A major factor limiting widespread application of wastewater monitoring is often an incomplete understanding of the variability and accuracy of wastewater detection data. Wastewater data can be highly variable. Limiting and understanding the causes of this variability are important tasks within the research community in order to improve confidence in this data source.
To encourage discussion on data quality and interpretation, we convened a workshop on May 17, 2021. During this workshop, our goals were to:
- discuss experiences in method development
- determine what quality control data are most valuable for increasing confidence in SARS-CoV-2 wastewater surveillance.
Prior to the workshop, a survey was distributed to better understand usage of various methodological controls, the perceived value of these controls, and the most frustrating issues encountered during method development and optimization. Below some results from the survey are highlighted:
Respondents also highlighted their most challenging quality controls issues, which included:
- Resource issues (e.g. accurately quantified virus recovery materials)
- Reaching LOD/LOQ
- Replicate variability
- Endogenous controls positive in method blanks
- Inhibition
- Extraction reproducibility
- Low/variable recovery
- Determining appropriate targets for endogenous/recovery controls
- Sample freeze-thaw
Workshop structure
Based on these responses, we developed a workshop consisting of two main sessions. In the first session, titled “Group Therapy” participants were split into virtual rooms for group troubleshooting of some of the issues identified in the pre-survey.
In the second session, participants returned to their virtual rooms for an activity to rank methodological controls based on both their value in increasing confidence in a certain result and their resource intensity. Each room was assigned a scenario based on typical surveillance program designs that rely on either positive/negative detection or quantitative trends (example below).
Each scenario was described as below:
- Positive Detection: You are monitoring a building complex weekly for SARS-CoV-2. Your samples for the past month have been negative. This week, your sample is positive. What other data/controls improve your confidence that this result is correct?
- Negative Detection: You are monitoring a building complex weekly for SARS-CoV-2. Your samples for the past month have been positive. This week, your sample is negative. What other data/controls improve your confidence that this result is correct?
- Increasing Trend: You are monitoring a community wastewater treatment plant twice per week. SARS-CoV-2 concentrations have been steady and relatively low for the past month. This week’s concentrations are about an order of magnitude higher. What additional data/controls improve your confidence that the community’s concentrations are increasing?
- Decreasing Trend: You are monitoring a community wastewater treatment plant twice per week. SARS-CoV-2 concentrations have been increasing over the past month and are starting to plateau. This week’s concentrations are about an order of magnitude lower. What additional data/controls improve your confidence that the community’s concentrations are decreasing?
The below interactive figures describe results from groups discussing each of the above scenarios. Click through the scroll bar at the bottom to view how each data/control type’s value changes for different interpretation scenarios. You can also hover over individual points to identify them.
As is demonstrated above, not all discussion groups ended with the same final list of possible data resources. Only five of the controls were shared across all of the scenarios by the end of the activity. Below, these five are compared with respect to their averaged resource intensity and confidence-increasing value for each scenario.
Conclusions
From the workshop activities, we engaged in meaningful discussion on the nuances of SARS-CoV-2 wastewater surveillance data interpretation and confidence levels. Some key conclusions are as follows:
- Sample replicates remain very important for all scenarios, but are also one of the most resource intensive controls to process.
- Different combinations of controls are valuable for different scenarios, reinforcing the value of a multi-layered control strategy to ensure sufficient data is available for interpretation.
- Data interpretation may not always be 100% clear, but controls are critical to increase confidence and minimize error.
- High-throughput methods with built-in control frameworks and/or reduced resource intensity would be valuable additions to the wastewater surveillance technology landscape.
Workshop Contributors
Moderator: Jeseth Delgado Vela (assistant professor, Howard University)
Facilitator: Mitham Al-faliti (PhD student, Howard University)
Discussion Group Leaders: Aaron Bivins (postdoc, University of Notre Dame), Catherine Hoar (postdoc, New York University), Rose Kantor (postdoc, UC Berkeley), Kendra Maas (scientist, UConn), Jill McClary-Gutierrez (postdoc, University of Notre Dame), Marlene Wolfe (postdoc, Stanford University)