Inherent bias of SARS-CoV-2 RNA quantification for wastewater surveillance due to variable RT-qPCR assay parameters

Department of Civil & Environmental Engineering & Earth Sciences, University of Notre Dame, Notre Dame, IN 46556. School of Chemical, Biological, & Environmental Engineering, Oregon State University, Corvallis, OR 97331. CSIRO Land and Water, Lucas Heights, NSW 2234, Australia. Medical Technology Research Center, Faculty of Health, Education and Social Care, Anglia Ruskin University, Chelmsford, Essex, CM1 1SQ, UK. United States Environmental Protection Agency, Office of Research and Development, 26W Martin Luther King Jr. Drive, Cincinnati, OH, 45268, USA. CSIRO Land and Water, Ecosciences Precinct, 41 Boggo Road, Dutton Park 4102, QLD, Australia. *Corresponding author. Warish Ahmed. Mailing address: Ecosciences Precinct, 41 Boggo Road, Dutton Park 4102, Queensland, Australia Tel.: +617 3833 5582; E-mail address: Warish.Ahmed@csiro.au Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 11 June 2021 doi:10.20944/preprints202106.0320.v1


Introduction
The screening of untreated wastewater and primary solids for severe acute respiratory syndrome Quantification by RT-qPCR is based on the quantification cycle (Cq), which is usually obtained by defining a baseline cycle range below which amplification is not recorded and uses a multiple of the standard deviation of the noise levels in the baseline to establish a threshold line which, upon crossing the amplification curves, generates Cq values. The Cq is inversely proportional to the initial template concentration, and samples are quantified by comparing their Cq values relative to assay-specific standard curves. These are constructed using a dilution series of standard control materials, typically four to six 10-fold dilution points, with a defined number of target sequences.
The standard curve is created by plotting the resultant Cq values against log transformed gene copy (GC) quantities from control materials and fitting a linear trend line to the data (y = mx + b). The precision of the quantitative data is strongly influenced by the quality of the standard curve as reflected by its slope (m), PCR efficiency, linearity (r 2 value) and y-intercept (b). PCR efficiency is calculated from the slope (m) of the trendline, with an optimal efficiency, resulting in a doubling of the PCR product after each cycle, characterized by a slope of -3.32. Although amplification efficiencies of between 90% and 110% are considered to be acceptable, a small change in PCR efficiency from 100 to 97% over 30 cycles equates to a 57% difference in input DNA calculation, while a change from 100 to 90% over the same range makes a 365% difference (Boulter et al. 2016). The r 2 and the standard error of the estimated amplification efficiency can be used to evaluate the quality of the efficiency determination, while the intercept of the standard curve on the y-axis gives a theoretical sensitivity of the assay, denoting the number of cycles required for the detection of a single unit of measurement. The quantities of target sequences in all samples can be calculated by comparing the respective Cq measurement to the corresponding standard curve calibration model. A standard curve can be constructed using a variety of control materials including plasmid DNA constructs, PCR amplicons, synthetic RNA or DNA, genomic DNA, cDNA, RNA or DNA from biological samples, and quantities may consist of certified, reference or information values (Bustin et al., 2009;Hou et al., 2010).
Although RT-qPCR offers a sensitive and specific technique for quantification of nucleic acid targets, reproducibility and reliability are critical and must be established for each assay.
Variations in protocols, reagents, sample quality, instruments, operators, analysts, data analysis and interpretation across laboratories can lead to the production of inaccurate quantitative data.
To circumvent these issues, Bustin  Herein, we review the SARS-CoV-2 wastewater surveillance literature to assess the appropriate use of RT-qPCR calibrators and associated performance parameters. All publications included in the analysis are compiled in an open access database as detailed below.

Screening the SARS-CoV-2 Wastewater Surveillance Literature
We screened a total of 125 preprint and peer-reviewed publications pertaining to wastewater surveillance for SARS-CoV-2 RNA as listed on the COVID-19 Wastewater-based Epidemiology Collaborative (WBEC) publication map (https://www.covid19wbec.org/publication-map) on 15 May 2021. Of those publications, 44 were excluded from further analysis due to use of platforms other than RT-qPCR for SARS-CoV-2 RNA quantification (e.g., digital PCR), reporting qualitative results (e.g., positive/negative or Cq values), or genomic rather than quantitative analysis. The remaining 81 publications (46 peer-reviewed, 35 pre-prints) reported quantitative results (GC) as measured by RT-qPCR. Where available, we extracted the following information for each RT-qPCR assay reported: SARS-CoV-2 gene target, one-step or two-step RT-qPCR protocol, standard curve parameters (y-intercept, slope, r 2 , efficiency), limits of detection and quantification, the dynamic range, control material used for standard curve (i.e., plasmid DNA, gBlock, PCR amplicon, etc.), the vendor of the control material, if pre-treatment of the control material was performed, and if independent quantification was performed prior to use. The included studies were divided evenly among three co-authors. Relevant data were extracted with duplicated review of 5 to 10 randomly selected entries by each author. The resulting database was independently reviewed by each author and discrepancies remedied by discussing to form a consensus. The consensus database was then independently reviewed and audited by a fourth author. The compiled database we used for the analysis can be found at https://osf.io/q7dnp/ doi: 10.17605/OSF.IO/Q7DNP. Our purpose in conducting this review is not to identify individual laboratories with questionable practices, but to highlight the importance of reporting standard curve parameters and to assess the performance of the wastewater surveillance community as a whole. For this reason, we have not provided citations when discussing individual publications.

RT-qPCR and Standard Curve Reporting
From the 81 selected publications, we extracted details pertinent to the assays used, standard curve parameters, and positive control materials for 208 separate quantitative assays (i.e., an average of 2.6 assays per publication). Only 26% of the total 208 RT-qPCR assays reported in the literature included all essential standard curve parameters, with 41% (86 of 208) reporting at least one standard curve parameter, i.e., 30,33,35, and 40% of studies detailing y-intercept, slope, r 2 values, and efficiency data, respectively. Among the 208 RT-qPCR assays, 130 targeted the N gene, 25 targeted ORF1, 23 targeted the E gene, 19 targeted RdRp, and 10 targeted the S gene, while one did not report any assay target. For assays targeting the N gene of SARS-CoV-2, the CDC N1 assay was applied most frequently (39%) followed by the CDC N2 assay (32%).
Together, these two assays accounted for 45% of the RT-qPCR assays reportedly used to quantify SARS-CoV-2 RNA in wastewater.
RT-qPCR assays are performed using two approaches: a separate RT reaction (i.e., cDNA synthesis) followed by qPCR (two-step) or a combined RT and qPCR reaction in the same tube (one-step). The two-step approach often uses random hexamer primers during RT followed by target-specific primers with the qPCR step offering more flexibility to optimize amplification conditions (Bustin and Nolan, 2017). But the two-step approach requires additional sample handling potentially leading to greater measurement variability and risk of contamination.
Conversely, the one-step approach utilizes gene-specific primers and minimizes sample handling by carrying out the RT and qPCR steps in the same microtube, reducing bench time and risk of contamination at the expense of less flexibility for assay optimization.
The majority (83%) of the reported RT-qPCR assays utilized the one-step protocol and 13% of RT-qPCR assays were performed as two-step assays. The remaining 4% did not report whether a one-step or two-step protocol was used. The performance of one-step and two-step RT-qPCR assays were compared in a previous study and the results indicated significant variation in quantification of the targets between the two protocols (Bustin and Nolan, 2017).
However, in an earlier study (Wackerd and Godard, 2005), both one-step and two-step protocols produced similar standard curves with RT-qPCR efficiencies close to 100% suggesting that discrepancies may be protocol and assay specific.

CDC N1 and N2 Standard Curves
To assess the impact of heterogeneity in reported RT-qPCR standard curves used for quantification of SARS-CoV-2 RNA in wastewater, we analyzed the reported y-intercepts, slopes, r 2 values, and efficiencies for the two most frequently used RT-qPCR assays, CDC N1 and N2 ( Figure 1). For review, the copy number in an unknown sample is calculated from the standard curve y-intercept and slope per equation (1).

Y-intercept values
Reported y-intercepts for CDC N1 (n = 21) and N2 (n = 18) standard curves ( Figure 1A) ranged from 36.1 to 42.5 and 37.8 to 53.5, respectively, with 30 and 24 publications, respectively not reporting this value. These intercepts indicate that anywhere from 36 to 43 RT-qPCR thermal cycles would be required to detect a single GC by the CDC N1 assay and 38 to 54 for the CDC N2 assay. However, these values were obtained using a variety of reference materials from different vendors and the y-intercept value depends on the concentration or copy number associated with that reference material. So, for example, starting with a nominally high copy number standard results in a lower y-intercept value compared with the same reference material that has been ascribed a lower copy number value. The most common source for standard material was plasmid from IDT (n = 13 and 11 for N1 and N2, respectively) and y-intercepts reported range from 36.1 to 42.5 (N1) and 37.8 to 53.5 (N2). Considering these are reported to be the same standard, presumably quality controlled prior to shipment and documented with the same ostensible copy number, this draws attention to the subsequent handling of this reference material.

Slope values and RT-qPCR efficiencies
qPCR efficiency values are derived from the slope of a standard curve per equation (2) While efficiency values often vary between templates, they are typically highly reproducible for the same template (A-Z of Quantitative PCR, 2004).
The ideal slope of a standard curve is -3.32, which indicates 100% RT-qPCR efficiency, although a range from -3.1 (110%) to -3.58 (90%) is typical for an optimized probe-based assay (A-Z of Quantitative PCR, 2004). Slope is also typically more reproducible between laboratories and instruments than the y-intercept (A-Z of Quantitative PCR, 2004). Reported RT-qPCR slopes for CDC N1 and N2 assays are displayed in Figure 1B. For the CDC N1 assay, the mean standard curve slope was -3.29, which is within the typical interval for a probed-based assay; however, the reported slopes ranged from -3.60 (90%) to -2.40 (161%); outside the acceptable range of -3.1 (110%) to -3.58 (90%). A similar pattern is seen for CDC N2 with a mean slope of -3.46 (95%) and a range from -4.48 (67%) to -2.72 (133%).
Reported and calculated RT-qPCR efficiencies (calculated with reported slope if not explicitly described in the publication) for the CDC N1 and N2 assays are shown in Figure 1D.
For reported CDC N1 standard curves, the mean efficiency was 101% (median 95.8%) and ranged from 89.6 to 161%. For CDC N2 assays the range was from 65 to 129%, with a mean efficiency of 95.8% (median 95%).

r 2 values
Standard curves should demonstrate strong linear fits with r 2 values usually ranging from 0.980 to 1.00 (A-Z of Quantitative PCR, 2004). The r 2 value of a standard curve is influenced by the precision of replicate standard material Cq measurements. Lower r 2 values indicate contributions to variation from sources other than the control material copy number (e.g., pipetting error, standard dilution preparation error). For CDC N1 and N2 assays as shown in Figure 1C

Standard Curve Control Materials
For RT-qPCR, performance is dependent on the standard materials used to produce the standard curve as well as good laboratory practices (A-Z of Quantitative PCR, 2004). Careful, application-specific optimization is required to maximize the performance of the standard materials and the subsequent quantification of genetic targets in samples.
In many cases, commercially available control materials were employed. Often these materials are provided at a vendor-specified titer, which is assumed to increase the likelihood of producing reliable standard curves. However, it is clear from the variability described above that either the dilutions or the pipetting steps are highly variable and operator-dependent, something that has been shown to be the case (Bustin, 2002).

Control Material Reporting
A description of the control material used was reported for 78% of the SARS-CoV-2 RT-qPCR assays in the wastewater surveillance literature. Although, the description was sometimes ambiguous including terms such as "genes encoding nucleocapsid protein", "synthetic oligonucleotide", and "cDNA standards". Such ambiguous descriptions can be especially problematic if the vendor is not specified, which was the case for 28% of the reported assays.
Using broad classifications, control materials included plasmids for 59 assays, synthetic cDNA for 29, synthetic RNA for 36, various forms of transcripts for 19, ambiguous or unclear for 19, and not reported for 46 RT-qPCR assays. Plasmids and synthetic oligonucleotides (cDNA or RNA) accounted for the majority of reported control materials with a nearly even split between the two (28 and 31%, respectively). Pre-treatment of control materials was reported for 7 RT-qPCR assays (heat inactivation of isolated strain in two and linearization of plasmid in the remaining five instances). Independent quantification of control materials was only reported for 6% (12 of 208) of RT-qPCR assays.

Effects of Control Material Type on Standard Curve Parameters
Previous studies have reported significant variations in Cq for standard curves produced using non-linearized plasmids compared to linear control materials (synthetic cDNA or RNA) ( Linearization was only reported for 8% of the RT-qPCR assays where a plasmid was used as the control material for standard curves. We considered the effects of control material type, linear materials (cDNA or RNA) versus plasmid materials (mostly circular), on reported RT-qPCR standard curves for the CDC N1 and N2 assays across the wastewater surveillance literature. From the literature, we were able to extract standard curve parameters for 21 CDC N1 assays (seven linear control materials and 14 plasmid), and 18 CDC N2 assays (six linear control materials and 12 plasmid). Figure 2 shows scatter plots of each standard curve parameter for CDC N1 and CDC N2 assays stratified by control material type. Standard curve performance parameters based on synthetic RNA or CDNA and plasmid materials were evaluated via a Mann-Whitney U test (Mann and Whitney, 1947). A significant difference was only observed between the standard curve slopes and efficiencies produced for the CDC N1 assays (p = 0.0402; Figure 2B, p = 0.0181; Figure 2D, respectively). Although most of the statistical tests imply no significant differences, the scatter plots in Figure 2 suggest that plasmid control materials consistently exhibit a broader range of variation across all standard curve parameters for both the CDC N1 and N2 assays. A coefficient of variation (CV = standard deviation/mean) was calculated for each standard curve parameter and standard material group combination for both CDC N1 and N2 assays, CV values were greater for curves produced using plasmid materials compared to linearized materials, further supporting this observation.
Differences were particularly pronounced for standard curve slopes, r 2 values, and RT-qPCR efficiencies. For the CDC N2 assay, the use of plasmid control materials resulted in a roughly 2fold increase in the CV of the standard curve slope, r 2 value, and efficiency. Whereas, for the CDC N1 assay the plasmid control materials resulted in an 5-fold, 9-fold, and 4-fold increase in CV for the slope, r 2 value, and efficiency, respectively. These observations suggest increased variability, and therefore, lower reproducibility, among CDC N1 and N2 standard curves produced using non-linearized plasmid controls materials compared to linear RNA and cDNA materials with the effect being particularly pronounced for the CDC N1 RT-qPCR assay. As shown in Figure 3,

Reported LODs and LOQs
The MIQE guidelines list "evidence for limit of detection" (LOD) and "Cq variation at lower limit" (LOQ) as of essential importance for reporting RT-qPCR experiments. LOD values were reported for only 34% of RT-qPCR assays, ranging between one and 500 GC per reaction (the most frequently used unit). LOD values were also reported in volumetric units (i.e., µL, mL, etc.), but the source of this volume (e.g., sewage, extract eluate, reaction mixture) was often unspecified. A method definition for the LOD determination was rarely provided and, in several cases, the reported LOD values were below the 95% LOD theoretical limit of 3 GC/reaction derived from the Poisson distribution (Bustin et al., 2009). An LOQ was reported for only 19% of RT-qPCR assays, about half as frequently as an LOD. LOQ values were also most frequently reported in GC per reaction with values ranging from 5 to more than 10,000. Similar to LOD, when volumetric units were used to report LOQ, the source of the relevant volume was often ambiguous. Furthermore, when multiple RT-qPCR assays were used in a single publication, reported LOD and LOQ values were often not linked to a corresponding assay.

Discussion
The quantitative potential of RT-qPCR assays has resulted in the broadening of its use from Furthermore, the departure from best practice in this important process has serious implications for environmental monitoring for high priority biothreat agents in general.
Basic and essential information for RT-qPCR assays including the standard curve parameters of y-intercept, slope and/or efficiency, and r 2 value were reported for only 26% of the RT-qPCR assays used for wastewater surveillance. Variation in these parameters is even more rarely reported being published for only 9% of assays. The reported standard curve data exhibit broad heterogeneity that, in turn, limits the reproducibility and reliability of RT-qPCR data for SARS-CoV-2 wastewater surveillance. For example, y-intercepts ranged from 36.1 to 53.5; slopes ranged from -2.4 to -4.5; r 2 values were as low as 0.700 with reported efficiencies ranging from 65 to 161%. Many of these are well outside the bounds of expected performance for optimized RT-qPCR assays. Variation in RT-qPCR performance seems further exacerbated by the prevalent use of a plasmid control material without linearization. This practice is likely to produce increased variation across all standard curve parameters with a particularly pronounced increase in the widely used CDC N1 RT-qPCR assay.
The published SARS-CoV-2 wastewater surveillance literature also contains other scenarios that reinforce the need for higher standards in practice and reporting. In some publications, "standard curves" were generated using one to three standard dilutions, while others made GC concentration estimates in samples falling well outside the defined dynamic range. Many publications provided no standard curve information at all. Another publication stated that the quantitative data were generated by a commercial lab and did not provide any pertinent RT-qPCR information, not even the assay used to generate the data. In a few cases, sophisticated epidemiological models such as susceptible-exposed-infectious-recovered (SEIR), vector autoregression, and Monte Carlo simulations were applied to quantitative data from RT-qPCR experiments with no standard curve parameters reported, making it essentially impossible to validate the utility of these approaches.
There are important limitations to this review. First, this is not a formal systematic review, instead publications were identified from a curated collection of SARS-CoV-2 wastewater surveillance publications. Although this review did not consider every SARS-CoV-2 RT-qPCR publication, the inclusion of 125 works proved sufficient to identify multiple deficiencies in standard curve performance and reporting practices. Second, this review relied on self-reported data from pre-print and published scientific literature, which could be subject to bias.
Additionally, it is important to note that variation between RT-qPCR replicates, experimental runs, instruments, and laboratories is expected to a degree, and is well documented for various SARS-CoV-2 RT-qPCR assays (Bustin et al., 2021). While it is not possible to independently account for this expected variation, the variability observed in this review often exceeded recommend best practices strongly suggesting that the wastewater surveillance community must make a concerted effort to improve both reporting of RT-qPCR parameters and optimization of SARS-CoV-2 RT-qPCR assays.

Conclusions and Recommendations
This brief review of the literature describing the use of RT-qPCR assays for monitoring wastewater has revealed significant variation in the standards of the reported results, although the paucity of quality control data makes it difficult to ascertain which reports are likely to be the most credible. Hence there is an urgent need to improve the transparency of reporting as well as disclose more information about the characteristics of the assays being used. We propose the following measures:  To produce reliable data for public health decision-making, the wastewater surveillance community should, at a minimum, aspire to achieve 100% reporting of all standard curve performance parameters (y-intercept, slope/efficiency and, r 2 value). If standard curve parameters do not fall within general data acceptance ranges, additional actions may be required such as optimization, calibration of pipets and/or thermal cycle instrumentation, among others until the appropriate performance is achieved.
 Because the reproducibility of RT-qPCR data relies, in part, on the standard curves produced by control materials, the type and vendor for such materials should be completely and unambiguously reported. Any pre-treatment or manipulation of these control materials prior to quantification should also be documented, especially for plasmid controls. Caveat emptor must be the motto when relying upon commercially available control materials.
 It would also be useful to independently confirm vendor-reported titers to ensure that materials were not degraded during shipment or due to mishandling in the laboratory.
In summary, to maximize the utility of wastewater surveillance for public health, it is time to "walk the walk" (Bustin and Nolan, 2017).

Disclaimer
Information has been subjected to U.S. EPA peer and administrative review and has been approved for external publication. Any opinions expressed in this paper are those of the authors and do not necessarily reflect the official positions and policies of the U.S. EPA. Any mention of trade names or commercial products does not constitute endorsement or recommendation for use.