Longitudinal Assessment of Diagnostic Test Performance Over the Course of Acute SARS-CoV-2 Infection

Abstract

Background

Serial screening is critical for restricting spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) by facilitating timely identification of infected individuals to interrupt transmission. Variation in sensitivity of different diagnostic tests at different stages of infection has not been well documented.

Methods

In a longitudinal study of 43 adults newly infected with SARS-CoV-2, all provided daily saliva and nasal swabs for quantitative reverse transcription polymerase chain reaction (RT-qPCR), Quidel SARS Sofia antigen fluorescent immunoassay (FIA), and live virus culture.

Results

Both RT-qPCR and Quidel SARS Sofia antigen FIA peaked in sensitivity during the period in which live virus was detected in nasal swabs, but sensitivity of RT-qPCR tests rose more rapidly prior to this period. We also found that serial testing multiple times per week increases the sensitivity of antigen tests.

Conclusions

RT-qPCR tests are more effective than antigen tests at identifying infected individuals prior to or early during the infectious period and thus for minimizing forward transmission (given timely results reporting). All tests showed >98% sensitivity for identifying infected individuals if used at least every 3 days. Daily screening using antigen tests can achieve approximately 90% sensitivity for identifying infected individuals while they are viral culture positive.

Adults newly infected with SARS-CoV-2 were sampled daily for saliva and nasal swab RT-qPCR, Quidel SARS Sofia antigen FIA, and viral culture. We compare test sensitivities at different stages of acute infection and as a function of testing frequency.

Frequent rapid diagnostic testing is critical for restricting community spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) by allowing the timely identification and isolation of infected individuals to interrupt the chain of transmission. Quantitative reverse transcription polymerase chain reaction (RT-qPCR)-based detection of viral RNA within nasal swab or saliva samples represents the gold standard for sensitivity in detecting the presence of SARS-CoV-2. Unfortunately, it has been difficult to achieve high testing frequency and volume with the rapid reporting of results needed to mitigate transmission effectively due to supply shortages, cost, and infrastructure limitations.

There is considerable interest in the potential of rapid, lateral flow antigen tests to expand diagnostic testing capacity due to their ease of use, availability, relatively low cost, and rapid time to results [1]. However, data for their use in screening asymptomatic individuals is sparse [2]. Enthusiasm for their widespread deployment has been further tempered by well-publicized examples of false-positive results in people with low pretest probability of infection, and by reports suggesting they lack sensitivity compared with RT-qPCR, potentially making them less effective at mitigating community spread [3–5].

To maximize the effectiveness of available testing resources, there is an urgent need to quantify the sensitivities of different testing platforms at different stages of infection and define how sensitivity can be enhanced through serial testing. To address this, we compared the sensitivities of nasal and saliva RT-qPCR tests with the Quidel Sofia SARS Antigen Fluorescent Immunoassay (FIA) over the course of mild or asymptomatic acute SARS-CoV-2 infection through daily sampling of individuals enrolled early during infection. We also estimated the effects of varying serial testing frequency on the sensitivities of both RT-qPCR and antigen tests.

METHODS

This study was approved by the Western Institutional Review Board, and all participants provided informed consent.

Participants

All on-campus students and employees of the University of Illinois at Urbana-Champaign are required to submit saliva for RT-qPCR testing every 2–4 days as part of the SHIELD campus surveillance testing program. Those testing positive are instructed to isolate, and were eligible to enroll in this study for a period of 24 hours following receipt of their positive test result. Close contacts of individuals who test positive (particularly those cohoused with them) are instructed to quarantine and were eligible to enroll for up to 5 days after their last known exposure to an infected individual. All participants were also required to have a documented negative saliva RT-qPCR result 7 days prior to enrollment in the study.

Individuals were recruited via either a link shared in an automated text message providing isolation information sent within 30 minutes of a positive test result, a call from a study recruiter, or a link shared by an enrolled study participant or included in information provided to all quarantining close contacts. In addition, signs were used at each testing location and a website was available to inform the community about the study.

Participants were required to be at least 18 years of age, have a valid university identity, speak English, have internet access, and live within 8 miles of the university campus. After enrollment and consent, participants completed an initial survey to collect information on demographics and health history, including suspected date of SARS-CoV-2 exposure. They were then provided with sample collection supplies.

Participants who tested positive prior to enrollment or during quarantine were followed for up to 14 days. Quarantining participants who continued to test negative by saliva RT-qPCR were followed for up to 7 days after their last exposure. All participants’ data and survey responses were collected in the Eureka digital study platform.

Sample Collection

Each day, participants were remotely observed by study staff collecting:

1. 2 mL of saliva into a 50mL conical tube

2. 1 nasal swab from a single nostril using a foam-tipped swab that was placed within a dry collection tube

3. 1 nasal swab from the other nostril using a flocked swab that was subsequently placed in a collection vial containing viral transport medium (VTM).

The order of nostrils (left vs right) used for the 2 different swabs was randomized. For nasal swabs, participants were instructed to insert the soft tip of the swab at least 1 cm into the indicated nostril until they encountered mild resistance, rotate the swab around the nostril 5 times, leaving it in place for 10–15 seconds. After daily sample collection, participants completed a symptom survey. A courier collected all participant samples within 1 hour of collection using a no-contact pickup protocol designed to minimize courier exposure to infected participants. All study protocols were consistent throughout the duration of the study.

Saliva RT-qPCR

After collection, saliva samples were stored at room temperature and RT-qPCR was run within 12 hours of initial collection. The protocol for direct saliva to RT-qPCR assay used has been detailed previously [6]. In brief, saliva samples were heated at 95°C for 30 minutes, followed by the addition of 2× Tris/borate/EDTA buffer (TBE) at a 1:1 ratio (final concentration 1× TBE) and Tween-20 to a final concentration of 0.5%. Samples were assayed using the Thermo Taqpath coronavirus disease 2019 (COVID-19) assay.

Quidel Assay

Foam-tipped nasal swabs were placed in collection tubes, transported with cold packs, and stored at 4°C overnight based on guidance from the manufacturer. The morning after collection, swabs were run through the Sofia SARS antigen FIA on Sofia 2 devices according to the manufacturer’s protocol.

Nasal Swab RT-qPCR

Collection tubes containing VTM and flocked nasal swabs were stored at −80°C after collection and were subsequently shipped to Johns Hopkins University for RT-qPCR and viral culture. After thawing, VTM was aliquoted for RT-qPCR and infectivity assays. One ml of VTM from the nasal swab was assayed on the Abbott Alinity per manufacturer’s instructions in a College of American Pathologist and Clinical Laboratory Improvement Amendments (CLIA)-certified laboratory.

Nasal Virus Culture

VeroTMPRSS2 cells were grown in complete medium consisting of Dulbecco’s Modified Eagle’s Medium (DMEM) with 10% fetal bovine serum (Gibco), 1 mM glutamine (Invitrogen), 1 mM sodium pyruvate (Invitrogen), 100 U/mL of penicillin (Invitrogen), and 100 μg/mL of streptomycin (Invitrogen) [7]. Viral infectivity was assessed on VeroTMPRSS2 cells as previously described using infection medium (identical to complete medium except the fetal bovine serum was reduced to 2.5%) [8]. When a cytopathic effect was visible in >50% of cells in a given well, the supernatant was harvested. The presence of SARS-CoV-2 was confirmed through RT-qPCR as described previously by extracting RNA from the cell culture supernatant using the Qiagen viral RNA isolation kit and performing RT-qPCR using the N1 and N2 SARS-CoV-2-specific primers and probes in addition to primers and probes for human RNaseP gene using synthetic RNA target sequences to establish a standard curve [9].

Data Analysis

At the time of analysis, nasal samples from 51 participants had been analyzed by virus culture and RT-qPCR. Eight individuals were removed from the analysis because their nasal virus culture was never positive, leaving 43 remaining participants. All confidence intervals around sensitivity were calculated using binconf from the Hmisc package in R version 3.6.2.

The sensitivity of each of the tests was analyzed in 3 ways. First, we calculated the daily sensitivity of each test across the course of the infection. Daily sensitivity was defined as the ability of each test (antigen, saliva RT-qPCR, or nasal RT-qPCR) to detect an infected person on a particular day, with day 0 defined as the day of first positive viral culture. Daily sensitivity was not calculated for time points with fewer than 5 observed person-days.

Second, we calculated the ability of each test to detect an infected person according to their viral culture status (status sensitivity). Viral culture status was defined as prepositive on days prior to the first positive viral culture result, positive on days for which viral culture results were positive, and postpositive on days with negative viral culture results that occurred after the first positive culture result. Status sensitivity was defined as the proportion of person-days with a positive result.

Finally, we calculated the ability of repeated testing over a 14-day period to detect an infected person (protocol sensitivity) using a value-of-information approach. Seven different testing frequencies were considered: daily, every other day, every third day, and so on, up to weekly sampling. For each individual, the result of testing on a given schedule was calculated for each potential starting date, with test results interpreted in parallel (all tests must be negative to be considered negative). For instance, each person contributed 2 observations to the every other day schedule, one starting on the first day of the study and comprising samples from days 1, 3, 5, 7, 9, 11, and 13, and the other starting on the second day of the study and comprising samples from days 2, 4, 6, 8, 10, 12, and 14. As each testing schedule was evaluated at each potential starting day, the number of potential schedules increased as testing frequency decreased. Protocol sensitivity was defined for individual testing schedules, where the numerator was the number of testing schedules resulting in at least 1 positive test and the denominator was the number of testing schedules examined, where a testing schedule was defined as a set of samples from 1 participant taken at a given frequency. The proportion of observations (or sets of samples) with a positive result (at least 1 positive test in the sampling time frame) was considered to be the sensitivity of that testing protocol (test and frequency combination).

Sensitivities were considered significantly different at P < .05. All statistics were calculated using binom.test or glm in R. All code used in analyses can be found at https://github.com/rlsdvm/CovidDetectAnalysis.

RESULTS

Table 1 shows demographic information for study participants reported here. The majority of participants (30/43, 69.8%) were non-Hispanic white and the average age was 32.3 years (SD 12.8; range, 19–73). Of the 43 participants, 23 provided 14 days of observations, 10 provided 13 days of observation, and only 3 provided fewer than 10 days of observation.

Table 1.

Demographic Information on Study Participants

Variable	Data (n = 43)
Age, y, mean (SD)	33.1 (12.8)
Race, No. (%)
Native American	0 (0.0)
Asian	1 (2.3)
Black	4 (9.3)
Other	4 (9.3)
Pacific Islander	0 (0.0)
White	34 (79.1)
Sex, No. (%)
Female	20 (46.5)
Male	23 (53.5)
Ethnicity, No. (%)
Hispanic	8 (18.6)
Non-Hispanic	35 (81.4)

The estimated daily sensitivities of nasal and saliva RT-qPCR and antigen tests relative to the day of first nasal swab viral culture positivity, which was used as a surrogate marker of infectious virus shedding, are shown in Figure 1 and Supplementary Table 1. For all 3 tests, daily and status sensitivity peaked during days in which infectious virus shedding was detectable, as would be expected. Antigen test daily sensitivity declined precipitously after infectious virus could no longer be detected in nasal swabs, dropping to 0.238 (95% confidence interval [CI], .135–.385) within a week after the onset of culture positivity, which was significantly lower (P < .001) than both nasal and saliva RT-qPCR platforms. Nasal and saliva RT-qPCR only showed minor decreases in sensitivity during this period, remaining at 0.857 (95% CI, .722–.933) and 0.690 (95% CI, .540–.809) after a week, respectively, and were not significantly different from each other (P = .07).

Figure 1.

Daily sensitivity of each test platform relative to the day of first positive viral culture result. Shaded areas represent the 95% confidence interval around the observed proportion. Abbreviation: RT-qPCR, quantitative reverse transcription polymerase chain reaction.

We also used the viral culture data to measure the status sensitivities of each test before, during, and after viral shedding (Figure 2). Prior to the first day of detectable shedding of infectious virus, nasal RT-qPCR tests had significantly higher (P < .05) sensitivity (0.650; 95% CI, .483–.794) than the antigen test (0.375; 95% CI, .227–.542). The sensitivity of saliva RT-qPCR (0.750; 95% CI, .588–.873) was not significantly different from that of nasal RT-qPCR (P = .14) or antigen (P = .07) prior to the first positive viral culture. On days when the viral culture was positive, there were no significant differences in sensitivity among the 3 testing modalities (P > .2). After viral culture was no longer positive, the sensitivity of the antigen test (0.454; 95% CI, .376–.534) was significantly lower (P < .001) than the sensitivity of the saliva (0.847; 95% CI, .782–.898) or nasal (0.945; 95% CI, .898–.974) RT-qPCR tests.

Figure 2.

Status sensitivity of each test platform relative to viral culture positivity. Bars indicate the 95% confidence interval around the observed proportion. Prepositive (n = 31) refers to samples taken on days before the first viral culture-positive sample collected from each individual. Positive (n = 153) refers to samples taken on days for which viral culture results were positive. Postpositive (n = 126) refers to samples taken on days with negative viral culture results that occur after the first positive culture result. Abbreviation: RT-qPCR, quantitative reverse transcription polymerase chain reaction.

We next estimated the protocol sensitivities, or how the ability of each of test platform to detect infected individuals was affected by differences in testing frequencies (Table 2 and Figure 3). In Figure 3A we show sensitivity to detect infected individuals at any stage of infection. For all 3 test platforms examined, protocol sensitivity remained >0.98 with testing at least every third day. When applied weekly, protocol sensitivity remained very high for nasal RT-qPCR at 0.987 (95% CI, .966–.996) and for saliva RT-qPCR at 0.963 (95% CI, .936–.982), but dropped to only 0.797 (95% CI, .747–.841) for the antigen test, which was significantly lower than either PCR test (P < .001).

Table 2.

Protocol Sensitivity of Each Testing Platform to Detect an Infected Person During a 14-day Testing Period Relative to the Frequency of Testing

Testing Frequency	No.	No. Before or While VC+a	Nasal Antigen				Saliva RT-qPCR				Nasal RT-qPCR
			Probability of Detection		No. Positive		Probability of Detection		No. Positive		Probability of Detection		No. Positive
			Any Timeb	Before or While VC+	Any Time	Before or While VC+	Any Time	Before or While VC+	Any Time	Before or While VC+	Any Time	Before or While VC+	Any Time	Before or While VC+
Daily	43	22	1	0.909	43	20	1	0.955	43	21	1	1	43	22
Every other day	86	44	1	0.841	86	37	0.988	0.909	85	40	1	0.909	86	40
Every third day	129	66	1	0.803	129	53	0.984	0.833	127	55	1	0.848	129	56
Every fourth day	172	88	0.959	0.739	165	65	0.983	0.761	169	67	1	0.784	172	69
Every fifth day	215	110	0.921	0.682	198	75	0.981	0.709	211	78	0.995	0.727	214	80
Every sixth day	258	132	0.864	0.621	223	82	0.965	0.644	249	85	0.992	0.667	256	88
Weekly	301	154	0.797	0.558	240	86	0.963	0.597	290	92	0.987	0.597	297	92

Abbreviations: RT-qPCR, quantitative reverse transcription polymerase chain reaction; VC+, viral culture positive,

^aBefore or while VC+ refers to detection of the individual before or during the time in which their viral culture was positive.

^bAny time refers to detection of the individual at any point in the 14-day testing period.

Figure 3.

Protocol sensitivity of each test platform to detect an infected person (A) at any time over a 14-day testing period or (B) before or on days where nasal samples were viral culture positive, relative to the frequency of testing. Bars indicate 95% confidence interval around the observed proportion. Abbreviation: RT-qPCR, quantitative reverse transcription polymerase chain reaction.

When we compared the abilities of different testing frequencies to identify individuals before or during the period when infectious virus was detectable in nasal samples (Figure 3B), we observed a clear reduction in protocol sensitivity for all testing modalities when testing frequencies decreased below daily, although the linear trend was not statistically significant (P > 0.05). The reduction in protocol sensitivity was most pronounced for the antigen test, which dropped to 0.739 (95% CI, .634–.827) with testing every fourth day. However, both RT-qPCR tests were only slightly better with both showing a sensitivity of 0.784 (95% CI, .684–.865) for nasal and of 0.761 (95% CI, .659–.846) for saliva.

Discussion

This is the first study to compare the longitudinal performance of rapid antigen and RT-qPCR tests with infectious virus shedding through daily testing early during SARS-CoV-2 infection. Our data clearly define how the sensitivities of RT-qPCR and antigen tests vary over the course of SARS-CoV-2 infection. Prior to the presumed infectious period (here defined as the period during which infectious virus could be detected in nasal swab samples), the daily sensitivities of nasal and saliva RT-qPCR tests were substantially higher than that of the Quidel Sofia SARS Antigen FIA, suggesting that RT-qPCR tests will be more effective than antigen tests at identifying infected individuals before they can transmit to others, provided that results reporting is rapid enough.

Both RT-qPCR and antigen tests peaked in daily and status sensitivities when infectious virus was detectable in nasal swab samples, suggesting that all 3 modalities can be effective at identifying individuals during the presumed infectious period. After this period, the daily sensitivity of RT-qPCR tests decreased gradually, consistent with the dynamics described previously for RT-qPCR [10, 11]. In contrast, the daily sensitivity of the antigen test declined very quickly, suggesting that this test will be less effective at identifying individuals during later stages of infection. The short duration of antigen positivity may limit diagnosis and contact-tracing efforts in test-limited environments.

Previous studies have suggested that frequent testing would maximize the ability of a given test modality to detect infected individuals at any stage of infection [12–14]. We found that all testing modalities showed >98% protocol sensitivity to detect infection if used at least every 3 days, which supports that conjecture. However, the results presented here are based on empirical data, rather than the modeling approaches previously used, and therefore give stronger confidence to these estimates.

Altogether, these data demonstrate the importance of frequent testing regardless of test modality for identifying individuals while they are contagious. It should be noted that while virus culture on nasal swabs represents the best proxy available for infectivity, it is likely imperfect. It is also possible that some samples taken from infectious individuals may have given negative results in the virus culture assay because they were below the limit of detection, especially given that the viral culture samples were subjected to a single freeze/thaw cycle prior to being assayed.

The sensitivities of particular testing protocols presented here assume that individuals will strictly adhere to these testing frequencies over time. This may be more feasible in more closed populations, such as schools or businesses, than in general public health settings where the population is more fluid. However, the results could also be applied at a personal level to assist concerned individuals in determining the best frequency at which to seek out testing. These results should not be applied to interpret the results of a single test outside the context of regular screening.

It should also be noted that participation in this study was limited to faculty, students, and staff of the University of Illinois at Urbana-Champaign, and that the participant population included here was primarily young, non-Hispanic white, and skewed slightly towards men. All infections were either mild or asymptomatic, and no participants were hospitalized for COVID-19. The limited demographic and clinical profiles of our study population must be considered when extending these results to groups with different risk profiles.

Altogether, our results indicate that frequent serial RT-qPCR testing with rapid results reporting is the optimal screening strategy for identifying asymptomatic or presymptomatic individuals before they can transmit the virus, thus mitigating community spread of SARS-CoV-2. In communities where serial RT-qPCR testing with rapid results reporting is not possible, then frequent serial antigen testing (at least every 3 days or twice weekly) represents the best alternative.

Supplementary Data

Supplementary materials are available at The Journal of Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Acknowledgments. We thank Shumon Ahmed, Carly Bell, Nate Bouton, Callie Brennen, Justin Brown, Coleco Buie, Emmaline Cler, Gary Cole, Trey Coleman, Lauren Engels, Savannah Feher, Kelsey Fox, Lexi Freeman, Yesenia Gonzalez, Montez Harris, Dan Hiser, Ayeshah Hussain, Daryl Jackson, Junko Jarrett, Michael Jenkins, Kalombo Kalonji, Syntyche Kanku, Steven Krauklis, Mary Krouse, Elmore Leshoure, Joe Lewis, Maggie Li, Angel Lopez, Guadalupe Lopez, Emily Luna, Chun Huai Luo, Colby Mackey, Skyler McLain, Yared Berhanu Melesse, Madison O’Donnell, Savanna Pflugmacher, Denver Piatt, Skyler Pierce, Jessica Quicksall, Gina Quitanilla, Ameera Samad, MacKenzie Scroggins, Monique Settles, Macie Sinn, Pete Varney, Evette Vlach, and Raeshun Williams-Chatman for their efforts supporting recruitment, enrollment, logistics, and sample collection. We also thank Jeffrey Olgin, Noah Peyser, and Xochitl Butcher for assistance with the Eureka platform, Michelle Lore for assistance with REDcap, Melanie Loots for assistance with administration, Gillian Snyder for assistance in development of study protocols and logistics, and Erin Iturriaga and Jue Chen for study protocol development.

Disclaimer. The views expressed in this manuscript are those of the authors and do not necessarily represent the views of the National Institute of Biomedical Imaging and Bioengineering; the National Heart, Lung, and Blood Institute; the National Institutes of Health; or the US Department of Health and Human Services. Sofia 2 devices and associated supplies were provided to Carle Foundation Hospital by Quidel; however, Quidel played no role in the design of the study or the interpretation or presentation of the data.

Financial support. This work was supported by the National Heart, Lung, and Blood Institute at the National Institutes of Health (grant number 3U54HL143541-02S2) through the RADx-Tech program.

Potential conflicts of interest. C. B. B. and L. W. are listed as inventors on a pending patent application for the saliva RT-qPCR test used in this study. All other authors report no potential conflicts. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.