ABSTRACT
Access to reverse transcription-PCR (RT-PCR) testing, the gold standard for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) detection, is limited throughout the world, due to restricted resources, available infrastructure, and high costs. Antigen-detecting rapid diagnostic tests (Ag-RDTs) overcome some of these barriers, but independent clinical validations in settings of intended use are scarce. To inform the World Health Organization’s (WHO) emergency use listing (EUL) procedure and ensure affordable, high-quality Ag-RDTs, we assessed the performance and ease of use of the SureStatus for SARS-CoV-2. For this prospective, multicenter diagnostic accuracy study, we recruited unvaccinated participants with presumed SARS-CoV-2 infection in India and Germany from December 2020 to March 2021, when the Alpha (B.1.1.7) variant was predominantly circulating. Paired swabs were performed for (i) routine clinical RT-PCR testing (sampling was either nasopharyngeal [NP] or combined NP and oropharyngeal [NP/OP]) and (ii) Ag-RDT (sampling was NP). Performance of the Ag-RDT was compared to RT-PCR overall and by predefined subgroups, e.g., cycle threshold (CT) value, symptoms, and days from symptom onset. To understand the usability, a system usability scale (SUS) questionnaire and ease-of-use (EoU) assessment were performed. A total of 1,119 participants were included in the analysis, of whom 205 (18.3%) were RT-PCR positive. SureStatus detected 169 out of 205 RT-PCR-positive participants, reporting a sensitivity of 82.4% (95% confidence interval [CI]: 76.6% to 87.1%) and a specificity of 98.5% (95% CI: 97.4% to 99.1%). In the first 7 days post-symptom onset, the sensitivity was 90.7% (95% CI: 83.5% to 94.9%), when CT values were low and viral loads were high. The test was characterized as easy to use (SUS, 85/100) and considered suitable for point-of-care settings, although quality concerns were raised due to visibly contaminated packaging of swabs included in the test kits. The SureStatus diagnostic test can be considered a reliable test during the first week of SARS-CoV-2 infection, with high sensitivity in combination with excellent usability.
IMPORTANCE Our manufacturer-independent, prospective diagnostic accuracy study assessed clinical performance in participants presumed to have a SARS-CoV-2 infection at three study sites in two countries. We assessed the accuracy overall and in predefined subgroups (CT values and symptom duration). SureStatus performed with high sensitivity. Its sensitivity was particularly high in the first 3 days after symptom onset and when CT values were low (i.e., the viral load was high). The system usability and ease-of-use assessment complements the accuracy assessment of the test and highlights critical factors to facilitate the widespread use of SureStatus in point-of-care settings. The high sensitivity demonstrated by the evaluated Ag-RDT within the first days of symptoms, when most transmission occurs, supports the role of Ag-RDTs for public health-relevant screening. Evidence from this study was used to inform the World Health Organization Emergency Use Listing procedure.
KEYWORDS: SARS-CoV-2, COVID-19, antigen-detecting rapid diagnostic tests, sensitivity, specificity
INTRODUCTION
Antigen-detecting rapid diagnostic tests (Ag-RDTs) are widely available and are used to complement the current gold standard, reverse transcription-PCR (RT-PCR), for diagnosis of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Ag-RDTs usage aims at increasing testing capacities and early isolation of infected individuals to minimize viral spread (1). Especially in the global south, usage of Ag-RDTs has increased, while access to the gold standard, RT-PCR, is limited due to the required infrastructure and personnel and the high cost. With this expanded usage of Ag-RDTs worldwide, the need for highly sensitive yet low-cost tests has increased (2, 3).
In 2020, the World Health Organization (WHO) initiated a global partnership to ensure affordable, high-quality Ag-RDTs for low- and middle-income countries (LMICs) to increase testing and reduce the spread of the virus (4). Currently, four Ag-RDTs are listed for emergency use (EUL) by the WHO: PanBio nasal and nasopharyngeal (Abbott), Standard Q (SD Biosensor), and SureStatus (PMC Private Limited) all meet the minimum standards of >80% sensitivity and >98% specificity (5). However, greater capacities for both the production of high-quality tests and lower prices are necessary to meet the rising demand in the global south (6).
The primary objective of this multicenter prospective accuracy study was to evaluate SureStatus for clinical diagnostic accuracy and ease of use (EoU) and its suitability as a diagnostic tool for global testing strategies. This study represents the first multicenter, manufacturer-independent diagnostic accuracy study for the Ag-RDT SureStatus. The study informed a large-scale investment from the WHO, the Foundation of Innovative New Diagnostics (FIND), and Unitaid, which enhanced production capacities and technology transfer, facilitating the production and delivery of over 250 million low-cost SureStatus tests to LMICs (7). In 2021, more than 10 million SureStatus tests were made available to the global south for under $2.55 per test (7).
RESULTS
Clinical diagnostic accuracy.
Patients were enrolled from 26 February to 25 March 2021 in Germany (26 February to 25 March 2021 in Heidelberg; 1 to 24 March 2021 in Berlin) and from 3 December 2020 to 23 April 2021 in India. During the enrollment period, a total of 1,196 eligible participants meting the inclusion criteria were screened for this study. Of these, 1,133 agreed to undergo a second swab for study purposes (Fig. 1). Across all sites, 13 participants were early exclusions due to a lack of patient information available. After data cleaning and the exclusion of one invalid PCR test result, a total of 1,119 participants were included in the analysis.
FIG 1.
Study flow.
The clinical and demographic characteristics of the enrolled participants are summarized in Table 1. The median age was similar across study sites (in India, 39 years; interquartile range [IQR], 28 to 52; in Germany, 36 years; IQR, 27 to 50). Overall, 38.3% of participants were female, with a higher female-to-male ratio in Germany (1.4:1) than in India (0.6:1). Of all participants, 20.0% reported comorbidities. In total, 454 participants (40.8%) reported having symptoms on the testing day. However, there were differences between sites; in India, 20.8% of participants presented with symptoms, and in Germany, 64.0% of participants reported symptoms. The median duration of symptoms was 5 (IQR, 2 to 7) days in India and 2 (IQR, 1 to 4) days in Germany. In total, 205 (18.3%) participants were diagnosed with a SARS-CoV-2 infection by RT-PCR testing during the enrollment period, with 19.3% out of 519 participants enrolled in Germany and 17.5% out of 600 enrolled participants in India testing positive. The median cycle threshold (CT) value was slightly higher in Germany, 20.3 (IQR, 17.5 to 23.7), than that in India, 19.0 (IQR, 16.5 to 25.0) (Table 1; Fig. 2).
TABLE 1.
Study population characteristics
| Characteristic | Data for study site(s) |
||
|---|---|---|---|
| All | Chennai, India | Heidelberg and Berlin, Germany | |
| No. of participants | 1,119 | 600 | 519 |
| Median age (IQR)a | 37 (27–51) | 39 (28–52) | 36 (27–50) |
| Gender (n [%])b | |||
| Female | 427 (38.3) | 176 (29.4) | 251 (48.7) |
| Male | 687 (61.7) | 423 (70.6) | 264 (51.3) |
| Comorbidities (n [%])c | |||
| Yes | 224 (20.0) | 99 (16.5) | 125 (24.1) |
| No | 895 (80.0) | 501 (83.5) | 394 (75.9) |
| PCR result (n [%])c | |||
| Positive | 205 (18.3) | 105 (17.5) | 100 (19.3) |
| Negative | 914 (81.7) | 495 (82.5) | 419 (80.7) |
| Reporting symptoms (n [%])b | |||
| Yes | 454 (40.8) | 125 (20.8) | 329 (64.0) |
| No | 660 (59.0) | 475 (79.2) | 185 (36.0) |
| Median symptom duration (days [IQR])d | −2 (1–5) | 5 (2–7) | 2 (1–4) |
| Median CT value (IQR)e | 20.0 (17.0 to 24.2) | 19.0 (16.5–25.0) | 20.3 (17.5–23.7) |
IQR, interquartile range.
Information available for n = 1,114 (of 1,119).
Information available for n = 1,119 (of 1,119).
Information available for n = 394 (of 454).
Information available for n = 203 (of 205). Information on two RT-PCR results was not available.
FIG 2.
CT value comparison between Germany and India. Assays performed in Germany: Allplex SARS-CoV-2 assay from Seegene, Roche Cobas SARS CoV-2 assay on the Cobas 6800 or 8800 systems, SARS CoV-2 assay from TIB Molbiol. Assay performed in India: Thermo Fisher TaqPath COVID-19 combo kit PCR on the Applied Biosystems platform. TP, true positive; FN, false negative.
The SureStatus test had an overall sensitivity of 82.4% (169/205 RT-PCR-positive cases detected; 95% confidence interval [CI]: 76.6% to 87.1%) and a specificity of 98.5% (14 false positives in 914 RT-PCR-negative cases; 95% CI: 97.4% to 99.1%) (Fig. 3). The sensitivity in Germany was 91.0% (91/100 RT-positive cases detected; 95% CI: 83.8% to 95.2%) and in India, 74.3% (78/105 RT-positive cases detected; 95% CI: 65.2% to 81.7%). The specificity in India was 99.6% (2 false positives; 95% CI: 98.5% to 99.9%), compared to 97.1% (12 false positives; 95% CI: 95.1% to 98.4%) in Germany.
FIG 3.
Overall performance and subgroup analysis post-symptom onset for SureStatus. Overall performance for sensitivity (top left) and specificity (top right); subgroup analysis by duration of symptoms (a.1, ≤3 days; a.2, >3 days; b.1, ≤7 days; b.2, >7 days) and by CT value (c.1, ≤25; c.2, >25; d.1, ≤30; d.2, >30). TP, true positive; FN, false negative.
Analysis of the performance of SureStatus using CT values (≤25, ≤30) showed the highest sensitivity in participants with a CT value of ≤25, 87.7% (95% CI: 78.7% to 93.2%) in India and 97.5% (95% CI: 91.3% to 99.3%) in Germany. In Germany, the sensitivity for participants with a CT value of ≤30 was 97.8% (95% CI: 92.3% to 99.4%) and for participants with a CT value of >30, 22.2% (95% CI: 6.3% to 54.7%). In India, the sensitivity was 78.6% (95% CI: 69.5% to 85.5%) in participants with a CT value of ≤30 and 0.0% (95% CI: 0.0% to 43.4%) in participants with a CT value of >30. In summary, in Germany, most false negatives had a CT value above 30 with few exceptions, while in India, SureStatus performed less well, with a larger number of false negatives also observed with lower CT values.
When the test performance was assessed by the duration of symptoms, SureStatus performed well overall in the first 7 days after symptom onset (sensitivity, 90.7%; 95% CI: 83.5% to 94.9%), with declining sensitivity thereafter (>7 days of symptoms: sensitivity, 66.7%; 95% CI: 46.1% to 82.4%). Less than 3 days post-symptom onset, the overall sensitivity was 95.5% (95% CI: 86.8% to 98.5%), with the sensitivity in Germany being slightly higher than that in India, 96.4% (95% CI: 87.9% to 99.0%) versus 90.0% (95% CI: 59.6% to 98.2%), respectively. During the first 7 days post-symptom onset, we found that the SureStatus sensitivity was substantially higher in Germany (sensitivity, 96.2%; 95% CI: 89.3% to 98.7%) compared to that in India (sensitivity, 75.9%; 95% CI: 57.9% to 87.8%). The opposite was shown when SureStatus testing was performed more than 7 days post-symptom onset, although the sample size was smaller and the confidence intervals overlapped (sensitivity in India, 76.9%; 95% CI: 49.7% to 91.8%; sensitivity in Germany, 54.4%; 95% CI: 28.0% to 78.7%).
Out of the 205 total RT-PCR-positive cases, 27 were asymptomatic high-risk contacts. Within this group, the sensitivity of the Ag-RDT was 74.1% (20/27; 95% CI: 55.3% to 86.8%), which was lower than that in the symptomatic participants, 84.2% (149/177; 95% CI: 78.1% to 88.8%). The mean CT value in the asymptomatic participants was 22.0 (IQR, 16.5 to 26.0), versus 20.7 (IQR, 17.1 to 24.0) in the symptomatic participants.
The high interrater reliability, measured by a kappa result of 1.0, indicates that the test results are clearly interpretable, and no discrepancies were experienced between readers.
Ease-of-use assessment.
The results of the ease-of-use (EoU) assessment and the system usability scale (SUS) questionnaire are summarized in Fig. 4. SureStatus was scored at 85 out of 100 points on the SUS, indicating a test that is easy to use. Laboratory staff indicated difficulty with applying the required 12 drops to the proprietary test tube. In addition, the handling of the buffer solution, meaning squeezing to apply an appropriate amount (three drops) onto the cassette-formatted test device, was considered tedious (Fig. 4). Further, issues were encountered with the quality of the proprietary swabs included in the test device box. Several swabs, only at the German testing sites, were not sterile, as contamination could be seen with the naked eye, and the swabs had to be discarded (on average, five swabs per testing kit). This problem was not experienced at the Indian testing site. In addition, the swabs were often significantly bent in shape and therefore unsuitable for sample collection. These swabs were discarded and not used.
FIG 4.
System usability scale questionnaire and ease-of-use assessment results.
DISCUSSION
This prospective multicenter clinical diagnostic accuracy study in representative high- and limited-resource settings shows that the SureStatus is a well-performing test with an overall sensitivity of 82.4% and a specificity of 98.5% compared to the reference standard, RT-PCR. The test is easy-to-use and feasible in point-of-care settings. This, combined with the low production costs and high production capacities, justifies the large-scale effort to make the test accessible.
Comparing the accuracy to other lateral-flow assays for SARS-CoV-2, the results were on par with those of the best performing tests (3). However, there were substantial differences observed in the performance between the two countries. One possible explanation for the lower performance could be differences in the population. In India, a higher number of participants presented without symptoms and in the second week of disease. Furthermore, the low performance in the group with <7 days of symptoms suggests underreporting or possibly different perceptions of the duration of symptoms in different cultures. Also, the impact of environmental conditions (i.e., high temperature and high humidity) could be considered (8, 9). Of note, FIND repeatedly observed lower test sensitivity in the evaluations performed at the study site in India. A root-cause analysis exercise, which included sampling and testing parameters, was conducted and did not return a conclusive cause for the lower performance compared to that at the other study sites. The higher temperature and humidity conditions observed in India are also considered to have contributed to the mixed performance (10). Furthermore, we acknowledge that different test systems were used for PCR testing, and variability between the test systems might have contributed to the differences between sites. Calibration against standards was not possible (11). Nevertheless, with a performance of 90.7% within the first 7 days post-symptom onset, the majority of transmission-relevant SARS-CoV2 infections are likely to be detected by the test, supporting recent published literature (3). The lower performance in the asymptomatic participants is in line with other reports (3) and likely due to the fact that more asymptomatic participants are captured in the second half of their illness, as viral kinetics and thus CT values in the first week of infection between asymptomatic and symptomatic adults are expected to be the same if tested at the same time (12).
Given that the evaluation was performed when mostly the Alpha variant was circulating (13), the question remains to what extent the findings can be translated to novel variants. While comparable results have been shown for Delta, data are also now emerging for Omicron that suggest that Ag-RDTs targeting nucleocapsid protein remain sensitive (14).
Considering the test’s ease of use and rapid turnaround time, along with its high specificity, it could be considered for several use cases on a global scale at low cost. (i) Mass screening, (ii) entry-testing to protect (e.g., in high-risk settings such as hospitals), (iii) testing to release (e.g., contact testing), and (iv) testing to enable (e.g., regular school or workplace testing) have been suggested in different studies, in addition to (v) the use in symptomatic patients when RT-PCR is not available, such as in low-resource settings, or in combination with RT-PCR, when a rapid decision is necessary. Furthermore, given that self-sampling from the anterior nose (AN) is a reliable alternative to professional nasopharyngeal sampling, scale-up of testing could be possible without requiring large numbers of trained health care workers, provided the result obtained with combined nasopharyngeal and oropharyngeal (NP/OP) sample collection here can be confirmed with AN sampling (15, 16).
Overall, our study has several strengths. Primarily, the population was enrolled in two representative settings in India and Germany, with a population enrolled that showed a broad spectrum of clinical disease (from asymptomatic with high-risk contacts to severely ill). SureStatus was performed at point of care (POC), thus mimicking the real-world challenges of POC testing in three different settings with different levels of resources. The ease-of-use assessment highlighted important points for operationalization of the test and suitability of use in low-resource settings.
However, the study also had several limitations. First, the prevalence of positive cases, and the percentage of symptomatic participants varied substantially between the countries. This likely reflects different phases of the pandemic but also could relate to differences in patient behavior and recommendations for testing. Furthermore, the differences in the reference standard, PCR, introduced limitations to the interpretability of the analysis by CT value. We also acknowledge that the RT-PCR reference standard has its limitations, as it is not always a meaningful test when considering viable virus and risk of transmission. Lastly, our study excluded vaccinated individuals and those with prior infections, given the uncertainty around the possibility of breakthrough infections at the time of enrollment. Lower viral loads in the case of breakthrough infections, especially in the second week, are likely to diminish the sensitivity of SureStatus in a vaccinated population overall; however, the tests should continue to detect the individuals with the highest viral loads, who are most likely to transmit the virus (12, 17).
In summary, the favorable ease-of-use results, low production costs, and limited infrastructure required for the Ag-RDT testing procedure, in addition to the high sensitivity for infections in the first week of illness, can empower control of population transmission on a global scale, if implemented in well-designed testing and screening programs (18–20).
MATERIALS AND METHODS
Ethics statement.
This study was approved by the ethics committees of Heidelberg University Hospital (registration number S-180/2020) and Charité University Hospital, Berlin (EA1/371/20) in Germany and by the Institutional Ethics Committee for Bio Medical Research of Apollo Hospitals, Chennai, India (IEC application number AMH-C-S-032/09-20). The study was registered in the German Clinical Trial Registry (DRKS00021).
Clinical diagnostic accuracy.
This study is reported following the Standards for Reporting Diagnostic Accuracy (STARD) (21).
(i) Study design and participants. This manufacturer-independent study was conducted in partnership with FIND, the WHO collaborating center for coronavirus disease 2019 (COVID-19) diagnostics. The study was conducted in India at a tertiary-care hospital in Chennai (Apollo Hospitals) and in Germany at two sites: (i) in Heidelberg at a SARS-CoV-2 drive-in testing center managed by the local health department (Rhein-Neckar Region) and (ii) in Berlin at an ambulatory SARS-CoV-2 testing facility of the Charité University Hospital. Participants were eligible for enrollment if aged ≥18 years, determined to be at risk for a SARS-CoV-2 infection by the local health department based on proven contact with a confirmed SARS-CoV-2 case, or having symptoms suggestive of infection, in accordance with WHO criteria (see Table S2 in the supplemental material). Exclusion criteria for the study were as follows: prior positive RT-PCR test for SARS-CoV-2 at any time during the pandemic, vaccination against SARS-CoV-2, inability to provide written informed consent due to limited knowledge of language (in Germany, German or English; in India, English or Tamil), hemodynamic instability, inability to provide a respiratory sample, and recent history of excessive nose bleeds.
(ii) Study procedures. Individuals presenting for routine testing and meeting the inclusion criteria were invited to participate in the study. After providing written informed consent, participants first underwent a routine swab for RT-PCR, directly followed by the study-specific swab for Ag-RDT testing, performed by trained study teams. Sampling for RT-PCR testing was performed with a nasopharyngeal (NP) swab in Heidelberg and India and a combined NP and oropharyngeal (OP) swab in Berlin, as per institutional procedure. Sampling for the Ag-RDT SureStatus was conducted using an NP swab; however, if NP swabbing was contraindicated for clinical reasons (e.g., risk of bleeding), an OP swab was performed. Laboratory personnel working on both the Ag-RDT testing team and the RT-PCR laboratory were blinded to the results of the other test.
(iii) RT-PCR testing. The samples collected for routine RT-PCR testing were stored in the provided Amies solution and processed in the referral laboratories following the established laboratory protocols. The RT-PCR assays used as reference standards were the Thermo Fisher TaqPath COVID-19 combo kit PCR on the Applied Biosystems platform (Waltham, MA, USA) in Chennai, India, the Allplex SARS-CoV-2 assay from Seegene (Seoul, South Korea) in Heidelberg, and the Roche Cobas SARS CoV-2 assay (Pleasanton, CA, USA) on the Cobas 6800 or 8800 system or the SARS CoV-2 assay from TIB Molbiol (Berlin, Germany) in Berlin. The Cobas assays apply multiplex RT-PCR with two genomic targets (the ORF1a and E genes), and the TIB Molbiol assay targets the SARS-CoV-2 E gene region. Both test systems comprise internal processing controls to test for inhibition during RNA purification and RT-PCR, as well as positive and negative controls to monitor the overall test performance. A sample was considered positive for SARS-CoV-2 RNA if a specific signal for one, or both in the case of the Cobas systems, was measured and all internal and external controls were valid. More specification (specificity, target specific technical limits of detection [LoD], etc.) of these CE-labeled commercial tests are given in the manufacturer’s instructions.
(iv) Test evaluated. The test evaluated was the SureStatus COVID-19 antigen card test (Premier Medical Corporation, Mumbai, India). SureStatus relies on the principle of lateral-flow rapid chromatographic immunoassay for detection of the SARS-CoV-2 nucleocapsid protein antigen in a cassette-based format. The test kit includes all required reagents and proprietary swabs for NP/OP sample collection, and the manufacturer’s instructions for use (IFU) were followed during the sampling and testing procedures. As indicated in the IFU, 12 drops of extraction buffer are added to a proprietary tube, and the swab is inserted and swirled 5 to 10 times. While squeezing the sides, the swab is removed, and three drops of the specimen are applied to the test device through a nozzle cap. Colloidal gold conjugated antinucleocapsid antibodies on the membrane strip bind to viral antigens, forming an antibody-antigen complex and generating a color change on the test strip that can be interpreted visually after 15 to 20 min.
(v) Ag-RDT testing. Ag-RDT testing was performed in immediate proximity to the sample collection for routine RT-PCR testing. The laboratory workstations were kept clean, contaminated materials were kept separate, and the laboratory personnel were trained in handling infectious material. The temperature and humidity at the testing sites were recorded daily. The Ag-RDT test was conducted immediately after sample collection by trained laboratory personnel and interpreted by eye after 15 min. Two readers interpreted each test, both blinded to the results of the other. In case of discrepant results, both readers reinterpreted the results and agreed on one final result. Invalid test results were repeated once with the remaining buffer solution.
(vi) Clinical data collection. All participants were asked to provide clinical information about their symptoms, symptom duration, and severity of disease (questionnaire available in Table S2). In Heidelberg, the participants indicated their contact preferences during enrollment and after leaving the drive-in testing site, were contacted by phone or email to complete the questionnaire. In Berlin and India, the questionnaire was completed on-site by a study team member prior to sample collection.
Data management.
All data were collected and managed using Research Electronic Data Capture (REDCap) tools hosted at Heidelberg University for the Heidelberg and Berlin study sites (22) and the OpenClinica 4 system (Waltham, MA) for the study site in Chennai, India.
System usability scale and ease-of-use assessment.
A standardized system usability scale (SUS) questionnaire and an ease-of-use (EoU) assessment were designed for this study to evaluate the usability and feasibility of the test and can be found in Questionnaires S3 and S4 in the supplemental material (23). Laboratory personnel from all study sites were invited to complete both questionnaires. An overall SUS score of 68 was interpreted as average (23). A heat map was generated to analyze aspects related to the EoU assessment, categorizing each as satisfactory, average, or dissatisfactory (Fig. S5).
Statistical analysis.
For the primary analysis, the pooled sensitivity and specificity of the Ag-RDTs were calculated using a fixed-effects model by comparing the Ag-RDT results to RT-PCR as the reference standard. The 95% confidence intervals (CIs) were calculated using Wilson’s method. Subgroup analyses combined data from all sites and included symptom duration (≥7 or <7 days, ≥3 or <3 days). Invalid Ag-RDT results were reported separately. Interoperator variability was assessed using Cohen’s κ statistic to calculate the agreement of positive and negative results between the two independent readers. The analysis was conducted using the statistical software R version 4.03 (R Foundation for Statistical Computing, Vienna, Austria).
For the usability assessment, the SUS score was calculated using the mean value of all answers for each test. For the EoU assessment, responses were scored on a predefined numerical scale, and the mean values were summarized in a heat map. Both assessments were analyzed using Microsoft Excel.
Data availability.
The deidentified raw data can be requested by contacting the corresponding author.
Supplementary Material
ACKNOWLEDGMENTS
This study was substantially supported by Angelika Sandritter and the KTS company. Further funding support came from Ministry of Science, Research and Arts, State of Baden-Wuerttemberg, Germany, internal funds from the Heidelberg University Hospital and Charité University Hospital Berlin, the UK Department of International Development, WHO, and Unitaid. The testing devices and all components were procured by FIND. FIND also provided input on study design and data analysis in the form of an academic exchange with the rest of the study group. T.C.J. is in part funded through NIAID-NIH CEIRS contract HHSN272201400008C.
Charité University Hospital Berlin is a corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin.
The manufacturer and study funders had no input into the study protocol, analysis, or interpretation of results. All authors had access to all data at all times. We declare no competing interests.
J.A.S., C.M.D., N.R.P., and L.J.K. designed the study. L.J.K., S.S., and C.M.D. wrote the manuscript, L.J.K., J.K., and C.M.D. supervised the study site in Heidelberg, O.N. managed all data, F.T. conducted the analysis, F.L. managed the SUS questionnaire and EoU assessment, P.S. conducted RT-PCR testing in Heidelberg, A.K.L. and F.P.M. supervised the study site in Berlin, O.N. was responsible for the Ag-RDT testing, V.M.C. and T.C.J. conducted RT-PCR testing in Berlin, B.K. and A.W. provided all resources in Heidelberg, J.A.S. and M.D.V. supported the study design and manuscript writing, S.O. and S.W. wrote the R code and conducted the meta-analysis, N.S., J.S., H.S., A.P., and M.T. supervised the Indian study site and conducted the RT-PCR testing, and P.L. supported the Indian team. C.M.D. made the final decision to submit the manuscript for publication.
The study team comprised the following members: Kholoud Assaad, Andrea Fuhs, Christopher Harter, Cristopher Schulze, and Gunter Schmitt (Department of Public Health Rhein Neckar Region, Heidelberg, Germany); Martina Fink, Maximilian Schirmer, Annika Small, Matthias Meinlschmidt, Valerie Dürr, Alina Schuckert, Salome Steinke, Henrik Ellinghaus, Alexander Penning, and Loai Abutaima (Division of Infectious Disease and Tropical Medicine, Heidelberg University Hospital, Heidelberg, Germany) Mandy Kollatzsch, Mia Wintel, Franka Kausch, Franziska Hommes, Alisa Bölke, Julian Bernhard, Claudia Hülso, and Elisabeth Linzbach (Institute of Tropical Medicine and International Health, Charité—Universitätsmedizin Berlin, Berlin, Germany); Heike Rössig (Medical Directorate, Charité—Universitätsmedizin Berlin, Berlin, Germany); Maximilian Gertler (Institute of Tropical Medicine and International Health, Charité—Universitätsmedizin Berlin, Berlin, Germany); Susen Burock (Charité Comprehensive Cancer Center, Charité—Universitätsmedizin Berlin, Berlin, Germany); Katja von dem Busche (Department of Pediatric Surgery, Charité—Universitätsmedizin Berlin, Berlin, Germany); and Stephanie Patberg (Berlin Institute for Clinical Teratology and Drug Risk Assessment in Pregnancy, Institute of Clinical Pharmacology and Toxicology, Charité—Universitätsmedizin Berlin, Berlin, Germany).
Footnotes
Supplemental material is available online only.
[This article was published on 6 September 2022 with an error in the abstract. The abstract was corrected in the current version, posted on 27 September 2022.]
Contributor Information
Claudia M. Denkinger, Email: claudia.denkinger@uni-heidelberg.de.
Loai Abutaima, Department of Public Health Rhein Neckar Region, Heidelberg, Germany.
Elisabeth Linzbach, Institute of Tropical Medicine and International Health, Charité—Universitätsmedizin Berlin, Berlin, Germany.
Heike Rössig, Medical Directorate, Charité—Universitätsmedizin Berlin, Berlin, Germany.
Maximilian Gertler, Institute of Tropical Medicine and International Health, Charité—Universitätsmedizin Berlin, Berlin, Germany.
Susen Burock, Charité Comprehensive Cancer Center, Charité—Universitätsmedizin Berlin, Berlin, Germany.
Katja von dem Busche, Department of Pediatric Surgery, Charité—Universitätsmedizin Berlin, Berlin, Germany.
Stephanie Stephanie, Berlin Institute for Clinical Teratology and Drug Risk Assessment in Pregnancy, Institute of Clinical Pharmacology and Toxicology, Charité—Universitätsmedizin Berlin, Berlin, Germany.
Heba H. Mostafa, Johns Hopkins Hospital
Collaborators: Kholoud Assaad, Andrea Fuhs, Christopher Harter, Cristopher Schulze, Gunter Schmitt, Martina Fink, Maximilian Schirmer, Annika Small, Matthias Meinlschmidt, Valerie Dürr, Alina Schuckert, Salome Steinke, Henrik Ellinghaus, Alexander Penning, Loai Abutaima, Mandy Kollatzsch, Mia Wintel, Franka Kausch, Franziska Hommes, Alisa Bölke, Julian Bernhard, Claudia Hülso, Elisabeth Linzbach, Heike Rössig, Maximilian Gertler, Susen Burock, Katja von dem Busche, and Stephanie Stephanie
REFERENCES
- 1.World Health Organization. 6 October 2021. Antigen-detection in the diagnosis of SARS-CoV-2 infection. World Health Organization, Geneva, Switzerland. https://www.who.int/publications/i/item/antigen-detection-in-the-diagnosis-of-sars-cov-2infection-using-rapid-immunoassays. Accessed 30 March 2022.
- 2.Denkinger CM, Brümmer LE, Katzenschlager S, Schmitz S. 31 August 2021. Diagnostics global health: rapid antigen tests for the diagnosis of a SARS-CoV-2 infection. Heidelberg, Germany. https://diagnosticsglobalhealth.org/. Accessed 30 March 2022.
- 3.Brümmer LE, Katzenschlager S, Gaeddert M, Erdmann C, Schmitz S, Bota M, Grilli M, Larmann J, Weigand MA, Pollock NR, Macé A, Carmona S, Ongarello S, Sacks JA, Denkinger CM. 2021. Accuracy of novel antigen rapid diagnostics for SARS-CoV-2: a living systematic review and meta-analysis. PLoS Med 18:e1003735. doi: 10.1371/journal.pmed.1003735. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Foundation of Innovative New Diagnostics. 2022. About the ACT-Accelerator. Foundation of Innovative New Diagnostics, Geneva, Switzerland. https://www.finddx.org/covid-19/act-accelerator/. Accessed 30 March 2022.
- 5.World Health Organization. 30 April 2021. WHO emergency use listing for in vitro diagnostics (IVDs) detecting SARS-CoV-2. World Health Organization, Geneva, Switzerland. https://extranet.who.int/pqweb/sites/default/files/documents/210430_EUL_SARS-CoV-2_product_list.pdf. Accessed 30 March 2022.
- 6.Foundation of Innovative New Diagnostics. 1 January 2022. SARS-COV-2 diagnostic pipeline. Foundation of Innovative New Diagnostics, Geneva, Switzerland. https://www.finddx.org/covid-19/pipeline/. Accessed 30 March 2022.
- 7.Rapid Microbiology. 5 February 2021. WHO give large contract for low-cost rapid COVID-19 tests to Indian manufacturer. Rapid Microbiology, Clonakilty, Ireland. https://www.rapidmicrobiology.com/news/who-give-large-contract-for-low-cost-rapid-covid-19-tests-to-indian-manufacturer. Accessed 30 March 2022.
- 8.Haage V, Ferreira de Oliveira-Filho E, Moreira-Soto A, Kühne A, Fischer C, Sacks JA, Corman VM, Müller MA, Drosten C, Drexler JF. 2021. Impaired performance of SARS-CoV-2 antigen-detecting rapid diagnostic tests at elevated and low temperatures. J Clin Virol 138:104796. doi: 10.1016/j.jcv.2021.104796. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Abdul-Mumin A, Abubakari A, Agbozo F, Abdul-Karim A, Nuertey BD, Mumuni K, Heuschen A-K, Hennig L, Denkinger CM, Müller O, Jahn A. 2021. Field evaluation of specificity and sensitivity of a standard SARS-CoV-2 antigen rapid diagnostic test: a prospective study at a teaching hospital in Northern Ghana. PloS Glob Public Health 1:e0000040. doi: 10.1371/journal.pgph.0000040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Foundation of Innovative New Diagnostics. 2022. FIND evaluation of SARS-CoV-2 antigen (Ag) detecting tests. Foundation of Innovative New Diagnostics, Geneva, Switzerland. https://www.finddx.org/sarscov2-eval-antigen/. Accessed 30 March 2022.
- 11.Matheeussen V, Corman VM, Donoso Mantke O, McCulloch E, Lammens C, Goossens H, Niemeyer D, Wallace PS, Klapper P, Niesters HG, Drosten C, Ieven M, RECOVER Project . 2020. International external quality assessment for SARS-CoV-2 molecular detection and survey on clinical laboratory preparedness during the COVID-19 pandemic, April/May 2020. Euro Surveill 25:2001223. doi: 10.2807/1560-7917.ES.2020.25.27.2001223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Jones TC, Biele G, Mühlemann B, Veith T, Schneider J, Beheim-Schwarzbach J, Bleicker T, Tesch J, Schmidt ML, Sander LE, Kurth F, Menzel P, Schwarzer R, Zuchowski M, Hofmann J, Krumbholz A, Stein A, Edelmann A, Corman VM, Drosten C. 2021. Estimating infectiousness throughout SARS-CoV-2 infection course. Science 373:eabi5273. doi: 10.1126/science.abi5273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Lindner AK, Krüger LJ, Nikolai O, Klein JAF, Rössig H, Schnitzler P, Corman VM, Jones TC, Tobian F, Gaeddert M, Burock S, Sacks JA, Seybold J, Mockenhaupt FP, Denkinger CM. 2021. SARS-CoV-2 variant of concern B.1.1.7: diagnostic accuracy of three antigen-detecting rapid tests. Microbiol Spectr 10:e0076321. doi: 10.1128/spectrum.00763-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Soni A, Herbert C, Filippaios A, Broach J, Colubri A, Fahey N, Woods K, Nanavati J, Wright C, Orwig T, Gilliam K, Kheterpal V, Suvarna T, Nowak C, Schrader S, Lin H, O'Connor L, Pretz C, Ayturk D, Orvek E, Flahive J, Lazar P, Shi Q, Achenbach C, Murphy R, Robinson M, Gibson L, Stamegna P, Hafer N, Luzuriaga K, Barton B, Heetderks W, Manabe YC, McManus D. 2022. Comparison of rapid antigen tests’ performance between Delta (B.1.61.7; AY.X.) and Omicron (B.1.1.529; BA1) variants of SARS-CoV-2: secondary analysis from a serial home self-testing study. medRxiv. doi: 10.1101/2022.02.27.22271090. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Lindner AK, Nikolai O, Rohardt C, Burock S, Hülso C, Bölke A, Gertler M, Krüger LJ, Gaeddert M, Tobian F, Lainati F, Seybold J, Jones TC, Hofmann J, Sacks JA, Mockenhaupt FP, Denkinger CM. 2021. Head-to-head comparison of SARS-CoV-2 antigen-detecting rapid test with professional-collected nasal versus nasopharyngeal swab. Eur Respir J 57:2004430. doi: 10.1183/13993003.04430-2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Abdulrahman A, Mustafa F, AlAwadhi AI, Alansari Q, AlAlawi B, AlQahtani M. 2020. Comparison of SARS-CoV-2 nasal antigen test to nasopharyngeal RT-PCR in mildly symptomatic patients. medRxiv. doi: 10.1101/2020.11.10.20228973. [DOI] [Google Scholar]
- 17.Levine-Tiefenbrun M, Yelin I, Katz R, Herzel E, Golan Z, Schreiber L, Wolf T, Nadler V, Ben-Tov A, Kuint J, Gazit S, Patalon T, Chodick G, Kishony R. 2021. Initial report of decreased SARS-CoV-2 viral load after inoculation with the BNT162b2 vaccine. Nat Med 27:790–792. doi: 10.1038/s41591-021-01316-7. [DOI] [PubMed] [Google Scholar]
- 18.Larremore DB, Wilder B, Lester E, Shehata S, Burke JM, Hay JA, Tambe M, Mina MJ, Parker R. 2021. Test sensitivity is secondary to frequency and turnaround time for COVID-19 screening. Sci Adv 7:eabd5393. doi: 10.1126/sciadv.abd5393. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.van Beek J, Igloi Z, Boelsums T, Fanoy E, Gotz H, Molenkamp R, van Kampen J, GeurtsvanKessel C, van der Eijk AA, van de Vijver D, Koopmans M. 2020. From more testing to smart testing: data-guided SARS-CoV-2 testing choices, the Netherlands, May to September 2020. Euro Surveill 27:2100702. doi: 10.2807/1560-7917.ES.2022.27.8.2100702. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Paltiel AD, Zheng A, Walensky RP. 2020. Assessment of SARS-CoV-2 screening strategies to permit the safe reopening of college campuses in the United States. JAMA Netw Open 3:e2016818. doi: 10.1001/jamanetworkopen.2020.16818. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, Lijmer JG, Moher D, Rennie D, de Vet HCW, Kressel HY, Rifai N, Golub RM, Altman DG, Hooft L, Korevaar DA, Cohen JF, STARD Group . 2015. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ 351:h5527. doi: 10.1136/bmj.h5527. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. 2009. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42:377–381. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Bangor A, Kortum PT, Miller JT. 2008. An empirical evaluation of the system usability scale. Int J Hum Comput Interact 24:574–594. doi: 10.1080/10447310802205776. [DOI] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental material. Download spectrum.01229-22-s0001.pdf, PDF file, 1.0 MB (952.4KB, pdf)
Data Availability Statement
The deidentified raw data can be requested by contacting the corresponding author.