Abstract
Background & objectives:
Coronavirus disease 2019 (COVID-19) has so far affected over 41 million people globally. The limited supply of real-time reverse transcription-polymerase chain reaction (rRT-PCR) kits and reagents has made meeting the rising demand for increased testing incompetent, worldwide. A highly sensitive and specific antigen-based rapid diagnostic test (RDT) is the need of the hour. The objective of this study was to evaluate the performance of a rapid chromatographic immunoassay-based test (index test) compared with a clinical reference standard (rRT-PCR).
Methods:
A cross-sectional, single-blinded study was conducted at a tertiary care teaching hospital in north India. Paired samples were taken for RDT and rRT-PCR (reference standard) from consecutive participants screened for COVID-19 to calculate the sensitivity and specificity of the RDT. Further subgroup analysis was done based on the duration of illness and cycle threshold values. Cohen's kappa coefficient was used to measure the level of agreement between the two tests.
Results:
Of the 330 participants, 77 were rRT-PCR positive for SARS-CoV-2. Sixty four of these patients also tested positive for SARS-CoV-2 by RDT. The overall sensitivity and specificity were 81.8 and 99.6 per cent, respectively. The sensitivity of RDT was higher (85.9%) in participants with a duration of illness ≤5 days.
Interpretation & conclusions:
With an excellent specificity and moderate sensitivity, this RDT may be used to rule in COVID-19 in patients with a duration of illness ≤5 days. Large-scale testing based on this RDT across the country would result in quick detection, isolation and treatment of COVID-19 patients.
Keywords: Antigen test, COVID-19, point-of-care test, SARS-CoV-2
In December 2019, a new coronavirus disease emerged in Wuhan, China, and rapidly spread throughout the world. Now formally called coronavirus disease 2019 (COVID-19), the causative virus has been named severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)1. For diagnosis, the SARS-CoV-2 genomic RNA is detected from upper and lower respiratory specimens2. The growing COVID-19 pandemic has led to a global crisis and crunch of laboratory-based molecular testing capacity and reagents3. This is especially true for developing countries, with a scarcity of health-resources. India currently has over 7.7 million cases of confirmed COVID-19 and the disease is rapidly spreading to smaller towns and villages4. The real-time reverse transcription-polymerase chain reaction (rRT-PCR) received emergency use authorization (EUA) by the Centers for Disease Control and Prevention (CDC) for the qualitative detection of SARS-CoV-2 nucleic acid from the respiratory specimens5. The rRT-PCR testing requires a sophisticated Biosafety level (BSL)-2/BSL-3 laboratory setup and trained technicians to run the test and interpret results. The rRT-PCR takes a minimum of 8-10 h from the collection of swab to reporting of results, which can further increase in resource-limited and high-burden settings. In small towns and cities, molecular diagnostic laboratories are non-existent and the reagents/viral transport medium (VTM) and resources are difficult to procure. Therefore, the need of the hour is to rapidly detect and isolate positive cases to contain the disease spread, to quickly triage patients with severe acute respiratory illness (SARI) in emergency departments (EDs) and to ramp up testing facilities. Many diagnostic test manufacturers are developing/have developed rapid diagnostic kits and devices to facilitate point-of-care testing. These sample kits are based on either antibody detection from blood/plasma/serum or the detection of SARS-CoV-2 antigens from respiratory samples.
However, there is very limited data on the performance and potential diagnostic utility of a rapid chromatographic immunoassay-based test for SARS-CoV-2 in suspected patients. Here, we report the evaluation of a rapid chromatographic immunoassay-based test for the qualitative detection of a specific antigen to SARS-CoV-2 in the nasopharyngeal (NP) swab for diagnosis of COVID-19 in India.
Material & Methods
A single-blinded, cross-sectional, single-centre study was conducted in a tertiary care, referral hospital in north India, following STARD 2015 guidelines for reporting diagnostic accuracy studies6 to evaluate the performance of a rapid chromatographic immunoassay-based test (index test) compared with a clinical reference standard (rRT-PCR).
Patient recruitment and clinical specimens: Patients eligible for inclusion were consecutive adults (>18 yr) with suspected COVID-19 infection, based on the Indian Council of Medical Research (ICMR) strategy for COVID-19 testing7. The following two types of patients were included: (i) patients symptomatic for COVID-19; and (ii) asymptomatic/pre-symptomatic contacts of laboratory-confirmed cases between 5 and 10 days of exposure.
The study was conducted at the 'COVID-19 screening and testing outpatient department (OPD)' at the 3000-bedded All India Institute of Medical Sciences (AIIMS), New Delhi, India, between May 31, 2020 and July 24, 2020. Almost 50 per cent beds have been allocated to COVID-19 care in the face of an increasing number of cases. All patients were evaluated in a consecutive manner at the Medicine out-patients department (OPD). Nasal and throat swabs were collected for rRT-PCR using nylon flocked swabs, and both the swabs were placed together in a 2 ml VTM tube for rRT-PCR; parallelly, NP samples were collected for the rapid diagnostic test (RDT). The rapid chromatographic immunoassay test was performed immediately in all the patients as per the manufacturer's instructions. A parallel sampling was done for rRT-PCR, and the sequence for specimen collection was random for both the samples. The samples for rRT-PCR were kept in an icebox at 4°C the laboratory. All suspected patients were advised to self-isolate themselves till the reporting of rRT-PCR results. Because rRT-PCR has the highest sensitivity for detection of SARS-CoV-2-specific gene targets, with the limit of detection (LOD) being as low as 0.91-3.1 copies/ml for different gene targets, it is considered a reference standard8.
The study was approved by the AIIMS Ethics Committee (IEC/537/5/2020) and informed consent was obtained from each patient.
rRT-PCR: This reference test was done on nasal and throat swabs collected in VTM and transported at 4-8°C as per the guidelines of the ICMR9. Total nucleic acid was extracted from the samples, using the MagMAX Viral Isolation Kit (Thermo Fisher Scientific, USA). A commercial rRT-PCR kit (BGI Genomics Co. Ltd., China, which has EUA from the US FDA and approval from the ICMR), was used to detect the SARS-CoV-2 ORF 1ab region of the genome, in an AriaMx real-time PCR instrument (Agilent, USA). The test also detects the human housekeeping gene β-actin as a control for confirming the adequacy of the sample, RNA extraction and rRT-PCR. The result was interpreted as positive or negative as per the manufacturer's instructions. The LOD of the kit was 100 copies/ml.
Rapid antigen detection test: The Standard Q rapid antigen detection test (SD Biosensor, Inc., Gurugram) was evaluated in this study. The test was conducted on an NP swab specimen. The RDT kit consisted of a sterile swab, viral extraction tubes with buffer, tube nozzles and a COVID-19 antigen test device.
Collection of specimens and antigen extraction: The test was conducted on an NP swab, and samples were taken from both sides of the nasopharynx using a swab provided with the kit to maximize the viral load in the sample. Before collecting samples, the patients were asked for a nose blow to remove excessive secretion. A sterile swab was inserted into the nasal cavity of the patient at an angle of 90° in a 50°-70° extended neck position to swab the surface of the posterior nasopharynx. The swab was kept in the nasopharynx for 5-10 sec to properly absorb secretions and gently removed while rotating it. The swab was inserted into the tube containing the extraction buffer provided with the kit and stirred into the buffer 5-6 times before squeezing and discarded. A nozzle was placed tightly onto this extraction buffer tube. Three drops of the extracted specimen were put onto the specimen well of the test device and was set aside.
Interpretation of results: The test results were read after 15-30 minutes. The test device develops red bands at two positions: 'C' control line and 'T' test line - SARS-CoV-2 antigen. If red bands appeared at the 'C' and 'T' positions, the test was interpreted as positive. All red bands, including the faint ones, were taken as positive results. If the red band appeared at only the 'C' position, it was interpreted as a negative result. The test was considered invalid if no red band appeared at the 'C' position and was repeated.
Statistical analysis: Data were recorded on a pre-designed proforma. Diagnostic characteristics such as sensitivity and specificity of the test with rRT-PCR as reference were calculated. Positive and negative predictive values of the test were also computed for both overall and various levels of pre-test probabilities (i.e. duration of illness ≤ 5 days, >5 days and asymptomatic). The rapid diagnostic test was also evaluated for the subgroups considering the viral load and days since infection. Agreement between RDT diagnosis of COVID-19 and rRT-PCR was evaluated using Cohen's kappa calculation (κ < 0.40, poor agreement; 0.40 ≤ κ < 0.60, moderate agreement; 0.60≤ κ <0.80, good agreement and κ ≥ 0.80, excellent agreement). For each of the summary measures, a 95 per cent confidence interval (CI) was also computed. Stata 14.0 statistical software (StataCorp LLC, TX, USA) was used for data analysis.
Results
A total of 990 swabs (one nasal and one throat swab for rRT-PCR and one NP swab for RDT) were collected from 330 participants during the study period. The median age of the study participants was 34.1±12.6 yr (231 males and 99 females, with a sex ratio of 0.42). According to rRT-PCR results (Table I), 77 were positive for SARS-CoV-2 RNA, with a mean cycle threshold (Ct) value of 21.4±5.0 (mean±SD, range: 10-35.4). Sixty four (83.1%) participants who tested positive in rRT-PCR, presented with symptoms suggestive of COVID-19, while 15.5 per cent (13/77) of the participants were asymptomatic. The median duration of illness at the time of testing among symptomatic patients was one day (range: 1-10). The most commonly presented symptoms among the screened participants were fever (31.5%) followed by cough (25.4%), fatigue/malaise (11.8%), headache (3.3%) and runny nose (3.3%), and 57 participants presented with the complaint of a sore throat but only two of them (3.5%) had COVID-19.
Table I.
Baseline characteristics of the study participants (n=330)
| Demography | |
|---|---|
| Age, yr (mean±SD) | 34.1±12.6 |
| Sex | |
| Male | 231 (70.0) |
| Female | 99 (30.0) |
| Ratio (female/male) | 0.42 |
| Symptomatic | 204 (61.8) |
| Asymptomatic | 126 (38.1) |
| Duration of illness (n=179) | |
| ≤5 days | 192 (58.1) |
| >5 days | 12 (3.6) |
| Clinical features | |
| Fever | 104 (31.5) |
| Cough | 84 (25.4) |
| Sore throat | 78 (23.6) |
| Fatigue/malaise | 39 (11.8) |
| Headache | 11 (3.3) |
| Runny nose | 11 (3.3) |
Values shown as n (%)
The rapid chromatographic immunoassay based RDT was positive in 64 (19.3%) and negative in 266 (80.6%) participants. Overall, among the positive test results (Table II), the rapid antigen test detected 63 true positives (19.0%) and gave one false-positive result, with respect to the reference standard. Among negative test results, 252 (76.3%) were true negatives and 14 (4.2%) were false negatives. The overall sensitivity and specificity of the test were 81.8 and 99.6 per cent, respectively, and the test accuracy was 95.4 per cent. The disease prevalence in the tested participants was 23.3 per cent with a kappa coefficient of 0.86 and an agreement of 95 per cent between both the tests. The likelihood ratio (LR) for positive test results was 207.0, and 0.18 for negative test results. The positive predictive value of the test was 98.4 per cent, and the negative predictive value was 94.7 per cent. The mean Ct value of truly positive RDT-positive samples was 21.1+4.8 (mean+SD, range: 10-35.4) and that of false-negative RDT samples was 25.8±5.0 (mean±SD, range: 15-34.1, P=0.0017). The sensitivity in participants with a duration of illness ≤ 5 days was 85.9 per cent (95% CI: 74.2-93.7).
Table II.
Diagnostic characteristics of rapid diagnostic test with reference to rRT-PCR: Overall and subgroup analysis
| Diagnostic characteristics | Overall | Subgroups | ||
|---|---|---|---|---|
| Duration of illness ≤5 days (n=191) | Duration of illness >5 days (n=12) | Asymptomatic (n=127) | ||
| TP | 63 | 49 | 5 | 9 |
| TN | 252 | 134 | 5 | 113 |
| FP | 1 | 0 | 0 | 1 |
| FN | 14 | 8 | 2 | 4 |
| Ct values (mean±SD) (minimum-maximum) | 21.4±5.0 (10-35.4) | 21.1±4.8 (10-35.4) | 25.1±4.8 (15.1-30.4) | 20.7±5.0 (16.9-34.1) |
| Sensitivity % (95% CI) | 81.8 (71.3-89.6) | 85.9 (74.2-93.7) | 71.4 (29.0-96.3) | 69.2 (38.5-90.9) |
| Specificity % (95% CI) | 99.6 (97.8-99.9) | 100 (97.2-100) | 100 (47.8-100) | 99.1 (95.2-99.9) |
| PPV % (95% CI) | 98.4 (88.8-99.7) | 100 | 100 | 90.0 (55.2-98.4) |
| NPV % (95% CI) | 94.7 (91.8-96.6) | 94.3 (89.7-96.9) | 71.4 (43.6-89.0) | 96.0 (92.5-98.4) |
| Accuracy % (95% CI) | 95.4 (92.6-97.4) | 95.8 (91.8-98.2) | 83.3 (51.6-97.9) | 96.0 (91.0-98.7) |
| LR+ (95% CI) | 207.0 (29.1-1468.0) | CNC | CNC | 78.9 (10.5-574.3) |
| LR− (95% CI) | 0.18 (0.11-0.29) | 0.14 (0.07-0.27) | 0.28 (0.08-0.92) | 0.31 (0.14-0.70) |
CI, confidence interval; Ct, cycle threshold; PPV, positive predictive value; NPP, negative predictive value; LR, likelihood ratio; CNC, could not be calculated (due to zero FP RDT result); RDT, rapid diagnostic test; TP, true positive; TN, true negative; FP, false positive; FN, false negative
Discussion
In the present study, the RDT was found to have an acceptable sensitivity of 81.8 per cent and a high specificity of 99.6 per cent. The analytical performance of RDT depends on the mixing of NP swab with buffer and the viral load in the sample, but the clinical performance of the test may be variable which depends on the technique of sample collection and the duration of illness of patients. Hence, the sensitivity will be average in asymptomatic patients because it is difficult to analyze the pre-test probability in asymptomatic patients.
Recently, Porte et al10 reported the evaluation of similar fluorescence immunochromatographic SARS-CoV-2 rapid antigen test (Bioeasy Biotechnology Co., China) using universal transport medium with NP and oropharyngeal swabs in 127 suspected COVID-19 cases. The overall sensitivity and specificity were 93 and 100 per cent, respectively, but a pre-print study by the manufacturer of the kit reported an overall sensitivity of 68 per cent and specificity of 100 per cent in NP swabs11.
The RDT in the present study showed a high positive LR of 207.0. This indicates a 207-fold increase in the odds of having infection with SARS-CoV-2 in participants with positive RDT results. Similarly, the negative LR for the RDT was 0.18, which means that the odds of having SARS-CoV-2 infection had decreased by 5.5-fold after a negative RDT result. However, to understand the utility of RDT, case-based estimation of pre-test probability is essential which depends on SARS-CoV-2 prevalence in the population, history of contact with the positive case and signs and symptoms in suspects (Fig. 1). The post-test probability calculated with the help of LR and pre-test probabilities, which turn out to be 98 per cent for a positive test and 5 per cent for a negative test result, would mean that 98 per cent times SARS-CoV-2 is present if the RDT is tested positive. In 5 per cent of cases, SARS-CoV-2 is present if RDT is tested negative. The application of LR in clinical settings requires an estimate of pre-test probability, which is often subjective, and the estimation of pre-test probability must be individualized and tailored to the suspect's symptoms. This is not easy to estimate in asymptomatic cases. Therefore, this test may not be good for surveillance purposes. This test can perform as well as RT-PCR in high-prevalence areas with high pre-test probability. Moderate sensitivity of the RDT leads to false-negative results, which must be taken into consideration while making diagnostic algorithms (Fig. 2).
Fig. 1.
A plot of the post-test probability against the pre-test probability of having COVID-19. The green and orange ribbons represent the 95% confidence interval around these values. The vertical line indicates the pre-test probability or prevalence of COVID-19 (23.3%). Where this vertical line cuts the green and orange lines, those points give the probabilities that if the result of the rapid diagnostic test is negative, then COVID-19 is absent, and if the result is positive, the disease is present.
Fig. 2.
A plot comparing cycle threshold (Ct) values of true-positive and false-negative cases and rRT-PCR-positive test results. The X-axis represents Ct values and the Y-axis represents the number of rRT-PCR-positive test results. The data table below the X-axis shows true-positive and false-negative rapid diagnostic test results.
The advantages of RDT such as yielding rapid results, being at a reasonable price and being safe due to viral inactivation and the fact that this does not require sophisticated laboratory set-up or technical expertise make it an ideal test to be rolled out in high-prevalence community settings. Based on these findings, this test has been adopted in the diagnostic algorithms for Indian hospitals and an advisory has been issued by the ICMR in this regard12. However, because negative results cannot rule out SARS-CoV-2 infection, all negative tests should be covered by rRT-PCR.
In India, and other populous developing nations facing a surge of cases, the rapid nature of this test has two main advantages: a positive patient can be immediately sent to a dedicated COVID-19 centre, which otherwise may be delayed by 2-3 days. Second, in the cramped EDs of many hospitals, while suspected patients are kept in a holding area until the PCR reports arrive, there is a high likelihood of COVID-19-negative SARI patients contracting COVID-19 due to cross-transmission. A rapid test may prevent that eventuality.
This study had several limitations. First, because participants were recruited from a screening OPD, they were either asymptomatic/pre-symptomatic or mildly symptomatic. The testing was not done in moderate-to-severe cases; therefore, it needs to be evaluated in this category of COVID-19 suspects. Second, the sensitivity of RDT was 81.8 per cent compared to a standard reference (rRT-PCR), but the standard reference itself had limited sensitivity in the initial test up to 83-89 per cent compared to the initial chest computed tomography. Thus, RDT can miss up to 25-30 per cent of COVID-19 cases13,14. Third, a positive RDT is not 100 per cent specific to SARS-CoV-2, as it shows cross-reactivity with SARS-CoV based on the analytical performance of the test provided by the kit manufacturer, but not evaluated further in the clinical setup15.
Our study had several strengths also. First, because the study was single blinded, those who collected swabs were unaware of the participant's clinical symptoms and rRT-PCR test report. Second, the NP swabs for RDT and nasal and throat swabs for rRT-PCR were collected simultaneously and randomly in some participants, and there was no time lag between the index and reference standard tests. Third, all participants underwent index and reference standard tests, so all positive and negative results were verified. Fourth, the test performance was also evaluated in asymptomatic contacts.
In conclusion, the rapid antigen test showed an excellent specificity to 'rule-in' COVID-19 patients within the first five days of illness and had a moderate sensitivity. Therefore, patients showing positive result need to immediately triaged and those with negative tests should be reconfirmed by an rRT-PCR.
Acknowledgment:
Authors thank Drs Thomas R Fanshawe, University of Oxford, Michael Power, Sara Graziadio, Newcastle upon Tyne Hospitals Foundation Trust, Newcastle upon Tyne, Joseé M Ordóñez-Mena, Nuffield Department of Primary Care Health Sciences, University of Oxford, John Simpson, Joy Allen, Newcastle University, Newcastle upon Tyne, UK, and British Medical Journal Evidence-Based Medicine for providing 'Clinical Accuracy and Utility' toolkit, for better visualization of pre- and post-test probability figures.
Footnotes
Financial support & sponsorship: This study was financially supported by the Indian Council of Medical Research, New Delhi (for the Regional Virus Research and Diagnostic Laboratory at the All India Institute of Medical Sciences, New Delhi).
Conflicts of Interest: None.
References
- 1.World Health Organization. Coronavirus disease 2019: Events as they happen. [accessed on June 29, 2020]. Available from: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/events-asthey-happen .
- 2.Wang W, Xu Y, Gao R, Lu R, Han K, Wu G, et al. Detection of SARS-CoV-2 in different types of clinical specimens. JAMA. 2020;323:1843–4. doi: 10.1001/jama.2020.3786. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.World Health Organization. Advice on the use of pointof- care immunodiagnostic tests for COVID-19: Scientific brief, 8 April 2020. [accessed on October 22, 2020]. Available from: https://apps.who.int/iris/handle/10665/331713 .
- 4.Government of India. India Fights Corona COVID-19. [accessed on July 14, 2020]. Available from: https://www.mygov.in/covid-19/
- 5.Centers for Disease Control and Prevention. CDC 2019-novel coronavirus (2019- nCoV) real-time RT-PCR diagnostic panel. Atlanta: CDC; 2020. [Google Scholar]
- 6.Cohen JF, Korevaar DA, Altman DG, Bruns DE, Gatsonis CA, Hooft L, et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open. 2016;6:e012799. doi: 10.1136/bmjopen-2016-012799. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Indian Council of Medical Research. Strategy for COVID-19 testing in India (Version 5, dated 18/05/2020) [accessed on July 14, 2020]. Available from: https://www.icmr.gov.in/pdf/covid/strategy/Testing_Strategy_v5_18052020.pdf .
- 8.Corman VM, Landt O, Kaiser M, Molenkamp R, Meijer A, Chu DK, et al. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Eurosurveillance. 2020;25:2000045. doi: 10.2807/1560-7917.ES.2020.25.3.2000045. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.ICMR- National Institute of Virology, Pune. Specimen collection, packaging and transport guidelines for testing 2019-Novel Coronavirus (2019-nCoV) (Version 1.0, dated 29/01/2020) [accessed October 22, 2020]. Available from: https://www.mohfw.gov.in/pdf/5Sample%20collection_packaging%20%202019-nCoV.pdf .
- 10.Porte L, Legarraga P, Vollrath V, Aguilera X, Munita JM, Araos R, et al. Evaluation of a novel antigen-based rapid detection test for the diagnosis of SARS-CoV-2 in respiratory samples. Int J Infect Dis. 2020;99:328–33. doi: 10.1016/j.ijid.2020.05.098. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Diao B, Wen K, Chen J, Liu Y, Yuan Z, Han C, et al. Diagnosis of Acute Respiratory Syndrome Coronavirus 2 Infection by Detection of Nucleocapsid Protein. medRxiv. 2020 doi: 10.1101/2020.03.07.20032524. [Google Scholar]
- 12.Indian Council of Medical Research. Advisory on use of rapid antigen detection test for COVID-19. [accessed on July 14, 2020]. Available from: https://www.icmr.gov.in/pdf/covid/strategy/Advisory_for_rapid_antigen_test14062020.pdf .
- 13.Kim H, Hong H, Yoon SH. Diagnostic performance of CT and reverse transcriptase polymerase chain reaction for coronavirus disease 2019: A meta-analysis. Radiology. 2020;296:E145–55. doi: 10.1148/radiol.2020201343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Long C, Xu H, Shen Q, Zhang X, Fan B, Wang C, et al. Diagnosis of the coronavirus disease (COVID-19): rRT-PCR or CT? Eur J Radiol. 2020;126:108961. doi: 10.1016/j.ejrad.2020.108961. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Standard Q COVID-19 Ag kit package insert and performance characteristics. [accessed on July 14, 2020]. Available from: http://sdbiosensor.com/xe/?module&file&act=procFileDownload&file_srl=18227&sid=34288e1ee19dcfe4303c44f79b8ffcfe&modu le_srl=513 .