Clinical evaluation of Sofia Rapid Antigen Assay for detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) among emergency department to hospital admissions

Abstract

Objective:

To determine the utility of the Sofia SARS rapid antigen fluorescent immunoassay (FIA) to guide hospital-bed placement of patients being admitted through the emergency department (ED).

Design:

Cross-sectional analysis of a clinical quality improvement study.

Setting:

This study was conducted in 2 community hospitals in Maryland from September 21, 2020, to December 3, 2020. In total, 2,887 patients simultaneously received the Sofia SARS rapid antigen FIA and SARS-CoV-2 RT-PCR assays on admission through the ED.

Methods:

Rapid antigen results and symptom assessment guided initial patient placement while confirmatory RT-PCR was pending. The sensitivity, specificity, positive predictive values, and negative predictive values of the rapid antigen assay were calculated relative to RT-PCR, overall and separately for symptomatic and asymptomatic patients. Assay sensitivity was compared to RT-PCR cycle threshold (Ct) values. Assay turnaround times were compared. Clinical characteristics of RT-PCR–positive patients and potential exposures from false-negative antigen assays were evaluated.

Results:

For all patients, overall agreement was 97.9%; sensitivity was 76.6% (95% confidence interval [CI], 71%–82%), and specificity was 99.7% (95% CI, 99%–100%). We detected no differences in performance between asymptomatic and symptomatic individuals. As RT-PCR Ct increased, the sensitivity of the antigen assay decreased. The mean turnaround time for the antigen assay was 1.2 hours (95% CI, 1.0–1.3) and for RT-PCR it was 20.1 hours (95% CI, 18.9–40.3) (P < .001). No transmission from antigen-negative/RT-PCR–positive patients was identified.

Conclusions:

Although not a replacement for RT-PCR for detection of all SARS-CoV-2 infections, the Sofia SARS antigen FIA has clinical utility for potential initial timely patient placement.

The pandemic of coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has affected people worldwide.¹ COVID-19 poses a public health threat that burdens the healthcare system, particularly hospitals. To best respond, diagnostic testing has been prioritized.² The ability of this virus to spread from individuals who are asymptomatic is especially problematic; thus, accurately diagnosing asymptomatic patients is of utmost importance.³ For this reason, admission testing of all hospitalized patients has become the standard at many US hospitals.

The current standard assay for SARS-CoV-2 is a reverse transcriptase polymerase chain reaction (RT-PCR) assay performed on a nasopharyngeal swab.⁴ These assays have several limitations including need for resources, trained laboratory personnel, and turnaround times that potentially exceed 24 hours.^5–7 Recent studies have demonstrated that fast turnaround times are critical for effective control of COVID-19.⁸ For this reason, the World Health Organization (WHO) emphasized rapid, point-of-care (POC) diagnostics as a top priority to contain COVID-19.⁹ Long turnaround times of RT-PCR tests have negatively impacted the emergency department (ED) to inpatient admission process at hospitals because the RT-PCR result is needed prior to inpatient placement. To overcome this problem, the Sofia rapid antigen fluorescent immunoassay (FIA) was introduced at our hospitals with the goal of rapid testing to guide initial bed placement of patients being admitted through the ED.

The Sofia SARS rapid antigen FIA has a manufacturer-published assay turnaround time of 15 minutes, and an emergency use authorization for SARS-CoV-2 diagnostic testing has been granted by the US Food and Drug Administration.¹⁰ Previous studies evaluating rapid antigen assays have suggested that these tests have low sensitivity at lower viral loads and among asymptomatic persons,¹¹ and published data on their utility among hospitalized patients are limited.

We conducted a real-world evaluation of rapid antigen testing among patients admitted through the ED through assessment of (1) the sensitivity, specificity, positive and negative predictive values of the antigen FIA compared to RT-PCR, (2) comparisons of turnaround times of antigen and RT-PCR testing, and (3) clinical correlates of antigen-negative/RT-PCR positive patients.

Methods

Setting and patients

This study was designed as a clinical quality improvement project at University of Maryland Upper Chesapeake Health, which includes 2 hospitals, the Upper Chesapeake Medical Center (220 beds with 29 double occupancy and an average of 32 daily admissions), and Harford Memorial hospital (104 beds with 49 double occupancy and an average of 12 daily admissions). From September 21, 2020, to December 3, 2020, all patients evaluated in the ED and considered candidates for hospital admission were told the purpose of the study and were asked to provide consent to undergo both Sofia SARS rapid antigen FIA and SARS-CoV2 RT-PCR assays. The study included both symptomatic (persons under investigation or PUIs) and asymptomatic patients.

Clinical practice and infection prevention measures during the study period

All hospitalized patients were required to undergo admission SARS-CoV-2 RT-PCR testing at the 2 hospitals. The test order required clinicians to indicate whether the patient was asymptomatic or a COVID-19 PUI. RT-PCR tests were performed for all patients. When caring for asymptomatic patients, staff were required to wear a medical-grade face mask and eye protection. For PUIs and laboratory-confirmed COVID-19 patients, staff were required to don a respirator, eye protection, gloves, and gowns.

Sample collection and testing

For each patient, trained staff collected 2 nasopharyngeal specimens. One specimen was placed in viral transport media for RT-PCR testing. The other specimen was placed dry in a tube for antigen testing. Results for both assays were reported in patient charts in the electronic medical record system and reported to the Maryland Department of Health.

SARS-CoV-2 RT-PCR assay

Three real-time RT-PCR-based methods were utilized for SARS-CoV-2 detection. The Abbott RealTime SARS-CoV-2 assay (Abbott, Des Plaines, IL) and the cobas SARS-CoV-2 (Roche Diagnostics, Indianapolis, IN) assay were performed at a centralized laboratory. The Xpress Xpert (Cepheid, Sunnyvale, CA) assay was utilized for patients presenting for emergency surgery, women presenting for delivery, prior to initial admission to behavioral health units, and for situations requiring rapid medical decision making based on the institutional testing protocol that guides the use of RT-PCR tests. The Roche cobas assay detects the ORF1 and a region of the E gene are specific to SARS-CoV-2. The Abbott RealTime assay detects RdRp and N genes. For Xpress Xpert, RT-PCR detects the pan-sarbecovirus E gene and the N2 region of N gene.¹³ The cycle threshold (Ct) values eere obtained for positive RT-PCRs tests as a surrogate measure for viral load for samples tested using the cobas and Xpress Xpert assays. Presumptive results were repeated by another method, and only the final results of “detected” or “not detected” were used in this study.

Sofia SARS rapid antigen FIA

This assay, which uses sandwich immunofluorescence-base lateral flow to qualitatively detect the SARS-CoV-2 nucleocapsid protein antigen, was performed according to the manufacturer’s guidelines.¹³ Sofia analyzers were used for detection.¹⁰

Patient placement

Patient room-placement guidelines were based on presenting symptoms and test results using a 2-step algorithm with the antigen assay available first and the RT-PCR over the following 24–48 hours. Briefly, antigen-negative, asymptomatic patients could be placed in any room with standard precautions and potentially in double occupancy rooms with other patients. Antigen-positive patients and symptomatic patients (regardless of antigen test results) were placed in a private room. Upon receipt of RT-PCR results, patients who were RT-PCR and antigen positive remained in private rooms, and antigen-negative patients who were found to be RT-PCR positive were moved to private rooms (Supplementary Table 1 online).

Turnaround times

Turnaround times were calculated as time (hours) between NP swab collection and result reporting in the electronic medical record.

Clinical evaluation

All antigen-negative, RT-PCR–positive patients were evaluated for the presence of COVID-19 symptoms, potential COVID-19 exposure in preceding 2 weeks, and potential SARS-CoV-2 transmission to patients and staff based on placement in a double occupancy room and compliance with institutional infection prevention protocols.

Outcome measures and statistical analyses

Daily positivity rates, 7-day moving averages, and trends in positivity based on RT-PCR results were plotted. Overall agreement, sensitivity, and along with 95% confidence intervals (CIs) were calculated for antigen testing with RT-PCR as the reference standard. Patients were stratified into symptomatic and asymptomatic and the aforementioned performance measures were calculated separately for both groups. Positive predictive value (PPV) and negative predictive value (NPV) of antigen testing were calculated for overall prevalence, symptomatic prevalence, and asymptomatic prevalence of COVID-19 based on RT-PCR, as well as for theoretical scenarios of 10% and 20% disease prevalence to evaluate WHO standards. The χ² test was conducted to statistically compare the performance measures among all patients, symptomatic patients, and asymptomatic patients.

The sensitivity of the antigen assay relative to RT-PCR was also calculated in different categories based on the following ranges of Ct values: <14, 14–17.9, 18–21.9, 22–25.9, 26–29.9, 30–33.9, and ≥34. Results from Abbott RealTime assay were excluded since the assay reports copy number instead of Ct values. Mean RT-PCR Ct values between true positives and false negatives and between symptomatic and asymptomatic patients were compared using the Student t test. Mean turnaround times for antigen and RT-PCR were compared using the Student t test. Statistical analyses and graphs were done using GraphPad Prism version 8 software (GraphPad, San Diego, CA).

This study was deemed non–human-subjects research by the University of Maryland Institutional Review Board.

Results

In total, 2,887 patients were enrolled in this study and received simultaneous SARS-CoV-2 RT-PCR and Sofia antigen testing; 235 patients were positive by RT-PCR, for an overall positive prevalence of 8.1%. For RT-PCR, 1,838 patients received the Roche cobas assay, 675 patients received the Xpress Xpert assay, and 374 received the Abbott RealTime assay. Of 1,675 patients presenting with COVID-19 symptoms, 193 (11.5%) were positive. Among 1,206 asymptomatic patients, 42 were positive (3.5%).

Positivity rates and trends

The 7-day moving average ranged from 2.5% to 20.9% during the study period, peaking between November 8 and November 24, and remained above 10% until the end of the study (Supplementary Fig. 1 online).

Sofia SARS rapid antigen FIA performance

Overall agreement, sensitivity, and specificity between antigen and RT-PCR for all participants were 97.9%, 76.6% (95% CI, 71%–82%), and 99.7% (95% CI, 99%–100%), respectively (Table 1). Among 1,675 symptomatic patients, the overall agreement was 97.1%, sensitivity was 76.2% (95% CI, 70%–82%), and specificity was 99.9% (95% CI, 99%–100%). Among 1,206 asymptomatic patients, the overall agreement was 98.9%, sensitivity was 78.6% (95% CI, 67%–91%), and specificity was 99.7% (95% CI, 99%–100%) (Table 1). Symptomatic and asymptomatic groups were not statistically different.

Table 1.

Comparison of Paired SARS-CoV-2 Rapid Antigen Assay and RT-PCR Assay Results Among Emergency Department Patients Presenting for Hospital Admission (N=2,887)

Patient Characteristic	PCR+/ Antigen+	PCR+/ Antigen−	PCR−/ Antigen−	PCR−/ Antigen+	Overall Agreement, %	Sensitivity, % (IQR)	Specificity, % (IQR)
Overall	180	55	2,645	7	97.9	76.6% (71%–82%)	99.7% (99%–100%)
Symptomatic	147	46	1,480	2	97.1	76.2% (70%–82%)	99.9% (99%–100%)
Asymptomatic	33	9	1,160	4	98.9	78.6% (67%–91%)	99.7% (99%–100%)

Note. RT-PCR, reverse-transcriptase polymerase chain reaction; PCR, polymerase chain reaction; +, positive; –, negative; IQR, interquartile range.

With an overall prevalence of 8.1% for this study, the PPV was 96.3% (95% CI, 92%–98%) and the NPV was 98.0% (95% CI, 97%–98%). Among symptomatic individuals, the prevalence was 11.5%, the PPV was 98.7% (95% CI, 97%–100, and the NPV was 97.0% (95% CI, 96%–98%). The prevalence among asymptomatic patients was 3.5%, the PPV was 89.2% (95% CI, 79%–99%), and the NPV was 99.2% (95% CI, 99%–100%). For WHO scenarios of 10% prevalence, the PPV was 96.5% and the NPV was 97.4 %. For 20% prevalence, the PPV was 98.4% and the NPV was 94.5% (Table 2).

Table 2.

Positive Predictive Value (PPV) and Negative Predictive Values (NPV) of SARS-CoV-2 Sofia SARS Rapid Antigen Assay for Prevalences of COVID-19

Scenarioa	Prevalence, %	PPV, %	NPV, %
Overall	8.1	96.3	98.0
Symptomatic	11.5	98.7	89.2
Asymptomatic	3.5	89.2	99.2
WHO hypothetical 1	10.0	96.5	97.4
WHO hypothetical 2	20.0	98.4	94.5

^a Scenarios include overall, symptomatic, and asymptomatic, prevalence in our sample, and 2 theoretical values that are used to determine quality of SARS-CoV-2 test by the World Health Organization.

Cycle thresholds were available for 166 RT-PCR–positive patients in this study (Fig. 1A): 146 from the Roche cobas assay and 20 from Xpress Xpert. The mean Ct value was 22.7 (95% CI, 21.8–23.6) for true positives and 32.1 (95% CI, 31.0–33.2) for false negatives (P < .001) on rapid antigen testing. At Ct values <17.9, the sensitivity was 100%. Between Ct values of 18 and 21.9, the sensitivity was 97.2%. For Ct values of 22–25.9, the sensitivity was 96.8%. For CT values of 26–29.9, the sensitivity was 71.0%. For Ct values of 30–33.9, the sensitivity was 29.2%. For Ct values ≥34, the sensitivity was 22.2% (Fig. 1B). We detected no difference in the mean Ct values between symptomatic patients (25.0; 95% CI, 24.1–25.9) and asymptomatic patients (25.4; 95% CI, 23.2–27.5; P = .78) (Fig. 2).

Fig. 1.

(A) Box plot comparing cycle thresholds of RT-PCR assays between true-positive and false-negative results for the Sofia Rapid antigen assay. True-positive antigen results had a significantly lower cycle threshold on corresponding RT-PCR assays than false-negative results. (B) Sensitivity of Sofia rapid antigen assay compared to RT-PCR assay based on cycle threshold. Sensitivity decreases as cycle threshold increases. Thus, at lower viral loads, the likelihood of false-negative antigen test results is higher.

Fig. 2.

Box plot comparing cycle thresholds of RT-PCR assays between symptomatic and asymptomatic individuals. No statistical differences were detected between the 2 groups (P = .78).

Turnaround time

The turnaround time for the Sofia SARS rapid antigen FIA was significantly shorter at 1.2 hours (95% CI, 1.0–1.3) compared to 20.1 hours (95% CI, 18.9–40.3) for RT-PCR (P < .001).

Clinical evaluation

In total, 55 patients with positive SARS-CoV-2 PCR assays were falsely negative on Sofia SARS rapid antigen FIA; of these patients, 38 (69%) were symptomatic. Symptomatic patients were placed in private room and treated as PUIs according to hospital policy, whereas 9 asymptomatic patients were placed in rooms with a roommate. Roommates of 3 patients still in house were tested for SARS-CoV-2 at 5–7 days from exposure; all were negative. No high-risk staff exposures were identified due to use of universal masking and eye protection. None of the staff or patient contacts developed symptoms. The median duration from symptom onset to testing for the 38 symptomatic false-negative results was 7 days (IQR, 2–13) and 17 (44%) received testing ≥8 days from symptom onset. Of 55 false-negative results, 9 (16%) had had prior a positive RT-PCR in the preceding 4-week period. Excluding those known prior positive test results, 20 (43%) patients had possible SARS-CoV-2 exposure risk.

Discussion

In this real-word evaluation among ED patients being considered for hospital admission, the Sofia rapid antigen FIA assay had a sensitivity of 76.6% (95% CI, 71%–82%) and specificity of 99.7% (95% CI, 99%–100%) compared to RT-PCR, with no significant differences between symptomatic and asymptomatic patients. Importantly, the predictive value estimates varied by prevalence, and the percent agreement increased with decreasing RT-PCR cycle thresholds, with 100% sensitivity with Ct values < 22. Average turnaround times were significantly lower for antigen versus RT-PCR testing. With limited testing of contacts, transmission to other patients or staff was not observed from antigen-negative/RT-PCR–positive patients.

Several studies have evaluated and compared rapid POC diagnostics for SARS-CoV-2. These assays can be administered via saliva, nasopharyngeal swab, or nasal swab, and they are antigen or molecular based.²⁰ Some current examples of rapid POC assays include BD Veritor, COVID-19 Ag Respi-Strip, Lumipulse, and the Abbott ID NOW.^21–26 These rapid assays have reported low sensitivities, especially at high Ct.^27–29 The Sofia SARS rapid antigen FIA has previously been compared to other POC diagnostics in symptomatic individuals, and it is either competitive with or outperforms other POC assays.^21–27

For a rapid SARS-CoV-2 diagnostic test to be considered acceptable by the World Health Organization (WHO), sensitivity and specificity must be 80% and 97%, respectively.¹⁴ The desirable thresholds are even higher at 90% and 99%, respectively. Although the overall sensitivity for the Sofia rapid antigen FIA assay does not meet WHO guidelines, it is only slightly lower than the acceptable level, with the 95% confidence interval crossing the threshold. For predictive values, the WHO requires a second or confirmatory assay for PPV <50%. By these standards, a second assay would not be required for the Sofia SARS rapid antigen FIA. Furthermore, for prevalence between 10% and 20%, the WHO-recommended acceptable ranges are >78%–89% for PPV and 95%–98% for NPV. At 10% prevalence, the PPV based on our study is 96.5% and the NPV is 97.4%; both meet acceptable criteria. At 20%, the NPV drops to 94.5%, slightly below guidelines. Based on recommended PPV and NPV ranges by the WHO, the ideal prevalence range for use of the Sofia SARS rapid antigen FIA assay is between 10% and 18.5%, and caution should be exercised when using this assay’s PPV at lower and NPV at higher prevalence of disease.

A significant proportion of transmission of SARS-CoV-2 occurs from asymptomatic and presymptomatic infected individuals.³ It is critical to properly identify these carriers in a timely manner. Previous studies lack thorough analysis of antigen-based assay performance in asymptomatic individuals. In this study, the Sofia SARS rapid antigen FIA was evaluated in both symptomatic and asymptomatic individuals, providing information not previously explored. When stratified by symptomatic and asymptomatic, overall agreement, sensitivity, and specificity were similar, demonstrating the Sofia SARS rapid antigen FIA is just as accurate in asymptomatic individuals as those who are symptomatic. Importantly, mean Ct values were not significantly different between asymptomatic and symptomatic individuals suggesting viral load and not symptoms as the primary determinant of test performance. These results differ from previous publications that have suggested its use only in symptomatic patients, and they reflect the wide spectrum of clinical manifestations of SARS-CoV-2 infection such that patients can be asymptomatic and highly infectious or severely ill and past their infectiousness.^16,17,26

We also observed a decrease in the sensitivity of the Sofia SARS rapid antigen FIA at lower viral loads as estimated by RT-PCR Ct values, and we observed significantly higher Ct in the antigen false-negative group. This finding is consistent with previous studies that have shown lower sensitivity at high Ct values.^11,15 Lack of sensitivity at high Ct values makes antigen testing less suitable than RT-PCR for detection of both very early and late SARS-CoV-2 infection when viral load is low. This aspect was noted in the clinical evaluation of discordant (antigen negative, RT-PCR positive) results in our study patients, with most representing either recent exposure or delayed shedding in prior RT-PCR positive patients. Although it is likely that antigen testing detects most patients with transmissible SARS-CoV-2 infection, infected individuals with low viral loads detected on NP swabs may still transmit the virus to others, particularly in the setting of early incubating disease.^16–19,30

No evidence of transmission from false-negative patients was found. Low or noninfectious viral shedding, lack of transmission was likely mitigated by infection control policies in place including universal masking, universal use of eye protection among staff providing patient care, and maintenance of symptomatic patients in appropriate precautions while RT-PCR test was still pending.

A significant reduction in turnaround time from sample collection to result was observed using the Sofia SARS rapid antigen FIA compared to RT-PCR with a shorter turnaround time by an average of 18.9 hours for antigen testing. Previous studies have suggested that time from sample to results is even more critical than sensitivity in reducing the spread of SARS-CoV-2.⁸ Many healthcare systems depend on assay results to decide patient placement. Delays in results could lead to patient placement in nonprivate rooms, spreading the virus, or unnecessarily isolating the patient, wasting limited hospital space and resource. During the study period, inpatient room placement was successful in 1,160 of 1,169 asymptomatic antigen-negative patients in this study, and only 9 false negative results required a re-evaluation of placement, reflecting the benefit of the high NPV of antigen testing in this setting. Collectively, this finding suggests that when used together with clinical symptoms, exposure history, infection control practices, and confirmatory RT-PCR testing, rapid antigen tests can be useful in guiding initial patient placement. Although not formally assessed, anecdotally, ED staff and inpatient providers expressed significant satisfaction in being able to make quicker decisions based on the significantly improved turnaround time of antigen testing relative to RT-PCR in a hospital with a relatively high proportion of semiprivate rooms.

Our study has several limitations. RT-PCR can provide inaccurate results and might not be the perfect comparison. However, RT-PCR remains the current recognized standard for SARS-CoV-2 diagnosis. Designations of symptomatic and asymptomatic were at provider discretion and subject to bias; however, this reflects real-world conditions of use of SARS-CoV-2 diagnostic testing. Furthermore, a higher prevalence or percent positivity in symptomatic versus asymptomatic patients in our sample indicates that designations were accurate within the inherent limitations of recognizing early or nonspecific symptoms of COVID-19 PCR Ct values are limited surrogate measures for viral load. However, these values do not provide an absolute count and are dependent on the assay, sample collection, and collection site. We tested only a small number of contacts of antigen-negative/RT-PCR–positive patients, which limits conclusions about transmission in this study. However, no symptomatic staff or patients were identified through contact tracing. Lastly, this study may not be generalizable to hospitals with different prevalences of SARS-CoV-2. Confirmatory testing of the Sofia SARS rapid antigen FIA may be required for negative tests in symptomatic individuals in high-prevalence populations and for positive asymptomatic patients in low-prevalence populations.

Despite not meeting the requirements to replace the RT-PCR assay for detection of SARS-CoV-2 infection, our findings suggest the usefulness of Sofia SARS rapid antigen FIA to guide initial patient placement in a burdened, limited rapid PCR-capacity hospital setting for both symptomatic and asymptomatic patients.

Financial support

No financial support was provided relevant to this article.

Conflicts of interest

All authors report no conflicts of interest relevant to this article.