Diagnostic performance of GenBody COVID-19 rapid antigen test for laboratory and non-laboratory medical professionals in real practice: A retrospective study

Abstract

Point-of-care tests for coronavirus disease 2019 (COVID-19) antigen detection have been widely used for rapid diagnosis in various settings. However, research on the diagnostic performance of the COVID-19 antigen test performed by non-laboratory personnel is limited. In this study, we aimed to elucidate the diagnostic performance of GenBody COVID-19 rapid antigen between laboratory professionals and non-laboratory staff. We retrospectively analyzed the data of patients who underwent both GenBody COVID-19 rapid antigen testing and reverse transcription polymerase chain reaction (RT-PCR) between November 01, 2021, and June 30, 2022. The diagnostic performance of the antigen test was compared between laboratory and non-laboratory operators, using RT-PCR as the gold standard. Sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, diagnostic odds ratio, positive predictive value, negative predictive value, and accuracy were calculated and sensitivity analysis was performed based on the PCR cycle threshold (Ct) value. Of the 11,963 patients, 1273 (10.6%) tested positive using real-time RT-PCR. The sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, diagnostic odds ratio, positive predictive value, negative predictive value, and accuracy of the GenBody COVID-19 rapid antigen test with 95% confidence interval were 79.92% (77.26%–82.39%), 99.23% (98.73%–99.57%), 103.25 (62.31–171.11), 0.2 (0.18–0.23), 510.18 (299.81–868.18), 98.11% (96.91%–98.85%), 90.75% (89.64%–91.75%) and 92.76% (91.76%–93.67%), respectively, for non–laboratory staff and 79.80% (74.78%–84.22%), 99.99% (99.94%–100.00%), 6983.92 (983.03–49617.00), 0.2 (0.16–0.25), 34566.45 (4770.30–250474.46) 99.58% (97.09%–99.94%), 99.32% (99.15%–99.46%), and 99.33% (99.13%–99.48%), respectively, for laboratory staff. Notably, when the PCR Ct value exceeded 25, the sensitivity of both the groups decreased to < 40%. The diagnostic performance of GenBody COVID-19 rapid antigen performed by non-laboratory staff was comparable to that of laboratory professionals. However, it should be noted that the sensitivity of the antigen tests decreased when the PCR Ct value exceeded 25. Overall, the GenBody COVID-19 antigen test is a viable option for non-laboratory staff during an epidemic.

1. Introduction

The emergence of the coronavirus disease 2019 (COVID-19) has been a major challenge to global public health.^[1] In Taiwan, 2 waves of COVID-19 were experienced in March 2020 and May 2021, respectively,^[2] with a peak in cases observed in May 2022.^[3] Rapid and accurate detection of the virus is one of the essential strategies for controlling its spread. Although reverse transcription polymerase chain reaction (RT-PCR) testing has become the gold standard for COVID-19 diagnosis owing to its high accuracy, sensitivity, and specificity,^[4] it requires trained personnel and is relatively time-consuming and expensive.^[5] Consequently, many healthcare systems worldwide have been exploring alternative diagnostic tools to meet the growing demand for rapid and affordable COVID-19 testing.

Point-of-care tests (POCTs) based on severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antigens have been increasingly used as a rapid alternative for diagnosis in various settings.^[6,7] Recent studies have highlighted the importance of accurate and timely detection of COVID-19, as early identification of cases can help prevent further transmission of the virus.^[8] However, the accuracy of POCTs is dependent on the expertise of the operator,^[9] with laboratory professionals generally considered to have more experience and training in performing diagnostic tests.

To address this concern, previous studies have examined the diagnostic performance of non-laboratory personnel for various diseases.^[10,11] However, there is relatively limited research specifically comparing the performance between laboratory professionals and non-laboratory staff in the context of COVID-19 antigen tests. In our previous study,^[12] we observed that the sensitivity of SARS-CoV-2 antigen testing conducted by non-laboratory personnel in community settings was comparable to that performed by laboratory professionals in the emergency department. These findings suggest the suitability of the SARS-CoV-2 antigen test for POCT. However, it is important to note that the study included a relatively small number of cases and different study populations.

In this study, we assumed that the performance of non-laboratory staff and laboratory professionals in utilizing the GenBody COVID-19 antigen test would exhibit similar characteristics. The primary objective of this investigation was to evaluate the clinical utility of the GenBody COVID-19 antigen test in practical settings, encompassing both non-laboratory staff and laboratory professionals. And, our aim was to assess the suitability of this antigen test as a POCT in non-laboratory settings.

2. Materials and methods

2.1. Subject selection and study design

This retrospective study was conducted at the Far Eastern Memorial Hospital in New Taipei City, Taiwan. From November 01, 2021 to June 30, 2022. The study included subjects who met the following criteria: Age ≥ 20 years and; Underwent both SARS-CoV-2 rapid antigen and real-time RT-PCR tests on the same day. The antigen and PCR samples were collected in the emergency department (ED), ward, or outpatient department (OPD). This study was approved by the Ethics Committee of Far East Memorial Hospital (approval date: November 18, 2022; approval number:111,270-E). Data were collected from the electronic medical records, including age, sex, department, antigen test operator, SARS-CoV-2 antigen test results, and RT-PCR test results. Informed consent was waived owing to the retrospective nature of the study and de-identified data. All the procedures adhered to the principles of the Declaration of Helsinki.

2.2. Determination of the SARS-CoV-2 antigen test

The GenBody COVID-19 antigen test, a qualitative test, was performed by either a trained medical laboratory technician or an on-site physician, both of whom were blinded to the RT-PCR results. The manufacturer’s instructions were followed.^[13] GenBody COVID-19 antigen is an immunochromatographic rapid diagnostic test for the detection of the nucleocapsid protein of SARS-CoV-2. The swabs of the collected specimens were inserted into the extraction solution. The swab was mixed by squeezing the tube and twirling it 8 to 10 times. The swab was then removed, and 4 drops (approximately 100 µL) of the solution were added to the center of the sample well of the test device. The test results were recorded at 15 to 20 minutes. The test result was interpreted as positive if both the control and test lines were visually present. Any faint visible reddish-purple test line was considered to be positive. The results were considered valid only when the control line was visible.

2.3. Determination of real-time RT-PCR

Real-time RT-PCR assays were conducted as reference standard tests to confirm the presence of SARS-CoV-2. The nasopharyngeal swab sample for RT-PCR was placed in universal transport medium. These samples were promptly transported to the laboratory, typically within a few hours of collection and mostly within an hour timeframe. RT-PCR was performed using an automated Roche Cobas 6800 SARS-CoV-2 test (Roche, Pleasanton, CA) with targets of the E gene and ORF-1ab gene.^[14,15] The results were considered positive when both the targets were detected.

2.4. Statistical analysis

Patient characteristics are presented as numbers (percentages) or means (standard deviations), as appropriate. Two-by-two tables were presented using RT-PCR results as the reference standard test and GenBody COVID-19 antigen as the index test. The sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, diagnostic odds ratio, positive predictive value, negative predictive value, and accuracy of the antigen test were calculated based on different antigen operators. A sensitivity analysis was conducted using cycle threshold (Ct) values of the E gene. The Ct values were categorized into 4 groups: < 15, 15 to 19.99, 20 to 24.99, and > = 25. Statistical analysis was conducted using an online statistical tool^[16] and IBM SPSS Statistics for Windows (version 19, IBM Corp., Armonk, NY).

3. Results

A total of 11,963 paired nasopharyngeal swabs were collected: 7335 (61.3%) from the ED, 4375 (36.6%) from the OPD, and 253 (2.1%) from the ward. Of these, 2914 antigen tests were conducted by non-laboratory staff and 9049 tests were conducted by laboratory professionals. The characteristics, RT-PCR results, and antigen test results of the study population are presented in Table 1.

Table 1.

Characteristics, polymerase chain reaction result and antigen result of the study population based on antigen operator.

	Non-laboratory staff	Laboratory professionals	Total
Patient number	2914	9049	11,963
Age	52 (IQR: 38–67)	57 (IQR: 42–70)	56 (IQR: 41–69)
Sex
Male	1359 (46.6%)	4279 (47.3%)	5638 (47.1%)
Female	1555 (53.4%)	4770 (52.7%)	6325 (52.9%)
Department
ED	1367 (46.9%)	5968 (66%)	7335 (61.3%)
OPD	1320 (45.3%)	3055 (33.8%)	4375 (36.6%)
Ward	227 (7.8%)	26 (0.3%)	253 (2.1%)
RT-PCR result
Positive	976 (33.5%)	297 (3.3%)	1273 (10.6%)
Negative	1938 (66.5%)	8752 (96.7%)	10,690 (89.4%)
E gene Ct value	18.85	19.69	19.06
	(IQR: 16.26–21.82)	(IQR: 17.84–22.1)	(IQR: 16.72–21.87)
Group of E gene Ct
<15	164 (5.6%)	9 (0.1%)	173 (1.4%)
15–19.99	439 (15.1%)	160 (1.8%)	599 (5%)
20–24.99	220 (7.5%)	91 (1%)	311 (2.6%)
25–29.99	82 (2.8%)	14 (0.2%)	96 (0.8%)
>=30	71 (2.4%)	23 (0.3%)	94 (0.8%)
Antigen result
Positive	795 (27.3%)	238 (2.6%)	1033 (8.6%)
Negative	2119 (72.7%)	8811 (97.4%)	10,930 (91.4%)

Ct = cycle threshold, ED = emergency department, IQR = interquartile range, OPD = outpatient department, RT-PCR = reverse transcription polymerase chain reaction.

Overall, 1273 (10.6%) of the RT-PCR results were positive (median E gene Ct value:19.06; IQR:16.72–21.87), while 10,690 were negative. The overall sensitivity and specificity of GenBody COVID-19 antigen were 79.89% (95% confidence interval [CI]: 77.58%–82.06%) and 99.85% (95% CI:99.76%–99.91%), respectively. The performances of the antigens based on different antigen operators are shown in Table 2. The sensitivity, specificity and negative likelihood ratio were comparable between the 2 groups, while the positive likelihood ratio, diagnostic odds ratio, positive predictive value, negative predictive value, and accuracy were lower in non-laboratory staff operators.

Table 2.

Performance of the GenBody coronavirus disease 2019 rapid antigen based on antigen operator.

	Non-laboratory staff (95% CI)	Laboratory professionals (95% CI)	Total (95% CI)
Sensitivity	79.92% (77.26%–82.39%)	79.80% (74.78%–84.22%)	79.89% (77.58%–82.06%)
Specificity	99.23% (98.73%–99.57%)	99.99% (99.94%–100.00%)	99.85% (99.76%–99.91%)
PLR	103.25 (62.31–171.11)	6983.92 (983.03–49617.00)	533.77 (326.87–871.63)
NLR	0.2 (0.18–0.23)	0.2 (0.16–0.25)	0.2 (0.18–0.22)
Diagnostic odds ratio	510.18 (299.81–868.18)	34566.45 (4770.30–250474.46)	2650.26 (1592.81–4409.74)
Prevalence	33.49% (31.78%–35.24%)	3.28% (2.92%–3.67%)	10.64% (10.09%–11.21%)
PPV	98.11% (96.91%–98.85%)	99.58% (97.09%–99.94%)	98.45% (97.50%–99.05%)
NPV	90.75% (89.64%–91.75%)	99.32% (99.15%–99.46%)	97.66% (97.39%–97.90%)
Accuracy	92.76% (91.76%–93.67%)	99.33% (99.13%–99.48%)	97.73% (97.44%–97.99%)

CI = confidence interval, NLR = negative likelihood ratio, NPV = negative predictive value, PLR = positive likelihood ratio, PPV = positive predictive value.

The overall sensitivity of the diagnostic test was 93.6%, 91.2%, 80.7%, and 30.5 for Ct values < 15, 15 to 19.99, 20 to 24.99, and equal or higher than 25, respectively (Fig. 1). The sensitivity analysis of each group for the GenBody COVID-19 antigen with respect to the Ct values of RT-PCR is shown in Figure 2. Laboratory professionals had better sensitivity for patients with Ct values < 15 and 15 to 19.99. For patients with Ct values between 20 and 24.99, sensitivity was comparable between the 2 groups. Non-laboratory staff had better sensitivity for patients with Ct values > 25.

Figure 1.

Sensitivity analysis of the GenBody COVID-19 antigen with respect to Ct values of RT-PCR for the E gene.COVID-19 = coronavirus disease 2019, Ct = cycle threshold, RT-PCR = reverse transcription polymerase chain reaction.

Figure 2.

Sensitivity analysis of the GenBody COVID-19 antigen and amount of antigen operated by different operator with respect to Ct values of RT-PCR for the E gene. COVID-19 = coronavirus disease 2019, Ct = cycle threshold, RT-PCR = reverse transcription polymerase chain reaction.

4. Discussion

This study evaluated the diagnostic performance of the GenBody COVID-19 rapid antigen test by different operators and in different healthcare settings. The test demonstrated high specificity (99.85%), although its sensitivity (79.89%) was lower than the manufacturer’s claim (96.8%, 95% CI:91.9%–99.1%).^[13] Similar findings were reported in previous studies.^[12,17] It was likely due to differences in Ct values. Notably, the sensitivity improved to 91.7% for patients with Ct values below 20. For subjects whose Ct values were below 25 and 30, the sensitivity rates were 88.6% and 85.2%, respectively. These results met the World Health Organization recommendation of ≥ 80% sensitivity and ≥ 97% specificity.^[18] In contrast, the sensitivity decreased to 13.8% for those with Ct values above 30, consistent with the manufacturer’s specifications that reported low sensitivity for Ct values > 30. Our findings are similar to those of studies using the GenBody COVID-19 antigen conducted in Hungary and Europe. In a study of 98 samples in Hungary, the overall sensitivity was reported to be 62%, which increased to 95.7% in cases with strongly positive results (Ct value ≤ 25).^[19] A European study involving 404 samples demonstrated that the sensitivity of the GenBody COVID-19 antigen test gradually declined as the Ct value was higher than 25, and further decreased when it exceeded 30.^[20] These findings emphasized the importance of considering Ct values when interpreting antigen test results.

Regarding the performance of antigen testing between non-laboratory staff and laboratory professionals, we found no marked differences in sensitivity and specificity. However, non-laboratory staff exhibited a higher false-positive rate (1.89%) and false-negative rate (9.25%) compared to laboratory professionals (0.42% and 0.68%, respectively). False-positive results can lead to unnecessary treatment, investigations, delayed surgeries and unnecessary isolation.^[21] The prevalence of COVID-19 was higher in the non-laboratory staff group (33.49% vs 3.28%). Generally, in high-prevalence situations, the false-positive rate is lower.^[21] However, our results showed a higher false-positive rate among non-laboratory staff. This might be due to the relative lack of training or experience of non-laboratory staff compared to that of laboratory professionals. More research is needed to verify this finding and identify the underlying reasons.

In the sensitivity analysis based on the Ct values, varying levels of sensitivity were observed among different operators. Non-laboratory staff demonstrated lower sensitivity than laboratory professionals for patients with Ct values below 20 but comparable sensitivity for those with Ct values between 20 and 24.99. However, non-laboratory staff showed higher sensitivity for patients with Ct values above 25. This may be due to several reasons. First, the number of patients with higher Ct values operated on by laboratory professionals was small, which might have affected the results. Second, patients with higher Ct values typically have lower levels of viral load and infectivity risk^[22] and thus may not be expected to yield positive antigen results. Therefore, the negative antigen results in these patients may not be false negatives by the operator, but rather represent the limitations of the antigen test. Thirdly, the observed differences could potentially be attributed to variations in sample collection, handling, and the timing of test interpretation. For instance, the misinterpretation of test results at 30 minutes instead of the recommended 15-minute mark could contribute to these findings. Additionally, it is worth noting that non-laboratory staff often have access to relevant clinical information, such as travel history, cluster history, contact history, or symptoms, which may influence their interpretation of test results. Moreover, the tested population handled by non-laboratory staff exhibited a higher prevalence of COVID-19, potentially leading to a tendency to interpret results as positive. These factors might partially account for the observed higher false-positive rate among non-laboratory staff.

A previous systematic review showed that antigen performance was brand-dependent but not operator-dependent.^[23] However, this review only included 1 study that involved self-swabbing by patients, while the other 14 studies were conducted by trained professionals. In a previous study conducted in the UK, the INNOVA SARS-CoV-2 antigen test was used to analyze 793 RT-PCR-positive cases in order to investigate the impact of operator expertise.^[24] The findings showed that laboratory professionals exhibited higher sensitivity (78.8%), followed by trained healthcare workers (70.0%), and self-trained individuals (57.5%). In our study, we observed a minimal difference of only 0.12% in overall sensitivity between non-laboratory operators and laboratory professionals. This comparable performance of the GenBody COVID-19 antigen test between non-laboratory staff and laboratory professionals underscores its potential as a POCT operated by non-laboratory healthcare workers. Implementing such an approach could enhance testing capacity and alleviate the strain on laboratory personnel, who have been facing significant stress and burnout amidst the demands of the ongoing pandemic. These considerations are in line with previous reports highlighting the need to optimize resources and support the well-being of laboratory personnel during these challenging times.^[25,26]

Our study had some limitations that should be considered when interpreting the findings. Firstly, the participants in this study did not undergo antigen testing by both laboratory and non-laboratory personnel. Due to the retrospective nature of this study, it was not feasible in clinical practice to perform simultaneous antigen testing using 2 different methods on each patient. However, this study collected a large sample size in an effort to mitigate the impact of this limitation. Secondly, medical information regarding patient risk factors for COVID-19 infection and the onset time of symptoms was not collected, resulting in an inability to demonstrate the balance of patient characteristics between the 2 groups. However, it is important to highlight that all antigen test results were compared to the gold standard RT-PCR test, and both the antigen and RT-PCR tests were conducted on the same day. These measures were implemented to reduce potential concerns associated with imbalanced groups and different sampling times. During the study period, it was a hospital policy to conduct both antigen and PCR tests for patients in ED. However, it was possible that not all subjects underwent both antigen and PCR tests in OPD and ward. This difference in test administration between ED, hospitalized and outpatient individuals was a limitation of the study. Despite these limitations, our study had several strengths. We used a large sample size and RT-PCR as the gold standard to evaluate the diagnostic accuracy of the antigen test. We also assessed the impact of operator factors and Ct values on diagnostic performance. Furthermore, in accordance with Taiwan policy, all participants underwent PCR testing, regardless of the antigen test results. This helped minimize selection bias. To our knowledge, this is the first study to examine the diagnostic performance of the GenBody COVID-19 rapid antigen test conducted by laboratory professionals and non-laboratory staff.

In conclusion, the GenBody COVID-19 rapid antigen test demonstrated high specificity and fair sensitivity when Ct values were < 25, regardless of whether it was conducted by laboratory professionals or non-laboratory staff. Using antigen tests as a POCT in non-laboratory settings is feasible and potentially effective during pandemics.

Author contributions

Conceptualization: Pei-Chin Lin, Fang-Yeh Chu.

Data curation: Pei-Chin Lin, Chun-Jung Huang, Yen-Ming Lu, Huei-Ling Huang, Zong-Ying Wu.

Formal analysis: Pei-Chin Lin, Chun-Jung Huang, Huei-Ling Huang, Zong-Ying Wu.

Funding acquisition: Pei-Chin Lin.

Investigation: Pei-Chin Lin, Chun-Jung Huang.

Methodology: Pei-Chin Lin, Chih-Chun Chang, Fang-Yeh Chu.

Software: Pei-Chin Lin, Chih-Chun Chang.

Supervision: Fang-Yeh Chu.

Validation: Fang-Yeh Chu.

Writing – original draft: Pei-Chin Lin, Yen-Ming Lu.

Writing – review & editing: Pei-Chin Lin, Chun-Jung Huang, Yen-Ming Lu, Huei-Ling Huang, Zong-Ying Wu, Chih-Chun Chang, Fang-Yeh Chu.