Performance of the FREND™ COVID-19 IgG/IgM Duo point-of-care test for SARS-CoV-2 antibody detection

ABSTRACT

Purpose: In the context of the current COVID-19 pandemic, multiple serological assays for the detection of severe acute respiratory syndrome 2 (SARS-CoV-2) immune response are currently being developed. This study compares the FREND^TM COVID-19 IgG/IgM Duo (NanoEntec) a point of care (POCT) assay with the automated Elecsys anti-SARS-CoV-2 electrochemiluminescent assay (Roche Diagnostics).

Methods: Serum samples (n = 81) from PCR-confirmed SARS-CoV-2 positive patients at different time points after the onset of symptoms were analyzed with both assays. An additional 24 serum samples with cross reactivity potential were also included.

Results: The sensitivity of the COVID-19 IgG/IgM Duo assay was higher as compared to the Elecsys anti-SARS-CoV-2 assay, especially when using the combined IgM/IgG result in samples analyzed within 6 days after the onset of symptoms (46.2% vs. 15.4%). The sensitivity of both assays increased with increasing time interval after the onset of symptoms and reached 100% for the COVID-19 IgG/IgM Duo assay in samples taken 14 days or more after symptom onset. Specificity of the COVID-19 IgG/IgM Duo assay was 95.8% for IgM, 91.7% for IgG and 87.5% for the combination of both.

Conclusion: This study shows that the sensitivity of both assays was highly dependent on the time interval between the onset of the COVID-19 symptoms and serum sampling. Furthermore, rapid serological testing for SARS-CoV-2 antibodies by means of the FREND^TM COVID-19 IgG/IgM Duo POCT assay showed a comparable diagnostic performance as the reference automated immunoassay.

Introduction

Due to the high mortality, morbidity and socio-economic burden, coronavirus disease 2019 (COVID-19) arising from severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection is currently a major international health concern. The burden on microbiology laboratories in providing rapid diagnosis in the context of timely patient isolation and consequently limiting virus transmission is considerable [1]. Detection of viral RNA by RT-qPCR in respiratory samples is considered the gold standard for the detection of SARS-CoV-2-infected individuals. However, the sensitivity of molecular testing is highly influenced by sampling technique and differences in viral load in various parts of the respiratory tract [2,3]. Hence, serology testing might help in the identification of SARS-CoV-2-infected patients with negative RT-qPCR results, especially when clinical suspicion of COVID-19 infection is high [4,5]. In addition, the global nature of this epidemic is associated with logistic challenges for diagnostic laboratories, which may hamper the use of the recommended RT-qPCR, thus requiring alternative methods. Furthermore, accessibility to PCR-techniques in some developing countries is not evident. Moreover, serological tests can provide essential data for epidemiological studies. In the present study, we compared the rapid point of care (POCT) FREND^TM COVID-19 IgG/IgM Duo assay from NanoEntec to the automated Elecsys anti-SARS-CoV-2 assay from Roche Diagnostics.

Material and methods

This retrospective study included 105 serum samples, stored at −20°C, from patients admitted at the University Hospital Antwerp. For sensitivity analysis, serum samples (n = 81) were selected from patients with confirmed SARS-CoV-2 by RT-PCR (in-house method adapted from Corman et al. [6]). For specificity analysis, serum samples with a potential cross-reactivity were selected, such as samples with antibodies against non-SARS-CoV-2 coronaviruses (HCoV 229E, HCoV NL63, HCoV OC43, n = 15) and other pathogens (n = 4) (presence of IgGs against Epstein Barr viral capsid, Hepatitis B surface antigen and Varicella zoster) and samples with high rheumatoid factor (>30 IU/ml, n = 5).

The COVID-19 IgG/IgM Duo is a fluorescent lateral flow immunoassay detecting both IgM and IgG antibodies against the SARS-CoV-2 nucleocapsid (N) protein separately. Samples were analyzed according to the manufactures instructions. In brief, 35 µL of the serum sample is diluted in sample buffer. Of this diluted sample, 35 µL is loaded on a cartridge which is hereafter inserted in the FREND^TM system, which provides an antibody ratio with corresponding negative/positive interpretation within approximately 5 minutes. Results were compared to results obtained by the Elecsys anti-SARS-CoV-2 assay from Roche Diagnostics on a Cobas e 801 module previously evaluated at the Antwerp University Hospital [7]. This reference serological assay also detects antibodies (including IgG) against the SARS-CoV-2 N protein but without distinction between IgG and IgM. For this 12 µL of serum sample is used for the antibody detection via an electrochemiluminescent immunoassay.

Sensitivity, at various time intervals after symptom onset, specificity, positive predictive value (PPV) and negative predictive value (NPV) with corresponding 95% confidence intervals were calculated for both serological assays. Data were analyzed using GraphPad Prism software. Statistical tests are mentioned in the Figure 1 legend. P < 0.05 was considered as statistically significant.

Figure 1.

IgM (A) and IgG (B) ratio distribution for COVID-19 IgG/IgM Duo assay and the Elecsys anti-SARS-CoV-2 assay (C) on serum samples from COVID-19 negative patients as compared to COVID-19 positive patients categorized by the sampling time after the onset of symptoms. Data are presented as mean±95%CI. One-way ANOVA **p < 0.01 ***p < 0.001

Results

Sensitivity

In total, 81 serum samples from 53 different patients with RT-PCR confirmed SARS-CoV-2 infection and 24 negative serum samples were analyzed. The diagnostic performance of both assays is shown in Table 1. The overall sensitivity of the COVID-19 IgG/IgM Duo assay seems higher when considering both the IgG results (82%) or the combined IgM/IgG results (89%) compared with the Elecsys anti-SARS-CoV-2 assay (74%). Overall, nine samples of patients with PCR confirmed SARS-CoV-2 infection classified as negative based on the combined result of IgM and IgG with the COVID-19 IgG/IgM Duo assay as compared to 21 samples with the Elecsys anti-SARS-CoV-2 assay.

Table 1.

Diagnostic performance of the COVID-19 IgG/IgM duo and Elecsys anti-SARS-CoV-2 assay

	COVID-19 IgG/IgM Duo			Elecsys anti-SARS-CoV-2
	IgM	IgG	IgM or IgG	Ab incl. IgG
Sensitivity (95% CI)	61.7% (50.2–72.1)	82.3% (72.4–99.7)	88.9% (79.5–94.5)	74.1% (62.9–82.9)
PPV (95% CI)	98.0% (88.2–99.9)	97.1% (89.0–99.5)	96.0% (88.0–99.0)	100.0% (92.5–100.0)
Specificity (95% CI)	95.8% (76.9–99.8)	91.7% (71.5–98.5)	87.5% (66.5–96.7)	100% (82.8–100.0)
NPV (95% CI)	42.6% (29.5–56.7)	61.1% (43.5–76.4)	70.0% (50.4–84.6)	53.3% (38.0–68.0)

Ab = antibodies

Hereafter, sensitivity results were stratified based on the date of positive PCR (Table 2) and considering the date of onset of the symptoms (Table 3). Sensitivity of both assays increased over time, reaching 100% for the IgG and combined IgG/IgM by COVID-19 IgG/IgM Duo and over 94% for Elecsys anti-SARS-CoV-2 on serum samples taken more than or equal to 14 days after PCR diagnosis or onset of symptoms. For earlier samples, a trend towards a higher sensitivity was obtained with the COVID-19 IgG/IgM Duo assay (81.0% vs. 57.1% on day 0–6 post PCR and 94.4% vs. 88.9% on day 7–13 post PCR for the combined IgG/IgM by COVID-19 IgG/IgM Duo and Elecsys anti-SARS-CoV-2, respectively.

Table 2.

Sensitivity of the COVID-19 IgG/IgM Duo and Elecsys anti-SARS-CoV-2 assay dependent on the time post positive PCR

		Days post PCR
		0–6	7–13	≥14
COVID-19 IgG-IgM Duo	IgM	45.2 (30.2–61.2)	66.7 (41.2–85.6)	90.5 (68.2–98.3)
	IgG	69.0 (52.8–81.9)	94.4 (70.6–99.7)	100 (80.8–100)
	IgM or IgG	81.0 (65.4–90.9)	94.4 (70.6–99.7)	100 (80.8–100)
Elecsys anti-SARS-CoV-2	Total ab (incl IgG)	57.1 (41.7–72.0)	88.9 (63.9–98.1)	95.2 (74.1–99.8)

Table 3.

Sensitivity of the COVID-19 IgG/IgM Duo and Elecsys anti-SARS-CoV-2 assay dependent on the onset of symptoms

		Days since onset symptoms
		0–6	7–13	≥14
COVID-19 IgG-IgM Duo	IgM	38.5 (15.1–67.7)	50.0 (32.2–67.8)	83.3 (66.5–93.0)
	IgG	23.1 (6.2–54.0)	87.5 (70.1–95.9)	100 (88.0–100)
	IgM or IgG	46.2 (20.4–73.9)	93.8 (77.8–98.9)	100 (88.0–100)
Elecsys anti-SARS-CoV-2	Total ab (incl. IgG)	15.4 (2.7–46.3)	75.0 (56.2–87.9)	94.4 (80.0–99.0)

Although both assays display a low sensitivity on samples taken within 6 days after the onset of symptoms, a trend towards a higher sensitivity was seen for the COVID-19 IgG/IgM Duo assay at this time point, especially when considering the combined result for IgM and IgG (46.2% vs. 15.4% for Elecsys anti-SARS-CoV-2) (Table 3).

Specificity

The specificity of the Elecsys anti-SARS-CoV-2 assay was excellent (100%) and therefore seemed better than the 87% obtained with the combined IgG/IgM test by COVID-19 IgG/IgM Duo (Table 1). Analysis of serum samples with potential cross-reactivity identified 3 false positive results with the COVID-19 IgG/IgM Duo assay (1 sample positive for IgM and 2 samples positive for IgG) while none of the samples were false positive with the Elecsys anti-SARS-CoV-2 assay. Among these false positive results, one was just above the cutoff value (1.03 with a cutoff of 1). False positive results were found in two samples with antibodies to other coronaviruses (NL63 and HKU1) and one sample with Epstein Barr viral capsid IgG.

Agreement

Table 4 summarizes the agreement between the COVID-19 IgG/IgM Duo and Elecsys anti-SARS-CoV-2 assay. For the COVID-19 IgG/IgM duo assay the test was considered positive when either IgM or IgG was positive. Overall there was agreement between the two assays; however, the agreement was highly dependent on the time of testing after the onset of the symptoms. The agreement was especially low on samples collected within the first week after the onset of symptoms and increased with samples collected at a later time point after onset of symptoms.

Table 4.

Agreement between the COVID-19 IgG/IgM Duo and Elecsys anti-SARS-CoV-2 assay depending on the onset of symptoms

Overall			Elecsys anti-SARS-CoV-2
			Pos		Neg		Total
COVID-19 IgG-IgM Duo (IgM or IgG)	Pos		30 (29%)		15 (14%)		45
	Neg		0 (0%)		60 (57%)		60
	Total		30		75		105
% overall agreement: 85.7% % positive percent agreement: 100.0% % negative percent agreement: 80.0% Kappa value (95% CI): 0.696 (0.560–0.831)
0–6 days			Elecsys anti-SARS-CoV-2
			Pos		Neg		Total
COVID-19 IgG-IgM Duo (IgM or IgG)	Pos		2 (5%)		7 (19%)		9
	Neg		0 (0%)		28 (76%)		28
	Total		2		35		37
% overall agreement: 81.1% % positive percent agreement: 100.0% % negative percent agreement: 80.0% Kappa value (95% CI): 0.302 (−0.032–0.635)
7–13 days				Elecsys anti-SARS-CoV-2
				Pos		Neg		Total
COVID-19 IgG-IgM Duo (IgM or IgG)		Pos		24 (43%)		9 (16%)		33
		neg		0 (0%)		23 (41%)		23
		total		24		32		56
% overall agreement: 83.9% % positive percent agreement: 100.0% % negative percent agreement: 71.9% Kappa value (95% CI): 0.687 (0.508–0.865)
≥14 days				Elecsys anti-SARS-CoV-2
				Pos		Neg		Total
VID-19 IgG-IgM Duo (IgM or IgG)		pos		34 (57%)		5 (8%)		39
		neg		0 (0%)		21 (35%)		21
		total		34		26		60
% overall agreement: 91.7% % positive percent agreement: 100% % negative percent agreement: 80.8% Kappa value (95% CI): 0.826 (0.683–0.970)

Although the COVID-19 IgG/IgM Duo is commercialized as a qualitative assay it allows quantitative detection of antibody levels by means of the ratio of fluorescence at the target zone to the reference zone. Overall, there was a significant positive correlation between the days of sampling since the onset of the symptoms and the IgG and IgM ratios (Pearson Correlation Coefficient 0.41 (IgG), 0.38 (IgM), p < 0.01 for both). When comparing the three sampling intervals, the antibody ratio differed significantly from the samples of COVID-19 negative patients from the time point of 7 to 13 days onwards after the onset of symptoms. Moreover, the antibody ratio in the group analyzed 0–6 days after symptoms was significantly lower as compared to more than 14 days after symptoms with both IgM as IgG although this difference was more pronounced with IgG.

Discussion

Up to date, numerous serological assays for SARS-CoV-2 are available or are being developed. Several indications for serological testing have been proposed such as providing epidemiological data and investigating seroconversion dynamics [8]. Although the diagnostic value of SARS-CoV-2 serological testing is still a matter of debate Pancrazzi et al. demonstrated that combining RT-PCR with serology increases the diagnostic sensitivity [5]. Indeed, false negative RT-PCR tests are described due to inappropriate sampling, low viral loads or unsuitable transport conditions [9,10]. In this diagnostic setting, rapid POCT SARS-CoV-2 serological assays could be valuable at for example emergency departments to evaluate the risk of infection and guide isolation measurements until results of RT-PCR analysis are available, to confirm SARS-CoV-2 infection in patients with false negative RT-PCR results with clinical and radiological findings compatible with COVID-19 [11] or to guide the interpretation of weakly positive RT-PCR results.

The present study compared SARS-CoV-2 IgM/IgG antibody detection via a POCT assay from NanoEntec with a fully automated CLIA assay from Roche. Although this lateral flow assay from NanoEntec provides a fast result (3 min) it has to be mentioned that the assay requires some technical actions (including the use of an micropipette) which can be a problem with non-laboratory staff.

Overall there was an agreement between the two assays, mainly in samples taken more than one week after the onset of the COVID-19 symptoms since considerable disagreement was observed in samples obtained at earlier time points. Overall, the diagnostic sensitivity of this POCT SARS-CoV-2 antibody assay seemed higher as compared to the standard chemiluminescent immunoassay from Roche Diagnostics (88.9% vs. 74.1%). Furthermore, sensitivity was highly dependent on the time when samples were obtained. Both assays showed an increase in sensitivity with increased time to onset of the COVID-19 symptoms, which is similar to other studies evaluating both lateral flow assays and automated immunoassays. Moreover, sensitivity and specificity of the COVID-19 IgG/IgM Duo assay was comparable with other lateral flow assays were a sensitivity of >91% 14 days after the onset of COVID-19 symptoms and a specificity of >85% was reached for the combined result of IgG and IgM [12–14]. The difference in sensitivity between the two assays was most pronounced for samples obtained within two weeks after the onset of symptoms. For samples collected at a later time point, the diagnostic sensitivity was comparable between the two methods (100% for COVID-19 IgG/IgM Duo vs 94.4% for Elecsys anti-SARS-CoV-2). When comparing the COVID-19 IgG/IgM Duo assay to other POCT assays described in literature in samples collected early after the onset of symptoms a large variability in sensitivity (26.7–77.8%) is observed. Although these results cannot directly be compared to ours with the COVID-19 IgG/IgM Duo assay because of the differences in sample collection time, the same trend in POCT assays of increasing sensitivity with increased sample time after the onset of symptoms was observed. However, it has to be noted that these other POCT assays are lateral flow assays where a result was based on the visual examination of a test line, which could be subject to variations in interpretation. The NanoEntec assay on the contrary provides an objective positive or negative result based on a predefined cut off index [14].

There was also a striking difference in sensitivity when analyzing the results based on time post positive PCR and based on the time of the onset of symptoms. This can be explained by the large variety in time between the onset of the symptoms and the positive PCR results (between 0 and 15 days with an average of 5,3 days). Because of this high sensitivity with increasing sampling time after the onset of symptoms serological assays can provide a complementary diagnostic value to PCR testing for which an opposite trend is observed [15]. Indeed, Zhao et al. demonstrated that RNA detectability with RT-PCR decreased from 66.7% in samples obtained within 7 days after the onset of symptoms to 45.5% in samples collected after 15 days after the onset of symptoms [16], which is in line with other data showing that in the second week following symptom onset serology testing would even be superior to PCR testing [11].

The specificity of the COVID-19 IgG/IgM Duo assay was slightly lower than the Elecsys anti-SARS-CoV-2 assay since no false positive results were obtained with the assay from Roche while 3 false positive results were obtained with the NanoEntek assay. However, in this particular study, samples with potential cross reactivity were specifically selected and therefore, the specificity on routine samples is presumed to be higher. Moreover, when using the NanoEntec POCT assay for diagnostic purposes, a high sensitivity is preferred over a high specificity. However, because the COVID-19 IgG/IgM Duo assay provides an antibody ratio, the cutoff can be adjusted using ROC-curve analysis when this assay is intended to be used for screening purposes (e.g. to collect epidemiological data).

Determination of both IgM and IgG antibodies has the ability to distinguish between patients with recent/active infection and associated contagiousness and non-contagious individuals [15]. However, seroconversion dynamics for SARS-CoV-2 are not completely clear. While in some studies IgM antibodies were detected prior to IgG antibodies [16,17], Liu X et al reported a concurrent detection of IgM and IgG [18]. Interestingly, a third type of seroconversion was described whereby IgG appears earlier than IgM [19]. In the present study, seroconversion of individual patients was not evaluated. However, sensitivity of IgM in COVID-19 IgG/IgM Duo was higher than IgG in samples taken within 6 days after onset of symptoms while IgG displayed a higher sensitivity in samples taken thereafter, indicating the occurrence of IgM antibodies prior to IgG antibodies. In a significant amount (37%) of patients no IgM antibodies could be detected while 100% of patients developed IgG antibodies more than 14 days after the onset of symptoms.

Serological assays are also useful in the evaluation of immune response after vaccination. This immune response is directed against the spike protein as the currently approved COVID vaccines are spike antigen based. An additional advantage of the NanoEntec POCT assay which is N protein based is the detection of antibodies originating from natural infection without interference of vaccine provoked antibodies [20].

To conclude, this study shows that rapid serological testing for SARS-CoV-2 by means of the FREND^TM COVID-19 IgG/IgM Duo POCT assay from NanoEntec provides accurate results with an equivalent sensitivity and specificity as the reference automated immunoassay. Sensitivity of both assays was highly dependent on the time interval between the onset of the COVID-19 symptoms and the serum sampling whereby sensitivity was nearly 100% in samples taken 14 days after the onset of symptoms allowing this serological assay to be of added value in a specific clinical context when RT-PCR results are unexpected, doubtful or unavailable.

Declarations

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

This work forms part of RECOVER (Rapid European COVID-19 Emergency Response research). RECOVER is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No 101,003,589.

Disclosure of potential conflicts of interest

No potential conflict of interest was reported by the author(s).