Diagnostic accuracy of Augurix COVID‐19 IgG serology rapid test

Abstract

Aims

To validate the diagnostic accuracy of the Augurix SARS‐CoV‐2 IgM/IgG rapid immunoassay diagnostic test (RDT) for COVID‐19.

Methods

In this unmatched 1:1 case‐control study, blood samples from 46 real‐time RT‐PCR–confirmed SARS‐CoV‐2 hospitalized cases and 45 healthy donors (negative controls) were studied. Diagnostic accuracy of the IgG RDT was assessed against both an in‐house recombinant spike‐expressing immunofluorescence assay (rIFA), as an established reference method (primary endpoint), and the Euroimmun SARS‐CoV‐2 IgG enzyme‐linked immunosorbent assays (ELISA) (secondary endpoint).

Results

COVID‐19 patients were more likely to be male (61% vs 20%; P = .0001) and older (median 66 vs 47 years old; P < .001) than controls. Whole blood IgG‐RDT results showed 86% and 93% overall Kendall concordance with rIFA and IgG ELISA, respectively. IgG RDT performances were similar between plasma and whole blood. Overall, RDT sensitivity was 88% (95% confidence interval [95%CI]: 70‐96), specificity 98% (95%CI: 90‐100), PPV 97% (95%CI: 80‐100) and NPV 94% (95%CI: 84‐98). The IgG‐RDT carried out from 0 to 6 days, 7 to 14 days and > 14 days after the SARS‐CoV‐2 RT‐PCR test displayed 30%, 73% and 100% positivity rates in the COVID‐19 group, respectively. When considering samples taken >14 days after RT‐PCR diagnosis, NPV was 100% (95%CI:90‐100), and PPV was 100% (95%CI:72‐100).

Conclusions

The Augurix IgG‐RDT done in whole blood displays a high diagnostic accuracy for SARS‐CoV‐2 IgG in high COVID‐19 prevalence settings, where its use could be considered in the absence of routine diagnostic serology facilities.

1. INTRODUCTION

Serology‐based assays have not only been shown to be clinically relevant to COVID‐19 diagnosis in specific cases, but can also indicate the true nature and extent of the population's exposure to SARS‐CoV‐2. Although SARS‐CoV‐2 serological response should not currently be considered a surrogate of individual protection, ¹ assessing broad‐scale individual COVID‐19 serology is likely to be a cornerstone of public health policies during the epidemic after the initial infection‐wave. In this context, the availability of simple, robust and reliable rapid diagnostic tests (RDT) for determining SARS‐CoV‐2 exposure will be an asset in the diagnostic arsenal for identification of previous SARS‐CoV‐2 infections. Although tens of RDTs have been developed, only a few validation studies addressing the performance of these tests compared to reference methods have been published so far. ² , ³ , ⁴ We identified three studies of interest. The first showed that the IgG results displayed a 95% specificity in a cohort of PCR‐confirmed COVID‐19 cases, without comparing the IgG serology results to results obtained with an IgG serology reference method. ² Another study had a prospective cohort of 150 patients displaying fever and/or respiratory symptoms and a RDT measuring combined IgM/IgG serology, which had a 71.1% sensitivity and 96.2% specificity against PCR‐confirmed COVID‐19 diagnosis, resulting in 97.2% PPV and 64.6% NPV. ³ A third study performed in an emergency department on 50 consecutive PCR‐confirmed COVID‐19 patients admitted for respiratory symptoms during the acute phase of COVID‐19, did not make a strong case for a RDT serology‐based diagnosis in these settings, with a sensitivity of 18.4%, a specificity of 91.7%, a NPV of 26.2% and a PPV of 87.5%. ⁴

These studies showed suboptimal performance that can be explained by the accumulation of different biological and analytical factors. At the biological level, the timing and the kinetics of the antibody response are key factors, and full seroconversion is likely to take 2‐3 weeks. Among analytical factors, the substantial heterogeneity of antigens used in the numerous available immunoassays (including RDTs) is of importance. Indeed, these assays rely either on the antibody response against the full spike protein (S), or against its subdomains 1 (S1) or 2 (S2), against the nucleocapsid (N) or against a combination thereof. ⁵ If and how the performance of these SARS‐CoV‐2 assays can be influenced by the antigens used is still unclear, but it can be expected that assays using the combination of full S and N antigens would be optimal at least in terms of sensitivity and may improve specificity. In this respect, a CE/IVD‐approved RDT using full S and N antigen (Augurix; GaDia, Switzerland) has recently been released, but a thorough evaluation of its performances is still lacking.

We aimed to validate this SARS‐CoV‐2 IgM/IgG serological RDT in a high COVID‐19 prevalence setting, using a recently established recombinant immunofluorescence assay as the reference method ⁶ derived from the reference method for MERS‐CoV diagnosis. ⁷

2. METHODS

2.1. Study population and blood sample collection

In accordance with our institution's ethical committee and national regulations, anonymised leftovers of whole blood EDTA (hereafter WB‐EDTA) were used for this method evaluation. Only laboratory‐based information was used in this study. The reporting of this study conforms to broad EQUATOR guidelines as published by Simera and colleagues. ⁸ We included blood samples of 46 real‐time RT‐PCR–confirmed COVID‐19 cases hospitalized at the University Hospitals of Geneva. These patients had COVID‐19 illness severities ranging from moderate to critical, since patients suffering from mild disease were not admitted in this hospital only for the purpose of isolation. We also included 45 unmatched control blood samples from asymptomatic donors without known exposure to SARS‐CoV‐2, who were not tested by RT‐PCR, since they did not meet the testing criteria of our institution. These healthy donors met the blood donation criteria at our institution: age 18‐65 years old, absence of known acute or chronic infection and without history of cancer, diabetes, or haematological disorders, as well as cardiovascular, autoimmune, inflammatory, chronic kidney or neurological disease. Leftovers from blood specimens (whole blood and plasma) from single patients or controls, collected at a single time point, were tested for various immunoassays. The control blood samples were obtained during the same period (April 2020). WB‐EDTA samples were used as a proxy of capillary blood and were centrifuged (3000 g 10 minutes) in parallel to generate EDTA plasma. All analyses (see below) were performed within 72 hours of blood sampling without any freezing‐thawing cycle. The 46 COVID‐19 samples were categorized according to the number of days post real‐time RT‐PCR positivity, days post‐diagnosis (DPD).

2.2. SARS‐CoV‐2 RT‐PCR analyses

SARS‐CoV‐2 RT‐PCR was performed according to manufacturers’ instructions on various platforms, initially including an in‐house method using eMAG (bioMérieux, France) and the Charité RT‐PCR protocol, ⁹ or the BD SARS‐CoV‐2 reagent kit for the BD Max system (Becton, Dickinson and Co, US) or Cobas 6800 SARS‐CoV‐2 RT‐PCR (Roche).

2.3. Augurix IgM/IgG immunochromatographic rapid cassette test

This commercially available SARS‐CoV‐2 IgM/IgG RDT can be used either with capillary blood, whole blood, plasma or serum. One IgM/IgG rapid test per sample was used. According to the manufacturers’ instructions, 20 µL of whole blood (as a proxy for one capillary blood drop) and 10 µL of plasma were applied in parallel for each samples. IgG and IgM responses were read after 10 minutes following manufacturer's instructions, blinded to the reference method results.

2.4. SARS‐CoV‐2 spike‐based recombinant immunofluorescence assay

Determination of the IgG antibody response against the complete spike (S) protein (both S1 and S2 domains) of SARS‐CoV‐2 was performed by recombinant immunofluorescence assay (rIFA) as previously validated for MERS‐CoV ⁷ , ¹⁰ and adapted to SARS‐CoV‐2. ⁶ Briefly, Vero B4 cells, transfected with the SARS‐CoV‐2 Spike protein expression vector pCG1‐SCOV2‐S (kindly provided by M. Hoffmann and S. Pöhlmann, DPZ), were seeded on multi‐test glass slides, washed and fixed. After appropriate rehydration and blocking, 30 μL of diluted plasma samples was applied on each spot and incubated for 60 minutes at room temperature (RT). Following washing, secondary goat anti‐human‐IgG antibody conjugated with Alexa488 (Jackson ImmunoResearch #109‐545‐088) was applied to each spot and incubated at RT for 45 minutes. ⁶ , ¹⁰ The rIFA were results were assessed by two independent readers, blinded to the COVID‐19 status, with good inter‐observer kappa coefficient. ⁶

2.5. SARS‐CoV‐2 IgG ELISA

Euroimmun IgG ELISA uses the S1‐domain of the spike protein of SARS‐CoV‐2 as antigen. EDTA plasma was diluted at 1:101 and assessed with the IgG CE‐marked ELISA (Euroimmun AG, Lübeck, Germany # EI 2606‐9601 G) according to the manufacturer's instructions. The assays were run on Dynex Agility (Ruwag) according to the manufacturer's protocol. After adding the conjugate, samples’ immunoreactivity was measured at an optical density of 450nm (OD450) and then adjusted to an internal calibrator to minimize inter‐assay variation. ⁶ , ¹¹ The quantitative results (ratios) were then expressed in arbitrary units and interpreted following the cut‐offs derived from our validation study: OD ratio: <0.5 = negative, ≥0.5 and < 1.5 = indeterminate, ≥1.5 = positive. ⁶

2.6. Study end points

The primary endpoint was to assess the accuracy of IgG detection in whole blood (as a surrogate for capillary blood) by this IgM/IgG RDT against the rIFA reference method (which identifies IgG targeting both S1 and S2 subunits of SARS‐CoV‐2 spike protein), within a cohort of 46 RT‐PCR–confirmed COVID‐19 and 45 control blood samples.

The secondary endpoints were to assess: (a) the IgG rapid test performances against a commercially available ELISA‐based IgG serological immunoassay (Euroimmun), (b) the concordance of the rapid IgM/IgG test results in whole blood vs EDTA plasma, c) the performance of the RDT against rIFA and IgG ELISA within each of the COVID‐19 DPD subgroups, 0‐6, 7‐14 and >14 days. Due to the absence of an available comparison method for IgM (not provided by Euroimmun) testing in our institution, we did not formally assess IgM detection performance by the rapid test and its potential added value in individuals infected with SARS‐CoV‐2.

2.7. Statistics

Vassarstats online tool (www.vassarstats.net) was used to calculate proportions, 95% confidence intervals, median and interquartile range (IQR), while significance (P‐values) was calculated using Fisher's exact test for categorical variables and the Mann‐Whitney U test for continuous variables. Concordance between immunoassays was assessed with Kendall's coefficient, which was calculated using Statistica (version 13.5.0.17, TIBCO Software Inc). Vassarstats online tool was also used to determine sensitivity (SN), specificity (SP), and positive and negative predictive values (PPV and NPV). As previously published, indeterminate rIFA IgG and ELISA IgG results were considered to be negative for test performance and concordance analyses, in order to maximize specificity. ⁶ Statistical significance was defined as P < .05.

3. RESULTS

3.1. Baseline characteristics

The baseline demographic characteristics of participants were as follows: the 46 RT‐PCR–confirmed COVID‐19 hospitalized patients were older (median 66 years old, IQR 50.5‐76) compared to the healthy blood donors (median 47 years old, IQR 39.5‐55; P = .0001). The COVID‐19 cohort had a higher proportion of males (n = 28, 60.9%) compared to the healthy controls (n = 9 males 20%; P = .0001). Among COVID‐19 patients, the median delay between a SARS‐CoV‐2 RT‐PCR diagnostic test and serology testing was 10 days (IQR 5‐15 days). The proportion of patients within each DPD subgroup (delta between their molecular testing and serological testing) was 43.5% for 0‐6 days (n = 20), 30.4% for 7‐14 days (n = 14) and 26.1% for >14 days (n = 12). In the majority of cases (45/46), the RT‐PCR tests were done in nasopharyngeal secretions. On one occasion, the RT‐PCR was carried out on a bronchial aspirate. The median CT value was 23.9 (IQR 19.45‐27.55).

3.2. rIFA IgG in RT‐PCR–confirmed COVID‐19 cases and negative controls

Overall, 67% of RT‐PCR–confirmed COVID‐19 patients were IgG seropositive by rIFA, as shown in Table 1. All patients within the DPD >14 days subgroup were seropositive by rIFA. As expected, the positivity rate of each tested antibody detection method increased in later DPD subgroups. Among the 45 negative controls, 44 (98%, 95%CI: 88‐100) were negative in rIFA; one sample was positive in rIFA, as well as in the IgG RDT (in both WB and plasma), and in ELISA IgG immunoassays (ratio 2.81). Detailed results of each immunoassay for both COVID‐19 and the control samples are available in Supplementary Materials.

Table 1.

SARS‐CoV‐2 IgM/IgG detection characteristics by various immunoassays

Samples	No, (%)	RDT IgM Positive No (%)		RDT IgG Positive No (%)		rIFA IgG Positive No (%)	ELISA IgG
		WB‐EDTA	plasma EDTA	WB‐EDTA	plasma EDTA		Positive a No (%)	Ratio Median (IQR)
Negative controls	45 (100)	0	0	1 (2)	1 (2)	1 (2)	1 (2)	0.35 (0.27‐0.43)
COVID‐19 patients
All	46 (100)	1 (2)	5 (11)	28 (61)	30 (65)	31 (67)	27 (59)	6.11 (0.40‐11.82)
DPD 0‐6	20 (44)	0	1 (5)	6 (30)	8 (40)	9 (45)	6 (30)	1.51 (0.24‐2.79)
DPD 7‐14	14 (30)	0	2 (14)	10 (71)	10 (71)	10 (71)	9 (64)	6.20 (0.51‐11.89)
DPD > 14	12 (26)	1 (8)	2 (17)	12 (100)	12 (100)	12 (100)	12 (100)	12.08 (10.60‐13.55)

Abbreviations: DPD, days post‐diagnosis; RDT, Augurix rapid diagnostic test; rIFA, recombinant immunofluorescence assays; WB, whole blood.

^a Cut‐offs < 0.5 = negative, ≥0.5 and <1.5 = indeterminate, and ≥1.5 = positive.

3.3. Diagnostic accuracy of IgG RDT against IgG S‐rIFA

IgG RDT accuracy against IgG identification by S‐rIFA immunoassays is shown in Tables 2 and 3. Both methods revealed similar results in 86 of 91 samples 94.5% (95%CI: 88%‐98%), and their concordance was evaluated at 0.86 (Kendall τ correlation coefficient, P < .0001). One discordant result showed an IgG RDT positive result while being negative by rIFA (false‐positive) and four samples tested negative by IgG RDT but tested positive by rIFA (false‐negatives). These resulted in an IgG RDT sensitivity (SE) of 88% (95% CI: 70‐96) and a specificity (SP) of 98% (95% CI: 90%‐100%), while the PPV was 97% (95% CI: 80‐100) and the NPV 94% (95% CI: 84‐98), in this unmatched 1:1 case‐control study.

Table 2.

Concordance between Augurix RDT vs rIFA and Euroimmun ELISA results

Kendall τ concordance coefficient a		rIFA IgG	ELISA IgG
IgG RDT	Whole blood EDTA	0.86	0.93
	Plasma EDTA	0.81	0.88

Abbreviation: RDT, Augurix rapid diagnostic test

^a P < .0001.

Table 3.

Augurix RDT performances against rIFA and Euroimmun ELISA

	SE % (95%CI)	SP % (95%CI)	PPV% (95%CI)	NPV% (95%CI)
All cases (n = 91), IgG rIFA seropositivity 35%
IgG RDT (WB) vs rIFA	88 (70‐96)	98 (90‐100)	97 (80‐100)	94 (84‐98)
IgG RDT (WB) vs ELISA	96 (80‐100)	97 (88‐99)	93 (76‐99)	98 (90‐100)
DPD 0‐6 and controls (N = 65), IgG rIFA seropositivity 15%
IgG RDT (WB) vs rIFA	70 (35‐92)	100 (92‐100)	100 (56‐100)	95 (85‐99)
IgG RDT (WB) vs ELISA	86 (42‐99)	98 (90‐100)	86 (42‐99)	98 (90‐100)
DPD 7‐14 and controls (N = 59), IgG rIFA seropositivity 18%
IgG RDT (WB) vs rIFA	91 (57‐100)	98 (88‐100)	91 (57‐100)	98 (57‐100)
IgG RDT (WB) vs ELISA	100 (66‐100)	98 (88‐100)	91 (57‐100)	100 (91‐100)
DPD > 14 and controls (N = 57), IgG rIFA seropositivity 23%
IgG RDT (WB) vs rIFA	100 (72‐100)	100 (90‐100)	100 (72‐100)	100 (90‐100)
IgG RDT (WB) vs ELISA	100 (72‐100)	100 (90‐100)	100 (72‐100)	100 (90‐100)

Abbreviations: NPV, negative predictive value; PPV, positive predictive value; RDT, Augurix rapid diagnostic test; rIFA, recombinant immunofluorescence assays; SE, sensibility; SP, specificity.

As shown in Table 1, most false‐negatives (3 out of 4) were tested in the DPD 0‐6 days’ subgroup, and none were observed after >14 days DPD. As a secondary end point, we analysed the IgG RDT performance in each DPD subgroup and in the negative controls. As expected, the longer the delay between a positive PCR and serology testing, the higher the IgG rIFA positivity rate was, as were the IgG RDT SE and NPV, resulting in 100% (both) in the DPD >14 subgroup (Table 3). From 7 days post RT‐PCR and beyond, the IgG RDT performances against IgG rIFA resulted in 96% SE (95% CI: 76‐100) and 98% NPV (95% CI: 88‐100).

3.4. Diagnostic accuracy of IgG RDT against IgG ELISA

IgG RDT (WB‐EDTA) and Euroimmun IgG ELISA thus showed a concordance coefficient of 0.93 (Kendall τ, P < .0001). One false‐negative IgG RDT and 1 false‐positive IgG RDT were identified, resulting in a SE of 96% (95%CI: 80‐100) and SP of 97% (95%CI: 88‐99). This resulted in 93% PPV (95%CI: 76‐99) and 98% NPV (95%CI: 90‐100). The subanalysis of each DPD is shown in Table 3.

3.5. IgG RDT in WB‐EDTA vs plasma

Out of 91 paired samples, IgG RDT carried out in WB‐EDTA compared to plasma gave similar results in 98% of cases (n = 89, 95%CI: 92‐99). Plasma IgG RDT against rIFA resulted in 88% SE (95CI%: 70‐96) and SP 95% (95%CI: 85‐99); thus, there was no obvious gain when compared to the performances of WB‐EDTA IgG RDT against the same reference method.

3.6. IgM RDT in WB‐EDTA vs plasma EDTA

IgM positivity rates within this cohort were low at 2% for WB‐EDTA and 11% for plasma (Table 1). We cannot establish here if these differences would indicate a significant sensitivity gain in plasma rather than WB (or capillary blood), due to the absence of a reference method (absence of IgM detection by Euroimmun ELISA). Nevertheless, we suspect that, in contrast to IgG, plasma might be more suitable for IgM detection by this IgM/IgG RDT immunoassay.

4. DISCUSSION

The key finding of the present validation study, using an unmatched 1:1 case‐control design including 50.5% laboratory‐confirmed COVID‐19 cases, is that the diagnostic accuracy of IgG Augurix RDT on whole blood when compared to rIFA on plasma displayed a SE of 88%, a SP of 98%, a PPV of 97% and a NPV of 94%. This good Kendall concordance coefficient ¹² retrieved with rIFA (0.86) can be explained by the fact that both assays target the immune response against full‐length S proteins of SARS‐CoV‐2. ⁵ Furthermore, our secondary analyses indicated the diagnostic accuracies of this RDT further increased when analyses were done on samples collected after 6 DPD and reached optimal NPV and PPV (100%) after 14 DPD. Such performances indicate that this RDT could be fit for purpose in clinical settings where a high prevalence of COVID‐19 prevails, but we need to be very cautious before promoting such tests at large scale. The performance in low prevalence populations still needs to be determined, and larger populations need to be tested in particular using whole blood. These results contrast with prior studies using other RDTs which indicated that the PPV and NPV were lower, albeit with a similar proportion of cases vs controls. ³ , ⁴ The reasons underlying such differences are still unknown but can at least be partly explained by inter‐assay analytical differences. In addition, one major difference is that the DPD might vary and be very short for the majority of cases in other studies. ⁴ Of note, the performance of the assay when carried out in the first week after diagnostic is imperfect and potentially limits its use as a potential diagnostic tool during the acute phase. Finally, here we used whole blood, and the test was performed in a laboratory environment; we may expect different results in real life at the bedside and using capillary blood.

The second notable finding of this study lies in the fact that for IgG serology, whole blood‐derived results—a validated proxy of capillary blood for antibody assessment ¹³ —performed as well as plasma. Although expected, this point still needed to be demonstrated for SARS‐CoV‐2 serology, because it establishes the proof of principle that capillary blood is an adequate medium for SARS‐CoV‐2 serology. Providing that adequate analytical quality is met by SARS‐CoV‐2 RDTs, the use of capillary blood alleviates two major pre‐analytical hurdles that usually prevent the use of point‐of‐care devices by the individuals themselves, such venous blood sampling and the need for sample centrifugation. Such operational optimization confers this RDT with the potential to represent an alternative to routine laboratory‐assessed COVID‐19 IgG serology. In addition, the test was easy to handle and provided clear results, without indeterminate or invalid results, and without inter‐operator variability. Whether the use of capillary blood would also be adequate to assess IgM SARS‐CoV‐2 serology nevertheless remains to be demonstrated.

There are several limitations to this study. First, we present here the results of a method validation study and not a seroprevalence study. Therefore, the PPV obtained here (based on a 50.5% proportion of cases defined as laboratory‐confirmed SARS‐CoV‐2 by RT‐PCR) will probably be lower in a low prevalence setting. We used the date of the RT‐PCR diagnosis to perform the DPD subanalysis, and this should not be misinterpreted as days after symptoms onset. Although it is likely an additional limitation of this study, it did provide an objective date to compare the serologies, when symptoms exact onset date can sometimes be difficult to establish. Nevertheless, our results also showed that when targeting a population after one week of diagnosis by RT‐PCR, PPV and NPV remained high (above 90%). One control blood sample turned out to be IgG positive by RDT, but also by rIFA and ELISA. Since the results were concordant across the immunoassays tested here, it suggests a possible SARS‐CoV‐2 serological scar within the control population of blood donors collected during the epidemic period. Although not fully established here, we observed that the RDT results were consistent with the other tests. Another limitation of this validation study lies in its limited sample size leading to broad 95% confidence intervals, requiring confirmation of these data at a larger scale. Finally, our present conclusions only apply to the Augurix RDT and must not be applied to any other RDTs currently available.

In conclusion, this RDT is not meant to replace a SARS‐CoV‐2 RT‐PCR diagnostic test in the first week of the disease, but could be a reliable option in the toolbox for assessing the SARS‐CoV‐2 serology in moderate to high COVID‐19 prevalence settings, especially in situations where ELISA are not available, or cannot be reliably used. Further investigations in low prevalence situations and using capillary blood are necessary, as are head‐to‐head comparisons among available RDTs.

CONFLICT OF INTEREST

The authors have no conflict of interest disclosure to report.

AUTHOR CONTRIBUTIONS

DOA, NV and LK designed the study. PC, BM, GT, SY, IAV, PLR and LF performed the research. AC, LM, TA, JLR, JS, IG and SS collected the data. DOA, CAS, IE, LK and NV analysed the data. DOA, TA, LK and NV wrote the paper.

Supporting information

Table S1‐S2

Andrey DO, Cohen P, Meyer B, et al. Diagnostic accuracy of Augurix COVID‐19 IgG serology rapid test. Eur J Clin Invest. 2020;50:e13357. 10.1111/eci.13357