Diagnostic Performance of SARS-CoV-2 IgM/IgG Rapid Test Kits for the Detection of the Novel Coronavirus in Ethiopia

Abstract

Purpose

Rapid severe acute respiratory syndrome coronavirus 2 test kits are crucial for bridging diagnostic gaps in health facilities and community screening mainly in resource limited settings. However, there is no objective evidence on their diagnostic performance. Thus, the study aimed to evaluate comparative diagnostic performance of three selected SARS-CoV-2 IgG/IgM rapid test kits in Ethiopia.

Methods

A cross-sectional study was conducted among 200 clients between May and July 2020 in Addis Ababa, Ethiopia. The performance of three SARS-CoV-2 rapid test kits EGENE, CTK BIOTECKs Onsite, and ACON Biotech were evaluated using blood specimens against RT-PCR on respiratory swabs. Sensitivity, specificity, and agreement with each other and to RT-PCR were computed using Vassarstats, MedCalc and SPSS version 23 statistical software.

Results

Test kits showed a heterogeneous comparative diagnostic performance in their sensitivity and specificity. The sensitivity was 61.18% (95% CI: 49.96–71.37%), 74.12% (95% CI: 63.28–82.74%) and 83.53% (95% CI: 73.57–90.38%) for kit A, B and C, respectively. Similarly, the specificity was 96.52% (90.81–98.88%), 94.78% (88.52–97.86%) and 94.78% (88.52–97.86%) for test kit A, B and C, respectively. The test kits have an agreement with RT-PCR with kappa value of 0.60 (0.48–0.83), 0.71 (0.65–0.93), and 0.80 (0.76–1.04) for A, B, and C, respectively. There was a significant difference on diagnostic performance among the three test kits and PCR with a p-value < 0.001 Cochran’s Q test.

Conclusion

The diagnostic performance of the test kits was promising and recommended for COVID-19 diagnostics in combination with RT-PCR to detect more infected patients. It allows determining the seroprevalence of the virus and true extent of SARS-COV-2 community spread in resource limited settings. We underline countries to evaluate rapid diagnostic test kits before diagnostic use.

Background

Corona virus disease 2019 (COVID-19) is a current public health pandemic and become the biggest threat of recent decades with 1,205,432 deaths from 46,834,497 cases globally as of 2 November 2020, since it was first reported in China, Wuhan, in late December 2019.^1–3 As many of the cases are asymptomatic, without laboratory testing, positive cases will not be identified and sources of infection could not be traced, WHO recommends “test, test, test” as a very important means of mitigation and control of the pandemic. Thus, affordable and user friendly⁴ laboratory test kits, mainly rapid tests which are more realistic for low-income countries that could be managed by minimal trained personnel and resources should be considered for this context.^5,6 Many rapid diagnostics that can detect SARS-CoV-2 have been and are under development, both in-house and commercially.^7–9 Despite its relatively low cost and simplicity of rapid laboratory test kits, their quality is under scrutiny that exposed and weakened the health systems and forced the countries and diagnostic companies to tousled for rapid tests, but prioritized as of its availability near to the patients.^10–12

The health system and laboratory diagnostic capacities in Africa were questionable in detecting outbreaks as early as possible, and the 2,030 sustainable development goal (SDG) begs the question of realization with big gaps at the diagnosis stage and so many people failing to get diagnosed, though many targets were accomplished.^13,14 Likewise, as of 20 October 2020, the number of COVID-19 laboratories in Ethiopia was not more than 46, with a longer turnaround time of results, which was not in line with the speed of the virus escalation.^15,16 In addition, the current recommended “gold standard” test for COVID-19 is based on real time reverse transcriptase PCR (rRT-PCR). It has limitations in its sensitivity and the procedure is relatively tedious and is prone to contamination, it also requires state-of-the-art-laboratory equipment with costly supplies and skilled professionals.^17,18

These challenges forced scientists to develop accurate, reliable and rapid COVID−19 diagnostic methods; however, evaluating their performance and introducing quality rapid test kits that can help to curve the COVID 19 pandemic by identifying, screening and tracing the source of infection is necessary. Thus, this study was done which aimed to compare and evaluate the diagnostic performance of selected SARS-CoV-2 IgG/IgM rapid test for the detection of the novel coronavirus against the currently established RT-PCR in Ethiopia.

Materials and Methods

Study Sites, Design and Period

A multicenter cross sectional study was conducted from May to July 2020 in Addis Ababa COVID-19 isolation and testing centers: EPHI national influenza reference laboratory, Ekka Kotebe general hospital, Yekatit 12 hospital medical college, Addis Ababa Health Bureau Public Health Research and Emergency Management (AAPHREM) center. These sites were amongst the first national COVID-19 testing and treatment centers and samples come from other referring health facilities of Addis Ababa.^19–21

Study Participants Eligibility Criteria

Overall, 540 sequentially ordered clients were screened with symptoms of COVID-19 in the study period. Among these, 200 clients who were volunteer to participate, gave written informed consent and assent for participation and having sign and symptoms of COVID-19 such as fever BT≥ 37.5, cough, sore throat, runny nose and sneezing during presentation to the health facilities were include.²² Critical patients who were unable to communicate were not included. The study participants were recruited as per the current WHO and Ethiopian COVID-19 management guideline through trained public health professionals.^22–24

Rapid Test Kit Selection

The rapid test kits include in this evaluation study were selected considering affordability, FDA approval or listed for approval for emergency use authorization and user friendliness as minimum criteria.²⁵ From ten local pharmaceutical distributors communicated, only three companies were interested and availed the test kits for the evaluation. Accordingly, three SARS-CoV-2 IgG/IgM rapid test kits: EGENE (A), CTK BIOTECKs Onsite (B), and ACON Biotech (C) were selected for performance evaluation to detect novel corona virus against RT-PCR method. For each rapid test kit, we receive 200 samples from the companies’ through their local representatives found here in Ethiopia. However, these pharmaceutical companies had not had any involvement with the research methodology design, analysis and write up of the research manuscript. SARS-CoV-2 IgG/IgM rapid test is a lateral flow chromatographic immunoassay which can detect antibodies against the SARS-CoV-2 virus. The test cassettes consists of: a colored conjugate pad containing SARS-CoV-2 recombinant antigens conjugated with colloidal gold (SARS-CoV-2 conjugates) and a nitrocellulose membrane strip containing an IgG line, an IgM line and the control line (C).^8,28

Data Collection Method

Participants’ socio-demographic characteristics were captured by using a pretested data collection tool. Nasal/throat swabs and blood specimen were collected by using viral transport medium (VTM) and ethylenediamine tetraacetic acid (EDTA) coated vacutainer tubes according to the national SOP with strict bio safety measures.^16,29 The collected swab specimens were sent immediately to molecular laboratories (EPHI) national influenza reference laboratory and Addis Ababa Public Health Emergency Management Center Laboratory (AAPHREM) through triple packaging for testing.

Laboratory Tests

The molecular RT-PCR was performed using Applied Biosystems 7500 real time PCR system and Abbott m2000sp/m2000rt real time PCR plat forms.^23,30 RDT was performed by three IgG/IgM rapid test kits (Nantong Egens Biotechnology EGENE labeled as A, CTK’s Onsite COVID-19 IgG/IgM Rapid Test, CTK BIOTECH labeled as B and ACON Biotech SARS-CoV-2 IgG/IgM rapid test labeled as C) following the manufacturer’s instructions. The results were interpreted by two independent readers.^26–28

Study participants’ PCR results were communicated through the established national emergency operation center, because the country have one channel of result communication for COVID-19, to manage the cases in a centralized manner, while RDT results were communicated at spot.SARS-CoV-2 IgG/IgM rapid test is a lateral flow chromatographic immunoassay which can detect antibodies against the SARS-CoV-2 virus. The test cassettes consists of: a colored conjugate pad containing SARS-CoV-2 recombinant antigens conjugated with colloidal gold (SARS-CoV-2 conjugates) and a nitrocellulose membrane strip containing an IgG line, an IgM line and the control line (C). It was performed following the respective manufacturers’ instructions and results were read at 10 to 20 minutes and did not read results after 20 minutes. Results were interpreted as, in addition to the presence of the C line, if only the G line is developed, the test result indicates the presence of IgG anti-SARS-CoV-2 virus. The result is IgG positive. In addition to the presence of the C line, if only the M line is developed, the test indicates the presence of IgM anti-SARS-CoV-2 virus. The result is IgM positive. In addition to the presence of the C line, if both G and M lines are developed, the test indicates the presence of IgG and IgM anti-SARS-CoV-2 virus, the result is IgG and IgM positive.^8,26–28

Data Quality Assurance

Data collectors were trained on how to collect the necessary data using the data collection tools and additional written guide was made available to them on interpreting each of the study variables. The molecular laboratories: EPHI national influenza reference laboratory and AAPHREM where the laboratory testing conducted are WHO and (Ethiopian National Accreditation Office (ENAO) ISO15189:2012 accredited, respectively.^24,31 RDTs contain inbuilt control feature, C line. Positive and negative controls were tested to ensure the proper performance of the assay.⁷ We used a calibrated micropipette for sample allocation.

Data Analysis and Interpretation

Data were double entered and analyzed using SPSS version 23, Vassarstats and MedCalc statistical software. Sensitivity and specificity of the RDTs were calculated and the performance agreements to RT-PCR were assessed using Kappa statistics. Cochran’s Q test was used to assess whether there is difference in performance among the rapid test kits and RT-PCR. P-values <0.05 were considered for statistical significance P-value< 0.05.

Ethical Consideration

To conduct this research, ethical approval was obtained from Addis Ababa University college of health sciences Institutional review board IRB (protocol # 029/20/Lab), Eka Kotebe hospital IRB protocol # Eka-150-5-4), and Addis Ababa Health Bureau IRB protocol # A/A/H11127/227. Permission to conduct the study was obtained from the concerned institutions: EPHI and Federal ministry of health COVID-19 task force (they registered it and give a permission to conduct the research work). During data and sample collection the data collectors inform each study participant about the purpose and anticipate benefits of the research project and also informed on their full right to refuse, withdraw or completely reject partly or all of their part in the study.

Finally, we obtained written informed consent from adult participants and parents or legal guardians of study participants under the age of 18 years to participate in the study and to use their files and records for the study. All participants’ identifiers were removed and only codes were used throughout the study to keep confidentiality. Moreover, this work was performed as per the Helsinki declaration.

Results

In this study, 200 study participants ranged from 1 month to 95 years with median of 27±13.75 years were included. The majority was male and 33 had co-morbidity conditions which can contribute and dreadful the case of the virus spread. The detail is illustrated in Table 1.

Table 1.

Socio-Demographic Characteristics of Study Participants, 2020 Addis Ababa, Ethiopia

Variables	Frequency	Percent
Age
<15	10	5%
15–30	64	32%
31–45	41	20.5%
46–50	29	14.5%
51–65	37	18.5%
>65	19	9.5%
Sex (%)
Male	120	60%
Female	80	40%
Occupation
Health professionals	32	16%
Student	19	9.5%
Drivers	26	13%
Privately-owned company	58	29%
Government employed	52	26%
Others	13	6.5%
Transport used
Private vehicle	33	16.5%
Government vehicle	36	19.5%
Public civil services vehicles	57	28.5%
Public bus	71	35.5%
exposure
Return from other countries	29	14.5%
Close contact	41	20.5%
Have no known contacts	130	65%
Co-morbidities
Hypertension	12	36.4%
Diabetes	9	27.3%
Cardiovascular disease	3	9.1%
COPD	4	12.1%
Malignancy	2	6%
Cerebrovascular diseases	2	6%
Others	1	3%
Have no known co-morbid conditions	167	83.5%

This study among the specimens, 85 were positive by RT-PCR. The evaluated test kits have a heterogeneous diagnostic performance, with a sensitivity of 61.2%, 74.1% and 83.53% of for A, B and C respectively. The detail is depicted in Tables 2–4.

Table 2.

Comparative Diagnostic Performance of Test Kit “A” with RT-PCR, 2020, Addis Ababa, Ethiopia

Test Kit A	RT-PCR		Totals
	Absent	Present
Test positive	4	52	56
Test negative	111	33	144
Total	115	85	200
	Estimated Value	95% Confidence Interval
		Lower Limit	Upper Limit
Sensitivity	0.611	0.499	0.714
Specificity	0.965	0.908	0.989
True positive (positive predictive value)	0.929	0.819	0.977
True negative (negative predictive value)	0.771	0.692	0.835

Table 4.

Comparative Diagnostic Performance of Test Kit “C” with RT-PCR, 2020, Addis Ababa, Ethiopia

Test Kit C	RT-PCR		Totals
	Absent	Present
Test positive	6	71	77
Test negative	109	14	123
Totals	115	85	200
	Estimated Value	95% Confidence Interval
		Lower Limit	Upper Limit
Sensitivity	0.836	0.736	0.904
Specificity	0.948	0.886	0.978
True positive (positive predictive value)	0.923	0.832	0.968
True negative (negative predictive value)	0.887	0.813	0.934

Table 3.

Comparative Diagnostic Performance of Test Kit “B” with RT-PCR, 2020, Addis Ababa, Ethiopia

Test Kit B	RT-PCR		Totals
	Absent	Present
Test positive	6	63	69
Test negative	109	22	131
Total	115	85	200
	Estimated Value	95% Confidence Interval
		Lower Limit	Upper Limit
Sensitivity	0.741176	0.632819	0.827397
Specificity	0.947826	0.88522	0.978614
True positive (positive predictive value)	0.913043	0.813984	0.964154
True negative (negative predictive value)	0.832061	0.754549	0.889553

The test kits diagnostic sensitivity performance were increased along with the date of clinical onset of the symptom of the patients, ranged from 8% CI95%: 1.39–27.50% to 61.68% CI95%: 45.51–75.25%, from 12.19% CI95%: 4.58–27.01%] to 73.47% CI95%: 57.66–84.60%, and from 13.04% CI95%: 5.42–26.95% to 83.87% CI95%: 65.52–95.90% with test kit A, B and C respectively, for patients having experienced their first symptoms from 0 to more than 15 days of clinical COVID−19 onset symptoms had before the date of testing performed, as depicted in Table 5.

Table 5.

Sensitivity Performance of the Rapid Test Kits with the Date of Clinical Onset of Symptoms of the Clients, 2020, Addis Ababa, Ethiopia

Name of Test Kits		No. of Days Since the Onset of COVID-19 Clinical Symptom
		≤5 Days		6–10 Days		11–15 Days		>15 Days
Test kit A	Pos	2	0	10	1	13	1	27	2
	Neg	23	0	58	0	46	0	17	0
	Sensitivity(95% CI)	8%(1.39–27.5%)		14.71%(7.6–25.85%)		22%(12.68–35%)		61.68%(45.51–75.25%)
Test kit B	Pos	3	0	10	1	17	2	33	3
	Neg	36	0	57	0	26	0	12	0
	Sensitivity(95% CI)	12.19%(4.58–27.01%		14.92%(7.76–26.2%)		39.53%(25.36–55.54%)		73.33%(57.66–84.60%)
Test kit C	Pos	6	0	16	1	23	3	26	2
	Neg	40	0	51	0	27	0	5	0
	Sensitivity(95% CI)	13.04%(5.42–26.95%)		23.88%(14.66–36.13%)		46%(32.06–60.54%)		83.87%(65.52–95.90%)

This study revealed as an overall comparative diagnostic performance of a sensitivity of 61.18% 95% CI: 49.96–71.37%), 74.12% (63.28–82.74%) and 83.53% (73.57–90.38%) of rapid test kit A, B and C respectively. It has an agreement of the three test kits with RT-PCR of kappa value of 95% CI 0.60 (0.48–0.83), 0.71 (0.65–0.93), 0.80 (0.76–1.04) for A, B, and C respectively. Across the three test kit and PCR, this study get a significantly different of test kit performance of asymptotic p-value < 0.001 by Cochran’s Q test, as illustrated at Table 6.

Table 6.

Overall Comparative Diagnostic Performance of Three Different Commercial Available SARS-CoV-2 IgG/IgM Rapid Test Against RT-PCR, 2020, Addis Ababa, Ethiopia

Manufacturer/Test Kit	Rate of Correct Positive Test Results (Percent Agreement with rt PCR)			Rate of Correct Negative Test Results (Percent Agreement with rtPCR)	Sensitivity (%), 95% CI	Specificity (%), 95% CI	Positive Predictive Value (%), 95% CI	Negative Predictive Value (%), 95% CI	Kappa Value*, 95% CI
	Total =52/85
	IgG&IgM	IgG	IgM
A	31	12	9	111/115	61.18%(49.96–71.37%)	97%(90.81–98.88%)	92.9(81.9–97.7%)	77.08%(69.19–83.49%)	0.60(0.48–0.83),
B	Total =63/85			109/115	74.12% (63.28–82.74%)	94.78%(88.52–97.86%)	91.30%(81.39–96.42%)	83.21%(75.45–88.95%)	0.71 (0.65–0.93)
	29	22	12
C	Total= 71/85			109/115	83.53% (73.57– 90.38%)	94.78%(88.52–97.86%)	92.21%(83.21–96.79%)	88.62%(81.32–93.41%)	0.80(0.76–1.04)
	16	33	22

Notes: A= EGENES, B= CTk onsite, C= ACON, *un-weighted kappa value.

Discussion

The recent ongoing global pandemic poses serious public health problems. Following its unprecedented urgent situation lead many companies for the development of a large range of diagnostics for SARS-CoV-2, including rapid test kits. In view of that, as of 15 August 2020 there are more than 777 SARS-CoV-2 rapid IgG/IgM test kits in the global market.²⁵ However, there is no documented evidence for its diagnostic performance done in resource limiting setting like Ethiopia, yet. Thus, the aim of this study was to evaluate the comparative diagnostic performance of three commercially available SARS-CoV-2 rapid IgG/IgM assays: EGENS (A), CTk Onsite (B) and ACON Biotech (C) SARS-CoV-2 IgG/IgM rapid test using human blood specimens against RT-PCR of respiratory specimens.

The evaluated SARS-CoV-2 rapid IgG/IgM assays have a heterogeneous comparative diagnostic performance with overall sensitivity of 61.18% (95% CI: 49.96–71.37%), 74.12% (95% CI: 63.28–82.74%) and 83.53% (95% CI: 73.57–90.38%) and a specificity of 96.52% (90.81–98.88%), 94.78% (88.52–97.86%) and 94.78% (88.52–97.86%) for test kit A, B and C, respectively, which was a lower finding and not in line with the respective manufacturer claim of sensitivity 96.80%, 88.24% and 99.1% and a specificity of 100%, 100% and 98.2% of assay A, B and C, respectively. The difference might be due to the number of samples and that the manufacturing company did not perform in real life of field work, as we did in different health facilities.^26–28

Test kit A has much lower sensitivity performance, whereas test kit B and C have a comparable diagnostic sensitivity performance with a study done from by Nicol et al, having an overall sensitivity of 80% and a study done by Porte et al, with an overall sensitivity and specificity of 93.9% (CI95% (86.5–97.4)) and 100% (CI95% (92.1–100)) respectively, and a Kappa coefficient of 0.9, which is concordance with Ckappa value of 0.8.^32,33

In terms of comparative diagnostic specificity, this study revealed a 96.52% (90.81–98.88%), 94.78% (88.52–97.86%) and 94.78% (88.52–97.86%) with test kit A, B, C respectively which was in line with a study done by Van Elslande et al, of a diagnostic specificity of 96.1% for IgG and also it showed an agreement with Jääskeläinen et al, a specificity of 95.1%.^34,35

This study revealed as 92%, 91% and 93% of positive predictive value of A, B and C test kits respectively, which was in-concordance with a study done by Irene Cassaniti et al, having a PPV of 87.5% using Viva Diag COVID‐19 IgM/IgG test of a very poor sensitivity among emergency patients and documented by Krüttgen et al, have a comparable result with our current study with a specificity and predictive value, with a specificity of 96.2%, 88.7%, 100%, and 100% of the four commercially assay. However, the study was focused only IgG and only 75 study sera, in our cases we evaluated both IgG/IgM and also our sample size was 200.^36,37

Onsite CTK Biotech assay had comparable performance with a study done by Pallett et al, of a diagnostic sensitivity of 88.2% (95% CI (81.6–93.1)) from 120 of 136; and a specificity of 94% (95% CI (87.46–97.8)). It indicate a reputable and almost similar finding across different geographic cases.³⁸

This study has a higher sensitivity than Tollånes et al, study.³⁹ It may be due to the number of study participants they used were less than what we have used. In the contrary, we get a lower comparative performance result with a study done by Roche with a diagnostic specificity and sensitivity 99.81% and 100% respectively using 5300 blood sample. The difference may be the sample size and they did the serological test after two weeks of confirmed positive PCR results, in our context we tried to test without knowing their status as presumptive as. On the other hand, the third test kit, ACON Biotech SARS-CoV-2 IgG/IgM rapid test, have showed a comparable diagnostic performance with Assure Tech. Assure COVID-19 IgG/IgM rapid test device and Biohit Healthcare (Hefei) Biohit SARS-CoV-2 IgM/IgG antibody test kit done at different site.^40,41

The sensitivity of the test performance increase with clients after the clinical onset of seven day, which is concordance finding with different study groups done by Cassanitietal., Xie J. et al, Jeffrey D.Whitman et al, it indicates that these test kits be better if we used for clients with chief complains of COVID-19 at the peripheral health facilities. We get relatively better specificity, which is expected countries having relatively low prevalence.^36,42,43

RT-PCR is the current established gold standard test for the diagnosis of SARS-COV-2 using respiratory specimens. However, it has limitations related to technical procedures, limit of detection, being prone to contamination and the tendency of negativity of test results after 10 days of clinical onset. In this study, we have been working to come up with alternative test methods having relatively minimal cost, easy to perform and deliver results within short turnaround time. Accordingly, the current finding showed that rapid IgG/IgM SARS-COV-2 point of care tests have a pivotal role for patient screening in resource scarce countries with limited number of established RT-PCR laboratories and it urges future research implication for the management of the ongoing COVID-19 pandemics.⁴⁴ On top of this, many low income countries have very low testing performance per population.⁴⁵ Thus, the availability of rapid and reliable screening test with alternative sample source for the virus detection has been marked as a critical opportunity to support the control of the pandemic and to curb the number of cases worldwide.^44,45 Furthermore, this might initiate researchers to conduct further studies and forward evidence based direction for policymakers on how to decide and when to re-open societies; and for program managers to develop testing algorithms and to design discharge protocols.^34,37,44

Readers should consider the following points while inferring our results as limitations. For the negative results, we did not collect second time specimens for confirmation. Moreover, we did not determine the SARS-COV-2 viral load amount in nasal/throat swabs and its limit of detection that might influence the performance of these rapid test kits.

Conclusion

Even though, the evaluated three commercially available SARS-CoV-2 IgG/IgM test kits showed a heterogeneous performance, the two test kits could be good alternative assert prevalence screening tool and could be used in combination of RT-PCR to detect more infected clients in resource limited countries like Ethiopian. This might help to minimize further spread of this deadly and gripped virus by testing and identifying potential source of infection.

Getting respiratory swab specimens are not easy from critical patients and their viral load become probably low and consequently their PCR result could be undetectable, which leads to false negativity. Hence, rapid test kits could be good alternative due to its relative easy to perform with in short turnaround time. Nonetheless, considering our study finding reinforces the need for the available rapid test kits should be evaluated prior to use in the particular population. We also recommend further studies on SARS-CoV-2 whole genome sequencing among Ethiopian isolates for designing and validating more sensitive and specific rapid test kits.

The whole study of the research process was summarized and depicted at Figure 1.

Figure 1.

A figure summarizing the whole study process of evaluating the diagnostic performance of SARS-CoV-2 IgM/IgG rapid test kits for the detection of the novel coronavirus in Ethiopia.

Funding Statement

This work becomes at this stage by the generous financial and material support of Addis Ababa University, from its limited budget, Ref #-PR/5.15/590/12/20. However, the funder had not any involvement with the research methodology design, analysis and write up of the manuscript.

Data Sharing Statement

Some of the data set like name of the presumptive and personalized data used and/or analyzed during the current study is not publicly available, to maintain privacy but can be available from the corresponding author on reasonable request. The rest, all relevant data are within the manuscript and there are no supplementary files.

Disclosure

We, the authors declare that we have no conflicts of interest for this work.