Validation of laboratory developed serology assays for detection of IgG antibody to severe acute respiratory syndrome coronavirus 2 in the South African population

Abstract

Serological assays for detection of IgG, IgM or IgA against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) play an important role in surveillance, antibody persistence, vaccine coverage and infection rate. Serological assays, including both ELISA and rapid lateral flow assays, are available commercially but the cost limits their accessibility for low resource countries. Although serological assays based on mammalian-expressed SARS-CoV-2 spike protein have been previously described these assays need to be validated using samples from local populations within the continent, or country, in which they will be used. Interpretation of results could be influenced by differences in specificity and potential for pre-existing cross-reactive antibodies. In this study, we investigated two laboratory developed serological assays, an enzyme linked immunosorbent assay (ELISA) and an immunofluorescent assay (IFA), developed using recombinant SARS-CoV-2 spike protein, for use in South African populations. The tests were compared with commercially available and South Africa Health Products Regulatory Authority (SAPHRA) approved assays. A panel of 100 residual diagnostic serum samples, collected prior to the pandemic, were tested on three separate occasions to determine a suitable cut-off value for differentiation of positive from negative samples. Specificity of 96 % and 100 % for ELISA and IFA respectively was demonstrated. A total of 82/89 serum samples collected between days 2–94 after onset of illness from patients with a positive molecular result were positive for IgG antibody. The sensitivity of the laboratory developed assays on samples collected > one week after onset of illness was shown to be 100 % and 98.8 % for ELISA and IFA respectively. Positive predictive values were 92.1 % for ELISA and 91.0 % for IFA using characterization of samples as positive based on confirmation of infection using RT-PCR. Serum samples (n = 62) collected from RT-PCR positive patients infected with either ancestral, or emerging variants such as Beta or Delta, tested positive for IgG antibody (62/62) using the laboratory developed assays confirming application of the assays regardless of currently circulating variant during the time of evaluation. High concordance was demonstrated between the laboratory developed assays and the commercial immunoassay among samples collected from South African populations, although the small sample size, especially for the comparison with commercial assays, must be noted. If all quality assurance controls are in place, the use of local laboratory developed assays for high-throughput screening in resource-constrained environments is a realistic alternative option.

1. Introduction

In December 2019 an outbreak of the novel coronavirus severe acute respiratory virus coronavirus 2 or SARS-CoV-2 associated with severe acute respiratory syndrome was identified in Wuhan, Hubei, China. The virus outbreak rapidly escalated into a pandemic with countries globally experiencing waves of infection and emerging variants.

Coronaviruses are single stranded positive sense RNA viruses that belong to the family Coronaviridae. Seven coronaviruses have been recognised as causing disease in humans, including two alpha-coronaviruses HCoVs-NL63 and HCoVs-229E and the beta-coronaviruses HCoVs-OC43, HCoVs-HKUI, SARS-CoV, MERS-CoV and the recently identified SARS-CoV-2. Laboratory confirmation is required to differentiate coronavirus disease 2019 (COVID-19) from other respiratory infections and multiple molecular platforms have been developed and are used globally. Rapid antigen tests have been validated and have application with caution on acceptance of a negative result and other indicators of infection due to perceived lower sensitivity compared with RT-PCR.

The use of serological assays for detection of IgG, IgM or IgA against SARS-CoV-2 has limited application for diagnosis however it has played an important role in surveillance, antibody duration studies and vaccine coverage. Acutely infected patients develop detectable IgG, IgM and IgA responses however the IgM and IgA responses appear to wane more rapidly and in some instances were shown to be undetectable as early as five months after infection (Mariën et al., 2021). Suggesting assays that detect IgG antibody rather than IgM or IgA will play a more useful role in serosurveillance studies. Serological assays are available commercially and include both ELISA and rapid lateral flow assays. An alternative for laboratories with recombinant technology expertise is the preparation of laboratory developed reagents. Hence the development and validation of laboratory developed assays will have an important role in the epidemiology of the outbreak. Various antigens, including spike antigen and spike subunits, nucleocapsid and receptor binding domain (RBD), have been used in serological assays. Comparison of performance does vary and use of different samples and assays may be considerations limiting overall comparisons, however ELISA antigens based on RBD or spike protein, performed with mean specificities of 99.6 % and 97.8 % respectively while the sensitivity of ELISA using spike protein (88.8 %) was higher than RBD (79.4 %) or nucleocapisd (72.4 %) (Makoah et al., 2021). Recombinant antigens provide a useful method for preparing SARS-CoV-2 spike protein for use in ELISA.

Serological assays based on mammalian expressed spike protein have been described (Amanat et al., 2020). However, the importance of validating assays using samples from populations within a continent or country in which the assays will be used has previously been highlighted (Ndaye et al., 2021). Interpretation of results could be influenced by differences in specificity and potential for pre-existing cross-reactive antibodies occurring in different populations. The occurrence of cross-reactivity with other seasonal coronaviruses in Africa has not been completely excluded (Ng et al., 2020). Many commercial assays are understandably validated using European populations. Hence in this study we aimed to investigate laboratory developed serological assays prepared using mammalian expressed recombinant SARS-CoV-2 spike protein for application in South African populations, to compare our laboratory developed assays with commercially available and South Africa Health Products Regulatory Authority (SAPHRA) approved assays and to confirm application against emerging variants.

2. Methods and materials

2.1. Serum samples

Blood samples were collected from patients during the first eight months (March–October 2020) of the outbreak including the first wave of infections (June–August, 2020) from a total of 89 patients in Bloemfontein, South Africa, who tested positive for SARS-CoV-2 using SAPHRA approved RT-PCR assays. Samples for the validation were collected from patients with onset of illness between March 2020 and October 2020. The samples were tested for anti-SARS-CoV-2 IgG using a laboratory developed enzyme linked immunosorbent assay (ELISA), and laboratory developed immunofluorescent assay (IFA). All patients completed informed consent documents and approval for the study was obtained from the University of the Free State Health Sciences Research Ethics Committee (UFS-HSD2020/0595/2605) and the Environment and Biosafety Ethics Committee (UFS-ESD2020/0072).

Forty-eight of these samples were randomly selected and tested using a commercial ELISA, Elecsys® Anti-SARS-CoV-2 (Roche Diagnostics GmbH, Mannheim, Germany), and commercial lateral flow assay, COVID-19 IgG/IgM Rapid Test cassette (Zhejiang Orient Gene Biotech Co., Ltd, Zhejiang, China). At the time limited assay kits were available to test complete cohort.

A panel of 100 negative control sera were obtained from residual diagnostic samples submitted to the Division of Virology, National Health Laboratory Service (NHLS), prior to January 2020 for various unrelated serological testing and stored. Permission to use the samples was obtained from the Business Manager NHLS Universitas Academic Laboratories. The negative panel was tested using the laboratory developed ELISA on three separate occasions.

After validation of the assays, additional samples (n = 62) were collected from patients with a positive RT-PCR and onset of illness between November 2020 and October 2021 during the second and third waves of infection.

A summary of the collection dates for samples and testing is shown in Table 1.

Table1.

A summary of collection dates for serum samples including stored pre-pandemic samples and samples from patients with positive RT-PCR result.

Collection period	Number of serum samples	Purpose	Assays performed
Pre-pandemic (prior to January 2020)	100	Negative controls for validation	Laboratory developed ELISA & IFA
March–October 2020	89	Validation	Laboratory developed ELISA & IFA
	48* (Subset of 89 serum samples)	Validation / Comparison	Elecsys® Anti-SARS-CoV-2 ELISA & COVID-19 IgG/IgM Rapid Test cassette
November 2020–October 2021	62	Performance testing	Laboratory developed ELISA & IFA

*48 serum samples were randomly selected from the panel of 89 serum samples collected between March and October 2020.

Data retrieved from public domain (National Institute for Communicable Diseases, (NICD), Surveillance Reports) were used to correlate the dates of illness of participants and the occurrence of waves of infection. (https://www.nicd.ac.za/diseases-a-z-index/disease-index-covid-19/surveillance-reports/national-covid-19-daily-report/).

2.2. ELISA and IFA control and internal quality control

A positive control (designated C++) was identified from an individual that had a laboratory confirmed infection using RT-PCR and was positive for IgG antibody using ELISA, IFA and rapid IgG assay.

A serum sample stored from a previous study prior to the onset of the pandemic was identified as a negative control and designated C-.

Internal quality control (IQC) data were generated according to standard recommendations (Blockx and Martin, 2001; Jacobson, 1998a, Jacobson, 1998b; Wiegers, 2000). Briefly, the internal control serum was tested on six occasions and additionally tested in six replicates on three plates on three different occasions to give a total of 60 determinants. Mean standard deviations of ELISA net optical density (OD) and percentage positivity (PP) for control serum, C++, were calculated from replicates of all controls in each plate and each run of the assay.

2.3. ELISA reagents

2.3.1. Recombinant antigen

A plasmid expressing a soluble spike protein of the SARS-CoV-2 isolate (GenBank MN908947.3) was available (Amanat et al., 2020).

Plasmid for transfection of mammalian cells was grown in LB broth with ampicillin and purified using an Endofree Plasmid maxi kit (QIAGEN GmbH, Hilden, Germany) according to manufacturer’s instructions. Recombinant antigen was prepared as described previously with some modifications (Amanat et al., 2020). Briefly, human embryonic kidney (HEK) 293 cells (ATCC CRL1573) (ATCC, Manasses, United States of America (USA)) were seeded at 2 × 10⁶ cells/ml in Dulbecco’s Minimal Essential Medium (DMEM) supplemented with L-glutamine (Lonza, Basel, Switzerland) and 10 % foetal bovine serum, grown to 80 % confluency and transfected with endotoxin free plasmid using Lipofectamine 3000 (Invitrogen™, Carlsbad, USA) according to manufacturer’s instructions. Cells were incubated at 37 °C in a 5 % carbon dioxide incubator for a further 72 h. Media were collected for purification of recombinant protein and cells were dissociated from flask surfaces using trypsin and washed for preparation of antigen slides. The cell culture supernatant was clarified at 4000g for 15 min and protein was purified from the supernatant using Protino Ni-2000 packed columns (Machery-Nagel, Düren, Germany) according to manufacturer’s instructions. Eluted protein was concentrated using Amicon® ultrafilters with a 3 kDa cut-off (MilliporeSigma, Burlington, USA) and the protein concentration measured using a Qubit 4 Fluorometer (Thermo Scientific, Waltham, USA) and Qubit Protein Assay Kit. Recombinant protein was aliquoted and stored at −80 °C.

2.3.2. Procedure for IgG direct ELISA

Recombinant SARS-CoV-2 antigen was used to develop an indirect ELISA for detection of IgG antibody directed against SARS-CoV-2 in serum samples collected from acute and convalescent patients. Checkerboard titrations of each reagent were used to optimize the ELISA protocol. Wells were treated with carbonate buffer and no antigen as controls to exclude non-specific reactivity and used to calculate net OD values for each sample. Unless stated otherwise, all volumes were 100 µl/well, plates were incubated for one hour at 37 °C and washed for 3 × 15 s with 0.1 % Tween 20 (Promega, Madison, USA) in phosphate-buffered saline (PBS) pH 7.0 (MilliporeSigma, Burlington, USA). Briefly, 96-well PolySorp microtitre plates (Nalgen Nunc International Corporation, Rochester, USA) were coated with 100 µl of purified recombinant SARS-CoV-2 diluted in carbonate buffer pH 8.6 (approximately 3.5 µg/ml protein) overnight at 4 °C. Control wells contained carbonate buffer without recombinant antigen. Plates were washed and blocked with 200 µl/well 10 % skimmed milk/PBS. Serum samples were diluted 1:50 in 2 % skimmed milk/PBS and added to antigen coated wells and control wells. On each plate duplicate positive (C++) and negative (C-) controls were included. Positive reactors were detected using anti-human IgG horse radish peroxidase (HRPO) conjugate diluted 1:2000 (SeraCare Life Sciences Inc., Milford, USA) and 2,2′-Azino di-ethyl-benzothiazoline-sulfonic acid peroxidase substrate (ABTS) (SeraCare Life Sciences Inc., Milford, USA). Plates were incubated for 10 min at room temperature (22–24 °C) in the dark and the OD values were read at 405 nm. Net OD values were determined for each sample as follows:

net OD = OD in wells with recombinant SARS-CoV-2 antigen minus OD in wells with no antigen.

The net OD₄₀₅ for each serum tested was calculated by subtracting the OD₄₀₅ reading of the sample from wells with no antigen from the cognate reading of that obtained using the recombinant antigen coated wells to account for any background absorbance. OD values are absolute measurements that are influenced by variables such as daily temperature fluctuations. To account for variability, results were expressed as a function of the reactivity of control samples included in each run. Therefore, net OD₄₀₅ readings were converted to a percentage of positive (C++) control serum (PP) value using the following equation (Paweska et al., 2005; Paweska et al., 2007):

(net OD of test sample/net OD of C++) × 100.

The cut-off PP value was determined from two graph receiver operating characteristics analysis (TG-ROC) of field samples characterised as positive based on molecular confirmation of recent infection, or negative based on collection prior to global circulation of the virus.

2.4. Immunofluorescent antibody tests for anti-SARS-CoV-2 IgG antibody

2.4.1. Preparation of antigen slides

SARS-CoV-2 antigen slides were prepared from HEK cells transfected with a plasmid expressing the spike protein of SARS-CoV-2 as described in Section 2.3.1. Briefly, after the media were removed from transfected cells for antigen purification, the cells were dissociated from the flasks using trypsin and resuspended in a minimal volume of 10 % foetal calf serum in PBS. A 10 µl aliquot of the cell suspension was spotted on each well of a 10 well slide and dried at 37 °C. The cells were fixed to the slides and permeated by immersion in cold methanol:acetone (Merck Group, Darmstadt, Germany) (1:1) mix for a minimum of 60 min. Slides were dried and stored frozen at −80 °C.

2.4.2. Procedure for IFA test

All serum samples were diluted 1:10 in PBS and tested for anti-SARS-CoV-2 IgG antibody using the laboratory developed antigen slides and anti-human IgG fluorescein isothiocyanate (FITC) (Invitrogen™, Carlsbad, USA) diluted 1:50 in 0.1 % Evans blue (Merck Group, Darmstadt, Germany) in PBS. Serum samples were incubated on the slides for 30 min at 37 °C in a humid environment, washed in PBS and positive reactors detected after incubation for 30 min with anti-human IgG FITC. Cells were washed, mounted with a coverslip and visualised using a Nikon fluorescent microscope.

2.5. COVID-19 IgG/IgM rapid test cassette

A total of 48 randomly selected sera were tested for anti-SARS-CoV-2 IgG and IgM using a commercial lateral flow assay, COVID-19 IgG/IgM rapid Test Cassette (Whole Blood/Serum/Plasma) (Zhejiang Orient Gene Biotech Co. Ltd, China). Briefly, these cassettes are lateral flow immunochromatographic assays in which a 5 µl aliquot of serum was added to the sample well followed with sample buffer as indicated by the manufacturers. Two test lines differentiated between IgG positive and IgM positive reactors. Samples for which both test lines appeared coloured were positive for IgG and IgM. The assay result was acceptable if the internal procedural control, the coloured line in the control line region, changes from blue to red.

2.6. COVID-19 ELISA

A total of 48 sera from the samples tested using the laboratory developed assays and the commercial lateral flow assays were tested using a commercially available ELISA, Elecsys® Anti-SARS-CoV-2 IgG assay and a Cobas e601 analyzer (Roche Diagnostics GmbH, Mannheim, Germany). The assay was performed according to manufacturer’s instructions. The assay is based on reactivity against the SARS-CoV-2 nucleoprotein.

3. Results

3.1. Internal quality control and repeatability

Upper and lower limits for internal controls obtained for the laboratory developed IgG ELISA are shown in Table 2 and Fig. 1. Briefly, internal quality control (IQC) data were generated using mean +/- two standard deviations (SD) of a total of 60 replicates of each control, performed over three runs with three plates per run.

Table 2.

Internal quality control data estimates for laboratory developed SARS-CoV-2 IgG ELISA.

	Mean optical density (OD405)	Standard deviation	Upper control level OD405	Lower control level OD405
Optical density (OD405) C++	0.751	0.199	1.149	0.353
Optical density (OD405) C-	0.0504	0.056	0.162	-0.062

Fig. 1.

Upper and lower IQC limits for optical density (OD) readings of positive control and negative control IgG ELISA.

The upper control limit (UCL) and lower control limit (LCL) provide useful limitations for verification of the ELISA based on acceptability of a run provided the control values fall within the specified UCL and LCL (Paweska et al., 2003).

3.2. Determination of cut-off values

Selection of a cut-off value for the IgG ELISA was determined using convalescent samples collected from patients with confirmed SARS-CoV-2 infections. All convalescent samples were from patients with positive molecular results. The time of collection of blood after onset of illness varied and in some instances was not known. Percentage positive values were calculated for normalization of data between runs.

The cut-off was determined by categorization of test samples as positive based on laboratory confirmation of infection using a molecular assay, or as negative based on using residual diagnostic samples collected in 2019 prior to global circulation of the virus.

A total of 89 serum samples from patients with positive molecular result were available for testing for IgG using the laboratory developed ELISA and 82/89 were positive for IgG antibody. In addition, a negative panel of 100 serum samples was available using residual diagnostic samples collected prior to the pandemic and tested on three separate occasions.

A comparison of OD values and PP values shown in Fig. 2 A and B, respectively, indicate differentiation of pre-pandemic control samples from convalescent samples with exception of four pre-pandemic control samples consistently showing cross-reactivity in ELISA.

Fig. 2.

Boxplots indicating A. optical density (OD₄₀₅) and B. percentage positive values (PP value) for negative control samples and convalescent samples. Negative samples = 100 repeated on three separate occasions and each value presented is average for each run. Positive samples = 89.

A cut-off value was determined using TG-ROC analysis shown in Fig. 3, and, taking into consideration highest true positivity rate a cut-off PP value of 30 % was considered suitable to differentiate positive samples from indeterminate samples. A PP value between 30 % and 56 % was considered to represent indeterminate outcome and below 30 % the samples can be considered as negative.

Fig. 3.

TG-ROC analysis of 89 convalescent samples and 100 negative control samples repeated on three separate occasions. All convalescent samples were from patients with positive molecular results. Vertical lines show 30 % and 56 % percent positive values. PP: percent positive, SE: sensitivity, SP: specificity.

3.3. Immunofluorescent IgG assay

The 89 serum samples from patients with a positive molecular result were tested for IgG antibody by IFA using the laboratory developed antigen slides with transfected cells ( Fig. 4).

Fig. 4.

Reactivity of antigen slides prepared from HEK cells transfected with plasmid expressing SARS-CoV-2 spike protein. A. serum sample showing positive reactor for IgG antibody B. negative serum sample (Nikon fluorescent microscope x10 magnification).

Based on the results from this validation, positive predictive values (PPV) were 92.1 % for ELISA and 91.0 % for IFA ( Table 3).

Table 3.

Detection of IgG antibody using laboratory developed IFA and ELISA to test samples from patients with positive RT-PCR result.

	IFA +ve	IFA –ve
ELISA +ve	78	4
ELISA –ve	3	4

The panel of negative pre-pandemic samples were tested using IFA and all samples were negative including the four that were consistently positive using the laboratory developed ELISA.

3.4. Commercial assays

Randomly selected 48 samples with positive diagnosis using a molecular assay were tested using the commercial lateral flow and Roche assay and compared to the laboratory developed ELISA ( Table 4, Table 5, Table 6). The commercial lateral flow detected 45 IgG positive samples of which 22 were positive for IgM.

Table 4.

Detection of IgG antibody using laboratory developed ELISA and commercial lateral flow assay to test samples from patients with positive RT-PCR result.

	Lateral flow IgG +ve	Lateral flow IgG –ve
ELISA +ve	44	2
ELISA –ve	1	1

Table 5.

Detection of IgG antibody using laboratory developed ELISA and commercial Roche assay to test samples from patients with positive RT-PCR result.

	Roche +ve	Roche –ve
ELISA +ve	42	4
ELISA –ve	1	1

Table 6.

Detection of IgG antibody using a commercial lateral flow assay and commercial Roche assay to test samples from patients with positive RT-PCR result.

	Roche +ve	Roche –ve
Lateral flow +ve	42	3
Lateral flow –ve	1	2

PPV were determined for each assay using only 48 samples and based on the outcome of positive molecular assays the laboratory developed ELISA had a PPV of 95.8 %, the Roche assay was 89.6 % and the lateral flow was 93.9 %.

3.5. Samples from subsequent waves of infection

After the validation, a further 62 samples were collected from patients with onset of illness between June 2020 and September 2021. The samples were tested using the laboratory developed IFA and ELISA. In summary, 62/62 samples were positive using both the IFA and ELISA. The same cut-off values as derived from the ELISA validation data were applied. The assay was accepted if the OD values of the positive and negatives controls fell within the validated range.

All participants had a positive PCR result at time of illness and 44/62 reported a history of vaccination. Fourteen received one dose of Johnson and Johnson’s Janssen vaccine, 28 received two doses of the Pfizer-BioNTech (COMIRNATY) vaccine, one received one dose of Pfizer-BioNTech and one received Vaxzevria vaccine (Astra-Zeneca). Within this cohort 18/62 were unvaccinated.

The dates of onset of illness and the dominant variant circulating in the region at that time is illustrated in Fig. 5. Using data retrieved from the NICD surveillance reports it was possible to estimate that the first wave continued from June and ending in August 2020. The second wave of infections increased from the end of November 2020 and declined early February 2021. The third wave lasted for a longer duration due to the emergence of Delta variant and extended from the middle of May 2021 to the end of September 2021. There are limitations associated with assuming which variant had caused the infection, however, considering the dates of emergence of each variant, samples collected during 2021 from the second and third waves were from patients infected with either the Beta or Delta variant which emerged and rapidly replaced the ancestral strain (National Institute for Communicable Diseases, 2022). The recombinant antigen was prepared using sequence data encoding for the ancestral strain glycoprotein. Despite mutations occurring within regions of the glycoprotein, as expected, there was sufficient cross-reactivity to conclude that the IFA and ELISA were able to detect antibody induced by both Beta and Delta variants of concern. Among the 18 unvaccinated participants, serum samples from 13 participants were collected during the second and third waves of infection supporting the application of the ELISA for detection of IgG from patients infected with emerging variants.

Fig. 5.

Dominant variants associated with COVID-19 infections in South Africa and date of onset of illness for 62 participants. The figure shows only samples collected after the validation. Variants and duration of dominance are represented by lines as follows: ancestral-, Beta- and Delta-. Participants infected during the different epidemic waves are represented by dots with vaccinated participants differentiated from unvaccinated participants.

4. Discussion

Serological assays play a role in determining the extent of naturally acquired immunity, taking into consideration occurrence of asymptomatic and mild infections, and vaccine induced immunity. In this study, recombinant spike protein expressed in a mammalian system was used to develop an ELISA and IFA for detection of antibody against SARS-CoV-2. Although commercial assays are readily available, the cost may be prohibitive for research and surveillance in low resource countries. Laboratory developed assays require facilities for recombinant protein expression, however if this is available then laboratory developed assays can be readily developed. Assays using the recombinant protein described in this study have been previously described (Amanat et al., 2020), but it remains important to validate assays using population specific samples. In this study the aim was to validate the assays using serum samples from local participants to define acceptable cut-off values to consistently differentiate between positive and negative samples.

Selection of a cut-off is a threshold value at and above which, findings can be interpreted as positive and/or provide a limit where samples are determined to be indeterminate. Net OD values were obtained using carbonate buffer coated wells to accommodate non-specific reactivity. Alternatively, a mock antigen could have been prepared using un-transfected cells. In many instances a cut-off is determined using two or three standard deviations above and below the mean value however this has limitations as it assumes a normal distribution of test values in the target population. The application of a TG-ROC provides an alternative method for determining a cut-off (Greiner, 1995). In this validation data normalization was achieved by expressing OD values as a percentage of a known positive control included in each run. Optical density values are absolute measurements that will be influenced by various factors including ambient temperatures, incubation times and even photometric instrumentation. Hence expression of results as a function of the reactivity of a control sample should account for variability between runs and even different laboratories.

A significant caveat in this analysis is the characterization of samples as positive based on confirmation of infection using a molecular assay. It is expected that the detection of antibody responses in samples collected less than one week after onset is less sensitive than in samples collected during convalescence. Selecting a gold standard for verification of assays poses a problem. Detection of neutralization antibodies, commonly used as a gold standard for serology, have not been fully investigated against all variants of SARS-CoV-2 infections and it is likely that neutralization antibody detection is also less sensitive during acute stages of illness. Hence the predictive values were determined by characterization of samples based on molecular results and keeping in mind that the inclusion of samples collected less than one week after onset of illness would impact on the PPV. Samples were collected from day 2 up to day 94 after onset of illness. However, for 7 samples an accurate date of onset and collection date of sample was not available. All samples with a positive molecular result that tested negative for IgG antibody by our laboratory developed ELISA were collected between days 2 and 6 after onset of illness. Overall, the sensitivity on samples collected > one week after onset of illness was shown to be 100 %. Additional limitations of this study include the use of the same panel of samples for determination of the cut-off values and validation and that the sample size, especially for the comparison of the commercial assays, is small.

The validation data were determined using samples collected from patients infected with the ancestral strain of SARS-CoV-2. New variants will continue to emerge and for any long-term application the ability to detect emerging variants would be important. Serological studies from several African countries are limited and therefore the true extent of immunity in these countries is not known (Tessema and Nkengasong, 2021). Laboratory developed assays with validation data could help to determine the true burden of disease in some of these countries. The emergence of new variants of concern, Beta and Delta, that dominated during the recent waves raised the question regarding ability of the assays to detect responses against emerging variants. Additional samples were collected to consider the serological cross-reactivity of new variants however at this time the vaccine had been rolled out and hence for 44/62 samples it would not be possible to differentiate a vaccine immune response from a naturally acquired response. However, although the numbers were small, samples were tested from 18 unvaccinated participants and an antibody response was detected in all serum samples available from unvaccinated participants infected during the second and third waves.

To determine the accuracy of our assays compared with SAPHRA approved assays, a cohort of the samples were randomly selected. Lateral flow assays are frequently considered as less sensitive than other laboratory assays and, although the numbers were small, the lateral flow had a higher PPV than the commercial ELISA. The assays differ in the target protein, with spike or nucleocapsid used respectively, which may contribute to the slightly different results. Lateral flow assays have the advantage of being rapid and requiring no sophisticated equipment. Hence there is a role for these assays provided the limitations there of are considered and negative results are interpreted with caution. The performance characteristics of the lateral flow assay provided by the manufacturer was generated using 36 samples from patients with confirmed infections based on positive molecular results and a sensitivity of 97.2 % with 100 % specificity. However, these are small sample sizes with no data to indicate when samples were collected after onset of illness. Similarly, the performance characteristics of the commercial ELISA were also based on samples derived from patients with a molecular diagnostic result. According to manufacturers and based on analysis of 10,453 negative samples in which 21 were false positive reactors, the overall clinical specificity was 99.8 % and the sensitivity, based on 496 samples from 102 symptomatic patients with a PCR confirmed infection, ranged from mean values of 60.2 % on days 0–6 after molecular confirmation of infection, 85.3 % on days 7–13 % and 99.5 % after day 14. A meta-analysis based on 33 articles describing various serological assays (ELISA, lateral flow and IFA) reported specificity ranging from 93 % to 99 % whereas the specificity of ELISA was on average 95 % (Makoah et al., 2021). Our laboratory developed ELISA detected one additional positive sample compared with the commercial ELISA and the reason for this is unclear but may represent differences in reactivity against the different viral proteins. However, the sample numbers are too limited to make any conclusive comments.

Overall, the results from the laboratory developed IFA correlated well with the laboratory developed ELISA. The laboratory developed IFA can be completed within two hours using antigen slides stored frozen at −80 °C and could be readily modified to detect IgM antibody, however, it has the disadvantage that it is not readily automated for large surveys and requires the use of a fluorescent microscope.

In summary, we have been able to develop a cost-effective laboratory developed ELISA and IFA with potential application in serosurveillance to determine seroprevalence within South African populations and neighboring countries.

Funding

This work was supported by South African Research Chair in vector-borne and zoonotic pathogens research hosted by the University of the Free State funded by Department of Science and Technology (DST) and administered by National Research Foundation (NRF). [Grant number U98346].

Ethics statement

All patients completed informed consent documents and approval for the study was obtained from the University of the Free State Health Sciences Research Ethics Committee (UFS-HSD2020/0595/2605) and the Environment and Biosafety Ethics Committee (UFS-ESD2020/0072).

CRediT authorship contribution statement

Litabe M: Investigation, Methodology, Data curation, Formal analysis. van Vuuren C: Investigation, Resources. Bester PA: Data curation, Formal analysis, Writing – review & editing. Goedhals D: Investigation, Resources, Writing – review & editing. Potgieter S: Investigation, Resources. Frater J: Investigation, Resources, Writing – review & editing. Thompson C: Investigation, Resources, Writing – review & editing. Wright D: Investigation, Resources, Writing – review & editing. Lambe T: Investigation, Resources. Gupta S: Investigation, Resources. Brink M: Investigation, Resources. van Jaarsveldt D: Investigation, Resources. Burt FJ: Conceptualization, Methodology, Writing – original draft, Writing – review & editing, Supervision, Project administration, Funding acquisition.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

Data will be made available on request.