Evaluation of HSV-2 serological tests for use with dried blood spots in Kenya

Abstract

We evaluated two assays to detect antibodies to HSV- 2 in dried blood spots (DBS) prepared from blood specimens submitted to a reference laboratory in Kenya. DBS did not perform well with the Kalon HSV-2 assay. Focus HerpeSelect 2 was 98.8% sensitive and 98.9% specific with dried blood spots.

Collection and storage of biological samples, including blood, are indispensable components of contemporary HIV and sexually transmitted infection (STI) research protocols, enabling detection of pathogen-specific antibodies and a wide variety of other diagnostic testing. Detection of antibodies to herpes simplex virus type 2 (HSV-2) glycoprotein G has become an increasingly common biomarker in HIV/STI prevention studies.^1–3 HSV-2 infection is a recognized cofactor for sexual transmission and acquisition of HIV,^4–8 and HSV-2 serology is useful as an objective measure to evaluate risk for HIV infection. In addition, because HSV-2 is primarily transmitted sexually and viral infection persists for life, detection of HSV-2 antibodies has been used as a marker of sexual activity in adolescent populations.^9–13

Traditionally, serological assays have been developed for use with serum prepared from whole blood obtained by venipuncture. Barriers imposed by the requirements of proper specimen collection, transport, handling and storage for serum and plasma are well-recognized, particularly in resource-limited settings, as are the benefits of dried blood spots (DBS) to facilitate the serological diagnosis of STIs.^{10, 14–19} Several enzyme-linked immunosorbent assays (ELISAs) for detection of anti-HSV-2 antibodies are commercially available and in widespread use. HerpeSelect 2 (Focus Diagnostics, Cypress, CA) is FDA-cleared and bears the CE mark for the qualitative detection of IgG antibodies to HSV-2 in human sera; the Kalon HSV-2 IgG ELISA (Kalon Biological, Guildford, UK) is offered for research use only. Both Focus and Kalon assays are validated by the manufacturers only for use with serum. Cherpes and colleagues showed excellent agreement in > 700 paired serum and plasma samples using the Focus ELISA.²⁰ This assay has also been widely used with DBS,^{10, 18, 19, 21–23} though published reports of DBS specimen validation are limited. In a study of 22 paired serum and DBS samples from healthy volunteers in the U.S. (including only 4 that were positive for HSV-2 antibodies), Hogrefe and colleagues demonstrated efficient elution of total IgG from DBS and good concordance of Focus assay results from DBS eluates with results for detection of HSV-2-specific antibody in serum.²² Here we report the results of an evaluation of both the Focus and Kalon ELISAs for detection of HSV-2 antibodies in DBS in the context of an ongoing HIV prevention intervention trial in Kenya.^{10, 11, 13}

The reference laboratory in Kenya obtained deidentifed excess specimens from healthy blood donors after routine screening tests were performed and from patients attending a comprehensive health clinic after routine diagnostic testing was performed. Samples were from 198 adults of both sexes living in Nairobi Kenya; no additional data were available for this laboratory assay validation. Whole blood was obtained by venipuncture and collected in tubes containing EDTA for routine testing; serum from blood without anticoagulant was not available for this study. In the reference laboratory, DBS were prepared from 50 μL of EDTA blood spotted onto S&S 903 filter paper (Schleicher & Schuell, Dassel, Germany); cards were dried for 18–24 hours in a biosafety cabinet, placed in a zip-lock bag with desiccant and a humidity indicator and stored at 4 °C until testing. Plasma was prepared from EDTA blood by centrifugation at 1000 × g for 10 minutes and stored at 4 °C until testing; samples were tested within 7 days of specimen collection.

For ELISA testing, a 6 mm disc from each DBS was punched into a microtiter plate well and constituents were eluted with 150 μL phosphate buffered saline at 4°C for 18–24 hours. For testing with the Kalon assay, 18 pairs of DBS and plasma samples were tested according to the manufacturer’s instructions. For testing with the Focus assay, we diluted DBS eluates 1:5 with the kit diluent and tested a total of 180 pairs of DBS and plasma samples according to all other manufacturer’s instructions. For both assays, results were expressed as index values calculated as the absorbance (450 nm wavelength) of the specimen divided by the mean absorbance values of the cutoff calibrator tested on the same microtiter plate. Both manufacturers specify interpretation of index values < 0.9 as negative, > 1.1 as positive and ≥ 0.9, ≤ 1.1 as equivocal.

We performed linear regression using SigmaStat software in SigmaPlot version 13 for Windows (Systat Software, Inc., Point Richmond, CA) to assess correlation of DBS and plasma results. Sensitivity, specificity, unweighted Cohen’s kappa statistic and 95% confidence intervals were calculated at vassarstats.net.

Reference laboratories in Kenya routinely use the Kalon HSV-2 ELISA for diagnostic testing, and our initial goal was to validate this assay with DBS to facilitate specimen collection, transport and storage for an ongoing longitudinal study to evaluate school support for prevention of HIV in orphan adolescents in western Kenya. Interim analysis of results from the first 18 sample sets tested with the Kalon assay suggested that the assay did not perform well with DBS in comparison with plasma. Using the manufacturer’s suggested cutoffs, 11/18 plasma samples were positive, and none of the corresponding DBS were positive; 7/18 plasma samples were negative, and all corresponding DBS were negative. Thus, estimated sensitivity of the Kalon assay with DBS was < 9.1% (95% CI: 0%, 25.9%); specificity was 100% (95% CI: 64.6%, 100%), and the unweighted kappa was < 0.068 (95% CI: 0, 0.369). We did not pursue further evaluation of this test for the needs of the ongoing study. It is possible that adjustment of DBS specimen elution or assay reagents could have improved the performance of the Kalon assay with DBS eluates. Subsequent discussions with a Kalon representative confirmed similar experiences to our own with in-house exploration of the potential use of the assay with DBS (M. Childerstone, personal communication). Furthermore, one other published report describing difficulties optimizing this assay for DBS²⁴ and documented success with the Focus ELISA and DBS from U.S. adults²² prompted us to proceed with validation of the Focus assay in the Kenyan samples.

In contrast to the poor performance of DBS eluates compared to plasma in the Kalon assay, results from the two sample types tesed using the Focus assay were highly correlated (Figure 1). Using the manufacturer’s suggested cutoffs, estimated sensitivity of the Focus assay with DBS was 98.8% (95% CI: 92.7%, 99.9%) and specificity was 98.9% (95% CI: 93.4%, 99.9%), compared to performance with plasma among 178 sample sets with either positive or negative results (in 1 sample set, both plasma and DBS were equivocal; in 1 sample set, plasma was positive and DBS was equivocal). Using the manufacturer’s suggested cutoffs to classify results from all 180 sample sets as positive, negative or equivocal, the unweighted kappa was 0.956 (95% CI: 0.913, 0.999).

Figure 1. Results from dried blood spots (DBS) were positively correlated with plasma results for detection of HSV-2 antibodies using the Focus HerpeSelect assay.

Linear regression analysis of results from 180 pairs of DBS and plasma from adults living in Kenya showed excellent correlation with r = 0.938. Vertical and horizontal lines set at the manufacturer’s suggested index cutoff value of 1.1 and an elevated cutoff of 1.5 illustrate potential false positive and false negative DBS results compared to plasma results under these conditions.

Numerous reports in the literature describe performance issues with HSV-2 ELISA tests in sub-Saharan African populations using manufacturers’ cutoffs, suggesting the use of elevated cutoff index values to improve test specificity and reduce the likelihood of false-positive results.^{19, 25–28} To address this issue, we explored the Focus assay performance with DBS using a cutoff index value of 1.5 (Figure 1), which we have used previously in algorithms for study-specific protocols with Kenyan adolescent populations.¹¹ Estimated sensitivity of the assay with DBS at the higher cutoff was 98.8% (95% CI: 93.6%, 99.8%) and specificity was 98.9% (95% CI: 94.2%, 99.8%), compared to performance with plasma among 178 sample sets with either positive or negative results (omitting 2 samples with indeterminant results). Using an index cutoff value of 1.5 to classify results from all 180 sample sets as positive, negative or equivocal, the unweighted kappa was 0.978 (95% CI: 0.978, 1). Thus, using either the manufacturer’s cutoff index value of 1.1 or the higher value of 1.5, the Focus assay demonstrated excellent agreement of results with DBS and plasma for detection of HVS-2 specific antibodies in these Kenyan samples.

This study has several important limitations. First, DBS were prepared in a laboratory setting from blood obtained by venipuncture, and our results may not be generalizable to DBS prepared from finger-stick samples in a clinic or field setting. Second, DBS were tested shortly after preparation for this study, and our results may not be generalizable to DBS stored for longer periods of time, as is typical in many real-world research settings. Additional research is needed to determine assay performance on DBS prepared in different settings and stored for extended periods of time. Third, plasma and DBS used in this study were stored at 4 °C for up to 7 days before testing, and antibody stability in these samples may have been better preserved with storage at or below −20 °C. Thus, Focus assay index values in this study may underestimate actual antibody levels. However, agreement between DBS and plasma stored under the same conditions is unlikely to be biased. Finally, we note that van Dyck et al. showed that the Kalon assay performed with greater specificity than the Focus assay in similar sub-Saharan African populations using serum according to all manufacturers’ instructions compared to a monoclonal antibody-blocking enzyme immunoassay as a reference standard.²⁸ Comparison of DBS and plasma samples using such a reference assay was beyond the scope of our study and represents another important limitation.

The convenience and utility of DBS for epidemiologic and program evaluation research aimed at reducing STIs including HIV cannot be overstated. Validation of these specimens for use with serological tests to identify persons with HSV-2 infection is an important element of the research toolkit, especially in resource-limited settings. Unless and until DBS are routinely evaluated by diagnostic test manufacturers using standardized protocols in the context of product development and approval by regulatory agencies, research studies such as this one, highlighting positive and negative outcomes, provide valuable information for investigators as they design study procedures for collection, transport, testing and storage of blood specimens.