Improved Performance of Enzyme-Linked Immunosorbent Assays and the Effect of Human Immunodeficiency Virus Coinfection on the Serologic Detection of Herpes Simplex Virus Type 2 in Rakai, Uganda

Abstract

Ugandan subjects (820) were tested by Focus HerpeSelect enzyme-linked immunosorbent assay (ELISA), Kalon herpes simplex virus type 2 ELISA, and BioKit rapid test, and the results were compared to those of Western blotting. Higher-than-standard-index cutoff values gave optimal sensitivity and specificity. Kalon ELISA was the optimal assay when an index value of 1.5 was used (sensitivity, 91.7%; specificity, 92.4%).

Herpes simplex virus type 2 (HSV-2) infection is one of the most common sexually transmitted diseases, and HSV-2 is the main cause of genital ulceration worldwide (4, 5, 17). Genital HSV-2 infections are associated with an increased risk for acquiring human immunodeficiency virus type 1 (HIV-1) (5). Diagnosis of HSV-2 infection in sub-Saharan Africa and developing countries elsewhere, where the prevalence is high, has been problematic. Serologic tests for HSV-2 that use an enzyme-linked immunosorbent assay (ELISA) or dot blot are technically simple and relatively inexpensive (15) but have been complicated by a variable rate of samples with positive HSV-2 ELISA and negative Western blot results (6, 8, 10, 15). In addition, the Western blot assay is expensive, difficult to read, and inefficient for evaluating large numbers of samples for clinical trials (2). We and others have previously demonstrated that a higher index cutoff value is required for optimal sensitivity and specificity for the Focus HerpeSelect HSV-2 ELISA (3, 10); however, the assay still lacks specificity. Therefore, Focus HerpeSelect ELISA, Kalon HSV-2 ELISA, and BioKit rapid test were compared to Western blotting as the gold standard.

The study utilized sera from 820 subjects (273 HIV positive and 547 HIV negative) collected in a population-based randomized control trial of presumptive sexually transmitted disease treatment among adults aged 15 to 19 in Rakai District, Uganda, from 1994 to 1998 (7, 10). All relevant institutional review boards in Uganda and at Johns Hopkins University and the National Institutes of Health approved the trial. Samples were collected at the participant's home, processed for sera, and stored at −70°C. The University of Washington HSV-2-specific Western blot analysis was performed as previously described (1, 12). The Focus HerpeSelect HSV-2 ELISA (Focus Technologies, Cypress, CA) and Kalon ELISA (Kalon Biological Ltd., Guilford, United Kingdom) were performed according to the manufacturers' protocols (11) with a few modifications. Samples and controls were run in either triplicate (Focus) or duplicate (Kalon). Mean index values were used for all calculations. Any samples with discordant results were run again. The Sure-Vue HSV-2 rapid test (BioKit USA Inc., Lexington, MA) was performed according to the protocol for serum samples. HIV status was determined using two different ELISAs (the Vironostika HIV-1 [Organon Teknika, Charlotte, NC] and one from Cambridge Biotech, Worcester, MA). Discordant results or new seroconversions were confirmed by HIV-1 Western blot assay (bioMerieux-Vitek, St. Louis, MO), as previously described (7). Statistical calculations and receiver operating characteristic curves were performed using Intercooled Stata 9.2 (StataCorp LP, College Station, TX).

To evaluate the performance of Focus HerpeSelect ELISA in detecting HSV-2 seroprevalence in sub-Saharan Africa, 820 subjects were tested. According to the manufacturer's instructions (index cutoff value, 1.1), the test had a sensitivity of 99.0% and a specificity of 50.7%. An index cutoff value of 3.2 was determined to provide the optimal sensitivity (88.4%) and specificity (80.8%).

Of the 820 subjects, 538 were also evaluated by Kalon ELISA. According to the manufacturer's instructions (index cutoff value, 1.1), the test had a sensitivity of 95.1% and a specificity of 87.6% (Table 1). An index value cutoff of 1.5 gave the optimal sensitivity (91.7%) and specificity (92.4%) (Table 1). The receiver operating characteristic curve demonstrated that Kalon was superior to Focus, with a greater area under the curve (Fig. 1).

TABLE 1.

Performance of the Kalon HSV-2 ELISA

Index value	ELISA/WBa				Sensitivity (%)	Specificity (%)
	+/+	+/−	−/+	−/−
0.7	286	2	63	187	99.3	74.8
0.8	285	3	45	205	99.0	82.0
0.9	283	5	38	212	98.3	84.8
1.1	274	14	31	219	95.1	87.6
1.3	268	20	26	224	93.1	89.6
1.5	264	24	19	231	91.7	92.4
1.8	250	38	14	236	86.8	94.4
2.0	240	48	11	239	83.3	95.6
2.5	220	68	6	244	76.4	97.6
3.0	197	91	5	245	68.4	98.0
3.5	188	100	4	246	65.3	98.4

^aShown are the numbers of subjects positive by ELISA and the University of Washington HSV-2-specific Western blot assay (WB) (+/+), positive by ELISA and negative by WB (+/−), negative by ELISA and positive by WB (−/+), and negative by ELISA and WB (−/−) for HSV-2.

FIG. 1.

Receiver operating characteristic curves for Focus and Kalon assays. The areas under the curve are 0.98 for Kalon and 0.93 for Focus.

Of the 820 subjects, 524 were also evaluated with the BioKit rapid point-of-care test. According to the manufacturer's instructions, which indicated that low-positive results should be considered positive, the sensitivity was 95.8% and the specificity was 56.1%. Adjustment of index values was not possible for this assay.

Due to the association between HIV-1 and HSV-2 (5, 13, 14), it is important to determine the effect of HIV-1 infection on HSV-2 antibody detection. For both ELISAs and the BioKit assay, there were higher proportions of subjects positive for HSV-2 among the HIV-1-infected than among the uninfected populations (P < 0.001). However, none of the assays were significantly affected by HIV-1 status (Table 2).

TABLE 2.

Performance of HSV-2 serology assays by HIV status^a

Assay	Sensitivity (%)	Specificity (%)	Concordance (%)
HIV+
Focus	89.0	80.6	67.2
Kalon	92.9	86.0	78.0
BioKit	97.6	52.1	57.0
HIV−
Focus	95.9	75.2	73.0
Kalon	92.5	94.5	86.5
BioKit	94.4	57.1	49.6

^aThe samples were stratified by HIV status by testing and were evaluated at the adjusted cutoff values. There was no statistical difference between statuses for each test (P > 0.05).

This study represents the largest investigation of the performances of three different commercial HSV-2 assays in sub-Saharan Africa. We demonstrated that the Kalon HSV-2 immunoglobulin G ELISA was improved at a higher index cutoff value of 1.5 than for subjects from the Western hemisphere and is superior to both the Focus HerpeSelect ELISA and the BioKit rapid point-of-care antibody test. In addition, we found that HIV status does not significantly affect the HSV-2 serologic diagnosis.

While adjusting the index values of the assays for subjects from sub-Saharan Africa significantly improved both the sensitivity and specificity, there were still both false-negative and false-positive results. The false negatives could have been due to an early infection, in which case the University of Washington HSV-2-specific Western blot was more sensitive. The false positives might have been due to cross-reactivity with HSV-1, which is >90% prevalent in sub-Saharan Africa (9, 16).

To create the best predictive testing strategy based on sensitivity and specificity compared to the University of Washington HSV-2-specific Western blot assay, different algorithms were analyzed. The optimal testing method with the highest combined sensitivity and specificity was to use the Focus ELISA with an index cutoff value of 1.1 and then test all positive samples with the Kalon ELISA, using an index cutoff value of 1.5. The sensitivity and specificity were then 92.0% and 92.8%, respectively. While this algorithm produced the best results overall, the results were not statistically significantly different from those using the Kalon test alone with a cutoff value of 1.5 (P = 0.7). Overall, the quickest, most economical, and most accurate method for HSV-2 detection was the Kalon ELISA but with an index cutoff higher than the manufacturer's recommendation.