Sensitivity and Specificity of Human Immunodeficiency Virus Rapid Serologic Assays and Testing Algorithms in an Antenatal Clinic in Abidjan, Ivory Coast

Abstract

To evaluate serologic testing algorithms for human immunodeficiency virus (HIV) based on a combination of rapid assays among persons with HIV-1 (non-B subtypes) infection, HIV-2 infection, and HIV-1–HIV-2 dual infections in Abidjan, Ivory Coast, a total of 1,216 sera with known HIV serologic status were used to evaluate the sensitivity and specificity of four rapid assays: Determine HIV-1/2, Capillus HIV-1/HIV-2, HIV-SPOT, and Genie II HIV-1/HIV-2. Two serum panels obtained from patients recently infected with HIV-1 subtypes B and non-B were also included. Based on sensitivity and specificity, three of the four rapid assays were evaluated prospectively in parallel (serum samples tested by two simultaneous rapid assays) and serial (serum samples tested by two consecutive rapid assays) testing algorithms. All assays were 100% sensitive, and specificities ranged from 99.4 to 100%. In the prospective evaluation, both the parallel and serial algorithms were 100% sensitive and specific. Our results suggest that rapid assays have high sensitivity and specificity and, when used in parallel or serial testing algorithms, yield results similar to those of enzyme-linked immunosorbent assay-based testing strategies. HIV serodiagnosis based on rapid assays may be a valuable alternative in implementing HIV prevention and surveillance programs in areas where sophisticated laboratories are difficult to establish.

Testing persons for antibodies to human immunodeficiency virus (HIV) is important in controlling the impact of the HIV and AIDS epidemic because it allows for the diagnosis and counseling of HIV-infected persons and facilitates the implementation of new prevention strategies. For instance, in Thailand, Ivory Coast, Burkina Faso, and Uganda, a short-course regimen of oral zidovudine or nevirapine administered to HIV-infected pregnant women has been found to reduce the rate of transmission of HIV-1 from infected mother to child by 38 to 50% (2, 5, 7, 12, 16). Co-trimoxazole administered together with standard tuberculosis therapy has been found to reduce mortality and morbidity by 40 to 45% among HIV-infected tuberculosis patients (17). In order to benefit from such therapies, HIV-infected persons must be identified in a timely fashion. Because of the long turnaround time, cost, and need for trained personnel, the standard HIV serodiagnostic algorithms that require that samples be tested by enzyme-linked immunosorbant assays (ELISAs) and by Western blotting are unsuitable for use in most field conditions in developing countries. In some settings in Africa, about 2 weeks are needed before individuals can return for results. Moreover, the percent of persons returning for HIV tests results after antibody testing is as low as 25 to 38% in some populations in Africa (3). This low rate of return can severely affect prevention efforts in developing countries. Rapid enzyme immunoassays (EIAs) may circumvent these limitations: these assays have improved considerably, and some do not presently require the reconstitution of reagents and refrigeration, thus making them very suitable for use in settings with limited resources. However, limited studies exist on the sensitivity and specificity of rapid assays in areas like West Africa, where both HIV-1 and HIV-2 are endemic. It is important to evaluate regularly the sensitivities and specificities of these assays before implementing them in any HIV prevention programs because antigens used in these assays are derived from HIV-1 subtype B viruses, which are uncommon in Africa, and some studies have shown a significantly lower sensitivity of these screening assays to detect antibodies to non-B subtypes during seroconversion (14). In this study, we evaluated the sensitivities and specificities of four rapid EIAs and their efficiency when used in combination in serial and parallel testing algorithms for the confirmation of HIV infection of patients in Abidjan, Ivory Coast.

MATERIALS AND METHODS

Study design.

We first determined the sensitivities and specificities of four rapid EIAs on a panel of 1,216 sera with known HIV serologic status and two panels of sera from recently infected patients. We then selected three of the four rapid EIAs (based on sensitivity, specificity, and performance of the seroconversion panels) to prospectively evaluate serial and parallel testing algorithms performed on sera obtained from 1,179 consecutive pregnant women attending an antenatal clinic in Abidjan.

HIV antibody assays.

The four rapid EIAs we evaluated were as follows: Determine HIV-1/2 (Abbott Laboratories, Tokyo, Japan), an immunochromatography assay; Capillus HIV-1/HIV-2 (Cambridge Diagnostics, Galway, Ireland), an agglutination assay; and two immunodot assays, HIV-SPOT (Genelabs Diagnostics, Singapore, Singapore) and Genie II HIV-1/HIV-2 (Sanofi Diagnostics Pasteur, Marne la Coquette, France). HIV-SPOT was the only rapid EIA that required reagent reconstitution.

The criteria used for selecting an assay were the capacity to detect HIV-1 (groups M and O) and HIV-2 and also to detect both immunoglobulin G and immunoglobulin M antibodies. All assays were performed as recommended by the manufacturers. Invalid results were reported as indeterminate, defined for the Determine assay as a weak or missing control band. All sera were also tested by a highly sensitive and specific algorithm that uses a combination of two ELISAs (Enzygnost Anti-HIV1/2 Plus [Behring Diagnostic, Marburg, Germany] and ICE 1.0.2 [Abbott, Murex, Dartford, United Kingdom]) for the serodiagnosis of HIV infection (11). In this testing algorithm, sera concordantly reactive or nonreactive by Enzygnost and ICE 1.0.2 were considered true positive or true negative, respectively. Sera with discordant results in the two assays were tested with the Vironostika HIV Uni-Form II Plus O (Organon Teknika bv, Boxtel, The Netherlands), and the outcome was considered definitive (11). The results of this ELISA-based testing algorithm were used as a “gold standard” for evaluating the rapid assays. Each assay was performed as recommended by the manufacturers.

All four rapid EIAs were tested on a panel of 1,216 sera with known serologic status that were collected between November 1998 and January 1999 in Abidjan, Ivory Coast. These sera were from different populations: 909 pregnant women, 102 tuberculosis patients, and 205 patients seeking care in an HIV outpatient clinic. These samples had been tested by a highly sensitive and specific ELISA-based algorithm (10, 11). Of the 1,216 sera, 793 (65.2%) were HIV negative and 423 (34.8%) were HIV positive.

Seroconversion panels.

Because the sensitivities of some EIAs are lower for sera collected early in HIV infection from persons infected with a non-B subtype (14), we tested the four rapid EIAs on two seroconversion panels. The first panel consisted of 11 sera obtained from persons recently infected with HIV-1 env subtype A, as determined by V3-loop peptide serology (9). These samples were considered to be seroconvertors based on detailed serologic and virologic testing (Table 1). The second seroconversion panel consisted of sera from two patients seroconverting with HIV-1 subtype B infection obtained from Boston Biomedical, Inc. (Table 2). Each panel had been well characterized using a variety of serologic and virologic assays (U.S. Food and Drug Administration- and/or European-licensed assays), including a p24 antigen assay, a viral load determination, and HIV-1 and HIV-2 Western blottings.

TABLE 1.

Results of rapid assays performed on a panel of samples from 11 persons recently infected with HIV-1 non-B subtype

Specimen	Results of assaysa
	Determine	Genie II	HIV- SPOT	Capillus	WB1	WB2	Ag p24	Viral load (log10 copies/ ml)
1	P	P	P	P	P	U	ND	4.9
2	P	P	P	P	P	U	P	6
3	P	N	P	N	P	N	N	BD
4	P	N	P	N	U	U	P	5.6
5	P	N	P	P	U	N	P	5.4
6	P	N	N	P	U	N	P	6
7	P	N	P	P	U	N	P	BD
8	P	N	N	N	U	N	P	6.2
9	P	N	N	P	N	N	P	5.2
10	P	N	N	P	P	N	P	6.4
11	P	P	N	P	N	N	P	6.5

^aWB1, Western blotting for HIV-1; WB2, Western blotting for HIV-2; Ag, antigen; P, positive; N, negative; ND, not done; U, indeterminate; BD, below detection. 

TABLE 2.

Results of rapid EIAs performed on samples from two patients (PRB911 and PRB914) seroconverting to HIV-1 subtype B antibody positive

Panel and sample no.	Day	Results of rapid assaysa				ODb ratio of p24 antigen	Result of Western blotting
		Genie II	Determine	HIV- SPOT	Capillus
PRB911
1	0	N	N	N	N	0.30	N
2	6	N	N	N	N	0.40	N
3	8	N	N	N	N	0.30	N
4	13	P	N	N	N	3.60	U
5	15	P	P	P	P	0.40	U
6	20	P	P	P	P	0.40	P
7	22	P	P	P	P	0.40	P
8	29	P	P	P	P	0.20	P
PRB914
1	0	P	P	P	P	0.40	U
2	4	P	P	P	P	0.50	P
3	7	P	P	P	P	0.50	P
4	25	P	P	P	P	0.40	P
5	31	P	P	P	P	0.40	P

^aN, negative; P, positive; U, indeterminate. 

^bOD, optical density. 

Prospective evaluation of parallel and serial testing algorithms.

From May 1999 through September 1999, sera collected from 1,179 consecutive pregnant women attending an antenatal clinic in Abidjan, Ivory Coast, were tested using parallel and serial testing algorithms with three of the four rapid EIAs (Determine, Genie II, and Capillus), which had been chosen based on their sensitivity, specificity, ability to detect the seroconversion panels, and requirement for reagent reconstitution. In the parallel algorithm, sera that were concordantly positive or negative by the two rapid EIAs (Determine and Genie II) were considered to be true positives or true negatives. Samples that yielded discordant results between the two tests were evaluated by a third test (Capillus) (Fig. 1A).

FIG. 1.

(A) Results of the parallel serology algorithm using rapid assays. A true-positive serum was defined as concordantly positive by both rapid EIAs. A true-negative serum was defined as concordantly negative by both rapid EIAs. (B) Results of the serial serology algorithm using rapid assays. All serum specimens that were positive by Determine were further tested by Genie II and, when positive, were considered to be true positives. Specimens that were negative by Determine were reported as true negatives.

In the serial testing algorithm, sera that reacted negatively in the Determine test were considered truly HIV negative and seroreactive samples were retested by Genie II, and the outcome was considered definitive (Fig. 1B).

RESULTS

Retrospective evaluation of sensitivity and specificity of rapid EIAs.

Of the 1,216 sera, 296 (24.4%) were HIV-1 seropositive only, 73 (6%) were dually HIV seropositive, 54 (4.4%) were HIV-2 seropositive only, and 793 (65.2%) were HIV seronegative. All four rapid EIAs correctly identified the 423 HIV-seropositive samples (100% sensitivity). The specificities of the assays were 100% for Genie II, 99.7% for Capillus, 99.6% for HIV-SPOT, and 99.4% for Determine (Table 3).

TABLE 3.

Sensitivity and specificity of the four rapid assays on the 1,216 sera tested by ELISA-based reference testing algorithm

Assays	No. true positivea	No. positive in rapid test	Sensitivity (95% CI)	No. true negativea	No. negative in rapid test	Specificity (95% CI)
Genie II	423	423	100 (99.13–100)	793	793	100 (99.53–100)
Determine	423	423	100 (99.13–100)	793	789	99.4 (98.71–99.86
HIV-SPOT	334b	334	100 (98.90–100)	793	790	99.6 (99.29–99.99)
Capillus	423	423	100 (99.13–100)	793	791	99.7 (99.09–99.96)

^aThe results of the highly sensitive and specific ELISA-based algorithm (11) were used as a gold standard. 

^bBecause of a lack of a sufficient number of reagents, only 334 HIV-seropositive samples were tested by the HIV-SPOT assay. 

Sensitivity of rapid EIAs in seroconversion panels.

We determined the sensitivity of the rapid EIAs in a panel of HIV-1 subtype B sera. For patient PRB911, antibodies were detected with Genie II on the 13th day, 2 days earlier than in the other assays. HIV-1 antibodies were detected on day zero with all four rapid EIAs for patient PRB914 (Table 2). We then determined the sensitivity of the rapid EIAs on a panel of HIV-1 non-B subtypes (Table 1). Of the 11 samples from patients recently infected with HIV-1 non-B subtypes, the Determine assay detected all (100%) as seropositive, Capillus detected 8 (73%), HIV-SPOT detected 6 (54.5%), and Genie II detected only 3 (27%) (Table 1).

Evaluation of the parallel and serial testing algorithms.

Based on the sensitivities, specificities, abilities of the assays to detect the seroconversion panel, and ease of reconstitution of reagents, three of the four rapid EIAs (Determine, Genie II, and Capillus) were selected and evaluated for their use in parallel and serial testing algorithms on sera collected from 1,179 consecutive pregnant women. Genie II was selected because of its ability to serotype HIV-1 and HIV-2.

Parallel testing algorithm.

Of the 1,179 sera, 169 (14.3%) yielded concordantly seropositive results by Determine and Genie II and 999 (84.7%) were concordantly seronegative with the two rapid EIAs. Eight samples (0.7%) yielded discordant results between Determine and Genie II and were seronegative by Capillus (Fig. 1A). Three samples yielded invalid results and were excluded from the analysis. Thus, 169 (14.3%) samples were classified as HIV seropositive and 1,007 (85.4%) were classified as seronegative. All of the 169 seropositive sera were reactive by the ELISA-based testing algorithm (100% sensitivity). Similarly, all of the 1,007 seronegative samples by the rapid testing algorithm were nonreactive by the ELISA-based testing algorithm (100% specificity).

Serial testing algorithm.

Of the 1,179 sera, 177 (15%) were seropositive by the Determine assay, 999 (84.7%) were seronegative, and 3 were invalid. Of the 177 seropositive sera, 169 (95.5%) were positive by Genie II and 8 (4.5%) were seronegative. This algorithm was 100% sensitive and specific compared with the ELISA-based testing algorithm (Fig. 1B).

Cost and turnaround time of rapid EIA algorithm.

In the prospective evaluation, the cost of the reagents to analyze the 1,179 sera using the parallel testing algorithm was $6,811, or $5.80 per sample, compared with $2,754, or $2.30 per sample, for the serial testing algorithm. This represents a 2.5-fold-higher cost for the parallel algorithm. The cost of the serial testing rapid EIA algorithm was similar to that of our ELISA-based standard algorithm: $2,758 ($2.33 per sample).

In order to assess the turnaround time, we used a hypothetical situation where 100 specimens were to be analyzed by the rapid test algorithm or the ELISA-based algorithm, assuming an HIV seroprevalance rate of 15%. According to this scenario, test results would be delivered on the same day to the patients, after 60 min for the serial testing algorithm and after 120 min for the parallel testing algorithm (Table 4). However, an ELISA-based algorithm would have required 240 min to produce the final results, and it usually takes about 10 days to transcribe and report results to the patients.

TABLE 4.

Turnaround times and costs of rapid assays and testing algorithms

Tests	Time (min) required by tests:				Cost of test or testing algorithm (US $)
	For pretest preparationa	For sample testing	For test interpretation	To produce final results
Rapid tests (10 samples)
Determine	—	5	3	10	2.00
Genie II	5	15	3	25	3.80
Capillus	—	5	3	10	4.00
HIV-SPOT	5	5	3	15	NKf
Testing algorithms (100 samples)b
Parallel (Determine, Genie II)	40	60	20	120	575c
Serial (Determine → Genie II)	10	30	15	60	257cd
Parallel ELISA-based algorithm (100 samples), Engygnost, ICE 1.0.2	30	180	5	240e	234c

^aTime required to prepare either reagents or dilute samples. —, no reagent preparation or dilution is required. 

^bDiscordant results were not considered in assessing the turnaround time. 

^cThe cost of testing did not include discordant samples. 

^dFor the serial algorithm, the cost was calculated assuming a 15% seroprevalence in the testing population. 

^eTime needed to analyze 100 samples; however, about 2 weeks are required to transcribe and report results to patients. 

^fNK, not known. 

DISCUSSION

HIV serodiagnosis is critical for limiting the impact of the rapidly expanding epidemic of HIV and AIDS in Africa. Although ELISA-based serodiagnostic strategies are highly cost-effective, their application in resource-poor settings is limited by several factors, including a need for trained personnel, availability of electricity, maintenance, and the cost of equipment. Our results demonstrate that rapid assays can have high sensitivity and specificity and, when used in combination with serial or parallel testing algorithms, can yield results with an accuracy comparable to that of results obtained by ELISA-based testing algorithms. The high sensitivity and specificity we observed in this study confirm and extend findings of others (1, 4, 6, 8; N. Dinat, M. Rayfield, D. K. Smith, T. Towindo, B. Nkala, J. McIntyre, H. Rees, and N. Mqoqi, XIII Int. Conf. AIDS, abstr. MoPeA2095, 2000) by providing information on the serodiagnosis of persons infected with HIV-1 or HIV-2, those dually infected with HIV-1 and HIV-2, and those seroconverting HIV-1 non-B subtypes.

Our findings of a high sensitivity and high specificity for some rapid EIAs, especially for samples from seroconverters of HIV-1 non-B subtypes, and the fact that reagents do not need to be reconstituted for some of the assays have important public health utility in Africa. First, by using rapid EIA testing algorithms, we have observed a remarkable increase in the number of women identified as HIV positive who were eligible to receive short-course zidovudine to reduce mother-child transmission of HIV (T. S. Sibailly, E. R. Ekpini, A. Kamelan-Tanoh, D. Yavo, C. Maurice, J. Nkengasong, T. H. Roels, S. Z. Wiktor, and T. L. Chorba, XIII Int. Conf. AIDS, abstr. WeOrC549, 2000). Second, testing strategies based on rapid assays could be very useful for voluntary testing and counseling in rural areas; indeed, recent studies have demonstrated the usefulness of voluntary testing and counseling using ELISAs in preventing HIV transmission in other less-developed countries (13, 15; G. Thaver, D. Moodley, J. Moodley, B. Ngubane, and I. Boehringe, XIII Int. Conf. AIDS, abstr. MoPeA2107, 2000). Lastly, we and others have shown that co-trimoxazole administered together with standard antituberculosis therapy reduces mortality and morbidity in HIV-infected tuberculosis patients by about 40% (17); however, the effective implementation of this program requires that patients be appropriately and timely diagnosed.

One limitation of the use of rapid EIAs in developing countries is the cost. Although both parallel and serial testing algorithms had the same performance, the reagents for the parallel algorithm were 2.5-fold more costly than those for the serial algorithm. Thus, in a resource-constrained setting, rapid-test serial algorithms may be preferable. However, testing samples in a serial algorithm can misdiagnose recently infected seroconverters, as shown in Table 1. A parallel strategy increases the chances of detecting persons who are at the seroconversion phase. Nonetheless, efforts should be made to reduce the cost of rapid testing globally through UNAIDS, the World Health Organization, and other international programs.

In summary, in an area where HIV-1 and HIV-2 both circulate, we have shown that rapid EIAs can have both high sensitivity and high specificity and, when used in either serial or parallel testing algorithms, can have high sensitivity and high specificity comparable to that of ELISA-based algorithms. HIV serodiagnosis based on rapid assays may be a valuable alternative in implementing HIV prevention and surveillance programs in areas where resources are constrained and sophisticated laboratories are difficult to establish.