Use of Stored Serum from Uganda for Development and Evaluation of a Human Immunodeficiency Virus Type 1 Testing Algorithm Involving Multiple Rapid Immunoassays

Abstract

We report the development and evaluation of a human immunodeficiency virus type 1 testing algorithm consisting of three rapid antibody detection tests. Stored serum samples from Uganda were utilized with a final algorithm sensitivity of 100% and a specificity of 98.9% (95% confidence interval, 98.6% to 99.3%).

Providing humanitarian or medical assistance in resource-limited environments poses challenges for health care workers, including occupational exposure to blood-borne pathogens. Evaluating the risk of human immunodeficiency virus type 1 (HIV-1) transmission includes the timely determination of source patient seroreactivity. Traditional HIV-1 serology by enzyme immunoassay (EIA) and Western blot analysis is very time-consuming and may be unavailable in resource-limited settings. Reflexive initiation of postexposure prophylaxis (PEP) consumes potentially limited stocks of antiretroviral drugs with the potential for toxicity. Rapid HIV-1 testing can assist PEP decision-making (13) in these time-critical and supply-constrained situations (3, 5, 6).

While the risk of occupational transmission is related to dose, route of exposure, and viral load of the source material, insufficient clinical evidence exists to definitively address the timing of PEP following exposure (2). Animal model data suggest that the decision to use antiretrovirals must be made efficiently. Dendritic cells are infected within 24 h in macaques mucosally exposed to simian immunodeficiency virus, with migration to regional lymph nodes in the subsequent 24 to 48 h (9). PEP failed in this model when initiated at a time beyond 48 h (11). HIV-1 screening in voluntary counseling and testing settings requires diagnostic algorithms based on HIV-1 prevalence, assay operating characteristics, and cost constraints. Postexposure testing must also be timely to complement PEP decisions.

Four HIV-1 rapid EIAs were evaluated for use in a multitest algorithm. We chose specimens from among 14,000 HIV-1 EIA-nonreactive and 1,500 HIV-1 EIA-reactive cryopreserved serum samples collected from an HIV-1 seroprevalence study conducted in the Rakai District of Uganda (14) with an HIV-1 prevalence of 16.9% (8). These samples were shipped frozen to the Walter Reed Army Institute of Research (WRAIR) in Rockville, MD, where the serum was stored at −70°C. The Rakai project, study number M-1356, was approved by the human-use review boards of the WRAIR, the U.S. Army Medical Research and Material Command, the Uganda Virus Research Institute, and the AIDS Research Subcommittee of the Uganda Council for Science and Technology.

All reference and rapid serologic testing was performed at the WRAIR. The OraQuick HIV-1 rapid antibody test (OraSure Technologies, Bethlehem, PA), Determine HIV-1/2 (Abbott Laboratories, Inc., Abbott Park, IL), Hema-Strip HIV-1/2 (Saliva Diagnostic Systems, Inc., Medford, NY), and InstantScreen rapid HIV-1/2 assay (German-American Institute For Applied Biomedical Research, Potsdam, Germany) were performed according to the directions on the package inserts. Test results were read by study investigators who were unaware of sample seroreactivity. Serum specimens were then subjected to EIA screening, confirmatory Western blotting (reference serology), and quantitative viral RNA testing (if indicated) (1, 7) essentially as described previously (4). Operating characteristics, predictive values, statistical simulation of the various permutations of rapid tests, and the performance of the final testing algorithm were calculated using STATA version 7.0 (STATA Corporation, College Station, TX).

Specimens for HIV-1 testing were subjected to testing in two phases. The first phase included 1,000 samples randomly selected from the pool of 15,500 samples. The operating characteristics of the rapid tests were compared to reference testing, and descriptive epidemiological characteristics of the tests were calculated (Table 1). The Hema-Strip HIV-1/2 was insensitive (92.5% sensitivity; 95% confidence interval [CI], 90.8 to 94.1%) compared to the 100% sensitivities observed for the other three rapid tests. The OraQuick test demonstrated the best specificity (97.6% specificity; 95% CI, 96.6 to 98.5%), with both the Determine HIV-1/2 and InstantScreen assays demonstrating specificities of <92%.

TABLE 1.

Rapid HIV-1 test operating characteristics^a

Test	Sensitivity		Specificity		PPV		NPV
	%	95% CI	%	95% CI	%	95% CI	%	95% CI
OraQuick	100	100	97.6	96.6-98.5	80.9	78.4-83.3	100	100
Hema-Strip	92.5	90.8-94.1	94.5	93.1-95.9	63.2	60.3-66.2	99.2	98.6-99.8
InstantScreen	100	100	91.8	90.1-93.5	55.7	52.6-58.8	100	100
Determine	100	100	91.7	90.0-93.4	55.4	52.3-58.4	100	100

^aTesting was accomplished through the use of 1,000 randomly selected Ugandan serum samples drawn from a panel of 15,500 samples (1,500 seropositive and 14,000 seronegative). Resultant HIV-1 seroprevalence in this phase was 9.3%.

Hypothetical evaluations of three-test serial and parallel designs were performed. The serial design (Fig. 1A) employs a single screening test. Should that test be nonreactive, the sample was considered HIV-1 negative. Should the first test be reactive, a second test was conducted. An HIV-1-positive final result was assigned in the case of concordance between the results of the first two tests. A third test was performed in cases of discordance, the result of which determined the HIV-1 status. The parallel design (Fig. 1B) utilized two tests performed simultaneously, and concordant reactive or nonreactive results would indicate HIV-1-positive or -negative results, respectively. The third test was utilized for discordant test pairs, the result of which would determine the HIV-1 status. All of the designs, including the combination of the OraQuick, InstantScreen, and Determine HIV-1/2 assays, had equivalent performances.

FIG. 1.

(A) Serial design. The sample is subjected to a single rapid test in which a nonreactive result is reported as seronegative. A reactive sample is subjected to a second rapid test, in which a concordant reactive result is reported as seropositive. Discordance is resolved by the reactivity of a third test, and serologic status is reported by best-of-three agreement. (B). Parallel design. The sample is subjected simultaneously to two rapid tests. Concordant seroreactivity or nonreactivity is reported as seropositive or seronegative, respectively. Discordance is resolved by the reactivity of a third test, and serologic status is reported by best-of-three agreement. This design was selected for use in the algorithm evaluation phase of this study.

Final algorithm selection was based on time to results and concern for false negatives seen with chronic HIV-1 infections (13) that are manifest in resource-limited areas. Therefore, the parallel design was chosen to diversify the first-line antigen detection by utilizing two different proprietary systems. The best-performing algorithms displayed the following performance characteristics: sensitivity, 100%; specificity, 99.6% (95% CI, 99.5 to 100%); positive predictive value, 97.9% (95% CI, 97.0 to 98.8%); and negative predictive value, 100% (prevalence, 9.3%). The decision to utilize the OraQuick-Determine-InstantScreen algorithm was based on the ease of use and reduced time to results (22 min versus 35 min) to complete each algorithm (Table 2). Use of the higher-specificity test earlier in the hypothetical algorithm made no difference in the results.

TABLE 2.

Performance of hypothetical algorithms calculated by statistical modeling (STATA version 7.0) of individual test operating characteristics derived from 1

Design	Algorithma	% Sensitivity (95% CI)	% Specificity (95% CI)	Time to resultsb(min)
Parallel	OraQuick-Determine-InstantScreen	100 (100)	99.6 (99.0-99.9)	22
	Determine-InstantScreen-OraQuick	100 (100)	99.6 (99.0-99.9)	35
Serial	Determine-InstantScreen-OraQuick	100 (100)	99.6 (99.2-100)	37
	InstantScreen-OraQuick-Determine	100 (100)	99.6 (99.0-99.9)	37
	OraQuick-Determine-InstantScreen	100 (100)	99.6 (99.0-99.9)	37

^aTest order is given from left to right.

^bTime to results by algorithm was calculated by actual performance of each algorithm.

Thirty-five hundred samples were randomly selected from the remaining pool of 14,500 specimens and subjected to the OraQuick-Determine-InstantScreen parallel algorithm, and the results were compared to those of reference serology. Of these, 474 specimens were concordantly HIV-1 positive, 2,993 specimens were concordantly HIV-1 negative, and 33 specimens were false positive by the rapid-test algorithm (Table 3). There were no false-negative results. This describes a sensitivity of 100% (95% CI, 100%) and a specificity of 98.9% (95% CI, 98.6 to 99.3%). The PPV and NPV, based on a prevalence of 13.5%, were 93.5% (95% CI, 92.7 to 94.3%) and 100% (95% CI, 100%), respectively. All rapid-test-algorithm false-positive sample results were below detection by viral RNA testing (data not shown).

TABLE 3.

Results of the parallel testing algorithm OraQuick-Determine-Instant-Screen^a

Rapid-test algorithm results	Double EIA-Western blot algorithm results
	No. of samples positive	No. of samples negative	Total no.
Positive	474	33	507
Negative	0	2,993	2,993
Total	474	3,026	3,500

^aTest order is given left to right. A total of 3,500 serum samples were randomly selected from the remaining panel of 14,500 samples (1,000 had been selected for the first phase of the study), with a resultant HIV-1 seroprevalence of 13.5%. Results of the parallel rapid-test algorithm are compared with those of serial EIA and confirmatory Western blotting with the resulting performance characteristics: sensitivity, 100% (95% CI, 100%); specificity, 98.9% (95% CI, 98.6 to 99.3); PPV, 93.5% (95% CI, 92.7 to 94.3); NPV, 100% (95% CI, 100%).

Parallel testing is not indicated for routine HIV-1 screening, especially in areas of high endemicity (15), as it is not cost-effective. However, postexposure HIV-1 testing addresses different goals. An ideal postexposure testing algorithm would include diverse antigen sources and different test principles (10, 12) to eliminate the possibility of false-negative results (13) and to provide accurate results within the shortest time (2, 15). The OraQuick-Determine-InstantScreen algorithm described here would cost approximately $12 versus $8 for testing with OraQuick alone, but it demonstrated increased specificity and would reduce the less easily defined costs of unnecessary administration of antiretrovirals with their attendant toxicities. This approach, in combination with risk factor analysis, might be especially relevant in resource-limited environments where available antiretroviral drugs are scarce.