Large-Scale Human Immunodeficiency Virus Rapid Test Evaluation in a Low-Prevalence Ugandan Blood Bank Population

Abstract

The use of rapid tests for human immunodeficiency virus (HIV) has become standard in HIV testing algorithms employed in resource-limited settings. We report an extensive HIV rapid test validation study conducted among Ugandan blood bank donors at low risk for HIV infection. The operational characteristics of four readily available commercial HIV rapid test kits were first determined with 940 donor samples and were used to select a serial testing algorithm. Uni-Gold Recombigen HIV was used as the screening test, followed by HIV-1/2 STAT-PAK for reactive samples. OraQuick HIV-1 testing was performed if the first two test results were discordant. This algorithm was then tested with 5,252 blood donor samples, and the results were compared to those of enzyme immunoassays (EIAs) and Western blotting. The unadjusted algorithm sensitivity and specificity were 98.6 and 99.9%, respectively. The adjusted sensitivity and specificity were 100 and 99.96%, respectively. This HIV testing algorithm is a suitable alternative to EIAs and Western blotting for Ugandan blood donors.

Rapid testing for human immunodeficiency virus (HIV) has become the standard in resource-limited settings in programs for voluntary counseling and testing and the prevention of mother-to-child transmission (3, 4, 5, 12, 14). The diagnosis of HIV infection by using “gold standard” serology, usually duplicate enzyme immunoassays (EIAs) with confirmation by Western blotting, is neither feasible nor practical in many developing-world locations due to technical and financial constraints (2). In Uganda and many other countries, several rapid testing algorithms, often based on the availability of test kits rather than the proven specifications of an algorithm, are being employed. The tests most commonly used in Uganda are Uni-Gold Recombigen HIV (Trinity Biotech, Wicklow, Ireland), HIV-1/2 STAT-PAK (Chembio, Medford, NY), Determine HIV-1/2 (Abbott, Tokyo, Japan), and OraQuick HIV-1 (Orasure, Bethlehem, PA; assembled in Thailand). Numerous studies in Africa have demonstrated that commercially available HIV rapid test kits yield favorable sensitivities and specificities (8, 13, 15). However, it is also apparent that laboratory diagnosis of HIV infection is challenged by confounding regional factors. For example, Determine HIV-1/2 specificity, while above 99% in studies from Cote d'Ivoire (13), The Central African Republic (16), and The Netherlands (18), was markedly lower (91.7%) in a study in Uganda (17).

The Uganda Blood Transfusion Service (UBTS) supplies over 140,000 U of blood per year to Ugandan hospitals. Blood donations are collected at the blood donation center in Kampala and at multiple donation centers throughout the country. In addition to the Nakasero Blood Bank in Kampala, there are four additional major regional blood banks located throughout Uganda: in Gulu in the north, Mbale in the east, Fort Portal in the west, and Mbarara in the south. The headquarters ensures that health sector policy on blood transfusion is implemented and ensures adherence to quality assurance and safety requirements at all blood collection sites. As a result of the blood bank's rigorous screening program and the declining national prevalence of HIV infection (10), HIV prevalence in blood donations is less than 2%, down from 14% in 1987 (Peter Kataaha, director, UBTS, personal communication). Collaboration between the Makerere University Walter Reed Project (MUWRP) in Kampala, Uganda, and the UBTS was initiated to address several laboratory service and health care issues in Uganda, including data-driven HIV rapid test utilization. The strength of a low-prevalence population for this type study is that it provides many clinically HIV-negative specimens for specificity determination.

The objective of this study was to generate performance data that could inform the rational selection of a rapid diagnostic approach for HIV infection in the Ugandan health care setting. In phase 1 of the study, we conducted a large side-by-side comparative study to define the operating characteristics of four commercial HIV rapid tests (Determine HIV-1/2, Uni-Gold Recombigen HIV-1, OraQuick HIV-1, and HIV-1/2 STAT-PAK) for Ugandan blood bank donors. These tests were chosen based on the ease of use, flexible storage requirements, and large-scale distribution and availability in Uganda. We then used phase 1 data to devise a serial rapid testing algorithm for large-scale prospective evaluation in phase 2 of the study.

MATERIALS AND METHODS

The study received approval from the institutional review boards in both the United States and Uganda. Anonymous donors over 18 years of age that passed the blood bank's screening process for blood donation were approached for participation in the study. Donors willing to participate were required to complete a donor affidavit form, agreeing to donate residual blood for research purposes. The only identifiers known to the laboratory were the age and sex of the donor, the date and time of sample collection, and the regional center where the blood was donated. The blood bank routinely performs HIV and hepatitis B virus testing; however, this information was unavailable to study personnel at the time of collection and at any time throughout the course of the study in order to maintain the anonymity of the blood donors. Samples for phase 1 of the rapid evaluation were collected in Kampala, while samples for phase 2 were obtained from the five regional collection centers in Uganda (Gulu, Mbararra, Fort Portal, Mbale, and Kampala).

After standard blood collection in a polyethylene donation bag, residual blood remaining in the tubing was collected into Vacutainers. The tubing was manually clamped at the bag to prevent the backflow of blood and/or anticoagulant from the bag into the tubing. Samples were transported at room temperature in sealed boxes to the MUWRP laboratory, located in Kampala, Uganda, within 24 to 48 h of collection. The MUWRP laboratory is a College of American Pathologists-accredited laboratory with an annual sample throughput of 15,000 to 20,000. Serum was separated and cryopreserved at −80°C. Anticoagulated whole blood was immediately used for HIV rapid testing.

In phase 1 of the study, the four individual HIV rapid tests were evaluated in parallel against reference serology. The HIV type 1 (HIV-1) rapid tests used were Uni-Gold Recombigen HIV, HIV-1/2 STAT-PAK, Determine HIV-1/2, and OraQuick HIV-1. All rapid tests were performed and interpreted according to the instructions of the manufacturers, with the exception that the allowed time between blood collection and testing was not feasible for collections occurring throughout Uganda.

For reference serology, initial EIAs of cryopreserved sera were performed using the Genetic Systems rLAV test (Bio-Rad Laboratories, Redmond, WA). Nonreactive samples were considered negative, and no further testing was performed. Samples reactive in the rLAV EIA were tested in duplicate with the Vironostika HIV-1 micro-enzyme-linked immunosorbent assay system (Organon Teknika, Durham, NC). Results for repeatedly reactive samples were confirmed with the Genetic Systems HIV-1 Western blotting kit (Bio-Rad Laboratories). All tests were performed and interpreted according to the manufacturers' instructions. Both the EIAs and the Western blotting test are approved by the U.S. Food and Drug Administration. Samples found to be indeterminate or reactive in the Western blotting assay were subjected to the Amplicor HIV-1 monitor test, version 1.5 (Roche Diagnostics, Indianapolis, IN), in the standard mode. Samples with indeterminate Western blotting results and those with viral loads below the level of detection were considered negative (Fig. 1a).

FIG. 1.

Algorithms for serology testing for HIV infection. (a) Gold standard serology algorithm. (b) Serial rapid testing algorithm.

Based on sensitivity, specificity, and cost and availability in Uganda, a serial testing algorithm was chosen based on phase 1 data and subsequently tested in the second part of the study (Fig. 1b). In the serial testing algorithm, samples that were nonreactive in the first rapid test were considered negative while reactive samples were subsequently tested in a second rapid test. Samples that were repeatedly reactive in the first two rapid tests were considered positive. A third rapid test was performed for samples with discordant results from the first two rapid tests and served as the tie breaker.

Sensitivity, specificity, and positive and negative predictive values were calculated using previously described methods (1) and expressed with binomial 95% confidence intervals (CI) (6). Because predictive values are so significantly affected by disease prevalence, generalizable test performance is best expressed using likelihood ratios (LRs), which do not vary substantially according to disease prevalence (1). The positive LR was defined as the ratio of sensitivity to the false-positive rate: LR⁺ = sensitivity/(1 − specificity). Similarly, the negative LR was defined as the ratio of the false-negative rate to specificity: LR⁻ = (1 − sensitivity)/specificity. Positive LRs above 10 and negative LRs below 0.1 have been noted to provide convincing diagnostic evidence, and those above 5 and below 0.2 give strong diagnostic evidence (11). Sample sizes were chosen to facilitate study objectives not described here.

RESULTS

Anticoagulated whole-blood samples (940) were evaluated in phase 1. Sixty (6.4%) of the 1,000 samples obtained could not be evaluated due to insufficient sample volume or missing identifiers. Of the donors included in the study, 747 (79.5%) were male and 193 (20.5%) were female, and donor ages ranged from 18 to 56 years, with a median age of 22 years. The operational characteristics of the rapid tests compared to the reference serology are summarized in Table 1. Ten donor samples (1.06%) tested positive by reference serology. No false-negative results were observed, resulting in a sensitivity of 100% for all four rapid tests. Determine had the lowest specificity (96.2%; 95% CI, 94.7 to 97.3%), with 35 false positives, followed by Uni-Gold, with a specificity of 99.1% (95% CI, 98.2 to 99.6%) resulting from 8 false positives. OraQuick and STAT-PAK were the most specific tests, with specificities of 99.8% (95% CI, 99.1 to 99.96%) and 99.6% (95% CI, 99.0 to 99.9%), respectively.

TABLE 1.

Operational characteristics of rapid tests used in parallel^a

HIV rapid test	No. of positive results		No. of negative results		% Sensitivity (95% CI)	% Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Positive likelihood ratio (95% CI)	Negative likelihood ratio
	True	False	True	False
Determine	10	35	895	0	100 (65.5-100)	96.2 (94.7-97.3)	22.2 (11.7-37.5)	100 (99.5-100)	26.6 (19.2-36.8)	0
Uni-Gold	10	8	922	0	100 (65.5-100)	99.1 (98.2-99.6)	55.6 (31.3-77.6)	100 (99.5-100)	116.3 (58.3-231.8)	0
OraQuick	10	2	928	0	100 (65.5-100)	99.8 (99.1-99.96)	83.3 (50.9-97.1)	100 (99.5-100)	310 (100.2-959.4)	0
STAT-PAK	10	3	927	0	100 (65.5-100)	99.6 (99.0-99.9)	76.9 (46.0-93.8)	100 (99.5-100)	465 (116.5-1,856.6)	0

^aPPV, positive predictive value; NPV, negative predictive value.

Based on the above-listed findings, we chose a serial testing algorithm with Uni-Gold as the screening test, followed by STAT-PAK and then OraQuick as the tie breaker, for the second part of the study (Fig. 1b). The two tests with the highest specificities were reserved for the second- and third-line testing. Although the specificities of OraQuick and STAT-PAK were equivalent (98.8% versus 98.6%), STAK-PAK was chosen based on its lower cost per test and widespread availability in Uganda. For the screening test, although Determine and Uni-Gold were equally sensitive, we felt that the rate of false positives with Determine was too high for the test to be included in the algorithm. The serial rapid testing algorithm was performed on 5,252 anticoagulated samples, and the results were compared with those of the reference EIA-Western blotting serology as described above. Of the donors, 3,913 (74.5%) were identified as male, 1,279 (24.3%) were identified as female, and 60 were not identified by sex. Donor ages ranged from 18 to 59 years, with a median age of 20 years. The sample collection locations were equally distributed among the five regional centers throughout Uganda, and the HIV prevalence ranged from 0.9 to 2.3%. Seventy-two samples (1.4%) were found to be positive by the reference serology, and there were not statistical differences in HIV prevalence among study sites (data not shown).

The initial analysis of data from the fresh-whole-blood serial rapid testing algorithm (Uni-Gold→STAT-PAK→OraQuick) revealed the operational test characteristics shown in Table 2. The algorithm resulted in a sensitivity of 98.6% (95% CI, 91.4 to 99.9%) due to one false negative and a specificity of 99.9% (95% CI, 99.76 to 99.96%) due to five false-positive results.

TABLE 2.

Operational characteristics of the serial rapid testing algorithm^a

Status of results	No. of positive results		No. of negative results		% Sensitivity (95% CI)	% Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Positive likelihood ratio (95% CI)	Negative likelihood ratio (95% CI)
	True	False	True	False
Unadjusted	71	5	5,175	1	98.6 (91.4-99.9)	99.90 (99.76-99.96)	93.4 (84.7-97.6)	99.98 (99.87-99.99)	1,021.6 (425.2-2,454.5)	0.014 (0.002-0.097)
Adjusted	72	2	5,178	0	100 (93.7-100)	99.96 (99.84-99.99)	97.3 (89.7-99.5)	100 (99.9-100)	2,590 (647.9-10,353.2)	0

^aPPV, positive predictive value; NPV, negative predictive value.

Due to the high volume of testing during the study (50 to 100 samples per day) and the chance for technical error, discrepant rapid testing and reference serology results were evaluated by repeating both types of tests using cryopreserved serum samples. The single false-negative sample tested positive in rapid tests upon repeat testing, and three of the five false-positive samples tested negative upon the repetition of the rapid tests with the cryopreserved serum samples.

Retrospective serostatus resolution resulted in an adjusted sensitivity of 100% (95% CI, 93.7 to 100%) and a specificity of 99.96% (95% CI, 99.84 to 99.99%).

DISCUSSION

The need for affordable, accurate, and feasible HIV testing is critical in the developing world. Numerous commercial rapid HIV tests are available; however, it cannot be assumed that all tests perform equally well. In the first part of the study, we carefully scrutinized the operational characteristics of four HIV rapid tests (Determine, Uni-Gold, STAT-PAK, and OraQuick) in a low (1.4%)-prevalence setting. All four tests demonstrated a sensitivity of 100%; however, the specificities of the four tests were more heterogeneous. The two tests with the highest specificities were OraQuick (99.8%) and STAT-PAK (99.6%). Determine, a test commonly used in developing countries, had the lowest specificity (96.2%) of the four tests. A recent study, also in Uganda, suggests that the lower specificities of Determine and Uni-Gold may be caused by unclear interpretation of weakly positive bands (9).

The serial testing algorithm (Uni-Gold→Stat-Pak→OraQuick) chosen for the second part of the study proved to be highly sensitive (98.6%) and specific (99.9%). Despite this exceptional performance with more than 5,000 samples, the desired sensitivity of 100% was not achieved. Upon further analysis of cryopreserved samples, it was determined that technical error likely caused the one false-negative result and three of the five false-positive results from the serial testing algorithm. Without the technical errors, the algorithm would have yielded an adjusted sensitivity of 100% and an adjusted specificity of 99.96%.

The suspected technical errors made in this study emphasize the limitations of rapid tests and the need for a stringent quality management program. Due to the nature of sample collection for this study, large batches of samples (50 to 100) were received in the laboratory and immediately tested using the rapid tests. Possible errors include labeling or transcription mistakes and/or sample mix ups. This discrepancy illustrates the difference between ideal and real-world testing conditions, a concept well illustrated by clinical pharmaceutical trials, in which efficacy is defined as how well the drug works under ideal conditions and effectiveness is defined as how well a treatment works given the real-world variabilities (7). Fortunately, rapid tests in developing countries are most often being used in the context of programs for voluntary counseling and testing and the prevention of mother-to-child transmission, in which real-time rather than batch testing is performed, reducing the chances for errors associated with high-throughput testing.

The data presented here strongly support HIV testing using a serial testing design with Uni-Gold Recombigen HIV-1, HIV-1/2 STAT-PAK, and OraQuick HIV-1 in the Ugandan health care setting. HIV EIA and Western blotting capability in Uganda may be best employed in a small number of Ugandan reference centers supporting high-throughput batch testing.