Field Evaluation of Diagnostic Accuracy of an Oral Fluid Rapid Test for HIV, Tested at Point-of-Service Sites in Rural Zimbabwe

Abstract

The objective of this study was to validate the use of OraQuick^® ADVANCE Rapid HIV-1/2 Antibody test (OraSure Technologies Inc., Bethlehem, PA) on oral fluid for a population-based HIV prevalence survey of rural youth in southeast Zimbabwe. The evaluation was conducted in patients presenting for voluntary counseling and testing at rural clinics. Each participant provided an oral fluid sample tested using OraQuick^® ADVANCE. In addition, dried blood specimens were collected and tested blind at the National Microbiology Reference Laboratory in Harare using two enzyme-linked immunosorbent assays (ELISA; Vironostika^®, Biomérieux BV, Boxtel, The Netherlands and Ani Labsystems, Ltd., Vantaa, Finland) with confirmatory Western blot (MP Diagnostics [formerly Genelabs Diagnostics], Medical Technology Promedt Consulting GMBH, St. Ingbert, Germany) for samples with discrepant results. Diagnostic accuracy of the oral fluid assay was determined against the ELISA/Western blot algorithm as gold standard. Five hundred and ninety-one participants took part in the study between February and July 2006. Sensitivity of the test on oral fluid was 100% (95% confidence interval [CI]: 97.9–100), and specificity was 100% (95% CI: 99.1–100). HIV prevalence based on the reference standard was 29.8% (95% CI: 26.1–33.5). This is one of the first validations of this rapid assay on oral fluid conducted in a general population to be reported in Africa. While there are some limitations with the assay (e.g., unlikely to detect those in early stages of HIV infection or with reduced viral load; altered accuracy in pregnancy) these limitations also apply to other rapid assays. The results showed the assay to be 100% accurate in determining HIV status, performed well in field settings, and can be considered suitable for use in epidemiologic surveys aiming to estimate HIV prevalence in general populations.

Introduction

Serologic surveys of HIV prevalence have traditionally required collection of venous blood, or whole blood collected onto filter paper as a dried blood specimen (DBS). The correlation between enzyme-linked immunosorbent assays (ELISA) for HIV on these specimens is excellent and well documented.^1–3 However, taking blood samples during population-based surveys can be problematic: skin puncture increases the complexity of equipment required and the risk of needle-stick injuries to staff. In some populations, blood draw is problematic for religious or cultural reasons. For example, Gregson et al.⁴ found that fear of Satanism was a common reason for refusing to provide DBS samples in an epidemiologic survey in Zimbabwe. The processing, storage, and transport of venous blood is also cumbersome.

In the last two decades, a number of rapid test assays have been developed that enable HIV antibody status to be determined in as little as 20 minutes at the point-of-care (POC). Most are designed to detect antibodies in several different body fluids including whole blood from finger-prick blood, plasma, urine, or saliva. Rapid tests are simple to perform, can be conducted in rural settings without laboratory equipment, and obviate the need to process and store specimens and transport them from the field. Using oral fluid eliminates the need for trained phlebotomists. These advantages mean that not only are these assays easier to utilize and implement for POC testing, particularly in urgent situations such as labor wards, but they also have potential benefits for research and surveillance purposes, such as undertaking population-based epidemiological surveys of HIV.^5–7 The OraQuick^® ADVANCE Rapid HIV-1/2 Antibody test (OraSure Technologies Inc., Bethlehem, PA) is one of these rapid assays and currently the only U.S. Food and Drug Administration (FDA)-approved and Clinical Laboratory Improvements Amendments Act of 1988 (CLIA)-waived rapid POC test for use with oral fluid.^5,8–10

For the last decade, the OraQuick test has been used as a rapid test with oral fluid at a number of testing sites funded by the Centers for Disease Control and Prevention (CDC) and by the Boards of Health for a number jurisdictions within the United States.^5,11,12 Independent studies to evaluate the performance of this test in real clinic settings, and postmarketing surveillance to monitor the products performance, have found the specificity of this rapid assay with whole blood and oral fluid overall, to be in accordance with the manufacturer's claims (i.e., 99.8% for oral fluid [95% confidence interval {CI}: 99.6–99.9] and 100% for whole blood [95% CI: 99.7–100]).^5,11–13 The preference and acceptability of this assay has also been shown by an increase in the number of persons who are testing at these sites and receiving their results.^5,11 However, since 2005 there have been occasional increases in the incidence of false-positive test results on oral fluid specimens reported at some sites.^5,11–13 These false-positives have occurred as isolated clusters, which have on occasions reduced the specificity of the test on oral fluid below the lower limit of the manufacturer's specifications and resulted in a lack of confidence in the test and suspension of its use by some Boards of Health (e.g., New York City), even though the overall specificity has not fallen below the FDA minimum threshold of 98% require for rapid HIV tests.¹¹ The CDC continues to encourage the use of this rapid test but is working with the FDA and manufacturer to investigate the causes of these clusters of false-positive results.^11,13

Data regarding the use and performance of oral fluid HIV assays are limited, particularly in Africa, and a number of those studies that have been conducted focus solely on testing pregnant women and women in labor.^14,15 Only five published studies from Africa could be located, three of which validated the OraQuick test.^16–21 The first study tested 377 adult inpatients with suspected tuberculosis in Botswana and validated results from gingival secretions and sputum samples against an enzyme-linked immunoassay (ELISA)/Western blot algorithm performed on sera.¹⁹ The second study tested 235 children (aged 11 to 18 months) born to HIV-positive women in South Africa and validated oral fluid results against a serum ELISA test with HIV DNA polymerase chain reaction (PCR) plus clinical evaluation.²⁰ Both studies involved high-risk populations and as such the test performance in these groups is likely to be different and less applicable to general populations.¹⁶ The third and most recent study, was a cross-sectional study (n = 273) conducted in pregnant women of unknown HIV status attending two antenatal clinics (ANC) in Namibia. This study validated oral fluid HIV tests against serial dual ELISA testing on blood plasma.²¹

In this article we report data from a study to validate the use of the OraQuick^® ADVANCE rapid assay using oral fluid, for a representative population-based HIV prevalence survey of rural youth in southeastern Zimbabwe, conducted in 2006. Participants were clients presenting for voluntary counseling and testing (VCT) at rural clinics in the population-based survey area.

Methods

Study setting and participants

The study was conducted in 39 rural health clinics in southeastern Zimbabwe that were taking part in an ongoing project.²² Participants were community members presenting for VCT between February and July 2006. VCT sessions included in the validation were chosen pragmatically in advance based on logistic considerations. VCT was administered by trained project nurses certified to undertake rapid HIV testing by the Ministry of Health and Child Welfare (MoHCW) in Zimbabwe.

All participants presenting at selected sessions, who consented to VCT were considered eligible. Each had HIV testing performed using the national HIV rapid testing algorithm (Uni-Gold^TM HIV Test; Trinity Biotech PLC, Bray, Ireland) and Abbott Determine HIV-1/2 Rapid test kit; Abbott Japan Co. Ltd., Tokyo, Japan) on whole blood from a finger-prick blood sample. In addition, five DBS samples were collected from that sample. Once VCT rapid test and DBS specimens had been taken, each participant also provided an oral fluid sample that was collected and tested onsite using the OraQuick test according to the manufacturer's package insert.²³ The age and gender of each participant was recorded.

Collection and testing of the oral fluid sample using OraQuick

Each OraQuick kit when opened was labeled with a unique bar code label and the VCT ID for that participant. The oral fluid specimen was collected by swabbing the flat pad of the test device around the outer surface of the gums; this was then placed in the developer solution. The swab was allowed to incubate in the vial for 20 minutes, timed with a stopwatch, after which the result was read from the “Results” window on the test device.

Collection of dried blood spot samples and laboratory analysis

DBS samples were collected onto cotton fiber-based filter papers (No. 903, Schleicher and Schuell, Riviera Beach, FL) from a finger-prick blood specimen according to standard protocol.²⁴ Once dried, samples were stored at room temperature until transported to the National Microbiology Reference Laboratory in Harare for HIV-1 antibody testing using a validated testing algorithm.²⁵ All specimens were tested using two ELISA tests (Vironostika^® HIV Uni-Form II Ag/Ab Microelisa System; Biomérieux BV, Boxtel, The Netherlands) and AniLabsytems HIV EIA; AniLabsystems Ltd., Vantaa, Finland), with Western blot (MP Diagnostics [formerly Genelabs Diagnostics], Medical Technology Promedt Consulting GMBH, St. Ingbert, Germany) used in the case of discrepant results. Laboratory personnel testing the DBS samples were blinded to the oral fluid and VCT rapid test results.

Sample size and statistical analyses

The study was powered to allow precise measures of the sensitivity and specificity of OraQuick when compared with the ELISA/Western blot algorithm as gold standard. With almost 600 participants and an assumed HIV prevalence of 20%, the study was able to measure sensitivity and specificity of 95% to within ± 4%.

Sensitivity, specificity, and predictive values with exact 95% confidence intervals (CIs), were calculated using standard formulae, in order to determine the diagnostic accuracy of the OraQuick test. The result of the ELISA/Western blot algorithm was taken as the reference standard.^26,27 Participants whose true infection status could not be determined (i.e., with indeterminate Western blot results) were excluded from the analyses. All analyses were conducted using Stata 10 (Stata Corporation, College Station, TX).

Ethics

All participants provided written informed consent, consenting to having blood taken and tested for HIV antibodies. HIV testing by project staff for the purpose of VCT and survey work was approved by ethics committees at University College London, the London School of Hygiene and Tropical Medicine, and the Medical Research Council of Zimbabwe.

Results

Figure 1 presents the flow of participants through the validation study. In total, 591 participants took part in the study between February and July 2006; 517 were female (79.9%). Median age of males was 36 years and for females it was 34 years (overall range, 16–80 years). None of the participants refused to provide any of the required specimens. Seven participants were excluded from the analyses as a result of mislabeled specimens (making it impossible to match dried blood and oral fluid results; n = 6), and one indeterminate ELISA/Western blot result; (OraQuick and VCT rapid test results indicated a negative result); these were collected from 4 different sites by 3 different nurses.

FIG. 1.

Flow diagram of validation study of OraQuick^® ADVANCE Rapid HIV-1/2 Antibody test (OraSure Technologies Inc., Bethlehem, PA) using oral fluid.

Diagnostic accuracy of the OraQuick rapid test

Comparison of results from the oral fluid test with results from the reference standard ELISA/Western blot algorithm for the 584 participants included in the analyses show that the OraQuick rapid test performed on oral fluid specimens had a diagnostic accuracy of 100% (sensitivity was 100% [95% CI: 97.9–100] and specificity was 100% [95% CI: 99.1–100]; positive and negative predictive values were both 100%). Oral fluid results also correlated 100% with final results of the VCT rapid tests on whole blood (data not presented).

Prevalence of HIV infection

The overall HIV prevalence determined using the reference standard algorithm was 29.8% (95% CI: 26.1–33.5) and was similar for male and female VCT attendees (male 30.5% [95% CI: 22.1–38.9] versus female 29.6% [95% CI: 25.4–33.8]).

Discussion

This study is the one of the first validation studies of the OraQuick^® ADVANCE Rapid HIV1/2 assay on oral fluid conducted in a general population to be reported in Africa, and is the largest validation study thus far conducted in the region.²¹ Based on results of this study, we found this assay was 100% accurate in determining HIV status and that its diagnostic accuracy was not only equivalent to rapid assays using whole blood currently used for providing VCT within Zimbabwe, but was also as accurate as using DBS specimens tested by two ELISAs. The OraQuick test has been validated on serum samples by the World Health Organization (WHO), but its use with oral fluid has not been validated.^16,28 The sensitivity and specificity results obtained in this study appear to meet WHO recommendations outlined for tests for surveillance purposes. These guidelines recommend that for surveillance testing in populations where HIV prevalence is greater than 10%, one assay with a specificity of at least 98% should be used (Strategy I).²⁸ Prior to this study and the Namibian study, none of the validation studies of oral fluid assays conducted in Africa had achieved sensitivity and specificity results that met these criteria.^16,21 Our sensitivity and specificity estimates are higher than those obtained in the Botswana study (sensitivity = 98.4% [95% CI: 96.5–99.4]; specificity 98.3% [95% CI: 91.9–99.9]), or in the South African study in infants and young children (87% sensitive; 97% specific [95% CI not reported]); and concur with results from a hospital-based study conducted in rural India and the Namibian ANC survey, both of which found the oral fluid test to be 100% accurate.^10,19–21 The larger sample size of this Zimbabwean study enables us to report these estimates of diagnostic accuracy with greater precision than any previous African studies. These data are also consistent with the manufacturer's clinical trial data reported to the FDA (sensitivity = 99.3% [95% CI: 98.4%–99.7%]; specificity = 99.8% [95% CI: 99.6%–99.9%]), as validated against ELISA/Western blot) and with other evaluations conducted in other settings (labor and delivery settings, clinics, outreach venues, and state and city health departments) in the United States.^5,12,23 Some studies have shown the OraQuick test with oral fluid to be marginally less specific than whole blood or serum tests.^5,12 Postmarketing surveillance of OraQuick reported by Wesolowski et al.¹² estimated the median specificity of Oraquick on whole blood to be 99.98% (95% CI: 99.73%–100%) compared to a specificity of 99.89% (95% CI: 99.44%–100%) on oral fluid. Delaney et al.⁵ estimated specificity to be 99.9% on whole blood and 99.6% with oral fluid in their evaluation of the performance of OraQuick from four CDC studies. Even with the occasional occurrence of isolated clusters of increased false-positive tests with oral fluid, the overall test performance has remained within the confidence interval limits specified by the manufacturer, and above the FDA minimum threshold for rapid HIV tests.¹¹

The consistent performance of OraQuick with oral fluid in these real-world settings suggests that its use to determine HIV status in population-based HIV prevalence surveys, or for other community-based epidemiologic surveys aiming to estimate the burden of HIV infection in rural and resource-limited settings, is both valid and appropriate.^5,12 Currently, the World Health Organization (WHO) recommends that for diagnostic purposes, two assays be used with a third test for discrepant results (Strategy II and III); the first test must have the highest sensitivity and the second test a similar or higher specificity.²⁸ If an additional oral fluid HIV test were approved, the performance characteristics of OraQuick suggest that it would also be suitable for inclusion as part of diagnostic VCT algorithms.

Client/participant preference for this non-invasive rapid assay has been widely reported.^{10,14–16,18,29–32} This evaluation of OraQuick performance was run in order to validate the use of this assay for a representative population-based HIV prevalence survey of rural Zimbabwean youth we conducted in 2006. While test preference was not specifically explored in this validation survey, in the house-to-house population-based survey we obtained very high recruitment rates: 92% of eligible participants (n = 3960) were recruited; no participant refused to provide an oral fluid sample. This suggests that this test was acceptable to the population being surveyed. In comparison, in the most recent Zimbabwe Demographic Health Survey for 2005–2006, which sought to estimate HIV prevalence based on ELISA/Western blot testing of DBS specimens, 15% of eligible participants recruited refused to allow a blood sample to be taken (reasons for refusal were not reported).³³ Similar high levels of refusing blood draw were also reported among participants in the Zimbabwe Young Adult Survey 2001–2002.³⁴

There are some limitations to the use of oral fluid for rapid HIV antibody testing. There is no stored sample for further or repeat testing, which can be done with whole blood, serum or plasma specimens. This assay is also unlikely to detect those in early stages of HIV infection, or those where viral load has been significantly reduced by antiretroviral therapy, and there is some evidence that pregnancy may alter the accuracy of HIV testing, although these points apply to all the rapid assays currently available.^8,15,23,35 Collection of oral fluid specimens which can be stored and tested at a later date or used for repeat or confirmatory testing, using appropriate oral fluid collection devices (e.g., Orasure^® HIV-1 oral specimen collection device), is also possible.³⁶ However, this process, while still noninvasive, reintroduces transport and storage requirements (specimens must be protected from impact and direct sunlight and stored for no more than 21 days at 4°C to 37°C), the need for trained laboratory personnel, and the various associated costs, aspects that are avoided with the POC rapid tests.

In conclusion, we found that the OraQuick^® ADVANCE Rapid HIV-1/2 Antibody test has excellent performance characteristics when compared with standard rapid HIV testing algorithms; it performs well in field settings and is suitable for use (either on its own or as part of dual testing) in epidemiologic surveys and other surveillance initiatives undertaken in similar resource-limited settings within sub-Saharan Africa.

Acknowledgements

The authors would like to acknowledge Hazvineyi Musabaike (NMRL Harare) who was responsible for undertaking all of the laboratory analysis for this study; Roycent Tumbare, the Field Coordinator for the study; and the entire Regai Dzive Shiri project staff involved in the collection of these data, and the participants themselves.

The authors acknowledge the support of OraSure Technologies Inc. for the donation of kits used in this study. Orasure Technologies Inc. did not provide funding for this study, nor did it have any role in the design or conduct of the study or in the preparation of the manuscript.

This study was part of a large study funded through the National Institute of Mental Health (RO1 MH-66570 PI Cowan Frances Mary).

Author Disclosure Statement

No competing financial interests exist.