Evaluation of Performance and Acceptability of Two Rapid Oral Fluid Tests for HIV Detection in Mozambique

Abstract

Simplified HIV testing based on oral fluid (OF) may allow the expansion of HIV infection counseling and testing (CT) while reducing the risk due to exposure to needles and blood collection. This study evaluated the performance and acceptability of two OF tests (the OraQuick Advance Rapid HIV-1/2 and the Chembio DPP HIV-1/2) from May to September 2009 in two CT sites in Maputo City, Mozambique, compared with results for the national testing algorithm. OF testing was conducted in parallel with whole blood-based testing according to the national HIV algorithm. Blood samples were collected as dried blood spot (DBS) specimens from all participants for quality assurance. HIV infection results were delivered according to the national algorithm. According to the national HIV algorithm, 512 (30.5%) samples were reactive, 1,151 (68.7%) were nonreactive, and 13 (0.8%) were discordant. All discordant cases were retested with an enzyme immunoassay followed by Western blotting, and five (38.5%) were confirmed as HIV positive. The OraQuick OF test showed 518 (30.9%) reactive samples and 1,158 (69.1%) nonreactive samples, with a sensitivity and specificity of 99.8% and 99.8%, respectively. The Chembio DPP OF test showed 519 (31.0%) reactive samples and 1,157 (69.0%) nonreactive samples with a sensitivity and specificity of 100% and 99.8%, respectively. The participants perceived blood testing (49.9%) to be more accurate than OF testing (46.8%). The OF tests showed high performance for the diagnosis of HIV infection when examined individually and in an algorithm, compared with results according to the national testing algorithm.

INTRODUCTION

Increasing the number of people who have knowledge of their HIV status is imperative for the containment of an epidemic that affects 11.5% of the population in Mozambique (1, 2). Since 2006, Mozambique has aimed to expand the coverage of HIV testing and counseling through integration into a broader health promotion package, including systematic screening and necessary referrals for tuberculosis, sexual transmitted infections, and hypertension, among other health issues. This approach, referred to as counseling and testing in health (CTH), is offered to walk-in clients specifically seeking HIV testing in health care facilities (CTH-F) and is also delivered in community settings (CTH-C) by mobile teams. HIV testing is also offered through a third modality, provider-initiated testing and counseling (PITC), which is routinely offered to clients seeking other health services. However, national coverage of HIV testing remains low, and testing does not reach the people in all communities, particularly those living in remote areas. The introduction of a rapid test based on oral fluid (OF) has the potential to have a major impact on program expansion, as it is painless and noninvasive (3, 4). Considering its ease of use, this technology is suitable for CTH, for surveillance purposes (3, 5), in antenatal care clinics for testing pregnant women, and in mobile testing units working in remote or hard-to-reach areas (6). OF-based testing can also reduce the risk to health workers of accidental exposure to HIV through contaminated blood and further reduce the need for disposal of biohazardous waste (5, 7).

Previous studies conducted in sub-Saharan Africa to assess the performance of the OraQuick Advance Rapid HIV-1/2 test demonstrated that the results are highly concordant with the results obtained using conventional blood-based HIV rapid tests (5, 8, 9). Recently the ChemBio DPP HIV-1/2 test was evaluated in Nigeria, demonstrating very promising results (10). However, no study has been conducted as yet to evaluate the performances of the OraQuick and the Chembio DPP tests in a testing algorithm using oral fluid specimens. Furthermore, neither of these tests has been assessed in Mozambique. In this context, the main objective of this study was to validate these two OF rapid tests (the OraQuick Advance Rapid HIV-1/2 and the Chembio DPP HIV-1/2) based on antibodies against HIV-1/2 at CTH sites in Mozambique, while comparing their performances with that of the whole blood-based national algorithm and assessing the acceptability of OF testing for HIV infection.

MATERIALS AND METHODS

Test devices.

The OraQuick test (OraSure Technology, Malaysia) is a qualitative immunochromatographic test for the detection of antibodies against HIV-1/2. It consists of two components: a pad for sample collection that is attached to a rapid test device and a vial of buffer (11).

The Chembio DPP test (Chembio Diagnostic Systems, USA) is a rapid test with a dual path platform for the detection of antibodies against HIV-1/2 in samples of oral mucosal transudate, whole blood, serum, or plasma. The test consists of an oral fluid collection device, a test device with two ports, a bottle cap for sample dilution, and a vial containing buffer solution (12).

Study settings and participants.

This cross-sectional study was conducted between May and September 2009 in two peri-urban health facilities in Maputo City, Mozambique: the Military Hospital and the 1st June Health Center. Both health facilities are enrolled in the National External Quality Assurance Program for HIV Serology (Qualimun-Ser) and have demonstrated high performances in all previous proficiency testing panels. These sites were selected because they have skilled personnel who can conduct tests in a reliable and reproducible manner, which is paramount in a validation study. Participants were recruited at the CTH units within the study sites. All consecutive clients fulfilling the eligibility criteria (18 years of age and the absence of any oral disease that might make the collection process uncomfortable) were enrolled in the study after signing specific informed consent forms and receiving a detailed description of the study procedures and pretest counseling. The study protocol was approved by the Mozambican National Health Bioethics Committee.

Sample collection and testing.

Each consenting participant was requested to provide oral samples and finger-prick blood samples for HIV testing. Each participant was tested using the two OF rapid tests and the national whole blood-based serial testing algorithm for HIV infection diagnosis. All rapid tests were performed onsite by designated trained counselors. The tests were conducted in the following order: (i) the first oral fluid sample test (OraQuick or DPP; see below), (ii) the Determine test using a finger-prick blood sample, (iii) if the Determine test is reactive, then the Uni-Gold test using a finger-prick blood sample, and (iv) finally the oral fluid sample test (DPP or OraQuick). Because there was no validated time interval between the two OF sample collections, the first OF assay was performed prior to the whole blood-based testing, which was then followed by the second OF test, to allow ample time (about 30 min) between the two OF sample collections. To avoid bias, the sequence of the OF tests was switched in the middle of the study. During the waiting time, a whole blood sample was collected by finger prick for HIV rapid testing according to the Mozambican national algorithm, which consists of serial testing using two immunochromatographic rapid tests. All samples were first screened by the Determine HIV-1/2 test (Abbott Laboratories, Japan), and all initially reactive samples were further tested using the Uni-Gold HIV-1/2 test (Trinity Biotech, Ireland) as a confirmatory test. All discordant samples were further tested using an enzyme immunoassay (EIA) (Vironostika HIV-1/2 Uni-Form II plus O; bioMérieux, Holland) and a Western blot (WB) assay (HIV Blot 2.2; Genelabs Diagnostics, USA). The WB results were interpreted according to the manufacturer's recommendations. For the purpose of quality control (QC), the whole blood samples collected from all participants were also used to prepare dried blood spot (DBS) specimens.

A questionnaire was offered to all participants to collect qualitative information on the acceptability of testing and their intention to use HIV rapid tests with different types of specimens and basic clinical and demographic data. The participants received their HIV results based on the whole blood-based testing per the Mozambican national algorithm and HIV CT guidelines.

Quality control.

Counselors from this study were trained to perform testing according to the instructions provided in the manufacturers' package inserts. The quality assurance measures for this study included the following. (i) For both the OraQuick and DPP tests, we purchased QC specimens provided by the respective manufacturers. The QC specimens are known HIV-positive and -negative specimens developed to verify the performance of a test and the testers prior to client testing. These QC samples were run daily at each study site. (ii) All testing was performed independently by counselors blinded to knowledge of the participant's HIV serostatus. (iii) Twenty percent of all DBS specimens were randomly selected and blind tested using EIA at the National Reference Laboratory.

Sample size and data analysis.

The sample size was calculated to be 1,600 participants, with 800 at each site. This sample size was determined by assuming a sensitivity and specificity of 99.7% for each OF test (11−13) and a prevalence of HIV infection of 30% at the CT sites in Maputo, with a power of 80% and a significance of 5% using a two-tailed test. The Pearson chi-square test was used to assess the interdependence between the categorical variables. A P value of ≤0.05 was considered statistically significant.

Data from the study were analyzed by Stata SE, version 9 (Stata Corporation, College Station, TX). A descriptive analysis was performed, and means, ranges, standard deviations, medians, and confidence intervals (CIs) were extracted for quantitative variables.

Sensitivity and specificity and their corresponding 95% CIs were calculated to evaluate the performance of the individual tests. The exact binomial confidence bounds were calculated. The Mozambique national algorithm for HIV infection diagnosis was considered the gold standard for this purpose. The qualitative questionnaire responses were used to calculate the preference for testing modalities and the intention of use or the “acceptability” of oral fluid testing. Data analysis was performed using the statistical package SPSS, version 17.0 (IBM, USA).

RESULTS

General characteristics of the study group.

A total of 1,690 individuals were initially enrolled in this study. The participants' median age was 29 years (range, 18 to 79 years). Fourteen participants were excluded due to incomplete data. Of the 1,676 samples tested, 59.3% (994/1,676) were from females. Likewise, the qualitative data collected from 1,620 study participants showed that 59.2% (959/1,620) were female.

Diagnosis with the national algorithm.

According to the gold standard for HIV infection diagnosis in Mozambique, 525 out of 1,676 (31.3%) samples were initially reactive using the Determine HIV-1/2 test. Of these, 512 were confirmed positive by the Uni-Gold HIV-1/2 test, yielding an overall HIV infection prevalence of 30.5% (95% CI, 28.6% to 33.1%). The remaining 13 samples (0.8%) showed discordant results and were further tested with EIA/WB. Of those, 5 (38.5%) were positive, while 8 (61.5%) were negative. The five WB reactive specimens had weak and partial banding patterns suggestive of seroconverting individuals (data not shown).

Diagnostic accuracy of OraQuick Advance Rapid HIV-1/2 test.

Of the 1,676 participants, 518 (30.9%) samples were reactive on the OraQuick test, while 1,158 (69.1%) were nonreactive. Two samples that were reactive on the OraQuick test were found to be nonreactive by WB and thus were classified as false positives. One sample that was nonreactive on the OraQuick test was found to be reactive by WB and thus was considered false negative (Table 1). Compared with the gold standard (the national algorithm confirmed by WB), the sensitivity of the OraQuick test was 99.8% (95% CI, 98.7% to 99.9%) and the specificity was 99.8% (95% CI, 99.3% to 99.9%).

TABLE 1.

Diagnostic accuracy of oral fluid and blood-based rapid tests compared to reference testing^a

Type of rapid test	Results for diagnostic accuracy of rapid tests
	True positive (no.)	False positive (no.)	True negative (no.)	False negative (no.)	Sensitivity (% [95% CI])	Specificity (% [95% CI])
Oral fluid test
OraQuick	516	2	1,157	1	99.8 (98.7–99.9)	99.8 (99.3–99.9)
Chembio DPP	517	2	1,157	0	100 (99–100)	99.8 (99.3–99.9)
Whole blood test
Determine	517	8	1,151	0	100	99.3
Uni-Gold	512	0	1,159	5	99.0	100

^an = 1,676.

Diagnostic accuracy of Chembio DPP HIV-1/2 test.

Of the 1,676 participants, 519 (31.0%) samples were reactive on the Chembio DPP HIV-1/2 test, while 1,157 (69.0%) were nonreactive. Two samples that were reactive on the Chembio DPP test were found to be nonreactive by WB and thus were classified as false positives. No false-negative result was found with the Chembio DPP test (Table 1). Compared with the national algorithm confirmed by Western blotting, the sensitivity of the Chembio DPP test was 100.0% (95% CI, 99.0% to 100.0%) and the specificity was 99.8% (95% CI, 99.3% to 99.9%).

Algorithm with two oral fluid tests.

The overall agreements between the OF tests (OraQuick and Chembio DPP) were 99.9% (1,675/1,676) and 99.8% (518/519) among the reactive results. These numbers were slightly higher than the overall agreement between the Determine and Uni-Gold tests (99.2%, [1,663/1,676]) and the positive agreement among the reactive results (97.5%, [512/525]). The accuracy of the serial OF-based testing algorithm was determined and compared to that of the national algorithm (Fig. 1 and Table 2). Both algorithms gave similar results for all participants' specimens tested except for one sample. This specimen would have been reported as indeterminate if the DPP test was used as the first test (DPP test reactive/OraQuick test nonreactive) but would have been considered negative if the OraQuick test was used at the first test.

FIG 1.

Comparison of HIV test results using the national algorithm and a serial oral fluid testing algorithm in Mozambique (n = 1,676). DPP, the Chembio DPP test; R, reactive; NR, nonreactive; POS, final result reported as positive; IND, final result reported as indeterminate; NEG, negative; EIA/WB, enzyme immunoassay/Western blot.

TABLE 2.

A 2 × 2 table showing comparison of the oral fluid-based testing algorithm with the national testing algorithm using blood-based rapid testing

Results for OF RT algorithma	Results for National testing algorithm
	Positive	Negative	Total
Positive	512	4	516
Negative	1	1,159	1,160
Total	513	1,163	1,676

^aOF RT, oral fluid rapid tests (Chembio DPP and OraQuick). Kappa value, 0.993 (95% CI, 0.987 to 0.999); agreement, 99.7%.

Intention of use or acceptability of oral fluid tests.

Taking into account the fact that OF tests are not currently in use in Mozambique, the investigators defined the term “acceptability” as the intention of use of an OF test device. The whole blood test was marginally more acceptable among the respondents with an intention of use of 49.9% (808/1,620) compared with 46.8% (758/1,620) for OF tests; the intention of use among the remaining 3.3% of participants was the venous blood test or other methods. Significant gender differences were noted, with 52.5% of women compared with 42.5% of men stating that OF testing was their preferred choice for future testing.

Preference regarding testing modalities.

Reasons provided for the OF testing preference included less pain and greater ease of use. Overall, the study population reported that whole blood testing was more reliable and prioritized the aspect of reliability as a determining factor when discussing testing modality preference.

DISCUSSION

This study is among the first studies describing the diagnostic accuracy of the Chembio DPP test and also represents the first field assessment of the diagnostic accuracy of oral fluid-based HIV rapid tests in Mozambique.

The data demonstrated that the performances of both tests are highly similar to those of the existing whole blood-based HIV rapid test algorithm currently used in Mozambique. Although both OF tests performed similarly regarding their specificity (99.8% for both), the Chembio DPP test displayed a slightly higher sensitivity (100%) than the OraQuick Advance Rapid HIV-1/2 test (99.8%).

The sensitivity and specificity of the OraQuick Advance Rapid HIV-1/2 test found in this study were similar to the manufacturer's data reported to the U.S. FDA (sensitivity, 99.3%; 95% CI, 98.4% to 99.7%; specificity, 99.8%; 95% CI, 99.6% to 99.9%) (11) and to those in other validation studies conducted in Zimbabwe (sensitivity, 100%; 95% CI, 98.0% to 100.0%; specificity, 99.7%; 95% CI, 98.4%- to 99.9%) (5) and in Botswana (sensitivity, 98.4%; 95% CI, 96.5 to 99.4; specificity, 98.3%; 95% CI, 91.9 to 99.9) (9).

The Chembio DPP test sensitivity (100%) was very high and consistent with the manufacturer's claim (12), The specificity was slightly lower than the sensitivity; however, it was similar to the manufacturer's information (99.8%; 95% CI, 99.6% to 99.9%). Beyond the manufacturer's evaluation, this is the first assessment using the Chembio DPP HIV-1/2 test for OF samples after that of Iregbu et al. (10).

Some operational characteristics such as the test kit packaging (all necessary testing components packaged in individual pouches) might help address some procurement problems and reduce the issues around shortages and stocking problems with buffer and capillary tubes, which may be procured separately for some HIV rapid test kits (7).

While these OF tests display high performance characteristics and operational benefits, the indications that host factors such as tobacco use, dentition, bleeding gums, nasal medication, and time since the last meal may influence the immunoglobulin (IgG) concentration in oral fluids affect the test performance and therefore might cause a setback in their full-scale implementation (14, 15). We switched the order of the tests half-way through the study to reduce a preferential bias of the first test over the second test. This switch may have introduced a systematic bias from the learning/Hawthorne effect, but we did not see this being reflected in the data. A true crossover design would randomize the order, either by patient, by day, or by week. However, this randomization would have been logistically challenging for this field evaluation and may have introduced other test-specific errors.

In this study, we found that 49.9% of respondents had a preference for the whole blood testing compared to 46.8% of them who preferred the OF testing. The primary reason provided by our study's participants for choosing OF-based testing was the lack of the pain associated with a finger prick. Similarly, a pilot study conducted in Kenya to assess the acceptability of the OraQuick test in five Voluntary Counseling and Testing (VCT) clinics reported that 271 out of 392 clients (69%) expressed a preference for an oral test when tested with both oral and finger-prick tests (16). The recommendations from this pilot study in Kenya indicated that there was a need to promote messages which emphasized the low risk of transmission of the virus through saliva even though HIV antibodies are detectable in the saliva. Moreover, because adequate outreach testing at the community level, particularly for hard-to-reach populations, is limited in developing countries (17), the use of OF tests might be an alternative strategy for decreasing the stigma associated with testing and increasing the testing uptake in these populations.(18, 19).

Although the observed HIV infection prevalence among the study population was high, the data generated should not be used as a proxy for the HIV infection prevalence in the general population in Mozambique. The prevalence rate of HIV infection among the study population was higher than both the national HIV infection prevalence in Mozambique and the regional HIV infection prevalence as related to the geographic location of study recruitment (1). This might be attributed to the fact that participants were selected from within CT services where the likelihood of HIV-positive clients is higher than in other health facility services and among the general population. In particular, as the implementation of provider-initiated CT is in the early stages of rollout in Mozambique, many clients seeking CT in the client-initiated settings where study recruitment took place were functionally referral cases from other clinical settings; these clients were seeking health care for other reasons, potentially related to the signs and symptoms of HIV infection. It is important to note that the high prevalence does not otherwise compromise the utility of the current study in demonstrating the feasibility and reliability of OF testing in Mozambique.

However, as HIV counseling and testing services are being scaled up to meet the World AIDS Day goals, it will be critical to assess the cost and the programmatic impact of introducing OF testing in the national health system. If demonstrated to be cost-effective, OF testing could an alternative approach for developing countries such as Mozambique to increase the number of individuals tested who know their HIV infection status and who receive early and adequate care and treatment (20).

In conclusion, the findings from this study show evidence that both OF tests, the Chembio DPP and the OraQuick Advance, have excellent diagnostic accuracy in field settings as individual tests in comparison with a serial algorithm comprising the Determine and Uni-Gold tests. OF testing could be an alternative as an HIV screening test because it is a less-invasive procedure. We recommend that additional feasibility and cost-effectiveness assessments be conducted in more remote and limited resource areas as a next step in considering rollout of OF testing in low-resource countries such as Mozambique.