Analysis of False-Negative Human Immunodeficiency Virus Rapid Tests Performed on Oral Fluid in 3 International Clinical Research Studies

Abstract

Background

The OraQuick Advance Rapid HIV-1/2 Test is a point-of-care test capable of detecting human immunodeficiency virus (HIV)-specific antibodies in blood and oral fluid. To understand test performance and factors contributing to false-negative results in longitudinal studies, we examined results of participants enrolled in the Botswana TDF/FTC Oral HIV Prophylaxis Trial, the Bangkok Tenofovir Study, and the Bangkok MSM Cohort Study, 3 separate clinical studies of high-risk, HIV-negative persons conducted in Botswana and Thailand.

Methods

In a retrospective observational analysis, we compared oral fluid OraQuick (OFOQ) results among participants becoming HIV infected to results obtained retrospectively using enzyme immunoassay and nucleic acid amplification tests on stored specimens. We categorized negative OFOQ results as true-negative or false-negative relative to nucleic acid amplification test and/ or enzyme immunoassay, and determined the delay in OFOQ conversion relative to the estimated time of infection. We used log-binomial regression and generalized estimating equations to examine the association between false-negative results and participant, clinical, and testing-site factors.

Results

Two-hundred thirty-three false-negative OFOQ results occurred in 80 of 287 seroconverting individuals. Estimated OFOQ conversion delay ranged from 14.5 to 547.5 (median, 98.5) days. Delayed OFOQ conversion was associated with clinical site and test operator (P < .05), preexposure prophylaxis (P = .01), low plasma viral load (P < .02), and time to kit expiration (P < .01). Participant age, sex, and HIV subtype were not associated with false-negative results. Long OFOQ conversion delay time was associated with antiretroviral exposure and low plasma viral load.

Conclusions

Failure of OFOQ to detect HIV-1 infection was frequent and multifactorial in origin. In longitudinal trials, negative oral fluid results should be confirmed via testing of blood samples.

The recognition that immunoglobulin G (IgG) is present in oral fluid at levels sufficient to allow serological diagnosis of human immunodeficiency virus (HIV) infection has led to the development of rapid HIV tests based on oral fluid [1]. In June 2004, the OraQuick Rapid HIV-½ Antibody Test was approved by the US Food and Drug Administration (FDA) for use in oral fluid (OFOQ), and in July 2012 the test became the first over-the-counter rapid HIV test approved by the FDA for home use. The availability of a noninvasive, rapid HIV test that can be performed by nonprofessionals has greatly increased the accessibility of HIV testing. Among men who have sex with men (MSM), home test kits are acceptable, lead to increased testing, and reduce high-risk sexual encounters [2, 3].

Rapid HIV tests may not perform as well as laboratory-based tests, and the advantages of oral fluid-based rapid testing must be weighed against possibly lower test accuracy. While the package inserts for OraQuick Advance and OraQuick HIV-½ (for use outside the United States) cite sensitivities of 99.3% and 100% and specificities of 99.8% and 99.9%, respectively, in oral fluid [4, 5], test performance may be affected by HIV prevalence, stage of illness, use of antiretroviral agents (ARVs), test operator error, and other factors [6–13]. In HIV prevention trials, preexposure prophylaxis (PrEP) and the potentially high proportion of early infections encountered during intensive sampling may result in failure to detect infection when relying on rapid oral fluid tests. The impact of a false-negative (FN) result is accentuated in this setting, as test recipients might receive PrEP regimens during established infection, or engage in unprotected sex under the false assurance of a negative result when HIV is most likely to be transmitted.

Between 2003 and 2006, the US Centers for Disease Control and Prevention (CDC) undertook 3 cohort studies in MSM and heterosexual persons at risk for acquiring HIV infection in Thailand and Botswana [14–16]. In these studies, OFOQ was selected for routine monthly HIV testing because of its ease of use, reported high sensitivity and specificity, and rapid turnaround time (Supplementary Figure 1). To further define the performance of the OFOQ test, we retrospectively examined OFOQ test result accuracy with reference to highly sensitive blood tests in all HIV seroconverting participants in these 3 studies.

METHODS

Ethical Review

The contributing studies were conducted with signed informed consent by participants and approval by the Botswana Health Research and Development Committee, the Ethical Review Committee of the Thai Ministry of Public Health, and the CDC Institutional Review Board.

Contributing Studies and Participant Characteristics

The Botswana TDF/FTC Oral HIV Prophylaxis Trial (TDF2) was a phase 3, randomized, double-blinded, placebo-controlled clinical trial of oral tenofovir disoproxil fumarate-emtricitabine (TDF-FTC) PrEP for reducing HIV incidence, performed on 1219 sexually active heterosexual adults in Francistown and Gaborone, Botswana [14]. The Bangkok Tenofovir Study (BTS) was a phase 3, randomized, double-blinded, placebo-controlled clinical trial of daily oral TDF HIV PrEP among 2413 adults who injected drugs in Bangkok, Thailand [15]. The Bangkok MSM Cohort Study (BMCS) investigated the epidemiology of HIV and prevention methods among 1377 Thai MSM in Bangkok [16]. HIV subtype was determined by consensus sequencing in pol (TDF2), or as described elsewhere [15, 17]. In the TDF2 study, all infections were due to HIV type 1 (HIV-1) subtype C, and subtype comparisons were not possible. In BTS and BMCS, infections were due to CRF01-AE, HIV-1 subtype B, and interstrain recombinants, and participants were categorized as infected with HIV-1 subtype B or a non-subtype B strain. PrEP was not given in BMCS. However, 5 BMCS seroconverters received antiretroviral therapy (ART) immediately after diagnosis [18].

OraQuick Testing

In all contributing studies, all test operators were trained in the use of the OFOQ test by a proficient laboratory staff member. Test operator competency was assessed at intervals using blinded model proficiency evaluation program and College of American Pathologists (CAP) proficiency panels containing known positive and negative samples (TDF2), a panel of blinded negative and positive samples provided by the Thai National Institute of Health (BTS), or CAP proficiency panels containing known positive and negative samples (BMCS). Because test strip interpretation relies on subjective interpretation of chromatographic bands, measures were taken to assure that testing was performed under lighting conditions assessed to be adequate prior to study initiation.

Laboratory Methods

We performed a look-back analysis on stored blood samples in all HIV-infected participants. For each seroconverting individual, samples collected and stored at −70°C at the time of, and prior to, the first reactive OFOQ test were tested in reverse chronological order until the first negative blood sample was obtained. Testing was performed by both enzyme immunoassay (EIA) and nucleic acid amplification test (NAAT) (Roche Cobas Amplicor HIV-1 Monitor Test, version 1.5 [TDF2] or Roche Cobas TaqMan version 1.0 [BTS and BMCS]). A reactive result on either test was considered sufficient to establish HIV infection.

Data Collection

OFOQ testing history was retrieved from all persons with 1 or more FN OFOQ results. Study records were queried for test date, test result, test kit lot number, test location (eg, clinic), test operator, operator workload on the day of testing, operator age and proficiency, plasma viral load (pVL), HIV subtype, participant ARV use (PrEP or ART), and participant demographic information.

Statistical Analysis

Nonreactive OFOQ tests were categorized with reference to a gold standard of EIA and/or NAAT results, or considered unconfirmed if no additional confirmatory testing was available. The primary outcome variable was the proportion of FN results among all negative OFOQ responses in newly infected individuals after the estimated time of infection. The relationships between primary outcomes and potential predictors (eg, participant age, sex, time to test kit expiration, test operator, operator workload, clinic site, ARV exposure, and HIV subtype) were summarized using prevalence ratios (PRs) and robust 95% confidence intervals (CIs) using generalized estimating equations [19–22]. In secondary analyses we considered (1) what proportion of negative OFOQ tests were obtained after the appearance of HIV-specific (or gp41-specific) antibodies in blood; (2) whether there were any significant differences in ARV exposure (treatment arm assignment), HIV pVL, and subtype distribution between participants with prolonged (ie, >180 days) and those without a prolonged history of FN results, and; (3) whether there were any interactions between participant, test center, test operator, and test kit lot in those with exceptionally frequent FN results (See Supplementary Materials).

RESULTS

Seroconverting Individuals

The TDF2, BTS, and BMCS studies provided 34, 53, and 200 newly infected individuals to this analysis, respectively (N = 287). In this group, geometric mean pVL at the time closest to seroconversion (median, 66.5 days [range, 14.5–934.5 days]) was 47 546 copies/mL (95% CI, 35 553–63 584 copies/mL). Twenty-four of 87 seroconverting individuals in TDF2 and BTS received PrEP consisting of either TDF or TDF-FTC. Five seroconverting individuals from the BMCS received combination ART immediately after recognition of infection.

FN Tests Across all Studies

Using the midpoint estimate, 233 FN results occurred in 80 seroconverting individuals (Supplementary Table 1). FN results were recorded by 39 of 63 test operators at 15 of 17 clinics. The median OFOQ conversion delay time in these tests was 98.5 days (range, 14.5–547.5 days). Eighty-one of 147 OFOQ test kit lots used in seroconverting individuals had 1 or more FN test results.

TDF2

Thirty-five FN tests occurred in 12 participants (Supplementary Table 1). The median estimated OFOQ seroconversion delay time (Figure 1) was 82 days (range, 20–307.5 days). FN results were significantly associated with test location (PR, 24.4 [95% CI, 4.1–145.3], P < .01), lack of operator proficiency (PR, 3.9 [95% CI, 1.6–9.6], P < .01), operator age <35 years (PR, 3.6 [95% CI, 1.3–9.9], P < .01), and use of test kits ≥100 days prior to kit expiration date (PR, 3.5 [95% CI, 1.7–7.2], P < .01). Participant age, sex, treatment arm, and operator workload were not significant factors (Table 1). All but 1 FN result occurred at 1 of 2 clinics. Thirteen operators performed testing: 4 operators accounted for 33 of 35 FN results. Of the 24 lots used for testing, 10 had FN results, 1 of which provided 10 of the 35 FN results.

Figure 1.

Distribution of estimated delay times between human immunodeficiency virus type 1 infection and oral fluid OraQuick (OFOQ) test positivity. A and B, Delay times for the Botswana TDF/FTC Oral HIV Prophylaxis Trial (TDF2) and Bangkok Tenofovir Study (BTS) participants, respectively. Participants received either tenofovir disoproxil fumarate (TDF) or TDF/emtricitabine (FTC)-based preexposure prophylaxis (black columns) or placebo (gray columns). C, Delay time probability distribution for TDF2 and BTS. Estimated delay time calculated using the midpoint method. Data are only shown for participants with delay >0 days. The x-axis shows delay time (days); the y-axis shows counts of individuals with false-negative tests by estimated delay time (A and B) and probability density (C).

Table 1.

Factors Associated With False-Negative Tests (Primary Analyses) and With Oral Fluid OraQuick Conversion Delay Times >180 Days (Secondary Analyses), in the Botswana TDF/FTC Oral HIV Prophylaxis Trial, the Bangkok Tenofovir Study, and the Bangkok MSM Cohort Study, According To Univariable Modeling Results Based on a Log-Binomial Regression Model

	Primary Analysis Factors
Factor	TDF2	BTS	BMCS
Participant age <25 y	1.45 (.4–5.26), P = .56	0.79 (.27–2.3), P = .67	1.1 (.58–2.1), P = .77
Participant sex	0.72 (.19–2.66), P = .62	0.42 (.13–1.31), P = .13	NA
High operator workload	1.9 (.86–4.17), P = .11	0.96 (.5–1.81), P = .88	0.83 (.41–1.7), P = .61
Operator age <35 y	3.64 (1.33–9.94), P < .01	0.89 (.44–1.78), P = .74	0.37 (.18-.75), P < .01
Operator proficiency	3.94 (1.61–9.63), P < .01	0.8 (.42–1.52), P = .48	NA
Time to kit expiration	3.46 (1.66–723), P < .01	2.41 (1.27–4.56), P < .01	4.3 (2.09–8.85), P < .01
Randomization to PrEP	1.56 (.38–6.41), P = .53	2.73 (1.22–6.09), P = .01	NA
HIV pVL (participants w/ vs w/o FN)	0.53	0.23	0.86
HIV subtype	NA	1.15 (.32–4.11), P = .83	0.95 (.42–2.14), P = .89
Operator	Yes	Yes	Yes
Location	Yes	Yes	NA
Secondary analysis (factors associated with delay >180 d)
Randomization to PrEP	P = 1.0	P = .26	NA
Low HIV pVL	P = .96	P < .016	P = .93
HIV subtype	NA	P = .55	P = 1.0

Quantitative measures are indicated with prevalence ratios (95% confidence interval) and/or P values. Dichotomous qualitative measures associated and not associated with FN tests are shown as “yes” and “no,” respectively. Significant P-values are shown in bold.

Abbreviations: BMCS, Bangkok MSM Cohort Study; BTS, Bangkok Tenofovir Study; FN, false-negative; FTC, emtricitabine; HIV, human immunodeficiency virus; MSM, men who have sex with men; NA, not applicable; OFOQ, oral fluid OraQuick test; PrEP preexposure prophylaxis; pVL, plasma viral load; TDF, tenofovir disoproxil fumarate; TDF2, Botswana TDF/FTC Oral HIV Prophylaxis Trial.

Bangkok Tenofovir Study

One hundred forty-five FN tests occurred in 32 participants (Supplementary Table 1). Median estimated OFOQ seroconversion delay time (Figure 1) was 126.5 days (range, 14.5–547.5 days). FN results were significantly associated with randomization to TDF prophylaxis (PR, 2.7 [95% CI, 1.2–6.1], P = .01) and use of test kits ≥100 days prior to kit expiration date (PR, 2.4 [95% CI, 1.3–4.6], P < .01), whereas participant age, sex, and HIV subtype were not significant correlates (Table 1). Among 38 operators performing testing, 23 obtained false-negatives (mean, 5.3 FN tests); however, 4 operators had a substantial number of FN tests (mean, 14.5 tests). Workload was not a significant factor in this study. Operator proficiency was assessed during the first 2.5 years of the study and was not a significant factor for these operators. Among 15 clinics, 13 contributed FN results; 4 clinics had elevated FN rates (mean, 21.5 FN tests/site). FN results were widely distributed across test kit lots, without obvious clustering.

Bangkok MSM Cohort Study

Fifty-three FN tests occurred in 36 participants (Supplementary Table 1). FN results were negatively associated with operator age <35 years (PR, 0.4 [95% CI, .2—.8], P < .01) and use of tests ≥100 days prior to test kit expiration (PR, 4.3 [95% CI, 2.1–8.9], P < .01). Participant age, HIV-1 subtype, and operator workload were not significant correlates (Table 1). Of 13 operators who performed OFOQ tests, 10 provided FN results and 2 had relatively high FN rates (12/24 [50%] and 6/6 [100%]). No test lots accounted for a disproportionate number of FN cases.

FN OFOQ Results Relative to Appearance of HIV-Specific Antibodies in Blood

Among 233 FN tests, 208 occurred after or on the same day as a positive EIA, while 11 FN confirmed by NAAT came before the last negative EIA and 14 came between last negative and first positive EIAs. Western blot data were available for 122 BTS participants; of these, 97 had a FN result (80%) after the appearance of a gp41 band, and 116 (95%) after the appearance of either a gp41 or a gp160 band.

Prolonged OFOQ False-Negativity and Randomization to PrEP, HIV pVL, and Subtype Distribution

Randomization to PrEP was positively associated with FN OFOQ tests only in the BTS (P = .01). The pVL was significantly lower among participants with OFOQ conversion delay times >180 days in BTS (2557 vs 50 555 copies/mL, P = .016), but not in the BMCS or TDF2 studies (Table 1). However, there was no significant association between OFOQ delay time >180 days and either ARV exposure or infecting HIV subtype in any study.

Clinic and Operator

We considered the relationship between clinics with higher-than-average FN results and the distribution of FN results among operators at these clinics. BMCS data were not included here because all tests were performed at a single clinic. In the TDF2 study, 33 of 34 FN results occurred at 1 clinic, where all 5 operators had higher than average rates of FN tests; in the BTS, 5 clinics had excess FN results. At one BTS clinic, 1 operator accounted for 20 of 25 (80%) FN results, suggesting that operator error may have been a significant contributing factor. At the remaining 4 BTS clinics, FN results were widely distributed across multiple operators.

Operators and Test Kit Lots

Across all 3 studies, 31 operators exceeded the mean operator FN rate, by study (mean, 5.9 FN tests; range, 1–20; standard deviation [SD], 4.7). Testing in these cases involved 182 FN tests and 66 lots (mean, 2.8 FN tests; range, 1–10; SD, 2.0). One outlying operator had 20 FN tests, but distributed across 15 lots. One outlying lot had 10 FN results, but distributed across 4 operators. The remaining FN test results were broadly distributed across kit lots and operators.

Operators and Participants

Finally, we considered the relationship between operators with frequent FN results and individual participants. Among the 31 operators with high FN rates, 58 participants contributed 182 FN tests (mean, 5.9 FN tests; range, 1–20; SD, 4.7). In this group, 3 outlying operators had a large number of FN tests (15/182, 13/182 and 20/182, tester FN percentiles 26.3, 48.1, and 24.1, respectively). In the first 2 operators, FN results occurred primarily in a single participant; the first operator obtained 14 of 15 FN results in 1 participant, 40% of results in this participant were FN, and all were performed by the same operator. The second operator obtained 13 of 13 FN results in a single participant. This participant had 3 additional tests performed by another operator, and all were also FN. In the remaining studies and operators, FN results were distributed across many participants.

DISCUSSION

We have examined the performance of HIV rapid testing in oral fluid in the context of 2 randomized placebo-controlled PrEP studies and 1 cohort study, involving study participants exposed to HIV through heterosexual contact, homosexual contact, and injection drug use. Eighty of 287 HIV-infected participants had 1 or more FN OFOQ results. Factors significantly associated with FN results in 1 or more studies included test operator, operator proficiency, test location, use of test kits 100 days or more prior to kit expiration date, and randomization to PrEP, while low pVL was associated with prolonged OFOQ conversion delay in 1 study. Operator age was a significant factor in TDF2 and BMCS, but had opposite effects in these 2 studies. Participant age and sex, operator workload, and HIV subtype were not associated with FN results (Table 1). Most FN results occurred after the appearance of HIV-specific antibody responses in blood.

The OFOQ package insert reports a test sensitivity of 99.6% [4]. Most studies cite OFOQ sensitivity values ranging from 97.8% to 100% [9, 11, 12, 23, 24], although a number of studies have reported lower sensitivity in some settings. Stekler et al noted an OFOQ sensitivity of 80% among 2479 high-risk men and transgender women in Seattle [7], and Pilcher et al observed a sensitivity of 86.6% among 127 participants in San Francisco [10]. In these 2 studies, failure to detect very early infection and a relative insensitivity of testing in oral fluid compared with equivalent testing in blood were important factors underlying FN OFOQ results. These findings are consistent with studies showing lower antibody titers and/or delayed appearance of antibodies in oral fluid in comparison with blood, and reduced sensitivity of oral fluid-based HIV tests compared with rapid tests in blood [25–28].

Because we examined incident HIV infections, early infection was a prevailing factor for at least some of the FN results obtained in each individual. However one striking observation in our report is the very prolonged OFOQ conversion delay time occurring in some individuals. In addition, a larger number of study participants had more modest delays exceeding 3 months, and factors other than testing during acute infection must be responsible for detection failures in these cases. One possible contributing factor is exposure to ART. Pavie et al noted an OFOQ sensitivity of only 86.5% among 200 study participants with HIV infection confirmed by Western blot [6]. Most were receiving ART (68.5%) and had plasma HIV-1 RNA <200 copies/mL (57.8%). Other studies also show that FN results are more likely during late-stage infection or during long-term suppressive therapy, where HIV-specific antibody titers may wane [8, 29]. These results are consistent with the significant association between FN results and randomization to PrEP, and the association between prolonged FN status and low pVL observed in the BTS.

In our analysis, certain operators seemed to be more likely than others to obtain a FN result. The OFOQ test requires subjective visual interpretation of chromatographic bands, which may be faint, and this may be challenging under some circumstances. These findings agree with other studies demonstrating the potential for operator error in the use of the OFOQ test [30, 31]. The direct association between time to kit expiration and likelihood of an FN OFOQ result was unexpected and counterintuitive, as it implies better test performance closer to the time of kit expiration. To our knowledge, this is the first report of decreased likelihood of a false-negative OFOQ result as time of testing approaches the kit expiration date. However, Facente et al reported a striking trend toward greater likelihood of a false-positive test result the closer the test was performed to the kit expiration date [32]. These 2 observations remain unexplained, but suggest that a positive test is more likely at times closer to kit expiration, possibly due to unrecognized test properties.

This analysis has several limitations. Because oral fluid and confirmatory testing was performed at intervals, the HIV infection dates were estimated based on testing dates. In participants with periods of loss to follow-up, the methods used to estimate time of infection may underestimate the true OFOQ delay time. Blood sampling was infrequent in the BMCS, resulting in a substantial uncertainty in the estimated OraQuick conversion delay time. In addition, routine EIA testing was implemented in the BMCS after February 2010. Although OraQuick tests were also performed at study visits, reactive EIA results prompted evaluation for HIV infection and obviated further OFOQ testing, thus placing an upper limit on the possible OraQuick delay time. Plasma viral load was measured infrequently in all studies, possibly obscuring the relationship between pVL and FN results. The OraQuick test requires subjective visual interpretation, and photographic records of test results were not routinely kept. It is therefore difficult to distinguish between the failure of operators to recognize faint bands and true test kit failure.

In this analysis, failure of the OFOQ to detect HIV infection was multifactorial in origin (Figure 2). The OFOQ test is accepted as a sensitive and specific test for the diagnosis of HIV infection in a variety of clinical settings, and has been approved for home use by nontechnical users. However, caution must be exercised when interpreting a negative OFOQ test in settings where acute infection is likely, and where PrEP use, ART-induced viral suppression, or profound immunosuppression may result in low HIV-specific antibody titers. In the setting of clinical trials, investigators should consider diagnostic algorithms relying on multiple tests for detection of infection.

Figure 2.

Frequency distribution of potentially influential factors associated with false-negative (FN) oral fluid OraQuick (OFOQ) tests. Barplot showing count (y-axis) of OFOQ tests performed among 287 seroconverters, according to the number of contributing factors present (x-axis), from among 8 potentially influential factors associated with OFOQ tests. The proportional contribution of each factor to total bar height (test count) in each test category (false negative [FN], true negative [TN], true positive [TP]) is shown as colored bar segments. Factors shown include time to kit expiration, test operator with excess FN results, clinic site with excess FN results, antiretroviral (ARV) exposure, OFOQ test done before appearance of Western blot gp41 band, OFOQ test done before enzyme immunoassay (EIA) conversion, OFOQ test performed within 90 days of estimated time of infection, plasma viral load (pVL) <30 000 copies/mL. Tests with no factors are shown with gray bars. Data are shown for each OFOQ test performed, and individual study participants generally contributed >1 test. Missing data were assumed to be negative.