The fourth generation Alere™ HIV Combo rapid test improves detection of acute infection in MTN-003 (VOICE) samples

Abstract

Background

Early and accurate detection of HIV is crucial when using pre-exposure prophylaxis (PrEP) for HIV prevention to avoid PrEP initiation in acutely infected individuals and to minimize the risk of drug resistance in individuals with breakthrough infection.

Objective

To determine if fourth-generation antigen/antibody (Ag/Ab) rapid diagnostic tests (RDT) would have detected HIV infection earlier than the third-generation RDT used in MTN-003 (VOICE).

Study design

5029 VOICE participants were evaluated with third-generation Alere Determine™ HIV-1/2, OraQuick ADVANCE^® Rapid HIV-1/2, Uni-Gold™ Recombigen^® HIV-1/2 and Bio-Rad GS HIV-1/2 + O EIA; and fourth-generation Alere Determine™ HIV-1/2 Ag/Ab Combo, Conformité Européene (CE)-Marked Alere™ HIV Combo and Bio-Rad HIV Combo Ag/Ab EIA. Multispot^®, GS HIV-1 Western Blot (WB) and Geenius™ (Bio-Rad) were also evaluated.

Results

Of 57 antibody-negative pre-seroconversion plasma samples with HIV RNA > 20 copies/mL identified, 16 (28%) were reactive by CE-Marked Alere™ HIV Combo (1 Ab; 9 Ag; 6 Ag/Ab reactive) and 4 (7%) by Alere Determine™ HIV-1/2 Ag/Ab Combo (2 Ab; 2 Ag; 0 Ag/Ab reactive) (p =0.0005). Multispot^® confirmed only 1 of 16 acute infections while WB and Geenius™ confirmed none. GS HIV Combo Ag/Ab EIA identified 27 of 57 (47%) pre-seroconversion RNA-positive samples.

Conclusion

In VOICE, 28% of infections missed by current third-generation RDT would have been identified with the use of CE-Marked Alere™ HIV Combo. Geenius™, Multispot^® and WB were all insensitive (< 10%) in confirming infections detected by fourth-generation assays. An improved diagnostic algorithm that includes a fourth-generation RDT with HIV RNA testing will be essential for efficiently identifying seroconverters on PrEP.

1. Background

Early and accurate detection of HIV is crucial when using pre-exposure prophylaxis (PrEP) for HIV prevention to avoid PrEP initiation in acutely infected individuals and to minimize the risk of drug resistance in individuals with breakthrough infection [1]. The World Health Organization recommends two sequential rapid tests for HIV diagnosis [2] however third generation rapid tests detect only HIV antibodies and may miss up to 75% of early acute HIV infection cases [3].

Fourth generation HIV rapid tests use a lateral flow cassette to separately assay for both anti-HIV antibodies and p24 antigen, however numerous studies have demonstrated the United States Food and Drug Administration (FDA)-approved Alere Determine™ HIV-1/2 Ag/Ab Combo to be insensitive for detection of acute infection. In a total of 54 acute seroconversion samples from 3 studies of acute infection from Malawi, Swaziland, Rwanda and Zambia, only 1 sample (1.9%) was positive for the p24 antigen component [4–6]. Reported sensitivity for antigen detection was higher in samples from the United Kingdom (50%), the United States (45.5% and 75.8%), Italy (88.2%) and in mixed-subtype seroconversion panels (86.6%) suggesting clade differences [3,7–10]. Evaluation of Gag-expressing virus-like particles (VLP) showed that 6 of 7 subtype B VLPs were detected by Alere Determine™ HIV-1/2 Ag/Ab Combo, compared to only 1 of 5 subtype C, and 3 of 31 non B/non-C-subtypes [11].

In February 2015, Alere released a re-formulated fourth generation rapid test kit, the Conformité Européene (CE)-Marked Alere™ HIV Combo. Only one study to date has evaluated this new assay, reporting an 88% sensitivity compared to the Abbott Architect HIV Ag/Ab Combo assay using stored plasma or serum samples from the United Kingdom [12]. Also unknown is the performance of HIV confirmatory tests such as Bio-Rad Multispot^® HIV-1/HIV-2 Rapid Test, Geenius™ HIV-1/2 Supplemental Assay and GS HIV-1 Western Blot in the context of an algorithm that includes fourth generation rapids. This is the first evaluation of the new CE-Marked Alere™ HIV Combo in pre-seroconversion samples collected in South Africa, Uganda and Zimbabwe.

2. Objectives

The objective of this study was to evaluate the performance of third and fourth generation HIV rapid diagnostic tests and confirmatory assays to detect acute infection in a large cohort of participants from the MTN-003 (VOICE) HIV prevention trial.

3. Study design

3.1. Study population and specimen collection

VOICE (ClinicalTrials.gov NCT00705679) was a randomized, Phase 2 B placebo-controlled trial to evaluate the safety and effectiveness of oral tenofovir disoproxil fumarate (TDF), oral TDF-emtricitabine (FTC) and vaginal tenofovir 1% gel for the prevention of HIV infection in 5029 HIV-uninfected women from 15 clinical sites in South Africa, Uganda and Zimbabwe. Population characteristics and trial results have been described previously [13].

In VOICE, participants were monitored monthly for seroconversion with one (Uganda) or two (South Africa and Zimbabwe) third generation HIV rapid diagnostic tests (RDT) performed point-of-care using venous or fingerstick-drawn whole blood at clinical research sites. RDTs used included Alere Determine™ HIV-1/2 (Alere Medical Co. Ltd., Matsudo, Japan), OraQuick ADVANCE^® Rapid HIV-1/2 Antibody Test (OraSure Technologies, Inc., Bethlehem, PA) and/or Uni-Gold™ Recombigen^® HIV-1/2 rapid test (Trinity Biotech™ Wicklow, Ireland). To diagnose HIV infection, clinical research sites followed an HIV algorithm during the study (Fig. 1). Positive or discordant RDT results were confirmed by GS HIV-1 Western Blot (Bio-Rad Laboratories, Redmond, WA) using plasma from a separate draw and HIV-1 RNA PCR using Abbott RealTime HIV-1 (Abbott Molecular, Inc., Des Plaines, IL) was performed on plasma following an indeterminate or negative Western blot result to assess acute infection (Fig. 1). Assay interpretation was defined by the manufacturer’s package insert for each test. In the VOICE study, plasma from the visit at which seroconversion was detected and stored plasma from the seroconverting participant’s enrollment visit were shipped to the University of Pittsburgh and re-tested to verify HIV infection using GS HIV-1/2 + O EIA (Bio-Rad), GS HIV-1 Western Blot and HIV-1 RNA PCR (Abbott).

Fig. 1.

Endpoint Algorithm for Diagnosis of HIV-1 Infection used at Clinical Sites during the VOICE Study. Participants were monitored monthly for seroconversion with one (Uganda) or two (South Africa and Zimbabwe) third generation HIV rapid diagnostic tests including Alere Determine™ HIV-1/2, OraQuick ADVANCE^® Rapid HIV-1/2 Antibody Test and/or Uni-Gold™ Recombigen^® HIV-1/2. Positive or discordant results were confirmed by GS HIV-1 Western Blot (Western Blot 1) using plasma from a separate draw. HIV-1 RNA PCR (HIV viral load) was performed on plasma following an indeterminate or negative Western blot result to assess acute infection. Participants confirmed as HIV-uninfected resumed study product and continued in VOICE while participants whose Western Blot tested positive returned to the study site to have a second Western blot performed on plasma from a separate draw. A second positive Western blot confirmed HIV infection and the participant was exited from the study. Discordant results underwent investigation and further testing by the Study Laboratory Management Group.

3.2. Identification of specimens from acutely infected VOICE participants

After completion of the VOICE study, stored plasma specimens collected up to 91 days prior to detection of seroconversion were shipped to the University of Pittsburgh. Pre-seroconversion plasma specimens were tested for the presence of HIV-1 RNA by the COBAS^® AmpliPrep/COBAS^® TaqMan^® HIV-1 Test, v2.0 with a limit of quantitation of 20 copies/mL (Roche Diagnostics, Indianapolis, IN). Specimens with detectable HIV-1 RNA (including results of < 20 copies/mL, detected) were re-tested with OraQuick ADVANCE^® and Uni-Gold™ Recombigen^® and if antibody negative, were considered to be from an acutely infected participant. Only those specimens identified as “pre-seroconversion” defined as having detectable HIV-1 RNA and negative antibody reactivity were used for the evaluation of fourth generation RDTs.

3.3. Third and fourth generation diagnostic test evaluations

Third generation RDT-negative samples were tested with the fourth generation FDA-approved Alere™ Determine™ HIV-1/2 Ag/Ab Combo rapid test (Alere North America, Waltham, MA) and the CE-Marked Alere™ HIV Combo Rapid Test (Alere Medical Co. Ltd., Matsudo-chi, Japan) (Supplementary Table S1). The Determine™ Combo and Alere™ HIV Combo RDTs are named similarly but have different formulations, different places of manufacture, and different performance specifications. Samples were also tested in duplicate by the third generation Bio-Rad GS HIV-1/2 + O Enzyme Immunoassay (EIA) and the fourth generation Bio-Rad GS HIV Combo Ag/Ab EIA, both FDA-Approved (Bio-Rad Laboratories).

3.4. Confirmatory test evaluations

Pre-seroconversion samples found to be positive by at least one of the above fourth generation tests were further tested for the presence of antibodies to HIV-1 and/or HIV-2 by Multispot^® HIV-1/HIV-2 Rapid Test, Geenius™ HIV-1/2 Supplemental Assay and GS HIV-1 Western Blot (Bio-Rad).

3.5. Statistical analysis

McNemar’s test was used to compare the proportion of positive/reactive test results between assays. For calculating sensitivity, specificity, positive and negative predictive value of the results of the third generation HIV RDTs performed in VOICE, the gold standard was defined by the results of the study algorithm (Fig. 1) which included additional WB and/or HIV-1 RNA testing by a local and/or central laboratory during the study. Samples that tested negative or indeterminate for RDT and/or WB but had detectable HIV-1 RNA were considered endpoints in the VOICE study. The gold standard for the same calculations on the results of the HIV tests evaluated in this study was detectable or undetectable HIV-1 RNA by Roche TaqMan. All confidence intervals are exact (Clopper-Pearson) 95% confidence intervals.

4. Results

4.1. Third generation rapid test performance in VOICE

Between 2009 and 2011, 5029 enrolled VOICE participants had 77,915 follow-up visits to screen for HIV infection. Of the 72,169 visits where two rapid tests were used, 55,346 (77%) were conducted with Uni-Gold™ Recombigen^® and Determine™ HIV-1/2 while 16,823 (23%) were conducted using OraQuick ADVANCE^® and Determine™ HIV-1/2. The rate of discordancy was similar for both rapid test combinations (0.15%). Ultimately, rapid testing correctly identified HIV infection status for 312 seroconverters and 4717 non-seroconverters at 72,055 of 72,169 (99.84%) follow-up visits in VOICE, however final HIV status differed from rapid test results at 114 visits (50 visits had a false positive rapid result while 64 visits had a false negative rapid result) (Fig. 2). Actual sensitivity for all 3 tests was markedly lower than manufacturer’s claims; Determine™ HIV-1/2 had the highest sensitivity at 88.78%, followed by Uni-Gold™ Recombigen^® at 85.86% and OraQuick ADVANCE^® at 61.54% (Table 1).

Fig. 2.

Third Generation Rapid Testing Results on Whole Blood Performed During Study Follow-Up Visits at Clinical Sites.

Table 1.

Sensitivity, Specificity, Positive and Negative Predictive Values of Rapid and Immunoassay Tests.

	Tests Performed (N)	Sensitivity (95% CI)	Specificity (95% CI)	Positive Predictive Value (95% CI)	Negative Predictive Value (95% CI)
Determine™ HIV−1/2a	72,196	n =401	n =71,795	n = 388	n =71,808
		88.78	99.96	91.75%	99.94%
		(85.27, 91.70)	(99.94, 99.97)	(88.56, 94.29)	(99.92, 99.95)
OraQuick® ADVANCEa	16,854	n = 26	n = 16,828	n = 17	n = 16,837
		61.54%	99.99%	94.12%	99.94%
		(40.57, 79.77)	(99.97, 100.00)	(71.31, 99.85)	(99.89, 99.97)
Uni-Gold™ Recombigen®a	61,075	n =389	n =60,686	n = 352	n =60,723
		85.86%	99.97%	94.89%	99.91%
		(82.00, 89.17)	(99.95, 99.98)	(92.04, 96.94)	(99.88, 99.93)
FDA-Approved Determine™ Combob	229	n = 71	n = 158	n = 19	n = 210
		23.94%	98.73%	89.47%	74.29%
		(14.61, 35.54)	(95.50, 99.85)	(66.86, 98.70)	(67.82, 80.05)
CE-Marked HIV Combob	229	n = 71	n = 158	n = 32	n = 197
		42.25%	98.73%	93.75%	79.19%
		(30.61, 54.56)	(95.50, 99.85)	(79.19, 99.23)	(72.84, 84.63)
GS HIV−1/2 + O EIAb	229	n = 71	n = 158	n = 23	n = 206
		30.99%	99.37%	95.65%	76.21%
		(20.54, 43.08)	(96.52, 99.98)	(78.05, 99.89)	(69.80, 81.85)
GS HIV Combo Ag/Ab EIAb	229	n = 71	n = 158	n = 42	n = 187
		57.75%	99.37%	97.62%	83.96%
		(45.44, 69.39)	(96.52, 99.98)	(87.43, 99.94)	(77.90, 88.91)

^aTests performed at trial sites during VOICE using fingerstick or venous-collected whole blood.

^bTests performed as part of post-trial analysis using stored plasma.

4.2. Pre-Seroconversion HIV-1 RNA

The low sensitivity of third generation rapid tests for detecting seroconversion in VOICE was investigated using pre-seroconversion plasma. Of 312 participants who seroconverted in VOICE, stored plasma collected within 91 days prior to the date of the first positive point-of-care rapid test was available for 215 HIV-1 seroconverters from 246 visits. HIV-1 RNA was performed on 230 of 246 (93%) specimens (14 of 16 not tested had an undetectable HIV-1 RNA result at a closer visit; 2 of 16 had specimen integrity issues). Of 230 specimens, HIV-1 RNA was detected in 32 of 48 (67%) collected 12–29 days pre-seroconversion, 33 of 98 (34%) collected 30–59 days pre-seroconversion, and 7 of 84 (8%) collected 60–91 days pre-seroconversion (Table 2).

Table 2.

Proportion of Positive or Reactive Test Result from Pre-Seroconversion Plasma Specimens.

	Number (%) of Positive or Reactive Test Results
Number of days that sample was collected prior to positive third generation rapid test result at the clinical site	12–29 days	30–59 days	60–91 days	TOTAL
	N=48	N=98	N=84	N = 230
Roche TaqMan, v2.0 HIV-1 RNA ≥20 copies/mL or < 20 copies/mL, detected	32 (67%)	33 (34%)	7 (8%)	72 (31%)
Uni-Gold™ Recombigen®	8 (17%)	5 (5%)	1 (1%)	14 (6%)
OraQuick ADVANCE®	6 (13%)	1 (1%)	2 (2%)	9 (4%)
	N=48	N=98	N=83a	N = 229a
CE-Marked Alere™ HIV Combo	16 (33%)	14 (14%)	2 (2%)	32 (14%)
FDA-Approved Determine™ Combo	12 (25%)	6 (6%)	1 (1%)	19 (8%)
GS HIV-1/2 + O EIA	13 (27%)	8 (8%)	2 (2%)	23 (10%)
GS HIV Combo Ag/Ab EIA	23 (48%)	16 (16%)	3 (4%)	42 (18%)
Multispot HIV-1/HIV-2	7 (15%)	4 (4%)	1 (1%)	12 (5%)
GS HIV-1 Western Blot	6 (13%)	3 (3%)	1 (1%)	10 (4%)
Geenius HIV-1/2 Supplemental	6 (13%)	4 (4%)	1 (1%)	11 (5%)

^aOne specimen was only tested by Roche TaqMan, Uni-Gold™ Recombigen^® and OraQuick ADVANCE^®.

4.3. 4th generation rapid tests and immunoassays

Of 72 RNA-positive pre-seroconversion plasma specimens, 15 were positive for Uni-Gold^® Recombigen^® and/or OraQuick Advance^® upon re-testing and were excluded from further analysis (Table 2). Of the remaining 57, 16 (28%) were positive by CE-HIV Combo RDT (1 Ab only, 9 Ag only, 6 dual Ag/Ab) and only 4 (7%) were positive by FDA-Determine ™ Combo RDT (2 Ab only, 2 Ag only, 0 dual Ag/Ab) (Table 3). The CE-HIV Combo RDT detected 21% more infections than the FDA-Determine™ Combo RDT (p = 0.0005). 27 of 57 (47%) were positive by HIV Combo EIA and 8 of 57 (14%) were positive by HIV-1/2 + O EIA. HIV Combo EIA detected 33% more infections than HIV-1/2 + O EIA (p < 0.0001) (Fig. 3). Specificity was high (> 98.7%) for all fourth generation tests, while sensitivity was highest for HIV Combo EIA (57.75%) followed by CE-HIV Combo RDT (42.25%) and Determine ™ Combo RDT (23.94%) (Table 1).

Table 3.

Rapid, Immunoassay and Confirmatory Test Results for HIV-1 RNA-Positive, Third Generation Rapid Negative and HIV Combo-Positive Pre-Seroconverter Specimens.

PID	Sample Collected # days Prior to Positive 3rd Generation Rapid Test	HIV-1 RNA (Copies/mL)	Rapid Test		Immunoassay		Confirmatory Tests
			CE-Marked HIV Combo	Determine™ HIV-1/2 Ag/ Ab Combo	GS HIV- 1/2 + O	GS HIV Combo Ag/Ab	Multispot HIV- 1/HIV-2	GS HIV-1 Western Blot (bands)	Geenius HIV-1/2 Supplemental (bands)
1	27	1,904,350	Ag/Ab	Ab	R	R	NR	Ind. (gp41, p24)	HIV Neg.
2	28	1,554,640	Ag/Ab	Ab	NR	R	NR	Ind. (gp41)	HIV Neg.
3	29	4,790,990	Ag	Ag	NR	R	NR	Ind. (p55/51, gp41)	HIV-2 Ind. (gp140)
4	32	17,546,710	Ag	Ag	NR	R	NR	Neg.	HIV Neg.
5	12	65,672,120	Ag	Neg.	NR	R	NR	Neg.	HIV Neg.
6	20	2,176,510	Ag	Neg.	NR	R	NR	Ind. (*gp41, *p18)	HIV Neg.
7	23	18,683,070	Ag/Ab	Neg.	R	R	HIV-1 R	Neg.	HIV Neg.
8	29	6,373,520	Ag/Ab	Neg.	R	R	NR	Neg.	HIV Neg.
9	30	20,594,620	Ag/Ab	Neg.	NR	R	NR	Neg.	HIV Neg.
10	30	3,426,650	Ag	Neg.	NR	R	NR	Ind. (gp41)	HIV Neg.
11	32	2660	Ag	Neg.	NR	NR	NR	Ind. (*gp41, p55/51)	HIV-2 Ind. (gp140)
12	35	2,180,160	Ag	Neg.	R	R	NR	Ind. (*gp41)	HIV Neg.
13	52	154	Ab	Neg.	R	R	NR	Ind. (*p55/51, *p41, p40, p24)	HIV Neg.
14	56	3,405,070	Ag	Neg.	NR	R	NR	Ind. (*gp41)	HIV Neg.
15	57	533,140	Ag	Neg.	NR	R	NR	Neg.	HIV Neg.
16	82	12,706,620	Ag/Ab	Neg.	NR	R	NR	Neg.	HIV Neg.

Abbreviations: Participant identifier (PID); Antigen (Ag); Antibody (Ab); Antigen/Antibody (Ag/Ab); Negative (Neg.); Reactive (R); Non-Reactive (NR); Indeterminate (Ind.).

Fig. 3.

Proportion of RNA Positive/3rd Generation Rapid Negative Samples Detected as HIV Positive by Third and Fourth Generation EIA and Fourth Generation Rapid Tests. 57 pre-seroconversion plasma samples with HIV RNA > 20 copies/ml that tested HIV negative using third generation antibody (Ab)-only rapid diagnostic tests were evaluated with Bio-Rad GS HIV-1/2 + O EIA, Bio-Rad HIV Combo Antigen (Ag)/Ab EIA, Alere Determine™ HIV-1/2 Ag/Ab Combo, and Conformité Européene (CE)-Marked Alere™ HIV Combo. 8 of 57 (14%) were positive by Bio-Rad GS HIV-1/2 + O EIA and 27 of 47 (47%) were positive by Bio-Rad HIV Combo Ag/Ab EIA (p < 0.0001). 4 of 57 (7%) were positive by Alere Determine™ HIV-1/2 Ag/Ab Combo (2 Ab only, 2 Ag only, 0 dual Ag/Ab) and 16 of 57 (28%) were positive by CE-Marked Alere™ HIV Combo (1 Ab only, 9 Ag only, 6 dual Ag/Ab).

4.4. Confirmatory tests

Because some HIV testing algorithms require confirmatory testing, we evaluated the concordance of fourth generation rapid test results with three Bio-Rad HIV confirmatory tests, Western blot, Multispot^® and Geenius™. Of 16 reactive CE-HIV Combo RDT samples, 9 were indeterminate by Western blot. Only one of 16 was reactive by Multispot^®, and no samples were reactive by Geenius™, with the exception of two specimens that were false reactive with a gp140 band yielding an HIV-2 indeterminate result (Table 3).

5. Discussion

Using pre-seroconversion plasma specimens from the VOICE study, we evaluated fourth generation RDT to determine if acute HIV infection would have been detected earlier compared to the third generation RDT used in the trial. The high incidence of acute infections in VOICE resulted in much lower sensitivities of third generation RDT (61.5% to 88.8%) relative to manufacturers’ claims (99.6% to 100%) and confirmatory tests also performed poorly in detecting acute infection. In our study, 28% of infections from participants with detectable HIV-1 RNA and negative third generation RDT were detected by CE-HIV Combo. These data support the use of CE-HIV Combo in HIV diagnostic algorithms in high incidence settings.

Numerous studies have found the antigen component in fourth generation RDT to be insensitive and have reported that these tests do not provide any advantage for the diagnosis of acute infection [4–6,9,10,14–17]. Notably, we found major differences in test performance using two different versions of the same rapid test from the same company; CE-HIV Combo detected 21% more acute infections than FDA-Determine™ Combo. This is the first study to directly compare both versions of the Alere™ HIV Combo tests, and it highlights the importance of using the appropriate kit in diagnostic algorithms. The 4th Generation HIV Combo EIA had the highest improvement in detection of any test evaluated (47%) (Fig. 3), however, cannot be used in algorithms requiring point-of-care testing.

Acute HIV detection is paramount for both PrEP clinical trials and PrEP rollout. In trials, delayed detection of infection can increase the number of person-years a participant is on an investigational product before reaching an endpoint, and more importantly, how long a participant continues taking an investigational product while already HIV infected. In five trials of TDF/FTC PrEP, the greatest risk of drug resistance occurred in participants who enrolled with undetected acute infection [1]. In VOICE, the majority of seroconverters obtained their HIV status within 3 months of acquiring HIV infection due to monthly testing, but the interval between infection and diagnosis in PrEP users outside the trial setting will be longer due to quarterly or less frequent HIV testing. Modeling studies are needed to evaluate the cost-benefit of implementing the CE-HIV Combo in PrEP programs.

Our study had several limitations. Plasma from seroconverters was stored quarterly, limiting the precision for calculating the exact number of days that the CE-HIV Combo could detect infection earlier than third generation RDT. We found that 15 of 230 specimens that tested negative by Uni-Gold™ Recombigen^® and/or OraQuick ADVANCE^® at clinical sites tested positive upon re-testing with stored plasma using the same RDT, suggesting that actual test sensitivity may be lower in point-of-care settings using fingerstick-collected blood compared to research settings due to imprecision in execution of test procedures.

The updated CE-Marked Alere™ HIV Combo Rapid Test has potential for improving current HIV algorithms for detection of acute HIV infection. Better confirmatory tests are needed to avoid discrepant results that require nucleic acid testing to resolve. Shortening the window period with antigen/antibody HIV tests will benefit individuals on PrEP for timely HIV diagnosis, and could help lower population-based incidence by decreasing transmission of HIV during acute infection.