Performance evaluation of the FDA-approved Determine^™ HIV-1/2 Ag/Ab Combo assay using plasma and whole blood specimens

Abstract

Background:

The Determine^™ HIV-1/2 Ag/Ab Combo (DC) rapid test can identify HIV-1 infection earlier than rapid antibody-only tests in plasma specimens.

Objectives:

We compared the performance of DC with a laboratory-based antigen/antibody (Ag/Ab) combo assay in plasma and evaluated antigen reactivity in whole blood specimens.

Study design:

We tested by DC 508 plasma specimens collected in a prospective study and 107 sequential plasma and simulated whole blood specimens from 20 seroconversion panels. Previous results using the ARCHITECT (ARC) Ag/Ab combo assay were compared to DC results. In seroconversion panels, the days from the first HIV1 RNA-positive test to first DC-reactive in plasma and whole blood was compared. McNemar’s and Wilcoxon signed rank tests were used for statistical analysis.

Results:

Of 415 HIV-positive samples, ARC detected 396 (95.4%) and DC 337 (81.2%) (p < 0.0001). DC was reactive in 50.0% of ARC-reactive/MS-negative, 78.6% of ARC-reactive/MS-indeterminate, and 99.6% of ARC-reactive/MS-HIV-1-positive or −undifferentiated specimens. DC antigen reactivity was higher among ARC-reactive/MS-negative than MS-indeterminate samples. In 20 HIV-1 seroconversion panels, there was a significant difference between DC reactivity in plasma (91.1%) and whole blood (56.4%) (p < 0.0001). DC with whole blood showed a significant delay in reactivity compared to plasma (p = 0.008).

Conclusions:

In plasma, DC was significantly less sensitive than an instrumented laboratory-based Ag/Ab combo assay. DC in plasma was significantly more sensitive compared to whole blood in early HIV-1 infections. With the U.S. laboratory-based diagnostic algorithm, DC as the first step would likely miss a high proportion of HIV-1 infections in early stages of seroconversion.

1. Background

In 2014 the Centers for Disease Control and Prevention (CDC) and the American Public Health Laboratories (APHL) published an updated HIV diagnostic algorithm for laboratory settings. The updated algorithm recommends screening with an HIV antigen/antibody (Ag/Ab) combination assay followed by a supplemental assay that differentiates between HIV-1 and HIV-2 antibodies. An HIV-1 RNA test is used to resolve discrepant results. With Food and Drug Administration (FDA) approval of new HIV diagnostic assays in the U.S., evaluation of the algorithm’s performance using these new assays is needed [1–4].

In the U.S., four laboratory-based immunoassays and one rapid test (RT) are approved by the FDA that can simultaneously detect HIV-1 p24 antigen and HIV-1/HIV-2 antibodies and potentially could be used as screening assays in the updated algorithm [5–11]. Of those assays, the Alere Determine^™ HIV-1/2 Ag/Ab Combo (DC) RT is the only point-of-care (POC) assay and is one of only two assays that can distinguish HIV-1 antigen from antibody reactivity [12]. DC received FDA approval in 2013 and received a Clinical Laboratory Improvement Amendments (CLIA) waiver in 2014. Evaluation of DC for HIV diagnosis using serum/plasma during early stages of infection has been limited and most results showed poor performance of antigen detection [11,13–16]. Other DC evaluations using whole blood from within and outside of the U.S. also indicated limited ability of the test to detect p24 antigen although some of those evaluations were performed with the previous version of DC [17–22]. A laboratory study using plasma specimens revealed that DC detected early HIV-1 infections later than laboratory-based Ag/Ab combo assays, but before laboratory tests that detect HIV IgM antibodies and approximately 15.5 days before HIV-1 western blot positivity [11].

2. Objectives

We evaluated the performance of DC with plasma specimens from persons enrolled in a study to identify acute infections as the first step of the CDC/APHL HIV laboratory diagnostic algorithm. Additionally, we evaluated both antigen reactivity in simulated whole blood made from commercial seroconversion panels and overall test performance in plasma and simulated whole blood early after infection.

3. Study design

3.1. Sample sets

3.1.1. Plasma specimens from the STOP study

The Screening Targeted Populations to Interrupt On-going Chains of HIV Transmission with Enhanced Partner Notification (STOP) study was a prospective study to evaluate methods to detect acute HIV infection (AHI) among high-risk populations in New York City, North Carolina and San Francisco. The study provided an unique opportunity to evaluate improved detection of early HIV infections with the CDC/APHL HIV laboratory diagnostic testing algorithm using the instrumented Abbott ARCHITECT HIV Ag/Ab Combo Assay (ARC) and the Bio-Rad Multispot HIV-1/HIV-2 Rapid test (MS) [3,23–25]. The STOP study was approved by the Institutional Review Boards for the University of California at San Francisco, the University of North Carolina at Chapel Hill, and the New York City Department of Health & Mental Hygiene (HSR 6193). Study participants were screened for HIV antibody with RTs on whole blood. To identify AHIs at each site, plasma specimens collected between September 2011 and October 2013 from RT-antibody negative persons were further tested with ARC (screening assay) and MS (supplemental assay) at all sites and with an HIV-1 RNA assay (Gen-Probe APTIMA HIV-1 RNA; (APTIMA)) in NYC and NC, and with a validated Abbott m2000 HIV-1 viral load (VL) assay in SF [25,26]. Plasma specimens from individuals that screened RT antibody positive were also tested with ARC and MS (established HIV-1 infections). MS was discontinued in 2016. For this evaluation, 508 available frozen plasma specimens (including 415 HIV-positive and 93 ARC-false reactive −ARC-reactive/HIV-1 RNA-negative-) were sent to CDC for further testing. A subset of 87 ARC-false reactive plasma specimens were also tested with the Bio-Rad GS HIV Combo Ag/Ab EIA (BRC) and DC to assess concordance of false reactivity.

3.1.2. Plasma and whole blood specimens from seroconversion panels

Twenty well-characterized, frozen HIV-1 seroconversion plasma panels from U.S. donors (presumably subtype B) were purchased from Zeptometrix, Inc. (Buffalo, NY) and BBI-SeraCare Diagnostics (Milford, MA) [4,11,27]. Plasma specimens were tested with DC, ARC, MS and APTIMA. A subset of 107 sequential plasma specimens from the 20 seroconverters described above were used to prepare simulated whole blood. To assess antigen detection with DC, the collection points selected were around the previously documented seroconversion phase. Plasma specimens were thawed and mixed at 40% hematocrit with washed red blood cells (Group O) from a single donor to simulate HIV-1-infected whole blood. Fifty microliters of the prepared whole blood was tested with DC on the day of preparation. Whole blood specimens were not tested with other FDA-approved tests.

3.2. HIV serologic assays at CDC

The Alere DC RT (Orgenics, Ltd., Yavne, Israel) was used as indicated in the package insert for plasma and whole blood specimens [12]. All specimens were tested in singlet and repeated only if invalid results (absence of the control line) were obtained. Results were read by a single operator between 20 and 30 min after adding the sample at ambient temperature. In ARC-false reactive samples, the BRC assay was used following the package insert (Bio-Rad, CA, USA).

3.3. Analysis

For this study, HIV-1 infection status was defined following the recommendations of the CDC/APHL laboratory-based HIV diagnostic algorithm [2]: AHI as ARC-reactive/MS-negative or −indeterminate/RNA-positive and established infection as ARC-reactive/MS-HIV-1 positive or −positive undifferentiated. False reactive specimens (HIV-1-negative) were defined as ARC-reactive/MS-negative or −indeterminate/RNA-negative. The RNA test was either APTIMA or a validated VL assay as described earlier. The different proportions in DC reactivity between screening tests (DC and ARC) and between plasma and whole blood specimens were analyzed using the McNemar’s test. The number of days since the first plasma collection date with APTIMA-reactivity (first available indication of HIV-1 infection) to the first date where DC was reactive for plasma and simulated whole blood specimens was calculated and compared statistically using the Wilcoxon signed rank test. A p value < 0.05 was considered statistically significant.

4. Results

4.1. DC performance on plasma specimens from the STOP study

Of 508 STOP study plasma specimens available, 142 were from NYC, 37 from NC and 329 from SF. 277 SF specimens were unique and 52 had two or three follow-up plasma specimens that initially were RT-antibody negative. Table 1 shows the distribution of plasma specimens classified by previous testing at different sites.

Table 1.

Previously collected HIV-1 test results on plasma specimens from the STOP study.^a

Results of previous testing	Total	New York City	North Carolina	San Francisco
HIV−1 RNA-positive/ARC-nonreactive/MSb	19	9	1	9
HIV-1 RNA-positive/ARC-reactive/MS-negative	110	26	10	74
HIV-1 RNA-positive/ARC-reactive/MS-indeterminate	14	3	0	11
ARC-reactive/MS-HIV-1 positive	271	67	2	202
ARC-reactive/MS-positive undifferentiatedc	1	0	0	1
HIV-1 RNA-negative/ARC-reactive/MS-negative or −indeterminate	93	37	24	32
Total	508	142	36	329

^aPlasma testing: HIV-1 RNA: GenProbe APTIMA or Abbott m2000 HIV-1 RNA viral load, ARC: Abbott ARCHITECT HIV Ag/Ab (screening assay), MS: Bio-Rad Multispot HIV-1/HIV-2 rapid test (supplemental assay). Tests were interpreted following the manufacturer’s package insert.

^bMS was not performed.

^cSpecimens with MS-positive undifferentiated results are positive for HIV-1 and HIV-2 antibodies and require further testing to confirm the type of infection. Samples were defined as HIV-1 positive or HIV-1 negative following the recommendations of the CDC/APHL laboratory-based HIV diagnostic algorithm [2].

None of the 508 plasma specimens tested invalid with DC. Of 415 HIV-positive samples, ARC detected 396 (95.4%) and DC 337 (81.2%) (Table 2). All 19 plasma specimens from AHIs that were positive in only the HIV-1 RNA assays were DC-negative. The difference in reactivity between the two screening assays was statistically significant (p < 0.0001). Overall DC detected 337 of 396 (85.1%) of HIV-1-positive plasma specimens that were ARC-repeatedly reactive and confirmed to be HIV-1 infections (Table 2). DC detected 50% (55/110) of ARC-reactive/MS-negative samples (p < 0.0001). There was no significant difference between the two screening assays when MS was either indeterminate, positive undifferentiated or HIV-1-positive. DC detected 78.6% (11/14) of ARC-reactive/MS-indeterminate specimens (p = 0.2482) and 99.6% (271/272) of ARC-reactive/MS-positive or −undifferentiated (p = 1). DC antigen detection was observed in 40.9% (45/110) of ARC-positive/MS-negative samples, 28.6% (4/14) of ARC-reactive/MS-indeterminate samples, and 2.9% (8/272) of ARC-reactive/MS-HIV-1-positive or −positive undifferentiated samples.

Table 2.

Reactivity of Determine Combo in plasma specimens from early and established HIV-1 infections from the STOP study.^a

	Total	Determine Combo resultsb				Overall DC positivity
		Ag-/Ab-	Ag+/Ab-	Ag+/Ab+	Ag-/Ab+
HIV−1 RNA-positive/ARC-nonreactive/MSc Total (%)	19 (4.6)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)
HIV-1 RNA-positive/ARC-reactive/MS-negativeTotal (%)	110 (26.5)	55 (50.0)	37 (33.6)	8 (7.3)	10 (9.1)	55 (50.0)
HIV-1 RNA-positive/ARC-reactive/MS-indeterminate Total (%)	14 (3.4)	3 (21.4)	1 (7.2)	3 (21.4)	7 (50.0)	11 (78.6)
ARC-positive/MS-HIV-1 positive or −undifferentiated Total (%)	272 (65.5)	1 (0.4)	0 (0)	8 (2.9)	263 (96.7)	271 (99.6)
Total (%)	415 (100)	59 (14.2)	38 (9.2)	19 (4.6)	280 (67.5)	337 (81.2)

^aPlasma testing: HIV-1 RNA: GenProbe APTIMA or Abbott m2000 HIV-1 RNA viral load, ARC: Abbott ARCHITECT HIV Ag/Ab (screening assay), MS: Bio-Rad Multispot HIV-1/HIV-2 (supplemental assay).

^bAg: antigen, Ab: antibody, −: negative, +: positive.

^cMS was not performed.

Of 124 ARC-reactive/MS-negative or −indeterminate samples, 123 had available VL results. Fig. 1 shows that detection of antigen by DC was mostly observed with high viremia in plasma specimens from early stages of HIV-1 infection. When the results were separated between ARC-reactive/MS-negative and ARC-reactive/MS-indeterminate (different stages of seroconversion), among 110 MS-negative samples, VL ranged from 3.1 × 10³ to >10⁷ copies/ml with a median of 10⁷ copies/ml. Of 13 MS-indwhich one was DC antigen-reactiveeterminate samples, VL ranged from 3.9 × 10⁴ to >10⁷ copies/ml with a median 7.5 × 10⁵ copies/ml (data not shown). Of 110 MS-negative, 45 samples were DC antigen-reactive with a median VL of 3.8 × 10⁶ copies/ml, of which 37 were DC antigen-reactive/antibody-nonreactive and eight were DC antigen/antibody-reactive (data not shown). Of 13 MS-indeterminate specimens, three were antigen-reactive samples with VLs > 10⁷ copies/ml, of which one was DC antigen-reactive/antibody-nonreactive and two were DC antigen/antibody-reactive (data not shown).

Fig. 1.

Distribution of 1 viral load and Determine Combo (DC) antigen results in plasma specimens from acute HIV-1 infections.

In plasma specimens that were repeatedly reactive with Abbott Architect (ARC-RR) and were either Multispot-negative or indeterminate, 123 had HIV-1 viral load results. For calculations of the median viral load and standard deviations, the values of >10⁷ were considered as 10⁷ copies/ml. DC Ag-Ab-: DC antigen-nonreactive/antibody-nonreactive, DC Ag+Ab-: antigen-reactive/antibody-nonreactive, DC Ag+Ab+: antigen-reactive/antibody-reactive, and DC Ag-Ab+: antigen-nonreactive/antibody-reactive.

Among 93 ARC-false reactive, HIV-1 RNA negative samples, DC testing showed antibody reactivity in three specimens (3.2%). Of 87 available plasma specimens tested with BRC, another laboratory-based Ag/Ab combo assay, six were repeatedly reactive (6.9%). Of those six, two specimens were also reactive with DC.

4.2. Comparison of DC performance on plasma and whole blood specimens from seroconversion panels

DC was nonreactive using plasma and whole blood in all eight specimens that were collected before HIV-1 RNA was detected. Of 14 plasma specimens that were HIV-1 RNA-positive and ARC/DC-nonreactive, one was antigen reactive and one was repeatedly invalid with whole blood.

The reactivity of DC among ARC-reactive plasma and whole blood specimens is shown in Fig. 2. In plasma, DC detected 87.5% (35/40), 100% (16/16), and 96.7% (29/30) of MS-negative, MS-indeterminate and MS-HIV-1-positive samples, respectively. In whole blood, DC detected 56.4% (22/39), 56.3% (9/16), and 96.7% (29/30) of MS-negative, MS-indeterminate and MS-HIV-1 positive samples, respectively. One MS-negative whole blood sample was repeatedly invalid with DC. In plasma specimens, antigen detection (with or without antibody reactivity) was observed in 84.6% (33/39) and 37.7% (6/16) of MS-negative and MS-indeterminate samples, respectively. For whole blood, a marked reduction of antigen detection (53.8%, 21/39) was observed among MS-negative samples compared to plasma, whereas in MS-indeterminate samples, antigen detection was 31.3% (5/16). Overall, DC reactivity in AHIs (MS-negative or −indeterminate specimens) was significantly higher in plasma (91.1%) than in whole blood (56.4%) (p < 0.0001).

Fig. 2.

Reactivity of Determine Combo using whole blood specimens from commercial seroconversion panels.

The selected bleeds were repeatedly reactive with the Abbott Architect test and were Multispot negative, indeterminate or HIV-1-positive in plasma specimens. The reactivity of Determine Combo for the antigen (Ag) or the antibody (Ab) line is indicated as nonreactive (−) or reactive (+) under the x axis. The number of total results is indicated by each bar for plasma and whole blood specimens.

An initial invalid DC result showing absence of the control line occurred in 19 (17.8%) of whole blood specimens. Two invalid DC result scenarios were observed: (1) the whole blood specimen did not flow through the pad to the control line and (2) the specimen had a very reactive antibody line but no control line. Seventeen samples were resolved and 2 specimens remained invalid after repeat testing (data not shown).

DC reactivity in plasma and whole blood was measured in days since the first available plasma APTIMA-reactive result (Fig. 3). Delay between specimen types was not observed in 11 of 20 seroconverters. Nine showed a delay in reactivity in simulated whole blood specimens, but one seroconverter had two repeatedly invalid DC results and never become DC-positive up to 28 days after the first available APTIMA-reactive result. The remaining 8 seroconverters had a median delay of six days between plasma and whole blood specimens. One seroconverter had delayed reactivity and a second negative DC phase over three consecutive bleeds in whole blood for a period of seven days after becoming antigen positive. The second negative phase was not observed in plasma with DC and ARC. Excluding the negative seroconverter whole blood specimen, DC reactivity with whole blood was observed significantly later, relative to the first available APTIMA-reactive result, than with plasma (p = 0.008, Wilcoxon signed rank test).

Fig. 3.

Reactivity of Determine Combo since first HIV-1 RNA positive result on plasma and whole blood specimens from commercial seroconversion panels.

The x-axis represents the number of days since the first available APTIMA HIV-1 RNA reactive result for each seroconverter (SC1-SC20) shown on the y-axis. SC20 was capped at 28 days because later time points were not available for whole blood specimens. SC1, SC5, SC8 and SC9 showed reactivity at the same time as the first available APTIMA HIV-1 RNA reactive results with both plasma and whole blood.

5. Discussion

This evaluation describes the comparison of DC with plasma specimens collected during a multi-site study in high-risk populations to a laboratory-based Ag/Ab combo assay used as a first step in the CDC/APHL HIV diagnostic algorithm for laboratory settings used in the U.S. [2]. We also describe the performance of DC in plasma and simulated whole blood during early stages of HIV-1 infection.

For the study, sequential plasma specimens were analyzed independently to assess DC reactivity in the context of the diagnostic algorithm. The difference between screening plasma specimens with ARC and DC was significant. These findings confirm prior results using U.S. seroconversion panels that were tested with another laboratory-based Ag/Ab combo assay, BRC, and DC [11]. Although DC detected significantly fewer AHIs than the laboratory-based assays, the antigen line was reactive (with or without antibody reactivity) for 40.9% of MS-negative samples and 28.6% of MS-indeterminate samples. These results indicate better antigen detection by DC using plasma/serum than previously published studies from outside of the U.S. [13–15]. Using DC with plasma/serum as the initial test in the laboratory algorithm might offer a fast turnaround time and antigen detection, especially in laboratories that lack an instrumented HIV Ag/Ab combo assay. However, in absence of antigen detection DC antibody reactivity may not be equivalent to that of a laboratory-based IgG/IgM antibody only assay [11] but is similar to that of an IgG/IgM antibody only RT [13].

Not unexpectedly DC detected more antigen reactive specimens early during HIV-1 infection with peak viremia and when potentially more unbound p24 antigen is present for detection. In MS-indeterminate specimens, when IgG antibodies are likely complexed with p24 antigens, DC antigen detection decreases.

Other studies have also shown that DC has a high proportion of antigen false reactivity [15,19,28]. Because different assays may have different false reactivity causes, we also tested 87 available ARC-false reactive (HIV-1 RNA-negative) plasma specimens with another Ag/Ab combo assay, BRC, and six were repeatedly reactive. ARC and BRC cannot distinguish between Ag and Ab reactivity. DC was antibody-reactive in three of the ARC-false reactive and two of those were also BRC-reactive. Interestingly, two of those three STOP study plasma specimens tested in North Carolina were also reactive with Bio-Rad GS HIV-1/2 Plus O, an IgG/IgM immunoassay, with very high signal to cutoff ratios, but had undetectable HIV-1 RNA. One of those two specimens had a MS-indeterminate result. Information collected during enrollment in the STOP study indicates the two participants were unaware of their HIV status, but the reactivity observed with different HIV tests described above may reflect that the use of antiretroviral therapy and/or HIV status may have been misreported, subject to recall bias, or some other factor [29]. These test results might be interpreted as the participants misreporting their status and having false-negative HIV-1 RNA results [30] or showing false reactivity in the different assays. DC did not show any reactivity in plasma or simulated whole blood specimens with samples collected before infection was detected by HIV-1 RNA tests. However, antigen false reactivity was observed in one simulated whole blood specimen from a plasma specimen that was HIV-1 RNA-positive but ARC/DC-nonreactive. In this laboratory study performed by trained staff, the proportion of false reactive results was low.

A limitation of this study is the use of simulated whole blood and the lack of comparative data between fingerstick and simulated whole blood. Although an evaluation with fingerstick whole blood is optimal, this method is an alternative to setting up costly field studies to identify persons during early stages of seroconversion. Our results may not reflect the true performance of DC in point-of-care settings [17,18], but they are similar to reports showing that DC testing with whole blood detects significantly less early HIV-1 infection than with plasma specimens [21]. In our study, a decreased sensitivity (and antigen detection) and a significant delay in reactivity between plasma and whole blood specimens was observed. The lower sensitivity of DC with whole blood supports the findings of poor p24 antigen detection by DC in cross-sectional surveys [17,19,20]. Using simulated whole blood we found a high proportion (17.8%) of initially invalid results. However, only two specimens remained invalid after repeat testing although this could be a limitation of the study design.

In this laboratory analysis, DC detected AHI in plasma specimens, but significantly fewer than with an instrumented laboratory-based Ag/Ab combo assay. Use of DC with plasma specimens may be beneficial in settings where there is no access to complex laboratory-based assays. However, despite the ease of use and fast turnaround time, routine use of DC in the CDC/APHL HIV diagnostic algorithm will require careful consideration of the potential usage circumstances and reported limitations.