Performance of a Fourth-Generation HIV Screening Assay and an Alternative HIV Diagnostic Testing Algorithm

Abstract

Objective

We evaluated the performance of the GS fourth-generation antigen/antibody assay and compared CDC’s proposed alternative algorithm (repeatedly reactive [RR] fourth-generation immunoassay [IA] followed by an HIV-1/HIV-2 differentiation IA and, if needed, nucleic acid testing [NAT]) with the current algorithm (RR third-generation IA followed by HIV-1 Western blot [WB]).

Design

A convenience sample of the following four specimen sets was acquired: 10,014 from insurance applicants, 493 known WB-positive, 20 known WB-indeterminate specimens, and 230 specimens from 26 HIV-1 seroconverters.

Methods

Specimens were tested with the GS third- and fourth-generation IAs, the Multispot HIV-1/HIV-2 differentiation IA, NAT, and WB. We applied the two algorithms using these results.

Results

Among insurance specimens, 13 (0.13%) specimens were IA RR: 2 were HIV-positive (RR by third- and fourth-generation IAs, and WB and Multispot positive); 2 third-generation RR and 9 fourth-generation RR specimens were false-positive. Third- and fourth-generation specificities were 99.98% (95%CI: 99.93%–100%) and 99.91% (95%CI: 99.84%–99.96%) respectively.

All HIV-1 WB-positive specimens were RR by third- and fourth-generation IAs. By Multispot, 491 (99.6%) were HIV-1 positive and 2 (0.4%) were HIV-2 positive.

Only eight (40%) WB-indeterminate specimens were fourth-generation RR: 6 were Multispot and NAT negative and 2 were Multispot HIV-1 positive but NAT negative.

The alternative algorithm correctly classified as positive 102 seroconverter specimens with the third-generation IA and 130 with the fourth-generation IA compared with 56 using the WB with either IA.

Conclusions

The alternative testing algorithm improved early infection sensitivity and identified HIV-2 infections. Two potential false-positive algorithm results occurred with WB-indeterminate specimens.

INTRODUCTION

The current HIV testing algorithm, which was recommended by the Centers for Disease Control and Prevention (CDC) in 1989, indicates “no positive test results should be given to clients/patients until a screening immunoassay (IA) has been repeatedly reactive (RR) on the same specimen and a supplemental, more specific test such as the Western blot (WB) has been used to validate those results.”¹ The WB detects anti-HIV antibody in a human serum sample infected with HIV; however, it cannot detect acute infections (period prior to detectable antibody) which have been associated with a higher probability of disease transmission compared with established infections.^2–4 The HIV-1 WB also misclassifies many HIV-2 infections as HIV-1, which is problematic because HIV-2 infections do not respond to many first-line antiretroviral agents, including non-nucleoside reverse transcriptase inhibitors and some protease inhibitors.⁵ In 2010 an alternative laboratory HIV diagnostic testing algorithm was proposed⁶ (Figure 1) that is designed to detect early infections, reduce indeterminate results, and identify HIV-2 infections.^7–10 The alternative diagnostic algorithm involves screening with a sensitive fourth-generation antigen/antibody HIV-1/2 IA, or if unavailable, a third-generation HIV-1/2 IA. When the screening IA is repeatedly reactive, it is followed with an HIV-1/HIV-2 antibody differentiation test. If the differentiation test is reactive, the result is positive for either HIV-1 or 2 antibodies or both. However, when the HIV antibody differentiation test results are negative, an HIV-1 nucleic acid test (NAT) is used to resolve infection status. Persons with a positive NAT and a negative differentiation test are considered to have acute HIV-1 infection.

Figure 1.

Alternative laboratory HIV diagnostic testing algorithm

To date, HIV NAT has not been used widely for diagnosis due to its labor requirements, cost, and uncertainty about whether acute infections would be identified in certain populations.^{4, 11} Recently, the Food and Drug Administration (FDA) approved fourth-generation IAs that detect p24 antigen and HIV-1 and HIV-2 antibodies.¹² These assays have the ability to detect more than 80% of acute HIV infections otherwise detectable only by NAT.^13–15 The commercial availability of fourth-generation HIV-1/2 assays will make simultaneous screening for both acute and established HIV infections feasible for most clinical laboratories. However, the specificity of these screening tests must be evaluated in low prevalence settings because of cost implications associated with NAT to resolve false-positive fourth-generation IA screening test results. In this study we evaluated the performance of the FDA-approved fourth-generation assay, the GS HIV Combo Ag/Ab IA (Bio-Rad Laboratories, Redmond, WA),¹⁶ as part of the alternative laboratory HIV diagnostic testing algorithm compared to the current algorithm (RR third-generation IA/ HIV-1 WB). The evaluation was conducted using specimens from a low prevalence population, persons with established infections, and seroconverters.

METHODS

Specimens

Quest Diagnostics obtained three sets of de-identified residual serum/plasma specimens and processed them at their Lenexa, Kansas facility: (1) 10,014 specimens from life insurance applicants, a population that typically has low HIV prevalence (<0.1%)¹⁷; (2) 493 previously tested GS HIV-1 WB-positive specimens from life insurance applicants; (3) 20 previously tested GS HIV-1 WB-indeterminate specimens submitted for diagnostic testing. In addition, CDC obtained 230 serial plasma specimens from 26 U.S. donors early in the process of HIV seroconversion (presumably infected with subtype B) from BBI-SeraCare Diagnostics (West Bridgewater, MA) and Zeptometrix, Inc. (Buffalo, NY).

Assays

At Quest Diagnostics, all specimens (except those from the HIV seroconversion panels) were tested according to package insert instructions with both the GS HIV-1/HIV-2 Plus O third-generation IA¹⁸ and the GS HIV Combo Ag/Ab fourth-generation IA (Bio-Rad Laboratories, Redmond, WA) using the EVOLIS Automated Microplate System (Bio-Rad Laboratories). Specimens that were initially reactive with the third- or fourth-generation IA were tested in duplicate on the same assay, and specimens reactive on one or both of the repeat tests were considered to be repeatedly reactive. Specimens repeatedly reactive (RR) by third or fourth generation IA were tested with a rapid HIV-1/HIV-2 differentiation assay, Multispot HIV-1/HIV-2 Rapid test (Bio-Rad Laboratories, Redmond, WA),¹⁹ and if negative, by the APTIMA HIV-1 RNA Qualitative assay (Gen-Probe Incorporated, San Diego, CA).²⁰ Specimens with HIV-2 positive results on the differentiation assay were also tested with HIV-2 IA (Bio-Rad Laboratories, Redmond, WA) and research use only (RUO) QualiCode™ HIV-1/2 Western Blot Kit (QualiCode, Immunetics, Boston, MA).

At the CDC, specimens from the HIV seroconversion panels were tested manually by GS third- and fourth-generation IAs in singlet, Multispot and APTIMA. Because all specimens used in this study were unlinked from personal identifiers, this study was determined by the CDC to be research not involving identifiable human subjects.

Data management and analysis

Test results generated at Quest Diagnostics were recorded in an Excel database that was subsequently provided to CDC via secure data network. For specimens from low HIV prevalence life insurance applicants, we calculated the specificities of third- and fourth-generation IAs and used the Mid-P Exact Test to calculate 95% confidence intervals (CI). For WB-positive and WB-indeterminate specimens, we compared the results of the current and alternative laboratory HIV diagnostic testing algorithms. Samples from all specimen sets were considered to be from HIV infected individuals if they were either WB or NAT positive. Analyses were performed using SAS 9.2 (SAS Institute, Inc., Cary, NC, USA) statistical software.

For seroconversion panels, the analyses were conducted as described previously.⁷ Differences in the number of HIV-1 infections detected by the current and alternative laboratory HIV diagnostic algorithms were statistically analyzed using McNemar’s test with one degree of freedom and continuity correction.

RESULTS

Specimens from a Low Prevalence Population

Thirteen (0.13%) of 10,014 specimens from life insurance applicants were RR on at least one third- or fourth-generation IA (Table 1). Of these, two specimens RR by both third- and fourth-generation IAs were HIV-1 positive by WB and Multispot (i.e., considered HIV-1 positive by both algorithms). Two third-generation IA and 9 fourth-generation IA results were false-positive based on available testing data (Table 1): one specimen RR only by third-generation IA and 8 RR only by fourth-generation IA were negative by WB, Multispot and NAT; one specimen RR only by third-generation IA and another RR only by fourth-generation IA were WB-indeterminate and negative by Multispot and NAT. Specificities of third-generation (10,010/10,012) and fourth-generation IAs (10,003/10,012) were 99.98% (95% CI: 99.93–100%) and 99.91% (95% CI: 99.84–99.96%), respectively. Specificity values for both assays were within the confidence bounds reported within the product package inserts for low risk populations. No acute HIV infections were identified among specimens from these life insurance applicants.

Table 1.

HIV-1 Western blot, Multispot, and nucleic acid test (NAT) results of specimens repeatedly reactive (RR) by third- or fourth-generation immunoassays in a low prevalence population (10,014 insurance applicants)

Number of specimens	Third-generation RR	Fourth-generation RR	Western blot	Multispot	Nucleic acid test
2††	Yes	Yes	Positive	Positive	Not Done
1§	Yes	No	Negative	Negative	Negative
8†	No	Yes	Negative	Negative	Negative
1§	Yes	No	Indeterminate	Negative	Negative
1†	No	Yes	Indeterminate	Negative	Negative

^§HIV-1 uninfected, false-positive on third-generation IA;

^†HIV-1 uninfected, false-positive on fourth-generation IA;

^††HIV-1 positive

HIV-1 Western blot-Positive Specimens

All 493 HIV-1 WB-positive specimens were third- and fourth-generation RR (i.e., classified HIV-1 positive by current algorithm). With the alternative algorithm, 491 (99.6%) were HIV-1 positive by Multispot and 2 (0.4%) were HIV-2 positive by Multispot, HIV-2 IA, and HIV-2 WB.

HIV-1 Western blot-Indeterminate Specimens

Twelve (60%) of the 20 HIV-1 WB-indeterminate specimens were RR by third-generation IA, but not by fourth-generation IA, and were negative by Multispot and NAT (i.e., classified as HIV-negative by the alternative diagnostic testing algorithm). Eight (40%) of the 20 WB-indeterminate specimens were RR by third- and fourth-generation IA. Of these, six were negative by Multispot and NAT (i.e., classified as negative by the alternative algorithm), and 2 were reactive by Multispot (i.e., classified as antibody-positive by the alternative algorithm) but NAT-negative. One of these two HIV-1 WB-indeterminate specimens had p24 and p55 WB bands present, and the other had a gp160 band.

Specimens from HIV-1 Seroconverters

The third -generation IA was reactive with 102 seroconverter specimens and the fourth -generation IA with 131 (Table 2a). The use of Multispot as a supplemental test improved the correct classification of HIV-1 infections compared with the WB regardless of the screening IA used. A total of 90 specimens were positive using Multispot as the supplemental test, whereas only 56 were confirmed positive by the WB (Table 2a). Two seroconverters with reactive NAT results at earlier bleeds had a total of four subsequent specimens that were negative by NAT but positive by Multispot and positive (n=3) or indeterminate (n=1) by WB. Multispot was negative in 12 seroconverter specimens that were reactive by third-generation IA and 41 that were reactive by fourth-generation IA. Use of the entire alternative algorithm including NAT further improved detection of HIV-1 infections compared with the WB, and correctly classified as positive 102 seroconverter specimens with the third-generation IA and 130 with the fourth-generation IA compared with 56 using the current algorithm with either IA (Table 2.b).

Table 2.

a. Performance of third- and fourth-generation IA, and the current and alternative laboratory HIV testing algorithms among 230 specimens from HIV-1 seroconverters
		Multispot		Western blot
		NR	R	negative	indeterminate	positive
Third-generation screening
R	n= 102	12	90	11	35	56
NAT negative		0	4†	0	1†	3†
NAT positive		12	86	11	34	53
NR	n= 128	128	0	n/d	n/d	n/d
Fourth-generation screening
R	n=131	41	90	36	39	56
NAT negative		1	4†	1	1†	3†
NAT positive		40	86	35	38	53
NR	n=99	99	0	n/d	n/d	n/d

b. Number of specimens detected by the current and alternative laboratory HIV testing algorithm among 26 HIV-1 seroconverters (230 specimens)
Screening IA	MS/NAT	WB
Third-generation	102* **	56**
Fourth-generation	130* **	56**

^†Same specimens, and had been obtained subsequent to NAT-positive specimens from the same seroconverters

The table shows the number of specimens detected by the combination of different tests in the context of the current and alternative laboratory.

HIV testing algorithm including nucleic acid testing (NAT); n: number; n/d: not determined; NR: non-reactive; R: reactive; NAT: nucleic acid test

^*p< 0.001 (102 vs. 130);

^**p< 0.001 (102 vs. 56 and 130 vs. 56).

MS: Multispot; NAT: nucleic acid test; WB: Western blot*

DISCUSSION

The GS HIV Combo Ag/Ab fourth-generation IA reduced the diagnostic window compared with the third-generation IA,^{7, 15} enabling detection of acute HIV infections only otherwise detected by amplified RNA methods. This screening assay also performed with high specificity in a low HIV prevalence population.²¹ Similar performance characteristics have been observed in studies of the only other FDA-approved fourth-generation IA on the U.S. market, the Abbott ARCHITECT HIV Ag/Ab Combo.¹⁴ In the current study, the alternative laboratory HIV testing algorithm using the GS HIV Combo Ag/Ab fourth-generation IA not only correctly resolved the infection status of specimens from individuals with established HIV infections that had been detected using the HIV-1 WB assay, but also identified 2 HIV-2 infections in specimens misclassified as HIV-1 by WB.

When used as part of the alternative algorithm in populations with persons at high risk of HIV during their early infection period, fourth-generation IAs will enable testing programs to detect HIV infections earlier. Even when a third-generation IA is used as the initial test in the alternative algorithm, more early infections will be detected than in the current algorithm when a third-generation IA is used with WB. Individuals in the acute stage of HIV infection are usually highly viremic, resulting in more virus shedding at mucosal sites, and are therefore more likely to transmit HIV infection through bodily secretions.⁴ Estimates of the proportion of total HIV transmissions attributable to early infections range widely from 6% to 49.4%.^{3, 22, 23} To use the alternate algorithm to its full advantage to detect early infections and reduce transmissions, laboratories must be prepared to conduct NAT testing with rapid turn-around and public health systems must be in place to conduct timely linkage to medical care and partner services.¹¹ Detection of early HIV infection may also play a key role in the success of treatment as prevention by offering available antiretroviral drugs to high-risk people before or immediately after HIV exposure or as a means of secondary prevention.²⁴

The Bio-Rad GS fourth-generation IA demonstrated high specificity in a very low-prevalence population. The specificity is similar to that obtained with third-generation IAs.²⁵ However, with the WB-indeterminate specimens, more false-positive results appeared to occur with third-generation IA compared with the fourth-generation IA, potentially due to cross reaction with the p24 antigen present in the third-generation assay, but not the fourth. This study demonstrated that few NATs will be required to resolve specimens with false-positive screening assay results when the alternative HIV diagnostic testing algorithm is utilized. Further, in our study, the alternative algorithm correctly identified the HIV infection status among all HIV-1 WB-positives, highlighting its utility in a variety of settings in the United States. In addition, even though HIV-2 is rare in the United States,²⁶ the alternative algorithm identified HIV-2 infections that would otherwise have been misclassified as HIV-1 infections by the HIV-1 WB, as has been noted in other studies.^{10, 27} With the availability of supplemental testing technologies that differentiate HIV-1 from HIV-2 and widespread use of the alternative algorithm, it is likely that more HIV-2 infections will be recognized.

The alternative HIV diagnostic testing algorithm has previously been shown to work well in high-risk individuals⁸ and in persons with established HIV-1 infection when used with a third-generation screening IA.⁹ In this study of over 10,000 specimens, excluding the seroconversion specimens, the alternative algorithm correctly resolved the infection status of all specimens except two. These two specimens were RR by third- and fourth-generation IA, and HIV-1 WB-indeterminate, Multispot HIV-1 reactive and NAT-negative, indicating discordance between serologic and virologic markers of infection. These specimens highlight the difficulty of resolving the true infection status without follow-up testing with a subsequent sample. It is possible that they represent false-positive results by the alternative algorithm. Alternatively, they might be similar to the four study specimens from seroconverters that were HIV-1 positive by serology but NAT-negative during follow-up. Other studies have shown that 3–5% of HIV-1 antibody-positive specimens^{28, 29} are negative by NAT. Persons classified as HIV antibody positive by the alternative algorithm (positive third- or fourth-generation IA and Multispot results) who have a negative NAT after entering care will need further diagnostic testing.³⁰

There are several theoretical limitations associated with these analyses. First, factors beyond assay performance may have been responsible for the limited number of false-positive and false-negative results. False-positive results may have occurred due to contamination of samples or specimen mix up, and false-negative NAT results in stored, frozen specimens tested retrospectively may have happened due to degradation of viral RNA by microbes or multiple freeze-thaws.^31–33 Second, for the HIV-1 WB-indeterminate specimens, we did not have the ability to obtain follow-up samples and thus the ultimate infection status is not known with certainty. Third, even though our study demonstrated the excellent performance of the alternative laboratory HIV diagnostic testing algorithm in selected specimen sets, we did not assess its performance with known HIV-2 positive specimens or in an actual population at high risk for acute infection. Finally, although the performance of the alternative diagnostic testing algorithm was assessed in this study using APTIMA, the only NAT approved by the FDA for diagnosis, it may be of additional benefit to assess the performance of quantitative HIV RNA assays, which are used widely in clinical settings for therapeutic monitoring.³⁰

In conclusion, the Bio-Rad fourth-generation HIV-1/2 IA demonstrated high sensitivity for both early and established infections, and high specificity in a low HIV prevalence population. When coupled with the alternative supplemental tests, testing jurisdictions will gain the ability to corroborate reactive screening test results for early infections, identify HIV-2 infections, and reduce indeterminate results. These features make this algorithm a good alternative to the current HIV testing algorithm, which utilizes WB or immunofluorescence assay (IFA) as the definitive supplemental test. However, as with any diagnostic testing, low numbers of false-positive results may occur that will warrant additional diagnostic testing.