Utility of the Signal-to-Cutoff Ratio and Antigen Index from Fourth- and Fifth-Generation HIV-1/HIV-2 Antigen/Antibody Assays for the Diagnosis of Acute HIV Infection: Applicability for Real-Time Use for Immediate Initiation of Antiretroviral Therapy

ABSTRACT

Identification of individuals with acute HIV infection (AHI) and rapid initiation of antiretroviral therapy (ART) are priorities for HIV elimination efforts. Fourth- and fifth-generation HIV-1/HIV-2 antigen (Ag)/antibody (Ab) combination assays can quickly identify patients with AHI, but false-positive results can occur. Confirmatory nucleic acid amplification testing (NAAT) may not be rapidly available. We reviewed the data for 127 patients with positive fourth-generation ARCHITECT and fifth-generation Bio-Plex immunoassay results who had negative or indeterminate confirmatory Ab testing results, which yielded 38 patients with confirmed AHI and 89 patients with false-positive results. The receiver operating characteristic (ROC) curves showed excellent discriminatory power, with an area under the curve (AUC) for the signal-to-cutoff (S/CO) ratio of 0.970 (95% confidence interval [CI], 0.935 to 1.00) and an AUC for the Ag index (AI) of 0.968 (95% CI, 0.904 to 1.00). A threshold of 3.78 for the S/CO ratio would maximize the sensitivity (96.3%) and specificity (93.4%). The threshold for AI was 2.83 (sensitivity of 100% and specificity of 96.4%). The S/CO ratio was significantly correlated with the viral load (Spearman correlation coefficient, 0.486 [P = 0.014]), but the AI was not. The viral loads were all high, with a median of >2.8 million copies/mL. Two false-positive results with AI and S/CO ratio values markedly higher than the medians were observed, indicating that biological false-positive results can occur. Review of the S/CO ratio or AI may be used to improve the accuracy of AHI diagnosis prior to confirmatory NAAT results being available.

INTRODUCTION

Identification of patients with acute HIV infection (AHI) and rapid initiation of antiretroviral therapy (ART) have become a priority for HIV screening and linkage-to-care programs. Fourth- and fifth-generation HIV combination antigen (Ag)/antibody (Ab) assays have enhanced the ability to diagnose these patients, and this is the rationale for the current testing algorithm endorsed by the CDC (1–4). Despite high specificities, these tests may yield false-positive results, especially when screening low-prevalence populations. Confirmatory tests for anti-HIV-1 and anti-HIV-2 Abs are very specific and can be completed rapidly. However, for patients with positive fourth- or fifth-generation HIV-1/HIV-2 combination Ag/Ab assay results and negative or indeterminate supplemental Ab testing results, quantitative HIV-1 RNA nucleic acid amplification testing (NAAT) is the confirmatory test (5). The availability and/or rapid turnaround time of reflex confirmatory NAAT varies across institutions. Additionally, we reported that the turnaround time for NAAT at the University of Chicago Medicine (UCM) was adversely impacted by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic, with shortages of supplies and reagents and increased workload demands on the clinical microbiology laboratory (6). Other sites, including Rush University Medical Center (RUMC), have seen similar delays (Sha B.E. and Moore N.M., personal communication). Although fourth- and fifth-generation assays have been approved only as qualitative tests, there has been interest regarding whether additional quantitative information, specifically, the signal-to-cutoff (S/CO) ratio or the Ag index (AI) from the fourth- and fifth-generation assays, respectively, can be used to determine whether the patient truly has HIV infection (5).

Other studies have suggested a relationship between the signal intensity with HIV combination Ag/Ab assays and confirmation of HIV infection with supplemental Ab testing and quantitative PCR. Jensen and colleagues reported on the ARCHITECT HIV combination Ag/Ab assay (Abbott Laboratories, Abbott Park, IL, USA), which can detect HIV-1 Abs, HIV-2 Abs, and HIV-1 p24 Ag (7). They were able to show that the positive predictive value of a true-positive result improved as the S/CO ratio increased. Their study, however, did not look specifically at patients with negative or indeterminate supplemental Ab testing results, i.e., patients with a false-positive result or true AHI. The S/CO ratio for these immunoassays has also been shown to accurately identify recent seroconverters (8). There are few data, however, on the accuracy of the S/CO ratio or AI from fourth- and fifth-generation HIV-1/HIV-2 Ag/Ab combination assays for the diagnosis of AHI before the results of reflex NAAT are available, and AHI cases represent the highest priority group for which immediate ART may be indicated even before the confirmatory PCR test results are available. A study by Ramos et al. looked at the S/CO ratio for the ARCHITECT HIV Ag/Ab combination assay to distinguish acute infection from established infection (9). They retrospectively looked at clinical test results from 2011 to 2013 and reported that the S/CO ratio for the assay accurately identified 15 patients with acute infection among 83 Ab-negative specimens. Despite that report, the S/CO ratio is not routinely used to establish the diagnosis of AHI in real time to make decisions about rapid initiation of ART.

The Bio-Plex 2200 HIV Ag/Ab assay (Bio-Rad, Hercules, CA USA) can differentiate between a positive result due to HIV-1 Abs, HIV-2 Abs, or HIV-1 p24 Ag, which is the definition of a fifth-generation assay. A positive Abbott ARCHITECT fourth-generation assay result does not specify whether the HIV-1 Abs, HIV-2 Abs, or HIV-1 p24 Ag is being detected. Thus, when the confirmatory HIV 1/2 Ab assay result is negative or indeterminate, the S/CO ratio may serve as a surrogate for HIV-1 p24 Ag testing. Here, we report that the AI value for the HIV-1 p24 Ag component of the Bio-Plex assay and the S/CO ratio for the ARCHITECT assay may be used to identify patients with AHI. We reviewed the positive results for fourth- and fifth-generation assays performed prospectively for patients as part of expanded HIV testing and linkage-to-care programs at UCM, RUMC, and Rush Oak Park Hospital (ROPH). We compared the AI of the HIV-1 p24 Ag component of the Bio-Plex assay and the S/CO ratio of the ARCHITECT assay for patients with negative or indeterminate confirmatory HIV-1/2 Ab test results and AHI confirmed by reflex HIV-1 RNA NAAT to those for patients with false-positive results, to assess whether these values could accurately distinguish between these patient groups; this would be useful in clinical decision-making regarding rapid initiation of ART prior to the availability of NAAT results.

MATERIALS AND METHODS

UCM, RUMC, and ROPH participate in an expanded HIV testing and linkage-to-care program supported by the Chicago Department of Public Health. The sites perform routine HIV screening in health care settings, including emergency departments (EDs), according to CDC recommendations (10–12). We performed a retrospective analysis of data for fourth- and fifth-generation HIV Ag/Ab assays that were repeatedly reactive but were nonreactive or indeterminate with a supplemental HIV 1/2 Ab assay. At UCM, the Bio-Plex 2200 HIV Ag/Ab assay was utilized and positive results were confirmed with the Geenius HIV 1/2 Ab assay between April 2018 and November 2019. At RUMC and ROPH, the ARCHITECT HIV Ag/Ab assay was utilized and positive results were confirmed with the Multispot HIV 1/2 Ab assay (Bio-Rad) between May 2015 and February 2017 and then with the Geenius assay between February 2017 and October 2021. For both the Bio-Plex and ARCHITECT HIV Ag/Ab assays, an AI or S/CO ratio of >1.00 is considered to indicate a reactive result. Quantitative HIV-1 RNA NAAT was performed on these reactive samples; the COBAS AmpliPrep/COBAS TaqMan HIV-1 PCR assay (Roche Diagnostics, Indianapolis, IN) was used at UCM, and the Abbott real-time HIV-1 RNA assay (Abbott Molecular, Des Plaines, IL) was used at RUMC and ROPH. Samples were classified as true-positive AHI if results were confirmed by a positive NAAT result or as a false-positive result if no HIV-1 RNA was detected. For each sample, if the initial screening result was reactive, then retesting was performed in duplicate according to each manufacturer’s package insert. At UCM, the reported AI values were averaged across replicates for each sample. At RUMC/ROPH, the first S/CO ratio was used.

Results are reported as the median and interquartile range (IQR). The Mann-Whitney-Wilcoxon test was used to compare the AI and S/CO ratio values between the patients with AHI and those with false-positive results. Receiver operating characteristic (ROC) curve analysis was used to determine the thresholds that optimized the sensitivity and specificity of these assays. The relationship between AI and S/CO ratio values and quantitative HIV-1 RNA NAAT results (i.e., viral load) were analyzed using the Spearman correlation coefficient. All analyses were completed using R version 4.0, and P values of <0.05 were considered statistically significant.

All procedures performed were in accordance with the ethical standards of our institutions. The UCM and RUMC/ROPH institutional review boards (IRBs) deemed the work to be exempt from IRB oversight.

RESULTS

Despite testing with different platforms, the results from RUMC/ROPH and UCM were quite similar. Table 1 shows the demographic features of the patients tested, according to institution, testing location, and HIV infection status. AHI was almost exclusively identified in the ED (37/38 patients). One inpatient at UCM was diagnosed with AHI. No outpatients were diagnosed with AHI. The RUMC/ROPH data set included 88 samples, 27 from patients with AHI and 61 false-positive samples. One patient was lost to follow-up monitoring, and his HIV status could not be confirmed by further testing. That patient presented with abdominal pain, engaged in receptive anal sex, and had a high S/CO ratio (42.52) but a negative confirmatory Ab test result and an initially negative HIV-1 RNA NAAT result. Repeat NAAT on the same sample was performed due to concern regarding AHI and results were positive, but the viral load was <40 copies/mL. There was no evidence that this undomiciled patient was receiving preexposure or postexposure HIV prophylaxis. It is also unlikely that he was HIV positive and receiving suppressive ART with loss of Abs. The individual was included with the false-positive results for the analysis. The false-positive samples had a median S/CO ratio of 1.61 (IQR, 1.25 to 2.06), compared to a median of 23.84 (IQR, 9.48 to 50.32) for the AHI samples (P < 0.001) (Fig. 1A). The ROC curve indicated that a S/CO ratio of 3.78 would maximize the sensitivity (96.3%) and specificity (93.4%), with an area under the curve (AUC) of 0.970 (95% confidence interval [CI], 0.935 to 1.00) (Fig. 2A). The median viral load for the patients with AHI was 6.54 log₁₀ copies/mL (IQR, 6.14 to 6.98 log₁₀ copies/mL). There was a significant correlation between the S/CO ratio and the viral load (Spearman correlation coefficient, 0.486 [P = 0.014]) (Fig. 3A).

TABLE 1.

Demographic features according to institution, testing location, and HIV infection status

Institution and parameter	Data for testing location of:				Data for infection status of:
	ED	Inpatient	Outpatient	Total	AHI	False-positive result	Total
RUMC/ROPH
No. of patients	63	0	25	88	27	61	88
Sex at birth (%)
Female	38.1		72	47.7	11.1	63.9	47.7
Male	61.9		28	52.3	88.9	36.1	52.3
Race/ethnicity (%)
Black, non-Hispanic	57.1		40	52.3	63	47.5	52.3
White, non-Hispanic	15.9		24	18.2	18.5	18	18.2
Hispanic	20.6		20	20.5	14.8	23	20.5
Other	6.3		16	9.10	3.7	11.5	9.1
Age (median [range]) (yr)	31 (16–64)		30 (3–68)	30.5 (3–68)	27 (17–59)	32 (3–68)	30.5 (3–68)
HIV status (%)
AHI	42.9		0	30.7
False-positive result	57.1		100	69.3
Testing location (%)
ED					100	59	71.6
Outpatient					0	41	28.4
UCM
No. of patients	31	3	5	39	11	28	39
Sex at birth (%)
Female	45.2	66.7	60	48.7	18.2	60.7	48.7
Male	54.8	33.3	40	51.3	81.8	39.3	51.3
Race/ethnicity (%)
Black, non-Hispanic	87.1	33.3	60	79.5	81.8	78.6	79.5
White, non-Hispanic	3.2	0	0	2.6	0	3.6	2.6
Hispanic	6.5	0	20	7.7	9.1	7.1	7.7
Other	3.2	66.7	20	10.3	9.1	10.7	10.3
Age (median [range]) (yr)	31 (17–77)	29 (15–71)	28 (24–72)	29 (15–77)	24 (15–47)	31.5 (18–77)	29 (15–77)
HIV status (%)
AHI	32.3	33.3	0	28.2
False-positive result	67.7	66.7	100	71.8
Testing location (%)
ED					90.9	75	79.5
Inpatient					9.1	7.1	7.7
Outpatient					0	17.9	12.8

FIG 1.

Box plots with median values, IQRs, and whiskers extending from the first and third quartiles to the smallest and largest points, respectively, within 1.5× IQR. (A) ARCHITECT assay S/CO ratios for patients with AHI and patients with false-positive results. (B) Bio-Plex assay AI values for patients with AHI and patients with false-positive results.

FIG 2.

(A) ROC curve for the ARCHITECT assay S/CO ratio, with the AUC and optimal threshold maximizing sensitivity and specificity. (B) ROC curve for the Bio-Plex assay AI, with the AUC and optimal threshold maximizing sensitivity and specificity.

FIG 3.

(A) Scatterplot of the relationship between ARCHITECT assay S/CO ratios and HIV-1 RNA levels. (B) Scatterplot of the relationship between Bio-Plex assay AI values and HIV-1 RNA levels.

For UCM, there were 11 patients with AHI and 28 with false-positive results. The false-positive samples had a median AI of 1.07 (IQR, 1.02 to 1.27), compared to a median of 6.43 (IQR, 5.05 to 10.37) for the AHI samples (P < 0.001) (Fig. 1B). The ROC curve for this cohort had an AUC of 0.968 (95% CI, 0.904 to 1.00), and a threshold of 2.83 resulted in sensitivity of 100.0% and specificity of 96.4% (Fig. 2B). The median viral load for the patients with AHI was 6.46 (IQR, 6.10 to 6.70 log₁₀ copies/mL), but there was no significant correlation between AI and viral load for this cohort (Spearman correlation coefficient, 0.150 [P = 0.66]) (Fig. 3B). There was one patient with a false-positive result who had a high AI (19.11). He was elderly (>65 years of age) and had widely metastatic prostate cancer, with a serum prostate-specific Ag (PSA) level of 2,595 ng/mL (normal level, ≤4 ng/mL).

DISCUSSION

The diagnosis of AHI has been enhanced using combination HIV Ag/Ab assays with AHI representing 15% of new HIV diagnoses in some EDs. This is a priority population for public health due to the role of acute and early infection in the sexual transmission of HIV (13, 14). We have also reported on our experience with the diagnosis of AHI in our expanded HIV testing and linkage-to-care program (15). It is feasible to rapidly initiate ART without additional resources in programs that have an established testing program. It is even feasible to initiate ART in real time in the ED based on a positive HIV-1/HIV-2 Ag/Ab combination assay result (16). However, there is a possibility of a false-positive result, especially with routine screening in low-prevalence settings or among low-risk individuals.

The clinical symptoms and signs of acute and recent HIV infection have been well described. Review of the clinical presentation and patient demographics may help improve the pretest probability of AHI, and some investigators have developed criteria and scoring tools for predicting the likelihood of AHI based on demographics, risk factors for HIV, and clinical features (17–19). The specificity of the clinical symptoms and signs at presentation or just prior to presentation has been studied but most often in men who have sex with men (MSM). The San Diego Symptom Score (SDSS) was derived from a study cohort in the San Diego Early Test screening program (19). Their summed scores for fever, myalgia, and weight loss of ≥2.5 kg yielded an ROC curve with an AUC of 0.851 (95% CI, 0.780 to 0.922), which was superior to a risk-based scoring system for the MSM subset of patients. Although that cohort did include non-MSM participants (261 of 998 participants [26.1%]), the single-site population composed mainly of White MSM might not be representative of other sites. Other studies have reported a wide range of specificities of clinical symptoms and signs (38% to 91%) (20). One study indicated a specificity of 65% for patients presenting with influenza-like illness (21). The symptoms of AHI, however, are often nonspecific and can overlap those of several acute viral infections, such as infectious mononucleosis and viral respiratory infections, including SARS-CoV-2 infection (22). This would mean that symptom scoring during respiratory virus season, or now during the SARS-CoV-2 pandemic, would likely be much less specific.

Our study supports review of the S/CO ratio or AI of the fourth- and fifth-generation HIV Ag/Ab assays on the ARCHITECT and Bio-Plex platforms to test for AHI. The ROC curves with AUC values of ≥0.968 suggest that these values are more accurate than symptom scoring. Patients with false-positive results on these assays usually have low S/CO ratio or AI values, and the ROC curves for the two assays were very similar, at least with respect to the threshold values maximizing sensitivity and specificity. We did have 2 patients with high S/CO ratios or AI values who had biological false-positive results. While uncommon, these false-positive results significantly above the median value are concerning and emphasize the need for confirmatory HIV NAAT. If ART is administered to a patient with a false-positive S/CO ratio result, then it can be discontinued within a few days when the viral load result is available, and brief therapy should not have adverse effects. Still, an inaccurate diagnosis of HIV has a few adverse effects other than the drug exposure and should be avoided if possible. Although it would be optimal for each testing site to analyze their own data to define an optimal AI or S/CO threshold, our data may be used as a starting point for programs that wish to use the AI or S/CO ratio. As our HIV testing programs evolved, we aggressively targeted outreach to ED patients with high S/CO or AI values to perform HIV NAAT if such testing could not be added to the initial blood draw.

The causes of false-positive fourth- and fifth-generation test results are not entirely defined. They may just reflect the performance characteristics of the tests, especially when the false-positive result is just above the threshold for a positive result. Laboratory errors and technical issues may also occur. There are, however, rare biological false-positive results. Such results are likely due to a diverse group of conditions. There are limited data on these biological false-positive results, but Liu et al. did report on 2 patients, both with neoplasms, with three false-positive test results using the ARCHITECT fourth-generation assay (23). Lee et al. reported on 34 instances of false-positive results with fourth-generation testing with the ADVIA Centaur HIV Ag/Ab assay (24). Patients with false-positive results had a variety of concurrent conditions, including autoimmune diseases, infections (including viral hepatitis and tuberculosis), cancer, and multiparity. There is also a theoretical possibility of infection with other retroviruses (e.g., human T-cell lymphotropic virus 1 [HTLV-1] and HTLV-2) that express p24 Ag. More recently, there have been rare reports of false-positive results for patients with SARS-CoV-2 infection (22). Thus, risk factor and clinical assessments remain important, and the need for follow-up HIV NAAT remains critical for definitively classifying AHI versus false-positive immunoassay results.

While our sample size of 127 patients was small, the 38 patients with AHI we had represent one of the larger data sets assessing the utility of the S/CO ratio and AI for differentiating false-positive results from AHI. Because the Bio-Plex and Abbott ARCHITECT Ag/Ab assays detect only HIV-1 groups M and O, our results may not be generalizable to parts of the world such as West Africa, where other HIV-1 groups circulate. While our patient selection for HIV testing was not random, all patients tested met CDC criteria for HIV screening.

Conclusion.

The use of the AI or S/CO ratio for p24 Ag in the combination HIV Ag/Ab assays can be helpful in identifying patients with AHI even before the HIV NAAT results are available. A quantitative HIV-1 RNA assay is needed to definitively differentiate false-positive HIV Ag/Ab assay results from AHI when the confirmatory HIV 1/2 Ab assay results are negative. This study provides further evidence that the S/CO ratio and AI from fourth- and fifth-generation HIV-1/HIV-2 Ag/Ab combination assays may be used to help guide early initiation of ART for patients with negative confirmatory HIV 1/2 Ab test results. Sites with expanded HIV testing and linkage-to-care programs, especially those in EDs, should utilize the S/CO or AI to help make decisions regarding initiation of ART prior to the results of NAAT when the confirmatory Ab testing results are negative or indeterminate.