The VACS Index Predicts Mortality In A Young, Healthy HIV Population Starting Highly Active Antiretroviral Therapy

Abstract

Background

The Veterans Aging Cohort Study (VACS) Index is a weighted combination of age and eight clinical variables. It has been well correlated with all-cause mortality among HIV-infected patients. The U.S. Military HIV Natural History Study (NHS) cohort provides a different validation population profile, being younger and healthier. A significant portion of the US HIV population is similarly composed, so evaluation of the VACS Index in this population is of great interest.

Methods

NHS subjects have medical history and laboratory data collected at six month visits. We performed an external validation of the VACS Index in the NHS evaluating correlation, discrimination, and calibration for all-cause mortality following HAART initiation (HI). We then tested whether combining longitudinal VACS Index values at different time points improves prediction of mortality.

Results

The VACS Index at one year after HI was well correlated with all-cause mortality (Harrell’s c-statistic 0.78), provided good discrimination (log-rank p <0.05), and was marginally well calibrated using Brier score. Accounting for VACS Index at HI and 6 months after HI significantly improved a standard model including only the VACS Index at 1 year after HI (Net Reclassification Improvement=25.2%, 10.9-48.9% 95% CI).

Conclusions

The VACS Index was well correlated and provided good discrimination with respect to all-cause mortality among HAART initiating subjects in the NHS. Moderate overprediction of mortality in this young, healthy population suggests minor recalibration could improve fit among similar patients. Considering VACS Index at HI and 6 months improved outcome prediction and allowed earlier risk assessment.

Introduction

With the success of highly active antiretroviral therapy (HAART), concerns regarding HIV infection have shifted toward comorbid illness, longer term outcomes, and aging, although premature death remains the most serious concern. Prognostic tools to help identify those at risk of these adverse outcomes have been developed and several have been validated, usually in study cohorts comprised of subsets of the general population, for example military service veterans, inner city gay men, intravenous drug users, and others. As HIV infection in the U.S. has spread broadly, evaluation of such tools among a population of young, otherwise healthy, ethnically diverse, physically active individuals is important to assess the generalizability of these models. The U.S. Military HIV Natural History Study (NHS) cohort is primarily composed of active duty service members found to be HIV infected upon routine screening and thus, have early diagnosis and entry to care. Healthcare is a military benefit provided along with medications at no cost to the individual. Active duty service is also associated with relatively stable income/socioeconomic status, education at a high school level or above, and near absence of IV drug use.

The VACS Index has been validated among many cohorts as an excellent predictor of all-cause mortality when applied to any single time point from one to five years after initiation of HAART in HIV infected individuals ^1,2 and strongly associated with biomarkers of inflammation ^3-6. However, the majority of subjects in these cohorts have been older or with a high prevalence of comorbidities. The Index is comprised of measures capturing standard HIV associated mortality risk (age, CD4, HIV viral load), as well as those identifying comorbidity and organ system dysfunction (hemoglobin, aspartate aminotransferase (AST), alanine aminotransferase (ALT), platelet count, and creatinine levels), and hepatitis C virus serostatus. Evaluation of the Index has yet to take into account previous VACS Index scores.

Our purpose was two-fold. We first sought to evaluate the predictive utility of the VACS Index for death in a young, otherwise health military HIV population. Secondly, we investigated whether considering VACS Index values at prior time points provides a better prediction of the five year mortality risk given the VACS Index value at one year.

Methods

Participants

The U.S. Military HIV Natural History Study (NHS) is a prospective continuous enrollment cohort study of consenting military beneficiaries with HIV infection including active duty personnel, retirees, and dependents ⁷. Active duty personnel must be HIV negative prior to entry into U.S. military service and subsequently undergo routine HIV screening every one to five years. Those identified with HIV infection are referred to military medical centers for evaluation and treatment and are invited to enroll in the NHS. Study visits occur approximately every six months when data including demographics, medical history, medications, and laboratory measurements are collected. The NHS has been IRB approved at all participating sites and all subjects provide written informed consent.

Definitions and inclusion criteria

Subjects in the NHS who initiated HAART between 1996 and 2011 and had available age and all components of the VACS Index at HAART initiation were included in these analyses. The VACS Index combines traditional factors associated with HIV outcome including age, CD4, HIV viral load, as well as additional clinical factors including hemoglobin, FIB-4, estimated glomerular filtration rate (eGFR), and hepatitis C virus serostatus, with a total possible score ranging from 0 to 164. VACS Index was calculated according to the method of Justice, et al. ². Briefly, clinically relevant variables associated with mortality among those with HIV infection are assigned points by value category, e.g. hemoglobin (values in g/dL) ≥14=0 pts, 12-13.9=10 pts, 10-11.9=22 pts, <10=38 pts, and Fib-4, a measure of liver fibrosis, <1.45=0 pts, 1.45-3.25=6 pts, >3.25=25 pts.

Deaths among NHS subjects are actively ascertained through annual search of a number of U.S. national databases including the National Death Index (NDI), Social Security Death Index (SSDI), Department of Defense (DoD), Veterans Administration (VA), and others. Those not known to be deceased were classified as living at the end of the 2011.

Statistical Analysis

The VACS Index was evaluated for each subject at HAART initiation (HI), 6 months after HI (6M), and one year after HI (1Y). The predictive value of the VACS Index was evaluated with respect to two outcomes: time to death (continuous variable), and five year mortality (binary variable) in those with at least five years of potential follow-up.

The external validation was conducted as follows ^8,9. Correlation between the VACS Index and the time to death was assessed using Harrell’s c-statistics. Discrimination was tested by evaluating differences in time to death or five year mortality among the VACS Index tertiles using the log-rank test and Greenwood’s formula after log-log transformation of the Kaplan-Meier survival curve respectively. Finally, calibration of the model was evaluated by comparing the observed and predicted five year mortality at HI, 6M and 1Y.

We then assessed whether there was additional predictive value by combining VACS Index information from the earlier two time points (HI and 6M) with that from the 1Y time point for both time-to-death and five year mortality. Only subjects with VACS Index values at all three time points were included in the analysis and multiple imputations were used to evaluate the effect of missing data. Cox models were employed, and model performance was evaluated in terms of goodness-of-fit (R-squared), statistical significance (p-values), Akaike information criteria (AIC), and ability to reclassify subjects in given risk groups (net reclassification improvement, NRI). Inverse probability weighting of the cases and controls was used to account for censoring when evaluating the NRI ^10,11.

Additional analyses (details included in the supplement) were performed to assess the sensitivity of our findings to the model considered. One approach evaluated a Cox model using principal components of the VACS index values at the three time points. Another approach categorized subjects into low and high risk groups according to their scores at HI and 6M. Results of the analyses were not sensitive to the approach used, and the more robust Cox model is presented herein.

Results

Baseline characteristics

Baseline characteristics for all subjects by VACS Index tertiles are shown in Table 1. Out of the 1659 subjects, 92% were male, 40% Caucasian, 45% African-American, and 14% Hispanic or other. Median CD4 count at HI was 332 (IQR 223-451), while median age was 34 (IQR 28-40). Those in the lowest tertile (<14) were more likely to be male and Caucasian. Those in the highest tertile (>23) were more likely to be female and/or African-American.

Table 1.

Baseline characteristics by tertiles of the VACS index at HAART Initiation, median (interquartile range) or number (percent).

		VACS Index at HAART Initiation, score tertiles
	Total	< 14	14 - 23	> 23
N	1659	664	478	517
Age	34 (28,40)	33 (27,38)	33 (28,38)	37 (31,45)
Gender
Male	1531 (92%)	654 (98%)	427 (89%)	450 (87%)
Female	128 (8%)	10 (2%)	51 (11%)	67 (13%)
Race
Caucasian	668 (40%)	314 (47%)	171 (36%)	183 (35%)
African-American	753 (45%)	240 (36%)	234 (49%)	279 (54%)
Hispanic/Other	238 (14%)	110 (17%)	73 (15%)	55 (11%)
CD4 (cells/μL)	332 (223,451)	392 (304,508)	356 (264,460)	182 (69,320)
VL (log10 copies/mL)	4.5 (3.8,5)	4.1 (3.4,4.6)	4.5 (3.9,5)	5 (4.3,5.4)
Hgb (g/dL)	14.3(13.3,15.2)	15.0 (14.5,15.7)	14.1 (13.4,15)	13.2 (11.9,13.9)
Platelets (×106/mL)	204 (169,242)	211 (180,245)	211 (172,246)	187 (148,231)
AST (IU/L)	31 (24,41)	29 (24,36)	30 (24,40)	36 (27,55)
ALT (IU/L)	33 (24,48)	32 (23,43)	32 (23,48)	36 (25,62)
Fib-4	0.92 (0.67,1.3)	0.81 (0.61,1.0)	0.88 (0.66,1.2)	1.3 (0.84,2.0)
Fib-4 >3.25	55 (3.3%)	0 (0%)	0 (0%)	55 (10.6%)
eGFR (mL/min)	99 (88,113)	99 (88,111)	99 (88,113)	100 (86,115)
HCV +	86 (5.2%)	2 (2.3%)	17 (19.8%)	67 (77.9%)

Abbreviations: Hgb, hemoglobin, ALT, alanine aminotransferase; AST, aspartate aminotransferase; eGFR, estimated glomerular filtration rate, HCV, hepatitis C virus.

External validation

Among the 1659 included subjects, the VACS Index was also calculated for 1594 at 6 months after HI, and 1724 at one year after HI, while 1089 subjects had VACS Index values at all three time points. Subjects without a calculated VACS Index were most often missing only one of the necessary components.

There were 176 deaths among the subjects with a VACS Index at HI, 145 deaths among the subjects with a VACS Index at 6M and 155 among those with a VACS Index at one year after HI, while 86 deaths were observed among the 1089 subjects with VACS Index values at all three time points. For the five year mortality (binary) endpoint, there were 83, 67, and 77 deaths among the subjects with a VACS Index value at HI, 6M, and 1Y respectively, and 40 deaths among the 1089 subjects with a VACS Index at all three time points. The VACS index discriminated risk of mortality at all three time points with Harrell’s c-statistics of 0.73, 0.77 and 0.78 at HI, 6M, and 1Y. Similar results were obtained after stratifying by age (supplementary Table S1). Time to death differed by VACS Index tertile at all three time points (log-rank p <0.05, supplementary Figure S2). Five year mortality was not significantly different between the first two VACS Index tertiles, but was different between the second and third tertiles (p <0.05).

Calibration of the VACS Index in this cohort was assessed by comparing predicted and observed mortality. These were 10.8%, 6.8% and 6.5% predicted for HI, 6M and 1Y versus 2.5% (1.7-3.3% 95% CI), 5.0% (3.8-6.3%) and 3.3% (1.6-5.0%) observed for the same time points respectively. Similar overestimates were also present when the analyses were stratified by age and VACS Index.

Combining VACS Index values over time

Among subjects with a five year follow-up, 86.0% and 91.8% of the subjects in the lowest VACS Index tertile at HI and 6 months respectively remained in the lowest tertile one year after HI, while 51.8% and 63.1% of subjects in the highest tertile at HI and 6 months remained in the highest tertile one year after HI (supplementary Tables S2, S3). Mortality was highest among subjects in the third VACS Index tertile at 1Y (29%), reinforcing the predictive power of the VACS Index at this time point. Mortality was lowest among the subjects in the two lowest 1Y tertiles (3.3%), particularly those who started in tertile 1 or 2 at HI and were in the lowest tertile at 6M (1.7%).

Combining these observations, Cox models were used to assess the importance of the VACS Index values at HI and 6M, in addition to the value at 1Y. The best model in terms of AIC included the VACS Index at HI, 6M, and 1Y, and an interaction between scores at HI and 6M (Table 2). This demonstrates additional independent information contained in earlier values and in change between HI and 6M. Interaction terms between the other time points were not significant.

Table 2.

Multivariable Cox model for time to all-cause mortality (89 deaths in 1089 patients) using the VACS Index values at HI, 6M, 1Y, and an interaction term between the Index values at HI and 6M.

VACS Index	Hazard Ratio*	Lower limit (95% CI)	Upper limit (95% CI)	p-value
HI	1.10	1.01	1.21	0.036
6M	1.32	1.15	1.52	<0.001
1Y	1.15	1.07	1.24	<0.001
Interaction HI * 6M	0.98	0.97	0.99	0.001

n=1089, d=89 (deaths);

^*per five point change in VACS Index

The independent contribution of VACS Index values at HI and 6M and their interaction (to the prediction based on VACS Index at 1Y was further evaluated using net reclassification improvement (NRI) for five year mortality for subjects with a VACS Index at all three time points (Table 3). Among those (n=40) who died during follow-up, the Cox model including all three time points resulted in 38.7% correctly reclassified to a higher risk group and 0% to a lower risk group, for a 38.7% net positive reclassification. Among those who did not die, 2.1% were correctly reclassified to a lower risk group and 15.6% were incorrectly reclassified to higher risk group, for a -13.5% net negative reclassification. Combining these, the overall net reclassification index was positive and significant, NRI = 25.2% (10.8-48.9% 95% CI) again confirming the additional information contained in the earlier Index values and the change between them. Similar results were obtained using multiple imputations to assess the effect of the missing VACS Index values (Supplement).

Table 3.

Net reclassification improvement for adding VACS Index values at HI and 6M and their interaction to the index value at one year after HI. Cells show numbers of patients reclassified in each group, using inverse probability weighting to account for censoring (blue indicates correctly reclassified, red incorrectly reclassified patients, green indicates no change).

	VACS Index at 1Y alone	VACS Index at HI, 6M, 1Y and VACS Index_HI*VACS Index_6M
	Tertiles of (Cox) Predicted Mortality	<2.28%	2.28-4.36%	>4.36%	Total (%)
Died during follow-up	<2.28%	8.2	5.9	0	14.1 (29%)
	2.28-4.36%	0	2.5	12.6	15.1 (32%)
	>4.36%	0	0	18.6	18.6 (39.0%)
	Total (%)	8.2 (17.2%)	8.4 (17.6%)	31.2 (65.2%)	47.8 (100%)
Did not die	<2.28%	734.4	73.6	6.9	814.9 (39%)
	2.28-4.36%	16.7	55.5	81.9	154.1 (33%)
	>4.36%	1.4	4.2	68.0	73.6 (28%)
	Total (%)	751.5 (72.1%)	133.3 (12.8%)	157.8 (15.1%)	1042.6 (100%)

Discussion

The VACS Index has been well validated among HIV-infected veterans as well as a broad aggregate of HIV cohorts in NA-ACCORD and ART-CC ^4,6, however younger, healthier subjects represent only a small minority of the studied subjects while accounting for a significant portion of those infected with HIV across the U.S. and in the world. We validated the VACS Index in a relatively young, healthy HIV-infected population with low prevalence of comorbidity, early diagnosis and entry into care in the military system with open access and free medications, low prevalence of injection drug use, relatively stable socioeconomic status, and at least a high school level of education. Using both 5-year and time to all-cause mortality, our findings showed good correlation and discrimination for the VACS Index, with adequate calibration although predicted mortality was modestly overestimated compared to observed. It is likely this results from underlying population age and health status differences. Future work in a larger sample of subjects will evaluate whether a generalization allowing model tuning or recalibration, for example an additional age cutoff below the age of 50 years (the current VACS Index model uses <50, 50-64, and ≥65), can be developed and validated. Such an addition could be useful, because only 13.1% of new HIV infections and 13.7% of new AIDS diagnoses in the U.S. are in those over 50 years of age while 50.3% of new HIV infections and 36.4% of new AIDS diagnoses occur in those 34 or younger (CDC 2010 data ¹²).

The VACS Index has been shown to be strongly associated with biomarkers of inflammation and highly predictive of all-cause mortality when applied at any point from one to five years after starting HAART ^3-6. We add to this knowledge, by considering the VACS Index history, along with the VACS Index value at 1Y.The benefit of following the VACS Index longitudinally is two-fold. First, it allows early identification of patients both at low and high risk of mortality beginning from HI. This is consistent with early reports of a similar index to predict mortality in HIV patients initiating HAART that was studied and validated at this time point ¹³ although subsequent work with the VACS Index has shown somewhat stronger predictions using later time points after the initial response to treatment ⁴. Second, our results suggest additional information can be gained by taking into account previous VACS Index values along with VACS Index value at 1Y. Addition of the VACS Index values at HI and 6M to that at 1Y provided good net reclassification improvement for those who died, improving prediction both in terms of risk magnitude and earlier identification of risk. Subjects with a lower risk at HI and 6M, had a significantly lower risk of death, both unadjusted and adjusted for the VACS Index at 1Y. This indicates that the VACS Index at these early time points can be used as an additional independent predictor of outcome in conjunction with the VACS Index at one year after HI. While we cannot yet determine the reason for this finding, we suspect that earlier changes in VACS Index score reflect differences in adherence—those with excellent adherence likely respond to ART initiation more rapidly.

The additional information from these early time points is also useful to allow identification of patients at higher mortality risk who were not identified as such by the VACS Index value at one year. An example is the 15% mortality among those subjects who moved to or remained in a higher VACS Index tertile (2 or 3). Identification of patients at high mortality risk is of particular interest, and addition of the earlier VACS Index information appears to increase sensitivity for detection. Such information provides a powerful tool that may become useful in management of HIV infection.

The VACS Index, a marker calculated from routinely obtained information, was shown to be well correlated with mortality and to provide good discrimination among HIV-infected subjects in the U.S. Military HIV Natural History Study cohort, a relatively young and healthy population representative of many people living with HIV in the U.S. VACS Index at the time of HAART initiation and six months later provides important additional information to the VACS Index at one year and helped identify high risk patients at HAART initiation.