Strengths and Limitations of the Veterans Aging Cohort Study Index as a Measure of Physiologic Frailty

Abstract

The Veterans Aging Cohort Study Index (VACS Index) is an index comprised of routine clinical laboratory tests that accurately and generalizably predicts all-cause mortality among those living with and without HIV infection. Increasing evidence supports its use as a measure of physiologic frailty among those aging with HIV because of its associations with frailty related outcomes including mortality, hospitalization, fragility fractures, serious falls, pneumonia, cognitive decline, delirium, and functional decline. In this review, we explore the evidence supporting the validity (construct, correlative, and predictive), responsiveness, and feasibility of the VACS Index as an early indicator of physiologic frailty. We also consider its limitations.

Background

Frailty, a core concept in geriatrics, is defined by its prognostic implication; it is the decreased ability to recover from injury or a loss of reserve capacity indicating that a relatively minor stress may result in future, disproportionate, adverse health outcomes.^1,2 Frailty-related outcomes include mortality, hospitalization, fragility fractures, serious falls, pneumonia, cognitive decline, delirium, and functional decline. Early frailty should trigger a full geriatric assessment to identify reversible causes.² Advanced frailty suggests the need to avoid stressors, including aggressive medical treatments,² which are likely to result in adverse, frailty-related health outcomes. Frailty increases with advancing chronologic age. However, frailty also differentiates risk of adverse health outcomes among those of similar age, that is, the difference between biologic and chronologic age.³ The etiology of frailty has not been definitively established, but is thought to result from the cumulative effects of molecular and cellular defects, chronic inflammation, immune exhaustion, sarcopenia, and cognitive decline.^1,4–7 Corresponding to these hypothesized etiologies, specific frailty associated biomarkers have been proposed.^8–11

For people aging with HIV (PAWH), major physiologic stresses occur at earlier ages. HIV infection and its treatment lead to microbial translocation, immune dysfunction, other viral coinfections, HIV-associated non-AIDS conditions, mitochondrial toxicity, and polypharmacy.^12–15 In addition, PAWH are more likely to use harmful substances, including alcohol, tobacco, and other drugs of abuse.¹⁶ As a result, PAWH often develop early signs of frailty well before their seventh or eighth decade.^8,17–25 PAWH with more advanced frailty may be susceptible to harm from aggressive medical management, including polypharmacy.

Multiple approaches to measuring frailty have been proposed. Fried describes a phenotype comprising any three of the following: weight loss, low physical activity, exhaustion, slowness (time to walk 15 feet), and weakness (by grip strength).²⁶ Rockwood proposed an accumulation of deficits, measured as proportion present of 30 or more conditions.²⁷ However, both the phenotype and accumulation of deficit approaches have limitations, in general and for PAWH.

In general, frailty measures lack standardization, have limited reproducibility, have limited agreement across measures, and have limited responsiveness to interventions.^2,6,28 In addition, studies rarely consider the extent to which the frailty measure used captures differences between chronologic and biologic age by either controlling for or stratifying by age.²⁹

Among PAWH, the Fried Frailty Phenotype and adaptations of it have been evaluated more than the Rockwood Accumulation of Deficits. When the Frailty Phenotype has been applied to PAWH, few patients demonstrate the full phenotype. Rather patients typically demonstrate only one or two of the criteria causing many studies to focus on “pre-frailty” rather than frailty.^17,21,30,31

Because of the above limitations, many have proposed specific frailty associated biomarkers.^2,9–11 But these biomarkers are not routinely available in clinical care. Investigators have also suggested that an index of biomarkers may provide an early indication of frailty (physiologic frailty) and might address limitations of previous measures—particularly if the biomarkers used are routinely measured. Of note, frailty is considered distinct from comorbidity and functional status, yet measures of comorbidity and/or functional status are included in over half the frailty metrics in common use, including the Frailty Phenotype which includes function.⁶ Consistent with this observation, some investigators have suggested that if the intended use is prognostic, it may be reasonable to include variables such as comorbidity or age known to improve accuracy of prediction for frailty associated outcomes.⁶

We have developed, and widely validated, using established measures of accuracy and generalizability (Appendix), the Veterans Aging Cohort Study Index (VACS Index), a physiologically-based index for predicting mortality. VACS Index is based on routine laboratory measures and age and has been shown to effectively discriminate risk of mortality among a wide variety of PAWH and several groups of uninfected individuals. Increasing evidence also supports its strong cross-sectional association with biomarkers and its predictive associations with other outcomes, considered indicative of frailty. As a result, many have suggested that the VACS Index may serve as a useful measure of frailty among PAWH.^{4,17,18,30,32–36} In this review, we explore the evidence supporting the validity (construct, correlative, and predictive), responsiveness, and feasibility of the VACS Index as an early indicator of physiologic frailty for people aging with and without HIV. We also consider its limitations.

Construct Validity

Our original conceptualization of the VACS Index was as an early summary indicator of multisystem injury reflecting reduced physiologic reserve across major organ systems (Fig. 1). Using an accumulation of deficit approach,²⁷ and in accordance with a previous laboratory based frailty measure,⁹ we used a variety of clinical laboratory tests. A score is calculated based on preassigned points for routinely monitored, Clinical Laboratory Improvement Amendments (CLIA) certified (highly reproducible and standardized) laboratory tests (Table 1).³⁷ Higher scores indicate increasing risk of mortality. Importantly, this approach is agnostic to specific disease diagnoses, which are susceptible to biases in ascertainment, and does not rely on self-report of health behaviors which can be susceptible to social desirability bias.^38,39

FIG. 1.

Conceptual model for VACS Index. VACS, Veterans Aging Cohort Study.

Table 1.

Veterans Aging Cohort Study Index 1.0 Components and Weights

Component	Level	Points
Age (years)	<50	0
	50–64	12
	≥65	27
CD4 (cells/mm3)	≥500	0
	350–499	6
	200–349	6
	100–199	10
	50–99	28
	<50	29
HIV-1 RNA (log copies/mL)	<500	0
	500–99,999	7
	≥1 × 105	14
Hemoglobin (g/dL)	≥14	0
	12–13.9	10
	10–11.9	22
	<10	38
FIB-4	<1.45	0
	1.45–3.25	6
	>3.25	25
eGFR (mL/min)	>60	0
	45–59.9	6
	30–44.9	8
	<30	26
HCV coinfection		5

eGFR, estimated glomerular filtration rate.

Nevertheless, in its original form, VACS Index (1.0) (Table 1) had important limitations. It categorized predictor variables, limiting its resolution and ability to respond to small changes within an individual. It was overly parsimonious; other routine laboratory tests were thought likely to increase its discrimination of mortality. We recently updated VACS Index 1.0–2.0 by adding albumin, white blood cell count, and body mass index (BMI) and treating all variables as continuous with somewhat complex functional forms (Table 2). While discrimination of the index improved with these modifications, our primary goal was to improve its sensitivity and resolution for detecting changes over time, thereby enhancing its utility for patient care and clinical research.

Table 2.

Veterans Aging Cohort Study Index 2.0 Cox Proportional Hazards Model, for 5-Year All-Cause Mortality, Estimated in Veterans Aging Cohort Study

n	28,390
Deaths	7,293
Parameter	PE	SE	χ2	p	HR	95% CI
Age (years), censored at 30–75, centered at (age—50)
X	0.056	0.012	22	<.0001	1.06	0.00–0.00
X2	−0.004	0.004	2	.22	1.00	0.00–0.00
X3	0.005	0.001	29	<.0001	1.01	0.00–0.00
CD4 cell count (cells/mL), censored at 0–1,000, as ln (1,000-CD4)
X	−0.056	0.025	5	.03	0.95	0.00–0.00
X2	−0.153	0.023	46	<.0001	0.86	0.00–0.00
X3	0.024	0.002	94	<.0001	1.02	0.00–0.00
HIV-1 RNA (log copies/mL), censored at 1.3–5.0, centered at (logVL—2)
X	0.513	0.033	247	<.0001	1.67	0.00–0.00
X2	−0.422	0.041	109	<.0001	0.66	0.00–0.00
X3	0.098	0.011	77	<.0001	1.10	0.00–0.00
Hemoglobin (g/dL), censored at 9–16, centered at (14—hemoglobin)
X	−0.134	0.011	141	<.0001	0.88	0.00–0.00
X2	0.026	0.006	16	<.0001	1.03	0.00–0.00
X3	0.005	0.001	10	.002	1.01	0.00–0.00
FIB-4, censored at 0.5–7.5
X	0.220	0.028	62	<.0001	1.25	0.00–0.00
X2	−0.009	0.003	7	.008	0.99	0.00–0.00
eGFR (mL/min), censored at 0–180a
X1	−0.031	0.028	1	.28	0.97	0.00–0.00
X2	−0.077	0.045	3	.0917	0.93	0.00–0.00
X3	0.106	0.027	16	<.0001	1.11	0.00–0.00
X4	0.133	0.034	15	.0001	1.14	0.00–0.00
Hepatitis C coinfection
Yes	0.342	0.028	147	<.0001	1.41	0.00–0.00
Albumin (g/dL), censored at 2–5, centered at (albumin—4)
X	−0.443	0.034	165	<.0001	0.64	0.00–0.00
X2	0.104	0.051	4	.04	1.11	0.00–0.00
X3	0.028	0.027	1	.30	1.03	0.00–0.00
White blood count (k/mL), censored at 2.5–11, centered at (WBC—5.5)
X	0.126	0.011	130	<.0001	1.13	0.00–0.00
X2	0.020	0.004	30	<.0001	1.02	0.00–0.00
X3	−0.004	0.001	23	<.0001	1.00	0.00–0.00
BMI, kg/m2, censored at 15–35, centered at (BMI—25)
X	−0.055	0.003	388	<.0001	0.95	0.00–0.00
X2	0.004	0.000	62	<.0001	1.00	0.00–0.00

^aX1 = eGFR/10, X2 = (eGFR-35)/10, X3 = (eGFR-65)/10, X4 = (eGFR-115)/10.

BMI, body mass index; CI, confidence interval; HR, hazard ratio; PE, parameter estimate; SE, standard error; WBC, white blood cell.

In support of construct validity, several variables included in the VACS Index were previously included in the laboratory-based Frailty Index proposed by Blodgett et al.⁹ These include albumin, creatinine, hemoglobin, platelets, and liver function tests (we use aspartate and alanine transaminase [AST and ALT]; they used alkaline phosphatase, lactate dehydrogenase, and bilirubin). Consistent with the wasting construct in the Frailty Phenotype, VACS Index 2.0 adds BMI. Immune exhaustion an important manifestation of frailty, especially among PAWH,^8,40,41 is represented by white blood cell count and CD4 cell count.⁴² Because hepatitis C virus (HCV), a common coinfection among PAWH, exacerbates inflammation, as well as directly causes injury to the liver and kidneys, we include an indicator for chronic HCV.^43–45

Finally, age independently predicts frailty associated biomarkers and outcomes and also modifies many other predictors. For example, age is incorporated in the estimation of liver fibrosis calculated with FIB-4, which includes AST, ALT, platelets, and age,⁴⁶ and in the estimation of creatinine clearance using creatinine and age.⁴⁷ To maximize discrimination and calibration, we retain age in the index and consider whether the VACS Index discriminates risk of frailty related outcomes among individuals of similar chronologic age (Fig. 2).

FIG. 2.

(a) Mortality by age and VACS Index 2.0. (b) Pneumonia events by age and VACS Index 2.0.

Correlative Validity

The VACS Index is strongly associated with a host of biomarkers and functional tests considered reflective of frailty.^32,34,36,48 Among PAWH, the index is correlated with markers of chronic inflammation, microbial translocation, and hypercoagulability (cystatin C, tumor necrosis factor alpha, interleukin-6, soluble CD14, soluble CD163, and D-dimer) in resource rich^49–52 and in resource limited settings.⁵³ VACS Index is more strongly associated with biomarkers of hypercoagulability than the Framingham Index.^51,53 The Index is associated with the Chronic Immune Activation and Senescence (CIADIS) score, composed of CD4 and CD8 activation, naive and terminally differentiated memory T cells, and CD57CD28 cells weighted by principal component analyses.⁵⁴ The VACS Index is also associated with concurrent measures of neurocognitive test performance,⁵⁵ functional performance,^56,57 sarcopenia,⁵⁸ and autonomic neuropathy.⁵⁹

Predictive Validity

While mortality is not the only adverse health outcome associated with frailty, it is objective, important, and highly patient relevant. We first consider predictive validity of the VACS Index for mortality among PAWH and uninfected. We then address prognostic validity for other important frailty-related outcomes, including: hospitalization, falls, fractures, cognitive decline, delirium, and functional decline.

Mortality among PAWH

VACS Index 1.0 was developed in veteran patients,³⁷ and its accuracy has been validated in other PAWH in North America and Europe.^37,60 It discriminates risk of mortality more effectively than an index restricted to CD4 count, HIV-1 RNA, and age especially among those with undetectable HIV-1 RNA and those 50 or more years of age.^37,60 The accuracy (discrimination and calibration) of the Index for predicting mortality among PAWH meets or exceeds that reported for indices currently used in clinical practice.^61–63 VACS Index is consistently accurate for any length of time on antiretroviral treatment and is robust among important subgroups, including women, people of color, those with HCV coinfection, and those over 50 years of age.^36,37,60 It also discriminates risk of mortality among young active duty military who are relatively free of comorbid disease⁶⁴ and among highly frail individuals initiating salvage antiretroviral therapy (ART).⁶⁵ It predicts cardiovascular mortality as accurately as all-cause mortality.⁶⁶

Discrimination improved and resolution increased in VACS Index 2.0. Its generalizability was demonstrated in a European and North American cross cohort collaboration, the Antiretroviral Therapy Cohort Collaboration (ART-CC).⁴² Compared with VACS Index 1.0, improvements in discrimination were seen across all cohorts and in subgroups defined by age, gender, race/ethnicity, HIV RNA suppression, HCV status, and among low versus high risk patients. Of note, VACS Index 1.0 and 2.0 scores are 86% correlated, suggesting that work demonstrating the validity of 1.0 with respect to cross-sectional associations with biomarkers and predictive associations with hospitalizations, falls, cognitive function, and physical function likely applies to 2.0.

Mortality among PAWH of similar chronologic age

An important validity check when considering a frailty metric is whether it differentiates risk of frailty related, adverse health outcomes among individuals of similar chronologic age. We calculated observed mortality rates for PAWH stratified by age and VACS Index score (Fig. 2a), using data from the article originally reporting VACS Index 2.0,⁴² updated to 2018. For those 60–69 years of age overall observed mortality was 8.5 deaths/100 person-years. After stratifying by VACS Index score, observed mortality ranged from 3.3 deaths/100 PY to 45 deaths/100 PY or in other words from less than half to more than five times the overall rate in those aged 65–69. Discrimination of risk was even greater for younger individuals. VACS Index 2.0 successfully differentiates risk of mortality among PAWH of similar chronologic age over a wide range of age strata.

Mortality among uninfected individuals

If you assume that those without HIV infection have no HIV-1 RNA and a CD4 cell count above 500 cells/mm³ (i.e., a normal value), the VACS Index predicts mortality among those without HIV equally well as among PAWH. This has been shown for 30-day mortality after hospitalization that included medical intensive care⁶⁷ and for long-term (median of 5 years) mortality⁶⁸ within the Veterans Administration Healthcare System and for hospitalization and mortality among women with and without HIV infection in the Women's Interagency HIV Study.^36,69 In the next section, we provide additional support for the accuracy of the VACS Index 2.0 in uninfected individuals compared with other risk indices.

Accuracy of predicted mortality exceeds other frailty and comorbidity metrics

Compared with an adapted version of the frailty related phenotype, the VACS Index more accurately predicted both all-cause mortality among PAWH and uninfected individuals.^17,36 VACS Index 2.0 discriminates risk of all-cause mortality substantially better than the Care Assessment Need Score⁷⁰ among veterans with HIV infection.⁷¹ More recent work in the Million Veteran Cohort, a general cohort of veterans in care volunteering for genetic research,⁷² demonstrated that VACS Index 2.0 discriminates risk of mortality more effectively than the Charlson Comorbidity Index (based on ICD coded diagnoses and age⁷³). This was true overall, among those over 45 years of age, among those with and without HCV infection, and among all racial and ethnic groups considered. There was no subgroup evaluated in which Charlson discriminated better than VACS Index 2.0. Similarly, among HIV infected and uninfected patients with cancer, the VACS Index discriminates risk of mortality better than the Charlson Index.⁷⁴ Given the long-standing use of Charlson Index in geriatric research, this finding is remarkable.

Other frailty-related outcomes

The VACS Index predicts other important frailty related adverse health outcomes, including hospitalization,^17,69,75 medical intensive care unit admission,⁷⁵ serious falls,⁷⁶ fragility fractures,^33,77 community acquired pneumonia,⁷⁸ neurocognitive compromise,^55,79–82 delirium, and functional decline. The index differentiates rates of pneumonia within age groups (Fig. 2b). When used in a time updated manner, the VACS Index predicts acute myocardial infarction better than time updated CD4 or HIV-1 RNA.⁸³ The index also predicted length of stay and readmission after hospitalization for bacterial pneumonia among HIV infected and uninfected (age 50+ years) veterans.⁸⁴ Finally, the VACS Index predicted delirium in the hospital setting among PAWH and uninfected individuals.⁸⁵

Differentiation by exposures and response to change

VACS Index scores also differ by important exposures, including level of smoking, alcohol consumption, and hypertension.^48,86,87 Values differ by number of non-antiretroviral (ARV) medications and physical function. Consistent with detecting reversible frailty, higher scores predict greater weight gain in the first 12 months after ART initiation.⁸⁸

VACS Index scores also respond to important changes in health and health behaviors. Scores change in response to antiretroviral initiation⁸⁹ and interruption⁶⁴ and discriminate among levels of ART adherence.⁸⁹ Changes in score correspond to changes in neurocognitive function.⁸⁰ When levels of alcohol consumption change among HIV infected subjects, the score changes.^90,91 Similarly, when HIV infected patients in treatment for substance abuse have positive urine toxicology screens, their scores are higher than when similar patients have negative toxicology screens (article in press). Because VACS Index scores rise during negative health behaviors (alcohol and stimulant use), it is likely that successful interventions in these domains would alter the VACS Index Score.

Feasibility

The VACS Index is in widespread use in research and clinical care. Online calculators can be found at the VACS website (http://vacs.med.yale.edu) and MDCalc (https://www.mdcalc.com/veterans-aging-cohort-study-vacs-index). These calculators have been accessed 88,000 times. In observational studies, the VACS Index has been used as a measure of frailty and as an adjustment for severity of illness.^{32,50,51,55,59,78,81,92–99} NIH funded alcohol intervention trials have included the VACS Index as an outcome¹⁰⁰ and another is underway. The AIDS Clinical Trials Group has used the VACS Index in a randomized trial.¹⁰¹ The Public Health-Seattle & King County, HIV/STD Program and the Washington State Department of Health are using the VACS Index to monitor risk of mortality and burden of disease among PAWH.

Several health systems have incorporated the VACS Index as a decision support tool in their electronic health record, including: Fenway Healthcare System in Boston; the San Francisco General Hospital HIV clinic; and University of California, San Diego Owen Clinic. The latter has incorporated the index into their live HIV registry and reviews VACS Index scores on their patients during care team meetings. Providers caring for over 60,000 HIV infected patients (and nearly 600,000 uninfected patients) in 79 health care sites in the Observational Pharmaco-epidemiology Research & Analysis (OPERA) database directly access VACS Index scores for PAWH and uninfected individuals through a physician portal. Information provided includes VACS Index score trends over time, overall and for each component of the index. Some sites use these data to target care to their sickest patients, while others use it in provider meetings to discuss overall patient management. The VACS Index is calculated on every patient seen at the University of Modena Metabolic Clinic (a clinic of over 4,000 HIV patients in Modena, Italy) using an automated application.

Limitations

While the VACS Index has demonstrated strong construct, cross-sectional, and predictive validity as a measure of physiologic frailty, it is not a substitute for geriatric assessment. Rather, the VACS Index score might provide a useful indicator for when a full geriatric assessment is indicated. Furthermore, other measures, including medication count, physical and cognitive function, and selected diagnoses, might improve the ability of the index to identify those developing early signs of frailty or prefrailty. For example, while the Charlson Index demonstrated poorer discrimination than did the VACS Index, a model which included both Charlson and VACS Index components discriminated risk of mortality better than either alone.⁷³ Although calibration of scores for VACS 1.0 is available, we are currently developing calibration curves for VACS Index 2.0 that will provide estimates of expected mortality over specified intervals of time from less than 1 year up to 10 years. Furthermore, we are still working on how to incorporate sustained viral response to HCV treatment and how to optimally handle missing data since healthier patients are less likely to have all required laboratory values for the VACS Index.

Conclusion

A “one size fits all” frailty index may not be a realistic goal. Rather, the frailty measure should be chosen based on its intended application. The VACS Index 2.0 may be a useful indicator of physiologic frailty indicating the need for more attentive management or a complete geriatric assessment. The VACS Index may also help alert providers and patients when life expectancy is short and end of life planning is indicated. The index reproducibly and generalizably discriminates risk of mortality in a wide variety of patient settings and it outperforms other general risk indices. It also predicts other frailty-associated outcomes, is cross-sectionally associated with frailty associated biomarkers, and differentiates risk of mortality and other frailty related outcomes among individuals of similar chronologic age. VACS Index 2.0 is based on readily available, standardized, clinical laboratory tests that are frequently obtained in routine care. Now that electronic health records make it easy to access these values and to automatically calculate scores using a validated algorithm, the VACS Index provides a practical and effective method to assess physiologic frailty.

Appendix

Accuracy and Generalizability

Accuracy has two components, discrimination and calibration.^A1 Discrimination is the ability to accurately rank individuals according to risk. It is commonly measured using C statistics. A C statistic of 0.50 corresponds to no discrimination, and C statistic of 1.00 corresponds to perfect discrimination^A2–A4; C statistics of 0.50–0.69 are considered fair, 0.70–0.79 are good, and >0.80 are excellent. Once a model has achieved good to excellent discrimination, it is difficult to improve the C statistic, since prognostic variables often covary. A change in c-statistic of 0.01 is usually clinically meaningful. Veterans Aging Cohort Study (VACS) Index 2.0 has demonstrated improved discrimination on the order of 0.02–0.03 over VACS Index 1.0, which previously demonstrated improved discrimination on the order of 0.05–0.10 over an index restricted to age, CD4 cell count, and HIV-1 RNA. Thus, the VACS Index offers substantial differentiation of risk controlling for age.

Calibration is the ability to consistently and accurately predict probability of an outcome over some specified interval of time. Calibration is often the more important component when using predictive indices in patient management. It can be evaluated using calibration curves or Kaplan–Meier Plots. VACS Index 1.0 has demonstrated consistent calibration, but this has yet to be determined for VACS Index 2.0.

Generalizability is the ability to be consistently accurate in a new, but plausibly related, sample.^A1 VACS Index 1.0 predicts all-cause mortality in a wide range of HIV infected populations, including those first initiating antiretroviral therapy (ART),^A5 after the first year of ART,^A6,A7 among highly treatment experienced patients^A8 and among young military recruits.^A9

Author Disclosure Statement

No competing financial interests exist.

Funding Information

The following funding was received from the National Institute on Alcohol Abuse and Alcoholism (NIAAA): U10 AA013566, U24 AA020794, U01 AA020790, and U24 A022001.