Evaluation of Frailty as an Unmeasured Confounder in Observational Studies of Antidiabetic Medications

Abstract

Background

It is unknown whether observational studies evaluating the association between antidiabetic medications and mortality adequately account for frailty. Our objectives were to evaluate if frailty was a potential confounder in the relationship between antidiabetic medication regimen and mortality and how well administrative and clinical electronic health record (EHR) data account for frailty.

Methods

We conducted a retrospective cohort study in a single Veterans Health Administration (VHA) healthcare system of 500 hospitalizations—the majority due to heart failure—of Veterans who received regular VHA care and initiated type 2 diabetes treatment from 2001 to 2008. We measured frailty using a modified frailty index (FI, >0.21 frail). We obtained antidiabetic medication regimen and time-to-death from administrative sources. We compared FI among patients on different antidiabetic regimens. Stepwise Cox proportional hazards regression estimated time-to-death by demographic, administrative, clinical EHR, and FI data.

Results

Median FI was 0.22 (interquartile range 0.18, 0.27). Frailty differed across antidiabetic regimens (p < .001). An FI increase of 0.05 was associated with an increased risk of death (hazard ratio 1.45, 95% confidence interval 1.32, 1.60). Cox proportional hazards model for time-to-death including demographic, administrative, and clinical EHR data had a c-statistic of 0.70; adding FI showed marginal improvement (c-statistic 0.72).

Conclusions

Frailty was associated with antidiabetic regimen and death, and may confound that relationship. Demographic, administrative, and clinical EHR data, commonly used to balance differences among exposure groups, performed moderately well in assessing risk of death, with minimal gain from adding frailty. Study design and analytic techniques can help minimize potential confounding by frailty in observational studies.

Observational studies using data collected for administrative purposes are an important method to evaluate the risks associated with common medication exposures. In a recent study of antidiabetic medications, we found that intensification of metformin monotherapy with insulin versus sulfonylurea was associated with an increased risk of all-cause mortality (1). Although we used propensity score adjustment to match patients between medication groups, we remained unsure how well these groups were balanced on frailty, a multidimensional syndrome of loss of reserves associated with increased risk of disability, healthcare utilization, and death (2,3). Frailty may be independently associated with antidiabetic medication exposure and risk of mortality which could introduce confounding. In studies that control for relevant measured confounders but have unmeasured or poorly measured frailty, the possibility of residual confounding by frailty exists. The prior study of intensification of metformin with insulin versus sulfonylurea was a motivating example to explore this concern in the current study.

Previous work aiming to account for frailty in observational studies using administrative data is limited (4,5). Faurot and colleagues found that in Medicare enrollees, codes for durable medical equipment (hospital beds, wheelchairs, and portable oxygen) were surrogates for dependence in activities of daily living (ADL), an indirect measurement of frailty (6). Another study found that use of individual healthcare services including nursing home stays and home care were associated with an increased probability of disability, another proxy for frailty (7).

Several instruments exist to measure frailty, but frailty is not routinely assessed in practice (8). Direct frailty measurements are not available in most administrative databases; thus controlling for its potential confounding influence remains challenging (4). We sought to determine if frailty was a potential confounder in the relationship between antidiabetic medication regimens and mortality, and to evaluate how well administrative and electronic health record (EHR) data account for frailty.

Methods

Study Design and Population

We conducted a retrospective cohort study of hospitalized patients sampled from a national observational cohort of Veterans with type 2 diabetes. The current study was nested within another study which validated an algorithm to identify hospitalizations due to heart failure (9). We sampled 500 hospitalizations; 400 met the heart failure algorithm criteria and 100 did not and each hospitalization was considered independent. Thus, a patient could contribute more than one hospitalization to the study if they fulfilled eligibility criteria.

Veterans were eligible for inclusion if they were hospitalized between October 2001 and September 2012 in one Veterans Health Administration (VHA) healthcare system and met the following criteria: aged 18 years or older, received regular VHA care (prescription fill or medical encounter at least once every 180 days), and had an incident prescription for an antidiabetic medication between 2001 and 2008. Sampled study hospitalizations could have occurred before or after the incident prescription for an antidiabetic medication to allow adequate sampling of hospitalizations meeting heart failure algorithm criteria. This study was approved by the Institutional Review Board with a waiver of informed consent.

Patient Frailty

Patient frailty at the time of study hospitalization was measured retrospectively by standardized chart abstraction using a modified version of the Canadian Study of Health and Aging Frailty Index (FI) (3,10–12). This method measures frailty as a count of deficits in health (symptoms, signs, diseases, disabilities); a higher number of deficits indicates greater frailty. We chose 60 variables for the FI that could be abstracted by chart review from the original FI which included 92 variables (11). Data for the FI were abstracted by one internist from the VHA’s integrated medical record system. Current symptoms and functional status data were abstracted from the admission history and physical, nursing notes, and physical and occupational therapy assessments limited to the first 72 hours of admission. Medical comorbidities were abstracted from discharge summaries, outpatient encounters notes, and structured problem lists for up to 2 years prior to the hospitalization. A protocol for data abstraction was developed to ensure independence and uniformity in the frailty evaluation (see Supplementary Appendix Table A1 for the chart abstraction form).

A patient’s FI (range 0 to 1) was obtained by dividing the number of deficits present by the total number of deficits measured. FI was valid if between 30 and 60 variables were measured; if not, an FI was not calculated (12). FI was treated as a continuous variable in our analyses but was also categorized for descriptive purposes using established cut-offs: non-frail (FI ≤ 0.10), vulnerable (0.10 <FI ≤ 0.21), frail (0.21 <FI ≤ 0.45), and most frail (FI >0.45) (3,13).

Antidiabetic Medication Regimen

A patient’s antidiabetic medication regimen at the time of study hospitalization was determined using the VHA pharmacy database. If a medication was filled by the patient within 180 days prior to or on the study hospitalization date, the patient was considered exposed to the medication. Based on these determinations, patients were categorized into mutually exclusive antidiabetic medication regimens: none, metformin alone, sulfonylurea alone, metformin plus sulfonylurea, metformin plus insulin, sulfonylurea plus insulin, insulin alone, or all other regimens.

Mortality

Mortality data were obtained from the National Death Index (NDI) and VHA vital status files. NDI data were used from the start of the study until December 31, 2011. After that, only VHA vital status data were available through study end, December 31, 2013. Agreement between these sources is excellent; sensitivity and specificity of the VHA vital status file are 98.3% and 99.8%, respectively, compared with the NDI (14). Follow-up continued from the time of study hospitalization until the date of death or end of the study.

Demographic, Administrative, and Clinical EHR Variables

We collected demographic, administrative, and clinical EHR variables for each patient from VHA sources supplemented with Medicare and Medicaid data for the 730 days preceding the patient’s study hospitalization. We collected the following demographic and administrative variables for each patient: age, sex, race, medical comorbidities, medication use, and healthcare utilization (Table 1). Additionally, we collected clinical EHR variables including laboratory values and vital signs.

Table 1.

Administrative and Clinical Electronic Health Record (EHR) Variables Included in Multivariable Regression Models

Model building steps	Variablesa
Step 0:	Frailty index
Step 1:	Demographics: age, race/ethnicity
Step 2:	Step 1 variables plus:
	- Medical comorbidities: malignancy, liver/respiratory failure, congestive heart failure, cardiovascular disease, serious mental illness including dementia, smoking, chronic obstructive pulmonary disease, cardiac valve disease, arrhythmia, Parkinson’s disease, fall and/or fracture, osteoporosis and/or use of bisphosphonate medication, dialysis, transplant
	- Medications: angiotensin-converting enzyme inhibitors or angiotensin II receptor blockers, beta blockers, calcium channel blockers, thiazide or potassium-sparing diuretics, loop diuretics, statins, antiarrhythmics, anticoagulants or platelet inhibitors, nitrates, aspirin, antipsychotics, oral glucocorticoids, antidiabetic medication regimen
	- Health care utilization: hospitalization in the previous year (in VHA, Medicare, or Medicaid data), hospitalization in the previous month (in VHA and Medicare data), admission date, Medicare use within the past year, Medicaid use within the past year, number of medications at admission, number of outpatient encounters in the previous year, home oxygen use
Step 3:	Step 2 variables plus clinical EHR variables:
	- Vital signs: systolic blood pressure (SBP), diastolic blood pressure (DBP), body mass index (BMI)
	- Laboratory values: hemoglobin A1c (A1c), low-density lipoprotein (LDL), creatinine, estimated glomerular filtration rate (eGFR), proteinuria
Step 4:	Step 3 variables plus:
	- Frailty index

Notes: VHA = Veterans Health Administration.

^aRare covariates occurring in less than 2% of the sample (sex, diagnosis of HIV) were excluded from multivariable models to allow for bootstrap model validation.

Statistical Analysis

Descriptive statistics characterized the study sample. We compared FIs across antidiabetic regimens using unadjusted ordinary least squares regression with robust standard errors and tested statistical significance with partial analysis of variance (F test). We performed multivariable Cox proportional hazards regression of time-to-death by demographics, administrative, clinical EHR data, and FI followed by bootstrap validation. We first assessed the unadjusted association of FI and time-to-death as Step 0 (Table 1). We subsequently used a stepwise model building approach including only age and race/ethnicity variables in Step 1, then adding administrative comorbidity, medication, and health care utilization data. These variables have been used in prior comparative effectiveness observational studies of antidiabetic medications to control for potential confounding (1,15,16). In Step 3, we added clinical EHR data (vital signs, laboratory values). In Step 4, we added back patient frailty (FI) to determine if it added to the predictive ability of the Step 3 model and tested the contribution of FI to the model by Wald Test. In the stepwise multivariable Cox proportional hazards regression modeling of time-to-death by demographics, administrative, clinical EHR data, and FI, we created restricted cubic splines for continuous variables to allow for nonlinear associations. Single imputation, using the R function transcan and conditioning on the other model covariates, was used to address missing data (vital signs and laboratory values) (17). Robust standard errors were included to account for repeated hospitalizations and nonindependence of FI measures for a small portion of patients. We report the c-statistic for each step of the above multivariable Cox proportional hazards model building; receiver operating characteristic (ROC) curves were plotted for each of step at the fixed median survival time of 3.4 years (see Supplementary Appendix 1 for additional methods) (18). To create ROC curves, we used the risksetROC function of the risksetROC R package; R2 and co-statistics were obtained using model performance measures from the validate function of the rms package (Supplementary Appendix Figure A1) (19). Adequacy of all prediction models was assessed by smooth calibration curves. Statistical analyses were completed using Stata Statistical Software: Release 14 (College Station, TX: StataCorp LP) and R software.

Motivating Example: Estimation of Residual Confounding Due to Frailty

In this study, we used the motivating example that intensification of metformin therapy with insulin versus with sulfonylurea was associated with increased risk of mortality to explore whether substantial residual confounding due to unmeasured frailty may have affected this comparison (1,20). While a large randomized controlled trial has shown neutral effect of insulin on the risk of cardiovascular events, in this study, we sought to examine the potential influence of frailty as a confounder, an unmeasured effect that may influence both the exposure and outcomes, in observational studies (21). To estimate the potential confounding by frailty, we estimated the hazard ratio of death between two FI values corresponding to the mean FI for patients on metformin plus insulin and for patients on metformin plus sulfonylurea. This hazard ratio represents the independent association of frailty variation and death approximating the factor by which a hazard ratio of death for metformin plus insulin versus metformin plus sulfonylurea should be reduced in a study that does not adjust for frailty. We performed this assessment using three strategies to control for confounding: (a) unadjusted, (b) direct covariate adjustment for variables reduced to five principal components, and (c) adjustment through a propensity score-based weighting strategy (see Supplementary Appendix 1, Supplementary Appendix Figure A2, and Supplementary Appendix Table A2 for additional methods).

Results

We sampled 500 of 10,766 eligible hospitalizations; 495 (99%) hospitalizations had sufficient documentation to assess frailty. The median age of patients was 65 years (interquartile range 58, 75). Patients were 98.8% male; 75.0% were white, 20% black (Table 2). When compared to the national VHA population with diabetes the current selected sample is similar in demographics but had more atherosclerotic cardiovascular disease than previously reported (63% vs 34%) in a national sample (1).

Table 2.

Patient Characteristics at Time of Study Hospitalization

Patient Characteristics	N = 495
Age in years, median (IQR)	65 (58, 75)
Male, n (%)	489 (99)
Race, n (%)
White	373 (75.0)
Black	99 (20.0)
Other	23 (5.0)
Hypertension, n (%)	414 (83.6)
Hyperlipidemia, n (%)	292 (59.0)
Atherosclerotic cardiovascular disease, n (%)	306 (62.8)
Type 2 diabetes, n (%)	430 (86.9)
Chronic kidney disease, stage 3–5, n (%)	206 (41.6)
Body mass index (kg/m2), mean (SD)	31.3 (7.3)
Mean hemoglobin A1c % (mmol/mol)	6.96 (53)

Note: IQR: interquartile range, SD: standard deviation.

Patient Frailty by Exposure (Antidiabetic Medications) and Outcome (Time-to-Death)

Overall, the mean FI was 0.23 (standard deviation [SD] 0.07) and median was 0.22 (interquartile range [IQR] 0.18, 0.27), which were in the frail range. Fifty-nine percent of patients were frail or most frail (FI > 0.21), 39% vulnerable (FI 0.1 to 0.21), and 2% non-frail (FI < 0.10).

Frailty differed significantly across the antidiabetic medication regimens as shown in Figure 1A (p < .001). Patients on insulin monotherapy and on insulin plus sulfonylurea were most frail with median FI of 0.25 (IQR 0.21, 0.32) and 0.25 (IQR 0.22, 0.28), respectively. Patients on metformin plus sulfonylurea had a lower median FI than patients on insulin plus metformin, median 0.20 (IQR 0.16, 0.24) versus 0.22 (IQR 0.19, 0.24).

Figure 1.

Relationship of patient frailty (frailty index) with antidiabetic medication regimen (exposure) and mortality (outcome), unadjusted analyses. (A) Boxplot of patient frailty (FI) by antidiabetic medication regimen categories. (B) Hazard ratio of outcome, time-to-death, by frailty index relative to median patient frailty (FI = 0.22) – unadjusted Cox proportional hazards model and boxplot of patient FI.

Median time-to-death after study hospitalization was 3.4 years. In the unadjusted Cox proportional model, higher FI was associated with an increased hazard of death (p < .001). An increase in FI of 0.05, from 0.22 to 0.27, increased the hazard of death by 1.45 (95% CI: 1.32, 1.60; Figure 1B).

Predicting Time-to-Death by Demographic, Administrative, Clinical EHR Variables and Patient Frailty

In stepwise multivariable Cox proportional hazard modeling, followed by bootstrap validation, we observed that FI alone accounted for 16.7% of the variation in time-to-death (R², Step 0). The Step 1 model with demographics only (age, race/ethnicity) accounted for 15.7% of variation in time-to-death. Step 2 (Step 1 plus administrative data) and Step 3 (Step 2 plus clinical EHR data) models accounted for 22.8% and 21.5% of variation in time-to-death, respectively. The final model added back patient frailty (Step 4) and accounted for 25.3% of variation in time-to-death. Frailty remained significantly associated with mortality in the final model adjusted for demographic, administrative, and clinical EHR data, p < .001.

Evaluating the Contribution of Patient Frailty to the Prediction of Time-to-Death

The ROC curves for each step of the multivariable Cox model evaluated the magnitude of contribution of demographic, administrative, clinical EHR, and frailty in predicting time-to-death. The predictive ability of the Step 4 model, which included patient frailty, was greater than other steps of the model building process, but only slightly greater than that of the Step 3 model, which included demographic, administrative, and clinical EHR data (Figure 2). After bootstrap validation, the corresponding c-statistics were estimated as Step 0 = 0.67 (frailty alone); Step 1 = 0.64; Step 2 = 0.70; Step 3 = 0.70; Step 4 = 0.72 (see Supplementary Appendix Figure A1). These results indicate that when the numerous administrative and clinical variables from Step 2 and 3 (Table 1) are in the model the addition of FI does not provide additional information to the model.

Figure 2.

Receiver operating characteristic curves for stepwise Cox proportional hazards models using administrative variables and patient frailty (FI) to predict time-to-death at median survival time of 3.4 years. Step 0 predicts time-to-death by chart-abstracted frailty unadjusted for any covariates. Additional steps build sequentially including: demographics (Step 1); administrative data (Step 2); clinical electronic health record (EHR) data (Step 3); chart-abstracted frailty index (Step 4). C-statistics for each step: Step 0 = 0.67 (frailty alone); Step 1 = 0.64; Step 2 = 0.70; Step 3 = 0.70; Step 4 = 0.72.

Motivating Example: Evaluation of Residual Confounding From Patient Frailty

In our previous study, which demonstrated that intensification of metformin with insulin compared with sulfonylurea was associated with increased risk of mortality (adjusted HR 1.44 [95% CI: 1.15, 1.79]), we used propensity score matching but did not control for frailty directly (1). In the current study, we evaluated the possibility that residual confounding due to frailty had affected that comparison of the risk of mortality between metformin plus insulin and metformin plus sulfonylurea groups in a sample of patients with direct measurement of frailty. Mean FI was 0.22 (SD 0.04) versus 0.20 (SD 0.06), for metformin plus insulin versus metformin plus sulfonylurea patients (p = .26). Although this implies no statistically significant difference in frailty between medication groups, the sample size was small, and this modest observed difference in frailty may be clinically significant.

The estimated unadjusted hazard ratio of death comparing the mean FI of the metformin plus insulin group with the mean FI of the metformin plus sulfonylurea group was 1.17 (95% CI: 1.12, 1.22, Figure 3). We then adjusted the model for demographic, administrative, and clinical EHR data using five principal components; the hazard ratio associated with the difference in mean FI between the medication groups decreased to 1.10 (95% CI: 1.04, 1.15). Finally, we balanced the metformin plus insulin and metformin plus sulfonylurea users by propensity score weighting and adjusted the model for the principal components; the hazard ratio was further reduced to 1.08 (95% CI: 1.04, 1.13). This indicates that a small residual independent association between frailty and risk of death remains when comparing patients receiving metformin plus insulin with patients receiving metformin plus sulfonylurea after adjustment for measured study covariates. Considering the motivating example, the small residual confounding by frailty indicates that the observed association between antidiabetic medication and death would be minimally attenuated, if that study included adjustment using a direct measurement of frailty.

Figure 3.

Hazard ratio of frailty index (FI) upon time-to-death when comparing mean FI of patients on metformin plus insulin versus the mean FI of patients on metformin plus sulfonylurea. Three analytic strategies were undertaken to control for confounding: (a) unadjusted for any confounding variables, (b) direct covariate adjustment for Step 3 variables (except for antidiabetic medication regimen) reduced to the first five principal components, and (c) the same direct adjustment using first five principal components on a propensity score weighted cohort.

Discussion

In our sample of hospitalized Veterans, we found that patient frailty differed by antidiabetic medication regimens and was associated with mortality. Imbalances in frailty between exposure groups may confound the assessment of the association between antidiabetic regimens and the risk of death. Nevertheless, we demonstrated that demographic, administrative, and clinical EHR data were associated with mortality (c-statistic of 0.70), and use of those data in regression models performed well in capturing frailty variability, as adding directly-measured patient frailty minimally improved model performance (c-statistic 0.72). Our findings were similar to those of Cuthbertson and colleagues who showed that 20 claims-based predictors from Medicare data showed good discrimination in predicting phenotypic frailty (c-statistic of 0.71) (22).

We explored the role of frailty as a potential confounder, an unmeasured effect that may be associated with both the exposure and outcome, in the association between two antidiabetic medication regimens (metformin plus insulin vs metformin plus sulfonylurea) and death. After balancing exposure groups by propensity score weighting and adjusting for covariates, we found a small independent association between frailty observed in the antidiabetic regimens and risk of death. This provides an estimate of the association that can be attributed to frailty differences between treatment groups in studies using these methods to account for potential confounders. While these results suggest that residual confounding due to unmeasured frailty in a previous study of antidiabetic medications was small, this cannot be generalized directly to other studies. The potential impact of not measuring frailty depends on the strength of the association between frailty and the outcome, the imbalance in frailty between the exposure groups, and the correlation between frailty and other measured covariates. We postulate that appropriate study design and analytical techniques can help minimize the potential confounding by frailty in observational studies (23).

When an unmeasured confounder is correlated with observed covariates, the potential for confounding can be mitigated through direct covariate adjustment and by creating a cohort that is balanced on observed covariates. These two approaches work on separate pathways. In the case of covariate adjustment, it is useful to think of how the effects estimated for the observed covariates will be biased. The estimates will be the covariates’ own independent effects plus some of the effect of the unmeasured confounder. Thus, the unmeasured confounder’s independent effect is partially absorbed by the observed covariates. How much of the effect is absorbed depends on the associations between the unmeasured confounder and the observed covariates. For the second pathway, a confounder must have associations between the outcome and the exposure. Where direct covariate adjustment helps break the path between the unmeasured confounder and the outcome, creating a balanced cohort helps break the path between the unmeasured confounder and the exposure. Balancing covariates that are associated with the unmeasured confounder will reduce the imbalance with that confounder and the exposure. Again, how much will depend on the strengths of the associations with the observed covariates. If the unmeasured covariate is independent of the observed covariates, the imbalance would be made better, worse, or unchanged by weighting purely by chance. Combining weighting with covariates adjustment is more effective than covariate adjustment alone in some but not all cases (24). The strengths and form of the associations, sample size, and method of analysis all play a role. Improved methods of combining weighting with covariate adjustment have been shown to perform particularly well (25,26).

Our study has several limitations. We conducted this study in one VHA health care system, and the patients were hospitalized—the majority due to heart failure—and thus likely represent a frailer sample than community-dwelling adults or the populations of patients on these antidiabetic medication regimens as a whole. This may limit the generalizability of these findings. Additionally, frailty was measured at time of hospitalization; while in the motivating example study, covariates were measured at the time of antidiabetic medication intensification and patients were likely less frail because they were not hospitalized. Though the patients may differ between the two studies, a frailer patient sample in this study demonstrates the worst case scenario for the degree of residual confounding by frailty. Data abstraction from medical records may be subject to error due to missing or low-quality information and/or interpretation of data from the electronic medical records despite standardized abstraction protocol. We assessed frailty retrospectively; while this method was used previously, it may not be as accurate as prospective assessment (6–8). In the frailty assessment, we counted a medical comorbidity as present if it was documented in the 2 years preceding the study hospitalization; we were not able to confirm the presence of each comorbidity at the time of study hospitalization. However, patients typically do not improve in many of the comorbidities noted in the FI, such as dementia; therefore the lookback period of 2 years was used to more accurately capture true levels of frailty that may be underreported at the time of admission. While we found direct frailty measurements to be associated with both antidiabetic regimen and mortality risk, we were unable to assess if the observed frailty was the result of the medication exposure, and potentially on the causal pathway between drug exposure and mortality. Finally, we did not assess the reliability of our FI assessments.

Implications

Frailty differs across commonly used antidiabetic medication regimens and is predictive of mortality; thus, frailty may act as a confounder in the study of antidiabetic medication regimens and mortality risk. However, we demonstrated that a combination of demographic, administrative, and clinical EHR variables have moderate discriminatory ability in predicting mortality, and direct measurement of patient frailty adds minimally to this. Our study suggests that residual confounding due to lack of direct frailty measurements can be minimized by study design and analytic techniques accounting for the aforementioned variables. Overall, our study adds to the understanding of the role of potential confounding by frailty in comparative effectiveness studies of antidiabetic medication regimens using data from administrative sources.

Funding

This work was supported by Veterans Affairs Clinical Science Research and Development (CX000570-06 to C.L.R.); by National Institutes of Health – National Institute on Aging (R01AG043471 to C.G.G.); by Vanderbilt Center for Diabetes Translation Research (P30DK092986 to C.L.R.). This work was supported by the VA Office of Academic Affiliations Quality Scholars Program (C.A.P., J.Y.M.) and the Department of Veterans Affairs, Veterans Affairs Health Services Research and Development Service, Veterans Affairs Information Resource Center (project numbers SDR 02-237 and 98-004; support for Veterans Affairs/Centers for Medicare & Medicaid Services data).

Conflict of Interest

None declared.