Assessing residual confounding of the association between antipsychotic medications and risk of death using survey data

Abstract

Objectives

Studies using linked claims databases found that conventional antipsychotic medications (APMs) were associated with short-term mortality compared with atypical APMs. It has been suggested that such results may be due to residual confounding by factors that cannot be measured in claims databases. Using detailed survey data we identified the direction and magnitude of such residual confounding.

Design

Cross-sectional survey data

Setting

Medicare Current Beneficiary Survey (MCBS)

Participants

17,776 survey participants ≥ 65 years

Measurements

To determine the association between conventional APM use and potential confounding factors we assessed 5 factors not measured in Medicare claims data but in the MCBS: body mass index, smoking, activities of daily living (ADL) score, cognitive impairment, and Rosow-Breslau physical impairment scale. We estimated adjusted associations between these factors and APM use. Combined with literature estimates of the independent effect of confounders on death we computed the extent of residual confounding caused by a failure to adjust for these factors.

Results

Comparing conventional APM users with atypical APM users, we found that not adjusting for impairments in ADL score led to an underestimation of the association with death (-13%) as did a failure to adjust for cognitive impairment (-7%). All 5 unmeasured confounders combined resulted in net confounding of -5% (range: -19% to +2%). After correction, the reported association between conventional APM use and death compared with atypical APM use was slightly increased from RR= 1.37 to RR = 1.44 (95% confidence interval: 1.33 to 1.56). Comparing any APM use with non-users would result in overestimations of > 50% if cognitive impairment remained unadjusted.

Conclusion

Claims data studies tend to underestimate the association of conventional APMs with death compared with atypical APMs because of residual confounding by measures of frailty. Studies comparing APM use with non-users may substantially overestimate harmful effects of APMs.

Background

Antipsychotic medications (APMs) are disproportionately used in the elderly and are prescribed to over a quarter of Medicare beneficiaries in nursing homes.^1,2,3 Reasons for this APM use include dementia, delirium, psychosis, agitation, and affective disorders, with much use outside FDA-approved indications.^4,5 In addition to rising use, there have been rapid shifts from first-generation conventional agents (e.g., haloperidol, phenothiazines, and butyrophenones) to heavily marketed second-generation atypical agents (e.g., aripiprazole, clozapine, olanzapine, quetiapine, risperidone, and ziprasidone).⁶

In a Public Health Advisory in April 2005, the US Food and Drug Administration (FDA) warned that atypical APMs nearly doubled the risk of death vs. placebo in 17 short-term randomized controlled trials among an elderly population with dementia.⁷ “Black box” warnings were added to labels of all atypical APMs describing these risks, but the advisory did not extend to conventional APMs, although the FDA noted that this is an important issue to study in the future.^7,8 Subsequent pharmacoepidemiologic studies showed a consistent 30% increase in short-term mortality (risk of death within 180 days) in new users of conventional APMs compared with atypical APMs.^9,10

Pharmacoepidemiologic studies that relied on large health care utilization data to be adequately powered for identifying a difference in short-term mortality and to reflect routine clinical prescribing of APMs have come under criticism for their lack of information on important potential confounders such as limitations in cognitive and physical functioning, and other patient factors that may be related to death.¹¹ It was argued that such factors may have led to selective prescribing of conventional APMs to patients closer to death,¹¹ which would result in an overestimation of the association between APM use and short-term mortality.¹² If one could assess the amount of residual confounding by unobserved factors in epidemiologic studies and correct the observed association for such bias, it would be possible to obtain better adjusted estimates of the association between conventional APM use and death. We have applied these methods to assess residual confounding caused by selective prescribing in other studies on drug effects.^13,14,15

We sought to assess the magnitude of confounding bias caused by factors not observed in Medicare utilization data, including information on body mass index (BMI), smoking, and functional and cognitive impairment that was available in a representative survey of Medicare beneficiaries. This information was used to correct estimates of an association between atypical and conventional APMs and short-term mortality based on existing claims data studies.^9,10

Methods

Design

Linked Medicare claims data contain information on physician services provided and diagnoses recorded, hospital discharge diagnoses and procedures, and detailed pharmacy dispensing information. We identified five patient characteristics not measured in Medicare utilization data but available in a representative survey of Medicare beneficiaries that could act as confounders in a study of conventional vs. atypical APM use and short-term mortality: BMI, current smoking status, activities of daily living (ADL) score, cognitive impairment, and physical impairment. We then assessed their association with the use of conventional APM, atypical APM, or neither, based on data from a detailed in-home survey of Medicare beneficiaries.

The Medicare Current Beneficiary Survey (MCBS) was conducted in a sample of beneficiaries selected each year to be representative of the current Medicare population, including both aged and disabled beneficiaries living in the community or in institutions.¹⁶ The MCBS slightly over-sampled disabled patients (under 65 years of age) and the oldest-old (85 years of age or over).¹⁶ Data were obtained from face-to-face interviews by trained interviewers in the beneficiaries’ homes or facilities. In the community interview, an effort was made to interview all subjects directly. If a person was unable to answer all questions, a proxy respondent, usually a family member or close acquaintance, was asked to answer the questions. All MCBS participants were interviewed every 4 months and followed for up to 4 years. Each year, a supplemental sample was drawn and persons were added to the MCBS sample to account for growth in the Medicare population and to replenish the sample for surveyed seniors who died, left the survey population after 4 years, or were lost to follow-up.¹⁶ The survey was reported to have a high response rate (between 85% and 95%) and very high data completeness.^17,18

Study Subjects

The MCBS sample was drawn from an enrollment list of all persons entitled to Medicare at the beginning of both study years, 2001 and 2002, and represented persons who were continuously enrolled throughout the full calendar year in the United States and Puerto Rico. The total sample size of the 2001 and 2002 MCBS was 25,587 subjects. For the present analysis, the study population was restricted to persons living in communities (23,120), which had 98.6% complete information of all critical items. Among these we further restricted the study population to MCBS respondents 65 years or older (17,776). The study was approved by the Center for Medicare and Medicaid Services and the Institutional Review Board of the Brigham and Women's Hospital.

Assessment of Medication Use

At every interview, drug names were recorded by the interviewer from medication bottles of participants in a free-text field of the questionnaire and subsequently checked and classified. ¹⁹ MCBS respondents were divided into three categories according to their drug use: (1) respondents who used conventional APMs in 2001 or 2002; (2) respondents who used atypical APMs in 2001 or 2002; and (3) respondents who used neither in 2001 and 2002 were classified as non-users. Users of conventional APMs who also used atypical APMs were excluded.

Assessment of Potential Confounders Measured in the MCBS

Potential confounders of interest assessed in the MCBS but unobserved in Medicare claims data included: BMI (weight in kg / [height in m]²), current smoking status, activities of daily living (ADL) score,²⁰ cognitive impairment,²¹ and Rosow-Breslau physical impairment scale.²² Body mass index was dichotomized at a cutpoint of 30, according to WHO's definition for obesity. Smoking status was categorized into current versus former and never. A modified and validated ADL score was obtained from the MCBS data. ²³ The Rosow-Breslau scale was computed according to the original instructions except for substituting “difficulty climbing stairs” with “difficulty stooping or kneeling” since the former but not the latter is a data item available in the MCBS. Cognitive impairment was ascertained from questions about Alzheimer's disease and memory loss.²¹

Statistical Analyses

Using the MCBS study population, we estimated the prevalence of exposure, p(E), and the prevalence of potential confounders, p(C), and the association between exposure and confounder, OR_EC. We used logistic regression to calculate the corresponding age-sex adjusted OR_EC, which was used for all subsequent analyses. We initially assumed the null hypothesis that there is no association between conventional APM use and death (RR_ED = 1). This assumption was only made to develop the formulas necessary to assess the magnitude of bias, but does not require that the true association really is 1.²⁴ We derived estimates of the confounder-disease associations, RR_CD, from the current medical literature. Literature estimates were derived from large cohort studies after an intensive literature search and expert consultations. If several valid literature estimates were identified, the average value was chosen for the base-case analysis. Based on these estimates and assumptions, a quadratic equation was derived to assess the direction and extent of residual confounding bias that would result from a failure to control for the list of these 5 possible confounders.¹⁵

We graphically explored the sensitivity to variations of our base-case literature estimate of RR_CD. The joint distribution of unmeasured confounders that were observed only in the MCBS was not assessed because literature estimates were not available for many confounder combinations. Instead, we summed bias estimates over all confounders weighted by the prevalence of each confounder in the MCBS population. Finally, we calculated the maximum range of bias by summing all negative biases to yield a lower-bound estimate and all positive confounders to yield an upper bound. A spreadsheet including all calculations is available at www.drugepi.org.²⁵

Results

Among the community sample respondents, 17,776 were 65 and older and 42% were men (3,655). Table 1 shows characteristics of atypical APM users compared with conventional APM users as well as non-users. Users of atypical APMs more frequently had an ADL score ≥1 (58%) than those using conventional (45%) or non-users (29%), indicating their greater limitations in performing activities of daily living. Atypical APM users were more likely to have cognitive impairments (61%) than conventional APM users (52%) or non-users (12%). The distribution of physical impairments according to the Rosow-Breslau score as well as high BMI status were fairly similar among users of APMs.

Table 1.

Body mass index, smoking, activities of daily living (ADL) score, cognitive impairments, and Rosow-Breslau work/walking impairment scale among community-dwelling Medicare beneficiaries 65 years or older using antipsychotic medications (APM) in 2001 and 2002 (n = 17,776).

	Atypical APM users (N=192)			Conventional APM users (N=101)			Non-users (N=17,483)
	N	%	95% CI	N	%	95% CI	N	%	95% CI
Gender
Female	116	60.42	[53.5, 67.3]	56	55.45	[45.8, 65.1]	10,039	57.42	[56.69, 58.15]
Male	76	39.58	[32.7, 46.5]	45	44.55	[34.9, 54.2]	7,444	42.58	[41.85, 43.31]
Age
65 to 74	67	34.90	[28.2, 41.6]	38	37.62	[28.2, 47.1]	7,421	42.45	[41.72, 43.18]
75 and older	125	65.10	[58.4, 71.8]	63	62.38	[52.3, 71.8]	10,062	57.55	[56.82, 58.28]
BMI
BM<30	169	88.02	[83.4, 92.6]	85	84.16	[77.0, 91.3]	14,140	80.88	[80.3, 81.5]
BMI≥30	23	11.98	[7.3, 16.5]	16	15.84	[8.7, 23.0]	3,343	19.12	[18.5, 19.7]
Smoking status
Current	12	6.28	[2.8, 9.7]	9	9.00	[3.4, 14.6]	1,824	10.45	[10.0, 10.9]
Former	96	50.26	[43.2, 57.4]	44	44.00	[34.3, 53.7]	8,500	48.68	[47.9,49.4]
Never	83	43.36	[36.3, 50.4]	47	47.00	[37.2, 56.8]	7,137	40.87	[40.1, 41.6]
ADL score
≥ 1 points	112	58.33	[51.4, 65.3]	45	44.55	[34.9, 54.2]	5,005	28.63	[28.0, 29.3]
0 points	80	41.67	[34.7, 48.6]	56	55.45	[45.8, 65.1]	12,478	71.37	[70.7, 72.0]
Cognitive impairment
Yes	115	60.85	[53.9, 67.8]	53	52.48	[42.7, 62.2]	2,045	11.73	[11.3, 12.2]
No	74	39.15	[32.2, 46.1]	48	47.52	[37.8, 57.3]	15,388	88.27	[87.8, 88.8]
Rosow-Breslau
≥ 1 impairments	163	85.79	[80.8, 90.8]	87	87.00	[80.4,93.6]	12,987	74.44	[73.8,75.1]
0 impairments	27	14.21	[9.3, 19.2]	13	13.00	[6.4, 19.6]	4,459	25.56	[24.9, 26.2]

Literature estimates of the associations between the individual possible confounders and the risk of death are shown in Table 2. Residual confounding bias in claims data risk estimates of an association between conventional APM use and the short-term risk of death were calculated separately for each of the five potential confounders (Table 3). Confounding bias due to failure to observe and adjust for each of five potential confounders was expressed as percent bias of the observed estimate from claims data. Uncontrolled ADL score was the strongest confounder, causing an underestimation by -13.0% when comparing conventional APM users with atypical APM users, followed by cognitive impairment (-7%, see Table 3). The net confounding bias, expressed as the sum of all component biases weighted by the population prevalence of each confounder, was -5%. The range from the most extreme negative bias to the most extreme positive bias was from -19% to +2%.

Table 2.

Relative Risk Estimates^* of the Associations Between Selected Potential Confounders and Death in Elderly Populations from the Medical Literature.

	Potential confounder variables
	Obesity (BMI ≥ 30)	Smoking (current vs. never)	ADL score (≥ 1 points)	Cognitive impairment (yes vs. no)	Rosow-Breslau (≥1 impairments)
EPESE studies †			3.70		1.37
Branch et al. ‡			3.66
Ramos et al. §			3.00
Corrada et al. ∥	1.12
Freedman et al. ¶	1.15
Hu et al. ¥	1.69
Doll et al. β		2.50
Dewey et al. Ψ				2.63
Mean	1.32	2.50	3.45	2.63	1.37

^*In case of conflicting literature estimates the mean estimate was used.

^†From the Duke and Iowa Established Populations for Epidemiologic Studies of the Elderly (EPESE), 10,297 community-dwelling adults 65 years and older.²⁸ The relative rate estimate of 3.7 for ADL score is the average estimate of the odds ratios for East Boston (4.0), Iowa (4.2), and New Haven (3.0, see Table 2 of original paper). The relative rate of 1.37 for the Rosow-Breslau Score is an estimate from the curves on Figure 3 of the original paper.

^‡From a prospective study on 1,625 community-dwelling elderly (age 65+) in Massachusetts, Table 2.²⁹

^§From a sample of 1,667 older (age 65+) urban residents in Brazil, Table 2.³⁰

^∥From a sample of 13,978 elderly residents in a retirement community in California, Table 2.³¹

^¶From a nationwide US cohort of 83,744 radiologic technologists aged 55+.³² The relative risk of 1.15 was calculated from data provided in Table 3 of original paper.

^¥From a cohort of 116,564 female registered nurses in the US.³³ Relative risk of 1.69 was calculated from exposure and death data provided in Table 3 of original article.

^βFrom the 50-year longitudinal study of 34,439 male British doctors.³⁴ Relative risk of 2.50 was calculated as the average from the mortality ratio of doctors aged 65-74 years (2.7) and mortality ratio of doctors aged 75-84 years (2.2), provided in Table 5 of original paper.

^ΨFrom a review of the available literature on cognitive impairment and mortality in community-dwelling elderly (age 65+), Table 4.³⁵

Table 3.

Quantitative Assessment of Confounding Bias in Risk Estimates of Conventional APM Users vs. Atypical APM Users and Death.

	RRCD	p(C)	Crude OREC	Adjusted OREC†	True RRED	p(E)	Apparent RRED‡	Percent bias§
Potential confounder:*
Obesity (BMI ≥ 30)	1.32	0.13	1.38	1.38	1.00	0.34	1.01	1.16
Smoking (current vs. never)	2.50	0.07	1.32	1.11	1.00	0.34	1.01	0.94
ADL score (≥ 1 points)	3.45	0.54	0.57	0.59	1.00	0.34	0.87	-13.19
Cognitive impairment (yes vs. no)	2.63	0.57	0.71	0.72	1.00	0.34	0.93	-6.68
Rosow-Breslau (≥ 1 impairments)	1.37	0.85	1.11	1.18	1.00	0.34	1.01	0.58

^*Potential confounder variable assessed in the MCBS but not in Medicare claims data.

^†Age- and sex-adjusted. The adjusted and not the crude OR_EC estimate was used in the subsequent analyses.

^‡Apparent relative risk between exposure (conventional APM use) and disease outcome (death) if the potential confounders were not controlled under the assumption that the true relative risk RR_ED equals 1. The apparent relative risk was also called ‘confounding risk ratio’³⁶ = apparent RR_ED / true RR_ED.

^§Percent bias = [apparent RR_ED – true RR_ED) / true RR_ED] * 100.

When comparing conventional APM users with non-users (Table 4), we computed a large positive bias caused mostly by an unobserved ADL score (+23%) and cognitive impairment (+57%), which resulted in a net confounding bias of +7% (-5% to +83%). When comparing atypical APM users with non-users (Table 4), there was a slightly stronger bias caused by unobserved ADL scores (+42%) and cognitive impairment (+68%), yielding a net confounding bias of +11% (-5% to 113%).

Table 4.

Quantitative assessment of confounding bias in risk estimates of APM use and death using different referent groups.

	% Bias**
	Conventional APM (101) vs. Atypical APM (192)	Conventional APM (101) vs. non-users (17,483)	Atypical APM (192) vs. non-users (17,483)
Potential confounder:*
Obesity (BMI ≥ 22)	1.16	-2.31	-0.80
Smoking (current vs. never)	0.94	-2.80	-3.95
ADL score (≥ 1 points)	-13.19	22.87	42.30
Cognitive impairment (yes vs. no)	-6.68	56.76	67.96
Rosow-Breslau (≥ 1 impairments)	0.58	3.70	3.22
Net confounding:
Sum of all negative biases:	-18.71	-5.11	-4.75
Weighted average:	-4.73	6.83	10.56
Sum of all positive biases:	1.52	83.33	113.48
Weighted average, excluding obesity†:	-5.11	12.77	17.87

^*Potential confounder variable assessed in the MCBS but not in Medicare claims data.

^**Bias = [apparent RR_ED – true RR_ED)/true RR_ED] * 100.

^†Because of the “U”-shaped relationship between body mass index and mortality that would violate the assumption of monotonicity of a dose-response relationship, we computed the weighted bias average without the binary obesity indicator variable.

Bias estimates of the potential confounder ADL impairment and cognitive impairment were somewhat sensitive to changes in the estimate of the confounder-death associations derived from the literature (Figure 1a, and b).

Figure 1.

Sensitivity of bias estimates towards misspecification of selected confounder-disease association (RR_CD).

Discussion

Two recent cohort studies using large linked health care utilization databases have consistently found that conventional APM use increases the risk of death in elderly patients compared with atypical APMs.^9,10 One of these studies has been questioned because of the lack of information in claims data on important potential confounders such as cognitive, physical, and functional impairment.¹¹ Our analysis demonstrates that confounding bias by patient factors that are not measured in claims data will not lead to an overestimation of the increased risk of death by conventional APMs but rather suggests a moderate underestimation. This result is mainly based on the observation that users of atypical APMs have more cognitive and functional impairment than those treated with conventional APMs.

A tendency that frail elderly patients with some behavioral disturbances are more likely to use APMs and are more likely to use atypical APMs has been observed before.⁴ In the present study we used three patient factors, cognitive, physical, and functional impairment, as proxies for frailty in elderly patients. Applying the resulting bias estimates to a recent cohort study based on Medicare claims data on the use of conventional versus atypical APMs⁵ resulted in an increase in the relative risk measure from 1.37 to 1.44 (Table 5). This result demonstrates the relatively small change of the association after adjusting for 5 important additional patient factors not observed in Medicare claims data. More importantly, it underlines that the observed increased risk of death in patients initiating atypical APMs is unlikely to be explained by confounding.

Table 5.

Reported Associations between Conventional APM Users vs. Atypical APM Users and Death in Elderly Patients.

	Relative risk estimates
	Original claims data analyses	Corrected claims data results using the weighted bias estimate	Corrected claims data results using the ADL bias estimate only	Corrected claims data results using the cognitive impairment bias estimate only
Wang et al. NEJM, 2005:	1.37 (1.27-1.49)	1.44* (1.33-1.56)	1.58 (1.46-1.72)	1.47 (1.36-1.60)
Schneeweiss et al. CMAJ, 2007:	1.32 (1.23-1.42)	1.39† (1.29-1.49)	1.52 (1.42-1.64)	1.41 (1.32-1.52)

^*1.37/(1-0.047) = 1.44

^†1.32/(1-0.047) = 1.39

Our approach to assess direction and magnitude of unobserved confounding in claims data makes several simplifying assumptions. Exposure, confounder, and outcome were all coded as dichotomous variables. While this may not be of concern for the outcome of interest (mortality) and the drug use categories, for some confounders it may be an oversimplification, e.g., ADL score. Choosing alternative cut-points in confounder variables like ADL score may change the strength of an association; this is not likely if the underlying dose-response relationship is monotonic.²⁶ Because of the “U”-shaped relationship between body mass index and mortality that would violate this assumption of monotonicity of a dose-response relationship, we computed the weighted bias average without the binary obesity indicator variable, which substantially increased the weighted bias estimate for the comparison between any APM and non-users. For our estimation of bias we assumed that the unobserved “true” drug-death association was 1. If the unknown true association is different from 1, our estimation of bias may be slightly inconsistent. However, the closer an association is to the null, the less our bias estimate will diverge from the true bias. Finally, we did not consider the joint distribution of unmeasured confounders. Instead, we computed a weighted sum of each potential confounder observed in the survey data as an approximation of the net bias. Although the extremes of this range assuming independence of individual biases are unlikely, their use will lead to a conservative interpretation of the data.²⁵

Our bias estimates regarding cognitive and functional impairment when comparing any APM versus non-users were somewhat sensitive to the choice of the independent effect of cognitive and functional impairment on mortality as derived from the medical literature. This may not be surprising since in older patients APMs are frequently prescribed to patients with dementia and behavioral disturbances who have a higher risk of death so that any small change in the independent effect of the confounders must have influenced the estimate of bias meaningfully. The confounding bias arising in the comparison between conventional versus atypical APMs was much less sensitive to variation in the literature estimate.

Valid bias assessment depends on the survey being performed in a representative sample of the main claims data study. Given that the MCBS was designed to be representative for Medicare beneficiaries as well as the high response rate and data completeness, it is a valid and readily available source for bias assessment. Generalizability is slightly compromised by the fact that the MCBS time periods were not entirely overlapping with the studies the results were applied to, and the fact that the oldest-old were oversampled in MCBS. This analysis is limited to 5 unobserved confounders and does not address other sources of bias that may affect each of the observational studies in different ways in addition to covariates that should be routinely adjusted in claims data studies, including prior medication use, co-medications, and health services use.²⁷ Each of the 5 unobserved confounders may have been reported with some degree of random misclassification, which would limit the ability to capture fully the confounding factor in the survey and would lead to an underestimation of bias. The survey used for this analysis was of limited size, occasionally resulting in wide confidence intervals. For the assessment of bias, the width of the confidence intervals as a measure of estimation precision has no implications as long as associations are estimated validly. Another limitation of MCBS is its cross-sectional nature that does not fully rule out the possibility that drug exposure resulted in changes of the measured characteristics and not vice versa.

Claims data studies tend to underestimate the association of conventional APMs with death compared with atypical APMs because of residual confounding by frailty measures. Studies comparing APM use with non-users may substantially overestimate harmful effects of APMs.