Instrumental variable applications using nursing home prescribing preferences in comparative effectiveness research

Abstract

Purpose

Nursing home residents are of particular interest for comparative effectiveness research given their susceptibility to adverse treatment effects and systematic exclusion from trials. However, the risk of residual confounding due to unmeasured markers of declining health using conventional analytic methods is high. We evaluated the validity of instrumental variable (IV) methods based on nursing home prescribing preference to mitigate such confounding, using psychotropic medications to manage behavioral problems in dementia as a case study.

Methods

A cohort using linked data from Medicaid, Medicare, Minimum Data Set and Online Survey, Certification and Reporting for 2001– 2004 was established. Dual-eligible patients ≥65 years who initiated psychotropic medication use after admission were selected. Nursing home prescribing preference was characterized using mixed-effects logistic regression models. The plausibility of IV assumptions was explored, and the association between psychotropic medication class and 180-day mortality was estimated.

Results

High- and low-prescribing nursing homes differed by a factor of 2. Each preference-based IV measure described a substantial proportion of variation in psychotropic medication choice (β(IV→treatment): 0.22–0.36). Measured patient characteristics were well balanced across patient groups based on instrument status (52% average reduction in Mahalanobis distance). There was no evidence that instrument status was associated with markers of nursing home quality of care.

Conclusion

Findings indicate that IV analyses using nursing home prescribing preference may be a useful approach in comparative effectiveness studies, and should extend naturally to analyses including untreated comparison groups, which are of great scientific interest but subject to even stronger confounding.

INTRODUCTION

Comparative effectiveness research (CER) studies of medications that rely on linked administrative data have been criticized for having incomplete information on potential confounders (predictors of study outcomes that might lead to selective prescribing), and thus result in biased estimates of effect ^{1, 2}. Instrumental variable (IV) methods have been proposed as a potential approach to control such confounding. An IV is a factor that is (i) associated with the treatment, and (ii) independent of the outcome given the treatment and the confounders. Substituting for the exposure (i.e., treatment) with an unconfounded instrument and then estimating the effect of the instrument on the study outcome will lead to an unbiased estimate, even if important confounding variables are unmeasured ^3–6. IV analyses rest on finding valid and reasonably strong instruments in the observed data. To date, instruments in CER have utilized naturally-occurring random exposure variation between high-level characteristics of typically hierarchically structured healthcare systems, including physician-level prescribing preference, hospital or health plan formulary structure, or geographic variation ^6–8.

Elderly nursing home (NH) patients represent a fast growing segment of the population that is of particular interest for CER, given these patients’ susceptibility to adverse treatment effects and their systematic exclusion from randomized controlled trial populations ⁹. However, in this population, the risk for residual confounding is high, even after adjusting with conventional multivariable and propensity score methods because of confounding by unmeasured or insufficiently characterized frailty. Frailty and other measures of declining health are poorly measured confounders in older adults, can promote treatment in some situations and discourage it in others ¹⁰, and such selective prescribing can lead to highly biased associations between drug use and outcomes ^{11, 12}. The feasibility and validity of using high-level variation in healthcare as an instrument in non-randomized studies involving NH populations has not previously been explored in a systematic way. The objective of this study was to explore the presence of unexplained between-NH variation in prescribing and to empirically evaluate the validity of instruments based on NH prescribing preference. The influences on prescribing in the US NH sector have been proven to be multi-factorial, and include the culture and context (e.g., federal regulations) within which the NH operates. A facility’s level of use of specific medication classes has been described as a visible artifact of deeper cognitive processes shared by different healthcare providers within that NH^{13, 14}. Our proposed IV aims to isolate the portion of the between-NH variation in prescribing attributable to such institutional ‘preference’. The use of different psychotropic medication classes to manage behavioral problems associated with dementia provides a useful context to explore this research question in light of the considerable uncertainty with respect to the preferred treatment leading to heterogeneous treatment patterns ^15–20 and persisting safety concerns ^20–29. Throughout this manuscript, we use the term ‘effectiveness’ to refer to all effects of a treatment, including intended and unintended effects. The empirical example is focused on mortality, an outcome whose risk most treatments try to actively reduce and not cause unintentionally.

METHODS

Source data and Study population

The study cohort was drawn from a merged dataset consisting of Medicare and Medicaid claims, the Minimum Data Set (MDS) and the Online Survey, Certification and Reporting (OSCAR) dataset in 8 US states (California, Florida, Georgia, Illinois, New Jersey, New York, Ohio, Texas) for the years 2001–2004. The claims data provided information on patient demographics, Medicaid eligibility, all physician services and hospitalizations with diagnostic and procedure coding, admissions to long-term care, and dispensings of prescription drugs. The MDS is a federally mandated health assessment tool used in US nursing homes that captures information on physical, psychological and psychosocial functioning, active clinical diagnoses, health conditions, treatments and services. OSCAR is a compilation of all the data elements collected during the inspection survey conducted at nursing facilities for the purpose of certification for participation in the Medicare and Medicaid programs, including NH operational characteristics and aggregate resident characteristics.

We assembled a cohort of subjects ≥65 years who were dually eligible for Medicare and Medicaid, entered a NH for a long-term stay, and initiated treatment with atypical antipsychotic medications (APM), antidepressants, or hypnotics. Patients residing in NHs that contributed five or fewer patients during the 4-year study period were excluded because such small clusters contribute little information to the estimate of between-NH variability in prescribing. (eAppendix A for list of medications).

Resident and facility characteristics

Socio-demographic characteristics included age, sex, race, and education. Clinical characteristics (i.e., case-mix variables) were determined based on the most recent MDS assessment before the first dispensing of a psychotropic medication in the NH, claims-based ICD-9 diagnostic and procedure codes associated with hospitalizations and physician visits before NH admission, as well as medication use prior to the first dispensing of a psychotropic medication. Clinical variables considered included psychiatric morbidity, cardiovascular morbidity, cerebrovascular disease, Parkinson’s disease, epilepsy, diabetes, obesity, and functional impairment. The Charlson Comorbidity score, number of physician visits, number of hospitalizations, and number of distinct prescription drugs — excluding psychotropic medications —were used as generic markers of co-morbidity.

Facility characteristics included variables such as number of beds, occupancy rate, availability of special care units, staffing, type of ownership, proportion of residents paid for by Medicare, Medicaid, or other sources, geographic region, as well as case-mix variables related to resident characteristics (e.g., proportion of residents with dementia, depression) and quality indicators (e.g., proportion of residents bed- or chair-bound, with restraints, with bedsores, number of deficits). For each NH, we also estimated the psychotropic medication prescribing rate, defined as the proportion of all residents who were admitted to the NH during the study period and who were prescribed a psychotropic medication (see eAppendixB for a complete list of resident- and facility-level covariates).

Statistical analysis

The three contrasts of interest were antidepressants versus atypical APMs, antidepressants versus hypnotics, and atypical APMs versus hypnotics. Since all patients in the cohort were treated, any potential effect of prescribing itself should apply equally to all cohort members.

Define instrumental variable: nursing home prescribing preference

To estimate NH-specific prescribing rates adjusted for case-mix and facility characteristics, mixed-effects logistic regression models were fit ^{30, 31}. The NH identifying variable was modeled as a random effect, with each of the random-effects parameters representing the NH-specific prescribing rate in relation to the mean rate predicted based on patient and facility characteristics. The resident and facility characteristics were modeled as fixed effects (eAppendix C). Because inclusion of a large number of covariates can lead to convergence problems in multilevel models with relatively small cluster sizes, adjustment for resident and facility characteristics was done through the use of summary propensity scores rather than individual variables. These propensity scores were derived from predicted probabilities of one medication class versus another estimated in logistic regression models. We centered the propensity scores on the mean and estimated the probability that the average patient, defined as the patient with a mean propensity score, would be treated with a particular medication class in each NH.

To create IV measures, we assessed the distribution of random intercepts from the fully-adjusted mixed-effects model and found the values that provided median, tertile and quartile cutoffs. Each patient was then uniquely assigned to median, tertile and quartile groups using these cutoffs, based on the random intercept of his/her NH. Patients assigned to the extreme groups (i.e., below vs. above median [IV₁]; lowest vs. highest tertile [IV₂]; lowest vs. highest quartile [IV₃]) were included in the respective IV analyses. Patients residing in NHs that fall in the highest quantile were assigned the index treatment (predicted treatment or IV status); patients residing in NHs that fall into the lowest quantile were assigned the reference treatment.

Explore plausibility of IV assumptions

To assess instrument strength, we computed the square of the semi-partial correlation between the instrument and the treatment, conditional on other covariates in the model. This semi-partial R² can be interpreted as the proportion of variance explained by the addition of the IV to the model.

Two falsification tests were conducted. To assess whether the instrument provided a natural experiment in treatment choice, we compared the distribution of measured covariates between patients residing in NHs with the highest and lowest quantile groups based on the random intercept. If the variation in prescribing captured by the instruments stems mainly from practice style, measured and unmeasured patient characteristics should be distributed similarly across patients grouped by instrument values. We measured change in imbalance of measured covariates, comparing the population as stratified by the treatment versus stratified by the IV. We used the % change in Mahalanobis distance ³² - a multivariable extension of the standardized mean difference which accounts for observed covariance – as a summary measure of change in covariate balance between exposure groups. The underlying assumption is that if the IV improves balance in observed patient characteristics, then it is reasonable to expect that it will also improve balance in unobserved variables ⁶.

To evaluate empirically whether the IV might be related to the outcome (other than through exposure), we examined the association between the IV and other markers of structural or process quality of care in nursing homes (e.g., the proportion of residents with facility acquired bedsores, the proportion on APMs, the number of documented deficits) that are likely associated with health outcomes.

Treatment effect estimates

Finally, we computed absolute differences in the risk of 180-day mortality following treatment initiation between different psychotropic medication classes, using a two-stage least squares regression with and without adjustment for measured covariates, implemented as a generalized linear model. In the Appendix, we also present relative risks based on a two-stage logistic regression model using the residual inclusion method ³³. We compare the findings with those from conventional multivariable outcome modeling using a first exposure carried forward approach for consistency with the instrumental variable approach. Because of the high variance of the IV estimates, treatment effects estimated with different instruments may diverge substantially, even if all the IV estimators are, on average, unbiased ³⁴. We therefore assessed whether the operational definition of the instrument (tertile, quartile, etc) affected the treatment effect estimates and their precision. All analyses were performed using the SAS system (SAS Institute Inc., Cary, NC).

RESULTS

We identified 52,062 long-stay residents in 4,483 NHs who initiated treatment with psychotropic medications following admission: 32.4% were treated with an atypical APM, 41.8% with an antidepressant, and 44.8% with a hypnotic agent (proportions add up to more than 100% since patients can be initiated on more than one psychotropic medication class). For each of the three comparisons, patients who simultaneously initiated both treatments of interest were excluded.

Nursing home prescribing preference

We observed large between-NH variation in the unadjusted prescribing rates for different psychotropic medication classes over the 4-year period, ranging from 0 to 100% with about 10% of NHs never prescribing a given class (eFigure 1).

If these observed differences in prescribing rates were due to chance variation alone or were fully explained by differences in measured resident and/or facility characteristics, the estimate of the between-NH variance (σ_b²) from a mixed-effects logistic regression model would be close to zero and all NHs would have a predicted prescribing rate around the mean. Depending on the comparison of interest, prescribing rates for the average patient in the adjusted model for the central 95% of the NHs ranged from 42.7% to 72.0% for antidepressants vs. atypical APMs, from 31.5% to 64.8% for antidepressants vs. hypnotics, and from 23.9% to 58.4% for atypical APMs vs. hypnotics (Table 1). Although the variability is reduced compared to the unadjusted models, these results indicate that substantial between-NH variation in prescribing practices remains after controlling for resident and facility characteristics (eFigure 2).

Table 1.

Estimated Prescribing Rates for Psychotropic Medications, Based on Mixed-Effects Logistic Regression Models With Patient- and Facility-Level Adjustments

Models	σb2 (b)	Predicted probability that the average patient will initiate treatment with a given psychotropic medication
		Average nursing home(c)	Range
			2.5 %ile	97.5 %ile
Antidepressants (index) vs. Atypical APMs (referent)		% Antidepressants
Unadjusted	1.17	60.7%	15. 7%	92.6%
Adjusted(a) for calendar year, patient characteristics and nursing home characteristics(d)	0.10	58.0%	42.7%	72.0%
Antidepressants (index) vs. Hypnotics (referent)		% Antidepressants
Unadjusted	0.18	48.2%	28.8%	68.2%
Adjusted for calendar year, patient characteristics and nursing home characteristics	0.12	47.9%	31.5%	64.8%
Atypical APMs (index) vs. Hypnotics (referent)		% Atypical APMs
Unadjusted	1.22	38.2%	6.7%	84.3%
Adjusted for calendar year, patient characteristics and nursing home characteristics	0.15	39.9%	23. 9%	58.4%

^(a)All adjustments made through estimation of a propensity score. The c-statistic for the propensity score regression models ranged between 0.52 and 0.82.

^(b)Estimate of the between-NH variation. The random intercept b_i is assumed to be normally distributed with mean 0 and variance σ_b². σ_b² represents the NH-specific deviation from β₀, the marginal (averaged across NHs) probability of initiating a given psychotropic medication class for a patient with the mean propensity score. With increasing levels of adjustment, there is less unexplained variation and σ_b² is expected to decrease.

^(c)Prescribing proportion for the ‘average’ patient, defined as a patient with a mean propensity score. The average differs slightly between models since different factors are being adjusted for in the various models.

^(d)Nursing home characteristics include quality of care indicators

Instrumental variable estimation

Instruments based on NH prescribing practices are quite strongly associated with actual treatment, explaining 6% to 13% of the variance in exposure in adjusted analyses (Table 2). The covariate balance improved for all instruments. For the comparison of antidepressants with atypical APMs, the reduction achieved in Mahalanobis distance was −53.4%, −50.5% and −49.1% for IV₁, IV₂ and IV₃ respectively. For the comparison of antidepressants with hypnotics, the corresponding changes were −60.8%, −55.0%, and −51.4%; and for the comparison of atypical APMs with hypnotics the changes were −53.2%, −50.9%, and −52.2%. eFigure 3 illustrates the covariate balance for selected patient and NH characteristics for actual exposure and instrument status, defined based on lowest and highest tertile groups. Although the covariates are relatively well balanced by actual treatment status, the difference in prevalence (or mean for continuous variables) is centered more closely around 0 for the instrument than for the actual treatment. The most notable example is a recorded diagnosis of dementia. The prevalence difference was reduced from −27.6% to −7.6% for antidepressants versus atypical APMs, from 4.2% to 1.0% for antidepressants versus hypnotics, and from 30.3% to 9.4% for atypical APMs versus hypnotics. Large reductions in imbalance were also observed for diagnosis of depression and psychotic disorders.

Table 2.

Strength of Instrumental Variables Based on Nursing Home Prescribing Preference in Predicting Psychotropic Medication Use for Individual Nursing Home Residents

	Unadjusted			Adjusted(a)
	Risk Difference(b)	95% CI	Semi-Partial R2	Risk Difference(c)	95% CI	Semi-Partial R2
Antidepressants versus Atypical APMs(d)
IV1: Median groups (lowest vs. highest quantile)	0.325	0.306, 0.343	0.107	0.223	0.215, 0.231	0.057
IV2: Tertile groups (lowest vs. highest quantile)	0.385	0.364, 0.406	0.150	0.279	0.269, 0.288	0.085
IV3: Quartile groups (lowest vs. highest quantile)	0.397	0.374, 0.421	0.161	0.309	0.299, 0.319	0.103
Antidepressants versus Hypnotics
IV1: Median groups (lowest vs. highest quantile)	0.261	0.251, 0.270	0.068	0.248	0.241, 0.256	0.062
IV2: Tertile groups (lowest vs. highest quantile)	0.329	0.318, 0.339	0.108	0.316	0.308, 0.324	0.100
IV3: Quartile groups (lowest vs. highest quantile)	0.370	0.358, 0.382	0.137	0.356	0.347, 0.365	0.128
Atypical APMS versus Hypnotics
IV1: Median groups (lowest vs. highest quantile)	0.327	0.308, 0.346	0.110	0.226	0.218, 0.234	0.060
IV2: Tertile groups (lowest vs. highest quantile)	0.392	0.371, 0.413	0.158	0.286	0.278, 0.295	0.091
IV3: Quartile groups (lowest vs. highest quantile)	0.407	0.384, 0.431	0.173	0.320	0.310, 0.330	0.112

^(a)Adjusted for propensity score including all resident and facility characteristics

^(b)1-stage Model: X = α₀ + α₁Z + ε, with X=exposure and Z=IV

^(c)1-stage Model: X = α₀ + α₁Z + α₂PS + ε, with X=exposure, Z=IV, and PS=propensity score

^(d)For the antidepressant versus atypical APM comparison, the risk difference represents the probability of being prescribed an antidepressant if residing in a nursing home with a preference for antidepressants (i.e., NH in highest quantile) relative to the probability of being prescribed an antidepressant if residing in a nursing home with a preference for atypical APMs (i.e., NH in lowest quantile) represented by α₁ in the Model. The interpretation is similar for the other comparisons.

There was no evidence that instrument status was associated with other indicators of quality of care in NHs or other facility characteristics, providing indirect empirical support for the assumption that IV status is unlikely to be related to the outcome, other than through exposure (Table 3).

Table 3.

Association Between Instrument Status and Potential Nursing Home Quality Indicators

	IV Status (predicted treatment based on the prescribing preference of the patient’s NH)
	Antidepressants vs. Atypical APM			Antidepressants vs. Hypnotics			Atypical APM vs. Hypnotics
	AD	Atypical APM	Diff	AD	Hypnotics	Diff	Atypical APM	Hypnotics	Diff
Quality indicators
Residents bedfast	0.05	0.04	0.01	0.05	0.05	0.00	0.04	0.05	−0.01
Residents chairfast	0.59	0.58	0.01	0.59	0.59	0.00	0.59	0.58	0.01
Residents with facility acquired bedsores	0.04	0.04	0.00	0.04	0.03	0.01	0.03	0.03	0.00
Residents with restraints	0.08	0.07	0.01	0.08	0.08	0.00	0.07	0.08	−0.01
Residents on psychoactive medicines	0.60	0.59	0.01	0.60	0.59	0.01	0.58	0.59	−0.01
Residents with contractures	0.12	0.11	0.01	0.11	0.12	−0.01	0.11	0.11	0.00
Total number of deficits	7.74	6.92	0.82	7.89	7.80	0.09	7.51	7.86	−0.35
Facility level
Number of beds in facility	161.16	184.04	−22.88	151.55	153.99	−2.44	187.08	162.18	24.90
Hospital based	0.03	0.04	−0.01	0.04	0.05	−0.01	0.03	0.04	−0.01
Part of multi-facility ownership structure	0.54	0.45	0.09	0.58	0.57	0.01	0.47	0.55	−0.08
For profit	0.80	0.73	0.07	0.81	0.79	0.03	0.77	0.80	−0.03
Government ownership	0.05	0.05	0.00	0.04	0.04	−0.01	0.05	0.04	0.01
Nonprofit ownership	0.15	0.21	−0.06	0.15	0.17	−0.02	0.18	0.16	0.02
Occupancy rate	0.86	0.87	−0.01	0.85	0.85	0.00	0.87	0.86	0.01
Residents on Medicaid	0.72	0.73	−0.01	0.71	0.70	0.01	0.73	0.72	0.01
Residents on Medicare	0.10	0.11	−0.01	0.10	0.11	−0.01	0.11	0.11	0.00
Private pay/private insurance residents	0.18	0.16	0.02	0.19	0.19	0.00	0.16	0.18	−0.02
Alzheimer’s special care unit	0.24	0.22	0.02	0.20	0.22	−0.02	0.22	0.22	0.00
Staffing
Team-based physician care	0.33	0.32	0.01	0.30	0.28	0.02	0.32	0.29	0.03
Mental health staffing	0.62	0.62	0.00	0.60	0.61	−0.01	0.62	0.63	−0.01
No physician available	0.11	0.10	0.01	0.11	0.12	−0.01	0.10	0.10	0.00
Residents
With dementia	0.46	0.46	0.00	0.45	0.44	0.01	0.46	0.43	0.03
With psychiatric diagnosis	0.20	0.20	0.00	0.20	0.20	0.00	0.20	0.20	0.00

All quality indicators are presented as proportions, except for total number of deficits and number of beds

Treatment effect estimates

Finally, Table 4 (eTable 1) compares effect estimates for 180-day all-cause mortality from conventional and IV approaches. Overall, results are consistent with a somewhat increased risk of death associated with hypnotic medication use (approximately 3 more deaths per 100 patients) compared with antidepressants and atypical APMs. The estimates for actual exposure and NH prescribing preference based instruments are very similar, suggesting there was little confounding to begin with, although the effects are slightly attenuated. As expected, the 95% confidence intervals are much wider for the IV analyses due to the known inefficiency of the 2-stage IV estimation approach.

Table 4.

Risk of Death Within 180 Days After Start of Psychotropic Medication

	N	Unadjusted		Adjusted(a)
		Risk Difference(b)(c) (/100 patients)	95% CI	Risk Difference(b)(d) (/100 patients)	95% CI
Antidepressants versus Atypical APMs (=referent)
Actual treatment status	25,129	1.60	0.56, 2.65	−0.82	−2.07, 0.42
IV status
IV1: Median groups (lowest vs. highest quantile)	25,129	−0.46	−4.00, 3.07	−3.70	−9.52, 2.11
IV2: Tertile groups (lowest vs. highest quantile)	19,333	0.20	−3.21, 3.61	−2.08	−7.29, 3.13
IV3: Quartile groups (lowest vs. highest quantile)	15,926	1.27	−2.33, 4.87	−0.09	−5.07, 4.89
Antidepressants versus Hypnotics (=referent)
Actual treatment status	32,901	−5.72	−6.66, −4.77	−3.69	−4.68, −2.70
IV status
IV1: Median groups (lowest vs. highest quantile)	32,901	−3.47	−7.64, 0.70	−2.81	−7.21, 1.60
IV2: Tertile groups (lowest vs. highest quantile)	24,228	−2.73	−6.60, 1.15	−2.18	−6.23, 1.86
IV3: Quartile groups (lowest vs. highest quantile)	19,533	−3.77	−7.64, 0.09	−3.36	−7.40, 0.68
Atypical APMS versus Hypnotics (=referent)
Actual treatment status	27,810	−7.07	−8.09, −6.04	−2.69	−3.90, −1.47
IV status
IV1: Median groups (lowest vs. highest quantile)	27,810	−6.05	−9.67, −2.43	−3.57	−9.25, 2.11
IV2: Tertile groups (lowest vs. highest quantile)	21,125	−5.39	−8.88, −1.91	−2.48	−7.49, 2.52
IV3: Quartile groups (lowest vs. highest quantile)	17,217	−5.22	−8.94, −1.49	−2.44	−7.35, 2.46

^(a)Adjusted for propensity score including all resident and facility characteristics

^(b)Exposure model based on (multivariable) linear regression model; IV models based on 2-stage least squares regression model

^(c)2-stage model: Y = β₀ + δÊ[X|Z] + ε, with Y = outcome, Ê[X|Z] = predicted values from first stage

^(d)2-stage model: Y = β₀ + δÊ[X|Z, PS] + β₁PS + ε, with Y = outcome, Ê[X|Z] = predicted values from first stage, PS = propensity score

DISCUSSION

In this population of 52,062 residents who initiated psychotropic medications at admission to NHs between 2001–2004, substantial between-NH variability in treatment choice remained after accounting for differences in patient and facility characteristics suggesting there are intangible factors – by some referred to as ‘nursing home culture’ – that meaningfully influence treatment choice. Instrumental variables that exploit this unexplained variability in prescribing were strongly associated with treatment, improved the balance of measured covariates and appeared unrelated to markers of quality of care in the NH. These findings suggest that the assumptions underlying IV methods are met (or at least cannot be falsified using the available evidence) ⁶, and IV analyses using NH prescribing preference may be a useful approach in CER in NH populations when not all confounders are measured or can be practically determined.

Overall, these conclusions seem consistent with the findings from Pratt et al ³⁵ who defined facility preference more narrowly based on the medication class (atypical vs. typical APM) most frequently prescribed over a 12-month period. To the best of our knowledge, this is the first study to examine the effect of both resident- and facility-level characteristics on medication choice within the NH. This detailed clinical and facility information allowed for good control of factors potentially related to prescribing patterns, increasing the probability that the observed residual variability is due to true differences in NH preference, rather than unmeasured factors.

If there is an underlying ‘nursing home culture’ that manifests itself in a tendency to prescribe one psychotropic medication class over another, there is no a priori reason to believe that this ‘culture’ would not also affect prescribing tendencies for other medication classes, particularly when there is a weak evidence base to guide the treatment choice. The purpose of the present study was to examine whether there is any evidence that such a prescribing ‘culture’ is present, and whether an IV approach exploiting this prescribing culture appears feasible and valid in general. As is the case for any IV analysis irrespective of instrument-type, the validity of the IV assumptions will have to be empirically verified in the dataset and study population at hand for any future applications.

Whether the bias/variance trade-off is likely to favor IV estimates in a given application depends on the strength of the unmeasured confounding, the size of the exchangeable group (‘marginal patients’) and the strength of the instrument’s impact on treatment choice. The two latter parameters are directly related to the strength of the association between the IV and the actual treatment, and it has been proposed, informed by simulation studies, that IV estimates are likely to be more accurate when the β coefficient from the first stage regression of the instrument on a dichotomous exposure is >0.2 ³⁴. The NH preference based IVs in our study meet this condition. Some variation in effect estimates was observed between different IV definitions. Two mechanisms may have contributed to this. First, focusing on the extremes of prescribing preference (i.e., quartile groups) is expected to create a better estimate of true underlying preference and therefore better quasi-randomization of patients to the predicted treatment groups. Some change in effect estimate is therefore anticipated. Second, IV methods measure the effect in the marginal patient rather than the effect in the entire cohort.³ By varying the size of the cohort as a result of restriction we may also have affected who the marginal patient would be. Any measures of effect drawn from these IV definitions may therefore not be directly comparable. Including NH fixed effects in a standard regression analysis to block nursing home level sources of confounding as a possible alternative approach to IV analyses would not be advisable because conditioning on an instrument may increase both the bias and variance of the exposure effect estimates.³⁶ We illustrated the IV approach in pairwise comparisons of one medication class versus another. However, if one were interested in drawing inferences about the comparative effectiveness of multiple medication classes in the subset of patients who could reasonably be treated with any of the drug classes (i.e., in the subset of patients in whom all treatments are exchangeable), all exposure groups can be included in a single analysis. Which analysis is most appropriate (pairwise versus multiway) is driven by the decision that needs to be informed and the particular causal contrast.

In general, the findings from the IV analyses were very similar to those from the conventional adjusted analysis. Whereas this is most likely due to limited unmeasured confounding present in the original data – as suggested by the covariate balance by treatment status – the possibility of residual confounding in the IV analyses cannot be excluded. The fact that PS adjustment affects the IV estimates is compatible with the explanation of some residual confounding (e.g., dementia, delirium). As for any IV analysis, it is inherently untestable whether there might be residual imbalance in unmeasured characteristics. Clinical knowledge and observed distributions of covariates provide no basis for predicting the direction of such residual bias, if present. As there is no randomized controlled trial that compares the safety of different psychotropic medications in older nursing home patients, the results from the IV analysis cannot be compared against such a “gold standard”. Although the availability of trial evidence would certainly be useful, it should be noted that the estimates from the two approaches are not necessarily directly comparable in the presence of treatment heterogeneity ³⁷.

In summary, our findings should encourage wider application of NH prescribing preference-based IV methods in CER involving the fast growing population of elderly NH patients if residual confounding is a concern. Such IV analyses should be presented as the primary analysis if findings confirm the presence of such confounding, or as a secondary confirmatory analysis if findings suggest that the concern was unwarranted. A major advantage of using NH, as opposed to physician, prescribing preference to define the IV is that a NH-based approach extends naturally to analyses including untreated comparison groups. A nursing home’s preference for prescribing a given medication versus not prescribing that medication could be characterized similarly as we illustrated here for active comparison groups. Such comparisons between active drug users and non-users are of great scientific interest but subject to even stronger confounding that may not be resolvable with conventional analytic approaches.

Take-home messages.

• There is substantial between-nursing home (NH) variability in treatment choice after accounting for differences in patient and facility characteristics, suggesting the presence of intangible factors (‘culture’) that meaningfully influence treatment choice.

• Instrumental variables (IV) that exploit this unexplained variability in prescribing were strongly associated with treatment, improved the balance of measured covariates and appeared unrelated to markers of quality of care in nursing homes, suggesting assumptions underlying IV methods are met.

• IV analyses using NH prescribing preference may be a useful approach in comparative effectiveness research when not all confounders are measured or can be practically determined; either as a primary or as a sensitivity analysis.

• This is the first study to examine NH prescribing preference while accounting for both resident- and facility-level characteristics.

• The feasibility and validity of this approach should be tested in other examples, and extension of the methods to include untreated comparison groups should be explored.