What is the Effect of Area Size When Using Local Area Practice Style as an Instrument?

Abstract

Many researchers have used local area practice style measures as instruments in instrumental variable analysis. What constitutes the size of a “local area” for measuring practice styles may affect the strength of the relationship between the instrument and treatment choice, and whether the instrument is related to unmeasured confounding factors. Among previous studies using local area practice style measures as instruments, only two reported whether their estimates were robust to changes in the local area size. There has been no discussion on how area size may affect IV estimates when local area practice style measures are used as instruments. The objective of this study is to discuss the tradeoffs inherent in choosing a local area size when using a measure of local area practice style as an instrument. We used the effectiveness of angiotensin converting-enzyme inhibitors and angiotensin receptor blockers on survival post acute myocardial infarction as an example. Across local area size definitions we contrasted treatment effect estimates in terms of (1) the strength of the relationship between local area practice styles and individual patient treatment choices; and (2) indirect assessments of the assumption of no correlation between local area practice style and unmeasured confounders.

Introduction

Instrumental variable (IV) estimators have been recognized as useful tools to assess the effectiveness of alternative treatments in healthcare using observational data.^{1, 2} The “instruments” required in IV studies must be measured factors that are strongly related to treatment choice but are unrelated to either study outcomes or other unmeasured factors related to study outcomes. Thus, instruments essentially provide an ex post randomization of treatment choice or exposure across patients.^3–9 IV methods yield estimates of the average treatment effect for the subset of patients whose treatment choices were mutable to changes in the instrument variable or “instrument” values.^{4, 10–13} IV estimates have been labeled a local average treatment effect (LATE) and are thought most suitable to assess the effect of treatment rate changes in a population.^{11, 13–19} Many researchers have used local area practice style measures as instruments in IV analysis^20–33 which conjectures that (1) patients residing in areas where physicians have stronger preferences for a particular treatment are more apt to receive that treatment; and (2) unmeasured confounding variables are unrelated to the differential patient access to physicians with distinct treatment preferences.

Only two studies that used a local area practice style measure as instruments have reported whether their estimates were robust to adjustments in the size of the local area used to measure practice style,^{28, 33} and there has been no discussion as to the potential effects of local area size on the properties of the resulting IV estimates. The size of the local area used to measure practice style may affect both the strength of the relationship between the instrument and treatment choice and whether the instrument is related to unmeasured confounding factors. One might expect that the larger the local area around a patient residence used to measure practice style, the weaker the relationship will be between the instrument and the treatment choices for individual patients. Confounding emerges when local area practice style measures are correlated with differences in average unmeasured patient characteristics or ecological factors across areas that are related to patient outcomes. A priori relationships between local area size patient and ecological factors that may confound estimates do not generally exist. Ecological factors have been categorized as aggregate attributes (e.g. smoking rates, average health behaviors), contagion factors (e.g. flu prevalence), environmental factors (e.g. pollution, weather, sunlight hours); patterns of interaction among area individuals (e.g. social networks); and global factors (e.g. local regulations or market structures).³⁴ One can envision that smaller local area sizes could introduce correlations between practice style and unmeasured neighborhood-level cultural and health behavior-related confounders. In contrast, use of larger local area sizes may introduce correlations with regional unmeasured confounders related to regulatory structures, regional healthcare systems, and climate. It is true, however, that because relationships between individual treatment choice and local area practice style measures weaken as local area size increases, this increases the potential for unmeasured patient and ecological factors to confound the treatment effect estimates. However, the only possible approach to validate assumptions of no correlation between local area practice style measures and potential confounding factors for specific area sizes and is to obtain secondary data sources describing these factors and directly estimate the correlations by area size.

If treatment effectiveness is heterogeneous across patients, another source of confounding related to local area size needs to be considered. “Essential heterogeneity” occurs when treatment effects are heterogeneous across patients and providers make treatment recommendations based on patient characteristics that are related to expected treatment effectiveness^{24, 35–38} If the patient characteristics related to treatment effectiveness are unobserved by the researcher, we theorize that local area practice style measures will be positively correlated with average treatment effectiveness across areas causing LATE estimates to be biased toward positive treatment effectiveness. It will appear that higher treatment rates will yield better outcomes when in fact areas with higher treatment rates simply contained more patients apt to gain from treatment. However, as we discuss in Appendix A, as the number of patients used to define a local area increases, the variation in average treatment effectiveness across local areas diminishes as will the favorable bias in LATE estimates.

The objective of this study is to discuss the tradeoffs involved in choosing a local area size when using local area practice style as an instrument. We use IV methods to estimate the LATE of renin-angiotensin system antagonists including angiotensin converting-enzyme (ACE) inhibitors and angiotensin receptor blockers (ARBs) on one-year patient survival among Medicare patients post acute myocardial infarction (AMI). ACE/ARB use post-AMI provides an interesting setting for this discussion because the benefits of ACE/ARBs are known to be heterogeneous across AMI patients with greater benefit for patients at higher risk of future cardiovascular events.³⁹ ACE/ARB survival benefit estimates from randomized controlled trials (RCT) vary from 50 lives saved per 1000 high-risk patients to 5 lives saved per 1000 low-risk patients.⁴⁰ In addition, evidence shows substantial variation in ACE/ARB prescribing as only 50% of Medicare patients from two states received an ACE/ARB post-AMI in 2004,⁴¹ and large geographic variation has been reported.⁴² If providers believe ACE/ARB benefits are heterogeneous across patients, and ACE/ARB treatments are sorted across AMI patients based on expected benefits -- “essential heterogeneity”^{24, 43} – we expect that local area ACE/ARB practice styles variation will reflect moderate risk AMI patients whose ACE/ARB choices are more discretionary than either high or low risk patients. As a result, IV estimates of LATE for ACE/ARB use on one-year survival for post-AMI patients should fall between the RCT estimates described above. In addition, all else equal, if essential heterogeneity is occurring we expect that LATE estimates will be biased high and that the bias will diminish as the size of the local area increases (see Appendix A). The consistency of our estimates is also conditional on the assumptions that local area ACE/ARB practice styles are unrelated to unmeasured average patient and ecological factors that affect cardiovascular patient outcomes such as general provider access,^{44, 45} characteristics of the healthcare delivery system,^{44, 45} area socioeconomic status and homogeneity,^44–50 pollution,⁵¹ social environment and support,^{45, 52, 53} and area health behaviors and disease prevention.⁵³

We used the driving area for clinical care (DACC) method to define local areas around Medicare AMI patient residence ZIP codes.^{28, 30, 33} The DACC method enables researchers to create local area practice style measures for alternative size definitions based on threshold numbers of patients living within a specified driving time of each ZIP code. Defining local areas based on the number of patients instead of distances alone helps account for urban/rural differences in healthcare access as rural patients routinely drive greater distances for healthcare. Across local area size definitions we contrasted ACE/ARB one-year survival effectiveness estimates in terms of (1) the strength of the relationship between local area practice styles and individual patient treatment choices; and (2) indirect assessments of the assumption of no correlation between local area practice style and unmeasured confounders.

Methods

Data and Sample

All Medicare claims files, enrollment information, and Part D prescription drug events for patients hospitalized for their first AMI in 2008 that did not have AMI in 2006 and 2007 were obtained. We applied the Chronic Care Warehouse definition of AMI as an inpatient stay with an ICD-9 code 410.xx, (excluding 410.x2) in the first or second diagnosis position of the claim.⁵⁴ AMI stay admission and discharge dates were based on all Medicare institutional claims (acute, long term care hospital, inpatient rehabilitation facility, critical-access hospital, and short-term nursing facility) with overlapping admission and discharge dates following an initial acute hospital admission with an AMI diagnosis. We restricted our sample to patients discharged alive; with continuous Medicare Part A and B fee-for-service enrollment 12 months prior to their index AMI admission and 12 months post-index discharge or until death; and with continuous Part D enrollment 6 months prior to admission and 12 months post-index discharge or until death. To ensure that all Part D events were observable during a 30-day post index treatment observation period, we further excluded patients who utilized hospice or skilled nursing care; were readmitted to inpatient care; or died within the 30-days post-index discharge. Finally, because driving times between ZIP codes may have inconsistent meaning for geographically non-contiguous areas, we restricted our sample to patients living in the continental United States at AMI admission. The final sample size was 68,236.

Measures

Patients were designated as having been prescribed an ACE/ARB post-AMI if they filled a prescription for an ACE/ARB within 30 days post AMI discharge (1 if an ACE/ARB prescription was filled within 30 days post AMI discharge, 0 otherwise).⁵⁵ The study outcome was a binary variable equaling 1 if the patient survived for one year post AMI discharge, 0 otherwise. Measured covariates included patient demographics; baseline medical conditions for both the year prior to the AMI admission and during the index AMI stay; medications used during the 180 days prior to the AMI admission; AMI diagnosis-type on admission; procedures during the AMI stay; complications during the AMI stay; other medications filled immediately post discharge (statins, beta blockers); Part D variables including premium levels and deductible phase at diagnosis; whether patients were Medicaid dual-eligible in their AMI index month; and socioeconomic characteristics for the patient residence zip code (per capita income, poverty rate, education level, English speaking percentage). Full definitions of these variables are included in Appendix B.

We defined local area practice style as the average intent of physicians in an area to prescribe ACE/ARBs for patients upon AMI discharge. Because measurement of treatment intent is less clear for patients already using ACE/ARBs when admitted for an AMI, to measure intent we used the patients who did not have ACE/ARBs available at home on their index AMI admission date based on previous prescription dates and days supplied figures on Part D claims (N=43,842). For these patients, we created a variable equaling 1 if the patient filled an ACE/ARB prescription within one day of their first prescription claim in the first 30 days post-AMI discharge, 0 otherwise. With these patients and this treatment definition we measured local area ACE/ARB practice style at the patient ZIP code-level using the DACC method. This method creates “local areas” around each patient residence ZIP code by consecutively adding patients from the next closest ZIP codes based on driving times between zip codes until a threshold number of patients is reached. Distinct local area sizes were created by varying the threshold number of patients from 10 to 200 patients in increments of 10. For the patients associated by the DACC method with each ZIP code for an area size definition, we calculated ZIP code-specific area treatment ratios (ATR) as the ratio of the number of patients receiving ACE/ARBs post-AMI over the sum of the probabilities of their receiving an ACE/ARB post-AMI. ACE/ARB probabilities were predicted using a multivariate logistic model of ACE/ARB choice over the patients with no ACE/ARBs available at index using the measured covariates described above as independent variables. Patients in our full sample (N=68,236) were then assigned local practice style ATR values for each of the 20 area size definitions based on their residence ZIP code.

Analytical Approach

We applied the linear two-stage least squares (2SLS) instrumental variable estimator that has been used in several previous IV studies.^{18, 23, 24, 26, 27, 33} Linear 2SLS yields consistent estimates that are robust to underlying error distributions unlike other estimators based on distributional assumptions that yield inconsistent estimates if the assumptions are wrong.^{56, 57} In the first stage of 2SLS, a linear probability model of ACE/ARB choice was estimated for the 68,236 patients in our sample. Independent variables in the first stage model were the measured covariates described above and the local area ACE/ARB practice style instrument. The instrument was specified empirically using nine binary variables that placed each patient residence ZIP code ATR within deciles of the distribution of ATRs across patients. In the second stage of 2SLS, a linear probability model of one-year survival was estimated. The second-stage model specified all the non-instrument covariates from the first-stage model as independent variables plus the predicted ACE/ARB choice probability from the first stage. Using this approach, the estimated parameter associated with the predicted ACE/ARB choice probability in the second stage model is the local average treatment effect (LATE). We repeated this 2SLS method using ACE/ARB ATR values across the 20 different local area sizes. For each local area size we estimated (1) the strength of the relationship between the instrument and ACE/ARB treatment choice via the Chow⁵⁸ F-value which tests whether the local area practice style instrument describes a statistically significant portion of the variation in ACE/ARB choice; (2) the ACE/ARB 1-year survival LATE estimate with respective standard error; (3) the Hausman over-identification test statistic⁵⁹ and (4) the correlation between ACE/ARB ATR and overall life expectancy for the county containing each patient ZIP code.⁶⁰ Statistics (3) and (4) provide indirect assessments of whether the instruments are related to unmeasured confounding variables. The Hausman over-identification statistic tests the null hypothesis that the direct exclusion of the instrument from the second stage one-year survival equation was appropriate. A high Hausman statistic rejects the null hypothesis, suggesting a relationship between the instrument and survival.⁵⁹ The correlation parameter between local area ACE/ARB practice style and local area life expectancy suggests whether ACE/ARB practice styles are related to unmeasured differences in baseline patient health at diagnosis.

Results

Table 1 describes the local areas defined using the DACC method by the number of threshold patients used. For 10-person local areas it took an average of 26.5 driving minutes from each ZIP code and 12.1 ZIP codes to find sufficient patients, whereas for the 200-person local areas it took an average of 87.5 driving minutes from each ZIP code and 201.2 ZIP codes to find sufficient patients. Figure 1 shows the relationship between the first-stage model Chow F-statistics assessing the strength of the local area practice style instrument and individual patient ACE/ARB choice by local area size. All F-statistics show statistically significant relationships between local area ACE/ARB practice styles and ACE/ARB choice but the values of the F-statistics fell (from 232 to 28) as the local area size increases. Figures 2a and 2b present maps of the northeastern United States illustrating the dispersion of ATR values for 10-patient and 200-patient areas, respectively. These figures show substantial geographic variation in the ACE/ARB use regardless of the local area size, but more practice style variation is revealed using the 10-patient sized local areas.

Table 1.

Statistics Describing ZIP Codes Within Local Areas Around Patient Residence ZIP Codes by the Threshold Number of Patient Defining a Local Area

	Driving Time in Minutes Required to Reach Patient Threshold Number			Number of ZIP Codes Required to Reach Patient Threshold Number
Threshold Number of Patients in Local Area	Mean	Minimum	Maximum	Mean	Minimum	Maximum
10	26.5	0.0	384.6	12.1	1	234
20	34.2	0.0	384.6	22.6	1	262
30	39.9	0.0	384.6	33.0	1	274
40	44.5	4.5	384.6	43.4	2	288
50	48.5	4.9	384.6	53.5	2	303
60	52.1	4.9	384.6	63.7	2	321
70	55.4	5.3	384.6	73.8	3	341
80	58.5	5.3	384.6	83.7	4	353
90	61.3	6.4	423.1	93.7	5	369
100	64.2	6.4	423.1	103.7	5	377
110	67.0	6.5	423.1	113.8	5	417
120	69.6	7.1	423.1	123.8	6	427
130	72.1	7.1	423.1	133.7	7	443
140	74.5	7.8	423.1	143.6	7	453
150	76.7	7.8	423.1	153.2	7	467
160	78.9	8.1	423.1	162.8	8	475
170	81.2	8.9	433.7	172.4	9	495
180	83.3	8.9	433.7	182.0	9	506
190	85.4	9.5	510.4	191.7	9	520
200	87.5	9.9	510.4	201.2	9	532

Figure 1.

First-Stage F-Statistics for the Effect of Local area Practice Style on ACE/ARB Use by Local Area Size (10 Patients to 200 Patients)

Figure 2.

Figure 2a: Northeast United States ACE/ARB Area Treatment Ratios Based on Local Areas Defined Using 10 Patients Around 5-Digit ZIP Codes.

Figure 2b: Northeast United States ACE/ARB Area Treatment Ratios Based on Local Areas Defined Using 200 Patients Around 5-Digit ZIP Codes.

Table 2 contrasts ACE/ARB treatment rates, unadjusted outcome rates, and selected measured covariate averages between patients grouped by ACE/ARB treatment choice, and the 1^st and 10^th deciles of the ACE/ARB ATR for 10-patient, 100-patient, and 200-patient defined local areas. We also report Cochran-Armitage trend tests for each measured characteristic across the ATR deciles groups for each local area size.^{61, 62} Of the patients in our sample, 50.5% received an ACE/ARB prescription in the 30-days post AMI discharge. Patients who received an ACE/ARB tended to be younger; had fewer comorbidities prior to AMI admission; had higher-risk AMIs; were more likely to have had a cardiac catheterization during their AMI admission; were less likely to have conditions related to ACE/ARB side effects either prior to or during admission; and were more likely to have used an ACE/ARB prior to their index AMI. Grouping patients by the ACE/ARB ZIP code ATRs revealed substantial ACE/ARB treatment variation but the extent of variation falls with the local area size. For example, ACE/ARB treatment rates varied from 35.0% to 64.0% moving from the 1^st to 10^th decile with the 10-patient local area size, but varied only from 44.2% to 56.3% using the 200-patient local area size. Compared with grouping patients by ACE/ARB treatment, grouping patients by ZIP code ATRs increased the balance in all measured confounders except patient race. Figure 3 presents the Hausman over-identification statistics for the IV model estimated for each local area size. The Hausman test statistic ranged from 0.47 to 2.21 and all were statistically insignificant except those estimated using the 60 and 100-person sized local areas. The Hausman test statistic appears to trend downward as local area size increases. Figure 4 presents estimated correlations between local area ACE/ARB practice style measures and local area overall life expectancy across local area sizes. A small positive correlation between ACE/ARB practice style and local area life expectancy is observed that trends higher as local area size increases.

Table 2.

Acute Myocardial Infarction Patient Characteristics by ACE/ARB Choice and Local Area Practice Styles Defined Using Different Area Sizes (10, 100, 200)

				Local Area Practice Style Area Size
	ACE/ARB in 30 days			10-Person Local Areas			100-Person Local Areas			200-Person Local Areas
	No	Yes	p-value (χ2)	1st decile	10th decile	Trend Testa	1st decile	10th decile	Trend Testa	1st decile	10th decile	Trend Testa
Number of Patients	33,757	34,479		6,824	6,824		6,825	6,825		6,821	6,821
	Column %			Column %			Column %			Column %
ACE/ARB Rx%b	0	100	<0.0001*	35.0	64.0	<0.0001*	42.8	57.1	<0.0001*	44.2	56.3	<0.0001*
1-year Survival %	81.7	87.6	<0.0001*	82.8	84.7	0.0054*	83.3	84.6	0.5427	83.8	84.3	0.4997
Age
66–75	35.6	41.6	<0.0001*	36.9	38.5	0.0482*	36.8	40.1	<0.0001*	36.4	38.8	0.0038*
76–85	39.5	39	0.2176	39.9	39.4	0.0507	40.6	37.3	<0.0001*	40.5	38.4	<0.0001*
85+	24.9	19.4	<0.0001*	23.2	22.1	0.9851	22.5	22.6	0.5585	23.1	22.8	0.2538
Race
White	84.5	82.5	<0.0001*	84.8	82.6	0.0163*	86.2	81.8	<0.0001*	87.9	79.8	<0.0001*
Black	7.5	8.0	0.0296*	7.6	9.4	0.2996	7.6	10.1	<0.0001*	6.9	11.1	<0.0001*
Other	8.0	9.6	<0.0001*	7.6	8.0	0.0299*	6.2	8.1	<0.0001*	5.2	9.1	<0.0001*
Metroc	71.2	68.7	<0.0001*	70.2	70.3	0.7837	70.6	71.4	<0.0001*	69.2	70.2	<0.0001*
Low Per Capita Incomed	48.3	50.8	<0.0001*	49.4	48.2	0.1770	49.1	49.7	<0.0001*	52.1	48.0	0.0012*
Charlson Scoree-- pre-index
0	31.2	35.7	<0.0001*	31.7	32.8	0.1944	31.7	32.8	0.5323	32.7	33.0	0.7787
1	22.5	24.6	<0.0001*	23.3	23.1	0.5597	23.3	23.4	0.5941	23.9	23.2	0.7940
2	15.2	14.0	<0.0001*	16.2	14.8	0.0640	15.0	13.7	0.2998	14.8	14.8	0.5170
3	11.1	10.2	<0.0001*	11.1	10.7	0.8890	11.1	11.4	0.4147	10.9	11.0	0.1480
4+	19.9	15.5	<0.0001*	17.8	18.6	0.5147	18.8	18.6	0.2869	17.6	17.9	0.9459
Anterior Wall AMIf	4.9	8.2	<0.0001*	5.8	6.5	0.4406	5.9	6.5	0.3104	5.7	6.6	0.4593
Non ST Elevation AMI (NSTEMI)g	78.7	72.8	<0.0001*	77.3	77.4	0.3574	77.8	76.7	0.4354	77.6	75.9	0.2170
ACE/ARB side effect condition -- pre-indexh	24.1	17.4	<0.0001*	21.6	21.4	0.1961	21.5	21.5	0.4023	20.6	21.2	0.9066
ACE/ARB side effect condition – indexh	34.8	25.3	<0.0001*	31.9	30.3	0.0252*	30.9	30.9	0.3121	30.6	30.7	0.7472
Cardiac Cath Index	50.0	62.8	<0.0001*	55.4	55.9	0.7898	54.8	55.2	0.3402	56.2	55.7	0.5336
ACE use in 180 days pre-index	28.2	43.1	<0.0001*	34.8	36.6	0.0201*	34.7	38.4	0.0005*	35.2	37.1	0.0165*

^aCochran-Armitage test of trend in characteristic value across patients grouped into deciles based on local area ACE/ARB practice style measure. For example, the p value Metro tests whether a linear trend in Metro residence exists across the ACE/ARB practice style-based patient groups.

^b% patients with an ACE/ARB prescription in 30-days post AMI discharge.

^cLived in a Metropolitan area according to Rural-Urban Continuum Codes developed by the USDA Economic Research Service (http://www.ers.usda.gov/topics/rural-economy-population/rural-classifications.aspx). 20 patients had unknown metro size.

^dMean per capita income in ZIP code below ZIP code mean.

^eKlabunde CN, Potosky AL, Legler JM, Warren JL. Development of a comorbidity index using physician claims data. Journal of Clinical Epidemiology, 2000 Dec; 53(12): 1258–67.⁶⁷

^fICD-9 410.0; 410.1

^gICD-9 410.7

^hPatient had either angioedema (995.1); hyperkalemia (276.7); acute renal failure/acute tubular necrosis (584.xx); acute glomerulonephritis (580.xx); of Chronic Kidney Disease (016.00, 016.01, 016.02, 016.03, 016.04, 016.05, 016.06, 095.4, 189.0, 189.9, 223.0, 236.91, 250.40, 250.41, 250.42, 250.43, 271.4, 274.1, 283.11, 403.01, 403.11, 403.91, 404.02, 404.03, 404.12, 404.13, 404.92, 404.93, 440.1, 442.1, 572.4, 582.xx, 583.xx, 585.xx, 586.xx, 588.xx, 591, 753.12, 753.13, 753.14, 753.15, 753.16, 753.17, 753.19, 753.20, 753.21, 753.22, 753.23, 753.29, 794.4)

Figure 3.

Over-Identification F-Statistics by Local Area Size (10 Patients to 200 Patients)

Figure 4.

Correlations of Local Area ACE/ARB Area Treatment Ratio and Local Area Overall Life Expectancy by Local Area Size (10 Patients to 200 Patients)

Figure 5 plots the LATE estimates and respective standard errors by local area size. Estimates of LATE ranged from −0.003 to 0.041 (i.e. 41 additional patients per 1000 treated survive for one-year post AMI). Only the IV estimate produced with the 10-person local area (0.041 or 41 patients per additional 1000 treated survive one year) is statistically different from zero. IV estimates trend downward moving from 10-person to 120-person local areas, but trend upward for local areas defined using greater than 120 person.

Figure 5.

Instrumental Variable Parameter Estimates and Standard Errors for the Effect of ACE/ARB Prescribing Post AMI on 1-Year Survival and Standard Error by Local Area Size (10 Patients to 200 Patients)

Discussion

Instrumental variable (IV) methods yield local average treatment effect (LATE) estimates that are representative of the subset of patients whose treatment choices were responsive to the instrument specified in the analysis.¹³ Methodologists have stressed that LATE estimates are best applied to policy questions that are closely aligned with the treatment variation generated by the instruments specified.⁶³ As such, our LATE estimates of ACE/ARBs on one-year survival post-AMI using local area practice styles would be best used to address policy questions related to whether changes in existing ACE/ARB rates will affect patient survival. For example, what survival benefits will be gained by moving from a 50.5% ACE/ARB treatment rate to a 55% rate, or what benefits, would be lost from reducing the ACE/ARB treatment rate to 45%? As has been argued, estimates of LATE may be more appropriate for answering these questions than estimates from RCTs.^{14, 19, 64, 65}

We found that the area treatment ratio (ATR)-based measures of local practice style describe a statistically significant portion of variation in ACE/ARB use across all local area sizes. All F-statistics across local area sizes were greater than 10 which is considered the threshold of “weak instruments” in the IV literature.²³ However, as seen in Figure 1, as the local area size expands, the strength of the relationship between local area practice style measures and individual patient ACE/ARB choice diminishes. A direct effect of this reduction can be seen in the increase in the standard errors of the ACE/ARB treatment effect estimates in Figure 5. If the treatment effect standard error found in the 10-patient local area were maintained across local area sizes, six of the 20 treatment effect estimates would have been statistically significantly different from zero at the .05 level.

We discussed two main sources of bias for LATE estimates using ATR-based measures of local area practice styles in IV estimation. We theorized in Appendix A that using ATR-based instruments will yield estimates of LATE with a positive bias but that this bias will diminish as local area size increases. It is also possible that for certain local area sizes ATR-based measures could be correlated with either average patient characteristics or ecological factors that affect outcome directly. Previous studies have suggested several potential confounding factors, but there are no a priori relationships that would associate these factors with ATR-based measures of local practice styles by local area size. We suggest researchers should look for anomalies in LATE estimates across local area sizes and search for secondary sources of information to assess the validity of estimates. We provide an example of this by estimating correlations between local area ACE/ARB ATR values and local area overall life expectancies. In future research medical charts from the index AMI could be abstracted for a sample of patients in the highest and lowest ACE/ARB ATR areas for several local area sizes to assess whether average patient characteristics such as AMI severity varies with ATR values across local area sizes.

With respect to the effect of ACE/ARBs on one-year survival post-AMI across local area sizes (Figure 5), we found that only the LATE estimate from the 10-person local area size was statistically different from zero and that LATE estimates trended downward as local area size increased from 10 to 120 patients. This trend is consistent with the theory we provide in Appendix A that ATR-based measures of local area practice style will lead to LATE estimates with positive bias but that this bias will diminish as local area size increases. However, this trend clearly does not extend to local area definitions greater than 120 patients as LATE estimates trended upward for local area sizes using from 120 to 200 patients. The U-shaped relationship between LATE estimates and local area size suggests that correlations between local area size and unmeasured confounders exist, but this U-shaped relationship alone is insufficient to pinpoint which local area sizes have the most prominent confounding problem. It is possible that correlations between unmeasured confounding factors and the ATR-based instruments exist in the larger-sized local areas that bias LATE treatment effect estimates in a positive manner. We did find small positive correlations between the ATR values and life expectancy that get stronger as local area size increases that support this conclusion. On the other hand, it is possible that correlations between unmeasured confounding factors and the ATR-based instruments exist in the smaller-sized local areas that bias treatment effects toward zero (independent of the positive bias associated with essential heterogeneity). The over-identification statistic estimates in Figure 3 trend lower with increased local area size which provides some weight to this idea, but secondary data sources are needed to further understand the source of LATE estimate variation across area sizes.

Just over 50% of the AMI patients in our sample filled an ACE/ARB prescription in the 30-days post discharge from their index AMI. If ACE/ARB prescribing is sorted across AMI patients based on expected benefits (essential heterogeneity) we expected that the treatment rate variation identified by local area practice style differences reflects treatment choices for patients whose expected treatment benefits from ACE/ARBs are less definitive. As a result, we expected our estimates of the local average treatment effect (LATE) of ACE/ARBs on one-year patient survival to be lower than the upper limit RCT estimates (50 lives saved per 1000 high-risk patients). Our LATE estimates using various local area sizes are all less than this upper limit. Our LATE estimates are best used to assess whether higher rates of ACE/ARB prescribing post-AMI will increase survival rates. If the IV assumptions hold, our results do not provide clear evidence that higher treatment rates will increase survival rates. While our local practice style-based instrument described a substantial portion of treatment variation across local area sizes and, for the most part, treatment effect estimates were positive, large treatment effect standard errors preclude the rejection of the null hypothesis that no survival benefits are available from higher ACE/ARB prescribing rates. Our results suggest that providers may be correctly sorting ACE/ARB treatments to AMI patients in practice and that increasing the ACE/ARB treatment rate beyond 50% may do little to improve survival rates. However, care should be taken not to generalize these results to all AMI patients especially those patients that receive ACE/ARBs in current practice.

Key Findings.

The use of smaller local areas to measure practice styles as instruments exploits more treatment variation and results in smaller standard errors. However, if treatment effects are heterogeneous, the use of smaller local areas may increase the risk that local practice style measures are dominated by idiosyncratic differences in average treatment effectiveness across areas resulting in treatment effect estimates that are biased toward greater effectiveness.

What this adds to what is known

Local area practice style measures can be useful instruments in instrumental variable analysis, but the use of smaller local area sizes to generate greater treatment variation may result in treatment effect estimates that are biased toward higher effectiveness. Assessment of whether ecological bias can be mitigated by changing local area size requires the use of outside data sources.

What is the implication, what should change now

Researchers should be aware that if treatment effects are heterogeneous across patients, the use of smaller-sized local areas as the basis to measure local practice style as an instrument may result in treatment effect estimates that are biased in favor of treatment.

Appendix A

To assess the properties of our local area practice style measures with regard to local area size we derive a model of local practice style model based on geographic variation in provider treatment effectiveness beliefs. Similar results can be found using other local area factors as the source of practice style differences such as variation in the value of treatment outcomes across areas.⁶⁶ Let expected patient outcome be described by the following model:

$$Y_i={\beta }_0+[({\beta }_{10}+{\beta }_{1s}{S}_i)·P_N]·T_{iN}+{\beta }_2{X}_i+{\beta }_2{E}_N+{\theta }_{iN},$$ A1

where Y_i is the outcome for patient “i”. N designates the size of the geographic area around patient “i” as defined by the shortest driving distance required to locate the N nearest patients with the same clinical condition. T_iN is the treatment choice for patient “i” living in the area N-sized. [(β₁₀ + β_1S S_i) · P_N] represents the range of treatment effectiveness beliefs across patients with characteristics S_i in living in the N-sized area. Under this specification treatment effectiveness is heterogeneous across patients based on S_i. If S_i is characterized in a manner so that higher values of S_i increase treatment effectiveness β_1S will be positive. P_N represents average provider treatment effectiveness beliefs in area N-sized. The larger P_N the more effective the providers in the area believe treatment is across the range of S_i. E_N are ecological factors in the N-sized area that effect patient outcomes. X_i represents the measured characteristics for patient “i” that affect the outcome directly, while θ_iN represents the accumulated unmeasured characteristics of patient “i” living in the N-sized that affect outcome.

Further, describe treatment choice for patient “i” living in the N-sized area as:

$$T_{iN}={\alpha }_0+{\alpha }_1{X}_i+{\alpha }_2{S}_i+{\alpha }_3{P}_N+{\mu }_i$$ A2

where T_iN, X_i, P_N′ and S_i are defined as above. Under essential heterogeneity S_i is specified in (A2) because providers are assumed to (1) assess S_i for each patient; (2) have knowledge of the effect of S_i on treatment effectiveness; and (3) recommend treatment choice based on the value of S_i for each patient. Higher S_i values increase treatment effectiveness so α₂ in (A2) will be positive. If P_N is measured so that larger P_N represents stronger treatment effectiveness beliefs, α₃ will also be positive. X_i has no direct relationship with treatment effectiveness in (A1) so the theoretical justification for specifying X_i in the treatment choice equation is unclear. However, we include X_i in (A2) to remain consistent with general convention. μ_i equals the accumulated unmeasured characteristics for patient “i” that affect treatment choice but do not affect outcome.

If was able to be measured directly, P_N could serve as a valid instrument under the following conditions (1) α₃ ≠ 0; (2) Corr(P_N, E_N) = 0; (3) Corr(P_N, S_i) = 0; and (4) Corr(P_N, θ_iN) = 0. Under these conditions IV estimation will yield a consistent estimate of the local average treatment effect (LATE) of T_i on Y_i − β_1PN -- that is specific to the distribution of S_i values for the subset of patient whose treatment choices were affected by P_N. β_1PN will likely represent patients with moderate S_i values as patients with very high S_i values will likely receive treatment regardless of P_N and patients with very low S_i values will likely not receive treatment regardless of P_N.

In this paper we measure P_N using area treatment ratios (ATRs) for the local area around each patient residence ZIP code of size N. T_iN, Y_i, X_i and are observed by the researcher. We assume S_i is not correlated with the underlying P_N and is distributed evenly across patients with a mean of μ_S and variance ${\sigma }_S^2$. However, our ATR approach to measure local treatment preferences for areas of size N solves to:

$${\mathit{ATR}}_N=\frac{{\sum }_{i=1}^N{T}_i/N}{{\sum }_{i=1}^N{\widehat{T}}_i/N}=\frac{{\alpha }_0+{\alpha }_1{\overline{X}}_N+{\alpha }_2{\overline{S}}_2+P_N+\overline{\mu }}{{\widehat{\alpha }}_0+{\widehat{\alpha }}_1\overline{X}}$$ A3

where T̂_i equals the predicted probability of treatment for patient “i” conditional on X_i. S̄_N equals the mean of S_i in the local area of size N. Local areas, out of chance, with a larger S̄_N will have more patients apt to gain from treatment and more patients choosing treatment conditional on X_i. As a result, our ATR_N measure will vary with both P_N and S̄_N. Therefore, when ATR_N is used as an instrument when estimating (A2) in the first stage of 2SLS we expect that the ATR_N value associated with each local area will be positively correlated with the unmeasured S_i values for the patients within each local area. This positive correlation will result in an IV estimate of β_1PN that is biased high relative to the true LATE for the subset of patients whose treatment choices were affected by P_N. However, S̄_N is distributed with mean μ_S and variance ${\sigma }_S^2/N$. Therefore, as N increases the variance in S̄_N across local areas will diminish and a larger portion of the treatment variation described by ATR_N will be generated from variation in P_N, and, all else equal, the positive bias associated with estimates of β_1PN will diminish.