A Comparison of Confounding Adjustment Methods for Assessment of Asthma Controller Medication Effectiveness

Lingling Li; William M Vollmer; Melissa G Butler; Pingsheng Wu; Elyse O Kharbanda; Ann Chen Wu

doi:10.1093/aje/kwt323

. 2014 Jan 23;179(5):648–659. doi: 10.1093/aje/kwt323

A Comparison of Confounding Adjustment Methods for Assessment of Asthma Controller Medication Effectiveness

Lingling Li ^*, William M Vollmer, Melissa G Butler, Pingsheng Wu, Elyse O Kharbanda, Ann Chen Wu

PMCID: PMC3983403 PMID: 24464909

Abstract

We compared the impact of 3 confounding adjustment procedures—covariate-adjusted regression, propensity score regression, and high-dimensional propensity score regression—to assess the effects of selected asthma controller medication use (leukotriene antagonists and inhaled corticosteroids) on the following 4 asthma-related adverse outcomes: emergency department visits, hospitalizations, oral corticosteroid use, and the composite outcome of these. We examined a cohort of 24,680 new users who were 4–17 years of age at the incident dispensing from the Population-Based Effectiveness in Asthma and Lung Diseases (PEAL) Network of 5 commercial health plans and TennCare, the Tennessee Medicaid program, during the period January 1, 2004, to December 31, 2010. The 3 methods yielded similar results, indicating that pediatric patients treated with leukotriene antagonists were no more likely than those treated with inhaled corticosteroids to experience adverse outcomes. Children in the TennCare population who had a diagnosis of allergic rhinitis and who then initiated the use of leukotriene antagonists were less likely to experience an asthma-related emergency department visit. A plausible explanation is that our data set is large enough that the 2 advanced propensity score–based analyses do not have advantages over the traditional covariate-adjusted regression approach. We provide important observations on how to correctly apply the methods in observational data analysis and suggest statistical research areas that need more work to guide implementation.

Keywords: asthma controller medications, confounding adjustment, Cox regression, electronic health care databases, high-dimensional propensity score, inhaled corticosteroids, leukotriene inhibitor, propensity score

Although randomized clinical trials are widely regarded as providing the highest level of evidence for answering questions related to clinical efficacy, they are often criticized for lacking generalizability (1). Pragmatic clinical trials that are more suited to answering broader questions of effectiveness are gaining in popularity, but many investigators still rely on large observational studies to assess the real-world impact of various clinical interventions and lifestyle behaviors (2). Deriving causal inference from observational data, however, is challenging because of the inherent confounding issues (3). Therefore, an appropriate analysis of observational data requires careful control for potential confounders, and multiple statistical methods have been developed to accomplish this (4–6).

Covariate-adjusted (CA) regression is the classic confounding adjustment method, which adjusts for potential confounders by directly including them in the regression model (7). An alternative method is propensity score (PS) analysis (8). The PS is defined as the conditional probability of receiving the exposure of interest given measured covariates (5). The PS is effectively a summary score that incorporates information from multiple covariates. The high-dimensional propensity score (hdPS) technique (6) builds upon the PS approach by ascertaining and selecting additional empirical covariates from electronic health care databases and then using these empirical covariates jointly with the predefined covariates in PS estimation to enhance confounding adjustment. Nevertheless, despite the increasing use of the PS-based methods and a growing amount of advanced methodological research in this area (9, 10), knowledge of how to correctly apply these methods and their potential impact on observational epidemiology is still limited (11, 12).

We explored and compared CA methods, PS adjustment, and hdPS adjustment as part of a comparative effectiveness study of the relative impact of 2 major classes of controller medications for asthma—leukotriene antagonists (LTRAs) versus inhaled corticosteroids (ICSs)—among a cohort of pediatric asthma patients in the Population-Based Effectiveness in Asthma and Lung Diseases (PEAL) Network population. The primary outcomes of interest were asthma-related exacerbations as assessed by emergency department visits, hospitalizations, and use of oral corticosteroids. National guidelines recommend the use of ICSs as first-line controller therapy rather than LTRAs, and randomized clinical trials (13–16) suggest that patients using ICSs are less likely to experience asthma-related exacerbations than patients using LTRAs. Nevertheless, anecdotal evidence suggests that LTRAs are frequently used as first-line controller therapy rather than ICSs, because adherence to oral medicines (i.e., LTRAs) seems to be better than adherence to inhalers (i.e., ICSs), and some families may prefer LTRAs because of concerns that inhaled steroids may stunt a child's growth. Using this example, we review the strengths and weaknesses of the 3 methods, discuss lessons learned from the implementation process, and identify knowledge gaps.

METHODS

Study design and population

The PEAL Network includes TennCare, the Tennessee Medicaid program, and the following 5 commercial health plans: Harvard Pilgrim Health Care, HealthPartners (Minneapolis, Minnesota), Kaiser Permanente (KP) Northern California, KP Georgia, and KP Northwest (17). The PEAL data warehouse includes information on individual demographic characteristics, enrollment records, health care utilization, and medication dispensing records (18, 19) for individuals with lung disease, as well as electronic medical records for a subset of these individuals.

Individuals were identified from claims records and electronic medical records and were potentially eligible for inclusion in the PEAL asthma population if they had any International Classification of Diseases, Ninth Revision, discharge diagnosis of code for asthma (code 493.xx) during an acute inpatient hospital stay, emergency department visit, ambulatory visit, or nonacute institutional stay during the period January 1, 2004, to December 31, 2010. This time window varied for each site by up to 1 year on the basis of data availability. Individuals were excluded if they had a diagnosis of cystic fibrosis, immunodeficiency, bronchiectasis, hereditary or degenerative diseases of the central nervous system, psychoses, mental retardation, congestive heart failure, pulmonary hypertension, or pulmonary embolism based on International Classification of Diseases, Ninth Revision, codes.

We identified 218,019 individuals in the PEAL Network who had at least 1 qualified asthma controller medication dispensing, meaning that, in the 12-month period prior to the dispensing date, they had continuous enrollment and uncontrolled asthma, which is defined as having at least 1 eligible health care encounter (hospitalization, emergency department visit, or dispensing of oral corticosteroids of 3 days or more). Patients who were dispensed individual ICSs and long-acting β agonist inhalers on the same day were placed in the ICS/long-acting β agonist group. We define the earliest dispensing date among all qualified dispensings as the index date and the date 12 months prior to this as the baseline period. We excluded 13,830 individuals who did not initiate monotherapy (or ICSs/long-acting β agonists) of 1 of the controller medications of interest on the index date, and 204,189 individuals remained. Of the 204,189 individuals, 84,044 patients were incident users (no controller medication use during the 12-month baseline period). In this analysis, we focus on the 24,680 pediatric patients aged 4–17 years on the index date who were incident users of either LTRAs (29%) or an ICS (71%).

Study outcomes

We examined the relative impact of LTRA versus ICS use on time from index date to first occurrence of the following: an asthma-related emergency department visit, an asthma-related hospitalization, dispensing of an oral corticosteroid burst pack, or any of these (composite outcome). The follow-up time was censored at disenrollment, 30 days after a patient augmented treatment (e.g., switched from LTRA to ICS or vice versa or added a long-acting β agonist to an ICS), or 365 days after the index date, whichever came first. We attributed all outcomes that occurred during the 30 days after augmentation to the initial controller medication, because medication augmentation is typically a sign of poor disease control by the initial controller medication and, thus, the adverse outcomes occurring soon after the augmentation should be attributed to the failure of the initial medication, not the newly augmented medication. We censored patients at 30 days after medication augmentation because it takes approximately 30 days for controller medications to become beneficial (20).

Time-varying adherence measure

We calculated a time-varying adherence measure for the initiated medication as the proportion of days covered (PDC) (21) based on a moving preceding 30-day window (i.e., the PDC on day t was calculated on the basis of the [t – 31, t – 1] window). We then dichotomized values as either ≥0.75 or <0.75 (22). Because the PDC methodology assumes that all medications are used as directed, all participants start with a guaranteed minimum of 30 days of good adherence. Any individuals who experienced the outcome of interest during this period were excluded from the analysis (i.e., the analysis was conditional on “survival” for the first 30 days).

Covariates

For the CA analysis, we included a variety of potential confounders, including patient demographic characteristics, prior asthma-related health care utilization, rescue medication use, and chronic medical conditions (Table 1). The claims-derived variables were created on the basis of clinical expertise as surrogate measures of asthma disease severity and level of control. These same variables were used to estimate the PSs for the PS analysis. The hdPS analysis further drew on a varying number of empirical covariates from the PEAL database. We describe the process below.

Table 1.

Baseline Characteristics of LTRA and ICS Users Among All Study Individuals From 5 Commercial Health Plans and TennCare, 2004–2010

Characteristic	Commercial Health Plan Subjects					TennCare Subjects
	LTRA (n = 1,286)		ICS (n = 13,505)		SD	LTRA (n = 5,867)		ICS (n = 4,022)		SD
	No.	%	No.	%	SD	No.	%	No.	%	SD
Age, years
4–11	955	74.3	9,657	71.5	0.04	4,587	78.2	3,070	76.3	0.03
12–17	331	25.7	3,848	28.5	0.04	1,280	21.8	952	23.7	0.03
Sex
Female	535	41.6	5,479	40.6	0.02	2,512	42.8	1,724	42.9	0.00
Male	751	58.4	8,026	59.4	0.02	3,355	57.2	2,298	57.1	0.00
Site
HPHC	506	39.3	2,031	15.0	0.40^a
HealthPartners^b	387	30.1	1,240	9.2	0.39^a
KP Northern California	287	22.3	7,964	59.0	0.57^a
KP Northwest	47	3.7	1,162	8.6	0.15^a
KP Georgia	59	4.6	1,108	8.2	0.11^a
Race
White	325	25.3	4,139	30.6	0.09	3,494	59.6	1,942	48.3	0.16^a
Asian	45	3.5	1,080	8.0	0.14^a	48	0.8	31	0.8	0.00
Black	65	5.1	1,587	11.8	0.17^a	1,840	31.4	1,656	41.2	0.15^a
Hispanic	81	6.3	2,178	16.1	0.22^a	254	4.3	204	5.1	0.03
Other	770	59.9	4,521	33.5	0.39^a	231	3.9	189	4.7	0.03
History of smoking	17	1.3	355	2.6	0.07	67	1.1	47	1.2	0.00
Experienced the following in prior 12 months
ED visit	240	18.7	3,243	24.0	0.09	2,013	34.3	1,671	41.5	0.11^a
Hospitalization	48	3.7	766	5.7	0.07	285	4.9	511	12.7	0.20^a
Outpatient visit	516	40.1	6,323	46.8	0.10^a	1,519	25.9	1,301	32.3	0.10^a
No. of oral corticosteroid dispensings in prior 12 months
0	106	8.2	1,899	14.1	0.13^a	938	16.0	825	20.5	0.08
1	919	71.5	9,745	72.2	0.01	3,546	60.4	2,183	54.3	0.09
≥2	261	20.3	1,861	13.8	0.12^a	1,383	23.6	1,014	25.2	0.03
No. of short-acting β agonist dispensings in prior 12 months
0	326	25.3	3,583	26.5	0.02	2,112	36.0	1,145	28.5	0.11^a
1–5	932	72.5	9,714	71.9	0.01	3,518	60.0	2,670	66.4	0.09
≥6	28	2.2	208	1.5	0.03	237	4.0	206	5.1	0.04
Medicaid coverage	60	4.7	768	5.7	0.03
Diagnoses in prior 12 months
Acute respiratory infection	979	76.1	9,416	69.7	0.10^a	4,501	76.7	2,825	70.2	0.10^a
Gastroesophageal reflux disease	25	1.9	185	1.4	0.03	150	2.6	84	2.1	0.02
Allergic rhinitis	516	40.1	2,569	19.0	0.34^a	1,038	17.7	538	13.4	0.08
PEAL Charlson Score >0^c	15	1.2	89	0.7	0.04	43	0.7	36	0.9	0.01
No. of generic drugs used in prior 12 months
≤1st quartile^d	469	36.5	5,888	43.6	0.10^a	1,701	29.0	1,457	36.2	0.11^a
(1st, 2nd] quartile^d	248	19.3	2,519	18.7	0.01	1,300	22.2	890	22.1	0.00
(2nd, 3rd] quartile^d	240	18.7	2,106	15.6	0.06	1,440	24.5	901	22.4	0.04
>3rd quartile^d	329	25.6	2,992	22.2	0.06	1,426	24.3	774	19.2	0.09

Open in a new tab

Abbreviations: ED, emergency department; HPHC, Harvard Pilgrim Health Care; ICS, inhaled corticosteroid; KP, Kaiser Permanente; LTRA, leukotriene antagonist; PEAL, Population-Based Effectiveness in Asthma and Lung Diseases; SD, standardized difference.

^a A standardized difference of 0.10 or greater indicates the existence of some imbalance.

^b Based in Minneapolis, Minnesota.

^c A modified Charlson score, which included all diseases that comprise the Charlson Comorbidity Index except chronic pulmonary disease.

^d Quartiles were estimated within the 2 subgroups with and without diagnosed allergic rhinitis.

Confounding adjustment methods

We used Cox regression (4) as the base model to analyze time-to-event outcomes adjusting for the time-varying PDC measure. We considered the following 3 methods that were used jointly with the Cox regression to adjust for baseline confounding: CA regression (4), PS regression (23), and hdPS regression (6).

CA regression

The CA approach adjusts for the covariates directly in the Cox regression model. Specifically, in this application, let A(t) denote the dichotomized PDC measure on day t, and let E denote the binary exposure variable (E = 1 for LTRA and E = 0 for ICS). In the CA analysis, we impose the following model for the hazard rate for each of the 4 outcomes:

(1)

where X denotes the vector of predefined covariates listed in Table 1. Exp(β₁ + β₂) denotes the parameter of interest, the hazard ratio between LTRA versus ICS when both controller medications were adhered to. The validity of the CA analysis requires that the imposed model 1 is correct.

PS regression

In this application, the PS is defined as the conditional probability of receiving LTRA given the predefined covariates (Pr(E = 1|X)). The PSs are unknown and were estimated with a logistic regression model regressing E on the predefined covariates X. Then the estimated PSs were adjusted for as quintile categories in the Cox regression, replacing the covariate vector X. The exposure variable E and the time-varying adherence measure A(t) were adjusted for in the same manner as in model 1.

The PS analysis may be biased if 1) the PS model is wrong, or 2) as with any regression analysis, the PS regression did not adjust for the PS appropriately (e.g., if the log of the hazard rate is in fact a linear function of PS instead of a step function (23)). Nevertheless, the PS analysis has a number of advantages. First, the PS model typically can include more covariates and more complex functional forms, such as higher order polynomials or interactions, than can the CA regression model because the number of exposed individuals is typically much larger than the number of adverse outcomes. Sometimes, there is prior knowledge of how physicians/patients made treatment selections, which can aid in PS modeling. Second, the goodness-of-fit of the imposed PS model can be assessed by comparing the distributions of the predefined covariates X between the exposure groups after adjusting for the estimated PSs (24). The confounders should be distributed equally between the exposure groups after adjustment. Third, the use of a single PS for confounding adjustment can lead to simpler regression models that are potentially easier to interpret and less subject to statistical issues such as incorrect functional forms or covariate collinearity. Moreover, PSs facilitate an often overlooked requirement for valid covariate adjustment: overlapping covariate values, or “common support,” across the exposure groups (25). Common support is required to prevent extrapolation beyond the range of the data. Covariate overlap is absent, for example, when 1 group includes individuals aged 45–65 years but the other is limited to those aged 45–55 years. Although it is difficult to assess multidimensional overlap among all of the covariates, it is relatively simple, as demonstrated below, to assess overlap in the PS.

HdPS regression

The hdPS analysis is identical to the traditional PS analysis, with the exception that the PSs generated in the hdPS analysis were estimated using both the predefined covariates X and a list of empirical covariates (6) derived from the PEAL database. These empirical covariates were derived and selected using a semiautomatic algorithm (26, 27) on the basis of their associations with the outcome. Thus, the set of empirical covariates varies by outcome. In our case, we used the following 7 data dimensions to create the list of empirical covariates: inpatient diagnosis, inpatient procedure, outpatient diagnosis, outpatient procedure, emergency department diagnosis, emergency department procedure, and medication dispensing during the 12 months prior to the index date. We capped the number of empirical covariates at floor (n / 50 − r), where n is the number of individuals, r is the number of main effect terms from the predefined covariates, and the floor function takes the biggest integer that is smaller than or equal to (n / 50 – r) to ensure that there were at least 50 individuals per main term in the logistic regression model for PS estimation to avoid overfitting (28).

Statistical analysis

Because of the striking differences in participant demographic characteristics, prior health care utilization, and use of LTRAs (Table 1) between the 5 commercial health plan and TennCare populations, we conducted separate analyses for these 2 populations. For each sample, we also conducted separate analyses for individuals who did or did not have a diagnosis of allergic rhinitis in the baseline period, because LTRAs could be used to treat both allergic rhinitis and persistent asthma (29) and because symptoms from allergic rhinitis could trigger asthma-related exacerbations (30). We defined the allergic rhinitis variable on the basis of the diagnosis codes during the baseline period. Thus, prior allergic rhinitis is a pretreatment variable and should not be on the causal pathway between the exposure and the outcome.

We calculated the PSs separately for the 3 KP sites (KP Georgia, KP Northern California, and KP Northwest) as a group and for the 2 remaining health plans (Harvard Pilgrim Health Care and HealthPartners) as a group to reflect the different prevalence rates of LTRA use in these 2 groups. For each group (KP sites, non-KP sites, and TennCare) we estimated the PSs separately for those with and without a diagnosis of allergic rhinitis. The number of empirical covariates included in PS estimation ranged from 0 to 144.

We assessed covariate balance, both before and after PS adjustment, using the standardized difference, which is defined as the difference in means or proportions divided by the pooled standard deviation (31). The standardized difference is insensitive to sample size and, thus, is more appropriate than a Student's t test or a χ² test when the overall sample size is large (31). A standardized difference of 0.10 or greater indicates the existence of some imbalance (31).

Theoretical guidance on determining the common support is not available, and we determined the common support region on an ad hoc basis. We plotted smoothed histograms of the PSs within each group on the basis of kernel density estimates. These plots show values of the PS for which each exposure group has at least a few observations, and we defined common support on that basis. We selected a range of PSs over which the estimated density estimates were 2% or greater. Observations outside the common support were excluded from further analyses. The selection of 2% as the cutoff point was arbitrary, so we repeated the analyses with a higher cutoff of 4%. The number of individuals within the common support regions further decreased, but results were similar (data not shown). All analyses were performed in SAS, version 9.3, software (SAS Institute, Inc., Cary, North Carolina).

RESULTS

We included data from 24,680 individuals, including 14,791 from the 5 commercial health plans and 9,889 from TennCare. The use of LTRA varied widely, with high use observed among TennCare participants (59.3%), low use among those from the 3 KP sites (3.7%), and moderate use among those from the 2 non-KP sites (21.4%). The use of LTRA was also higher among those with a prior diagnosis of allergic rhinitis than among those without allergic rhinitis (65.9% vs. 58.1% for TennCare, 8.2% vs. 2.7% for the KP sites, and 33.1% vs. 17.5% for the non-KP sites). Characteristics of LTRA and ICS users are shown in Table 1 for the health plans and TennCare populations, respectively. Similar numbers for the PS common support subsample are shown in Appendix Table 1. Following PS adjustment, covariate balance was improved on race, health plan, prior health care use, and prior diagnosis of allergic rhinitis as the standardized difference values decreased after PS adjustment. The improvement was greater among the TennCare population with a much higher proportion of LTRA users. The hdPSs improved covariate balance in a similar manner (data not shown).

Figures 1–3 depict PS kernel density estimates among LTRA and ICS users in each of the 6 subgroups defined by health plan (3 KP sites, 2 non-KP sites, and TennCare) and diagnosis of allergic rhinitis. As seen in the figure, the kernel densities were fairly similar for the 2 medication arms within each population subgroup. The dotted vertical lines in each panel define the region of common support and exclude approximately 10% of patients.

Figure 1. — LTRA (leukotriene antagonist) versus ICS (inhaled corticosteroid) propensity score kernel density estimates and common supports by allerigic rhinitis diagnosis among subjects from the Kaiser Permanente Georgia, Kaiser Permanente Northern California, and Kaiser Permanente Northwest commercial health plans, Population-Based Effectiveness in Asthma and Lung Diseases cohort, 2004–2010. (The term “common support” refers to the range over which the smoothed histograms of the propensity scores within each exposure group were 2% or greater.) A) Subgroup with no allergic rhinitis, B) subgroup with diagnosed allergic rhinitis. The solid curves indicate the propensity score kernel density estimates for the LTRA group. The dotted curves indicate the propensity score kernel density estimates for the ICS group. The gray dotted horizontal line indicates a cutoff of 2%. The gray dotted vertical lines indicate the boundaries of the within-group common support.

Figure 2. — LTRA (leukotriene antagonist) versus ICS (inhaled corticosteroid) propensity score kernel density estimates and common supports by allerigic rhinitis diagnosis among subjects from the Harvard Pilgrim Health Care and HealthPartners (Minneapolis, Minnesota) commercial health plans, Population-Based Effectiveness in Asthma and Lung Diseases cohort, 2004–2010. (The term “common support” refers to the range over which the smoothed histograms of the propensity scores within each exposure group were 2% or greater.) A) Subgroup with no allergic rhinitis, B) subgroup with diagnosed allergic rhinitis. The solid curves indicate the propensity score kernel density estimates for the LTRA group. The dotted curves indicate the propensity score kernel density estimates for the ICS group. The gray dotted horizontal line indicates a cutoff of 2%. The gray dotted vertical lines indicate the boundaries of the within-group common support.

Figure 3. — LTRA (leukotriene antagonist) versus ICS (inhaled corticosteroid) propensity score kernel density estimates and common supports by allerigic rhinitis diagnosis among subjects from the Tennessee Medicaid program, Population-Based Effectiveness in Asthma and Lung Diseases cohort, 2004–2010. (The term “common support” refers to the range over which the smoothed histograms of the propensity scores within each exposure group were 2% or greater.) A) Aubgroup with no allergic rhinitis, B) ubgroup with diagnosed allergic rhinitis. The solid curves indicate the propensity score kernel density estimates for the LTRA group. The dotted curves indicate the propensity score kernel density estimates for the ICS group. The gray dotted horizontal line indicates a cutoff of 2%. The gray dotted vertical lines indicate the boundaries of the within-group common support.

Table 2 presents the crude incidence rates (per 100,000 person-days) of the 4 asthma-related adverse outcomes and the associated 95% confidence intervals among the LTRA and ICS users. Table 3 provides results from the 3 adjustment procedures (i.e., the hazard ratio estimates among compliers). To assess the impact of restricting analyses to the common support region, we present results for the CA regression both for the full cohort and for the PS common support restricted subsample. Results are shown by care delivery model (commercial health plans vs. TennCare) and by allergic rhinitis status.

Table 2.

Crude Incidence Rates (per 100,000 Person-Days) for the 4 Asthma-Related Adverse Outcomes Among LTRA and ICS Users, 2004–2010

Outcome	Commercial Health Plan Subjects^a				TennCare Subjects
	No Allergic Rhinitis		Allergic Rhinitis		No Allergic Rhinitis		Allergic Rhinitis
	Rate	95% CI	Rate	95% CI	Rate	95% CI	Rate	95% CI
ED visit
LTRA users	17.3	11.4, 23.1	13.3	7.2, 19.5	37.3	34.0, 40.7	28.6	22.2, 34.9
ICS users	14.3	13.0, 15.7	13.3	10.7, 16.0	56.7	51.6, 61.8	49.4	37.0, 61.7
Hospitalization
LTRA users	3.1	0.6, 5.5	0.7	0.0, 2.2	4.7	3.5, 5.8	3.2	1.1, 5.2
ICS users	2.4	1.9, 2.9	2.4	1.3, 3.5	7.0	5.3, 8.7	6.1	1.9, 10.3
Use of oral corticosteroids
LTRA users	24.4	17.4, 31.4	24.7	16.3, 33.1	42.0	38.4, 45.5	47.3	39.1, 55.6
ICS users	18.4	17.0, 19.9	21.1	17.8, 24.4	46.4	41.9, 51.0	52.4	39.6, 65.1
Composite outcome
LTRA users	38.3	29.5, 47.2	34.1	24.2, 44.1	71.7	66.9, 76.5	66.6	56.6, 76.5
ICS users	29.7	27.8, 31.6	32.3	28.1, 36.4	91.1	84.5, 97.8	87.3	70.4, 100.0

Open in a new tab

Abbreviations: CI, confidence interval; ED, emergency department; ICS, inhaled corticosteroid; LTRA, leukotriene antagonist; TennCare, Tennessee Medicaid program.

^a Includes subjects enrolled in the following 5 commercial health plans: Harvard Pilgrim Health Care, HealthPartners (Minneapolis, Minnesota), Kaiser Permanente Northern California, Kaiser Permanente Georgia, and Kaiser Permanente Northwest.

Table 3.

Estimated Hazard Ratios for LTRA Users Versus ICS Users From the CA Regression Among All Study Individuals, the CA Regression Among Individuals on the PS Common Supports^a, the PS Regression Among Individuals on the Corresponding Common Supports, and the hdPS Regression Among Individuals on the Corresponding Common Supports, 2004–2010

Outcome	CA Regression on all Study Individuals		Restricted CA Regression on PS Common Supports			Regular PS Regression			hdPS Regression
Outcome	HR	95% CI	HR	95% CI	θ^b	HR	95% CI	θ^b	HR	95% CI	θ^b
Allergic rhinitis
ED visit
Health plans^c	1.18	0.48, 2.90	1.34	0.54, 3.37	1.2	1.41	0.58, 3.43	1.2	1.26	0.50, 3.20	1.1
TennCare	0.43	0.21, 0.88^d	0.41	0.20, 0.85^d	1.0	0.46	0.22, 0.94^d	1.1	0.44	0.21, 0.93^d	1.1
Hospitalization
Health plans	Missing^e	Missing^e	Missing^e	Missing^e	Missing^e	Missing^e	Missing^e	Missing^e	Missing^e	Missing^e	Missing^e
TennCare	1.98	0.19, 20.2	1.64	0.16, 16.6	0.8	1.69	0.17, 16.6	0.8	1.88	0.19, 18.8	0.9
Use of oral corticosteroids
Health plans	0.81	0.39, 1.72	0.84	0.40, 1.79	1.1	0.88	0.42, 1.83	1.1	0.77	0.35, 1.72	1.0
TennCare	0.74	0.39, 1.41	0.65	0.34, 1.24	0.9	0.66	0.34, 1.25	0.9	0.64	0.34, 1.22	0.9
Composite
Health plans	0.89	0.49, 1.63	0.95	0.51, 1.75	1.1	1.00	0.55, 1.83	1.1	0.90	0.48, 1.68	1.1
TennCare	0.60	0.36, 1.00^d	0.53	0.32, 0.89^d	0.9	0.54	0.33, 0.91^d	0.9	0.54	0.32, 0.93^d	1.0
No allergic rhinitis
ED visit
Health plans	1.39	0.75, 2.58	1.27	0.65, 2.47	1.0	1.37	0.71, 2.64	1.1	1.72	0.91, 3.24	1.3
TennCare	0.88	0.63, 1.21	0.91	0.63, 1.31	1.2	0.90	0.63, 1.29	1.1	1.10	0.74, 1.62	1.5
Hospitalization
Health plans	0.77	0.10, 5.90	0.81	0.10, 6.28	1.1	0.70	0.09, 5.34	0.9	0.75	0.10, 5.67	1.0
TennCare	1.00	0.42, 2.38	1.86	0.50, 6.91	3.3	1.89	0.51, 7.02	3.3	0.89	0.29, 2.73	1.3
Use of oral corticosteroids
Health plans	1.04	0.62, 1.70	1.01	0.58, 1.75	1.0	1.17	0.68, 2.01	1.2	1.04	0.56, 1.93	1.2
TennCare	1.12	0.81, 1.55	1.16	0.81, 1.60	1.2	1.13	0.79, 1.62	1.1	1.32	0.88, 1.98	1.5
Composite
Health plans	1.17	0.76, 1.81	1.11	0.70, 1.77	1.0	1.21	0.78, 1.95	1.1	1.26	0.77, 2.09	1.3
TennCare	1.09	0.85, 1.39	1.12	0.85, 1.48	1.2	1.10	0.83, 1.45	1.1	1.26	0.95, 1.69	1.4

Open in a new tab

Abbreviations: CA, covariate-adjusted; CI, confidence interval; ED, emergency department; hdPS, high-dimensional propensity score; HR, hazard ratio; ICS, inhaled corticosteroid; LTRA, leukotriene antagonist; PS, propensity score; TennCare, Tennessee Medicaid Program.

^a The range over which the smoothed histograms of the PSs within each exposure group were 2% or greater.

^b For each of the restricted CA, PS, and hdPS models, we also show the ratio of the 95% confidence interval width for that model to the 95% confidence interval width from the corresponding CA regression among all study individuals (θ) to examine the impact of sample size reduction on estimation efficiency.

^c Five commercial health plans, which included Harvard Pilgrim Health Care, HealthPartners (Minneapolis, Minnesota), Kaiser Permanente Northern California, Kaiser Permanente Georgia, and Kaiser Permanente Northwest.

^d The 95% confidence interval indicates significance.

^e Missing results because the Cox regression model did not converge because of a small number of hospitalization events.

Hazard ratio estimates were generally similar across the 4 adjustment procedures, and significant results were seen consistently in all 4 regression models. All of the confidence intervals overlapped considerably. We also observed no evidence that the hazard ratio estimates from any particular analytical approach were consistently higher or lower than those from other approaches. It is true, however, that the restricted CA, PS, and hdPS analyses had consistently higher standard error estimates for log(hazard ratio) than the CA analyses fit to the full cohort (mostly by a factor of <20%). The point and interval estimates were slightly unstable for the hospitalization outcome, likely because of the small number of events. The results from PS regression and hdPS regression remained similar (data not shown) in 2 additional analyses in which we adjusted for PSs 1) via the linear and quadratic terms of the continuous PS, and 2) via the quintile categorization and the linear and quadratic terms of the continuous PS.

Among patients with a diagnosis of allergic rhinitis, individuals in TennCare who were treated with LTRAs were less likely to experience emergency department visits (hazard ratio = 0.43, 95% confidence interval: 0.21, 0.88 from CA regression on all study individuals) compared with individuals treated with ICS. For all other comparison groups, whether individuals had allergic rhinitis or not, individuals treated with LTRA were just as likely to experience emergency department visits or hospitalizations or to need oral corticosteroids as individuals treated with ICS. Refer to our companion paper for more detailed discussion on the clinical implications of the results (A.C.W., Harvard Pilgrim Health Care Institute and Children's Hospital, unpublished manuscript).

DISCUSSION

We examined the impact of 3 confounding adjustment methods (CA regression, PS regression, and hdPS regression) in an observational study of the impact of asthma controller medication use (LTRA vs. ICS) on 4 asthma-related adverse outcomes.

In each of 4 population subgroups defined by insurance type (commercial health plans vs. TennCare) and allergic rhinitis diagnosis, these 3 adjustment approaches gave essentially equivalent results with no consistent differences in point estimates and highly overlapping confidence intervals. Consistency in results brings reassurance about appropriate model selection and indicates that the hdPS approach did not identify any additional highly influential confounder.

Our finding that LTRAs may be just as effective as ICSs in preventing asthma-related exacerbations in real-life settings is contrary to the results of randomized clinical trials in which patients using ICSs were found to experience fewer asthma-related exacerbations than patients using LTRAs (15). One possible reason for our findings is that additional nonadherence beyond that adjusted for by the time-varying PDC played a role (13). Alternatively, persistent unmeasured confounding could also explain our results. We ascertained the predefined and empirical covariates from electronic health care databases that were designed for nonresearch purposes and, thus, may have limited information on confounders such that all adjustment procedures are subject to residual bias due to uncontrolled confounding. In Tenncare patients with a diagnosis of allergic rhinitis, we found that those who were treated with LTRAs were less likely to experience asthma-related exacerbations. This result is consistent with prior studies (32) and may be due to the use of LTRAs as treatment for allergic rhinitis, which can be a trigger of asthma-related exacerbations. Interestingly, this finding was not observed in the commercial health plan population. When comparing TennCare with commercial health plan populations, 2 important distinctions should be noted. First, for the most part, children with public insurance are more likely than those who are privately insured to use the emergency department for nonurgent conditions or, in this case, for minor asthma exacerbations. Thus, in the TennCare population, a patient who experiences an asthma-related emergency department visit is likely to be less ill than a patient in the commercial health plan population who had an emergency department visit. Also, TennCare patients were more likely than commercial health plan patients to be prescribed an LTRA. One possible reason is that adherence to oral medications, such as LTRAs, may be higher than adherence to inhaled medications, such as ICSs. The increased effectiveness of LTRAs in the TennCare population may reflect real differences in “effectiveness” between the cohorts (because of underlying differences in asthma severity and health-seeking behaviors) or increased power to detect an effect (because of increased number of LTRA-exposed individuals); alternatively, it may have been a spurious finding due to residual confounding bias or imperfect adjustment of medication adherence.

It is unclear how generalizable our findings are to other data sets or analytical questions. Nevertheless, the PS and hdPS analyses have been found to be useful in other studies (6, 33–35). We expect these approaches to be used more commonly because of the analytical advantages we described above and the increasingly available electronic health care data. Thus, our critical observations on how to correctly apply these methods in observational data analyses are important. We describe the 4 most important observations below.

First, even though accounting for covariate overlap did not have a substantial impact in this specific application, likely because covariate nonoverlap was minimal, we recommend adopting this as a routine analysis step, even for traditional CA methods. The validity of comparative effectiveness analysis is suspect when covariate overlap is absent. Common support is required to prevent extrapolation beyond the range of the data. We found in a different application that failure to assess covariate overlap can result in substantially different results and conclusions (36). However, theoretical guidance on determining the common support is not available, at least to the best of our knowledge. We determined the common support region on an ad hoc basis. In the absence of objective, evidence-based guidelines, we suggest varying the cutoff points for the common support selection and assessing the robustness of analytical results to such changes.

Second, it is important to identify confounders that have a strong impact on medication use patterns and to conduct separate logistic regression models among the subgroups defined by the confounders. In our example, allergic rhinitis and the health plans played this role. A combined logistic regression model adjusting for the confounders as independent variables will not suffice for proper PS estimation and overlap assessment. For example, in this application, had we pooled all 5 commercial health plans, adjusted for health plan as a covariate in the PS model, and estimated the PS kernel densities within all 5 health plans, the PS density estimates for the non-KP sites would have been so small (because of the much larger number of patients from the KP sites) that all of them would have fallen outside the common support region and hence been excluded from subsequent analyses. Thus, for the purpose of common support selection, it is important to separate individuals with different PS ranges.

Third, capping the number of empirical covariates may be important. During our data analyses, we discovered a recent study that was conducted by a group of researchers including the inventors of the hdPS approach (28). This study suggests capping the number of empirical covariates such that there are at least 50 individuals per main effect term in the logistic regression model. We adopted this recommendation because we agree that it is important to determine the number of empirical covariates in the hdPS analysis to avoid overfitting in the logistic regression model for hdPS estimation. Nonetheless, we think this recommendation needs to be evaluated more thoroughly.

Fourth, although we tried to better understand the results across different methods, we realize that the current hdPS program does not output the diagnosis, procedure, or national drug codes from which the selected empirical covariates were derived. It prevents the investigators from modifying the empirical covariate definitions on the basis of their clinical expertise, which may result in better performance of the hdPS approach.

The administrative claims databases we used have limitations, including the lack of detailed information on clinical measures of disease severity and level of control, no medication order data, potential missing data from health care encounters and medication dispensing not captured by the claims systems, loss to follow-up due to patients switching health plans, and claims coding errors. However, these issues are not unique to the PEAL database. Another limitation of our analyses is that the use of asthma controller medication is not a 1-time decision. Parents of pediatric asthma patients may often consider whether to stop, resume, or decrease the frequency of controller medication use on the basis of multiple factors, including the level of disease control while their children are on therapy. Some of these factors may affect the risk of exacerbation outcomes. The electronic health care databases, unfortunately, have very limited information on the level of disease control and, thus, do not allow us to adjust for time-varying confounding appropriately. Hence, we used the 3 confounding adjustment methods only for the purpose of controlling for baseline confounding. We then used the time-varying PDC measure to adjust for adherence because we were interested in comparing the effects of the controller medications when patients were adhering to the medications.

In summary, confounding adjustment in observational data is challenging. In comparing 3 approaches, we found that the results in our application were robust to method selection and implementation factors. Our main findings are important observations on how to correctly apply these methods in observational data analysis. In addition, we have identified key areas in which more work is needed to guide implementation.

ACKNOWLEDGMENTS

Author affiliations: Center for Child Health Care Studies, Department of Population Medicine, Harvard Pilgrim Health Care Institute, Boston, Massachusetts (Lingling Li, Ann Chen Wu); Harvard Medical School, Boston, Massachusetts (Lingling Li, Ann Chen Wu); Center for Health Research—Northwest, Kaiser Permanente, Portland, Oregon (William M. Vollmer); Center for Health Research—Southeast, Kaiser Permanente Georgia, Atlanta, Georgia (Melissa G. Butler); Department of Biostatistics, Vanderbilt University School of Medicine, Nashville, Tennessee (Pingsheng Wu); HealthPartners Institute for Education and Research, Minneapolis, Minnesota (Elyse O. Kharbanda); and Division of General Pediatrics, Department of Pediatrics, Children's Hospital, Boston, Massachusetts (Ann Chen Wu).

The Population-Based Effectiveness in Asthma and Lung Diseases Network is supported by the Agency for Healthcare Research and Quality (grant 1R01HS019669).

We thank Dr. Gurvaneet Randhawa at the Agency for Healthcare Research and Quality for his support and advice. This project relied on the hard work and intellectual contributions of many other members of the PEAL Network study team, including Drs. Tracy A. Lieu, Stephen Soumerai, Robert L. Davis, Tina Hartert, James Nordin, and John Hsu. We thank Irina Miroshnik for preparing the analytical data set for us. We are grateful to the suggestions of the PEAL Network National Advisory Committee, which included Drs. David Au, Marie Griffin, Jerry Krishnan, Robert F. Lemanske, Richard Platt, and Michael Schatz. We are indebted to the Tennessee Bureau of TennCare of the Department of Finance and Administration and the Tennessee Department of Health, Office of Policy, Planning and Assessment, for providing the TennCare Medicaid data.

Conflict of interest: none declared.

Appendix Table 1.

Baseline Characteristics of LTRA and ICS Users Among Subjects on the PS Common Supports^a From 5 Commercial Health Plans and TennCare, 2004–2010

Characteristic	Commercial Health Plan Subjects					TennCare Subjects
	LTRA (n = 1,190)		ICS (n = 12,401)		PS-adjusted SD^b	LTRA (n = 5,239)		ICS (n = 3,435)		PS-adjusted SD^b
	No.	%	No.	%	PS-adjusted SD^b	No.	%	No.	%	PS-adjusted SD^b
Age, years
4–11	881	74.0	8,931	72.0	0.07	4,084	78.0	2,606	75.9	0.04
12–17	309	26.0	3,470	28.0	0.07	1,155	22.0	829	24.1	0.04
Sex
Female	494	41.5	5,040	40.6	0.06	2,258	43.1	1,473	42.9	0.04
Male	696	58.5	7,361	59.4	0.06	2,981	56.9	1,962	57.1	0.04
Site
HPHC	486	40.8	1,837	14.8	0.07
HealthPartners^c	346	29.1	1,111	9.0	0.07
KP Northern California	270	22.7	7,345	59.2	0.08
KP Northwest	40	3.4	1,110	9.0	0.12^d
KP Georgia	48	4.0	998	8.0	0.13^d
Race
White	319	26.8	3,998	32.2	0.06	3,058	58.4	1,757	51.1	0.06
Asian	40	3.4	1,013	8.2	0.09	40	0.8	27	0.8	0.03
Black	48	4.0	1,145	9.2	0.08	1,695	32.4	1,324	38.5	0.07
Hispanic	75	6.3	2,024	16.3	0.06	236	4.5	173	5.0	0.03
Other	708	59.5	4,221	34.0	0.08	210	4.0	154	4.5	0.01
History of smoking	14	1.2	312	2.5	0.06	53	1.0	38	1.1	0.03
Experienced the following in prior 12 months
ED visit	214	18.0	2,790	22.5	0.06	1,898	36.2	1,452	42.3	0.06
Hospitalization	42	3.5	654	5.3	0.07	90	1.7	91	2.6	0.03
Outpatient visit	479	40.3	5,856	47.2	0.11^d	1,407	26.9	1,140	33.2	0.05
No. of dispensings in prior 12 months
Oral corticosteroids
0	97	8.2	1,581	12.7	0.09	848	16.2	608	17.7	0.06
1	867	72.9	9,144	73.7	0.07	3,140	59.9	1,948	56.7	0.03
≥2	226	19.0	1,676	13.6	0.10^d	1,251	23.9	879	25.6	0.03
Short-acting β agonists
0	291	24.5	3,300	26.6	0.10^d	1,689	32.2	903	26.4	0.09
1–5	879	73.9	8,968	72.3	0.08	3,333	63.6	2,356	68.6	0.09
≥6	20	1.7	133	1.1	0.08	217	4.1	176	5.1	0.01
Medicaid coverage	44	3.7	569	4.6	0.09
Diagnoses in prior 12 months
Acute respiratory infection	914	76.8	8,731	70.4	0.09	3,976	75.9	2,454	71.4	0.04
Gastroesophageal reflux disease	23	1.9	134	1.1	0.10^d	123	2.3	76	2.2	0.03
Allergic rhinitis	489	41.1	2,335	18.8	0.00	1,005	19.2	512	14.9	0.03
PEAL Charlson Score >0^e	11	0.9	78	0.6	0.07	37	0.7	27	0.8	0.02
No. of generic drugs used in prior 12 months
≤1st quartile^f	450	37.8	5,364	43.3	0.04	1,562	29.8	1,144	33.3	0.06
(1st, 2nd] quartile^f	230	19.3	2,321	18.7	0.09	1,191	22.7	790	23.0	0.04
(2nd, 3rd] quartile^f	226	19.0	1,972	15.9	0.10^d	1,303	24.9	808	23.5	0.05
>3rd quartile^f	284	23.9	2,744	22.1	0.11^d	1,182	22.6	639	20.2	0.04

Open in a new tab

^a The range over which the smoothed histograms of the PSs within each exposure group were 2% or greater.

^b The PS-adjusted standardized difference was calculated as the weighted average of the 10 within-stratum standardized differences. The 10 strata were defined by 2 subgroups (subjects in health plans/TennCare with and without allergic rhinitis diagnosis) and the within-group PS quintiles. The weights were the proportions of subjects in these strata.

^c Based in Minneapolis, Minnesota.

^d A standardized difference of 0.10 or greater indicates the existence of some imbalance.

^e A modified Charlson score, which included all diseases that comprise the Charlson Comorbidity Index except chronic pulmonary disease.

^f Quartiles were estimated within the 2 subgroups with and without diagnosed allergic rhinitis.

REFERENCES

1.Sox HC, Greenfield S. Comparative effectiveness research: a report from the Institute of Medicine. Ann Intern Med. 2009;151(3):203–205. doi: 10.7326/0003-4819-151-3-200908040-00125. [DOI] [PubMed] [Google Scholar]
2.Nallamothu BK, Hayward RA, Bates ER. Beyond the randomized clinical trial: the role of effectiveness studies in evaluating cardiovascular therapies. Circulation. 2008;118(12):1294–1303. doi: 10.1161/CIRCULATIONAHA.107.703579. [DOI] [PubMed] [Google Scholar]
3.Hernan MA, Hernandez-Diaz S, Robins JM. A structural approach to selection bias. Epidemiology. 2004;15(5):615–625. doi: 10.1097/01.ede.0000135174.63482.43. [DOI] [PubMed] [Google Scholar]
4.Platt RW, Joseph KS, Ananth CV, et al. A proportional hazards model with time-dependent covariates and time-varying effects for analysis of fetal and infant death. Am J Epidemiol. 2004;160(3):199–206. doi: 10.1093/aje/kwh201. [DOI] [PubMed] [Google Scholar]
5.Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55. [Google Scholar]
6.Schneeweiss S, Rassen JA, Glynn RJ, et al. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology. 2009;20(4):512–522. doi: 10.1097/EDE.0b013e3181a663cc. [DOI] [PMC free article] [PubMed] [Google Scholar]
7.Casella G, Berger RL. Statistical Inference. Pacific Grove, CA: Duxbury; 2002. [Google Scholar]
8.Rubin DB. Estimating causal effects from large data sets using propensity scores. Ann Intern Med. 1997;127(8 Pt 2):757–763. doi: 10.7326/0003-4819-127-8_part_2-199710151-00064. [DOI] [PubMed] [Google Scholar]
9.Sturmer T, Joshi M, Glynn RJ, et al. A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. J Clin Epidemiol. 2006;59(5):437–447. doi: 10.1016/j.jclinepi.2005.07.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Glynn RJ, Schneeweiss S, Sturmer T. Indications for propensity scores and review of their use in pharmacoepidemiology. Basic Clin Pharmacol Toxicol. 2006;98(3):253–259. doi: 10.1111/j.1742-7843.2006.pto_293.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
11.Austin PC. A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003. Stat Med. 2008;27(12):2037–2049. doi: 10.1002/sim.3150. [DOI] [PubMed] [Google Scholar]
12.Stuart EA. Developing practical recommendations for the use of propensity scores: discussion of ‘A critical appraisal of propensity score matching in the medical literature between 1996 and 2003’ by Peter Austin, Statistics in Medicine. Stat Med. 2008;27(12):2062–2065. doi: 10.1002/sim.3207. [DOI] [PubMed] [Google Scholar]
13.Chauhan BF, Ducharme FM. Anti-leukotriene agents compared to inhaled corticosteroids in the management of recurrent and/or chronic asthma in adults and children. Cochrane Database Syst Rev. 2012;5:CD002314. doi: 10.1002/14651858.CD002314.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Garcia Garcia ML, Wahn U, Gilles L, et al. Montelukast, compared with fluticasone, for control of asthma among 6- to 14-year-old patients with mild asthma: the MOSAIC Study. Pediatrics. 2005;116(2):360–369. doi: 10.1542/peds.2004-1172. [DOI] [PubMed] [Google Scholar]
15.Sorkness CA, Lemanske RF, Jr, Mauger DT, et al. Long-term comparison of 3 controller regimens for mild-moderate persistent childhood asthma: the Pediatric Asthma Controller Trial. J Allergy Clin Immunol. 2007;119(1):64–72. doi: 10.1016/j.jaci.2006.09.042. [DOI] [PubMed] [Google Scholar]
16.Szefler SJ, Baker JW, Uryniak T, et al. Comparative study of budesonide inhalation suspension and montelukast in young children with mild persistent asthma. J Allergy Clin Immunol. 2007;120(5):1043–1050. doi: 10.1016/j.jaci.2007.08.063. [DOI] [PubMed] [Google Scholar]
17.Wu AC, Li L, Fung V, et al. Leukotriene inhibitors may be more effective than inhaled corticosteroids in preventing asthma-related exacerbations. Am J Respir Crit Care Med. 2013;187:A2332. [Google Scholar]
18.Hornbrook MC, Hart G, Ellis JL, et al. Building a virtual cancer research organization. J Natl Cancer Inst Monogr. 2005;(35):12–25. doi: 10.1093/jncimonographs/lgi033. [DOI] [PubMed] [Google Scholar]
19.Wagner EH, Greene SM, Hart G, et al. Building a research consortium of large health systems: the Cancer Research Network. J Natl Cancer Inst Monogr. 2005;(35):3–11. doi: 10.1093/jncimonographs/lgi032. [DOI] [PubMed] [Google Scholar]
20.National Asthma Education and Prevention Program. Expert Panel Report 3 (EPR-3): guidelines for the diagnosis and management of asthma-summary report 2007. J Allergy Clin Immunol. 2007;120(5 suppl):S94–S138. doi: 10.1016/j.jaci.2007.09.043. [DOI] [PubMed] [Google Scholar]
21.Peterson AM, Nau DP, Cramer JA, et al. A checklist for medication compliance and persistence studies using retrospective databases. Value Health. 2007;10(1):3–12. doi: 10.1111/j.1524-4733.2006.00139.x. [DOI] [PubMed] [Google Scholar]
22.Williams LK, Peterson EL, Wells K, et al. Quantifying the proportion of severe asthma exacerbations attributable to inhaled corticosteroid nonadherence. J Allergy Clin Immunol. 2011;128(6):1185–1191. doi: 10.1016/j.jaci.2011.09.011. e2. [DOI] [PMC free article] [PubMed] [Google Scholar]
23.Kurth T, Walker AM, Glynn RJ, et al. Results of multivariable logistic regression, propensity matching, propensity adjustment, and propensity-based weighting under conditions of nonuniform effect. Am J Epidemiol. 2005;163(3):262–270. doi: 10.1093/aje/kwj047. [DOI] [PubMed] [Google Scholar]
24.Rosenbaum PR, Rubin DB. Reducing bias in observational studies using subclassification on the propensity score. J Am Stat Assoc. 1984;79(387):516–524. [Google Scholar]
25.Ho DE, Imai K, King G, et al. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Polit Anal. 2007;15(3):199–236. [Google Scholar]
26.Rassen JA, Glynn RJ, Brookhart MA, et al. Covariate selection in high-dimensional propensity score analyses of treatment effects in small samples. Am J Epidemiol. 2011;173(12):1404–1413. doi: 10.1093/aje/kwr001. [DOI] [PMC free article] [PubMed] [Google Scholar]
27.Toh S, Garcia Rodriguez LA, Hernan MA. Confounding adjustment via a semi-automated high-dimensional propensity score algorithm: an application to electronic medical records. Pharmacoepidemiol Drug Saf. 2011;20(8):849–857. doi: 10.1002/pds.2152. [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Neugebauer R, Schmittdiel JA, Zhu Z, et al. High-dimensional propensity score algorithm in comparative effectiveness research with time-varying interventions. (http://diabetestranslation.org/en/news_publications/Reports/RNeugebauer_hdPS_StatMed.pdf?view=publicationpage. ). (Accessed January 9, 2014) [DOI] [PubMed]
29.Diamant Z, Mantzouranis E, Bjermer L. Montelukast in the treatment of asthma and beyond. Expert Rev Clin Immunol. 2009;5(6):639–658. doi: 10.1586/eci.09.62. [DOI] [PubMed] [Google Scholar]
30.Yawn BP. Importance of allergic rhinitis management in achieving asthma control: ARIA update. Expert Rev Respir Med. 2008;2(6):713–719. doi: 10.1586/17476348.2.6.713. [DOI] [PubMed] [Google Scholar]
31.Mamdani M, Sykora K, Li P, et al. Reader's guide to critical appraisal of cohort studies: 2. Assessing potential for confounding. BMJ. 2005;330(7497):960–962. doi: 10.1136/bmj.330.7497.960. [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Price D, Musgrave SD, Shepstone L, et al. Leukotriene antagonists as first-line or add-on asthma-controller therapy. N Engl J Med. 2011;364(18):1695–1707. doi: 10.1056/NEJMoa1010846. [DOI] [PubMed] [Google Scholar]
33.Braitman LE, Rosenbaum PR. Rare outcomes, common treatments: analytic strategies using propensity scores. Ann Intern Med. 2002;137(8):693–695. doi: 10.7326/0003-4819-137-8-200210150-00015. [DOI] [PubMed] [Google Scholar]
34.Rosenbaum P. Observational Studies. New York, NY: Springer-Verlag; 2002. [Google Scholar]
35.Biondi-Zoccai G, Romagnoli E, Agostoni P, et al. Are propensity scores really superior to standard multivariable analysis? Contemp Clin Trials. 2011;32(5):731–740. doi: 10.1016/j.cct.2011.05.006. [DOI] [PubMed] [Google Scholar]
36.Tse SM, Li L, Butler MG, et al. Statin exposure is associated with decreased asthma-related emergency department visits and oral corticosteroid use. Am J Respir Crit Care Med. 2013;188(9):1076–1082. doi: 10.1164/rccm.201306-1017OC. [DOI] [PMC free article] [PubMed] [Google Scholar]

[KWT323C1] 1.Sox HC, Greenfield S. Comparative effectiveness research: a report from the Institute of Medicine. Ann Intern Med. 2009;151(3):203–205. doi: 10.7326/0003-4819-151-3-200908040-00125. [DOI] [PubMed] [Google Scholar]

[KWT323C2] 2.Nallamothu BK, Hayward RA, Bates ER. Beyond the randomized clinical trial: the role of effectiveness studies in evaluating cardiovascular therapies. Circulation. 2008;118(12):1294–1303. doi: 10.1161/CIRCULATIONAHA.107.703579. [DOI] [PubMed] [Google Scholar]

[KWT323C3] 3.Hernan MA, Hernandez-Diaz S, Robins JM. A structural approach to selection bias. Epidemiology. 2004;15(5):615–625. doi: 10.1097/01.ede.0000135174.63482.43. [DOI] [PubMed] [Google Scholar]

[KWT323C4] 4.Platt RW, Joseph KS, Ananth CV, et al. A proportional hazards model with time-dependent covariates and time-varying effects for analysis of fetal and infant death. Am J Epidemiol. 2004;160(3):199–206. doi: 10.1093/aje/kwh201. [DOI] [PubMed] [Google Scholar]

[KWT323C5] 5.Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55. [Google Scholar]

[KWT323C6] 6.Schneeweiss S, Rassen JA, Glynn RJ, et al. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology. 2009;20(4):512–522. doi: 10.1097/EDE.0b013e3181a663cc. [DOI] [PMC free article] [PubMed] [Google Scholar]

[KWT323C7] 7.Casella G, Berger RL. Statistical Inference. Pacific Grove, CA: Duxbury; 2002. [Google Scholar]

[KWT323C8] 8.Rubin DB. Estimating causal effects from large data sets using propensity scores. Ann Intern Med. 1997;127(8 Pt 2):757–763. doi: 10.7326/0003-4819-127-8_part_2-199710151-00064. [DOI] [PubMed] [Google Scholar]

[KWT323C9] 9.Sturmer T, Joshi M, Glynn RJ, et al. A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. J Clin Epidemiol. 2006;59(5):437–447. doi: 10.1016/j.jclinepi.2005.07.004. [DOI] [PMC free article] [PubMed] [Google Scholar]

[KWT323C10] 10.Glynn RJ, Schneeweiss S, Sturmer T. Indications for propensity scores and review of their use in pharmacoepidemiology. Basic Clin Pharmacol Toxicol. 2006;98(3):253–259. doi: 10.1111/j.1742-7843.2006.pto_293.x. [DOI] [PMC free article] [PubMed] [Google Scholar]

[KWT323C11] 11.Austin PC. A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003. Stat Med. 2008;27(12):2037–2049. doi: 10.1002/sim.3150. [DOI] [PubMed] [Google Scholar]

[KWT323C12] 12.Stuart EA. Developing practical recommendations for the use of propensity scores: discussion of ‘A critical appraisal of propensity score matching in the medical literature between 1996 and 2003’ by Peter Austin, Statistics in Medicine. Stat Med. 2008;27(12):2062–2065. doi: 10.1002/sim.3207. [DOI] [PubMed] [Google Scholar]

[KWT323C13] 13.Chauhan BF, Ducharme FM. Anti-leukotriene agents compared to inhaled corticosteroids in the management of recurrent and/or chronic asthma in adults and children. Cochrane Database Syst Rev. 2012;5:CD002314. doi: 10.1002/14651858.CD002314.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]

[KWT323C14] 14.Garcia Garcia ML, Wahn U, Gilles L, et al. Montelukast, compared with fluticasone, for control of asthma among 6- to 14-year-old patients with mild asthma: the MOSAIC Study. Pediatrics. 2005;116(2):360–369. doi: 10.1542/peds.2004-1172. [DOI] [PubMed] [Google Scholar]

[KWT323C15] 15.Sorkness CA, Lemanske RF, Jr, Mauger DT, et al. Long-term comparison of 3 controller regimens for mild-moderate persistent childhood asthma: the Pediatric Asthma Controller Trial. J Allergy Clin Immunol. 2007;119(1):64–72. doi: 10.1016/j.jaci.2006.09.042. [DOI] [PubMed] [Google Scholar]

[KWT323C16] 16.Szefler SJ, Baker JW, Uryniak T, et al. Comparative study of budesonide inhalation suspension and montelukast in young children with mild persistent asthma. J Allergy Clin Immunol. 2007;120(5):1043–1050. doi: 10.1016/j.jaci.2007.08.063. [DOI] [PubMed] [Google Scholar]

[KWT323C17] 17.Wu AC, Li L, Fung V, et al. Leukotriene inhibitors may be more effective than inhaled corticosteroids in preventing asthma-related exacerbations. Am J Respir Crit Care Med. 2013;187:A2332. [Google Scholar]

[KWT323C18] 18.Hornbrook MC, Hart G, Ellis JL, et al. Building a virtual cancer research organization. J Natl Cancer Inst Monogr. 2005;(35):12–25. doi: 10.1093/jncimonographs/lgi033. [DOI] [PubMed] [Google Scholar]

[KWT323C19] 19.Wagner EH, Greene SM, Hart G, et al. Building a research consortium of large health systems: the Cancer Research Network. J Natl Cancer Inst Monogr. 2005;(35):3–11. doi: 10.1093/jncimonographs/lgi032. [DOI] [PubMed] [Google Scholar]

[KWT323C20] 20.National Asthma Education and Prevention Program. Expert Panel Report 3 (EPR-3): guidelines for the diagnosis and management of asthma-summary report 2007. J Allergy Clin Immunol. 2007;120(5 suppl):S94–S138. doi: 10.1016/j.jaci.2007.09.043. [DOI] [PubMed] [Google Scholar]

[KWT323C21] 21.Peterson AM, Nau DP, Cramer JA, et al. A checklist for medication compliance and persistence studies using retrospective databases. Value Health. 2007;10(1):3–12. doi: 10.1111/j.1524-4733.2006.00139.x. [DOI] [PubMed] [Google Scholar]

[KWT323C22] 22.Williams LK, Peterson EL, Wells K, et al. Quantifying the proportion of severe asthma exacerbations attributable to inhaled corticosteroid nonadherence. J Allergy Clin Immunol. 2011;128(6):1185–1191. doi: 10.1016/j.jaci.2011.09.011. e2. [DOI] [PMC free article] [PubMed] [Google Scholar]

[KWT323C23] 23.Kurth T, Walker AM, Glynn RJ, et al. Results of multivariable logistic regression, propensity matching, propensity adjustment, and propensity-based weighting under conditions of nonuniform effect. Am J Epidemiol. 2005;163(3):262–270. doi: 10.1093/aje/kwj047. [DOI] [PubMed] [Google Scholar]

[KWT323C24] 24.Rosenbaum PR, Rubin DB. Reducing bias in observational studies using subclassification on the propensity score. J Am Stat Assoc. 1984;79(387):516–524. [Google Scholar]

[KWT323C25] 25.Ho DE, Imai K, King G, et al. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Polit Anal. 2007;15(3):199–236. [Google Scholar]

[KWT323C26] 26.Rassen JA, Glynn RJ, Brookhart MA, et al. Covariate selection in high-dimensional propensity score analyses of treatment effects in small samples. Am J Epidemiol. 2011;173(12):1404–1413. doi: 10.1093/aje/kwr001. [DOI] [PMC free article] [PubMed] [Google Scholar]

[KWT323C27] 27.Toh S, Garcia Rodriguez LA, Hernan MA. Confounding adjustment via a semi-automated high-dimensional propensity score algorithm: an application to electronic medical records. Pharmacoepidemiol Drug Saf. 2011;20(8):849–857. doi: 10.1002/pds.2152. [DOI] [PMC free article] [PubMed] [Google Scholar]

[KWT323C28] 28.Neugebauer R, Schmittdiel JA, Zhu Z, et al. High-dimensional propensity score algorithm in comparative effectiveness research with time-varying interventions. (http://diabetestranslation.org/en/news_publications/Reports/RNeugebauer_hdPS_StatMed.pdf?view=publicationpage. ). (Accessed January 9, 2014) [DOI] [PubMed]

[KWT323C29] 29.Diamant Z, Mantzouranis E, Bjermer L. Montelukast in the treatment of asthma and beyond. Expert Rev Clin Immunol. 2009;5(6):639–658. doi: 10.1586/eci.09.62. [DOI] [PubMed] [Google Scholar]

[KWT323C30] 30.Yawn BP. Importance of allergic rhinitis management in achieving asthma control: ARIA update. Expert Rev Respir Med. 2008;2(6):713–719. doi: 10.1586/17476348.2.6.713. [DOI] [PubMed] [Google Scholar]

[KWT323C31] 31.Mamdani M, Sykora K, Li P, et al. Reader's guide to critical appraisal of cohort studies: 2. Assessing potential for confounding. BMJ. 2005;330(7497):960–962. doi: 10.1136/bmj.330.7497.960. [DOI] [PMC free article] [PubMed] [Google Scholar]

[KWT323C32] 32.Price D, Musgrave SD, Shepstone L, et al. Leukotriene antagonists as first-line or add-on asthma-controller therapy. N Engl J Med. 2011;364(18):1695–1707. doi: 10.1056/NEJMoa1010846. [DOI] [PubMed] [Google Scholar]

[KWT323C33] 33.Braitman LE, Rosenbaum PR. Rare outcomes, common treatments: analytic strategies using propensity scores. Ann Intern Med. 2002;137(8):693–695. doi: 10.7326/0003-4819-137-8-200210150-00015. [DOI] [PubMed] [Google Scholar]

[KWT323C34] 34.Rosenbaum P. Observational Studies. New York, NY: Springer-Verlag; 2002. [Google Scholar]

[KWT323C35] 35.Biondi-Zoccai G, Romagnoli E, Agostoni P, et al. Are propensity scores really superior to standard multivariable analysis? Contemp Clin Trials. 2011;32(5):731–740. doi: 10.1016/j.cct.2011.05.006. [DOI] [PubMed] [Google Scholar]

[KWT323C36] 36.Tse SM, Li L, Butler MG, et al. Statin exposure is associated with decreased asthma-related emergency department visits and oral corticosteroid use. Am J Respir Crit Care Med. 2013;188(9):1076–1082. doi: 10.1164/rccm.201306-1017OC. [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

A Comparison of Confounding Adjustment Methods for Assessment of Asthma Controller Medication Effectiveness

Lingling Li

William M Vollmer

Melissa G Butler

Pingsheng Wu

Elyse O Kharbanda

Ann Chen Wu

Abstract