Colorado Asthma Toolkit Implementation Improves Some Process Measures of Asthma Care

Abstract

Background:

The Colorado Asthma Toolkit Program has been shown to improve processes of care with less evidence demonstrating improved outcomes.

Objective:

To model the association between pre-and-post Colorado Asthma Toolkit Program (CATP) status and asthma related process and outcome measures among patients ages 5–64 years receiving care in safety-net primary care practices.

Methods:

This is an implementation study involving secondary pre-post analysis of existing structured clinical, administrative, and claims data. Nine primary care practices in a federally qualified health center network implemented the CATP. Processes of care and health and utilization outcomes were evaluated pre-post implementation in a cohort of patients with asthma using generalized linear mixed models.

Results:

The study cohort included 2678 patients age 5–64 years with at least one visit to one of the nine participating practices during the study period (3/12/2010 to 12/1/2012). A comparison of 12 months pre- and post-CATP implementation showed improvement in some process measures of asthma care associated with the intervention, including the rate of asthma severity measurement, although no change in two Healthcare Effectiveness Data and Information Set (HEDIS) measures: asthma medication ratio and medication management for people with asthma. We also found no change in asthma outcomes measured across multiple domains: exacerbations, utilization, symptom scores and pulmonary physiology measures.

Conclusions:

Implementation of the Colorado Asthma Toolkit Program in a primary care setting led to some improved processes of asthma care, but no changes in measured outcomes. Recommendations for future work include supplemental follow up training including case review.

Introduction

Asthma is one of the most common chronic illnesses in the United States. In 2015, the overall prevalence of actve asthma was 8.4% among children (6.2 million)¹ and 7.6% among adults (18.4 million),² and asthma is the lead diagnosis for 2.0 million ED visits³ and 340,000 hospitalizations³ annually. When managed appropriately in primary care settings, asthma is a controllable disease,⁴ such that most asthma exacerbations—episodes of increased airway obstruction—are preventable.⁵ As asthma exacerbations are the reason for asthma-related ED visits, hospitalizations, missed school/work days, and deaths, evidence on effective disease management strategies is a priority for policy-makers,^6,7 payers,⁸ and researchers,^9,10 with most interventions targeting the primary care setting.^4,11

National asthma guidelines¹² and subsequent reviews^13–15 provide consistent guidance on what constitutes high quality asthma care; nevertheless, clinicians’ guideline concordance is inconsistent in primary care settings.^16–18 The need for guidelines-focused training in primary care is well established,^19–21 but training programs have met with mixed success. Interventions designed to enhance diagnosis and management of respiratory diseases in primary care settings have had variable success and have not been implemented broadly.^22,23 The Colorado Asthma Toolkit Program (CATP) was introduced as a provider-level intervention to improve implementation of guidelines for high-quality asthma care through addressing known barriers and utilizing known facilitators of implementation.²⁴ Barriers to guideline implementation include lack of knowledge of guideline recommendations, doubt about the evidence base, and lack of time and resources.^25,26 Facilitators of guideline implementation include improved teamwork, comprehensive training and improvements in electronic record systems.^26–28

To improve guideline-concordance of asthma care in primary care, investigators from National Jewish Health (NJH) partnered with local stakeholders to develop both the content and implementation strategy for an intervention to address the known barriers to and facilitators of asthma treatment guideline concordance. The implementation strategy and content of the intervention-- the CATP—was first developed in 2007 and addresses all 4 categories of provider-level asthma intervention.²⁹ To date, 500 providers at 170 Colorado primary care practices serving safety-net patients have been trained using the CATP.³⁰ As part of this program, NJH partnered with the Metro Community Provider Network (MCPN)³¹ to implement the CATP in nine practices in 2011.³² MCPN is a network of health centers serving safety-net populations throughout metropolitan Denver.³¹ The MCPN CATP implementation was adapted from an earlier, stakeholder-designed version of the CATP that successfully changed practice behavior in 57 primary care offices in rural eastern Colorado.²⁴ The CATP has been shown to improve guideline-concordance of care processes related to spirometry, asthma action plans and inhaled corticosteroids.³³

Assessment of the impact of the CATP on processes and outcomes has been limited by the minimal resources typically present in safety-net practices for tracking asthma outcomes longitudinally.^32,34 However, MCPN participates in the Scalable Architecture for Federated Translational Inquiries Network (SAFTINet),^35,36 an Agency for Healthcare Quality and Research (AHRQ)-funded practice-based distributed research network supported by a sophisticated technical infrastructure that standardizes clinical and claims data to a common data model and links electronic health record (EHR) and claims data at the patient level to facilitate patient centered outcomes research.^37,38 The SAFTINet partner databases include both the clinical EHR data and Medicaid claims data necessary for operationalizing standard asthma outcome measures.³⁹ The objective was to perform secondary data analysis of asthma process and outcome measures associated with implementation of the CATP at MCPN practices, using existing data from SAFTINet. We performed a pre-post single-group study of the CATP implementation, using generalized linear mixed models (GLMM) to evaluate pre-post changes in processes and outcomes of care at the patient level (Figure 1). Specifically, these measures include use of spirometry, asthma severity assessment, and asthma action plan use among all asthmatics, and controller medication use and medication ratios among persistent asthmatics.^40,41 We hypothesized that patients in participating practices would have improved asthma control and fewer exacerbations after CATP implementation compared to pre-implementation.

Figure 1.

Colorado Asthma Toolkit Program Conceptual Model. Key content areas were introduced in three education and feedback/audit sessions, which were then reinforced via three other approaches: 1) Organizational change to optimize efficieny integrating CATP processes into practice workflow; 2) Decision support tools for asthma management; and 3) Online resources to help reinforce and sustain practice changes.

Methods

Study Design and Setting

We performed an implementation study involving secondary pre-post analysis of existing structured clinical, administrative, and claims data. The data network from which study data was obtained is SAFTINet, a clinical and claims distritubed data network designed to securely share electronic health data to support quality improvement and research. The SAFTINet databases include existing administrative, clinical, Medicaid claims, patient-reported outcome data including the Asthma Control Test,⁴² and enrollment data collected during routine clinical care. Data are standardized to the OMOP common data model (Version 4).⁴³ The data are limited data sets as defined by the Health Insurance Portability and Accountability act (HIPAA), de-identified with the exception of service dates, birth dates for those < 90 years old, and zip codes.

We requested data from the MCPN SAFTINet database for patients who had experienced at least one encounter during the study period at at least one of the the nine participating practices. Electronic health data from the SAFTINet databases was used to select patients for inclusion in the cohorts, to operationalize patient-level outcomes and covariates, and to associate patients with practices.

Cohort Selection

Inclusion criteria included age between 5 and 64 years by the start of the study period (3/12/2010 to 12/1/2012); at least one visit to an MCPN practice during the study period; at least one month of Medicaid eligibility during the study period; and either an asthma diagnosis or an asthma-specific medication. Given the chronic nature of asthma, for asthma diagnosis events, we included any asthma diagnostic code or string assigned for the patient during the study period or five years before study inception. As asthma-specific medications, we included any prescription or fulfillment of at least one medication from the Healthcare Effectiveness Data and Information Set (HEDIS) list “Asthma Controller and Reliever Medications” that occurred during the study period.⁴⁴ We included the same cohort in both the pre- and post-intervention study periods. For more details on the definitions used for asthma diagnosis events and asthma-specific medications, please see the Technical Appendix.

Intervention

Four provider-level asthma intervention categories²⁹—decision support, organizational change, audit and feedback, and education—have been shown to be effective in improving the quality of primary asthma care processes.^5,45 These were all incorporated into the CATP implementation as follows.

1. Education and feedback/audit: All clinical care team members were invited to 3 training sessions led by a nurse practice coach. A practice-coaching strategy was used to emphasize teamwork and to develop a practice-specific asthma protocol. Session 1 was 7 hours, with content in 6 modules: 1) assessment and monitoring, including use of the Asthma Control Test (ACT), 2) spirometry administration and interpretation, 3) asthma tools and techniques, 4) adherence and patient education, 5) non-pharmacologic management, and 6) case studies. Sessions 2 and 3 were 3 hours long and reviewed module content; spirometry technique; collaborative self-management; and specific cases. For the ACT, clinical care teams administered the childhood version for children age 5–11 years (asking the parent to complete it on behalf of the child) and the adult version for ages 12–64 years.

2. Organizational change: At each session, the practice coach used interactive discussion to develop a practice-specific asthma protocol aimed at optimizing efficiency in integrating CATP processes into practice roles and workflow. Practice members were also asked to identify an “asthma champion” to help lead implementation of the practice-specific asthma protocol.

3. Written and computerized clinical decision support: Each provider received a provider manual containing all content from the training, a spirometry guide, and copies of the Asthma Control Test and an asthma action plan The CATP electronic health record asthma template includes severity assessment tools and decision support guiding initial asthma management and recommended actions for ongoing management based on asthma control.

4. Online resources: Providers were given access to online support to help reinforce and sustain practice changes, including an Asthma Toolkit website and Facebook page. These were made accessible to trained and new MCPN providers. The website contains training module videos and provider training manuals. The Facebook page permits providers and the NJH CATP team to interact about the CATP.

Study periods

The intervention period for the CATP was March 12, 2011 to December 1, 2011. We selected a one-year pre- and post-intervention period (March 12, 2010 to March 11, 2011 and December 2, 2011 to December 1, 2012) for our analysis.

Process measures related to guideline-concordance of asthma care

To evaluate the implementation, we selected and assessed several process measures directly related to the content in the CATP training program, as shown in the study’s conceptual model (Figure 1).³³ Each process measure was assessed at the patient level, resulting in separate, dichotomous variables indicating concordance (yes/no) for the pre- and post-CATP periods. The investigators selected process measures for inclusion based on their use in the initial evaluation of the MCPN CATP intervention, their consistency with the conceptual model (Figure 1), their relevance to the NIH asthma guidelines, and their endorsement by the National Quality Forum (NQF).

The two NQF-endorsed HEDIS asthma process measures were the Medication Management for People with Asthma (the percentage of individuals 5–64 years of age who were identified as having persistent asthma and were dispensed appropriate asthma controller medications that they remained on for at least 75% of the period) and the Asthma Medication Ratio (the percentage of individuals 5–64 years of age who were identified as having persistent asthma and had a ratio of controller medications to total asthma medications of 0.50 or greater during the measurement year).⁴⁶ The denominator for these two measures is “persistent asthma”, defined via both HEDIS measure specifications as having at least one of the following during each of two successive years: an asthma-specific ED visit or inpatient visit, at least four outpatient asthma visits plus at least two asthma-specific medications, or at least four asthma-specific medications. We used only Medicaid claims data to calculate HEDIS measure values, as instructed in the HEDIS measure specifications^46–48 (more details in Technical Appendix). We chose not to also include electronic health record data in calculating these measures to avoid duplication of medications or visits.

We included asthma control measurements—both spirometry and ACT—as process measures how many were assessed in each period and as outcome measures based on the value of each measure. We assessed spirometry performance based on procedure codes.

Outcome measures

To evaluate the effectiveness of the CATP, we evaluate a variety of health and utilization outcomes. The study used four categories of outcomes recommended for observational studies by the Asthma Outcomes Workshop convened by the National Institute of Health and Agency for Healthcare Research and Quality.³⁹ These include exacerbations⁴⁹ (primary outcome), healthcare utilization,⁵⁰ composite scores⁵¹ and pulmonary physiology measures.⁵²

We defined the primary outcome measure, asthma exacerbations, using the recommended observational study standards for exacerbation measurement from the Asthma Outcomes workshop.⁴⁹ Specifically, we defined an exacerbation as at least one of the following qualifying events, with qualifying events occurring within 14 days of each other defining a single exacerbation.

1. An emergency department or inpatient visit with a primary diagnostic code for asthma.

2. A cluster of two visits of at least one of the following two types within 1 and 14 days: (a) an outpatient visit (primary care or subspecialty) with a primary diagnostic code for asthma or (b) an ED, IP or outpatient visit with any diagnostic code (not just primary) for asthma. We excluded visits on the same day to avoid double-counting the same visit.

3. Any ED, IP or outpatient visit with an ICD-9 code for asthma that has a 5^th digit, or a diagnosis text string, indicative of an exacerbation.

4. Any prescription or fulfillment of a systemic corticosteroid medication from 0 to 3 days after the date of any type of visit where the visit includes a diagnostic code for asthma. Medication codes for systemic steroids were identified using Generic Product Identifier (GPI) codes. As prescription duration data showed high rates of missing data, we did not include steroid prescription duration in this definition.

For exacerbation criteria events, and two events within a 14-day span were considered the same exacerbation; thus, three or more events could create an exacerbation with a duration of more than 14 days. This is based on evidence showing a median 10–14-day duration of typical asthma exacerbations.⁵³ For further details of the exacerbation definition, please see the Technical Appendix.

In accordance with the Asthma Outcomes Workshop recommendations, we also provide descriptive statistics on the specific components of the exacerbation definition: asthma-specific ED, IP and OP visits, and steroid prescriptions. We also provide summary statistics for the other categories of asthma medications prescribed in each period.

The second category of outcome measures is healthcare utilization events. Here we report each of the three visit type categories—inpatient, emergency department and outpatient—as well as asthma-specific medication use (reliever or controller medications). Due to duplicate medications occurring within both the clinical and claims data, we de-duplicated medication records by person, start date, and by Generic Product Identifier (GPI) code to avoid double-counting the same prescription.

The third category of outcome measures is composite symptom scores—namely, the Asthma Control Test values. The range for the Asthma Control Test (ACT) is 5–25 and the Childhood ACT range is 0–27. For both versions, a score less than 19 is considered poor control and thus ACT results were dichotomized at 19.

The fourth category of outcome measures is physiologic measures, including the results of spirometry (FEV1/FVC ratio percent predicted values^54,55) and of peak expiratory flow monitoring (PEF). For descriptive statistics, report median for patients with >1 observation recorded. For analyses, we dichotomized FE at 80% of the FEV1 predicted value.

Covariates

We adjusted for those variables available in the dataset that were potential confounders of the association between the CATP and asthma control. These include patient demographics (age, sex, race/ethnicity), healthcare insurance coverage and chronic medical complexity. We categorized age using the categories defined in the age-specific guideline recommendations in the National Asthma Education and Prevention Program guideline on asthma management.⁵ We categorized race and ethnicity using the categories present in MCPN’s EHR. Race and ethnicity are documented through each clinic’s patient registration process at the time of initial registration in the MCPN system.

We defined healthcare insurance coverage as the number of months of Medicaid eligibility for each patient during the study period. Because we limited the cohort to patients with at least one month of Medicaid eligibility, our sample included no patients with zero months of eligibility.

To measure medical complexity of each patient in the cohort, we used the Healthcare Cost and Utilization Project’s Chronic Condition Indicator (CCI) tool’s 17 body system categories. For each patient, we counted the number of body systems affected by CCIs, excluding the category for diseases of the respiratory system, as every cohort member had a condition in this category (asthma).⁵⁶

Statistical Methods

Generalized linear mixed models (GLMM) were used to evaluate pre-post changes in process, health, and utilization outcomes within-patient. A random effect was included to account for correlation among observations from the same clinic. Models where comparisons were made between pre-intervention and post-intervention on paired data from individuals also included a random effect for individual-level correlation. Count variables were assumed to follow Poisson distributions when the means were approximately equal to the variances. For all binary outcomes, binomial GLMMs were fit with logit link functions. Data cleaning was performed in R⁵⁷ and SAS⁵⁸, and all models were fit in SAS using the PROC GLIMMIX procedure.

Results

Sample characteristics

After applying inclusion and exclusion criteria, the resulting asthma study cohort included 2,687 patients (Figure 2). The median age was 23 years, with 58.3% of patients in the adult age range (18–64 years), and 66.5% were female. (Table 1) More than half (51.8%) had unknown race, with 32.5% reported as white race and 10.9% as black race. By ethnicity, 31.8% were Hispanic/Latino, 46.7% were not Hispanic/Latino and 21.6% had unknown ethnicity. The count of comorbidity categories were as follows: 12.6% had none and 52.1% had more than 2. The median duration of Medicaid eligibility was the full study period duration—33 months—with 89.5% having at least 20 months of eligibility.

Figure 2.

The study cohort flow diagram.

Table 1:

Asthma cohort demographics

Variable	Asthma Cohort (n =2678)
Age group [n (%)]
5–11 years	747 (27.8%)
12–17 years	374 (13.9%)
18–64 years	1566 (58.3%)
Sex
Male	899 (33.5%)
Female	1788 (66.5%)
Race [n (%)]
White	874 (32.5%)
Black/ African American	294 (10.9%)
Other	128 (4.8%)
Unknown	1391 (51.8%)
Ethnicity
Hispanic/ Latino	854 (31.8%)
Not Hispanic/ Latino	1254 (46.7%)
Unknown	579 (21.6%)
Comorbidity count [n (%)]*
0	340 (12.7%)
1	522 (19.4%)
2	425 (15.8%)
>2	1400 (52.1%)
Medicaid eligibility [months of 33 study months, (median, IQR)]	33 (31–33)

^*Number of Healthcare Cost and Utilization Project (HCUP) Chronic Condition Indicator (CCI) “body systems” wherein the patient has one or more comorbidities, excluding respiratory.

Process Outcomes

For process measures, rates pre- and post-intervention were mixed (See Table 2. Both types of asthma severity assessment—spirometry and ACT performance—increased significantly from pre- to post-intervention. (Table 2) The two NQF-endorsed HEDIS process measures-- asthma medication ratio and medication management for people with asthma—did not change significantly from pre- to post-intervention, although the percentage of patients meeting the HEDIS definition criteria for “persistent asthma” (the measure denominator for both measures) did increase significantly from 4.8% to 8.5% of the cohort, largely driven by more patients meeting medication-dispensing criteria for asthma in the post-intervention period. Another change noted between pre- and post-intervention periods was the rise in the proportion of patients meeting inclusion criteria for HEDIS definition of persistent asthma.

Table 2:

Process Measures

	Pre-Intervention	Post-Intervention	p-value
Asthma severity assessed
Spirometry performed (#; % of patients)	32 (1.2%)	272 (10.1%)	0.0031,2
ACT assessed (#; % of patients)	0 (0.0%)	310 (11.5%)	N/A
HEDIS measures
Patients meeting inclusion criteria for HEDIS definition of persistent asthma (% of study sample)	129 (4.8%)	228 (8.5%)	<0.00011
Asthma medication ratio: percentage identified as having persistent asthma who had a ratio of controller medications to total asthma medications of 0.50 or greater	52.3%	58.5%	0.3
Medication management for people with asthma: percentage identified as having persistent asthma who were dispensed an asthma controller medication that they remained on for at least
75% of their treatment period	27.6%	34.2%	0.2
50% of their treatment period	56.9%	63.2%	0.3

All p-values in this table obtained using generalized linear mixed models with random effects to control for variability between clinics. Random effects for subject were included when applicable (1), and random treatment effects by clinic were included when significant (2).

Health Outcomes

Exacerbations

As shown in Table 3, during the study period, 581 (21.6%) patients experienced at least one asthma exacerbation, with a total of 1063 exacerbations; 356 experienced 1 exacerbation, 119 experienced 2, 52 experienced 3, and the remaining 54 experienced 4 or more exacerbations. The rate of exacerbations did not change between pre- and post-intervention, nor did the proportion with each of the four exacerbation qualifying events in our study definition of exacerbation.

Table 3:

Outcome Measures

	Pre-Intervention Period	Post-Intervention Period	p-value
Exacerbation events
Total Exacerbations	386	414
N (%) of patients with no exacerbations	2410 (89.7%)	2389 (88.9%)	0.31
Median (IQR) (min, max) exacerbations per person among those with 1 or more exacerbations	1 (1,1)(1,9)	1 (1,1)(1,8)	1.01
Percent of exacerbations with a qualifying event of type:*
Type 1: Single ED or IP visit with primary diagnosis of asthma	112/386 (29.0%)	109/414 (26.3%)	0.31
Type 2: Cluster of 2 visits with 1–14 days between, both with an asthma diagnosis	85/ 386 (22.0%)	89/414 (21.5%)	0.51
Type 3: ICD code (5th digit) or string indicating exacerbation	220/386 (57.0%)	245/414 (59.2%)	0.31
Type 4: Systemic corticosteroid within 0–3 days after any type of visits for asthma	265/386 (68.7%)	275/414 (66.4%)	0.41
Healthcare utilization
1. IP visits with primary asthma diagnosis (number of visits)	5	1	-
2. ED visits with primary asthma diagnosis (number of visits)	131	121	-
3. Outpatient visits with an asthma diagnosis (number of visits)	1888	1910	-
4. Asthma-specific medication use (n of prescriptions)**	5,221	7,116	-
4a. Controller use	2,224	2,987
4b. Reliever use	2,997	4,129
Healthcare utilization--number of patients with at least 1 of the following:
1. IP visit with primary asthma diagnosis	5/2687 (0.2%)	1/2687 (0.0%)	0.1
2. ED visit with primary asthma diagnosis	83/2687 (3.1%)	92/2687 (3.4%)	0.4
3. Outpatient visits with an asthma diagnosis	609/2687 (22.7%)	649/2687 (24.2%)	0.1
4. Asthma-specific medication use**	1094/2687 (40.7%)	1334/2687 (49.6%)	<0.0001
4a. Controller use	468/2687 (17.4%)	585/2687 (21.8%)	<0.0001
4b. Reliever use	1005/2687 (37.4%)	1240/2687 (46.2%)	<0.0001
Composite scores: ACT
Number of observations	0	543	-
ACT score = “in control” [n (% of all ACTs)]	n/a	325 (60.2)	-
ACT score = “poorly controlled” [n (%)]	n/a	132 (24.4)	-
ACT score = “very poorly controlled” [n (%)]	n/a	83 (15.4)	-
Pulmonary physiology: FEV1/FVC percent predicted
Number of observations	13	456	-
Mean value (percent predicted, (standard deviation))	82.5 (11.1)	77.9 (13.1)	0.11

^*Percentages add up to greater than 100% as an exacerbation may have more than one qualifying event.

^**This includes controllers and relievers but not steroids.

Of the four qualifying events in the study definition of an exacerbation, the most common was a Type 4 event (systemic corticosteroid medication, 728 events), although we found that many exacerbations were characterized by more than one event type (Figure 3). For example, of the 728 exacerbations that included a Type 4 event, 433 included at least one other qualifying event type. None of the event types changed in frequency between the pre- and post-intervention periods (Table 3).

Figure 3:

Distribution of exacerbations by qualifying event.

Type 1: Single ED or IP visit with primary diagnosis of asthma

Type 2: Cluster of 2 visits with asthma diagnosis <14 days apart

Type 3: ICD code or string indicating exacerbation

Type 4: Systemic corticosteroid within 0–3 days after any type of asthma visit

Note: exacerbations occurring within 14 days of each other were grouped into a single exacerbation

Utilization Outcomes

As shown in Table 3, for the utilization outcomes, the number of ED, IP and outpatient visits for asthma were similar pre- and post-intervention. The number of asthma-specific medication events (prescriptions, dispensing events) rose by 36.3% from 5,221 to 7,116. This increase occurred for both controller medications (34.3% rise) and relievers (37.7% rise).

We could not analyze composite asthma score values as an outcome as there were no ACTs measured in the pre-intervention period. This was expected, as the ACT was part of the intervention. For our pulmonary physiology outcome, we found no change in the FEV1/FVC percent predicted values pre- to post-intervention.

Discussion

In this pre-post analysis of the CATP in one network of federally qualified health centers, we found improvement in some process measures of asthma care associated with the intervention: increased use of spirometry and administration of the ACT measure. At the same time, no change occurred in the two HEDIS asthma process measures: asthma medication ratio and medication management for people with asthma. We also found no change in asthma outcomes measured across multiple domains: exacerbations, utilization, symptom scores and pulmonary physiology measures.

These findings are similar to those of prior studies of asthma care processes associated with CATP implementation. One prior study of the CATP found that practices reported they increased their use of inhaled corticosteroids, increased their use of asthma action plans, and initiated or increased their use of spirometry.²⁴ An analysis of the MCPN CATP intervention using only EHR data found increased use of spirometry, severity assessments, asthma action plans and prescription of inhaled corticosteroids.³² Although we could not measure asthma action plan use in our current study, we confirmed using SAFTINet data that both spirometry and ACT measurement improved. Unlike these two prior CATP studies, the current study also assessed asthma outcomes but found no change.

A plethora of evidence using the RE-AIM Framework has shown that consistent implementation of an intervention (e.g., CATP process of care) is prerequisite for realizing effectiveness in real-world care settings.^59,60 The lack of effects on outcomes in this study may be attributable to the limited changes in guideline-adherent process of care. While spirometry and ACT use increased, simply measuring outcomes was not enough to change outcomes. Further attention to taking action upon observing poor control is needed. The finding that measuring symptoms is not sufficient to make an impact reflects findings from other domains of primary care – for instance, implementation of the PHQ-9, a recommended tool for measuring symptoms of depression, does not in and of itself improve depression outcomes.⁶¹

Among our secondary outcomes, the only change was a rise in the number of asthma controller and reliever medications prescribed or fulfilled. This increase also explained the rise in the number of patients meeting HEDIS measure denominator criteria (persistant asthma). This is consistent with the rise in controller medication administration reported in the two prior CATP studies.^24,32

Targeting specific provider behaviors reflecting change in processes of care is important because practice change is an essential step toward changing outcomes. Nonetheless, as reflected in these findings, changing provider behaviors is no guarantee of change in outcomes, which are influenced by a host of variables including patient-level adherence to treatment⁶² and the unique characteristics of the patient population served by the practice and the characteristics and context of the practices themselves.⁶³ Further improvements in provider behaviors may yield greater effectiveness at the patient-level. A survey of 784 primary care providers concluded that continuing medical education programs addressing respiratory disease management had not adequately reached primary care practices, and that clinicians were less interested in attending distant sessions with opinion leaders, preferring live presentations, information they could access from their office, and content relevant to their daily practice.⁶⁴ Tailored approaches that engage providers and include multiple opportunities for interaction and feedback are markedly more likely to change practice behavior.⁶⁵ These training guidelines have been adopted in the CATP.

Still, additional research is needed to further improve CATP training and with it to improve patient outcomes. Results from this study of processes of change can help inform such refinements to the CATP, continuing to improve care delivery with the objective of decreasing asthma symptoms and healthcare utilization. Informal feedback from providers participating in CATP have indicated a desire for more follow up training to include spirometry interpretation and decision support through case studies. Future directions include exploration of which elements of the provider-level CATP implementation are most effective in improving process measures, and in collecting process and outcome measures more specifically related to the components of the CATP intervention. For example, if future training involves more emphasis on spirometry interpretation and decision support, future process and outcome measures could quantify provider concordance with recommended responses to patient-level spirometry values, and the associated asthma symptom burden or exacerbation in the weeks following each spirometry measurement.

The current study also describes the distribution of asthma exacerbation events operationalized using hybrid (claims-and-EHR) data. Although prior studies have used the Asthma Outcomes standards for exacerbations,^49,66 we found no prior studies detailing the distribution of exacerbations defined using observational data by type of qualifying event. The fact that 295 exacerbations were characterized by systemic corticosteroid prescribing or prescription-filling alone suggests that these may represent a different, milder form of exacerbation than those exacerbations also characterized by asthma-related ED and/or inpatient events.

Several types of bias are possible given the pre-post, non-experimental design, including selection bias and confounding. Selection bias (non-random selection into the study population) could be introduced by improved asthma detection—perhaps of more with milder asthma severity—following the intervention. As we included the same study population in both pre- and post-intervention study populations, this bias is unlikely to impact our findings. Confounding may have been introduced by a secular trend—e.g., if the pre- or post-intervention periods differed in the severity of asthma triggers in the regional environment, such as seasonal allergens or circulating respiratory viruses. Given the lack of a control group, any number of other influences on asthma processes of care may have led to the effects observed in this study. Subsequent research should include pragmatic trial and quasi-experimental designs to test both the implementation and the effectiveness of CATP in diverse clinical settings.⁶⁷

In conclusion, although we found improvement in some process measures of asthma care in this pre-post analysis of the CATP in one network of practices, we found no change in asthma outcomes measured across multiple domains: exacerbations, utilization, symptom scores and pulmonary physiology measures. Future directions may include a randomized trial of the toolkit in another setting, and subgroup analysis of findings based on exacerbation qualifying events.