Abstract
PURPOSE We wanted to examine whether a collaborative to improve asthma care influences process and outcomes of care in asthmatic adults.
METHOD We undertook a preintervention-postintervention evaluation of 185 patients in 6 intervention clinics and 3 matched control sites that participated in the Institute for Healthcare Improvement Breakthrough Series (BTS) Collaborative for asthma care. The intervention consisted of 3, 2-day educational sessions for teams dispatched by participating sites, which were followed by 3 action periods during the course of a year.
RESULTS Overall process of asthma care improved significantly in the intervention compared with the control group (change of 10% vs 1%, P = .003). Patients in the intervention group were more likely to attend educational sessions (20% vs 5%, P = .03). Having a written action plan, setting goals, monitoring peak flow rates, and using long-term asthma medications increased between 2% and 19% (not significant), but asthma-related knowledge was unchanged for the 2 groups. Patients in the BTS Collaborative were significantly more likely to be satisfied with clinician and lay educator communication (62% vs 39%, P = .02). Health-related quality of life, asthma-specific quality of life, number of bed days caused by asthma-related illness, and acute care service use were not significantly different between the 2 groups.
CONCLUSIONS The intervention was associated with improved process-of-care measures that have been linked with better outcomes. Patients benefited through increased satisfaction with communication. Follow-up of patients who participated in the intervention may have been too brief to be able to detect significant improvement in health-related outcomes.
Keywords: Asthma, evaluation studies, chronic disease, breakthrough series collaborative, chronic care model, process of care, outcomes and process assessment (health care)
INTRODUCTION
Asthma is a widespread chronic disease. The National Center for Health Statistics estimates the prevalence of asthma in the United States to be 75 in 1,000 for non-Hispanic white adults, 86 in 1,000 for non-Hispanic blacks, and 59 in 1,000 for Hispanics.1 Rates are higher for females (83 in 1,000) than for males (64 in 1,000). Moreover, asthma affects the quality of life of afflicted patients.2–4
In 1998, to foster improvement in the care of patients with chronic illnesses, the Robert Wood Johnson Foundation initiated a national program called Improving Chronic Illness Care (ICIC), which is based in Seattle, Wash, at the MacColl Institute for Healthcare Innovation at Group Health Cooperative. Researchers involved in this program had developed the Chronic Care Model based on their experience and the clinical literature5,6 (http://www.improvingchroniccare.org).
In 1998, ICIC joined with the Institute for Healthcare Improvement to offer a Breakthrough Series (BTS)7 Collaborative, using the Chronic Care Model as its foundation, the BTS Collaborative brought together organizations hoping to improve care for their asthma patients. Institute for Healthcare Improvement BTS Collaboratives promote a method for inducing and sustaining organizational change based on an explicit agreement on improvement goals, monitoring ongoing progress through specific measures, developing change ideas for the participating teams based on best practices, and the plan-do-study-act approach as a framework for monitoring and implementing continuous quality improvement.
This study represents the first evaluation of a BTS Collaborative for asthma. Its purpose is to evaluate whether the BTS Collaborative improved processes and outcomes of care for adult patients receiving asthma care at sites participating in the collaborative. Cretin et al give more details on the Chronic Care Model, the ICIC collaboratives, and our evaluation.8
METHODS
The BTS Collaborative for asthma began on February 1, 2000, and ended on March 1, 2001. Twenty-six health care organizations volunteered to participate in the collaborative and paid a fee to attend. All asthma organizations were invited to participate in the RAND evaluation before the start of the collaborative. Six of these organizations agreed to participate in the RAND evaluation discussed here. These 6 organizations include rural and urban sites in the Northwest, East, and Puerto Rico. Three of the 6 organizations provided a control site for comparison. The control sites were always at locations physically separate from the intervention sites. The sites included rural and urban sites in the Northwest, West, and East. Four of the 6 BTS intervention sites (BTS sites) and 2 of the 3 control sites were Bureau of Primary Healthcare community health care centers. These clinics primarily serve patients of low socioeconomic status without access to private sector health care services. The remaining sites primarily served privately insured patients. Within each site all eligible patients were chosen for participation in the RAND evaluation.
Each intervention site sent a team to 3, 2-day learning sessions taught by BTS faculty. Most teams had 3 or 4 members, though a few were larger. The learning sessions were followed by 2- to 6-month action periods. We evaluated the influence of the collaborative on process of care and outcomes for the patients based on 2 data sources: a patient survey and medical records.
Survey Data Collection
Participating clinical organizations created an asthma patient registry based on medical record review. To be entered on a clinic’s asthma registry, patients had to have an encounter for asthma care during the 12 months prior to when the organization began participating in the collaborative. The internal control sites selected by the organizations participating in the RAND study also created asthma patient registries. All sites (both control and intervention) participating in the RAND study attempted to obtain consent from all patients on their asthma registries to release the contact information and diagnosis to the RAND survey research group. The RAND survey research group telephoned each consenting patient, confirmed that the patient had asthma, and conducted a 30-minute interview about the patient’s asthma. The telephone interviews began shortly after the conclusion of the BTS Collaborative on March 13, 2001, and were completed on December 31, 2001. The survey questions were offered in English, Spanish, and Mandarin. The English version of the survey instrument for this evaluation study is available at: http://www.rand.org/health/ICICE. The research effort was approved by local institutional review board (IRB) for each participating site as well as the RAND IRB.
Medical Records Data Collection and Scoring
In each telephone interview, eligible respondents—even those who refused to participate in the survey—were asked whether they would allow their medical record to be examined for this study. Upon gaining permission, a RAND-trained abstractor abstracted the patient’s outpatient medical record on site. The abstractors were not told whether the clinic was an intervention or control site. We examined care received during the 12 months before the BTS intervention and care received between the 4th and the 15th month after the BTS intervention started (March 1, 1999, to June 30, 2001). The 3-month delay was chosen in advance to allow the sites time to begin making changes.8
The computer-based abstraction tool was developed for this study by a team that included experts in asthma care, in computer-based medical record abstraction tool development, and in quality-of-care measurement. This abstraction tool includes 11 quality indicators for asthma care, 9 of which had adequate variation for meaningful analysis. The indicators were based on the 1997 National Heart Lung Blood Institute guideline for asthma care9 and the RAND Quality Assessment Tool.10 For example, each participant’s medical record was scored on the following quality indicator: “All patients should have a written action plan in the medical record that is based on changes in symptoms or peak flow measurements.” All participants would be eligible to receive a score of pass (1) or fail (0) on this indicator. A summary process-of-care score for each patient can also be assessed as the ratio of the number of indicators for which the patient is eligible (denominator) to the number of times they received the indicated care (numerator).
Measurement
Survey-Based Variables of Process of Care
We measured 4 aspects of patient self-management: peak flow monitoring (“Do you check your peak flows at home?”), goal setting (“Did you work with one of your asthma doctors or nurses to set personal goals for your asthma treatment?”), presence of a written action plan (“Have your asthma doctors or nurses worked with you to develop an asthma action plan so that you know how to take care of your asthma?” “Do you have a copy of this plan in writing?”), and whether the patient had attended asthma education sessions (“In the past 6 months, did you attend any group classes to learn more about asthma?”).
Asthma knowledge was assessed by 10 questions adapted from the National Asthma Education Program asthma knowledge questionnaire.9 The knowledge variable counts the number of right answers to the 10 knowledge questions.
Patients were asked at the time of the telephone interview whether they were currently using a long-term asthma-control medication, such as cromolyn or inhaled steroids. If they responded yes, they were asked the name of the inhaler. If they did not know the name, they were asked the color of the inhaler canister, and that color was matched by the interviewer to a predetermined list of canister colors of long-term asthma inhaler medications. If the patient named a long-term asthma inhaler medication or named a canister color that matched the color of one of these inhalers, the patient was coded as currently using a long-term asthma-control medication.
Survey-Based Outcome Variables
We used the physical health summary scale from the SF-12 Health Survey11 to assess generic health-related quality of life (HRQoL) and an adapted version of the Marks’ Asthma Quality of Life questionnaire12 to assess asthma-specific HRQoL. The Marks quality-of-life scale ranges from 0 to 10. Higher numbers refer to a greater impact of asthma on one’s life (ie, higher scores indicate worse quality of life).
To assess satisfaction with communication in care, we adapted 4 items from the provider communication composite of the Consumer Assessment of Health Plans asking about satisfaction with various aspects of communication in the previous 6 months, and we calculated an overall score as the mean of the 4 items (score range: 0 to 3, higher scores indicate greater satisfaction).13
Acute care service use was measured by summing the total number of visits to an emergency department for asthma and the total number of admissions to the hospital lasting 1 or more night(s) during the previous 6 months. We used the term acute to distinguish this care of service use from planned ambulatory care follow-up visits, which theoretically should prevent acute care service use.
We measured lost productivity as the number of days the patient’s health kept him or her in bed in the previous month.
Covariates
All models that relate the intervention to key outcomes were adjusted for asthma severity, the number of comorbidities, education, annual income, insurance type, race or ethnicity, age, and sex.
We controlled for 3 levels of asthma severity: mild intermittent asthma, mild persistent asthma, and moderate or severe persistent asthma. Asthma severity categories were constructed based on guidelines from the National Heart Lung Blood Institute.9
Patients were categorized as having mild intermittent asthma if they experienced asthma symptoms between 0 and 2 days per week and if they experienced asthma symptoms between 0 and 2 nights per month. Patients were categorized as having mild persistent asthma if they experienced asthma symptoms between 3 to 6 days per week or if they experienced asthma symptoms between 3 to 4 nights per month. Patients with daily asthma symptoms or with asthma symptoms on 5 nights per month or more were categorized as having moderate to severe persistent asthma.9 We asked the patients whether they had any of the following comorbidities: hypertension, diabetes, chronic lung disease other than asthma, or peptic ulcer disease, or whether they ever had a heart attack, heart failure, or depression.
Survey Data: Imputations
To retain observations with 1 or 2 missing values, we imputed those values by grouping observations into up to 16 different categories based on demographic variables. Within each stratum, missing values were imputed using the hotdeck procedure.14 For indicator variables (and categorical variables more generally), the hotdeck procedure has the advantage that the imputed values are constrained to be the original response choices “yes” and “no.” The following variables had imputed values (number in parentheses): peak flow monitoring (1), goal setting (4), appropriateness of care (1), education (1), income (5).
Analysis
Performance of Participating vs Nonparticipating Sites
To find out whether sites participating in both the BTS Collaborative and the evaluation performed better than sites participating in only the collaborative, we analyzed site ratings by the faculty. Site ratings ranged from 1 to 5 with 5 being best. We use a t test to compare the sites that participated with those that did not.
Surveys
We used 2-level hierarchical regression models with patients nested within sites. We used normal (Gaussian) regression for generic and asthma-specific HRQoL. We used logistic regression for satisfaction with communication. We used Poisson regressions to model the number of bed days and acute care service use. For improved interpretability, we report the effect of the intervention on the original scale: we give multivariate adjusted proportions, days, or scores. The multivariate adjusted proportions and counts were computed as the average predictions from the regression models of 2 slightly altered data sets: for one data set it was assumed that nobody received the intervention; for the other, that everyone received the intervention.15
Medical Records
To determine whether the BTS intervention improved processes of care, we compared changes in the processes of care between intervention and control groups (a differences-in-differences analysis). The preintervention period consisted of the 12 months before the intervention began (March 1, 1999, to March 1, 2000). The postintervention 12-month period was deliberately delayed for 3 months (July 1, 2000, to June 30, 2001) to ensure that participating organizations had time to implement patient registries and other recommended changes. For all patients preintervention and postintervention scores were calculated for each of 9 quality indicators and for the overall process-of-care score.
The difference between before and after scores for both the intervention and control groups were calculated. Because quality improvements are naturally interpreted as differences rather than as ratios (a change from 10% to 40% in beneficial process is preferred to a change from 1% to 4%, even though both changes represent the same ratio or odds), we used a linear probability model in the analysis. Adjusted differences between the intervention and control groups and the adjusted P values are reported for each individual indicator and for the summary process-of-care scores.
RESULTS
When we compared the sites participating in both the collaborative and the evaluation with the sites participating in only the collaborative, we found that participating sites had an average faculty rating of 3.81 and the nonparticipating sites had an average faculty rating of 3.92. The average difference in faculty ratings, −0.11, is not statistically significant (P = .58) and is not large enough to be of practical relevance. There is no reason to believe that participating sites self-selected into the evaluation because they were high performers.
Changes Implemented
To assess changes made in the collaborative sites, we analyzed summary progress reports submitted in the 10th month of the collaborative year and the 2 monthly reports afterward. The analysis method was reported elsewhere.16 On average, the sites made 36.2 changes to their systems (ranging from 27 to 51) during the course of the collaborative. All sites worked on guideline implementation, clinician and lay educator education, building a patient registry, use of information for care management, acquiring leadership support, and seeking or linking community resources. The areas with little or no implementation were a culture of practice team, availability of clinical guidelines to patients, a system of expert consultation support, and provision of financial or organizational motivation to increase clinician and lay educator participation in quality improvement. During the collaborative period, 2 control sites also had active quality improvement programs or committees that addressed all chronic diseases, including asthma. The remaining control sites were not involved in any organized quality improvement activities. Some individual physicians implemented small changes independent of any official quality improvement activity.
Participation and Demographic Characteristics
Twenty-six health care organizations accepted an open invitation to participate in the BTS Collaborative. Six organizations providing asthma care for adults and children agreed to provide a site for the adult evaluation, and 3 of these agreed to provide a control site. Survey data were obtained for 123 patients in 6 intervention sites and for 62 patients in 3 control sites. Medical record data were obtained for 109 patients in 3 intervention sites and for 76 patients in 2 control sites.
Demographic information by intervention or control group based on the survey is shown in Table 1 ▶. The response rate was 76% (using the American Association for Public Opinion Research definition RR4).17 Of those surveyed, 75 patients from the intervention group (61%) and 54 patients from the control group (92%) also agreed to have their medical record abstracted, as did an additional 34 patients from the intervention group and 22 patients from the control group who refused to participate in the survey interview. We therefore had medical records data for 109 patients in the intervention group and 76 patients in the control group. There were 3 sites (2 intervention and 1 control) that contributed survey data but not medical records (n = 32).
Table 1.
Demographic Characteristics of the Survey Sample
| Variable | Intervention n = 123 | Control n = 62 | Overall n = 185 | PValue |
| Age, y, mean (SD) | 42.3 (17.4) | 43.9 (15.7) | 42.8 (16.8) | .53 |
| Male, % | 17.9 | 24.2 | 20 | .31 |
| Respondent’s education, % | – | – | – | .55 |
| < High school | 29.3 | 23.0 | 27.2 | – |
| High school | 29.3 | 27.9 | 28.8 | – |
| > High school | 41.5 | 49.2 | 44.0 | – |
| Household income ≥ $30,000, % | 26.8 | 22.6 | 28.9 | .53 |
| Race or ethnicity, % | – | – | – | .32 |
| White (non-Hispanic) | 65.9 | 75.8 | 69.2 | – |
| Hispanic | 17.1 | 9.7 | 14.6 | – |
| Black (non-Hispanic) | 5.7 | 8.1 | 6.5 | – |
| Other + multiracial + Asian | 11.4 | 6.5 | 9.7 | – |
| Comorbidities, No., mean (SD) | 1.32 (1.23) | 2.08 (1.53) | 1.57 (1.38) | .007 |
| Asthma severity, % | – | – | – | .11 |
| Intermittent | 33.9 | 43.9 | 40.5 | – |
| Persistent (mild) | 19.4 | 25.2 | 23.2 | – |
| Persistent (moderate or severe) | 46.8 | 30.9 | 36.2 | – |
| No insurance, % | 9.8 | 8.1 | 9.2 | .70 |
Note: P values stem from χ2 tests for categorical variables and from t tests for age. To avoid nearly empty or empty cells for the χ2 tests, cells with 4 or more comorbidities were combined.
Table 1 ▶ shows that patients in the control clinics had significantly more comorbidities than those in intervention clinics (2.1 vs 1.3, P = .007). The number of comorbidities ranged from 0 (43 patients, 23%) to 7 (2 patients, 1%). All other variables for the intervention and control groups in Table 1 ▶ are not significantly different.
Medical Records Results: Association Between the BTS Collaborative and Processes of Care
Table 2 ▶ shows the postintervention scores (percentage of patients passing each indicator) and the change from the initial (preintervention) score. The control sites showed little or no improvement between the preintervention and postintervention periods. The largest improvement among control sites was a 7.1% increase in the percentage of patients being instructed in how to use metered-dose inhalers among patients who received inhaled medications. For 4 indicators the performance in the intervention clinics was significantly higher than at the control clinics. The 4 improved indicators all reflect patient self-management. The overall score at intervention sites also improved significantly more (8.8%, P = .003) than the overall score at control sites. The intraclass cluster correlation for the overall score was 0.21. Other intraclass cluster correlations were as low as 0.02.
Table 2.
Adjusted Difference of Differences for the Intervention and Control Groups for Each Quality Indicator Based on Medical Record Abstractions
| Intervention Group (n = 101) | Control Group (n = 64) | |||||
| Quality Indicator | Final Passing (%) | Change From Initial (%) | Final Passing (%) | Change From Initial (%) | Adjusted Differences of Differences | AdjustedPValue for Differences of Differences |
| 1. All patients should have a β2-agonist prescribed for symptomatic relief | 83 | 2 | 89 | 6 | −4 | .68 |
| 2. Peak expiratory flow rate (or spirometry) should be measured in all patients at least annually | 28 | 21 | 14 | 0 | 17 | .03* |
| 3. No β-blocker should be prescribed for patients with diagnosed asthma | 92 | −2 | 92 | −2 | 0 | .94 |
| 4. All patients should have a written action plan in the medical record based on changes in symptoms or peak flow measurements | 27 | 26 | 0 | 0 | 26 | <.0001† |
| 5. Patients with asthma should have at least 2 routine planned follow-up visits for asthma annually | 77 | −8 | 91 | 2 | −7 | .41 |
| 6. Patients should be educated by physician in self-management of asthma | 37 | 15 | 10 | −3 | 15 | .07 |
| 7. Patients prescribed inhaled medications should be instructed in use of metered- dose inhalers | 22 | 15 | 7 | 7 | 13 | .04 |
| 8. Evidence of collaborative goal setting between patient and clinician and lay educator should be recorded at least annually | 7 | 7 | 0 | 0 | 7 | .03* |
| 9. Overall asthma process of care summary score | 46 | 10 | 38 | 1 | 8 | .003† |
* Significant at P <.05.
† Significant at P <.01.
Survey Results: Association Between the Intervention on the BTS Collaborative and Processes of Care
Table 3 ▶ shows adjusted and unadjusted survey-based measures of process of care for both the control and intervention groups. The percentages shown are adjusted for covariates as described earlier. Patients in the intervention group were more likely than patients in the control group to attend educational sessions (20% vs 5%, respectively, P =.03). Having a written action plan, goal setting, peak flow monitoring, and use of long-term asthma control medications also increased in the intervention group (between 2% and 19%), but the differences were not statistically significant. Asthma-related knowledge was unchanged for the 2 groups.
Table 3.
Adjusted and Unadjusted Survey-Based Process-of-Care Measures for Patients in Control and Intervention Groups
| Unadjusted | Adjusted* | ||||||
| Variables | Control (n = 62) | Intervention (n = 123) | Difference | Control (n = 62) | Intervention (n = 123) | Difference | PValue |
| Patient self-management, % | |||||||
| Peak flow monitoring | 38 | 60 | 23 | 44 | 57 | 13 | .21 |
| Goal setting | 40 | 53 | 13 | 47 | 50 | 3 | .74 |
| Written action plan | 24 | 43 | 19 | 25 | 44 | 19 | .058 |
| Education sessions attended | 2 | 11 | 10 | 5 | 20 | 15 | .028† |
| Knowledge (0–10 scale) | 7.7 | 7.5 | −0.2 | 7.6 | 7.6 | 0.0 | .93 |
| Taking long-term medication (yes/no), % | 66 | 70 | 4 | 67 | 69 | 2 | .85 |
* Adjustments are based on multiple regressions adjusted for race/ethnicity, education, sex, income, severity of asthma, and number of comorbidities.
† Significant at P <.05.
Survey Results: Association Between the BTS Collaborative and Patient-Level Outcomes
A comparison between various outcomes measures of patients in the control groups and patients in the BTS Collaborative intervention is summarized in Table 4 ▶. Patients receiving care in the BTS sites were significantly more likely to be satisfied with clinician and lay educator communication (62% vs 39%, P = .02). Satisfaction with clinician and lay educator communication was significantly lower for the Asian, Pacific Islander, multiracial group (not shown).
Table 4.
Comparison Between Various Survey-Based Outcomes Measures of Patients in Control Groups and Patients in Breakthrough Series Collaborative Intervention
| Outcome | Control (n = 123) | Intervention (n = 62) | Difference | PValue |
| General quality of life | 39 | 40 | 2 | .29 |
| Asthma-specific quality of life* | 4 | 4 | 0 | .73 |
| Satisfaction with clinician and lay educator communication | 39 | 62 | 23 | .02† |
| Number of emergency department visits and hospital admissions | 1 | 2 | 1 | .08 |
| Number of days in bed due to asthma-related illness | 2 | 2 | 0 | .77 |
* Scale between 0 and 10 where 0 is best (no impact) and 10 is worst (large impact).
† Significant at P <.05.
The use of acute care services was higher for patients receiving the intervention (1.72 vs 0.92 average visits); however, that difference did not reach statistical significance (P = .08). Patients with more comorbid conditions used significantly more acute care services.
Receiving care in a BTS clinic was not significantly associated with HRQoL, asthma-specific HRQoL, or the number of bed days resulting from asthma-related illness. HRQoL was significantly worse for older patients and for those with mild, moderate, or severe persistent asthma. High income was associated with significantly higher HRQoL. More comorbid conditions and having mild, moderate, or severe persistent asthma were associated with worse asthma-specific HRQoL. Moderate to severe persistent asthma and more comorbid conditions were associated with a significantly more bed days. Patients with higher income and patients with less education had significantly fewer bed days.
DISCUSSION
In a collaborative intervention, small teams from health organizations are encouraged to transform their larger organizations by implementing ideas from a menu of possible changes. In the real-world setting of our evaluation, these small teams must contend with daily on-the-job crises and organizational inertia. A recent study of quality improvement collaboratives across many sites for patients infected with human immunodeficiency virus showed a statistically insignificant improvement in process of care.18 An accompanying editorial noted that there was general enthusiasm for Breakthrough Series initiatives, but there was scarce evidence in the literature to support that enthusiasm.19
Nevertheless, this evaluation study indicates that participants in the BTS Collaborative successfully improved process of care for several aspects of patient self-management. Evidence from the chart review of greater improvement in the participating sites was corroborated by patient reports on attending educational class(es) and self-management practices. Because of their previously established commitment to improved health outcomes in asthmatics, the faculty stressed these self-management strategies during the 3 BTS Collaborative learning sessions.
There is evidence all areas of process improvement—attending educational sessions, having a written action plan, goal setting, and peak flow monitoring—can improve clinical outcome measures. Ford found that patients randomized to an intervention consisting of 3 educational sessions significantly reduced the number of visits to an emergency department.20 For an educational intervention consisting of 6 monthly sessions, deOliveira et al found that patients who received the intervention had significantly fewer emergency department visits and significantly better quality of life.21 Although we found that the intervention increased attendance at educational sessions, the intervention did not translate into a better quality of life, possibly because we had no information about the actual number of educational sessions attended.
Gibson found that optimal self-management, when it included a written action plan, led to a significant reduction in hospitalizations for asthma-related illness.22 Our study does not find such a reduction, possibly because patients who manage their asthma better are more aware of their exact physical condition. Greater awareness may lead patients to perceive a greater need of acute care services. A recent study of quality of care in the United States found underuse to be more a problem than overuse.10 A written action plan was linked with improved health outcomes in the meta-analysis by Bodenheimer et al.23 When Toelle and Ram24 conducted a meta-analysis of the few randomized trials on the effect of a written action plan on clinical outcomes, however, they concluded that “available trials are too small and the results too inconsistent” to yield any definite conclusions on clinical outcomes. Finally, in an intervention that emphasized peak flow monitoring, Lahdensuo et al found that patients with the intervention had significantly fewer direct medical costs, fewer unscheduled outpatient visits, and fewer work days missed because of asthma-related illness, as well as a significantly better quality of life.25,26
Overall, our findings on process of care are similar to the findings obtained for the pediatric population in the same study.27 For the pediatric population more effects were significant because the sample sizes were larger.
The BTS Collaborative increased patient satisfaction with clinician and lay educator communication. Satisfaction is a more proximate outcome than HRQoL or asthma-specific HRQoL, and it was easier and quicker to affect than more distal outcomes, especially because the BTS faculty emphasized improved physician-patient communication through goal setting, having a written action plan, and peak flow monitoring. The literature supports the link between improved physician-patient communication and increased satisfaction with care.21,28–32
The BTS Collaborative did not have a significant effect on other health outcome measures in our study. The lack of significance does not appear to be due to insensitive measures, as these outcomes were consistently associated with worse asthma and comorbidities. The lack of measured influence of the BTS Collaborative on these outcomes might be explained, however, by the brief follow-up period in our study. Patients were interviewed during the 9 months after the collaborative had ended. Whereas processes of care had clearly improved, the influence on health-related outcomes might not have been able to be detected for several months after the improved processes were implemented. It is also possible that the control population had better HRQoL or asthma-specific HRQoL at baseline compared with the intervention population. If such were the case, improvements in outcomes for the intervention group would not be detected, as we were able to measure outcomes only during the postintervention phase of the collaborative.
Other studies also have reported lack of evidence that an asthma intervention has an effect on health outcomes. A recent meta-analysis related to the Chronic Care Model reviewed 27 articles about asthma. The study found evidence that the Chronic Care Model reduced emergency department use but did not find improved quality of life.33
The effect of the intervention on some survey-based measurements of process of care was large but not statistically significant. The lack of statistical significance was due to insufficient power. To detect an effect of the magnitude indicated in Table 3 ▶, we had 48% power for written action plan and 24% power for peak flow monitoring. With the exception of satisfaction with clinician and lay educator communication, the survey-based estimated effects on outcome measures were small or none. The study was not underpowered for the continuous quality-of-life measures: the study had 80% power to detect an improvement in HRQoL of 4.4 or an improvement in asthma-specific HRQoL of 1.0.
Although the survey did include a control group, the survey analysis is limited by the lack of preintervention measurements. This limitation is mitigated through the availability medical records with preintervention and postintervention data. These data confirm survey results and show that there is no secular trend. We were not able to obtain preintervention measurements for the survey; dealing with the local institutional review boards (IRBs) took on average 14 months, far longer than anticipated. By the time we had resolved all issues and obtained consent to contact patients, the intervention was already in place, thus preventing us from taking any preintervention measurements.8
This BTS Collaborative implementing the Chronic Care Model to improve asthma care had a positive effect on patient self-management practices that have previously been linked to improved health outcomes. The BTS Collaborative significantly improved satisfaction with clinician and lay educator communication. Studies that are large enough to measure gains in all health outcomes from quality improvement may have to be conducted in large health care systems.
Acknowledgments
We would like to acknowledge Catherine M. Cruz, who assisted in the preparation of the manuscript.
Conflicts of interest: none reported
Funding support: This research was supported by the Robert Wood Johnson Foundation, which funded this work through grants #034984 and #035678.
REFERENCES
- 1.Centers for Disease Control and Prevention. National Center for Health Statistics (NCHS). Available at: http://www.cdc.gov/nchs/products/pubs/pubd/hestats/asthma/asthma.htm. Accessed June 17, 2004.
- 2.U.S. Department of Health and Human Services, Public Health Services. Action against asthma: a strategic plan for the Department of Health and Human Services. Washington, DC 2000. Available at: http://aspe.hhs.gov/sp/asthma/.
- 3.Institute of Medicine. Clearing the Air: Asthma and Indoor Air Exposures. Washington, DC: National Academy Press; 2000. [PubMed]
- 4.Lara M, Nicholas W, Morton S, Vaiana ME, Genovese B, Rachelefsky G. Improving Childhood Asthma Outcomes in the United States. Santa Monica, Calif: RAND; 2001. [DOI] [PubMed]
- 5.Wagner EH, Austin BT, VonKorff M. Organizing care for patients with chronic illness. Milbank Q. 1996;74:511–544. [PubMed] [Google Scholar]
- 6.Wagner EH, Austin BT, Davis C, Hindmarcsh M, Schaefer J, Bonomi A. Improving chronic illness care: translating evidence into action. Health Aff (Millwood). 2001;20:64–78. [DOI] [PubMed] [Google Scholar]
- 7.Kilo CM. Improving care through collaboration. Pediatrics. 1999; 103(Suppl):384–393. [PubMed] [Google Scholar]
- 8.Cretin S, Shortell SM, Keeler EB. An evaluation of collaborative interventions to improve chronic illness care: framework and study design. Evaluation Rev. 2004;28;28–51. [DOI] [PubMed] [Google Scholar]
- 9.National Asthma Education Program, Office of Prevention, Education, and Control. National Heart, Lung, and Blood Institute; 1991. August. Publication No. 91–3042. Bethesda, MD: National Institutes of Health.
- 10.McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348:2635–2645. [DOI] [PubMed] [Google Scholar]
- 11.Ware JE, Kosinski MA, Keller SD. How to Score the SF-12 Physical and Mental Health Summary Scales. 2nd ed. Boston, Mass: The Health Institute, New England Medical Center; 1995.
- 12.Marks GB, Dunn SM, Woolcock AJ. A scale for the measurement of quality of life in adults with asthma. J Clin Epidemiol. 1992;45:461–472. [DOI] [PubMed] [Google Scholar]
- 13.U.S. Department of Health and Human Services/Public Health Services. Agency for HealthCare Research and Quality. CAHPS 2.0 Survey and Reporting Kit. AHRQ Publication Number 99-0039, Oct 1999.
- 14.Brick JM, Kalton G. Handling missing data in survey research. Stat Methods Med Res. 1996;5:215–238. [DOI] [PubMed] [Google Scholar]
- 15.Graubard BI, Korn EL. Predictive margins with survey data. Biometrics. 1999;55:652–659. [DOI] [PubMed] [Google Scholar]
- 16.Pearson ML, Wu S, Schaefer J, et al. Assessing the Implementation of the Chronic Care Model in Quality Improvement Collaboratives. Health Serv Res. In press. [DOI] [PMC free article] [PubMed]
- 17.The American Association for Public Opinion Research. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 3rd ed. Lenexa, Kan: AAPOR; 2004. Available at http://www.aapor.org/pdfs/standarddefs2004.pdf.
- 18.Landon BE, Wilson IB, McInnes K, et al. Effects of a quality improvement collaborative on the outcome of care of patients with HIV infection: the EQHIV study. Ann Intern Med. 2004;140:887–896. [DOI] [PubMed] [Google Scholar]
- 19.Mittmann BS. Creating the evidence base for quality improvement collaboratives. Ann Intern Med. 2004;140:897–901. [DOI] [PubMed] [Google Scholar]
- 20.Ford ME, Havstad SL, Tilley BC, Bolton MB. Health outcomes among African American and Caucasian adults following a randomized trial of an asthma education program. Ethn Health. 1997;2:329–339. [DOI] [PubMed] [Google Scholar]
- 21.deOliveira MA, Faresin SM, Bruno VF, de Bittencourt AR, Fernandes ALG. Evaluation of an education programme for socially deprived asthma patients. Eur Respir J. 1999;14:908–914. [DOI] [PubMed] [Google Scholar]
- 22.Gibson PG, Coughlan J, Wilson AJ, et al. The effects of limited (information-only) asthma education on health outcomes of adults with asthma. Cochrane Database Syst Rev. 2002;(2):CD001005. [DOI] [PMC free article] [PubMed]
- 23.Bodenheimer T, Lorig K, Holman H, Grumbach K. Patient self-management of chronic disease in primary care. JAMA. 2002;288:2469–2475. [DOI] [PubMed] [Google Scholar]
- 24.Toelle BG, Ram FSF. Written individualised management plans for asthma in children and adults. Cochrane Database Syst Rev. 2004;(2): CD002171. [DOI] [PubMed]
- 25.Lahdensuo A, Haahtela T, Herrala J, et al. Randomised comparison of guided self management and traditional treatment of asthma over one year. BMJ. 1996;312:748–752. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Lahdensuo A, Haahtela T, Herrala J, et al. Randomised comparison of cost effectiveness of guided self management and traditional treatment of asthma in Finland. BMJ. 1998;316:1138–1139. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Mangione-Smith R, Schonlau M, Rosen M, Chan K, Louis T, Keeler E. Measuring the effectiveness of a collaborative for quality improvement in pediatric asthma care: does implementing the chronic care model improve processes and outcomes of care? Ambul Pediatr. 2005;5:75–82. [DOI] [PubMed] [Google Scholar]
- 28.Brody DS, Miller SM, Lerman CE, Smith DG, Lazaro CG, Blum MJ. The relationship between patients’ satisfaction with their physicians and perceptions about interventions they desired and received. Med Care. 1989;27:1027–1035. [DOI] [PubMed] [Google Scholar]
- 29.Froehlich GW, Welch HG. Meeting walk-in patients’ expectations for testing: effects on satisfaction. J Gen Intern Med. 1996;11:470–474. [DOI] [PubMed] [Google Scholar]
- 30.Robbins JA, Bertakis KD, Helms J, Azari R, Callahan EJ, Creten DA. The influence of physician practice behaviors on patient satisfaction. Fam Med. 1993;25:17–20. [PubMed] [Google Scholar]
- 31.Bertakis KD, Roter D, Putnam SM. The relationship of physician medical interview style to patient satisfaction. The J Fam Practice. 1991;32:175–181. [PubMed] [Google Scholar]
- 32.DiMatteo MR, Hays RD. The significance of patients’ perceptions of physician conduct: a study of patient satisfaction in a family practice center. J Commun Health. 1980;6:18–34. [DOI] [PubMed] [Google Scholar]
- 33.Tsai AC, Morton SC, Mangione CM, Keeler E. A meta-analysis of interventions to improve chronic illness care. Paper presented at Academy Health Annual Research Meeting, San Diego, Calif, June 2003.