Characteristics of Successful Asthma Programs

SYNOPSIS

Objective

We identified characteristics of interventions associated with positive asthma outcomes to understand how programs can be improved.

Methods

We identified asthma interventions from the peer-reviewed literature or through a nomination process for unpublished programs. Initially, we identified 532 interventions. Of those, 223 met our eligibility criteria (e.g., focus on asthma, completed an evaluation, and demonstrated at least one asthma-related health outcome) and provided information on program components and processes, administration, evaluation, and findings through telephone interviews, program documents, and published reports. We analyzed bivariate relationships between programmatic factors and outcomes using Chi-square statistics, Fisher's exact tests, and unconditional logistic regression. We confirmed findings for all programs by analyzing the subset with published results in peer-reviewed journals.

Results

Our findings indicated that programs were more likely to report a positive impact on health outcomes if they (1) were community based, (2) engaged the participation of community-based organizations, (3) provided program components in a clinical setting, (4) provided asthma training to health-care providers, (5) collaborated with other organizations and institutions and with government agencies, (6) designed a program for a specific racial/ethnic group, (7) tailored content or delivery based on individual health or educational needs, and (8) conducted environmental assessments and tailored interventions based on these assessments.

Conclusions

Positive asthma outcomes were associated with specific program characteristics: being community centered, clinically connected, and continuously collaborative. Program developers and implementers who build these characteristics into their interventions will be more likely to realize desired asthma outcomes.

The alarming prevalence of asthma worldwide^1–3 and the increasing burden of the disease, especially on low-income and minority populations,⁴ has in recent decades led to a proliferation of asthma-related clinical services and educational programs. For example, in the U.S. there are more than 200 coalitions of asthma-related organizations devoted to community-wide improvement of asthma policies, services, and programs. Virtually every clinical care institution, directly or indirectly through partners, has services and programs for people with asthma. These amount to thousands of efforts around the world.

A number of asthma interventions designed to enable people to manage the clinical, behavioral, and social aspects of their condition have been subjected to rigorous randomized, controlled trial (RCT) evaluation, and several models have been tested in replication studies.^5–18 Many more programs have been developed and evaluated locally, but results have not been published. These unpublished programs reach very large numbers of adults and children with asthma.

Experts have discussed elements of asthma interventions thought to enhance their efficacy,^19,20 and both the guidelines of the National Asthma Education and Prevention Program²¹ and Global Initiative on Asthma²² describe qualities of effective interventions. Nonetheless, the specific aspects of programs associated with accomplishment of desired asthma outcomes have not been explored empirically in a large-scale investigation. This study comprised a survey of evaluated asthma programs around the world and an analysis to identify the characteristics of those achieving success. We examined the types of strategies implemented and the programmatic characteristics most significantly associated with programs reaching targeted outcomes. The outcomes of interest in the study included health-care use, quality of life (QOL), school days absent, work loss, behavioral changes, and clinical status. To obtain an accurate reflection of the state-of-the-art in actual practice, we included both published and unpublished programs.

METHODS

The University of Michigan Health Sciences Institutional Review Board approved all study procedures.

Eligibility criteria

To be included in the study, a program had to (1) focus on asthma, (2) include an environmental component (e.g., education on trigger identification, control or avoidance strategies, or environmental policy interventions), or (3) have conducted an evaluation that showed a positive impact on at least one of the targeted health outcomes.

Data collection

Data collection occurred in two stages from September 2003 through July 2005. In the first stage, we identified published programs from the Computer Retrieval of Information on Scientific Projects (CRISP) database,²³ the Centers for Disease Control and Prevention's bibliography of potentially effective interventions,²⁴ Ovid MEDLINE®,²⁵ the Cochrane Central Register of Controlled Trials,²⁶ the Cochrane Database of Systematic Reviews,²⁷ and PubMed search terms (e.g., asthma, trigger, outcome, environment, indoor air, and intervention).

The second phase of data collection entailed preliminary nomination of programs by key informants. We undertook an extensive nomination process to identify local and community-based efforts that might have been less likely to publish their results. We elicited nominations from a wide range of individuals through announcements in e-mail listservs, organizational newsletters, targeted direct mailings, and outreach at various conferences and meetings. These included representatives of federal agencies, community coalitions and similar groups working in asthma, international organizations, school districts, health plans, large clinical settings, and professional organizations. We sent forms to 2,769 key informants, and we received 450 nominations.

To determine eligibility, we sent a profile form to 532 programs, 111 of which had been identified from the literature. Subsequently, 362 programs provided information, and we excluded 105 programs because they did not focus on asthma or they did not have an environmental component. Of the remaining programs, 233 had completed an outcome evaluation and met all other study inclusion criteria, and we collected data on all but 10 of the 233 eligible programs. We did not include these 10 eligible programs in the study because a representative could not be reached for interview, and no published data from the programs were available. The types of evaluations conducted varied. Among all programs (n=223), 35% used RCT designs, 7% used comparison groups without randomization or matching, 60% had pre/post or time-series designs, and 3% were case studies. The programs were located in 46 U.S. states and 30 countries.

Program description data

We collected complete descriptions of the eligible programs and their outcomes using a structured questionnaire. We based questionnaire items on findings in the literature on principles of effective disease prevention and health promotion programming,^28–32 and we developed them in collaboration with members of an expert panel. The interview questionnaire (available at http://cmcd.sph.umich.edu/asthma-health-outcomes-project-ahop.html) consisted of 117 primarily closed-ended items about program context, planning and design, implementation, evaluation (including health outcomes), and program sustainability. We asked respondents if the program measured and demonstrated a positive impact on any of the following health outcomes: hospitalizations, emergency department (ED) visits, urgent care visits, sick (unscheduled) office visits, well (scheduled) office visits, general health-care utilization, QOL for children, QOL for parents, QOL for adults, functional status, school absences, work loss, symptoms, lung function, medication use, self-management skills, use of asthma action plan, peak flow meter usage, change in clinical actions (e.g., asking a patient to demonstrate the use of metered-dose inhaler, or prescribing inhaled anti-inflammatory medicine), and other.

We considered both content and process factors, including, for example, the types of stakeholders involved in planning and implementation, the theoretical basis of the program, the types of activities implemented, the qualifications of staff, and the populations reached. Several open-ended items captured qualitative data regarding unintended impact, other factors contributing to program success, assessments of the programs' strengths, and barriers to implementation. Trained interviewers conducted telephone interviews with a representative from each program. Interviews were successfully completed with 223 programs and/or needed data extracted from reports published in the literature.

Data analysis

We analyzed bivariate relationships between programmatic factors and health outcomes with Chi-square statistics using Fisher's exact tests (p<0.05) in SAS® software.³³ Outcomes were dichotomized as “improvement” or “no improvement.” We calculated odds ratios (ORs) as estimates of the association between programmatic factors and health outcomes, and we obtained 95% confidence intervals (CIs) using unconditional logistic regression. To increase sample size to measure QOL outcomes, we constructed a variable that included any combination of QOL for children with asthma, parents/caregivers of children with asthma, or adults with asthma. In the few cases with zero cells in the two-by-two tables, we added a 0.5 continuity correction to each cell to enable estimation of the OR.³⁴

Subanalyses.

In preliminary analyses among all 223 programs, we noted that the programs with less rigorous evaluations (e.g., lack of a comparison or control group, lack of sample-size calculations, or use of less advanced statistical procedures) were more likely to report more positive results. To increase confidence in the overall findings, we repeated these analyses in the 111 programs published in peer-reviewed journals using the same methods described previously and compared those findings with all 223 programs.

Among the 111 peer-reviewed programs, 65 were reports of RCTs. We conducted bivariate analyses using the methods described separately with the RCT studies and compared the results with findings in the peer-reviewed and overall program analyses. The primary difference we noted in these subanalyses was the strength of significance (p-value). For only three items was significance evident in one subgroup and not the other. These analyses increased confidence that the overall analysis uncovered associations of importance, and results of the peer-reviewed programs are the ones reported in this article.

Qualitative data analysis.

Using an approach adapted from Patton et al.,³⁵ we coded, categorized, and summarized open-ended questionnaire items from all programs that were related to successes, barriers, and unintended impact. We used these data to aid interpretation of quantitative findings.

RESULTS

The Figure offers definitions of the program characteristics that were significantly associated with health outcomes. Table 1 presents results related to programmatic efforts to (1) be community-based and (2) engage participation of community-based organizations (CBOs), as they were associated with health-care use. Programs that collaborated with CBOs and involved them in planning were significantly more likely to see reductions in patients' health-care use for asthma than those that did not engender such participation. Lower patient health-care use (hospitalizations, ED visits, and in general) was also significantly associated with having an office located in the target community.

Figure.

Definitions of program characteristics associated with positive outcomes in the Asthma Health Outcomes Project

Table 1.

Association of health-care utilization with community-centered asthma programs in the Asthma Health Outcomes Project

^aN = number of programs that measured both variables

^bA general measure reported by key informants that may include any combination of hospitalization, ED visits, urgent care visits, or sick office visits

^cDirection of relationship confirmed through analysis of peer-reviewed programs that employed randomized, controlled trial evaluations

CI = confidence interval

ED = emergency department

Table 2 shows that offering program components in a clinical setting and providing asthma training to health-care providers (including school nurses) were significantly associated, respectively, with fewer asthma ED visits and fewer school absences by children with asthma.

Table 2.

Association of school absences and ED visits with programs that included a clinical focus or location in the Asthma Health Outcomes Project

^aN = number of programs that measured both variables

^bDirection of relationship confirmed through analysis of peer-reviewed programs that employed randomized, controlled trial evaluations

ED = emergency department

CI = confidence interval

Table 3 provides findings regarding items related to the collaborative nature of programs. Those that collaborated with other organizations and -institutions and with government agencies had, respectively, less health-care use (in general) and fewer ED visits (in particular) by their participants. Specific types of collaboration were associated with significant results. Collaborating on policy action with other organizations and institutions was associated with significant results related to fewer school absences and improved use of asthma medicines by patients.

Table 3.

Association of health-care utilization, school absences, and medicine use with programs that involved collaborative activity with other agencies, organizations, or institutions in the Asthma Health Outcomes Project

^aN = number of programs that measured both variables

^bDirection of relationship confirmed through analysis of peer-reviewed programs that employed randomized, controlled trial evaluations

^cA general measure reported by key informants that may include any combination of hospitalizations, ED visits, urgent care visits, or sick office visits

^dCalculations of the odds ratios were performed with the use of a zero-cell correction of 0.5.

CI = confidence interval

ED = emergency department

Tables 4 and 5 provide findings related to programs' efforts to respond to particular participants. As shown in Table 4, designing programs for a specific racial/ethnic group and tailoring the program content or delivery to individual participants' health or educational needs were significantly associated with enhanced QOL and symptom control for asthma sufferers.

Table 4.

Association of quality of life and symptoms with programs that included tailoring for participants in the Asthma Health Outcomes Project

^aN = number of programs that measured both variables

^bDirection of relationship confirmed through analysis of peer-reviewed programs that employed randomized, controlled trial evaluations

^cCalculations of the odds ratios were performed with the use of a zero-cell correction of 0.5.

^dIncludes any combination of QOL for children with asthma, parents/caregivers of children with asthma, or adults with asthma

CI = confidence interval

QOL = quality of life

Table 5.

Association of quality of life with programs that included assessments of environmental triggers in the Asthma Health Outcomes Project

^aN = number of programs that measured both variables

^bDirection of relationship confirmed through analysis of peer-reviewed programs that employed randomized, controlled trial evaluations

^cIncludes any combination of QOL for children with asthma, parents/caregivers of children with asthma, or adults with asthma

^dCalculations of the odds ratios were performed with the use of a zero-cell correction of 0.5.

CI = confidence interval

QOL = quality of life

Table 5 illustrates that assessing environmental triggers and tailoring interventions based on these assessments were significantly associated with enhanced QOL for participants. A further analysis showed that conducting a needs analysis or resource assessment in general correlated significantly with positive school and work-loss reductions (OR=22.09, 95% CI 2.25, 216.60, p=0.02).

The number of programs among those surveyed in which the characteristics presented in the Figure and Tables were evident ranged from 44 to 168, depending on the characteristics in question. Tailoring the content or delivery method based on individual health or educational needs was the most common characteristic associated with a positive health outcome, followed by assessing exposure to asthma triggers. For engaging communities, we found an association between being physically present in the target community and positive outcomes. Tailoring the intervention based on an assessment of sensitivity to environmental asthma triggers was the least practiced strategy.

DISCUSSION

The characteristics of asthma programs that emerged in this study as significantly associated with desired asthma outcomes can be summarized as being community centered, collaborative, clinically connected, and responsive to participants' needs (as uncovered through preliminary demographic, health, educational, and environmental assessments).

There are a number of likely reasons that the program characteristics evident in these findings were significantly associated with asthma outcomes—that is, why programs reflecting certain elements achieved success. A growing body of work illustrates how collaborating with organizations and individuals who live in the targeted area and intimately understand the targeted health problem (e.g., asthma) can enhance the quality of an intervention.³⁶ This participatory approach to program design and implementation seems to be especially important in vulnerable communities. Collaboration that includes the program providers actually being physically located in the target community enhances a program's visibility and collaborative intentions.³⁷ Further, broader collaboration in general (e.g., among voluntary, governmental, academic, and other groups interested in asthma control) has been associated with the ability to mobilize and deploy resources needed for asthma control.³⁸

In recent years, government involvement, particularly in local programs, has been encouraged through, for example, the National Asthma Education and Prevention Program of the National Heart, Lung and Blood Institute and the Centers for Disease Control and Prevention.²¹ Further, technical assistance provided to individuals and families trying to control asthma—often in the form of visits to the home and provision of technical advice about environmental control and asthma monitoring—has been shown to be effective in generating positive changes in health status and health-care use.¹⁰ Tailoring interventions based on assessments of individuals' sensitivity to environmental precipitants is another type of asthma program proven to successfully achieve positive health outcomes.³⁹

Asthma prevalence data and data regarding disparities in health status show that inequities are most evident among people living in vulnerable communities, including those in low-income and diverse racial/ethnic areas. Programs that are especially designed to consider the social and economic needs in such communities and the culture in which asthma is managed have been shown to achieve positive results in a number of studies.^40,41

Addressing the training needs of health-care providers regarding the effective clinical management of asthma has been shown in previous work to be directly associated with improvements in asthma outcomes.^14,42 In addition, a number of studies of interventions conducted in clinical settings, including the physician's office, have produced positive results.^15,43

This study examined characteristics of a large number of programs around the world and gives evidence of the importance of a few principles in achieving health outcomes of value to individuals, families, and the public health system. Findings suggest that asthma programs could enhance their potential for success by intentionally reflecting characteristics identified in this research.

Limitations

This study had several limitations. There may be important program characteristics that were not covered in the questionnaire developed for data collection. Nonetheless, it was extensive and tried to be comprehensive regarding accepted concepts in asthma and in program design and evaluation^21,44,45 and included open-ended queries where respondents could offer information. Further, the study examined a large number of programs and did not independently evaluate the effectiveness of any one program. Therefore, caution is needed when applying the collective findings to individual programs. We excluded programs that reported no positive outcomes from the study, and this exclusion could have resulted in bias. However, many if not all of the programs studied measured more than one outcome; therefore, analyses included data from programs that experienced both successful and unsuccessful efforts to demonstrate improvement.

Data for analyses were provided through self-report in telephone interviews when published reports were not available. However, findings from these two different sources were only different in that unpublished program data tended to produce more significant statistical values. Only evaluations demonstrating at least one positive outcome were included in the study and exclusion of interventions with no outcomes could have biased results. “No result” studies are rarely published in the literature and are very difficult to otherwise locate. However, the large number of programs included in the research and the variety of outcomes produced lend confidence to our findings. Finally, only programs that were comprehensive regarding asthma by including an environmental component were eligible for the study. Caution in generalizing findings beyond this type of program should be exercised.

CONCLUSIONS

A number of features made this study unique. It went beyond published articles to reflect the broader field of practice in a large number of communities in the U.S. and abroad. It explored efforts across programs to determine successful characteristics of a wide variety of interventions. It used an iterative process for data analysis, including both published and unpublished reports to ensure confidence in results.

There is no question that the global burden of asthma is very high. The results of this study suggest that a few characteristics of programs enhance their potential to help people with asthma to control the effects of the disease. When activities are implemented to engage CBOs, results are evident. When there is a close connection to the clinical setting and efforts to enhance skills of health-care providers, change occurs. Collaborating broadly with the range of institutions and agencies that hold a stake in asthma control makes a difference. Carefully assessing particular asthma problems as faced by specific groups of individuals and shaping a program to address their needs leads to success. Findings from this research add to the understanding of how programs can be improved to better serve the goal of asthma control.