Abstract
Objective
The evaluation sought to determine if a comprehensive, school-based asthma management program in a small, rural school district helped students improve asthma control.
Methods
To determine if students in the asthma program demonstrated better asthma control than students in a comparison school district, the evaluation team used a quasi-experimental, cross-sectional design and administered questionnaires assessing asthma control (which included FEV1 measurement) to 456 students with asthma in the intervention and comparison districts. Data were analyzed for differences in asthma control between students in the two districts. To determine if students in the intervention experienced increased asthma control between baseline and follow-up, the evaluation team used a one-group retrospective design. Program records for 323 students were analyzed for differences in percent of predicted forced expiratory volume in one second (FEV1) between baseline and follow-up.
Results
Students with asthma in the intervention district exhibited significantly better asthma control than students with asthma in the comparison district. Percent of predicted FEV1 did not change significantly between baseline and follow-up for the intervention participants; however, post hoc analyses revealed students with poorly-controlled asthma at baseline had significantly higher FEV1 scores at follow-up, and students with well-controlled asthma at baseline had significantly lower FEV1 scores at follow-up.
Conclusions
Findings suggest the comprehensive school-based program led to improvements in asthma control for students with poorly controlled asthma at baseline, and school-based programs need mechanisms for tracking students with initially well-controlled asthma in order to ensure they maintain control.
Keywords: Asthma, schools, evaluation, youth, children, asthma control
INTRODUCTION
Asthma is a leading chronic illness among youth in the United States. Estimates show, in 2011, more than 10 million children in the United States had been diagnosed with asthma at some point in their lives [1], an increase of approximately 1 million children from a decade earlier [2]. In addition, 2011 data indicate an estimated 7 million children currently live with asthma [1].
Asthma—especially poorly controlled asthma—places considerable burden on those diagnosed with it and on their families, communities, and schools; schools feel the burden of asthma in a unique way. Asthma is a leading cause of illness-related school absenteeism [3], and in 2008, was estimated to have led to approximately 14.4 million missed school days [4]. It can lead to decreased academic performance among students with asthma and classroom disruptions for other students and staff [3]. Furthermore, because asthma is potentially life-threatening if not properly managed [5], school staff must be prepared to respond to asthma-related emergencies and implement strategies to reduce students’ likelihood of experiencing exacerbations at school.
Fortunately, schools are well-positioned to help students better manage asthma. Schools offer efficient points of access to youth ages 5-18 years, and often already employ nurses, teachers, and other staff who can help identify and address asthma difficulties. Teachers and other school personnel who interact regularly—often daily—with students can observe changes in behavior, appearance, and health status. Furthermore, school nurses and other health professionals have relevant clinical backgrounds that make them well-suited to deliver asthma management interventions. When schools address environmental triggers and students receive appropriate asthma education and care, students can control their asthma with few, if any, symptoms [3], thereby improving students’ health status and potential for academic success.
For the past decade, the Centers for Disease Control and Prevention (CDC) has encouraged schools to address students’ asthma through six key strategies:
(1) establish management and support systems for asthma-friendly schools; (2) provide appropriate school health and mental health services for students with asthma; (3) provide asthma education and awareness programs for students and school staff; (4) provide a safe and healthy school environment to reduce asthma triggers; (5) provide safe, enjoyable physical education and activity opportunities for students with asthma; and (6) coordinate school family, and community efforts to better manage asthma symptoms and reduce school absences among students with asthma [6].
In line with this guidance, many schools have adopted programs to increase asthma education for students with asthma (e.g., American Lung Association’s Open Airways for Schools [7]) and reduce asthma triggers in the school environment (e.g., Environmental Protection Agency’s Tools for Schools [8]).
Although approaches such as education and reduction of triggers are essential for managing asthma, it is also critical that children receive appropriate clinical care and medications [9, 10]. Fortunately, schools also can play a role in helping students access clinical care they need [10]. Through activities such as assessing lung function, monitoring medication adherence, teaching medication administration techniques, or securing students’ appointments with providers, school-based programs can directly provide some basic care and connect students to community-based health care providers for medications and more advanced care.
In light of the critical nature of clinical care and appropriate pharmacotherapy, recommendations have been made that school-based asthma programs use coordinated and multiple components to address the range of needs experienced by students with asthma [10]. Such comprehensive programs not only provide education, they can also help students with asthma gain access to necessary clinical care, ensure students have access to appropriate medications, provide intensive case management, and work directly with students’ families to help students gain and maintain better asthma control [10]. In implementing programs with such comprehensive scope, suggested strategies call for programs with limited resources to focus efforts first on students with the most poorly controlled asthma [6, 10].
The program evaluated in this study used a comprehensive model that combined education for students with asthma, training for school staff, home visits and education for families of students with asthma, and intensive case management that emphasized linking students to community-based clinical providers and providing information on students’ asthma (including clinical assessments of asthma control and difficulties) to those providers so that their care could be better-informed and more targeted. In addition, the program followed current recommendations to focus program resources such that all students with asthma interfaced with the program, but more intensive effort was given to those with the greatest asthma management difficulties.
The program was conducted in a rural, agriculturally-focused town of approximately 11 000 people in the central United States. The school district’s five schools served approximately 1974 students from two counties during the 2011-2012 school year. Compared to the state average, poverty levels in both counties are high [11, 12], and approximately half of the district’s students are Medicaid eligible.
Purpose
The evaluation examined whether a comprehensive model of a school-based asthma management program in a small, rural school district helped students improve their asthma control. To determine this, evaluators posed two key questions: (1) “Did students in the asthma program demonstrate better asthma control than students in a comparison group?” and (2) “Did students in the asthma program experience improvements in indicators of asthma control between baseline and follow-up?”
METHOD
Evaluation design
The evaluators used two design approaches. To determine if students in the asthma program demonstrated better asthma control than students in a comparison group, evaluators used a quasi-experimental, cross-sectional design to collect asthma control data from students with asthma in the intervention school district and a comparison school district. To assess whether students in the program experienced improvements in asthma control between baseline and follow-up, evaluators used a one-group, retrospective, longitudinal design, collecting data from nurse records of students in the asthma program at the intervention school district.
Evaluators selected the comparison school district based on its similarity to the intervention school district in terms of location, number of students (1573 in the 2011-2012 school year), student demographics, and rural, agriculturally-focused setting. It was also selected because it had no formal asthma management program.
The evaluation was initially planned for the 2010-2011 school year but was extended to include the 2011-2012 school year after flooding closed the comparison school district the week evaluators were present for data collection. As a result, an initial round of data collection happened in May 2011, and a second data collection round happened the following school year in October 2011. This manuscript reports findings of the second, complete round of data collection, which included data from students with asthma in both the intervention and comparison districts. Program record data analyzed for this manuscript were from students who returned consent forms for the October data collection.
The evaluation was approved by administrators at both school districts, and ICF International’s Institutional Review Board (IRB) for the Protection of Human Subjects, which has a current Federal Wide Assurance (FWA00000845) with the Office for Human Research Protections. When the protocol was changed due to flooding, the IRB approved the amendment.
The intervention
The asthma program in the intervention district was organized around four critical components: (1) identification of students with asthma; (2) asthma education for students with asthma; (3) case management for students with asthma; and (4) asthma training for staff. An expanded intervention description with implications for replication is provided in the full evaluation report [13].
Identification of students with asthma
Asthma program staff (e.g., school nurses, the asthma educator, the district’s health services director) identified students with asthma in several ways. Most frequently, they were notified of students’ asthma through emergency information cards and health inventory sheets. Sometimes, students with asthma were identified when they presented to nurses with asthma-like symptoms or when parents called about their children’s asthma. Additionally, local hospital staff, with consent from parents/guardians, notified district staff if they saw students with asthma-related problems.
Asthma education
Asthma education, particularly for students with asthma and their families, was another important intervention component, and it included both formal and informal strategies. IMPACT Asthma—Kids! is a computer-based program used during the school day for kindergarten through fifth grade students with asthma to educate them about asthma, medicines, triggers, and proper response during exacerbations [14]. An asthma support group allowed third through fifth grade students time to discuss asthma and support one another while learning to manage their asthma more effectively. The Asthma Academy offered another opportunity for asthma management education. This all-day workshop was provided 1-4 times a year by program staff and a well-known expert on guidelines-based treatment of asthma. The expert trainer, working alongside program staff and local health care providers, educated students and their parents/families about asthma management, including triggers and medications. The Asthma Academy also provided opportunities for physical assessments, pulmonary function tests, and education about the meaning of forced expiratory volume in one second (FEV1) and peak expiratory flow (PEF) values in addition to expert care that can be challenging to access in rural communities. In addition to formal education opportunities, asthma program staff conducted more informal, one-on-one education with students, frequently teaching students individually about asthma triggers, medications, and medication inhalation technique.
Case management
In addition to one-on-one education, school nurses and the asthma educator provided case management for students with asthma. Because the intervention district had a school nurse in each of its 2 elementary schools, 1 middle school, and 1 high school, nurses had regular access to students, allowing them to implement several beneficial activities. Some activities (i.e., ensuring asthma action plans were on file; administering medications as necessary) were part of basic nursing duties, but program staff also implemented more intensive case management activities for students at all grade levels. These intensive activities included: assessing asthma control through PEF, FEV1, and the asthma control test (ACT); conducting home visits to educate families and identify asthma triggers; arranging extensive home environmental assessments (including remediation suggestions and/or assistance); and communicating students’ symptoms with parents and healthcare providers. In addition, some education components (e.g., teaching inhalation technique) were implemented in the context of case management. EPR-3 guidelines [15] provided parameters for all program components (including case management), but the guidelines were particularly important in facilitating clear, effective communication with healthcare providers, a key and innovative aspect of the program.
Training for staff
The intervention district focused heavily on providing asthma training for the program staff, other school staff, and community healthcare providers. Program staff received in-depth, EPR-3-based, clinical training in partnership with Asthma Ready Communities®. In the training, staff learned to take and interpret FEV1 readings and to teach inhalation technique using the In-Check DIAL®. In addition, the school district provided asthma training bi-annually to teachers and other school staff and annually to sports coaches and physical education teachers to teach early signs and symptoms of exacerbations, as well as how to prevent and respond to asthma exacerbations. In partnership with Asthma Ready Communities®, the program staff also brought training opportunities to local health care providers and their staff; these trainings focused on asthma care and treatment in accordance with the EPR-3 guidelines.
Asthma activities in the comparison school district
The comparison school district had no formal asthma program, but school nurses conducted basic activities to address students’ asthma as part of normal nursing duties. Like the intervention district, the comparison district had one school nurse assigned to each school. Nurses used emergency information forms to identify students with asthma, and they sometimes discovered additional students with asthma when students presented with symptoms. Though the district had no formal or systematic approach to asthma case management, they did store asthma inhalers and administer medicines as needed. They also conducted some physical assessments (i.e., listening to students’ lungs with stethoscopes, checking oxygen levels in the blood with pulse oximeters) but did not assess FEV1, PEF, or ACT values. Furthermore, nurses in the comparison district had not received routine, up-to-date asthma training (though one nurse attended an asthma-focused workshop two years prior to the evaluation), and they reported helping students manage asthma exacerbations with techniques that have not been proven effective and are not recommended (e.g., having students drink cold water, warm coffee, or warm soda). The district did not provide asthma education for students.
Measures
Asthma Control Questionnaire (ACQ) score
The Asthma Control Questionnaire (ACQ) [16, 17] score was the measure of asthma control used to examine cross-sectional differences in students in the intervention and comparison districts. Evaluators used a 9-item questionnaire to quantify both students’ self-perceived asthma management and control and an objective assessment of lung function measured through spirometry. The questionnaire included seven questions that make up the Asthma Control Questionnaire (ACQ), an instrument previously validated [16, 18] for use with children (test-retest reliability in the validation study revealed an intraclass correlation coefficient of 0.79) [16]. Five of the seven ACQ questions ask about symptoms—being woken at night by symptoms, waking in the mornings with symptoms, limitation of daily activities, shortness of breath, and wheezing. The sixth question covers quick-relief medication use and the seventh was completed by trained data collectors using spirometers to assess students’ FEV1, a standard lung function measure.
All ACQ questions are equally weighted, rendering a score that is the average of the responses. The possible range of scores on the ACQ is 0 to 6, with lower scores indicating better asthma control. Any student scoring 0.0 for the seven questions on the ACQ was considered to have “totally-controlled” asthma, and a student scoring 6.0 was considered to have “severely uncontrolled” asthma [17].
Percent of predicted FEV1
FEV1 readings at designated baseline and follow-up time points allowed objective assessment of asthma control changes among intervention district students in the asthma program. The variable used for analysis was percent of predicted FEV1, which provided standardized interpretation of FEV1 scores. Percent of predicted FEV1 was calculated using raw FEV1 scores from program records. Because of variation in the number and timing of FEV1 assessments in students’ program records, evaluators established criteria for baseline and follow-up measures. Baseline raw FEV1 was defined as the first recorded assessment of FEV1 with an associated date and height recorded (all three variables—FEV1, date, and height—were needed to calculate percent of predicted FEV1). The date of the baseline varied for students, depending on when they entered the asthma management program. Consequently, a student’s baseline value could be from the 2007-2008, 2008-2009, 2009-2010, or 2010-2011 school year. Follow-up raw FEV1 was defined as the first FEV1 measure from the annual assessment for the 2011-2012 school year.
Once raw FEV1 scores were identified for baseline and follow-up, evaluators used data on each student’s sex, race, date of recorded FEV1 score, and height at the time of the recorded FEV1 score to determine the predicted FEV1 for other individuals of similar age, sex, height, and race; this was done using spirometric reference values by Hankinson et al. [19]. Once predicted FEV1 was identified, the raw FEV1 score was compared to determine each student’s percent of predicted FEV1 at baseline and follow-up; percent of predicted FEV1 provided a standardized way of comparing FEV1 scores across students and years.
Asthma control classification
For post-hoc analyses related to well-controlled or poorly-controlled asthma, evaluators used ACQ score or percent of predicted FEV1 to classify students’ asthma control. When using ACQ scores for comparing asthma control classification among students in the intervention and comparison districts, evaluators followed guidelines provided by ACQ creators to use ACQ scores to classify students as having well-controlled asthma or not-well-controlled asthma. The guidelines were derived from the GOAL study [20] and based on recommendations for clinical practice from the Global Initiative for Asthma and the National Institutes of Health. Accordingly, if a student had a total ACQ score of 0.75 or less, there was an 85% chance his or her asthma was well controlled. For purposes of analysis, students with ACQ scores of 0.75 or less were classified as having well-controlled asthma. Students with ACQ scores higher than 0.75 were classified as having not-well-controlled asthma.
When using FEV1 scores for comparing baseline and follow-up asthma control classifications of students in the intervention, evaluators followed criteria referenced in the EPR-3 guidelines [15] to classify level of asthma control. EPR-3 guidelines describe asthma as being: “well controlled” when a person’s percent of predicted FEV1 is greater than 80%, “not well controlled” when percent of predicted FEV1 is 60-80%, and “very poorly controlled” when percent of predicted FEV1 is less than 60% [15]. For analysis, evaluators combined the “not well controlled” and “very poorly controlled” categories into a single category labeled “poorly controlled” (i.e., students with FEV1 scores of 80% or less of their predicted FEV1 values were classified as having “poorly controlled asthma”).
Participants
Evaluators analyzed existing data from program records of students with asthma in the intervention district and collected data on asthma control from students with asthma in the intervention and comparison districts. Nurses in both districts used emergency cards and other nurse records to identify students with asthma. Nurses gave those students letters describing the evaluation and consent forms for parents, asking permission to disclose their children’s asthma status to the evaluators and consent to their children’s participation in the evaluation. Nurses called parents/guardians of students who did not return consent forms. Nurses in each district provided the evaluators a list of students whose parents consented to their participation. In both districts, those students were recruited for participation in the cross-sectional, asthma control data collection, and in the intervention district, those students’ existing program records were used for additional data analysis. Due to unexpected flooding, the consent process occurred in May 2011 and again in October 2011. Because a key data collection instrument had been validated for use with children age six and older, evaluators recruited kindergarten students who were at least six years old and students in the first through twelfth grade who attended regular, mainstream classrooms.
In the intervention district, 350 students in four schools were identified as having asthma. In October 2011, 323 of the 350 students returned consent forms and 299 participated in cross-sectional data collection, yielding a district response rate of 85.4% (see Table 1). In the comparison district, 235 students in five schools were identified as having asthma. Of those, 159 students returned consent forms and 157 participated in data collection, yielding a district response rate of 66.8%. In the intervention district, 16 kindergarten students returned consent forms but were not six years of age by the time of questionnaire administration, and therefore, were not eligible to participate. Across both districts, 10 students returned consent forms but were absent during data collection.
Table 1.
Student Questionnaire Response Rates Among Students in the Intervention and Comparison School Districts
| School District School |
Number of Students Identified as Having Asthma |
Number of Students Who Returned Consent Forms |
Number of Students Who Completed the Questionnaire |
Response Rate, Expressed as a Percentage |
|---|---|---|---|---|
| Intervention school district | 350 | 323 | 299 | 85.43% |
| Elementary schools | 172 | 162a | 144 | 83.72% |
| Middle school | 82 | 77 | 74 | 90.24% |
| High school | 96 | 84 | 81 | 84.38% |
|
| ||||
| Comparison school district | 235 | 159 | 157 | 66.81% |
| Elementary schools | 78 | 67 | 67 | 85.90% |
| Middle school | 63 | 38 | 38 | 60.32% |
| High school | 94 | 54 | 52 | 55.32% |
After returning consent forms, the evaluation team discovered that 16 of the elementary students (kindergarten students) were not six years old by the time of questionnaire administration. Therefore, these students were not eligible to participate in the cross-sectional questionnaire.
For the retrospective analyses of program records among students with asthma in the intervention district, evaluators pulled records for 323 students with completed consent forms. Only students with all the data needed for calculating percent of predicted FEV1 at baseline and follow-up were included in analyses. Of students for whom data were abstracted, approximately 39% (n=127) did not have complete records for the FEV1 measures and were not included in analyses. Missing data were managed using list-wise deletion to limit the data set to students with complete data, resulting in an analysis sample of 196 students.
Data Collection
Questionnaire administration
Trained data collectors administered the 7-item ACQ to students with asthma. Data collectors read each question aloud to every student to ensure comprehension. To complete the last item on the questionnaire, data collectors measured each student’s FEV1 three times with Vitolgraph asma-1 handheld asthma monitors and measured the student’s height. Each student’s best raw FEV1 measurement was converted to his or her percent of the predicted FEV1 value for individuals of similar age, sex, height, and race.
Document review
Administrative records from both districts provided age, sex, and race for students with asthma who completed the questionnaire. These data, together with spirometry results, were used to calculate percent of predicted FEV1.
Among students in the intervention school district, program records for the previous five years provided students’ FEV1 readings at baseline and follow-up points. To ensure consistency between two data abstractors in recorded FEV1 values, evaluators assessed interrater reliability. Each data abstractor recorded data from the same set of 15 students, approximately 5% of the sample. Interrater reliability for the 15-record abstraction was 94.8%. To address inconsistencies, data abstractors discussed where they found each data point and their rationales for entering specific values and came to a consensus.
Data analysis
Data were analyzed using SAS version 9.3. To look for a difference in asthma control between students in the asthma program and comparison group students, evaluators first assessed demographic differences between students in the two groups. The team used one-way analysis of variance (ANOVA) tests to look for differences in age and grade and chi-square analyses to look for differences in race and gender between students in the intervention and comparison school districts. To determine if asthma control differed between the two groups, evaluators conducted a t-test to look for differences between ACQ scores in students in the two districts. Evaluators also conducted a posthoc multivariate logistic regression to examine the odds of having well-controlled asthma in the intervention district compared to the comparison district.
For analyses of baseline and follow-up data from students in the intervention group, missing data were handled by list-wise deletion. Then, evaluators looked for differences between students with missing data and students without missing data by using t-tests to examine age and percent of predicted FEV1 at follow-up and chi-square analyses to examine gender and race. To determine if students in the asthma program experienced improvements in asthma control between baseline and follow-up, evaluators conducted a t-test (planned a priori) to look for change in percent of predicted FEV1 between baseline and follow-up. Evaluators later ran post hoc analyses to explain findings of the original t-test. Post hoc analyses included: t-tests to look for change in the percent of predicted FEV1 between baseline and follow-up in students with well-controlled asthma at baseline and students with poorly-controlled asthma at baseline, and a McNemar’s chi-square test to look for a difference in the distribution of students into well-controlled and poorly-controlled asthma classifications between baseline and follow-up.
Missing Data Analysis
Evaluators looked for differences between baseline and follow-up asthma control among students in the intervention by using data from program records. Therefore, the analysis sample was limited by incompleteness of some students’ existing records (e.g., missing raw FEV1 readings or associated dates, heights, or race). To explore potential bias, evaluators compared participants with complete data (the analysis sample) to the group of participants with missing data identify any differences in race, gender, age at time of baseline FEV1 assessment, and percent of predicted FEV1 at follow-up. Results of a t-test showed students in the analysis sample were significantly older at the time of baseline FEV1 assessment (M = 10.51, SD = 3.31) than those excluded from analysis (M = 9.53, SD = 3.52) (t(314) = −2.51, p = .01). Despite the slight (approximately 1 year) difference in age at baseline FEV1 assessment, no significant differences emerged between the groups with respect to race [White (X2 (1, N = 323) = .31, p = .58); African American (X2 (1, N = 323) = .0052, p = .94); Asian (X2 (1, N = 323) = .10, p = .76); Hispanic (X2 (1, N = 323) = 1.1981, p = .27)], gender (X2 (1, N = 323) = .96, p = .33), or percent of predicted FEV1 at follow-up (t(294) = 1.25, p = .21). Of these variables, percent of predicted FEV1 at follow up was particularly important because that measure took age into consideration and was critical for answering the evaluation questions. Given there was no significant difference in the percent of predicted FEV1 at follow-up between the missing data sample and the analysis sample, evaluators concluded students with missing values were no more likely to have higher (or lower) percent of predicted FEV1 than students with complete data, and consequently, the evaluators felt comfortable the analysis sample constituted a valid sample for additional analyses.
In addition, one student’s records revealed an extremely high percent of predicted FEV1 at baseline; evaluators ran analyses with and without this outlier, and because overall conclusions remained the same and it was not possible to determine whether the value was truly not valid, the evaluators included this student in the analyses. The resulting analysis sample for the evaluation question about changes between baseline and follow-up for students in the asthma program consisted of 196 students.
Missing data were not problematic in the analysis of cross-sectional asthma control data from intervention and comparison district students. All students completed the questionnaire without any missing responses.
RESULTS
Differences in asthma control among intervention vs. comparison district students
Demographic variables
Demographic characteristics of students in the intervention school district and students from the comparison district are presented in Table 2. ANOVA results revealed no significant differences in age (F (1, 456) = 1.00, p = 0.31) or grade (F (1, 456) = 1.39, p = 0.238) between students in the two districts. Chi-square analyses revealed no significant differences in race (X2 (1, 456) = 4.6052, p < 0.2334) or gender (X2 (1, 456) = 0.0012, p = 0.9729) between the two groups.
Table 2.
Demographic Characteristics for Students with Asthma in the Intervention and Comparison School Districts
| Demographic Characteristic |
Students in the
Intervention School District (n=299) |
Students in the
Comparison School District (n=157) |
|---|---|---|
| Age (in years) | ||
|
| ||
| Mean age (standard deviation) | 12.04 (SD=3.44) | 12.38 (SD=3.53) |
|
| ||
| Range | 6.19-19.30 | 6.54-18.93 |
|
| ||
| Gender | ||
|
| ||
| Male | 56.5% (n=169) | 56.7% (n=89) |
|
| ||
| Female | 43.5% (n=130) | 43.3% (n=68) |
|
| ||
| Race | ||
|
| ||
| White | 55.2% (n=165) | 63.7% (n=100) |
|
| ||
| Black | 40.1% (n=120) | 34.4% (n=54) |
|
| ||
| Hispanic | 4.0% (n=12) | 1.9% (n=3) |
|
| ||
| Asian | 0.7% (n=2) | 0.0% (n=0) |
|
| ||
| Grade Level | ||
|
| ||
| K | 1.3% (n=4) | 0.0% (n=0) |
|
| ||
| 1 | 12.7% (n=38) | 9.6% (n=15) |
|
| ||
| 2 | 10.0% (n=30) | 12.1% (n=19) |
|
| ||
| 3 | 3.7% (n=11) | 7.0% (n=11) |
|
| ||
| 4 | 9.7% (n=29) | 6.4% (n=10) |
|
| ||
| 5 | 10.7% (n=32) | 7.6% (n=12) |
|
| ||
| 6 | 7.7% (n=23) | 7.0% (n=11) |
|
| ||
| 7 | 8.0% (n=24) | 12.7% (n=20) |
|
| ||
| 8 | 9.0% (n=27) | 4.5% (n=7) |
|
| ||
| 9 | 6.4% (n=19) | 8.9% (n=14) |
|
| ||
| 10 | 12.4% (n=37) | 7.6% (n=12) |
|
| ||
| 11 | 4.0% (n=12) | 12.7% (n=20) |
|
| ||
| 12 | 4.3% (n=13) | 3.8% (n=6) |
ACQ scores
A t-test analysis revealed students from the intervention and comparison districts had significantly different ACQ scores. Students in the intervention district had lower mean scores (M = .93, SD = .73 points), indicating better asthma control, than students in the comparison district (M = 1.14, SD = .84 points) (t (456) = −2.65, p = .0085). Intervention district students had, on average, an ACQ score 0.21 points lower than comparison district students. Even when controlling for race, grade, gender, and age, students from the intervention district had significantly lower ACQ scores than students from the comparison district (F (1, 456) = 8.17, p = .0045) (see Table 3).
Table 3.
Multivariate regression results for differences in Asthma Control Questionnaire scores between students in the intervention district and students in the comparison district.
|
Possible
Score Rangea |
Students in the
Intervention School District (n=299) |
Students in the
Comparison School District (n=157) |
||||||
|---|---|---|---|---|---|---|---|---|
| Measure | M | SD | M | SD | F | df | p | |
| Asthma Control Questionnaire Scoresb |
0-6 | .93 | .73 | 1.14 | .84 | 8.17 | 456 | .0045 |
Lower scores indicate better asthma control.
Regression analysis controlled for race, grade, gender, and age.
Asthma control classification based on ACQ scores
Based on their ACQ scores, 51.8% of the students from the intervention district had well-controlled asthma, whereas 40.1% of students from the comparison district had well-controlled asthma. Results of a multivariate logistic regression controlling for race, age, grade, and gender revealed a statistically significant difference in asthma control classification between students in the intervention district and the comparison district (X2 (1, N = 456) = 4.1479, p = .0417); the odds of having well-controlled asthma were 54.8% higher for students in the intervention school district than for students from the comparison district (OR = 1.548; 95% CI, 1.017 to 2.358).
Changes in asthma control among students in the intervention
Percent of predicted FEV1 at baseline and follow-up
Among students in the asthma program, the mean percent of predicted FEV1 at baseline was 88.25 (SD = 21.38), and the mean percent of predicted FEV1 at follow-up was 87.80 (SD = 16.55). Findings of a paired t-test showed no significant change in percent of predicted FEV1 between baseline and follow up (M = −.46, SD = 19.07, t(195) = −.33, p = .74).
After finding no overall change in percent of predicted FEV1 between baseline and follow-up, evaluators conducted post hoc analyses to separately look at changes in percent of predicted FEV1 among students with well-controlled asthma and those with poorly-controlled asthma. Students with poorly-controlled asthma at baseline (n=70) experienced a significant mean increase of 10.11 percentage points at follow-up (t(69) = 5.48, p < .01), and students with well-controlled asthma at baseline (n=126) experienced a mean decrease of 6.33 percentage points at follow-up (t(125) = −3.86, p < .01) (see Table 4).
Table 4.
t-test results for differences in percent of predicted FEV1a from baseline to follow-up for students who participated in the intervention.
| Baseline Percent of Predicted FEV1 |
Follow-up Percent of Predicted FEV1 |
||||||
|---|---|---|---|---|---|---|---|
| Students included in t-test | M | SD | M | SD | t | df | p |
| All students in the intervention (n=196) | 88.25 | 21.38 | 87.80 | 16.55 | −0.33 | 195 | 0.74 |
|
| |||||||
| Students with poorly-controlled asthma at baseline (n=70) |
68.29 | 9.75 | 78.40 | 12.79 | 5.48 | 69 | <0.01 |
|
| |||||||
| Students with well-controlled asthma at baseline (n=126) |
99.34 | 17.70 | 93.01 | 16.13 | −3.86 | 125 | <0.01 |
Higher values for percent of predicted FEV1 indicated better asthma control.
Asthma control classification based on percent of predicted FEV1
Evaluators conducted another post hoc analysis to examine transition in asthma control classification (based on percent of predicted FEV1) between baseline and follow-up. Although a McNemar’s chi-square test revealed no overall difference in the distribution of students into well-controlled and poorly-controlled classifications between baseline and follow-up (X2 = 1.528, N = 196, p = .216), examination of descriptive frequencies revealed there were students who moved from one classification to the other between baseline and follow-up. As shown in Table 5, 44.3% (n=31) of students with poorly-controlled asthma at baseline had well-controlled asthma at follow-up; however, the remaining 55.7% of students with poorly-controlled asthma at baseline remained classified as having poorly-controlled asthma at follow-up. Of students with well-controlled asthma at baseline, 82.5% had well-controlled asthma at follow-up, while 17.5% (n=22) had poorly-controlled asthma at follow-up.
Table 5.
Percent and Number of Students from the Intervention District’s Program Record Analysis Sample Who Were in Each Asthma Control Classification at Baseline and Follow-up
|
Students’ Asthma Control at Follow-up
|
||
|---|---|---|
|
Students’ Asthma Control at
Baseline |
Well Controlled (n=135) | Poorly Controlled (n=61) |
|
|
||
| Well Controlled (n=126) | 82.54% (n=104) | 17.46% (n=22) |
| Poorly Controlled (n=70) | 44.29% (n=31) | 55.71% (n=39) |
DISCUSSION
Key Findings and Implications
Existing guidelines and research offer numerous strategies school staff can use to help students improve asthma control [6, 10]. For years, schools have taught students with asthma about asthma triggers and management techniques through group and individualized educational programs [21, 22], but recently, and in light of EPR-3 guidelines [15], health and education professionals have given increased attention to finding ways schools can improve students’ access to asthma-related healthcare, a critical foundation for successfully managing asthma [21, 23]. The collective evidence provides support for school-based programs to use comprehensive approaches (i.e., including education and trigger reduction for students and families, case management, and linkage to clinical care) to improve students’ asthma control [6, 10].
The school-based asthma management program in this evaluation took a comprehensive approach to help students better manage their asthma and used EPR-3 guidelines to set parameters for all activities and communications. Activities encompassed many recommended strategies (i.e., education for students with asthma and their families, intensive case management, assessment of environmental triggers, and linkage to clinical care) and emphasized intensive case management activities such as teaching medication inhalation technique, providing regular clinical assessments of asthma control using FEV1 measures, and ensuring relevant information about students’ asthma (including FEV1 values) were communicated to clinical providers. Because students may experience variability in their asthma over time [15], these types of program components—particularly clinical monitoring and sharing assessments with providers—enable providers to address variations in asthma control with appropriate treatment adjustments. These approaches, combined with the program’s role in facilitating access to quality training for community health care providers, illustrate ways schools can provide both care coordination and limited care delivery, and help ensure student access to evidence-based clinical care.
Evaluation findings revealed a subset of students in the intervention school district experienced improvements in asthma control during participation in the asthma program. Although there was no change in percent of predicted FEV1 between baseline and follow up for the full sample of program participants, post hoc analyses revealed students who initially had poorly-controlled asthma appeared to experience slight improvement in asthma control at follow-up. Furthermore, analyses examining transition in asthma control classification from baseline to follow-up confirmed a substantial percentage (44.3%) of students with poorly-controlled asthma at baseline transitioned to well-controlled by follow-up, representing a subgroup of students who potentially were helped by the program. This finding aligns with the program staff’s focus on working most intensely with students who had the most difficulty managing their asthma.
Findings also revealed intervention district students with asthma exhibited better asthma control (measured by ACQ) than comparison district students with asthma. When controlling for race, age, gender, and grade, the odds of having well-controlled asthma were 54.8% higher for students in the intervention school district than for students in the comparison school district. Together, these findings are consistent with existing scientific literature that supports schools as an important and potentially effective venue for helping students improve asthma management [21, 22].
However, asthma control decreased among some students who started with well-controlled asthma at baseline. Furthermore, analyses examining transition in asthma control classification revealed 17.5% of the students who began the program with well-controlled asthma had poorly-controlled asthma at follow-up. These results suggest students with well-controlled asthma may need additional attention or more intensive staff support to maintain asthma control. This finding is particularly important because the program was following best practice guidance by focusing resources on students with the greatest need [6, 10], and although findings from this one evaluation alone do not warrant a change to existing guidance, they do suggest there may be needs unmet by focusing primarily on students with greatest asthma morbidity. In light of this, it is critical to explore ways to better monitor and assist students with well-controlled asthma so they can maintain that control.
This evaluation highlights the important role of evaluation in understanding and supporting effective interventions. In this school district, existing program records of FEV1 assessments provided valuable data for evaluation analyses, and by introducing new tools such as the ACQ across both intervention and comparison sites, evaluators were able to better assess program impact. The combination of measures incorporating both perceived symptoms and objective assessments of lung function is a strength of this evaluation and could be replicated with relative ease in other school-based programs.
Limitations
A number of limitations should be considered when interpreting this evaluation’s findings. Key limitations include unexpected data collection challenges, variations in response rates, data availability, and characteristics of the design and measures.
Evaluators faced unexpected challenges when original data collection plans were interrupted by unscheduled school closures in the comparison district. When the Mississippi River experienced record levels of flooding in May 2011, both school districts were impacted, but only the comparison district cancelled classes. Evaluators spoke with staff from both districts, asthma experts from the Environmental Protection Agency, and academic experts on the impact of environmental conditions on asthma to gather information about the flood’s potential impact. Input from these people, combined with the discovery the comparison school district had been closed primarily due to road closures rather than homes or schools being flooded, led evaluators to return the following October to collect data from students in the comparison district and re-collect data from students in the intervention district. Consequently, many intervention district students experienced data collection twice. However, questionnaire repetition should not create test-retest bias on most items because the questionnaire specifically asked respondents to consider symptoms in the past week. Test-retest bias could occur on spirometry readings, given that FEV1 and PEF assessments are effort-dependent, but because intervention district students conducted these assessments with nurses regularly, it is unlikely a single additional assessment produced much difference.
A second limitation was the variation in response rates. Response rate was good among intervention students, but was substantially lower among control district students. Response rates were similar across elementary schools in both districts, but response rates in the comparison district’s middle and high schools were lower than in the intervention district. Despite this limitation, the lack of statistically significant differences in age, race, or gender between intervention and control students provided support for using these data in additional analyses.
A third limitation was completeness of existing asthma program records. Program records collected over five years had not been designed for evaluation; consequently, when abstracting data, evaluators discovered substantial amounts of missing data. Because analyses required certain pieces of data, a sizeable group of students (n=127) with missing data was not included in the final analysis sample. Analyses related to missingness revealed differences in age at baseline between students in the analysis sample and those with missing data; however, evaluators decided to continue analyses on the data set because the critical variable reflective of asthma control was not significantly different between groups.
Finally, design characteristics and methods posed additional limitations. Design limitations imposed by initiating an evaluation of a program already underway prevented evaluators from collecting baseline data from a comparison site. Furthermore, completion of the evaluation revealed additional variables that would have been helpful to measure. For example, the evaluators did not measure dosage of individual program components, and therefore, were unable to link changes in asthma control to specific program components. In addition, evaluators did not assess students’ access to health care. Anecdotally, conversations with staff in both districts led evaluators to sense comparison district students had less access to health care than intervention district students despite both groups being in rural areas, but without a related measure, evaluators could not control for access to care in the analyses. Furthermore, evaluators did not include specialized analyses to control for declines in asthma symptoms that may have occurred naturally during childhood [24]. However, pre-post FEV1 assessment analyses did control for age, and cross-sectional findings from the intervention and comparison districts would not have been impacted by any such declines.
Future evaluations could address these limitations and answer additional questions. For example, evaluations should be designed to measure the dosage students receive of individual program components in order to determine which components are associated with improvements in asthma control. Future evaluations also could seek to explain why asthma control decreased among some students who began the program with well-controlled asthma.
Conclusions/key findings
Combined findings, considered in context of limitations, suggest the program model used in the intervention school district led to improvements in asthma control, particularly for students who began the program with poorly-controlled asthma. In addition, findings suggest school-based asthma programs may need additional strategies for staff to track and monitor students who initially had well-controlled asthma in order to ensure they maintain sufficient asthma control.
ACKNOWLEDGMENTS
The authors would like to thank Ngozi Kamalu and Doryn Chervin for their assistance in evaluation design and data collection oversight. We also thank Hollie Huckabee for her assistance in abstraction of data from program records, and Robert Stephens, Ruchika Agrawal, and Ye Xu for their data analysis assistance. We thank Eric Armbrecht, Ben Francisco, Peggy Gaddy, and Cindi Jones for their consultation and assistance throughout the evaluation. And last, but certainly not least, we thank the school districts’ staff, as well as the students with asthma and their families, for their willingness to participate in this evaluation.
DECLARATION OF INTEREST
Technical assistance for this evaluation was funded by the Centers for Disease Control and Prevention’s (CDC) Division of Adolescent and School Health (DASH) under contract #200-2009-30503 with ICF International.
Footnotes
Disclaimer: The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
REFERENCES
- 1.Bloom B, Cohen RA, Freeman G. Hyattsville, MD: 2012. Summary health statistics for U.S. children: National Health Interview Survey, 2011. [PubMed] [Google Scholar]
- 2.Bloom B, Cohen RA, Vickerie JL, Wondimu EA, editors. Summary health statistics for U.S. children: National Health Interview Survey, 2001. Hyattsville, MD: 2003. [PubMed] [Google Scholar]
- 3.National Asthma Education and Prevention Program . Managing asthma: A guide for schools. National Heart, Lung, and Blood Institute; 2003. [Google Scholar]
- 4.American Lung Association Trends in asthma morbidity and mortality. 2012.
- 5.American Lung Association Asthma and children fact sheet: American Lung Association. 2012 [updated October 2012; cited 2013 June 28]. Available from: http://www.lung.org/lung-disease/asthma/resources/facts-and-figures/asthma-children-fact-sheet.html.
- 6.Centers for Disease Control and Prevention . Strategies for addressing asthma within a coordinated school health programs, with updated resources. Atlanta, Georgia: 2006. [cited 2013 July 1]. Available from: http://www.cdc.gov/healthyyouth/asthma/strategies/asthmacsh.htm. [Google Scholar]
- 7.American Lung Association Open Airways for Schools Washington, DC: American Lung Association. 2013 [cited 2013 July 1]. Available from: http://www.lung.org/lung-disease/asthma/in-schools/open-airways/open-airways-for-schools-1.html.
- 8.United States Environmental Protection Agency Creating healthy indoor environments in schools: United States Environmental Protection Agency. [cited 2013 July 1]. Available from: http://www.epa.gov/iaq/schools/index.html.
- 9.Rossier Markus A, Lyon M, Rosenbaum S. Changing pO2licy: The elements for improving childhood asthma outcomes. The George Washington University School of Public Health and Health Services; 2010. [Google Scholar]
- 10.Wheeler LS, Merkle SL, Gerald LB, Taggart VS. Managing asthma in schools: Lessons learned and recommendations. J Sch Health. 2006 Aug;76(6):340–4. doi: 10.1111/j.1746-1561.2006.00125.x. [DOI] [PubMed] [Google Scholar]
- 11.U.S. Census Bureau . American fact finder: Selected economic characteristics, 2007-2011 American Community Survey 5-year estimates. Pemiscot County, Missouri: [cited 2013 July 1]. Available from: http://factfinder2.census.gov/bkmk/table/1.0/en/ACS/11_5YR/DP03/0500000US29155%7C0400000US29. [Google Scholar]
- 12.U.S. Census Bureau . American fact finder: Selected economic characteristics, 2007-2011 American Community Survey 5-year estimates. Dunklin County, Missouri: [cited 2013 July 1]. Available from: http://factfinder2.census.gov/bkmk/table/1.0/en/ACS/11_5YR/DP03/0500000US29069%7C0400000US29. [Google Scholar]
- 13.ICF International . Asthma management program at Kennett Public Schools: Rapid evaluation final report. Atlanta, GA: 2013. [Google Scholar]
- 14.Krishna S, Francisco BD, Balas EA, Konig P, Graff GR, Madsen RW. Internet-enabled interactive multimedia asthma education program: A randomized trial. Pediatrics. 2003;111(3):503–510. doi: 10.1542/peds.111.3.503. [DOI] [PubMed] [Google Scholar]
- 15.National Heart, Lung, and Blood Institute . Expert panel report 3: Guidelines for the diagnosis and management of asthma. Rockville, MD: 2007. [cited 2013 July 1]. Available from: http://www.nhlbi.nih.gov/guidelines/asthma/asthgdln.pdf. [Google Scholar]
- 16.Juniper EF, Gruffydd-Jones K, Ward S, Svensson K. Asthma Control Questionnaire in children: Validation, measurement properties, interpretation. Eur Respir J. 2010;36(6):1410–6. doi: 10.1183/09031936.00117509. [DOI] [PubMed] [Google Scholar]
- 17.Juniper EF, Bousquet J, Abetz L, Bateman ED, The GOAL Committee Identifying 'well-controlled' and 'not well-controlled' asthma using the Asthma Control Questionnaire. Respir Med. 2006;100:616–21. doi: 10.1016/j.rmed.2005.08.012. [DOI] [PubMed] [Google Scholar]
- 18.Juniper EF, O'Byrne PM, Guyatt GH, Ferrie PJ, King DR. Development and validation of a questionnaire to measure asthma control. Eur Respir J. 1999;14(4):902–7. doi: 10.1034/j.1399-3003.1999.14d29.x. [DOI] [PubMed] [Google Scholar]
- 19.Hankinson J, Odencrantz J, Fedan K. Spirometric reference values from a sample of the general U.S. population. Am J Resp Crit Care Med. 1999;159:179–87. doi: 10.1164/ajrccm.159.1.9712108. [DOI] [PubMed] [Google Scholar]
- 20.Bateman ED, Boushey H, Bousquet J, Busse W, Clark T, Pauwels R, et al. Can guideline-defined asthma control be achieved? The Gaining Optimal Asthma Control Study. Am J Respir Crit Care Med. 2004;170:836–44. doi: 10.1164/rccm.200401-033OC. [DOI] [PubMed] [Google Scholar]
- 21.Bruzzese J-M, Evans D, Kattan M. School-based asthma programs. J Allergy Clin Immunol. 2009;124(2):195–200. doi: 10.1016/j.jaci.2009.05.040. [DOI] [PubMed] [Google Scholar]
- 22.Coffman JM, Cabana MD, Yelin EH. Do school-based asthma education programs improve self-management and health outcomes? Pediatrics. 2009;124(2):729–42. doi: 10.1542/peds.2008-2085. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Halterman JS, Fagnano M, Montes G, Fisher S, Tremblay P, Tajon R, et al. The school-based preventive asthma care trial: Results of a pilot study. J Pediatr. 2012;161(6):1109–15. doi: 10.1016/j.jpeds.2012.05.059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Ko YA, Song PS, Clark NM. Declines with age in childhood asthma symptoms and health care use. An adjustment for evaluations. Ann Am Thorac Soc. 2014;11(1):54–62. doi: 10.1513/AnnalsATS.201304-093OC. [DOI] [PMC free article] [PubMed] [Google Scholar]