Abstract
Objective
To determine the effectiveness of the Spirometry 360™ distance learning quality improvement (QI) program for enhancing the processes and outcomes of care for children with asthma.
Methods
Cluster randomized controlled trial involving 25 matched pairs of pediatric primary care practices. Practices were recruited from two practice-based research networks: the Slone Center Office-based Research Network at Boston University, Boston, MA and the Puget Sound Pediatric Research Network in Seattle, WA. Study participants included providers from one of the 50 enrolled pediatric practices and 626 of their patients with asthma. Process measures assessed included: spirometry test quality and appropriate prescription of asthma controller medications. Outcome measures included: asthma-specific health-related quality of life, and outpatient, emergency department and inpatient utilization for asthma.
Results
At baseline, 25.4% of spirometry tests performed in control practices and 50.4% of tests performed in intervention practices were of high quality. During the six month post-intervention period, 28.7% of spirometry tests performed in control practices and 49.9% of tests performed in intervention practices were of high quality. The adjusted difference-of-differences analysis revealed no intervention effect on spirometry test quality. Adjusted differences-of-differences analysis also revealed no intervention effect on appropriate use of controller medications or any of the parent/patient-reported outcomes examined.
Conclusion
In this study, the Spirometry 360™ distance learning QI program was ineffective in improving spirometry test quality or parent/patient-reported outcomes. QI programs like the one assessed here may need to focus on practices with lower baseline performance levels or be tailored for those with higher baseline performance.
Keywords: Asthma, Spirometry, Outcomes, Quality Improvement
Introduction
Asthma affects an estimated 6 million children in the United States (U.S.).1 Rates of emergency department use for asthma are on the rise2 with three million children suffering a severe exacerbation annually1 resulting in 113,840 pediatric hospitalizations, costing approximately $433 million.3 Thus, it is crucial to maximize the quality of care provided for this condition. According to the National Asthma Education and Prevention Program Expert Panel Report-3 (EPR-3) and the Global Initiative for Asthma 2017 report, spirometry tests should be performed at the initial visit for asthma, after treatment is started, during periods of progressive loss of asthma control, and at least 1–2 times annually depending on response to therapy.4,5 However, fewer than one-fifth of pediatric primary care providers routinely perform spirometry as part of their asthma diagnosis and assessment.6–8 Providers cite a lack of appropriate training in performing and interpreting spirometry tests as a key barrier to its routine use.7,9,10 Addressing this training deficit is the main focus of the Spirometry 360™ distance learning quality improvement (QI) program.11 By enhancing knowledge, skills, and self-efficacy around the performance and interpretation of spirometry tests, the program attempts to address barriers to routinely utilizing spirometry for patients with asthma.
The objectives of this study were to determine the effectiveness of the Spirometry 360™ program on: 1) increasing the quality of spirometry tests performed; 2) increasing appropriate prescription of asthma controller medications; and 3) improving outcomes for children with asthma.
Methods
Practice Participants
We conducted a cluster randomized controlled trial (cRCT) among 50 pediatric practices located throughout the U.S. Practices were recruited from two practice-based research networks: the Slone Center Office-based Research Network (SCOR) at Boston University and the Puget Sound Pediatric Research Network (PSPRN) in Seattle, WA.
SCOR and PSPRN practices were sent standard mail or email study invitations. Eligible practices had to exclusively serve pediatric patients and have internet access. Additionally, a minimum of one person who orders and interprets spirometry tests at the practice (subsequently called providers) and one person who coaches patients through the tests (subsequently called coaches) had to agree to study participation. Providers could be physicians or nurse practitioners and coaches could be physicians, nurse practitioners, nurses, medical assistants, or physician assistants.
SCOR and PSPRN research staff identified pairs of practices matched on the following criteria: number of pediatric providers in the practice (+/− 1 for practices with ≤ 4 providers; and +/− 2 for practices with > 5 providers); percent of Medicaid-eligible patients (+/− 20%); and type of practice (solo private, group private, community clinic/community health center, or other). Each practice pair was randomized so that one practice was assigned to the intervention group and one was assigned to the control group. Practices were blinded to the other practice pair member. If a matched practice could not be identified for an interested provider-coach pair, their practice was ineligible for study participation.
Parent and Patient Participants
Prior to randomization, using billing data, all practices identified a list of children 5 to 16 years-old seen by the participating providers for asthma during the prior 12 months. The parents of identified children were sent a letter describing the study, and given the opportunity to “opt out” with a refusal card. If the opt out card was not received, the research team contacted the parents to determine eligibility and enroll them into the study. Families were eligible if their child had asthma according to a validated screening questionnaire12, was between 5 and 16 years-old, and they reported having internet access. Eligible families who verbally agreed to participate were sent a written consent form to review, sign, and return. Family recruitment calls for each enrolled practice continued until 30 eligible families were enrolled or until study enrollment ended, whichever occurred first.
All study procedures were reviewed and approved by the Seattle Children’s Hospital and Boston University Medical Center Institutional Review Boards.
Study Procedures: Pediatric Practices
Each participating provider and coach in a practice completed a practice enrollment survey that collected data on their level of personal experience using spirometry (LOW = ≤2 months experience or ≤ 10 tests performed in the last year or ≤ 25 tests performed during professional career versus HIGH = > 10 tests performed in the past year or > 25 tests performed during professional career). Information was also collected on practice size (1, 2–4, 5–10, or >10 providers), practice type, and how long the practice had been using spirometry (never, ≤ 6 months, ≤ 1 year, > 1 year),
Prior to randomization, practices received an ndd EasyOne spirometer and software from the University of Washington (UW) based Spirometry 360™ program11 which facilitated the transmission of de-identified spirometry testing data to the project’s central spirometry database. Training and user support for the EasyOne spirometer was delivered by ndd sales representatives to all participating practices using their standard protocol.
After randomization, both intervention and control practices electronically transmitted their spirometry test data on a weekly basis to the project’s central database. The decision to perform a spirometry test on a patient was at the discretion of his/her provider and was not dictated by the research protocol. After all practices (both control and intervention) transmitted spirometry data for two months (baseline data collection period), intervention practices began the Spirometry 360™ training program. For intervention practices only, spirometry test quality was assessed for all data transmitted during the four-month intervention to facilitate the production of feedback reports on their performance.
The Spirometry 360™QI Program
This intervention is a multifaceted four-month long training program, where providers and coaches are trained in skills to perform and interpret spirometry tests and then use the results to aid in the management of their patients with asthma. The training includes 1–2 hours of Spirometry Fundamentals™ viewing, 3 hours of training during two interactive web-based Spirometry Learning Lab sessions over a 4–5 week period, and 30 minutes/month to review tailored feedback reports from spirometry experts.
Spirometry Fundamentals™ is a web-based, self-paced training tool that includes ten short individual learning modules (2 to 11 minutes each) on how to properly obtain and interpret spirometry tests.
The Spirometry Learning Labs are delivered using WebEx which is a computer-based virtual meeting tool that allows interactive image-based teaching along with conference calls. The learning sessions reinforce principles introduced in Spirometry Fundamentals™ through case-based examples and emphasize using symptom frequency (day and night), exacerbation history, controller mediation use history, and lung function together when determining both asthma severity and control. The importance of planned asthma visits is also reviewed.
Spirometry Feedback reports include performance measures, stratified into two pediatric age groups (5–7 years-old and ≥ 8 years-old) and a summary measure that includes all age groups. The performance measures include the proportion of spirometry tests that are of high quality, reasons for test failure, and example screen shots. The quality performance measure employs a grading scheme based on criteria set forth by the American Thoracic Society (ATS) and European Respiratory Society.13 Spirometry tests with grades of A or B are considered high quality and grades of C, D or F are considered low quality. De-identified spirometry data are uploaded from the point of care for expert review and feedback. For this study, a trained certified respiratory therapist (RT) who was blinded to study group assignment assessed the spirometry tests submitted by practices and assigned grades according to the ATS criteria. In cases where the RT was unsure of how to grade a test, the case was flagged and reviewed by one of the course faculty members. The RT and faculty member would then come to consensus on how to grade the test.
After the four-month Spirometry 360™ intervention was completed, all practices (intervention and control) continued to send spirometry test data to the UW database for an additional 6 months. No feedback reports were provided to any of the practices during this period.
Study Procedures: Parents and Children
Participating parents of children 5 to 16 years-old were asked to complete an online survey at three time points: at enrollment (pre-intervention) and during months 6 and 12 (i.e. immediately following and 6 months post-intervention). Children aged 8 to 16 years-old who provided assent were also asked to complete online surveys at the same three time points. Surveys were offered in English or Spanish.
Family Survey
Using a 6-month look-back period, the survey collected process and outcome measures including: whether spirometry testing had been performed, whether a controller medication had been prescribed, the child’s asthma-specific health related quality of life (HRQOL) using the Pediatric Quality of Life Inventory Asthma Module Short Form Version 3.0 (PedsQL™ 3.0 SF-22 Asthma Module),14 and the number of planned and unplanned outpatient visits, ED visits, and hospitalizations for asthma. The survey also assessed daytime symptoms during the past two weeks and night-time symptoms during the past month. Demographic variables were only collected at enrollment and included: child gender, race/ethnicity , age, asthma severity (based EPR-3 criteria4), parent education level and age, annual household income, and medical insurance type.
Appropriate Prescription of Controller Medications
Appropriate prescription of controller medications was assessed using parent survey reports of asthma severity and receipt of controller medication prescriptions. Asthma severity was assessed using the EPR-3 day and night-time symptom criteria,4 activity limitation (not at all limited, slightly limited, moderately limited, severely limited), and frequency of quick relief medication use (≤ 2 days/week, > 2 days/week but not daily, daily but ≤ 2 times/day, daily and > 2 times/day). Since child asthma severity was dynamic over time, we focused on the inappropriate under-prescription of controller medication. Appropriate controller medication prescription was defined as parent reports of being prescribed a controller medication for children deemed to have mild to severe persistent asthma. To assess the validity of parent reports, a sample of 327 medical records for study subjects (~50% random sample from intervention and control arms) were abstracted to compare parent reports of controller medications prescribed to the provision of prescriptions according to medical record documentation. The same was done to examine validity of parent/patient reports of spirometry tests being performed. Sensitivity and specificity of parent reports were calculated using the abstracted medical records data as the gold standard. Sensitivity and specificity of parental reports of receiving controller medication prescriptions were 86% (241/280) and 45% (24/53), respectively. Sensitivity and specificity of parental reports for spirometry tests being performed were 88% (56/65) and 67% (138/205). Overall, the level of agreement for both aspects of care was 79%.
Analytic Methods
We conducted two sets of analyses to address the study objectives. The first set examined intervention effects on spirometry test quality, in which individual spirometry tests were the unit of analysis. The second set examined intervention effects on various processes and outcomes, in which patients were the unit of analysis. Mixed effects regression models were used for the analyses, with various random effects to account for the clustering at different levels (e.g., patient, practice) in order to avoid under-estimation of standard errors and artificially significant p-values.15 Hypothesis tests were considered statistically significant if they produced two-sided p-values < 0.05. We performed intention-to-treat comparisons of intervention and control groups for all analyses including all study practices, regardless of their level of participation in the Spirometry 360™ program.
To test whether exposure to the Spirometry 360™ program increased the likelihood of high quality spirometry test performance we constructed a hierarchical mixed effects logistic regression model. The outcome was the binary indicator of whether or not a spirometry test was assigned a quality grade of A or B. Predictors included treatment assignment (intervention/control), study period (pre versus post intervention), treatment-by-period interaction, child age group (5–12 vs. 13–16 years) and gender. We included multiple random effects (practice and practice-within-pair) to account for clustering within practice and practice pair matching. The primary predictor was the interaction term between study-arm and period allowing for a difference-of-differences evaluation of intervention effectiveness on spirometry test quality.
To assess whether the intervention increased the frequency of appropriate controller medication prescription, we constructed a hierarchical mixed effects logistic regression model. The model included an interaction term between study-arm and period (pre and post-intervention) as the primary predictor of interest allowing for a difference-of-differences evaluation, and random effects to account for correlations due to clustering within practices and repeated measures within patients.
Assessment of whether patients receiving care in intervention practices reported better outcomes, involved fitting a series of hierarchical models. Outcomes considered included: PedsQL™ 3.0 SF-22 Asthma Module 11-item symptoms and 11-item treatment scale scores (0–100 scale); measures of unplanned healthcare utilization for asthma including outpatient, ED and inpatient visits. For this series of models, the primary predictor was the interaction term between study-arm and period allowing for a difference-of-differences evaluation of intervention effectiveness on these outcomes. For patients ≥ 12 years-old, we used patient-reported PedsQL data when available and parent data when patient-reported data were missing for a given time-point.
A priori power calculations indicated that a sample of 22 practices per study arm with an average of 27 patients enrolled per practice would yield 86% power (alpha 0.05) to detect a 25 percentage point increase in the rate of high quality spirometry tests performed between the baseline (pre-) and post-intervention periods in the intervention arm versus (vs) no change in the control arm, 92% power to detect a 12.5% increase from a 70% baseline rate of appropriate controller medication prescription in the intervention arm vs no change in the control arm, >95% power to detect a 3-point increase in PedsQL™ scores from a baseline of 85 in the intervention arm vs no change in the control arm, an 11% decrease in ED utilization, and a 15% decrease in unplanned outpatient asthma visits and hospitalizations in the intervention arm vs no change in the control arm.
Results
Study invitation letters were sent to 161 practices in the SCOR and PSPRN networks. When called by study staff, 50 practices enrolled, 14 were deemed ineligible, 29 declined, and 68 could not be successfully contacted after multiple attempts resulting in a practice response rate of 34% (50 /147). Practices provided names and contact information for 22–196 (mode = 62) patients who were between 5–16 years-old and had at least one visit with the billing code 493.xx during the prior 12 months. Of 3409 families with potentially eligible children sent study invitation letters, 308 (9%) chose to opt out of the study. Among the 3101 not opting out, 1733 (56%) were successfully contacted by research staff and screened for eligibility. Of those screened, 303 (17%) were ineligible and 652 consented to participate for an eligible response rate of 46% (See Figure 1, Consort Diagram). Thus we enrolled patients from all 50 practices with an average of 12 patients/practice (range 1–33).
Figure 1.
Consort Diagram
Of the 50 enrolled practices, one intervention practice dropped out before any spirometry data could be collected from them and three control sites did not submit any post-intervention spirometry data, leaving 22 control sites and 24 intervention sites for the spirometry quality analysis. Because we conducted intention-to-treat analyses, data contributed by patients enrolled from practices not included in the spirometry quality analysis were still included in all patient-based analyses.
All 25 intervention practices were exposed to at least two of the three Spirometry 360™ QI program components. Sixty-four percent (16/25) of the practices participated in all aspects of the program. Another 16% (4/25) missed one of the two Spirometry Learning Lab webinars, but otherwise participated in all program components.
Patients sampled from intervention and control practices were similar in terms of their age, race/ethnicity, gender distribution, and asthma severity. Parent characteristics were also similar between the two study groups (Table 1).
Table 1.
Patient and Family Characteristics
| Patient and Family Baseline Characteristics* | Control (N = 328) | Intervention (N = 298) |
|---|---|---|
|
|
||
| N = 626 | N (%) | N (%) |
|
| ||
| Child gender | ||
| - Male | 197 (60) | 166 (56) |
|
| ||
| Child race/ethnicity | ||
|
| ||
| - Hispanic/Latino | 22 (7) | 24 (8) |
| - White | 225 (68) | 199 (66) |
| - African-American | 35 (11) | 35 (12) |
| - Other/Multiracial | 44 (15) | 38 (15) |
|
| ||
| Asthma severity | ||
|
| ||
| - Intermittent | 118 (36) | 115 (39) |
| - Mild persistent | 128 (39) | 103 (35) |
| - Moderate persistent | 55 (17) | 42 (14) |
| - Severe persistent | 21 (6) | 32 (11) |
|
| ||
| Child age; mean (SD) | 9.3 (3.0) | 9.5 (3.1) |
|
| ||
| Parent education level | ||
|
| ||
| - High school of less | 51 (16) | 37 (12) |
| - Some college | 120 (37) | 118 (40) |
| - 4-year degree or higher | 152 (46) | 137 (46) |
|
| ||
| Parent age | ||
|
| ||
| - 18–34 | 78 (24) | 67 (23) |
| - 35–44 | 161 (49) | 152 (51) |
| - 45+ | 84 (26) | 75 (25) |
|
| ||
| Household income | ||
|
| ||
| - < 15k | 18 (6) | 13 (4) |
| - 15–30k | 40 (12) | 36 (12) |
| - 30–50k | 58 (18) | 45 (15) |
| - >50k | 204 (62) | 197 (66) |
|
| ||
| Insurance type | ||
|
| ||
| - Private | 234 (71) | 210 (71) |
| - Public | 76 (23) | 63 (21) |
| - Other | 6 (2) | 7 (2) |
| - None | 5 (2) | 11 (4) |
The distribution of these characteristics did not vary significantly between control and intervention. Cell totals may not add to 100% due to missing responses.
Intervention and control practices were similar regarding number of providers and staff, having ≥ 1 staff member with high levels of spirometry experience at baseline, percent of patients publicly insured, and geographic location (Table 2).
Table 2.
Practice Characteristics
| Practice Characteristics * | Control (N = 25) | Intervention (N = 24) |
|---|---|---|
|
|
||
| N = 49 | N (%) | N (%) |
|
| ||
| Number of recruited patients | ||
| - Mean (SD) | 13 (6) | 12 (7) |
| - (min, max) | (4, 33) | (1, 26) |
| - IQR | (9–15) | (6–16) |
|
| ||
| Practice type | ||
| - Solo private | 8 (32) | 6 (25) |
| - Group private | 16 (64) | 16 (67) |
| - Community clinic or health center | 0 (0) | 1 (4) |
| - Other | 1 (4) | 1 (4) |
|
| ||
| Number of physicians and nurse practitioners | ||
| - 1 | 7 (28) | 3 (13) |
| - 2–4 | 9 (36) | 6 (25) |
| - 5–10 | 8 (32) | 9 (38) |
| - >10 | 1 (4) | 6 (25) |
|
| ||
| Number of medical assistants and nurses | ||
| - 1 | 1 (4) | 1 (4) |
| - 2–4 | 11 (44) | 11 (46) |
| - 5–10 | 11 (44) | 6 (25) |
| - >10 | 2 (8) | 6 (25) |
|
| ||
| How long practice has been using spirometry | ||
| - Never | 13 (52) | 12 (50) |
| - 6 months | 1 (4) | 1 (4) |
| - 1 year | 1 (4) | 1 (4) |
| - >1 year | 10 (40) | 10 (42) |
|
| ||
| Practice with ≥ 1 staff member having a high level of spirometry experience at baseline** | ||
|
| ||
| % of practice patients insured by Medicaid*** | 9 (36) | 12 (50) |
|
| ||
| - > 0–10% | 8 (32) | 7 (28) |
| - 11–20% | 5 (20) | 6 (24) |
| - 21–39% | 6 (24) | 4 (16) |
| - 40–75% | 6 (24) | 8 (32) |
|
| ||
| Network | ||
|
| ||
| PSPRN | 1 (4) | 3 (12) |
| SCOR | 24 (96) | 22 (88) |
|
| ||
| Geographic Setting | ||
|
| ||
| Rural | 6 (24) | 3 (12) |
| Suburban | 11 (44) | 12 (48) |
| Urban | 8 (32) | 10 (40) |
The distribution of these characteristics did not vary significantly between control and intervention.
High level of experience = > 10 tests performed in the past year or > 25 tests performed during professional career.
This estimate includes all practice patients not just those enrolled in the study.
Spirometry Testing Quality
During the two-month pre-intervention period, 25.4% of spirometry tests performed in control practices received grades of A or B, while 50.4% of tests performed in intervention practices received A or B grades. During the six month post-intervention period, 28.7% of spirometry tests performed in control practices were graded A or B, while 49.9% of tests performed in intervention practices received A or B grades. The adjusted difference-of-differences analysis revealed no intervention effect on the frequency of high quality spirometry tests performed by intervention practices after exposure to the Spirometry 360™ program (Table 3).
Table 3.
Quality of Spirometry Testing
| Outcome | Control | Intervention | Adjusted Difference of Differences | ||||
|---|---|---|---|---|---|---|---|
| Pre3 (n=410) | Post3 (n=884) | p-value1 | Pre3 (n=719) | Post3 (n=1103) | p-value1 | p-value2 | |
| Spirometry tests with passing grade (AB criterion): N (%) | 104 (25.4%) | 254 (28.7%) | 0.21 | 362 (50.4%) | 550 (49.9%) | 0.84 | 0.71 |
P-values are based on Chi-squared test comparing pre and post-intervention periods within each arm.
P-value is for the study group (intervention/control)-by-period interaction term in a mixed effects logistic regression model, controlling for age group (5–12 vs. 13–16 years), and gender. Within matched-pair correlations and within-site correlations were accounted for via random effects.
Pre-period for control practices: Jan–Apr, 2011; Pre-period for intervention practices Jan–Feb, 2011; Post-period for both control and intervention practices: Jul–Dec 2011.
Spirometry Testing Frequency
Patients obtaining asthma care from intervention practices were no more likely to have a spirometry test performed than were patients cared for in control practices. At 6 to 12 months after enrollment in the study, 55% and 51% of patients from control practices and 53% and 57% from intervention practices had undergone spirometry testing, respectively (Table 4).
Table 4.
Processes and Outcomes of Care by Study Arm and Study Period
| Total N | Month | Control | Intervention | Adjusted Difference of Differences* |
|---|---|---|---|---|
|
| ||||
| Baseline | 328 | 298 | ||
| 6-month | 286 | 269 | ||
| 12-month | 267 | 255 | ||
| Processes of Care | p-value | |||
|
| ||||
| % appropriately prescribed a controller medication | Baseline | 65 | 57 | |
| 6-month | 86 | 86 | 0.23 | |
| 12-month | 87 | 86 | 0.44 | |
|
| ||||
| % reporting any spirometry test in past 6 months | Baseline | 46 | 42 | |
| 6-month | 55 | 51 | 0.67 | |
| 12-month | 53 | 57 | 0.07 | |
|
| ||||
| Outcomes of Care | ||||
|
| ||||
| symptom days in past 2 weeks; mean (SD) | Baseline | 3.6(4.1) | 3.5(4.0) | |
| 6-month | 2.7(2.6) | 2.6(2.5) | 0.87 | |
| 12-month | 2.8(3.1) | 3.0(3.5) | 0.39 | |
|
| ||||
| % reporting "0" days with day symptoms in past 2 weeks | Baseline | 37 | 37 | |
| 6-month | 44 | 46 | 0.45 | |
| 12-month | 40 | 42 | 0.56 | |
|
| ||||
| % reporting to "never" wake at night with trouble breathing in the past month | Baseline | 46 | 48 | |
| 6-month | 52 | 56 | 0.53 | |
| 12-month | 60 | 60 | 0.98 | |
|
| ||||
| PedsQL asthma symptoms subscale score mean (SD)** | Baseline | 66.1(18.2) | 67.7(18.0) | |
| 6-month | 72.7(18.1) | 71.9(19.8) | 0.26 | |
| 12-month | 73.7(17.5) | 73.8(18.6) | 0.47 | |
|
| ||||
| PedsQL asthma treatment problems subscale score mean (SD)** | Baseline | 83.9(14.3) | 85.2(15.4) | |
| 6-month | 87.4(12.8) | 88.3(13.8) | 0.94 | |
| 12-month | 86.9(12.9) | 87.5(13.9) | 0.6 | |
|
| ||||
| % classified as "intermittent" severity | Baseline | 37 | 39 | |
| 6-month | 50 | 47 | 0.27 | |
| 12-month | 49 | 51 | 0.74 | |
|
| ||||
| Hospitalizations in past 6 months; mean (SD) | Baseline | 0.07 (0.7) | 0.07 (0.4) | |
| 6-month | 0.04 (0.4) | 0.004 (0.06) | 0.06 | |
| 12-month | 0.02 (0.1) | 0.02 (0.1) | 0.77 | |
|
| ||||
| Unplanned outpatient visits in past 6 months; mean (SD) | Baseline | 1.22 (0.6) | 1.26 (0.5) | |
| 6-month | 1.08 (0.3) | 1.06 (0.2) | 0.45 | |
| 12-month | 1.07 (0.3) | 1.15 (0.4) | 0.74 | |
|
| ||||
| Emergency Department visits in past 6 months; mean (SD) | Baseline | 0.24 (0.9) | 0.19 (0.7) | |
| 6-month | 0.17 (1.0) | 0.06 (0.3) | 0.23 | |
| 12-month | 0.06 (0.3) | 0.08 (0.4) | 0.61 | |
|
| ||||
| Planned outpatient visits in past 6 months; mean (SD) | Baseline | 0.74 (0.7) | 0.66 (0.7) | |
| 6-month | 0.52 (0.6) | 0.48 (0.5) | 0.56 | |
| 12-month | 0.48 (0.6) | 0.46 (0.6) | 0.33 | |
P-value is for the study group (intervention/control)-by-period interaction term in a mixed effects logistic regression model. Within matched-pair correlations and within-site correlations were accounted for via random effects.
In a prior study children with mild intermittent, mild persistent, or moderate/severe persistent asthma had average PedsQL™ 3.0 Asthma Symptom subscale scores of 84.0 (standard deviation [SD] 13.5), 64.8 (SD 14), and 50.5 (SD 18.1) respectively on the 0–100 scale. Scores on the SF-22 Treatment subscale were 92.3 (SD 9.3), 84.1 (14.0), and 79.6 (16.5) in these three groups respectively.14 The minimal clinically important difference (MCID) for scores on the Asthma Symptoms subscale is 8.96 points for child self-report and 8.18 points for parent proxy report on the 0–100 scale.17 The MCID for the Asthma Treatment subscale has not previously been determined.
Appropriate Controller Medication Prescription
Pre-intervention, 65% of children with mild to severe persistent asthma from control practices were being appropriately prescribed controller medications, while 57% of children with persistent asthma from intervention practices had been prescribed a controller medication. At 6 and 12 months after study enrollment, a majority of children with persistent asthma from both control and intervention practices had been prescribed controller medications (Table 4). Adjusted analysis revealed no intervention effect on the frequency of appropriate controller medications prescribing.
Asthma Care Outcomes
Obtaining asthma care from an intervention practice did not result in any improvements in the following outcomes at 6 and 12 months after enrollment: parent/patient reported daytime symptoms during the past two weeks, night-time symptoms during the past month, asthma-specific HRQOL, number of planned or unplanned outpatient visits, ED visits, or hospitalizations for asthma during the prior 6 months (Table 4).
Discussion
This cRCT did not demonstrate positive effects of the Spirometry 360™ QI program on asthma processes or outcomes of care in this group of pediatric practices. The lack of effect on quality of spirometry testing was surprising given the results of a prior cRCT which demonstrated a positive effect on testing quality during participation in a similar distance learning spirometry QI program.16 However, in that trial, the number of tests performed was substantially smaller than in the current trial (1028 versus 3116 tests) and there were no post-intervention assessments of spirometry quality; all assessments were performed at baseline and during intervention participation. It is possible that had we assessed spirometry test quality during intervention participation, we may have observed differences in performance between control and intervention practices that waned over time and were no longer present 6 to 12-months after the conclusion of QI program participation.
Another possible explanation for the lack of intervention effect observed on spirometry quality was the imbalance between control and intervention practices regarding the quality of their pre-intervention spirometry performance with only 25% of control practice tests versus 50% of intervention practice tests receiving A or B grades respectively. This study was powered to detect an improvement of 25 percentage points in the rate of performing high quality spirometry tests, thus the lack of balance in this characteristic between control and intervention practices likely diminished our ability to find an intervention effect. It is possible that the Spirometry 360™ QI program has a “ceiling effect” in terms of how much improvement is achievable for a given practice. If a practice starts at the level of 50% of their spirometry tests being high quality before program participation, they may not obtain the same benefits as practices starting at a level of 25%. The program may be most effective for those practices that are spirometry naïve.
This study was novel in that we attempted to examine the effects of the Spirometry 360™ QI intervention not only on the process of performing spirometry tests, but also on patient-centered outcomes. We observed a marked improvement in several parent/patient-reported care processes and outcomes assessed between the baseline and 6-month follow-up surveys and these improvements were sustained at the 12-month follow-up time-point (Table 4). However, these improvements were similar in both the control and intervention groups. Although all surveys administered had a 6-month recall period, it is possible that parents and patients reflected further back than this for the baseline assessment in terms of assessing asthma-specific HRQOL and heath care utilization.
The lack of intervention effect on outcomes may also be due to their being too distal from the key processes of care that the QI intervention aimed to improve. However, the targeted care processes: performance of higher quality spirometry testing and more appropriate prescription of controller medications, were not observed to change as a result of the intervention. So it is not surprising that the outcomes assessed also did not change as a function of the intervention.
This study has several limitations. The participating practices belonged to practice-based research networks having primarily white, privately insured, non-poor patients who largely had mild intermittent (37%) or mild persistent (37%) asthma. It is possible the Spirometry 360™ QI program may have been more effective in a higher risk population of children with asthma. The study may have been underpowered to identify true effects of the Spirometry 360™ QI program on the quality of spirometry testing given the lack of balance for this characteristic between the control and intervention practices at baseline and potential ceiling effects of intervention effectiveness. Future studies attempting to assess the effectiveness of such QI programs should collect baseline spirometry performance data and block-randomize practices based on this characteristic. Provider knowledge, skills, and self-efficacy around performance and interpretation of spirometry tests were not assessed; thus we do not know if participation in the QI program altered these provider characteristics. Overall patient/family participation rates were low. Because we were only able to recruit an average of 12 patients/practice, we may have been under-powered to detect differences in outcomes between the study arms.
Conclusions
In this study, the Spirometry 360™ distance learning QI program was ineffective in improving the quality of spirometry testing or patient-reported outcomes. Unfortunately, the lack of balance between the practice groups in baseline spirometry performance limited our ability to assess this primary outcome, which underscores the importance of ascertaining performance of key care processes prior to randomization, and utilizing randomization methods that ensure balance. QI programs like the one assessed here may need to focus on practices with lower baseline levels of performance or be tailored to be effective for practices with higher baseline levels of performance.
What’s New?
The Spirometry 360™ distance learning QI program was ineffective in changing provider spirometry performance or asthma outcomes. Further tailoring of the intervention is likely necessary to enhance asthma management and outcomes.
Acknowledgments
We would also like to acknowledge the contributions of Megan Fesinmeyer, PhD, MPH, former data analyst at Seattle Children’s Research Institute, who conducted the data analysis for this study under the direction of author Dr. Chuan Zhou, Statistician, Center for Child Health, Behavior, and Development. This study was funded by the National Heart, Lung, Blood, Institute, grant # 1R01HL094579-01A1, Principal Investigator: Rita Mangione-Smith. The funders had no role in the study design; in the collection, analysis, or interpretation of data; in the writing of the report; nor in the decision to submit the article for publication.
Funding: This study was funded by the National Heart, Lung, Blood, Institute, grant # 1R01HL094579-01A1, Principal Investigator: Rita Mangione-Smith. The funders had no role in the study design; in the collection, analysis, or interpretation of data; in the writing of the report; nor in the decision to submit the article for publication.
Footnotes
Potential Conflicts of Interest: The authors have no conflicts of interest relevant to this article to disclose.
Trials Registration: Name: Improving Asthma Outcomes through Spirometry Distance Learning, URL: clinicaltrials.gov, Identifier: NCT01168635
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.CDC. [Accessed 12/20/2016];Data, statistics, and surveillance: Most recent asthma data. https://www.cdc.gov/asthma/most_recent_data.htm.
- 2.Nath JB, Hsia RY. Children's emergency department use for asthma, 2001–2010. Acad Pediatr. 2015;15(2):225–230. doi: 10.1016/j.acap.2014.10.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Leyenaar JK, Ralston SL, Shieh MS, Pekow PS, Mangione-Smith R, Lindenauer PK. Epidemiology of pediatric hospitalizations at general hospitals and freestanding children's hospitals in the United States. J Hosp Med. 2016;11(11):743–749. doi: 10.1002/jhm.2624. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.National Asthma E, Prevention P. Expert Panel Report 3 (EPR-3): Guidelines for the diagnosis and management of asthma-summary report 2007. J Allergy Clin Immunol. 2007;120(5 Suppl):S94–138. doi: 10.1016/j.jaci.2007.09.043. [DOI] [PubMed] [Google Scholar]
- 5.Global Inititative for Asthma. [Accessed June 2017];Global strategy for asthma management and prevention, updated 2017. http://ginasthma.org/2017-gina-report-global-strategy-for-asthma-management-and-prevention.
- 6.Cabana M, Slish KK, Nan B, Leo H, Bratton SL, Dombkowski KJ. Outcomes associated with spirometry for pediatric asthma in a managed care organization. Pediatrics. 2006;118(1):e151–156. doi: 10.1542/peds.2005-2352. [DOI] [PubMed] [Google Scholar]
- 7.Dombkowski KJ, Hassan F, Wasilevich EA, Clark SJ. Spirometry use among pediatric primary care physicians. Pediatrics. 2010;126(4):682–687. doi: 10.1542/peds.2010-0362. [DOI] [PubMed] [Google Scholar]
- 8.Banasiak NC. Spirometry in primary care for children with asthma. Pediatr Nurs. 2014;40(4):195–198. [PubMed] [Google Scholar]
- 9.Doerschug KC, Peterson MW, Dayton CS, Kline JN. Asthma guidelines: an assessment of physician understanding and practice. Am J Respir Crit Care Med. 1999;159(6):1735–1741. doi: 10.1164/ajrccm.159.6.9809051. [DOI] [PubMed] [Google Scholar]
- 10.O'Dowd LC, Fife D, Tenhave T, Panettieri RA., Jr Attitudes of physicians toward objective measures of airway function in asthma. Am J Med. 2003;114(5):391–396. doi: 10.1016/s0002-9343(03)00007-x. [DOI] [PubMed] [Google Scholar]
- 11.Spirometry 360. [Accessed 2/7/17];2017 www.spirometry360.org.
- 12.Redline S, Gruchalla RS, Wolf RL, et al. Development and validation of school-based asthma and allergy screening questionnaires in a 4-city study. School Nurse News. 2004;21(5):12–14. [PubMed] [Google Scholar]
- 13.Miller MR, Hankinson J, Brusasco V, et al. Standardisation of spirometry. Eur Respir J. 2005;26(2):319–338. doi: 10.1183/09031936.05.00034805. [DOI] [PubMed] [Google Scholar]
- 14.Chan KS, Mangione-Smith R, Burwinkle TM, Rosen M, Varni JW. The PedsQL: reliability and validity of the short-form generic core scales and Asthma Module. Med Care. 2005;43(3):256–265. doi: 10.1097/00005650-200503000-00008. [DOI] [PubMed] [Google Scholar]
- 15.Donner A, Klar N. Design and analysis of cluster randomized trials. Statistics in medicine. 2001;20:329–496. doi: 10.1002/1097-0258(20010215)20:3<329::AID-SIM794>3.0.CO;2-0. [DOI] [PubMed] [Google Scholar]
- 16.Stout JW, Smith K, Zhou C, et al. Learning from a distance: effectiveness of online spirometry training in improving asthma care. Acad Pediatr. 2012;12(2):88–95. doi: 10.1016/j.acap.2011.11.006. [DOI] [PubMed] [Google Scholar]
- 17.Seid M, Limbers CA, Driscoll KA, Opipari-Arrigan LA, Gelhard LR, Varni JW. Reliability, validity, and responsiveness of the Pediatric Quality of Life Inventory™ (PedsQL™) Generic Core Scales and Asthma Symptoms Scale in vulnerable children with asthma. Journal of Asthma. 2010;47(2):170–177. doi: 10.3109/02770900903533966. [DOI] [PubMed] [Google Scholar]