Unexpected Results of a Randomized Quality Improvement Program for Children with Severe Asthma

Abstract

Objective,

To assess whether an asthma intervention program reduces treatment days outside the home among children with severe asthma receiving comprehensive care (CC) in our center.

Methods,

Between October 21, 2014 and September 28, 2016, children with severe asthma were randomized to receive CC alone (n=29) or CC plus the asthma intervention program (n=34) which involved collaboration with pharmacists and school nurses, motivational interviewing, and tracking the one-second forced expiratory volume at home. All patients were followed through March 31, 2017. Frequentist and Bayesian intent-to-treat analyses were performed.

Results,

The asthma intervention program doubled the telephone calls between the staff and families (753 vs 356 per 100 child years for the intervention group vs. control group; Rate Ratio [RR], 2.11 [95% confidence interval, 1.29–3.45]). Yet, we found no evidence that it reduced the composite number of days of healthcare outside home which includes, clinic visits, ED visits, and hospital admissions (1179 vs 958 per 100 child-years in the intervention group vs. control group; [RR], 1.23 [95% CI, 0.82–1.84]) or secondary outcomes which are individual components (clinic visits, ED visits, hospitalizations, PICU admissions and school absences; RR 1.15 – 2.30; p>0.05). Bayesian analysis indicated a 67% probability that the intervention program increases total treatment days outside the home and only a 14% probability of a true decrease of >20% as originally hypothesized.

Conclusion,

A multi-component intervention program provided to children with severe asthma failed to reduce and may have increased days of healthcare outside home and school absenteeism.

Introduction

Asthma is the most common chronic illness in children (1) and a leading cause of childhood hospitalization and school absenteeism (2)(3). Despite advances in the understanding of its pathophysiology and the availability of effective therapies, adverse effects on health, school attendance, academic achievement, and family life remain high (4)(5)(6), particularly among children with severe asthma in minority or low income families (7). Innovative approaches to improve asthma outcomes in children and reduce healthcare utilization are needed.

We have developed a new model of comprehensive care (CC) for high-risk children with asthma and other chronic illnesses that involves multiple features to promote prompt effective care at all hours. Acute same-day care, chronic care, and specialty care are all provided in the same clinic. Same-day appointments are available 40 hours/week in our clinic, and parents have 24/7 cell phone access to the primary care provider (PCP) on call (8)(9).Our previous randomized trial showed that relative to usual care, CC reduced total emergency department (ED) visits and hospital days by 52–64%, and health system costs by $10,258 per child-year (8). A subgroup analysis of the trial children with severe asthma identified a similar reduction of 40– 60% in ED visits and hospitalizations (9).

Nevertheless, many of our asthma patients continued to experience high rates of exacerbation despite our measures to identify and treat the comorbidities that exacerbate asthma and to educate families about the importance of treatment adherence (10)(11)(12). We conducted a survey of the patients with poorly controlled severe asthma in our clinic, 40% (8 of 20 respondents) indicated poor adherence (defined as taking <50% of the prescribed controller medication). Previous studies have shown that interventions such as motivational interviewing, involvement of school nurses, collaboration of pharmacists or portable pulmonary testing have shown to improve adherence in children with severe asthma (13)(14)(15). However, none have studied the effect of an integrated approach to improve outcomes. Given this information, the tendency for self-reports to exaggerate adherence (16), and the evidence from multiple studies that poor adherence contributes to asthma exacerbations (11)(17)(18)(19), we developed a multi-component asthma intervention program (AIP) to improve asthma outcomes. We then conducted days of satisfaction among children with uncontrolled severe asthma receiving CC in our program. This is a pragmatic study designed to evaluate the effectiveness of interventions in real-life routine practice conditions.

Methods

Population

Study candidates were identified from the High-Risk Children’s Clinic at The University of Texas at Houston (UTH) McGovern Medical School. All children in this clinic are medically complex with a chronic illness, history of high utilization of healthcare services (≥2 hospitalizations, or ≥1 pediatric intensive care unit [PICU] admission in the year prior to enrollment), and a high estimated risk of hospitalization in the next year (>50% as judged by the clinic’s medical director [R.A.M.] based on the children’s diagnoses, current medical status, past medical history, and socioeconomic risk factors). We included patients aged 2–18 years in the trial if they met at least one of the following NIH guidelines criteria (20) for uncontrolled severe asthma, ≥ 2 ED visits and/or ≥1 hospitalization for respiratory symptoms in prior year; >2 rounds of oral steroids in the previous year; asthma symptoms multiple times/day, more than 2 days/week or asthma-triggered nighttime awakenings more than twice/month; any significant asthma interference with normal activity; short-acting beta-agonist use for symptom control >2 days/week; ≤60% of predicted forced expiratory volume in one second (FEV1) peak flow (% of personal best); and/or ≤75% of FEV1/forced vital capacity on routine visit. We assessed eligibility by reviewing patients’ electronic medical records. We excluded patients with cystic fibrosis, interstitial lung disease, bronchiolitis obliterans, tracheostomy, mechanical ventilation, or any neuromuscular disorder. Our research coordinator and providers reviewed patients’ charts to identify eligible patients (poorly controlled asthmatics), then subjects were invited to take part of our study during a clinic visit or on a telephone call.

Our Institutional Review Board approved this study as a randomized QI study, for which verbal informed consent was permitted (21). Partly because of the differences in the AIP according to patient age, participants were stratified by age (2–5 years; 6–18 years). They were also stratified by the total days of respiratory-related medical care provided outside the home (in a clinic, ED, or hospital) in the prior year (≤5 days; >5 days) and then randomized to either CC or CC+AIP using the REDCap randomization module. If two siblings were found to be eligible, both were assigned to the same group.

Comprehensive Care (CC)

Comprehensive care was provided in our High Risk Children’s Clinic as a medical home augmented by multiple measures to improve care and outcomes for children with medical complexity (9)(22).’The clinic maintains a patient-to-provider ratio of ≤75. Clinic visits typically last 45–60 minutes, and providers frequently communicated with parents by telephone and email. Parents have direct access to their child’s assigned PCP 40 hours/week in the clinic and 24/7 cell phone access to the provider on call (either the general pediatrician or one of the 3 pediatric nurse practitioners who are each familiar with all patients and have access to their medical records at all hours). The medical director, a Spanish-speaking pediatric pulmonologist, provides continuous backup, a second pediatric pulmonologist serves as a PCP, and multiple specialists attend the clinic at least half a day/month. Same-day care is provided on weekdays for acute illnesses or complications, and our providers communicate directly with the attending physician in the ED, pediatric floor, or PICU whenever the children are referred to Children’s Memorial Hermann Hospital, our affiliated teaching hospital.

Multi-Component Asthma Intervention Program

Both groups were treated by the same primary care team. The AIP was designed to maximize treatment adherence and more promptly treat exacerbations in patients with severe asthma using all appropriate measures for the patient’s age. It added the following components to CC:

1. Involvement of school nurses. For school-age children (≥6 years of age), we routinely requested consent to contact the school nurse. Because of concerns that controller medication was often not given before school, we instructed the family and school nurses that the first dose of the daily controller medication on school days must be administrated by the nurse at school to ensure a minimum of 5 doses/week. We also cautioned parents to verify that the children were receiving the medication on both weekends and weekdays. In addition, the school nurses were called every 2–3 months to report school absences.

2. Collaboration with pharmacists. Once a patient was enrolled and until the end of the study, the research coordinator called the pharmacy monthly a week after the refills were due to confirm that the medication was refilled. If the patient had not picked up the medication, the families were called to be reminded. If they failed to pick up for two consecutive months, the providers were informed to call the families.

3. Motivational Interviewing. Motivational interviewing (MI) is a client-centered intervention that focuses on enhancing intrinsic motivation to change a particular behavior and exploring and resolving ambivalence about behavior change (13)(23). There is evidence that MI can improve adherence in asthma patients (13).The PCPs in our clinic were trained in MI techniques to better understand, address, and remove barriers that may impact medication adherence (23). MI techniques used included open-ended questions, reflective listening, empathy and support for the decision making of patients in the CC+AIP group. In order to avoid contamination to the CC only patients, Providers were trained and instructed to review the patient’s chart prior to the visit. Although, having separate centers would help avoid contamination, we only had one site and therefore we only advised the providers to be careful about who they conducted MI for.

4. Portable Pulmonary Testing. The PiKo-1 is a portable, hand-held device that can be used to measure FEV1 (15). Children who were ≥6 years and capable of performing this test were given a PiKo-1 at enrollment. The patients and their families were instructed on how to perform the test, and to ensure reliability of the testing, the results were compared with clinic spirometry results at their first and all subsequent scheduled clinic visits. The families were instructed to measure the FEV1 whenever the child had asthma symptoms and to call the clinic emergency cell phone whenever they found that the FEV1 was <60% of the predicted value. Families were also asked once a month by telephone about the PiKo-1 readings. The PCPs used this information to modify treatment or to advise families to seek prompt or immediate care in the clinic or ED.

Process Measures

The staff recorded all telephone calls to the families and pharmacies. In addition, we recorded the number of children that received MI, patients who failed to pick up medication for two consecutive months, number children who were received PiKo-1 device, and children for whom the school nurses confirmed that he/she would administer controller medications on weekdays.

Parental Satisfaction

Research personnel uninvolved in patient care assessed parental satisfaction with outpatient care by administering 5 preselected questions from the Consumer Assessment of Healthcare Providers and Systems Child 12-Month Survey (24). These questions were administered’ in English or Spanish (25) 12 months after enrollment or at 6–12 months for children who participated for less than a year in the study.

Outcome measures

The primary outcome is the composite number of days of healthcare outside home which includes (clinic visits, ED visits, and hospital admissions). The secondary outcomes is the individual components of the primary outcome; clinic visits, ED visits, hospital admissions, PICU admissions, and hospital days. The staff recorded all clinic visits and hospital electronic medical records were reviewed each weekday to identify ED visits or hospital admissions. We also asked parents at all clinic visits about any medical services received and/or missed days of school (with or without respiratory symptoms). Other secondary outcomes included school absences, and changes in FEV1 (assessed using clinic spirometer) between enrollment and 12 months post enrollment or at 6–12 months for children who were followed for less than a year.

Statistical Analyses and Sample Size

Under intent-to-treat principles, total treatment days outside the home, clinic visits, ED visits, hospitalizations, hospital days, school absences, and phone calls were analyzed using negative binomial regression models. Changes in FEV1 between baseline and 12 months after enrollment in both groups were compared using a multilevel linear mixed model. Parental satisfaction ratings, CAHPS rating of 9 or 10 (highest scores) vs. any other CAHPS rating were analyzed using a logistic regression model. All models were adjusted for the trial’s stratifying variables (age at enrollment [≤ 5 or >5 years]; baseline risk [≤5 or >5 treatment days outside the home in the year prior to enrollment]), and length of follow-up (included as an offset in all models).

All analyses were performed using Stata software version 15.1 (StataCorp) and R software version 3.1.1 (R Foundation for Statistical Computing). A 2-sided P < .05 was considered statistically significant. Separate Bayesian analyses were performed to estimate the probability that CC+AIP reduced the total treatment days outside the home. We used a neutral prior probability (risk ratio = 1.0 [95% credible interval, 0.3–3.0], which encompasses the largest likely effect size for major outcomes observed in randomized trials) (26)(27)(28).

We originally planned to enroll 80 patients in two years to have 82% power to identify a 40% reduction from the 10 treatment days outside the home per year observed before the trial. However, enrollment was halted at the end of the second year of study due to funding constraints and unanticipated expenses to replace multiple devices lost by the families during the study.

Results

Enrollment occurred from October 2014 through September 2016. We screened 114 medically complex children with asthma in our program; 71 met all eligibility criteria. Parental consent was obtained for 63; 29 were randomized to CC alone and 34 to CC+AIP (Figure 1). The groups were at similarly high baseline risk (Table 1). All patients were followed for at least 6 months (through April 2017) with median length of follow-up of 1.5 years, and a total of 51 child-years in the CC group and 62 child-years in the CC+AIP group.

Figure 1.

Treatment Assignment for Children with Uncontrolled Severe Asthma.

Table 1.

Baseline Characteristics by Treatment Group.

Characteristic, No. (%)	Treatment Group, No. (%)
	CC + AIPa (n=34)	CCb (n=29)
Age Ranges
2–5 y	9 (26)	7 (24)
6–18 y	25 (74)	22 (76)
Male sex	19 (56)	18 (62)
Race/ethnicity
African American	20 (59)	17 (59)
Asian	2 (6)	0 (0)
Caucasian Non-Hispanic	0 (0)	2 (7)
Hispanic	11 (32)	8 (28)
Other	1 (3)	2 (7)
No. of unique families	32	27
Maternal education level
HS Diploma or GED	28 (82)	25 (86)
No HS Diploma	5 (15)	4 (14)
Unknown	1 (3)	0 (0)
Gestational age
37 weeks or greater	24 (71)	16 (55)
Less than 37 weeks	10 (29)	13 (45)
Total respiratory days of care in year prior to enrollment
5 days or less	13 (38)	11 (38)
6 days or more	21 (62)	18 (62)
Medicaid insurance	28 (82)	26 (90)

^aThere were 62 child-years.

^bThere were 51 child-years.

Process Measures (Table 2).

Table 2.

Process Measures and Parental Rating of Care by Treatment Group.

Measures	CC + AIP (n=34) a	CC (n=29) b	Adjusted Difference (95% CI)c	P Valuec
Process Measures
Phone calls (per 100 child-years) between health care staff and families, No.	753	356	2.11 (1.29–3.45)	0.003
Families who received motivational interviewing at least once, No. (%)	34 (100%)	N/A	N/A	N/A
Patients who received 6 or more consecutive monthly communication with the pharmacy, No. (%)	26 (76%)	N/A	N/A	N/A
Patients who failed at least once to pick-up their medication for 2 consecutive months, No. (%)	27 (79%)	N/A	N/A	N/A
Children ≥6 years of age who were able to perform the portable pulmonary function test, No. (%)	19 (76%)d	N/A	N/A	N/A
Children ≥6 years for whom the school nurses confirmed that he/she would administer controller medications on weekdays, No. (%)	20 (80%)d	N/A	N/A	N/A
Parental Satisfaction e
Clinician always explained things in a way that was easy to understand, No. (%)	30/31 (97%)	25/25 (100%)	0.20 (−0.38–0.44)	0.106
Clinician always listened carefully, No. (%)	31/31 (100%)	25/25 (100%)	N/A	N/A
Clinician always gave easy to understand information about health questions or concerns, No. (%)	27/27 (100%)	18/18 (100%)	N/A	N/A
Clinician always showed respect, No. (%)	31/31 (100%)	25/25 (100%)	N/A	N/A
Clinician rating of 9 or 10f	30/31 (97%)	25/25 (100%)	−0.32 (−0.09–0.2)	0.365

^aThere were 62 child-years.

^bThere were 51 child-years.

^cAdjusted for age (2–5 years; 6–18 years), days of respiratory-related medical care provided outside the home in the prior year before the study (≤5 days; >5 days), and length of follow up.

^dThere were 25 children aged 6 years or older in the CC+AIP group

^eThe Consumer Assessment of Healthcare Providers and Systems Child 12-Month Survey was administered to parent of each participant 12 months after study enrollment (6–12 months after enrollment for patients enrolled during the last study year). There were a total of 34 possible survey respondents in the intervention group and 29 in the usual care group. The P values were obtained from the 2-sample z test for proportions.

^fOn a scale of 0–10 with 10 representing the highest score possible.

Telephone calls between clinic staff and families were higher with CC+AIP than with CC alone (753 vs 356 per 100 child-years) (RR, 2.11; [1.29–3.45], P=0.003). Among the CC+AIP children, 100% received at least one session of MI, 76% received ≥’6 consecutive pharmacy calls, 79% failed at least once to pick-up medication for 2 consecutive months, and among those ≥’6 years, 76% received a PiKo-1 device, while 80% made an agreement with the school nurses to administer the medication. By study protocol, no communication with pharmacists or school nurses occurred for children given only CC.

Parental Satisfaction (Table 2).

Parental satisfaction ratings of 9 or 10 (highest scores) vs. any other rating, were high and similar in both groups (97% vs 100%) (RR, −0.32; [−0.09–0.2], P=0.365).

Outcomes (Table 3).

Table 3.

Outcome Measures by Treatment Group.

Outcome Measure	CC + AIP (n = 34)a		CC (n = 29)b		Adjusted Rate Ratio (95% Cl)c	P Value c
	No.	Adjusted Rate/100 Child-Yearsc	No.	Adjusted Rate/100 Child-Yearsc
Treatment days provided outside the home d	391	1179	285	958	1.23 (0.82–1.84)	0.31
Clinic visits	338	970	247	842	1.15 (0.81–1.64)	0.44
ED visits	38	107	29	97	1.10 (0.53–2.31)	0.80
Hospital admissions	12	38	10	33	1.13 (0.32–4.01)	0.85
Hospital Days	29	137	18	59	2.30 (0.65–8.11)	0.20
PICU admissions	0	0	0	0	n/a	n/a
Missed school days	445	1285	245	823	1.56 (0.93–2.61)	0.09
Missed school days with respiratory symptoms	309	932	187	637	1.46 (0.81–2.62)	0.21

^aThere were 62 child-years.

^bThere were 51 child-years.

^cAdjusted for age (2–5 years; 6–18 years), days of respiratory-related medical care provided outside the home in the prior year before the study (≤5 days; >5 days), and length of follow up.

^dPrimary outcome

Contrary to our hypothesis, we found no evidence that the AIP reduced the total number of treatment days outside the home below that with CC alone (1,179 vs 958 days per 100 child-years; rate ratio [RR], 1.23 [95% CI, 0.82–1.84]). There was no evidence of a treatment-interaction with patient age or days of care before enrollment. Bayesian analysis using a neutral prior probability indicated only a 14% probability that the intervention program decreased total treatment days outside the home by 20% or more as hypothesized. Surprisingly, the findings indicated a 67% probability that the intervention actually increased treatment days outside the home.

We also found no evidence that the intervention program reduced clinic visits relative to CC alone (970 vs. 842 per 100 child-years ; RR, 1.15 [0.81–1.64]), ED visits (107 vs. 97 per 100 child-years; RR 1.10 [0.53–2.31]), hospitalizations (38 vs. 33 per 100 child-years; RR 1.13 [0.32–4.01]), and hospital days (137 vs. 59 per 100 child-years; RR, 2.30 [0.65–8.11]). No PICU admissions occurred in either group. There was also no evidence of a reduction in missed school days (1,285 vs. 823 per 100 child-years; RR 1.56 [0.93–2.61]) or missed school days with respiratory symptoms (932 vs. 637 per 100 child-years; RR 1.46 [0.81–2.62]).

Changes in pulmonary function tests from baseline to one year after enrollment did not differ significantly between the two groups. The mean FEV1 for the CC+AIP group was 74% of predicted (for age, weight, height, and race) at enrollment (SD, 4.95, [95% CI, 65.8–84.61]) and 79% at one-year post enrollment (SD, 3.97, [71.2–87.6]). The CC group had a mean FEV1 of 76% of predicted at enrollment (SD, 4.22, [−67.2–84.8]) and 83% at one year post enrollment (SD, 2.49, [78.4–88.7]). Using the mixed-effects model to account for missing data, we found no significant difference between CC+AIP and CC in changes of predicted FEV1 from baseline to one-year post enrollment (mean increase of 2.8% vs. 3.3%, p=0.99).

Discussion

This multi-component AIP was designed to maximize medication adherence and more promptly and effectively treat exacerbations and thereby reduce medical care outside the home and school absenteeism among high-risk children with severe asthma receiving comprehensive care. Yet, we found no evidence of any such reductions. Indeed, Bayesian analyses identified a 67% probability that the AIP increased the total days of care outside the home. We also found no evidence that the AIP resulted in improvement in FEV1 or in parental satisfaction ratings over CC alone.

This intervention program was designed to enhance the outcomes obtained by our comprehensive care model which demonstrated decrease of hospital utilization (hospitalizations, ED visits, clinic visits and PICU admissions). The total days of healthcare was chosen as a primary outcome to determine if AIP did in fact enhance our CC model. The absence of a discernible incremental benefit from the AIP may partly reflect the reduction in ED visits, PICU admissions, and hospitalizations as well as the high parental satisfaction ratings resulting from CC itself (8)(9)(22). It is likely that we have reached a ceiling effect with PICU admissions and parental satisfaction ratings with CC alone. The AIP might be beneficial if implemented in other models of care.

Our study resulted in high rates of failure to pick up medication refills, which may be partially explained by patients having extra medication. Patients were provided two sets of controller medication monthly, one for home and one for school (used only Monday to Friday), which is more than the usual one set per month. The two sets of controller medication would likely have had sufficient dosing for 6 weeks at home and 10–12 weeks at school. School nurses were instructed to contact us with any problems with adherence during school days including shortage of medication. There was only one instance in which a school nurse contacted us about medication deficiency, suggesting that most patients had sufficient medication at school despite the high rates of failure to pick up medication refills.

We believe that communication is key to improve adherence, so we called parents 15 days after they failed to pick up their medication. Improved communication on the consequences of non-adherence to both parents and patients, as developmentally appropriate, may also contribute to better adherence. Future studies may want to focus on identifying and addressing barriers to compliance that may improve pick up rates (e.g. developing a system where medication may be mailed to parents limited ability to access pharmacies).

Asthma is a complex disease. Factors such as unresponsiveness to therapies may cause severe exacerbations, lead to poor adherence, and contribute to the absence of a discernible benefit from even an intensive program to increase adherence in patients with poor compliance. However, in attempting to maximize treatment adherence, we may have inadvertently increased the likelihood of school absences and medical treatment outside the home by increasing child or parental anxiety. The intervention may have led families to be more aware of their children’s symptoms and seek higher levels of care that in turn contributed to significantly increase acute care events. Despite training the parents and the child in assessing FEV1 using the PiKo-1, home use of this device may well have resulted in falsely low FEV1 values, lowered parents’ thresholds to contact our PCPs, and made our PCPs more likely to appoint the patients to our clinic or refer them to the ED.

Asthma generally improves over time. Rates of ED visits and hospital admissions in both groups have improved over time.(8)(9)(22) These findings reflect sustained longitudinal improvement in the health status of these asthma patients (8)(9). Had this study been designed as a pre- and post-comparison often used in QI studies, we would have falsely concluded that the multi-component AIP improved outcomes. Although this was a failed trial, we learned that well-designed studies to minimize bias – preferably randomized trials – are important to verify the benefits of QI interventions presumed to be beneficial. Additional lessons learned include the importance of communication with school nurses and motivation of subjects to maintain study devices e.g. using incentives provided to subjects who were able to keep their devices for the duration of the trial. Our study also highlights the need for more creative measures to improve adherence.

Our study has several limitations. Adherence is extremely difficult to assess accurately (11). While we assessed administration of medication in the school and provision by the pharmacy, we could not verify adherence in the home. Our findings may not be generalizable to other settings outside a medical home like ours or to lower risk populations. Although, our study was underpowered, we used Bayesian analyses to overcome this issue. In addition, the increase in the absenteeism in the intervention group could be due to a recall bias. School absenteeism was family reported in both groups, however, in the intervention group the research coordinator contacted the school nurses to give verbal attendance report as well.

In conclusion, a multi-component intervention program to increase adherence to asthma medication among children with severe asthma failed to reduce and may have somewhat increased school absenteeism and treatment days outside the home, including hospitalizations.