Simultaneously implementing pathways for improving asthma, pneumonia, and bronchiolitis care for hospitalized children: Protocol for a hybrid effectiveness-implementation, cluster-randomized trial

Abstract

Background:

Asthma, pneumonia, and bronchiolitis are the top causes of childhood hospitalization in the United States, leading to over 350,000 hospitalizations and ≈$2 billion in costs annually. The majority of these hospitalizations occur in general/community hospitals. Poor guideline adoption by clinicians contributes to poor health outcomes for children hospitalized with these illnesses, including longer recovery time/hospital stay, higher rates of intensive care unit transfer, and increased risk of hospital readmission. A prior single-center study at a children’s hospital tested a multicondition clinical pathway intervention (simultaneous implementation of multiple pathways for multiple pediatric conditions) and demonstrated improved clinician guideline adherence and patient health outcomes. This intervention has not yet been studied in community hospitals, which face unique implementation barriers.

Objective:

To study the implementation and effectiveness of a multicondition pathway intervention for children hospitalized with asthma, pneumonia, or bronchiolitis in community hospitals.

Methods:

We will conduct a pragmatic, hybrid effectiveness-implementation, cluster-randomized trial in community hospitals around the United States (1:1 randomization to intervention vs. wait-list control). Our primary outcome will be the adoption of two to three evidence-based practices for each condition over a sustained period of 2 years. Secondary outcomes include hospital length of stay, ICU transfer, and readmission.

Discussion:

This hybrid trial will lead to a comprehensive understanding of how to pragmatically and sustainably implement a multicondition pathway intervention in community hospitals and an assessment of its effects. Enrollment began in July 2022 and is projected to be completed in September 2024. Primary analysis completion is anticipated in March 2025, with reporting of results following.

INTRODUCTION

Asthma, pneumonia, and bronchiolitis are the top causes of childhood hospitalization in the United States, leading to over 350,000 hospitalizations and ≈$2 billion in costs annually.¹ Clinicians face many challenges adopting evidence-based guidelines (e.g., lack of familiarity, low confidence in ability to adhere),² and poor guideline adoption by clinicians contributes to poor health outcomes for children hospitalized with these respiratory illnesses, including longer recovery time/hospital stay, higher rates of transfer to intensive care units, and increased risk of hospital readmission.^3–6 General hospitals, such as community hospitals, primarily provide care for adults but also provide care for >70% of hospitalized children nationally.⁷ Unlike dedicated children’s hospitals, community hospitals face unique challenges to achieving guideline adoption and high-quality care for children, including less access to specialized pediatric services and limited resources for pediatric care and quality improvement.^8,9 A scoping review of pediatric quality and safety found several areas of concern when comparing community hospitals to free-standing children’s hospitals, including increased risk of neonatal mortality in low birth-weight infants, unnecessary radiation exposure from imaging studies for children, and suboptimal drug selection and inadequate drug toxicity monitoring in children.¹⁰

Pathways can improve clinicians’ adoption of evidence-based practices/guidelines in both children’s and community hospital settings.^11–13 Pathways are simple, visual diagrams that guide clinicians step-by-step through the evidence-based care of a specific medical condition (accessed via paper or electronically). They have been shown to improve care for children hospitalized with asthma, pneumonia, and bronchiolitis.^11–19 For example, in a national analysis of asthma pathways in 42 children’s hospitals from 2006 to 2015, pathways were associated with reductions of ≈4000 unnecessary antibiotic prescriptions, ≈22,000 hospital days, and ≈$18,000,000 in costs.¹⁶

Most hospitals implement pathways for a single medical condition at a time (e.g., asthma). However, a prior single-center study at a children’s hospital tested a multicondition clinical pathway intervention (simultaneous implementation of multiple pathways for multiple pediatric conditions). This intervention led to sustained guideline adoption, decreased length of stay, and decreased costs^14,20–22; and, these effects were comparable to those shown with single-condition pathway implementation.^11–19 This multicondition pathway intervention has not yet been studied in community hospitals, which face unique implementation barriers.^8,9 Understanding how this pathway intervention can be successfully implemented in community hospitals will fill critical gaps that can enable rapid, broad-scale improvements in guideline adoption and health outcomes for hospitalized children.

Our objective is to study the implementation and effectiveness of a multicondition pathway intervention for children hospitalized with asthma, pneumonia, or bronchiolitis in community hospitals. We are conducting a pragmatic, hybrid effectiveness-implementation, cluster-randomized, superiority trial in community hospitals around the United States (1:1 randomization to intervention vs. wait-list control). This study will provide a comprehensive understanding of how to sustainably implement the multicondition pathway intervention in community hospitals and an assessment of its effects, which can support improvements in care and outcomes for children with respiratory illnesses.

METHODS

Design, participants, interventions, and outcomes

Trial design

We will conduct a pragmatic, type 3 hybrid effectiveness-implementation trial, in which the primary goal is evaluating implementation and a secondary goal is determining effects on clinical outcomes.²³ The design involves a parallel, cluster-randomized trial (1:1 randomization of hospitals to intervention vs. wait-list control). The trial intervention period will be 2 years. At the end of the 2-year trial period, wait-list control hospitals will receive the intervention (Figure 1). A parallel, cluster-randomized trial best addresses seasonal variation in outcomes for these respiratory illnesses by providing a concurrent control group and collecting data over five winter/respiratory viral seasons, three seasons before implementation and two seasons after implementation.

FIGURE 1.

Study timeline.

Hospitals will be randomized via computer-generated allocation, stratified by type (community hospital with pediatric beds vs. nested/non-freestanding children’s hospital).^7,8 Due to the nature of the intervention, the central study team, site PIs, health care providers, chart reviewers, and participants are not blinded to the intervention (use of multicondition clinical pathway intervention in care provision).

Study setting

The study will take place in general hospitals around the United States. General hospitals are defined as community hospitals or nested children’s hospitals (not freestanding).^7,8 Nested children’s hospitals share physical space and leadership structures with larger medical centers that primarily focus on adult care; whereas, freestanding children’s hospitals operate independently in both of these respects. Hospitals were recruited via collaborative efforts with our partner organizations, including the Society of Hospital Medicine, the American Academy of Pediatrics, and the Pediatric Research in Inpatient Settings Network. A final list of study sites will be listed in the trial publication and is available upon request.

Eligibility criteria

Hospitals will be eligible for involvement in the study if they have inpatient pediatric units and utilize electronic health records for order placement. Free-standing children’s hospitals will be excluded. Children will be eligible for inclusion and analysis if admitted to the inpatient pediatric wards of study hospitals with a primary diagnosis of asthma, pneumonia, or bronchiolitis during the study period. We will identify eligible children using ICD-10 codes that have been previously validated as part of the NIH-funded PRIMES Studies.²⁴ Detailed inclusion criteria, exclusion criteria, and rationale are outlined in Table 1.

TABLE 1.

Study inclusion/exclusion criteria.

Inclusion criteria
Primary diagnosis of asthma AND age >2 to <18 years old at time of admission to the hospital.
Primary diagnosis of pneumonia AND age >2 months and <18 years at time of admission to the hospital.
Primary diagnosis of bronchiolitis AND age <2 years at time of admission to the hospital.
Exclusion criteria and rationale
Secondary diagnosis of SARS-CoV-2 (due to lack of evidence-based guidelines on management).
Transfer in from or out to an inpatient facility (due to inability to accurately determine length of hospital stay).
Pre-existing chronic lung disease, cardiovascular disease, airway anomalies, immunodeficiency, or neurologic disorders.a

^aThe pathway intervention excludes these children because it is intended for routine management of the targeted respiratory conditions, not for children who have underlying chronic illness.

Intervention

The multicondition pathway intervention consists of evidence-based clinical pathways that guide the care of children with asthma, pneumonia, or bronchiolitis hospitalized in inpatient pediatric units/wards (Supporting Information). A national advisory board guided intervention development. Advisory board members included clinical experts for each condition of interest, implementation scientists, and front-line pediatric clinicians, including clinicians from community hospitals. Intervention design was guided by these clinician partners because clinician behavior is the focus of the intervention; patient/public partners were not involved. Our overall intervention plan combines the implementation strategies outlined in the prior single-center study of the intervention²⁰ with implementation strategies specified for mentored implementation, a well-established framework for multisite implementation of healthcare interventions.²⁵ Implementation strategies include:

Audit and feedback

Site leaders (inpatient pediatric physicians/hospitalists) will be provided with monthly run charts/performance reports on guideline adoption outcomes (Table 2), and patient-centered outcomes (ICU transfer, length of hospital stay, 30-day hospital readmission or ED revisit). Reports will contain hospital-specific performance and aggregate performance across all intervention sites. Local implementation teams will share and review these reports with pediatric clinicians.

TABLE 2.

Guideline adoption outcomes/evidence-based practices.

Condition	Evidence-based practices
Asthma	Early use of metered-dose inhalers (MDIs) Use of asthma pathways/protocols for bronchodilator titration Prescription of inhaled corticosteroid at hospital discharge for children ≥5
Pneumonia	Administration of narrow-spectrum antibiotic in hospital No prescription of macrolide antibiotic at discharge
Bronchiolitis	No administration of albuterol during admission No chest radiography performed during admission

PDSA cycles

Local implementation teams will review inpatient clinical workflows in relation to current performance and plan PDSA cycles, in which teams identify and test workflow changes to improve performance.²⁶ PDSA cycles facilitate progressive, structured, and iterative change.

Integration of pathway content into electronic order sets

Local implementation teams will integrate guidance on the evidence-based practices of interest into electronic order sets. The core study team will provide these teams with order set templates to guide this process, and mentors will support site leaders in planning and addressing barriers.

Mentor meetings/external facilitation

Mentors/external facilitators will meet virtually every month for the first 12 months of the implementation period. Meetings will involve: (1) reviewing performance reports on adoption of evidence-based practices, (2) planning PDSA cycles, (3) identifying key milestones/targets, and (4) monitoring intervention fidelity. Mentors will also conduct an in-person site visit, in which they promote the intervention to local hospital leaders and monitor fidelity to the intervention.

Learning collaborative

Site leaders will participate in a series of educational seminars. These seminars will provide didactics as well as facilitate cross-site problem solving and sharing of insights on barriers and facilitators of implementing the multicondition pathway intervention.

Comparator

In Year 1, all hospitals (both intervention and wait-list control) will collect baseline data. In Years 2–3 all hospitals will continue data collection, but intervention hospitals will implement the multicondition pathway intervention while wait-list control hospitals do not implement any new interventions related to the care of children with respiratory illness. They will instead continue current standards of care. In Year 4, we will implement the multicondition pathway intervention in the wait-list comparator control group (Figure 1). Implementation studies often assess outcomes over ≤1 year, but we will assess outcomes over a 2-year period to better assess sustainability.

Outcomes

Our primary outcome will be adoption of evidence-based practices. We selected seven evidence-based practices (Table 2),^27–30 with input from our national advisory board using the following criteria: (1) mean adoption rates are below 80% in US community hospitals and significant variability in adoption exists, (2) pathways are associated with significantly increased adoption of the practice, and (3) improved adoption is associated with improvements in patient-centered outcomes, such as length of stay or hospital readmission. Secondary study outcomes will include adoption of each evidence-based practice (analyzed separately), length of hospital stay in hours (defined as the time from admission order placement to discharge order placement), transfer to an intensive care unit, and hospital readmission or emergency revisit within 30 days of hospital discharge. The outcomes selected align with national quality measures, including the PRIMES guideline adoption measures and the National Quality Forum-endorsed 30-day pediatric respiratory illness readmission measure.^24,31,32 We will also measure fidelity (use of implementation strategies as intended) and barriers of facilitators of implementation via monthly surveys of site leaders and field visits to intervention hospitals. These surveys will collect both quantitative and qualitative data.

Participant timeline, recruitment, and retention

Participant enrollment in the study will occur a minimum of 30 days after the hospitalization event, when the hospitalization is reviewed and data is collected via chart review. Data collection will occur a minimum of 30 days after the hospitalization event to determine if a 30-day hospital readmission or ED revisit occurred. No participant recruitment is required for this study, and participants do not need to be retained, as all trial data will be collected via retrospective chart review.

Data collection, management, and analysis

Data collection and management

Trained chart reviewers at each hospital will collect data through a monthly chart review of the electronic health records of enrolled patients. We will collect data on the hospitalization event, including our primary and secondary study outcomes, participant demographics, and factors known to be associated with the severity of these respiratory illnesses in children (e.g., need for oxygen support). The full data collection form is provided in the Supporting Information. Reviewers will review all eligible admissions each month, to a maximum of 10 per condition of interest (total of 30 admissions per month, randomly sampled).

To ensure data quality, the core study team will train chart reviewers at each study site using the Medical Record Abstraction Quality Assurance and Control Framework.³³ Principles of the framework include: (1) quality assurance—prospective actions taken such as abstractor training, standard procedures, and job aids to ensure adequate accuracy, and (2) quality control—measurement of error or discrepancy rates and use of the measurements to guide adjustments to controllable inputs to the abstraction process, such as abstraction tools, procedures, and training. A second chart reviewer at each site will review a subset of charts (10%) each month for quality control, providing feedback to the primary reviewer and core study team about any discrepancies (to facilitate additional training). Reviewers will enter data into the REDCap electronic database (Version 14, Nashville, TN) accessible for all study sites. To maximize data quality, we will use REDCap data validation algorithms, which restrict input to plausible values and check for potentially erroneous outliers. Core study team staff will check for suspicious or missing data weekly and contact chart reviewers to make any needed corrections.

No protected health information will be collected in any part of this study. To ensure confidentiality and compliance with the Health Insurance Portability and Accountability Act (HIPAA), all data and records will be kept confidential, all data will be stored on secure servers, and access will be restricted to members of the IRB-approved research team.

We will also collect data on intervention fidelity at intervention sites and potential contamination at control sites. During the first 12 months of intervention implementation, we will collect fidelity data via monthly electronic surveys of local implementation leaders. Surveys will measure utilization (yes/no) of core implementation strategies in the prior month. Each month, mentors will verbally verify these yes/no survey responses (during scheduled mentor meetings). In addition, during scheduled mentor on-site visits to sites, mentors will visually verify the integration of pathways into electronic order sets. We will also collect data on potential contamination from wait-list control sites via annual electronic surveys of local implementation leaders. Surveys will ask about the use of clinical pathways or any of the implementation strategies (e.g., PDSA cycles) in quality improvement efforts for children with asthma, pneumonia, or bronchiolitis. They will also ask about any other pediatric quality improvement efforts.

Statistical methods

Our primary outcome is mean adherence the evidence-based practices of focus (Table 2). To analyze our primary outcome, we will compare changes in mean adherence from the baseline to the intervention period between the pathway intervention and control groups (difference-in-differences analysis). The baseline period will include the three winter/respiratory illness seasons before the trial, and the trial period will span 2 years (two winter/respiratory illness seasons). The unit of analysis is each participant, who will have two to three of the evidence-based practices applicable to their care during the hospitalization. We will analyze this primary outcome using multilevel linear regression models. We will include hospital-specific random effects to account for clustering within hospitals. We will include hospital-level covariates including hospital size, teaching status, and geographic region. We will also include patient-level covariates that have been associated with severity of respiratory illnesses. We have defined this list a priori based on the literature (e.g., age, sex, need for supplemental oxygen), and we will include any that are unbalanced (p < .05). We will also conduct subgroup analyses by condition (asthma, pneumonia, bronchiolitis).

We will analyze secondary outcomes using multilevel regression models with hospital-specific random effects and the same hospital and patient characteristics included in the primary analysis. We will analyze length of stay using gamma regression, and transfer to intensive care and 30-day hospital readmissions/emergency revisits using logistic regression. We will exclude data from months 1–3 of the implementation period from all these analyses to allow for intervention implementation.

We will also prospectively collect data on fidelity in intervention sites (as described above) and potential contamination in control sites. If concerns arise about poor fidelity and/or contamination, we will conduct an as-treated sensitivity analysis using these data. Missing data will be minimized via the data quality efforts described above, and we will address any residual missing data using multiple imputation.

Sample size and power

For sample size calculations, we used an alpha of .05, effect size estimates based on prior literature, and intraclass correlation coefficients (ICCs) specific to each outcome.^11,34 We used admission volume estimates based on prior multicenter studies involving community hospitals^11–13: 10 children with asthma, 15 children with pneumonia, and 15 children with bronchiolitis per year per hospital. We found that inclusion of 32 sites would give us >80% power to detect previously reported effect sizes in mean adherence to these evidence-based practice measures. We determined a final sample size of 40 sites, to account for up to a 20% dropout of hospitals (highest dropout rate reported in studies of pathways in community hospitals).^11–13 If 32 hospitals remain in the trial, we will have an estimated minimal sample size of 3840 admissions in the study (32 hospitals × 40 admissions per year × 3 years [12 months of baseline period and 2-year intervention period]).

Monitoring

This study involves hospitals implementing a pathway intervention that provides decision support to clinicians to promote current standards of care. The intervention does not involve any new therapies or diagnostic tests in participants. Participants are not expected to experience serious adverse effects. The progress of the study will be monitored by an independent Data Safety Monitoring Board (DSMB). On a semiannual basis, the core study team will prepare a written report for the DSMB on the progress of the study including data on enrollment, comparison of target to actual enrollment, overall status of study participants, information on race/ethnicity and sex, and information on any serious adverse events (SAEs). Following review of these reports, the DSMB will recommend any modifications that are required to ensure safety and enhance enrollment. Participating hospitals each have designated safety officers as well, who will help ensure the safety of participants. The study presents minimal risks to participants and interim analyses of trial outcomes are not considered crucial for the protection of human subjects.

Given the nature of this intervention, SAEs have been defined as: (1) events that lead to patient harm/morbidity (reported via hospital safety/incident reporting systems), or (2) death/mortality. Site PIs at participating hospitals will be notified to contact the core study team immediately should SAEs occur, and they will also be contacted monthly throughout the trial period via survey to inquire about any SAEs. If any SAEs are reported, the PI is responsible for reporting to the core study team, DSMB, and the UCSF IRB within 48 h. The core study team, DSMB, Site PI, and Local Safety Officers will then use root cause analysis (RCA) methods to review any events to determine attribution to the SIP intervention. A central tenet of RCA is to identify underlying problems that increase the likelihood of errors while avoiding the trap of focusing on mistakes by individuals. RCA thus uses the systems approach to identify both active errors (errors occurring at the point of interface between humans and a complex system) and latent errors (the hidden problems within healthcare systems that contribute to SAEs).

Ethics and dissemination

Research ethics approval

This trial was approved by the UCSF Single IRB (#20–30418). In line with prior similar studies,^11–13 we were granted waiver of consent. An IRB may waive the requirements to obtain informed consent provided the IRB finds and documents that: the research involves no more than minimal risk to the subjects, the waiver or alteration will not adversely affect the rights and welfare of the subjects, the research could not practicably be carried out without the waiver or alteration, and whenever appropriate, the subjects will be provided with additional pertinent information after participation.

Protocol amendments

The study protocol is available on clinicaltrials.gov (NCT05206695). The PI is responsible for communicating any protocol amendments to all relevant parties (core study team, UCSF IRB, clinicaltrials.gov, NIH, and Site PIs).

Dissemination

We have engaged the five leading national organizations involved in the care of hospitalized children in dissemination efforts. These include the Society of Hospital Medicine, the American Academy of Pediatrics, the Pediatric Research in Inpatient Settings Network, America’s Hospital Essentials, and the National Improvement Partnership Network. These partners have all committed to disseminate our study findings via organizational websites, e-mail listservs, social media, newsletters, and national seminars.

If the multicondition pathway intervention is effective, we will assemble an implementation toolkit for dissemination by these national organizations. The toolkit will contain all necessary materials for future implementation efforts, including a brief/overall summary of our findings; the multicondition pathways; details about mentored implementation; educational seminar materials; specifications on measuring evidence-based practice adoption outcomes; and guidance on implementation strategies, barriers and facilitators, and adaptions to the intervention. SHM will distribute this toolkit, and prior SHM projects have each had 400–800 downloads of such toolkits and widespread use. In addition, we will share order sets via the Agency for Healthcare Research and Quality’s Clinical Decision Support Repository. We will also seek to publish our findings in peer-reviewed manuscripts, present them at academic meetings, and present them to national, regional, and local clinicians, health administrators, and QI audiences.

The Final Research Data (the data set necessary to document and support research findings) will be made available for sharing after the main research findings from the final data set have been accepted for publication in a peer-reviewed journal (upon request to the PI). Before sharing, data will be redacted to strip all direct identifiers of hospitals (no identifiers of patients/individuals will be collected).

DISCUSSION

This hybrid effectiveness-implementation trial will evaluate a multicondition pathway intervention for children hospitalized with asthma, pneumonia, or bronchiolitis in community hospitals. Our expected outcomes will be a comprehensive understanding of how to pragmatically and sustainably implement the multicondition pathway intervention in community hospitals and an assessment of its effects on children hospitalized with respiratory illnesses, the leading causes of childhood hospitalization.

One potential limitation of the trial is the focus on inpatient care. Given hospitalized children are more ill and care is more complex, this is the focus of the SIP trial. However, we recognize that care for children with these conditions is initiated in the emergency room setting, and we will encourage partnership in implementation efforts via recruitment of an emergency room champion/leader and provision of supplemental educational resources for emergency clinicians.

The SIP trial explicitly focuses on general/community hospitals, where the majority of children in the United States are hospitalized. Thus, the trial can have an important positive impact by providing evidence on an intervention that can leverage implementation resources by tackling multiple pathways and rapidly improve care and outcomes for children with respiratory illnesses. Understanding how to successfully implement the multicondition pathway intervention in community hospitals will fill a critical knowledge gap that can enable broad-scale improvements in guideline adoption and health outcomes for hospitalized children across a wide range of common conditions.