Rationale and methodology of a collaborative learning project in congenital cardiac care

Abstract

Background

Collaborative learning is a technique through which individuals or teams learn together by capitalizing on one another’s knowledge, skills, resources, experience, and ideas. Clinicians providing congenital cardiac care may benefit from collaborative learning given the complexity of the patient population and team approach to patient care.

Rationale and development

Industrial system engineers first performed broad-based time-motion and process analyses of congenital cardiac care programs at 5 Pediatric Heart Network core centers. Rotating multidisciplinary team site visits to each center were completed to facilitate deep learning and information exchange. Through monthly conference calls and an in-person meeting, we determined that duration of mechanical ventilation following infant cardiac surgery was one key variation that could impact a number of clinical outcomes. This was underscored by one participating center’s practice of early extubation in the majority of its patients. A consensus clinical practice guideline using collaborative learning was developed and implemented by multidisciplinary teams from the same 5 centers. The 1-year prospective initiative was completed in May 2015, and data analysis is under way.

Conclusion

Collaborative learning that uses multidisciplinary team site visits and information sharing allows for rapid structured fact-finding and dissemination of expertise among institutions. System modeling and machine learning approaches objectively identify and prioritize focused areas for guideline development. The collaborative learning framework can potentially be applied to other components of congenital cardiac care and provide a complement to randomized clinical trials as a method to rapidly inform and improve the care of children with congenital heart disease.

Background

Collaborative learning is the process by which 2 or more people attempt to learn something together. As opposed to individual learning, collaborative learning promotes sharing experience, resources, skills, and techniques to enhance performance.^1,2 Collaborative learning is based on a model whereby knowledge can be created within a group where members interact and share experiences about a specific process.³ Its application in medicine is more closely aligned with benchmarking, a method of comparing services or outcomes at an individual center with other leading centers.^4–7 Experience from adult cardiac care, such as the New England Cardiovascular Disease Study Group and the Michigan Society of Thoracic and Cardiovascular Surgeons Quality Collaborative, suggests that many “best” institutional practices can be disseminated and adopted across multiple participating centers through a collaborative learning model, leading to improvement in quality of care.^8,9 Within pediatric cardiology, the National Pediatric Cardiology Quality Improvement Collaborative has created a national network using both providers and parents to promote collaborative learning to improve outcomes.^10–15

Success in collaborative learning requires careful coordination and stewardship commitment among the different stakeholders. It remains underused in pediatrics, but congenital cardiac care in particular stands to benefit from collaborative learning because of the small number of patients, the complexity and heterogeneity of disease processes, and geographically dispersed sites of care. All these factors create challenges for more traditional research efforts. In addition, collaboration is a natural aspect of the multidisciplinary nature of the clinical teams that care for these patients, which include anesthesiologists, cardiologists, intensivists, surgeons, nurses, and others. Collaborative learning involves evaluating and disseminating beneficial practices quickly. This is especially true for complex processes that require the input of many disciplines.

The Pediatric Heart Network (PHN), sponsored by the National Heart, Lung, and Blood Institute, was formed expressly to conduct clinical research. The Single Ventricle Reconstruction Trial, which examined the outcomes of infants with hypoplastic left heart syndrome palliated with 2 different operative techniques, required sharing of some institutional experience among investigators to develop a study protocol that could be carried out across the network.¹⁶ However, such institutional sharing was incidental to the development of the protocol and not the primary focus. Several recent investigations, including post hoc analysis of data from the Single Ventricle Reconstruction Trial, highlighted dramatic variation in clinical practice and outcomes among participating centers.^17–20

Interest in reducing this variation as a potential means of improving outcomes provided the impetus for exploring the collaborative learning model within the PHN. Although collaborative learning and site visits have been applied on a local geographic level in other surgical fields,^8,9 this process has not yet been applied to pediatric cardiac care at a national level. Furthermore, engaging system engineers to objectively observe and prioritize potential areas for focused collaborative learning had not yet been used. Therefore, we designed this study to test the feasibility of collaborative learning with the ultimate goal of developing a clinical practice guideline (CPG) that minimized variation and improved clinical outcomes at participating sites. This process included data sharing and meta analytics, benchmarking, machine learning to prioritize potential areas, site visits, and practice dissemination at 5 large academic congenital heart centers in order to change practice, test hypotheses, and improve outcomes (Figure 1).

Figure 1.

Collaborative learning process.

Study development

Rationale

The Collaborative Learning Study endeavored to identify outcome variation among centers related to differences in perioperative care that could be modifiable through a collaborative learning intervention and would minimize variation and improve quality of care. The initiative targeted infants undergoing heart surgery because this population accounts for a large proportion of morbidity, mortality, and resource utilization in pediatric cardiac centers. The entire care process was observed and evaluated including diagnosis, surgical planning, intraoperative technique, postoperative care, and the transition to outpatient care. All clinical processes that were interdependent and logically associated with patient outcomes were included in the initial evaluation. In choosing metrics for outcome measurement, factors such as center-specific surgical volume, acuity, mortality, and postoperative length of stay were taken into account.

Initial site visits

Five core PHN sites were enrolled as Collaborative Learning Study sites (Table I). Each site appointed a multidisciplinary team consisting of a cardiac intensivist and/or cardiac anesthesiologist, a cardiothoracic surgeon, a respiratory therapist, an intensive care unit (ICU) nurse, and a quality improvement specialist. These 5 clinical sites, along with a team of industrial systems engineers from the Georgia Institute of Technology (Georgia Tech), formed the collaborative learning team. The system engineers played an integral role in our study. They provided an unbiased comprehensive evaluation of the existing system processes at each center and prioritized potential areas for deep learning and transformation. Using their expertise in quality improvement, operations efficiency, and patient safety, the engineers studied workflow, clinical care, and process interdependencies at each center to identify strengths and limitations.

Table I.

Pediatric Heart Network core sites participating in CPG

1. Children’s Hospital of Philadelphia

2. Emory University

3. Texas Children’s Hospital

4. University of Michigan

5. University of Utah–Primary Children’s Hospital

CPG, Clinical practice guideline.

The strategy was to observe a broad range of activities and to pair the resulting information with center-specific outcome data to identify high-leverage practices for CPG development that would impact clinical outcomes such as postoperative length of stay (LOS). Pairing of information gathered with outcomes was guided by previously published studies that associated clinical variables with outcomes, and prior experience of the system engineering team observing and studying pediatric, adult, and Veteran’s Administration health care systems.²¹

The collaborative learning project began with site visits to each participating center by a group of physicians from Emory University and the industrial engineering team from Georgia Tech. These comprehensive visits lasted 5 days and involved detailed system and process observations by the engineers on resource availability and usage, decision paths, and care coordination surrounding the intra- and postoperative care of patients. Specifically, the system engineers directly observed and analyzed ICU processes including clinical decision making by treatment teams during rounds and handoffs, coordination of discharge planning including transfer of care from ICU to step-down unit, and family education before discharge. Video analysis documented the efficiency of key components of clinical care, such as operating room (OR) to ICU handoff. Interviews conducted with physician, nursing, and respiratory staff gathered information about system processes in place at each participating institution. A system workflow model was established for each site. These models were subsequently contrasted to identify practice variance and potential areas for clinical experimentations with the ultimate goal of optimizing safety, quality, and cost-effectiveness.

Other specific areas of analysis included ancillary resource availability such as respiratory therapy, timeline for postoperative extubation, collaboration between surgical and ICU teams, surgical decision making in the ICU, postsurgical care in the ICU, and sedation and postoperative feeding protocols. The engineering team compiled the information from all sites, documented commonalities, and captured differences in practices and protocols. They created process maps for each site that established interdependence, decision points, and potential outcome factors. Consistent modeling was used across the participating institutions. This included time-motion studies of OR, ICU, and step-down unit care. For all observations, the engineers identified the parties responsible for a particular component of care.

Participating centers shared their clinical pathways and protocols for perioperative management. Surgical and ICU clinical outcomes data from the Society of Thoracic Surgeons and Virtual PICU Systems databases, respectively, were collected. The data from the site visits including team logs, interview results, industrial system engineer analysis, center-specific protocols, and outcomes data were compiled and analyzed to identify a practice where collaborative learning might be applied to minimize variation and improve quality of care.

CPG development

Once the initial site visits were completed, site reports were drafted, and institutional clinical data were analyzed. The reports categorized clinical practices such as extubation practices and feeding protocols based upon outcome variability present among centers (Figure 2). To identify a target practice for collaborative learning, the team considered several critical variables. These criteria are shown in Table II.

Figure 2.

Clinical practices considered for collaborative learning initiative.

Table II.

Criteria to identify clinical practice suited for collaborative learning initiative

1. Varies among centers

2. Correlated to important clinical outcome

3. Is systems based

4. Difficult to disseminate by publication alone

5. May result in reduced resource use

Practice variation among centers for relevant clinical practices was stratified by 3 tiers: high, moderate, and low. Clinical outcomes were correlated to relevant clinical practice using standard tests of significance including linear and logistic regression, contingency tables, and rank sum tests. For the purposes of this project, systems-based was defined as “involving multiple providers from various disciplines.” Only systems-based practices were considered for the collaborative initiative.

Using the relevant comprehensive site visit data and extensive discussions, the group worked to establish a consensus focal area for practice change and implementation to observe potential impact on quality and patient care. The analysis included a 2-staged approach: (1) an objective system-engineering approach to mine the data, process interdependencies, and outcomes and (2) a clinical approach via subjective practice experience and deep learning to establish a hypothesis for potential implementation.

System approach via machine learning to prioritize various factors and potential benefits

Combining observations of the site teams, video analysis, interviews, and both subjective and objective data from each center, we identified 7 major system factors that were possible targets for development of a CPG: preoperative preparation, surgical procedure, time to extubation, surgical lines and drain removal, ICU and step-down unit care, and discharge planning.

Machine learning analyses pinpointed the key factors (among the 7) that were most predictive of the LOS. Specifically, 24 different classifiers were used, including optimization based discriminant analysis model (DAMIP)^22,23, logistic model decision tree, random forest, support vector machine, naive Bayesian, k-nearest neighbors, and Bayesian network. Running feature selections on these 7 key factors on all the classifiers, we selected the top 50% of all the models and calculated the Gini importance²⁴ on the features to determine their significance in predicting/influencing LOS. Table III shows that ICU care, step-down unit care, and early extubation were the top 3 impactful features. ICU and step-down care (Table IV) consist of varying care protocols, multiple resource use and staffing, and complex medication and care coordination. Their similarities and complexities among the 5 sites made them less appealing as targets for establishing a new CPG.

Table III.

Gini importance scores for factors predicting the LOS (the higher the score, the more important the factor)

Factor	Discharge planning	Chest tube removal	Duration of mechanical ventilation	ICU care	Step-down unit care
Total Gini importance score	70	72	82	89	93

Table IV.

Intensive care unit and step-down unit protocols

	ICU care	Step-down unit care
Similarities	Rounding team composition Weekend coverage Blood transfusion parameters Postoperative: • Fluid management • Inotropic support	High-flow nasal cannula allowed Chest tubes allowed Select intravenous inotropes allowed Discharge coordinator involvement Pharmacist educator
Differences	Overnight coverage Neonatal feeding advancement protocol Ventilator weaning protocol	Arterial line allowed Care team composition Discharge education and timeline

Time to extubation, however, showed significant variation concerning practice approach, staffing models, and actual timing across all the institutions. Data regarding duration of postoperative ventilation gathered from institutional databases for a sample period of 2010-2011 showed an institutional median duration of mechanical ventilation post–tetralogy of Fallot repair of 19.8 hours with institution-specific ranges of 0-26.1 hours and institutional median duration of mechanical ventilation following coarctation repair of 20.6 hours with institution-specific ranges of 0-27.8 hours.

Time to extubation is a process that can be conducive to adoption of a new protocol after which its impact on duration of mechanical ventilation, LOS, and other clinical metrics can be determined. Based on the machine-learning results, we synthesized a potential clinical strategy to reduce LOS through shortening the duration of mechanical ventilation. Simulation was then performed to analyze the potential improvement over the current practice.

Clinical approach via rotational site visit and deep learning

In addition to the objective system engineering data, we observed that one participating center consistently extubated infants and children safely immediately following cardiac surgery based on data available at the time of study development. Based on this clinical observation and the results from the machine learning analysis, we therefore chose practices related to timing of extubation as a target for the collaborative learning project. Early successful extubation has the potential to reduce ICU and hospital LOS and improve other outcomes.^25,26

Rotational site visits

Multidisciplinary team rotational site visits were used to explore early extubation for potential CPG development. The rotational model allowed each institution to visit and host another participating institution (Figure 3). By design, the institution visited and the institution hosted were not the same. Site visits were 2-3 days and were completed within a 6-week period to facilitate uninterrupted knowledge exchange, deep learning, and practice comparison and to heighten the value of the collaborative learning experience.

Figure 3.

Map of study centers for rotational site visits. Total distance traveled: 4500 miles/7242 km.

Rotational site visits included observation in the operating room and ICU, formal interviews with staff at each site, and informal discussion between teams. Each visiting clinical team member was paired with a team member at the host institution to facilitate informal shadowing and observation in the clinical setting. In addition, there were formal meetings arranged with directors in clinical areas that would be key to CPG implementation. These experiences together provided the optimal opportunity for observation of clinical practices and protocol sharing (Table V).

Table V.

Sample itinerary for rotational site visit

Day 1
6:30–7:30 AM	Meet and greet breakfast
7:30–8:30 AM	Paperwork for visiting team
8:30–11:30 AM	CICU medical rounds Operating room observation Cardiac catheterization laboratory observation
11:30 AM–12:30 PM	Lunch with Director of Inpatient Nursing
12:30 PM–2:00 PM	Tour of inpatient heart center
2:00 PM–3:00 PM	Meeting with Medical Director of Quality
4:30 PM–5:15 PM	Meeting with Chief of Pediatric Cardiothoracic Surgery
7:00 PM–9:00 PM	Dinner at restaurant with visiting and host teams
Day 2
6:30 AM–7:30 AM	CICU surgical rounds
8:30 AM–11:30 AM	CICU medical rounds OR observation Cardiac catheterization laboratory observation
11:30 AM–12:30 PM	Lunch with Medical Director Cardiac Step-Down Unit
1:00 PM–1:45 PM	Meeting with Director of Pediatric Cardiac Anesthesia
2:00 PM–2:45 PM	Meeting with Medical Director of CICU
3:00 PM	Depart for airport

CICU, Cardiac intensive care unit.

In parallel, questionnaires developed by the Emory University team were sent to all participating sites. These included specific anesthesia, OR, and ICU practice questions about perioperative mechanical ventilation and extubation. The data collected from each center were merged with the system engineers’ observation summary into a database for comparison of various practices. The data were analyzed with the goal of developing a CPG for postoperative extubation that could be implemented within the existing staffing, scheduling, and care framework at each institution (an example of the ICU questionnaire is included in Appendix A).

Clinical practice guideline

Bimonthly conference calls took place through the period of rotational visits followed by an in-person meeting of all team members from all 5 sites. This allowed all the centers to share experiences, including viewing a video of extubation of an infant immediately following a cardiac procedure. Data from questionnaires, experiential learning, and data collected by the engineering team formed the foundation for development and implementation of a CPG to promote early extubation after infant cardiac surgery (Figure 4).

Figure 4.

Timeline for collaborative learning project development.

The CPG set a goal to extubate patients within 6 hours of ICU admission after 2 common infant cardiac surgical procedures: complete repair of tetralogy of Fallot and repair of isolated coarctation of the aorta via thoracotomy. Patients with these lesions tend to have an uncomplicated early postoperative course and typically have physiology that would both allow for and benefit from early extubation. The CPG was designed to promote an approach to anesthesia, OR, and ICU practices for the index operations that allows for early extubation.

The CPG was implemented at each individual center after review by the PHN’s Data Safety Monitoring Board, approval from the individual center’s quality committees, and institutional review board approval for data collection from subjects. The need for informed consent was waived by the institutional review board because of the quality improvement nature of the CPG and the agreement among all participants that the practice change should lead to better overall patient care.

Summary of analytic plan

The overarching analytic plan involves comparing early extubation rates at hospitals participating in the collaborative learning process before and after CPG implementation. Five other core PHN centers were used as control sites, as they did not participate in the collaborative learning activities. These sites were included to help control for secular trends in postoperative extubation practices during the study period. Data collected at the control sites include retrospective and prospective data concurrent with the time periods at the study sites. Data collection was completed in May 2015, and analysis is currently ongoing and will include any changes in outcomes for survival and morbidity, and a cost-benefit analysis based upon the shortened durations of mechanical ventilation. Compliance and knowledge of the CPG by the care team will also be captured. The results of this data analysis will be published separately.

Discussion

We report the design of a national collaborative learning project in congenital cardiac care that included a unique approach of multidisciplinary team site visits, protocol sharing, data collection and analysis, formal and informal discussions and interviews, and objective system and process evaluations. Computer simulation and machine learning were used to objectively analyze data to identify and prioritize potential areas for clinical practice change. The collaborative learning method facilitated the development of a consensus CPG for early extubation after 2 common infant congenital heart surgical procedures across 5 centers where there was variation in both practice and outcomes surrounding perioperative ventilation. In some centers, significant practice changes were required from multiple disciplines to execute the CPG.

The PHN’s collaborative learning protocol represents an attempt to improve outcomes of children with congenital heart disease using a time-tested approach. We started with the basic premise that sharing information between centers performing congenital heart surgery is valuable. Formal and informal sharing of practices and protocols, even those outside the parameters of the CPG, highlights an important aspect of the collaborative learning approach. Teams returned from their rotational visits with new information and ideas for potential implementation into their own practice model.

Our group emulated seminal work in the field of collaborative learning performed by the New England Cardiovascular Disease Study Group⁸ by employing industrial system engineers from Georgia Tech. Engineering observations from video analysis and interviews provided the collaborative teams with unbiased process maps and system perspectives across each site. These process maps identified hospital system interdependencies and areas of inefficiency within the workflow model in each participating center, many of which contributed substantively to CPG development. The system simulation model and machine learning approaches offered objective analysis to identify and prioritize the area of focus for CPG development.

Innovative approaches

There are 2 unique aspects of this study. First, this is the first time machine learning was applied within a collaborative learning framework to prioritize clinical processes for potential CPG development. This system-engineering approach offers an unbiased understanding of potential benefits and tradeoffs before actual implementation is determined. Second, this study involved rotational site visits. These proved to be a valuable deep learning experience for all participants. We considered several options for conducting the site visits. One potential model involves identifying a primary center of excellence and having all other centers visit that center. The major downside to this approach is that the reciprocal nature of a true learning collaborative may be lost. Every institution has some aspect of care that is novel and may represent an improvement over conventional approaches. This is especially important if the learning collaborative intends to identify a series of practices that may be addressed in future studies. For example, one center may be very innovative in the process of postoperative extubation, whereas another center may have an extremely effective family teaching and discharge process. Rotational site visits increase the likelihood of observing these practices.

Although we chose to use a rotational model in which each center traveled to one other center and each hosted one other center, ideally, one would undertake visits to all other centers in the collaborative. The main barriers to such a model are logistics and expense, but our experience was that there was minimal burden on the hosting site.

Conclusion

We report the successful development of a national collaborative learning project with 5 large pediatric congenital cardiac centers cooperating to develop a CPG aimed at reducing variation in the duration of postoperative mechanical ventilation. All participating centers implemented the CPG even though this represented a significant change in practice at some sites. In addition, the in-person site visits and frequent conference calls have provided learning and practice changing opportunities outside the clinical domain encompassed by the CPG. Our process has set the stage for ongoing collaboration and data sharing among centers after completion of this initial collaborative learning project.

We would like to acknowledge Becky Kinkead, PhD, Emory University School of Medicine, for her contributions to this article, as well as Niquelle Brown, Cory Girard, Jinha Lee, Kevin Yee, TsungLin Wu, and Ruilin Zhou from Georgia Tech for performing time-motion and system process study on this project.

Appendix A. Cardiac ICU questionnaire

Cardiac ICU checklist

Case mix
Preoperative patients managed in CICU
Preoperative monitoring
Preoperative infection prophylaxis for sternal wound and central line infection (CHG baths and mupirocin)
Preoperative management of high Qp:Qs in HLHS (sub ambient O2, inhaled CO2, etc)
Preoperative parent education
Preoperative central venous access for TPN
Staffing
Training & board certification
Number of full time attendings
Weekday coverage
Weekend coverage
Night time coverage
Fellows/trainees
Number
Role
Type
Use of midlevel practice providers
Role of neonatology in management of neonates
Role of cardiology and cardiology subspecialists (EP, Adult Congenital, Heart failure, etc) in CICU
Rounds
Review of daily CXRs, echos and telemetry on rounds
Surgical rounds vs medical rounds
Role of surgeons in decision making
Team composition on rounds
Delegation of responsibility
Use of EMR and/or supplementary data forms to present summary data
Written summary of daily plan of care
Time per patient
Teaching
Daily quality checklist: need for CVL, arterial line, Foley, labs, meds, etc, reviewed on a daily basis
Family presence and participation in rounds
Family presence during emergent situations—specifically codes
Documentation
Documentation platform
Responsibility
Detail
Timeliness
CICU database—what is tracked and who maintains this
Handoffs
Required participants
Protocol for handoffs
Communication strategy
Required elements to handoff
Time for handoff
Postoperative order sets in EMR
Monitoring
Hemodynamic monitoring for common lesions
Norwood
NIRS
Continuous mixed venous
Arterial line
Umbilical vs radial
Left atrial line
Right atrial line
Other central lines
Pacing wires
Blood gas (arterial, venous) and lactate frequency
Usage of ETCO2 monitoring
Tetralogy
NIRS
Continuous mixed venous
Arterial line
Left atrial line
Right atrial line
Other central lines
Pacing wires
Blood gas (arterial, venous) and lactate frequency
Usage of ETCO2 monitoring
Postoperative ventilation
Delegation of responsibility for weaning/extubation
Ventilator weaning protocols
Use and strategy of “sprints” or CPAP trials
Role of noninvasive support
Use of extubation readiness assessment tools
Criteria procedures and personnel for removing
Pacing wires
Atrial lines
Chest-tube/Jackson Pratt drain use and removal criteria
Postoperative antibiotic standard use
Anticoagulation practices for indwelling catheters
Anticoagulation for shunt dependent physiology
Lab and CXR frequency
Fluid management
Standard postoperative fluid administration (neonate)
POD 1
POD 2
Use of peritoneal drains and dialysis in infants undergoing surgery with
CPB
Routine/Periodic/Never
Diuretic usage—how early and how often? Continuous infusion vs bolus
Nutrition
Preoperative feeding
Oral vs tube
Enteral feeds
With UA catheter
With PGE
Feeding readiness evaluation
Formal evaluation of swallowing
Speech and Occupational therapy
Use of feeding protocols
Postoperative feeding advance
Protocol
Adherence to protocols
Use of TPN
Use of nasogastric feeds (neonatal surgery with CPB)
Use of g-tube
Use of Nissen
Personnel procedures in CICU
Intubation
Chest tubes
Parent education in CICU
General: infant CPR, medication administration, tube feedings, etc
Lesion-specific education including protocols for shunt-dependent patients
Involvement of ancillary services and consultants: PT, OT, Speech, Lactation, Social Work, Palliative Care
Discharge to step-down unit
Defined criteria for transfer
Communication of management plan