Implementation of a Rapid Evidence Assessment Infrastructure during the Coronavirus Disease 2019 (COVID-19) Pandemic to Develop Policies, Clinical Pathways, Stimulate Academic Research, and Create Educational Opportunities

In early 2020, hospitals across the US, including Children's Hospital Colorado (CHCO), began preparing for the regional impact of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic via an Incident Command Team (ICT), a unified approach to hospital operations including the command, control, and coordination of incident management.1, 2, 3 The ICT at CHCO recognized the need for a scientific partnership for rapid uptake and synthesis of quickly accruing medical and public health literature to inform institutional policies and clinical care. Therefore, the ICT established a coronavirus disease 2019 (COVID-19) Scientific Advisory Council (SAC),⁴ a multidisciplinary team of clinician–scientists from relevant CHCO clinical divisions and CHCO's Clinical Effectiveness (CE) team members.⁵ The SAC leadership developed an academic–hospital partnership and set of processes for identification of issues, priority-setting, rapid evidence assessment, as well as synthesis and dissemination of findings. To support other organization's efforts to create a similar academic–operational partnership, this paper describes the SAC's innovative infrastructure: the diverse team and the processes the SAC developed for rapid evidence synthesis, development of recommendations and guideline documents, and dissemination of findings: the Children's Colorado Rapid Evidence Analysis and Dissemination System (CCREADS).

Organizing Framework

We used the Replicating Effective Programs (REP) framework to organize this report. The REP framework is an implementation science framework commonly used in quality improvement and health services research to guide selection, adaptation, and implementation of evidence-based interventions in real-world care settings.6, 7, 8 The REP has 4 phases: the preconditions phase involves identification of the need and determining most effective strategies; preimplementation incorporates the development, orientation, and logistics of the program; the implementation phase encompasses the roll out and dissemination of the various components; and the maintenance and evolution phase involves organizational steps to sustain the intervention in the short and long term. Figure 1 (available at www.jpeds.com) summarizes application of each phase for development and implementation of CCREADS.

Figure 1.

Application of each phase for development and implementation of CCREADS.

Preconditions Phase

Identifying the Need

In March 2020, the ICT called for rapid review of the emerging literature for high-priority questions, including screening and treatment of children with suspected or confirmed SARS-CoV-2 infection and protection of healthcare workers. To achieve this goal, CHCO assembled a team of clinician–scientists by issuing a hospital-wide request for volunteers representing all CHCO clinical and scientific sections.

Defining the SAC Mission

Soon after SAC co-leaders were selected, they met with ICT to establish a mission statement:

The mission of the SAC is to advise CHCO incident command, faculty providers, and referring provider community on clinical aspects of the COVID-19 epidemic. The SAC will review existing literature and provide timely feedback on high priority questions to incident command.

Based on input from its members, the SAC mission evolved from purely addressing the needs of the ICT to developing and updating clinical guidance for COVID-19 evaluation, management, and healthcare worker protection that were not already being addressed by other hospital teams.

Identifying Potential Barriers

We considered potential barriers to implementation of SAC and CCREADS, including the rapid growth of the literature, including accelerated peer-reviewed publications from diverse international settings and preprints not yet peer-reviewed, required a rapid evidence assessment approach. Such an approach needed to evolve from traditional literature review standards, be nimble and efficient, and consider varied levels of evidence, settings, and study populations. Although rapidly evolving, the literature featured minimal evidence specific to pediatric populations, requiring the SAC to make recommendations at times in the absence of robust data, with the need to re-evaluate recommendations as new scientific literature emerged. In a novel pandemic context, the hospital's existing clinical pathways planning and development processes needed to be significantly accelerated to respond to demand from clinicians given limited local experience, the challenges associated with a rapidly-evolving body of literature, and the absence of published guidelines.

Ensuring Fit of SAC Processes and Outputs to the Organizational Context

The SAC aligned with the ICT goals of team member safety and an evidence-based approach to patient care. In addition, the SAC aligned with the CHCO mission to encompass research and education as well as clinical and operational needs; and with the academic mission of the University of Colorado Anschutz Medical Campus. The SAC leveraged Campus scientific resources including health sciences librarians, ethicists, and dissemination and implementation scientists.

Preimplementation Phase

Formation of the Team

The SAC team initially assembled by the ICT was a multidisciplinary translational team, characterized by diversity in: (1) clinical setting—inpatient, critical care, emergency department, outpatient; (2) subspecialty—infectious disease, rheumatology, surgery, etc; (3) research expertise across the entire translational spectrum, from basic science to clinical science to public health; (4) training—including medical and surgical subspecialties, microbiology, epidemiology, respiratory therapy, pharmacology, health psychology, nursing, public health, communications, and CE; and (5) career stage—including senior and mid-career researchers as well as junior faculty, trainees and students. All who volunteered to be on the SAC were included. The SAC leadership worked with hospital and academic division leaders to fill gaps in representation by setting and specialty.

The SAC leadership recognized that the ICT's evidentiary needs generally fell into 4 categories and established 4 corresponding working groups: Clinical Course and Epidemiology, Clinical Treatment, Diagnostic Testing, and Infection Control. Members of the SAC were asked to select 1 or more working groups and then to decide on a leader within each group. Within working groups, smaller groups with relevant expertise reviewed publications as they were released to identify those of highest possible quality and relevance to the clinical questions being addressed by the SAC.

Development of Organizational Structure and Processes

After the ICT presented the first set of urgent questions to the SAC, SAC leadership developed a process for clarifying the scientific issues, priority-setting, rapid evidence assessment, and reporting findings to the ICT (Figure 2 ).

Figure 2.

Rapid evidence assessment process flow.

The SAC and ICT leadership recognized the need for an organized set of tools for the organization and dissemination of the work products, which overlapped with existing functions of the hospital's CE program. The CE team includes clinical and operational leaders, process improvement professionals, and data analysts working collaboratively with front-line caregivers to develop and oversee the Clinical Pathways program, lead large-scale quality improvement initiatives, and align clinical outcomes with emerging payment models. CE identified and assigned key team members to facilitate SAC efforts, including adaptation of existing templates and web-based development tools to improve effectiveness and efficiency overall. The CE and SAC teams developed an internal guidance to this set of processes and templates—the CCREADS—on a team web-based collaborative platform site.

Identifying the Most Effective Implementation Strategy

The SAC assessed existing resources available for addressing hospital needs and overcoming barriers. We then identified the most effective approach to rapid assessment of the literature and for adapting CE workflows to meet SAC goals.

A process for rapid review relevant to COVID-19 was established (Table I; available at www.jpeds.com), and was reviewed and refined by a team of Dissemination and Implementation experts at the University of Colorado, with resources suggested by evidence review experts at the Agency for Healthcare Research and Quality. CCREADS includes intranet Web site resources, templates, project management tracking tools, and a standardized approval process flow—all documented on the team's Team collaboration software site and accessible to SAC team members. Because of the rapidly evolving evidence, the SAC developed a scheduled updating process that included timestamps on each guidance update. The SAC's rapid guidance development documents and processes were adapted from existing CE templates and workflows, including templates for internal communication, rapid evidence review, and recommendations/guidance documents. For clinical guidelines, we created AgileMD⁹ clinical pathway algorithms and embedded in these algorithms hyperlinks to longer-form SAC guidance statements. This approach allowed us to include the more rapidly updating details of clinical management in the hyper-linked statements with fewer changes to the base algorithm. The process we developed for the SAC and ICT to review guidance documents and select dissemination strategies included a process flow map and involved coordination with CHCO communications team members. To support other academicly-affiliated hospitals in implementing an SAC to support ICT decision-making, we developed a toolkit that includes SAC processes, spreadsheets, and document templates. The SAC toolkit is available for free download from the CHCO Web site (available at: childrenscolorado.org/ScientificAdvisoryToolkit).

Implementation Phase

The entire SAC met between twice weekly to once monthly depending on the level of ICT response activation to review the status of guidance documents being developed, updates by the working groups, and incoming questions; to set team priorities; and to address working group needs for support from SAC leadership or the CE team. All SAC members reviewed each document, and feedback was used to improve the scientific rigor of each guidance document as well as the templates and processes used in their preparation. The SAC leadership met with the CE team after each meeting to further discuss implementation of the action items from each meeting and to prepare for the next ICT report-out. Report-out meetings with ICT were initially twice weekly and included a report-out template.

Clinical Pathways and Recommendations

By leveraging the collective expertise of a diverse group of experts, the SAC iteratively developed 20 clinical guidance resources to guide local care teams on screening, monitoring, treatment,¹⁰ and escalation of patients with suspected or confirmed SARS-CoV-2 or multisystem inflammatory syndrome in children (MIS-C), including 3 institutional clinical pathways (Table II; available at www.jpeds.com). Clinical pathways translate best-available evidence into an actionable format to help physicians make decisions in specific clinical circumstances.¹¹ These pathways include dynamic algorithms integrated in the electronic health record and available at the point-of care, as well as externally available to peer organizations via the hospital's Web site. They required weekly reassessment and amendment incorporating emerging scientific literature, guidelines, and local institutional data. Updates to guidance documents were logged in a SAC tracking document. This document provided the input for each SAC team meeting agenda, and action items from each meeting were added to the tracking document. New incoming questions were adapted based on a template. As a result, a rapid-cycle improvement framework was applied using CE and research community expertise to identify, critically appraise and apply findings from key studies to the local context.

Research

The SAC infrastructure afforded opportunities to conduct expedited research and quality improvement to answer questions related to COVID-19 in pediatric populations. For example, despite an extensive rapid evidence assessment, there remained uncertainty regarding the reliability of preoperative SARS CoV-2 testing in the pediatric setting, which was affecting hospital policies regarding personal protective equipment, clearance wait times, and testing protocols. We formed a perioperative taskforce comprising members of the SAC and conducted a quality assurance initiative by comparing preoperative upper respiratory samples with intraoperative upper and lower respiratory tract samples. After obtaining necessary approvals, we were able to rapidly design a protocol, implement the collection of samples, and analyze data over a 2-month period (publication under review). The SAC's multidisciplinary team meetings enable a cross-pollination of ideas and research questions, with opportunities for ongoing future collaborations. To date, SAC members have published 4 peer-reviewed manuscripts related to SAC work, as well as 2 non–peer-reviewed articles, and 2 non–peer-reviewed meeting abstracts.12, 13, 14, 15, 16

Return on Investment

Because of the benefit of the SAC to hospital operations, the SAC's structure and processes have been written into the institution's “Pandemic Playbook.” In the case of another novel threat, the SAC can then be stood up in a shorter amount of time. This was agreed upon by hospital leadership based on the following value added: (1) the SAC demonstrated that it could rapidly assess the evidence and release recommendations in a short time period—sometimes as short as 3-4 days, in contrast to the institution's typical time period of 8 months—and (2) the SAC allowed for rapid collaborations on scientific grants and national organization statements that gave the institution academic exposure. The SAC created no additional cost to the hospital as the SAC drew on members of the existing CE team. For faculty members of the SAC, the time invested was time they would have spent figuring out COVID-19–related policies and procedures for their own divisions; the SAC coordinated those efforts so that divisions were not duplicating efforts. The SAC also provided faculty the opportunity to pursue several activities relevant to academic advancement, including working on clinical committees and policy development. This contribution was supported in a letter of recognition SAC leadership sent to the department chair on behalf of each member.

Education

The SAC's evidence synthesis process became the basis for a medical student elective and included several postgraduate trainees. When traditional third-year clinical rotations were closed because of social distancing, the School of Medicine designed a COVID-19 elective course that placed students in diverse COVID-19–related settings including the SAC. Students, residents, and subspecialty fellows on the SAC worked in project teams to help research and draft initial SAC guidance. Students and trainees were mentored by senior SAC members, providing an opportunity for professional development and academic mentorship.

Information Dissemination

Additional approaches to disseminating information from the SAC's clinical guidance documents and pathways included multiple lectures given within the University. Updates on SAC activities and relevant documents were presented at section meetings, as well as the hospital's medical staff, epidemiology, and pediatric community Town Hall meetings. Materials from SAC guidance documents were incorporated into the hospital's internal frequently-asked-questions site for staff. Information generated for SAC guidance documents were incorporated into national guidelines through its members who were involved in several national organizations, including the Sharing Antimicrobial Reports for Pediatric Stewardship (SHARPS) collaborative and the International PICU COVID-19 Collaborative.¹⁷

Since inception, the median number of times SAC pathway documents are opened is 368 per month, with an IQR from 333 to 523. The most frequently accessed document is the clinical pathway for patients suspected to have MIS-C.

Maintenance and Evolution Phase

Update/refine Pathways and Guidance as Needed

Given the rapidly changing evidence related to COVID-19, the SAC continues to update its guidance statements quarterly or as new key evidence is released. Content experts continue to monitor evolving evidence and update guidance and pathways monthly or more often as applicable. The SAC is also using process and outcome metrics to further adapt the COVID-19 and MIS-C clinical pathways.

Planning for Sustainability

Based on epidemiologic projections and the clinical research pipeline—including more than 2900 COVID-19–related studies in ClinicalTrials.gov—the SAC projects that the COVID-19 pandemic curve and ongoing COVID-related scientific discovery will extend into 2022. Anticipating the potential need to scale up efforts in response to pandemic surges, the SAC will operate with flexibility depending on institutional needs as well as the pace of scientific discovery to meet future needs.

Conclusions

The SAC was developed in response to our health system–driven demand for evidence. We engaged a diverse team of scientific experts and hospital leaders in a well-coordinated and timely pandemic response that met operational and academic goals with an infrastructure that is customizable. We have developed a process for prioritizing clinical needs, developed a rapid evidence assessment method, and adapted existing CE processes for rapid implementation and dissemination. Our iterative experience can inform not only our institution's CE processes to match future evidence delivery needs, but, in addition, through sharing our processes and templates on public Web sites, we plan to disseminate our learned experience with other healthcare systems interested in replicating the SAC's framework in their specific settings.

Appendix

Table I.

Process flow for review, synthesis, approval, and dissemination of SAC guidance

Stages of CCREADS	Key components of stage
1. Framing the question	•COVID-19 question needing scientific review identified by SAC and ICT as appropriate for SAC review •Clinical/operational question distilled into research question •Question assigned to one of the working groups
2. Literature review and creation of guidance document	•Multidisciplinary team of clinicians/students conduct literature review •Team summarizes evidence using tracking tool •Team compiles evidence into guidance document using standardized format to answer question and report back to ICT
3. Guidance document review and dissemination	•Guidance document reviewed by ICT •Finalized clinical guidance document incorporating feedback from ICT •Guidance document posted internally on pathway site and disseminated hospital-wide •Documents reviewed and updated at least monthly based on new evidence

Table II.

Guidance documents and clinical pathways developed by the SAC

Formats	Clinical area	Topic
Guidance documents	Therapy	• Antiviral therapy • Steroids • Immune modulation • Anticoagulation • Convalescent plasma • Ibuprofen • Chloroquine and hydroxychloroquine • ACE inhibitors and ACE-receptor blockers • IL-6 inhibition • IL-1 inhibition • Complement inhibition • Intravenous immunoglobulin
	General management	• Tracheostomy/ventilator management
	Infection control	• Transmission risk mitigation during minimally invasive surgery • Transmission risk mitigation through pre operative testing • Thermometry methods
Clinical pathways		• Multisystem inflammatory syndrome in children (MIS-C) • Acute management of COVID-19—emergency department • Acute management of COVID-19—inpatient

ACE, angiotensin-converting enzyme; IL, interleukin.