Implementation Science Research in Pediatric Critical Care Medicine

Abstract

Objective:

Delay or failure to consistently adopt evidence-based or consensus-based best practices into routine clinical care is common, including for patients in the pediatric intensive care unit (PICU). PICU patients can fail to receive potentially beneficial diagnostic or therapeutic interventions, worsening the burden of illness and injury during critical illness. Implementation science (IS) has emerged to systematically address this problem, but its use of in the PICU has been limited to date. We therefore present a conceptual and methodologic overview of IS for the pediatric intensivist.

Design and Methods:

The members of ECLIPSE (Excellence in Pediatric Implementation Science; part of the Pediatric Acute Lung Injury and Sepsis Investigators Network) represent multi-institutional expertise in the use of IS in the PICU. This narrative review reflects the collective knowledge and perspective of the ECLIPSE group about why IS can benefit PICU patients, how to distinguish IS from quality improvement (QI), and how to evaluate an IS article.

Results:

IS requires a shift in one’s thinking, away from questions and outcomes that define traditional clinical or translational research, including QI. Instead, in the IS rather than the QI literature, the terminology, definitions, and language differs by specifically focusing on relative importance of generalizable knowledge, as well as aspects of study design, scale, and timeframe over which the investigations occur.

Conclusions:

Research in pediatric critical care practice must acknowledge the limitations and potential for patient harm that may result from a failure to implement evidence-based or professionals’ consensus-based practices. IS represents an innovative, pragmatic, and increasingly popular approach that our field must readily embrace in order to improve our ability to care for critically ill children.

Introduction

Historically, in adult medical practice it is concluded that 17 years elapses before an evidence-based practice (EBP) is routinely adopted into clinical care (1, 2). As such, recommendations may easily be out of date before they reach their intended audience or, possibly, never reach their audience at all. The uptake of EBPs is challenged by academia’s traditional focus on rigorous clinical studies in highly specific patient populations without prioritizing the next step of wider implementation for broad impact. Hence, addressing this “research to practice gap” is the core concept underlying the field of implementation science (IS) (3–5). Academic and research funding bodies increasingly recognize the value of IS to advance more widespread public health impact of clinical research and quality improvement (QI) initiatives (6, 7).

In 2021, one of the articles celebrating the 50^th Anniversary of the journal Critical Care Medicine reviewed “The coming of Age of Implementation Science Research in Critical Care Medicine” (8). In fact, work in adult intensive care units (ICUs) suggests specific reasons why critical care research often fails to translate into clinical practice (e.g., the complexity of ICU team dynamics when attempting behavior change, and challenges in overcoming ICU culture) and how IS can help to overcome these obstacles. While the practice of pediatric critical care likely faces some of these challenges, the pediatric ICU (PICU) differs from adult ICUs in important ways. For example: 1) the comparatively limited evidence-base to guide care; 2) the logistical and ethical obstacles to traditional randomized trials in children; and 3) the relative lack of resources available for pediatric research. The ECLIPSE (Excellence in Pediatric Implementation Science) group of the Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) Network(9) believes that many of the research challenges faced by the pediatric intensivist can and should be addressed using IS methodology. However, there is a paucity of IS research in pediatric critical care.

Therefore, in this narrative review we aim to describe core elements of IS investigations that may help improve utilization of IS methodology in the PICU. Hence, this review will: 1) define IS; 2) describe the basic terminology used in IS research; 3) review the distinctions between IS and QI; and 4) guide readers of Pediatric Critical Care Medicine through their evaluation of an IS study.

Defining Implementation Science

In general, IS is defined as “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice, and, hence, to improve the quality and effectiveness of health services” (10). As such, IS falls within the scope of translational research, that occurs at the end of the research pipeline (4, 11).

Initially, potential interventions are tested in efficacy trials using homogenous populations and highly controlled conditions. Effectiveness trials are used subsequently to determine if the intervention works in a more realistic context (11), but with ongoing supervision and support from researchers. Thereafter, in the past, it was assumed that effective interventions would automatically be broadly applied in clinical practice. However, health care researchers have repeatedly demonstrated that this assumption is flawed. Even the most effective interventions require systematic efforts to achieve integrated and sustained use within the real-world clinical setting. For example, one major limitation in the conventional clinical research pipeline is at the time of facilitating widespread uptake of an effective practice or intervention. There is often no additional resource, such as staff, for implementing any new evidence once the research protocol has been completed. This funding gap means that without specific examination of how to implement a new EBP in the real world, researchers will not understand what parts of the new intervention either prove troublesome or are straightforward outside the ideal setting of a research project.

The field of IS was developed as an answer to this problem. That is, the need to understand how to ensure widespread use of EBPs beyond simple publication. Put simply, the early part of the EBP research pipeline focuses on patient-level health outcomes, such as mortality rate or duration of mechanical ventilation, and answers the question “what is the best treatment for disease X?”. IS comes at the end of this pipeline addresses how frequently a new EBP is used, how well it is used, and how to make its use sustainable in the real world. That is, IS asks “how do I ensure all eligible patients receive the current evidence-based treatment for disease X?”

The language used in IS research

Learning about IS requires familiarity with its core terminology and understanding the subtleties that distinguish concepts like “intervention,” “implementation,” and “dissemination.” These terms are often used interchangeably outside of IS but have specific meaning and implications when applied in IS work (Table 1). In addition, IS employs familiar-sounding concepts like “theories,” “frameworks,” and “models” in highly specific and precisely defined ways to produce generalizable knowledge. IS theories, frameworks, and models can be categorized as serving to create generalizable knowledge of three types: 1) to describe or guide the process of translating research into practice; 2) to understand or explain what influences implementation success or failure; and 3) to evaluate the implementation process (12–22). IS also focuses on outcomes that are distinct from those encountered in typical clinical research, including acceptability, feasibility, and fidelity (21) (Table 2).

Table 1.

Glossary of implementation science

Term	Definition	Example
Intervention, implementation, and dissemination
Intervention	The evidence-based, or consensus-based, clinical practice, policy, or program that a particular patient population will benefit from but is currently under-utilized(13)	The timely use of antibiotics for septic shock
Implementation	The integration of an intervention into a specific setting or context(13)	Sepsis huddle to ensure no barriers to timely administration (such as IV access or stat ordering)
Dissemination	The distribution of an intervention to a specific audience (13)	Creation of a toolkit to facilitate widespread adoption of sepsis huddles for timely antibiotics in pediatric critical care centers
Theory, model, and framework
Theory	A set of analytical principles or statements designed to structure our observation, understanding, and explanation of the world.	Theory of diffusion (14) Organizational theory(22)
Model	A deliberate simplification of a phenomenon or a specific aspect of a phenomenon. Models can be described as theories with a more narrowly defined scope of explanation; a model is descriptive, whereas a theory is explanatory as well as descriptive.	The Knowledge-to-Action model(15), the Ottawa model(16), the ACE Star Model of Knowledge Transformation(17)
Framework	A structure, overview, outline, system or plan consisting of various descriptive categories, (e.g., constructs or variables) and the relations between them that are presumed to account for a phenomenon. Frameworks do not provide explanations; they only describe empirical phenomena by fitting them into a set of categories.	Consolidated Framework for Implementation Research (CFIR)(18); Theoretical Domains Framework (TDF)(19); Promoting Action on Research Implementation in Health services (PARIHS)(20); Proctor outcomes framework(21)

Table 2.

Definition of outcomes used in Implementation Science

Outcome	Definition
Acceptability	The perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable, or satisfactory
Adoption / Uptake	The intention, initial decision, or action to try or employ an innovation or evidence-based practice
Appropriateness	The perceived fit, relevance, or compatibility of the innovation or evidence-based practice for a given practice setting, provider, or consumer; and/or perceived fit of the innovation to address a particular issue or problem.
Cost	The cost impact of an implementation effort, related to the costs of the particular intervention, the implementation strategy used, and the location of service delivery
Feasibility	The extent to which a new treatment, or an innovation, can be successfully used or carried out within a given agency or setting
Fidelity	The degree to which an intervention was implemented as it was prescribed in the original protocol or as it was intended by the intervention developers
Penetration	The integration of a practice within a service setting and its subsystems; can be calculated in terms of the number of providers who deliver a given intervention, divided by the total number of providers trained in or expected to deliver the intervention, or by the number of eligible patients who receive the intervention divided by the total number of patients eligible for the intervention
Sustainability	The extent to which a newly implemented treatment is maintained or institutionalized within a service setting’s ongoing, stable operations

Distinguishing IS from quality improvement

There is an important overlap between IS and QI: both have a shared goal of improving the quality of care delivered to a patient. However, the two fields diverge in how to accomplish this goal (Table 3). At its core, IS seeks to shorten the research-to-practice-gap of EBPs as the primary mechanism to improve patient outcomes, while QI is designed to solve problems that may or may not relate to a specific EBP(23, 24). IS seeks to develop generalizable knowledge that can be applied beyond the area of study and in multiple settings. QI in contrast does not seek to generate generalizable knowledge, but instead focuses on improving a specific problem in a system of healthcare practice (5). An IS approach offers insight into the mechanisms of behavior change and examines contextual factors that impact implementation, grounded by specific scientific theories, frameworks, or models (25); while such detailed study of context is typically not a part of QI projects. Unlike outcomes that are examined in QI (such as a particular clinical outcome, i.e., rates of unplanned extubations; as well as process and balancing metrics, i.e., rates of reintubation), IS outcomes pertain to adoption and use of the intervention itself, and include measures such as feasibility, fidelity, and sustainability (Table 2). While QI tends to move quickly, with rapid tests of change and repeated plan-do-study-act cycles, IS requires more time to complete formal assessment of context and to generate theory-based generalizable knowledge (23).

Table 3.

Quality improvement vs implementation science

	Quality improvement	Implementation Science
Underlying assumption	Evidence-based practices, benchmarks, guidelines exist to optimize care or there is variability in care or outcomes	Evidence-based practices, benchmarks, guidelines exist to optimize care
Primary problem to address	Improvement needed in the performance of a specific, local problem; at times based on evidence-based practice, benchmarks, or guidelines	Evidence-based practices, benchmarks, guidelines have not been widely adopted by clinicians or patients
Generalizability	Applicability to the local setting is the focus, accounting for specific context; often not replicable in different contexts or at a large scale	Applicability to multiple settings is essential, as is characterization of context
Use of theoretical models	Sometimes, but not essential	Essential
Tools	Improvement models: Six Sigma, IHI Model for Improvement Specific tools: key driver diagrams, process mapping, audit and feedback	Theories, Models, and Frameworks: Consolidated Framework for Implementation Research (CFIR), Exploration, Preparation, Implementation, Sustainment (EPIS), Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) and many others
Study structure	Hypothesis-generating studies (often qualitative); prospective experimental and quasi- experimental studies focused on effectiveness outcomes (qualitative/ quantitative/ mixed methods)	Hypothesis-generating studies (often qualitative); prospective experimental and quasi- experimental studies focused on implementation outcomes or hybrid outcomes (qualitative/quantitative/ mixed methods)
Timeframe	Short term initially: improvement can be rapid with small tests of change (Plan-Do-Study-Act cycles). Subsequently, a longer-term focus on sustainability.	Medium to long term initially: the scientific approach (planned implementation accounting for contextual factors and use of mixed methods) can lengthen time. Subsequently, a longer-term focus on sustainability.
Measures	Outcomes, processes, and balancing metrics displayed in control and run charts	Effectiveness and/or implementation outcomes depending on study design and statistical analysis plan; implementation outcomes include measures such as acceptability, feasibility, appropriateness and others.

Essential elements of an IS publication

To standardize the quality and reporting of IS studies, an international group of IS experts developed the Standards for Reporting and Implementation Studies (StaRI) statement (26). We highlight the core components of the StaRI statement and provide some examples of IS work in the PICU.

The manuscript should first clearly describe the EBP of interest, the evidence behind it, and demonstrate the research-to-practice gap now justifying the IS approach.

Lane-Fall et al offer a useful schematic highlighting that efficacy and effectiveness research should ideally be completed prior to formal implementation studies (4). Hence, by definition, IS first requires a substantial body of evidence before to embarking on studies, i.e., the “what to do” should be well-established before asking questions about “how.” In many PICU-based interventions, however, there may be some benefit from an IS perspective without first demonstrating the strength of evidence for both efficacy and effectiveness in critically ill children. The clinical reality is that there are relatively few large randomized clinical trials in critically ill children(9, 27, 28) which means that many ‘informed’ decisions in pediatric critical care likely use adult data or lower quality pediatric evidence (29). Furthermore, our field has developed a variety of consensus and EBP guidelines to shape practice (30–35). Therefore, studying the implementation of these adult data-driven approaches, or pediatric guidelines are appropriate uses IS methods in pediatric critical care. Therefore, in the PICU, an IS publication should either make a clear case that the practice of interest has a reasonable evidence base to justify studying its implementation, or to make a compelling case for minimal harm/high likelihood of benefit for studying the implementation of a consensus-based guideline.

The methods should identify what aspect of IS is under study, and how.

For example, is the paper evaluating context (i.e., barriers and facilitators to implementing the EBP/guidelines), strategies (e.g., specific approaches to better integrate the EBP/guideline into routine clinical care), or implementation outcomes (see Table 2)? The study design can vary widely: often, qualitative or mixed methods are employed to assess context and outcomes, while a prospective randomized clinical trial may be used to study if a particular strategy is superior or inferior to another strategy. When the evidence underlying an EBP/guideline is still being developed, a hybrid trial, which concurrently collects data on clinical outcomes as well as implementation outcomes, is an increasingly promising approach (36). Like any clinical trial, the methods should describe intervention allocation and planned analysis including any subgroups. Detailed description of the specific setting in which the EBP is being used or evaluated is critical, as is a thorough explanation of the necessary components of any implementation strategy. If applicable, a reader should also be able to identify the theory, framework, or model serving as the scientific basis for the approach taken by the authors, as well as justification for which one was chosen (Table 1).

The outcome metrics, whether they are a hybrid of clinical/implementation or implementation only, should be defined, and reported in the results section.

The results sections of IS studies often include the fidelity to core components of the implementation strategy and EBP, as well as any adaptions to the EBP or contextual changes that may have occurred during the study.

The publication should pay attention to generalizability of its findings, remembering that applicability to multiple settings – attuned to contextual issues – is a critical defining feature of IS. Depending on the focus of the work, take-away points may include key barriers/facilitators that should be addressed for improved EBP use, if/how the EBP can be adapted to work better in specific contexts, or optimal strategies for EBP implementation.

Conclusions about IS within pediatric critical care and a call to action

To date, IS research within the PICU includes a range of studies: to increase uptake of a specific guideline (e.g., blood product transfusion); to characterize the general process of clinical practice change in the PICU via an IS framework (37); to explore determinants and contextual barriers to implementation (30, 31, 38); and, to study implementation strategies (39–41). These studies, in addition to answering specific questions about the implementation of unique interventions, highlight the usefulness of pragmatic tools and strategies to aid in implementation of best practices. These efforts are important, but much more work is needed to ensure all critically ill children consistently receive beneficial interventions.

In conclusion, all PICU practitioners can contribute to shortening the research to practice gaps in our field. Researchers trying to understand and discover “what” is best for our patients can and should simultaneously consider “how” to implement these approaches. IS methodologies allows us to think systematically about the feasibility, acceptability, and adoptability of proposed interventions. Hybrid studies concurrently examining both the “what” the “how” are particularly important given the challenges pediatric intensivists face when using high-quality evidence to guide practice. Systematically asking and answering questions using the IS perspective, and applying its tools and methodologies has the potential to transform our practice of pediatric critical care and significantly improve outcomes for all critically ill children.

Supplementary Material

Supplemental Data File (.doc, .tif, pdf, etc.)

Supplement 1. Authorship appendix, ECLIPSE subgroup of the PALISI Network