Research as a standard of care in PICU

Abstract

Background

Excellence in clinical care coupled with basic and applied research reflects the maturation of a medical subspecialty, advances that field, and provides objective data for identifying best practices. Pediatric intensive care units (PICU) are uniquely suited for conducting translational and clinical research. Moreover, multiple investigations have reported that a majority of parents are interested in their children’s participation in clinical research, even when the research offers no direct benefit to their child. However, such activity may generate ethical conflict with bedside care providers trying to acutely identify the best approach for an individual critically ill child. Ultimately, this conflict may diminish enthusiasm for the generation of scientific evidence that supports application of evidence-based medicine into PICU clinical standard work.

Objective

Provide an overview of current state PICU clinical research strengths, liabilities, opportunities, and barriers, and contrast this with an established pediatric hematology-oncology iterative research model that constitutes a learning healthcare system.

Design

Narrative review of medical literature published in English.

Conclusions

Currently most PICU therapy is not evidence-based. Developing a learning healthcare system in the PICU integrates clinical research into usual practice and fosters a culture of evidence-based learning and continual care improvement. As PICU mortality has significantly decreased, identification and validation of patient-centered, clinically relevant research outcome measures other than mortality is essential for future clinical trial design. Because most pediatric critical illness may be classified as rare diseases, participation in research networks will facilitate iterative, collaborative, multi-institutional investigations that over time identify best practices to improve PICU outcomes. Despite real ethical challenges, critically ill children and their families should have the opportunity to participate in translational/clinical research whenever feasible.

Current and Ideal States

Although the need to investigate clinical practice to improve outcomes of critically ill patients is widely recognized (1–3), current evidence supporting both preventive and therapeutic interventions for pediatric critical care (PCC) remains sparse (4). This state of affairs seems paradoxical, as nowhere else in the hospital exists a more ideal environment for conduct of clinical research than the pediatric intensive care unit (PICU). Here, a dedicated, well-trained, multidisciplinary care team is immediately available. In addition to the electronic medical record (EMR), extensive electronic physiological monitoring represents usual practice. A variety of biosamples are readily collected in the PICU because various invasive devices are routinely employed. Laboratory and imaging studies, obtained for clinical decision-making, are also available for research.

Despite this plethora of investigational resources, research in the PICU has not flourished as might be expected. In fact PICU research may be less likely, because high clinical intensity may leave less time for academic pursuit. Health care providers engaged in clinical research have described their struggle to appropriately balance potentially conflicting responsibilities associated with being a clinician versus a researcher (5–7). Moreover, the critical care provider’s personality may be more inclined to action rather than deliberate investigation. With an ever-imminent threat of clinical deterioration leading to death or disability, there is need for urgent decision-making in the PICU. Thus, critical care providers may be reluctant to contemplate unfamiliar interventions related to a research protocol, with preference for more familiar clinical decision making (8) as summarized in Figure 1.

Figure 1.

Elements of clinical decision making (adapted from reference 8). Most decision making in pediatric critical care is currently not related to evidence based medicine derived from clinical research, but rather physiology and knowledge acquired during training and personal experience.

Research activities in the PICU may be compared with those in pediatric hematology/oncology or hematopoietic stem cell transplantation units, where virtually every patient is enrolled in one or more research protocols. This latter model encompasses the definition of a, “learning healthcare system”, where knowledge generation is so embedded into usual medical practice that it becomes a natural outgrowth of the health care delivery process, fosters a culture of evidence-based learning, and facilitates continual care improvement (9). This review will examine strengths, liabilities, opportunities and barriers related to PICU research particularly in relation to implementing a PICU learning healthcare system, where research becomes fully integrated with clinical care. In the ideal state every PICU patient benefits directly or indirectly from research; every patient/family and all PICU staff understand research as value-added work; and each patient has the opportunity to participate in quality research. Ultimately such research informs clinical care and clinical observations drive future research.

Probably the most convincing example of a learning healthcare system success is iterative randomized controlled trials (RCT) translating into gradual improved outcomes for children with acute lymphocytic leukemia (ALL) (10). This approach reflects the healthy tension that should exist between standardization and creativity as schematically depicted in Figure 2. Standardization represents the foundation for iterative improvement, and without standardization, measurements in improvement are not possible (11). In reality, standardization provides control for nuisance variables that adversely affect study signal-to-noise ratio (12). Protocolized care itself may improve outcomes and reduce costs (13, 14). Cancer research protocols have typically involved current, standardized, best practice in both study arms (15), with alteration of one or more treatment factors in the interventional arm. This model embraces significant standardization of care for both groups, with testing of a potentially beneficial novel component within the treatment arm.

Figure 2.

Balance of standardization and creativity in iterative research. Without standardization, measurements in improvement are not possible. Without creativity and discovery, standard work and associated outcomes cannot improve.

In 1950, ALL was generally fatal within three months. There was a distrust of clinical trial protocols, characterized as “cookbook medicine”, and pessimism and provincialism prevailed. An iterative research process was revolutionary at this time, but simply reflected serial, small step, trial-and-error experiments not unlike contemporary continuous quality improvement science, albeit without the rapid cycle change (16). Utilizing this methodology, long-term ALL survival increased to 90% by 2006 (10). Development of clinical trial infrastructure supported the clinical research, and gradually raised the standard of care for all cancer patients. Five-year event-free survival for children with ALL now approaches 99% (17).

In fairness, remarkable decreases in mortality have also been realized for major diagnoses treated in the PICU, although the reasons for these improved outcomes are not so readily discernible. Introduction of aggressive volume replacement, early antimicrobial therapy, resuscitation guidelines implementation (18) and regionalization of care have been responsible in part for the decline in pediatric sepsis mortality from near 100% in the 1960s to approximately 10% currently (19). This progress is schematically depicted in Supplemental Digital Content, Figure SDC 1. Over the past several years, serial research based initially on whole genome mRNA expression has substantially expanded knowledge of pediatric sepsis (20–25). Similarly there has been a decrease in mortality for pediatric acute respiratory distress syndrome (26), as schematically summarized in Supplemental Digital Content, Figure SDC 2, and pediatric trauma (27). These important improvements in outcomes following pediatric critical illness have likely resulted from general advances in delivery of PCC, but relative to hematology-oncology are inadequately understood beyond biological plausibility. As noted by Rivera, et al, “The fact that nearly every child in America with cancer is enrolled in a study reflects both the rigor and the research training of academic members in the subspecialty as well as the marked improvement in survival for many forms of childhood cancer”(28).

Barriers and Opportunities

Declining clinical revenues and paucity of funding for both career development of young faculty and support for mid-career faculty have threatened long-term viability of many pediatric academic departments (29). Although the NIH budget doubled between 1998–2003, and pediatric research funding increased 12.8% annually, the proportion of the NIH total budget directed towards pediatric research actually decreased over this same interval (30). As fewer pediatric faculty members identify as clinician-scientists, and funding for junior faculty development diminishes, such departments may exhibit profound declines in academic productivity (31). With inflation adjusted dollars, the NIH budget has remained “flat,” and even decreased, over the past 10 years. The success rate for R01 equivalent funding has decreased from ~30% in 2000 to ~17% in 2013 [http://report.nih.gov/nihdatabook/Charts/Default.aspx?chartid=202].

Despite these changes, the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) has provided a research home to pediatric critical care researchers. Recognizing the need for high-quality research in PCC and the need to develop the next generation of researchers, the NICHD created and currently supports the Collaborative Pediatric Critical Care Research Network (CPCCRN) (http://cpccrn.org/), the Pediatric Critical Care and Trauma Scientist Development Program (PCCTSDP) (http://www.pccsdp.org/), and most recently created the new Pediatric Trauma and Critical Illness Branch (PTCIB) within the NICHD (29). Creation of programs such as CPCCRN and PCCTSDP demonstrates that the development of research capacity as a prominent feature of PCC is possible even in when resources may be constrained.

Research is uniquely challenging in the PICU, as the environment, patient condition, and trial design are typically and simultaneously complex (32). Accepting the logistic, financial, and ethical challenges to conducting RCTs that enroll children (33–35), it is also important to acknowledge the supportive nature of PCC. Generally, PCC is neither curative nor preventive, but is characterized by support for threatened or actual organ dysfunction. With time, healing progresses and most children can be separated from advanced life support and survive. Nevertheless, common tools of the critical care provider include interventions that are only feasible in the PICU and enhance the likelihood or speed of organ recovery. However, such PCC support strategies remain largely individualized according to patient response, available resources, and personal practice style and experience (8). Developing the scientific basis for practice techniques, strategies, execution, and timing remains limited (18, 36).

As summarized in Table 1, substantial differences distinguish experimental care from standard practice utilizing innovative therapy outside of research (35). Here innovative therapy is defined as one that drifts into standard practice with unproven effectiveness and unknown adverse effect profile, but is still undertaken in the presumed best interest of the patient (37). A number of innovative therapies subsequently evolved into standard practice without adequate testing of safety and efficacy, and later were found to cause harm (38). For example, significant animal data suggested that therapeutic hypothermia was beneficial in traumatic brain injury, and this therapy was widely adapted in the PICU. When pediatric RCTs were completed, this adjunctive therapy was actually shown to provide no benefit and potentially be harmful (39). Similarly, like our adult counterparts, pediatric critical care providers were convinced of the efficacy of activated protein C for pediatric septic shock, until the RCT was actually conducted, that demonstrated no benefit in hastening organ dysfunction resolution or reducing mortality, but twice the risk of intracranial hemorrhage among the youngest subjects (40). Accordingly, innovative use of a drug, device, or biologic may be more risky to patients than the same use in the context of an appropriately designed and conducted clinical trial (41). It has been argued that the ethics and regulatory requirements for clinical practice, quality improvement and clinical research should be identical (see Supplemental Digital Content, Table SDC 1) (42, 43).

Table 1.

Itemized standards for research typically absent in innovative but standard practice care [adapted from 35]

Competition for and review by a funding agency Requirement for adherence to federal regulations Systematic literature review involved Pre-clinical data typically available Investigator credentials and experience reviewed Informed parental permission/assent required Institutional review board scrutiny Data safety monitoring board for adverse events Academic center environment Abstract and manuscript composition with peer review Open discussions of results Protocolized care with potential for improved care for both placebo and intervention groups

Discussion of the barriers to research as a standard of practice in PCC would be incomplete without alluding to the potential harm for children. Although research in the PICU is, “… conducted at the uncertain boundary between life and death”(44), as noted above, usual care is often based on pathophysiologic rationale and individual clinical experience in the absence of scientific evidence (8). However, utilizing an open-label approach for management essentially means that every patient treated is an undocumented experiment with n=1, with no data accrued regarding either efficacy or safety. For example, in the absence of evidence-based medicine, but with strong personal opinions, there persists a general lack of consensus regarding indication, type, dosing, duration, and weaning of adjunctive corticosteroids for septic shock in children (45–47). Extrapolating information from studies conducted in adults, and generalizing to children may also be dangerous (48, 49).

Overall the practice of PCC is not well supported with scientific evidence (18, 36). Clinical research involving critically ill children should be a healthcare priority because PICU care is expensive and associated with high morbidity and mortality (50). If the relative safety and efficacy of complex PICU therapies are unknown, PCC care providers have an ethical imperative to generate such knowledge; without doing so, “practice as usual” remains empiric and perhaps more hazardous than it needs to be. Balancing these ethical imperatives in PCC clinical research is illustrated in Supplemental Digital Content, Figure SDC 3.

As indicated in Table 2, several ethical requirements should be present for clinical trials conducted in PICUs to move forward (51). Utilizing these guidelines, pediatric intensivists face two major problems in terms of including their patients in clinical trials: namely, identification of practical, patient-centered, clinically meaningful primary outcome measures and maintenance of relative equipoise regarding the research question.

Table 2.

Ethical requirements for clinical trials (from reference 50)

Important social value Scientific validity (rigorous reliable results, adequate numbers) Fair subject selection Favorable risk/benefit ratio (minimizes risk, maximizes potential benefit) Independent review Informed consent Respect for subjects

Identification of patient-centered, clinically meaningful outcome measures that ideally do not require large study populations is essential to the future success of PICU clinical research (52). An outcome measure must be accurately determined, easy to record, responsive to change, demonstrate a causal relationship to the disease process under study (biological plausibility), and must be clinically relevant to patients, family, and providers. Surrogate outcomes must have valid relationships with clinically meaningful measures and probably should only be used in phase 2 screening trials (53). Relatively few RCTs conducted in the ICU setting, utilizing mortality as a primary outcome measure, have shown a beneficial impact of the experimental intervention. Accordingly, there has been an evolving consensus to identify primary end points other than crude differences in all-cause mortality (54). In this regard, various measures of long-term health-related quality of life appear promising (55).

It has been argued that the critical aspect of equipoise relates to lack of agreement within a relevant clinical or scientific community (56). However, when an unstable child is spiraling towards death or disability, individual physician uncertainty is more likely to be colored by bias based on personal anecdote and experience (57, 58). In a survey of 415 pediatric intensivists, 88% believed RCTs are the most scientifically appropriate study design for assessing new therapies for critically ill children. However, 90% reported that they had experienced ethical conflict within this experimental design, and 84% indicated that published data from uncontrolled trials could bias them towards use of an (unproven) investigational therapy (59). Accordingly, individual physician equipoise may be ephemeral in PICUs, and RCTs involving life-sustaining therapies for children may be biased and protocol adherence flawed. For success of an interventional trial both the medical community and individual physicians must possess equipoise for the research question (60).

Approach

Conduct of clinical trials differs fundamentally between adults and children (61): 1) physiology and biochemistry of children differs from that of adults; 2) disease processes in childhood differ from those of adults; and 3) pediatric diseases and treatments need to account for a child’s growth and development.

Single-center trials continue to predominate PCC research as they are logistically easier to conduct, less expensive, do not typically require prolonged negotiation for study design or funding, utilize simplified data collection tools, enroll a less heterogeneous population, permit better planning for definitive trials, and are useful for hypothesis generation (62). Problems with single-center trials frequently include limited external validity, implausible hypothesized effect size, actual or unintended bias, possible lack of blinding, loss of equipoise for definitive trials and unwarranted evolution into “standard of care” (62, 63). However, launching large, multicenter trials without performing pilot, single-center trials would be foolhardy and wasteful.

Prior to conduct of a formal RCT, pre-RCT investigations are almost always beneficial for refining various aspects of the research design. This approach typically involves descriptive, epidemiologic, and observational studies (64). Systematic reviews and meta-analyses provide key RCT epidemiologic data, including incidence and risk factors (65, 66). Scenario based questionnaires can identify practice variability and establish if community and individual equipoise for the research question exists (47). Supporting observational/descriptive trials preceding the RCT may include studies examining population demographics, description of the health care burden, temporal changes, questions regarding efficacy vs. effectiveness and evaluation of potential outcome measures. Such studies will facilitate consensus development and identify research collaborations (67). For single-institution studies, clinical databases derived from the electronic medical record will be a good source for pilot data (68). Larger epidemiologic studies employing larger databases can expand preclinical trial information beyond a single institution (69, 70). Pilot studies prior to the RCT can evaluate enrollment and protocol feasibility and logistics. The ‘programmatic research’ approach of the Canadian Clinical Trials Group, summarized in SDC Table 2, provides an useful context for considering a variety of research methodologies that facilitate the success of an RCT (71).

Traditionally, the prospective, double-blind, randomized placebo-controlled trial has represented the gold standard for clinical trial design (62, 72), and some have advocated that, “… whenever practical, the RCT should remain the foundation for evidence-based practice” (73). Given the history of critical care evidence-based medicine, it has been argued that at least two beneficial RCTs are necessary with at least one being a confirmatory trial (74). Suggestions have been offered for successful conduct of RCTs in the critical care setting (32), where implementation is frequently difficult (75). As discussed above, if some clinicians view assignment of their patients to a placebo arm as a potential threat, individual physician equipoise will need to be reexamined and established to assure success of a trial.

Utilizing a ‘scoping’ methodology, Duffett, et al recently performed a comprehensive state-of- the-art review of PCC RCTs (72). The authors analyzed publications emanating from 248 RCTs, from 31 countries, conducted over the interval 1986–2013. Most RCTs were single center (82%), with a majority conducted in North America and Western Europe. Most trials enrolled small sample numbers, examined medications (63%) and employed intermediate or surrogate outcome measures, although primary outcome measures were identifiable in only 67%. Low risk of bias was determined for only 11% of RCTs, and 57% of studies did not report a planned sample size. The authors concluded a need for more rigorous RCT methodology, use of appropriate outcome measures and improvement in the quality of reporting. In addition the authors established a valuable clinical trials resource, namely an online data base of PCC RCTs that is updated quarterly [http://epicc.mcmaster.ca/].

According to the Institute of Medicine, the purpose of comparative effectiveness research (CER) is to assist consumers, clinicians, purchasers and policymakers in making informed decisions that will improve health care at both the individual and population levels (76). A key aspect of clinical investigation for many years, CER includes many research designs including RCTs, and has recently been in vogue because of CER-specific federal funding (77). In the United States the Patient-Centered Outcomes Research Institute (PCORI) [http://www.pcori.org/] has been instrumental in promoting CER. Globally, the International Initiative for Traumatic Brain Injury Research, an international multidisciplinary collaborative, will focus on CER methodology to rapidly improve clinical practice for traumatic brain injury for both adults and children. This initiative is supported by the European Commission, NIH and the Canadian Institutes of Health Research (78).

Although RCTs emphasizing strict protocol adherence are essential for demonstrating the efficacy of a particular approach, they may not address effectiveness in more generalized practice settings (79). On the other hand, observational studies are recognized as being inherently limited by indication bias and effects of unmeasured confounding variables (80). An evidence-based medicine approach to patient-oriented research needs to recognize both the strengths and limitations of RCTs as well as observational studies (81).

The key role of individual clinical research performance sites, where the actual work of clinical research happens, has been relatively underappreciated. Competing agendas including conflict of commitment, financial pressure, regulatory burdens, risk aversion, and multiple research priorities, stress the local research mission and infrastructure (82). Strategies to address these clinical research impediments include: 1) instituting a process of clinical research improvement methodology; 2) responding to the actual needs of site-based research; 3) identifying clinical research as a key mission of the institution; 4) establishing a clear process for reviewing and assigning research priorities; and 4) improving local public understanding of the role of clinical research.

For research in the PICU the role of families as surrogate decision makers takes on even greater meaning. At a time when high anxiety compromises understanding, families appreciate the process of shared decision making (83). Trust in the medical and research teams is fundamental to why families agree to participate in research (83–85). When family values, preferences and perspectives are understood by researchers (86, 87), most families will cite altruism in allowing their children to participate in research (85, 88, 89), even in the absence of potential direct benefit (90).

Because most diseases encountered in PICUs can be viewed as rare diseases (91), many clinical trials are challenged by the need to recruit large numbers of subjects to ensure adequate power. Organized research networks facilitate enrollment of subjects and enhance collaboration and organization among investigators. Characteristics of successful multi-center clinical research include a cohesive spirit, sense of mission, and the importance of organizational goals rather than individual priorities. For grass roots research networks, good ideas may be more important initially than funding (1). Research networks that have included PCC research are summarized in Supplemental Digital Content Table SDC 3.

The “Pediatric Rule”, first legislated in 1997 through the Food and Drug Administration Modernization Act (FDAMA), was adapted to become the Best Pharmaceuticals for Children Act (BPCA) [http://www.fda.gov/AboutFDA/WhatWeDo/History/Milestones/ucm128305.htm]. The BPCA provides an incentive to pharmaceutical companies to conduct pediatric studies (at the request of the FDA) by providing an additional six months of patent exclusivity (92). Whereas BPCA offers a “carrot” to the drug and device industry by extending patent protection, the Pediatric Research Equity Act essentially legislates penalties for not performing pharmaceutical research studies in children, where indicated. BPCA was established to enhance the likelihood that children would be given proven, innovative therapy without untoward outcomes. Recently, BPCA has expanded its mission to consider not only priority drugs in need of evidence-based medicine for children, but also common childhood conditions with significant pharmaceutical knowledge gaps (93). Details of similar international legislative directives to encourage pediatric pharmacology research have been discussed in detail (94). Conduct of the, Clopidogrel To Lower Arterial Thrombolytic Risk In Neonates and Infants Trial (CLARINET), that enrolled over 900 infants from 134 countries, represents one tribute to the power of international government-pediatric medicine collaboration to hasten identification of best practices in the PICU (95).

Increasingly, the pharmaceutical and device industry fund the large, expensive, definitive assessment, phase 3 efficacy trials. However, because of commercial motives, actual or potential bias represents a significant integrity risk for such studies (96). Site investigators can maximize the benefit of industry collaboration in clinical research by insisting on the key principles summarized in Table 3 (97).

Table 3.

Requirements for industry-sponsored research (adapted from reference 97)

Performance site investigators are involved in the study design. Results of the study are published regardless of the outcomes. An independent data safety monitoring board with an independent statistician is utilized. Post-study access to the database and biorepository is assured. Site investigators are involved in composing manuscripts, abstracts and other presentations. All investigators report their potential conflicts of interest. Clinically meaningful study endpoints are employed as primary specific aims.

Over the interval 1985–2005, 1347 RCTs enrolled critically ill adults as compared to 157 RCTs that enrolled critically ill children (98). In a commentary to these findings, four suggestions were offered (28): 1) improve the rigor of research training during fellowship training; 2) expand T32 training programs in child health research; 3) embrace RCTs as a key tool for improving quality of care, cost effectiveness, and comparative effectiveness research; and 4) involve clinician-educators working alongside physician-scientists to conduct RCTs (99). The last suggestion may be especially important in terms of expanding research in the PICU. As one method for enhancing a PICU iterative research model and encouraging development of a learning healthcare system in the PICU, critical care providers might consider implementing a novel quality measure:

$$\frac{\mathrm{\sum }\text{PICU}\text{subject}\text{clinical}\text{trials}\text{days}}{\mathrm{\sum }\text{PICU}\text{patient}\text{days}}\times 100$$

RCT performance monitoring tools (e.g. schedule performance index, cost performance index, protocol compliance rates, safety risk scores) should also be implemented at the outset to maximize the return-on-investment from PCC research (100–103). Ideally PICU-based research would focus on interventions that are both “quality-improving” and “cost-lowering” (104). Such activity should also include efforts to expand the type, scope, and enrollment in PCC research (105).

Conclusions

Research as a pathway to the truth can be time consuming, difficult, and expensive. Challenges to conducting high-quality research in critically ill children are significant, but such barriers can be overcome (106). With these challenges also exist tremendous opportunities given the PICU environment (107, 108). Hematologists/oncologists developed a culture of a learning healthcare system for their patients because they realized that this was essential for improving outcomes. This article has emphasized the concept of the iterative research model, the obligation of inviting patient/family participation in PICU research, the value of research networks in facilitating multi-institutional PCC studies, and the importance of all critical care providers in supporting the goals of evidence-based medicine by maintaining equipoise on important research questions. If every PICU admission is considered as a potential opportunity for clinical trial enrollment, iterative improvements in PICU care delivery and patient-centered outcomes will follow.

Supplementary Material

1. Supplemental Digital Content, Figure SDC1.

Longitudinal reduction in pediatric sepsis mortality over time, as reported in pediatric sepsis investigations, 1968–2001.

1968, Gram-negative sepsis at the University of Minnesota; 1985, all-organism septic shock at Children’s Hospital National Medical Center; 1999, national estimate for severe sepsis in the United States; 2000, Neisseria meningococcal septic shock at St. Mary’s Hospital (UK); and 2001, Dengue shock in Vietnam. From: Carcillo JA (19) with permission.

2. Supplemental Digital Content, Figure SDC2.

Longitudinal reduction in pediatric acute lung injury mortality over time, as reported in pediatric investigations, 1980–2008.

Pediatric acute lung injury (ALI) mortality reported since 1980. Each data point represents one study, with appropriate weighting based on the number of cases reported in order to generate the trend line (total n = 978; Pearson’s correlation coefficient = −0.750). From Zimmerman, et al (26) with permission.

3SDC3. Supplemental Digital Content, Figure SDC3.

Balancing ethical imperatives in pediatric critical care clinical research. On the one hand, for a variety of reasons, critically ill children represent a particularly vulnerable population. On the other hand evidence-based pediatric critical care medicine is sparse, and accordingly critically ill children should not be deprived of participation in research that may ultimately benefit themselves or others.