What Can a Learning Healthcare System teach us about Improving Outcomes?

Abstract

Purpose of review:

This review describes the learning healthcare system paradigm, recent examples, and future directions. Patients, clinicians, and health systems frequently encounter decisions between available treatments, technologies, and healthcare delivery methods with little or no evidence about the comparative effectiveness and safety of the available options. Learning healthcare systems endeavor to recognize such knowledge gaps, integrate comparative effectiveness research – including clinical trials – into clinical care to address the knowledge gaps, and seamlessly implement the results into practice to improve care and patient outcomes.

Recent findings:

Recent studies comparing the effectiveness of diagnostic tests and treatments, using information technology to identify patients likely to experience an outcome or benefit from an intervention, and evaluating models of healthcare delivery have demonstrated how a learning healthcare system approach can reduce arbitrary variation in care, decrease cost, and improve patient outcomes.

Summary:

Learning healthcare systems have the potential to answer questions of importance to patients, clinicians, and health system leaders, improve efficiency of healthcare delivery, and improve patient outcomes. Achieving this goal will require realignment of the culture around clinical care, institutional and federal investment, expanded stakeholder engagement, tailored ethical and regulatory guidance, and methodologic advances in information technology and biostatistics.

Introduction

Over the last century, biomedical research has provided patients, clinicians, and health systems with an astonishing array of new medications, technologies, and healthcare delivery methods, helping improve survival and quality of life for a wide range of acute and chronic conditions [1,2]. As an unexpected consequence, in clinical care today patients and clinicians frequently must choose between numerous treatments for a given condition, with little or no evidence about the comparative effectiveness and safety of the available options. As a result, the treatment that an individual patient would receive varies with the clinician, health system, formulary, and geographic region. This arbitrary variation in care (i.e., variation driven by factors other than knowledge of which treatment is best for the patient) systematically and unknowingly exposes some patients to ineffective or harmful therapies and represents a “profoundly serious moral problem” for current clinical practice [3]. Just as the biomedical research system is ideally positioned to discover new treatments, the healthcare systems that deliver clinical care are uniquely positioned to generate and apply evidence on the comparative effectiveness of available therapies. The paradigm of a ‘learning healthcare system’ – in which researchers, patients, clinicians, and health system leaders collaborate to identify knowledge gaps, integrate comparative effectiveness research into clinical practice, and systematically implement best evidence into practice – has been proposed as a method for reducing potentially harmful arbitrary variation in care, decreasing cost, and improving patient outcomes.

The Problem

More than a decade ago the National Academy of Medicine (NAM) established the goal of having “90 percent of clinical decisions…supported by accurate, timely, and up-to-date clinical information [reflecting] the best available evidence” [4]. Yet, today the vast majority of clinical decisions faced by patients, clinicians, and health systems still lack high-quality evidence [5–8**]. In critical care, common, simple questions like “what is the best vasopressor agent for septic shock?”, “what is the best treatment for alcohol withdrawal?”, and “what is the best oxygen saturation target during mechanical ventilation?” go unanswered for decades and millions of patients each year receive therapies which may be ineffective or harmful without contributing to new knowledge or improving care for future patients.

Generating evidence to inform common comparative effectiveness decisions has been limited by the cost, inefficiency, and inadequate representativeness of traditional explanatory randomized trials. Explanatory trials, typified by drug development trials, are trials conducted to determine if a therapy demonstrates the hypothesized mechanism of action under idealized conditions. Explanatory trials are ill-suited to informing the choice between available treatments, technology, and healthcare delivery methods for several reasons. First, the use of research personnel and data systems separate from the personnel and data systems of clinical care is inefficient, expensive, and time-consuming. Phase III trials of a new drug typically cost more than $100 million and require 3–5 years to conduct [9,10]. At this pace and cost, generating evidence for the hundreds of decisions clinicians face each day would take centuries and cost trillions of dollars. Second, pharmaceutical companies’ financial investments in trials of novel drugs and devices are offset by the potential to earn billions of dollars in profit [11], whereas no similar incentive exists to compare interventions already common in clinical care. Third, while proving a new drug is better than placebo in a homogenous patient population might require enrolling hundreds or thousands of patients, understanding the optimal choice between two common therapies for the broad range of patients receiving them in practice might require enrolling tens of thousands of patients. Fourth, site selection, narrow eligibility criteria, and difficulty recruiting and retaining patients frequently result in explanatory trial populations that do not represent the diversity of age, sex, race, ethnicity, economic status, health literacy, and comorbidities needed to understand the effectiveness and safety of interventions in real-world clinical care [12–14]. Fifth, many knowledge gaps about the best use of a technology (e.g., electronic health record [EHR] alerts) or method of delivering care (e.g., nursing staffing) are context-dependent, delivered to clusters of patients or clinicians, and not suited to evaluation in traditional, patient-level, placebo-controlled explanatory trials.

A Learning Healthcare System as a Potential Solution

The concept of empowering healthcare systems to integrate clinical research and clinical care to improve outcomes for patients was first articulated by the National Academy of Medicine (NAM) [15]. NAM defined a learning healthcare system as one in which “science, informatics, incentives, and culture are aligned for continuous improvement and innovation, with best practices seamlessly embedded in the care process, patients and families as active participants in all elements, and new knowledge is captured as an integral by-product of the care experience”. A learning healthcare system would ideally [1] recognize knowledge gaps, [2] integrate comparative effectiveness research – including randomized clinical trials – into clinical care to address the knowledge gaps, and [3] seamlessly implement the results into practice to improve care and patient outcomes. The patients, clinicians, and leaders within a healthcare system are often better positioned than external researchers, funders, or industry representatives to identify areas of uncertainty and controversy that represent a comparative effectiveness knowledge gap. In a learning healthcare system, arbitrary variation in clinical care can be recognized as a marker of clinical equipoise, highlighting an opportunity for comparative effectiveness research (Figure 1). Transforming arbitrary variation in clinical care into structured variation in the form of effectiveness trials embedded within clinical care (sometimes referred to as pragmatic trials) is a potentially inexpensive, efficient, and inclusive method for health systems to close knowledge gaps and improve care and patient outcomes.

Figure 1. Arbitrary Variation in Clinical Care vs. Structured Variation in a Learning Healthcare System Trial.

The left panel shows a schematic of arbitrary variation in clinical care. When two or more therapies are available for a common condition without evidence to inform which is best for a given patient, clinicians are forced to make an arbitrary choice between the two therapies. One clinician may choose therapy A while another would choose therapy B. The patient experiences all the benefits and risks of whichever therapy is received, but no knowledge is gained, and care is not improved for future patients. When one of the two therapies is superior, continuing to allow arbitrary variation in care systematically exposes patients to ineffective or harmful treatments. The right panel shows a schematic of how a learning healthcare system can address such knowledge gaps. Ideally, arbitrary variation in clinical care is recognized as a marker of clinical equipoise, prompting studies that structure the variation between therapy A and therapy B through a comparative effectiveness trial. Because both therapies are routinely used in clinical care and neither therapy is known to be better for the patient, randomizing patients to therapy A and therapy B may not significantly change the overall number of patients receiving each therapy or the risks and benefits experienced by patients, compared to clinical care. However, by structuring variation through a clinical trial, the learning healthcare system generates evidence regarding which therapy is superior for which patients and applies that knowledge to improve care for future patients.

COVID-19: Learning Healthcare Systems vs Traditional Clinical Care and Clinical Trials

The coronavirus disease 2019 (COVID-19) pandemic has dramatically highlighted the contrasts between the approach of integrating pragmatic trials into clinical care in a learning healthcare system and the traditional approach of conducting explanatory trials segregated from clinical care. As COVID-19 emerged in early 2020, the prevalence and lethality of the virus made an urgent priority of identifying which of the available potential treatments were safe and effective [16].

In the United States (US), the comparative effectiveness knowledge gaps in the treatment of COVID-19 resulted two parallel, and sometimes conflicting [17**], responses: [1] a rush to conduct explanatory randomized trials and [2] widespread arbitrary variation in the treatments provided to patients in clinical care. For example, the antimalarial medication hydroxychloroquine was proposed as a potential treatment for COVID-19 early in the pandemic [16]. In March of 2020, with unprecedented efficiency, the “Outcomes Related to COVID-19 Treated With Hydroxychloroquine Among Inpatients With Symptomatic Disease (ORCHID)” trial was designed, federally funded, and approved by the U.S. Food and Drug Administration and a central Institutional Review Board in just 15 days [18*]. Dedicated research staff for the blinded, placebo-controlled trial enrolled 479 patients from 34 academic hospitals in the Prevention and Early Treatment of Acute Lung Injury (PETAL) Network, before the trial was stopped for futility. Between March when the ORCHID trial was designed and November when the results from the 479 patients were published, hundreds of thousands of patients across the US were treated with hydroxychloroquine as a part of clinical care [19*], experiencing the risks and benefits of the drug without improving knowledge or care for future patients.

In the United Kingdom (UK) during the same period, the National Health Service designed the Randomised Evaluation of COVID-19 Therapy (RECOVERY) trial, which also aimed to compare the effectiveness of available therapies for COVID-19. RECOVERY evaluated hydroxychloroquine using a radically different, learning healthcare system approach. At the direction of the National Health Service, health system leaders encouraged patients and clinicians to participate in the RECOVERY trial to facilitate rapid evaluation of untested therapies and discouraged the use of these therapies outside of trials. RECOVERY trial procedures were embedded within clinical care and conducted by treating clinicians after a brief training session. Data were collected from the medical record. By embedding the pragmatic trial in clinical care, RECOVERY enrolled approximately 11,000 patients between March 25 and June 5, 2020 [20*] – approximately 23-times as many patients as the ORCHID trial enrolled during the same period [21**], despite case counts that were approximately one-tenth those of the US [22]. While the ORCHID trial was designed to answer only one question, the RECOVERY trial was designed to concurrently evaluate the full range of comparative effectiveness questions in COVID-19 treatment, including use of corticosteroids, azithromycin, and antivirals [21**,23*,24*]. Consistent with the paradigm of a learning healthcare system, the National Health Service hospitals continuously identified new knowledge gaps to evaluate, like the use of convalescent plasma and tocilizumab, and implemented findings from each completed trial arm into treatment guidelines for future patients to continually and iteratively improve outcomes [25].

Ideal Research Questions for a Learning Healthcare System

No single method is best for all clinical research. Even if the role for learning healthcare systems expands to address many common comparative effectiveness questions, traditional explanatory trials likely will remain the primary method for conducting drug development and translational research. The following sections describe examples of three types of knowledge gap that learning healthcare systems may be well-suited to address (additional recent examples in Table 1).

Table 1.

Recent Trials in a Learning Healthcare System Paradigm

Study Type	Trial	Description and Comment
Patient-level comparative effectiveness research	Preventing Hypoxemia with Manual Ventilation during Endotracheal Intubation (PreVent) Trial [40]	Randomized trial of bag-mask ventilation to prevent hypoxemia during emergency airway management in the ICU. Enrollment, randomization, delivery of the intervention, and data collection were embedded into clinical care.
	Preventing Cardiovascular collaPse With Administration of Fluid Resuscitation Before Endotracheal Intubation (PrePARE) [41*]	Randomized trial of a pre-intubation fluid bolus to prevent hypotension and cardiac arrest during emergency airway management in the ED and ICU Enrollment, randomization, delivery of the intervention, and data collection were embedded into clinical care.
	The RECOVERY Trial [20,21,23,24]	Adaptive platform randomized trial evaluating therapies against COVID-19 in National Health Service hospitals in the UK. Screening, consent, enrollment, delivery of the intervention, and data collection were embedded into routine care.
	Pragmatic, Randomized Clinical Trial of Gestational Diabetes Screening [42**]	Randomized trial conducted by Kaiser Permanente Northwest and Kaiser Permanente Hawaii comparing one-step vs two-step screening for gestational diabetes. Screening, enrollment, randomization, intervention delivery, and outcome assessment all embedded within the EHR.
	Urgent vs. Early Endoscopy in High-Risk Patients With Upper Gastrointestinal Bleeding Trial [43*]	Randomized trial of timing of endoscopy for acute upper gastrointestinal bleeding. Screening, enrollment, randomization, and intervention delivery all embedded within routine care.
Unit-level comparative-effectiveness research	Isotonic Solutions and Major Adverse Renal Events Trial (SMART) [26]	Cluster-randomized, multiple-crossover trial comparing balanced crystalloids with saline for intravenous fluid administration among critically ill adults admitted to five intensive care units. Screening, enrollment, randomization, intervention delivery, and outcome assessment all embedded within the EHR.
	Saline against Lactated Ringer’s or Plasma-Lyte in the Emergency Department (SALT-ED) Trial [44]	Single-center cluster-crossover trial comparing balanced crystalloids with saline for intravenous fluid administration among noncritically ill adults in the ED Screening, enrollment, randomization, intervention delivery, and outcome assessment all embedded within the EHR.
	Proton Pump Inhibitors vs Histamine-2 Receptor Blockers for Ulcer Prophylaxis Treatment in the Intensive Care Unit (PEPTIC) trial [45**]	Cluster-randomized, crossover trial comparing proton pump inhibitors to histamine-2 receptor blockers for peptic ulcer prophylaxis among critically ill adults admitted to 50 intensive care units. Screening, enrollment, randomization, intervention delivery, and outcome assessment all embedded within routine care and outcomes were collected from a pre-existing registry.
	Neonatal Resuscitation With Supraglottic Airway (NeoSupra) Trial [46]	Cluster-randomized, multiple-crossover trial comparing laryngeal mask airway (LMA) to face-mask ventilation during neonatal resuscitation in low-income countries. Screening, enrollment, randomization, and intervention delivery all embedded in routine care.
	Chlorhexidine Bathing Trial [47]	Cluster-randomized, multiple-crossover trial trial of once-daily bathing of all patients with 2% chlorhexidine compared to nonantimicrobial cloths among critically ill adults admitted to 5 intensive care units. Screening, enrollment, randomization, intervention delivery, and outcome assessment all embedded in routine care.
Hospital-level comparative-effectiveness research	High-Sensitivity Troponin in the Evaluation of Patients With Acute Coronary Syndrome (High-STEACS) Trial [48]	Step-wedge trial to evaluate implementation of high-sensitivity cardiac troponin assays to identify patients at high-risk of cardiovascular events. Screening, enrollment, randomization, and intervention delivery were all embedded within the EHR
	Active Bathing to Eliminate Infection (ABATE Infection) Trial [49]	Cluster-randomized trial of 53 hospitals comparing routine bathing to decolonization with universal chlorhexidine and nasal mupirocin. Screening, enrollment, randomization, intervention delivery, outcome assessment all embedded within clinical care.
	Head Positioning in Acute Stroke (HeadPost) Trial [50]	Cluster-randomized, crossover trial of head positioning after acute stroke, conducted at 114 hospitals in 9 countries. Screening, enrollment, randomization, and intervention delivery, all embedded within clinical care.
Trials using information technology to identify patients likely to experience an outcome or benefit from an intervention	Advanced Alert Monitor at Kaiser Permanente Northern California [35**]	Used a model embedded within the EHR to identify patients at risk of unplanned ICU transfer within 12 hours (afferent limb), triggering a clinical evaluation (efferent limb). Results were compared to control patients from other hospitals for whom the model was functional but not linked to a clinical team alert (no efferent limb). Screening, enrollment, randomization, intervention delivery, and outcome assessment all embedded within the EHR.
	Optimizing Electronic Alerts for Acute Kidney Injury [36**]	Used a statistical model embedded within the EHR to identify patients at risk of acute kidney injury (afferent limb), triggering an order set for best management practices (efferent limb) Screening, enrollment, randomization, intervention delivery, and outcome assessment all embedded within the EHR.
	Evaluating processes of care and outcomes of children in hospital (EPOCH) [51]	Cluster-randomized trial at 21 hospitals comparing a bedside pediatric early warning score to usual care with regard to in-hospital mortality. Screening, enrollment, intervention delivery, and outcomes assessment were all embedded within clinical care.
Trial evaluating models of healthcare delivery	Hotspotting trial [38**].	Trial randomly assigned patients with medically and socially complex conditions to receive or not receive assistance from a care-transition program composed of nurses, social workers, and community health workers
	Vanderbilt ICU Recovery Program Pilot (VIP) Trial [52]	Trial randomly assigned critically ill patients to enrollment in an ICU recovery program compared or usual care. Screening, enrollment, randomization, and intervention delivery all embedded within routine care.
	Improving Patient and Family Centered Care in Advanced Critical Illness [53]	Stepped-wedge, cluster-randomized trial that randomized critically ill patients in five intensive care units to a multicomponent family-support intervention compared to usual care. Screening, enrollment, randomization, and intervention delivery all embedded within routine care.
	Strategies to Reduce Injuries and Develop Confidence in Elders (STRIDE) Trial [54*]	Cluster-randomized trial at ten clinical sites to evaluate the effectiveness of a multifactorial intervention to prevent fall injuries. Screening, randomization, and intervention delivery embedded within routine care.
	Promoting Successful Weight Loss in Primary Care in Louisiana (PROPEL) Trial [55*]	Cluster-randomized trial at 18 primary clinics to evaluate the effectiveness of a high-intensity, lifestyle-based program for obesity treatment. Screening, randomization, intervention delivery, and outcome assessment, all embedded within routine care.
	Randomized Order Safety Trial Evaluating Resident-Physician Schedules (ROSTERS) Trial [56]	Cluster-randomized trial of resident work schedules conducted in six pediatric intensive care units. Screening, enrollment, randomization, and intervention delivery all embedded within routine care
	Trial of Method of Contraception Prescription Delivery (Bridge-It) [57*]	Cluster-randomized crossover trial in 29 pharmacies evaluating the effectiveness of providing women with a bridging supply of the progestogen-only pill compared to the usual care (referring patients to usual contraceptive provider). Screening, enrollment, randomization, and intervention delivery all embedded within routine care
	Cut Your Pressure Too: The Los Angeles Barbershop Blood Pressure Study [58]	Cluster-randomized trial of blood pressure control conducted in 52 black-owned barbershops evaluating the effect of an embedding a pharmacist-led intervention within a nontraditional health care setting.

Comparing the Effectiveness of Diagnostic Tests or Treatments

Several recent high-profile trials have demonstrated the power of learning healthcare systems to efficiently compare the effectiveness of diagnostic tests and treatments by embedding pragmatic trials into clinical care at the level of the patient, provider, unit, or hospital (Table 1).

Administration of intravenous fluids is the most common treatment received by hospitalized patients. For more than a century, two basic classes of isotonic fluid have existed: 0.9% sodium chloride (saline) and crystalloid solutions with electrolyte compositions closer to that of plasma (balanced crystalloids). Until recently, no large trials had compared balanced crystalloids to saline with regard to patient outcomes. Between June 1, 2015 and April 30, 2017, the Learning Healthcare System at Vanderbilt University Medical Center conducted the Isotonic Solutions and Major Adverse Renal Events Trial (SMART), a pragmatic, un-blinded, cluster-randomized, multiple-crossover trial comparing balanced crystalloids with saline among critically ill adults [26]. Trial screening, enrollment, group assignment, intervention delivery, and outcome assessment were all performed by software applications within the EHR. As shown in Figure 2, both fluids were commonly used in the health system before the trial and conduct of the trial did not significantly change the total amount of each crystalloid used in the health system. After the trial but before the results were known, treating clinicians returned to using slightly more saline than balanced crystalloids. Once the trial’s analysis demonstrated that balanced crystalloids led to improved clinical outcomes compared to saline, the learning healthcare system implemented the trial results via the same software applications used during the trial, resulting in a rapid change to balanced crystalloids for nearly all future patients.

Figure 2. Phases of Research in a Learning Healthcare System.

This figure demonstrates the ‘observation, intervention, implementation’ phases of research in a learning healthcare system as exemplified by the SMART trial [26]. The y-axis shows the percent of isotonic fluid administered in the Vanderbilt medical ICU that was saline from 2014 to 2019. Before the SMART trial, 60–75% of the isotonic fluid administered was saline, with some clinicians administered as much as 100% saline and other administering as little as 10% saline. During the trial, the choice of isotonic fluid alternated monthly between saline and balanced crystalloids, achieving high rates of compliance with the assigned fluid without markedly changing the total proportion of fluid that was saline, compared to clinical care before the trial. After the trial was completed but before the results were known, clinicians returned to using slightly more saline than balanced crystalloids. Once the trial’s analysis demonstrated improved clinical outcomes with balanced crystalloids compared to saline, the learning healthcare system implemented the trial results using the same software applications used to deliver the assigned intervention during the trial. This mechanism allowed rapid integration of study findings and within one month of implementation, nearly all the fluid administered in the study unit was balanced crystalloids.

Using Information Technology to Identify Patients likely to Experience an Outcome or Benefit from an Intervention.

Advances in technology, risk modeling, and machine learning have created an opportunity to embed prognostic and predictive models within the EHR with the aim of improving patient outcomes, health care quality, and equity. Learning healthcare systems are well-suited for the iterative process of defining the problem, developing algorithms, implementing tools, and evaluating the performance of the tools and their effect on patient care and outcomes. Recent examples of embedding prognostic or predictive models for sepsis, critical illness, and acute kidney injury into the EHR have highlighted significant opportunities and challenges.

Hospital-acquired sepsis is an attractive use-case for applying predictive tools to improve care delivery. Several EHR-based early warning systems have been developed to alert clinicians in real-time to inpatients meeting criteria for sepsis. Early efforts to deploy warning systems, however, did not improve clinical outcomes or processes of care [27–31]. Consistent with the iterative process of a learning healthcare system, early models are being refined to better define the clinical problem and incorporate temporally granular EHR data, perspectives of bedside clinicians, and tenets of implementation science [32,33].

An example is the hourly Advanced Alert Monitor at Kaiser Permanente Northern California, which identifies patients at risk of unplanned ICU transfer within 12 hours. The model was originally implemented as proof-of-concept [34], was iteratively updated and refined, and then was deployed and evaluated in a prospective, quasi-experimental study of nearly 44,000 hospitalizations [35**]. In this study, the model was deployed at a staggered fashion at 19 hospitals over 3 years. Combining the model to identify patients at risk of deterioration with a triggered clinical review and response from a rapid-response team led to decreases in 30-day mortality, ICU transfer, and hospital length of stay, compared to a control group composed of patients from hospitals where the model had been deployed but not yet linked to a clinical response.

Another recent study, which evaluated an EHR alert for acute kidney injury, underscores the importance of rigorously evaluating the effects of implementing predictive and prognostic models. In a randomized trial at six hospitals, Wilson and colleagues found that an alert notifying clinicians of patients meeting diagnostic criteria for acute kidney injury with a link to an order set for best management practices was effective in changing clinician behavior but unexpectedly led to increased mortality compared to usual care [36**].

Collectively, these examples demonstrate the important potential role that a learning healthcare system can play in evaluating the clinical effects of implementing new predictive models, decision support, or other technology. They highlight the importance of considering both the model (afferent limb) and the clinical actions taken in response to the model (efferent limb). They also highlight the value of rigorously evaluating the effects of implementing the model, either to help refine the intervention or to recognize and de-implement an intervention found to be ineffective or harmful.

Evaluating Models of Healthcare Delivery

Healthcare systems are also ideally positioned to evaluate models of healthcare delivery. Historically, healthcare systems have designed and implemented interventions such as appointment reminder cards, post-discharge phone calls, and programs targeting patients with high healthcare utilization without high-quality evidence of efficacy [37**]. Such interventions are rarely formally evaluated, and even ineffective programs may be newly implemented or continued for years at significant financial cost and burden to patients, clinicians, and the health system. The value of formally evaluating models of healthcare delivery was recently demonstrated by the Health Care Hotspotting trial [38**]. This trial randomly assigned patients with medically and socially complex conditions to receive or not receive assistance from a care-transition program composed of nurses, social workers, and community health workers. This team visited patients, coordinated outpatient care, and linked patients with social services. Health systems globally had implemented similar resource-intense programs based on promising observational studies. In the 800-patient Health Care Hotspotting trial, however, there was no difference in readmission or any other outcome between patients assigned to the post-discharge program and those assigned to usual care.

Challenges to the Development of Learning Healthcare Systems

Developing robust learning healthcare systems will require solving several challenges. To succeed, learning healthcare systems must find new ways to meaningfully engage patient-, clinician-, and health system stakeholders in each phase of research from prioritization to implementation and dissemination [39]. These stakeholders will need to transition from viewing arbitrary variation in treatments as an acceptable characteristic of clinical care to viewing it as an opportunity to improve outcomes. An ethical and regulatory framework will need to be tailored to the incremental risk of comparing common treatments that patients are already receiving, rather than to the development of novel drugs with unknown safety profiles. Learning healthcare systems must extend across multiple hospitals or health systems to produce generalizable knowledge and improve clinical care at scale. Doing so will require addressing challenges related to information technology, data inter-operability, privacy and confidentiality, and health system priorities and governance. Finally, learning healthcare systems must develop the advanced statistical modeling techniques to use data from large pragmatic trials to estimate the effects of treatments for individual patients, rather than the average effect of treatment across the population. Addressing these challenges may fulfill the promise of a learning healthcare system in which patients, clinicians, and health system leaders across multiple hospitals collaborate to identify knowledge gaps, use information technology to embed pragmatic trials into clinical care to address those knowledge gaps, and implement individual treatment effect estimates from those trials to help patients and clinicians make evidence-based, personalized treatment decisions for the myriad of choices that comprise routine medical care.

Conclusion

Learning healthcare systems have the potential to improve patient outcomes, answer questions of importance to patients, clinicians, and health system leaders, and improve efficiency of healthcare delivery. Achieving this goal will require expanded stakeholder engagement, institutional and federal investment, tailored ethical and regulatory guidance, methodologic advances in information technology and biostatistics, and realignment of the culture around clinical care.

Key Points:

• Traditional explanatory trials in the biomedical research system frequently occur separately from clinical care and are poorly suited to comparing available treatments, contributing to the arbitrary variation in clinical care that exposes patients to ineffective or harmful therapies.

• A learning healthcare system can: 1) observe arbitrary provider practice variation and other opportunities to improve care, 2) intervene to determine the best treatment for patients (often by comparing available treatments in comparative effectiveness clinical trials), and 3) rapidly implement knowledge generated through research to improve the care of future patients.

• Clinical care and research during the COVID-19 pandemic exemplified the differences between a learning healthcare system (e.g. the UK model in which use of untested therapies outside of trials was discouraged and efforts were made to include all patients in efficient, pragmatic clinical trials implemented by treating clinicians) and segregated models of care (e.g. the US model in which small, independent research teams separate from treating clinicians enrolled a fraction of patients into trials while clinicians commonly administered untested therapies to hundreds of thousands of patients in clinical care outside of trials).

• Five key challenges facing the development of learning healthcare systems are: 1) institutional and federal investment in infrastructure for learning healthcare systems, 2) expanded engagement of patient, clinician, and health system stakeholders, 3) ethical and regulatory guidance tailored to the incremental risk inherent in learning healthcare system research, 4) methodologic advances in information technology to enable multicenter learning healthcare system and in biostatistics to allow trials to deliver evidence-based individual treatment effects, and 5) realignment of the culture of clinical care to see comparative effectiveness research as inherent to choosing treatments, technologies, and methods of healthcare delivery that produce the best outcomes for patients.