What role for learning health systems in quality improvement within healthcare providers?

Abstract

Introduction

Recent decades have seen a focus on quality in healthcare. Quality has been viewed across 6 dimensions—safe, effective, patient‐centred, timely, efficient and equitable. As IT has enabled the transformation of other industries, there has been an increasing interest in the potential for learning health systems (LHS) to improve quality in healthcare. We are not aware of any systematic attempt to investigate the potential impacts of different types of LHS on quality within healthcare providers.

Methods

A review of the limited LHS literature informed the topics for 25 expert interviews, 6 focus groups, and 2 site visits. A deductive thematic analysis was conducted to identify the different types of LHSs and their potential impacts across the 6 dimensions of quality.

Results

Six types of LHS were identified—intelligent automation, clinical decision support, predictive models, positive deviance, surveillance, and comparative effectiveness research. The thematic analysis identified that the 6 types of LHS could potentially have a broad range of positive impacts across the 6 dimensions of quality. However, they also identified the potential for negative impacts on quality and highlighted that many of the potential impacts have not been substantiated through rigorous evaluation.

Conclusions

These findings suggest that LHSs may represent an evolution of existing quality improvement techniques or even fundamentally new capabilities within quality improvement. However, they also highlight the need for further research to evaluate the impacts.

1. INTRODUCTION

Political, financial, and demographic pressures, combined with increasing public expectation and transparency, have resulted in a focus on quality in healthcare over the last 2 decades. In Crossing the Quality Chasm, the US Institute of Medicine1 (IoM) defined quality as

The degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.

The IoM definition went on to identify 6 dimensions of healthcare quality (see Box 1).

BOX 1: Six dimensions of healthcare quality.

• Safe

• Effective

• Patient‐centred

• Timely

• Efficient

• Equitable

At the same time, information technology has enabled the transformation of industry after industry. A realisation that these trends would not pass healthcare by led the IoM2 to identify a new sociotechnical concept, the learning health system (LHS)—one in which

“science, informatics, incentives, and culture are aligned for continuous improvement and innovation, with best practices seamlessly embedded in the delivery process and new knowledge captured as an integral by‐product of the delivery experience.”

While learning has always been possible within organisations, LHSs harness the rapidly developing opportunities, presented by informatics, to learn from every patient who is treated. Learning health systems are usually described in a cycle3, 4 in which data are collected and analysed to address a question and then fed back into the health system to drive some improvement. This fits neatly with most definitions of a quality improvement cycle, such as the plan, do, study, act model.5

Learning health systems operate at different scales, that is, they may operate at the level of a national or international health system, a provider or a clinical microsystem, such as a specialty team.

There have been significant efforts to demonstrate how national and international LHSs can improve quality by closing gaps in the evidence base,6 but despite efforts to identify such practice, we are not currently aware of any systematic attempts to identify how LHSs, at the provider scale, might address the 6 dimensions of quality outlined in Box 1. This may be because LHSs are not always recognised as being tools for quality improvement and that quality improvement practitioners are often not aware of the new methods at their disposal.

Conversely, the limited adoption of LHS thinking at a provider level may indicate a difficulty in implementing this approach in a provider setting. We make a first attempt to identify how LHSs could address each of the dimensions of healthcare quality within provider organisations.

2. METHODS

2.1. Phase 1

A review of the academic, grey, and commercial literature on LHSs was conducted in February 2015. An initial search was conducted using the Medline bibliographic database. Because of the limited indexing of relevant articles, only 2 free text terms, “Learning Health Systems” and “Learning Healthcare Systems,” were required. Articles were chosen, by TF, from citations identified, if they met one of the following criteria:

1. Citation contained an example meeting the broad IoM2 definition of a LHS.

2. Citation contained generalizable insights into the development or implications of LHSs, as defined by the IoM.2

This was followed by a citation search of the references within identified papers to identify further eligible papers. Google was used to search for commercial and grey literature on the topic. Further papers were identified by the expert participants. In total, 92 references were identified, as shown in Figure 1.

Figure 1.

PRISMA flow diagram outlining search strategy

An inductive thematic analysis7 of the relevant papers was conducted, by a single reviewer, TF, to identify key themes in the field of LHSs. Abstracts and references to all papers that informed the analysis were uploaded to the project website.8 The findings of the inductive analysis were used to inform the interview/focus group topic guide.

2.2. Phase 2

Experts in the field of LHSs, based in the United Kingdom and United States, were identified, from the literature review. Further experts were identified by the first group of experts and so on, in a snowball approach, until the key experts (those referred to by several participants) emerged. These individuals were approached and invited to participate in either an in‐depth, semistructured interview or in a focus group. Individuals were engaged until no new concepts or views were expressed in interviews (ie, we judged that data saturation had been achieved).

Interviews were typically 1‐hour long and usually face to face. A number of interviews had to be conducted by telephone, as a result of illness, adverse weather conditions, and other unforeseen circumstances. Each participant was asked a selection of questions from the interview/focus group topic guide, based on their area of expertise. Topics covered the trends, ethics, regulation, workforce, and service implications of LHSs but did not explicitly cover the 6 dimensions of quality. A synopsis of each interview was written up, agreed with the participant, and published on the project website.8

Focus groups were structured around areas of ambiguity within the field of LHSs, and participants were chosen on the basis of their expertise. Each focus group was written up, and the synopsis, along with a full video recording (where consent was received), was published on the project website.8

Supplementing the interviews and focus groups, site visits were conducted to organisations that were repeatedly cited, by participants or in the literature, as having implemented important aspects of LHSs. The purpose of the site visits was to address key questions identified from the literature, interviews, and focus groups.

2.3. Phase 3

A final deductive thematic analysis of the literature, interviews, focus groups, and site visits was conducted by TF. This was used to determine the types of LHSs and how each could address the 6 dimensions of healthcare quality, identified by the IoM.

We sought advice from the Newcastle University Faculty of Medical Sciences Ethics Committee and were advised that ethical approval was not required for this study. The study protocol, consent forms, and participant information sheets were submitted.

3. RESULTS

3.1. Data available for analysis

3.1.1. Phase 1

Of the 92 references identified, there were 34 peer‐reviewed papers, 20 nonpeer reviewed monographs (eg, think tank reports), 10 government reports, 9 commercial publications, 9 websites, 6 books, 2 press articles, and 2 presentations (references available on the project website8).

As Figure 2 illustrates, the number of publications per year has increased significantly since 2008. Publications before 2007 did not mention LHSs explicitly but were mostly methodological references. Figure 2 excludes 5 references from before 1995. References from before 1995 include Florence Nightingale's 1863, Notes on Hospitals,9 and Eric Codman's 1913, Standardization of Hospitals,10 which were early examples of health outcomes and benchmarking research.

Figure 2.

Publications by year

The literature identified was predominantly from US‐ and UK‐based sources; however, the search did not include non‐English language sources, so some relevant work may not have been identified.

3.1.2. Phase 2

Twenty five people were interviewed and 15 people were engaged in 6 focus groups. Site visits were conducted at Geisinger Health System in Pennsylvania and at the IBM Watson research facility in New York. Where consent was received, these were published on the project website.8

3.2. Summary of findings

Thematic analysis of the literature, interviews, focus groups, and site visits suggested the existence of 6 broad and overlapping types of LHS (Box 2). The same data were reanalysed to identify were participants and the literature suggested that each type of LHS could impact on each of the 6 dimensions of quality (Box 2).

BOX 2: Types of LHSs identified.

• Intelligent automation

• Positive deviance

• Predictive models

• Clinical decision support systems

• Surveillance systems

• Comparative effectiveness research

3.3. Intelligent automation

Intelligent automation makes use of electronic health record (EHR) and other data to automate routine processes, previously performed by clinicians, such as prepopulating order sets and clinic notes and summarising case notes prior to consultations.

Participants reported the potential for such systems to improve the “safety” and “effectiveness” of care, by preventing clinicians from missing important actions, such as relevant investigations. Advanced systems for summarising case notes, such as that being developed by the IBM Watson team,11 could make care more “person‐centred,” by ensuring that clinicians are aware of all relevant information, even for patients with complex histories and voluminous notes.

Intelligent automation has been used by providers, such as Geisinger,12 to make care more “timely” and “efficient,” by removing administrative tasks from clinicians, allowing them to focus on tasks that require their specific skills and experience.

3.4. Positive deviance

Routinely collected outcomes can be used to benchmark providers or teams, identifying those who deliver safer, more effective, person‐centred, timely, or efficient care. These cases can be studied in depth, and their good practice can be disseminated, through a methodology known as positive deviance13 or safety II (learning from care that works well).14 Such an approach has been increasingly advocated as the best way of improving patient safety.

This approach is only possible in situations where there are reliable performance measures and where there is natural variation in performance. It also requires openness about practice and an engaged constituency who are prepared to take up new practices.13 By focusing on a positive change in practice, the change in behaviour may be more likely to be initiated and maintained.

Critical to this is the presence of valid outcome or quality measures. These have been traditionally lacking, but organisations such as the International Consortium for Health Outcomes Measurement15 are now systematically creating sets of outcome measures for all major conditions that can be implemented as part of routine care. The International Consortium for Health Outcomes Measurement claims to have developed standard sets15 for over 35% of the developed world disease burden and aims for 50% by 2017.

3.5. Predictive models

Predictive models are algorithms that can identify instances where there is a high risk that “unsafe,” “delayed,” or “inefficient” care will occur and can estimate how effective (and potentially cost‐effective) interventions are likely to be in particular instances.16 For example, Geisinger have developed predictive models that use metrics, from their EHR, to predict system level events such as spikes in hospital activity and patient level events such as patients not attending when scheduled. Action can then be taken to mitigate the impact, while further models can predict instances or patients who will be most amenable to particular interventions. Thus, interventions can be focused on patients where they are likely to be effective. This approach is analogous to the move towards stratified and personalised medicine in the drug discovery arena.

Participants did recognise a risk that this could threaten the “equity” of care. For example, models that prioritise patients who are likely to respond well to intervention are likely to exclude some of the most vulnerable groups in society, such as those with poor language skills, drug and alcohol problems, or cognitive impairment. Providers need to be aware of this possibility.

Participants noted that such models could be enhanced by inclusion of genetic and social care data and even data from social media and wearable technologies. This is becoming increasingly common in the area of stratified medicine, where such data enable the identification of relevant patient subgroups where the effectiveness of a therapy may be markedly different than the whole patient population. Such information enables more targeted therapies to be developed and delivered.

It was also pointed out that predictive models are a form of screening and are therefore subject to false positives and false negatives that can impact the quality of care. It was proposed that they should be appraised against a modified form of the Wilson and Jungner WHO screening criteria17 before implementation.18

3.6. Clinical decision support systems

Clinical decision support systems (CDSSs) have been defined as “an electronic system designed to aid directly in clinical decision making, in which characteristics of individual patients are used to generate patient‐specific assessments or recommendations that are then presented to clinicians for consideration.”19 Participants felt that this could improve safety and effectiveness in dealing with unfamiliar or high‐risk situations.

There were conflicting accounts, among participants and in the literature, regarding the success of existing systems, with a recent systematic review concluding that, across clinical settings, new generation CDSS had only a small impact on safety, effectiveness, and efficiency.20 This general finding may hide benefits that occur within particular settings or circumstances. Indeed, participants suggested that “decision supportive” systems would be more acceptable than “decision directive” systems, with one noting that “doctors don't go to university to be told, by a computer, what to do.” “Would you use your satnav to tell you how to drive to work every day?”

Some participants felt that such systems would become more important as the amount of genetic, social, and monitoring data exceeds the ability of clinicians to weigh it systematically.

Decision‐making relies on a combination of information and preferences. Information may be objective, but the preferences are subjective and involve values.21 The CDSS may enable more person‐centred care, but only if the patient is aware of and able to influence the preferences that drive the system.

3.7. LHSs beyond the provider

The final 2 types of LHS, identified in this study, have the potential to significantly impact on the 6 dimensions of quality, but they tend to operate at the regional, national, or international level, rather than solely within provider organisations. These systems have received much of the attention directed towards LHSs and are well explored elsewhere22:

1. Real‐time surveillance systems can track epidemiological phenomena and adverse events related to treatments in a much more timely fashion than passive reporting systems. This can have significant implications for the safety of care.

2. Comparative effectiveness research in an LHS can take the form of observational, quasi‐experimental, and innovative experimental study designs. These methods can use routinely collected data to fill gaps in the evidence base, ensuring more effective care, in a more timely and efficient way than would be possible with traditionally designed RCTs. They can achieve this by reducing the recruitment and data collection burden both on patients, health services, and research teams. These designs also avoid some of the safety risks associated with participating in tightly controlled phase 3 RCTs, such as experimental exposure and withholding standard treatment. They can be more equitable because their findings can be applicable to patient groups that are often excluded from RCTs, such as those with multiple co‐morbidities. How well these approaches work is a matter for methodological debate, but there was consensus that they would become more important as more routinely collected data and outcome measurements become accessible.

4. DISCUSSION, LIMITATIONS, AND FURTHER RESEARCH

Participants and the literature, identified potential impacts across the 6 dimensions of quality, as illustrated by Figure 3. Not all impacts will be positive, they will not occur every time that a specific type of LHC system is implemented, and many have not yet been substantiated by rigorous evaluations. There will also be inevitable trade‐offs between the dimensions. However, this study provides a strong rationale for LHS designers to consider the potential impacts of their systems within a framework such as the IoM 6 dimensions and to evaluate them post implementation.

Figure 3.

Types of LHSs and the dimensions of quality that they might impact, as highlighted by participants and the literature

It is unlikely that a universal evaluation method for a LHS could be developed. However, there will most likely be a role for well‐designed RCTs or quasi‐experimental studies in evaluating LHSs. These rigorous evaluative designs are likely to be more appropriate once the particular LHS has reached a level of maturity and stability.23 This is because that the iterative developments may make the implementation of such designs overly complex. Before stability is reached, rapid prototyping and iterative evaluation methods are likely to be more appropriate in identifying beneficial impacts and problems in a more timely and cost‐effective fashion. Ultimately, it is important not to fall into the same trap as has been observed for other health technologies, “that it is always too early for evaluation until it is too late.” The evaluation methods should be linked to the design of the LHS and whether it is safe to fail or must be fail safe.24 There is a need for further research to explore these issues and to build an evidence base for different types of LHS.

It could be argued that several types of LHS simply represent the evolution of existing QI techniques, with LHSs essentially comprising continuous QI systems.4 It could also be argued that the advent of LHSs, with such broad impacts on quality, actually represents fundamentally new capabilities within QI, such as the ability to process large and clinically rich datasets in near real‐time. This would suggest that QI practitioners will require new skillsets or will have to work more closely with other professionals, such as informaticians.25 It would also suggest that organisations that promote QI will need to broaden the scope of their activity.

Risks to quality from EHR implementations are well documented,26 but these results have also highlighted potential risks to quality from LHSs. There were particular risks within the “equitable” domain, where systems, such as predictive models could inadvertently disadvantage certain patient groups. This suggests a need for greater focus on equity, within the LHS community.

None of the LHSs identified are simple IT products. They are invariably sociotechnical systems that require the reengineering of care delivery, particularly at the provider level. This, in part, may account for their broad impact on quality but also suggests the need for careful design and organisational planning, to avoid the unintended consequences for quality and professional satisfaction that have been associated with EHR implementation.27, 28 Further work is required to understand how providers can effectively reengineer care delivery around their new capabilities.

Although some providers, such as Geisinger, have embraced the LHS philosophy, none have yet implemented LHSs across all aspects of care. The reasons are likely to be multifactorial and may include organisational culture, lack of interoperability, lack of funding, skills shortage, and competing priorities,29 as well as lack of reusable technology platforms, information governance concerns, and lack of economic and other evaluation. These will need to be better understood and overcome before LHSs become part of the mainstream QI strategies of most providers.

5. CONCLUSION

Recent decades have seen a focus on quality in healthcare. Quality has been viewed across 6 dimensions—safe, effective, patient‐centred, timely, efficient, and equitable. There have been significant efforts to demonstrate how LHSs can improve quality by closing gaps in the evidence base, but we are not currently aware of any systematic attempts to identify how LHSs, at the provider scale, might address the 6 dimensions of quality.

Six types of LHS were identified from a thematic analysis of literature, interviews, focus groups, and site visits. Although further research is required to test the validity of this typology, with participants and with the broader LHS community, it provides a useful framework against which to understand the potential impacts of different LHSs on quality within a provider.

Participants and the literature reported that LHSs could have potential positive and, in some cases negative, impacts on quality. This provides direction for future research to evaluate these impacts and to identify the facilitators and barriers to LHS implementation.

Foley TJ, Vale L. What role for learning health systems in quality improvement within healthcare providers? Learn Health Sys. 2017;1:e10025 10.1002/lrh2.10025