Skip to main content
PMC home page
Health Research Policy and Systems logoLink to Health Research Policy and Systems
. 2021 Jul 19;19:102. doi: 10.1186/s12961-021-00746-6

Aiming for quality: a global compass for national learning systems

Diana Sarakbi 1,3,✉, Nana Mensah-Abrampah 2, Melissa Kleine-Bingham 2, Shams B Syed 2
PMCID: PMC8287697  PMID: 34281534

Abstract

Introduction

Transforming a health system into a learning one is increasingly recognized as necessary to support the implementation of a national strategic direction on quality with a focus on frontline experience. The approach to a learning system that bridges the gap between practice and policy requires active exploration.

Methods

This scoping review adapted the methodological framework for scoping studies from Arksey and O’Malley. The central research question focused on common themes for learning to improve the quality of health services at all levels of the national health system, from government policy to point-of-care delivery.

Results

A total of 3507 records were screened, resulting in 101 articles on strategic learning across the health system: health professional level (19%), health organizational level (15%), subnational/national level (26%), multiple levels (35%), and global level (6%). Thirty-five of these articles focused on learning systems at multiple levels of the health system. A national learning system requires attention at the organizational, subnational, and national levels guided by the needs of patients, families, and the community. The compass of the national learning system is centred on four cross-cutting themes across the health system: alignment of priorities, systemwide collaboration, transparency and accountability, and knowledge sharing of real-world evidence generated at the point of care.

Conclusion

This paper proposes an approach for building a national learning system to improve the quality of health services. Future research is needed to validate the application of these guiding principles and make improvements based on the findings.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-021-00746-6.

Keywords: Learning health system, Health policy, Implementation science, Quality improvement

Introduction

Universal health coverage (UHC) is a global health priority that is part of the 2030 Agenda for Sustainable Development endorsed by the United Nations [1]. The goal of UHC is not only to provide access to a national, publicly funded health system but also to deliver quality health services that are effective, safe, people-centred, timely, equitable, integrated, and efficient [2]. There is no single pathway to achieving quality UHC. Each country will need to learn how to transform its health system into a learning one to deliver quality health services to patients, families, and the broader community [3–5]. A learning system is defined by the Institute of Medicine, the most common reference cited in the literature, as a system that learns from itself, where “science, informatics, incentives, and culture are aligned for continuous improvement and innovation, with best practices seamlessly embedded in the delivery process and new knowledge captured as an integral by-product of the delivery experience” [6, 7]. Real-world evidence generated at the point of care could help national health authorities understand the systemic barriers for delivering quality health services and the strategies and policies needed to address them [8, 9].

The WHO Global Learning Laboratory (GLL) for Quality UHC welcomes people from across the world to share their experiences and expertise, challenge each other, and spark innovation for improving the delivery of quality health services. One of the priority areas for the GLL is to share how countries are learning to develop, implement, and refine their national policies and strategies for quality based on the frontline experiences of health professionals and the patients, families, and communities they serve. This strategic approach to learning and decision-making based on context is a continuous cycle of collaboration, feedback, and improvement that requires the engagement of multiple stakeholders across the health system to bridge the gap between practice and policy [10–12].

According to WHO, a country’s health system is formed by the people and organizations who are receiving, delivering, and overseeing health services, consisting of three main levels of stakeholders: (1) patients/families, health professionals, and health organizations, (2) subnational health authorities guiding the delivery of quality health services, and (3) national health authorities setting policies/strategies for quality [13, 14]. A national learning system for improving the delivery of health services involves stakeholders at each level of the health system to form a common vision for quality (organizational, subnational, and national). Having a health system perspective of quality means recognizing that health professionals are not working in silos but are part of a health organization within a subnational and/or national health system that is accountable to patients, families, and the broader community. A national learning system targeting stakeholders across the health system could potentially be effective at implementing sustainable policy changes by uncovering systemic barriers to quality at the point of care [15, 16].

A scoping review was conducted to identify guiding principles for building a national learning system to improve the quality of health services. The first objective of the scoping review was to provide a high-level overview of strategic learning across the health system (organizational, subnational, and national) to better understand how the concept of “learning systems” fits within this overall landscape. The second objective was to complete an in-depth review of the literature on “learning systems” by expanding on the definition provided by the Institute of Medicine and identifying common themes for building a continuous cycle of collaboration, feedback, and improvement across all levels of the health system.

Methods

Scoping reviews are a common approach for clarifying definitions, mapping the key themes of a topic based on research evidence, and highlighting knowledge gaps in the literature [17]. This type of literature review was selected given the overall aim of this paper to explore guiding principles for strategic learning to improve the quality of health services across the health system. A protocol was developed based on the methodological framework from Arksey and O’Malley [18] and the Joanna Briggs Institute (JBI) Reviewer’s Manual for scoping reviews [19]. This was an iterative process that included an exploratory search for articles on learning systems using free-text terms in the JBI Database of Systematic Reviews and Implementation Reports, Cochrane Database of Systematic Reviews, and MEDLINE to develop, pilot, and refine the search strategy based on the aim of the scoping review [20]. The scoping review consisted of five stages, as outlined in Fig. 1: (1) developing the research question and search objectives based on the priority areas for the WHO GLL for Quality UHC and initial search results, (2) identifying the inclusion criteria, screening and selecting relevant studies, (3) extracting and analysing data from selected articles, (4) summarizing and reporting the findings, and (5) completing an online consultation with experts in healthcare improvement to obtain feedback on the recommendations.

Fig. 1.

Fig. 1

Open in a new tab

Scoping review framework

Research questions

The following research questions were explored in the literature.

Primary

What are the guiding principles for building a national learning system to inform policies and strategies for quality grounded in frontline realities?

Secondary

What are the levels for strategic learning across the health system focused on quality improvement (organizational, subnational, and national)? What are the common themes for forming a learning system to deliver quality health services?

Search strategy

As recommended by JBI, the PCC (Population, Concept and Context) framework was used to define the search terms based on the research question and objectives of the scoping review. The PCC framework was also used to define the inclusion criteria for selecting relevant articles (Table 1). Four databases pertinent to the scoping review topic were searched on 17 June 2019 for publications between 2009 and 2019 (Global Health, MEDLINE, HealthSTAR, and CINAHL) using the search terms in the PCC framework as follows: (“Health” OR “Healthcare”) AND “Quality” AND (“Improvement” OR “Improvements”) AND “Learning”. A more conservative approach was used for the search terms to obtain a broader range of articles after piloting the initial search strategy. For example, the terms “health/healthcare” were used instead of “health system” to identify studies at multiple levels of the health system (e.g., health professionals/organizations). Similarly, the term “learning” was used on its own instead of “learning systems” to yield broader results that cover the topic of strategic learning in general across the health system, in addition to learning systems, and to account for the variability in terminology. Finally, the terms “quality” and “improvement” were selected to capture articles that addressed the continuous cycle of feedback and learning for the purposes of improving the delivery of quality health services.

Table 1.

Search strategy

Objectives Inclusion criteria Search terms

Learning levels

(High-level review):

Identify the levels for strategic learning to deliver quality health services from a health system perspective

Learning systems

(Focused review):

Identify common themes for building a learning system for delivering quality health services

P (Population)

Any health condition/population

P (Population)

Health/Healthcare

C (Concept)

Improving the quality of health services

C (Concept)

Quality

Improvement/Improvements

C (Context)

Collaborative learning at any level of the health system

Any country/language

Published between 2009 and 2019

C (Context)

Learning
Open in a new tab

Study selection

The search and screening results are reported in Fig. 2 using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) guidelines [21]. The search strategy resulted in 3507 records after the duplicates were removed. These records were screened using a two-step process based on the following inclusion criteria: (1) publications describing collaborative learning approaches to improve the delivery of health services from a system perspective (conceptually and/or empirically), (2) in the context of any health condition or healthcare setting, (3) at any level of the health system (organizational, subnational, and national), and (4) published between January 2009 and June 2019. Articles were excluded if the learning methods were unclear, the focus was on individual learning or learning technology (simulation, artificial intelligence, and complex data analysis), or the context of learning was outside the health sector. The articles were screened by titles and abstracts according to the inclusion criteria, where 101 articles were selected for the high-level “learning levels” synthesis. Of the 101 articles on “learning levels”, 48 records covered the topic of “learning systems” and were retained for a full-text review. Of the 48 records, 35 articles provided clear recommendations for building a learning system and were retained for the “learning systems” synthesis. Additional file 1 has the references of included articles by learning level, and Additional file 2 has the citations of excluded articles during the full-text review.

Fig. 2.

Fig. 2

Open in a new tab

PRISMA study flow diagram

Data analysis

The characteristics of each study were extracted and charted in a data extraction table that identified the title and authors, publication year, country, learning level to improve the quality of health services, and key findings. The authors met regularly to validate the data extraction and discuss the data analysis based on the priority areas for the WHO GLL for Quality UHC. A descriptive analysis was completed to define strategic learning at each level of the health system: “health professional level”, “organizational level”, “subnational/national level”, “multiple levels”, and “global level”. The characteristics of the learning levels were summarized in a table that included an amalgamated definition from a health system perspective, the number of articles, the list of countries, the types of settings, and the main topics (Additional file 3).

A thematic analysis was completed for articles on “learning systems” to identify common themes for building a continuous system of collaboration, feedback, and improvement across the health system. The full-text assessment was done independently by two reviewers, where the characteristics of each eligible study were presented in a summary table that included the title, the country, the main focus, the level of the health system (organizational, subnational, and national), and the definition of “learning system” and its key features (Additional file 4).

Consultation

A summary of the scoping review results and recommendations was prepared for an online consultation with experts in healthcare improvement. The four experts referenced in the acknowledgements section of this paper were invited to answer open-ended questions on the definition of a learning system, the learning levels, and the guiding principles for forming a national learning system. They noted the importance of this work and that the nationwide framework provided a structure for quality improvement efforts across the health system. They also provided recommendations that complemented the findings from the scoping review. This included emphasizing the complexity of the health sector, the role of the public in setting priorities for the national learning system, and the importance of learning from unsuccessful quality improvement initiatives.

Results

The first objective of the scoping review was to understand at a high level the levels for strategic learning to deliver quality services across the health system. Given the complexity of the health sector, it is important to have a global view of learning to understand the stakeholders involved in delivering quality health services and their role in a national learning system. A total of 101 articles met the inclusion criteria and were retained for the “learning levels” synthesis. All the articles were published in English. These articles were grouped into five categories: “health professional level” (19%), “health organizational level” (15%), “subnational/national level” (26%), “multiple levels” (35%), and “global level” (6%).1 The abstracts of these articles were reviewed to identify at a high level the characteristics of each learning level from a health system view. The full text of the article was reviewed when this information was not available in the abstract (or was unclear). The results are summarized in Additional file 3, including an amalgamated definition developed for learning at each level of the health system based on common trends.

The second objective of the scoping review was to complete a focused review of the literature on “learning systems” and identify common themes for building a continuous system of feedback, learning, and improvement across the health system. A total of 35 articles were retained for the “learning systems” synthesis. All the articles were published in English. Thirty-one articles were from high-income (HI) countries (89%) and four were from low- and middle-income countries (LMIC) (11%). The common objectives of the articles were to summarize the key features of a learning system to improve the quality of health services. Of the 35 articles, one was a quantitative study (3%), two were quality improvement reports (6%), and the other 32 were perspective/commentary papers (91%). Seven of the articles covered learning systems at the organizational level (20%), five at the subnational level (14%), 16 at the national level (46%), and one at the global level (3%). The other six articles did not have a defined scope for the learning system (17%). The results are summarized in Additional file 4.

Health professional level

Definition

Health professionals are responsible for assessing, diagnosing, treating, and preventing health conditions based on the local needs of their community. Examples include medical doctors, nursing professionals, midwifery professionals, dentists, and pharmacists. Health professionals play an essential role in a national learning system by identifying and implementing improvements and recognizing systemic barriers to quality based on their frontline experiences [22, 23]. Health professionals need to be supported to develop the competencies required to lead changes at the point of care in a complex health system.

Strategic learning

Pre-service training in quality improvement is a pathway for building the competencies of future health professionals to continuously improve the quality of health services as part of their undergraduate and graduate training. Health professionals would benefit from early training to develop their knowledge, skills, and attitudes towards quality improvement and learning how to work as part of a broader interprofessional team within a larger health system.

Application

There is an increased focus on quality improvement training in medical and nursing education. Training methods address knowledge, skills, attitudes, and interprofessional collaboration in healthcare improvement. Training in practice-based learning and improvement helps trainees develop the competencies needed to continuously identify and implement improvements based on data in the service environment including hospital and primary care [24–30]. While students are trained in quality improvement methods to help them identify service improvement opportunities, a health system view is also needed to recognize systemic barriers as a limitation of improvement efforts [23, 31]. Systems-based practice requires trainees to have a broader understanding of the health system and associated challenges, and participate in system-level improvement initiatives [32, 33]. Combining practice-based learning and improvement with systems-based practice is an emerging concept to help prepare trainees to effectively function within a national learning system [34]. Interprofessional learning is another important theme in healthcare improvement where learning becomes more relationship-based and complex in the service environment [35–37]. Health professionals are expected to work as part of a team but are often trained in silos. More opportunities are needed for interprofessional learning to improve the quality of health services [38]. The role of pre-service training in quality improvement needs more attention in the literature [39, 40].

Health organizational level

Definition

Health organizations are responsible for delivering quality health services to a defined community, including hospitals, primary care clinics, and community health centres. As the learning cycle in a national learning system begins with the interaction between the patient and the health professional, it is important to develop the characteristics needed to support learning and improvement at the organizational level [41].

Strategic learning

Participating in quality improvement initiatives is a pathway for a health organization to become a learning one by promoting accountability and building a culture of continuous feedback and improvement in the service environment based on successes and failures. Healthcare organizations are knowledge-intensive and rely on extensive professional experience, skills, and knowledge to be effective [42]. Having an organization-wide data-driven approach to patient care promotes continuous collaboration, learning, and improvement [43].

Application

A foundational characteristic of a learning organization is psychological safety, where people feel comfortable raising concerns and trying new approaches to improve health services and are accountable as a team for the results [41, 44–46]. Another important characteristic is interprofessional collaboration to improve communication, create positive learning experiences, and facilitate improvement [47–50]. Health professionals gain a deeper understanding of how their collective tasks are interdependent and impact the overall quality of health services [50]. Other common themes for promoting a culture of continuous collaboration, learning, and improvement include leadership commitment, creative problem-solving, and performance measurement [42, 51–54]. The digitization of patient care facilitates real-time sharing of data and generates new knowledge for learning and improvement [55].

Health organizations are characterized by people, knowledge, and information technology, and are key components of an effective national learning system. The focus of health organizations is on identifying and addressing organization-specific areas for improvement. However, there are mutual benefits for adopting a health system perspective and exchanging learnings with other organizations to leverage experiences and spread improvements for common challenges.

Subnational/national level

Definition

Multiple health organizations overseen by health authorities at the subnational or national levels to guide the delivery of health services through supportive policies and strategies on quality. The specific roles will vary depending on the structure of a country’s health system.

Strategic learning

Learning collaboratives are a pathway for multiple health organizations within a subnational or national health system to become a learning one by participating in quality improvement initiatives. These are temporary initiatives where a group of healthcare organizations learn together to improve a common priority area generally at the subnational or national level. The scoping review focused on learning collaboratives that leveraged opportunities for group learning. As healthcare is a complex adaptive system, it is important to study not only whether a quality improvement initiative works within a collaborative, but also how and why [56].

Application

Learning collaboratives are generally short-term initiatives that bring together healthcare teams to spread a specific improvement by applying a change package of interventions. Learning collaboratives covered various clinical settings including primary care, community care, acute care, and specialized services. They were used by HI and LMIC. Models for improvement generally included an aim statement, a project team, team-based learning using training materials (e.g., change package and online platform) and activities (e.g., in-person workshop and coaching), and performance evaluation. The Institute for Healthcare Improvement Breakthrough Series collaboratives were referenced as an evidence-based model for improvement through several studies. Topics ranged from adherence to specific clinical practice guidelines for chronic diseases (e.g., diabetes and obesity) to broader improvements to integrated care (e.g., improving mental health services in primary care). One study conducted a series of mini-collaboratives to build quality improvement capacity across several states and prepare for national accreditation [57].

Successful collaboratives included leadership support, clear objectives and time frames, opportunities for peer-to-peer learning, evidence-based improvements, psychological safety, and development of transferrable competency in quality improvement [56–65]. Learning collaboratives are temporary in nature and seem to be effective for single-target interventions [66–70]. However, practice transformation such as integration of care is complex and requires a multilevel approach to achieve improvement and adaptation to the local context [71–74]. More complex learning collaboratives provided opportunities to improve interorganizational communication, and connections between services where health professionals view themselves as part of a larger system of care [63, 75–77]. Learning collaboratives also provided opportunities to build relationships, create networks of experts to exchange best practices, and improve implementation of quality improvement initiatives [78–80].

Barriers included protected time to participate in the collaborative, few opportunities to co-design the collaborative, lack of information sharing and coordination of care between sectors, and lack of data on the sustainability of the results due to the temporary nature of the collaborative [58, 81, 82]. The evaluation tends to focus more on the outcomes rather than the learning methodology and associated costs.

While learning collaboratives have more of a subnational/national health system perspective towards learning by broadening the reach of improvements, they do not address systemic barriers that are outside the control of participating organizations because they tend not to involve subnational/national health authorities as stakeholders.

Multiple levels

Definition

Quality improvement initiatives involving multiple stakeholders learning together to continuously improve the quality of health services based on the frontline experience.

Strategic learning

A learning system is formed by a group of stakeholders operating at an organizational, subnational, national, or global level to continuously improve the quality of health services based on real-world evidence. The purpose of the learning system is to provide the best care at the lowest cost by having researchers partner with patients, families, practitioners, and other stakeholders to co-create knowledge based on real-world evidence and promote mutual learning for improving the quality of health services and ensuring integrated patient-centred care [83–86]. Additional file 4 references the definitions found in the literature for learning systems, where several studies have used or adapted the definition provided by the Institute of Medicine [87].

Application

“Learning systems” is an emerging concept that was first formally defined by the Institute of Medicine in 2007 [87]. Therefore, there is some, albeit limited, evidence on the positive impact of learning systems on patient outcomes [85, 88, 89]. However, there is guidance on the core features of learning systems and the process used to develop them based on examples from both HI and LMIC.

The first common feature of a learning system is designating a network of stakeholders responsible for the design, operation, and governance of the system, including establishing a model for improvement and fostering a collaborative environment. Stakeholders include patients, families, health professionals, administrators, researchers, and policy-makers [90–95]. Learning is needed at multiple levels of the health system, from practice to policy, to evaluate the quality of health services at the population level [96]. It is recommended to start with a prototype for a learning system that could be replicated by other areas in healthcare and gradually scale to the national or global level based on continuous learning and improvement [86, 94, 97, 98].

The second feature of a learning system is having a common goal and commitment to improving one or more areas in the delivery of health services. This shared purpose fosters a sense of community among participants of the learning system where collaboration is a fundamental requirement for learning [84, 85, 94, 98–101].

The learning cycle starts with the patient–clinician interaction at the point of care [102]. The third feature of a learning system is generating standardized approaches to care and quality measures based on patient data collected at the point of care combined with research and expertise [103, 104]. Quality measures include processes of care, patient experience, and patient outcomes. They are used to test new ideas, in assessing performance against best practices, and for benchmarking and improvement across the health system [104–106]. Quality measures and incentives are needed to encourage continuous learning and improvement and achievement of common quality goals [83, 105, 107–109].

The fourth feature of a learning system is leveraging technology (e.g., electronic health records and patient registries, and virtual platforms) to collect “real-world evidence” at the point of care, provide real-time access to knowledge including clinical decision support tools and treatment options, and facilitate the flow of knowledge between the participants of the learning system [94, 104, 108, 110–114].

The fifth feature of a learning system is building trust, transparency, and accountability among all stakeholders. Public reporting is required to share knowledge on the quality of health services with the public and other stakeholders, including national or subnational comparisons and benchmarks. Transparency helps build trust and accountability and incentivizes learning and improvement [105, 107, 109, 115]. It is important that a learning system is designed in a way that protects the privacy, confidentiality, and security of patient data [92, 93, 95].

Learning relies on trust and collaboration. One of the challenges with a learning system is fostering a sense of community with mutual benefits within a fragmented health system [116]. When health services are organized and funded in silos, this creates misaligned interests and does not reflect the path of the patient who needs an integrated health system. Learning systems for military health services have demonstrated that having an integrated health system promotes aligned interests and facilitates the flow of information within the learning system [109, 117]. It is recommended that the health system be designed in a way that promotes learning from the beginning, specifically in LMIC, to avoid retrofitting the health system at a later stage [91].

Learning systems serve similar needs in HI and LMIC, which are to ensure the delivery of quality health services based on continuous learning and improvement. Poor quality of care is one of the leading causes of mortality in LMIC, and contributed to five million deaths based on data from the 2016 Global Burden of Disease study [118]. However, additional challenges are noted for LMIC, including limited resources to electronically collect data at the point of care and evaluate the impact of quality improvement initiatives on patient outcomes [119]. There is also little standardization of patient health records and hospital forms, which makes the use of routine clinical data a challenge [90]. Another challenge is translating evidence into practice. Quality improvement initiatives in LMIC tend to be externally led, with limited insight into contextual barriers and/or lacking local evidence. It is important that learning systems include representation from interprofessional teams working at the point of care and that initiatives are adapted to the local context [105, 119].

Global level

There is potential to form a network with other learning systems globally around a common, complex healthcare challenge to foster cross-learning between countries as they build or strengthen their learning systems. A learning network is a group of learning systems operating at a national or global scale for mutual learning and improvement on a common priority theme. Learning networks rely on collaboration and a bottom-up approach to learning and improvement [120–122]. The concept of a “learning network” was used to refer to either learning collaboratives or learning systems [122, 123]. However, learning networks could also be defined as a method for bringing learning systems together around a common theme such as primary care [86, 124, 125]. There are few studies in the literature on learning networks, in particular on a global scale.

Discussion

A health system perspective is needed to continuously improve the quality of health services at the point of care while simultaneously addressing systemic barriers. Having a health system perspective of learning and improvement means recognizing that health professionals are part of an interdisciplinary team within a health organization that is part of a subnational/national health system that is accountable to patients, families, and the broader community [83, 98]. While it is unclear whether one quality improvement model is more effective than another, some of the success factors seem to include having a systemwide approach that considers the local context and involves multiple stakeholders learning together to achieve improvement based on a systematic review of quality improvement models [86, 126, 127]. A national learning system involves stakeholders across the health system and uses a collaborative process of translating patient data into knowledge that can be used to make changes within and beyond health organizations to improve the quality of health services through supportive strategies and policies grounded in the frontline experiences of patients, family members, and health professionals [96].

A global compass for national learning systems

Dr. Don Berwick highlights the fact that “more than ever before, [we need] a system devoted to continual learning and improvement of patient care, top to bottom and end to end” [128]. This paper recommends guiding principles for transforming a country’s health system into a national learning system based on the scoping review results and consultation with experts in the field. The premise of this approach is that the national learning system is built from the ground up based on frontline experiences of patients, families, and health professionals and the broader needs of the community to guide a national strategy for quality health services [88, 89, 102].

The compass of the national learning system is centred on four cross-cutting themes across the health system: alignment of priorities, systemwide collaboration, transparency and accountability, and knowledge sharing of real-world evidence generated at the point of care.

The following is a summary of these four themes illustrated in Fig. 3, where a governance model is needed to set the stage for learning at the national level to facilitate the flow of information across the various levels of the health system [91, 95, 105]. The multiple arrows illustrate how the flow of learning and improvement across the entire health system can influence direction at all levels (top-down, bottom-up, and end to end).

Fig. 3.

Fig. 3

Open in a new tab

Global compass for national learning systems

Alignment: Who are the learners?

Investing in shared priorities for patients, families, health professionals, administrators, researchers, and policy-makers facilitates the uptake of best practices at the point of care [84, 90, 101, 117]. Priorities should be in line with the overall national direction for quality health services [98, 105]. This helps standardize the delivery of quality health services at the subnational level (e.g., district, regional, or county) and minimize variations as much as possible. Adaptations may be needed at the organizational level based on the local context [96, 103, 110].

Collaboration: How are we learning?

Providing integrated health services based on the patient pathway facilitates cross-sector learning and improvement. This includes the spread of best practices to other organizations at the subnational level (or other districts), and interprofessional collaboration and patient/community engagement to harness learnings and improvements at the organizational level [86, 99, 101]. Examples of patient/community engagement include participating in advisory committees to inform quality policies/strategies, research design, and implementation of quality improvement initiatives [94, 117].

Accountability: Why are we learning?

Promoting a sense of trust, transparency, and shared accountability for learning and improvement across the health system includes creating financial incentives to focus on patient experience/outcomes, publicly sharing reports on the health system’s performance, and having a safe space for health professionals to share variations from best practices and/or lessons learned [92, 100, 105, 115, 116].

Knowledge: What are we learning?

Training on “systems thinking” established as part of medical education can support health professionals to recognize systemic barriers based on their frontline experience [34, 111]. Data are generated across all levels of the health system, including patient stories, to evaluate performance at the point of care (including costs), strengthen the health system to inform service planning efforts at the subnational level, and inform the national quality strategy [90, 91, 108, 109, 117].

National learning system levels

As the learning cycle begins with the interaction between patients and health professionals, the perspectives of these stakeholders are essential to guide the national direction on quality, the systemwide implementation of quality improvement initiatives, and the delivery of quality health services at the point of care [88, 89, 102]. Therefore, it is important to prepare health professionals and health organizations for their role within a national learning system. Health professionals can develop competencies in quality improvement and help identify systemic barriers to delivering quality health services [32–34]. Health organizations can become learning organizations by participating in quality improvement initiatives, including learning collaboratives, to cultivate a culture of continuous improvement, patient partnership and community engagement, and shared learning [42, 43].

The following is an overview of the guiding principles for learning and improvement at each level of the health system illustrated in Fig. 4. It is recommended to start with a prototype for a learning system in a specific health priority area and gradually scale to the national level [94]. An example is provided for illustrative purposes on how to apply these guiding principles to mental health services in primary care [129].

Fig. 4.

Fig. 4

Open in a new tab

National learning system to improve the quality of health services

Health organizational level: delivery of quality health services

As a national learning system is built from the ground up based on the frontline experiences of patients, families, and health professionals, stakeholder input is obtained from health organizations to support adherence to evidence-based guidelines and data collection requirements, team-based approach to quality health services, shared learning, and measurement [88, 103]. For example, in a national learning system focused on quality mental health services in primary care, the following interventions could be identified at the organizational level to achieve quality integrated care: patient registry, collaboration between primary care and mental health clinicians including ongoing mentorship and training, adherence to evidence-based guidelines with routine screening and active follow-up to support treatment adherence, comprehensive treatment options including evidence-based psychotherapy and/or medications, stepped approach to care based on patient’s response to treatment, and leveraging patient data in electronic health records to learn from patient outcomes [129].

Subnational level: organization of quality health services

In a national learning system framework for improving the quality of health services, the organization of quality health services at the subnational level involves standardizing the delivery of health services, integrating health services based on the patient pathway, reporting performance results to the public, and monitoring performance and using the results to make improvements [97, 107]. In the example of a national learning system for mental health services in primary care, the subnational health system would be responsible for the systemwide implementation of integrated care for mental health services in primary care clinics within its jurisdiction. Examples of interventions at the subnational level to achieve quality integrated care include the following: standards and measures for mental health services in primary care clinics, publicly funded treatment options based on evidence with costs tracked, electronic health record database with patient data documented across the integrated care pathway from screening to follow-up visits, primary care clinics incentivized to provide integrated plans in consultation with mental health clinicians, routine screening policy informed by electronic health record data, and reporting of patient outcomes for mental health services in quality improvement plans at the subnational health system level [129].

National level: strategy for quality health services

In a national learning system for improving the quality of health services, the strategy for quality health services is overseen at the national health policy level, including identifying and setting priorities for public policy, incentivizing a collaborative focus on patient experiences and outcomes, and using frontline data to review and improve the strategy [90, 95, 98, 109]. In a national learning system for mental health services in primary care, the following interventions could be identified at the national level to support quality integrated care: national guidelines for mental health services in primary care, policy on public coverage of evidence-based treatments, national electronic health record database for primary care clinics, action plan to support the integration of mental health professionals in primary care, and screening recommendations [129].

Future directions

There is a gap in the literature on how to operationalize learning systems, specifically at the national level. The proposed guiding principles for building a national learning system is the first step in learning to develop, implement, and refine national policies and strategies for quality based on the frontline experiences of patients, families, and health professionals. More research is required to evaluate the impact of a national learning system on patient outcomes. The national learning system will need to be applied, evaluated, and reviewed based on implementation results. As national health systems continue to face common challenges, there are opportunities to exchange learnings with other countries by forming a global network of learning systems.

Possible applications of the national learning system depend on country priorities. However, a common challenge is for countries to refocus their current model to invest in prevention and quality health services in the community. An example would be building a national learning system to understand the policies and strategies needed to improve the quality of mental health services in primary care.

Strengths and limitations

The first limitation of this scoping review is that the articles were screened by one reviewer. However, two reviewers independently completed the full-text review and extracted the relevant data from the articles. The data summary tables were also shared with the authorship team for validation against the research question and objectives of the scoping review and the inclusion criteria. The second limitation is the adaptation of the methodological framework for scoping studies from Arksey and O’Malley based on the objectives of this scoping review. The learning levels for improving the quality of health services were identified by reviewing the abstracts to understand their respective role in building a national learning system. The full text was reviewed when this information was not available in the abstract (or was unclear).

The high-level review of all the “learning levels” provided a more holistic view of learning to improve the quality of health services and complemented the detailed review of the “learning systems” category by allowing for richer recommendations. Another strength of this scoping review is the consultation with experts in healthcare improvement to complement the results and recommendations based on expertise and experience.

Conclusion

“Every system is perfectly designed to get the results it gets” is a popular quotation in the world of healthcare improvement [130]. Government health authorities need to consider lessons from the point of care when setting their national direction on quality health services. Therefore, it is important to understand the layers for strategic learning across the health system. Building a national system of learning and improvement requires shared priorities, collaboration, public trust and accountability, and knowledge sharing within and between countries. This paper proposes guiding principles for building a national learning system that bridges the gap between practice and policy based on the needs of patients, families, and the broader community. Future research is needed to validate the application of this approach, adapt the guiding principles to a country’s context and priorities, and make improvements based on the findings.

Key messages

  • There is an urgent need to understand how different health systems are developing the learning architecture required to implement national strategic direction on quality health services.

  • This scoping review provides granular-level information that can be considered by those responsible for developing a learning architecture for quality health services.

  • This scoping review has led to an approach to building a national learning system that requires shared priorities, collaboration, public trust and accountability, and knowledge sharing within and between countries.

  • Future research is needed to validate the application of this approach and make improvements based on the findings.

Supplementary Information

12961_2021_746_MOESM1_ESM.docx (48.1KB, docx)

Additional file 1. References of articles included by “Learning Level”.

12961_2021_746_MOESM2_ESM.docx (44.6KB, docx)

Additional file 2. References of articles excluded during full-text review.

12961_2021_746_MOESM3_ESM.docx (196KB, docx)

Additional file 3. Summary of studies included by “Learning Level”.

12961_2021_746_MOESM4_ESM.docx (61.3KB, docx)

Additional file 4. Characteristics of studies included for the “Learning Systems” synthesis.

Acknowledgements

The authors would like to thank the experts from the World Health Organization, Bénédicte Walter and Sebastian Kevany, and Canadian healthcare leaders, Dr Danielle Martin (Vice-President, Women's College Hospital) and Wendy Nicklin (President, International Society for Quality in Healthcare), for their valuable feedback on the scoping review findings. A special thanks to Dr Thomas Rotter (Associate Professor, Health Quality Programs, Queen’s University) for his advice on this paper, and Seham Rabaa, M.Sc., for her support in reviewing the articles.

Abbreviations

UHC

Universal health coverage

GLL

Global Learning Laboratory

JBI

Joanna Briggs Institute

PCC

Population, Concept and Context

PRISMA

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

HI

High-income

LMIC

Low- and middle-income countries

Authors’ contributions

All authors contributed to the design of the scoping review and have reviewed and approved the submitted version of the manuscript. DS completed the data collection and analysis. All authors read and approved the final manuscript.

Funding

Not applicable.

Availability of data and materials

Available in additional documents.

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors have no conflict of interest to declare.

Footnotes

1

Percentages may not add up to exactly 100% due to rounding.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.United Nations. The 2030 agenda for sustainable development. 2015. https://sustainabledevelopment.un.org/?menu=1300.
  • 2.World Health Organization OECD & International Bank for Reconstruction and Development/The World Bank . Delivering quality health services: a global imperative for universal health coverage. Washington: The World Bank Group; 2018. [Google Scholar]
  • 3.Akhnif E, et al. Are our ‘UHC systems’ learning systems? Piloting an assessment tool and process in six African countries. Health Res Policy Syst. 2018;16(1):78. doi: 10.1186/s12961-018-0340-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Braithwaite J. Changing how we think about healthcare improvement. BMJ. 2018;361:k2014. doi: 10.1136/bmj.k2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Akhnif E, et al. Scoping literature review on the Learning Organisation concept as applied to the health system. Health Res Policy Syst. 2017;15(1):16. doi: 10.1186/s12961-017-0176-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Ferguson TB., Jr The Institute of Medicine committee report "best care at lower cost: the path to continuously learning health care". Circ Cardiovasc Qual Outcomes. 2012;5(6):e93–e94. doi: 10.1161/CIRCOUTCOMES.112.968768. [DOI] [PubMed] [Google Scholar]
  • 7.Platt JE, Raj M, Wienroth M. An analysis of the learning health system in its first decade in practice: scoping review. J Med Internet Res. 2020;22(3):e17026. doi: 10.2196/17026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Kruk ME, et al. High-quality health systems in the Sustainable Development Goals era: time for a revolution. Lancet Glob Health. 2018;6(11):e1196–e1252. doi: 10.1016/S2214-109X(18)30386-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Makady A, et al. What is real-world data? A review of definitions based on literature and stakeholder interviews. Value Health. 2017;20(7):858–865. doi: 10.1016/j.jval.2017.03.008. [DOI] [PubMed] [Google Scholar]
  • 10.National Academies of Sciences Engineering and Medicine . Crossing the global quality chasm: improving health care worldwide. Washington, DC: The National Academies Press; 2018. [PubMed] [Google Scholar]
  • 11.O'Sullivan MJ. Strategic learning in healthcare organizations. Hosp Top. 1999;77(3):13–21. doi: 10.1080/00185869909596526. [DOI] [PubMed] [Google Scholar]
  • 12.Akhnif E, Macq J, Meessen B. The place of learning in a universal health coverage health policy process: the case of the RAMED policy in Morocco. Health Re Policy Syst. 2019;17(1):21. doi: 10.1186/s12961-019-0421-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Syed S, et al. Improving the quality of health care across the health system. Bull World Health Organ. 2018;96:799. doi: 10.2471/BLT.18.226266. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.World Health Organization. Effective health system governance for Universal Health Coverage. n.d.; https://www.who.int/health-topics/health-systems-governance#tab=tab_1.
  • 15.Cleary PD, et al. Multilevel interventions: study design and analysis issues. J Natl Cancer Inst Monogr. 2012;2012(44):49–55. doi: 10.1093/jncimonographs/lgs010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Ferlie EB, Shortell SM. Improving the quality of health care in the United Kingdom and the United States: a framework for change. Milbank Q. 2001;79(2):281–315. doi: 10.1111/1468-0009.00206. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Pham MT, et al. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5(4):371–385. doi: 10.1002/jrsm.1123. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32. doi: 10.1080/1364557032000119616. [DOI] [Google Scholar]
  • 19.Aromataris E, et al. Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. Int J Evid Based Healthc. 2015;13(3):132–140. doi: 10.1097/XEB.0000000000000055. [DOI] [PubMed] [Google Scholar]
  • 20.Budrionis A, Bellika JG. The learning healthcare system: where are we now? A systematic review. J Biomed Inform. 2016;64:87–92. doi: 10.1016/j.jbi.2016.09.018. [DOI] [PubMed] [Google Scholar]
  • 21.Liberati A, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700. doi: 10.1136/bmj.b2700. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Smith KL, et al. Residents contributing to inpatient quality: blending learning and improvement. J Hosp Med (Online) 2012;7(2):148–153. doi: 10.1002/jhm.945. [DOI] [PubMed] [Google Scholar]
  • 23.Chen CA, et al. How we used a patient visit tracker tool to advance experiential learning in systems-based practice and quality improvement in a medical student clinic. Med Teach. 2016;38(1):36–40. doi: 10.3109/0142159X.2014.975193. [DOI] [PubMed] [Google Scholar]
  • 24.Anagnostou VK, et al. A longitudinal, collaborative, practice-based learning and improvement model to improve post-discharge heart failure outcomes. Conn Med. 2014;78(1):33–36. [PubMed] [Google Scholar]
  • 25.Yazdani S, Farajpour A, Shakerian S. Practice-based learning and improvement (PBLI) from the perspective of Iranian medical education experts: a thematic content analysis. Iran Red Crescent Med J. 2017;19(7):e55664. doi: 10.5812/ircmj.55664. [DOI] [Google Scholar]
  • 26.Kataoka SH, et al. MAP as a model for practice-based learning and improvement in child psychiatry training. J Clin Child Adolesc Psychol. 2014;43(2):312–322. doi: 10.1080/15374416.2013.848773. [DOI] [PubMed] [Google Scholar]
  • 27.O'Connor ES, et al. Developing a practice-based learning and improvement curriculum for an academic general surgery residency. J Am Coll Surg. 2010;210(4):411–417. doi: 10.1016/j.jamcollsurg.2010.01.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Huntington JT, et al. A standardized curriculum to introduce novice health professional students to practice-based learning and improvement: a multi-institutional pilot study. Qual Manag Health Care. 2009;18(3):174–181. doi: 10.1097/QMH.0b013e3181aea218. [DOI] [PubMed] [Google Scholar]
  • 29.Yang Y-Y, et al. A model of four hierarchical levels to train Chinese residents' teaching skills for "practice-based learning and improvement" competency. Postgrad Med. 2015;127(7):744–751. doi: 10.1080/00325481.2015.1080588. [DOI] [PubMed] [Google Scholar]
  • 30.Ross AM. Learning about practice-based evidence in outcomes management. Commun Nurs Res. 2010;43:165–165. [Google Scholar]
  • 31.Wittich CM, et al. Perspective: transformative learning: a framework using critical reflection to link the improvement competencies in graduate medical education. Acad Med. 2010;85(11):1790–1793. doi: 10.1097/ACM.0b013e3181f54eed. [DOI] [PubMed] [Google Scholar]
  • 32.Pensa M, Frew P, Gelmon SB. Integrating improvement learning into a family medicine residency curriculum. Fam Med. 2013;45(6):409–416. [PubMed] [Google Scholar]
  • 33.Tomolo AM, et al. Pilot study evaluating a practice-based learning and improvement curriculum focusing on the development of system-level quality improvement skills. J Grad Med Educ. 2011;3(1):49–58. doi: 10.4300/JGME-D-10-00104.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Wysham NG, et al. Development and Refinement of a Learning Health Systems Training Program. EGEMS (Washington, DC) 2016;4(1):1236. doi: 10.13063/2327-9214.1236. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Wong AKC, et al. The effect of interprofessional team-based learning among nursing students: a quasi-experimental study. Nurse Educ Today. 2017;53(ned, 8511379):13–18. doi: 10.1016/j.nedt.2017.03.004. [DOI] [PubMed] [Google Scholar]
  • 36.Kenaszchuk C, et al. Positive and null effects of interprofessional education on attitudes toward interprofessional learning and collaboration. Adv Health Sci Educ. 2012;17(5):651–669. doi: 10.1007/s10459-011-9341-0. [DOI] [PubMed] [Google Scholar]
  • 37.Portney L, Johnson AF, Knab M. Preparing future health professionals for interprofessional collaborative practice part 1: the context for learning. Semin Speech Lang. 2017;38(5):335–341. doi: 10.1055/s-0037-1607344. [DOI] [PubMed] [Google Scholar]
  • 38.Fowler TO, et al. Alignment of an interprofessional student learning experience with a hospital quality improvement initiative. J Interprof Care. 2018;9205811:1–10. doi: 10.1080/13561820.2018.1455649. [DOI] [PubMed] [Google Scholar]
  • 39.Wehbe-Janek H, et al. Preparing academic medical centers for the clinical learning environment review: alliance of independent academic medical centers National Initiative IV Outcomes and Evaluation. Ochsner J. 2016;16(2):166–171. [PMC free article] [PubMed] [Google Scholar]
  • 40.Butler JM, et al. "It feels like a lot of extra work": resident attitudes about quality improvement and implications for an effective learning health care system. Acad Med. 2017;92(7):984–990. doi: 10.1097/ACM.0000000000001474. [DOI] [PubMed] [Google Scholar]
  • 41.Nembhard IM, Tucker AL. Applying organizational learning research to accountable care organizations. Med Care Res Rev. 2016;73(6):673–684. doi: 10.1177/1077558716640415. [DOI] [PubMed] [Google Scholar]
  • 42.Soklaridis S. Improving hospital care: are learning organizations the answer? J Health Organ Manag. 2014;28(6):830–838. doi: 10.1108/JHOM-10-2013-0229. [DOI] [PubMed] [Google Scholar]
  • 43.Liu VX, et al. Data that drive: closing the loop in the learning hospital system. J Hosp Med. 2016;11:S11–S17. doi: 10.1002/jhm.2651. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Lyman B, et al. Organizational learning in a cardiac intensive care unit. Dimens Crit Care Nurs. 2017;36(2):78–86. doi: 10.1097/DCC.0000000000000233. [DOI] [PubMed] [Google Scholar]
  • 45.Ortega A, et al. Enhancing team learning in nursing teams through beliefs about interpersonal context. J Adv Nurs. 2013;69(1):102–111. doi: 10.1111/j.1365-2648.2012.05996.x. [DOI] [PubMed] [Google Scholar]
  • 46.Albritton JA, et al. The role of psychological safety and learning behavior in the development of effective quality improvement teams in Ghana: an observational study. BMC Health Serv Res. 2019;19(1):385. doi: 10.1186/s12913-019-4234-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.McNamara DA, Rafferty P, Fitzpatrick F. An improvement model to optimise hospital interdisciplinary learning. Int J Health Care Qual Assur. 2016;29(5):550–558. doi: 10.1108/IJHCQA-10-2015-0131. [DOI] [PubMed] [Google Scholar]
  • 48.Shamshirsaz AA, et al. Multidisciplinary team learning in the management of the morbidly adherent placenta: outcome improvements over time. Am J Obstet Gynecol. 2017;216(6):612.e1–612.e5. doi: 10.1016/j.ajog.2017.02.016. [DOI] [PubMed] [Google Scholar]
  • 49.Bunniss S, Gray F, Kelly D. Collective learning, change and improvement in health care: trialling a facilitated learning initiative with general practice teams. J Eval Clin Pract. 2012;18(3):630–636. doi: 10.1111/j.1365-2753.2011.01641.x. [DOI] [PubMed] [Google Scholar]
  • 50.Hovlid E, et al. Sustainability of healthcare improvement: what can we learn from learning theory? BMC Health Serv Res. 2012;12(101088677):235. doi: 10.1186/1472-6963-12-235. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Lyman B, et al. Organizational learning in hospitals: a realist review. J Adv Nurs. 2019(7609811, h3l). [DOI] [PubMed]
  • 52.Nembhard IM, Cherian P, Bradley EH. Deliberate learning in health care: the effect of importing best practices and creative problem solving on hospital performance improvement. Med Care Res Rev. 2014;71(5):450–471. doi: 10.1177/1077558714536619. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Nystrom M. Characteristics of health care organizations associated with learning and development: lessons from a pilot study. Qual Manag Health Care. 2009;18(4):285–294. doi: 10.1097/QMH.0b013e3181bee19e. [DOI] [PubMed] [Google Scholar]
  • 54.Nystrom ME, et al. Exploring the potential of a multi-level approach to improve capability for continuous organizational improvement and learning in a Swedish healthcare region. BMC Health Serv Res. 2018;18(1):376. doi: 10.1186/s12913-018-3129-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Schilling L, et al. Kaiser Permanente's performance improvement system, part 4: creating a learning organization. Jt Comm J Qual Patient Saf. 2011;37(12):532–543. doi: 10.1016/s1553-7250(11)37069-9. [DOI] [PubMed] [Google Scholar]
  • 56.Wells S, et al. Are quality improvement collaboratives effective? A systematic review. BMJ Qual Saf. 2018;27(3):226. doi: 10.1136/bmjqs-2017-006926. [DOI] [PubMed] [Google Scholar]
  • 57.Beitsch LM, Pattnaik A, Madamala K. The Multi-state Learning Collaborative storyboards: quality improvement lessons learned from 162 projects. Florida Public Health Review. 2013;10(Florida State University College of Medicine, Center for Medicine and Public Health, Tallahassee, Florida, USA.):25–34.
  • 58.Freney E, Johnson D, Knox I. Promoting breastfeeding-friendly hospital practices: a Washington State learning collaborative case study. J Hum Lact. 2016;32(2):355–360. doi: 10.1177/0890334415594381. [DOI] [PubMed] [Google Scholar]
  • 59.Nembhard IM. All teach, all learn, all improve?: the role of interorganizational learning in quality improvement collaboratives. Health Care Manage Rev. 2012;37(2):154–164. doi: 10.1097/HMR.0b013e31822af831. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Beitsch LM, et al. The quantitative story behind the quality improvement storyboards: a synthesis of quality improvement projects conducted by the Multi-State Learning Collaborative. J Public Health Manag Pract. 2013;19(4):330–340. doi: 10.1097/PHH.0b013e3182629054. [DOI] [PubMed] [Google Scholar]
  • 61.Godfrey MM, Oliver BJ. Accelerating the rate of improvement in cystic fibrosis care: contributions and insights of the learning and leadership collaborative. BMJ Qual Saf. 2014;23 Suppl 1(101546984):i23–i32. doi: 10.1136/bmjqs-2014-002804. [DOI] [PubMed] [Google Scholar]
  • 62.Nembhard IM. Learning and improving in quality improvement collaboratives: which collaborative features do participants value most? Health Serv Res. 2009;44(2 Pt 1):359–378. doi: 10.1111/j.1475-6773.2008.00923.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Starkey M, Wiest D, Qaseem A. Improving depression care through an online learning collaborative. Am J Med Qual. 2016;31(2):111–117. doi: 10.1177/1062860614555883. [DOI] [PubMed] [Google Scholar]
  • 64.Werdenberg J, et al. Successful implementation of a combined learning collaborative and mentoring intervention to improve neonatal quality of care in rural Rwanda. BMC Health Serv Res. 2018;18:941. doi: 10.1186/s12913-018-3752-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Gillespie SM, et al. Pioneering a nursing home quality improvement learning collaborative: a case study of method and lessons learned. J Am Med Dir Assoc. 2016;17(2):136–141. doi: 10.1016/j.jamda.2015.08.014. [DOI] [PubMed] [Google Scholar]
  • 66.Baum RA, King MA, Wissow LS. Outcomes of a statewide learning collaborative to implement mental health services in pediatric primary care. Psychiatr Serv. 2019;70(2):123–129. doi: 10.1176/appi.ps.201800163. [DOI] [PubMed] [Google Scholar]
  • 67.McHugh M, et al. Changes in patient flow among five hospitals participating in a learning collaborative. J Healthc Qual. 2013;35(1):21–29. doi: 10.1111/j.1945-1474.2011.00163.x. [DOI] [PubMed] [Google Scholar]
  • 68.Nordstrom BR, et al. Using a learning collaborative strategy with office-based practices to increase access and improve quality of care for patients with opioid use disorders. J Addict Med. 2016;10(2):117–123. doi: 10.1097/ADM.0000000000000200. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.Rand CM, et al. A learning collaborative model to improve human papillomavirus vaccination rates in primary care. Acad Pediatr. 2018;18(2 Suppl.):S46–S52. doi: 10.1016/j.acap.2018.01.003. [DOI] [PubMed] [Google Scholar]
  • 70.Nadeem E, et al. A literature review of learning collaboratives in mental health care: used but untested. Psychiatr Serv. 2014;65(9):1088–1099. doi: 10.1176/appi.ps.201300229. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Cleary S, et al. The everyday practice of supporting health system development: learning from how an externally-led intervention was implemented in Mozambique. Health Policy Plan. 2018;33(7):801–810. doi: 10.1093/heapol/czy051. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72.Hirschhorn LR, et al. Learning before leaping: integration of an adaptive study design process prior to initiation of BetterBirth, a large-scale randomized controlled trial in Uttar Pradesh, India. Implement Sci. 2015;10(101258411):117. doi: 10.1186/s13012-015-0309-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.Okafor M, et al. Explication of a behavioral health-primary care integration learning collaborative and its quality improvement implications. Community Ment Health J. 2018;54(8):1109–1115. doi: 10.1007/s10597-017-0230-8. [DOI] [PubMed] [Google Scholar]
  • 74.Shaw EK, et al. Effects of facilitated team meetings and learning collaboratives on colorectal cancer screening rates in primary care practices: a cluster randomized trial. Ann Fam Med. 2013;11(3):220–228. doi: 10.1370/afm.1505. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 75.Russ SA, et al. Improving follow-up to newborn hearing screening: a learning-collaborative experience. Pediatrics. 2010;126(Supplement 1):S59–S69. doi: 10.1542/peds.2010-0354K. [DOI] [PubMed] [Google Scholar]
  • 76.Gerrish K, Keen C, Palfreyman J. Learning from a clinical microsystems quality improvement initiative to promote integrated care across a falls care pathway. Prim Health Care Res Dev. 2018;100897390:1–6. doi: 10.1017/S1463423618000567. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77.Vannoy SD, et al. A learning collaborative of CMHCs and CHCs to support integration of behavioral health and general medical care. Psychiatr Serv. 2011;62(7):753–758. doi: 10.1176/ps.62.7.pss6207_0753. [DOI] [PubMed] [Google Scholar]
  • 78.Bunger AC, et al. Advice-seeking during implementation: a network study of clinicians participating in a learning collaborative. Implement Sci. 2018;13(1):101. doi: 10.1186/s13012-018-0797-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 79.Kotecha J, et al. Influence of a quality improvement learning collaborative program on team functioning in primary healthcare. Fam Syst Health. 2015;33(3):222–230. doi: 10.1037/fsh0000107. [DOI] [PubMed] [Google Scholar]
  • 80.Prabhakaran S, Lee J, O'Neill K. Regional learning collaboratives produce rapid and sustainable improvements in stroke thrombolysis times. Circ Cardiovasc Qual Outcomes. 2016;9(5):585–592. doi: 10.1161/CIRCOUTCOMES.116.003222. [DOI] [PubMed] [Google Scholar]
  • 81.Shellhaas C, et al. The Ohio gestational diabetes postpartum care learning collaborative: development of a quality improvement initiative to improve systems of care for women. Matern Child Health J. 2016;20:71–80. doi: 10.1007/s10995-016-2170-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 82.Lipman PD, Aspy CB. Local learning collaboratives to improve quality for chronic kidney disease (CKD): from four regional practice-based research networks (PBRNs) J Am Board Fam Med JABFM. 2016;29(5):543–552. doi: 10.3122/jabfm.2016.05.160049. [DOI] [PubMed] [Google Scholar]
  • 83.Hagland M. lOM report: 'The path to continuously learning healthcare in America'...Paul Tang, M.D. Healthc Inform. 2012;29(9):30–33. [PubMed] [Google Scholar]
  • 84.McIntosh B. Developing a national learning health system. Br J Healthc Manag. 2017;23(7):304–305. doi: 10.12968/bjhc.2017.23.7.304. [DOI] [Google Scholar]
  • 85.Shah A, et al. Improving access to services through a collaborative learning system at East London NHS Foundation Trust. BMJ Open Qual. 2018;7(3):e000337. doi: 10.1136/bmjoq-2018-000337. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 86.Bailie R, et al. A systems-based partnership learning model for strengthening primary healthcare. Implement Sci. 2013;8(101258411):143. doi: 10.1186/1748-5908-8-143. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 87.Institute of Medicine Roundtable on Evidence-Based Medicine. The National Academies Collection: Reports funded by National Institutes of Health. In: Olsen LA, Aisner D, McGinnis JM, editors. The Learning healthcare system: workshop summary. 2007. National Academies Press (US) National Academy of Sciences.: Washington (DC). [PubMed]
  • 88.Lowes LP, et al. 'Learn from every patient': implementation and early results of a learning health system. Dev Med Child Neurol. 2017;59(2):183–191. doi: 10.1111/dmcn.13227. [DOI] [PubMed] [Google Scholar]
  • 89.Noritz G, et al. "Learn from every patient": how a learning health system can improve patient care. Pediatr Qual Saf. 2018;3(5):e100. doi: 10.1097/pq9.0000000000000100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 90.Irimu G, et al. Approaching quality improvement at scale: a learning health system approach in Kenya. Arch Dis Child. 2018;103(11):1013–1019. doi: 10.1136/archdischild-2017-314348. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 91.English M, et al. Building learning health systems to accelerate research and improve outcomes of clinical care in low- and middle-income countries. PLoS Med. 2016;13(4):e1001991. doi: 10.1371/journal.pmed.1001991. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 92.Brooks D, et al. Developing a framework for integrating health equity into the learning health system. Learn Health Syst. 2017;1(3):e10029. doi: 10.1002/lrh2.10029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 93.Roberts DW. Improving care and practice through learning health systems. Nurs Manage. 2013;44(4):19–22. doi: 10.1097/01.NUMA.0000428194.99141.85. [DOI] [PubMed] [Google Scholar]
  • 94.Forrest CB, et al. PEDSnet: how a prototype pediatric learning health system is being expanded into a national network. Health Aff. 2014;33(7):1171–1177. doi: 10.1377/hlthaff.2014.0127. [DOI] [PubMed] [Google Scholar]
  • 95.Bernstein JA, et al. Ensuring public health's future in a national-scale learning health system. Am J Prev Med. 2015;48(4):480–487. doi: 10.1016/j.amepre.2014.11.013. [DOI] [PubMed] [Google Scholar]
  • 96.Stucki G, Bickenbach J. Functioning information in the learning health system. Eur J Phys Rehabil Med. 2017;53(1):139–143. doi: 10.23736/S1973-9087.17.04612-3. [DOI] [PubMed] [Google Scholar]
  • 97.Perla RJ, Finke B, DeWalt DA. Learning systems at scale: where policy meets practice. JAMA. 2015;314(20):2131–2132. doi: 10.1001/jama.2015.15079. [DOI] [PubMed] [Google Scholar]
  • 98.Forrest CB, et al. PEDSnet: a national pediatric learning health system. J Am Med Inform Assoc. 2014;21(4):602–606. doi: 10.1136/amiajnl-2014-002743. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 99.Kamal AH, et al. A person-centered, registry-based learning health system for palliative care: a path to coproducing better outcomes, experience, value, and science. J Palliat Med. 2018;21(S2):S61–S67. doi: 10.1089/jpm.2017.0354. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 100.Britto MT, et al. Using a network organisational architecture to support the development of Learning Healthcare Systems. BMJ Qual Saf. 2018;27(11):937–946. doi: 10.1136/bmjqs-2017-007219. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 101.Moffatt-Bruce S, et al. IDEA4PS: the development of a research-oriented learning healthcare system. Am J Med Qual. 2018;33(4):420–425. doi: 10.1177/1062860617751044. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 102.Deans KJ, Sabihi S, Forrest CB. Learning health systems. Semin Pediatr Surg. 2018;27(6):375–378. doi: 10.1053/j.sempedsurg.2018.10.005. [DOI] [PubMed] [Google Scholar]
  • 103.Smoyer WE, Embi PJ, Moffatt-Bruce S. Creating local learning health systems: think globally, act locally. JAMA J Am Med Assoc. 2016;316(23):2481–2482. doi: 10.1001/jama.2016.16459. [DOI] [PubMed] [Google Scholar]
  • 104.Johnson LC, et al. Fostering collaboration through creation of an IBD learning health system. Am J Gastroenterol. 2017;112(3):406–408. doi: 10.1038/ajg.2017.9. [DOI] [PubMed] [Google Scholar]
  • 105.Jiang L, et al. Achieving best outcomes for patients with cardiovascular disease in China by enhancing the quality of medical care and establishing a learning health-care system. Lancet. 2015;386 North American Edition(10002):1493–505. [DOI] [PMC free article] [PubMed]
  • 106.Wallace PJ, et al. Optum labs: building a novel node in the learning health care system. Health Aff. 2014;33(7):1187–1194. doi: 10.1377/hlthaff.2014.0038. [DOI] [PubMed] [Google Scholar]
  • 107.Sullivan T. Improving quality and performance in Ontario's cancer services: lessons for constructing a learning healthcare system. Healthc Q. 2015;17 Spec No(101208192):5–9. doi: 10.12927/hcq.2014.24002. [DOI] [PubMed] [Google Scholar]
  • 108.Etheredge LM. Rapid learning: a breakthrough agenda. Health Aff. 2014;33(7):1155–1162. doi: 10.1377/hlthaff.2014.0043. [DOI] [PubMed] [Google Scholar]
  • 109.Atkins D, Kilbourne AM, Shulkin D. Moving from discovery to system-wide change: the role of research in a learning health care system: experience from three decades of health systems research in the Veterans Health Administration. Annu Rev Public Health. 2017;38(Veterans Health Administration, US Department of Veterans Affairs, Washington, DC 20420, USA.): 467–87. [DOI] [PubMed]
  • 110.Serena TE, et al. A new approach to clinical research: Integrating clinical care, quality reporting, and research using a wound care network-based learning healthcare system. Wound Repair Regen. 2017;25(3):354–365. doi: 10.1111/wrr.12538. [DOI] [PubMed] [Google Scholar]
  • 111.Bindman AB. The agency for healthcare research and quality and the development of a learning health care system. JAMA Intern Med. 2017;177(7):909–910. doi: 10.1001/jamainternmed.2017.2589. [DOI] [PubMed] [Google Scholar]
  • 112.Sledge GW, et al. ASCO's approach to a learning health care system in oncology. J Oncol Pract. 2013;9(3):145–148. doi: 10.1200/JOP.2013.000957. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 113.Psek WA, et al. Operationalizing the learning health care system in an integrated delivery system. EGEMS (Washington, DC) 2015;3(1):1122. doi: 10.13063/2327-9214.1122. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 114.Mason AR, Barton AJ. The emergence of a learning healthcare system. Clin Nurs Spec J Adv Nurs Pract. 2013;27(1):7–9. doi: 10.1097/NUR.0b013e3182776dcb. [DOI] [PubMed] [Google Scholar]
  • 115.McElwee NE, Dubois RW. From methods to policy: Enthusiasm for rapid-learning health systems exceeds the current standards for conducting it. J Comp Eff Res. 2013;2(5):425–427. doi: 10.2217/cer.13.51. [DOI] [PubMed] [Google Scholar]
  • 116.Scoap Collaborative, W.G.f.t.S.C. et al. Creating a learning healthcare system in surgery: Washington State's Surgical Care and Outcomes Assessment Program (SCOAP) at 5 years. Surgery. 2012;151(2):146–152. doi: 10.1016/j.surg.2011.08.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 117.Kilbourne AM, et al. Accelerating research impact in a learning health care system: VA's quality enhancement research initiative in the choice act era. Med Care. 2017;55 Suppl 7 Suppl 1(0230027, lsm):S4–S12. doi: 10.1097/MLR.0000000000000683. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 118.Kruk ME, et al. Mortality due to low-quality health systems in the universal health coverage era: a systematic analysis of amenable deaths in 137 countries. Lancet. 2018;392(10160):2203–2212. doi: 10.1016/S0140-6736(18)31668-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 119.Beane A, et al. A learning health systems approach to improving the quality of care for patients in South Asia. Glob Health Action. 2019;12(1):1587893. doi: 10.1080/16549716.2019.1587893. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 120.Cresswell K, Cunningham-Burley S, Sheikh A. Creating a climate that catalyses healthcare innovation in the United Kingdom—learning lessons from international innovators. J Innov Health Inform. 2017;23(4):882. doi: 10.14236/jhi.v23i4.882. [DOI] [PubMed] [Google Scholar]
  • 121.Weaver SJ, et al. A collaborative learning network approach to improvement: the CUSP learning network. Jt Comm J Qual Patient Saf. 2015;41(4):147–159. doi: 10.1016/s1553-7250(15)41020-7. [DOI] [PubMed] [Google Scholar]
  • 122.Adams S, Titler MG. Building a learning collaborative. Worldviews Evid-Based Nurs. 2009;7(3):165–173. doi: 10.1111/j.1741-6787.2009.00170.x. [DOI] [PubMed] [Google Scholar]
  • 123.McCannon CJ, Perla RJ. Learning networks for sustainable, large-scale improvement. Jt Comm J Qual Patient Saf. 2009;35(5):286–291. doi: 10.1016/s1553-7250(09)35041-2. [DOI] [PubMed] [Google Scholar]
  • 124.McLinden D, et al. The learning exchange, a community knowledge commons for learning networks: qualitative evaluation to test acceptability, feasibility, and utility. JMIR Form Res. 2019;3(1):e9858. doi: 10.2196/formative.9858. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 125.Holve E, Segal C. Infrastructure to support learning health systems: are we there yet? Innovative solutions and lessons learned from American Recovery and Reinvestment Act CER investments. J Comp Eff Res. 2014;3(6):635–645. doi: 10.2217/cer.14.72. [DOI] [PubMed] [Google Scholar]
  • 126.Minkman M, Ahaus K, Huijsman R. Performance improvement based on integrated quality management models: what evidence do we have? A systematic literature review. Int J Qual Health Care. 2007;19(2):90–104. doi: 10.1093/intqhc/mzl071. [DOI] [PubMed] [Google Scholar]
  • 127.Powell AE, Rushmer R, Davies HTO. A systematic narrative review of quality improvement models in health care. Edinburgh: NHS Quality Improvement Scotland; 2009. [Google Scholar]
  • 128.Berwick D. A promise to learn—a commitment to act: improving the safety of patients in England. National Advisory Group on the Safety of Patients in England. 2013.
  • 129.Sarakbi D, Sears K. Scoping review on quality integrated care for adolescent depression. 2021. (Manuscript in Preparation). [DOI] [PMC free article] [PubMed]
  • 130.Institute for Healthcare Improvement. Like Magic? (“Every system is perfectly designed…”). 2015. http://www.ihi.org/communities/blogs/origin-of-every-system-is-perfectly-designed-quote.

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

12961_2021_746_MOESM1_ESM.docx (48.1KB, docx)

Additional file 1. References of articles included by “Learning Level”.

12961_2021_746_MOESM2_ESM.docx (44.6KB, docx)

Additional file 2. References of articles excluded during full-text review.

12961_2021_746_MOESM3_ESM.docx (196KB, docx)

Additional file 3. Summary of studies included by “Learning Level”.

12961_2021_746_MOESM4_ESM.docx (61.3KB, docx)

Additional file 4. Characteristics of studies included for the “Learning Systems” synthesis.

Data Availability Statement

Available in additional documents.

Articles from Health Research Policy and Systems are provided here courtesy of BMC

RESOURCES