Development of the Learning Health System Researcher Core Competencies

Abstract

Objective

To develop core competencies for learning health system (LHS) researchers to guide the development of training programs.

Data Sources/Study Setting

Data were obtained from literature review, expert interviews, a modified Delphi process, and consensus development meetings.

Study Design

The competencies were developed from August to December 2016 using qualitative methods.

Data Collection/Extraction Methods

The literature review formed the basis for the initial draft of a competency domain framework. Key informant semi‐structured interviews, a modified Delphi survey, and three expert panel (n = 19 members) consensus development meetings produced the final set of competencies.

Principal Findings

The iterative development process yielded seven competency domains: (1) systems science; (2) research questions and standards of scientific evidence; (3) research methods; (4) informatics; (5) ethics of research and implementation in health systems; (6) improvement and implementation science; and (7) engagement, leadership, and research management. A total of 33 core competencies were prioritized across these seven domains. The real‐world milieu of LHS research, the embeddedness of the researcher within the health system, and engagement of stakeholders are distinguishing characteristics of this emerging field.

Conclusions

The LHS researcher core competencies can be used to guide the development of learning objectives, evaluation methods, and curricula for training programs.

Learning organizations seamlessly share knowledge, evaluate the impact of their actions, and continuously learn to improve outcomes (Senge 1990; Garvin 2000). Although this model of innovation and transformation has been used by several manufacturing and service firms (Garvin 2000), the health sector has been slow to adopt it. Reasons for this include the immense volume and rapidly changing medical information underlying care, the complexity of clinical decision making, and a limited capacity to evaluate the effects of decisions on health, costs, and care experiences. Nonetheless, several trends are converging to create the context for the emergence of health systems that operate as learning organizations, or what has been termed the learning health system (LHS) (Institute of Medicine 2013).

Health care organizations have made large investments in quality assessment and improvement programs to address the pervasive and persistent gaps between available evidence and its application in clinical practice (McGlynn et al. 2003; Mangione‐Smith et al. 2007). The US Department of Health and Human Services’ Office of the National Coordinator of Health Information Technology and Centers for Medicare and Medicaid Services have stimulated the adoption of electronic health records (EHRs), in use by 88 percent of office‐based physicians and 98 percent of hospitals in the United States as of 2015 (The Office of the National Coordinator for Health Information Technology 2017). Electronic health records digitize health care systems in ways that allow for repurposing of clinical information to support quality improvement, research, and health system analytics (Cimino 2007). Innovative approaches for improving health care have shown that significant advances in outcomes can be achieved by engaging patients and physicians in communities that cross organizational boundaries and are committed to a common purpose, such as improving the health of patients with a particular disease (Crandall et al. 2012).

As defined by a 2013 Institute of Medicine report, a LHS can be any type of health care delivery system that combines research, data science, and quality improvement, yielding knowledge as a by‐product of the patient–clinician interaction (Institute of Medicine [IOM] 2013). An essential and distinguishing attribute is engagement of patients and families in governance, research, and improvement projects (Berwick 2009). There have been calls to create a national‐scale LHS (Friedman, Wong, and Blumenthal 2010), to align academic medical centers around the vision of the LHS (Grumbach, Lucey, and Johnston 2014), and to develop specialty‐specific networks organized to promote the LHS across institutions (Kwon et al. 2012; Forrest et al. 2014). For any of these to succeed, a cadre of researchers will be needed to both build the medical evidence base and study innovations in implementation of these practices in health care organizations.

This manuscript is based on work sponsored by the US Department of Health and Human Services Agency for Healthcare Research and Quality (AHRQ). The Agency for Healthcare Research and Quality has long invested in efforts that promote the development, synthesis, and rapid movement of new evidence into practice including the training of researchers. With the emergence of the LHS and the growing need for researchers ready to spearhead the adoption of evidence on a systematic basis to improve decision making throughout organizations in a consistent way (Bindman 2017), AHRQ commissioned this a project to develop and prioritize a set of core competencies to guide the design, implementation, and evaluation of training programs for LHS researchers.

Prior competencies have been articulated for health services researchers (Forrest et al. 2009); however, there has been limited attention given to the unique skills and knowledge that LHS researchers need to be successful and to contribute optimally to the development of health systems. The project adopted an existing definition of competencies as knowledge‐ or skill‐based assets that trainees should acquire during their training (Forrest et al. 2009). Core competencies are knowledge and skill sets that should be characteristics of training programs for LHS researchers. Each competency will have associated learning objectives, describing the educational and experiential approaches for achieving it. Although the end results (i.e., competencies) of training programs should be similar, the approaches used to attain them (i.e., learning objectives, format, and evaluation methods) will differ depending on the setting and the needs and prior experiences of the learner.

Methods

The development of the LHS researcher competencies occurred in three phases. Most of the work was completed during three consensus development meetings with a 19‐member expert panel. The panelists included individuals with expertise in patient‐centered outcomes research, statistics, epidemiology, health services research, quality improvement and implementation science, and informatics. These experts were drawn from the public and private sectors, and represented researchers, practicing clinicians, patients, and health system leaders. Most panelists had in‐depth experience working with C‐suite‐ level health care executives.

We started by drafting a LHS researcher definition and a domain framework to organize the competencies. This first phase involved a literature review, semi‐structured interviews with key informants, and a consensus development meeting with the expert panel. During the second phase, the panel formed writing teams and drafted specific competencies. These were reviewed and revised during a second consensus development meeting. The final list was winnowed based on a modified Delphi process of the expert panel, followed by a final consensus development meeting. Each of these activities is described below.

Literature Review

We conducted a literature review in August 2016 to identify LHS conceptual frameworks and definitions and to generate an initial competency domain framework. Starting with seminal articles and reports that describe structures and functions of the LHS (Etheredge 2007; Friedman, Wong, and Blumenthal 2010; Greene, Reid, and Larson 2012; Institute of Medicine 2013; Forrest et al. 2014), we identified relevant medical subject heading (MeSH) terms that have been used to index LHS literature in MEDLINE to construct search strategies. The MeSH terms were combined with free‐text terms using Boolean operators to search PubMed. Because the field of LHS research is young, no time restrictions were applied to the search results. The search was limited to the English language, humans, and the United States. We supplemented the list of articles retrieved from MEDLINE with additional searches using Google Scholar, a review of reference lists of identified articles, and articles identified by expert panel members. These methods identified 197 articles. The full text of each article was examined by two members of the project team; those that described a conceptual framework or provided a definition of a LHS (n = 23) were studied more thoroughly. We extracted a list of LHS functions from these articles, produced a preliminary competency domain framework, and drafted an initial definition of an LHS researcher.

Key Informant Interviews

To ensure completeness and clarity of the domain framework, we conducted eight semi‐structured interviews with key informants with expertise in LHS research or health services research. Additional feedback was obtained from six health services research training directors, each of whom led a training program funded by the AHRQ. These 14 individuals provided a critical review of the LHS researcher definition, the competency domains, and the specific competencies. Their feedback was reviewed and discussed during the second and third expert panel meetings.

Expert Panel Meetings

We convened an expert panel (n = 19 members) to develop a definition of an LHS researcher, the competency domain framework, and the specific competencies. The panel integrated the information from the literature review and key informant interviews during its deliberations. The panel met three times—twice for in‐person, day‐long meetings, and once by teleconference.

During the first meeting, the expert panel agreed to adopt the Institute of Medicine's 2013 definition of a LHS in order to guide further work: [A system in which] “science, informatics, incentives, and culture are aligned for continuous improvement and innovation, with best practices seamlessly embedded in the care process, patients and families active participants in all elements, and new knowledge captured as an integral by‐product of the care experience” (Institute of Medicine 2013). After examining the literature review, the panel produced an initial competency domain framework and the definition of an LHS researcher.

Development of Individual Competencies

For each of 10 initial competency domains, the expert panel formed writing teams that were charged with drafting specific competencies relevant to each domain. A total of 91 competencies were produced at this stage. We then asked the 19 panel members to provide importance ratings on a five‐point Likert scale for each competency. These ratings and additional comments from the full panel review were used during the second consensus development meeting of the expert panel. During that meeting, the 91 competencies were winnowed to 67, often by combining similar concepts.

Modified Delphi Process

Between the second and third expert panel meetings, we conducted a modified Delphi process in which panel members rated the importance of each of 67 competencies on a scale from 1 (low) to 9 (high). These ratings were used to produce a prioritized list of competencies during a final in‐person expert panel meeting. For each competency, the median and the percentage of respondents giving a competency a high rating, defined as between 7 and 9, were computed. Only those competencies with a median of at least 7 and ≥75 percent of panel members rating it between 7 and 9 were evaluated for retention, so that the final list reflected near‐consensus of the diverse panel members.

Results

Definition of an LHS Researcher

Guided by the 2013 Institute of Medicine's definition of a LHS (Institute of Medicine 2013), the literature review, and expert panel deliberations, we produced the following definition of a LHS researcher:

An individual who is embedded within a health system and collaborates with its stakeholders to produce novel insights and evidence that can be rapidly implemented to improve the outcomes of individuals and populations and health system performance.

Conventionally, researchers tend to be isolated from the health systems they study. The embeddedness of a LHS researcher, however, is an essential and distinguishing feature. The researcher must be part of the system when conducting the research, either as an employee or as an invited partner, who leads or assists with the development, conduct, implementation, and dissemination of research designed to address questions of interest to the stakeholders of the health system. This embeddedness allows the LHS researcher to execute research in such a way that does not disrupt day‐to‐day operations and ensures that the investigator has an appreciation for the perspectives of those operating the system and those it serves. Researchers who merely use data collected within health systems, without clear links to the priorities and operations of the system, would not be considered LHS researchers.

The expert panel took a broad view of health system as one or more organizations that provide services to restore or promote individual or population health. They could be primary care centers, academic medical centers, hospitals, retail clinics, entire delivery systems ranging from integrated systems providing care and coverage to conventional fee‐for‐service systems operating hospitals and clinics, or other organizational models that focus on improving health. The term stakeholder refers to patients, caregivers, clinicians, system leaders, and other individuals who interact to carry out the functions of the health system. Finally, the word rapidly was used to connote the need in LHS research to ensure prompt and efficient knowledge generation and application. The importance of timely provision of research evidence has been highlighted by clinical systems leaders convened by the Institute of Medicine (Institute of Medicine 2013).

Domain 1: Systems Science

During the early stages of competency development, several themes related to systems science were highlighted. These included the following: using systems thinking to develop and implement LHS projects; understanding the structures, functions, and outcomes of health systems and how they interrelate; and using systems theories in LHS research. The panel considered systems theory as the collection of interdisciplinary frameworks for understanding systems, which are composed of interconnected and interrelated parts that interact to produce emergent and complex behaviors and results. During the final consensus development meeting, the expert panel concluded that systems science was an essential and distinguishing attribute of LHS research, meriting its own domain (Table 1). A deep understanding of systems science was believed necessary to conduct research effectively and implement its findings in the context of complex health systems.

Table 1.

Systems Science Competency Domain

Domain Definition

To understand how health systems are financed and operate and how to apply systems theory to research and implementation.

Competencies

1.1: Demonstrate knowledge of how systems theories can be used to understand how the interactions of the parts of health systems operate to produce value for stakeholders.

1.2: Demonstrate systems thinking in the design and conduct of research and implementation of its findings within the context of complex health systems.

1.3: Demonstrate knowledge of the financing, organization, delivery, and outcomes of health care services and their interrelationships.

1.4: Demonstrate ability to assess the extent to which research activities will likely contribute to the quality, equity, or value of health systems.

Learning health system researchers must be able to design and conduct research within the context and complexity of an operating health system. This ability requires an understanding of how health systems are led, financed, and managed; how health care is delivered; the effects of financial and nonfinancial incentives on health system employees; and how the varied components of the health system work together to produce care and outcomes. Learning health system researchers must also be able to rapidly apply medical evidence in health systems, producing real value, including financial value, to the organizations in which they are embedded. The ability to facilitate rapid implementation of new knowledge generated from research requires an understanding of the complexity of health systems and an ability to demonstrate how research activities will contribute to quality, equity, and value of the health system.

Domain 2: Research Questions and Standards of Scientific Evidence

This competency domain addresses the need to ask meaningful research questions, engaging stakeholders in their elicitation and prioritization, and critically analyzing scientific evidence with special relevance to a health care organization (Table 2). Learning health system researchers should investigate questions that are important not only to scholars and experts in the field, but also to the health care organization, its patient population, and clinicians. Research questions should build on extant research, although questions that confirm prior evidence or provide insights into subpopulations served by the LHS may be appropriate. Research questions must be meaningful to patients, families, clinicians, or system leaders. This means that the LHS researcher must understand the features of a health system, such as its structures and functions and the financial and nonfinancial incentives that affect stakeholder behavior. Although LHS research can and should be rigorous, its focus is the applicability of findings to the care settings and populations of the health system.

Table 2.

Research Questions and Standards of Scientific Evidence Competency Domain

Domain Definition

To ask meaningful questions relevant to health system stakeholders and evaluate the usefulness of scientific evidence and insights.

Competencies

2.1: Demonstrate the ability to compose feasible and timely research questions and hypotheses, incorporating stakeholder priorities, to generate evidence that informs meaningful clinical and policy decisions.

2.2: Demonstrate the ability to engage with all relevant stakeholders (patients, families, clinicians, and system leaders) in the elicitation and prioritization of research questions that address current and future stakeholder needs.

2.3: Demonstrate the ability to critically analyze and assess available scientific evidence from peer‐reviewed articles, systematic reviews, meta‐analyses, and gray literature to identify novel LHS questions and to judge the applicability of the evidence to a local care setting

Domain 3: Research Methods

This competency domain addresses the use of models and frameworks, evaluation and selection of appropriate study designs, outcome measurement, and data analysis for research performed within the context of a health system (Table 3). All LHS researchers should be able to design and analyze intervention studies using experimental (individual‐ and cluster‐level randomization) and quasi‐experimental approaches (Wagenaar and Komro 2011; Stürmer and Brookhart 2013) that permit the estimation of the effect of treatments or interventions. They also must know when to use mixed methods to understand how and why a given intervention works, for whom, and in what contexts. Based on the specific research question, researchers should be able to explain the criteria for choosing an analytic approach. They should also be able to interpret and explain a study's conclusions with respect to a particular LHS.

Table 3.

Research Methods Competency Domain

Domain Definition

To conduct research within the context of real‐world health systems using appropriate study designs and analytic methods to assess outcomes of interest to health systems stakeholders.

Competencies

3.1: Demonstrate ability to use theory and conceptual models in the design and interpretation of LHS research.

3.2: Demonstrate ability to develop an appropriate observational, quasi‐experimental, or experimental study design while mitigating threats to internal and external validity for research that is minimally disruptive to operations in real‐world health systems and practices.

3.3: Demonstrate knowledge of mixed methods and how they can be used to improve LHS research studies.

3.4: Demonstrate knowledge of how to assess multilevel determinants of health and health care disparities when designing studies.

3.5: Demonstrate ability to select and interpret appropriate clinical, financial, and patient‐centered outcomes of interest based on the concepts they measure and their measurement properties.

3.6: Demonstrate ability to apply the principles of hypothesis testing and statistical inference to data collected routinely through the course of care as well as supplemental data from patients, providers, and health systems.

Learning health system researchers must be able to understand and apply principles of measurement science to the selection and assessment of outcomes as well as to moderators and mediators of these outcomes within an LHS. Although most LHS researchers will not develop new measures or need to be expert in modern psychometric theory, they do need to be able to identify, incorporate, and interpret a range of health outcome measures appropriately and understand the limits of the measures available. In addition to measurement error, reliability, and validity, LHS researchers should be familiar with a variety of psychometric properties, including normative reference samples and responsiveness of measures to clinical change.

Domain 4: Informatics

Informatics focuses on the science of information, and the technologies, processes, and people involved in the use of information. The LHS is a paradigm for the continuous transformation of data to knowledge to action, and for which informatics is a core capability. Informatics is by nature multidisciplinary, and LHS researchers must be able to communicate effectively about information systems with health system leaders, clinicians, and technical experts (Table 4).

Table 4.

Informatics Competency Domain

Domain Definition

To know how to use information systems to conduct LHS research and improve patient and health system outcomes.

Competencies

4.1: Demonstrate ability to use data derived from electronic health records and other clinical information sources for research and quality improvement.

4.2: Demonstrate knowledge about additional data sources that can be linked to health system clinical data in order to augment exposure and outcome ascertainment.

4.3: Demonstrate ability to assess data quality and apply data quality assurance processes, including error prevention, data cleaning, data monitoring, documentation, and relevant data standards.

4.4: Demonstrate knowledge of population health informatics, including disease surveillance, monitoring of community health, assessment of social and behavioral determinants of health, and geographic information systems.

4.5: Demonstrate knowledge of clinical information systems, including electronic health records, clinical documentation, computerized physician order entry (CPOE), clinical decision support systems, electronic prescribing, medical imaging, and clinical/population dashboards.

Learning health system researchers need to understand how to use data from EHRs for quality improvement and research, while recognizing the limitations of these data and ways to improve the quality of data entered at the point of care. In addition, they should be abreast of advances in clinical decision support and the most effective ways for applying knowledge when and where it will be useful to patients and clinicians. They should know how to collect data directly from patients and other stakeholders (e.g., patient‐reported outcomes and care experiences) while minimizing burden on respondents and clinical care processes. Learning health system researchers should be able to identify datasets that can be linked to the EHR to expand the measures available (e.g., area‐level data on environmental influences of health linked via geocodes). Finally, LHS researchers should be able to identify and incorporate data from the rapidly expanding digital traces that patients provide from personal wearable and home sensors.

Domain 5: Ethics of Research and Implementation in Health Systems

Ethics training for LHS researchers must include attention to activities on the borderline between research and improvement and the ethical issues related to each (Finkelstein et al. 2015). Training for clinical and health services research has long had an ethics component, and specific standards for training in the responsible conduct of research are required of clinical research trainees. Areas of focus of research ethics training typically begin with protection of human subjects. It is critical that researchers understand the underlying principles (and their origins in the Belmont report and subsequent policies and regulation) and how the current regime of institutional review boards, with federal oversight, operationalizes them. Privacy of health information is covered by specific regulations (Health Insurance Portability and Accountability Act), but it is important that the ethical principles underpinning these are understood. Research ethics training should provide knowledge and skills to understand and manage potential conflicts of interest, and issues of research integrity.

The above ethical principles and concepts generally apply to all clinical and health services research. However, additional ethics training and skills for conducting research and implementation embedded in LHSs will be needed (Table 5). The categories outlined above will largely be the same, but the competencies within each may differ. For example, pragmatic, cluster‐randomized trials in which whole clinical units may be assigned to one or another care delivery mode raise new issues regarding appropriate notification and consent (Platt, Kass, and McGraw 2014).

Table 5.

Ethics of Research and Implementation in Health Systems Competency Domain

Domain Definition

To ensure that research and quality improvement performed in health care settings adheres to the highest ethical and regulatory standards.

Competencies

5.1: Demonstrate ability to apply ethical principles in the engagement of health systems including issues of business ethics and importance of publishing both positive and negative findings in the public domain.

5.2: Demonstrate knowledge of what activities constitute research as opposed to quality improvement activities and seek appropriate oversight for each.

5.3: Demonstrate knowledge of specific Health Insurance Portability and Accountability Act (HIPAA) requirements associated with varied data sources used in health systems research activities and seek appropriate approvals.

5.4: Demonstrate ability to identify and minimize potential conflict of interests in the design, conduct, and reporting of research conducted in health systems.

5.5: Demonstrate knowledge of ethical and legal considerations when engaging in multisystem studies for compliant collaboration and study conduct.

Ethical issues in engagement of patients, clinicians, and health system leaders in research and implementation activities may also require additional attention in training programs. While privacy issues have always been part of research ethics, new uses of large amounts of LHS data collected in the course of routine care require different training for researchers. Finally, a defining feature of LHSs is the ability, when appropriate, for rapid implementation of research evidence and further rapid‐cycle improvement in systems to reliably implement best practices for patients, tailored to individual circumstances and preferences.

Domain 6: Improvement and Implementation Science

Learning health system researchers need the skills to integrate knowledge sets generated from research performed in health care settings as well as the capacity to apply knowledge using quality improvement and other methods (Table 6). Functioning LHSs generate new knowledge as a natural outgrowth of patient care (Institute of Medicine 2013). The results of near real‐time data aggregation, analysis, and evaluation will naturally prompt changes that feed back into the system to improve care delivery (Wysham et al. 2016). LHS researchers should understand traditional quality improvement methods and the relationship to research performed in the LHS. Researchers should also have a basic understanding of implementation science, which is defined by the NIH as “the scientific study of the use of strategies to adopt and integrate evidence‐based health interventions into clinical and community settings in order to improve patient outcomes and benefit population health (NIH PAR 16‐238).” Quality improvement involves the use of systems engineering processes and control tools to improve overall teamwork, leadership, engagement, and change management methodologies in a clinical care setting. In contrast, implementation science involves the rigorous study of methods or strategies (e.g., audit and feedback, financial incentives, coaching, mentoring) used to change provider behavior in order to enhance the uptake of an evidence‐based clinical treatment or practice (Powell et al. 2015). It is critical that researchers in LHSs consider specifically how implementation of innovations or improvements can mitigate health and health care disparities to promote health system equity.

Table 6.

Improvement and Implementation Science Competency Domain

Domain Definition

To reduce avoidable variation in process and outcome and ensure the systematic uptake of research findings in a health system.

Competencies

6.1: Demonstrate the ability to employ specific quality improvement methods to reduce avoidable variation in clinical processes and outcomes in routine practice.

6.2: Demonstrate the ability to employ specific implementation science or quality improvement methods to study and promote systematic uptake of research findings and other effective clinical interventions into routine practice.

6.3: Demonstrate knowledge regarding when to mount larger efforts to scale‐up, spread, and sustain successful interventions based on strength of clinical evidence and organizational and provider readiness to change and adopt interventions

Domain 7: Engagement, Leadership, and Research Management

Special leadership skill that LHS researchers must employ includes being able to engage diverse health system stakeholders in all aspects of the research process and the ability to integrate multiple perspectives from a variety of disciplines and stakeholders (Table 7). Engagement refers to the deliberate practice of identifying and integrating diverse stakeholders as active participants in the planning, execution, and application of research studies and results. The unique perspectives, experiences, and skills of each stakeholder group are recognized as valuable and critical to achieving study goals and improving outcomes. Dissemination involves the distribution of new knowledge and effective communication of the implications of this knowledge across stakeholder groups and to the public at large. Engagement and dissemination within a LHS require a unique set of skills because of the diversity of the stakeholder groups, the complexity of the health care system, and the need to improve care in a timely manner.

Table 7.

Engagement, Leadership, and Research Management Competency Domain

Domain Definition

To engage stakeholders in all aspects of the research process and effectively lead and manage LHS research teams and projects.

Competencies

7.1: Demonstrate the ability to build and lead research teams with diverse health system stakeholder representation.

7.2: Demonstrate knowledge of the values and communication mechanisms used by stakeholder groups involved in research in health systems.

7.3: Demonstrate ability to translate, disseminate, and communicate the value proposition and business case for research to diverse health system stakeholders.

7.4: Demonstrate ability to conduct effective team‐based project management, employing skills in leadership, communication, negotiation, consensus building, and problem‐solving.

7.5: Demonstrate ability to develop protocols consistent with health systems needs and timelines, employing patient and clinician engagement, and using a mix of conventional and alternative funding sources.

7.6: Demonstrate ability to implement protocols aligned with health systems operations and integrated into clinical settings, including engaging clinicians in the research process.

7.7: Demonstrate knowledge of participatory research approaches that foster participation and engagement of vulnerable populations.

Learning health system research takes a pragmatic approach toward evidence generation and application. It requires researchers to be keenly aware of clinical and research operations and organizational governance and culture related to research. Because the results are practical and meaningful to system stakeholders, the LHS researcher may rely on institutional financial support as a source of sponsorship. Finally, researchers must conduct their work and disseminate their findings on shorter timelines than conventional clinical research.

Discussion

The LHS researcher core competencies described in this manuscript were developed using an iterative, multimethod process to elicit, refine, and prioritize 33 knowledge‐ and skill‐based capacities that all LHS researchers should acquire during their training. The core competencies and domain structure generated from this process are intended to provide a framework for training programs that will prepare a cadre of LHS researchers ready to consistently address real‐world health care delivery challenges, drive improvements, and promote individual, population, and system outcomes. They should be considered a first attempt to define the scope of training for LHS researchers and will evolve as the field matures. The individual competencies focus on specific assets needed to generate and apply new knowledge within the context of health systems. The real‐world milieu of LHS research, the embeddedness of the researcher within the health system, and engagement of stakeholders in all aspects of research and improvement are distinguishing characteristics of this emerging field.

The core competencies are meant to guide the development or expansion of existing training programs. They do not prescribe the pedagogical or experiential methods by which the skills and knowledge are to be acquired and evaluated. Individual training programs will determine their specific learning activities (teaching strategies, curricula, and participation in LHS activities), approaches for assessing competency achievement, and areas of specialized emphasis. For example, one training program may choose to focus more in‐depth training in implementation science, while another may focus more on experimental designs for patient‐centered outcomes research. It should be noted that LHS research requires multiple disciplines to work together to produce the science and translate results; thus, it is not expected that any one individual will be an expert in all competency domains. However, the LHS researcher must be able to assemble and lead multidisciplinary teams of experts who collectively bring expertise in each of the domains outlined in the framework to realize the broader goals of the LHS.

The expert consensus panel identified foundational knowledge and skills that LHS researcher trainees should either possess coming into an LHS training program or acquire in parallel during their participation in a training program. These competencies include existing health services research competencies (Forrest et al. 2009) and basic knowledge in epidemiology, biostatistics, clinical research, and behavioral and social sciences relevant to health care. However, the expert panel highlighted the importance of not requiring a set of foundational competencies as prerequisites for entry into a LHS researcher training program. This approach could have the unintended consequence of limiting training programs to postgraduate health services researchers. However, basic competencies in research design, epidemiology, and biostatistics must be achieved either before or as part of training as an LHS researcher.

There are several distinct attributes of research in LHSs that have implications for training their researchers. First, LHS research is concerned with the direct application of results to the context of an individual system (i.e., external validity). For health systems, research evidence may not need to meet the same standards as medical evidence for clinical decision making; often good enough evidence is sufficient to make strategic, operational, or financial decisions. Balancing the desire of health systems for rapidly generated, practical evidence with the rigors of peer‐review and scientific standards is one of the key challenges for LHS researchers. Second, the timeline for LHS research often does not fit the timeline of conventional research awards. LHS research may capitalize on an imminent policy or program phenomenon needing evaluation and study using a rapid‐cycle approach. It is often an iterative process with continuous cycles of analysis and feedback and output, rather than a one‐and‐done study. Third, this cycle of LHS research often does not fit well with conventional grant funding opportunities. LHS researchers may rely on health systems to invest in their research or awards to support infrastructure.

The LHS researcher domain framework and core competencies serve as a starting point for further work and discussion. The core competencies are meant to evolve together with learning health systems experiences, science, and research. Greater understanding of how the competencies are operationalized with regard to curricula and teaching strategies will inform future iterations. We encourage the use and refinement of the competencies among health systems that are dedicated to the LHS system approach and that seek to train individuals in the methods for rapidly advancing and applying new knowledge to improve patient and system outcomes.

Supporting information

Appendix SA1: Author Matrix.