How the VA is training the Next‐Generation workforce for learning health systems

Abstract

Objectives

The U.S. Department of Veterans Affairs (VA) has been a national leader in Learning Health System (LHS) implementation due to its combined mission of research, education, clinical care, and emergency preparedness. We describe the current VA LHS training ecosystem within the Veterans Health Administration's Office of Academic Affiliations (OAA), Office of Research and Development (ORD), ORD's Health Services Research and Development (HSR&D) program, and Innovation Ecosystem (IE), including lessons learned regarding their sustainment.

Methods

The VA LHS training ecosystem is based on the Learning Loop and HSR&D Quality Enhancement Research Initiative (QUERI) Roadmap, which describes VA learning opportunities, underlying infrastructures, and core competencies.

Results

VA‐focused LHS educational programs include data‐to‐knowledge initiatives in health sciences and analytics, for example, OAA/HSR&D health services and informatics research fellowships; knowledge‐to‐performance opportunities in implementation and quality improvement, for example, QUERI Learning Hubs and IEs' Diffusion of Excellence Initiative; and performance‐to‐data embedded opportunities, for example, IE's entrepreneur fellowship programs and QUERI's Advancing Diversity in Implementation Leadership. These training programs are supported by combined VA research and clinical operations investments in funding, informatics, governance, and processes. Lessons learned include ongoing alignment of research funding with operational priorities and capacity, relentless recruitment and retention of implementation, system, and information scientists especially from under‐represented groups, sustainment of data infrastructures suitable for research and quality improvement, and ensuring sustainable funding opportunities for researchers to work on system‐wide health care problems.

Conclusions

There is an urgent need to expand training opportunities in LHSs, especially as health care is increasingly driven by multiple interested parties, impacted by persistent health disparities exacerbated by emerging public health threats, and rapid technology growth. With ongoing alignment of research and clinical goals, foundational support through research funding, underlying clinical operations infrastructures, and active engagement interested parties, VA's LHS training ecosystem promotes a more LHS‐savvy, 21st century workforce.

1. INTRODUCTION

The United States (US) health care is changing, and there is an urgent need for training for the health care research and clinical workforce to meet the changing needs of the health systems and the people they serve. The National Academy of Medicine's Future of Health Services Report in 2018 ¹ strongly emphasized that research and clinical practice should focus on addressing the complex, multilevel problems facing health care systems to improve quality, promote health equity, and have a public health impact. Many of these recommendations call for the adoption of a Learning Health System (LHS), which is the process of using data to continuously identify and solve real world problems facing organizations through the use of advances in informatics, socio‐technical and human factors system design, and implementation science. ² , ³

For over 30 y, the US Department of Veterans Affairs (VA) has been a national leader in LHS‐driven health services research primarily because of its embedded research program within a national health care system that is actively informed by policymaker, provider, and consumer (eg, Veteran Service Organization) priorities. ⁴ , ⁵ VA is also a national leader of LHS‐driven research methods in implementation science, informatics, quality improvement, and socio‐technical infrastructures. ⁵ Nearly three‐quarters of all the US physicians receive training in VA. Overall, VA has over 150 training sites with academic medical schools and 21 advanced fellowship programs that offer post‐residency, post‐doctoral, and post‐masters interdisciplinary programs for physicians and dentists, as well as associated health professions, including psychologists, social workers, and pharmacists. ⁶ , ⁷ As one of the largest single employers of health professionals, VA has over 420 000 employees who care for over 6 million Veterans per year, many of whom represent marginalized and at‐risk populations, including those experiencing mental disorders, homelessness, physical disabilities, or multiple chronic conditions. ⁴ A unique feature of the VA is its close connection to academic health systems, which enables trainees and faculty to obtain advanced training to conduct research that directly impacts the care they provide to their patients in a national network of hospitals and clinics. ⁸

Nonetheless, current trends in health care point to the urgent need to expand training programs in LHS beyond traditional academic researchers and to ensure these training programs are preparing the next generation of scientific workforce for health systems. Notably, health care is becoming increasingly consumer‐driven with a wide variation in access to, sources of, and trust in health information. This has been especially pronounced during the COVID‐19 pandemic among at‐risk populations, who have historically been marginalized or experienced health care disparities. ⁹ , ¹⁰ The persistent gaps in health outcomes among populations that have been historically marginalized call for a more focused approach to addressing health disparities in systematic ways that extend beyond the clinic walls. In addition, a substantial amount of innovative health care research is happening beyond the traditional academic health system, through new public‐private models of health care (eg, CityBlock), ¹¹ virtual care provision, and the expansion of health care to non‐clinical settings such as community organizations and schools (eg, Black Health Movement). ¹² Finally, the COVID‐19 pandemic challenged current health care infrastructures ¹³ , ¹⁴ to respond more quickly to emerging public health crises, with accelerated adoption of virtual care, which also revealed significant gaps in the evidence base and data infrastructure.

There are also perssistent barriers to translating research into practice that call for more LHS‐focused training initiatives, especially for academic researchers. It takes years, if not decades, ¹⁵ for effective innovations to be adopted by practitioners in routine care because of a lack of training in how to do real‐world implementation. Most clinical research studies occur in highly resourced settings, which are poor analogues for the communities that could benefit from the research. Researchers often lack the training to elicit active input from end‐users, or training in the organizational acumen to get processes in place to facilitate adoption of innovations. Health systems may also make decisions without waiting for research and often have limited information regarding the value propositions of many innovations developed from research. Researchers often lack the skills to not only communicate study results back to community members and end‐users but to also support them to change practice or develop implementation roadmaps to sustain improvements for research funding support.

LHS‐driven training programs that support both research and practice, especially within large or multi‐networked health care systems, can address these gaps. First, LHS competencies can foster a focus on addressing real‐world problems through training in organizational acumen, informatics, implementation, quality improvement, and effective engagement of multiple interested parties. Second, LHSs involve ongoing input and engagement from interested parties, who form a learning community from the beginning of the process to determine the prioritization, design, and communication of research study impacts. Third, LHS‐oriented projects run simultaneous research and quality improvement cycles, involving the groundwork to build the data infrastructure and evidence, while also implementing rapid and rigorous data feedback to providers to foster immediate quality improvement changes, as well as “piggyback” research protocols involving deep analyses of mechanisms and determinants of the interventions or processes. Finally, there are ongoing opportunities for on‐the‐job training through the development of implementation plans that are used and owned by the practitioners to sustain the study results if proven effective once the project ends.

To this end, this article describes current LHS‐focused educational and competency‐building initiatives in VA. In doing so, we propose a description of a VA LHS training ecosystem and core competencies, how it fits within the larger context of VA LHS research and practice, and lessons learned based on the challenges and opportunities in sustaining these programs that inform the key ingredients for sustaining similar initiatives in other health systems. Most of the programs discussed exist under the VA's Veterans Health Administration Discovery, Education, and Affiliated Networks (DEAN) program and are available to researchers, practitioners, and leaders in real‐world settings. DEAN includes the Office of Academic Affiliations (OAA), which oversees clinical and research graduate, post‐doctoral, and clinical training programs, Office of Research and Development (ORD), which funds research conducted by VA investigators, ORD's Health Services Research and Development (HSR&D) program, Quality Enhancement Research Initiative (QUERI), which funds investigators to conduct high‐priority quality improvement initiatives for Veterans that are grounded in implementation science, and Innovation Ecosystem (IE), which supports VA employees in the development, scale‐up, and sustainment of new programs and practices that improve Veteran care.

2. UNDERLYING FRAMEWORK FOR THE VA LHS TRAINING ECOSYSTEM

Figure 1 provides an overview of the LHS‐focused VA training programs that are available across the distinct phases of an LHS cycle. This framework is based on the Friedman Learning Loop ¹⁶ and further adapted using the QUERI Roadmap for implementation and quality improvement. ¹⁷ , ¹⁸ The foundation of these training programs is based on ORD research priority goals, ¹⁹ which support high‐caliber science through a strong community of researchers across the translation spectrum in close partnership with clinical operations. Specific strategic goals include using VA data for Veterans' healthcare improvement, increasing Veteran access to high‐quality clinical trials, enhancing the substantial real‐world impact of research, promoting diversity, inclusion, and equity in VA's sphere of research, and building a research community.

FIGURE 1.

VA training program ecosystem: enhancing core scientific processes across the learning health system cycle. ADIL, advancing diversity in implementation leadership; EHR, electronic health record; IE, innovation ecosystem; HSR&D, health services research and development; OAA, office of academic affiliations; ORD, office of research and development; QUERI, quality enhancement research initiative; RIVRs, research to impact for veterans initiatives; SGIM, society for general internal medicine

These training programs rely on a network of research and clinical services infrastructures that provide sustained research and clinical operations funding, access to national electronic medical record and population health databases, standard governance processes supporting data access/curation, as well as research oversight and adherence to ethical standards, and access to methodological expertise, primarily through ORD/HSR&D national centers. Key examples of infrastructures provided by VA research and clinical operations that supports LHS trainees are available in Table 1. Formation of an LHS learning community happens when trainees are hired to collaborate and lead research or quality improvement efforts as part of a larger research center (eg, HSR&D Centers of Innovation, QUERI National Programs) ⁴ , ²⁰ or VA national clinical program office to conduct preliminary research or quality improvement work.

TABLE 1.

Core Infrastructures Supporting Learning Health System (LHS)‐focused Learning Opportunities in VA

LHS infrastructure domain	Examples
Funding	HSR&D Centers of Innovation HSR&D Consortia of Research (COREs) HSR&D Research to Impact for Veterans Initiatives (RiVRs) ORD/HSR&D Investigator‐initiated Research, Career Development Awards, Diversity Supplement funding mechanisms ORD Research Career Scientist awards QUERI Programs, Partnered Implementation/Evaluation Centers ORD Cooperative Studies Program Centers (coordinating centers and funding mechanisms) VA Program Office or regional Integrated Service Network project funding opportunities
Informatics	VA National Patient Care Database and Corporate Data Warehouses VA Support Service Center Capital Assets data portal (VSSC) National Program Office population health registries HSR&D VA Information Resource Center (VIREC) ORD VA Informatics and Computing Infrastructure (VINCI) ORD Million Veteran Program (MVP) Centralized Interactive Phenomics Resource (CIPHER)
Governance	ORD Guidance documents regarding publication, data integrity, human subjects, technology transfer, research/non‐research protocols and policies VA Data Access Request Tracker System (DART) QUERI Memoranda of Understanding for Partnered Evaluation and Implementation projects
Processes	QUERI national Network of Programs (and Mentoring Cores) QUERI‐VA Integrated Service Network Partnered Implementation Initiatives QUERI Partnered Evaluation Initiatives QUERI Evidence‐based Policy Centers
Methods resources	HSR&D Health Economics Resource Center (HERC) QUERI Center for Evaluation and Implementation Resources (CEIR) HSR&D Evidence Synthesis Program (ESP) HSR&D Center for Information Dissemination and Education Resources (CIDER) ORD/QUERI Evidence, Policy, and Implementation Center (EPIC) QUERI Partnered Evidence‐based Policy Resource Center (PEPReC)

Figure 1 also depicts core VA LHS educational programs along with examples of scientific areas of inquiry based on the LHS cycle. Training programs focused on knowledge generation in an LHS (data‐to‐knowledge) include the OAA/HSR&D Advanced Fellowship in Health Services Research ²¹ for post‐doctoral level investigators as well as initiatives through the Office of Research and Development National Artificial Intelligence Institute ²² to recruit big data scientists to VA. Examples of LHS‐focused projects led by trainees include quantitative analyses such as predictive analytics, quantitative and qualitative studies focused on the lived experiences of patients, providers, and other interested parties, as well as evidence syntheses and treatment intervention trials.

Training programs focused on knowledge‐to‐performance include the QUERI Implementation and Quality Improvement Strategy Learning Hubs ²³ that are open to investigators and practitioners and provide training in practical methods to scale up and spread effective treatments in real‐world settings. In addition, HSR&D and the Society for General Medicine (SGIM) collaborated to develop a Partnered Research Fellowship to give VA investigators mentored training to collaborate with clinical leaders in developing their research areas. ²⁴ The Innovation Ecosystem's Diffusion Academy ²⁵ provides training to VA employees who have been selected by clinical operations leaders to further implement a program or innovation across different settings. These initiatives often involve training in the design or testing of different implementation or quality improvement strategies to enhance the uptake of effective treatments or programs. ²⁶

Innovative performance‐to‐data training opportunities have been spearheaded through the IE's entrepreneurship training programs through the OAA innovation fellowship programs. In addition, the QUERI Advancing Diversity in Implementation Leadership (ADIL) program ²⁷ provides mentored support for VA investigators or staff interested in leading implementation, evaluation, or quality improvement efforts in an LHS. Unique features of ADIL include its focus on real‐world implementation and evaluation methods, and it is open to employees with a health care background regardless of whether they have a terminal degree, widening the tent to engage a more diverse and experienced talent pool. ²⁸ The HSR&D Researcher or Evaluator in Residence (REiR) program ²⁹ provides research funding to embed investigators and their research teams to work on health care problems identified by a national clinical program office. LHS methods such as natural language processing, systems science, or community engagement are often used in these studies.

VA LHS training opportunities also rely on strong learning communities to support trainees, namely through the HSR&D Centers of Innovation (COINs), Consortia of Research (COREs), and QUERI centers. The HSR&D COINs are a national network of health services research centers designed to build capacity for advanced methods akin to LHS that address the organization, financing, delivery, and quality of health care. HSR&D COREs foster a research agenda including pilot funding opportunities focused on a national research priority (eg, access to care, suicide prevention, and opioid/pain treatment). The national network of QUERI Programs, ³⁰ which are multisite centers that deploy implementation strategies to scale up and sustain evidence‐based practices to achieve a clinical impact goal, also sponsors trainees through their Mentoring Cores to lead real‐world implementation, evaluation, or quality improvement projects. Trainees, including ADIL or HSR&D career development awardees, may also work on QUERI Partnered Implementation Initiatives, which are national quality improvement initiatives co‐led by investigators and VA regional health system leaders to actively implement evidence‐based practices that address quality gaps. The QUERI Partnered Evaluation Initiatives are also learning communities primarily funded by VA clinical operations to conduct national evaluations of programs or policies. The Innovation Ecosystem's Diffusion of Excellence initiatives (eg, Diffusion Academy, Diffusion Marketplace, Innovators Network) also support mentoring and training for VA employees from different job echelons who developed new program or technologies for potential national rollout. ²⁵

3. VA LHS TRAINING ECOSYSTEM INITIATIVES AND CORE COMPETENCIES

We highlight how specific VA training programs deliver LHS core competencies, the underlying infrastructures that enable their sustainment, and their application to routine health care settings.

3.1. OAA/HSR&D advanced fellowship in health services research and learning health systems

Since 1991, the OAA/HSR&D Advanced Fellowship Program (AFP) has provided collaborative training and mentoring opportunities for clinicians, researchers, clinician‐investigators, and operational partners. The program provides 2 y of research, education, and clinical learning opportunities to eligible post‐doctoral nurses, associated health professionals, and post‐residency physicians. Fellows spend approximately 80% of their time in research and education and 20% in clinical care (or healthcare improvement activities for non‐clinicians) at competitively selected VA sites.

The HSR&D AFP engages the expertise, mentoring, and educational infrastructure primarily at HSR&D Centers of Innovation (COINs) to provide advanced interprofessional research training opportunities for fellows. Other OAA fellowship programs affiliated with COINs that teach key LHS core competencies relevant to quality and patient safety include the Quality Scholars and National Center for Patient Safety programs. ⁵

In 2020, OAA and HSR&D agreed to formally build LHS core competencies into a nationally disseminated curriculum. This was motivated by a desire to further integrate the AFP sites and by the conclusions from a 2019 co‐sponsored VA‐Kaiser Permanente State of the Art Conference on Embedded Research Opportunities that emphasized the need for more problem‐focused research in health systems. ³¹ Specifically, the updated AFP includes training in LHS core competencies including management, team building, communications, consensus‐building, implementation and quality improvement science, informatics, systems thinking, and engagement skills consistent with the Agency for Healthcare Research and Quality (AHRQ) LHS core competencies through their K12 program. ³² , ³³ To facilitate this transition to a formalized national curriculum, a program review and re‐competition were conducted in 2021 for existing and aspiring AFP sites. As a result of this process, 14 AFPs were awarded to HSR&D COINs at VA facilities in Boston‐Bedford, MA, Charleston, SC, Denver, CO, Durham, NC, Houston, TX, Indianapolis, IN, Iowa City, IA, Los Angeles, CA, Minneapolis, MN, Portland, OR, Providence RI, Salt Lake City, UT, Seattle, WA, and West Haven, CT. As COINs, these sites provide the crucial infrastructure and capacity needed to effectively train fellows to become successful health services researchers. They also offer the common LHS curriculum in combination with localized expertise in relevant topics such as rural health care, health disparities, and medical informatics. Sites are authorized to recruit two trainees per year for the 2‐year program, creating a total national cohort of approximately 56 trainees annually.

The HSR&D Advanced Fellowship Coordinating Center (AFCC), led by AcademyHealth, focuses on the development and implementation of a model curriculum for the LHS core competencies and assessment of measurable learning objectives for each LHS domain for fellows, training sites, and the overall LHS Fellowship program. The Curriculum is organized around the seven AHRQ LHS Core Competencies Domains, ³² , ³³ and the CC added equity as an eighth in 2022 (Table 2). In addition, in consultation with the Fellowship site faculty, the CC will organize supplemental skill‐based training such as orientations to VA resources, military culture, and Veteran engagement, as well as professional communications, data visualization, user‐centered design, and other practical topics.

TABLE 2.

Learning health system (LHS) core competencies in the office of academic affiliations‐health services research and development advanced fellowship program

LHS domain	Description
Systems science	How complex health systems operate and using systems thinking in research and practice; Understanding how organization, delivery, outcomes, and payment processes inter‐relate
Research questions and standards of scientific evidence	Asking research questions that seek to solve a real‐world practice issue in a timely way, especially those identified by end‐users and practitioners
Research methods	Study designs and analytic methods that take into account complex health systems that assess outcomes of interest to end‐users (ie, veterans and families) and practitioners
Informatics	Practical use of information systems and communication technologies to improve quality and outcomes in health care, public and community health, and health systems research and analytics
Ethics of research and implementation in health systems	Ensure that studies in care settings adhere to the highest ethical standards, avoiding conflicts of interest and emphasizing informed consent, transparency, privacy, and security of personal health information
Improvement and implementation science	Reduce avoidable variations in clinical services by using evidence‐based interventions and improvement strategies; ensure the systematic uptake of effective innovations and discoveries in a complex health system
Engagement, leadership, and research management	Engage end‐users, practitioners, and other interested parties in all aspects of the research and QI process and effectively build and lead multi‐sector and interprofessional project teams
Health equity	Incorporating the lived perspectives of end‐users, practitioners, and leaders to inform strategies that promote equality and inclusion in the research workforce, supporting a diverse project team, and working to reduce disparities in health outcomes of clinical and community interventions

The goals of the AFCC include integrating sites and building a collaborative learning community around the common national curriculum with increased communication and resource‐sharing across sites; enhancing and systematizing the development of embedded researchers, building on the existing expertise among fellowship site faculty and their academic and community partners; and the evaluation of learners' overall program success and diversity in recruitment.

3.2. ORD national artificial intelligence institute (NAII)

As the largest integrated health care system in the country, VA has established several big data repositories, including the largest genomic knowledge base in the world linked to health care information. As a joint initiative by ORD and the Office of the Secretary's Center for Strategic Partnerships, the NAII collaborates and provides training on large‐scale artificial intelligence (AI) research initiatives focused on advancing AI methods for real‐world impact and outcomes to ensure Veteran health and well‐being. ²²

The NAII is also helping VA build AI research capacity from basic to clinical research, notably through the creation of an AI Tech Sprint handbook that will allow other teams and organizations to orchestrate a sprint to introduce innovative ideas and solutions. The Big Data Scientist Training Enhancement (BD‐STEP) Advanced Fellowship Program is another affiliated 2‐y fellowship utilizing data science to advance healthcare research and patient care that focuses on recruiting big data scientists to the VA. Established in 2015, BD‐STEP was launched in collaboration with the National Cancer Institute (NCI) and connects early‐career data scientists with VA researchers and clinicians, as well as NCI cancer experts, to apply VA's big data resources to translate findings to improve patient care. Notably, BD‐STEP has embarked on a transition to include enhanced competency development in supporting VA operational data analytic needs, in addition to the traditional focus on health care research. Thus, fellowship graduates will be increasingly attuned to VA clinical and operational needs and able to provide data analytic support for systemic transformation.

3.3. Quality enhancement research initiative (QUERI)

Established in 1998, the mission of QUERI is to improve Veterans' health by accelerating the implementation of research findings into real‐world practice. ¹⁷ , ²⁰ QUERI funds VA investigators to conduct national‐level implementation initiatives that are designed to evaluate and optimize quality improvement to inform practice and policy for national health care priorities. To build capacity for more rapid and effective implementation, QUERI offers several training opportunities that are available online or through its national network of centers and programs.,

Central to the implementation of QUERI training resources and opportunities is the Center for Evaluation and Implementation Resources (CEIR). CEIR is VA's central resource for VA for training, consultation, and mentoring in implementation and evaluation methods, serving as the connector for VA employees and researchers to access online resources and other training opportunities offered by QUERI and other VA programs. For example, CEIR connects learners to the QUERI Learning Hubs (Table 3) located across the United States, which provide a unique opportunity for leaders, providers, and researchers across VA to gain practical experience and skills needed to lead care improvement. They also provide training for researchers to learn how to deploy rigorously designed implementation strategies. There are currently eight QUERI Learning Hubs, and each Hub follows the general framework of the QUERI Roadmap but also teaches strong implementation strategies for different settings. CEIR also commissioned several self‐directed resources including the QUERI Roadmap for Implementation and Quality Improvement and ¹⁸ the Implementation Training Resource Matrix. ³⁴

TABLE 3.

Quality enhancement research initiative (QUERI) implementation and quality improvement strategy learning hubs

Learning hub	Description of learning opportunities
Adaptation	Tailor implementation efforts using iterative stakeholder engagement, workflow mapping, and adaptive problem solving (ie, people, process, and problems)
Design for dissemination & implementation (D4D&I)	Core components of the design‐for‐dissemination and implementation strategy (pre‐implementation assessment, stakeholder engagement, intervention adaptation, and implementation evaluation) to increase the adoption, implementation, and sustained use of evidence‐based interventions with an emphasis on care coordination programs
Evidence‐based quality improvement (EBQI)	Support individuals and teams of leaders, providers, and staff in identifying, addressing, and solving problems using an evidence‐based, multi‐level, end‐user‐driven approach to quality improvement
Implementation facilitation (IF)	Participatory approach comprised of preparatory work that includes instruction on key implementation facilitation roles, interactive role‐plays, and group exercises to train participants in applying an evidence‐based strategy involving interactive problem‐solving
Leading healthcare improvement (LHI)	Employs interactive learning activities to train frontline providers and leaders in leadership principles and improvement strategies
Leadership and Organizational Change for Implementation (LOCI)	Evidence‐based combination of workshops and coaching that strengthens leadership skills and fosters individual and organizational capacity to implement effective practices for mental health
Learn. Engage. Act. Process. (LEAP)	Hands‐on virtual learning in quality improvement methods to clinical teams, packaging multiple implementation strategies into a single, structured program
Teamwork learning hub	Fundamentals of LOCK (Learn from the bright spots, Observe, Collaborate in huddles, and Keep it bite‐size) and its implementation (including rapid cycle quality improvement)

CEIR also trains Innovation Ecosystem fellows in implementation, quality improvement, and evaluation methods through the Diffusion Academy. CEIR also serves as the point of contact for the VA‐sponsored positions in the AcademyHealth Delivery System Science Fellowship program, supporting the placement of early‐career investigators into embedded VA training positions with the explicit goal of learning skills related to applied, translational health service research. Placements involve direct collaborations with VA researchers conducting research studies funded by HSRD/QUERI and directed toward health care improvement.

3.4. Innovation ecosystem LHS training opportunities

While the OAA Advanced Fellowship program focuses on training health services researchers, VA also realized that there was an unmet need to train frontline providers to be leaders in health care quality improvement and practice change consistent with an LHS. There was also increased awareness that the most creative and innovative ideas for improving health care can originate from front‐line, real‐world practice settings. IE's Diffusion of Excellence and Innovator's Network initiatives ²⁵ , ³⁵ enable frontline practitioners and employees to submit and develop innovations for potential national spread. The Diffusion of Excellence fellows are selected via a “Shark Tank” competition format, and then receive quality improvement and implementation training through CEIR at the Diffusion Academies.

In 2020, to meet growing employee demands for innovation and implementation training, IE established the Entrepreneur in Residence and Senior Innovation Fellowship programs ³⁶ to build the innovative leaders of tomorrow and to spread mission‐driven advances in healthcare delivery across VA. These programs provides unique experiential learning opportunities for emerging and accomplished leaders from real‐world VA practice settings who were provided strategic guidance and relationship development to become leaders capable of implementing effective innovations to improve healthcare for Veterans.

The Entrepreneur in Residence Fellowship is for emerging VA employee leaders who are prepared to implement an innovative project that leverages employees, consumers, and leaders at all levels at their local institutions and beyond. The Senior Innovation Fellowship seeks to engage accomplished leaders prepared to implement a national scale project that leverages internal and external interested parties throughout the organization, government, academia, and industry. Both programs provide fellows with experiential learning in core competencies including communication strategies, storytelling, building and expanding professional networks, developing a business case, and organizational acumen (systems thinking) that enables them to develop and scale their innovative initiatives and practices.

4. DISCUSSION

Training in LHS core competencies is essential to not only improve health but to retain and empower a national health care workforce that provides optimal care to patients. The key distinction between LHS‐oriented training and traditional research training is an emphasis on using real‐world data, working in close partnership with health system clinical partners, implementing results into practice, and using a variety of research and quality improvement designs to generate timely, relevant, and actionable information. LHS core competency goals are also aligned with the Quintuple Aim goals (eg, improving population health, healthcare consumer and workforce experiences, reducing costs and improving health equity) ³⁷ that ultimately maximize and sustain improved health for Veterans and other populations. LHS training is also needed to meet the growing demand for research in groundbreaking areas such as health informatics, implementation science, socio‐technical infrastructures, community‐engaged research, and complexity science. Recent NIH funding initiatives such as AIM‐AHEAD emphasize these LHS‐focused emerging methods to recruit a more diverse research talent pool. ³⁸ Growing evidence ³⁹ suggests that a key reason for funding disadvantages among Black scientists is that the topics and settings ⁴⁰ that comprise their applications are more likely to involve questions related to real‐world and community‐based population health problems, rather than focused on singular mechanism of action (such as cellular processes).

In the US, LHS‐focused training programs similar to VA's include the AHRQ K12 training program, which is also based on the LHS core competencies. ³² , ³³ The National Cancer Institute's Transdisciplinary Research Centers of Excellence, notably the Implementation Science Centers in Cancer Control (ISC3) program ⁴¹ also funds implementation‐focused LHS research to promote uptake of cancer treatment and prevention. Akin to HSR&D COIN and QUERI centers, the ISC3 uses implementation laboratories to deploy treatments across different settings while also developing, testing, and measuring different implementation research methods and strategies for generalizable use.

Several features of the VA LHS training ecosystem can potentially be adopted elsewhere. For example, similar innovation fellowships are being replicated across medical schools such as the Dell Medical School ⁴² and organizations such as the Institute for Healthcare Improvement ⁴³ provide quality improvement learning opportunities. Others have described optimizing LHS‐focused clinical education in large health systems. ⁴⁴ , ⁴⁵ , ⁴⁶ , ⁴⁷ However, to date, few entities have described how LHS training can better align research and clinical care priorities to inform both quality improvement and research impacts over time.

There are several strengths in the VA LHS training ecosystem. Notably, it uses learning communities with strong clinical and research infrastructures that are actively involved in innovation and quality improvement. Learning communities are further supported through the VA's common mission of supporting Veterans. Second, some of the programs (eg, IE, QUERI ADIL) are open to employees without a terminal degree, enabling greater diversity and growth of innovations from the field by reducing academic credential or economic barriers to participation. Third, it focuses on real‐world learning opportunities through active engagement across multiple audiences (eg, IE) and the use of pragmatic implementation and evaluation tools (eg, QUERI Learning Hubs, HSR&D REiRs).

There are also challenges that inform opportunities and lessons learned from the implementation of the VA LHS training ecosystem. These challenges are currently being addressed in the updated VA ORD research strategic priorities ¹⁹ and may also be salient to other health systems interested in initiating similar programs. Notably, they include ongoing recruitment and retention of implementation, informatics, and systems scientists, especially from under‐represented groups, ongoing data infrastructure and maintenance to support both research and quality improvement initiatives and providing sustainable research funding opportunities for investigators that enhance their career trajectory focused on LHS research that addresses system‐wide problems facing healthcare providers.

In general, recruitment and retention of LHS investigators, especially those from under‐represented groups, is an ongoing challenge. Recruitment is especially needed in LHS‐focused areas such as implementation science, informatics, and systems science. In some situations, the lag in the federal hiring process can lead to lost recruitment opportunities. Moreover, federal salaries for investigator‐track careers are not as competitive as in the private sector or universities. To address the salary gaps, ORD is in the process of reforming the hiring process to enable LHS‐focused specialists in informatics and data scientists the opportunity to complete for higher government (GS) pay scales. However, more effort is needed to support competitive GS pay scales for implementation and quality improvement scientists as well. Federal employment through VA can also provide opportunities for investigators to pursue higher‐level leadership or management positions, such as in VA national program offices that provide more stable funding sources to pursue LHS work.

To promote the retention of LHS investigators, funding opportunities must also be aligned with LHS core competencies. Figure 2 provides a roadmap with examples of ongoing VA LHS‐focused funding opportunities that enable trainees to advance in their careers by addressing real‐world clinical problems in VA. Funding opportunities under capacity‐building, for example, are available through HSR&D, QUERI, and IE, and include independent or collaborative projects with additional mentoring by more established investigators and clinical leadership. Opportunities under Research and Evaluation Mechanisms highlight VA funding opportunities to support the transition from early to mid‐career LHS investigators through independent research or partnered implementation or evaluation initiatives with clinical operations leaders. Finally, under Scale up and Sustain, examples include multi‐site, team science opportunities that focus on capacity‐building as well as protected time through the Research Career Scientist award.

FIGURE 2.

Examples of funding opportunities for LHS learners post‐fellowship. ADIL, advancing diversity in implementation leadership; EHR, electronic health record; IE, innovation ecosystem; HSR&D, health services research and development; LHS, Learning Health System; OAA, office of academic affiliations; ORD, office of research and development; QUERI, quality enhancement research initiative; RIVRs, research to impact for veterans initiatives; SGIM, society for general internal medicine

Second, VA research and clinical leaders need to enhance the curation and sustainment of national clinical data infrastructures suitable for both research and quality improvement. VA investigators have access to national health care data through the VA's corporate data warehouse, which aggregates data from the electronic health record and other resources (Table 1). Still, these data often lack comprehensive clinical information in near‐real time that are available from clinical operations, thereby limiting opportunities to conduct pragmatic clinical trials or generating real‐world evidence. Regulatory barriers also complicate the use of research‐derived data sets that have been cleaned and augmented with clinically derived data. ORD informatics initiatives such as the Centralized Interactive Phenomics Resource (CIPHER) allow researchers to share code and algorithms they use to define specific clinical phenotypes (eg, metastatic prostate cancer) or outcomes (eg, hospitalization due to post‐surgical complications) using electronic medical record data. ⁶ , ⁴⁷ Another barrier includes information technology firewalls, which limits the availability of software for surveys and qualitative data that are essential to obtaining more comprehensive data on the lived experiences of patients, providers, caregivers, and other individuals. Currently, ORD has worked with the VA Information Technology office to enhance access to qualitative software on a national level, especially tools that can enhance efficiencies such as automated transcription services.

On a national scale, ORD is also developing new Actively Managed Portfolios (AMPs) that are more focused on solving specific problems in partnership with clinical operations leaders and managing the research process toward pre‐specified goals. The goal of AMPs is to align the LHS infrastructure, including data curation, and the learning community, to promote areas of research that can also enhance outcomes for Veterans. They also build upon similar initiatives such as the University of Michigan‐Blue Cross Blue Shield Clinical Quality Collaboratives ⁴⁸ , ⁴⁹ and QUERI‐partnered initiatives ⁵⁰ by also building the LHS infrastructure and learning communities to sustain ongoing discovery, testing, and improvement in the research area over time. One example of an ORD AMP involves developing data and partnerships to increase Veterans' access to validated precision oncology‐focused treatments. AMPs will require a standardized governance and process for distinguishing between work that is considered research by an institutional review board vs activities that can fall under non‐research or quality improvement protocols, and will streamline this determination process to facilitate a more rapid translation of inquiry into practice.

The third major challenge for LHS programs is ensuring that investigators can maintain successful careers as embedded researchers in health systems by providing LHS‐focused funding opportunities. ³¹ Currently, many of the goals in an LHS are not aligned with the traditional benchmarks of success in academic health systems, where publication volume and grant funding are valued over impacts on health care quality or policy. Even with many opportunities for embedded research, such as in VA, the development of the learning community and curation of data are challenging without an underlying investment in the infrastructure and partnerships, as well as ongoing trainee mentoring and support. Yet these activities, while informed by scientific methods and frameworks derived from organization theory, community‐based participatory research, and implementation science, for example, do not have clear funding sources. As a result, promising real‐world research ideas may fail to spread beyond their initial descriptive studies. Investigators also need the security and confidence that clinical operations partnerships required for the research will yield fruitful scientific products necessary for promotion and tenure, and some might be less willing to take on risky or complex systematic questions of most interest to clinical operations leadership without funding sources that enable the building of scientifically informed LHS learning communities and infrastructures.

To better incentivize investigators to work on real‐world, impactful clinical initiatives, QUERI's Partnered Implementation Initiative provides phased support for work with a clinical operations partner to develop, deploy, and evaluate implementation strategies to scale up effective programs or practices that are benchmarked using national performance measures (eg, CMS Hospital Compare). The Innovation Ecosystem's Diffusion of Excellence Academy also trains practitioners selected by VA healthcare leaders to replicate and spread innovations across multiple sites.

In the future there, will be emerging challenges at the national level that LHS training programs in VA and elsewhere will need to address. First, with the rapid growth of Veterans receiving care outside the traditional VA clinical settings, there will be more demand for community‐engaged research methods and knowledge and competency in using data from health information exchanges and other sources. Second, VA is implementing a new electronic health record system that will enable more opportunities to incorporate artificial intelligence, machine learning, and quality improvement monitoring tools, especially in mixed‐methods research. This will require a workforce with more sophisticated computational knowledge and programming skills. Third, the development of multiple learning communities and interest groups will necessitate prioritization of research and quality improvement topics and ensure active engagement across interested parties. For example, the VA has recently adopted a process developed by QUERI ⁵⁰ based on the VA Strategic Plan ⁵¹ to prioritize scientific funding using multi‐level input from consumers, providers, and leaders of VA health care.

Overall, the VA LHS training ecosystem is poised to inform how health systems can engage researchers and practitioners in partnering to improve care delivery and discover novel treatments and innovations that can have a substantial public health impact. Critical ingredients include having learners embedded in the health system with a shared agenda among clinical operational leaders so that the learning is largely experiential and mission‐driven. This is primarily accomplished with a solid infrastructure foundation that provides phased and stable sources of funding to learners as they advance in their research or quality improvement careers, supportive technologies including advanced informatics, user‐friendly governance that adheres to high ethical standards, and reliable processes that enhance research and quality improvement methods that achieve health impacts. The VA LHS training ecosystem ultimately informs a pragmatic and sustainable roadmap towards improving healthcare research utilization and can serve as a model for similar efforts within other organizations.

AUTHOR CONTRIBUTIONS

Amy M. Kilbourne drafted the manuscript and provided content on key training initiatives; Joel Schmidt and David Atkins edited the manuscript and provided content on advanced fellowship and related research and operations programs; Margo Edmunds provided content related to the LHS core competencies and made edits to the manuscript; Ryan Vega edited and provided content related to the innovation‐focused LHS content; and Nicholas Bowersox made edits and wrote core components of the training opportunities. All authors reviewed and approved the manuscript.

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

Kilbourne AM, Schmidt J, Edmunds M, Vega R, Bowersox N, Atkins D. How the VA is training the Next‐Generation workforce for learning health systems. Learn Health Sys. 2022;6(4):e10333. doi: 10.1002/lrh2.10333