Training the Workforce to Conduct Embedded Pragmatic Clinical Trials to Improve Care for People Living with Dementia and Their Caregivers

Abstract

The National Institute on Aging IMbedded Pragmatic Alzheimer’s Disease and Alzheimer’s Disease-Related Dementias Clinical Trials (IMPACT) Collaboratory serves as a national resource for the conduct of embedded pragmatic clinical trials to improve the care of people living with dementia (PLWD) in partnership with the healthcare systems that serve them. Inherent in this objective is the need to train and support a cadre of investigators prepared to conduct this work now and in the future. The Training Core of the IMPACT Collaboratory supports the training of investigators to become experts in this field through three objectives: (1) curricula development and dissemination; (2) network generation and navigation; and (3) a career development award program. The innovative approach of the Training Core will require developing content and providing training experiences that recognize the unique challenges of research at the intersection of health systems, pragmatic trials, and PLWD and their caregivers. Ultimately, we seek to build the nation’s capacity to conduct research that bridges the gaps between efficacy studies to effectiveness research to implementation science. Although foundational resources in the methods of each of these areas are already available, few actually focus on pragmatic trials embedded within healthcare systems that focus on PLWD. To bring new interventions for PLWD from efficacy to widespread implementation, researchers must build diffusability, adaptability, heterogeneity, and scalability into the design of the intervention. In achieving these objectives, the Training Core will utilize the network of investigators, institutions, and stakeholders represented in the IMPACT Collaboratory.

INTRODUCTION

As consumers and customers, we expect continual technological innovation as well as reduced costs in the products and services that we purchase. Innovative products and services often seem to appear suddenly, the result of a disruptive idea from a single individual. In reality, most of these innovations stem from research, trial and error, and incremental improvements by many individuals and large networked teams over a period of multiple decades.¹ In the field of health care, patients and their families learn about early ideas for a cure or promising model of care decades before its widespread availability. Patients and families struggling with progressive illness, such as Alzheimer’s disease (AD) and AD-related disorders (ADRDs), are especially frustrated by these repeated cycles of unfulfilled hope.² Although some of this frustration stems from ideas that simply do not work, some results from an unacceptable delay in getting ideas that do work to people who could benefit.³ Recent meta-analysis, meta-reviews, and consensus workshops confirm that new models of care demonstrate efficacy in improving outcomes for people living with dementia (PLWD) and their caregivers.^4–6 Although over 5 million Americans are living with AD/ADRD, none of these evidence-based models has been widely integrated into routine clinical practice.^7,8 Overcoming the barriers and delays in making efficacious treatments available is a primary focus of research that focuses on effectiveness, implementation, and dissemination.⁹

Characteristics of care models, in addition to their efficacy, contribute to the likelihood that innovation reaches PLWD. Rogers’ “diffusion of innovations” provides a useful framework for understanding why some innovations diffuse readily, whereas others do not.¹⁰ This framework proposes six features of an innovation that facilitate diffusion: relative advantage, compatibility, complexity, “trial-ability,” adaptability, and observed effectiveness. Rogers also stressed that end users (e.g., PLWD, caregivers, and healthcare systems) judge the value of these features, rather than the inventors.¹⁰ Several groups have adapted this framework to improve diffusion of innovation in complex healthcare systems.^11,12 In early work, investigators seeking to improve diffusion in medical care find that “diffusability” must be built into the design and testing of a new care model.^3,11,13 To build in diffusability, investigators must better understand the needs of the heterogeneous group of end users and the practical realities of an ever-changing complex healthcare system.¹⁴ Embedded trials evaluate the effectiveness of an intervention in a real-world setting. Investigators begin with identifying questions and outcomes important to decision makers, designing the trial with stakeholder input to integrate the intervention into the flow of routine care, and enrolling representative subjects under real-world conditions. Embedded pragmatic clinical trials (ePCTs) seek to build in diffusability. Unfortunately, there are few investigators with the knowledge and experience to design and conduct such trials.

The National Institute on Aging IMbedded Pragmatic Alzheimer’s Disease and Alzheimer’s Disease–Related Dementias Clinical Trials (IMPACT) Collaboratory serves as a national resource for the conduct of ePCTs that improve the care of PLWD in partnership with the healthcare systems that serve them. Inherent in this objective is the need to train and support a cadre of investigators prepared to conduct this work now and in the future. The Training Core of the IMPACT Collaboratory supports the training of investigators to become experts in this field.

OBJECTIVES OF THE IMPACT COLLABORTORY TRAINING CORE

The Training Core of the IMPACT Collaboratory focuses on three objectives:

1. Curricula Development and Dissemination. The Training Core will develop and implement structured training activities that address all aspects of conducting ePCTs among PLWD, with particular attention to building and delivering curricula suitable for a variety of trainees in a variety of educational settings. This includes “internal” trainees, such as those funded by the IMPACT Collaboratory career development and pilot award programs, as well as “external” trainees from the broader research community who wish to learn skills in this field.

2. Network Generation. The Training Core will facilitate a national network to help both internal and external trainees access the combined talents, expertise, mentors, and clinical resources, and learn the perspectives of investigators, stakeholders, PLWD, and their advocates within the IMPACT Collaboratory Cores and Teams. The goal of this network is to promote professional expertise and individual productivity specifically in the field of ePCTs among PLWD. This network recognizes the distributed expertise in this area of study and the reality that few academic programs or healthcare systems encompass all of the expertise needed to design and conduct ePCTs among PLWD.

3. Career Development Award (CDA) Program. The Training Core will develop, fund, and coordinate a 2-year junior CDA program for doctor of medicine (MD) and doctor of philosophy (PhD) early career trainees who seek to build their career conducting ePCTs among PLWD.

UNIQUE CHALLENGES CONDUCTING ePCTs

ePCTs seek to address barriers to diffusion by focusing especially on: (1) addressing the priorities for research as identified by end users and stakeholders; (2) engaging end users and stakeholders in the design of the research; (3) enrolling representative populations and conducting trials under real-world conditions; and (4) measuring outcomes important to end users and stakeholders.^15–17 Each of these four features requires special research training and infrastructure not typical of current resources available for students of traditional randomized clinical trials. In addition, ePCTs accept adaptability as a design feature because the healthcare system and the study subject change. Studies must adapt to this change to avoid irrelevance.

Features of ePCTs, compared with traditional randomized trials, exist along a continuum; and investigators make judgments in deciding how to balance these features in any given study. The PRagmatic-Explanatory Continuous Indicator Summary (PRECIS) framework describes ePCTs along nine design features, including: eligibility, recruitment, setting, organization, flexibility in delivering the intervention, fidelity to the intervention, success in subject follow-up, selection of the primary outcome, and integrity of the primary analysis.^18,19 Because ePCTs seek both credible results and generalizability, decisions around these nine design features have a major impact on the relevance of the trial results. Balancing these features requires training typically absent from the curricula of programs seeking to train researchers in the rigorous and reified designs of explanatory research.

“Embeddedness” also exists along a continuum and, like other features, levels of embeddedness by necessity result in trade-off in methodological rigor. At one end of this continuum, the existing literature includes studies led by nonclinicians and conducted completely outside the realm of clinical care yet seeking to eventually influence clinical care. Such a design allows for strict experimental control with the attendant excellent internal validity, but with dubious external validity and often major challenges to diffusability. At the opposite end are those studies conducted by clinicians and staff who are deeply embedded in a single complex healthcare system yet seeking to eventually influence clinical care in other healthcare systems. Such a design offers excellent clinical relevance and local generalizability but often results in dubious internal validity and may or may not offer external validity. ePCTs seek to move further along the continuum to clinical relevance and adoptability by addressing the exigencies of real-world clinical care and the needs of end users and other stakeholders in the earliest stages of trial design.

Embeddedness is a design aspiration driven largely by interpersonal relationships. This feature of pragmatic trials thereby offers unique training challenges not inherent in training on trial design or analysis. Because embeddedness requires building and sustaining relationships, which, in turn, requires time and space within the targeted healthcare system, learning its language and culture. Coaching scientists to develop these relationships will be a key mission of the training core. Investigators must also account for that fact that given the time investment, one investigator is likely to achieve embeddedness in a limited number of places. Scientists already in an embedded relationship face an underdeveloped national infrastructure of other embedded scientists at other health systems that would be needed to conduct multisite studies within the real-world settings of diverse healthcare systems. The Training Core will contribute to the development of this national network of embedded scientists.

UNIQUE CHALLENGES OF ePCTs AMONG PLWD

Conducting ePCTs among PLWD and their caregivers adds further complexity to the design and conduct of ePCTs because of the special personal, dyadic, cultural, and clinical considerations of AD/ADRD and the changing nature of health and social needs as the disease naturally progresses. In the real world of care for PLWD, clinical and social heterogeneity is the rule.²⁰ Heterogeneity exists among the PLWD, of course, but also among the one or multiple caregivers. Some PLWD do not need caregivers, and some who need caregivers do not have one even late into the course of the illness. Thus, all interventions seeking to improve the quality of life for PLWD and their caregivers must be tailored at the outset, and they must be tailorable over the changing course of the disease.²¹ PLWD and their advocates demand and deserve input into the conduct of such studies, but there is also an enormous opportunity cost at the level of individual research projects in building and maintaining these relationships. The IMPACT Collaboratory seeks to build, along with other advocates, a national resource for gaining stakeholder input and a national resource to develop training guides so that local programs can develop their own local resources for input from PLWD. Trainees seeking to conduct ePCTs among PLWD also require an understanding of the unique challenges posed in settings, such as nursing homes, homes, and assisted living. The frontline providers in these settings and the priorities of the care setting as well as the variability in resources across settings present new challenges to the durability of the intervention and study design. Interventions, for example, may target the PLWD, the frontline caregiver, and/or the care system itself, and each of these targets may present different outcome measures. On a pragmatic level, different structures of accountability are needed to monitor research staff productivity, research protocol fidelity, and research subject outcomes in settings remote from the home research office.

Training in research ethics for ePCTs must also consider the distinctive ethical features of these studies. For example, to enhance feasibility in the clinical setting, the requirement for informed consent may often be waived. However, input from key stakeholders, including health systems clinicians, patients, and caregivers, remains important. Engaging stakeholders with respect for their interests and autonomy is a key concern of ePCTs. Because outcome assessment must also be pragmatic, outcomes may be obtained from existing administrative or clinical data rather than patient or family report, raising questions about the importance of these outcomes to the PLWD. In addition to these general concerns, PLWD are a vulnerable population due to their impaired capacity to consent for research, which requires researchers to consider additional protections as well as surrogate consent, if appropriate.^22,23

Examples of special considerations in conducting ePCTs among PLWD and their caregivers are shown in Table 1 and grouped by typical features of the PRECIS-2 framework.¹⁹ This is not meant to be an exhaustive list but rather to demonstrate some design challenges unique to research among ePCTs among PLWD and to describe some training objectives. These objectives can be met through any of the training core objectives (e.g., by a CDA recipient learning skills through a mentored research project or by an experienced investigator making use of the IMPACT network for consultation on study design). Table 2 provides a broad outline of a draft curriculum structured around the cores of the IMPACT Collaboratory. As a first step in curriculum development, we will create an introductory, online series of videos that provide a broad introduction to the major topics in the design and implementation of ePCTs in PLWD and that serve as a basis for training CDA recipients and other investigators designing these studies. In addition to the topic areas described in these tables, we also envision using case studies of successful and unsuccessful trials as an important vehicle for learning. Case studies will allow learners to see how research teams addressed barriers and opportunities in the field. To reach a wide audience, we plan to provide this curriculum as both a series of webinars for distance and asynchronous instruction and face-to-face meetings conducted in a variety of settings, such as seminars, national meetings, and consultations.

Table 1.

Design Considerations in Conducting Pragmatic Trials Among PLWD and Their Caregivers

Design elementa	Examples of design challenges	Examples of training objectives
Eligibility	• Assessment of cognition impairment • Prevalence of comorbidity • Role and availability of caregivers	∘ Innovate pragmatic assessment tools based on study goals and setting ∘ Utilize existing data sources to assess eligibility ∘ Strategize to minimize exclusion criteria ∘ Develop approaches to including PLWD who do not have a caregiver
Recruitment	• Small numbers of eligible subjects in single clinical sites • Large percentage of undiagnosed PLWD • Developing processes for consent or assent from PLWD	∘ Organize multisite studies ∘ Communicate the resources available within the IMPACT Collaboratory ∘ Learn how to identify undiagnosed subjects who are potentially PLWD using existing data ∘ Understand how to respect the voice and goals of the PLWD in research participation
Setting	• Variety of settings where PLWD live and receive care	∘ Review barriers and opportunities for research in sites, ranging from homes, assisted living, nursing homes, and hospitals, among others ∘ Strategize for tracking PLWD across sites ∘ Develop relationships and skill to conduct intersectoral study designs
Organization	• PLWD receive care across multiple HCS and community-based organizations	∘ Learn to design interventions based on the typical resources of the study site ∘ Consider opportunities for partnering in research with community-based organizations
Flexibility	• Heterogeneity of PLWD, caregivers, and sites of care	∘ Understand how to design interventions to allow tailoring and to anticipate heterogeneity among study subjects and research sites ∘ Demonstrate how to capture the process of care to understand dose and duration of intervention
Fidelity	• Care is often delivered by a range of providers • Standard of care changes over time • External environment changes over time	∘ Use design and measurement attributes that bring durability to study processes and facilitate dissemination and implementation
Follow-up	• PLWD will decline in health over time and move between settings and payers • Caregivers change over time and experience changes in health	∘ Plan for attrition due to multiple factors, including mortality ∘ Increase the frequency of follow-up and allow for varying intervals of data collection and existing data sources
Outcomes	• Outcomes important to PLWD and their caregivers change over time • Outcomes important to other stakeholders may not align with those of PLWD and their caregiver	∘ Build relationships and understanding to engage stakeholders in selecting outcome measures ∘ Incorporate a range of outcome measures ∘ Declare a primary outcome measure
Analysis	• Patient and/or caregiver may not be unit of analysis • Missing data due to loss to follow-up • Lack of randomization	∘ Learn to use alternative design and analysis, such as stepped-wedge designs ∘ Describe options for data imputation ∘ Learn the alternatives for analyses of observational data, including comparator groups

^aUsing PRagmatic-Explanatory Continuous Indicator Summary (PRECIS)-2 framework.¹⁹

Abbreviations: HCS, Health Care System; IMPACT, IMbedded Pragmatic Alzheimer’s Disease and Alzheimer’s Disease–Related Dementias Clinical Trials; PLWD, people living with dementia.

Table 2.

Major Content Areas of New Curriculum in Conducting ePCTs Among PLWD and Their Caregivers

Design and statistics Features of pragmatic trials for PLWD Study designs for ePCTs in PLWD Pilot studies and feasibility testing Statistical analytic considerations in ePCTs Grant writing for ePCTs

Stakeholder engagement Identifying stakeholders in research on PLWD Strategies for eliciting input from stakeholders from pilot projects to dissemination Engaging stakeholders

Health equity Core concepts of health equity Health equity in dementia care Equity considerations in ePCTs among PLWD

Healthcare systems Sites of care for PLWD (outpatient, nursing facility, hospital, and hospice) Structure of healthcare systems Partnering with your health system in ePCTs Payment models and related rewards

Outcomes PCROs Selecting PRCO ePCTs in PLWD Methods and sources for getting PCROs for ePCTs in PLWD

Pilot studies What is a pilot study for ePCTs Assessing readiness for ePCTs among PLWD

Technical data Health system data sources for doing ePCTs in dementia How to use secondary data to conduct your ePCTs How to extract and link data sources Data sharing

Regulation and ethics Informed consent and surrogate consent in ePCTs among PLWD Assessing minimal risk (intervention risk and use of protected health information) Principles of data quality, safety, and security

Dissemination and implementation Implementation science Building dissemination and implementation into study design Measuring adherence and fidelity in ePCTs

Abbreviations: ePCT, embedded pragmatic clinical trial; PCRO, patient and caregiver reported outcome; PLWD, people living with dementia.

EXISTING RESOURCES AND LITERATURE

Although the IMPACT Collaboratory faces new challenges in designing and delivering curricula to address the issues outlined above, educational materials for randomized controlled trials, secondary data analyses, or other experimental or observational designs are not irrelevant. As embraced by the PRECIS-2 framework, features of pragmatic trials exist along a continuum.¹⁹ Thus, we view training in ePCTs among PLWD as building from a trainee’s education in these other research design approaches. Indeed, the Training Core curricula assume competency in clinical trials, health services research, research on the clinical care of PLWD, and research on clinical and community-based support of caregivers. We also do not seek to recreate the existing methodological literature on ePCTs, of which the work of the National Institutes of Health (NIH) Health Care System (HCS) Collaboratory is emblematic.^17,19,24 The NIH HCS Collaboratory’s Living Textbook, for example, includes a rich set of resources on the design, conduct, and dissemination of ePCTs.

When searching for existing literature at the intersection of these three areas (pragmatic trials, health systems embedded trials, and ADRD trials), we find scant existing training materials. This is true due to the infancy of this field and the infancy of the language within NIH search engines, such as PubMed. The relevant PubMed medical subject heading (MeSH) term for pragmatic clinical trials (not introduced until 2014) is “pragmatic clinical trials as topic.” The MeSH is defined for indexers as “Works about randomized clinical trials that compare interventions in clinical settings and which look at a range of effectiveness outcomes and impacts.” Similar entry terms in MeSH include variations on the word pragmatic, including “practical,” “naturalistic,” or “real world.” Trainees might also access relevant materials through a PubMed search for methods closely related to pragmatic trials, such as “cluster analysis.” Our scoping review identified 93 relevant articles in PubMed by searching various combinations of the terms “pragmatic,” “dementia,” and “embedded” as well as through the MeSH term “pragmatic clinical trials as topic.” Of these identified studies, 49 were reports of original research studies, 37 were protocols for original research studies, and only 7 focused on methods relevant to ePCTs among PLWD.

Within this scoping review, and within the experience of the IMPACT Collaboratory investigators, we also find examples of exemplar studies. Such studies are important building blocks toward our goal to build curricula suitable for a variety of trainees and settings. We will build from these existing educational resources, with a particular focus in moving the field forward in ePCTs among PLWD. Figure 1 illustrates the intersection of these key areas and highlights the narrower focus of the IMPACT Collaboratory at the intersection of these areas.

Figure 1.

The IMbedded Pragmatic Alzheimer’s Disease and Alzheimer’s Disease–Related Dementias Clinical Trials (IMPACT) Collaboratory Training Core focuses training on the design, methods, and analysis at the intersection of pragmatic trials, embedded healthcare system trials, and trials seeking to improve outcomes among persons living with dementia. AD, Alzheimer’s disease; ADRD, AD-related disorder.

INNOVATION

The IMPACT Collaboratory is the first initiative to specifically network a national team of investigators and stakeholders to address training in ePCTs among PLWD. Building from the lessons learned from the NIH HCS Collaboratory, this project is also unique in establishing a specific core and focus on training. The training function is embodied in the curriculum development and dissemination but also along three other innovative features: (1) a formal network of researchers organized to provide support for trainees; (2) a nationally competitive CDA program; and (3) a pilot project program led by the Pilot Core. We are designing trainee materials to help investigators understand the characteristics of a rigorous pilot project to support ePCTs among PLWD. We are also designing training materials to define the “readiness” of an intervention for ePCTs among PLWD.²⁵

NEXT STEPS

The Training Core and their colleagues in the other IMPACT Collaboratory Cores and Teams will soon complete the environmental survey and curation of existing materials. These materials, as well as early products and webinars from the IMPACT Collaboratory, are already available on the Collaboratory website.²⁶ At this time, we find several components of an effective curricula that the IMPACT Collaboratory must develop de novo. These include the following areas:

1. Identification and curation of lessons learned from prior and ongoing ePCTs, including examples of real-world failures and the post-hoc failure analysis of these studies;

2. Training in the business acumen, language, and culture needed by scientists to build relationships and codesign relevant interventions with the potential to effect quicker change within healthcare systems;

3. Training in research acumen, language, and culture needed by health system partners to effectively participate in research design and implementation;

4. Training in the relational and operational skills needed to capitalize on emerging and changing opportunities within healthcare systems and to build interventions capable of embracing change; and

5. Training in understanding the unique challenges of conducting ePCTs among PLWD and their family caregivers, including strategies to hear their voices and perspectives in the design of clinical research.

In addition to these next steps in building a novel and adaptable curriculum, we have also released the first Request for Proposals for Career Development Awards. We envision these CDAs as “pre-K” awards, ultimately leading to an NIH K award. These will be mentored awards, and eligibility and application requirements are available on the Collaboratory website. In brief, qualified candidates will have PhD or MD degrees and will have completed postdoctoral training in basic clinical research methods. Candidates must have or be eligible for faculty appointments at academic institutions at the time of funding. Candidates will be required to demonstrate their commitment to a career in conducting ePCTs among PLWD, and they must demonstrate their access to, and engagement with, healthcare system partners.

CONCLUSIONS

The overarching goal of the Training Core is to help build the nation’s capacity to conduct ePCTs among PLWD. The Training Core addresses this goal by building from existing training materials, exploiting the combined expertise and resources of the IMPACT Collaboratory, and developing new training materials. We anticipate the need to develop curricula on basic design and analysis features of ePCTs. We are, however, proposing that those aspects of our challenge are more straightforward and build to a great extent from existing content and approaches to adult learning in research design. In contrast, the larger challenges will come from developing competencies in relationship building, embeddedness, and scalability. These approaches depend greatly on growing the capacity, depth, and interconnectedness or the network represented by the people and institutions represented in the IMPACT Collaboratory.