Pragmatic Trials in Long-Term Care: Implementation and Dissemination Challenges and Opportunities

Abstract

Randomized controlled trials are considered the most rigorous research design in efficacy and effectiveness research; however, such trials present numerous challenges that limit their applicability in real-world settings. As a consequence, pragmatic trials are increasingly viewed as a research design that overcomes some of these barriers with the potential to produce findings that are more reproducible. Although pragmatic methodology in long-term care is receiving increasing attention as an approach to improve successful dissemination and implementation, pragmatic trials present complexities of their own. To address these complexities and related issues, experts with experience conducting pragmatic trials, developing nursing home policy, participating in advocacy efforts, and providing clinical care in long-term care settings participated in a virtual consensus conference funded by the National Institute on Aging in Spring 2021. Participants identified 4 cross-cutting principles key to dissemination and implementation of pragmatic trial interventions: (1) stakeholder engagement, (2) diversity and inclusion, (3) organizational strain and readiness, and (4) learn from adaptations. Participants emphasized that implementation processes must be grounded in the perspectives of the people who will ultimately be responsible for implementing the intervention once it is proven to be effective. In addition, messaging must speak to long-term care staff and all others who have a stake in its outcomes. Although our understanding of dissemination and implementation strategies remains underdeveloped, this article is designed to guide long-term care researchers and community providers who are increasingly aware of the need for pragmatism in disseminating and implementing evidence-based care interventions.

Randomized controlled trials are considered the most rigorous research design in efficacy and effectiveness research.¹ However, such trials present numerous challenges that limit their applicability in real-world settings. For one, participants are typically required to meet strict enrollment criteria, which may exclude some individuals who would benefit from the intervention. Also, controlled trials have intervention protocols that may be impractical to implement in the broader community. For these reasons, it is not uncommon that efficacious interventions subsequently implemented under less controlled conditions among more diverse participants lack the benefits observed in the original trial. For example, several well-designed trials recently conducted in nursing homes lacked efficacy when expanded beyond initial testing despite promising initial results.^2–5 Importantly, these unexpected results prompted additional analyses to understand how differences in implementation may have contributed to variability in findings across study sites.⁶ Implementation challenges identified by these and other studies conducted in long-term care include high staffing turnover, regulatory constraints, lack of culture focused on scientific inquiry, and reluctance to innovate.^7,8

Pragmatic trials are increasingly viewed as a more practical research design that overcomes some of these barriers and has the potential to produce data that are relevant when moved into typical care settings. A recent National Academies of Sciences, Engineering and Medicine report⁹ called for more pragmatic trials, and the applicability of pragmatic methodology in long-term care is receiving increasing attention.¹⁰ A pragmatic approach may be particularly relevant in long-term care settings such as nursing homes and assisted living communities because these are complex care environments in which enrolling subjects in research studies may be complicated by cognitive impairment requiring second-party consent.⁹ In addition, enrollment may be limited by short remaining life-years.^11,12

That said, pragmatic trials present complexities of their own, including that many do not fully meet the criteria necessary to constitute a true pragmatic trial. By way of example, a recent JAMDA editorial examined 6 pragmatic trials and found that nursing home recruitment and eligibility had restrictions that limited their generalizability.⁵ All too often, implementation—the integration of a new practice within a specific setting or context—is not fully informed by those participating in the trial itself, and the literature is virtually silent on issues related to dissemination—distribution of an intervention or innovation to a specific audience. To address these and related topics, the National Institute on Aging funded a virtual consensus conference titled “Pragmatic Trials in Long-Term Care: A Consensus Conference” in spring 2021. The conference included presentations by researchers with experience conducting pragmatic trials in long-term care,¹³ followed by 2 expert panel presentations that discussed implementation and dissemination. The Implementation Panel consisted of 5 individuals with expertise in nursing home policy, advocacy, and clinical care; they were charged with describing how researchers can support adoption of evidence-based interventions. The Dissemination Panel consisted of 4 members from different geriatric professional organizations, who were charged with describing how researchers can make evidence available to relevant audiences.

These panels were included in the conference for 2 key reasons. First, pragmatic trials must be informed by and involve the stakeholders who know best about dissemination and implementation. Unfortunately, there has been limited feedback from direct stakeholders to guide trials in these areas and so limited related guidance. Second, program evidence must be made available beyond the scientific community, and opportunities to participate in pragmatic trials must be widely communicated, which has received scant attention. The purpose of this article is to present the challenges and opportunities identified by these expert panel participants along with tools identified in the literature that might inform future implementation and dissemination approaches specific to pragmatic trials in long-term care.

Cross-cutting Challenges and Opportunities in Implementation and Dissemination

The panelists raised 4 key challenges and opportunities relevant to both implementation and dissemination of evidence-based practices: (1) stakeholders engagement, (2) ensure diversity and inclusion, (3) assess organizational strain and readiness, and (4) learn from adaptations (Table 1).

Table 1.

Implementation and Dissemination Opportunities in Long-Term Care

Opportunities	Description
Cross-cutting opportunities to support adoption and availability of evidence-based practices in long-term care settings
Stakeholder engagement	Promote bidirectional input between researchers and long-term care staff and other stakeholders at all phases of research
Diversity and inclusion	Accommodate staff from diverse backgrounds to inform design, adaptations, and dissemination of results
Organizational strain and readiness	Consider the readiness of a setting for implementation, keeping in mind that settings the least ready may benefit the most from dissemination to their environment
Learn from adaptations	Identify core elements to measure fidelity and track within intervention changes to noncore elements
Implementation opportunities to support adoption of evidence-based practices
Workflow integration	Design interventions that are compatible with existing workflows
Agility and responsiveness	Develop an infrastructure, including funding strategies, that allow for rapid implementation of evidence-based interventions
Dissemination opportunities to make evidence-based practices available
Package the message for the audience	Match language and content of dissemination materials to individual audiences
Engage diverse audiences	Make evidence available beyond typical scientific audiences
Apply dissemination and diffusion tools	Gain momentum by using existing tools and stakeholder engagement strategies to design for dissemination

Stakeholder Engagement

Participants noted a paucity of engagement of nursing home staff, residents, family members, and long-term health care system leaders in long-term care research. They emphasized the importance of obtaining buy-in from those who will be implementing a new practice and also the value of learning about important gaps and ineffective care practices from the perspective of direct care and clinical and managerial staff, residents, and their families. These individuals can collaborate to generate pragmatic strategies to address identified shortcomings in care and inform implementation strategies. Such engagement allows for identification of opinion leaders and respected peers within the organization who can influence adoption of new practices or necessary adaptation of existing practices to make them more palatable and scalable. In a qualitative analysis of interviews with nursing home leaders, the keys to effective partnership included a desire for clear processes to prioritize and implement new interventions, an interest and belief in the value of the project, a positive impact on their facility and residents, and flexibility.¹⁴

Unique examples of successful stakeholder engagement include the Living Lab for Aging and Long-Term Care in Maastricht, Netherlands, and Abe’s Garden in Nashville, Tennessee.¹⁵ Both of these programs were developed as models of a sustainable academic partnership with nursing homes and stakeholders.¹⁶ In the case of the Living Lab, a fundamental concept supporting successful dissemination and implementation are “Linking Pins”—scientific and practice-based personnel who have joint appointments in a nursing home organization and a local university. Linking Pins are senior researchers who work 1 or 2 days weekly in the nursing home to implement research activities; the practice-based Linking Pins are employees of the nursing home organization that funds 1 day per week of scientific research engagement at the collaborating university. Following the success of the initial Living Lab, similar labs have been established in 5 other locations throughout the Netherlands. In the case of Abe’s Garden, long-term care researchers at Vanderbilt University are designing research in partnership with long-term care staff. Successful implementation for both the Living Lab and Abe’s Garden hinge on reciprocity to permit a deeper understanding of both the long-term care and research environments.

Reciprocity was also noted as a key feature of stakeholder engagement in the Pragmatic Trial of Video Education in Nursing Homes (PROVEN), one of the first large cluster randomized pragmatic clinical trials in nursing homes.¹⁷ In this study of 119 experimental and 241 control nursing homes, the intervention offered advance care planning videos. Researchers interviewed staff in facilities with high and low adherence rates, finding that a key component to successful implementation was reciprocal facilitation in which stakeholders engaged in the design of the intervention while researchers and organizational leaders supported implementation.

Key stakeholder engagement resources are available from the Patient-Centered Outcomes Research Institute, which provides a framework and toolkit to identify target audiences and guidance on including stakeholders throughout the implementation and dissemination process.¹⁸ Another resource is the INSPIRE Research Portal, an online library of tools focused on active engagement throughout the research process.¹⁹ In addition, the CERTAIN Patient Advisory Network may be useful; it is composed of patients, caregivers, and family members who serve as “Patient Advisors” to partner in informing research through a learning health system.¹⁹ Although many of the tools can be adapted to engage stakeholders in long-term care, they are not specifically designed for the populations typically enrolled in studies of older, frail, cognitively impaired individuals.

Ensure Diversity and Inclusion

An important step in the design of implementation and dissemination processes is accommodating nursing home staff from diverse ethnic, racial, and educational backgrounds. Diversity is clearly a valuable means of obtaining a wide range of ideas; at the same time, it requires tailoring meeting structure and study materials to different backgrounds. For example, staff may not be native speakers of the language spoken by others in the organization and may benefit from translation or adaptation of materials to match their language skills. Not all communication need be written, though, and work with marginalized individuals from resident councils in Dutch nursing homes found that storytelling workshops were an effective means of eliciting perspectives prior to research, obtaining feedback throughout the trial and sharing results.²⁰ Diverse and inclusive research teams have the potential to improve implementation and dissemination to a broader audience.

Organizational Strain and Readiness

In assessing fidelity of the PROVEN Trial intervention, the authors recommended formal assessment of organizational strain and champion readiness prior to and throughout implementation so as to increase the likelihood of maintaining fidelity and triggering remediation when appropriate.¹⁷ Despite this recommendation, conference attendees did not reach consensus on the value of assessing organizational readiness and limiting inclusion to those settings most ready. Some panelists thought doing so was critical to implementation success, whereas others recognized that dissemination and implementation to those less ready is important to reach lower-functioning homes. Further, a nursing home that is ready at one point may be ill-equipped a short time later owing to turnover and, thus, readiness may need periodic assessment. Researchers must anticipate the need to adapt to organizational strain as they implement and scale interventions.

Learn From Adaptations

Participants noted that although researchers traditionally rely on systematic reviews or meta-analyses to choose evidence-based interventions, details about within-intervention adaptations are critical to fully inform implementation because in the real world, implementation will vary somewhat. Therefore, emphasizing the components of implementation that are core (critical) to success and the degree to which noncore elements are adapted for ease of implementation is essential. Such information is necessary to untangle the key ingredients to success in multicomponent interventions and also to disentangle why a trial may be successful in one setting but not in another.

Publications rarely focus on why interventions fail, despite the value of learning from those failures. One panel participant commented that he has supported many nursing home trials and within minutes of implementation, he and the research team immediately realize adaptations are required. Systematic tracking of adaptations to understand the implementation and dissemination potential of a given intervention may be far more useful than simply understanding that an intervention was effective.²¹ In addition, it highlights the benefit of maximizing the expertise of stakeholder engagement and reciprocity.

Panel participants recommended integrating assessments of adaptations that permit fidelity while allowing for necessary changes in implementation. Fidelity was addressed by researchers who implemented the PROVEN Trial, finding that while it was essential to deliver core elements of the intervention, successful implementation was also dependent on site adaptation to noncore elements.¹⁷ Similarly, the Helping Invested Families Improve Veterans’ Experience Study (HI-FIVES) used results from their original randomized controlled trial to adapt the intervention prior to implementation at additional sites. For this effort, the Replicating Effective Programs implementation framework guided rapid adaptations while retaining core intervention components.²² Ideally, core elements and mechanisms are identified prior to implementation. Of course, it is not always clear what constitutes the “core” elements until findings discern a lack of success, making it that much more important to track fidelity.

Implementation Challenges and Opportunities

Panelists identified 2 key challenges and opportunities related specifically implementation or adoption of evidence-based practices: (1) workflow integration and (2) agility and responsiveness (Table 1).

Workflow Integration

Panel participants described compatibility with existing workflows and experiences as important for implementation. They encouraged researchers to learn from and integrate with Quality Assurance and Performance Improvement programs mandated by the Centers for Medicare & Medicaid Services.²³ These programs are intended to continually conduct systematic, comprehensive, and data-driven tests of change. Quality improvement programs are designed to improve care by involving residents, families, and staff in problem-solving; in so doing, they provide an opportunity to learn what issues are of greatest concern to stakeholders. These programs have the potential to provide insight into how the organization typically addresses barriers, challenges, and concerns, and how to support successes.

Some studies have leveraged the quality improvement design structure and may serve as a model for future trials embedded in nursing homes. Indeed, some pragmatic trials, if effectively integrated and embedded into nursing home operations, may be indistinguishable from quality improvement efforts and thus may reduce some regulatory thresholds that hinder the conduct of pragmatic trials (eg, complex informed consent procedures).²⁴ Other opportunities to implement interventions compatible with existing workflow include substitution rather than addition of new tasks and use of micro-learning on handheld devices for younger direct care workers.²⁵ Deimplementing ineffective and inefficient care processes are also key to permitting integration of new evidence-based tasks. Interventions should impact quality of life and quality of work while eliminating or placing less emphasis on nonessential tasks that detract from focal areas.

Agility and Responsiveness

Successful research partnerships require stable leadership from individuals who can clearly prioritize implementation of new initiatives and interventions.¹⁴ However, consensus panel participants noted that in pursuing research funding, there is an unacceptable time lag between when a proposal is written and when funding is received to launch the trial. During this prolonged interval, leadership may change, and a new leadership team may not be as interested in what might seem an “outdated” initiative or one that no longer aligns with the strategy/philosophy of new leadership. Interestingly, lessons learned during the COVID-19 pandemic revealed that research funding models may be made more nimble under rapid review options. The experience of rapidly developing interventions and responding to new information needs in long-term care highlights the opportunity for more agile intervention implementation, dissemination, and funding models.⁸ COVID-19 also shifted the focus of many long-term care executives to actively work with researchers in search of evidence-based research to inform clinical care and policy.²⁶ Sustaining an agile and responsive implementation context provides an opportunity for bidirectional communication about priorities and to design pragmatic trials that can thoughtfully integrate dissemination and implementation plans in a timely manner.

The recent emphasis on Learning Health Systems may also serve as a sustainable means of integrating a culture of research that is less dependent on health care crises such as a pandemic, and less vulnerable to individual leadership priorities.²⁷ The Agency for Healthcare Research and Quality defines a learning health system as “a health system in which internal data and experience are systematically integrated with external evidence, and that knowledge is put into practice.”²⁷ These systems foster a culture of generating evidence to guide care and have the potential to transcend the challenges associated with shifting priorities and turnover among individual leaders and staff. They also focus on identifying clinician scientists who undergo training in learning health systems and are more fully “embedded” into the health system as a nexus between intervention development/testing and their implementation into real-world care.

It is important to note that organizations that engage in continuous quality improvement tend to have stronger learning and knowledge cultures and are more open to research partnerships, whereas those in greatest need of quality improvement are often the least likely to be able to adopt and sustain evidence-based programs. Therefore, part of the challenge is promoting agility in long-term care settings that might benefit the most.

Dissemination Challenges and Opportunities

The panelists described 3 key challenges and opportunities in dissemination: (1) package the message for the audience, (2) engage diverse audiences, and (3) apply dissemination and diffusion tools (Table 1).

Package the Message for the Audience

Panel participants cautioned that language used for an academic audience may not resonate with, or worse, may generate fear among, direct care providers. One participant stated that many of his staff equate the word research with adding to their already overwhelming daily tasks. Thus, messaging is critically important to effective dissemination.

Tools such as the NIH Checklist for Communicating Science and Health Research to the Public offer guidance on describing the research and why doing so is important.²⁸ The NIH Collaboratory Living Textbook of Pragmatic Clinical Trials also provides practical guidance on dissemination to different audiences ranging from the scientific community to patients and health care systems.²⁹ As mentioned in the stakeholder engagement section, messaging need not always be written, and storytelling with user-friendly language can be a powerful means of dissemination. Storytelling increases sensitivity to an experience and, as a consequence, serves as a pragmatic modality to make the information relevant to the people affected by the problem.³⁰

Engage Diverse Audiences

Panel participants noted the tendency of researchers to limit dissemination efforts to academic publications and presentations to peers in geriatrics and gerontology. Worse is that many of those publications are not freely accessible on the internet, and so even if clinicians or administrators learn of a publication, they are not able to read it. As a result, the vast majority of research is not integrated into practice and the research that does inform policy or clinical care typically does so only after several decades following initial publication.³¹ Participants recommended expanding efforts to reduce this lag by collaborating with other organizations to disseminate research findings more broadly and tailoring the message to different audiences. Opportunities to disseminate results beyond traditional outlets include connecting with advocacy groups and identifying social and news media sites frequented by those most likely to implement new programs or benefit from the study results.

A first step includes gathering information from a broad range of stakeholders—including nursing home residents, families, and staff—about where they seek information and how to effectively deliver the information to them. Returning to the earlier recommendation of engaging stakeholders, this process should begin from the initial stages of study design. In this planning, the stakeholders should collaborate with researchers to develop a plan of how participants will receive study results, what information should be included and when communication should occur. Although not a traditional practice, the National Academies of Sciences, Engineering and Medicine recommends that research results be shared with research participants to foster transparency, trust, and respect for participants.³² The Multi-Regional Clinical Trials Center has produced toolkits to guide this process.³³

Apply Dissemination and Diffusion Tools

Participants noted the increased focus on implementation frameworks in recent years and described dissemination and diffusion theories/frameworks as of value. These theories and frameworks can strengthen pragmatic trials if considered early in the intervention process. Beyond requirements to register and report data on ClinicalTrials.gov and other reporting sites, dissemination toolkits with guidelines, worksheets, and case studies are available to guide researchers through a variety of theories and frameworks.^18,34 As an example, the Patient-Centered Outcomes Research Institute (PCORI) provides a Dissemination and Implementation Framework and Toolkit on their website. This resource provides links to a 1-page guide with NIH recommendations for use of plain language along with descriptions of how to use storytelling as a dissemination tactic and how to return results to research participants.¹⁹

Another fundamental concept mentioned by panel participants that is distinct from dissemination as typically conceived are diffusion processes. Developed by E.M. Rogers in 1962, Rogers’ Diffusion of Innovation Theory organizes diffusion into stages by which innovations are adopted and dissemination occurs.³⁵ Diffusion is defined as a social process that explains how an idea or innovation gains momentum to spread through a social system. Importantly, most innovations fail to diffuse after dissemination of information.³⁶ As with implementation and traditional dissemination, designing diffusion strategies ideally occurs early and purposefully by engaging key stakeholders to increase the likelihood of successful diffusion in the shortest time possible. Multipronged efforts have been successful in diffusion of long-term innovations and, unsurprisingly, nursing homes with more resources and higher quality care tend to serve as the innovators.^21,37,38

The Diffusion of Excellence Initiative, launched by the Veterans Health Administration in 2015, aims to accelerate awareness and adoption of clinical innovations.³⁹ This initiative is accompanied by a web-based Diffusion Marketplace where evidence-based practices are featured, including practices such as age-friendly health systems and the life-sustaining treatment initiative in long-term care.⁴⁰ This marketplace represents an important model in scaling promising practices across a health system and may serve as a prototype for a similar online platform featuring long-term care practices ready for scaling and spread.

Implications for Practice, Policy, and Research

Panel participants acknowledged the challenges of pragmatic trial implementation in the complex long-term care setting. At the same time, unprecedented opportunities exist that were uncovered during the COVID-19 pandemic that generated global awareness of the interconnectedness of community health and long-term care, specifically, nursing home care.

This article summarized challenges, opportunities, and potential next steps to address key implementation and dissemination issues related to conducting pragmatic trials in long-term care settings. If an intervention from any pragmatic trial is to be adopted, 2 critical areas must be addressed: (1) its implementation processes must be grounded in the perspectives of those who will ultimately be responsible for implementing it once it is proven to be pragmatic, and (2) its messaging must speak to those same individuals, and to all others who have a stake in its outcomes. Clearly, much work remains to understand how to accomplish these 2 goals in the design and conduct of pragmatic trails. Our understanding of dissemination and implementation strategies for long-term care settings is underdeveloped and will require systematic evaluation to permit integration of evidence-based strategies into these complex care environments.