Applying a model of stakeholder engagement to a pragmatic trial for people with mental disorders

Abstract

Stakeholder engagement is an important component of pragmatic trials testing programs to improve mental healthcare in real world settings. Models of stakeholder engagement outline the benefits of involving a diverse array of partners in all phases of research. The authors describe a stakeholder engagement plan for a comparative effectiveness pragmatic trial focused on increasing linkage between emergency departments and outpatient treatment at community mental health centers. Benefits of stakeholder engagement include meaningful input on program design and implementation, insights into balancing the characteristics of each site while keeping study fidelity in mind, and early discussions about program sustainability and dissemination.

Stakeholder partnerships represent a critical element in patient-centered trials. Involving stakeholders throughout all phases of research can help ensure study feasibility and appropriateness of the program being tested, and help balance fidelity of the program with the need to adapt to differences across complex clinical sites. Stakeholders can include patients, caregivers and family members, providers, purchasers, payers, advocacy organizations, and policy makers (1,2). The goals of engagement are to develop meaningful and effective partnerships, increase the relevance of research to patients, facilitate the use of research results in healthcare settings, and improve patient health (3).

While stakeholder engagement is valuable throughout the development, implementation, and dissemination of a research project, there are few examples of partnerships in pragmatic trials aimed at improving mental healthcare. Here we describe a stakeholder engagement strategy for a multisite comparative effectiveness study. The Engaging Patients in Care study aims to increase linkage between emergency departments (EDs) and outpatient treatment at Community Mental Health Centers (CMHCs) for patients with mental disorders. The study compares care navigation provided by mental health professionals and peer support specialists (PSS) in facilitating linkage from the ED to outpatient treatment and supporting ongoing engagement in care. Care Navigators in the Engaging Patients in Care program use shared decision making, motivational interviewing, and goal setting to guide patients in identifying and addressing barriers they experience to engaging in outpatient care. In this column, we describe partnerships throughout the study phases, with a focus on who is involved, how stakeholders are engaged, and lessons learned.

Stakeholder Engagement

Our stakeholder engagement approach, which is based on the Patient Centered Outcomes Research Institute’s (PCORI) model for patient-centered outcomes research (3,4), incorporates the following principles: Stakeholders are involved in every step of the study, from design to dissemination and sustainability; we seek to actively engage a variety of stakeholders; with our partners, we aim to foster an environment of open communication, co-learning, and mutual respect; and stakeholders are full partners in all key study decisions.

This study was developed as a partnership between Emory University, the University of South Carolina, and the South Caroline Department of Mental Health (SCDMH). Stakeholders include patients who use mental health emergency care; mental health providers; public sector healthcare payers and purchasers, including the SCDMH and the South Carolina Department of Health and Human Services (SCDHHS); and mental health workforce and advocacy organizations. Partnerships with these groups are essential for mental health programs given the continued fragmentation between the mental health, substance use, and medical healthcare systems and the challenges in providing services that may fall outside of the typical reimbursed services (5). Below, we describe how we have engaged these stakeholder groups. See Appendix 1 for additional information on engagement strategies, roles, and results of the different partnerships.

Establishing and Maintaining Partnerships

The study is guided by a national Advisory Board and state Implementation Team, which advise the academic researchers. The purpose of these boards is to ensure the relevance of the research findings to a variety of audiences and to plan for program sustainability (3). Existing collaborators at state governmental and advocacy agencies helped us to identify and approach members for these two teams, including a patient and PSS from a local CMHC. We have not encountered difficulties in finding patients and providers willing to serve on the board and have experienced minimal turnover so far.

The Advisory Board, which meets annually, oversees key study decisions and will spearhead planning for program sustainability and national dissemination of results. The Advisory Board is chaired by the director of a consumer-run mental health agency and includes four additional members from national-level behavioral healthcare systems management, patient advocacy, and mental health workforce development organizations. The Advisory Board recommended adding mental health recovery as an outcome measure, which was incorporated into the study proposal, and provided input that was incorporated into the care navigator program.

The Implementation Team oversees the planning and execution of the study, with the aim of ensuring that findings are applicable to patients and to the public mental health system. The Implementation Team also facilitates co-learning between the partners and the research staff; stakeholders gain insight into the research process through involvement in study planning and decision-making, and the research staff learn how to best engage patients and incorporate patient-centeredness in the local context. The five members represent patients and providers from local CMHCs in South Carolina, a patient advocate from NAMI South Carolina, and representatives from SCDMH and SCDHHS. The Implementation Team has contributed to the refinement of recruitment and enrollment protocols, and development of the intervention manual and materials. It has also provided the study team with a greater understanding of the state mental healthcare system, such as which billing codes apply for care navigation strategies (See Appendix 2 for examples).

Site visits are another strategy to initiate partnerships with stakeholders involved in participant recruitment or program implementation. The research team made site visits to build rapport with partners, such as mental health liaisons, ED nurses, telepsychiatrists, and CMHC clinic staff; learn about site-specific contextual factors; and finalize training with care navigators. CMHC clinic directors helped to identify professionals and peers at their clinics who could serve as care navigators. These visits also resulted in adjustments to the care navigators’ training, patient recruitment protocols, and warm-handoff phone call, which is the first interaction between the care navigator and patient in the ED (See Appendix 2 for examples). Speaking with site representatives led to the development of site-specific back-up plans, such as training extra staff to step in when the normal care navigator is sick or on vacation. Mental health liaisons proved to be valuable partners because they are DMH employees who split their time between the ED and CMHC, thus providing a linkage between the two systems. However, not every CMHC has a mental health liaison and staff turnover has been a challenge because it necessitates forming new relationships.

Gathering and Using Stakeholders’ Perspectives

We are using qualitative methods to gather patients’ and clinicians’ perspectives and to inform the development, implementation, and revisions to the program. Qualitative methods allow us to gain in-depth information about patients’ and clinicians’ experiences with care linkage and engagement, and feedback on the care navigator program.

During the development of the care navigator program, we conducted patient focus groups and key informant interviews to ensure the feasibility, patient-centeredness, and applicability of the program. Two focus groups were conducted with 12 patients at two CMHCs who had previous experience in an ED. Key suggestions from these focus groups included that care navigators should treat each patient as an individual and not assume that all experiences are the same. Participants recommended that care navigators convey compassion and genuine caring. As one patient explained,

...you need to listen, you need to pay attention to us, you need to be aware where you’re coming from and not judge us… We need you to listen and feel, because sometimes it’s hard for us to explain these things.

Interviews with seven key informants provided insights into the current practices in the ED or CMHC in which they worked and any linkages between the two. Key informants discussed the importance of validating patients’ concerns, as well as being nonjudgmental and supportive. As one key informant noted, “I would think to try to find some way to say [to patients], ‘We’re here for you’.” They also suggested that when care navigators talk to patients in the ED, they should describe what the patients can expect when they arrive at the CMHC for their first appointment after discharge. One key informant said,

And the other thing that might be helpful for new people is actually having brochures about the mental health center and what we can do. ‘Cause a lot of times they’re like, “I was just told to show up here, I don’t know what I’m doing”. That doesn’t mean they weren’t told, but when you’re in a crisis, you don’t hear and process well.

The recommendations from patients and key informants were incorporated into the care navigator program, training, and materials. We revised wording to clearly convey empathy and support, added potential barriers to receiving mental health care to the manual, and developed materials to give to the patients in the ED.

Qualitative data collection is continuing annually with patients and care navigators throughout the study. The goal of the participant interviews is to examine patients’ experiences with care navigation and assess acceptability of the care navigator program. Participants discuss the barriers they face in engaging in mental health care, whether and how the care navigator program has been helpful in overcoming these barriers, and recommended improvements for the program. Interviewing care navigators provides an in-depth understanding of their experiences in delivering the program. Care navigators describe encounters with participants, how well the program fits with their clinical duties, successes and challenges with program delivery, and feedback on how to improve the program. Each year, findings from the interviews will inform program implementation, with adjustments being made as needed.

Lessons Learned

Based on our experiences so far, three main lessons regarding stakeholder engagement are the benefits of early stakeholder involvement, gathering insight into the particular characteristics of each site while keeping overall study fidelity in mind, and early initiation of discussions with stakeholders about program sustainability.

First, early involvement from a diverse range of stakeholders allows for guidance on key aspects of the study, including the choice of outcome measures, program design, and logistical planning for recruitment and program implementation. The nature of our study, which focuses on transitions of care, necessitated partnerships across a range of settings, including CMHCs, EDs, and the relevant state-level departments. Patient input was invaluable to understanding their experiences in the ED and being linked to the CMHC. Likewise, providers at the CMHCs provided insight into how the care navigator program would best fit into their daily work. Our other stakeholders helped us to better understand how this program would work in the state healthcare system. Important adaptations to the intervention that were based on our stakeholders’ input included training on billing codes, preparing the patient for their first visit to the CMHC, and ensuring discussions of barriers to care and goal setting were meaningful and easy to follow. Having multiple ways in which to engage stakeholders and elicit feedback has been central to our goals of ensuring the patient-centeredness and feasibility of the care navigator program.

Second, stakeholder perspectives have been important in the process of balancing the need to implement the program with fidelity while also allowing for flexibility to meet the diverse needs of local sites and patients. Procedures differed across ED and CMHC sites in multiple ways, including staffing, patient populations, connections between the ED and CMHC, and appointment scheduling methods. Therefore, we needed to develop the care navigator program in such a way that it could be adapted to different sites and personalized for patients, while still maintaining the same core structure. We designed the encounters between care navigators and patient participants to be implemented similarly across sites, with other aspects, such as the logistics of making appointments, being based on the specifics of the site. Qualitative data collection and documentation of the differences between sites will aid in determining in which settings and for which patients the care navigation program works best.

Finally, early engagement of stakeholders in discussions on issues within the healthcare system, such as identifying reimbursement methods for care navigation services, is setting the course for longer-term planning for program dissemination and sustainability. The Implementation Team has provided valuable insight into how care navigators can currently bill for services. It will lead planning for dissemination and sustainability of the care navigator program within the state. The Advisory Board will be central in determining how to roll out the program nationally.

Conclusions

Pragmatic trials examining the effectiveness of programs to improve mental healthcare benefit from the involvement and insights of diverse stakeholders. Establishing a stakeholder engagement plan facilitates early, intentional, and meaningful stakeholder involvement. Stakeholders can provide valuable input on the design and implementation of interventions and study designs, particularly around the heterogeneous factors at the patient, provider, and system levels that can affect treatment linkage and engagement. Continued involvement can contribute to program refinement, dissemination of results, and sustainability planning.

Appendix

Appendix 1.

Engagement strategies, roles, and results for the different stakeholders in a pragmatic trial for people with mental disorders

Stakeholders	Engagement strategies	Role	Results
Patients
• Patients at community mental health centers, particularly those who have been to the ED	• Developmental focus groups • Annual interviews with participants • Members on Implementation Team	• Provide feedback on program design • Provide feedback on program implementation	• Revisions to Care Navigator manual
Clinicians
• Certified peer specialists and mental health professionals at CMHCs	• Key informant interviews • Site visits • Member on Implementation Team • Annual interviews with Care Navigators	• Provide feedback on program design • Provide feedback on development and implementation of Care Navigator training • Provide feedback on program implementation	• Revisions to Care Navigator manual • Tailored program to fit in with their work in the CMHC
• Mental health professionals at EDs	• Key informant interviews • Site visits	• Provide feedback on program design • Inform recruitment strategies	• Refined recruitment protocols • Developed a script for providers involved in referring patients to the study • Developed protocol for ED staff to distribute study materials to participants once they enroll
Implementation Team
• Patient • Provider: Certified peer specialist • Patient advocacy organization • Payer: State Department of Health and Human Services • Policy maker: State Department of Mental Health	• Co-learning between Implementation Team partners and research team • Open communication	• Oversee program planning and implementation • Facilitate access to sites and telepsychiatry • Ensure appropriateness of the program for the CMHCs and EDs • Ensure patient-centeredness • Facilitate dissemination of results within South Carolina	• Refinement of recruitment and enrollment protocols • Understanding of billing codes that apply to intervention activities • Refinements to Care Navigator manual
Advisory Board
• Peer workforce non-profit organization/consumer • Purchaser/consumer/ patient advocate: Public sector behavioral health managed care company • Provider/payer: Behavioral healthcare systems management • Provider/Mental health workforce organization • Provider	• Decisions by consensus • Open communication	• Input on study proposal • Input on program planning and implementation • Oversee key study decisions • Spearhead planning for program sustainability • Facilitate dissemination of results to national audiences	• Addition of mental health recovery as an outcome measure • Refinements to Care Navigator manual

Abbreviations: CMHCs, Community Mental Health Clinics; EDs, Emergency Departments

Appendix

Appendix 2.

Changes to the Engaging Patients in Care Program based on stakeholder engagement and input

Component of the Engaging Patients in Care Program	Changes resulting from stakeholder engagement
Recruitment and Enrollment Process	• Materials were developed for the mental health professionals who are involved in referring patients to the study team. These materials included cards with information about the study and the eligibility criteria. • Materials were developed to give to the patient when they were still in the ED. These materials include a figure describing the steps of the study, a card with the contact information for their Care Navigator, and directions to get to the CMHC. • Worked with stakeholders at each ED to establish a person who would be responsible for overseeing and storing study materials.
Care Navigators – Training	• Incorporated more information on billing codes for professionals and peers. • Added an activity for Care Navigators to identify procedures for making appointments at their clinic. • Allowed for training of additional individuals who could serve as back-up Care Navigators in case the main Care Navigator was on vacation or medical leave.
Care Navigators – Program Manual	• Ensured that the scripts for the Care Navigators clearly conveyed empathy and support. For example, the script emphasized that the Care Navigator was looking forward to meeting the patient and that the Care Navigator and the patient would be closely working together. • For the warm hand-off phone call between the Care Navigator and patient while the patient is in the ED, the script was revised to include information about what the patient can expect to happen at the first visit at the CMHC. • Added to the list of attitudinal and logistic barriers to care that Care Navigators discuss with patients. Example additions include: Symptoms bothering me too badly to come; Fear that staff will find out I am not taking my medication; Discomfort with sharing private details with strangers. • Revised the sheet that Care Navigators use with patients to guide them through the goal setting process to address a barrier to care. To make it more user friendly, the goal setting sheet was simplified and an example was included. • Ensured that participant encounter documentation was brief and easy to complete, while capturing necessary information, to minimize the burden on Care Navigators.

Abbreviations: CMHCs, Community Mental Health Clinics; EDs, Emergency Departments