Facilitators and Barriers to Patient-Centered Outcomes Research Partnership Sustainability in the United States

Abstract

Purpose

Engaging patients in research can enhance relevance and accelerate implementation of findings. Despite investment in patient-centered outcomes research (PCOR), short-term funding cannot maintain such efforts beyond the program timeframe. Sustained interaction between researchers, practitioners, patients, and other stakeholders is needed to sustain use of evidence-based practices and achieve maximum benefit. While previous literature describes components of public health program sustainability, such factors do not necessarily apply to the partnerships that implement those programs, and facilitators are likely to differ across disciplines. We sought to determine facilitators and barriers to PCOR partnership sustainability from participant experiences with sustainable and unsustainable community-academic partnerships across the United States.

Methods

From 2017 to 2019, a collaboration representing public health institutes, community-based organizations, and academic organizations convened PCOR partnership members in virtual focus groups and conducted qualitative analysis to identify facilitators and barriers to partnership sustainability. A grounded theory framework, which applied a combination of a priori codes (barriers, facilitators, sustainable, not sustainable) and open coding, guided participant selection, data collection, and analysis across all project stages.

Results

There was no single definition of partnership sustainability. Common facilitators of sustainability were investing time in relationships, connector role to promote communication and trust, equal power dynamics, shared motivation for participation, partnership institutionalization, and reciprocity. Barriers to partnership sustainability included external factors influencing participation and operations, funding-related challenges, and lack of institutionalization.

Conclusions

PCOR partnerships should incorporate an early and ongoing focus on relationship development through intentional efforts to collaborate with specific partners and stakeholders according to the goals of the research. This would allow more patients to access the evidence-based practices resulting from research investments.

There is growing attention toward engaging patients, community members, and other stakeholders in research to enhance the relevance of findings and accelerate implementation of evidence-based practices.^1–3 Patient-centered outcomes research (PCOR) methodology emerged with the purpose of producing evidence that is meaningful to patients and treatment decisions. PCOR incorporates patient perspectives into evaluating outcomes and risks of health care interventions across a variety of populations and settings.^4,5 A key element of PCOR is collaboration between community- or patient-serving and academia-based stakeholders to ensure that both research methodology and patient perspectives are rigorously employed.⁶

Despite U.S. investment in PCOR, programs supported by short-term funding generally cannot survive beyond the program timeframe.⁷ This is problematic because few health interventions are sustained to fulfill their maximum benefit.^8,9 Sustained interaction between researchers, practitioners, patients, and other stakeholders is needed to sustain use of evidence-based practices.¹⁰ While previous literature describes components of public health program sustainability, contributing factors are likely to differ according to the field and goals of the program,⁷ and such factors do not necessarily apply to the sustainability of the partnerships that implement those programs. Understanding best practices to sustain partnerships for PCOR is crucial to maintaining any positive outcomes achieved by such efforts.

Sustainability is emphasized as a key component of research and collaborative partnerships, but there is little evidence to suggest how specific factors might promote or hinder research partnership sustainability.^11–13 Most evidence on program and research sustainability comes from the field of community-based participatory research (CBPR), and the majority concerns sustainability of programs and interventions rather than the partnership itself.^14–16 Key contributors to program sustainability fall into project design and implementation factors, such as presence of a program champion and effectiveness; organizational factors, including fit with the organization’s goals and leadership support; and broader community environmental factors, such as community and stakeholder support.^10,13,17 PCOR and CBPR share similar guiding principles around community engagement. However, PCOR is distinct from CBPR in its particular emphasis on health care settings and patient stakeholders.^5,18 The extent to which recommendations for sustainability from CBPR and public health programs apply to PCOR partnerships is unclear. Despite the growing base of research regarding PCOR methodology, there are no guidelines to support the sustainability of PCOR efforts to complement those methodological standards.¹⁹

To address this, we conducted qualitative analysis of focus groups and conference proceedings with PCOR stakeholders on community-academic partnership sustainability, incorporating the contributions of both individual and organizational-level relationships to the sustainability of partnerships. Participants with varied and extensive PCOR expertise from community organizations and academic institutions across the United States discussed particularly sustainable or unsustainable community-academic partnerships. This paper presents a summary of common facilitators and barriers to PCOR partnership sustainability identified by participants. Results can provide important implications for new and existing PCOR partnerships to enhance the positive impact of research findings on patients and communities.

METHODS

Our research team received a Eugene Washington Engagement Award from the Patient-Centered Outcomes Research Institute (PCORI) to conduct a Conference on PCOR Sustainability. The aim was to engage experienced and novice PCOR practitioners to identify common facilitators and barriers to community-academic partnership sustainability. Between July 2017 and June 2019, we applied an iterative engagement strategy to promote community-led participation and decision-making across four distinct phases. This phased approach was situated within a larger grounded theory framework that guided participant selection, data collection, and analysis across all project stages.

Grounded theory is used to develop theory about processes that is grounded in systematic data collection and analysis, rather than being interpreted through existing theories.^20,21 Given the discrete differences between PCOR and other somewhat similar research methodologies, a grounded theory approach permitted the team to avoid excessive focus on the longer-established but distinct field of CBPR, thereby allowing for the possibility of unique facilitators and barriers to partnership sustainability to emerge. This grounded theory framework also allowed initial project stages to inform later data collection and analysis, further situating findings within the context of input from PCOR practitioners and participants.

This work was determined exempt from human subjects review by the Colorado Multiple Institutional Review Board (Aurora, CO).

Phase I: PCOR Sustainability Panel

Phase I assembled expert stakeholders in PCOR partnerships to provide guidance on the structure and content of subsequent phase activities designed to explore partnership sustainability. From July through December 2017, we convened 10 individuals in a PCOR Sustainability Panel for a series of 3 in-person meetings (with 1 remote participant) to prioritize topic areas in the large field of sustainability and then develop a discussion guide on partnership sustainability in PCOR. In a grounded theory framework, participants at each stage are purposively sampled according to project goals and questions. We used a purposeful sampling approach to target potential participants with past experience leading or participating in PCOR projects to ensure that participants had applicable background and knowledge to inform the topic areas and questions that should guide focus groups on partnership sustainability. Research liaisons sent an invitation email to their extensive network of stakeholders meeting this criteria, and followed up using individual email and telephone communication to recruit approximately even representation across community and academic sectors, race and ethnicity, experience level, and gender. We recruited within Colorado to minimize travel for in-person meetings.

While guided by a grounded theory framework, our team acknowledged that findings from related fields on partnership and program sustainability may be relevant to this project and felt it was important to use existing literature as a starting point to maximize the efficiency of time spent with participants. We incorporated findings of a literature review on factors that influence community-academic partnership sustainability to develop a brief web-based survey for initial topic area prioritization. Panel members were asked to select 3 topics of highest priority from a list of 8–10 topics in each of the following categories: general partnership sustainability (eg, defining sustainability, planning for sustainability, evaluation practices for sustainability in PCOR partnerships); partnership organization and structure (eg, partnership functioning and operations, decision-making, leadership, funding); relationships and collaboration (eg, diverse membership, member engagement, trust, relationships among members); and partnership efforts (eg, media and promotion, achieving goals, celebrating successes, policy change efforts). The survey also included space to share any additional comments. We administered the survey prior to the first panel meeting to allow the opportunity for participants to inform the prioritization process from project onset. The qualitative lead summarized the number of respondents that selected each topic and identified the most common priority areas in each category. This summary of topic areas prioritized by participants was used as a starting point for PCOR Sustainability Panel discussions.

At the first panel meeting, we facilitated an unstructured discussion of priority topics related to PCOR partnership sustainability based on survey responses. We presented the most commonly identified priority areas to the panel for elaboration on their relationship to sustainability, in addition to asking specific questions about additional factors that may not have been included on the questionnaire. We used the resulting input to develop and refine a semi-structured focus group guide. Our team facilitated a pilot test of the focus group guide with panel members divided into 2 groups of 5 participants each. Upon completion of their phase I participation, panel members received a $150 Visa gift card.

Phase II: Shared Learning Collaborative Discussions

In phase II, we conducted focus groups on PCOR partnership sustainability using the discussion guide developed in phase I. In contrast to phase I, in which we recruited participants with PCOR leadership experience to guide subsequent data collection, phase II participants were recruited with the goal of obtaining a wide range of participants to maximize variation in perspectives on partnership sustainability. Using purposeful maximum variation sampling,²² we invited individuals with experience as PCOR partnership members to participate in small focus groups. We aimed to recruit 25 to 30 individuals for these focus groups, allowing for variation across gender, experience levels, racial and ethnic identification, rural and urban geography, and community and academic representatives in order to obtain a wide breadth of perspectives for input on the topic of sustainability.

Community research liaisons recruited participants with PCOR partnership experience from their professional network via telephone and email. Additionally, recruitment flyers and emails were distributed across multiple email listservs to reach a nationwide network of PCOR professionals. This network-based recruitment method meant that some participants had some degree of past professional relationship with community research liaisons or the project manager, though the qualitative lead had no past relationship with participants. From February through April 2018, we convened PCOR stakeholders from across the United States in 9 separate focus groups of 2–5 participants each using web-based video conferencing (Zoom). Groups were kept to a maximum of 5 participants to ensure sufficient time for all participants to share adequate detail about their experience with partnership sustainability.

We used a structured questionnaire to gather participant characteristics of sectors represented (community and/or academic), experience in community engagement and PCOR, employment in a university setting, gender, race, and ethnicity. One week prior to each discussion, we prepared participants by sending the discussion guide and brief biographies for other focus group participants. The community research liaisons (C.B.O., G.P.J.) and qualitative lead (T.L.H.), with assistance from Trailhead Institute staff (C.F., J.S.), co-facilitated focus groups. No one outside of project team members and participants was present for focus groups. At least one member of the analysis team (T.L.H., C.F.) co-facilitated each focus group to ensure that all relevant analytic topics were covered and to contribute to efficiency of the analysis process by allowing analysts to begin familiarizing themselves with the data as soon as it was collected.

Focus group facilitators provided a brief introduction describing to participants their professional training and experience, interest in PCOR, and an overview of the project’s origins. To frame the discussion and familiarize participants with its focus, each session began by asking participants to reflect and provide feedback on the adapted definition of sustainability previously developed in phase I. To develop an initial understanding of participant experiences and set the tone for equal contribution to the discussion, we then invited each participant to share a brief 3–5-minute introductory story using the opening prompt: “Focus on your most or least sustainable partnership. What was the partnership’s topic or focus? How did partnership members come together? What was your role in the partnership? In what way(s) was this partnership sustainable or unsustainable?” Facilitators posed follow-up questions to the group for more details about identified facilitators and barriers. To complete each discussion, we offered the opportunity for each participant to respond to the final question: “Thinking about the partnership example you shared, what do you wish you would have had to facilitate sustainability?”

We provided each participant with a $100 Visa gift card via mail upon completing the focus group. Focus groups lasted 60–90 minutes. Facilitators took notes on participant comments and emergent themes throughout the focus groups. Topics and concepts that emerged in later focus groups were increasingly similar to previous focus groups as the series continued, a concept known as data saturation, giving analysts confidence that added recruitment and data collection would not add value to the dataset. All focus groups were audio-recorded and professionally transcribed.

Phase III: Conference on PCOR Sustainability

This Conference on PCOR Sustainability, held in October 2018, was led by a collaborative team representing public health institutes, community-based organizations, and academic organizations. Each partner led different aspects of the project and provided support across all areas. Project management, coordination, and fiscal administration was led by a nonprofit public health institute. Community outreach, participation engagement, group facilitation, and consensus building was led by two community research liaisons from a local nonprofit community organization (G.P.J.) and a school of public health (C.B.O.), each with more than a decade of experience and extensive training in community-engaged work.

Phase IV: Qualitative Analysis

All team members participated in interpretation of findings. Two team members conducted qualitative analysis — an MPH-trained qualitative lead (T.L.H.) employed as a research assistant at a university and an MPH-trained project manager (C.F.) at a nonprofit public health institute. The qualitative lead brought 10 years of experience conducting qualitative analysis for public health and health services research and program evaluation at the time this project was conducted, in addition to academic training on qualitative inquiry as part of recent coursework toward a PhD in health and behavioral sciences and annual continuing education through conference and workshop attendance. ATLAS.ti 8.3 data analysis software (ATLAS.ti Scientific Software Development GmbH) was used to review, code, and analyze transcript text.

We applied a combination of a priori, or predetermined, codes and open coding that defines traditional grounded theory.²⁰ We used a priori codes of barriers, facilitators, and sustainable or not sustainable to ensure we captured these elements at the heart of questions about contributors to partnership sustainability that guided this project. We then used open coding to identify additional relevant codes from the focus group data, iteratively reviewing text and holding meetings of the analysis team to develop a coding structure. This coding structure consisted of the 4 aforementioned a priori codes in addition to 10 additional codes capturing descriptions of participants’ past experience, characteristics of partnerships described, and corresponding funding mechanisms and communication methods. Table 1 describes the coding structure and associated code definitions used in this analysis.

Table 1.

Coding Structure and Code Definitions

Code	Description
Barriers	Factors that challenged or prevented partnership sustainability.
Facilitators	Factors that supported partnership sustainability.
Sustainable	Participant’s description of whether and how the partnership they are describing was sustainable and the form of sustainability (partnership existence, relationships of individual or organizational members, effects/impacts of activities).
Unsustainable	Participant’s description of whether and how the partnership they are describing was unsustainable.
Participant characteristics	Background information and individual characteristics of participants, such as sector represented (community, academic, multiple sectors) and years of experience.
Partnership operations	Structural, functional, and/or operational aspects of the partnership, including decision-making processes, evaluation of efforts, formal agreements, defined roles, and diversity of membership.
Power dynamics	State of power dynamics within partnership (equal or unequal power in relationships of members of various sectors), factors contributing to power dynamics, effects of equal or unequal power dynamics.
Communication	Methods and types of communication across the partnership, including formal communication (eg, regular conference calls or emails) and dissemination of results of efforts.
Mindset, motivation, personal value	Mindset and personal values that individual participants bring to the partnership, sometimes their motivation for ongoing participation. Often includes explicit description of commitment to partnership efforts and/or the relationship. Participant’s personal history may contribute. In some cases, institutions recognize the importance of a particular external value at the organizational level, which may contribute to institutionalization.
Funding mechanisms	The particular type and process of partnership funding; the role that funding plays in partnership sustainability.
Relationships	Description of relationships between individual and organizational partners across the partnership life cycle — including beginning partnerships, maintaining relationships, evolving relationships — and an individual in a bridge/connector role that brings different people or organizations together.
Capacity-building	Building capacity within individuals, person-to-person, and at the partnership/organizational level. Often involves reciprocity across partnership members, in which partners are each able to bring something to one another that they otherwise wouldn’t have access.
Institutionalization	Capacity-building in terms of organizational systems and infrastructure, beyond the individual level, so that sustainability is not dependent on having a specific person in a particular role. Sometimes described as creating a legacy with partnership efforts or accomplishments.
Social determinants / External factors	External factors and determinants, out of individual’s control, that affect participation, such as geography.

Upon developing a stable set of codes and consistent understanding of code meanings across the analysis team, we applied those codes to relevant text of all transcripts. Once all transcripts were coded, we reviewed the text associated with each code to identify themes across similar responses in each category, paying particular attention to facilitators and barriers to partnership sustainability. The analysis team periodically met with the larger project team to solicit feedback on emerging themes and interpretation. To avoid overburdening participants, transcripts were not provided back to participants for review and comment; however, we used member checking to support validity²³ by presenting initial findings to discussion participants and incorporating their feedback into the final summary report.

RESULTS

Participant Characteristics

PCOR Sustainability Panel members represented varying sectors: 3 worked in community, 3 represented academic institutions, and 4 represented both community and academic sectors. The panel consisted of 3 male and 7 female members; 5 were employed by a university and 5 were not. All 10 panel members were local to Colorado and represented the West region of the United States because phase I took place at the project location of Denver, Colorado, and participants were highly encouraged to attend in-person.

In phase II, a total of 29 individuals participated in 9 small focus groups. About two-thirds of discussion participants reported representing academic or research sectors, while just over one-third were patients, community members, or other stakeholders. More than half did not have experience working at a university. Most had extensive experience in community engagement of more than 15 years. Participants were geographically located in the U.S. Midwest, West, South, or Northeast. See Table 2 for full characteristics of PCOR Sustainability Panel and focus group participants.

Table 2.

Number of Phase I and II Participants by Region and Sector

Participant characteristic	Phase I participants, n (%)	Phase II participants, n (%)
Total	10 (100%)	29 (100%)
Sector
Academic, research, or project lead	2 (20%)	18 (62%)
Patient, community, or stakeholder	2 (20%)	11 (38%)
Combination of above	6 (60%)	0 (0%)
University affiliation
Yes	5 (50%)	12 (41%)
No	5 (50%)	17 (59%)
Years of experience
<5 years	4 (40%)	4 (14%)
5 to 15 years	1 (10%)	3 (10%)
>15 years	5 (50%)	22 (76%)
Region
Midwest	0 (0%)	3 (10%)
West	10 (100%)	10 (34%)
South	0 (0%)	7 (24%)
Northeast	0 (0%)	9 (31%)
Gender
Female	7 (70%)	17 (59%)
Male	3 (30%)	12 (41%)
Race/Ethnicity
White	4 (40%)	11 (38%)
Hispanic/Latino/a	2 (20%)	7 (24%)
African American	1 (10%)	6 (21%)
Other race/ethnicity	3 (30%)	5 (17%)

Defining “Sustainability”

In phase I, PCOR Sustainability Panel members wanted to identify a shared definition of sustainability before discussing other specific topic areas to guide focus groups. They first reflected on a definition from the U.S. Department of Health and Human Services, which conceptualizes sustainability as program continuation, institutionalizing activities, creating a legacy, or maintained impacts.²⁴ Panel members adapted this definition by discussing additional important elements and specificity in an iterative process, eventually reaching consensus around a shared definition of partnership sustainability that acknowledged the wide range of forms that sustainability might take. Box 1²⁵ displays the panel’s shared definition of sustainability for PCOR partnerships.

Box 1. Definition of Partnership Sustainability by the PCOR Sustainability Panel.

There are many different ways of defining sustainability. Sustainability may include:

• Creating a legacy through continuing organizational ideas, policy change, memory through mentoring, and intergenerational thinking through dialogue

• Shared core vision

• Trust

• More than just funding

• Strengthening existing relationships or partnerships, including subgroups or other partners, and through shared power

• Continuing programs, services, activities or impacts

• Integration of systems and services

• Evaluation and maintenance of consistent outcomes

• Developing adaptability, flexibility

Focus group participants reacted to the definition of partnership sustainability compiled by panel members in the previous project phase (Box 1). Participants did not come to consensus around a single definition of sustainability for PCOR partnerships. Thematic analysis indicated that participants generally agreed strongly with particular aspects of the panel’s definition. However, the combination of these components considered most important often differed from person to person. Participants suggested that the relative importance of components of the sustainability definition depended on the context of the partnership.

Facilitators and Barriers to Partnership Sustainability

In phases III and IV, we identified the following common facilitators of sustainability in PCOR partnerships: investing time to develop relationships in partnerships’ preliminary stages; shared motivation for participating in the partnership; a “connector role” to facilitate cross-sector collaboration; equal power dynamics in partnership operations; reciprocity between partners from different sectors; and institutionalization of the partnership or its goals. Conversely, we identified the following common barriers that hindered partnership sustainability: external factors that affect partnership participation and operations (ie, history, organizational structure); the conflicting role of funding; and lack of institutionalization. The next sections describe these facilitators and barriers to partnership sustainability in more detail.

Facilitators to Partnership Sustainability

Investing Time to Develop Relationships at Beginning of Partnership

Participants indicated that their PCOR partnerships initiated in several different ways, including those spurred by a funding opportunity, generated through a connection from an individual working in both community and academic sectors, or built on historical institutional relationships. Participants described how substantial initial time investment in the development of relationships among individual and organizational partners helped facilitate long-term sustainability. Participants highlighted the importance of investing time early in the relationship for members to get to know each other and identify points of common interest to achieve sustainable relationships. This was in contrast to experiences that were spurred primarily by a funding opportunity and focused only on funder objectives. Without developing and investing in personal connection, these “business-only” partnerships were more likely to dissolve over time.

Shared Motivation for Partnership Participation

Having shared motivation across individuals or organizations in partnership or project goals was associated with sustainable partnerships. These shared motivations may include shared personal experiences such as their own or a family members’ health conditions but also broader motivations such as commitment to long-term collaboration, vision of creating a legacy, or importance of equal power dynamics across partnership members. Participants from all sectors described the importance of entering into new partnerships with an explicit commitment to long-term relationships and collaborating over time. These participants demonstrated the mindset that the partnership’s ultimate goals and vision would persist over time, beyond any funding that may or may not exist.

“Connector Role” to Facilitate Cross-Sector Collaboration

Participants described the value and importance of an individual acting in what they called a “connector” or “bridge” role across the various sectors represented in a partnership. Individuals in this connector role worked across multiple sectors, generally both academic and community, to initiate and develop relationships. Connectors had commonly resided in the community or collaborated with partner organizations on a long-term basis prior to the partnership in question. Connectors engaged with partnerships organically through their attendance at partnership events or recommendations for recruitment from existing members, rather than being recruited specifically for the role. The connector’s involvement built goodwill and facilitated the building of trust to both sides of the partnership. Critically, the connector role was identified as especially important in attempting to address historical distrust between community members and academic partners. The value and trust that an individual in a connector role contributes to partnership sustainability was equally emphasized by participants representing patient or community as well as academic and research sectors.

Equal Power Dynamics in Partnership Operations

Prioritizing and striving for equal power dynamics across partners and sectors was identified to be critical for facilitating partnership sustainability. Transparency was highlighted as a key tactic in working toward equal power dynamics. To this end, participants described utilizing transparent communication practices, such as honestly communicating funding requirements, timelines, and limitations and sharing feedback on funding applications regardless of the outcome. In addition to transparency in communication practices, participants described the importance of transparency in terms of clear intentions, roles, expectations, and goals for future partnership efforts. Additionally, to support equalizing power dynamics, academic and research sector representatives reported efforts to allow community members and patients to truly take leadership in partnership operations by deferring to their knowledge and preferences in decision-making. Participants described data collection and sharing findings in formats that would resonate with community members and researchers alike by exploring less traditional methodologies, such as photovoice. Other participants described using physical space and location symbolically as a strategy to shift the power balance more equally toward patients and community members. Some partnerships intentionally held meetings in community spaces, particularly in the early stages of the partnership, to engage stakeholders in a manner that felt comfortable to community members and patients.

Reciprocity Between Partners From Different Sectors

Relationships described as sustainable often included reciprocity between partners representing different sectors, often between community members and university representatives. Reciprocity was frequently defined as each party sharing resources and providing access to benefits they wouldn’t have had otherwise. This included program evaluation training, accounting and financial services, and staff support. Creating opportunities for everyone to benefit fostered a culture of reciprocity to help ensure that different members received similar value from partnership involvement. Reciprocity was less commonly discussed than other themes but was a notable contributor to sustainability for some participants.

Institutionalization of Partnership or Its Goals

Participants described institutionalization as establishing organizational systems and infrastructure to support the partnerships or its goals beyond the individual level. This helps ensure that sustainability is not dependent on having a specific person in a particular role. Some referred to this as creating a legacy through partnership efforts or accomplishments. Institutionalization took such forms as a parent organization funding partnership activities upon completion of external grant funding or demonstrating a culture that values community-academic collaboration by allowing dedicated time for community work and relationship-building. Institutionalization was often achieved at the same time that organizational leadership members acknowledged the value of partnership efforts and subsequently took action to dedicate resources, time, money, or infrastructure to sustain the partnership or its work.

Importantly, many attendees of the Conference on PCOR Sustainability felt that the word “institutionalization” held negative connotations that did not perfectly express this theme, though they agreed with its sentiment. However, participants did not come to consensus around a single alternative term over the course of several discussions. Suggested alternatives to the word “institutionalization” were “systemizing,” “systematizing,” “succession,” and “normalization.”

Barriers to Partnership Sustainability

External Factors Influencing Participation and Operations

External factors outside of PCOR partnerships’ control affected individual participation in addition to partnership operations as a whole. Geography and history influenced partnership members’ ability to participate in meetings and activities. Participants described how geography and transportation in some areas influenced participation, particularly for community members and patients; the lack of public transportation or large area represented by some communities made it difficult to identify a central and accessible venue for all to attend partnership meetings.

The negative legacy of unethical research and historical trauma further affected partnership participation. Community members and patients sometimes hesitated or declined to engage with partnerships, citing the potential for harm as inflicted on marginalized populations through research in the past. Community members, patients, and similar stakeholders generally discussed the need to consider, acknowledge, and incorporate these external factors influencing participation more extensively compared to academic or research sector representatives.

Beyond the impact on participation, such external factors also hindered partnership efforts and sustainability more broadly. Misalignment between partnership efforts and local or governmental agencies’ existing organizational structure and policies impeded some partnerships’ progress toward goals. Structural factors, such as the state of communication systems between community organizations and agencies, affected partnership abilities to implement activities and services. At times, the policies of large organizations or governmental agencies conflicted with or lacked flexibility to address community needs and wants, preventing true community input and affecting the feasibility of partnership goals. For example, one participant described attempts to improve communication between community-based organizations and local government in a formalized manner by sharing information on communities being served. However, governmental systems were technologically incapable of integrating the information collected by community organizations in terms of format and contents. This prevented any further progress on the effort to improve cross-sector collaboration to better serve the community.

Conflicting Role of Funding

Participants described being somewhat conflicted over the role of funding in partnership sustainability. Many acknowledged that a partnership cannot be successful by focusing only on funding as a form of sustainability. However, they also suggested that the ability for partnership work to continue in some way often relies or is heavily influenced by the ability to secure ongoing funding, either from an external funding agency or by being institutionalized by a member organization. Members of PCOR partnerships often must strike a careful balance between their mission and the pursuit of continuing funding for long-term sustainability.

Lack of Institutionalization

Some participants described the inability to convince their organizations at large of the value and importance of partnership efforts. These participants described unsuccessful attempts at partnership institutionalization after they approached their organizations and were denied support at the completion of external funding. This threatened partnership sustainability and, in some cases, resulted in dissolution of the partnership.

Tables 3 and 4 display select illustrative quotations for each of the themes identified as facilitators (Table 3) and barriers (Table 4) to sustainability in PCOR partnerships.

Table 3.

Common Themes in Facilitators to Partnership Sustainability and Corresponding Illustrative Quotations From Focus Group (Phase II) Participants

Theme	Theme description	Illustrative quote(s)
Investing time to develop relationships at the beginning of the partnership	Substantial initial time investment in the development of relationships helped facilitate long-term sustainability.	“We asked some really deep questions, and so we got to know each other. We built friendships…. We all had different backgrounds, and we all represented different parts of the community. Usually these people that came together we would not — our lives would not have intersected anywhere other than here ... so I think that’s a part of the sustainability is having people that understand each other at a, like, real inside part of your soul kinda level.” [Participant 1B]
Shared motivation for partnership participation	Shared motivation for joining and contributing to partnerships across individuals or organizations in partnership or project goals was associated with sustainable partnerships.	“… acknowledging each other’s context and being kind of honest and open about that so that really what happens over time, 4 or 5 years, was a real sense of shared ownership. This isn’t about the system anymore, protecting children from bad families, it was about the system and families and community working together to benefit children, and that was a kind of paradigm shift that happened because of these ongoing difficult and humble conversations.” [Participant 5A]
A “connector role” to facilitate cross-sector collaboration	Individuals in this connector role worked across multiple sectors, generally both academic and community, to initiate, develop, and maintain relationships.	“I think you have the folks in the community that are, they’re born there. They’re raised there. They marry, they have their kids. They’re active in the community. They own their community, and this person that came to us, who’s just a force to be reckoned with, had enough sense to go, ‘Here’s other people who are doin’ the work, and I just need somebody to mesh us together and make us all realize we’re all trying to do the same thing instead of competing.’” [Participant 7B] “I was excited about bringing my colleagues into the community group that I was familiar with. I mean, that is just what I do … it’s my passion. It’s what fuels me. So, for me, I was really excited to get this goin’ ‘cause my intent is always, with people, that it will continue even if I’m not a part of that partnership anymore, but that they will continue. So I like to really build, like, this network of partnerships … [and] I was really excited to introduce them to sort of what I do on my side.” [Participant 1A]
Equal power dynamics in partnership operations	Prioritizing and striving for equal power dynamics across partners and sectors was identified to be critical for facilitating partnership sustainability.	“I also think that the sustainability was because we were treated like equals at that table, even though there was MDs at that table and there were PhDs at that table.” [Participant 1B]
Reciprocity between partners from different sectors	Relationships described as sustainable often included reciprocity in the form of shared resources and access to benefits between partners representing different sectors.	“That’s, to me, kind of one of the critical elements of sustainability is creating and facilitating win-wins for the partners that are involved in the process.” [Participant 8C]
Institutionalization of the partnership or its goals	Establishing organizational systems and infrastructure to support the partnership or its goals beyond the individual or group level.	“These aren’t just the relationships between the two partners. It’s also their institutional relationships ‘cause people come and go, but you really have to get the buy-in and the support of the larger institutions.” [Participant 4C]

Table 4.

Common Themes in Barriers to Partnership Sustainability and Corresponding Illustrative Quotations From Focus Group (Phase II) Participants

Theme	Theme description	Illustrative quote(s)
External factors influencing participation and operations	External factors outside of partnership’s control that affected individual participation and partnership operations, such as geographic distance and historical trauma related to research participation.	“What has helped us in regard to continuing relationship is some critical awareness of the forces that are making things hard for people. Concepts of shared social justice … there is a broader picture when you’re interacting with populations who have been on the social exclusion historical experience, you need to include, and respect, and instill that it’s a systemic challenge that we face together, and that we want to change and not just some sort of Band-Aid.” [Participant 9E] “When we looked to put the community health workers into community organizations and have them pair with our health care partners in our clinics, they don’t have — they can’t process the data. They can’t do HIPAA-compliant management of the data. They can’t talk to the electronic health record systems, and those are the big barriers that our community-based organizations have.” [Participant 2D]
Conflicting role of funding	Acknowledgment that partnership sustainability depends heavily on the ability to secure ongoing funding, but partnerships cannot make an impact on patients’ lives and health by focusing only on funding as a form of sustainability.	“Yes, it is more than just funding, but you always have to remember that funding is a part of the partnership sustainability, so is it joint funding or joint looking for funding or is that based on one of the partners is going to work on funding and the other ones are just a part of the partnership? Because even though we don’t want to say that it’s based on funding, really, the projects are based on funding whether we like it or not.” [Participant 1B] “The work is our vision, and we’ll find a way, and somehow, we did [laughs]. But it was always something and remains looming in the periphery about whether we can make long-term goals or should be focused on what, you know, funding there is.” [Participant 9E]
Lack of institutionalization	Inability to secure ongoing funding from parent organizations, sometimes leading to dissolution of partnership.	“I think that [the community health worker model] is a piece that institutions have really not gotten, even though it’s been around for so many years and it’s been used in Latin America, Central America, and China, I mean, just hundreds of years that this model’s been out there. So, for me, that was a failure in sustaining partnerships and something that I felt was so important, and it just crushed my heart to know that at this institutional level it wasn’t important enough to sustain in any form, even the partnership piece of it, afterward.” [Participant 2B]

DISCUSSION

Findings highlight the positive influence of careful, intentional, ongoing relationship development, members’ shared motivation, a “connector role” facilitating communication and trust across sectors, striving for equal power dynamics, cross-sector reciprocity, and institutionalizing partnership activities and achievements on partnership sustainability. In contrast, findings also highlight the tangible challenges that members of research and health promotion partnerships face, such as reliance on external funding opportunities for ongoing progress.

The lack of a single common definition of sustainability for PCOR partnerships is consistent with the variety of definitions identified for program sustainability.^13,17 This diverse range of definitions for sustainability reinforces subtle differences in the factors that contribute to sustainability across disciplines and purposes, highlighting the need to incorporate facilitators specific to sustaining PCOR partnerships. The definition’s resemblance to a list more so than a single concise definition may reflect participants’ reluctance to disagree with or omit elements that other participants found important, and this wide-ranging definition may have contributed to focus group participants’ inability to agree on a single description of partnership sustainability. A more rigorous consensus-building approach from facilitators of the discussion may have proved more beneficial in this regard.

These findings on PCOR partnership sustainability are consistent in part with evidence on program sustainability from public health practice and other health research areas. Our findings that relationship development, equal power dynamics, reciprocity, and institutionalization facilitate PCOR partnership sustainability overlap with elements of the CBPR evidence base.^26,27 However, our findings indicate greater importance than previously identified of a connector role to support relationships and early investment in long-term relationship development. Given the need for ongoing collaboration to sustain evidence-based practices to improve health,¹⁰ supporting partnership sustainability through an overall focus on initial and ongoing relationship development may be a critical component of getting the maximum benefit out of research findings and corresponding health promotion programs.

The “connector role” is particularly novel to PCOR, though similar concepts have emerged in adjacent disciplines. A connector facilitates relationship development across sectors in the partnership or the larger community, particularly for subpopulations that have experienced harm through imposition of unethical research practices. This contrasts with community health workers who link community members with the health care system to improve health outcomes²⁸ or community coalition champions who contribute broadly to awareness, relationship development, and promotion of the partnership.²⁹ While the connector role tended to emerge organically rather than being intentionally integrated as a formal role from the project’s beginning, deploying a connector role has potential as a strategy to foster challenging, less tangible facilitators to PCOR partnership sustainability, such as trust and strong relationships. Clinical and health services researchers should consider how to intentionally and meaningfully incorporate project team members and partners to fulfill this connector role, particularly when trying to collaborate with populations who are historically underrepresented in PCOR or who experience the brunt of inequitable health outcomes.

For funding organizations, fiscal mechanisms should be designed to support the time necessary to create sustainable and impactful research partnerships. This may include preaward support for partnership development, longer funding timelines, and no-cost extensions to account for unanticipated delays. Funders should incorporate mechanisms to directly support time and efforts of the connector role, even if that role does not align with traditional personnel expenses.

Limitations

There are several limitations of these findings. This project was designed to convene stakeholders with experience in PCOR, distinct from community-engaged research as a whole or other fields of inquiry. Themes result from numerous projects with a variety of funding sources, topic areas, member demographics, and geographic locations. The extent to which participants are representative of the larger group of PCOR practitioners and participants, or the generalizability of findings to specific types of research partnerships, is unclear. Participants’ retrospective assessment may introduce recall bias regarding sustainability or the relative influence of facilitators and barriers. Given the nature of recruitment methods, which included flyers and listservs, it was not feasible to obtain a reliable number of individuals approached or to describe those who declined to participate, which could have offered additional context as to participant representativeness and generalizability of results.

Due to scheduling limitations, some focus groups were fairly small and consisted of 2 or 3 participants, which may have limited the degree to which participants could build off of one another’s responses and generate new insights on the topic of partnership sustainability.

CONCLUSIONS

Patient-centered outcomes research partnerships and research collaborations more generally should incorporate an early and ongoing focus on relationship development through intentional efforts to collaborate with specific partners and stakeholders according to the goals of the research. Ensuring support for wider and more sustained health improvements would allow more patients to access the evidence-based practices resulting from these research investments, which is a primary goal of PCOR partnerships.

Patient-Friendly Recap

• Many health research projects and findings demonstrating new ways to improve care never get fully implemented or made available to all patients.

• Though difficult to achieve, a sustained partnership among patients, researchers, and clinicians — often from multiple organizations — leads to wider adoption of patient-centered advances.

• The authors queried those involved in patient-centered outcomes research (PCOR) to identify facilitators and barriers to sustaining these multistakeholder partnerships long term.

• Early and ongoing relationship building, treating patient partners as equals, and aligning partner collaborations to the goals of the research are keys to sustained success.