The costs of partner engagement in research: Opportunities for pragmatic research, implementation science, and the science of engagement

Abstract

There is growing emphasis on the importance of active partner engagement in research and the design, planning, and implementation of new programs. Indeed, the science of engagement is gaining is receiving more attention from researchers, being formally included in research studies more often, and increasingly being required by funders. Financial costs and economic implications are well-known drivers of successful adoption, implementation, and sustainment of evidence-based programs; however, partner engagement costs are often excluded from economic evaluations even when elements of engagement may be relevant to sustainment and replication. Resource constraints in both research and practice highlight the need to better understand engagement costs to ensure that scarce resources are put towards the richest engagement activities. As part of the Rapid and Rigorous Patient-Centered Program (R2P2) in Colorado, we designed an approach to systematically document and analyze partner engagement costs across four prototype projects. We describe this approach, summarize needed adaptations, share an interactive data collection template, synthesize the relevant engagement costs during implementation, replication, and sustainment, and discuss lessons learned. Our experiences in R2P2 in costing initial engagement activities across four diverse projects offer direction for future research and practice that balances high-quality partner engagement with efficiency in addressing resource constraints facing researchers and practitioners.

Introduction

Active and equitable partner engagement have become priority issues in the field of implementation science. The science of multi-sector engagement, including methods development in community-based participatory research, has grown extensively over the last two decades [1–4]. Engagement in research has been defined as “meaningful involvement and partnership of stakeholders throughout the research process – from planning and conducting the study to ultimately disseminating study results.” [5] Active and meaningful engagement of community members, patients, individuals with lived experience, health delivery organizations in which many interventions are tested, front line and middle management staff, and policymakers (i.e., partners) is recognized as important for successful implementation and replication of research in community settings [6–8].

Partner engagement is a core aspect of pragmatic and implementation research to ensure its relevance to multiple partners, including those delivering a program and those making decisions about replication and sustainment. Researchers use multiple strategies to engage their partners, including conducting focus groups, surveys or interviews to document partner perspectives, formally incorporating partners into the study team, creating advisory groups, and other targeted presentations and outreach efforts [9]. Partners play a meaningful role in the design or adaptation of an intervention (including through co-creation, identification of outcomes and metrics, recruitment and retention of participants, and various other aspects of the research and dissemination process) [10].

Partner engagement in research fulfills several critical goals. First, it provides a process through which researchers can better understand the context of the environment in which an intervention or program is to be implemented. Engagement efforts should include community and organizational leaders and community members who have experienced challenges to accessing needed care, including functional, financial, and policy-driven barriers. Understanding this context from the perspectives of those who may implement a program, those who have delivered or received similar programs in the past, and those who may benefit from it is critical to designing interventions that are both feasible and acceptable.

Second, thoughtful engagement can enhance the fit of an intervention and implementation strategies to local context and partners can inform adaptations that may be necessary for successful implementation. Co-creation among partners, as a type of active and ongoing engagement, can help researchers meet those goals [6]. Active partner engagement can also foster trust and support the adoption, replication, and sustainment of programs.

Like everything, engagement in research and program implementation has associated costs. While there is acknowledgement that engagement takes significant resources [11], particularly the time people invest in recruitment of partners, maintaining ongoing communication, and preparing them for an active engagement, there is limited empirical work on the realized costs of engagement to those designing and implementing new interventions and to those they engage. Further, as the science of engagement expands, it is important to understand the costs and cost-effectiveness of different engagement strategies to ensure maximum value and efficiency of engagement.

Time, costs, and other economic-related concerns are of importance to community partners and patient populations, and the fields of health services research, pragmatic research, and implementation science face a growing demand to assess program costs and to communicate those to relevant stakeholders and the field [12–15]. Several studies have conducted micro-costing evaluations to this effect [16–19], including the HEALing Communities Study, which reported community costs of over $63 million to implement evidence-based practices across four states [20]. However, to our knowledge, none have examined the costs of engagement in research.

To date, most implementation science and pragmatic research have focused on the costs of program delivery [21,22]. Often the engagement component of this work is considered a research or planning cost, which is traditionally excluded from economic analyses. But engagement is not just a research cost and is likely a function (or implementation strategy) in implementation science and pragmatic research that extends far beyond an initial study. For full intervention fidelity and presumably successful outcomes, engagement to at least some extent will need to be done with other settings, organizations, populations, and communities wanting to adopt a program or practice.

Strengthening our knowledge of the economics of engagement serves two purposes. First, it helps further the theory and evidence within the field of implementation science by understanding the costs associated with successful engagement and can help design programs that are replicable, acceptable across diverse communities, sustainable, and efficient in terms of costs and benefits. Second, recognizing that research funding is constrained, we can contribute to the science of engagement by measuring and quantifying, to the extent possible, the cost-effectiveness of different engagement strategies, which may inform decisions about engagement strategies in future research. Thus, the development of a broadly applicable, pragmatic costing tool that can support these team efforts is critical.

The purposes of this paper are to: 1) conceptualize the costs relevant to multidisciplinary and multilevel partner engagement in research; 2) provide examples of variations in initial engagement approaches and engagement costing from four diverse projects funded as part of an innovation research center in Colorado, and 3) discuss key considerations and the potential contributions of this type of work to implementation science and the science of engagement.

Materials and methods

In early 2024, a research team at the University of Colorado was awarded a large internal program grant to support research focusing on translational and pragmatic, patient-centered methods to transform care and care delivery. The Rapid and Rigorous Patient-Centered Program (R2P2) is one of nine Anschutz Acceleration Initiative Awards at the University of Colorado Anschutz Medical Campus and is designed to impact the health of Coloradans within two to three years. R2P2 currently supports four pragmatic research projects, all employing implementation science methods and a focus on enhancing patient-centered care. It will fund several more in coming years and is producing and iteratively evaluating a series of resources and guidance documents to support patient-centered projects in accomplishing their goals in ways that are both rapid and rigorous.

One key component of the R2P2 program is partner engagement in research planning, implementation, evaluation, and dissemination. The R2P2 program provides support to funded projects through resource development and consultation with experts in engagement and economic evaluation issues, among other areas of technical expertise. Teams are encouraged to incorporate partner engagement, through structured approaches such as advisory boards and co-creation partner sessions, into the design, implementation, and evaluation aspects of their studies. They are also provided with one-on-one consultation sessions with our engagement team leads (MPJ and SR), tailored materials such as worksheets for tracking engagement activities, evidence-based resources (including the 7P framework [23]), and access to a Spanish-speaking advisory panel comprised of patient and community members with lived experiences). Through these projects, the overall R2P2 program seeks to understand approaches that optimize engagement outcomes related to partner engagement.

Teams are also supported with resources and expertise related to economic evaluation by the R2P2 leads (LC and JC). These two team leads (engagement and economic evaluation) met several times to discuss an approach to developing an engagement costing tool. Principles of Community-Based Participatory Research [24] and an already field-tested engagement tracking tool [25] informed the first spreadsheet prototype. This initial prototype was revised several times after being presented to the R2P2 teams, through several feedback loops, before being pilot tested by each team. The costing tool was developed with a focus on brevity, easy and efficient use, and flexibility to accommodate nuances from each team’s project. During the current pilot phase of the costing tool, all teams are incorporating measures of implementation costs that inform sustainment and adoption decision-making and some include more advanced analyses of the value of their intervention (e.g., cost-effectiveness).

Project teams, primarily the project manager, document their engagement activities and outcomes in a tracking spreadsheet, which is intended to facilitate analysis of the planning, implementation, and impact of engagement activities. The tracker is updated on an ongoing basis, usually once every quarter to accommodate project managers’ workflow while setting expectations for regular data documentation. In the same spreadsheet, project teams document information relevant to the costs of engagement, including partner role and perspective (e.g., physician, patient, community member, etc.), participant incentives paid, and engagement-related meeting costs (e.g., food, venue, travel, etc.). Table 1 summarizes the key costs included in the tracking template and their intended use in data analysis. The full interactive template is provided in Supplement 1.

Table 1.

Cost elements of engagement costing template

Table 1 long description.

Cost element	Use in analysis
Partner’s organization	Estimating hourly rate; characterizing cost variations
Partner’s role/profession	Estimating hourly rate; characterizing cost variations
Duration of the meeting in minutes	Estimating time costs
Main topic/agenda	Characterizing cost variations
Compensation/incentive provided to each partner	Estimating other costs
Total cost and description of any other resources used	Estimating other costs
Any relevant notes/context for understanding numbers/costs/activities	Context; characterizing cost variations

These partner engagement costs are used to estimate total and average per project costs and demonstrate variation across diverse projects and engagement approaches. Of note, we use the time a partner is engaging with a research team to quantify the opportunity cost of them being involved in the study instead of spending those resources elsewhere [26]. All analysis of engagement cost data is performed by members of our economic evaluation team.

In R2P2, we have adapted our approach to different project needs and contexts, while prioritizing a core set of data elements with which we are piloting our analysis of engagement costs and effectiveness. Guided by our experiences with this engagement costing approach during the planning phases of projects in the R2P2 program, the results section below summarizes an approach for considering and documenting the costs of engagement in implementation and pragmatic research and provides examples from the R2P2 program. We provide suggestions for other research on relevant engagement costs to track; likely outcomes of engagement; and considerations for others as they seek to measure and analyze the costs of engagement in research.

Results

As with many types of economic analyses, the costs and outcomes measured are informed by the perspective for which we are conducting the analysis [18]. In engagement costing, we argue that a broader scope is most relevant, given the substantial investment of time and resources made by external partners, including clinicians, patients, community members, organizational leaders, and policy decision-makers. We have identified costs relevant to both engagement in research and engagement in implementation and replication and provide examples in Table 2. As can be seen, we think it important to categorize these costs based on the phase of a study in which they are incurred. It is important to track engagement costs throughout all phases of a project, we only report data on only the initial planning phase in this report. The primary engagement costs incurred relate to participant time, partner incentives, and meeting-related expenses. However, roles of partners often change across the research phases. We have provided examples to further clarify the roles partners may plan as they are engaged throughout a study. Table 2 summarizes these costs.

Table 2.

Summary of potential engagement costs by research phase (note: not all research teams will have engagement in each phase)

Table 2 long description.

Phase of research or implementation	Engagement costs	Example(s) of engagement activities
Proposal preparation Pilot research	Planning time Initial outreach Group facilitator training	Time to identify and reach out to each partner/organization, and plan a recruitment approach
Establishing research question(s)	Participant time Participant incentives Meeting expenses	Generating feedback on research questions relevant to local community
Designing the intervention or program	Participant time Participant incentives Meeting expenses	Co-creation or co-refinement of the intervention or program Input on feasibility of and requirements for IT/EHR build-outs; approval of these Input on implementation strategies
Designing the study	Participant time Participant incentives Meeting expenses	Input on patient-centered outcomes Input on scope of usual care or comparator Input on implementation strategies
Recruiting and retaining participants	Participant time Participant incentives	Input on and co-development of patient recruitment given current practice workflows
Implementing the intervention or program	Participant time Participant incentives Meeting expenses	Feedback on local context relevant to implementation Ongoing involvement to contribute to necessary adaptations. Input on implementation strategies
Collecting and analyzing data	Participant time Participant incentives Meeting expenses	Interpretation of results
Disseminating the study findings	Participant time Participant incentives Meeting expenses	Distribution of study results to other community members, organizations, and clinical groups Presentation of findings from partner perspective and co-authorship on papers

In the R2P2 program, projects are currently at the planning or early implementation phase, and we have found great variation in engagement strategies undertaken and related costs. For example, some projects have electronic health record integration and have required significant time and engagement of IT committees and teams across multiple, diverse organizations with varying capacity and structures. One project regularly engages with a partner site at another university. We also observed differences in participant engagement incentives offered (range: $50.00 to $100 per partner hour) and accepted, efforts planning and preparing for engagement encounters, such as developing presentation materials, and approaches to recruiting engagement partners.

We also find that cost measurement depends on the context of the engaged partner(s), particularly if they represent an organization. For example, some large organizational partners such as health systems have standing advisory boards or committees that study teams often actively engage in their work. The project-related discussions are added to standing agendas and therefore do not have the same opportunity costs of time that may be real for ad-hoc meetings of partners or establishment of new groups. Both of these examples demonstrate the importance of a flexible but structured approach for documenting engagement costs, so we accurately capture engagement activities, their associated costs, and the reasons for observed variations. In addition, this variation informs sensitivity analyses that can help decision-makers considering sustaining or adopting programs under various assumptions and delivery options.

The preliminary analyses of initial engagement-related costs in R2P2 reflect these variations (see Table 3). We observe variations in costs of study team time doing engagement work, the time of external partners, and the participant incentives. The range of engagement costs related research personnel time is from $845 to $12,962 across the first 18 months of the projects. Total partner incentives also vary ($1250 to $7000) as do total direct costs ($2095 to $23,475) and opportunity costs for partners ($2968 to $20887).

Table 3.

Summary of variations in preliminary engagement costs across four projects

Table 3 long description.

	Minimum	Maximum	Median	Mean
Research personnel time costs	$ 845.27	$ 12,962.41	$ 1696.09	$ 4299.97
Partner compensation (incentive) costs	$ 1250.00	$ 7000.00	$ 5130.00	$ 4627.50
Total direct project costs	$ 2095.27	$ 23,457.52	$ 7221.09	$ 9998.74
Partner time/opportunity costs	$ 2968.61	$ 20,887.13	$ 7045.75.65	$ 9486.76

Importantly, all projects have at least half or more of their total costs identified as opportunity costs of their external partners. In some cases, these costs are only partially offset by the direct incentives paid to partners. This could mean that partners did not accept incentives offered, that the incentive does not adequately compensate for partners’ time, or that there were no incentives offered. The latter is an important cost consideration when teams engage standing bodies or groups in which there is no expectation of an incentive payment (e.g., a standing IT review committee in a health system).

One key question that arose in our work in R2P2 is “which subset of engagement costs are most important to replication and sustainment as implementation outcomes”? It likely varies by intervention or program needs and resources, but it is likely that there are several engagement activities and related costs that may be most emphasized during sustainment and new replications or scale-up of a program. Of note, specific research-related engagement costs are excluded here (e.g., obtaining IRB approval and consent, establishing and implementing data collection systems, etc.) as they are unlikely to be ongoing costs for either replication or sustainment outside of research projects. However, even some of the engagement costs that are often considered research costs would be necessary during sustainment and replication and therefore should be included [21,22]. For example, recruitment, data collection, and dissemination remain important for many communities and organizations that must continue to demonstrate the value of a program to funders and leadership, and ongoing multi-sector partner engagement both internally and externally may be an important function in successfully fulfilling these activities. Figure 1 summarizes the engagement activities relevant to research, program delivery, sustainment, and replication.

Figure 1.

Figure 1 long description.

Application of engagement-related activities to research, program delivery, sustainment, and replication.

Readers may note the addition of adaptations to Figure 1. We include this as a design-related activity where partner engagement can be more intense and, if partner-engaged adaptations are made during sustainment or replication, there will be engagement-related costs incurred [27,28].

Discussion

Sincere, multisector partner-centered engagement is important for success in research and implementation [6–8]. However, this critically important activity takes resources, including time and money, that are then not available for other purposes. With few exceptions, these costs are seldom reported – and if so generally quantify resources (e.g., time) but do not estimate financial costs [16]. We argue that more transparent collection and reporting of the estimated costs of engagement are important for three reasons: 1) to better inform budgets for implementation and pragmatic research and for replication and sustainment of effective programs, 2) to advance the science of engagement, which rarely incorporates costs as empirical measures or outcomes, and 3) to facilitate the use of value assessments (costs relevant to benefits) in partner engagement.

There are several key considerations for and lessons learned in engagement costing in multiple settings. First, engagement approaches depend on partner types and the purpose of the partnership. By design, partner engagement is considerate of partner needs and motivations for engaging. Financial incentives can honor the time people volunteer for a study; but incentives may not be a primary motivator, or even allowed, for someone who is driven primarily by the mission alignment or organizational rules. The opportunity costs of time are important to incorporate since most partners take time away from other things to engage in research or implementation. However, monetary opportunity costs will vary greatly based on who is engaged (e.g., hospital CEO, chief of surgery, primary care medical assistant, retired life experience committee member) and how their time is valued financially (e.g., actual wage data versus using wage estimates from public sources). We recognize that applying a financial cost to time is a societal value judgment and does not imply anything about the relative importance of member contributions from different partners. We do feel it important to include these engagement participation costs, but to do so as opportunity costs, rather than direct intervention costs, so it is clear these are important to consider as costs regardless of who bears that cost directly.

Second, there are nuanced and context-dependent considerations, including the history of collaboration, power dynamics, expectations of partners, and organizational and community culture and beliefs. For example, partner engagement is prone to repeated invitations to the same partners and risks tokenism that may diminish the voice of some partners [29]. Being intentional about recruiting from and ensuring diversity in perspectives of partners is important, even if more time consuming and costly.

Third, researchers must be open to flexibility and adaptations in partner engagement, some of which may be driven by resource constraints in program funding. These adaptations will influence associated costs in various ways but research teams should be careful of both 1) over-considering the costs and under-considering the value lost by not effectively engaging and 2) not making most effective use of limited resources if an activity is not necessary or does not produce incremental value. Additional research in this area may highlight the most impactful and efficient engagement activities.

Finally, there are important considerations around research- and program delivery-related costs compared to the engagement-related costs of 1) sustaining or 2) replicating a program (Figure 1). Implementation science emphasizes sustainment and replication or scale-up as key outcomes [19,21]. As with most economic analyses, engagement costing should consider the role of engagement activities that address these key outcomes. As noted in the figure, both sustainment and replication actions should include a transparent summary of results [19], including costs (with estimated or different types and levels of engagement and possible adaptations with relevant organizational decision-makers).

Considering these issues, and collecting data on them, can help to understand the ways in which partner engagement can produce the best outcomes most efficiently. We encourage research teams to adopt a broad perspective when conducting economic evaluations of engagement in research. Although the value of partners’ time may not be a line item in a budget, even with incentives research partners could be doing something else with their time and that has real value in their own decision-making about serving as a research partner. This approach helps us to understand whose costs and what types of costs may influence the success of partner engaged research.

To answer these important scientific questions, the field needs an agreed upon framework and ideally standardized methods with which these engagement activities and associated costs are collected and reported. We do not propose that research projects should be doing more or less engagement; rather we suggest that we should do more to understand the costs and value of engagement, at different levels of intensity and through different strategies. Indeed, to further understand the value (costs relative to outcomes and benefits) of partner engagement, future large scale, multi-site research could study variations in engagement strategies and resultant engagement and implementation outcomes.

There are challenges and limitations of this work. A central issue is that determining a causal relationship between level and degree of engagement and longer-term outcomes is fraught with challenges and beyond the scope of this paper [27,28]. Because pragmatic research occurs in real-world settings, it lacks the controlled environment of more classic clinical trials. It will be challenging to attribute a certain percent of improvement in outcomes to engagement. There is also the issue of “whose costs for which outcomes under what conditions”? It is probably more reasonable to estimate the impact of costs on proximal outcomes such as partner perceptions of engagement or co-creation [2,28] or implementation outcomes such as reach and adoption [27,28] than trying to attribute more distal outcomes to engagement versus the myriad of other factors related to distal outcomes.

Our initial work on engagement costing in the planning stage of R2P2 projects is a starting point and a suggested foundation for growth and collaboration across scientific disciplines – pragmatic research, implementation science, economics, and partner engagement. We propose that these fields can come together to generate empirical evidence that helps balance limited resources, competing research priorities, and a need to move fast but also intentionally and with concerns for equity and necessary fit to local context [30]. Further, with a lens of longitudinal impact, increased attention to engagement cost data will advance the science of engagement as there is more work to tie together costs of and the benefits derived from engagement.

In conclusion, we think multi-sector engagement is critical for successful implementation, sustainment, and scale-up. We also think that there are limited time and resources available in any project and that researchers have a responsibility to be good stewards of these resources. Standard, transparent reporting of engagement strategies, costs, and outcomes will allow us to compare across diverse projects and interventions, identify where the primary costs of engagement lie. This in turn will help the field to consider opportunities to make engagement more efficient without compromising impact.

Supporting information

Creel et al. supplementary material

Creel et al. supplementary material

Table 1. Long description

Seven cost elements are listed, including: Partner’s organization, Partner’s role/profession, Duration of the meeting in minutes, Main topic/agenda. Compensation/incentive provided to each partner, Total cost and description of any other resources used, and Any relevant notes/context for understanding numbers/costs/activities. Each is linked to ways the element may be used in analysis, including: Estimating hourly rate, characterizing cost variations, Estimating time costs, and Context

Navigate back to Table 1..

Table 2. Long description

Engagement-related costs are crosswalked with eight phases of research, including Proposal preparation and pilot research, Establishing research questions, Designing the intervention or program, Designing the study, Recruiting and retaining participants, Implementing the intervention or program, Collecting and analyzing data, and Disseminating the study; Engagement costs allocated across phases include planning time, outreach, facilitator training, participant time, participant incentives, and meeting expenses; examples are also provided.

Navigate back to Table 2..

Table 3. Long description

Summary statistics (minimum, maximum, median, and mean) on engagmeent costs across four projects; costs are separated into four categories: Research Personnel Time Costs, Partner Compensation (Incentive) Costs, Total Direct Project Costs, and Partner Time/Opportunity Costs.

Navigate back to Table 3..

Figure 1. Long description

The venn diagram shows where there are overlapping engagement-related activities. Research is represented in the top purple circle, replication is in the right blue circle, sustainment is in hte left orange circle, and program delivery is in the center where all three overlap. The following activities are exclusive to research: establish research question, design the intervention, design the study. These activities are exclusive to replication: bring ideas to decision-makers and consider needed adaptations. These activites are exclusive to sustainment: feedback to decision-makers and feedback on budgeting. Overlapping all three in program delivery are: recruit and retain participants, implement the intervention, collect and analyze data, disseminate results, and adapt the intervention.

Navigate back to Figure 1..

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/cts.2026.10784.

Author contributions

Liza M. Creel: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Supervision, Writing – original draft, Writing – review & editing; John Cronin: Data curation, Formal analysis, Methodology, Project administration, Writing – original draft, Writing – review & editing; Daniel D. Matlock: Conceptualization, Funding acquisition, Project administration, Writing – review & editing; Russell E. Glasgow: Conceptualization, Funding acquisition, Methodology, Writing – original draft, Writing – review & editing; Samantha Rubio: Data curation, Project administration, Writing – review & editing; Monica Perez Jolles: Conceptualization, Supervision, Writing – original draft, Writing – review & editing.

Funding statement

The Rapid and Rigorous Patient-Centered Program (R2P2) and the research reported in this publication was funded and supported by the Anschutz Acceleration Initiative (AAI), a partnership between The Anschutz Foundation and the University of Colorado Anschutz.

Competing interests

The authors declare none.