Transforming healthcare by prioritizing qualitative and quantitative clinical trial evidence: evaluating the Aging, Community and Health Research Unit’s Community Partnership Program for Older Adults (ACHRU-CPP)

Kathryn Fisher; Soo Chan Carusone; Rebecca Ganann; Maureen Markle-Reid; Melissa Northwood; Diana Sherifali

doi:10.1186/s13063-025-08839-1

. 2025 May 13;26:154. doi: 10.1186/s13063-025-08839-1

Transforming healthcare by prioritizing qualitative and quantitative clinical trial evidence: evaluating the Aging, Community and Health Research Unit’s Community Partnership Program for Older Adults (ACHRU-CPP)

Kathryn Fisher ^1,^✉, Soo Chan Carusone ¹, Rebecca Ganann ¹, Maureen Markle-Reid ¹, Melissa Northwood ¹, Diana Sherifali ¹

PMCID: PMC12070794 PMID: 40361198

Abstract

Background

This study aimed to test the effectiveness and implementation of a complex integrated care intervention for older adults. We collected both quantitative and qualitative data, which is recommended in evaluating complex interventions to gain a comprehensive understanding of key success factors. Often, congruence is sought and considered desirable when integrating the findings from both data types. However, data are not always congruent, nor is it suboptimal when incongruence occurs, as we illustrate in this case study. We present the divergent findings from a large community-based implementation-effectiveness hybrid type II trial, and how the struggle to reconcile incongruent results yielded rich insights informing the next steps for translational research on the intervention being tested.

Methods

Previous foundational research, including a pilot study and randomized controlled trial (RCT), showed promising results and supported proceeding with a multi-site pragmatic hybrid type II effectiveness-implementation RCT. This recent RCT was undertaken and quantitative and qualitative data were collected to inform the effectiveness and implementation evaluation. To synthesize the findings and guide integration of this large body of evidence, we developed a conceptual model which combined two existing frameworks: the Consolidated Framework for Implementation Research and Quintuple Aim. We used this model to identify the evidence and relate it to relevant implementation and intervention determinants/outcomes. We then synthesized the evidence to distall the main messages regarding the future of the intervention, which involved reconciling apparently discrepant findings from the quantitative and qualitative approaches.

Results

The current RCT showed no statistically significant effect for participants for the primary (or secondary) outcomes yet the implementation evaluation consistently found perceived benefits of the intervention for patients, providers, and the healthcare system. Qualitative evidence was critical in understanding contextual factors potentially responsible for the absence of a treatment effect (e.g., COVID-19), strategies to overcome challenges experienced in participant engagement and intervention delivery, and recent policy/practice setting changes which showed strong alignment with the intervention and supported its future implementation.

Conclusions

With the goal of the hybrid type II effectiveness and implementation study in mind, stakeholders encouraged proceeding with a scalability assessment to consider the evidence from the current trial within the context of our prior research, the broader literature for similar interventions, and the ever-changing policy context.

Trial registration

clinicaltrials.gov NCT03664583. Registration date: September 10, 2018.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-025-08839-1.

Keywords: Effectiveness-implementation trial, Hybrid type II RCT, Pragmatic trial, Quantitative and qualitative data integration, Patient-oriented intervention, Older adults

Introduction

“Not everything that counts can be counted, and not everything that can be counted counts.” [1], p. 13).

The above quote challenges the notion that anything that cannot be readily quantified is valueless. It has been used as a cautionary remark in many fields including business, sociology, physics, statistics, and medicine. The architect of the randomized controlled trial (RCT), Sir Bradford Hill, surely had this in mind in stating that any belief that the statistical results from an RCT were the only form of evidence “would mean not that the pendulum had swung too far, but that it had come right off the hook” [2], p. 108). The Medical Research Council (MRC) framework, which assigns a central role to the RCT, underscores similar concerns and thus recommends other research methods such as qualitative approaches for evaluating complex interventions to explain unexpected findings and understand how context influences trial outcomes [3]. The emergence of effectiveness-implementation hybrid designs is another signal that an understanding of both healthcare intervention effectiveness and implementation are required to more readily move evidence into practice [4]. Calls for increased use of implementation research and mixed methods approaches that combine qualitative and quantitative data are also seen in the literature on the evaluation of integrated care interventions [5–7], to respond to the inconsistent findings seen across many RCTs [8–11]. The complex and dynamic nature of integrated care is thought to account for the disappointing findings from traditional RCTs, leading to recommendations for more multidimensional approaches that better capture how integrated care works and its implementation facilitators and barriers [7, 12], and to increase consideration of patient-important experiences and outcomes [13, 14]. Collectively, these (and similar) declarations serve as a cogent reminder that statistical trial findings are part of a broader collaborative creation that integrates the findings with clinical and contextual observations to inform healthcare decision-making and research uptake.

Yet, evidence hierarchies routinely place RCTs at the summit, with guideline developers and policymakers inferring “strength” in making causal claims from this dominant position and assigning prominence to quantitative RCT results in criteria used to grade intervention evidence. This elevated position of RCTs has been referred to as an “undeserved pedestal,” with calls for replacing the hierarchy with “accepting—indeed embracing—a diversity of approaches” [15], p. 580). Accepting diverse forms of evidence, while intuitively appealing, is challenging to implement in hybrid trials where both effectiveness and implementation data are collected because of the lack of clarity on how to integrate and synthesize these different types of evidence. Sparse guidance on how to merge the two types of data stem from a recognized gap in the mixed methods literature [16], and international working groups tasked with addressing this gap have yet to publish formal recommendations or guidelines [17, 18]. Even when all forms of evidence converge, “acceptance of the significance of context—that different people in different circumstances will tend to do things differently” implies that the evidence is relative and complicates efforts to render solid claims regarding effect, causality, and generalizability [19], p. 517). This challenge is further amplified when faced with divergent findings, which occurs in the majority of mixed methods research [20]. The extent of divergent findings in RCTs that employ mixed methods to evaluate complex interventions (including integrated care interventions) is difficult to estimate as it is rarely reported. This is likely due to the discrepancies being masked by the dominant post-positivist methods used in RCTs [21] and the tendency to analyze and publish quantitative and qualitative results separately with minimal effort at integrating the two [22].

This paper presents a case example of divergent findings from a large community-based implementation-effectiveness trial, the dilemma that these findings present, and how efforts to understand incongruent results yield rich insights that can inform the next steps for translational research. Alas, the dilemma we faced was this: an effectiveness and implementation study showed no statistically significant effect for participants for the primary (or secondary) outcomes despite previously demonstrated trial evidence of effectiveness, and yet the implementation evaluation consistently found perceived benefits of the intervention for patients, providers, and the healthcare system.

The community-based RCT discussed in this paper was part of a program of research with three phases:

Phase 1: population-based study using administrative data to compare high system users with general members of the target population (older adults with diabetes) on key socio-demographic, health status, and health service use characteristics;
Phase 2: a multi-site implementation-effectiveness type II hybrid RCT [4] evaluating a self-management intervention for older adults with diabetes and at least one other chronic condition,and
Phase 3: a scalability assessment of the intervention [23].

This paper is one of a series of publications resulting from the program of research. A separate manuscript has been submitted for publication pertaining to the results from the Phase 2 effectiveness evaluation [24], and the paper describing the results from the Phase 3 scalability assessment has been published [25]. A qualitative study exploring the experiences and perceived impacts of the intervention by older adults has also been published [26]. This paper provides a detailed discussion of the implementation evaluation findings from the clinical trial and their interpretation within the context of broader trial findings, which include the results from the effectiveness evaluation (briefly summarized in this paper). The case example presented in this paper is intended to illustrate an approach for combining and reconciling discrepant results, yielding a sum that is greater than the individual parts and helping to clarify critical next steps in the program of research. Rather than use findings as rationale for discrediting the results of either quantitative or qualitative analyses, we attempted to understand the circumstances under which such discrepancy (or discrepancies) could arise and then consider the implications of this for future research.

Methods

Foundational studies

The intervention, known as the Aging, Community and Health Research Unit – Community Partnership Program (hereinafter CPP), was originally developed based on best practice guidelines for diabetes and empirical evidence of diabetes care for older adults and was co-designed with older adults with diabetes, their caregivers, and community service providers. It was theoretically informed using Bandura’s Social Cognitive Theory [27], which recognizes the central role of self-efficacy in behavior. Program components were designed to capture core constructs in Bandura’s model (e.g., key sources of self-efficacy, such as social modeling and mastery) and reinforced via group sessions and home visits. The intervention was also grounded in the principles of integrated care, where person-centered, holistic care is emphasized and addressed by an interprofessional team of health and social service providers.

A feasibility study was conducted to examine the feasibility and preliminary effects of the CPP in real-world clinical practice. The results showed that it was feasible to deliver, acceptable to participants and providers, and with some adaptations to the intervention and study methods it could proceed to testing in a large-scale clinical trial [28]. A RCT was then conducted at four sites in two Canadian provinces (Ontario and Alberta, Canada). In the Ontario sites, the intervention resulted in greater improvements in quality of life and self-management and reduced depressive symptoms compared to usual diabetes care [29]. In the Alberta sites, there was no difference in outcomes between the intervention and usual care groups,however, usual care was changing during the study and becoming similar to the intervention, thus diluting the treatment effect [30]. Collectively, the findings from these prior studies were promising, thus further study of the CPP in diverse settings was warranted resulting in the current trial.

Current trial

The RCT reported in this paper was conducted to evaluate the implementation and effectiveness of the CPP in more diverse populations and settings (e.g., higher needs, different ethnic groups, rural), and to inform scale-up efforts. The study design, intervention and trial evaluation are described briefly here having been reported in detail previously [31].

The study was a parallel, 2-arm, patient-level randomized hybrid type II implementation-effectiveness study [32]. This design was chosen to ensure equal attention was given to both effectiveness and implementation outcomes, and pragmatic features were emphasized in the design using the PRECIS-II tool [33] to facilitate translation of the findings into practice. Six diabetes care/education sites were selected, two from each of three Canadian provinces (Ontario, Quebec, PEI). The target sample size was n = 264 (44 participants per site) and the inclusion criteria were: community-dwelling older adults 65 years of age, diagnosed with type 1 or type 2 diabetes and at least one other chronic condition. Caregivers ≥ 18 years of age were also invited to participate in the study together with the older adult. Participants randomized to the intervention group were offered the 6-month CPP in addition to usual healthcare services. The CPP was delivered by an interprofessional team of providers in each site including registered nurses (RNs), registered dietitians (RDs)/Nutritionists, and a Program Coordinator (PC) from a community partner organization (e.g., a kinesiologist or exercise specialist). The core components of the CPP included up to three in-home/virtual visits by the RN and/or RD/Nutritionist; up to six monthly in-person/virtual group wellness sessions; monthly case conferences where the intervention team discussed and evaluated the participant’s plan of care; and ongoing RN-led care coordination/system navigation to support clients to access other health care professionals and community services in alignment with their needs and goals. The intervention differed from usual care in many ways; key differences included an emphasis on interprofessional team-based care and self-management of multiple chronic conditions, development of unique insights into participant needs through in-home visits, and inclusion of social determinants of health in developing participant care plans. Strategies used to implement and monitor delivery of the CPP at each site included provider training, outreach meetings between researchers and the intervention team to discuss progress and address challenges, and an audit and feedback strategy to validated the delivery of the intervention components and assess fidelity. Online resources and virtual methods were used in this RCT unlike in prior trials; these were needed to address the challenges related to the COVID pandemic and are described in detail in the published protocol [31].

Additional file 1 shows the governance structures used in the RCT and highlights the roles of two groups that were critical sources for the qualitative evidence included in this paper: the Steering Committee and Community Advisory Boards (CABs). The CABs met regularly to provide information on community resources and gaps to inform local intervention adaptations and develop key messages to be shared with their networks. The Steering Committee met biannually and guided research directions, intervention adaptations to enable integration within primary and community-based settings, and development of knowledge translation products.

Outcomes and associated measures/tools, data collection timepoints, and analytical methods used to evaluate effectiveness and implementation are provided Tables 1 and 2 of the protocol paper [31], thus briefly summarized here. Data were collected at baseline and 6 months, and analysis of covariance (ANCOVA) was used to evaluate effectiveness for group differences in the change from baseline for the following participant outcomes: mental functioning (primary), physical functioning, diabetes self-care, depressive symptoms, anxiety, social support, physical activity, activities of daily living, and eating/nutrition risk. Health and social service use and associated costs were analyzed using non-parametric methods (e.g., Mann–Whitney for costs, relative risk, and risk difference for acute care service use). The key provider outcome was interprofessional team functioning, measured at 3 months and 6 months for providers delivering the intervention, and analyzed using descriptive and inferential statistics. CFIR was used to guide development of interview/focus group guides, data coding, and directed content analysis to evaluate enablers and challenges to intervention implementation. Quantitative and qualitative methods were also used to assess specific implementation outcomes (adoption, acceptability, feasibility, fidelity).

Table 1.

Conceptual framework domain: implementation—intervention

Domain	Themes	Source, theme (# sources, # quotes)	Evidence—sample quote(s)
Aim	Incorporating Social Determinants of Health into Care Model	CFIR B.1 (Intervention—Aim) -care for older adults (10,12) -diversity, health inequities (3,13)	- “this is a second round of testing and expanding the intervention to see what works in different communities and groups of people.” [PATIENT]
Aim	Improving Access to Integrated Care Services	CFIR B.1 (Intervention—Aim) -integrated care for older adults with diabetes (8,12)	- “its providing access to other community resources and more of an integrated experience for patients.” [PATIENT]
Source	Building on Prior Studies done by McMaster University	CFIR B.3 (Intervention—Source) -source (McMaster University), building (17,26)	- “I believe…this is through McMaster.” [MNG] - “this program comes from, basically Ontario…to test the program in various sites.” [PRV-RN]
Difference from usual care	Home Visits	CFIR B.2 (Intervention—Difference) -home visits-unique view (15,24)	- “we usually are not inside the home and we are not doing group sessions…that’s the difference.” [PRV-RN]
	Group Sessions	CFIR B.2 (Intervention—Difference) - “all-in-one” care (14,22)	- “group wellness sessions might be available in bigger cities…they’re actually non-existent here.” [PRV-PC]
	Integrated Care	CFIR B.2 (Intervention—Difference) -rigor, coordination (13,19) -new provider relations (10,14)	- “its [usual care] is not coordinated to the extent that this program is…its more collaborative, this program has intentionally brought 2 services together.” [MNG]
*Relative advantage—patients***	Group Sessions (Education, Exercise, Access)	CFIR B.4 (Intervention—Rel. Adv.) -socialization, education (14, 28) -physical activity/exercise (7,7) -group session, new info. (3,8)	- “it’s a program that’s offered free and there’s…support for individuals to ensure that they are engaging in physical activity that is safe.” [MNG] - “there’s always a need for community programs like this…it’s downtown, its accessible.”[MNG]
	Home Visits (Access, Socialization)	CFIR B.4 (Intervention—Rel. Adv) -home visits—access, socialization (11,18) -home visits—access (9,18)	- “…because of age, they’re not able to and they don’t want to do something…but when they have everything together, they are ready to accept…easy accessibility is one of the great advantages of this program.” [PRV-RN]
	Holistic Care	CFIR B.4 (Intervention—Rel. Adv.) -holistic, tailored (12,17)	- “we take a very holistic approach…with all of their barriers, strengths, any limitations…what goals they want to set and what’s important for them.” [PRV-RD]
	System Navigation	CFIR B.4 (Intervention—Rel. Adv.) -resource connections (6,12)	- “…we can connect them…to different resources, because we have time to do that.” [PRV-RD]
*Relative advantage—providers***	Home Visits (Comprehensive Assessment)	CFIR B.4 (Intervention—Rel. Adv.) -home visits—more comprehensive screen/assessment (15,32)	- “you go to the home and see there’s not enough food, you can talk about vegetables all day long, but getting them into the home is the real issue.” [PRV-RN]
*Relative advantage—providers***	Access to Specialized Expertise	CFIR B.4 (Intervention—Rel. Adv.) -provider access to diverse interprofessional team (3,3)	- “having a diverse interprofessional team is one of the… greatest advantages of this program and something that has really worked well. [PRV – RN]
*Relative advantage—system***	Integrated Care (Reduced Fragmentation)	CFIR B.4 (Intervention—Rel. Adv.) -intersectoral relationships (4,5)	- “as community partners they can help identify some of the gaps…it brings an awareness of the diversity and various levels of impact.” {MNG]
*Relative advantage—system***	Potential for Reduced Health Care Costs	CFIR B.4 (Intervention—Rel. Adv.) -reduced acute care service use (4,4)	“…length of stay…readmissions…a system that gives people knowledge, skills…and partnerships to engage in self care…entire health care system benefits.” [MNG]
Adaptations	None (beyond allowable tailoring)	CFIR B.5 (Intervention—Adaptations) -none(5,6) -extra/own notes (7, 9) or tools (1,1)	“nothing to me that would change the direction of the program.” [MNG]
Complexity	Complexity (System Issues, Intervention not Complex)	CFIR B.6 (Intervention—Complexity) -system/setting is complex (6,7) -intervention itself not complex (4,5)	- “when you’re looking at tertiary care settings there’s a lot of red tape to go through to sort our legalities and outreach accountabilities…so that’s the only piece I think that is complex…the program itself was very straightforward and easy to execute.” [PPRP]

Open in a new tab

Table 2.

Conceptual framework domain: implementation—inner setting, outer setting

Domain	Themes	Source, theme (# sources, # quotes)	Evidence—sample quote(s)
INNER SETTING
Readiness	Enabler: Strong Leader/Senior Management Support	CFIR D.1.2 (Inner—Leadership) -leaders/manager support(12,33)	- “We were very supportive…we gave them the flexibility in terms of time.”[MNG]
	Enabler: Strong Team Support	CFIR D.1.1 (Inner—People Resources) -support from provider team, research team, organizations (7, 10)	- “…the obvious is our community partner site. Without them this program would not be possible…the study team and everyone else’s role on the team … everybody has been supportive.” [MNG]
	Enabler: Strong Provider Training Program	CFIR D.1.3 (Inner—More Knowledge) -training enjoyable/useful (7, 21) -refresher sessions useful (5, 10) -online resources useful (8,18)	- “we received more than enough training and it was very clear, straight to the point.” [PRV-PC]
	Barrier: Insufficient Funds	CFIR D.1.1 (Inner—Resources/$) -insufficient funds for exercise equipment, printing (9, 20) -need central office, phone (4, 21)	- “Lots of hours volunteered. Also, I give folders in which I put educational materials, so I buy that with my money…there is also printing.” [PRV-NUT]
	Barrier: Facility Booking Challenges	CFIR D.1.1 (Inner—Resources/$) -facility booking challenges (4,5)	- “I think the big part is space…we had to look around to see when space was available…we don’t have enough space to have like a class full of people. [PRV-PC]
Climate	Enabler: Aligned with Existing Policies, Programs, & Patient Needs	CFIR D.2 (Inner—Climate) -compatibility with partner organization goals/aims (24, 80) -fits with community pgms (25, 53) -program seen as priority (7, 8) -aligns with patient needs (11, 19)	- “I think it’s [the intervention] a priority…its trying to provide patients with access to integrated, like bundle of services that meet their unique needs…that’s our vision at our site where our community partner is providing integrated care to clients.” [MNG]
Climate	Barrier: Potential Misalignment with Existing Programs or Priorities	CFIR D.2 (Inner—Climate) -incompatibility with current programs (7, 8)	- “..so in terms of outcome measures that we measure in our department, we look at their lipid profile, blood pressure and AIC and those are the drivers that we use to help to co-ordinate our efforts.” [MNG]
Networks and communications	Enabler: Proximity and/or Past Relationships	CFIR D.3 (Inner—Communications) -past relationships, proximity (4,4) -strong team communication (4,10)	- “It is because there was a certain openness to contributing to it given that we have a physician here who contributes to this research [MNG]
Networks and communications	Barrier: Communication Challenges with Primary Care	CFIR D.3 (Inner—Communications) -challenges with primary care (5,12)	- “I reached out [to physician] a couple of times with depression and medication alerts and never once received anything back so I just follow up with the participants.” [PRV-RD]
OUTER SETTING
Patient needs and resources	Enabler: Increased Health Literacy	CFIR C.1.3 (Outer—Patient Needs) -health literacy (diet) (17,30)	- “my main need was identifying portions…that I needed to eat as a diabetic…the second was identifying foods that were the least recommended for diabetics.” [PATIENT]
	Enabler: Increased Socialization	CFIR C.1.3 (Outer—Patient Needs) -socialization (9,10)	- “we’re trying to get people, especially seniors, they tend to be really hesitant to socialize…but once they do begin socializing…they see the benefit.” [MNG]
	Enabler: Increased Access to Providers & Services	CFIR C.1.3 (Outer—Patient Needs) -access-health providers (9,14) -access-community services (8,17) -financial challenges (8,11)	- “some patients had never met a nutritionist even though they have lived with diabetes for many years.” [RESEARCH TEAM MEETING NOTES, PROVINCE 2]
	Neutral: Main Health Issues Identified	CFIR C.1.1 (Outer—Patient Health) -diabetes, co-morbidities (13, 27) -mental health (9,19) -self-management (9,12)
	Barrier: COVID Impact (Socialization, Service Access)	CFIR C1.2 (Outer—COVID challenges) -challenges due to COVID: -family-friend dynamics (9,15) -limited socialization (7,8) -limited service access (8, 12) -limited exercise resources (5,7)	- “…you were so looking forward to actually meet other people in-person….that was the whole intent of my signing up and it didn’t happen.” [PATIENT] - “the exercise was good because it helps!…everything is closed now so I can’t even go to the exercise program.” [PATIENT]
Cosmopolitanism	Enabler: Accessed Other Community-Based Services	CFIR 2 (Outer—Cosmopolitanism) -accessed other practitioners beyond intervention team (9,12) -did not need to access others (6,7)	- “we’ve had to get a little bit more comprehensive in the inclusion of other community resources outside of the ones just established for this study.” [PPRP] - “I think it was a psychiatrist…there was a person that needed a lot of support…we finally got a hold of somebody and I just know that helped a lot.” [PRV-PC]
Cosmopolitanism	Barrier: Challenges in Connecting with Other Services	CFIR 2 (Outer—Cosmopolitanism) -challenges with other providers (3,7)	- “…a lot of time we went into the home, we see there’s a lot of social issues…but it is really hard to connect to the social worker …or get OT/PT to do a home assessment.” [PRV-RN]
External policy and incentives	Enabler: Intervention Aligns with System Priorities & Models	CFIR 3 (Outer—Policy and Incentives) -aligns with provincial models and system priorities	- “in principle it’s aligned…I don’t know that there are any Ontario Health Teams that are specifically looking at diabetes but certainly looking at chronic conditions.” [PPRP]

Open in a new tab

Qualitative evidence sources from current trial

Data sources for the qualitative evidence were described in the protocol paper. The volumes of each data source that informed the evaluation in this paper include provider focus groups (n = 11), manager interviews (n = 8), Community Advisory Board (CAB) meetings (n = 9) and meeting minutes (n = 11), patient interviews (n = 11), research team meeting minutes (n = 55), and Steering Committee meeting minutes (n = 6). Directed content analysis [34] was conducted on all of these data sources, informed by the CFIR [35, 36]. Focus group, interview, and meeting data were audio-recorded and transcribed by a professional transcriptionist. Transcripts and meeting minutes were uploaded to NVivo V. 12 to facilitate the analysis process. Two researchers conducted line-by-line coding of the data, assigning a short-phrase to each section of relevant content. These codes were then grouped into similar topic areas and further collapsed into single topics by one researcher. Finally, these topic areas were organized by the constructs within the CFIR domains and themes by domains were generated. Data that was not aligned with the CFIR domains were related to the potential scalability of the intervention. Two researchers reviewed the topic groupings and made adjustments by consensus. The resultant themes were reviewed by the research team and finalized.

Conceptual framework guiding evaluation of implementation and intervention outcomes

Two frameworks were combined to guide the implementation evaluation and integrate these findings into an assessment of key outcomes aligned with the aim of the research program: the Consolidated Framework for Implementation Research (CFIR) [35] and the Quintuple Aim framework [37, 38]. CFIR was used to categorize and present the evidence relating to determinants of implementation, and Quintuple Aim was used to consider both the effectiveness- and implementation-related evidence in relation to the key outcomes of interest in interventions like the CPP (i.e., improving the quality of health care). The CPP has the potential to transform healthcare and thereby improve the quality of healthcare in a number of ways, with its emphasis on interprofessional, collaboration (increasingly recognized as critical yet often missing in usual care), holistic care that prioritizes and integrates health and social care sectors (not simply narrowly defined clinical measures), and self-management of multimorbidity (rather than siloed and sometimes conflicting management of individual conditions experienced concurrently).

Figure 1 provides an overview of the broad categories of evidence that we present below, categorized as implementation and/or intervention determinants and outcomes. Our conceptual framework was informed by the recent work of Damschroder and colleagues [36], which clarified their view of the CFIR determinant framework and its relationship with intervention outcomes (re-labelled as innovation outcomes in their Addendum), specifically: CFIR determinants inform antecedent assessments (i.e., acceptability, appropriateness, feasibility, implementation climate, and readiness), and these, in turn, inform key implementation outcomes (adoption/adoptability, implementation/implementability, sustainment/sustainability), which are distinct from intervention outcomes (but related in ways yet to be fully understood). We adapted Fig. 1 of the CFIR Outcomes Addendum diagram [36] to refer to the CFIR determinants we captured, the implementation outcomes, and the intervention outcomes of interest in our study. We retained four of the five main CFIR domains (Intervention, Outer Setting, Inner Setting, Process), and added a fifth domain for perceived impacts (patients, providers, system), which we thought fit best under intervention determinants. We avoid the ongoing debate on how to classify CFIR constructs (e.g., determinants versus outcomes) by assuming the classifications presented by Damschroder & colleagues (i.e., CFIR determinants and assessments inform key implementation outcomes), which in turn shape the ultimate success of the intervention (achievement of intervention outcomes). We also note that the connections between CFIR constructs, related implementation strategies and implementation outcomes are unclear, as are the links between implementation outcomes and intervention outcomes [39]. These uncertainties mean cause-effect linkages cannot be clearly identified,thus we do not attempt this but instead focus on the higher-order goal of synthesizing/reconciling the collective evidence to inform next steps in a research program and translation of the findings.

Fig. 1 — Conceptual Model for the Evaluation of the ACHRU-CPP Hybrid Type II (EffectivenessImplementation) RCT (adapted from [36])

Results

Conceptual model domain: implementation determinants

The evidence regarding implementation determinants is briefly summarized below using CFIR domains and constructs, with the supporting details (i.e., themes, data source volumes, sample quotes) provided in Tables 1, 2, and 3, respectively (located at end of the manuscript).

Table 3.

Conceptual framework domain: implementation—process

Domain	Themes	Source, theme (# sources, # quotes)	Evidence—sample quote(s)
Engaging-opinion leaders and champions	Enabler: Engaged-Champions (Providers, Org. Managers, Research Team)	CFIR 2. (Process—Engaging) -interventionists, managers (4,15) -research team (3,6)	- “…my CEO is fantastic…you can’t help but be impacted by enthusiasm from our leaders.” [MNG] - “…when we meet…the meetings are very positive. There’s energy to get things done.” [MNG]
Engaging-opinion leaders and champions	Barrier: Unengaged-Opinion Leaders	CFIR 1.(Process—Engaging) -unsure—opinion leaders (3,3)	- “I don’t think others [opinion leaders, provincial officials] really know about it or the impact that it has had …they are important people.” [MNG]
Executing-components delivered and not delivered	Enabler: Services Delivered	CFIR 2. (Process—Executing) -delivered intervention components including home/phone visits (49,124), heath literacy (46,106), group sessions (46,91), self-management support (20,34), system navigation (20,30), chronic conditions mgt (14,25), interdisciplinary care (12,23)
	Enabler: Services Not Delivered—Not Needed	CFIR 2. (Process—Executing) -patients had no needs or problems (39,82), or no needs for services such as community services (32,33), mental health (24,24), other services (10,10)
	Barrier: Services Not Delivered—Desired	CFIR 2. (Process—Executing) -patients wanted more information (6,8), mental health support (4,4), or care coordination (4,4)
Executing—delivering *group sessions*	Enabler: Participants Valued Food, Exercise & Relationships (Peers, Providers)	CFIR F.3.6.2 (Process—Group Enablers) -sessions helpful, enjoyable (29,46) -enjoyed lunches (7,8) -diet/exercise education useful (19,25) -exercises enjoyable (28,45) -socialization valued (16,23) -providers built rapport (7,7) -facility, equipment worked well (6,12) -wanted more/longer sessions (9,11)	- “…the lectures were all good, they had nice handouts …and had good ideas and I thought it was well organized and well prepared.” [PATIENT] “…the session was about 2 h…we could socialize while eating…really nice.” [PATIENT] “…They [participants] hoped that the program would last longer so that they could maintain contact with us and the people that they met during the wellness sessions. That was really one of the strong points of this program.” [PRV-NUT]
	Enabler: Providers Valued Relationships (Participants)	CFIR F.3.6.2 (Process—Group Enablers) -build rapport with clients (7,7)	- “the wellness sessions [were good] because it helps us to get to know them for the home visit.” [PRV-RD]
	Barrier: Provider Coordination and Assessment Challenges	CFIR F.3.6.1 (Process—Group Barriers) -coordination challenges (24,39)	- “some people when we do exercises they wouldn’t do them as a group,..its tough to gauge how they’re progressing or regressing.” [PRV-PC]
	Barrier: Participant Lack of Participation	CFIR F.3.6.1 (Process—Group Barriers) -lack of participation (29,38) -insufficient tailoring (23, 38) -transportation, parking issues (19,35) -personality clashes (12,14)	- “I’m not a group meeting kind of guy.” [PATIENT] - “when I was asking if people would be attending…one would always say it depended on the weather, like if it was too cold she wouldn’t or if it was too icy she wouldn’t even think to go outside.” [PRV-PC]
Executing—delivering *home visits*	Enabler: Increased Access to Services	CFIR F.3.6.3 (Process—Home Enablers) -improved access to care (6,7)	- “It is just in the manner of offering it, offering it at home rather than the patient has to come to the medical ward.” [PRV-RN]
	Enabler: Improved Participant Care Experience (in-person visits)	CFIR F.3.6.3 (Process—Home Barriers) -patients comfortable (11,19) -prefer in-person visits (5,8) -information better from in-person vs phone visits (5,6)	- “…when you go see your doctor, you know that it is about 15 min per patient…at home, you do not feel that pressure.” [PATIENT] - “…it was a little more relaxing than in an office, …you are comfortable where you are.” [PATIENT]
	Barrier: Provider Burden (in-person visits)	CFIR F.3.6.3 (Process—Home Barriers) -lengthy to complete (10,22) -scheduling challenges (7,9) -burden, completing forms (8,9) -safety concerns (7,7) CFIR F.3.7 (Process—Program Barriers) -burden, completing forms (11,24)	- “at home people…feel like talking…we are easily interrupted…took more time than plan.” [PRV-RN] - “a lot of papers I found, did I do this one…that was sometimes a challenge as often they didn’t have a table you could spread out on.” [PRV-RN] - “Some people don’t want us there…someone who is a hoarder and someone who had cats, like maybe an unsafe environment…” [PRV:RD]
Executing—delivering *care conferences*	Enabler: Providers valued Team Approach to Care Planning	CFIR F.3.6.5 {Process—Conf. Barriers) -worked well (to discuss/plan) (21,39)	- “the more players there are, the more relevant it is to communicate…it was quite nice to exchange points of view.” [PRV-RN]
Executing—delivering *system navigation*	Enabler: Providers Able to Help Patients find Resources	CFIR F.3.6.7 (Process—System Barriers) -system navigation went well (4,5) -tough finding community service (4,7)	- “I am quite comfortable with it [system navigation]…I work in many sectors with the local community service centre.” [PRV-RN]
Executing—delivering *program*	Enabler: Collaborative Provider Team	CFIR F.3.8 (Process—Program-Enablers) -team-based approach (11,22) -provider consistency (4,6)	- “well-trained…before the program started and great collaboration among us is making the program to go very easy.” [PRV-PC]
	Enabler: Strong Administrative Support	CFIR F.3.8 (Process—Program Enablers) -research team support (12,17) -follow-ups, patient reminders (5,7)	- “the Research Coordinator has had a huge role in guiding the big moving parts…logistical.” [PRV-RD] - “we received tremendous support from the research team.” [PRV-MNG]
	Enabler: Provider availability	CFIR F.3.8 (Process—Program Enablers) -patients can contact providers (4,4)	- “the [PRV-RN- 01/02] are really nice. I actually have her extension number, so if there’s anything, she told me I could call her any time.” [PATIENT]
	Enabler: Virtual Delivery (Increased Access)	CFIR F.3.8 (Process—Program Enablers) -virtual program-convenient (10,13)	- “where it’s out of town like I am then the online is really good because it saves a lot of travelling…I don’t do too much driving anymore.” [PATIENT]
	Barrier: Virtual Delivery (Decreased Access)	CFIR F.3.7 (Process—Program Barriers) -challenges with access/use of technology (14,21)	- “inequity…their technology background… a participant who does not have the ability to join with internet…may be disadvantaged.” [PRV-RN]
	Barrier: Language/Hearing Challenges	CFIR F.3.7 (Process—Program Barriers) -hearing/language challenges (13,19)	- “I know there are people who had trouble hearing …when people have low/soft voices.” [PATIENT]
	Barrier: Restricted Provider Access to Medical Records	CFIR F.3.7 (Process—Program Barriers) -ltd. access to medical records (12,17)	- “I had some who had been to the ER recently…to access the health files may be beneficial, cause they are ill and don’t recall everything.” [PRV-RN]
	Barrier: Premature Service Termination	CFIR F.3.7 (Process—Program Barriers) -program ends, no care (8,17) -limitations on max. visits/contacts leaves patients without care (5,10)	- “we’ve got six months and then we can’t continue, so continuity of care is gone.” [PRV-RD] - “we feel the person still has needs, but we are limited by the guideline from the project as well as the budget. So we stop.” [PRV-NUT]

Open in a new tab

Intervention (Table 1)

Aim

Commonly reported aims understood by providers and managers (all 3 provinces) were to improve care for older adults with diabetes, test the intervention in diverse communities (particularly those experiencing health inequities), and assess an integrated care intervention. The majority of providers and managers understood the study’s aim, with the exception of a few that were mainly from province 2.

Source

Many providers, managers, CAB members, and a few patients (all sites) understood that the intervention was building on a previous study conducted by McMaster University. Some further explained that the intervention was an extension of previous research, done specifically to test for its scale-up potential and transferability to other communities with known patient needs.

Differences from usual care

Providers, managers, CAB members, and patients (all 3 provinces) identified home visits, group sessions, and integrated care as key elements that were not a part of usual care during the time of the study. A few participants (mainly province 2) felt usual care did not differ significantly from the intervention.

Relative advantage (for patients, providers, system)

Patients

Group sessions were seen by patients and providers (all sites) as improving social connections and education. Providers (most sites, all provinces) felt that the home visits improved access to services and provided an opportunity for socialization. Many providers (provinces 1 and 3) felt that the intervention’s holistic approach was beneficial because it considered a broader set of health and social determinants and patient goals/preferences in care planning. Some CAB members and providers (province 1) noted that system navigation was a benefit as it supports patients’ self-management via connections/access to community health and social services.

Providers

Many providers, managers, CAB members, and patients (all 3 provinces) viewed the home visits as beneficial as they allowed providers to conduct a more comprehensive assessment than would occur in the clinic, including increasing their understanding of the patients’ context.

System

A few providers and managers (provinces 1 and 2) noted that the intervention facilitated intersectoral relationships across acute, primary, and community care. A few CAB members and providers (mainly province 3) felt that the intervention could reduce acute care costs via improved self-management and education.

Adaptations

A few managers and providers (provinces 1 and 2) noted that no adaptations were needed; however, some adaptations were made (allowable as part of tailoring intervention to site and context) including COVID-related changes (e.g., pivot to virtual); provider availability or site proximity (e.g., one provider team for 2 sites, switching providers for continuity of care); increased funds for sites due to underestimated costs (e.g., healthier meals, longer home visits); site-specific needs (e.g., topics tailored, case conferences combined for 2 sites); and using additional assessments (e.g., those used in usual care). The most potentially impactful adaptations were those made in response to COVID, at the sites that could continue to offer the intervention (2 sites dropped out due to provider re-deployment and insufficient health human resources to continue the RCT, as a result). The adaptations made to the intervention at continuing sites are detailed in previously published work [31], and briefly highlighted here to illustrate their extensive impact on the intervention. Additional File 2 provides the timeline for the trial within the context of COVID. It shows the time span for the trial (4 years: 2019–2022) and the differential impact of COVID at each site (e.g., intervention delays/disruptions, format changes) depending on the stage sites were at when the lockdown occurred. The switch to virtual delivery was done for all sites where the trial was underway (and could continue) or about to begin when the pandemic lockdown began (March 2020); intervention changes and COVID-related changes beyond the intervention (e.g., availability/access to community services) impacted the following intervention components: (1) some assessments that could not be delivered virtually were eliminated from the home visits (potentially impacting the care plans); (2) group wellness sessions were reduced from 2 to 1 h (to reduce online fatigue) and meals were eliminated. While internet/wifi and tablets were provided to all participants that needed these, not all participants/providers were comfortable with technology and other potential pros/cons were introduced (e.g., improved access, reduced opportunity for socialization, meals no longer provided); (3) virtual monthly case conferences enabled continued contacts among providers but also changed the dynamics of team collaboration; and (4) many community-based services were paused/eliminated, which limited our ability to assist participants in connecting to supportive services and meant overall per person health and social service use (and costs) decreased.

Complexity

Some CAB members (provinces 1 and 3) thought the program was complex mainly due to features of the healthcare systems/settings (e.g., professional silos where collaboration is rare). Regarding the program itself, CAB members (mainly province 1) viewed the program as straightforward and easy to implement.

Inner setting (Table 2)

Readiness

Many managers and providers viewed senior leadership as supportive, in terms of allowing providers time and autonomy to schedule and deliver the intervention components. The university research team was cited as supportive and responsive in resolving implementation challenges. Providers (all 3 provinces) indicated that the training was comprehensive, relevant (as it was tailored to the unique needs/characteristics of the settings), and enjoyable, including the refresher training offered (e.g., to address covid-related impacts) and online training resources (e.g., Moodle website). Barriers were mainly related to coordination challenges (e.g., booking rooms for group sessions) and insufficient resources for certain components (e.g., exercise equipment, printing, central office space).

Climate

Many managers and providers (all 3 provinces) noted how the intervention aligns with current programs (e.g., interprofessional primary care programs, bundled care provision, diabetes and chronic disease management programs, exercise programs, smoking cessation classes). They also indicated that the intervention was seen as a priority, as it was addressing key needs in older adults pertaining to socialization, physical activity, and chronic conditions self-management. Only a few providers felt that program was not a priority because they were already providing similar services, had a different mandate (e.g., community partner with housing mandate), or based care planning solely on biophysical measures such as HbA1c/blood pressure/cholesterol rather than other social determinants of health.

Networks and communications

Providers indicated that past relationships and proximity of providers to one another facilitated communication among the intervention team. Providers (provinces 1 and 2) reported specific challenges in communicating with primary care (3rd province had no challenges as clinician from primary care was engaged with research); many providers reported that they sent mental health/medication alerts to family physicians but did receive any response to know whether the issues were being addressed.

Outer setting (Table 2)

Patient needs and resources

The most common patient health issues cited were managing diabetes (e.g., insulin management, foot care), co-morbidities (e.g., arthritis, cardiovascular, dementia), mental health (e.g., challenges navigating system, few mental health services), and social isolation (not just during COVID). The intervention was generally viewed by patients as addressing these issues through increasing their self-management (e.g., lay language explanations of self-care activities and their importance), socialization, service access, and system navigation. However, it was challenging to address these needs/issues due to COVID, which resulted in limited opportunities to socialize, exercise in group/community settings, and access community resources. It was noted that some intervention group participants continued to face financial challenges and barriers to accessing relevant/desired services (e.g., exercise classes targeting seniors, availability or affordability of recommended foods identified in wellness sessions).

Cosmopolitanism

Some providers (all 3 provinces) indicated that they did not use resources external to the intervention, yet others reached out to specific resources (e.g., Provincial Diabetes Coordinator, Health Authority, physicians) for advice on mental health and home care. A few providers (provinces 1 and 3) noted challenges connecting with other providers (e.g., OT/PT, social work, primary care physicians).

External policy and incentives

A few CAB members and a manager (province 1) viewed the intervention as closely aligned with emerging provincial models, particularly those of Ontario Health Teams focusing on integrated care for older adults, diabetes or chronic conditions, and the provincial government’s emphasis on self-management.

Process (Table 3)

Engaging

Opinion leaders (e.g., senior leadership at the sites and/or regional level, provincial health policy, and decision-makers) were unaware of the program, and/a few managers from each province noted they were unsure about what opinion leaders were saying about it. In contrast, a few managers and providers (all provinces) noted champions among those directly involved with program implementation (e.g., interventionists, site managers, research team).

Executing (Components-Delivered; Not Delivered—Not Needed; Not Delivered—Desired)

Many providers (all 3 provinces) cited the following components as received by patients: home/phone/web-based visits, health literacy, group sessions, self-management support, system navigation, managing multiple chronic conditions (including referrals), and interdisciplinary care. A significant number of patients did not receive all components for various reasons (e.g., lack of need and/or interest) yet other patients expressed a need for better care coordination and more information on specific topics compared to what was received from the intervention team (e.g., housing, medications, exercise, mental health supports, COVID constraints limiting access to materials/services).

Executing (Components Delivered–Enablers/Barriers)

Group sessions

Many patients valued the lunches, education, diet/exercise experience, and the relationships they built with peers and providers. Some even wanted more or longer sessions. Some providers noted a benefit in getting to know participants at the group sessions before the home visits; benefits included establishing a trusted relationship with participants and gaining insight into relevant needs and services. However, low participant attendance was cited by many providers (all 3 provinces) and this resulted in major challenges in coordinating the sessions (e.g., a minimum number of participants is needed for group sessions to be economical and facilitate meaningful socialization) and assessing participants. Participants attributed the low attendance to a dislike of a group format, personality clashes, virtual delivery; preference for more tailored/individual approaches; and transportation challenges. Providers (all sites) also found it challenging to source quality, diabetic-friendly meals for the group program, and exercise-appropriate space for in-person delivery.

Home visits

Most patients embraced home visits and welcomed providers into their homes, expressing a preference for in-person versus phone/virtual visits. Many providers (all provinces) thought home visits improved access to care and provided deeper insights into how patients live. Implementation barriers cited by some providers (all provinces) included the burden of scheduling/conducting home visits, limitations of phone assessments (e.g., absence of visual observation/verification), personal safety concerns, technology-challenges with virtual delivery, and challenges completing program documents (e.g., time required).

Care conferences

Many providers (all provinces) viewed the care conferences as valuable, citing the benefits of a team-based approach to enacting care plans. A few providers found it challenging to rush to care conferences from their care delivery roles; scheduling these to directly follow group sessions was done at some sites to address this issue.

System navigation

A few providers (provinces 1 and 3) indicated challenges with finding community services to address patient-identified needs, challenges in accessing sites/services, during COVID, and the learning curve needed to acquire knowledge about community resources, yet they also agreed that this component seemed to be valuable to many participants.

Executing (Program Delivery–Enablers/Barriers)

Some patients and providers (all provinces) noted that strong team collaboration enabled successful program delivery, which was related to maintaining a consistent provider team to support relationship-building; pre-existing relationships among providers were also enablers. A few patients (all provinces) noted that providers were accessible to reach and were encouraged to contact them. Strong administrative support was also recognized as key to program delivery by some managers and providers (all provinces); reference was made to both timely research team support and dedicated efforts by some providers to issue reminders of follow-up visits and group sessions. Virtual delivery was seen as both an enabler and barrier; it improved access to services for some (e.g., those with transportation challenges, preferring home comfort, seeking technology skills), yet reduced access for others (e.g., those without internet, computers, or technology skills).

Hearing/language challenges with in-person sessions, exacerbated by the pivot to virtual care (e.g., masking made it more difficult to hear/understand), were also noted by some patients and providers (provinces 1 and 3). Some providers and CAB members (all provinces) also noted problems accessing patient medical records, which would have helped to obtain accurate additional health assessment data and/or develop care plans. Across all provinces, some providers, patients, and CAB members (all provinces) expressed concern about being unable to continue care for patients that still needed help due to program limitations (e.g., maximum number of visits, 6-month program duration). A few challenges were encountered in securing/maintaining team providers (e.g., staffing turnovers).

Implication: implementation outcomes

We begin by briefly assessing the antecedent constructs and impact on the key implementation outcomes. Data on actual adoption and implementation at the RCT sites is cited, as well as assessment of anticipated implementation outcomes based on qualitative feedback (e.g., implementation enablers, barriers).

Acceptability

This construct captures the extent to which patients and providers like the program, approve of it, and have no serious objections to it [40]. The qualitative evidence summarized above offers evidence of acceptability. Most patients and providers indicated that they liked the program, with both groups seeing it as able to increase health literacy, self-management of diabetes and co-morbidities, access to services, and social opportunities. The program’s appeal and benefit to older adults was often cited, a group prone to social isolation and service access barriers. The appeal of the program was evident from the disappointment some patients expressed when the program ended or when group sessions were cancelled (e.g., because of the pandemic). Even patients who did not need the program’s services did not object to it, as they saw the value for others who may need and benefit from it. There is also quantitative evidence from dose statistics to suggest patients found the program acceptable. For home visits, the proportion receiving at least 1 visit over 6 months was high, ranging from 86 to 89% across sites. The mean number of home visits was 2.2 (offered 3) for the sites in provinces 1 and 3, and 1.3 for the sites in province 2 (where the intervention terminated due to COVID). In-person versus virtual (phone/video) attendance varied depending on how far along the study was when the COVID lockdown occurred: in province 1, at site 1 the study was almost complete thus most patients (77%) received in-person visits, whereas site 2 was just beginning the study thus almost all patients (95%) received virtual. In the first cohort in province 3, the study was almost complete at both sites thus most patients (86%) received in-person home visits, and in the second cohort, the study started after COVID thus all participants received virtual home visits. For group sessions, the mean number of group sessions ranged from a high of 2.53 (maximum of 6) for province 1 to a low of 0.61 in province 2 (intervention terminated at both sites). As with home visits, in-person/virtual varied—e.g., in province 1, the majority of site 1 patients (73%) attended in-person sessions whereas the majority at site 2 (90%) attended virtual assessments; in province 2, most patients in the first cohort (68%) attended in-person sessions, whereas the majority of cohort 2 (73%) attended virtual sessions. Important barriers that need to be addressed to facilitate acceptability include ensuring service continuity (e.g., during pandemics, poor weather) and tailoring services (e.g., virtual sessions, individualized services).

Appropriateness

This construct captures the extent to which the program is viewed by patients and providers as suitable, applicable, and relevant [40]. The overall feedback from many providers was that the program was applicable and relevant for most patients. Group sessions and home visits provided valuable knowledge/insights, access to integrated care, and social opportunities for patients. The team approach to developing care plans was seen as efficient in meeting patient needs. It was also clear that group sessions and home visits were not suitable for all patients, thus it is important to allow for program tailoring to meet individual needs/preferences. At some sites, there were existing organizations which already offered similar services or focused on clinical outcomes in care planning (not social determinants), thus the program may not be necessary/suitable at these sites as the intervention may not offer a relative advantage compared to existing services.

Feasibility

This construct captures the extent to which the program is viewed by providers as doable, workable, and easy to use/deliver [40]. While the intervention itself was not viewed as complex, providers cited resource constraints, difficulties scheduling group sessions due to poor attendance or facility constraints, documentation burden, and poor communication (particularly with primary care). Addressing these barriers will likely require a multifaceted approach, e.g., additional funding (if possible), tailoring services (virtual options, alternatives to in-home visits/group sessions), engagement efforts to improve primary care relations, and securing partners with an aligned mandate and existing infrastructure. Many of these issues are beyond the control of any single study and aligned with the concept of integrated care.

Fidelity

The CFIR Process-Executing construct provides evidence of the delivery of core components of the intervention. Many data sources (see Table 3), as well as home visit and group session records maintained by providers, captured the delivery of three core intervention components: home visits, group sessions, and system navigation. Monthly outreach meetings involving the research and intervention team and records kept by the provider team showed that care conferences (a core component) were held monthly while the intervention ran at each site; each intervention patient was discussed at six care conferences. Patient and provider qualitative interviews describe the core components many times, providing further evidence that they were delivered by the providers. Fidelity concerns arose at two sites (intervention terminated due to COVID), where challenges in securing a suitable and accessible community site for the group wellness sessions resulted in poor attendance by participants. Concerns also arose in the second cohort at site 3, where the dietitian conducted the majority of home visits due to the limited availability of the RN. To address these concerns in future implementation efforts, more time upfront is likely required to ensure access to a suitable community site for group wellness sessions and secure commitment from providers to ensure providers have sufficient time available to prioritize intervention delivery (recommended even though sites were given funds to allow providers to have dedicated time to deliver the intervention).

Readiness and climate

Synthesis of the feedback suggests that the intervention is ready to implement. Enablers include strong senior management support, strong provider team collaboration, and that all providers receive the training program (including access to online tools). Senior management support was present for the sites in the RCT, and opinion leaders beyond this level such as policy and decision makers at the provincial level were engaged in knowledge mobilization efforts towards the end of the trial. Ongoing involvement of representatives at the practice setting and policy level was seen as critical for future implementation, thus continued and broader promotion of the program is needed to secure and sustain higher-level support. At a program delivery level, provider team collaboration can be enabled and reinforced by selecting a provider group with pre-existing relationships, all team members attending the training program, and collaborative delivery of the intervention elements (e.g., group sessions, care conferences). Booking challenges and additional resource needs can be addressed by implementing the program in sites/locations where policies/initiatives/mandates are already aligned, tailoring (e.g., focus on those that prefer group format, virtual option), and collaborating with partners having existing infrastructure.

Main implementation outcomes

The program was adopted and implemented at all study sites (though implementation was terminated in province 2 due to COVID), with supporting evidence including the dose statistics (home visits, group sessions), CFIR Process-Executing data, and many project documents (e.g., provider records). Dose statistics in general show high uptake, although group sessions may not suit all patients and virtual options are likely needed to address mobility/transportation barriers that limit in-person attendance. Looking at the program’s future potential, it appears to be adoptable and implementable if: focuses on patients with greater need for the program’s resources, tailored to holistically meet the individual needs of patients, supported by an intervention team comprised of appropriate partners (e.g., aligned mandates, infrastructure to leverage), sustained by efforts to engage senior leaders and primary care, and backed by efforts to collect data to continue building the evidence base. Sustainability should follow from the collection and sharing of continued evidence that supports the program, with such evidence potentially suggesting priorities in terms of beginning implementation at specific sites or with particular patient subgroups.

Conceptual model domain: intervention determinants (Table 4)

Table 4.

Conceptual framework domain: intervention outcomes (JOINT TABLE)

Quintuple outcome	Theme	Quantitative results	Qualitative results, construct, theme (# sources, # quotes), sample quotes (if available)
Patient experience	No Treatment Effect (Patient Experience)	ANCOVA Results—[Mean Difference (95% CI), p-value]—COLLABORATE: − 0.66 (− 2.20, 0.87), p = 0.39	CFIR B.H.1.2—Perceived Impacts on Patients, Negative Impacts-Patient Experience: -patient needs not considered/understood (9,11) -patient needs not met (3,6) -can’t recall if needs met (11,24) -irrelevant information, group sessions (11,18)
	Providers Understood Patient Needs, Answered Questions Well		CFIR B.H.1.1.—Perceived Impacts on Patients, Positive Impacts–Patient Experience: -patients heard providers well (48, 49) -providers answered questions well (50,101) -patients felt understood/respected (48,62) -patients preferences/needs considered, including shared decision-making (37,49)
	Patients Valued Personalized (One on One) Care		CFIR B.H.1.1.—Perceived Impacts on Patients, Positive Impacts—Patient Experience: - patients valued personalized care (45,82) - home visits beneficial (19, 24) - patients valued provider follow-ups (11,13)
	Consistent Provider Messaging		CFIR B.H.1.1.—Perceived Impacts on Patients, Positive Impacts—Patient Experience: -consistent messaging across providers (42,42)
Population health	No Treatment Effect (Overall)		CFIR B.H.1.2—Perceived Impacts on Patients, Negative Impacts—Health Outcomes: -no perceived impacts (6,9)
	No Treatment Effect (Social Support)	ANCOVA Results—Mean Difference (95% CI), p-value]—DSSI: − 0.27 (− 1.00, 0.46), p = 0.47	CFIR B.H.1.1.2—Perceived Positive Impacts-Healthcare Outcomes: -improved socialization mostly from group sessions, some from home visits (24,43) CFIR F.2.6.1—Group Sessions, Enablers: -group interactions support socialization and sense of belonging (16,23) - “..there was about 10 of us, we could socialize while eating … it was really nice.” [PATIENT] CFIR B.4 – Intervention, Relative Advantage: -group sessions, access & socialization (14, 28) -home visits, access & socialization (11,18) - “The first advantage of this program, and the objective that was reached, to get people out of isolation…elderly people, over age 65, they are isolated and those who accepted to see us…all had someone to socialize with.” [PVR-Nutr]
	No Treatment Effect (Functional Limitations)	ANCOVA Results—Mean Difference (95% CI), p-value]—OARS: − 0.07 (− 0.22, 0.37), p = 0.62
	No Treatment Effect (Nutrition)	ANCOVA Results—Mean Difference (95% CI), p-value]—SCREEN II: − 0.25 (− 1.64, 1.14), p = 0.72	CFIR B.H.1.1.2—Perceived Positive Impacts-Healthcare Outcomes: -self-mgt of nutrition, exercise, self-care (46,110) CFIR F.2.6.1—Group Sessions, Enablers: -educational content on nutrition and exercise, enjoyable and reinforcing (11,15) - “they gave us advice about nutrition and we saw the new food guide…” [PATIENT]
	No Treatment Effect (Self-Management)	ANCOVA Results—Mean Difference (95% CI), p-value]—SDSCA: − 0.42 (− 1.88, 2.72), p = 0.72	CFIR B.H.1.1.2—Perceived Positive Impacts-Healthcare Outcomes: -self-mgt of nutrition, exercise, self-care (46,110) -gained knowledge about diabetes and self-mgt from providers & peers (35,76) -community-based referrals, connections (24, 60) -did not gain new knowledge: (3,3)
	No Treatment Effect (HRQoL)	ANCOVA Results—Mean Difference (95% CI), p-value]—SF- 12 (PCS) & SF- 12 (MCS), respectively: − 0.97 (− 2.96, 1.02), p = 0.34 − 0.56 (− 2.49, 1.37), p = 0.57
	No Treatment Effect (Depressive Symptoms)	ANCOVA Results—Mean Difference (95% CI), p-value]—CESD: − 0.24 (− 0.82, 1.3), p = 0.66	CFIR F.2—Executing, Services, Patients did not receive, not needed: -mental health supports not needed (24,24) CFIR F.2—Executing, Desired Services not delivered: -wanted more mental health services (4,4) CFIR C.2—Cosmopolitanism: -challenges connecting with other providers (3,7): - “I tried to get, I think it was a psychiatrist…there was a person that needed a lot of support…we finally go hold of somebody and I know that helped a lot, but I know we had to go through a lot to get to that stage.” [PRV-PC]
	No Treatment Effect (Anxiety)	ANCOVA Results—Mean Difference (95% CI), p-value]—GAD: − 0.02 (− 0.61,0.66), p = 0.94	- see above (Depressive Symptoms)
	No Treatment Effect (Physical Activity)	ANCOVA Results—Mean Difference (95% CI), p-value]—PASE: − 6.07 (− 17.0, 4.83), p = 0.27	CFIR B.H.1.1.2—Perceived Positive Impacts-Healthcare Outcomes: -self-mgt of nutrition, exercise, self-care (46,110) CFIR F.2.6.1—Group Sessions, Enablers: -exercises, enjoyable & worked well (28, 45) -interactive, passive teaching approaches for exercises worked well (19,25) - “I liked doing the exercises, they felt good.” [PATIENT] - “they showed us the importance of exercising … then we would go and exercise … I really liked that.” [PATIENT]
Cost	Increased Costs	Mann–Whitney U Test Results: -median difference, total costs: $664.7 (p < 0.0001) -median (IQR) intervention costs: $559.2 ($417.4, $793.4)	CFIR B.4—Relative Advantage: A few providers and managers noted that the intervention might overload the healthcare system such that the costs exceed the benefits
Cost	Potential to Reduce Long-Term Costs		CFIR B.4—Relative Advantage: -could reduce acute care service use (4,4) - “That would be a benefit to the hospital in terms of length of stay, in terms of readmission and all those things. So, if we can have a system in place that would give people the knowledge, skills, ability and partnerships that they need to take better care of themselves, I think eventually that the entire health care system benefits.” [MNG]
Provider experience	Improved Team Collaboration	Paired T-Test Results [Mean Difference for T2-T1, p-value]: PSAT subscales: -management and administration: 0.49 (0.03) -synergy: 0.42 (0.04) -leadership: 0.37 (0.03) -efficiency: 0.17 (0.56) -non-financial resources: 0.03 (0.88) -financial resources: − 0.25 (0.13)	CFIR B.4—Relative Advantage -access to diverse interprofessional team (3,3) - “I think the collaboration of different professionals was like the biggest benefit for sure.” [PRV-PC] - “The advantages are that we’re able to collaborate with a partner that’s actually skilled and knowledgeable about exercise regimens and different modalities…” [MNG]
	Work Perceived as Important and Effective		CFIR B.4—Relative Advantage -addressing older adult needs (10,15) -link patients with community supports (6,12) -home visits yield provider insights (15, 32) - “The advantage is that we can…see, we can help, we can see more, help them out more…because we have time to do that. In the clinic…we really don’t have it.” [PRV-RD] “…also giving them other tools and resources…a place where they can go for food…information on adult day programs, perhaps where they can really interact with other people. I think it’s so important for people and their health and well-being. It’s like a win–win situation.” [MNG]
	Building Capacity		CFIR B.4—Relative Advantage -information on community resources, patient needs (3,3) - “This program can really give us some kind of guidelines of what to do with diabetes completely to have better engagement in areas of what they really need.” [PRV-PC] - “…ever since, I’m now working with community partners…my community partner…is now coming to do presentations to patients and caregivers. And, I think that’s the point of the program is to build capacity.” [MNG]
Health equity	Target Population Representation	Baseline characteristics, key results: -RCT participants broadly similar to general target population on demographic, health status and healthcare services use (See Additional File 3 for details)
	Improved Access to Services		CFIR B.4—Relative Advantage -improved access, general population (14,28) and older adult population (10,15) - “..the advantages are that there is no other program like this one so its definitely filling a gap for the needs of that population…I find that the community partner is … really accessible.” [MNG] - “…the advantage is that it’s a program that offered for free and that there’s facilitators that are there to support individuals to ensure that they are engaging in physical activity that is safe but will achieve good results.”[MNG] - “…because of age…they don’t want to do something because it’s like, ‘oh that’s too far’…but when they have everything together, they are more than ready to kind of accept so easy accessibility is one of the great advantages of this program for them.” [PRV-RN]
	Need to Tailor to Cultural Needs		CFIR B.4—Relative Advantage -language/cultural needs may need attention (6,13) - “that’s one of the things I could see not meeting their needs, is probably looking at languages, the different languages that people speak in diverse populations [MNG] “…it’s just not something they could feel comfortable with unless it was somebody … a group that was similarly maybe culturally to them…So…if you’re offering something to a Chinese population together, …you know my parents don’t feel comfortable with…with typical Anglophones.” [PATIENT]
	Reduced Access to Services during COVID		CFIR C.1—Patients Needs and Resources -service access challenges (8,12) -limited socialization (7,8) -limited access to exercise sites (5,7) - “So, even before the pandemic, you were looking forward to meeting people in-person so you could connect with them.” [INTERVIEWER]…”Yeah. That was the whole intent of my signing up and it didn’t’ happen.” [PATIENT] - “…I won’t go out of the house with this COVID going around …and I won’t go in doctor’s offices either. [PATIENT] - “…we had an exercise that we an do in the building, but with the virus it close…Everything is closed now. So, I can’t even go to the exercise program anymore.” [PATIENT]

QUAL

Many providers and patients noted that equity considerations were built into the design of the intervention, and improved accessibility to services, particularly for older adults who are reluctant or unable to travel and find multiple provider visits challenging. Enhanced accessibility was thought to result from both home visits and the group sessions, which exposed patients to a interprofessional team in one location/session, allowed flexibility for patients to start group sessions at any point, were in close proximity to transportation services, were cost-free, and included linking people with social services (e.g., food banks, government programs) to address food and financial insecurity. While the aim of the RCT was to test the CPP in diverse communities and settings, some providers thought the program required further tailoring to engage and sufficiently meet the unique needs of specific ethnic groups (e.g., provision of program in languages other than English/French, providers with same ethnicity, focus on foods/exercise preferences of specific cultures). For participants involved with the intervention during COVID, accessibility was noted as limited or challenging. As noted above, various adaptations were made to the intervention in response to COVID, and while participants were provided with technical support as needed (including a tablet if needed), these adaptations nevertheless reduced the opportunity for socialization and complicated delivery of the intervention due to technical challenges and inherent limitations associated with virtually assessing and delivering care to patients. This was accompanied by a general inability or reluctance for participants to access primary care, hospital/clinic services, and community-based resources.

Implications: intervention outcomes

Our task of evidence synthesis began by searching for quantified approaches that could be applied to the constructs in CFIR, Quintuple Aim, and ultimately our conceptual model. Our search revealed that CFIR users mainly employ qualitative methods to assess the constructs [52]. Additionally, the tenets of mixed methods research encourage awarding comparable weight to diverse ways of understanding [53] and stress the importance of embracing multidimensionality rather than reductionist numerical approaches. Highlighting multidimensionality is particularly critical when faced with contradictory results, a common occurrence in the evaluation of complex interventions and complex problems in general [21, 54–56]. Accordingly, we developed a rating scale similar to the “strength” (weak to strong) ratings that a few researchers have applied to CFIR constructs, but did not generate an overall summative score [36]. Based on a synthesis of all qualitative and quantitative evidence concerning each of the Quintuple Aims, we arrived at an assessment of the overall strength of the evidence and categorized it as Level I (strong), Level II (moderate), and Level III (weak). While all levels of evidence were impacted by COVID, differences still existed regarding the strength of the evidence pertaining to each outcome. This assessment was developed by our team in order to help us distal from the large body of evidence the key messages to inform future implementation efforts and translation of the findings into policy and practice. Like any analysis that aims to extract meaning from a vast amount of data, it is possible that other researchers could identify other insights and explanations. Nonetheless, our efforts to consider all forms of evidence and reconcile discrepant findings led us to new and nuanced understandings that helped to clarify the next steps in implementing the CPP.

An outcome at Level I (strong) represents one where the quantitative and qualitative evidence are consistent and few barriers exist to achievement of the outcome. An outcome at Level II (moderate) is one where the quantitative and qualitative evidence are inconsistent and barriers to achievement of the outcome exist; however, viable strategies were identified to address the barriers, suggesting that achievement of the outcome is likely. An outcome at Level III (weak) represents one where both the quantitative and qualitative evidence are primarily consistent in pointing to challenges/barriers in achieving the outcome, with strategies to address these remaining unclear.

Provider experience was assessed as Level I, meaning achievable with few (if any) constraints. This was the only outcome where the quantitative and qualitative evidence was consistent and positive with no negative comments or suggestions for ways to improve the partnership. Providers indicated that the team approach provided them with additional knowledge and skills, and they saw the work as meaningful, rewarding, and able to positively impact patients.

Three outcomes were categorized at Level II—Patient Experience, Population Health, and Equity. For these outcomes the quantitative and qualitative evidence typically diverged, but feedback pointed to enablers and various strategies that could potentially address barriers. Below is a brief summary for each outcome:

Patient experience: the quantitative results did not show a treatment effect, and qualitative feedback indicated that many patients needed and enjoyed the intervention, yet some patients expressed not learning anything new, did not benefit from the intervention or did not need the intervention. The main recommendations to improve patient experience include focusing on delivering the intervention to patients who need the intervention (e.g., complex health and/or social needs), further tailoring to ensure individual needs/preferences are addressed, and offering delivery options that include virtual care for those who desire it or face mobility/transportation constraints.
Population health: the quantitative results did not show a treatment effect, yet providers and patients expressed many positive perceptions of the intervention and reported perceived benefits. COVID was considered a significant confounding factor in our study by providers, CAB members, and the Steering Committee. While no treatment effect was seen in the sensitivity analyses that was stratified by a major COVID-driven change (delivery format—virtual vs in-person), the study was underpowered for these analyses. Moreover, it is unlikely that statistical solutions can fully account for the complex interactions and impacts of COVID on the range of implementation domains. COVID was responsible for threats to both internal and external validity that resulted in considerable skepticism regarding the treatment effect and its generalizability, ultimately contributing to increased interest and reliance on the qualitative evidence.
Equity: various sources suggest that participants in our RCT were similar to the general population of older adults with diabetes. Our results may not be representative of those with high needs (due to health, ethnic/cultural preferences). Most of the qualitative feedback was positive, with the intervention seen as offering improved access to various services (compared to usual care) for many participants (e.g., those needing guidance on safe and effective physical activity or nutrition advice). As noted above, some providers highlighted the need for more tailoring to address specific cultural and linguistic needs/preferences. Strategies moving forward include considering more tailoring or focusing on specific population subgroups, with a close look at costs and potential benefits.

One outcome was categorized at Level III—Cost. The quantitative and qualitative data largely agreed and pointed to cost/resource concerns. While the intervention cost was lower (by half) compared to our prior trials due largely to the intervention being delivered virtually (at some sites), a return to in-person may result in increased intervention costs. Other concerns were also noted (e.g., home visits take more time than clinic visits, costs for centralized supplies/facilities were not covered, organizing group sessions is time consuming). At present, it is difficult to determine whether targeting home visits and group sessions to those in need of them (e.g., socially isolated) versus a broader population will address cost concerns. Prior trials have found the intervention to be cost neutral, but costs could increase if the groups targeted by the intervention are truly high needs (e.g., increased service use occurs to address the needs). Alternatively, a longer time horizon may show that investment in upstream prevention lowers downstream acute care costs. Future studies are needed that include a detailed focus on costs. For example, a comprehensive cost–benefit analysis with an appropriate timeframe would help to determine whether the benefits outweigh the added costs of a more intensive, integrated approach to enhance self-management and address both health and social factors.

Summary—comparing quantitative and qualitative quintuple aim findings

Our quantitative findings showed no statistically significant treatment effect across a range of health outcomes, yet our qualitative findings consistently showed perceived benefits of the intervention. Overall, these results appear divergent, but when we look more closely at Table 4 we see three types of results when we compare findings from the two strands:

One strand complements the other: This occurs when one strand captures a different dimension of a construct. Examples can be seen in the Health Equity construct, specifically improved access to services and the need to tailor the intervention to address cultural differences, which came up often in the qualitative findings but were not measured in the quantitative strand. Another example can be seen in the Population Health construct, specifically HRQoL and functional limitations, which were measured quantitatively but were not mentioned in the qualitative interviews.
One strand corroborates the other on some level(s) of a construct: Examples can be seen in the Cost and Provider Experience constructs, where both strands agree that cost impacts were potentially negative and team collaboration improved with the intervention.
One strand contradicts the other on some level(s) of a construct: Examples can be seen in the Patient Experience construct, where the quantitative results showed no difference despite many positive reports on experience in the qualitative data. Another example is the Population Health construct, where there were many positive reports in the qualitative findings on physical activity, self-management ability, nutrition, and social support despite no differences seen in the quantitative findings.

Ultimately, findings may appear contradictory or corroborative, but caution is needed in interpreting them as such as they may be the product of methodological differences between methods. Method differences also prevent using one set of findings as causal evidence in support of the other, or assigning priority to certain findings (e.g., assigning more importance to corroboratory versus contradictory results). These issues are further explored in the next section.

Discussion

We chose a hybrid design for the RCT to maximize the information obtained from the trial and facilitate the goal of informing policy, practice, and research related to the CPP intervention. This choice allowed us to showcase a key strength of the design, which was the ability to collect high-quality, context-rich qualitative data to in a rapidly and dramatically changing environment to expand our understanding beyond the quantitative findings. We developed a conceptual framework to help us comprehensively identify and integrate all the various forms of evidence to achieve this goal. Our efforts to synthesize the evidence resulted in discrepant findings that can be traced to a number of potential root causes, which we highlight below.

Quantitative and qualitative research—paradigm differences

We should acknowledge that epistemological and methodological differences between quantitative and qualitative methods may account for differences in the two sets of findings. There are clear differences between the two research paradigms and methods including the underlying research questions, types of data collected, sample size, sampling strategies, and analytic methods. Quantitative approaches define the issue under study, the relevant outcomes, and the analytical methods a priori and document these in a study protocol that is not easily adapted to the contextual changes that occur during clinical trials. Qualitative approaches however, are emergent and more flexible, allowing participants to introduce new topics, explain unexpected responses, and provide conditions/caveats that highlight variability in behavior and context that are unmeasured or unable to be inferred from quantitative research [57]. An important rationale for the hybrid trial design is to triangulate the findings from these two approaches, resulting in a more complete understanding of the intervention and the key implementation strategies needed for success. Importantly, not all research projects that collect and analyze quantitative and qualitative data produce conflicting results despite the epistemological and methodological differences, suggesting that discordant findings may have other causes. We believe this to be the case in our study, and we discuss some of the key factors below.

COVID-related impacts

Impacts of the COVID- 19 pandemic were seen by many as a confounding factor potentially responsible for the absence of a treatment effect. The negative findings from the effectiveness evaluation did not come as a surprise to the research team, CAB members, or the Steering Committee. Rather than stopping the trial altogether, we chose to make the necessary adaptations to the intervention to allow our trial to continue, based on limited evidence supporting virtual delivery of similar interventions [31]. The key changes to the intervention were highlighted in Section 'Intervention (Table 1)', sub-section 'Adaptations' and encompassed a shift to a virtual delivery modality that was extensive and affected participant recruitment and retention, as well as implementation of all intervention components. While these changes alone were significant, they fail to capture the full range of impacts and complex interactions that took place in response to the COVID- 19 pandemic. The study by Massazza and colleagues explored the impact of the pandemic on the delivery of non-COVID RCTs (psychological interventions), and found that the impacts cited by researchers spanned all CFIR domains [58]. The same was true in our case, with examples from our study including:

Intervention characteristics: changed from in-person to online, generalizability of results challenged by uniqueness of context during COVID;
Outer setting: mobility and access issues experienced by participants, distancing restrictions prevented in-person meetings, funding agency issued extension to identify and implement trial adaptations, closure, or pausing of community-based services;
Inner setting: budget extensions needed at some sites to re-train providers after delays, budget increases needed to supply tablets/internet to participants, workforce challenges resulted in the loss of 2 sites due to provider re-deployment, provider team dynamics and skills changed; and
Implementation process: eliminating assessments that could not be done virtually, logistical and ethical issues experienced in shifting to virtual.

The effect of these changes across the range of CFIR domains is extremely challenging to fully explicate, yet many dismissed the findings due to concerns that the COVID- 19 pandemic may have threatened the internal and external validity of the trial results.

The impact of the pandemic on the conduct of clinical trials has been recognized in the literature. A study that captured the impacts of COVID on Non-COVID clinical trials in a 2020 query of the ClinicalTrials.gov database found that, compared to the pre-pandemic period, there was a significantly higher number of stopped trials and a significantly lower number of newly initiated trials [59]. Another study in 2020 found that the number of stopped trials increased significantly during the pandemic (p < 0.001) with the proportion of stopped trials by country ranging from 1 to 17.1% [60].

This is exacerbated by researchers’ varying views on the ability of a RCT to handle contextual changes of this magnitude, with some seeing the RCT’s inherent design features (e.g., randomization to achieve balance in contextual changes across groups) as limiting the impacts on study results, and others seeing the RCT’s design as offering little insulation as it is “particularly vulnerable to drastic shifts in context…that clash with the unpredictable and ever-changing circumstances dictated by COVID- 19” [58], p. 3). While changes in some CFIR domains were likely experienced by both trial arms (e.g., outer setting, inner setting, implementation process for services delivered in both arms), changes to the intervention and its implementation may have reduced the relative advantage of the intervention compared to usual care that would have existed outside of the pandemic. The most directly relevant evidence we have regarding the CPP’s relative advantage during non-pandemic conditions comes from our prior multi-site, pragmatic RCT which tested the in-person delivery of the CPP in one of the 3 provinces involved in the current trial. This RCT was run prior the pandemic and demonstrated the effectiveness of the CPP in improving quality of life and self-management, and reducing depressive symptoms [29].

In our current study, research team members viewed the impact of COVID as intrinsically disruptive, leading to concerns about the validity of the quantitative results. Ultimately, the impact of the changes that were made to the delivery of the intervention are unknown, and we cannot know whether we would have seen an effect if these changes had not been made. This skepticism extended to the statistical methods (e.g., sensitivity analyses stratified by modality), which were introduced to explore the effect of COVID; despite these methods being among those endorsed to help protect trial integrity [61, 62]. Most researchers in this study thought that statistical methods offered insufficient control over the myriad factors (many of which defy measurement) and changes that occurred during COVID.

The qualitative evidence was invaluable and offered relevant insights into the negative impact of COVID on intervention delivery (which could explain the lack of a treatment effect) as well as evidence for the positive impact of the intervention as perceived by patients and providers. Qualitative evidence highlighted that many perceived that the pandemic negatively impacted intervention delivery and resulted in the absence of a treatment effect in the quantitative findings, but also revealed positive perceived impacts (e.g., improved access to services for some participants, cost savings in delivering parts of the intervention, and facilitation of provider collaboration for those located remotely) that highlight the importance of adapting the CPP to include the option of a virtual delivery modality in future implementation initiatives (virtual/hybrid delivery options were not in the original design of the intervention).

RCT designs—reconciling effectiveness and implementation

A key aim of the CPP was delivering patient-centered care. We were challenged in determining whether we achieved this or, more importantly, whether we captured the right measures/information to evaluate whether we have achieved this. Certainly, delivering person-centered care was the intent of the CPP and inherent in its design, but we faced challenges in measuring all aspects of this construct and found that qualitative evidence played an important role in assessing whether it had been achieved. The qualitative findings pointed to a difference between patients and researchers/clinicians regarding how care should be delivered and what outcomes are most important. RCTs typically focus on a limited set of primary outcome(s), which is a reductionist approach that can be both a pro and a con in relation to patients’ interests (e.g., it limits patient burden yet also limits the outcomes evaluated). Choosing outcomes that matter most to patients may require different approaches to measurement, data captured from various sources (e.g., clinical trial data, administrative data), and the priority assigned to different forms of data. Advancements continue in relation to the development and validation of tools to measure patient experience and satisfaction with healthcare in various settings (e.g., acute care, primary care, specialty care). Yet, there are also limits on the extent to which quantitative measures can capture all relevant aspects of a complex domain such as person-centered care, thus the collection of qualitative data to augment quantitative findings is likely to remain both realistic and wise.

The pragmatic nature of our trial presented various challenges. We intended to mimic the real-world in designing the intervention (e.g., few extra efforts to recruit/engage patients, implementation in practice settings and by providers similar to usual care, provider flexibility to deliver the intervention using professional judgment similar to how they would in usual care, adopting outcomes relevant to patients) [33]. However, pragmatic trials create a paradoxical situation: delivering an intervention outside the research context is more likely to provide a truer representation of the results that are likely to be seen in real-world practice, yet effects cannot be proven without the research [63]. Pragmatic trials by design have little control over real-world changes that impact the effectiveness of the intervention. For example, our trial ran over the 4-year period 2019–2023, and during this period significant changes occurred in practice and healthcare policy that may have reduced the gap between the usual care and the intervention, including the COVID pandemic (resulting in restricted access to care and a shift to virtual care in both groups), increased emphasis on integrated care (CPP is an integrated care intervention that was more novel at the beginning of the trial compared to now) [64], recommendations to consider health equity and incorporate social prescriptions into health care plans [65] (CPP holistically integrates strategies to address social determinants in patient care plans), and increased recognition of the importance of addressing multimorbidity (CPP holistically considers all chronic conditions not just diabetes) when treating chronic illness [66]. Pragmatic trials also introduce the dilemma of ensuring that an intervention is delivered with fidelity and allowing flexibility, which is not straightforward when attempting to describe core components and where to allow for flexibility [63]. For example, in our trial, group wellness sessions are a core component,our protocol clarifies the aim of the sessions, the core content to be included in them, and how they can be tailored to the interests/needs of the patients in the sessions. However, we did not envision virtual delivery (necessitated during COVID, so we did not originally specify this modality in the protocol, and at the time of modification there was very limited literature on how to best deliver virtual care or evidence of its effectiveness. The emphasis in pragmatic trials on outcomes of importance to patients creates further challenges, not simply ones related to measurement but also adoption of outcomes that may be less amenable to change or routinely captured in health systems data. This was seen in the results of the 3D trial (the largest trial of optimal multimorbidity management as of 2018), where the intervention did not improve quality of life (considered an important patient outcome). This was further confirmed through a meta-analysis that pooled eight trials, concluding no statistically significant improvement in quality of life [67]. Interestingly, patient experience was positive in the 3D trial (as in ours), and the authors suggest that multimorbidity interventions may improve patients’ perceptions of care quality but not the quality of their lives, and that these perceptions of care should still rank as important in terms of patient-centered care and as an aim in the Quintuple Aim model.

We ourselves experienced challenges overcoming conventional views on the forms of evidence that matter in RCTs. Conventional views prioritizing the effectiveness results prevailed initially among most members of the research team; many saw the quantitative trial results as a failure, until we developed and applied the conceptual model presented in this paper to look closely at all the evidence and attempt to synthesize it. One wonders whether we would even be writing this paper if the situation had been reversed—i.e., the quantitative results showed evidence of effect, and the other aims showed neutral or negative effects? Perhaps we would have glossed over the neutral/negative findings and elected to align with conventional norms and promote the promising results of the effectiveness evaluation? Ultimately, researchers are obliged to honor the intent of the chosen design. Our study used a pragmatic RCT hybrid type II design, which assigned equal weight to the mixed methods findings and attempted to mimic important contextual conditions encountered in practice. We intentionally collected both quantitative and qualitative data; thus, we were obliged to ensure that both play an important role in the evaluation of the intervention, while acknowledging the limitations that pertain to each. Importantly, the hybrid design offered the opportunity to collect more nuanced information that was helpful to guide the adaptation/tailoring/focus of future efforts to scale and spread the intervention.

Recent literature supports the need for more evolved approaches to RCT design. While acknowledging many of the challenges we described above in relation to hybrid and pragmatic designs, a recent systematic review emphasizes a growing interest in these designs and the rationale for them. A declining interest in efficacy trials for complex interventions has been linked to a lack of fit with the experimental framework, inclusion of features that cannot be strictly standardized, and the need for more information on implementation and context as often these are a determinant of the results [68]. Instead of efficacy trials and individual RCTs, researchers are turning to “adaptive” RCTs (e.g., pragmatic designs, cluster randomization, process analyses, realist evaluations), with these designs being more than twice as prevalent (45% versus 22%) as individual RCTs in a systematic review of methods used to evaluate complex interventions [68].

Bypassing efficacy trials is also particularly common in studies evaluating community-based interventions like the CPP, where the research agenda prioritizes findings from real-world settings to avoid the artificiality of experimental conditions with the aim of expediting and translation to practice [32]. Developers of the original type I/II/III hybrid design typology note that this framework has not kept pace with the intervention development and research approaches now used in public health, health promotion/prevention, and community-based service research. To address this concern, they have identified a series of questions to provide more concrete guidance to researchers in choosing a hybrid design that best addresses the goals of the research and the needs of the stakeholders [32]. While this guidance was not available when we designed our RCT, we have reflected on these questions and feel the type II hybrid design we selected was appropriate for the stage of the research on the CPP intervention and our knowledge users’ goals.

Next steps in the research program

With the goal of the hybrid type II effectiveness and implementation study in mind, stakeholders of the CPP encouraged proceeding with the following research and implementation agenda:

Conduct a scalability assessment to consider the evidence from the current trial within the context of our prior research, the broader literature for interventions similar to the CPP, and the ever-changing political/policy context.
Continue to build an evidence-base by adopting a phased approach that starts with implementing and testing the CPP in high-risk subgroups.
Develop and validate measures that are important to patients (patient-reported outcome measures, or PROMs).
Ensure that study durations are sufficient in length to enable accurate evaluation of effects and factors shaping successful implementation.
Continue to evolve RCT methods to better adapt to the dynamic and complex nature of integrated care.

The overall position of our stakeholders was that the CPP could be effective outside of a pandemic context and would benefit from additional study to inform the tailoring and focusing of the CPP on priority populations. This viewpoint was driven largely by the mixed findings from this research program, the recognition of the mixed results seen in the broader literature for integrated care interventions for older adult and recent changes in healthcare policy and practice that align with the CPP (e.g., value based care, integrated care, health equity, multimorbidity). Strong evidence of support for the CPP also comes from one of the RCT sites, which is currently integrating the CPP into their diabetes integrated care pathway. Importantly, many of the next steps outlined above align with the recommendations in the integrated care literature, including using a phased approach to build evidence for the intervention [7], focusing on high-risk groups rather than large heterogeneous populations [8], lengthening study durations [69], and assigning a higher priority to patient-oriented outcomes [13, 14].

Conclusions

The next steps in the CPP research program will be to explore and respond to the contextual changes using an evolved RCT approach that is dynamic, preserves scientific integrity, and respects the goals of effectiveness and implementation evaluation. This evolved approach recognizes that research practice is far less predictable and controlled than textbooks lead us to believe, resulting in “wicked problems” that will continue to make mixed methods relevant [17, 18, 53]. An adaptive strategy that mixes methods will ensure that results yield current, practical, and contextually relevant guidance for policy-makers and clinicians, thereby optimizing care for older adults with multiple chronic conditions. Guidance on exactly how this mixing should be done is currently scarce, creating both challenges and opportunities. Challenges also remain in linking implementation determinants and outcomes with intervention outcomes. It is unclear how to map/relate constructs from implementation models like CFIR with intervention models like Quintuple Aim. Our conceptual model helped to recognize the different forms of evidence and guide us in linking the evidence to key implementation and intervention outcomes, but we struggled with developing a more concrete understanding of cause-and-effect relationships between the two models. Relationships between the two models are most likely bi-directional, but more detail is needed. There are ongoing efforts in the implementation science field to better understand these relationships, with the most recent CFIR addendum [36] being an important step in clarifying the distinction between implementation and intervention/innovation outcomes.

Supplementary Information

13063_2025_8839_MOESM1_ESM.pdf^{(139.9KB, pdf)}

Additional file 1. ACHRU CPP Program of Research Governance StructureGovernance structure for the research program.

13063_2025_8839_MOESM2_ESM.pdf^{(40.5KB, pdf)}

Additional file 2. Timeline for Intervention Delivery and Data Collection at Trial Sites. Start and end time for delivery of intervention and data collection at each RCT site.

13063_2025_8839_MOESM3_ESM.pdf^{(53.5KB, pdf)}

Additional file 3. Comparison of RCT participants with provincial populations. Statistics from various sources comparing RCT participants to provincial populations on socio-demographic and health status indicators.

Acknowledgements

Not applicable.

Abbreviations

ACHRU: Aging, Community & Health Research Unit
ANCOVA: Analysis of covariance
CAB: Community Advisory Board
CESD: Center for Epidemiologic Studies on Depression Scale
CFIR: Consolidated Framework for Implementation Research
CPP: Community Partnership Program
GAD: Generalized anxiety disorder
LSM: Least Squares Mean
OARS: Older Americans Resources and Services
PC: Program Coordinator
PEI: Prince Edward Island
QUAL: Qualitative
QUAN: Quantitative
PASE: Physical Activity in Seniors Scale
PRECIS: PRagmatic Explanatory Continuum Indicator Summary
PSAT: Partnership Self-Assessment Tool
RCT: Randomized controlled trial
RD: Registered Dietitian
RN: Registered Nurse
SDSCA: Summary of Diabetes Self-Care Activities

Authors’ contributions

KF prepared an initial draft of the manuscript, developed the conceptual model, analyzed and interpreted the effectiveness data, and incorporated the implementation evaluation findings into a synthesis of the evidence based on the detailed qualitative report prepared by a qualitative expert on the research team. SCC, RG, MMR, MN, and DS made significant contributions to conceptualizing and scoping the manuscript and actively contributed to revisions of the manuscript through a number of internal review rounds. All authors read and approved the final manuscript.

Funding

This study is supported, in part, by funding from the Canadian Institutes of Health Research Strategy for Patient-Oriented Research (SPOR) Primary and Integrated Health Care Innovations Network: Programmatic Grants (Funding Reference Number: KPG- 156883) in partnership with: Diabetes Action Canada, a Canadian Institutes for Health Research (CIHR) Strategy for Patient-Oriented Research Network in Chronic Disease (project reference #1.1.1 ACHR); McMaster Institute for Research on Aging (Hamilton, ON); McMaster University School of Nursing; Réseau- 1 Québec; Fonds de Recherche du Québec (FRQS); Scarborough Health Network Foundation. This research was also undertaken, in part, thanks to the funding from Dr. Markle-Reid’s Tier 2 CIHR Canada Research Chair and the McMaster Collaborative for Health and Aging. The funders of this study had no role in study design, data collection, data analysis, data interpretation, or writing the manuscript.

Data availability

The data that support the findings of this study are not openly available due to reasons of sensitivity and confidentiality, but anonymized versions of the data may be available from the corresponding author upon reasonable request.

Declarations

Ethics approval and consent to participate

Institutional ethics approval was obtained from the following: the Hamilton Integrated Research Ethics Board (#5101); the Scarborough Health Network Research Ethics Board (#NEP- 18–014); the Unity Health Toronto Research Ethics Board (#18- 336); University of Prince Edward Island Research Ethics Board (#6008019); Prince Edward Island Research Ethics Board; and Centre intégré universitaire de santé et de services sociaux (CIUSSS) de la Capitale-Nationale (MP13 - 2019–1670). Ethics approval was renewed on an annual basis as required for the study duration. Informed consent will be obtained from participants (older adults, caregivers, providers, managers, public and community partners) by the research assistant before study enrolment.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Footnotes

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

1.Cameron WB. Informal sociology - a casual introduction to sociological thinking. New York: Random House; 1963. [Google Scholar]
2.Hill AB. Heberden oration - reflections on the controlled trial. Annals of Rheumatology Disorders. 1966;25:107–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
3.Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., Petticrew, M., & Medical Research Council, G. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337: a1655. 10.1136/bmj.a1655. [DOI] [PMC free article] [PubMed] [Google Scholar]
4.Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–26. 10.1097/MLR.0b013e3182408812. [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Billings J, de Bruin SR, Baan C, Nijpels G. Advancing integrated care evaluation in shifting contexts: blending implementation research with case study design in project SUSTAIN. BMC Health Serv Res. 2020;20(1):971. 10.1186/s12913-020-05775-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
6.ihub. Evaluating new models of care: a guide for integrated care providers. 2017. Scotland Retrieved from ihub.scot. file:///C:/Users/fishe/Downloads/Evaluating%20new%20models%20of%20care-1.pdf.
7.Koziel A, Gorgens M, Chawla M, Krol-Jankowska A, Kononiuk A. Measuring integrated care - methodological reflections from monitoring and evaluation process of the PHC plus pilot program in Poland. Int J Integr Care. 2023;23(2):1. 10.5334/ijic.6646. [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Hoogendijk EO. How effective is integrated care for community-dwelling frail older people? The case of the Netherlands. Age Ageing. 2016;45(5):585–8. 10.1093/ageing/afw081. [DOI] [PubMed] [Google Scholar]
9.Hughes G, Shaw SE, Greenhalgh T. Why doesn’t integrated care work? Using Strong Structuration Theory to explain the limitations of an English case. Sociol Health Illn. 2022;44(1):113–29. 10.1111/1467-9566.13398. [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Kirst M, Im J, Burns T, Baker GR, Goldhar J, O’Campo P, Wodchis WP. What works in implementation of integrated care programs for older adults with complex needs? A realist review. Int J Qual Health Care. 2017;29(5):612–24. 10.1093/intqhc/mzx095. [DOI] [PMC free article] [PubMed] [Google Scholar]
11.Zhang Y, Stokes J, Anselmi L, Bower P, Xu J. Can integrated care interventions strengthen primary care and improve outcomes for patients with chronic diseases? A systematic review and meta-analysis. Health Res Policy Syst. 2025;23(1):5. 10.1186/s12961-024-01260-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Harnett P, Kennelly S, Williiams P. A 10 step framework to implement integrated are for older persons. Ageing Int. 2020;45:288–304 Retrieved from doi: 10.1007/s12126-019-09349-7. [Google Scholar]
13.Hayes C, Manning M, Fitzgerald C, Condon B, Griffin A, O'Connor M, et al. Effectiveness of community-based multidisciplinary integrated care for older adults with general practitioner involvement: a systematic review and meta-analysis. Health Soc Care Commun. 2024;2024(Article ID 6437930):23. Retrieved from 10.1155/2024/6437930.
14.Liljas AEM, Brattstrom F, Burstrom B, Schon P, Agerholm J. Impact of integrated care on patient-related outcomes among older people - a systematic review. Int J Integr Care. 2019;19(3):6. 10.5334/ijic.4632. [DOI] [PMC free article] [PubMed] [Google Scholar]
15.Rawlins M. De Testimonio: on the evidence for decisions about the use of therapeutic interventions. Clin Med (Lond). 2008;8(6):579–88. 10.7861/clinmedicine.8-6-579. [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Dawadi S, Shrestha S, Giri R. Mixed-methods research: a discussion on its types, challenges and criticisms. J Pract Stud Educ. 2021;2(2):25–36. 10.46809/jpse.v2i2.20. [Google Scholar]
17.Adu J, Owusu M, Martin-Yeboah E, Gavidia L, Gyamfi S. A discussion of some controversies in mixed methods research for emerging researchers. Methodological Innovations. 2022;15(3):321–30. 10.1177/20597991221123398. [Google Scholar]
18.Mertens D, Bazeley P, Bowleg L, Fielding N, Maxwell J, Molina-Azorin J, Niglas K. Expanding thinking through a kaleidoscopic look into the future: implications of the mixed methods international research associations task force report on the future of mixed methods. J Mixed Methods Res. 2016;10(3):1–7. 10.1177/1558689816649719. [Google Scholar]
19.Blackwood B, O’Halloran P, Poter S. On the problems of mixing RCTs with qualitative research: the case of the MRC framework for the evaluation of complex interventions. J Res Nurs. 2010;15(6):511–21. 10.1177/1744987110373860. [Google Scholar]
20.Plano Clark VL, Ivankova NV. Mixed methods research: a guide to the field. Thousand Oaks: SAGE Publications; 2016. [Google Scholar]
21.Mannell J, Davis K. Evaluating complex health interventions with randomized controlled trials: how do we improve the use of qualitative methods? Qual Health Res. 2019;29(5):623–31. 10.1177/1049732319831032. [DOI] [PubMed] [Google Scholar]
22.Richards DA, Bazeley P, Borglin G, Craig P, Emsley R, Frost J, O’Cathain A. Integrating quantitative and qualitative data and findings when undertaking randomised controlled trials. BMJ Open. 2019;9(11):e032081. 10.1136/bmjopen-2019-032081. [DOI] [PMC free article] [PubMed] [Google Scholar]
23.Milat A, Lee K, Conte K, Grunseit A, Wolfenden L, van Nassau F, Bauman A. Intervention scalability assessment tool: a decision support tool for health policy makers and implementers. Health Res Policy Syst. 2020;18(1):1. 10.1186/s12961-019-0494-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
24.Fisher K, Ploeg J, Markle-Reid M, Valaitis R, Ganann R, Chambers T, et al. Effectiveness of the Aging, Community and Health Research Unit's Community Partnership Program (ACHRU-CPP) for older adults with diabetes and multiple chronic conditions: a multi-site, pragmatic randomized controlled trial. [Submitted for Publication ]. Trials, (TRLS-D-24_01478). 2024.
25.Northwood M, Chambers T, Fisher K, Ganann R, Markle-Reid M, Yous ML, Whiteside C. Readiness for scale up following effectiveness-implementation trial: results of scalability assessment of the Community Partnership Program for diabetes self-management for older adults with multiple chronic conditions. BMC Health Serv Res. 2025;25(1):284. 10.1186/s12913-025-12378-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Yous, M. L., Ganann, R., Ploeg, J., Markle-Reid, M., Northwood, M., Fisher, K., … Team, A.-C. R. Older adults’ experiences and perceived impacts of the Aging, Community and Health Research Unit-Community Partnership Program (ACHRU-CPP) for diabetes self-management in Canada: a qualitative descriptive study. BMJ Open. 2023;13(4): e068694. 10.1136/bmjopen-2022-068694. [DOI] [PMC free article] [PubMed] [Google Scholar]
27.Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84(2):191–215. 10.1037/0033-295x.84.2.191. [DOI] [PubMed] [Google Scholar]
28.Markle-Reid M, Ploeg J, Fisher K, Reimer H, Kaasalainen S, Gafni A, Upshur R. The Aging, Community and Health Research Unit-Community Partnership Program for older adults with type 2 diabetes and multiple chronic conditions: a feasibility study. Pilot Feasibility Stud. 2016;2:24. 10.1186/s40814-016-0063-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
29.Markle-Reid M, Ploeg J, Fraser KD, Fisher KA, Bartholomew A, Griffith LE, Upshur R. Community program improves quality of life and self-management in older adults with diabetes mellitus and comorbidity. J Am Geriatr Soc. 2018;66(2):263–73. 10.1111/jgs.15173. [DOI] [PMC free article] [PubMed] [Google Scholar]
30.Miklavcic JJ, Fraser KD, Ploeg J, Markle-Reid M, Fisher K, Gafni A, Upshur R. Effectiveness of a community program for older adults with type 2 diabetes and multimorbidity: a pragmatic randomized controlled trial. BMC Geriatr. 2020;20(1):174. 10.1186/s12877-020-01557-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
31.Ploeg, J., Markle-Reid, M., Valaitis, R., Fisher, K., Ganann, R., Blais, J., … Team, A.-C. R. The Aging, Community and Health Research Unit Community Partnership Program (ACHRU-CPP) for older adults with diabetes and multiple chronic conditions: study protocol for a randomized controlled trial. BMC Geriatr. 2022;22(1):99. 10.1186/s12877-021-02651-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Curran GM, Landes SJ, McBain SA, Pyne JM, Smith JD, Fernandez ME, Mittman BS. Reflections on 10 years of effectiveness-implementation hybrid studies. Front Health Serv. 2022;2: 1053496. 10.3389/frhs.2022.1053496. [DOI] [PMC free article] [PubMed] [Google Scholar]
33.Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350: h2147. 10.1136/bmj.h2147. [DOI] [PubMed] [Google Scholar]
34.Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88. 10.1177/1049732305276687. [DOI] [PubMed] [Google Scholar]
35.Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4: 50. 10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
36.Damschroder LJ, Reardon CM, Opra Widerquist MA, Lowery J. Conceptualizing outcomes for use with the Consolidated Framework for Implementation Research (CFIR): the CFIR Outcomes Addendum. Implement Sci. 2022;17(1):7. 10.1186/s13012-021-01181-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
37.Itchhaporia D. The evolution of the quintuple aim: health equity, health outcomes, and the economy. J Am Coll Cardiol. 2021;78(22):2262–4. 10.1016/j.jacc.2021.10.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
38.Nundy S, Cooper LA, Mate KS. The quintuple aim for health care improvement: a new imperative to advance health equity. JAMA. 2022;327(6):521–2. 10.1001/jama.2021.25181. [DOI] [PubMed] [Google Scholar]
39.Proctor EK, Bunger AC, Lengnick-Hall R, Gerke DR, Martin JK, Phillips RJ, Swanson JC. Ten years of implementation outcomes research: a scoping review. Implement Sci. 2023;18(1):31. 10.1186/s13012-023-01286-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
40.Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, Halko H. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12(1):108. 10.1186/s13012-017-0635-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
41.Elwyn G, Barr PJ, Grande SW, Thompson R, Walsh T, Ozanne EM. Developing CollaborRATE: A fast and frugal patientreported measure of shared decision making in clinical enounters. Patient Educ Couns. 2013;93:201–107. 10.1016/j.pec.2013.05.009. [DOI] [PubMed] [Google Scholar]
42.Koenig HG, Westlund RE, George LK, Hughes DC, Blazer DG, Hybels C. Abbreviating the Duke Social Support Index for use in chronically ill elderly individuals. Psychosomatics. 1993;34(1):61–9. 10.1016/S0033-3182(93)71928-3. [DOI] [PubMed] [Google Scholar]
43.Fillenbaum GG, Smyer MA. The development, validity, and reliability of the OARS multidimensional functional assessment questionnaire. J Gerontol. 1981;36(4):428–34. 10.1093/geronj/36.4.428. [DOI] [PubMed] [Google Scholar]
44.Keller HH, Goy R, Kane SL. Validity and reliability of SCREEN II (Seniors in the community: risk evaluation for eating and nutrition, Version II). Eur J Clin Nutr. 2005;59(10):1149–57. 10.1038/sj.ejcn.1602225. [DOI] [PubMed] [Google Scholar]
45.Toobert DJ, Hampson SE, Glasgow RE. The summary of diabetes self-care activities measure: results from 7 studies and a revised scale. Diabetes Care. 2000;23(7):943–50. 10.2337/diacare.23.7.943. [DOI] [PubMed] [Google Scholar]
46.Gandek B, Ware JE, Aaronson NK, Apolone G, Bjorner JB, Brazier JE, Sullivan M. Cross-validation of item selection and scoring for the SF-12 Health Survey in nine countries: results from the IQOLA Project. Int Qual Life Assessment J Clin Epidemiol. 1998;51(11):1171–8. 10.1016/s0895-4356(98)00109-7. [DOI] [PubMed] [Google Scholar]
47.Andresen EM, Malmgren JA, Carter WB, Patrick DL. Screening for depression in well older adults: evaluation of a short form of the CES-D (Center for Epidemiologic Studies Depression Scale). Am J Prev Med. 1994;10(2):77–84. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/8037935. [PubMed] [Google Scholar]
48.Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092–7. 10.1001/archinte.166.10.1092. [DOI] [PubMed] [Google Scholar]
49.Washburn RA, Smith KW, Jette AM, Janney CA. The Physical Activity Scale for the Elderly (PASE): development and evaluation. J Clin Epidemiol. 1993;46(2):153–62. 10.1016/0895-4356(93)90053-4. [DOI] [PubMed] [Google Scholar]
50.Weiss ES, Anderson RM, Lasker RD. Making the most of collaboration: exploring the relationship between partnership synergy and partnership functioning. Health Educ Behav. 2002;29(6):683–98. 10.1177/109019802237938. [DOI] [PubMed] [Google Scholar]
51.Gruneir A, Griffith LE, Fisher K, Perez R, Favotto L, Patterson C, Upshur R. Measuring multimorbidity series. An overlooked complexity - Comparison of self-report vs. administrative data in community-living adults: paper 3. Agreement across data sources and implications for estimating associations with health service use. J Clin Epidemiol. 2020;124:173–82. 10.1016/j.jclinepi.2020.04.018. [DOI] [PubMed] [Google Scholar]
52.Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci. 2016;11:72. 10.1186/s13012-016-0437-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
53.Creamer E, Collings M, Poth C. The implications of a mixed methods way of thinking to practice. Methodological Innovations. 2025;18(1):51–60. 10.1177/2059799251325470. [Google Scholar]
54.Mason J. Mixing methods in a qualitative driven way qualitative research. 2006;6(1):9–25. Retrieved from. 10.1177/1468794106058866.
55.May V. What to do with contradictory data? Realities Toolkit #12. 2010. Retrieved from https://eprints.ncrm.ac.uk/id/eprint/1322/1/12-toolkit-contradictory-data.pdf.
56.Poth C. The contributions of mixed insights to advancing technology-enhanced formative assessments within higher education learning environments: an illustrative example. Int J Educ Technol High Educ. 2018;15(9):1–19. 10.1186/s41239-018-0090-5. [Google Scholar]
57.Wagner KD, Davidson PJ, Pollini RA, Strathdee SA, Washburn R, Palinkas LA. Reconciling incongruous qualitative and quantitative findings in mixed methods research: exemplars from research with drug using populations. Int J Drug Policy. 2012;23(1):54–61. 10.1016/j.drugpo.2011.05.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
58.Massazza A, Roberts B, Fuhr DC, Woodward A, Park A-L, Sondrop E, McDaid D. A qualitative study on the impacts of COVID-19 on the delivery of randomised controlled trials evaluating lay-delivered psychological interventions in five countries. SSM-Mental Health. 2023;4:1–11. 10.1016/j.ssmmh.2023.100251. [Google Scholar]
59.Audisio K, Lia H, Robinson NB, Rahouma M, Soletti G Jr, Cancelli G, et al. Impact of the COVID-19 pandemic on non-COVID-19 clinical trials. J Cardiovasc Dev Dis. 2022;9(1). 10.3390/jcdd9010019. [DOI] [PMC free article] [PubMed]
60.Gaudino M, Arvind V, Hameed I, Di Franco A, Spadaccio C, Bhatt DL, Bagiella E. Effects of the COVID-19 pandemic on active non-COVID clinical trials. J Am Coll Cardiol. 2020;76(13):1605–6. 10.1016/j.jacc.2020.07.051. [DOI] [PMC free article] [PubMed] [Google Scholar]
61.Fleming TR, Labriola D, Wittes J. Conducting clinical research during the COVID-19 pandemic: protecting scientific integrity. JAMA. 2020;324(1):33–4. 10.1001/jama.2020.9286. [DOI] [PubMed] [Google Scholar]
62.Meyer RD, Ratitch B, Wolbers M, Marchenko O, Quan H, Li D, Hale M. Statistical issues and recommendations for clinical trials conducted during the COVID-19 pandemic. Stat Biopharm Res. 2020;12(4):399–411. 10.1080/19466315.2020.1779122. [DOI] [PMC free article] [PubMed] [Google Scholar]
63.Mann C, Shaw ARG, Guthrie B, Wye L, Man MS, Chaplin K, Salisbury C. Can implementation failure or intervention failure explain the result of the 3D multimorbidity trial in general practice: mixed-methods process evaluation. BMJ Open. 2019;9(11): e031438. 10.1136/bmjopen-2019-031438. [DOI] [PMC free article] [PubMed] [Google Scholar]
64.Sullivan-Taylor P, Suter E, Laxton S, Oelke ND, Park E. Integrated people-centred care in Canada - policies, standards, and implementation tools to improve outcomes. Int J Integr Care. 2022;22(1):8. 10.5334/ijic.5943. [DOI] [PMC free article] [PubMed] [Google Scholar]
65.Hough K, Kotwal AA, Boyd C, Tha SH, Perissinotto C. What are “Social Prescriptions” and how should they be integrated into care plans? AMA J Ethics. 2023;25(11):E795-801. 10.1001/amajethics.2023.795. [DOI] [PubMed] [Google Scholar]
66.van Blarikom E, Fudge N, Swinglehurst D. Multimorbidity: a problem in the body, or a problem of the system? Br J Gen Pract. 2023;73(735):443–4. 10.3399/bjgp23X735045. [DOI] [PMC free article] [PubMed] [Google Scholar]
67.Salisbury C, Man MS, Chaplin K, Mann C, Bower P, Brookes S, Thorn J. In A patient-centred intervention to improve the management of multimorbidity in general practice. Southampton (UK): the 3D RCT; 2019. [PubMed] [Google Scholar]
68.Minary L, Trompette J, Kivits J, Cambon L, Tarquinio C, Alla F. Which design to evaluate complex interventions? Toward a methodological framework through a systematic review. BMC Med Res Methodol. 2019;19(1):92. 10.1186/s12874-019-0736-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
69.Seamer P, Lloyd T, Conti S, O’Neill S. The long-term impacts of an integrated care programme on hospital utilisation among older adults in the south of England: a synthetic control study. Int J Integr Care. 2023;23(3):10. 10.5334/ijic.6475. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

13063_2025_8839_MOESM1_ESM.pdf^{(139.9KB, pdf)}

Additional file 1. ACHRU CPP Program of Research Governance StructureGovernance structure for the research program.

13063_2025_8839_MOESM2_ESM.pdf^{(40.5KB, pdf)}

Additional file 2. Timeline for Intervention Delivery and Data Collection at Trial Sites. Start and end time for delivery of intervention and data collection at each RCT site.

13063_2025_8839_MOESM3_ESM.pdf^{(53.5KB, pdf)}

Data Availability Statement

[CR1] 1.Cameron WB. Informal sociology - a casual introduction to sociological thinking. New York: Random House; 1963. [Google Scholar]

[CR2] 2.Hill AB. Heberden oration - reflections on the controlled trial. Annals of Rheumatology Disorders. 1966;25:107–13. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR3] 3.Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., Petticrew, M., & Medical Research Council, G. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337: a1655. 10.1136/bmj.a1655. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR4] 4.Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–26. 10.1097/MLR.0b013e3182408812. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR5] 5.Billings J, de Bruin SR, Baan C, Nijpels G. Advancing integrated care evaluation in shifting contexts: blending implementation research with case study design in project SUSTAIN. BMC Health Serv Res. 2020;20(1):971. 10.1186/s12913-020-05775-5. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR6] 6.ihub. Evaluating new models of care: a guide for integrated care providers. 2017. Scotland Retrieved from ihub.scot. file:///C:/Users/fishe/Downloads/Evaluating%20new%20models%20of%20care-1.pdf.

[CR7] 7.Koziel A, Gorgens M, Chawla M, Krol-Jankowska A, Kononiuk A. Measuring integrated care - methodological reflections from monitoring and evaluation process of the PHC plus pilot program in Poland. Int J Integr Care. 2023;23(2):1. 10.5334/ijic.6646. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR8] 8.Hoogendijk EO. How effective is integrated care for community-dwelling frail older people? The case of the Netherlands. Age Ageing. 2016;45(5):585–8. 10.1093/ageing/afw081. [DOI] [PubMed] [Google Scholar]

[CR9] 9.Hughes G, Shaw SE, Greenhalgh T. Why doesn’t integrated care work? Using Strong Structuration Theory to explain the limitations of an English case. Sociol Health Illn. 2022;44(1):113–29. 10.1111/1467-9566.13398. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR10] 10.Kirst M, Im J, Burns T, Baker GR, Goldhar J, O’Campo P, Wodchis WP. What works in implementation of integrated care programs for older adults with complex needs? A realist review. Int J Qual Health Care. 2017;29(5):612–24. 10.1093/intqhc/mzx095. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR11] 11.Zhang Y, Stokes J, Anselmi L, Bower P, Xu J. Can integrated care interventions strengthen primary care and improve outcomes for patients with chronic diseases? A systematic review and meta-analysis. Health Res Policy Syst. 2025;23(1):5. 10.1186/s12961-024-01260-1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR12] 12.Harnett P, Kennelly S, Williiams P. A 10 step framework to implement integrated are for older persons. Ageing Int. 2020;45:288–304 Retrieved from doi: 10.1007/s12126-019-09349-7. [Google Scholar]

[CR13] 13.Hayes C, Manning M, Fitzgerald C, Condon B, Griffin A, O'Connor M, et al. Effectiveness of community-based multidisciplinary integrated care for older adults with general practitioner involvement: a systematic review and meta-analysis. Health Soc Care Commun. 2024;2024(Article ID 6437930):23. Retrieved from 10.1155/2024/6437930.

[CR14] 14.Liljas AEM, Brattstrom F, Burstrom B, Schon P, Agerholm J. Impact of integrated care on patient-related outcomes among older people - a systematic review. Int J Integr Care. 2019;19(3):6. 10.5334/ijic.4632. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR15] 15.Rawlins M. De Testimonio: on the evidence for decisions about the use of therapeutic interventions. Clin Med (Lond). 2008;8(6):579–88. 10.7861/clinmedicine.8-6-579. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR16] 16.Dawadi S, Shrestha S, Giri R. Mixed-methods research: a discussion on its types, challenges and criticisms. J Pract Stud Educ. 2021;2(2):25–36. 10.46809/jpse.v2i2.20. [Google Scholar]

[CR17] 17.Adu J, Owusu M, Martin-Yeboah E, Gavidia L, Gyamfi S. A discussion of some controversies in mixed methods research for emerging researchers. Methodological Innovations. 2022;15(3):321–30. 10.1177/20597991221123398. [Google Scholar]

[CR18] 18.Mertens D, Bazeley P, Bowleg L, Fielding N, Maxwell J, Molina-Azorin J, Niglas K. Expanding thinking through a kaleidoscopic look into the future: implications of the mixed methods international research associations task force report on the future of mixed methods. J Mixed Methods Res. 2016;10(3):1–7. 10.1177/1558689816649719. [Google Scholar]

[CR19] 19.Blackwood B, O’Halloran P, Poter S. On the problems of mixing RCTs with qualitative research: the case of the MRC framework for the evaluation of complex interventions. J Res Nurs. 2010;15(6):511–21. 10.1177/1744987110373860. [Google Scholar]

[CR20] 20.Plano Clark VL, Ivankova NV. Mixed methods research: a guide to the field. Thousand Oaks: SAGE Publications; 2016. [Google Scholar]

[CR21] 21.Mannell J, Davis K. Evaluating complex health interventions with randomized controlled trials: how do we improve the use of qualitative methods? Qual Health Res. 2019;29(5):623–31. 10.1177/1049732319831032. [DOI] [PubMed] [Google Scholar]

[CR22] 22.Richards DA, Bazeley P, Borglin G, Craig P, Emsley R, Frost J, O’Cathain A. Integrating quantitative and qualitative data and findings when undertaking randomised controlled trials. BMJ Open. 2019;9(11):e032081. 10.1136/bmjopen-2019-032081. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR23] 23.Milat A, Lee K, Conte K, Grunseit A, Wolfenden L, van Nassau F, Bauman A. Intervention scalability assessment tool: a decision support tool for health policy makers and implementers. Health Res Policy Syst. 2020;18(1):1. 10.1186/s12961-019-0494-2. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR24] 24.Fisher K, Ploeg J, Markle-Reid M, Valaitis R, Ganann R, Chambers T, et al. Effectiveness of the Aging, Community and Health Research Unit's Community Partnership Program (ACHRU-CPP) for older adults with diabetes and multiple chronic conditions: a multi-site, pragmatic randomized controlled trial. [Submitted for Publication ]. Trials, (TRLS-D-24_01478). 2024.

[CR25] 25.Northwood M, Chambers T, Fisher K, Ganann R, Markle-Reid M, Yous ML, Whiteside C. Readiness for scale up following effectiveness-implementation trial: results of scalability assessment of the Community Partnership Program for diabetes self-management for older adults with multiple chronic conditions. BMC Health Serv Res. 2025;25(1):284. 10.1186/s12913-025-12378-5. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR26] 26.Yous, M. L., Ganann, R., Ploeg, J., Markle-Reid, M., Northwood, M., Fisher, K., … Team, A.-C. R. Older adults’ experiences and perceived impacts of the Aging, Community and Health Research Unit-Community Partnership Program (ACHRU-CPP) for diabetes self-management in Canada: a qualitative descriptive study. BMJ Open. 2023;13(4): e068694. 10.1136/bmjopen-2022-068694. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR27] 27.Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84(2):191–215. 10.1037/0033-295x.84.2.191. [DOI] [PubMed] [Google Scholar]

[CR28] 28.Markle-Reid M, Ploeg J, Fisher K, Reimer H, Kaasalainen S, Gafni A, Upshur R. The Aging, Community and Health Research Unit-Community Partnership Program for older adults with type 2 diabetes and multiple chronic conditions: a feasibility study. Pilot Feasibility Stud. 2016;2:24. 10.1186/s40814-016-0063-1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR29] 29.Markle-Reid M, Ploeg J, Fraser KD, Fisher KA, Bartholomew A, Griffith LE, Upshur R. Community program improves quality of life and self-management in older adults with diabetes mellitus and comorbidity. J Am Geriatr Soc. 2018;66(2):263–73. 10.1111/jgs.15173. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR30] 30.Miklavcic JJ, Fraser KD, Ploeg J, Markle-Reid M, Fisher K, Gafni A, Upshur R. Effectiveness of a community program for older adults with type 2 diabetes and multimorbidity: a pragmatic randomized controlled trial. BMC Geriatr. 2020;20(1):174. 10.1186/s12877-020-01557-0. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR31] 31.Ploeg, J., Markle-Reid, M., Valaitis, R., Fisher, K., Ganann, R., Blais, J., … Team, A.-C. R. The Aging, Community and Health Research Unit Community Partnership Program (ACHRU-CPP) for older adults with diabetes and multiple chronic conditions: study protocol for a randomized controlled trial. BMC Geriatr. 2022;22(1):99. 10.1186/s12877-021-02651-7. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR32] 32.Curran GM, Landes SJ, McBain SA, Pyne JM, Smith JD, Fernandez ME, Mittman BS. Reflections on 10 years of effectiveness-implementation hybrid studies. Front Health Serv. 2022;2: 1053496. 10.3389/frhs.2022.1053496. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR33] 33.Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350: h2147. 10.1136/bmj.h2147. [DOI] [PubMed] [Google Scholar]

[CR34] 34.Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88. 10.1177/1049732305276687. [DOI] [PubMed] [Google Scholar]

[CR35] 35.Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4: 50. 10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR36] 36.Damschroder LJ, Reardon CM, Opra Widerquist MA, Lowery J. Conceptualizing outcomes for use with the Consolidated Framework for Implementation Research (CFIR): the CFIR Outcomes Addendum. Implement Sci. 2022;17(1):7. 10.1186/s13012-021-01181-5. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR37] 37.Itchhaporia D. The evolution of the quintuple aim: health equity, health outcomes, and the economy. J Am Coll Cardiol. 2021;78(22):2262–4. 10.1016/j.jacc.2021.10.018. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR38] 38.Nundy S, Cooper LA, Mate KS. The quintuple aim for health care improvement: a new imperative to advance health equity. JAMA. 2022;327(6):521–2. 10.1001/jama.2021.25181. [DOI] [PubMed] [Google Scholar]

[CR39] 39.Proctor EK, Bunger AC, Lengnick-Hall R, Gerke DR, Martin JK, Phillips RJ, Swanson JC. Ten years of implementation outcomes research: a scoping review. Implement Sci. 2023;18(1):31. 10.1186/s13012-023-01286-z. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR40] 40.Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, Halko H. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12(1):108. 10.1186/s13012-017-0635-3. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR41] 41.Elwyn G, Barr PJ, Grande SW, Thompson R, Walsh T, Ozanne EM. Developing CollaborRATE: A fast and frugal patientreported measure of shared decision making in clinical enounters. Patient Educ Couns. 2013;93:201–107. 10.1016/j.pec.2013.05.009. [DOI] [PubMed] [Google Scholar]

[CR42] 42.Koenig HG, Westlund RE, George LK, Hughes DC, Blazer DG, Hybels C. Abbreviating the Duke Social Support Index for use in chronically ill elderly individuals. Psychosomatics. 1993;34(1):61–9. 10.1016/S0033-3182(93)71928-3. [DOI] [PubMed] [Google Scholar]

[CR43] 43.Fillenbaum GG, Smyer MA. The development, validity, and reliability of the OARS multidimensional functional assessment questionnaire. J Gerontol. 1981;36(4):428–34. 10.1093/geronj/36.4.428. [DOI] [PubMed] [Google Scholar]

[CR44] 44.Keller HH, Goy R, Kane SL. Validity and reliability of SCREEN II (Seniors in the community: risk evaluation for eating and nutrition, Version II). Eur J Clin Nutr. 2005;59(10):1149–57. 10.1038/sj.ejcn.1602225. [DOI] [PubMed] [Google Scholar]

[CR45] 45.Toobert DJ, Hampson SE, Glasgow RE. The summary of diabetes self-care activities measure: results from 7 studies and a revised scale. Diabetes Care. 2000;23(7):943–50. 10.2337/diacare.23.7.943. [DOI] [PubMed] [Google Scholar]

[CR46] 46.Gandek B, Ware JE, Aaronson NK, Apolone G, Bjorner JB, Brazier JE, Sullivan M. Cross-validation of item selection and scoring for the SF-12 Health Survey in nine countries: results from the IQOLA Project. Int Qual Life Assessment J Clin Epidemiol. 1998;51(11):1171–8. 10.1016/s0895-4356(98)00109-7. [DOI] [PubMed] [Google Scholar]

[CR47] 47.Andresen EM, Malmgren JA, Carter WB, Patrick DL. Screening for depression in well older adults: evaluation of a short form of the CES-D (Center for Epidemiologic Studies Depression Scale). Am J Prev Med. 1994;10(2):77–84. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/8037935. [PubMed] [Google Scholar]

[CR48] 48.Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092–7. 10.1001/archinte.166.10.1092. [DOI] [PubMed] [Google Scholar]

[CR49] 49.Washburn RA, Smith KW, Jette AM, Janney CA. The Physical Activity Scale for the Elderly (PASE): development and evaluation. J Clin Epidemiol. 1993;46(2):153–62. 10.1016/0895-4356(93)90053-4. [DOI] [PubMed] [Google Scholar]

[CR50] 50.Weiss ES, Anderson RM, Lasker RD. Making the most of collaboration: exploring the relationship between partnership synergy and partnership functioning. Health Educ Behav. 2002;29(6):683–98. 10.1177/109019802237938. [DOI] [PubMed] [Google Scholar]

[CR51] 51.Gruneir A, Griffith LE, Fisher K, Perez R, Favotto L, Patterson C, Upshur R. Measuring multimorbidity series. An overlooked complexity - Comparison of self-report vs. administrative data in community-living adults: paper 3. Agreement across data sources and implications for estimating associations with health service use. J Clin Epidemiol. 2020;124:173–82. 10.1016/j.jclinepi.2020.04.018. [DOI] [PubMed] [Google Scholar]

[CR52] 52.Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci. 2016;11:72. 10.1186/s13012-016-0437-z. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR53] 53.Creamer E, Collings M, Poth C. The implications of a mixed methods way of thinking to practice. Methodological Innovations. 2025;18(1):51–60. 10.1177/2059799251325470. [Google Scholar]

[CR54] 54.Mason J. Mixing methods in a qualitative driven way qualitative research. 2006;6(1):9–25. Retrieved from. 10.1177/1468794106058866.

[CR55] 55.May V. What to do with contradictory data? Realities Toolkit #12. 2010. Retrieved from https://eprints.ncrm.ac.uk/id/eprint/1322/1/12-toolkit-contradictory-data.pdf.

[CR56] 56.Poth C. The contributions of mixed insights to advancing technology-enhanced formative assessments within higher education learning environments: an illustrative example. Int J Educ Technol High Educ. 2018;15(9):1–19. 10.1186/s41239-018-0090-5. [Google Scholar]

[CR57] 57.Wagner KD, Davidson PJ, Pollini RA, Strathdee SA, Washburn R, Palinkas LA. Reconciling incongruous qualitative and quantitative findings in mixed methods research: exemplars from research with drug using populations. Int J Drug Policy. 2012;23(1):54–61. 10.1016/j.drugpo.2011.05.009. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR58] 58.Massazza A, Roberts B, Fuhr DC, Woodward A, Park A-L, Sondrop E, McDaid D. A qualitative study on the impacts of COVID-19 on the delivery of randomised controlled trials evaluating lay-delivered psychological interventions in five countries. SSM-Mental Health. 2023;4:1–11. 10.1016/j.ssmmh.2023.100251. [Google Scholar]

[CR59] 59.Audisio K, Lia H, Robinson NB, Rahouma M, Soletti G Jr, Cancelli G, et al. Impact of the COVID-19 pandemic on non-COVID-19 clinical trials. J Cardiovasc Dev Dis. 2022;9(1). 10.3390/jcdd9010019. [DOI] [PMC free article] [PubMed]

[CR60] 60.Gaudino M, Arvind V, Hameed I, Di Franco A, Spadaccio C, Bhatt DL, Bagiella E. Effects of the COVID-19 pandemic on active non-COVID clinical trials. J Am Coll Cardiol. 2020;76(13):1605–6. 10.1016/j.jacc.2020.07.051. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR61] 61.Fleming TR, Labriola D, Wittes J. Conducting clinical research during the COVID-19 pandemic: protecting scientific integrity. JAMA. 2020;324(1):33–4. 10.1001/jama.2020.9286. [DOI] [PubMed] [Google Scholar]

[CR62] 62.Meyer RD, Ratitch B, Wolbers M, Marchenko O, Quan H, Li D, Hale M. Statistical issues and recommendations for clinical trials conducted during the COVID-19 pandemic. Stat Biopharm Res. 2020;12(4):399–411. 10.1080/19466315.2020.1779122. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR63] 63.Mann C, Shaw ARG, Guthrie B, Wye L, Man MS, Chaplin K, Salisbury C. Can implementation failure or intervention failure explain the result of the 3D multimorbidity trial in general practice: mixed-methods process evaluation. BMJ Open. 2019;9(11): e031438. 10.1136/bmjopen-2019-031438. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR64] 64.Sullivan-Taylor P, Suter E, Laxton S, Oelke ND, Park E. Integrated people-centred care in Canada - policies, standards, and implementation tools to improve outcomes. Int J Integr Care. 2022;22(1):8. 10.5334/ijic.5943. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR65] 65.Hough K, Kotwal AA, Boyd C, Tha SH, Perissinotto C. What are “Social Prescriptions” and how should they be integrated into care plans? AMA J Ethics. 2023;25(11):E795-801. 10.1001/amajethics.2023.795. [DOI] [PubMed] [Google Scholar]

[CR66] 66.van Blarikom E, Fudge N, Swinglehurst D. Multimorbidity: a problem in the body, or a problem of the system? Br J Gen Pract. 2023;73(735):443–4. 10.3399/bjgp23X735045. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR67] 67.Salisbury C, Man MS, Chaplin K, Mann C, Bower P, Brookes S, Thorn J. In A patient-centred intervention to improve the management of multimorbidity in general practice. Southampton (UK): the 3D RCT; 2019. [PubMed] [Google Scholar]

[CR68] 68.Minary L, Trompette J, Kivits J, Cambon L, Tarquinio C, Alla F. Which design to evaluate complex interventions? Toward a methodological framework through a systematic review. BMC Med Res Methodol. 2019;19(1):92. 10.1186/s12874-019-0736-6. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR69] 69.Seamer P, Lloyd T, Conti S, O’Neill S. The long-term impacts of an integrated care programme on hospital utilisation among older adults in the south of England: a synthetic control study. Int J Integr Care. 2023;23(3):10. 10.5334/ijic.6475. [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Transforming healthcare by prioritizing qualitative and quantitative clinical trial evidence: evaluating the Aging, Community and Health Research Unit’s Community Partnership Program for Older Adults (ACHRU-CPP)

Kathryn Fisher

Soo Chan Carusone

Rebecca Ganann

Maureen Markle-Reid

Melissa Northwood

Diana Sherifali

Abstract

Background

Methods

Results

Conclusions

Trial registration

Supplementary Information

Introduction

Methods

Foundational studies

Current trial

Table 1.

Table 2.

Qualitative evidence sources from current trial

Conceptual framework guiding evaluation of implementation and intervention outcomes

Fig. 1.

Results

Conceptual model domain: implementation determinants

Table 3.

Intervention (Table 1)

Aim

Source

Differences from usual care

Relative advantage (for patients, providers, system)

Adaptations

Inner setting (Table 2)

Readiness

Climate

Networks and communications

Outer setting (Table 2)

Patient needs and resources

Cosmopolitanism

External policy and incentives

Process (Table 3)

Engaging

Executing (Components-Delivered; Not Delivered—Not Needed; Not Delivered—Desired)

Executing (Components Delivered–Enablers/Barriers)

Executing (Program Delivery–Enablers/Barriers)

Implication: implementation outcomes

Acceptability

Appropriateness

Feasibility

Fidelity

Readiness and climate

Main implementation outcomes

Conceptual model domain: intervention determinants (Table 4)

Table 4.

Patient experience

QUAN

QUAL

Population health

QUAN

QUAL

Costs

QUAN

QUAL

Provider experience

QUAN

QUAL

Health equity

QUAN

QUAL

Implications: intervention outcomes

Summary—comparing quantitative and qualitative quintuple aim findings

Discussion

Quantitative and qualitative research—paradigm differences

COVID-related impacts

RCT designs—reconciling effectiveness and implementation

Next steps in the research program

Conclusions

Supplementary Information

Acknowledgements