Abstract
Background
Delivering evidence-based interventions remains challenging, particularly for complex conditions like chronic musculoskeletal pain. Non-pharmacologic treatments are recommended for many pain conditions, but implementing these can be difficult due to their complexity and resource demands. Pragmatic trials, especially embedded designs, provide a method to see how interventions are being implemented and adapted in real-world settings throughout the trial process. This study explored how intervention delivery complexity and adaptations differ between non-pharmacologic pain trials and non-pain trials to provide guidance on future treatment delivery and implementation.
Methods
From July to October 2023, an online survey was distributed to members of three NIH Trial Collaboratories to assess intervention delivery complexity and adaptations during their pragmatic trials. Participants rated their trial's intervention delivery complexity using a 7-item tool and reported any adaptations to intervention delivery throughout the trial process. Data analysis compared complexity and adaptations between the two trial types to explore differences and relationships between intervention delivery complexity and adaptations.
Results
We analyzed 12 pain and 12 non-pain trials and found that intervention delivery complexity was not discernibly different between the two trial types, however, pain trials did have a slightly higher average intervention delivery complexity, overall. Pain trials also had more adaptations in the workflow domain compared to non-pain trials, while adaptations across other domains were similar between the two types. Workflow emerged as the most challenging domain for adaptation among all trials.
Conclusion
Intervention delivery complexity may be higher for pragmatic trials that are investigating non-pharmacologic pain interventions versus non-pain trials, but only in very specific areas.
Keywords: Pragmatic trials, Implementation science, Adaptations, Non-pharmacologic pain treatments, Intervention delivery
1. Background
Delivering evidence-supported patient interventions remains a challenge across healthcare settings. The volume of clinical guidelines and treatment recommendations has increased exponentially, and research on delivering intervention has also advanced. Yet, translating these evidence-based clinical guidelines and interventions into consistent, high-quality patient care is elusive, especially for complex conditions like chronic musculoskeletal pain. For the last three decades musculoskeletal pain has been a leading cause of global disability [[1], [2], [3]]. Over 80 % of Americans experience musculoskeletal pain in their lifetime and the cost of treating low back pain alone is estimated to be $635 billion each year [4]. While medication is indicated in some cases, non-pharmacologic interventions like physical therapy, cognitive behavioral therapy, and chiropractic care are now widely recommended in clinical practice guidelines as first line treatments for many pain conditions [[4], [5], [6]]. However, implementing these types of interventions can be challenging due to each of them having multiple core intervention components, various implementation strategies, and the need for interactions with multiple people throughout the intervention life cycle.
Non-pharmacologic pain interventions have been demonstrated as efficacious and have a low risk of side-effects; however, these interventions can be resource intensive and complicated to implement due to staffing needs and multistep feedback loops. Thus, they represent a challenge to deliver “at scale” in many care settings [7]. These interventions have been well-studied in highly controlled clinical trials, but little is known about the challenges of moving them into routine clinical practice [7]. Similarly, while explanatory trials provide valuable insights into pain treatments within tightly controlled settings, pragmatic trials offer a more nuanced perspective on how they may perform in real world settings [8]. Pragmatic trials examine how pain interventions fare when delivered in diverse clinical environments and by studying pain intervention delivery in its natural habitat we can uncover the impacts of important considerations like contextual factors [9].
Adaptation is one common occurrence during pragmatic trial intervention delivery, regardless of the treatment being studied. Adaptation is the process of making purposeful modifications to the design or delivery of an intervention, with the goal of improving fit or effectiveness [10,11]. When an intervention is not aligned with the context in which it is being implemented, implementors may make changes either to the intervention itself or change how it is being delivered [12]. Adaptations can be a part of the implementation context as context is anything that interacts or influences an intervention and its implementation [13]. However, these adaptations are often done in a “black box” where changes are made organically, without proactive consideration of how or why they were made. Additionally, when we fail to see expected improvements in clinical outcomes at trial completion (i.e. when efficacy does not result in effectiveness), it's hard to know if the lack of improvement is because of failed delivery, adaptation of the intervention, or the intervention itself. Gaining a better understanding of how adaptation occurs to the intervention or implementation strategies used during pragmatic trials could be a critical next step to improving treatment delivery of non-pharmacologic pain interventions.
Evaluating intervention delivery adaptations for non-pharmacologic pain treatments within real-world settings could help to improve clinical outcomes, reduce unnecessary testing like imaging, and limit exposure to higher risk treatments by increasing the likelihood of implementation success through proactive implementation strategy mapping and resource planning [14]. However, we know very little about the intersection of intervention delivery for pain treatment and adaptations. Since non-pharmacologic pain interventions are often complex by involving individualized, multistep processes with different types of providers and multiple feedback loops, there is a high likelihood that adaptations will be made during implementation. Thus, comparing intervention delivery complexity and adaptations in pragmatic trials for non-pharmacologic pain interventions to those that are investigating interventions that are potentially less complex (e.g., reminders to take a medicine or an electronic record alert) could contribute valuable insights into future implementation approaches. This comparison may reveal more specific aspects of the intervention delivery complexity and adaptations relationship. If the evidence is clear that non-pain interventions are easier to deliver and require less modifications throughout a trial, then that is an important take-away to inform future attempts at improving delivery of non-pharmacologic pain interventions.
This study investigated the connection between intervention delivery complexity and adaptations within pragmatic trials investigating non-pharmacologic pain interventions and to provide context for those trials we also included trials investigating non-pain interventions. Understanding the relationship between intervention delivery complexity and adaptations may guide future investigations of the effectiveness of non-pharmacologic pain interventions by illuminating potential implementation challenges. Additionally, knowing which intervention delivery complexity domains are likely to require adaptations may help with planning future pragmatic trials as well as moving the intervention into real-world practice through informing implementation resource allocation or highlighting where intervention fidelity issues are likely to arise. Including two different trial types in this study may allow for identification of factors specific to the delivery of non-pharmacologic pain interventions.
Therefore, we will explore: 1) differences in intervention delivery complexity between nonpharmacologic pain and non-pain trials, 2) types and frequencies of adaptations across intervention delivery complexity domains between nonpharmacologic pain and non-pain trials, and 3) the relationship between intervention delivery complexity and delivery adaptations for seven domains (i.e., workflow, training demand, number of intervention components, number of healthcare systems, number of clinics, dependency on setting for implementation, and the number of steps between the intervention and the outcome) within the two types of trials.
2. Methods
2.1. Participants
From July to October 2023, principal investigators and/or implementation team members of three NIH Clinical Trial Collaboratories were invited to complete an online survey about intervention delivery complexity and adaptations during their pragmatic trial. Trial team members were contacted by the NIH Pragmatic Trials Collaboratory leadership. Inclusion criteria included membership of the NIH Pragmatic Trial Collaboratory, Pain Management Collaboratory, or IMPACT Collaboratory as an upcoming, ongoing, or completed trial. This study was approved by the Duke University Health System Institutional Review Board (Protocol number 00113504).
2.2. Procedures
Emails inviting study participation were sent to members of the three NIH Trial Collaboratories outlined above and were sent by NIH Pragmatic Trials Collaboratory leadership. The invitation email described the purpose of the study, provided contact information for the study team, brief instructions on who should fill out the survey (i.e., anyone who was familiar with the plans for intervention delivery implementation), and included an option to opt out of future emails. The invitation email was sent three times to all members of the collaboratories over a three-month period. For those interested in study participation, they completed an electronic informed consent. Participants completed the survey via a link to a Qualtrics Survey and all results were blinded. Due to blinding we are unable to list all trials that completed surveys so representative trials from each Collaboratory are summarized in Table 1.
Table 1.
Sample of interventions being tested from participating NIH Collaboratories.
| Pragmatic Trials Collaboratory [15] | Pain Management Collaboratory [16] | Impact Collaboratory [17] |
|---|---|---|
|
Pragmatic Trial of Acupuncture for Chronic Low Back Pain in Older Adults (BackInAction18): The study will compare a standard 3-month course of acupuncture with an enhanced course of acupuncture (3-month standard course, plus 3-month maintenance course) to usual medical care for cLBP. |
Co-operative Pain Education and Self-management: Expanding Treatment for Real-world Access (COPES ExTRA19): The trial objective is to compare an interactive voice response-based form of CBT-CP called Co-operative Pain Education and Self-management (COPES) versus in-person CBT-CP provided by clinicians previously trained through VHA's evidence-based psychotherapy program. |
Collaborative Care Coordination Program for Alzheimer's Disease and Related Dementias (Co-CARE-AD): The Dementia Care Consultation (DCC) program, delivered by a care consultant over a 6- month period, provides an in-depth, personalized service for individuals and families facing ADRD. The program consists of comprehensive needs assessments, creation and implementation of personalized care plans, monitoring and revising care plans, disease education and support coaching, referrals to community-based organizations for service and support, and access to assistance. |
|
Fibromyalgia TENS in Physical Therapy Study (FM-TIPS20): The trial objective is to assess the feasibility of using TENS in addition to physical therapy for treatment of patients with fibromyalgia using a cluster-randomized pragmatic clinical trial of routine physical therapy with or without TENS. |
Targeting Chronic Pain in Primary Care Settings Using Behavioral Health Consultants21: The trial objective is to assess the effect of monthly telehealth booster contacts on long-term Brief Cognitive Behavioral Therapy for Chronic Pain (BCBT-CP) pain outcomes compared to BCBT-CP without a booster in a sample of Military Health Systems (MHS) beneficiaries referred to a Behavioral Health Consultant (BHC) for pain management using BCBT-CP. |
Improving How People with Dementia are Selected for Care Coordination: A Pragmatic Clinical Trial Embedded in an Accountable Care Organization22: The intervention is a novel approach to assign care coordinators to PLWD whose care partners report problems with care coordination based on a survey of perceptions of care coordination. By contrast, usual care assigns PLWD to care coordinators after hospital discharge. |
|
Personalized Patient Data and Behavioral Nudges to Improve Adherence to Chronic Cardiovascular Medications (Nudge23): The trial objective is to employ population-level pharmacy data and to deliver nudges via cell phone text messaging and artificially intelligent (AI) interactive chat bot to improve medication adherence and patient outcomes in 3 integrated healthcare delivery systems (HCS). |
Improving Veteran Access to Integrated Management of Low Back Pain (AIM Back)24: The trial compares the effectiveness of two LBP care pathways designed to enhance access to nonpharmacologic pain treatments according to the most recent practice guidelines and a biopsychosocial approach to care; the first pathway is a sequenced, multi-modal integrated approach incorporating pain modulatory treatment, tailored behavioral treatment, and home-based activity. The second pathway is care management via pain navigator that facilitates coordinated use of existing pain management resources. |
Mitigation of Postoperative Delirium in High-Risk Patients: The intervention consists of clinical decision support alerts in the electronic health record directed towards anesthesiologists caring for patients with preexisting cognitive impairment. This intervention will promote 12 evidence-based best practices during care for perioperative patients. |
2.3. Survey measures
Participant demographic and background characteristics were assessed by self-report in the Qualtrics survey and included NIH Collaboratory membership, trial role, phase, and type, experience with conducting pragmatic trials, primary training discipline, level of implementation science training, and level of familiarity with the concept of intervention adaptation.
2.4. Intervention delivery complexity
The Intervention Delivery Complexity Calculator was developed by the NIH Pragmatic Trials Collaboratory as a standardized way to address the challenges of intervention delivery within a pragmatic trial [25]. Survey participants were asked to rate their trial's intervention delivery complexity across seven different items. The 7-item tool consists of internal and external factors. Internal factors pertain to the intervention itself and include workflow, training, and the number of intervention components. External factors are related to intervention delivery at the system level including differences in healthcare systems, the dependency on setting for implementation, and the number of steps between the intervention and the outcome [25]. Output from the Intervention Delivery Complexity Calculator is calculated on a scale and results in a horizontal bar graph to show where self-reported intervention delivery complexity falls on each of the seven domains.
2.5. Intervention delivery adaptations
Survey participants were asked if any adaptations were made within each of the seven domains from the Intervention Delivery Complexity Calculator (i.e. workflow, training, number of intervention components, etc.). Questions about intervention delivery adaptations were modified from two existing frameworks: the FRAME (Framework for Reporting Adaptations and Modifications to Evidence-Based Interventions) and the FRAME-IS (Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies) [11,26]. When adaptations were made, respondents were asked about the primary goal of the adaptation, an example of an adaptation, and perceived success of the adaptation. Participants were also asked to rank how challenging it was to make adaptations across the seven intervention delivery complexity items (from 1 (most challenging) to 7 (least challenging).
2.6. Survey tool development
We developed the survey to separately assess intervention delivery complexity and associated adaptations by asking study investigators (n = 8) and implementation team members (n = 7) of multiple pragmatic trials (n = 4) for verbal and written feedback on an initial version of survey measures that included the NIH Intervention Delivery Complexity Calculator, an adapted version of the Framework for Reporting Adaptations and Modifications to Evidence-Based Interventions (FRAME), and demographic information [11,25]. Feedback was given over two months by the same study investigators and implementation team members. Each time new survey iterations were developed (approximately 10) until feedback consensus was reached. The final version of the survey was 42 questions and was estimated to take approximately 15 min to complete.
2.7. Analysis of survey data
2.7.1. Intervention complexity
We investigated differences in intervention delivery complexity between pain and non-pain trials. Descriptive differences between trial type and intervention complexity were reported as the frequency of intervention complexity reported within each domain. To further investigate differences between trial types, we created a summative quantitative variable to compare intervention complexity between trial types for each of the tool domains. We recoded the seven categorial variables representing each domain (i.e., workflow, training, number of intervention components, etc.) into interval variables. Each domain was recoded (1–4). One represents least complex and 4 represents most complex (e.g., a “low complexity” score would be as low as 7, “high complexity” score could be as high as 27). We then calculated sum and mean scores to create an “intervention delivery complexity” score and reported means with standard deviations with 95 % confidence intervals.
2.7.2. Intervention delivery adaptations
We investigated the difference in the number and type of adaptations within each intervention delivery complexity domain and between trial types. We reported the frequencies of adaptations selected within each domain. A site could select more than one adaptation per domain. Next, we quantitatively compared adaptations between trial types by creating a summative variable, “Number of Adaptations,” that represents the total number of adaptations across all seven domains. We compared the total number of adaptations by trial groups using independent t-tests. We then calculated the sum of the number of adaptations for each of the seven domains. Next, we ran independent t-tests to look at differences in the sum of adaptations between pain and non-pain trials.
2.7.3. Intersection of intervention complexity and intervention delivery adaptations
We investigated the relationship between intervention delivery complexity and number of delivery adaptations required to deliver the intervention within each of the seven domains. We collapsed the number of delivery adaptation into two binary variables (0 = no adaptation reported, 1 = one or more adaptations reported). We then compared the frequency of adaptations by domain (i.e., overall complexity) using a chi-squared analysis. The intervention delivery complexity domain, workflow, did not have any “zero adaptations” answers, therefore we analyzed the workflow adaptations individually. Normality was tested using the Shapiro-Wilk test and 2-sided P-values.
3. Results
We collected surveys for 12 pain studies and 12 non-pain studies (out of 30 total studies, response rate = 80%). Study investigators were most likely to complete the survey and intervention versus usual care (A vs. A+) was the most common trial type for both pain and non-pain trials. Medicine was the most common training discipline and levels of implementation science training and familiarity with adapting intervention delivery were at similar levels for both types of trials (Table 2).
Table 2.
Survey participant characteristics.
| Pain | Non-Pain | |
|---|---|---|
| N | 12 | 12 |
| Trial Role, category (%) | ||
| Investigator | 9 (75.0) | 9 (75.0) |
| Implementation Specialist | 1 (8.3) | 3 (25.0) |
| Other | 1 (8.3) | 0 (0.0) |
| Unknown | 1 (8.3) | 0 (0.0) |
| Trial Phase, category (%) | ||
| Complete | 3 (25) | 5 (41.7) |
| Ongoing | 8 (66.7) | 4 (33.3) |
| Planning | 0 (0.0) | 3 (25.0) |
| Unknown | 1 (8.3) | 0 (0.0) |
| Trial Description, category (%) | ||
| Intervention vs usual care | 7 (58.3) | 8 (66.7) |
| Comparative effectiveness | 3 (25.0) | 3 (25.0) |
| Other | 1 (8.3) | 1 (8.3) |
| Unknown | 1 (8.3) | 0 (0.0) |
| Pragmatic Trial Experience | ||
| None | 4 (33.3) | 4 (33.3) |
| 1 Trial | 2 (16.7) | 2 (16.7) |
| 2–3 Trials | 3 (25.0) | 2 (16.7) |
| 4-5 Trials | 2 (16.7) | 0 (0.0) |
| >5 Trials | 0 (0.0) | 0 (0.0) |
| Unknown | 1 (8.3) | 4 (33.3) |
| Primary training discipline | ||
| Medicine | 4 (33.3) | 5 (41.7) |
| Physical Therapy | 3 (25.0) | 0 (0.0) |
| Psychology | 2 (16.7) | 1 (8.3) |
| Public Health | 1 (8.3) | 2 (16.7) |
| Health Services | 1 (8.3) | 0 (0.0) |
| Unknown | 1 (8.3) | 4 (33.3) |
| Level of implementation science training | ||
| None | 3 (25.0) | 1 (8.3) |
| A little | 2 (16.7) | 2 (16.7) |
| Moderate amount | 7 (58.3) | 4 (33.3) |
| A lot Unknown |
0 (0.0) 0 (0.0) |
1 (8.3) 4 (33.3) |
| Level of familiarity with concept of intervention adaptation | ||
| None | 2 (16.7) | 0 (0.0) |
| A little | 2 (16.7) | 2 (16.7) |
| Moderate | 7 (58.3) | 5 (41.7) |
| A lot | 1 (8.3) | 0 (0.) |
| Unknown | 0 (0.0) | 5 (41.7) |
3.1. Intervention delivery complexity between pain and non-pain trials
Table 3 shows the self-reported intervention delivery complexity for both types of trials. When aggregated, the average overall intervention delivery complexity score was not discernibly different between the two trial types, however descriptively, pain trials did have a slightly higher average complexity (i.e., pain = 3.02, SD = 0.46, non-pain = 2.79, SD = 0.44). The range for pain trials was 1.16 and non-pain trials was 1.33. New tasks with modified workflows were common experiences, with 100% of pain trials and 92% of non-pain trials reporting new tasks with modified workflows. Fig. 1 illustrates there were no differences between the two trial types for intervention delivery complexity at the domain level, however, the workflow domain had the biggest descriptive difference between pain and non-pain trials (i.e., pain = 3.58, non-pain = 3.08). Similarly, 58% of pain trials reported three or more intervention components in their trial versus 33% of non-pain trials. The training demand, number of health systems, number of steps for intervention delivery, and dependency on setting showed similar complexity across pain and non-pain trials. Table 3 shows the self-reported intervention delivery complexity for both types of trials.
Table 3.
Intervention delivery complexity by domain.
| Pain | Non-Pain | |
|---|---|---|
| Workflow | ||
| No changes were made to this domain | 0 (0.00 %) | 0 (0.00 %) |
| Modified workflow, no new tasks | 0 (0.00 %) | 1 (8.33 %) |
| Modified workflow, new tasks | 5 (41.67 %) | 8 (66.67 %) |
| New workflow, new tasks | 7 (58.33 %) | 3 (25.00 %) |
| Missing | 0 (0.00 %) | 0 (0.00 %) |
| Total | 12 (100.00 %) | 12 (100.00 %) |
| Training Demand | ||
| No training | 1 (8.33 %) | 2 (16.67 %) |
| Refresh of existing skills | 3 (25.00 %) | 3 (25.00 %) |
| Training for a new skill | 6 (50.00 %) | 5 (41.67 %) |
| Training of multiple new skills | 2 (16.67 %) | 2 (16.67 %) |
| Missing | 0 (0.00 %) | 0 (0.00 %) |
| Total | 12 (100.00 %) | 12 (100.00 %) |
| Intervention Components | ||
| Three + components | 7 (58.33 %) | 4 (33.33 %) |
| Two components | 1 (8.33 %) | 2 (16.67 %) |
| One component | 4 (33.33 %) | 6 (50.00 %) |
| Missing | 0 (0.00 %) | 0 (0.00 %) |
| Total | 12 (100.00 %) | 12 (100.00 %) |
| Number of Health Systems | ||
| 4 or more systems | 7 (58.33 %) | 6 (50.00 %) |
| 3 systems | 1 (8.33 %) | 2 (16.67 %) |
| 2 systems | 0 (0.00 %) | 1 (8.33 %) |
| 1 system | 4 (33.33 %) | 3 (25.00 %) |
| Missing | 0 (0.00 %) | 0 (0.00 %) |
| Total | 12 (100.00 %) | 12 (100.00 %) |
| Number of Steps in Pathway | ||
| Pathway is short (only one or two steps between intervention and outcomes), direct, and linear | 4 (33.33 %) | 5 (41.67 %) |
| Pathway is longer (three or more steps between intervention and outcomes) but still linear | 3 (25.00 %) | 3 (25.00 %) |
| Non-linear (including the potential for more than one provider) | 1 (8.33 %) | 1 (8.33 %) |
| Variable steps, long pathway, multiple providers | 4 (33.33 %) | 3 (25.00 %) |
| Missing | 0 (0.00 %) | 0 (0.00 %) |
| Total | 12 (100.00 %) | 12 (100.00 %) |
| Number of Clinics | ||
| 4 or more clinics | 10 (83.33 %) | 9 (75.00 %) |
| 3 clinics | 0 (0.00 %) | 0 (0.00 %) |
| 2 clinics | 1 (8.33 %) | 0 (0.00 %) |
| 1 clinic | 1 (8.33 %) | 1 (8.33 %) |
| Missing | 0 (0.00 %) | 2 (16.67 %) |
| Total | 12 (100.00 %) | 12 (100.00 %) |
| Dependency on Setting | ||
| Not dependent on setting (could be delivered in any setting) | 1 (8.33 %) | 2 (16.67 %) |
| Minimally dependent on setting (could be delivered in low resource setting) | 4 (33.33 %) | 4 (33.33 %) |
| Moderately dependent on setting | 5 (41.67 %) | 5 (41.67 %) |
| Largely dependent on setting (could only be delivered in a high resource setting) | 2 (16.67 %) | 1 (8.33 %) |
| Missing | 0 (0.00 %) | 0 (0.00 %) |
| Total | 12 (100.00 %) | 12 (100.00 %) |
Fig. 1.
Overall complexity and adaptations by domains for pain trials vs non-pain trials
Note: Negative numbers on X-axis in Fig. 1 are illustrative only and meant to highlight differences between trial types.
3.2. Adaptations across intervention delivery complexity domains between pain and non-pain trials
Similar to overall intervention delivery complexity, total adaptations by domain showed no discernible difference between pain and non-pain trials (Table 4). However, pain trials reported a higher average number of adaptations to workflow (i.e., pain = 2.42, non-pain = 1.58). Pain trials reported 29 adaptations to the workflow domain and non-pain trials reported 21 adaptations. The most common adaptation to workflow was a change in how patients were recruited or enrolled for both trial types, followed by a change in communication strategies between trial and intervention teams. The most common reason for adapting workflows for pain trials was to increase reach or enrollment versus to improve fit with recipients for non-pain trials. The overall number of adaptations for training demand, intervention components, number of healthcare systems/clinics, number of steps, and dependency on setting were similar across the two trial types. Finally, neither type of trial reported that their adaptations were unsuccessful. When aggregated, the total number of adaptations by domain for both trial types were highest for number of healthcare systems/clinics, changes to pathways, and workflow (Table 5).
Table 4.
Total adaptations by domain.
| Pain |
Non-Pain |
||||||
|---|---|---|---|---|---|---|---|
| Domain | M | SD | n | M | SD | n | p (95 %) |
| Workflow | 2.42 | 1.38 | 12 | 1.58 | 1.24 | 12 | 0.134 |
| Training Demand | 0.83 | 0.58 | 12 | 0.75 | 0.45 | 12 | 0.698 |
| Intervention Components | 1.00 | 1.13 | 12 | 0.67 | 0.65 | 12 | 0.385 |
| Number of Health Systems/Clinics | 2.92 | 1.44 | 12 | 2.92 | 1.31 | 12 | 1.000 |
| Number of Steps in Pathway | 2.42 | 1.31 | 12 | 2.17 | 1.27 | 12 | 0.640 |
| Changed Setting | 3.58 | 0.99 | 12 | 3.50 | 1.168 | 12 | 0.850 |
Table 5.
Total number of adaptations by domain for both types of trials.
| Domain | M | SD | n | SEM | Mdn | Min | Max |
|---|---|---|---|---|---|---|---|
| Workflow | 2.00 | 1.35 | 24 | 0.28 | 2 | 0 | 5 |
| Training Demand | 0.79 | 0.51 | 24 | 0.1 | 1 | 0 | 2 |
| Intervention Components | 0.83 | 0.92 | 24 | 0.19 | 1 | 0 | 4 |
| Number of Health Systems/Clinics | 2.29 | 1.27 | 24 | 0.26 | 2 | 1 | 4 |
| Number of Steps in Pathway | 2.92 | 1.35 | 24 | 0.28 | 4 | 1 | 4 |
| Changed Setting | 3.54 | 1.06 | 24 | 0.22 | 4 | 1 | 4 |
3.3. Relationship between intervention delivery complexity and adaptations for seven domains
When looking at adaptations categorically (i.e., no changes at all versus at least one change), workflow was the only domain among pain and non-pain trials where there were zero reports of “no changes” (i.e., all pain trials adapted their workflows in some way). Finally, when asked which domain was the most challenging to adapt, the most challenging was workflow (54 %), followed by training demand (15 %), and adapting the number of intervention steps (9 %) across both trial types.
4. Discussion
Non-pharmacologic pain interventions have the potential to have high intervention delivery complexity and the need for more adaptations to successfully deliver the intervention. Persistent pain is difficult to treat and non-pharmacologic interventions that have been guideline recommended as early treatment options (e.g. physical therapy, cognitive behavioral therapy, chiropractic care) often involve multifaceted treatment plans. Thus, introducing these non-pharmacologic interventions into existing workflows within a health system may be potentially more complicated than implementing simpler interventions like pharmacologic treatments. In this study, we found that there were some specific differences based on trial type for the complexity of intervention delivery and adaptations made during a pragmatic trial (Fig. 1). However, when we examined these phenomena collectively, we found no wide-ranging differences in intervention delivery complexity and the associated adaptations between pain and non-pain trials.
Workflow did have the highest signal for intervention delivery complexity and need for adaptations, especially among pain trials. This finding aligns with other research regarding the complicated nature of implementing non-pharmacologic pain interventions, however our study is one of the first to investigate intervention delivery complexity using a previously described assessment tool [25,27,28]. Prior research has focused on provider perspectives of challenges to implementing pain interventions or issues with payment models for pain care. For example, Ng et al. conducted a systematic review to investigate barriers and facilitators to adopting biopsychosocial approaches to pain care and found barriers at the individual practitioner level including misconceptions about practice guidelines, at the mesolevel including funding models and workforce training issues, and at the macrolevel including health policy and organizational, factors [29]. Similarly, George et al. have discussed barriers to implementing non-pharmacologic pain care such as primary care providers citing short consultation times that prevent them from exploring non-pharmacologic options or patient level barriers like the belief that imaging must be done before proceeding with non-pharmacologic pain interventions [30]. Both studies cite multilevel workflow domain factors that may account for higher intervention delivery complexity and the need to adapt workflows within pain trials.
We identified evidence that pain trials had increased intervention delivery complexity due to adding new tasks, modifying existing workflows, and having an additional number of intervention components. However, a lack of discernible differences between the two trial types may indicate that the burden of making any type of change, no matter the trial type, will affect other processes. This could be because any time a new task or process is added to a system, regardless of the magnitude of the change, other areas of the system are affected. Healthcare delivery happens within an interconnected system where individuals have the autonomy to act in ways that may not follow predictable patterns and these individual actions influence contextual factors throughout that system [31,32]. Thus, while it may seem that more complex intervention processes, like many non-pharmacologic treatment options, would require a greater number of adaptations, our study results did not strongly support such an assertion. Generally, non-pharmacologic pain interventions may show isolated areas of increased intervention delivery complexity but overall, they may not be more complicated to implement than any other type of intervention. Similarly, what is “complicated” to implement for one healthcare system may be easier for another system that has implemented similar interventions previously. Future research could investigate these areas of increased delivery complexity and could look at adaptations over time to see if they become static or are constantly evolving. There could be some intervention delivery domains that are easily implemented and minorly adapted, while others may require constant monitoring.
There are several potential explanations as to why pain trials were not discernibly different from non-pain trials in intervention delivery complexity in our study. First, we chose to survey the NIH Collaboratories that focus on pragmatic trials and this could have resulted in selection bias for higher complexity trials, regardless of intervention type. Using surveys from all NIH funded trials could also limit generalizability to non-NIH funded work or beyond larger health systems where these types of complex studies take place. Additionally, all trials from the Pain Management Collaboratory took place in either the Department of Veteran's Affairs or the Military Health System which are both closed, hierarchical systems. It could be less complicated to implement interventions in these types of systems which could also contribute to the lack of discernible differences in intervention delivery complexity between the two trial types. Future research can address this issue by studying complexity and adaptation outside of established clinical trial collaboratories. Another potential explanation is that our survey did not capture the level of detail required to expose a true difference in intervention delivery complexity or related adaptations between the two types of trials. For example, the Intervention Delivery Complexity Calculator was originally designed to be concise and facilitate communication between the trial team and health system partners, not to provide an exhaustive complexity list. While using this tool in our survey was helpful to show high-level information related to seven intervention delivery complexity domains, we were not able to capture the intricacies of intervention delivery for each of the individual trials. Future research could highlight potential modifications to the tool for a more comprehensive picture of intervention delivery complexity. Similarly, development of a standardized tool to capture intervention delivery adaptations throughout the trial process could help to provide a more systematic way of identifying common delivery modifications. Finally, all trials in our study were pragmatic in some form. Intervention delivery complexity and subsequent adaptations could be similar across pragmatic trials regardless of the treatment being tested.
Additionally, information about intervention delivery complexity and adaptations were captured retrospectively for 8 completed trials and prospectively for 3 trials in the planning phase (out of 24). While having trials in multiple phases allowed us to see adaptations across a spectrum, gathering this type of information retrospectively for the completed trials may not provide the most accurate information, specifically, teams may focus only on adaptations that were successful. This recall bias could give a false impression that all adaptations were successful when in reality there were multiple attempts to get to the successful one. Similarly, asking trials that are in their planning phase to comment on intervention delivery complexity does not allow for consideration of what happened during the trial. It may be that trials rated themselves less complex during the planning phase. Importantly, we did not ask trial teams about fidelity consistency with their reported adaptations. Adaptations that are made proactively during pre-implementation based on what's anticipated to happen could be inherently different than adaptations that are made during the implementation process. Adaptations made during pre-implementation are more likely to be fidelity consistent versus those that are made in response to real-time health system or patient needs. Future research could investigate a higher number of trials in each phase to capture a more complete picture of complexity and delivery modifications across the trial lifespan. Also, the information gathered on the survey is subjective and different trial team members may have differing perspectives on delivery complexity or adaptation. Investigators accounted for 75 % of survey participants for both type of trials and an investigator's perspective on adaptations could be different from members of the implementation team as they may be more aware of everyday implementation challenges.
Future investigations on implementing non-pharmacologic pain interventions should have a real-time method of capturing intervention delivery adaptations rather than relying on retrospective data collection. Based on our investigation, capturing adaptations retrospectively may provide a limited perspective. Real-time assessment may be more accurate and objective given our survey seemed to only capture adaptations that were successful. Trial specific adaptation checklists could be added to electronic health records or trial team data captures where designated team members could gather detailed information about why an adaptation is being made, who made the adaptation, whether it was successful, etc. Use of a framework on adaptations is helpful for deciding what adaptation data to capture [11,26]. Gathering this type of implementation data will allow other investigators and implementation team members to see what may or may not work as they're trying to implement a similar intervention. However, findings from even the most well-designed pragmatic trials may be limited in their ability to guide implementation studies in different health systems due to the nuances and specificity of workflows in other health systems. Relatedly, using an adaption framework to capture real-time adaptation data may help with choosing future implementation strategies that may be a better fit for that type of intervention delivery which could improve implementation success. Other ways to shed more light on the relationship between intervention delivery complexity, adaptations, and implementation success could be to conduct primary implementation studies to look at differences in adaptations between comparison groups that were implementing the same intervention or to gather qualitative data about adaptations to get a more nuanced perspective on what is happening during intervention delivery.
5. Conclusion
This study investigated the relationship between intervention delivery complexity, adaptations, and implementation by comparing non-pharmacologic pain and non-pain intervention pragmatic trials. Our findings suggest that intervention delivery complexity may be higher for pragmatic trials that are investigating non-pharmacologic pain interventions versus non-pain trials but only in very specific areas. Additionally, changes in workflow was an important consideration for intervention delivery for all trials in our study. Future research should capture detailed, real-time information about the nature of intervention delivery complexity, adaptations, and implementation success to help improve delivery of non-pharmacologic pain interventions.
CRediT authorship contribution statement
Lindsay A. Ballengee: Writing – review & editing, Writing – original draft, Visualization, Validation, Software, Project administration, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Maggie E. Horn: Writing – review & editing, Visualization, Validation, Software, Methodology, Formal analysis, Data curation. Trevor A. Lentz: Writing – review & editing, Visualization, Validation, Supervision, Methodology, Investigation. Devon Check: Writing – review & editing, Visualization, Validation, Supervision, Methodology, Investigation. Leah L. Zullig: Writing – review & editing, Visualization, Validation, Supervision, Methodology, Formal analysis, Conceptualization. Steven Z. George: Writing – review & editing, Visualization, Validation, Supervision, Methodology, Investigation, Formal analysis, Data curation, Conceptualization.
Data availability
Data will be made available upon request.
Funding
This work was supported within the National Institutes of Health (NIH) Pragmatic Trials Collaboratory through cooperative agreement U24AT009676 from the National Center for Complementary and Integrative Health (NCCIH), the National Institute of Allergy and Infectious Diseases (NIAID), the National Cancer Institute (NCI), the National Institute on Aging (NIA), the National Heart, Lung, and Blood Institute (NHLBI), the National Institute of Nursing Research (NINR), the National Institute of Minority Health and Health Disparities (NIMHD), the National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS), the NIH Office of Behavioral and Social Sciences Research (OBSSR), and the NIH Office of Disease Prevention (ODP). This work was also supported by the NIH through the NIH HEAL Initiative under award number U24AT010961 The content is solely the responsibility of the authors and does not necessarily represent the official views of the NCCIH, NIAID, NCI, NIA, NHLBI, NINR, NIMHD, NIAMS, OBSSR, or ODP, or the NIH or its HEAL Initiative.
Declaration of competing interest
The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Lindsay Ballengee reports financial support was provided by NIH Pragmatic Trials Collaboratory. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
The authors would like to thank the NIH Pragmatic Trials Collaboratory, specifically Karen Staman and Tammy Reece for their help with survey design and administration. We would also like to thank members of the NIH Pain Management Collaboratory and NIH IMPACT Collaboratory for their participation in this project. Research reported in this publication was also made possible by Grant Number U24 AT009769 from the National Center for Complementary and Integrative Health (NCCIH), and the Office of Behavioral and Social Sciences Research (OBSSR). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NCCIH, OBSSR, and the National Institutes of Health. This manuscript is a product of the NIH-DOD-VA Pain Management Collaboratory. For more information about the Collaboratory, visit https://painmanagementcollaboratory.org/.
References
- 1.Jackson T., Thomas S., Stabile V., Shotwell M., Han X., McQueen K. A systematic review and meta-analysis of the global burden of chronic pain without clear etiology in low- and middle-income countries: trends in heterogeneous data and a proposal for new assessment methods. Anesth. Analg. 2016;123(3):739–748. doi: 10.1213/ANE.0000000000001389. [DOI] [PubMed] [Google Scholar]
- 2.Wu A., March L., Zheng X., et al. Global low back pain prevalence and years lived with disability from 1990 to 2017: estimates from the Global Burden of Disease Study 2017. Ann. Transl. Med. 2020;8(6):299. doi: 10.21037/atm.2020.02.175. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kim L.H., Vail D., Azad T.D., et al. Expenditures and health care utilization among adults with newly diagnosed low back and lower extremity pain. JAMA Netw. Open. 2019;2(5) doi: 10.1001/jamanetworkopen.2019.3676. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Simon L.S. Relieving pain in America: a blueprint for transforming prevention, care, education, and research. J. Pain Palliat. Care Pharmacother. 2012;26(2):197–198. doi: 10.3109/15360288.2012.678473. [DOI] [Google Scholar]
- 5.Qaseem A., Wilt T.J., McLean R.M., Forciea M.A. Noninvasive treatments for acute, subacute, and chronic low back pain: a clinical practice guideline from the American college of physicians. Ann. Intern. Med. 2017;166(7):514–530. doi: 10.7326/M16-2367. [DOI] [PubMed] [Google Scholar]
- 6.WHO guideline for non-surgical management of chronic primary low back pain in adults in primary and community care settings. https://www.who.int/publications-detail-redirect/9789240081789 [PubMed]
- 7.Ballengee L.A., Zullig L.L., George S.Z. Implementation of psychologically informed physical therapy for low back pain: where do we stand, where do we go? J. Pain Res. 2021;14:3747–3757. doi: 10.2147/JPR.S311973. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Ford I., Norrie J. Pragmatic trials. N. Engl. J. Med. 2016;375(5):454–463. doi: 10.1056/NEJMra1510059. [DOI] [PubMed] [Google Scholar]
- 9.Concannon T.W., Guise J.M., Dolor R.J., et al. A national strategy to develop pragmatic clinical trials infrastructure. Clinical and Translational Science. 2014;7(2):164–171. doi: 10.1111/cts.12143. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wiltsey Stirman S., Gamarra J.M., Bartlett B.A., Calloway A., Gutner C.A. Empirical examinations of modifications and adaptations to evidence-based psychotherapies: methodologies, impact, and future directions. Clin. Psychol. Sci. Pract. 2017;24(4):396–420. doi: 10.1111/cpsp.12218. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Wiltsey Stirman S., Baumann A.A., Miller C.J. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement. Sci. 2019;14(1):58. doi: 10.1186/s13012-019-0898-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Chambers D.A., Norton W.E. The adaptome: advancing the science of intervention adaptation. Am. J. Prev. Med. 2016;51(4, Supplement 2):S124–S131. doi: 10.1016/j.amepre.2016.05.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Pfadenhauer L.M., Gerhardus A., Mozygemba K., et al. Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework. Implement. Sci. 2017;12(1):21. doi: 10.1186/s13012-017-0552-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Chou R., Deyo R., Friedly J., et al. Nonpharmacologic therapies for low back pain: a systematic review for an American college of physicians clinical practice guideline. Ann. Intern. Med. 2017;166(7):493–505. doi: 10.7326/M16-2459. [DOI] [PubMed] [Google Scholar]
- 15.NIH Collaboratory Rethinking Clinical Trials - The Living Textbook. Rethinking Clinical Trials. Accessed November 15, 2023. https://rethinkingclinicaltrials.org/.
- 16.PMC3 | NIH-DoD-VA Pain Management Collaboratory. Accessed November 13, 2019./home/pmc3/. [DOI] [PMC free article] [PubMed]
- 17.Demonstration Projects Program NIA IMPACT collaboratory. https://impactcollaboratory.org/grants-and-training/demonstration-projects/demonstration-grants-program/
- 18.DeBar L.L., Justice M., Avins A.L., et al. Acupuncture for chronic low back pain in older adults: design and protocol for the BackInAction pragmatic clinical trial. Contemp. Clin. Trials. 2023;128 doi: 10.1016/j.cct.2023.107166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Heapy A.A., Driscoll M.A., Buta E., et al. Co-operative pain education and self-management (COPES) expanding treatment for real-world access (ExTRA): pragmatic trial protocol. Pain Med. 2020;21(12 Suppl 2):S21–S28. doi: 10.1093/pm/pnaa365. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Post A.A., Dailey D.L., Bayman E.O., et al. The fibromyalgia transcutaneous electrical nerve stimulation in physical therapy study protocol: a multisite embedded pragmatic trial. Phys. Ther. 2022;102(11) doi: 10.1093/ptj/pzac116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Goodie J.L., Kanzler K.E., McGeary C.A., et al. Targeting chronic pain in primary care settings by using behavioral health consultants: methods of a randomized pragmatic trial. Pain Med. 2020;21(Suppl 2):S83–S90. doi: 10.1093/pm/pnaa346. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Luers E. NIA IMPACT Collaboratory. May 25, 2022. Lisa kern, MD, MPH.https://impactcollaboratory.org/lisa-kern-md-mph/ [Google Scholar]
- 23.Mccamic E. Grand rounds november 17, 2023: personalized patient data and behavioral nudges to improve adherence to chronic cardiovascular medications: results from the nudge study (michael Ho, MD, PhD; sheana bull, PhD) Rethinking Clinical Trials. November 21, 2023 https://rethinkingclinicaltrials.org/news/grand-rounds-november-17-2023-personalized-patient-data-and-behavioral-nudges-to-improve-adherence-to-chronic-cardiovascular-medications-results-from-the-nudge-study-michael-ho-md-phd-sheana-bu/ [Google Scholar]
- 24.George S.Z., Coffman C.J., Allen K.D., et al. Improving veteran access to integrated management of back pain (AIM-Back): protocol for an embedded pragmatic cluster-randomized trial. Pain Med. 2020;21(Suppl 2):S62–S72. doi: 10.1093/pm/pnaa348. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Staman K.L., Check D.K., Zatzick D., et al. Intervention delivery for embedded pragmatic clinical trials: development of a tool to measure complexity. Contemp. Clin. Trials. 2023;126 doi: 10.1016/j.cct.2023.107105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Miller C.J., Barnett M.L., Baumann A.A., Gutner C.A., Wiltsey-Stirman S. The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare. Implement. Sci. 2021;16(1):36. doi: 10.1186/s13012-021-01105-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Delitto A., Patterson C.G., Stevans J.M., et al. Stratified care to prevent chronic low back pain in high-risk patients: the TARGET trial. A multi-site pragmatic cluster randomized trial. eClinicalMedicine. 2021;34 doi: 10.1016/j.eclinm.2021.100795. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Middleton A., Fitzgerald G.K., Delitto A., Saper R.B., Gergen Barnett K., Stevans J. Implementing stratified care for acute low back pain in primary care using the STarT Back instrument: a process evaluation within the context of a large pragmatic cluster randomized trial. BMC Muscoskelet. Disord. 2020;21(1):776. doi: 10.1186/s12891-020-03800-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Ng W., Slater H., Starcevich C., Wright A., Mitchell T., Beales D. Barriers and enablers influencing healthcare professionals' adoption of a biopsychosocial approach to musculoskeletal pain: a systematic review and qualitative evidence synthesis. Pain. 2021 doi: 10.1097/j.pain.0000000000002217. Published online January 29. [DOI] [PubMed] [Google Scholar]
- 30.George S.Z., Lentz T.A., Goertz C.M. Back and neck pain: in support of routine delivery of non-pharmacologic treatments as a way to improve individual and population health. Transl. Res. 2021 doi: 10.1016/j.trsl.2021.04.006. Published online April 23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Plsek P.E., Greenhalgh T. Complexity science: the challenge of complexity in health care. BMJ. 2001;323(7313):625–628. doi: 10.1136/bmj.323.7313.625. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Matlow A.G., Wright J.G., Zimmerman B., Thomson K., Valente M. How can the principles of complexity science be applied to improve the coordination of care for complex pediatric patients? Qual. Saf. Health Care. 2006;15(2):85–88. doi: 10.1136/qshc.2005.014605. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Data will be made available upon request.