Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 May 2.
Published in final edited form as: Contemp Clin Trials. 2018 Mar 6;67:91–99. doi: 10.1016/j.cct.2018.02.015

Interdisciplinary team-based care for patients with chronic pain on long-term opioid treatment in primary care (PPACT) – Protocol for a pragmatic cluster randomized trial

Lynn DeBar a,*,1, Lindsay Benes a,b, Allison Bonifay a, Richard A Deyo c, Charles R Elder a, Francis J Keefe d, Michael C Leo a, Carmit McMullen a, Meghan Mayhew a, Ashli Owen-Smith e,f, David H Smith a, Connie M Trinacty g, William M Vollmer a
PMCID: PMC5931339  NIHMSID: NIHMS958284  PMID: 29522897

Abstract

Background

Chronic pain is one of the most common, disabling, and expensive public health problems in the United States. Interdisciplinary pain management treatments that employ behavioral approaches have been successful in helping patients with chronic pain reduce symptoms and regain functioning. However, most patients lack access to such treatments. We are conducting a pragmatic clinical trial to test the hypothesis that patients who receive an interdisciplinary biopsychosocial intervention, the Pain Program for Active Coping and Training (PPACT), at their primary care clinic will have a greater reduction in pain impact in the year following than patients receiving usual care.

Methods/design

This is an effectiveness-implementation hybrid pragmatic clinical trial in which we randomize clusters of primary care providers and their patients with chronic pain who are on long-term opioid therapy to 1) receive an interdisciplinary behavioral intervention in conjunction with their current health care or 2) continue with current health care services. Our primary outcome is pain impact (a composite of pain intensity and pain-related interference) measured using the PEG, a validated three-item assessment. Secondary outcomes include pain-related disability, patient satisfaction, opioids dispensed and health care utilization. An economic evaluation assesses the resources and costs necessary to deliver the intervention and its cost-effectiveness compared with usual care. A formative evaluation employs mixed methods to understand the context for implementation in the participating health care systems.

Discussion

This trial will inform the feasibility of implementing interdisciplinary behavioral approaches to pain management in the primary care setting, potentially providing a more effective, safer, and more satisfactory alternative to opioid-based chronic pain treatment.

Clinical Trials Registration Number: NCT02113592

Keywords: Chronic pain, Interdisciplinary treatment, Pragmatic clinical trial

1. Introduction

Chronic pain is one of the most common, disabling, and expensive public health problems in the United States [1]. It is one of the primary reasons patients seek medical care and is often associated with high levels of physical disability and emotional suffering [1,2]. Medical management of patients with chronic pain is fragmented, with patients seeking a wide variety of primary and specialty care services [1,3]. This fragmentation of care leads to poorer outcomes and higher health care costs as patients may receive unneeded diagnostic and medical procedures [3,4]. Many patients could benefit from multidisciplinary behaviorally-oriented approaches that emphasize pain management over a cure, and improved function over pain relief [5,6]. However, although such approaches have been found to be more effective than less comprehensive or single-modality interventions [1,2], most patients lack access to such treatment models. The majority are seen by primary care providers in settings where medications are the mainstay of pain treatment. Indeed, prescription opioid use has increased dramatically in recent years [7,8].

Opioid use has been shown to carry significant risks, and its application for pain management has caused mounting concern [919]. In response the Centers for Disease Control (CDC) issued widely publicized guidelines for prescribing opioids for chronic pain in the primary care setting [20]. As a result, health care providers, including primary care providers (PCPs), increasingly are seeking strategies to incorporate nonpharmacological treatment options into care plans for patients with chronic, non-cancer-related pain. However, they often lack readily available, systematic, integrated, and interdisciplinary treatment options. Furthermore, most PCPs have neither the time nor the training in pain management to effectively balance these patients' treatment needs [21].

Interdisciplinary pain management treatment approaches, particularly those employing a biopsychosocial framework, have been among the most successful approaches in helping patients with chronic pain reduce symptoms and regain functioning [2225]. These approaches combine a variety of therapeutic modalities and typically rely on teams of physicians, behavioral specialists, nurse case managers, and physical therapists to help patients develop skills to actively cope with and self-manage their condition [2633]. Interdisciplinary care for chronic pain is commonly situated in specialty care settings; however, study findings suggest that such approaches can be effectively delivered in the primary care setting [27,31,32,34,35]. The aim of our cluster-randomized pragmatic trial is to build on these studies by testing a program for its feasibility and sustainability within primary care to help patients adopt self-care and coping skills for managing chronic pain, limit use of opioid medications, and identify exacerbating factors amenable to treatment. Our main hypothesis is that patients of PCPs who are randomized to receive this interdisciplinary behavioral intervention, the Pain Program for Active Coping and Training, or “PPACT,” at their primary care clinic will have a greater reduction in pain impact in the year following compared to usual care. Our secondary hypothesis is that the integration and coordination of such services will result in lower overall health care and utilization costs for such patients compared to those in usual care.

2. Methods

2.1. Overview

We are conducting an effectiveness-implementation hybrid [36] pragmatic clinical trial in which we randomize clusters of primary care providers (PCPs) and their patients with chronic pain who are on long-term opioid therapy to either receive an interdisciplinary behavioral treatment intervention in conjunction with their current health care (intervention) or continue with their current health care services (usual care). Additionally, the PPACT trial includes robust process and formative evaluation components employing mixed methods to better understand the context for the intervention's implementation in the participating health care systems.

2.1.1. Primary care clinic eligibility

We have enrolled PCPs and patients in three regions of the Kaiser Permanente health care system: Kaiser Permanente Georgia (KPGA), Kaiser Permanente Hawaii (KPHI), and Kaiser Permanente Northwest (KPNW). Collectively, these sites offer both geographic and demographic diversity. All sites have comprehensive electronic health record (EHR) systems that utilize the same software platform (Epic); this allows the capture of the same health care utilization data across all three sites. Engagement of primary care clinics within each region is high. Operational and primary care leaders were interested in having all their primary care clinics and PCPs participate. However, the study limited enrollment in the much larger KPNW region (561,000 members) to allow a larger and thereby a somewhat more comparable number of patients and PCPs to participate from the two smaller regions (KPGA with 250,000 members and KPHI with 22,000 members). All primary care clinics approached, agreed to participate representing all 23 primary care clinics in KPGA, 13 of the 15 primary care clinics (87%) in KPHI and 9 of 19 clinics (47%) in KPNW. In both KPHI and KPNW, the selection of clinics was based on health care leaders' perception of service need and feasibility of implementation. Recruitment for the pragmatic trial began at the KPGA and KPNW sites in 2014 and at the KPHI site in 2016; study recruitment ended in the first quarter of 2017.

2.1.2. Primary care provider eligibility and enrollment

PCPs participating in the trial include Kaiser Permanente (KP) medical doctors, doctors of osteopathic medicine, physician assistants, and nurse practitioners who have established panels of patients at the participating primary care clinics. To recruit and enroll primary care providers, a member of the PPACT management team worked with participating clinics to present information about the trial to all eligible clinicians at a regular clinic staff meeting. The presentations included an overview of the study and intervention and information on the actions requested of providers during their participation, and covered all elements of informed consent. Providers could opt out of participating in the trial. Of the 338 PCPs invited to participate in the study, 99% agreed to do so (only 2 opted out).

2.1.3. Patient eligibility and enrollment

Patients were identified for the PPACT recruitment pool if their PCP agreed to participate in the study and the patients met the following eligibility criteria based on a query of the EHR: 1) adult (age 18 years or older), 2) current KP health plan member with at least 180 days of membership, 3) receipt of long-term opioid treatment defined by at least two dispensings of long-acting opioids in the 6 months prior to recruitment or at least a cumulative 90-day supply of short-acting opioids during any 4-month period within the 6 months prior to recruitment, and 4) any type of pain diagnosis recorded in the EHR within the previous year. While the chronicity of pain was not explicitly part of the EHR eligibility criteria, the fact that patients were required to have both a pain diagnosis and receipt of long-term opioid treatment suggests the pain is chronic. Patients were excluded based on the EHR query if they: 1) had a current malignant cancer diagnosis, 2) had received hospice or other end-of-life palliative care within past year, 3) were enrolled in current intensive addiction medicine treatment services or had active substance dependence diagnosis, or 4) had a cognitive impairment severe enough to preclude their participation in a behavioral/lifestyle change program. Exclusionary criteria based on EHR review were purposively kept to a minimum to ensure that study participants resemble those most in need of these services in the broader population.

2.1.3.1. Patient identification

PCPs were sent a secure staff message through the EHR that listed eligible patients on their panel. Providers were asked to review their list of potentially eligible patients and respond, indicating any patients who should not be invited to participate because they were ill-suited for the intervention. This practice is in line with the pragmatic nature of the trial. To date, PCPs have asked to exclude < 2% of identified potentially eligible patients, so this practice is not expected to introduce significant bias in our results. Patients who met the eligibility based on EHR review and provider approval were mailed a brochure or letter that provided an overview of the study, then a member of the study team attempted to contact patients by phone to provide an overview of the study and address any questions.

Clinical and operational partners in the three health care systems explicitly requested that we target more complex patients, that is, those whom PCPs frequently struggle to manage independently without additional support. Accordingly, recruitment was prioritized for patients who were: on high-dose opioids (morphine equivalence ≥120 mg); concurrently using opioids and benzodiazepines; high utilizers of primary care services (≥12 contacts within the last three months); and/or identified by their PCP as having a high need for additional pain management services. These characteristics were considered by our health system partners to indicate patients with more complex conditions for whom they felt this program was most critical. “Prioritization” meant that recruitment follow-up phone contacts began with patients meeting these three criteria, and a greater effort was expended to recruit such individuals.

During the recruitment phone calls, study staff further screened patients for eligibility by using the pain interference general activity item from the PEG (“On a scale from 0 to 10, with 0 meaning that pain does not interfere and 10 meaning that pain completely interferes, how has pain interfered with your general activity during the past 7 days?”) [36]. Patients were included if they reported a pain interference level of 4 or higher [36,37]. Patients were excluded if they: 1) did not speak English, 2) exhibited cognitive impairment, 3) were no longer paneled to the participating primary care provider or planning to change providers in next 12 months, 4) were moving out of the area, or 5) would not be able to attend the scheduled intervention group session times for their PCP.

2.1.4. Consent

The Institutional Review Boards (IRBs) at all three KP sites retained oversight and approved the trial procedures. The participating IRBs agreed that the study posed minimal risk as intervention activities involve the coordination of clinical services already available to most (KP) members and therefore presented no more risk of harm than what already existed for patients undergoing usual care treatment for chronic pain.

We did not obtain explicit consent from the PCPs; their receipt of the study information along with no request to opt out constituted informed consent and is most consistent with the goals and methods of pragmatic studies (i.e., folding the process into ongoing clinical care rather than carving out a subset of enthusiastic providers for participation). The IRBs at all three sites determined this to be an appropriate consent procedure because of the minimal risk posed to the providers and because the study involves services typically offered in a clinical setting to their patients. For patients, a waiver of signed informed consent and an alteration of the Health Insurance Portability and Accountability Act (HIPAA) authorization was granted at all sites, and verbal consent and authorization were obtained.

2.1.5. Randomization

PCPs were grouped into clinic-based clusters of 1 to 6 providers. Providers with a small number of eligible patients were grouped into multiprovider clusters (generally within clinics) to approximate the target of 8 patients per cluster. Of the 106 clusters, 21 (20%), consisted of a single provider and 85 clusters (80%) consisted of two or more PCPs. Clusters with fewer PCPs include providers who have a large number of potentially eligible patients on their panel whereas clusters with more PCPs include providers who have fewer of these patients on their panels. Clusters were randomized to either the intervention or usual care at a 1:1 ratio. A total of 106 clusters were randomized, representing 273 PCPs with patients in the study. (Note: Although 336 providers agreed to participate in the study, not every PCP had patients who consented to participate.) Recruitment and randomization occurred in 20 waves from March 2014 through February 2017, with 2 to 6 clusters per wave at any given site (Fig. 1). The variability in timing of recruitment waves and schedule reflects our efforts to accommodate regional needs. Each recruitment wave focused on provider clusters within a given clinic or small number of clinics in close demographic proximity; hence, the distribution of cluster characteristics, including the number of providers per cluster, should be similar across the two arms. The design does not formally require that these characteristics be balanced.

Fig. 1.

Fig. 1

PPACT study waves.

A total of 851 patients are participating in the study; 67% of enrolled patients are female, 13% are Black or African American, 3% are Hawaiian or Pacific Islander and the mean age of enrollees is 60.3 (SD 12.1). According to diagnoses documented in the EHR, many of those enrolled have multiple chronic co-morbidities. Specifically, 53% have two or more of the following chronic conditions, cardiovascular disorder, hypertension, diabetes and chronic pulmonary disease, in addition to their chronic pain. Notably, 33% have been diagnosed with depression, 21% with anxiety and 5% with post-traumatic stress disorder. Drug abuse was documented for 5% of enrollees and alcohol abuse for 4%. All patients are on long-term opioid treatment, since it is a condition of eligibility, and 20% were prescribed a daily morphine equivalent dose at or above 90 in the 6 months prior to randomization and 27% were concurrently taking benzodiazepines. Close to three-quarters (72%) had at least two types of pain conditions and close to half (46%) had three or more pain conditions diagnosed in the 6 months prior to enrollment. These characteristics are consistent with our intent to reach complex patients with more severe chronic pain presentations.

3. Intervention

The intervention integrates a number of services—behavioral services, nursing care management, physical therapy, and pharmacy consultation—into the primary care environment with the goal of helping patients develop skills to self-manage their condition. Rather than patients being seen by providers from these disciplines sequentially—which could result in variable coordination, as is often the case in clinical care settings—the goal of the intervention is to integrate the work of these providers to promote better coordinated and highly patient-centered care. The intervention consists of 1) a comprehensive intake evaluation, 2) cognitive behavioral therapy (CBT)-based pain coping skills training and adapted movement practice provided in 12 weekly group sessions, and 3) PCP consultation and patient outreach.

This approach is consistent with chronic care models of care [3842], previous collaborative care and interdisciplinary behavioral approaches to the management of chronic pain [27,35,4346], the many studies done on the pain coping skills training approach [29,30,4749], and chronic pain treatment guideline criteria [5052]. A visual depiction of patient flow through the intervention is shown in Fig. 2.

Fig. 2.

Fig. 2

PPACT intervention description.

The goal of the intake is to orient patients to the program, identify possible factors contributing to the pain and functional impairment, and tailor core intervention goals so that they are consistent with the patients' specific circumstances and preferences. Intakes were conducted by either the nurse case manager or behavioral specialist (2 sessions). Screening focuses on identifying common comorbid conditions that often exacerbate pain and impact functioning and for which treatment is available within these health care settings (i.e., depression, anxiety, substance disorders, post-traumatic stress disorder, and sleep apnea). PCPs are provided information about any comorbid condition so that treatment services could be provided, if warranted. The intake also includes a pharmacist chart review of medications with the goal of advising PCPs about potential alternatives to opioids or other adjustments of psychotropic medications. Lastly, as part of the intake a physical therapist meets with patients (1 session) to identify any adaptations that may be warranted for them to participate in the adapted movement portion of the intervention and to help the patient develop physical activity goals.

The 12 weekly group sessions that focus on coping-skills training and adapted-movement practice comprise the core of the intervention. The group sessions were designed to be co-led by the nurse case manager and behavioral specialists but with the recognition that sometimes sessions would be led by a single individual from one of these disciplines depending on current staffng availability at the site. Group activities and content are designed to 1) enhance patients' self-efficacy in using coping skills to control pain, 2) decrease maladaptive pain catastrophizing, 3) decrease fear of movement, and 4) increase the level and range of social and physical activities. Specific evidence-based pain-coping skills training (PCST) skills taught include: progressive muscle relaxation and minipractices (brief applied relaxation), activity-rest cycling, pleasant activity scheduling, guided imagery and other distraction techniques, emotional regulation skills, identifying/challenging negative thoughts and use of calming self-statements, problem-solving, and relapse prevention and maintenance. Each group session includes instruction and practice in yoga-based movement, which is designed to enhance core strength and coordination and thereby enhance confidence in participating in broader physical activity goals. The in-session practice is limited to seated and supported standing poses and uses the “Relax Into Yoga” DVD, which is based on Yoga of Awareness trials with vulnerable populations [5356]. The same DVD is provided to intervention participants to guide their home practice. PPACT interventionists contact participants between each session to discuss progress and adherence to the home-practice component of the program.

Finally, PPACT interventionists meet with intervention participants' PCPs after their patients complete the intake process but have not yet started group sessions (to review intake summary), and at the end of the intervention to review patients' progress and maintenance plan. If PCPs are willing, an additional telephone session is scheduled between them and their patient-participants to discuss patients' self-identified functional goals and reinforce their self-management efforts.

Core PPACT interventionist team (behavioral specialists and nurse care managers) were expected to participate in a two-day in-person training led by study investigators (FJK, LB, LLD) with PPACT physical therapists and pharmacists participating in relevant portions of the training. Each interventionist was required to reach proficiency in the manualized intervention approach which was assessed through review of audio-recorded mock and actual sessions prior to beginning work with study participants. In addition, interventionists received biweekly telephone consultation with clinical investigators on the study (FJK, LB, LDD) who reviewed audio recordings of group sessions and provided feedback regarding treatment quality and adherence to the study protocol.

4. Primary and secondary outcomes

4.1. Pain impact and pain-related disability

Our primary measure of pain impact is the PEG [57,58], a validated three-item measure derived from the short form of the Brief Pain Inventory (BPI-SF) [36,37]. The original 9-item version of the BPI-SF has been widely adopted for clinical pain assessment, epidemiological studies, and studies of pain treatment effectiveness [57,58]. Pain impact is a composite measure of pain intensity (question 1: how bad is your pain?), pain interference with enjoyment of life (question 2), and pain interference with general activity (question 3) as rated over the previous 7 days on an 11-point Likert scale [57,58]. The brevity of the PEG makes it more acceptable for use by PCPs and their support staff in busy clinical practice settings. We worked closely with clinical stakeholders in each of the participating health care systems during the preparatory phase of the project to establish use of an assessment that had acceptable psychometrics, was brief enough, focused on functioning and was easily interpretable. The PEG fulfilled these criteria whereas the lengthier BPI-SF did not.

Pain-related disability, a secondary outcome, is assessed using the 24-item Roland Morris Disability Questionnaire (RMDQ), which has been validated in patients with low back pain and other chronic pain conditions [5963]. Items on the RMDQ are rated as yes/no.

4.2. Utilization of health care services

We will also assess utilization of health care services of specific relevance for the PPACT target patient population. These utilization variables include: the amount of opioids dispensed (in morphine equivalent dispenses), both aggregated and disaggregated primary care contacts (i.e., outpatient visits, e-mail contacts, telephone contacts), use of specialty pain services (i.e., physiatry, pain medicine, physical therapy, and occupational therapy services), inpatient services related to the participant's pain condition (i.e., surgeries, implementation of pain-related devices), receipt of pain and psychotropic medications, and overall outpatient utilization.

4.3. Patient satisfaction

We worked with KP regional stakeholders to identify patient satisfaction questions that they believed to be of importance in evaluating the intervention. We are using two questions, one assessing patients' satisfaction with their primary care services, and a second assessing their satisfaction with overall pain-related services provided by the health plan. The questions assess satisfaction over the past 3 months on a 5-point Likert scale (“very dissatisfied” to “very satisfied”).

4.4. Data collection methods and schedule

The PEG and RMDQ are collected quarterly during patients' 12-month study participation: at baseline and at 3, 6, 9, and 12 months. Quarterly data collection was chosen to ensure adequate data for analyses but not overburden patients. There are three tiers to the data collection process. First, patients are sent a message via the patient EHR portal, kp.org., where they can manage medical appointments and prescriptions as well as send electronic communications to clinic staff. Surveys completed via kp.org are stored directly in patients' medical records. Next, if the kp.org survey is not completed within 1 week, patients are contacted via KP's interactive voice response (IVR) system and asked to complete the survey by phone. If surveys are not completed via IVR within another week, a third data-collection method is used: patients are called by research staff and asked to complete the survey. Patient-reported satisfaction with health care services is collected twice: at baseline and 5–6 months after enrollment, which is shortly after the conclusion of the intervention.

All other data to be used for secondary analyses are extracted from the EHR. These data are extracted for the 12 months prior to enrollment and the 12 months of study participation. A summary of data collected and the schedule of data collection is included in Table 1.

Table 1.

PPACT outcome variables.

Measure Source Schedule of assessment

Up to 12 months preceding patient enrollment Study month

0 3 6 9 12
Patient-reported outcomes
PEG Primary outcome Study survey
Roland Morris Disability Questionnaire Secondary outcome Study survey
Patient Satisfaction Survey Secondary outcome Study survey
Medication-related outcomes
Opioids dispensed Secondary outcome EHR graphic file with name nihms958284t1.jpg
% of patients with morphine equivalents ≥90 and morphine equivalents ≥50 Secondary outcome EHR graphic file with name nihms958284t1.jpg
Benzodiazepines dispensed Health service utilization Secondary outcome EHR graphic file with name nihms958284t1.jpg
Primary care utilization (outpatient visits, emails, telephone contacts and total) Secondary outcome EHR graphic file with name nihms958284t1.jpg
Emergency and urgent care services Secondary outcome EHR graphic file with name nihms958284t1.jpg
Use of specialty pain services (physiatry, pain clinic, physical and occupational therapy) Secondary outcome EHR graphic file with name nihms958284t1.jpg
Overall outpatient service utilization Secondary outcome EHR graphic file with name nihms958284t1.jpg
Inpatient services related to pain condition Secondary outcome EHR graphic file with name nihms958284t1.jpg

5. Data safety and monitoring plan

Due to the low risk posed to patients by the PPACT pragmatic clinical trial, the data safety and monitoring plan relies on close monitoring by the principal investigator in conjunction with experienced clinicians on the investigative team, and by the National Institutes of Health-approved independent monitor. Reviews of hospitalizations and deaths among active subjects are conducted by the independent monitor every 6 months for the duration of the study.

6. Analytic approach for primary and secondary outcomes

The following analytic framework will be used for our primary and secondary outcome analyses. All analyses will be performed using an intention-to-treat framework, and tests will be evaluated at a two-tailed alpha level of 0.05. Because of the nested structure of the data (observations nested within patients nested within provider groups), we will use a three-level hierarchical linear model (HLM: mixed models, random effects regression, and multilevel models) to account for the intraclass correlation that results from the nesting [6466]. An advantage of multilevel modeling is that unlike repeated measures analysis of variance, it does not require the same number of data points from all patients, thus all patients with at least a baseline measure can be included in the analysis. The first level of the model will include time as a predictor (five timepoints, representing the number of weeks since baseline), thus modeling the within-person trajectories across time. We will use two parameters (linear and quadratic slope) to characterize change across time, with linear slope capturing initial rate of change and quadratic slope reflecting the degree to which the change slowed (or increased) over time. The second level of the model may include patient-level covariates as predictors of the baseline PEG score and the slope parameters for time. Randomization is expected to balance most potential patient-level covariates, however, in the case of remaining residual imbalances, covariates will be included in the model. These may include variables such as substance use problems/history, number of pain conditions and type, and other comorbid medical and mental health conditions. The third level will include a dummy variable for arm as the predictor of the patient-level intercept and slope parameters for time. A significant coefficient for arm on the slope(s) of time would indicate that there are different trajectories across time for each arm. A pattern in which those in PPACT demonstrate a greater reduction in pain impact over time than those in the usual care arm would provide support for the effectiveness of PPACT. We will use the same analytical framework for the RMDQ. Because there are only two timepoints available for satisfaction, we will be limited to a two-level model of the difference scores between 6 months and baseline of patients nested within provider groups.

Level-1 model

Ytij=π0ij+π1ij(LIN_TIMEtij)+π2ij(QUAD_TIMEtij)+etij

Level-2 model

π0ij=β00j+β01j(Patient_covariatesij)+r0ijπ1ij=β10j+β11j(Patient_covariatesij)+r1ijπ2ij=β20j+β21j(Patient_covariatesij)+r2ij

Level-3 model

β00j=γ000+γ001(ARMj)+u00jβ01j=γ010β10j=γ100+γ101(ARMj)+u10jβ11j=γ110β20j=γ200+γ201(ARMj)+u20jβ21j=γ210

where: Ytij is the outcome for person i under provider j at time t, π are level 1 (occasion) regression coefficients, etij is the random error associated with person i under provider cluster j at time t, Lin_Time is the number of weeks since baseline and Quad_Time is the number of weeks since baseline squared, β are level 2 (patient) regression coefficients, r are level 2 random effects, γ are level 3 (provider cluster) regression coefficients, u are level 3 random effects, and arm is an indicator variable.

Hierarchical generalized linear modeling (HGLM) will be used to test the secondary outcomes: opioids dispensed, pain treatment and diagnostic procedures, emergency/urgent care visits, primary care visits, and specialty care visits over 12 months. These will be two-level models, with patients forming the first level of the model and clinics the second level. Patient-level covariates will be included in the first level, and PPACT versus a usual-care dummy variable will form the second level of the model. This will allow us to test whether the secondary outcomes differ for the two groups, controlling for differences in patient characteristics. Because the utilization variables are likely to follow non-normal distributions, we will use Poisson, Negative Binomial, or Gamma distributions as appropriate for the distribution of each secondary outcome variable.

Level-1 model

ηij=β0j+β1j(Patient_covariatesij)

Level-2 model

β0j=γ00+γ01(ARMj)+u0jβ1j=γ10

where: ηij is the outcome defined by the identity link (log) and distribution (gamma, Poisson, or Negative Binomial), β are level 1 (person) regression coefficients, γ are level 2 (provider cluster) regression coefficients, u is the level 2 random effect for the level 1 intercept, and arm is an indicator variable.

6.1.1. Power for primary outcome

We calculated power using the PASS software program, which applies the formulas from Donner and Klar [67] and assumes a simple ANOVA framework with no covariate adjustment. Based on direct estimates of the intraclass correlation coefficient (ICC) of PEG slopes clustered within provider groups that we derived from historical data from the KPNW region, we estimate the ICC to be 0.0013. In the calculations presented below we conservatively use ICCs of 0.002, 0.005 and 0.01. From the literature, we also expect standardized effect sizes to range from 0.022 to 0.54 [27,31,32,37,68], and therefore conservatively calculated power for effect sizes ranging from 0.16 to 0.24 standard deviation units (SDUs).

Our initial study design nominally called for 120 total clusters of 10 patients each. In practice, however, we randomized 106 PCP clusters, and cluster sizes have varied from 3 to 13, with a mean of 8 and interquartile range of 6–10. As seen in Table 2, we constructed our power calculations to accommodate the possibility of such smaller cluster sizes. With the likely ICC of 0.002 and 106 clusters with an average cluster size of 8, we should have 93% to detect a standard effect size of 0.24 and 88% power to detect an effect size of 0.22.

Table 2.

Power for detecting given effect sizes under various design scenarios.

Number of clusters Patients per cluster ICC = 0.002 ICC = 0.005 ICC = 0.01



Effect size (in SDUs) Effect size (in SDUs) Effect size (in SDUs)



0.16 0.18 0.20 0.22 0.24 0.16 0.18 0.20 0.22 0.24 0.16 0.18 0.20 0.22 0.24
120 8 68% 78% 86% 92% 96% 68% 78% 86% 91% 95% 66% 76% 84% 90% 95%
106 8 63% 73% 82% 88% 93% 62% 72% 81% 88% 93% 61% 71% 80% 87% 92%

7. Economic evaluation

In our economic analysis of the PPACT intervention, we will assess the resources and costs necessary to deliver the PPACT intervention in routine clinical practice, and the cost-effectiveness of the PPACT intervention compared with usual care. Costs will be reported at three levels: 1) costs related to the intervention delivery, 2) medical care costs related to pain control, and 3) the total cost of medical care. Intervention costs will be estimated using EHR data, supplemented with data collected directly from intervention team staff. A sampling of clinical visits and interviews with intervention delivery staff will be used to determine the time needed to deliver the intervention. We will also consider costs related to the administration of the PPACT intervention in practice, including project management, training, and additional team meetings.

Using EHR data, we will aggregate medical care events related to pain control and total costs at meaningful levels in order to demonstrate how the intervention impacts medical care resource use, specifically inpatient stays, outpatient procedures, clinic visits, and pharmacy dispenses. We will identify medical care utilization events that are related to pain control and the intervention using ICD-9CM, ICD-10CM, and CPT codes. To facilitate costing, we will examine the number and type of health care encounters participants receive over the course of their 12-month participation in the study. We will capture inpatient stays by extracting the information in the discharge abstract, including ICD-9 and ICD-10 codes and procedures and length-of-stay information necessary to cost the event.

Medical care utilization events will be analyzed using mixed effects Poisson regression analysis, with primary care provider cluster as a random effects factor and follow-up time as an offset variable. We will estimate quantities of medical care events (i.e., inpatient stays, out-patient procedures, clinic visits, and pharmacy dispenses) using separate regression models.

8. Process and formative evaluation

The PPACT process evaluation assesses fidelity of intervention delivery (the extent to which the intervention is delivered as intended), the intervention dose (how much of the intended intervention is delivered), and the reach to the groups targeted by the intervention (the proportion of intended recipients who actually participate in an intervention) using the RE-AIM framework [69,70] as a guide.

For the study's formative evaluation framework, we use PRISM [71], created to complement RE-AIM and focused on delineating criteria for successful implementation of interventions in health systems. Further, the structure, staffng, and analysis of formative evaluation data is guided by the Rapid Assessment Process (RAP) [72,73], which employs ethnographic assessment by teams to gather and analyze information quickly to build an evolving understanding of conditions related to a planned or existing intervention. Data used for the process evaluation include: journal entries compiled by the study team (to document the conversations, current practices, and PPACT-related concerns that arise in the course of their interactions with stakeholders, project staff and teams) as well as patient surveys and telephone interviews with patients, clinicians and operational leaders. As part of RAP, the qualitative team meets regularly to review data collection and incremental data analyses to compile an emerging picture of the progress of the intervention, and the results of these analyses become part of debriefing meetings and progress reports to the larger research team.

9. Discussion

The PPACT pragmatic clinical trial addresses one of the most pervasive and costly public health problems in the United States: chronic pain. The trial tests the effectiveness of integrating an interdisciplinary behavioral intervention within a primary care environment. The intervention is based on three key principles: 1) Patients learn active pain-coping skills and adapted movement, as distinguished from passive approaches such as pharmacotherapy and/or procedural interventions; 2) there is a focus on patients' improvements in function, rather than solely on pain relief; and 3) interdisciplinary team members make efforts to actively collaborate with and support primary care providers with the goal of enhancing the providers' skills and confidence in working with these patients. Our study builds upon previous multimodal behavioral pain management approaches [27,32,35,74] by conducting the research within a pragmatic trial framework that is focused on applicability, broad inclusion, flexibility in intervention implementation, and attention to the outcomes most meaningful to key stakeholders.

Importantly, the impetus for this study emerged from clinical and administrative leaders of the participating health plans who acknowledged struggling with the provision of adequate nonpharmacological pain management services to patients with complex pain receiving long-term opioid treatment. These leaders sought partnership with our research team to implement and evaluate a primary care-based interdisciplinary behavioral pain management program. The health plan leaders identified patients on long-term opioid treatment as those for whom they were willing to extend and potentially sustain the staffng resources needed for this relatively intensive intervention by clinical care standards. The interest and commitment of these leaders has resulted in a stronger partnership than might have otherwise developed.

A further strength of the trial is its large and diverse sample. The sample is geographically, racially, and ethnically diverse. Further, input from health plan and clinical leaders shaped study decisions, including extending trial eligibility to those with common types chronic pain who were being treated with long-term opioid therapy. The biopsychosocial model and previous studies suggest that the type of multimodal behavioral treatment offered in this trial can be effective with a range of pain conditions [1,2,24,35,45,46]. To our knowledge, this is the largest sample ever recruited for an interdisciplinary pain management trial and uniquely includes patients presenting with a variety of chronic pain conditions and comorbid medical and mental health conditions. The PRagmatic-Explanatory Continuum Indicator Summary (PRECIS-2) is a tool developed to describe how “pragmatic” a specific trial is with respect to nine key domains [75]. Fig. 3 illustrates where the PPACT pragmatic clinical trial falls on this continuum for each domain, as judged collectively by the research team who are authors of this paper. This illustrates that although the trial is very pragmatic on many of these domains (eligibility, setting, primary outcome, and primary analysis) others are less “pragmatic” by design as appropriate for a more intensive behavioral intervention with a complex population (recruitment, flexibility: delivery, flexibility: adherence).

Fig. 3.

Fig. 3

PPACT PRECIS-2 scoring.

In summary, the PPACT study is a novel, Type 1, effectiveness-implementation, cluster-randomized pragmatic clinical trial. The study is designed to evaluate an intervention that has the goal of helping patients learn to apply an array of coping and self-care skills for managing chronic pain and limit use of opioid medications, and helping PCPs identify and treat complicating factors amenable to treatment in primary care settings. In addition to testing the effectiveness of a multimodal behavioral pain management intervention, our goal is to identify, evaluate, and optimize strategies that enhance the sustainability of such an intervention in the everyday clinical work flow of primary care. We hope this work will inform feasible approaches to pain management in the primary care setting, potentially providing a more effective, safer, and more satisfactory alternative to the current over-reliance on opioid-based chronic pain treatment.

Acknowledgments

The authors gratefully acknowledge Reesa Laws (Kaiser Permanente Center for Health Research) for her work to develop and implement the data collection infrastructure for the PPACT trial. We also gratefully acknowledge the efforts of Kelly DeGraffenreid, MD (KPGA), Stacy Honda, MD (KPHI) and Adrianne Feldstein, MD (KPNW) for facilitating the integration of the PPACT trial into the health care system at each site.

Funding

The PPACT study is supported by the National Institutes of Health (NIH) Common Fund, through a cooperative agreement (UH2AT007788, UH3NS088731) from the Office of Strategic Coordination within the Office of the NIH Director. The views presented here are solely the responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health.

References

  • 1.Institute of Medicine. Relieving Pain in America: A Blueprint for Transforming Prevention, Care, Education and Research. Washington DC: 2011. [PubMed] [Google Scholar]
  • 2.Pain Management Task Force. Final Report: Providing a Standardized DoD and VHA Vision and Approach to Pain Management to Optimize the Care for Warriors and their Families. 2010 [Google Scholar]
  • 3.Roth RS, Geisser ME, Williams DA. Interventional pain medicine: retreat from the biopsychosocial model of pain. Transl Behav Med. 2012;2(1):106–116. doi: 10.1007/s13142-011-0090-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Cheatle MD, Klocek JW, McLellan AT. Managing pain in high-risk patients within a patient-centered medical home. Transl Behav Med. 2012;2(1):47–56. doi: 10.1007/s13142-012-0113-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Ballantyne JC, Sullivan MD. Intensity of chronic pain–the wrong metric? N Engl J Med. 2015;373(22):2098–2099. doi: 10.1056/NEJMp1507136. [DOI] [PubMed] [Google Scholar]
  • 6.Schneiderhan J, Clauw D, Schwenk TL. Primary care of patients with chronic pain. JAMA. 2017;317(23):2367–2368. doi: 10.1001/jama.2017.5787. [DOI] [PubMed] [Google Scholar]
  • 7.Kuehn BM. Opioid prescriptions soar: increase in legitimate use as well as abuse. JAMA. 2007;297(3):249–251. doi: 10.1001/jama.297.3.249. [DOI] [PubMed] [Google Scholar]
  • 8.Caudill-Slosberg MA, Schwartz LM, Woloshin S. Office visits and analgesic prescriptions for musculoskeletal pain in US: 1980 vs. 2000. Pain. 2004;109(3):514–519. doi: 10.1016/j.pain.2004.03.006. [DOI] [PubMed] [Google Scholar]
  • 9.Staats PS, Markowitz J, Schein J. Incidence of constipation associated with long-acting opioid therapy: a comparative study. South Med J. 2004;97(2):129–134. doi: 10.1097/01.SMJ.0000109215.54052.D8. [DOI] [PubMed] [Google Scholar]
  • 10.Swegle JM, Logemann C. Management of common opioid-induced adverse effects. Am Fam Physician. 2006;74(8):1347–1354. [PubMed] [Google Scholar]
  • 11.Angst MS, Clark JD. Opioid-induced hyperalgesia: a qualitative systematic review. Anesthesiology. 2006;104(3):570–587. doi: 10.1097/00000542-200603000-00025. [DOI] [PubMed] [Google Scholar]
  • 12.Davis PE, Liddiard H, McMillan TM. Neuropsychological deficits and opiate abuse. Drug Alcohol Depend. 2002;67(1):105–108. doi: 10.1016/s0376-8716(02)00012-1. [DOI] [PubMed] [Google Scholar]
  • 13.Kalso E, et al. Opioids in chronic non-cancer pain: systematic review of efficacy and safety. Pain. 2004;112(3):372–380. doi: 10.1016/j.pain.2004.09.019. [DOI] [PubMed] [Google Scholar]
  • 14.Manchikanti L, Singh A. Therapeutic opioids: a ten-year perspective on the complexities and complications of the escalating use, abuse, and nonmedical use of opioids. Pain Physician. 2008;11(2 Suppl):S63–S88. [PubMed] [Google Scholar]
  • 15.Deyo RA, et al. Overtreating chronic back pain: time to back off? J Am Board Fam Med. 2009;22(1):62–68. doi: 10.3122/jabfm.2009.01.080102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Harden RN. Chronic pain and opiates: a call for moderation. Arch Phys Med Rehabil. 2008;89(3 Suppl 1):S72–S76. doi: 10.1016/j.apmr.2007.12.013. [DOI] [PubMed] [Google Scholar]
  • 17.McLellan AT, Turner B. Prescription opioids, overdose deaths, and physician responsibility. JAMA. 2008;300(22):2672–2673. doi: 10.1001/jama.2008.793. [DOI] [PubMed] [Google Scholar]
  • 18.Trescot AM, et al. Opioid guidelines in the management of chronic non-cancer pain. Pain Physician. 2006;9(1):1–39. [PubMed] [Google Scholar]
  • 19.Von Korff M, Deyo RA. Potent opioids for chronic musculoskeletal pain: flying blind? Pain. 2004;109(3):207–209. doi: 10.1016/j.pain.2004.02.019. [DOI] [PubMed] [Google Scholar]
  • 20.Dowell D, Haegerich TM, Chou R. CDC guideline for prescribing opioids for chronic pain — United States, 2016. MMWR Recomm Rep 2016. 2016;65(RR-1):1–49. doi: 10.15585/mmwr.rr6501e1. [DOI] [PubMed] [Google Scholar]
  • 21.Upshur CC, Luckmann RS, Savageau JA. Primary care provider concerns about management of chronic pain in community clinic populations. J Gen Intern Med. 2006;21(6):652–655. doi: 10.1111/j.1525-1497.2006.00412.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Goldenberg DL. Multidisciplinary modalities in the treatment of fibromyalgia. J Clin Psychiatry. 2008;69(Suppl 2):30–34. [PubMed] [Google Scholar]
  • 23.Scascighini L, et al. Multidisciplinary treatment for chronic pain: a systematic review of interventions and outcomes. Rheumatology (Oxford) 2008;47(5):670–678. doi: 10.1093/rheumatology/ken021. [DOI] [PubMed] [Google Scholar]
  • 24.Sarzi-Puttini P, et al. Multidisciplinary approach to fibromyalgia: what is the teaching? Best Pract Res Clin Rheumatol. 2011;25(2):311–319. doi: 10.1016/j.berh.2011.03.001. [DOI] [PubMed] [Google Scholar]
  • 25.Smith BH, Torrance N. Management of chronic pain in primary care. Curr Opin Support Palliat Care. 2011;5(2):137–142. doi: 10.1097/SPC.0b013e328345a3ec. [DOI] [PubMed] [Google Scholar]
  • 26.Dixon KE, et al. Psychological interventions for arthritis pain management in adults: a meta-analysis. Health Psychol. 2007;26(3):241–250. doi: 10.1037/0278-6133.26.3.241. [DOI] [PubMed] [Google Scholar]
  • 27.Dobscha SK, et al. Collaborative care for chronic pain in primary care: a cluster randomized trial. JAMA. 2009;301(12):1242–1252. doi: 10.1001/jama.2009.377. [DOI] [PubMed] [Google Scholar]
  • 28.Keefe FJ, et al. Recent advances and future directions in the biopsychosocial assessment and treatment of arthritis. J Consult Clin Psychol. 2002;70(3):640–655. doi: 10.1037//0022-006x.70.3.640. [DOI] [PubMed] [Google Scholar]
  • 29.Keefe FJ, Somers TJ. Psychological approaches to understanding and treating arthritis pain. Nat Rev Rheumatol. 2010;6(4):210–216. doi: 10.1038/nrrheum.2010.22. [DOI] [PubMed] [Google Scholar]
  • 30.Keefe FJ, et al. Effects of coping skills training and sertraline in patients with non-cardiac chest pain: a randomized controlled study. Pain. 2011;152(4):730–741. doi: 10.1016/j.pain.2010.08.040. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Kroenke K, et al. Optimized antidepressant therapy and pain self-management in primary care patients with depression and musculoskeletal pain: a randomized controlled trial. JAMA. 2009;301(20):2099–2110. doi: 10.1001/jama.2009.723. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Von Korff M, et al. A trial of an activating intervention for chronic back pain in primary care and physical therapy settings. Pain. 2005;113(3):323–330. doi: 10.1016/j.pain.2004.11.007. [DOI] [PubMed] [Google Scholar]
  • 33.Waters SJ, McKee DC, Keefe FJ. Cognitive behavioral approaches to the treatment of pain. Psychopharmacol Bull. 2007;40(4):74–88. [PubMed] [Google Scholar]
  • 34.Dorfiinger LM, et al. Integrating interdisciplinary pain management into primary care: development and implementation of a novel clinical program. Pain Med. 2014;15:2046–2054. doi: 10.1111/pme.12554. [DOI] [PubMed] [Google Scholar]
  • 35.Bair MJ, et al. Evaluation of stepped care for chronic pain (ESCAPE) in veterans of the Iraq and Afghanistan conflicts: a randomized clinical trial. JAMA Intern Med. 2015;175(5):682–689. doi: 10.1001/jamainternmed.2015.97. [DOI] [PubMed] [Google Scholar]
  • 36.Krebs EE, et al. Development and initial validation of the PEG, a three-item scale assessing pain intensity and interference. J Gen Intern Med. 2009;24(6):733–738. doi: 10.1007/s11606-009-0981-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Krebs EE, et al. Comparative responsiveness of pain outcome measures among primary care patients with musculoskeletal pain. Med Care. 2010;48(11):1007–1014. doi: 10.1097/MLR.0b013e3181eaf835. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Wagner E. The Chronic Care Model. 2006 [Google Scholar]
  • 39.Wagner EH, et al. Improving chronic illness care: translating evidence into action. Health Aff (Millwood) 2001;20(6):64–78. doi: 10.1377/hlthaff.20.6.64. [DOI] [PubMed] [Google Scholar]
  • 40.Coleman K, et al. Evidence on the chronic care model in the new millennium. Health Aff (Millwood) 2009;28(1):75–85. doi: 10.1377/hlthaff.28.1.75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Wagner EH. Chronic disease management: what will it take to improve care for chronic illness? Eff Clin Pract. 1998;1(1):2–4. [PubMed] [Google Scholar]
  • 42.Wagner EH, Austin BT, Von Korff M. Organizing care for patients with chronic illness. Milbank Q. 1996;74(4):511–544. [PubMed] [Google Scholar]
  • 43.Loeser J. Multidsciplinary pain management. In: Merskey H, Loeser J, Dubner R, editors. The Paths of Pain. IASP; Seattle: 2005. pp. 503–511. [Google Scholar]
  • 44.Turk D, Swanson K. Efficacy and cost-effectiveness treatmentfor chronic pain: an analysis and evidence-based synthesis. In: Schatman Michael E, Campbell Alexandra., editors. Chronic Pain Management: Guidelines for Multidisciplinary Program Development. Informa; New York: 2007. pp. 15–38. [Google Scholar]
  • 45.Von KM, et al. A randomized trial of a lay person-led self-management group intervention for back pain patients in primary care. Spine (Phila Pa 1976) 1998;23(23):2608–2615. doi: 10.1097/00007632-199812010-00016. [DOI] [PubMed] [Google Scholar]
  • 46.Kroenke K, et al. Telecare collaborative management of chronic pain in primary care: a randomized clinical trial. JAMA. 2014;312(3):240–248. doi: 10.1001/jama.2014.7689. [DOI] [PubMed] [Google Scholar]
  • 47.Keefe FJ, et al. Pain coping skills training in the management of osteoarthritic knee pain: a comparative study. Behav Ther. 1990;21(1):49–62. [Google Scholar]
  • 48.Keefe FJ, et al. Pain coping skills training in the management of osteoarthritic knee pain-II: follow-up results. Behav Ther. 1990;21(4):435–447. [Google Scholar]
  • 49.Keefe FJ, et al. Effects of spouse-assisted coping skills training and exercise training in patients with osteoarthritic knee pain: a randomized controlled study. Pain. 2004;110(3):539–549. doi: 10.1016/j.pain.2004.03.022. [DOI] [PubMed] [Google Scholar]
  • 50.Chou R, Huffman LH. Nonpharmacologic therapies for acute and chronic low back pain: a review of the evidence for an American Pain Society/American College of Physicians clinical practice guideline. Ann Intern Med. 2007;147(7):492–504. doi: 10.7326/0003-4819-147-7-200710020-00007. [DOI] [PubMed] [Google Scholar]
  • 51.Chou R, et al. Interventional therapies, surgery, and interdisciplinary rehabilitation for low back pain: an evidence-based clinical practice guideline from the American Pain Society. Spine (Phila Pa 1976) 2009;34(10):1066–1077. doi: 10.1097/BRS.0b013e3181a1390d. [DOI] [PubMed] [Google Scholar]
  • 52.American Society of Anesthesiologists Task Force on Chronic Pain Management, American Society of Regional Anesthesia and Pain Medicine. Practice guidelines for chronic pain management: an updated report by the American Society of Anesthesiologists Task Force on Chronic Pain Management and the American Society of Regional Anesthesia and Pain Medicine. Anesthesiology. 2010;112(4):810–833. doi: 10.1097/ALN.0b013e3181c43103. [DOI] [PubMed] [Google Scholar]
  • 53.Wren AA, et al. Yoga for persistent pain: new findings and directions for an ancient practice. Pain. 2011;152(3):477–480. doi: 10.1016/j.pain.2010.11.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Carson JW, et al. A pilot randomized controlled trial of the Yoga of Awareness program in the management of fibromyalgia. Pain. 2010;151(2):530–539. doi: 10.1016/j.pain.2010.08.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Carson JW, et al. Yoga of awareness program for menopausal symptoms in breast cancer survivors: results from a randomized trial. Support Care Cancer. 2009;17(10):1301–1309. doi: 10.1007/s00520-009-0587-5. [DOI] [PubMed] [Google Scholar]
  • 56.Carson JW, et al. Yoga for women with metastatic breast cancer: results from a pilot study. J Pain Symptom Manag. 2007;33(3):331–341. doi: 10.1016/j.jpainsymman.2006.08.009. [DOI] [PubMed] [Google Scholar]
  • 57.Cleeland CS, Ryan KM. Pain assessment: global use of the Brief Pain Inventory. Ann Acad Med Singap. 1994;23(2):129–138. [PubMed] [Google Scholar]
  • 58.Keller S, et al. Validity of the brief pain inventory for use in documenting the outcomes of patients with noncancer pain. Clin J Pain. 2004;20(5):309–318. doi: 10.1097/00002508-200409000-00005. [DOI] [PubMed] [Google Scholar]
  • 59.Roland M, Morris R. A study of the natural history of back pain. Part I: development of a reliable and sensitive measure of disability in low-back pain. Spine (Phila Pa 1976) 1983;8(2):141–144. doi: 10.1097/00007632-198303000-00004. [DOI] [PubMed] [Google Scholar]
  • 60.Roland M, Fairbank J. The Roland-Morris Disability Questionnaire and the Oswestry Disability Questionnaire. Spine (Phila Pa 1976) 2000;25(24):3115–3124. doi: 10.1097/00007632-200012150-00006. [DOI] [PubMed] [Google Scholar]
  • 61.Davidson M, Keating JL. A comparison of five low back disability questionnaires: reliability and responsiveness. Phys Ther. 2002;82(1):8–24. doi: 10.1093/ptj/82.1.8. [DOI] [PubMed] [Google Scholar]
  • 62.Jordan K, et al. A minimal clinically important difference was derived for the Roland-Morris Disability Questionnaire for low back pain. J Clin Epidemiol. 2006;59(1):45–52. doi: 10.1016/j.jclinepi.2005.03.018. [DOI] [PubMed] [Google Scholar]
  • 63.Stratford PW, et al. Defining the minimum level of detectable change for the Roland-Morris questionnaire. Phys Ther. 1996;76(4):359–365. doi: 10.1093/ptj/76.4.359. discussion 366–8. [DOI] [PubMed] [Google Scholar]
  • 64.Hox JJ. Multilevel Analysis: Techniques and Applications. Routledge; 2010. [Google Scholar]
  • 65.Raudenbush SW, Bryk AS. Hierarchical Linear Models: Applications and Data Analysis Methods. 2. Sage Publications Inc; Thousand Oaks, CA: 2002. [Google Scholar]
  • 66.Snijders T, Bosker R. Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. Sage; Thousand Oaks, CA: 1999. [Google Scholar]
  • 67.Donner A, Klar N. Design and Analysis of Cluster Randomization Trials in Health Research. Arnold Publishers; London: 2000. [Google Scholar]
  • 68.Dworkin RH, et al. Interpreting the clinical importance of group differences in chronic pain clinical trials: IMMPACT recommendations. Pain. 2009;146(3):238–244. doi: 10.1016/j.pain.2009.08.019. [DOI] [PubMed] [Google Scholar]
  • 69.Glasgow RE. RE-AIMing research for application: ways to improve evidence for family medicine. J Am Board Fam Med. 2006;19(1):11–19. doi: 10.3122/jabfm.19.1.11. [DOI] [PubMed] [Google Scholar]
  • 70.Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–1327. doi: 10.2105/ajph.89.9.1322. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Feldstein AC, Glasgow RE. A practical robust implementation and sustainability model (PRISM) for integrating research findings into practice. Jt Comm J Qual Patient Saf. 2008;34(4):228–243. doi: 10.1016/s1553-7250(08)34030-6. [DOI] [PubMed] [Google Scholar]
  • 72.McMullen CK, et al. Rapid assessment of clinical information systems in the healthcare setting: an efficient method for time-pressed evaluation. Methods Inf Med. 2011;50(4):299–307. doi: 10.3414/ME10-01-0042. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.Beebe J. Rapid Assessment Process: An Introduction. Alta Mira Press; Walnut Creek, CA: 2001. [Google Scholar]
  • 74.Kroenke K, et al. Stepped Care for Affective Disorders and Musculoskeletal Pain (SCAMP) study: design and practical implications of an intervention for comorbid pain and depression. Gen Hosp Psychiatry. 2007;29(6):506–517. doi: 10.1016/j.genhosppsych.2007.08.005. [DOI] [PubMed] [Google Scholar]
  • 75.Loudon K, et al. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350:h2147. doi: 10.1136/bmj.h2147. [DOI] [PubMed] [Google Scholar]

RESOURCES