Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jul 1.
Published in final edited form as: J Pain. 2019 Nov 21;21(7-8):798–807. doi: 10.1016/j.jpain.2019.11.007

Integrated Behavioral Treatment for Veterans With Co-Morbid Chronic Pain and Hazardous Opioid Use: A Randomized Controlled Pilot Trial

Kevin E Vowles *, Katie Witkiewitz , Karen J Cusack , Wesley P Gilliam §, Karen E Cardon , Sarah Bowen , Karlyn A Edwards *, Mindy L McEntee , Robert W Bailey **
PMCID: PMC7542006  NIHMSID: NIHMS1545433  PMID: 31760109

Abstract

Opioid prescription in the treatment of chronic pain is frequent and carries a risk of increased morbidity and mortality in a clinically significant number of patients, particularly those who are using opioids in a hazardous manner. Few treatment options are available that target both pain-related interference and hazardous opioid use among patients with chronic pain. In military Veterans, this issue is of particular importance as numerous reports indicate continued high rates of opioid prescription for chronic pain, as well as significant opioid-related problems. The overall aim of the present study was to determine the feasibility of an integrated psychosocial treatment in Veterans with chronic pain, who also have evidence of hazardous opioid use. To examine this aim, a random design was used to assess the feasibility and initial efficacy of integrating 2 empirically supported interventions: Acceptance and Commitment Therapy for chronic pain and Mindfulness Based Relapse Prevention for opioid misuse. Half of participants were randomized to the integrated treatment group and all participants received usual care through a Veteran’s Administration co-occurring disorders medical clinic to treat chronic pain and opioid misuse. In total, 37 participants were randomized and included in intent-to-treat analyses and 32 individuals were included in per protocol analyses with 6-month follow-up serving as the primary study endpoint. Feasibility indicators included recruitment, retention, and treatment completion rates. Recruitment fell short of targeted enrollment, although retention and completion were excellent. Primary outcome measures were opioid misuse, pain interference, and pain behavior. Simultaneous multiple regression analyses controlled for pain duration, baseline opioid dose, and baseline value for outcome measures. Results of both the intent-to-treat and per protocol indicated a significant effect in favor of the integrated intervention for opioid misuse, pain interference, and pain behavior. Results support the feasibility of providing an integrated treatment for both opioid risk and pain interference.

Keywords: Chronic pain, opioids, veterans, integrated treatment, acceptance and commitment therapy, mindfulness-based relapse prevention


Chronic pain, defined as pain that persists longer than 3 months, is common, costly, and debilitating.10,26,35,50 The preponderance of the available evidence suggests that problems associated with chronic pain have been exacerbated by exponential increases in prescribed opioids over the past few decades.3,4 Overall opioid prescription rates have doubled in this century, from 11% to 20% of all pain-related ambulatory and office-based medical visits, while overall rates of pain as a primary symptom and prescription rates of nonopioid analgesics remained unchanged.16

While prescription rates have decreased moderately over the past few years, rates remained approximately 300% higher in 2016 compared to 2006.13,54 Furthermore, significant adverse consequences related to opioids remain, with the January 4, 2019 edition of the Center for Disease Control’s Morbidity and Mortality Weekly Report48 indicating continued increases in opioid-related mortality in 2017, with a total of 47,600 overdose deaths. The adverse impact of aberrant opioid use is widespread enough in the United States that it is now considered a public health crisis. In chronic pain patients, hazardous opioid use appears to occur in a clinically significant proportion of patients.23,37,59

In U.S. military Veterans, the issues of chronic pain and opioid-related problems are more pronounced than in nonveterans. Chronic pain is common, distressing, and debilitating, particularly in those that have served since the first Gulf War, with prevalence estimates reaching as high as 68%.15,28,31,36,51 As in civilian healthcare settings, opioids were increasingly used in Veteran healthcare settings for the treatment of chronic pain during the early part of this century.15,41,49 While the veterans affairs (VA) Opioid Safety Initiative52 and other efforts have led to decreasing prescription rates, the most recent 2018 VA Opioid Prescribing Data indicate that high rates of opioid prescribing remains a significant problem in the VA.70 Furthermore, there is evidence of a vicious cycle of comorbidity amongst chronic pain, opioid misuse (ie, using not as prescribed), and adverse events in Veterans.20,46 For example, a chronic pain diagnosis increases the risk for prescription drug misuse6 and opioid prescription for chronic pain increases risk of adverse clinical outcomes (eg, accidents resulting in wounds, opioid-related accidents, overdose, violence-related injuries) in a manner independent of mental health diagnoses or other prescribed medications.49 Clinicians in VA hospitals are well aware of these issues and have noted both frustration and uncertainty in treating persons with chronic pain, as well as the lack of evidence-based approaches that simultaneously treat problems related to both chronic pain and opioid misuse.18,45

Given the combined prevalence, cost, and disruptive nature of both chronic pain and problematic opioid use, particularly in Veterans, a crucial healthcare objective at present involves the identification of coordinated and efficacious treatments for patients with both chronic pain and opioid use disorder. Such treatments will need to simultaneously reduce both the problematic interference of chronic pain on functioning and problematic opioid use. While previous work has been shown to reduce opioid misuse behaviors in those with chronic pain,34 there are only a handful of interventions that have targeted both pain interference and hazardous opioid use in a coordinated fashion.65 Two recent studies incorporating cognitive behavioral treatment (CBT) for substance use disorders with medication assisted treatment for opioid-treated chronic pain failed to have a significant effect on pain interference.5,29 Naylor et al.43 used an Interactive Voice Response system as a follow-on to CBT for chronic pain and found it to have beneficial effect on both analgesic consumption and disability in comparison to no follow-up, although they did not measure opioid misuse specifically. To our knowledge, the only example of a successful integrated intervention are 2 trials by Garland and colleagues,24,25 who found that Mindfulness-Oriented Recovery Enhancement successfully reduced pain interference and opioid misuse in comparison to a support group control. Of note, participants in these trials did not have to present with evidence of opioid misuse for inclusion, and the latter trial24 specifically excluded these individuals.

There is clearly a need to examine interventions suitable for people with chronic pain who are presently using opioids in a hazardous manner. Integration of treatments that have shared theoretical frameworks and focus on adaptive behavior change may be both feasible and efficacious for chronic pain and opioid misuse. Two candidate interventions for integration include Acceptance and Commitment Therapy (ACT) for chronic pain30,39 and Mindfulness-Based Relapse Prevention (MBRP)8,68 for opioid misuse. These 2 interventions are excellent candidates for integration for several reasons. First, they each have evidence of effect. In those with chronic pain, ACT is effective at reducing pain interference1,32,44,53 and MBRP has established effectiveness in substance use disorder populations.7,21,69 Second, these 2 interventions come from a shared theoretical background, where the primary purposes of treatment include altering response to aversive experiences to allow for better functioning over the longer term. For example, ACT seeks to improve adaptive responses to the pain experience by increasing willingness to have pain in the service of valued life domains.57,62 Relatedly, MBRP seeks to alter response to substance cravings.7,9,21,67,69 Third, both interventions share an emphasis on the cultivation of present-focused awareness and willingness to have present experiences as a manner of achieving adaptive responding to pain and cravings specifically.8,30 The present study sought to test the feasibility and initial efficacy of an integrated treatment that combined ACT and MBRP to address opioid misuse, pain interference, and pain behavior in among Veterans with chronic pain who were prescribed opioids. Participants were randomized to 1 of 2 treatment arms: 1) physician care through a co-occurring disorders clinic for patients with chronic pain and evidence of opioid misuse (usual care; UC) or 2) the integrated treatment plus UC. Each group was followed for 12 weeks during the active intervention phase and the primary study endpoint was a 6-month follow-up appointment.

Methods

Participants

To be eligible for study inclusion, participants were required to be receiving treatment for chronic pain through the Albuquerque Veteran’s Affairs Medical Center, be prescribed at least 1 opioid medication for the treatment of chronic pain, and show evidence of current opioid misuse. Opioid misuse was defined as at least one of the following: a score on the current opioid misuse measure (COMM12) in excess of the established cut point of 9 or greater11,12 and/or meeting criteria for opioid use disorder using the structured clinical interview for DSM 522 (SCID). Participants also had to speak and read English and consent to randomization. Exclusion criteria included history of suicide attempt in the preceding 12 months, current buprenorphine prescription, or evidence of uncontrolled psychosis. All participants referred to the study were asked to complete a telephone screening, which provided study details and entailed an initial assessment in relation to inclusion criteria. Those who passed the phone screen were invited for an in-person screening session, where they completed the SCID, study questionnaires, and informed consent. Randomization occurred following this baseline assessment. A block randomization method was used with a block size of 4. This study was registered with clinicaltrials.gov (NCT02423772). This study was approved by both VA and University Institutional Review Boards.

Integrated Intervention

A 12-session treatment protocol was constructed using the methods described in other established protocols for ACT60,64 and MBRP.8 The study team are primary developers of ACT for chronic pain (KEV) and MBRP for substance use disorders (KW, SB). These individuals facilitated the training of study clinicians (MLM, RWB, KAE).

Sessions were held weekly for 90 minutes each, with every session including content from both ACT and MBRP, such that responses to pain and opioid craving were discussed during each session. Appendix 1 details the content of each session. In brief, the primary treatment objectives of ACT for chronic pain were to decrease pain’s interference on functioning, particularly with regard to meaningful activities.38 Treatment included exposure-based exercises, skills training, and behavioral experimentation designed to augment willingness and acceptance of pain, develop persistence and flexibility in the pursuit of behavioral goals, and increase effective responding to pain.38,61 Further, MBRP was designed to target experiences of craving and substance use.8,68 Intervention included skills training in relapse prevention (eg, coping skills training) and in mindfulness meditation. The mindfulness practices were intended to increase awareness and acceptance, with a specific focus on uncomfortable or unwanted experiences related to substance use. The practices teach clients to observe discomfort (including pain) without judgment and without engaging in an automatic or habitual reaction. Treatment was provided in a group format with a minimum group size of 4 participants.

Usual Care

All participants received physician directed care through a VA co-occurring disorders clinic that primarily provided medical oversight of opioid prescription and directed patients to other necessary pain interventions. For all participants, UC interventions were guided by the treatment plan and not manipulated by the research team. Consistent with national and VA standards,14,17 care typically consisted of noninvasive interventions, such as analgesic pain medications (eg, opioids, non-steroidal anti-inflammatories, antiepileptics), topical solutions (eg, lidocaine), physical therapy, and massage, as well as limited invasive interventions (eg, injections, radiofrequency denervation).

Measures

Feasibility

The feasibility of offering an integrated intervention for co-morbid chronic pain and opioid misuse was evaluated via 3 primary indicators. The first was feasibility of recruitment. We planned to randomize 120 individuals to the study. We aimed to randomize 80% of individuals meeting study inclusion and exclusion criterion. Second, we assessed feasibility of retention by evaluating treatment attrition. Given the results of other behavioral treatments for chronic pain, we expected an attrition rate of 20%.58,63,64,66 Finally, we assessed both session attendance rates and individual participant attendance rates. For both of these latter measures, we counted adequate attendance as 75% or greater.

Therapist Adherence

All study therapists completed a multiday workshop in ACT and MBRP and were doctoral trainees in a clinical psychology program. All clinical sessions were audio-taped and reviewed in full for adherence to protocol by the senior author (KEV), who also met with study therapists weekly for clinical supervision. It was planned that any therapist demonstrating poor adherence to the protocol would repeat the training workshops to ensure fidelity to the intervention.

Treatment Outcome

Primary outcome measures were risk of hazardous opioid use, as assessed by the COMM,12 pain interference, as measured by PROMIS short form 8a,2 and pain behavior, as measured by the PROMIS V1.1 short form 7a.47 The COMM is a 17-item measure designed to aid in the identification of opioid misuse. Scores range from 0 to 68 with higher scores indicating greater risk of misuse. As noted, a cut score of 9 or greater was used to identify the presence of opioid misuse. The PROMIS pain interference assesses the impact of pain on important aspects of social, physical, cognitive, and recreational activities. It consists of 8 items and scores range from 8 to 40 with higher scores indicating greater pain interference. The PROMIS pain behavior measures self-reported frequency of verbal and nonverbal behaviors indicating that one is experiencing pain. It consists of 7 items with scores ranging from 7 to 35 with higher scores indicating more frequent pain behavior.

There were 2 secondary outcomes analyzed. First, prescribed opioid dose was extracted from medical records and converted to morphine milligram equivalent dose (MED) per day. Second, usual pain intensity over the preceding 7 days was assessed via a 0 (no pain)-10 (worst pain possible) numerical rating scale.

Analytic Approach

For measures of adherence, we calculated study dropout rates and, for those randomized to the integrated intervention, session attendance rates, and percent of participants who completed treatment (75% or greater session attendance). Therapist protocol adherence was also calculated as a percent adherence to session content.

Intent-to-treat (ITT) and per protocol analyses were conducted using simultaneous linear regression equations, where follow-up score for each outcome was regressed onto baseline outcome score and group membership. Data management were performed using SPSS version 25.0,33 as was calculation of descriptive information. Simultaneous regression analyses used Mplus version 8.142 using maximum likelihood estimation. Maximum likelihood estimation is a preferred method for missing data handling, assuming data are missing at random. To control for any effects of treatment cohort, a weighted maximum likelihood function with robust standard errors was computed using a sandwich estimator (ie, standard errors were adjusted for cohort membership in the data).

Results

Participants

Participants were primarily male (86%) with relationship status split fairly evenly across 3 categories, including married/cohabitating (39%), single (32%), and divorced/separated (29%). Ethnically, participants identified as non-Hispanic white (51.4%), Latinx (28.6%), Native American (17.1%), and other (2.9%). Years of education averaged 14.5 (SD: 2.1) with identified categories of educational achievement including, "some college" (57.1%), college degree (14.3%), technical/ trade school degree (11.4%), high school degree (14.3%), and postgraduate degree (2.9%). All participants were unemployed, with the largest proportion of participants retired or not working due to pain (78.1%), and the remainder of individuals not working due to reasons unrelated to pain.

Across randomized participants, pain duration averaged 17.2 years (SD: 8.7). Primary pain location included low back (65.7%), leg/hip (14.3%), whole body (11.4%), and neck (8.6%). Secondary pain locations were reported by 85.7% of participants. Regarding compensation status, participants were receiving social security disability (57.1%), VA service connection (51.4%), and other (eg, workers compensation; 5.7%).

Feasibility

Fig 1 displays participant flow throughout the study. With regard to feasibility of recruitment, 115 individuals in total were referred to the study, of which 85 completed a phone screening. Forty-two individuals completed the in-person screen. Of these, 35 individuals met inclusion criteria and consented to randomization with 17 randomized to the integrated treatment and 18 randomized to UC. In total, 28 of these individuals, 80%, began the allocated treatment, with 88.2% and 72.0% beginning the integrated intervention and UC only arm, respectively. Of those who began the allocated intervention, 78.6% provided data at 6-month follow-up, including 80.0% and 76.9% completion rates for the integrated intervention and UC only arms, respectively.

Figure 1.

Figure 1.

Participant flow through the study

Of the 15 patients who began the integrated intervention arm, total treatment attendance across all 12 sessions was 77%. One participant attended only the first session; average attendance was 82% after excluding this individual. Thirteen participants had attendance rates in excess of 75% (ie, 9+ sessions) with the remaining 2 individuals attending 8% (1 session) and 17% (2 sessions). Both of these latter individuals dropped out of treatment.

In total, 4 treatment groups were conducted. Treatment group size ranged from 4 to 6. All groups ended with at least 3 participants. Three therapists were trained in the study protocol. Following full review of audio recordings of each therapy session, adherence rates to the protocol were rated as high, at 96%. Specifically, therapists provided the necessary components of treatment and the specific interventions identified at each session with high fidelity.

Outcomes Analyses

Table 1 displays demographic and baseline measure details for randomized individuals. At baseline, no group differences were indicated for any variable with the sole exception of prescribed MED. For this variable, the integrated intervention group began treatment with a higher prescribed daily MED dose in comparison to the UC group, F (1, 34) = 6.9, P ≤ .01. Boxplot inspection indicated 3 participants had MED values that were outliers. All 3 participants were randomized to the integrated treatment condition and had MED values of 318, 300, and 240 per day. A Winsorizing procedure replaced the outlying MED values for these 3 participants with the value of the next non-outlying MED values, which was 180 per day, and another ANOVA using the Winsorized data was conducted, which continued to indicate a significant between group difference, F (1, 34) = 7.1, P ≤ .01. Therefore, primary outcome analyses used MED as a covariate. Two of the participants with outlying baseline MED data completed follow-up and their raw (ie, non-Winsorized) baseline values were used in outcome analyses to ensure accurate reporting of any change in MED.

Table 1.

Demographic and Baseline Means (SD) for All Randomized Participants (n = 37)

Measure Total Sample Integrated Intervention UC
Age (y) 50.5 (10.5) 48.3 (11.6) 53.3 (8.6)
Pain duration (y) 16.1 (8.4) 14.7 (9.3) 17.8 (7.2)
Education (y) 14.5 (2.1) 14.5 (1.9) 14.5 (2,3)
Unemployment duration (y) 10.1 (7.4)   9.1 (7.9) 11.3 (6.7)
Current opioid misuse 15.9 (8.6) 15.3 (8.6) 16.7 (7.0)
Pain interference 66.3 (5.9) 66.7 (6.0) 65.7 (6.0)
Pain behavior 62.7 (4.7) 62.6 (4.1) 62.8 (5.5)
Pain intensity   6.5 (1.5)   6.4 (1.4)   6.6 (1.6)
Prescribed opioid dose 86.1 (79.9) 99.1 (65.3) 77.0 (58.7)*

Measures: Current opioid misuse: Current Opioid Misuse Measure, Pain interference: PROMIS Pain Interference short form 8a; Pain behavior: PROMIS Pain Behavior v1.1 short form 7a; Pain intensity: 0 to 10 Numerical Rating Scale of average pain for past week; Prescribed opioid dose: Morphine Equivalent Dose for all prescribed opioids.

*

Significant between group difference at baseline indicated.

As noted above, 35 individuals were randomized, of which 28 began the allocated intervention, and 22 completed 6-month follow-up. No baseline differences were indicated between participants who dropped out after randomization and who completed follow-up, all Fs < 3.3, all Ps > .08, with the sole exception of pain duration for which there was a significant between group difference indicated, F(1, 34) = 5.9, P =.02. Individuals who dropped out reported shorter average pain duration, 12.1 years (SD = 6.9), than those who completed follow-up, 18.7 years (SD = 8.4). Given this difference, pain duration was used as a covariate in all regression analyses.

With regard to primary outcome measures, simultaneous regression analyses were performed regressing follow-up values for primary and secondary outcome measures onto baseline values for the measures, prescribed MED, pain duration, and group membership. Standardized beta-weights and unstandardized betas were calculated, as was variance for each model.

Table 2 displays both ITT and per protocol results. In the case of 4 of 5 outcomes, including current opioid misuse, pain interference, pain behavior, and usual pain intensity in the past week, treatment group accounted for significant variance in 6-month values. In each case, membership in the integrated intervention group was associated with lower scores on each outcome. For prescribed MED, treatment group membership did not account for significant variance in follow-up dose.

Table 2.

Results of Simultaneous Multiple Regression Analyses Predicting 6 Month Follow-up

β/B (SE)
Predictor Intent-to-Treat (n = 35) Per Protocol (n = 28)
Current opioid misuse
 Baseline .76****/.76 (.10)**** .77****/.76 (.11)****
 MED .13/.01 (.01) .14/.01 (.01)
 Pain duration .03/.002 (.01) .03/.002 (.01)
 Group .36***/5.39 (1.71)*** .36****/5.39 (1.56)****
Model R2 .76 (.08)**** .75 (.12)****
Pain interference
 Baseline .55****/.75 (.27)*** .59***/.75 (.27)***
 MED .15/.02 (.02) .16/.02 (.02)
 Pain duration .16/.01 (.01) .17/.01 (.01)
 Group .41***/6.50 (2.02)** .40***/6.5 (2.5)**
Model R2 .54 (.13)**** .56 (.15)****
Pain behavior
 Baseline .36*/.39 (.14)** .34*/.39 (.14)***
 MED .51****/.03 (.01)*** .57****/.03 (.11)***
 Pain duration −.10/−.01 (.01) −.10/−.01 (.01)
 Group .41***/4.19 (2.07)* .42***/4.19 (2.07)*
Model R2 .37 (.15)* .36 (.13)*
Pain intensity
 Baseline .30/.36 (.21) .31/.36 (.21)
 MED .09/.002 (.01) .10/.002 (.01)
 Pain duration .13/.002 (.003) .14/.002 (.003)
 Group .53****/1.94 (.68)*** .54****/1.94 (.68)***
Model R2 .40 (.13)**** .39 (.13)***
Prescribed opioid dose
 Baseline .95****/.94 (.06)**** .96****/.94 (.06)****
 Pain duration .05/.04 (.04) .05/.04 (.04)
 Group −.04/−6.24 (5.76) −.04/− 6.24 (5.76)
Model R2 .95 (.02)*** .96 (.02)***
*

P ≤ .05,

**

P ≤ .01,

***

P ≤ .005,

****

P ≤ .001.

Measures: Current opioid misuse: Current opioid misuse measure, Pain interference: PROMIS pain interference short form 8a; Pain behavior: PROMIS pain behavior v1.1 short form 7a; Pain intensity: 0 to 10 numerical rating scale of average pain for past week; Prescribed opioid dose: Morphine equivalent dose for all prescribed opioids.

Group (0 = integrated intervention, 1 = usual care).

Table 3 presents descriptive information for all outcomes for each treatment group. For current opioid misuse, COMM scores for the UC group remained fairly stable across the 2 assessment points, while the score decreased for the integrated intervention group. All patients had baseline COMM scores in excess of the recommended cut-point of 9 or above.12 At follow-up, 6 individuals (50%) in the integrated intervention group and 1 individual (10%) in the UC group had COMM scores that were below this cutoff. The effect size difference between the integrated intervention and UC at follow-up indicated a medium-to-large effect size of the integrated intervention on COMM scores (d = .73).

Table 3.

Means (SD) for Primary and Secondary Measures for All Completers (n = 22)

Group Baseline Follow-up Cohen’s d for Integrated vs Usual Care at Follow-up
Current opioid misuse
Integrated intervention 16.2 (8.8) 10.6 (6.5) 0.73
Usual care 17.4 (7.8) 16.1 (8.5)
Pain interference
Integrated intervention 66.7 (6.6) 64.3 (9.2) 0.79
Usual care 65.2 (6.8) 70.1 (4.9)
Pain behavior
Integrated intervention 66.7 (6.6) 64.3 (9.2) 0.30
Usual care 66.0 (6.6) 66.8 (8.0)
Pain intensity
Integrated intervention 6.6 (1.5) 5.6 (1.9) 1.08
Usual care 6.7 (1.8) 7.4 (1.4)
Prescribed opioid dose
Integrated intervention 96.4 (70.1) 91.3 (72.1) 0.83
Usual Care 51.8 (45.6) 41.6 (43.1)

Measures: Current opioid misuse: Current Opioid Misuse Measure, Pain interference: PROMIS Pain Interference short form 8a; Pain behavior: PROMIS Pain Behavior v1.1 short form 7a; Pain intensity: 0 to 10 numerical rating scale of average pain for past week; Prescribed opioid dose: Morphine Equivalent Dose for all prescribed opioids.

For both pain interference and usual pain intensity, scores for the integrated intervention group decreased while scores for the UC group increased between baseline and follow-up with large effect size differences between groups at follow-up (pain interference: d = .79; pain intensity: d = 1.08). For Pain Behavior, scores in the integrated intervention group reduced modestly and stayed steady in UC, with a small between group effect size at follow-up (d = .30). Prescribed opioid dose decreased over this same time period for both groups and the integrated intervention had a higher prescribed opioid dose at both baseline and follow-up (d = .83).

Discussion

A clinical pilot study was conducted to examine the feasibility and initial efficacy of an integrated behavioral treatment for co-morbid chronic pain and opioid misuse. In total, 35 Veterans were randomized, with 28 included in per protocol analyses. Two evidence-based interventions were utilized in the integrated treatment group, ACT for chronic pain and MBRP for substance misuse. All individuals received usual care (UC) of physician directed treatment though a specialized clinic to address chronic pain and opioid misuse. In addition to assessing feasibility of recruitment and retention, primary outcomes were opioid misuse, pain interference, and pain behavior. All participants evidenced opioid misuse behaviors, as indicated by baseline COMM scores, and all were unemployed with the majority also receiving some type of wage replacement (eg, social security disability, VA service connection). Average pain duration was quite pronounced with an average of 16 years (+8).

With regard to feasibility, 2 primary results are apparent. First, the trial was under-recruited, as we randomized 29.2% of our intended sample size of 120. There were several factors that decreased recruitment rates, including 1) Veteran hesitation to participate in a trial of behavioral therapy addressing opioid use, 2) the exclusionary criterion of buprenorphine prescription, which was used increasingly over the study period, and 3) the requirement of in-person treatment attendance, which was not feasible for some Veterans, such as those that had to travel long distances to the study facility. Thus, the feasibility of recruitment was less than expected. It may be possible to enhance recruitment through further integration with the UC arm of treatment, which was physician led in the current study and fully independent of the integrated intervention. Such interdisciplinary care is the recommended standard for chronic pain treatment19,27 and, for ACT specifically, there is evidence of larger treatment effects for interdisciplinary chronic pain treatment compared with psychology-only for disability, psychosocial impact, and depression (Vowles, 2018). Such integrated care may help to allay patient concerns about the possibility that treatment will force opioid dose reduction. Furthermore, the inclusion of patients on buprenorphine or the use of telehealth services may aid in increasing the number of potential participants.

The second result supported feasibility of retention – once patients were assessed for eligibility, a majority were randomized (35 of 42, 83.3%), began the study postrandomization (28 of 35, 80.0%), and provided data at 6-month follow-up (22 of 28; 78.6%). Furthermore, in those randomized to the integrated treatment arm, session attendance was 77% and 13 of 15 participants had attendance rates in excess of 75%.

Outcome analyses for this pilot trial provided initial support for the efficacy of the integrated intervention in comparison to UC alone across the 2 of the primary outcome measures, opioid misuse and pain interference with medium to large effect size differences between groups at follow-up. The third primary outcome, pain behavior, was more modestly related to treatment group, with a small between group difference at follow-up. Assignment to the integrated intervention group was associated with lower opioid misuse and pain interference scores at 6-month follow-up in comparison to assignment to the UC group, after controlling for baseline values, baseline prescribed MED, and pain duration. Standardized betacoefficients were statistically significant for group membership in a moderate range (β range: .35-.45). The pattern of findings was the same for the secondary outcome of pain intensity, but not for MED, which decreased for both groups over the study enrollment period.

As noted above, previous work has examined integrated behavioral treatment for co-morbid chronic pain and opioid misuse. Two recent trials of CBT failed to find evidence of reduction in pain interference.5,29 The successful trials of Garland and colleagues24,25 did not specifically recruit participants with evidence of ongoing opioid misuse. The results of these intervention trials serve to underscore the complexity in managing co-occurring chronic pain and opioid misuse, where treatment targets include altering responses to persistent pain and opioid use cues, such as withdrawal or opioid craving. At the onset of the present trial, we hypothesized that ACT and MBRP would be particularly well-suited for integration in this clinical population. Each treatment prioritizes effective functioning with ongoing aversive experiences, such as pain and substance craving.39,67 These experiences are likely to continue to be a part of living for those with chronic pain who have used opioids in a hazardous manner. Furthermore, both seek to cultivate aspects of present-focused awareness and acceptance in order to facilitate effective responding to ongoing aversive experiences. Finally, each has an established evidence base. While these data are preliminary, they serve to support our hypotheses and suggest that a fully powered trial is worth pursuing.

There are limitations to consider. First, this was a pilot trial intended to primarily test feasibility and provide a preliminary test of efficacy of an integrated behavioral treatment. The sample size was small, which places limits on generalizability and power. Second, the UC condition did not include a psychotherapy comparator, which means that effects are not necessarily attributable to the integrated treatment and may be due to other nonspecific therapeutic factors. Relatedly, we did not collect extensive baseline information in relation to behavioral health diagnoses or pain-related aspects of psychosocial functioning (eg, pain-related fear). Third, we were able to collect data on prescribed MED only, which is not the same as consumed MED. While opioid dose is likely an important variable, there are also data to suggest that it is potentially more important to examine the manner in which opioids are consumed or the outcomes that patients are intending to achieve through consumption.12,40,55,56 Fourth, there was a failure of randomization in that average MED was approximately 25% higher in the treatment group compared to UC. While this difference at baseline between the arms of the study is important, it also potentially means the integrated intervention group was somewhat more complex than individuals in the UC arm. Fifth, as already noted, the study was under-recruited, which indicates poorer than expected feasibility and a need for more effective approaches to enrolling participants into the study. Finally, we did not assess therapist competency in treatment delivery. Such an assessment will be useful to consider in future studies.

In conclusion, chronic pain is a complex health condition that can have multifaceted negative effects on functioning. In at least a proportion of individuals prescribed opioids for the treatment of chronic pain, opioids seem to exacerbate the negative impacts of chronic pain. These individuals require effective intervention for both pain-related interference and problematic opioid use. The present study integrated ACT and MBRP to address this need by training participants to more effectively: 1) notice habitual responding to pain and craving in the present, 2) “make room” for negative feelings and sensations without engaging in ineffective strategies to control them, and 3) engage in personally valued activities to facilitate greater quality of life. The results of this study provide preliminary evidence that such an approach has promising feasibility and efficacy.

Supplementary Material

1

Perspective:

Opioid misuse occurs in some opioid-prescribed individuals with chronic pain. Few treatment options exist that target both pain interference and opioid misuse. This study examined feasibility and initial efficacy of an integrated behavioral treatment for Veterans. Feasibility was supported, except recruitment. Efficacy was supported compared to usual care.

Acknowledgments

Funding: This work was supported by funding from the National Center for Complementary and Integrative Health (R34 AT08398, Vowles, PI).

Footnotes

Supplementary data

Supplementary data related to this article can be found at https://doi.org/10.1016/j.jpain.2019.11.007.

References

  • 1.American Psychological Association - Division 12 Society of Clinical Psychology (Institution/Organization): Acceptance and commitment therapy for chronic pain [Internet]. [cited 2018 Apr 3]. Available at: http://www.div12.org/PsychologicalTreatments/treatments/chronicpain_act.html. Accessed April 5, 2019
  • 2.Amtmann D, Cook KF, Jensen MP, Chen W, Choi S, Revicki D, Cella D, Rothrock N, Keefe F, Callahan L, Lai J: Development of a PROMIS item bank to measure pain interference. Pain 150:173–182, 2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Bailey RW, Vowles KE: Chronic noncancer pain and opioids: Risks, benefits, and the public health debate. Prof Psychol Res Pract 46:340–347, 2015 [Google Scholar]
  • 4.Ballantyne JC, Fleisher LA: Ethical issues in opioid prescribing for chronic pain. Pain 148:365–367, 2010. Accessed April 5, 2019 [DOI] [PubMed] [Google Scholar]
  • 5.Barry DT, Beitel M, Cutter CJ, Fiellin DA, Kerns RD, Moore BA, Oberleitner L, Madden LM, Liong C, Ginn J, Schottenfeld RS: An evaluation of the feasibility, acceptability, and preliminary efficacy of cognitive-behavioral therapy for opioid use disorder and chronic pain. Drug Alcohol Depend 194:460–467, 2019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Becker WC, Fiellin DA, Gallagher RM, Barth KS, Ross JT, Oslin DW: The association between chronic pain and prescription drug abuse in Veterans. Pain Med 10:531–536, 2009 [DOI] [PubMed] [Google Scholar]
  • 7.Bowen S, Chawla N, Collins SE, Witkiewitz K, Hsu S, Grow J, Clifasefi S, Garner M, Douglass A, Larimer ME, Marlatt A: Mindfulness-Based Relapse Prevention for Substance Use Disorders: A Pilot Efficacy Trial. Subst Abus 30:295–305, 2009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Bowen S, Chawla N, Marlatt GA: Mindfulness-Based Relapse Prevention for Addictive Behaviors: A Clinician’s Guide. New York, NY, Guilford Press, 2010 [Google Scholar]
  • 9.Bowen S, Witkiewitz K, Clifasefi SL, Grow J, Chawla N, Hsu SH, Carroll HA, Harrop E, Collins SE, Lustyk MK, Larimer ME: Relative efficacy of mindfulness-based relapse prevention, standard relapse prevention, and treatment as usual for substance use disorders. JAMA Psychiatry Am Med Assoc 71:547, 2014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Breivik H, Collett B, Ventafridda V, Cohen R, Gallacher D: Survey of chronic pain in Europe: Prevalence, impact on daily life, and treatment. Eur J Pain 10:287–333, 2006 [DOI] [PubMed] [Google Scholar]
  • 11.Butler SF, Budman SH, Fanciullo GJ, Jamison RN: Cross validation of the current opioid misuse measure (COMM) to monitor chronic pain patients on opioid therapy. Clin J Pain 26:770–776, 2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Butler SF, Budman SH, Fernandez KC, Houle B, Benoit C, Katz N, Jamison RN: Development and validation of the current ppioid misuse measure. Pain 130:144–156, 2007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Centers for Disease Control and Prevention: U.S. Opioid Prescribing Rate Maps [Internet]. 2018. [cited 2018 Jan 7]. Available at: http://https//www.cdc.gov/drugoverdose/maps/rxrate-maps.html. Accessed April 5, 2019
  • 14.Chou R, Qaseem A, Snow V, Casey D, Cross JT, Shekelle P, Owens DK: Diagnosis and treatment of low back pain : A joint clinical practice guideline from the American College of Physicians and the American Pain Society. Ann Intern Med 147:478–491, 2007 [DOI] [PubMed] [Google Scholar]
  • 15.Clark JD: Chronic pain prevalence and analgesic prescribing in a general medical population. J Pain Symptom Manage 23:131–137, 2002 [DOI] [PubMed] [Google Scholar]
  • 16.Daubresse M, Chang H, Yu Y, Viswanathan S, Shah ND, Stafford RS, Kruszewski SP, Alexander GC: Ambulatory diagnosis and treatment of nonmalignant pain in the United States, 2000-2010. Med Care 51:870–878, 2013 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Department of Veterans Affairs: VHA Pain Management Strategy [Internet]. Available at: http://www.va.gov/PAINMANAGEMENT/VHA_Pain_Management_Strategy.asp. Accessed April 5, 2019
  • 18.Dobscha SK, Corson K, Flores JA, Tansill EC, Gerrity MS: Veterans affairs primary care clinicians’ attitudes toward chronic pain and correlates of opioid prescribing rates. Pain Med 9:564–571, 2008 [DOI] [PubMed] [Google Scholar]
  • 19.Dowell D, Haegerich T, Chou R: CDC guidelines for prescribing opioids for chronic pain - United States, 2016. 1:1–49, 2016. [DOI] [PubMed] [Google Scholar]
  • 20.Edlund MJ, Steffick D, Hudson T, Harris KM, Sullivan M: Risk factors for clinically recognized opioid abuse and dependence among veterans using opioids for chronic noncancer pain. Pain 129:355–362, 2007 [DOI] [PubMed] [Google Scholar]
  • 21.Elwafi HM, Witkiewitz K, Mallik S, Iv TAT, Brewer JA: Mindfulness training for smoking cessation: Moderation of the relationship between craving and cigarette use. Drug Alcohol Depend 130:222–229, 2012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.First MB, Williams JB, Karg RS, Spitzer RL: Structured Clinical Interview for DSM-V Disorders - Research Version. (SCID-V-RV). American Psychiatric Association, 2016 [Google Scholar]
  • 23.Fishbain DA, Cole B, Lewis J, Rosomoff HL, Rosomoff RS: What percentage of chronic nonmalignant pain patients exposed to chronic opioid analgesic therapy develop abuse/addiction and/or aberrant drug-related behaviors? A structured evidence-based review. Pain Med 9:444–459, 2008 [DOI] [PubMed] [Google Scholar]
  • 24.Garland EL, Hanley AW, Riquino MR, Reese SE, Baker AK, Salas K, Yack BP, Bedford CE, Bryan MA, Atchley R, Nakamura Y, Froeliger B, Howard MO: Mindfulness-oriented recovery enhancement reduces opioid misuse risk via analgesic and positive psychological mechanisms: A randomized controlled trial. J Consult Clin Psychol 87:927–940, 2019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Garland EL, Manusov EG, Froeliger B, Kelly A, Williams JM, Howard MO: Mindfulness-oriented recovery enhancement for chronic pain and prescription opioid misuse: Results from an early-stage randomized controlled trial. J Consult Clin Psychol 82:448–459, 2014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Gaskin DJ, Richard P: The economic costs of pain in the United States. J Pain 13:715–724, 2012 [DOI] [PubMed] [Google Scholar]
  • 27.Gatchel RJ, Okifuji A: Evidence-based scientific data documenting the treatment and cost-effectiveness of comprehensive pain programs for chronic nonmalignant pain. J Pain 7:779–793, 2006 [DOI] [PubMed] [Google Scholar]
  • 28.Gironda RJ, Clark ME, Massengale JP, Walker RL: Pain among veterans of operations enduring freedom and Iraqi freedom. Pain Med 7:339–343, 2006 [DOI] [PubMed] [Google Scholar]
  • 29.Guarino H, Fong C, Marsch LA, Acosta MC, Syckes C, Moore SK, Cruciani RA, Portenoy RK, Turk DC, Rosenblum A: Web-based cognitive behavior therapy for chronic pain patients with aberrant drug-related behavior: Outcomes from a randomized controlled trial. Pain Med 19:2423–2437, 2018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Hayes S, Strosahl K, Wilson K: Acceptance and Committment Therapy: The Process and Practice of Mindful Change, 2nd ed. New York, Guilford Press, 2012 [Google Scholar]
  • 31.Helmer DA, Chandler HK, Quigley KS, Blatt M, Teichman R, Lange G: Chronic widespread pain, mental health, and physical role function in OEF/OIF veterans. Pain Med 10:1174–1182, 2009 [DOI] [PubMed] [Google Scholar]
  • 32.Hughes LS, Clark J, Colclough JA, Dale E, McMillan D: Acceptance and commitment therapy (ACT) for chronic pain: A systematic review and meta-analysis. Clin J Pain 33:552–568, 2017 [DOI] [PubMed] [Google Scholar]
  • 33.IBM Corp (Institution/Organization): SPSS Statistics for Windows. Armonk, NY [Google Scholar]
  • 34.Jamison RN, Ross EL, Michna E, Chen LQ, Holcomb C, Wasan AD: Substance misuse treatment for high-risk chronic pain patients on opioid therapy: A randomized trial. Pain 150:390–400, 2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Johannes CB, Le TK, Zhou X, Johnston JA, Dworkin RH: The prevalence of chronic pain in United States adults: Results of an internet-based survey. J Pain 11:1230–1239, 2010 [DOI] [PubMed] [Google Scholar]
  • 36.Kerns RD, Otis J, Rosenberg R, Reid MC: Veterans’ reports of pain and associations with ratings of health, health-risk behaviors, affective distress, and use of the healthcare system. J Rehabil Res Dev 40:371–379, 2003 [DOI] [PubMed] [Google Scholar]
  • 37.Martell BA, O’Connor PG, Kerns RD, Becker WC, Morales KH, Kosten TR, Fiellin DA: Systematic review: Opioid treatment for chronic back pain: Prevalence, efficacy, and association with addiction. Ann Intern Med 146:116–127, 2007 [DOI] [PubMed] [Google Scholar]
  • 38.McCracken LM: Contextual Cognitive-Behavioral Therapy for Chronic Pain. Seattle, WA, IASP Press, 2005 [Google Scholar]
  • 39.McCracken LM, Vowles KE: Acceptance and commitment therapy and mindfulness for chronic pain: Model, process, and progress. Am Psychol 69:178–187, 2014 [DOI] [PubMed] [Google Scholar]
  • 40.Meltzer EC, Rybin D, Saitz R, Jeffrey H, Schwartz SL, Butler SF, Jane M: Identifying prescription opioid use disorder in primary care: Diagnostic characteristics of the current opioid misuse measure (COMM). Pain 152:397–402, 2011 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Morasco BJ, Duckart JP, Carr TP, Deyo RA, Dobscha SK: Clinical characteristics of veterans prescribed high doses of opioid medications for chronic non-cancer pain. Pain 151:625–632, 2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Muthén LK, Muthén BO: Mplus Users Guide (Version 8). CA, Muthén & Muthén, 2017 [Google Scholar]
  • 43.Naylor MR, Naud S, Keefe FJ, Helzer JE: Therapeutic interactive voice response (TIVR) to reduce analgesic medication use for chronic pain management. J Pain 11:1410–1419, 2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Öst L-G: The efficacy of acceptance and commitment therapy: An updated systematic review and meta-analysis. Behav Res Ther 61, 2014 [DOI] [PubMed] [Google Scholar]
  • 45.Pade PA, Cardon KE, Hoffman RM, Geppert CMA: Prescription opioid abuse, chronic pain, and primary care: A co-occurring disorders clinic in the chronic disease model. J Subst Abuse Treat 43:446–450, 2012 [DOI] [PubMed] [Google Scholar]
  • 46.Reid MC, Engles-Horton LL, Weber MB, Kerns RD, Rogers EL, O’Connor PG: Use of opioid medications for chronic noncancer pain syndromes in primary care. J Gen Intern Med 17:173–179, 2002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Revicki DA, Chen W-H, Harnam N, Cook KF, Amtmann D, Callahan LF, Jensen MP, Keefe FJ: Development and psychometric analysis of the PROMIS pain behavior item bank. Pain 146:158–169, 2009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Scholl L, Seth P, Kariisa M, Baldwin G: Drug and opioid-involved overdose deaths — United States, 2013–2017. Morb Mortal Wkly Rep 67:1419–1427, 2019. [Internet] Available at: www.cdc.gov/mmwr/volumes/67/wr/mm675152e1.htm. Accessed April 5, 2019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Seal KH, Shi Y, Cohen G, Cohen BE, Maguen S, Krebs EE, Neylan TC: Association of mental health disorders with prescription opioids and high-risk opioid use in US Veterans of Iraq and Afghanistan. J Am Med Assoc 307:940–947, 2012 [DOI] [PubMed] [Google Scholar]
  • 50.Smith BH, Elliott AM, Chambers WA, Smith WC, Hannaford PC, Penny K: The impact of chronic pain in the community. Fam Pract 18:292–299, 2001 [DOI] [PubMed] [Google Scholar]
  • 51.Thomas HV, Stimpson NJ, Weightman A, Dunstan F, Lewis G: Pain in veterans of the Gulf War of 1991: A systematic review. BMC Musculoskelet Disord 7:74–86, 2006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.US Department of Veterans Affairs, Department of Veterans Affairs: VHA Pain Management [Internet]. Opioid Saf. Initiat. Toolkit. 2014. Available at: http://www.va.gov/PAINMANAGEMENT/VHA_Pain_Management_Strategy.asp. Accessed July 22, 2017
  • 53.Veehof MM, Trompetter HR, Bohlmeijer ET, Schreurs KMG: Acceptance- and mindfulness-based interventions for the treatment of chronic pain: A meta-analytic review. Cogn Behav Ther 45:1–27, 2016 [DOI] [PubMed] [Google Scholar]
  • 54.Volkow ND: America’s addiction to opioids: Heroin and prescription drug abuse [Internet]. 2014. Available at: http://www.drugabuse.gov/about-nida/legislative-activities/testimony-to-congress/2015/americas-addiction-to-opioids-heroin-prescription-drug-abuse. Accessed April 5, 2019
  • 55.Vowles KE, Ashworth J: Is opioid withdrawal necessary within comprehensive pain rehabilitation programs? Pain 152:10–12, 2011 [DOI] [PubMed] [Google Scholar]
  • 56.Vowles KE, Bailey RW, McEntee ML, Pielech M, Edwards KA, Bolling LA, Rivers WE: Using analgesics for emotional modulation is associated with increased distress, depression, and risk of opioid and alcohol misuse: Initial evaluation and component analysis of the reasons for analgesic use measure (RAUM). Clin J Pain 34:975–982, 2018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Vowles KE, McCracken LM: Acceptance and values-based action in chronic pain: A study of treatment effectiveness and process. J Consult Clin Psychol 76:397–407, 2008 [DOI] [PubMed] [Google Scholar]
  • 58.Vowles KE, McCracken LM, O’Brien JZ: Acceptance and values-based action in chronic pain: A three-year follow-up analysis of treatment effectiveness and process. Behav Res Ther 49:748–755, 2011 [DOI] [PubMed] [Google Scholar]
  • 59.Vowles KE, McEntee ML, Julnes PS, Frohe T, Ney JP, van der Goes DN, Siyahhan P, Frohe T, Ney JP, van der Goes DN: Rates of opioid misuse, abuse, and addiction in chronic pain : A systematic review and data synthesis. Pain 156:569–576, 2015 [DOI] [PubMed] [Google Scholar]
  • 60.Vowles KE, Sorrell JT: Life with chronic pain: An acceptance-based approach. 2007.
  • 61.Vowles KE, Sowden G, Ashworth J: A comprehensive examination of the theoretical framework underlying acceptance and commitment therapy for chronic pain. Behav Ther 45:390–401, 2014 [DOI] [PubMed] [Google Scholar]
  • 62.Vowles KE, Sowden G, Hickman J, Ashworth J: An analysis of within-treatment change trajectories in valued activity in relation to treatment outcomes following interdisciplinary acceptance and commitment therapy for adults with chronic pain. Behav Res Ther 115:46–54, 2019 [DOI] [PubMed] [Google Scholar]
  • 63.Vowles KE, Wetherell JL, Sorrell JT: Targeting acceptance, mindfulness, and values-based action in chronic pain: Findings of two preliminary trials of an outpatient group-based intervention. Cogn Behav Pract 16:49–58, 2009 [Google Scholar]
  • 64.Vowles KE, Witkiewitz K, Sowden G, Ashworth J: Acceptance and commitment therapy for chronic pain: Evidence of mediation and clinically significant change following an abbreviated interdisciplinary program of rehabilitation. J Pain 15:101–113, 2014 [DOI] [PubMed] [Google Scholar]
  • 65.Wachholtz A, Ziedonis D, Gonzalez G: Comorbid pain and opioid addiction: Psychosocial and pharmacological treatments. Subst Use Misuse 46:1536–1552, 2011 [DOI] [PubMed] [Google Scholar]
  • 66.Wetherell JL, Afari N, Rutledge T, Sorrell JT, Stoddard JA, Petkus AJ, Solomon BC, Lehman DH, Liu L, Lang AJ, Hampton Atkinson J: A randomized, controlled trial of ccceptance and commitment therapy and cognitive-behavioral therapy for chronic pain. Pain 152:2098–2107, 2011 [DOI] [PubMed] [Google Scholar]
  • 67.Witkiewitz K, Bowen S, Douglas H, Hsu SH: Mindfulness-based relapse prevention for substance craving. Addict Behav 38:1563–1571, 2013 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Witkiewitz K, Marlatt GA, Walker D: Mindfulness-Based relapse prevention for alcohol and substance use disorders. J Cogn Psychother Marlatt 19:211–228, 2005 [Google Scholar]
  • 69.Witkiewitz K, Warner K, Sully B, Barricks A, Stauffer C, Thompson BBL, Luoma JBJB, Steckler G, Thompson BBL, Luoma JBJB: A randomized trial comparing mindfulness-based relapse prevention with relapse prevention for women offenders at a residential addiction treatment center. Subst Use Misuse 49:536–546, 2014 [DOI] [PubMed] [Google Scholar]
  • 70.Department of Veterans Affairs Opioid Prescribing Data. 2018.

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1

RESOURCES