Treatment engagement in an internet-delivered cognitive behavioral program for pediatric chronic pain

Abstract

Internet-delivered cognitive-behavioral therapy (iCBT) is a promising treatment for chronic pain among youth, but effect sizes are small, and strategies aimed at enhancing treatment effects are needed. Participants' engagement with the program may be an important factor in determining treatment outcomes. The primary aim of the current study was to examine the relationship between treatment engagement and treatment outcomes. Secondarily, we sought to characterize participant engagement in an iCBT program for adolescents with chronic pain and their parents. Participants included 134 adolescents randomized to the intervention arm of a controlled trial examining iCBT for chronic pain. Overall engagement with the intervention by adolescents and parents was high. Parental engagement (number of modules completed by parents and number of parent logins) predicted adolescent activity limitations change scores at post-treatment. Contrary to our expectation, adolescent treatment engagement was not predictive of treatment outcomes. Results indicate that parental engagement with the program may be an important predictor of treatment outcomes. Further research is needed to better understand influences of treatment engagement on outcomes in iCBT for youth.

Highlights

• •Engagement with the intervention by adolescents and parents was high.
• •Parental engagement predicted adolescent activity limitations change scores at post-treatment.
• •Adolescent treatment engagement was not predictive of treatment outcomes.

1. Introduction

Chronic pain is common in childhood, impacting as many as 1 in 4 youth (King et al. 2011). A subset of youth with chronic pain experience significant pain-related disability as well as anxiety and depressive symptoms, sleep disturbance, and reduced quality of life (Huguet and Miro 2008; Valrie et al. 2013; Vinall et al. 2016). Over 30 years of research supports the efficacy of cognitive-behavioral therapy for pain management to reduce pain and disability among youth with chronic pain (Fisher et al. 2014). However, many youth with chronic pain do not receive psychological treatment due to barriers such as lack of access to trained professionals, burden associated with missed work and high costs for parents, and stigma against mental health treatment (Chandra and Minkovitz 2006; Ernst et al. 2015). Internet-delivered cognitive-behavioral therapy (iCBT) interventions for youth with chronic pain have the potential to address these barriers to care and a recent systematic review found that a few programs have been developed which show beneficial but small effects (Fisher et al. 2015). Research is needed to identify strategies that can enhance the effectiveness of Internet-delivered treatment protocols for youth with chronic pain.

There is a general assumption that greater engagement with Internet delivered treatment programs will lead to behavior change and positive treatment outcomes. Indeed, conceptual models have been developed to define engagement with Internet-delivered interventions and propose how this relates to treatment response (Mohr et al. 2014; Ritterband et al. 2009). Across these models, program use (e.g., logins, completion of program modules, completion of assignments) and human support (e.g., communication with an online therapist or coach) have been identified as important mediators of treatment efficacy. These conceptual models have been supported by numerous studies of adult populations, which have demonstrated that greater program use (i.e., completion of program modules, completion of assignments) is associated with improved treatment outcomes (e.g. Glasgow et al. 2011; Hadjistavropoulos et al. 2016; Hedman et al. 2015). Available data on the impact of human support on treatment efficacy are mixed. For example, some studies have indicated that iCBT is more effective with human support (Andersson et al. 2013), while others, including a study on iCBT for chronic pain among adults, have shown similar outcomes when comparing iCBT with and without human support (Dear et al. 2015).

Little is known about the impact of treatment engagement on response to iCBT for youth with chronic pain. To our knowledge, our research team has published the only evaluation of treatment engagement in iCBT among youth with chronic pain (Law et al. 2012). In this secondary data analysis of a small pilot RCT, we focused on the role of human support using a narrative coding scheme to categorize the content of asynchronous messages sent by adolescents to online coaches during the treatment program. The coding scheme specified three types of message content: rapport, treatment skills, and technical issues. Results showed enhanced treatment outcomes for adolescents who sent messages with rapport or treatment content but not technical issues (Law et al. 2012), indicating certain types of engagement with human support may improve treatment efficacy for these youth. However, associations between program use and treatment outcomes were not examined in this small pilot study, and research in this area is needed.

There is evidence of some early work examining Internet program use in other pediatric populations such as youth with anxiety (Morgan et al. 2018), which found that parental completion of assignments is associated with greater improvements in children's treatment outcomes. Although preliminary, these findings highlight the need to consider the impact of both parent and child program use on the efficacy of iCBT interventions for pediatric populations. To our knowledge, no prior published RCTs of iCBT for youth with chronic pain have reported on associations between program use and treatment outcomes.

To address this gap, the aim of this study was to examine the impact of treatment engagement on treatment response among adolescents who received iCBT for chronic pain. We have previously reported on the outcomes for this multisite randomized controlled trial (RCT), which demonstrated positive effects of iCBT on reducing adolescent activity limitations, but not pain intensity, at six-month follow-up compared to Internet-delivered pain education (Palermo et al. 2016). For this analysis, we hypothesized that greater adolescent and parent treatment engagement would predict greater reductions in adolescent's activity limitations and pain intensity at post-treatment and 6-month follow-up. Treatment engagement was defined as program use (module completion, assignment completion, logins) and human support (content of messages to the online coach with rapport or treatment skills content). As a secondary aim, we sought to characterize adolescent and parent engagement in the iCBT program.

2. Methods

2.1. Study design

Participants were randomized to the active treatment arm in a RCT evaluating the efficacy of iCBT for pediatric chronic pain. We have previously published manuscripts from this trial, including two manuscripts reporting on treatment outcomes (Law et al. 2018; Palermo et al. 2016) and one manuscript examining longitudinal associations between parent and child functioning (Law et al. 2017). Here we report for the first time on adolescent and parent engagement with the iCBT program and examine associations between engagement and treatment outcomes.

Families were enrolled from 15 multidisciplinary pediatric pain management clinics in the United States and Canada. Only participants randomized to the iCBT arm of the trial have been included (n = 134) in the current study. All families received the iCBT program, which was adjunctive to the standard care that participants received from their local pain clinic. Further information on the design, procedures, and participants of this RCT can be found in our prior publication reporting primary outcome analyses (Palermo et al. 2016).

The study was approved by the primary site's Institutional Review Board and the Institutional Review Boards at each referring center. Parents gave informed consent and adolescents gave assent prior to initiating study procedures. The clinical trial was registered and the full protocol is available (ClinicalTrials.gov Identifier NCT01316471).

2.2. Procedures

2.2.1. Internet-delivered cognitive behavioral therapy (iCBT)

All participants received iCBT for pain management (Palermo et al. 2016). Treatment content is grounded in cognitive-behavioral and social learning theories of pain management. The program includes eight treatment modules, designed to be completed in 30 min each, over 8–10 weeks. The program includes separate websites for adolescents and parents. The program is travel themed and has four main sections that are accessible from the home page: 1) treatment modules, 2) a message center, 3) a progress tracker, and 4) audio files of relaxation exercises. Core treatment components include education about chronic pain, training in behavioral (e.g., deep breathing, relaxation) and cognitive coping skills (e.g., reducing negative thoughts), instruction in increasing activity participation, and education about pain behaviors as well as parent operant and communication strategies.

Within each module, participants complete interactive fields in response to queries about their treatment goals, challenges they have faced due to their pain problem, and action plans for practicing treatment skills. These responses are then repopulated later in the modules to provide tailored and personalized instruction for behavioral assignments. At the end of each module, participants complete interactive quizzes to test knowledge acquisition. The program also has audiovisual features, including videos of peer models and audio files of relaxation exercises that participants can listen to via the program website or download onto their own device.

In six of the eight modules, participants are given a behavioral assignment focused on practicing skills learned in that module. Participants are asked to work on each assignment for one week, and then complete the assignment by logging back into the program and answering questions about their experience. Each assignment must be completed before participants are allowed to progress to the next module. An online coach reviews all completed assignments. In this trial, there were five online coaches (four were PhD-level psychology post-doctoral fellows and one with a master's degree) who all had previous experience delivering CBT. The online coach provided asynchronous feedback on each assignment via an online message center. Participants could also initiate messages to the online coach, although this was not required. Coaches used a previously developed manual to guide their responses to assignments and participant messages.

2.2.2. Assessment protocol

Assessments were completed at pre-treatment, immediate post-treatment, and 6-month follow-up through the secure study website. Adolescents and parents were instructed to complete assessments independently and privately.

2.3. Measures

2.3.1. User demographics

Parents reported on parent and adolescent's age, sex, race and family income at pre-treatment.

2.3.2. Treatment engagement

The web program automatically recorded the number of times adolescents and parents completed a module, completed an assignment, logged into the program, or sent a message to the online coach. To examine the content of e-mail messages sent by adolescents and parents to the online coach, we conducted automated textual analysis using Latent Dirichlet Allocation (LDA) (Blei et al. 2003) available through the Mallet toolkit (McCallum 2002). In LDA, documents are represented as random mixtures over latent topics, where each topic is characterized as a distribution over words (Blei et al. 2003). Automated textual analysis facilitates the evaluation of a larger body of content than is possible manually, and characterizes language use in measurable terms that can in turn serve as a consistent basis for comparison (Chen 2014). LDA has been used in a variety of applications relevant to health-related research, including the identification of health-related topics in social media data (Chen et al. 2015; Prier et al. 2011), and to examine topics in electronic medical records (Cohen et al. 2013).

In this report, automated textual analysis with LDA resulted in a 40-topic solution. The topic with the highest predicted likelihood was assigned to each message. To facilitate interpretation, a primary coder examined the messages that were included in each of the 40 topics and categorized each topic under one of the following broader themes based on the operational definitions used in our prior research (Law et al. 2012): 1) rapport content, 2) treatment content, or 3) study/technical issues content. A secondary coder reviewed 20% of the messages to confirm agreement that the topics were categorized into the appropriate theme. The total number of messages within each theme was calculated to provide total rapport content, treatment content, and technical issues content scores. See Table 1 for detailed definitions and examples of each theme.

Table 1.

List of categories for content analysis of adolescent and parent messages.

Category	Definition	Example
Rapport	Content that included sharing of personal information and efforts to relate to the online coach, sharing of personal information not related to pain or treatment, asking personal questions about the online coach, use of emoticons or SMS language (e.g., “lol”), expressions of gratitude.	Hi, thank you for the email. Happy Thanksgiving to you and your family. We are having friends and neighbors for dinner. I am really looking forward to the day. Thank you again for all of your advice.
Treatment skills	Content directly related to the skills being taught in the program, the adolescent's treatment goals, descriptions of the adolescent's pain problem.	It's nice to meet you to! The strategies that have been given to me have been helping me a lot. The third destination I find has really helped me relax. I just started my 2nd year of high school on Wednesday so I'm trying to get back into a routine of going to bed early and I find the muscle relaxation exercises have really been helping me calm down and sleep better.
Study/technical issues	Content containing questions and statements about study coordination and technical issues related to the website, questions about mailing reimbursement, questions about how to complete measures, reports of delays in progressing through the program, reports of technical problems, inquiries about whether online assignments were received, feedback about the website.	What do I do now that I've finished all of the diary entries? It won't tell me what to do now.

2.3.3. Treatment outcomes: Activity limitations and pain intensity

At each assessment time point, adolescents reported on activity limitations and pain intensity using a 7-day prospective online daily diary.

2.3.4. Activity limitations

Activity limitations due to pain were assessed using the Child Activity Limitations Interview (CALI), which is a reliable and valid self-report measure of pain-related disability in adolescents (Palermo et al. 2004). Adolescents selected eight activities that were important to them and rated how difficult each activity was to perform because of pain on a 5-point scale ranging from 0 (no difficulty) to 4 (extreme difficulty). Total scores range from 0 to 32, with higher scores indicative of greater limitations in daily activities due to pain. Average activity limitation scores were calculated for each assessment period.

2.3.5. Pain intensity

Pain intensity was assessed using an 11-point numerical rating scale (NRS; 0 = no pain and 10 = worst pain), a widely used measure of pain intensity in youth with chronic pain that has strong psychometric properties (von Baeyer 2009). Average pain intensity scores were calculated across each assessment period.

3. Data analytic plan

Descriptive statistics were used to calculate demographic characteristics and to describe treatment engagement. We used hierarchical linear regression to examine associations between program use (i.e., number of modules completed, percentage of assignments completed, number of logins) and human support (i.e., number of messages with rapport vs. treatment skills vs. technical content), and the treatment effect (i.e., the change in pain intensity and activity limitations from pre-treatment to post-treatment, and from pre-treatment to 6-month follow-up). Residualized change scores were calculated to determine change in pain intensity and activity limitations scores from pre- to post-treatment and from pre- to 6-month follow-up. For adolescents and parents, separate linear regressions were conducted with pre- to post-treatment and pre- to 6-month follow-up pain intensity and activity limitations residualized change scores as dependent variables. All models controlled for pre-treatment activity limitations and pain intensity scores. Independent variables included number of modules completed, percentage of assignments completed, number of logins, and total scores for the rapport, treatment, and study/technical issues message content themes. Statistical significance was set at p < .05.

4. Results

4.1. User characteristics and descriptive statistics for treatment outcomes

Demographic characteristics are shown in Table 2. Adolescents and parents were primarily female, Caucasian, and middle class. Means and SDs for activity limitations and pain intensity at pre-treatment, post-treatment, and 6-month follow-up are shown in Table 3. As shown, most of the adolescents endorsed moderate to high pre-treatment activity limitations and pain intensity.

Table 2.

User characteristics (n = 134 parent-adolescent dyads).

Characteristic	n (%)/M (SD)
Adolescent age (years)	14.63 (1.61)
Adolescent sex
Female	106 (79.1%)
Male	28 (20.9%)
Adolescent race
Caucasian	124 (92.5%)
Black or African American	2 (1.5%)
Other	7 (5.2%)
Missing	1 (0.8%)
Primary pain location
Head	11 (8.2%)
Abdominal	15 (11.2%)
Musculoskeletal	50 (37.3%)
Multiple	58 (43.3%)

Table 3.

Descriptive statistics for treatment outcomes.

Outcome	Mean (SD)
	Pre-treatment	Post-treatment	6-month follow-up
Activity limitations (CALI)	7.42 (4.52)	5.68 (4.38)	5.46 (4.32)
Pain intensity (NRS)	6.23 (1.72)	5.87 (2.05)	5.85 (1.97)

Note. SD = standard deviation; CALI = Child Activity Limitations Interview; NRS = numerical rating scale.

4.2. Descriptive information for adolescent and parent treatment engagement

Descriptive information about treatment engagement is shown in Table 4. Nearly all of the adolescents and parents initiated the treatment program by completing at least one module (99.3%, n = 133; 100%, n = 134, respectively). Adherence to the intervention was high, with 85.8% of adolescents (n = 115) and 82.8% of parents (n = 111) completing at least six out of eight modules. Most adolescents and parents completed all eight treatment modules (73.9%, n = 99; 73.1%, n = 98, respectively). Assignment completion was also high; on average, youth completed 82% of assignments (SD = 21.89), and parents completed 81% of assignments (SD = 26.56). On average, adolescents logged into the program more often than parents (M = 53.13, SD = 17.08 vs. M = 30.90 times, SD = 17.01, respectively).

Table 4.

Descriptive statistics for adolescent and parent treatment engagement.

Measure	Adolescents		Parents
	M (SD)	Range	M (SD)	Range
Program use
Number of modules completed	7.13 (1.78)	0–8	7.02 (1.90)	1–8
Assignments
% completed	81.71 (21.89)	0–96.67	81.32 (26.56)	0–100.00
Word count	25.85 (19.91)	0–121.17	43.21 (27.46)	0–144.83
Number of logins	53.13 (17.08)	10–102	30.90 (17.01)	5–123
Messages
Number sent	2.84 (2.91)	0–14	2.63 (2.37)	0–11
Word count	151.82 (198.74)	0–1132	232.59 (274.05)	0–1502
Human support
Rapport message content	1.43 (1.62)	0–7	0.87 (1.15)	0–5
Treatment skills message content	1.04 (1.39)	0–6	1.15 (1.30)	0–5
Study/technical issues message content	0.42 (0.79)	0–4	0.61 (1.00)	0–5

Most adolescents (75%, n = 100) and parents (73%, n = 98) initiated a message to the online coach. On average, adolescents sent 2.84 (SD = 2.91) messages (range 0–14), and parents sent 2.63 (SD = 2.37) messages (range 0–11). For adolescents, messages were most commonly classed as having rapport content (M = 1.43, SD = 1.62), followed by treatment skills content (M = 1.04, SD = 1.39), and study/technical issues content (M = 0.42, SD = 0.79). In contrast, for parents, messages were most commonly categorized as having treatment skills content (M = 1.15, SD = 1.30), followed by rapport content (M = 0.87, SD = 1.15), and study/technical issues content (M = 0.61, SD = 1.00).

4.3. Adolescent treatment engagement as a predictor of treatment response

Contrary to our hypothesis, results indicated that after controlling for pre-treatment activity limitation scores, adolescent treatment engagement (i.e., program use and human support factors) did not significantly predict activity limitations change scores at post-treatment or follow-up (β = −0.08–0.11, ps > 0.05). Similarly, after controlling for pre-treatment pain intensity scores, adolescent treatment engagement variables were not associated with pain intensity change scores at post-treatment or follow-up (β = −0.29–0.26, ps > 0.05).

4.4. Parent treatment engagement as a predictor of treatment response

As shown in Table 5, we found that after controlling for pre-treatment activity limitations scores, parent treatment engagement (i.e., program use and human support factors) predicted activity limitations change scores. However, the pattern of these findings only partially supported our hypothesis. Specifically, consistent with our hypothesis, greater parental module completion was associated with improvements in activity limitations at post-treatment (β = 0.28 p < .05). However, contrary to our hypothesis, we also found that greater parental logins was associated with worsening activity limitations at post-treatment (β = −0.14, p < .05). Furthermore, also contrary to our hypothesis, treatment engagement was not associated with activity limitations change scores or pain intensity change scores at 6-month follow-up (ps > 0.05; see Table 5).

Table 5.

Parent treatment engagement predicting change in treatment outcomes.

Predictor	Activity limitations				Pain intensity
	Post-treatment		6-month follow-up		Post-treatment		6-month follow-up
	Δ R2	β	Δ R2	β	Δ R2	β	Δ R2	β
Step 1	0.70⁎⁎⁎		0.71⁎⁎⁎		0.54⁎⁎⁎		0.62⁎⁎⁎
Control variablesa		0.84⁎⁎⁎		0.84⁎⁎⁎		0.74⁎⁎⁎		0.79⁎⁎⁎
Step 2	0.03⁎		0.02		0.02		0.01
Control variablesa		0.85⁎⁎⁎		0.86⁎⁎⁎		0.71⁎⁎⁎		0.76⁎⁎⁎
Program use factors
Number of modules completed		0.28⁎		0.16		−0.12		−0.17
% assignments completed		−0.22		−0.01		0.09		0.17
Number of logins		−0.14⁎		−0.06		−0.08		0.04
Human support factors
Rapport message content		0.03		−0.06		0.01		−0.06
Treatment skills message content		−0.04		−0.02		0.12		0.06
Study/technical issues message content		−0.07		−0.03		0.03		0.00
Total R2	0.73⁎⁎⁎		0.74⁎⁎⁎		0.56⁎⁎⁎			0.63⁎⁎⁎

Note.

^⁎p < .05.

^⁎⁎p < .01.

^⁎⁎⁎p < .001.

^a Control variables include pre-treatment activity limitations and pain scores, respectively.

5. Discussion

In this secondary data analysis, we sought to comprehensively describe adolescent and parent engagement with an iCBT pain management program and to examine the relationship between treatment engagement (defined as program use and human support) and treatment outcomes. Among adolescent and parent users, we found a high level of initiation and adherence to the iCBT program. These findings indicate that iCBT programs for pediatric chronic pain can be feasibly delivered to patients (the adolescent) as well as other family members (parents).

Our hypothesis that adolescent treatment engagement would predict patient responses to iCBT was not supported. We found that adolescent treatment engagement was not associated with activity limitations or pain intensity change scores at post-treatment or 6-month follow-up. These findings are inconsistent with our prior work examining human support in an earlier, pilot version of our iCBT program (Law et al. 2012). In that study we found that adolescents had enhanced treatment outcomes (greater improvements in activity limitations and pain) when they sent a greater number of messages to the online coach with treatment or rapport content. In contrast, our current findings suggest that adolescent engagement with the online coach may not necessarily influence adolescents' treatment response. There are several potential explanations for the differences in findings between these two studies. First, the sample size in our current report (n = 134) was much larger than in our pilot study (n = 26), which provides a more representative sample and increased power to detect true associations. Second, in the current report we used automated textual analysis to categorize each message into themes whereas the prior analysis used a more traditional qualitative coding approach, which may have contributed to the difference in findings. Our results are also in contrast to the study by Morgan et al. (2018) where more self-reported homework practice by parents predicted reduced child anxiety at follow-up. In the current study, percentage of behavioral assignments completed by parents or adolescents, did not significantly predict any of the treatment outcomes.

We also examined parent treatment engagement as a predictor of adolescents' treatment outcomes. Consistent with our hypothesis, we found that greater parental module completion was significantly associated with improvements in post-treatment activity limitations although this was not maintained at 6-month follow-up. This is similar to prior studies of engagement with iCBT programs for adults, which have also found associations between greater module completion and improved treatment outcomes (e.g. Hadjistavropoulos et al. 2016; Hedman et al. 2015). Research is needed to determine whether the impact of treatment engagement on treatment response is more integral for immediate post-treatment improvements and less relevant for maintenance of treatment gains over the long-term.

Contrary to our hypothesis, however, we found that greater parental logins was associated with worsening activity limitations at post-treatment. The direction of this relationship was unexpected and there are several possible explanations. When parents perceive greater difficulties in their adolescents' functioning they could log into the treatment program more frequently in attempts to seek help; alternatively, more frequent parental logins could reflect a lack of accomplishing the treatment material or of understanding it. Although unexpected, there is one prior study of iCBT for adults with chronic pain where greater user logins was associated with a decreased likelihood of clinical improvement in depression at post-treatment (Dear et al. 2016). Similar to adolescents, and contrary to our expectation, parental treatment engagement was not associated with adolescents' pain intensity change scores. Further research is needed to determine whether (and at what point during the treatment period) the rate of logins or other treatment engagement factors could be used to identify patients at risk for poor response to treatment. Research is also needed to determine how to best leverage parental treatment engagement to enhance treatment outcomes for iCBT programs targeting adolescents.

Taken together, our findings lend support to the notion that simply more engagement in an iCBT program may not necessarily lead to symptom reduction (Yardley et al. 2016). It is also possible that, in studying engagement in Internet-delivered interventions, we are failing to measure other variables that may influence treatment outcomes such as the use of treatment skills in real-world settings. Research is needed to understand the impact of these self-management behaviors on treatment response.

As this is the first study to comprehensively evaluate treatment engagement as a predictor of response to iCBT for youth with chronic pain, our results should be interpreted cautiously and in the context of several limitations. For example, although consistent with clinical populations of youth with chronic pain, our sample was largely homogenous such that it was primarily white and upper middle class, thus, generalization of the current findings to minority groups is unknown. Also, although our sample size was larger than prior studies, it is possible that an even larger sample size is required to detect very small effects between treatment engagement and treatment outcomes.

Taken together, these limitations and considerations highlight the need for more research examining treatment engagement in relation to treatment outcomes in iCBT programs designed for adolescents. Additional data on treatment engagement and predictors of outcomes among adolescents would also further inform and contribute to the theoretical model of Internet interventions (Ritterband et al. 2009).