Barriers Associated with the Implementation of Homework in Youth Mental Health Treatment and Potential Mobile Health Solutions

Abstract

Background:

Homework, or between-session practice of skills learned during therapy, is integral to effective youth mental health TREATMENTS. However, homework is often under-utilized by providers and patients due to many barriers, which might be mitigated via mHealth solutions.

Methods:

Semi-structured qualitative interviews were conducted with nationally certified trainers in Trauma Focused Cognitive Behavioral Therapy (TF-CBT; n=21) and youth TF-CBT patients ages 8–17 (n=15) and their caregivers (n=12) to examine barriers to the successful implementation of homework in youth mental health treatment and potential mHealth solutions to those barriers.

Results:

The results indicated that many providers struggle to consistently develop, assign, and assess homework exercises with their patients. Patients are often difficult to engage and either avoid or have difficulty remembering to practice exercises, especially given their busy/chaotic home lives. Trainers and families had positive views and useful suggestions for mHealth solutions to these barriers in terms of functionality (e.g., reminders, tracking, pre-made homework exercises, rewards) and user interface (e.g., easy navigation, clear instructions, engaging activities).

Conclusions:

This study adds to the literature on homework barriers and potential mHealth solutions to those barriers, which is largely based on recommendations from experts in the field. The results aligned well with this literature, providing additional support for existing recommendations, particularly as they relate to treatment with youth and caregivers.

Introduction

Homework, or between-session practice of skills learned during therapy, is one of the most integral, yet underutilized components of high-quality, evidence-based mental health care (Kazantzis & Deane, 1999). Homework activities (e.g., self-monitoring, relaxation, exposure, parent behavior management) are assigned by providers in-session and completed by patients between sessions with the goal of “practicing” therapeutic skills in the environment where they will be most needed (Kazantzis, Deane, Ronan, & L’Abate, 2005). There are numerous benefits to the implementation of homework during mental health treatment (Kazantzis et al., 2016; Kazantzis, Deane, & Ronan, 2004). Homework enables the generalization of skills and behaviors learned during therapy, facilitates treatment processes, provides continuity between sessions, allows providers to better grasp patients’ learning, and strengthens that learning, leading to improved maintenance of treatment gains (Hudson & Kendall, 2002; Scheel, Hanson, & Razzhavaikina, 2004). Meta-analytic and systematic reviews have shown that homework use by providers and adherence by patients predict increased treatment engagement, decreased treatment dropout, and medium-to-large effects on improvements in clinical outcomes for use (Cohen’s d=.48–.77) and adherence (d=.45–.54) (Hudson & Kendall, 2002; Kazantzis, Deane, & Ronan, 2000; Kazantzis & Lampropoulos, 2002; Kazantzis, Whittington, & Dattilio, 2010; Mausbach, Moore, Roesch, Cardenas, & Patterson, 2010; Scheel et al., 2004; Sukhodolsky, Kassinove, & Gorman, 2004). Simply put, 68% vs. 32% of patients can be expected to improve when therapy involves homework (Kazantzis et al., 2010).

Despite its many benefits, homework is implemented with variable effectiveness in mental health treatment. Only 68% of general mental health providers and ~55% of family providers report using homework “often” to “almost always” (Dattilio, Kazantzis, Shinkfield, & Carr, 2011; Kazantzis, Lampropoulos, & Deane, 2005). Further, providers report using homework in an average of 57% of sessions, although this rate is higher for CBT practitioners (66%) vs. non-CBT practitioners (48%). Moreover, only 25% of providers report using expert recommended systematic procedures for recommending homework (i.e., specifying frequency, duration, and location; writing down homework assignments for patients) (Kazantzis & Deane, 1999). A national survey revealed that 93% or general mental health providers estimate rates of patient adherence to homework to be low to moderate (Kazantzis, Lampropoulos, et al., 2005), and research studies report low to moderate rates of youth/caregiver adherence during treatment (i.e., ~39–63%; (Berkovits, O’Brien, Carter, & Eyberg, 2010; Clarke et al., 1992; Danko, Brown, Van Schoick, & Budd, 2016; Dattilio et al., 2011; Gaynor, Lawrence, & Nelson-Gray, 2006; Helbig & Fehm, 2004; Lyon & Budd, 2010; Simons et al., 2012).

Numerous barriers to the successful implementation of homework during mental health treatment have largely been suggested by experts in the field, rather than specifically measured (Dattilio et al., 2011), and have generally been classified as occurring on the provider-, patient-, task-, and environmental-level (Kazantzis & Shinkfield, 2007). Provider-level barriers can relate to the therapeutic relationship and the degree to which a collaborative approach is used, provider beliefs about homework and the patient’s adherence, and providers’ ability to effectively design homework tasks (Callan et al., 2012; Coon, Rabinowitz, Thompson, & Gallagher-Thompson, 2005; Friedberg & Mcclure, 2005; Garland & Scott, 2002; Kazantzis & Shinkfield, 2007). Patient-level barriers can include patient avoidance and symptomatology, negative beliefs toward the task, not understanding the rationale or how to do the task, forgetting, and beliefs about their ability to complete homework tasks. (Bru, Solholm, & Idsoe, 2013; Callan et al., 2012; Dattilio et al., 2011; Friedberg & Mcclure, 2005; Garland & Scott, 2002; Hudson & Kendall, 2005; Kazantzis & Shinkfield, 2007; Leahy, 2002). Relatedly, core beliefs central to the patients’ psychopathology can be activated during homework–thereby triggering withdrawal and avoidance patterns (Kazantzis & Shinkfield, 2007). Task-level barriers include poor match between tasks and therapy goals, tasks that are perceived as vague or unclear, tasks that are perceived as too difficult or demanding in terms of time or effort, tasks being viewed as boring, and general aversiveness of the idea of completing homework (Bru et al., 2013; Callan et al., 2012; Dattilio et al., 2011; Friedberg & Mcclure, 2005; Garland & Scott, 2002; Hudson & Kendall, 2005). Environmental factors have been noted to include practical obstacles, lack of family/caregiver support, dysfunctional home environments, lack of time due to busy schedules, and lack of reward or reinforcement (Callan et al., 2012; Dattilio et al., 2011; Hudson & Kendall, 2005).

The advancement and ubiquitousness of technologies such as mHealth resources (e.g., mobile- and web-based apps) provide a tremendous opportunity to overcome barriers to homework use and adherence and resultantly, improve the quality of mental health treatment. mHealth solutions to improve access and quality of care, have been widely investigated, are effective in facilitating behavior change, practical, desired by patients and providers, and available at low cost (Amstadter, Broman-Fulks, Zinzow, Ruggiero, & Cercone, 2009; Boschen & Casey, 2008; Donker et al., 2013; Ehrenreich, Righter, Rocke, Dixon, & Himelhoch, 2011; Hanson et al., 2014; Heron & Smyth, 2010; Krebs & Duncan, 2015; Luxton, McCann, Bush, Mishkind, & Reger, 2011; Ruggiero, Saunders, Davidson, Cook, & Hanson, 2017). Existing mHealth resources include features that can support homework implementation (e.g., voice and SMS reminders and feedback, self-monitoring and assessment, and modules and activities that can be used to facilitate between-session practice; Bakker, Kazantzis, Rickwood, & Rickard, 2016; Tang & Kreindler, 2017), but these resources were not designed with the express intention of addressing barriers to homework implementation, particularly for youth and family patient populations.

The extant literature on barriers to homework implementation is limited in that it is largely based on expert recommendations. Therefore, the first aim of this study was to explore provider, youth, and caregiver patient perspectives on barriers to the successful implementation of homework during youth mental health treatment. Further, mHealth solutions to those barriers have not been explored, especially for youth and family patients. Thus, the second and third aims of this study were to obtain suggestions for mHealth solutions to homework barriers and explore perceptions on the benefits and challenges associated with those mHealth solutions.

Methods

Design

Institutional Review Board approval was obtained prior to enrolling any participants in the study. The approach for this study was based on the constructivist grounded theory, which acknowledges the researcher’s prior knowledge and influence in the process and supports and guides conceptual framework development to understand interrelations between constructs (Charmaz, 2006). This qualitative study used a thematic analysis of semi-structured interviews in a sample of nationally certified trainers in Trauma-Focused Cognitive Behavioral Therapy (TFCBT; Cohen, Mannarino, & Deblinger, 2017), youth who had engaged in TF-CBT, and their caregivers. The initial goal was to conduct interviews with 15–20 interviewees in each group to achieve theoretical saturation (i.e., no new information was derived), consistent with a prior study by members of the research team which used similar semi-structured interviews with national TF-CBT trainers (Hanson et al., 2014), and recommendations by Morse (2000) given the relatively narrow scope and clear nature of the study. Interviews were conducted until interviewers and the study lead determined that no new pertinent information was being obtained.

Participants

National Trainers.

Twenty-one national trainers in TF-CBT were interviewed. National trainers are mental health providers who completed a 15-month TF-CBT Train-the-Trainer program led by the TF-CBT developers. Trainers work extensively with numerous community mental health providers to problem-solve common barriers to clinical practice and thus, provide a unique perspective on the barriers to successful homework implementation and possible mHealth solutions to those barriers. An e-mail invitation was sent to a list of approved TF-CBT trainers. Twenty-four trainers responded to this e-mail, 22 of whom agreed to participate in an interview, one of whom was unreachable after initial scheduling. Interviews were completed with a total of 21 trainers, who received a $25 gift card in compensation for their time.

Trainers had been treating children for an average of 23.29 years (SD=8.80) and had been training providers for an average of 14.95 years (SD=8.98). In the year prior to the interview, they led an average of 17 provider trainings (SD=21.67) and trained roughly 345 providers (SD=339.90). All trainers were licensed, and the majority were Clinical Psychologists (47.6%) and Social Workers (33.3%). The average age of trainers was 47.48 years (SD=13.63) and the majority were female (71.4%), white (95.2%), and non-Hispanic/Latino (85.7%; see Table 1).

Table 1.

Trainer Demographics

Variable	M	SD
Age	47.48	13.63
Years Treating Children	23.29	8.80
Years Training Providers	14.95	8.98
Workshops in Past Year	17.00	21.67
Providers Trained in Past Year	345.52	339.90
Estimated % of Providers Experiencing Difficulty with Homework	76.40%	17.10%
	n	%
Sex
Female	15	71.4
Male	6	28.6
Race
White	20	95.2
Asian	1	4.8
Ethnicity
Non-Hispanic/Latino	18	85.7
Hispanic/Latino	3	14.3
Discipline
Physician (MD)	1	4.8
Clinical Psychologist (PhD, PsyD)	10	47.6
Counselor (LPC)	1	4.8
Social Worker (LCSW, MSW)	7	33.3
Other	2	9.5
Currently Licensed	21	100.0

Families.

Twelve families were interviewed for this study. Families were included if they had one or more youth between the ages of 8 and 17 years-of-age and a caregiver who had engaged in at least four sessions for TF-CBT. These criteria were chosen because TF-CBT is typically recommended for youth between the ages of 8 and 17 years-of-age and it was estimated that four sessions would have likely allowed for adequate time for patients to have received homework assignments, consistent with the authors’ experience and prior TF-CBT literature (Deblinger, Pollio, & Dorsey, 2016; Scheeringa, Weems, Cohen, Amaya-Jackson, & Guthrie, 2011). Families were recruited via advertisements online and at local community mental health clinics, and from a participant pool from a prior study (Davidson et al., 2019). Twenty-nine families initially expressed interest in participating in the study. Six families were ineligible because they had not received TF-CBT and contact was lost with six families after their initial contact. Seventeen families were scheduled for an interview, five of which were unreachable after initially being scheduled, and interviews were completed with 12 families. Written informed consent from caregivers and assent from youth above the age of 15 were obtained in-person for four families and via a telemedicine-based teleconsent platform (i.e., https://musc.doxy.me) for eight families. Families received a $30 gift card in compensation for their time.

A total of 15 youth who had engaged in TF-CBT, and their caregivers (n=12; three families had two youth who had received treatment) were interviewed. Six youth were still in treatment at the time of their interview and nine had finished treatment an average of 49 weeks (SD=42.32) prior to the interview. The average age of youth was 13.20 years (SD=3.19), roughly half were female (53.3%), the majority were white (80%), and all were non-Hispanic/Latino. The average age of caregivers was 44.83 years (SD=7.90), 66.7% were female, and all were White and non-Hispanic/Latino. Youth and caregivers rated their comfort with technology, in general, on a 10-point Likert scale (i.e., 1–10) with higher scores representing higher levels of comfort. Youth reported being very comfortable with technology (M=9.62, SD=1.12), as did their caregivers (M=7.83, SD=2.63; see Table 2).

Table 2.

Family Demographics

Variable	Youth n=15		Caregivers n=12
	M	SD	M	SD
Age	13.20	3.19	44.3	7.90
Time Since Ending Treatment in Weeks*	49.00	42.32	-	-
Comfort with Technology	9.62	1.12	7.83	2.63
	n	%	n	%
Sex
Female	8	53.3	8	66.7
Male	7	46.7	4	33.3
Race
White	12	80.0	12	100
Black/African American	2	13.3	0	0.0
Native American/American Indian	1	6.7	0	0.0
Ethnicity
Non-Hispanic/Latino	15	100	12	100
Hispanic/Latino	0	0.0	0	0.0
Education
High School	-	-	1	8.3
High School	-	-	4	33.3
Some College	-	-	5	41.7
College Degree	-	-	3	25.0

Note.

^*= 6 families were still in treatment at the time of their interview and were not included in this average.

Trauma-Focused Cognitive Behavioral Therapy

TF-CBT is a well-established and widely disseminated mental health treatment (Cohen et al., 2017; Deblinger, Mannarino, Cohen, Runyon, & Steer, 2011; Silverman et al., 2008; Wethington et al., 2008). It is a conjoint youth-caregiver mental health treatment typically conducted over ~12, 90-minute sessions that address nine major treatment components (i.e., Psychoeducation; Parenting Skills; Relaxation Skills; Affective Expression and Modulation Skills; Cognitive Coping and Processing Skills; Trauma Narration and Processing; In Vivo Exposure; Conjoint Child Parent Activities; and Enhancing Future Safety and Development). TF-CBT also addresses a broad range of symptom domains including trauma- and stress-related disorders, disruptive behavior disorders/behaviors, depression/depressive symptoms, and anxiety disorders (Cohen et al., 2017). TF-CBT was chosen as a model treatment for this study because of its broad symptom focus, inclusion of treatment components used in a variety of youth mental health treatments, and involvement of youth and their caregivers, offering potential to improve the applicability of the study’s results to a range of youth mental health treatment approaches.

Procedures for Data Collection

Interviews were conducted via telephone for trainers, and either in-person or via telephone for families based on their preference. A postdoctoral fellow and masters-level research assistant conducted the interviews, which were audio-recorded and transcribed using a professional transcription service. Interviews included three major components. The first component included demographic questions. The second included a brief orientation to the goal of the study, which was to develop a new technology-based resource to help providers and patients during the implementation of homework during mental health treatment. The third component included questions that aimed to assess perspectives on barriers to homework implementation, elicit suggestions for mHealth solutions to those barriers, and examine perceptions on the benefits and challenges associated with mHealth solutions to homework barriers. The average duration of interviews was 41 minutes for trainers and 37 minutes for families. See Supplementary Materials for complete interviews.

Data Analysis

Transcribed interviews were coded using NVivo qualitative analysis software. NVivo was used to identify common themes (nodes) as they related to (1) patient-, provider-, task-, and environmental-barriers to homework implementation, (2) suggestions for mHealth solutions to homework barriers, and (3) benefits and challenges associated with mHealth homework solutions. Initial and secondary coding passes were conducted to identify and refine theme classifications as they emerged and impose a data-derived hierarchy to the nodes identified. Focused coding was used to refine the coding and ensure that data were coded completely with minimal redundancy (Miles & Huberman, 1994). Themes were initially proposed by the first author and reviewed by an expert in qualitative and mixed methods research (the second author) and an internationally recognized expert in the implementation of homework and related barriers during CBT (the fourth author). Divergent perspectives on theme descriptions (n=2) and classifications (n=1) were compared until agreement was reached.

Results

Results are organized by the main topics explored in this study, including: 1) barriers to the successful implementation of homework, coded on provider, patient, task, and environmental levels; 2) potential mHealth solutions to those homework barriers; and 3) perceived benefits and challenges of those potential mHealth solutions. Results within each of these topics are presented first from the perspectives of trainers and second from the perspectives of families.

Barriers to the Successful Implementation of Homework

Trainer Perspectives.

As displayed in Table 3, trainers identified several barriers to homework implementation on the provider-, patient-, task-, and environmental-level.

Table 3.

Trainer Perspectives on Homework Barriers

Levels	Themes	No. of Trainers Raising Theme	No. of References to Theme
Provider
	Difficulty engaging patients and or discouraged by low engagement	15	23
	Don’t see homework as an integral part of therapy or important	14	17
	Don’t know what to assign	12	14
	Forget	11	13
	Too busy or lack of time	11	18
	Don’t know how to effectively assess and assign homework	11	14
	Don’t effectively assess patient barriers	10	11
	Difficulty individualizing homework to specific patients	10	13
	Difficulty with consistency	9	10
	Assignments are too difficult or overwhelming	6	9
	Don’t want to distress or put too many demands on the patient	6	7
	Difficulty planning ahead for homework	5	6
	Lack resources	4	5
Patient
	Don’t see homework as an integral part of therapy or important	15	19
	Forget	5	6
	Don’t understand the rationale	5	5
	Patient avoidance or symptoms	4	5
	Don’t understand practical implementation (i.e., when, where, and how)	4	4
Task
	Homework is an aversive term	7	7
	Assignment does not align with patient values or treatment goals	7	12
Environment
	Home life is busy and chaotic; no time	15	17
	Lack of caregiver involvement	13	18
	Lack of reward or reinforcement	7	10

Provider-Level Barriers.

Many trainers felt that providers tend to have difficulty engaging patients in assigned tasks, leading some providers to become discouraged by low levels of engagement. As stated by one trainer,

“I think they recognize that [homework assignments] do have value, but in terms of what I feel, a lot of clinicians are not having success with families completing homework, so it’s diminishing the sense of value…something they’ve tried to put into place and they are not feeling there’s any success in it.”

Trainers also noted that many providers do not see homework as an integral part of therapy. One trainer commented,

“I think there are a lot of concrete barriers, but to me probably the biggest barrier will be the–I think that still to this day [providers] like to think that therapy happens in that one hour.”

Other interrelated difficulties faced by providers related to their capacity to effectively and consistently develop, assess, and assign meaningful and patient-centered homework exercises.

As stated by one trainer,

“I see a lot of that just shooting from the hip, kind of off the cuff, ‘let’s do this,’ but yet, it’s not backed by anything concrete or tangible…I think probably one of the biggest pieces again is the failure on the clinician’s part to follow that up and too often review it at the end of the session.”

Another said,

“I think clinicians don’t always appreciate how hard it is to actually do homework that requires you to make some behavioral change.”

Barriers also related to providers’ time and resources for implementing homework, as conveyed by one trainer’s comment,

“I mean, these people…every minute of every day is filled up with doing, billing, writing, charting, going to meetings, getting supervision, and seeing patients, and then they go home exhausted.”

Patient-Level Barriers.

Many trainers stated that, similar to some providers, patients often do not see homework as an important part of therapy. Put by one trainer,

“I think that some [patients] just feel that coming to the session is enough and that should resolve everything, and that you know, doing homework is just kind of an extra thing…I don’t really need to do it to benefit from the therapy.”

Perhaps relatedly, trainers also noted that patients generally forget to do homework assignments, and often forget why, how, when, and where assignments should be done.

Task-Level Barriers.

Task-level barriers noted by trainers included assignments not always aligning with patient values or treatment goals and that the term ‘homework’ being aversive to patients of all ages. One trainer commented,

“I think it has to be something that [patients] see the value in. And again, we go back to that engagement and them trusting you as well as you explaining to them why this could be helpful…If it didn’t help, we need to change it.”

Another trainer laughed while stating,

“when we use the word homework, we might as well just throw a stink bomb in the room.”

Environmental-Level Barriers.

Finally, on the environmental-level, many trainers suggested that patients’ home lives are busy and chaotic, leaving little-to-no time for homework.

Explained by one trainer,

“I think that for parents…they have many other things in their life; work, parenting, partnerships that they are working on, just day to day chores or things that they have to do in terms of their family or other responsibilities. So, [homework] often feels like, I think for families, to add another thing…it just feels like a lot.”

Associated barriers included limited caregiver involvement and reinforcement for completing homework assignments. One trainer commented,

“So, let’s not forget that the parents need to be encouraged and checked on to make sure the kid is doing it. They have to work at it – It’s not going to just happen. So, helping the parents to see that they’re going to need to work to make sure the kids do it, because again, the kids would rather eat ice-cream than do the work. I mean change is hard.”

Another stated,

“I would say, lack of reinforcement for homework, so maybe for getting what you assign for homework and not reviewing it or the kiddo or the family learning pretty quickly, you know, why do it, because there’s not a lot of support around it. You know, if [patients] don’t get reinforced, whether tangibly or verbally, they may not continue that.”

Family Perspectives.

Families identified several barriers to homework implementation on the patient-, task-, and environmental-level which were similar to many of those noted by national trainers (see Table 4).

Table 4.

Family Perspectives on Homework Barriers

Levels	Themes	No. of Families Raising Theme	No. of References to Theme
Patient
	Patient avoidance or symptoms	10	23
	Forget	9	14
	Don’t understand practical implementation (i.e., when, where, and how)	8	12
	Don’t understand the rationale	8	13
	Don’t see homework as an integral part of therapy or important	5	5
	Disinterested or don’t care	4	5
Task
	Assignment is viewed as boring	10	12
	Assignment does not align with patient values or treatment goals	3	3
	Paperwork is inconvenient	2	2
Environment
	Home life is busy and chaotic; no time	12	21
	Lack of reward or reinforcement	7	7
	Lack of caregiver involvement	2	2
	Lack of provider enthusiasm	2	2

Patient-Level Barriers.

Families believed that patients often avoid homework as a result of their symptoms. In other words, the patient’s unhelpful coping strategies are being triggered.

One caregiver commented,

“Sometimes people don’t even want to dig into their feelings even to do the assignment either, you know. It stirs up things. You know, when you’re dealing with feelings, sometimes you don’t want to experience that feeling…you shut down. You don’t want to feel that at that time.”

Another stated,

“When you already have a child that has ADHD or behavior problems, it’s hard to get them motivated and to get them to do these exercises at home.”

Families also felt that patients simply forget to complete homework or bring it to their next session. One child stated,

“That’s my problem, she’ll give me homework, we met once a week, basically, and I would forget it because I’ve got a lot going on, and when I come in and she’s like, ‘Did you do your homework,’ I’m like, ‘Oh man’.”

Similar to trainers, families felt that patients often forget why, how, when and where assignments should be done. As stated by one caregiver,

“I think sometimes it can also be just, like maybe not fully understanding what is being asked of them to do. I know the therapist will ask them in the office, ‘do you understand?’ and of course the kids always go, ‘yes I do, can I go home now’?”

Task-Level Barriers.

With respect to task-level barriers, most families viewed homework assignments as boring. General consensus from families was that patients–particularly youth– would more often than not just rather be doing something more interesting.

Environmental-Level Barriers.

On the environmental level, all families noted that the home-life of patients is busy and chaotic, leaving little perceived time for homework. Everyday responsibilities such as schoolwork, employment, household chores, and familial responsibilities often take precedence. One caregiver stated,

“Well I think it sounds good in the office and then you get home and you just get quite busy and it gets pushed aside.”

Another commented,

“But I know what he’s saying…sometimes seven-and-a-half hours at school and then sometimes his therapy would be an hour-and-a-half. And thank goodness, his teacher was so flexible that on days he has therapy he did not have homework [for school], but he was just so emotionally and physically drained. When he got home, all he wanted to do was just rest or play. Because that’s the therapy, it can be just exhausting.”

Families also believed that that there is often a lack of reinforcement for completing homework assignments.

mHealth Solutions to Homework Barriers

Trainer Suggestions.

Trainers provided several suggestions for mHealth solutions to homework barriers (Table 5). Most trainers felt that reminders and schedules to help patients remember to complete homework assignments would be a crucial feature. One trainer suggested, “Maybe some kind of reminder feature, something that would kind of record into their daily calendars that they use, or an alarm, or something like a daily reminder…set to the times they are most likely to do the homework.”

Table 5.

Trainer Suggestions for mHealth Solutions to Homework Barriers

Themes	No. of Trainers Raising Theme	No. of References to Theme
Reminders and schedules for patients	16	26
Reports or activity summaries	16	24
Behavior and symptom tracking	13	21
Interactive and fun activities	13	15
Include a variety of homework activities to choose from	12	14
Easy to use and easy navigation to relevant resources	11	19
Resources for caregivers and caregiver engagement	11	20
Patient centered and developmentally appropriate	10	17
Reward system	10	22
Reminders and schedules for providers	7	9
Clear instructions on how to do assignments	4	6
Information on rationale for homework	2	3

Trainers also suggested including reports or activity summaries of homework completion along with behavior and symptom tracking tools. One trainer thoughtfully commented, “If the homework app can somehow help to provide some data on the actual implementation of certain skills during the week that would be very valuable because I think the constructive feedback and the positive feedback that’s offered by therapists about performance of those skills between sessions can be really valuable.”

Trainers suggested including a variety of interactive, fun, and rewarding activities that engage children and caregivers. For example, one trainer stated,

“I think the more interactive you can make it between parent and child and the more of a game you can make it…kids are more likely to do that and to kind of use those skills.” All trainers (n=21) felt that a text message-based system for reminding patients to complete homework assignments would be beneficial.

Family Suggestions.

Families suggested that the main function of the resource should serve to provide reminders (e.g., text messages or push notifications) for patients to complete homework assignments as well as instructions for how and when they should be completed. Another common suggestion was to include a reward system within the resource to reinforce engagement with homework assignments. Some suggestions for this reward system included coins, experience points, levels, and customizable avatar characters. One child thoughtfully related,

“there could be a digital reward system like stars or gems or something. Then it could be redeemed or something in the therapist’s office. Like I remember it was a while ago, I remember my therapist said if I was able to do something that I was having trouble with, we would have like brownies or something the next visit.”

Families also recommended that the resource include interactive and fun activities. The most common suggestion was to “gamify” homework assignments to make them more fun and interesting to patients. For example, a caregiver noted,

“I think that if you are able to play a game or level up after you did your activity…I don’t think you would have a problem with them doing the activity. They would be so excited to be able to play the game.”

Families providers also recommended reports and activity summaries so that progress could be tracked and reported to providers to be reviewed during the next treatment session (Table 6). All families (n=12) felt that a text message-based system for reminding patients to complete homework assignments would be beneficial.

Table 6.

Family Suggestions for mHealth Solutions to Homework Barriers

Themes	No. of Families Raising Theme	No. of References to Theme
Built-in reward system	12	53
Instructions via video, text, audio	12	28
Reminders and schedules	12	24
Interactive and fun activities	10	18
Reports or activity summaries	9	20
Colorful	7	12
Resources for caregivers and caregiver engagement	4	5
Access without internet	4	5

Benefits and Challenges of mHealth Solutions to Homework Barriers

Trainer Perspectives.

The majority of trainers responded that an mHealth solution to homework barriers would increase provider use of (n=20; 95.2%) and family adherence to (n=21; 100%) homework during mental health treatment. The majority of trainers also responded that such a resource would positively affect the therapeutic relationship (n=15; 71.4%), increase treatment efficiency (n=18; 85.7%), and improve treatment effectiveness (n=18; 85.7%). Neutral responses were provided by all trainers who did not respond affirmatively to these questions (i.e., no negative responses were provided). Trainers also commented on the potential clinical utility of an mHealth homework resource in that it would help providers with tracking and assigning homework and patients with skill development while promoting high levels of engagement in youth patients. Access, comfort with technology, and convenience were also noted benefits (See Table 7). One trainer commented,

Table 7.

Trainer Perspectives on Benefits and Challenges relating to mHealth Solutions to Homework Barriers No. of Trainers

Theme	Sub-themes	No. of Trainers Raising Theme	No. of References to Theme
Benefits
Clinical utility		13	21
	Youth like technology and would engage with it	9	11
	Would help with keeping track of homework	3	4
	Will help youth develop skills and maintain treatment gains	3	4
	Will help providers with developing and assigning homework	2	2
Access, comfort, convenience		9	14
	Most people have mobile phones providing easy access	6	7
	Some providers are really good with technology	5	5
	Having exercises on the phone would be convenient	2	2
Challenges
Confidentiality issues		13	15
Access, comfort		12	26
	Some families may not have access to the technology	8	9
	Youth access to device might be restricted	4	6
	Providers might not have access to the technology	3	3
	Some families may not have internet access	3	3
	Some providers are not good with technology	3	3
	Some caregivers might not feel comfortable with it	2	2
Negative impacts on therapy		10	11
	Might promote social isolation	4	4
	It may add to the provider's administrative load	3	3
	Providers will still need to use clinical judgement	3	3
	Exercises might not fit with provider preferences	1	1

“I feel like so many people now enjoy so much more doing things on electronics and so definitely in sessions with kids I’m often recommending having a clinician use apps…sometimes technology is the way to really hook families in and engage them.”

Another stated,

“You know everybody has a phone and if we can have some apps where…I mean it’s so exciting to me what you are talking about. I can’t think of a better idea, I really can’t. I mean people always have their phones on them even if you are really, really poor, people tend to have a phone.”

Challenges identified by trainers centered around confidentiality, access and comfort with technology, and potential negative impacts on the therapeutic process. For example, one trainer stated,

“I do not know if people worry about if somebody else saw the app and wondered, ‘oh you are in therapy, oh what happened to you?’ So, some things around privacy issues and confidentiality, but those will be pretty easy to fix.”

Family Perspectives.

The majority of families believed that the an mHealth homework resource would make practicing therapy skills at home more fun or interesting (n=11; 91.7%), would help families practice skills more often (n=12; 100%), would positively affect the therapeutic relationship (n=12; 100%), and would improve treatment effectiveness (n=11; 91.7%). Neutral responses were provided by all families who did not respond affirmatively to these questions (i.e., no negative responses were provided). Families also suggested that an mHealth homework resource would have excellent clinical utility, helping to improve communication between providers and families, make treatment and homework more rewarding, encourage more engagement from youth One caregiver commented,

“I think it would encourage the kids to get [homework] done even before the parents. The kids would want to do it on the phone, they love messing with phones.”

Another stated,

“I think by having the reminders, as well as having something there that’s interactive for the kids and the caregivers both. I think it would be a huge help.”

Similar to trainers, challenges noted by families related to confidentiality and some families not having access to the technology or the internet. Additional family perspectives on benefits and challenges are provided in Table 8.

Table 8.

Family Perspectives on Benefits and Challenges relating to mHealth Solutions to Homework Barriers

Themes	Sub-themes	No. of Families Raising Theme	No. of References to Theme
Benefits
Clinical utility		11	60
	Will lead to better communication between providers and families	10	21
	Would help make treatment and homework more rewarding	7	9
	Youth like technology and would engage with it	7	7
	Would help families remember to do assignments	5	7
	Would help reinforce skills learned in therapy	4	6
	Could help to bring families together	4	4
	Would help treatment go faster	4	5
	Would help families remember why homework is beneficial	1	1
Challenges
Confidentiality issues		11	20
Access, comfort		9	15
	Some families may not have access to the technology	6	6
	Some families may not have internet access	4	4
	Some families aren’t good with technology	2	2
	Youth access to device might be restricted	2	2
	Some families might have concerns about data or storage space	1	1

Discussion

The aims of this study were to assess barriers to the successful implementation of homework during youth mental health treatment, obtain suggestions for mHealth solutions to those barriers, and explore perceptions on the benefits and challenges associated with mHealth solutions to homework barriers through semi-structured qualitative interviews with relevant stakeholders. National trainers in TF-CBT provided a unique perspective on the common challenges met by mental health providers and their patients as well as potential solutions to those challenges, particularly given their extensive experience problem-solving common clinical challenges with community mental health providers. Interviews with youth TF-CBT patients and their caregivers provided important perspectives from those most affected by homework barriers in mental health treatment.

Perspectives on Barriers to the Successful Implementation of Homework

Trainer and family perspectives on the various barriers to the successful implementation of homework during mental health treatment aligned well with the heuristic proposed by Kazantzis and Shinkfield (2007), which classifies barriers as occurring on the provider-, patient-, task-, and environmental-levels. Most of the provider-level barriers noted by trainers were consistent with expert recommendations from the research literature, such as providers’ beliefs relating to homework and patient engagement in homework (Coon et al., 2005; Friedberg & Mcclure, 2005; Garland & Scott, 2002), difficulty designing homework activities and individualizing them to specific patients (Kazantzis & Shinkfield, 2007), forgetting about homework and running out of time during the session (Friedberg & Mcclure, 2005), difficulty with consistency and not wanting to put too many demands on patients (Coon et al., 2005), and difficulty effectively assessing patient barriers (Kazantzis & Shinkfield, 2007). Experts have proposed a model for practice that directly addresses many of these provider-level barriers by proposing an ideal process for facilitating engagement in homework (Kazantzis, MacEwan, & Dattilio, 2005).

Trainer and family perspectives on the most common patient-level homework barriers were similar and were also consistent with the extant literature. These included patients’ avoidance or symptoms (Coon et al., 2005; Dattilio et al., 2011; Friedberg & Mcclure, 2005; Garland & Scott, 2002; Hudson & Kendall, 2005; Leahy, 2002), forgetting to complete assignments (Coon et al., 2005; Hudson & Kendall, 2005), not understanding when, where, or how to do assignments or the rationale (Dattilio et al., 2011; Friedberg & Mcclure, 2005; Garland & Scott, 2002), and beliefs about homework tasks and their ability to complete them (Dattilio et al., 2011; Kazantzis & Shinkfield, 2007). Interestingly, whereas the most commonly endorsed patient-level barrier by trainers was patients not seeing homework as an integral part of therapy or important, the most commonly endorsed barriers by families included avoidance or symptoms, forgetfulness, and lack of understanding about assignments, reflecting differing views on the more significant barriers faced by patients. This discrepancy in the trainers/providers vs. families’ perspectives regarding between session assignments suggests the importance of therapists’ focusing more time on explaining assignments, discussing potential challenges, emphasizing the benefits of completing assignments in overcoming symptoms/difficulties and ultimately inspiring follow through.

Task-level barriers reported by both trainers and families included assignments not aligning with patient values or treatment goals (Coon et al., 2005; Dattilio et al., 2011; Hudson & Kendall, 2005). Many trainers reported that the word “homework” is an aversive term to patients, particularly to youth patients. Perhaps relatedly, many families reported that children view homework assignments are boring. Negative associations with homework may be addressed by referring to “homework” as practice assignments, experiments, exercises, or action plans, as recommended by a recent Beck Institute blog post by Drs. Judith Beck and Francine Broder (Beck & Broder, 2016).

Finally, environment-level barriers noted by trainers and families included the home lives of patients being busy and chaotic – leaving little time to complete homework assignments; a lack of caregiver involvement in the case of youth; and a lack of reward or reinforcement for completing homework assignments, all of which have been previously noted (Bru et al., 2013; Coon et al., 2005; Dattilio et al., 2011; Kazantzis & Shinkfield, 2007). In sum, trainer and family perspectives on barriers to the successful implementation of homework were largely consistent with those suggested by experts. Further, there was a general agreement between trainers and families with respect to those barriers. It is important to note the interrelatedness of several barriers within various levels. For example, patients not understanding the importance of homework or seeing it as an integral part of therapy could very much reflect a mismatch in alliance, tasks needed to achieve therapy goals, or a poor therapist rationale and opportunity for client feedback and discussion. Further, a patient’s understanding of the rationale for homework might be dependent on the provider’s skill in its explanation.

mHealth Solutions to Homework Barriers

Trainers and families provided numerous suggestions for mHealth solutions to homework barriers. These functionality and content suggestions included: reminders and schedules to overcome barriers to forgetting; behavior and symptom tracking and reports or activity summaries to assist providers in assessing homework completion; a variety of homework activities to choose from to help providers struggling with developing activities; resources for caregivers to improve caregiver support; and an integrated reward system to make completing homework rewarding and reinforcing for patients. Other suggested features related more to user interface and user experience. For example, interviewees felt that the mHealth resource should allow easy navigation to relevant resources; include clear instructions via video, text, and audio to help patients understand and remember how to do assignments; include interactive and fun activities to help make the assignments less boring and less like “homework;” and be patient-centered and developmentally appropriate. Trainers and families also felt that a text message-based system for reminding patients to complete homework assignments would be beneficial, indicating that this approach would provide a good alternative to a purely app-based resource.

As outlined in recent reviews, there are several studies on mHealth resources that include the functionality and content features suggested in this study and can also be used to facilitate homework implementation (Bakker et al., 2016; Tang & Kreindler, 2017). For example, a number of mHealth resources can be used for self-monitoring and symptom tracking, and many have engaging activities that can be used to support between-session learning and skill development in the areas of relaxation, cognitive therapy, imaginal exposure, and parent behavioral management (Bunnell et al., 2019; Jungbluth & Shirk, 2013; Kristjánsdóttir et al., 2013; Newman, Przeworski, Consoli, & Barr Taylor, 2014; Reger et al., 2013; Shapiro et al., 2010; Whiteside, Ale, Vickers Douglas, Tiede, & Dammann, 2014). SMS- and app-based reminders and feedback on progress can also be used to encourage continued engagement in skills practice (Aguilera & Muñoz, 2011; Harrison et al., 2011; Reger et al., 2013; Wiederhold, Boyd, Sulea, Gaggioli, & Riva, 2014). However, as stated previously, most of these resources were not designed with the express intention of addressing barriers to homework implementation, particularly for youth and family patient populations, leaving room for future work in this area.

Benefits and Challenges of mHealth Solutions to Homework Barriers

Trainers and families expressed very positive views on mHealth solutions to homework barriers. Trainers felt that mHealth would increase provider use and family adherence to homework, positively affect the therapeutic relationship, and increase treatment efficiency and effectiveness. Families felt that it would make practicing therapy skills at home more fun or interesting, help families practice skills more often, positively affect the therapeutic relationship, and improve treatment effectiveness. A potential benefit commonly noted by trainers and families was a high likelihood that youth would engage with the resource given their generally strong interest in technology, and that this would help to reinforce the practice of skills learned during therapy. A particular benefit noted was increased access to helpful resources between-sessions. Trainers and families expressed concerns about issues relating to confidentiality. While they did not view this as a fatal flaw of the resource, they suggested implementing appropriate safeguards to protect patient privacy and clearly explaining data protection to encourage use.

Limitations

There are several limitations to this study. Regarding generalizability of results, the selection of trainers and families interviewed was based on experience with TF-CBT, a specific treatment protocol for childhood trauma. Although interview questions were kept general during interviews, referring to mental health treatment rather than solely to TF-CBT, the views expressed by interviewees may relate more to TF-CBT than other child mental health treatments. However, a strength of this research is that TF-CBT has a broad symptom focus (e.g., PTSD, anxiety, depression, anger, disruptive behavior) and includes treatment components used in numerous youth mental health treatments (e.g., psychoeducation, relaxation, cognitive coping, affective modulation, exposure), which suggests that results would be applicable to a range of child mental health treatments. Additionally, national trainers in TF-CBT have consistent exposure to working closely with community mental health providers and regularly help them to problem-solve common barriers in clinical practice. This added insight into difficulties experienced by numerous mental health providers rather than asking individual providers about their experience. This is a strength of this study but also a potential limitation as not directly measured, thus an assumption. The views of trainers may not be completely representative of the every-day challenges to homework implementation experienced by community mental health providers. Given the small samples size and lack of diversity, the results should be interpreted with caution as they may not reflect the experiences or views of therapists and patients who utilize homework across different treatment approaches, therapy settings, and populations.

With respect to interview questions and results, they tended to focus on barriers and challenges and provided less of an opportunity for trainers and family members to share factors that may have led to successes with homework assignments. Such information could also importantly support the development and presentation of mhealth solutions by therapists. Relatedly, families were asked about barriers faced by youth and caregivers, and not by providers, which would have provided interesting data on family perspectives on providers’ limitations. Although comfort with technology in general was assessed in youth and caregivers, it was not specified as comfort with mHealth, and ratings were not collected from trainers. As such, a potential limitation of this study is that participants’ comfort specifically with mHealth was unknown. Furthermore, this study focused specifically on mHealth without a comparison to other low-tech solutions, which might have resulted in inflated levels of interest in mhealth solutions to homework barriers. A final limitation is that interviews were coded by the first author, and there is potential for variability in coding that was not accounted for (i.e., the same themes might have been classified in different ways). Despite this limitation, themes were reviewed and by an internationally recognized expert in the implementation of homework and related barriers during CBT (the fourth author) and compared until agreement was reached, supporting the derived themes.

Conclusions

This study provides important new information on barriers to the successful implementation of homework during youth mental health treatment, based on perspectives of providers, youth, and caregivers with that treatment experience. This study adds to the literature on these barriers, which has been based largely on recommendations from experts in the field. The results of this study aligned well with this literature, providing additional support for these recommendations. Valuable insights on potential mHealth solutions to these homework barriers were also provided. These data are being used to inform the development of an mHealth resource that aims to address homework barriers in hopes of improving provider use and patient adherence to homework during youth mental health treatment, with the ultimate goal of improving the quality of care received by patients in community mental health settings.