Caregiver Use of the Core Components of Technology-Enhanced Helping the Noncompliant Child: A Case Series Analysis of Low-Income Families

Abstract

Children from low-income families are more likely to develop early-onset disruptive behavior disorders (DBDs) compared to their higher income counterparts. Low-income families of children with early-onset DBDs, however, are less likely to engage in the standard-of-care treatment, behavioral parent training (BPT), than families from other sociodemographic groups. Preliminary between-group findings suggested technology-enhanced BPT was associated with increased engagement and boosted treatment outcomes for low-income families relative to standard BPT. The current study used a case series design to take this research a step further by examining whether there was variability in use of, and reactions to, the smartphone enhancements within technology-enhanced BPT and the extent to which this variability paralleled treatment outcome. Findings provide a window into the uptake and use of technology-enhanced service delivery methods among low-income families, with implications for the broader field of children’s mental health.

There is growing interest in the role of technological innovations to better meet the needs of mental health consumers (e.g., Aguilera & Muench, 2012; Jones, 2014; Kazdin & Blasé, 2011). Enthusiasm regarding the role of technology is multifaceted, but stems largely from its promise to increase engagement and adherence by facilitating efficient communication between clinicians and clients, as well as access to resources to generalize the content of sessions to the client’s daily life (see Aguilera & Muench, 2012; Enock & McNally, 2013; Jones et al., 2013, for reviews). In turn, preliminary findings suggest the promise of technology to increase the reach and impact of evidence-based treatments (e.g., Comer et al., 2014; Duncan, Velasquez, & Nelson, 2014; Jones et al., 2014); however, as efficacy research continues to evolve, new questions emerge regarding the real-world acceptability and sustainability of technology-delivered or enhanced service delivery methods (e.g., Nelson, Bui, and Velasquez, 2011; Ritterband et al., 2003; Wu, Steele, Connelly, Palermo, & Ritterband, 2014).

In part, answers regarding acceptability and sustainability depend on supplementing randomized controlled trial (RCT) between-group designs (i.e., standard of care vs. technology-enhanced arms) by elucidating the extent to which levels of use within technology-enhanced treatments appear to correspond with variability in treatment outcomes (e.g., Ritterband et al., 2003; Waller & Gilbody, 2009). Such within-group research responds to calls to maximize knowledge generation from the relatively costly research and development in technology-enhanced services research (Riley et al., 2011; Rothwell, 2005; Wu et al., 2006) and further builds upon the long-standing tradition of case series designs in cognitive behavioral and pilot services research (e.g., Coughtrey, Shafran, Lee, & Rachman, 2013; Daughters, Magidson, Schuster, & Safren, 2010; Norberg, Perry, Mackenzie, & Copeland, 2014). This study aims to investigate levels of and attitudes toward technology use, as well as links between use, attitudes, and outcomes among caregivers randomized to one technology-enhanced intervention, Technology-Enhanced Helping the Noncompliant Child (TE-HNC; Jones et al., 2013). Findings from this study have the potential to provide insight into the sustainability and acceptability of technology-enhanced interventions in real-world clinic settings where average differences between groups in RCTs may tell us relatively little about how individual clients interface with technology. Preliminary findings from the pilot RCT comparing TE-HNC to the standard of care behavioral parent training (BPT) program upon which it was based, Helping the Noncompliant Child (HNC; McMahon & Forehand, 2003), suggested the promise of cost-effectively improving engagement and boosting treatment outcomes among low-income families of youth with disruptive behavior disorders (DBDs). The TE-HNC intervention used a range of proof-of-concept smartphone components to enhance connection with and support to low-income families between clinic-based sessions. Building upon this foundational work, we believe TE-HNC provides an ideal exemplar for within-group research on technology-enhanced services for several reasons.

First, DBDs, characterized by noncompliance, aggression, and defiance, are the second most common (19.1%) reason for child referral to mental health services, and low-income youth are more likely to be referred than their higher-income counterparts (Heiervang et al., 2007; Merikangas et al., 2010; Merikangas, Nakamura, & Kessler, 2009; also see Forehand, Jones, & Parent, 2013, for a review). Second, the standard-of-care for early onset DBDs, BPT (also called Parent Management Training, PMT), includes a constellation of programs with common history, theory, and treatment techniques firmly rooted in the core tenets of behaviorism, including skill modeling, skill practice in and out of session, and tailored feedback (Abraham & Michie, 2008; Jones et al., 2013; Patterson, 2005; Reitman & McMahon, 2013). Therefore, research on one technology-enhanced BPT program should generalize to other BPT programs and other behaviorally oriented treatments for children as well.

Third, BPT, like other evidence-based treatments, tends to be underutilized by families who may benefit the most from intervention, including low-income families (Eyberg, Nelson, & Boggs, 2008; Gardner et al. 2009; Jones et al., 2013; Shaw, 2013). Barriers to engagement in BPT are varied and complex, but include acute and chronic socioeconomic-related stressors that make navigating time-intensive and demanding clinic-based BPT services (i.e., 12 to 28 session hours, midweek telephone check-ins, daily home practice of skills) more challenging (see Eyberg et al., 2008; McMahon & Forehand, 2003; Reyno & McGrath, 2006; Thomas & Zimmer-Gembeck, 2007, for reviews). Although the digital divide certainly merits some caution in technology-enhanced services research, low-income homes are more likely than other sociodemographic groups to rely entirely on smartphones in particular, given the diverse and relatively cost-effective functionalities bundled into one increasingly affordable platform and service plan options (see Anderson & Subramanyam, 2011; Davies, 2011; Snider, 2011, for reviews).

Accordingly, with the aim of enhancing our understanding of the potential uptake and sustainability of technology-enhanced service delivery models, the current study builds on previously reported findings comparing TE-HNC and HNC by using a case series design. Specifically, we examine caregiver variability in use of and attitudes toward the smartphone components within caregivers randomized to TE-HNC, as well as the extent to which this variability corresponds with variability in treatment outcome. Of note, the literature on uptake of treatment, including technology-enhanced treatment, suggests that client-level factors, such as attitudes, influence both use and engagement (e.g., Reed, Messler, Coombs, & Quevillon, 2014; Venkatesh, Morris, Davis, & Davis, 2003; Waller & Gilbody, 2009). Accordingly, it is expected that there will be variability in use of and attitudes toward the range of smartphone components within and between caregivers randomized to TE-HNC, that higher levels of use of each smartphone component will parallel more positive attitudes toward the component, and that use and attitudes will correspond with improvement in disruptive behaviors at posttreatment, as well as the efficiency with which families complete the mastery-based HNC program.

Method

Participants

Low-income (i.e., adjusted gross income did not exceed 150% of the federal poverty limit) caregiver-child dyads were included in the pilot RCT if they had a 3- to 8-year-old child (i.e., range for which HNC was developed and tested) and the child met or exceeded clinical cutoffs on the Eyberg Child Behavior Inventory (ECBI; Eyberg & Pincus, 1999). Families were excluded if the child had a developmental or physical disability that precluded the use of HNC (e.g., physical disability precluded use of time-out), the caregiver had a current diagnosis of substance abuse/dependence, mood, or psychotic disorder; and/or the family was involved with Department of Social Services related to abuse/neglect.

Interested and eligible families were randomized to HNC (n = 11) or TE-HNC (n = 11). Of the 11 families randomized to TE-HNC, two served as practice cases for project therapists, resulting in 9 families for these analyses (see Table 1). Caregivers in TE-HNC were 91% female, 91% biological parents, 64% married or in a long-term relationship, and on average 33 years old (SD = 6.71). Nearly half (45%) of youth in TE-HNC were male (M = 5 years old, SD = 1.18).

Table 1.

Caregiver and Child Demographics

	Case 1		Case 2		Case 3		Case 4		Case 5		Case 6		Case 7		Case 8		Case 9
	CG	TC	CG	TC	CG	TC	CG	TC	CG	TC	CG	TC	CG	TC	CG	TC	CG	TC
Age	35	4	34	6	29	6	31	4	44	4	47	7	28	4	32	4	37	6
Gender	F	F	F	M	F	F	M	M	F	F	F	M	F	F	F	M	M	F
Race
American Indian or Alaska Native
Asian						X
Black or African American								X		X							X	X
Native Hawaiian/Other Pacific Islander					X	X
White	X	X	X	X	X	X	X	X	X		X	X	X	X	X	X
Ethnicity
Hispanic or Latino												X				X	X
Not Hispanic or Latino	X	X	X	X	X	X	X	X	X	X	X		X	X	X			X

Notes. CG = Caregiver; TC = Target Child; M = Male; F = Female.

Procedure

Families were recruited through agencies (e.g., schools, YMCAs, churches), doctors’ offices, advertisements (e.g., university-wide informational emails, bus displays, brochures), and word-of-mouth (e.g., participants telling other families about the project). A brief phone screen to determine initial eligibility on key criteria (i.e., 3- to 8-year-old child, disruptive behaviors, low-income) was first completed, then eligible and interested caregiver-child dyads were scheduled for a more extensive baseline assessment to complete caregiver consent for their own and their child’s participation, confirm eligibility criteria, and to obtain additional information about the family. Caregiver-child dyads that met eligibility criteria were then randomized. Procedures at postassessment were similar to the baseline procedures. Caregivers were compensated $50 per assessment, and participants in the TE-HNC group received a $100 bonus for smartphone return.

Technology-Enhanced HNC Program

Consistent with Hanf Model BPT programs (Reitman & McMahon, 2013), HNC (McMahon & Forehand, 2003) is a two-phase program designed to teach effective child management strategies to caregivers of young children (3 to 8 years old) with clinically significant disruptive behaviors. In Phase I, Differential Attention, caregivers learn to increase the frequency and range of social attention to the child’s positive behavior and to ignore minor inappropriate behavior. In Phase II, Compliance Training, caregivers are taught the difference between unclear and clear instructions, to give the “Clear Instruction” sequence, and to use a nonphysical discipline procedure, “Time–Out,” for occasions of noncompliance and other inappropriate behavior. Advancement to the subsequent skill (e.g., Attends to Rewards) or phase (e.g., Phase I to Phase II) is dependent on caregivers meeting specific behavioral performance criteria for the prior skill and set of skills during structured play (i.e., Child’s Game or Parent’s Game), which is observed and coded by therapists at each session. On average, caregivers require 8 to 12 sessions, as well as daily home practice of skills (at least 15 minutes per day recommended), to meet mastery for all program skills (McMahon & Forehand, 2003).

The proof-of-concept TE-HNC smartphone components were designed to complement the theory and treatment techniques of the HNC program and were created in conjunction with: (a) researchers with expertise in providing BPT to underserved families; (b) a Clinician Advisory Panel (20% male; 20% racial minority) whose members practice at least one BPT program; (c) an industry partner with experience in developing tailored and sustainable technological applications; and (d) health economists with expertise in health care efficiency, effectiveness, and value.

Core smartphone-enhancements included: (a) Midweek Videoconference between therapists and caregivers regarding skills practice, including problem-solving obstacles to progress (e.g., selecting an ideal Time-Out location); (b) caregiver completion and submission of a Daily Survey of skills practice and progress used by therapists to guide Midweek Videoconferences and weekly sessions; (c) caregiver daily review of the relevant Skills Videos from the Skills Video Series, which included psychoeducation and modeling of BPT skills; (d) caregiver Weekly Video recording of at least one home practice of skills for therapist review and feedback; and (e) Personalized Text Message Reminders from therapist to client regarding weekly sessions, Midweek Videoconferences, and home practice. Promising trends from a pilot RCT comparing TE-HNC to the standard HNC program suggest that these smartphone enhancements increased the engagement of low-income families in service (i.e., session attendance, participation in Midweek Videoconference, and home practice higher in TE-HNC than HNC), enhanced the efficiency of services (i.e., TE-HNC families required fewer weeks to master program skills than HNC), and boosted treatment outcomes (i.e., effect size and analyses of clinical significance favored TE-HNC relative to HNC) (Jones et al., 2014).

Measures

The following information is reported for all 9 TE-HNC families available for analyses, regardless of completion status:

Demographics

Caregivers reported demographics, including information about themselves (e.g., education and age), the child (e.g., age), and family (e.g., household income).

Client Technology Use

Use was defined by dividing the actual use of each component by the number of opportunities for use to obtain an average score for each family:

Daily Surveys use was defined by the number of surveys completed between sessions, divided by the number of days between sessions throughout treatment. Daily Surveys asked questions about skill practice and were responsive to caregiver input (e.g., if caregiver said s/he practiced skills at home on a given day, the survey asked about the quality of practice).

Skills Videos Series use was defined by the number of times a caregiver reported watching Skills Videos, divided by the number of opportunities to watch Skills Videos throughout treatment. The Skill Videos Series included one approximately 3-minute video per each of 5 skills, which included psychoeducation, modeling, and reminders to practice.

Caregivers were instructed to video record one home practice per week. Therefore, Video recording of Skills Practice use was defined by assessing the proportion of videos recorded between sessions to the number of opportunities for video recording throughout treatment. Therapists watched the video recording in order to tailor feedback.

How often a caregiver completed the scheduled midweek call divided by the number of opportunities to complete a call throughout treatment defined Midweek Videoconference use. As is typical of HNC midweek calls, the videoconference was used to check in regarding skill practice and progress; however, the smartphone allowed it to be face-to-face.

The final TE-HNC components, Text Message Reminders regarding sessions, Midweek Videoconference calls, and skills practice, were not included in these analyses because of the inability to track whether or not caregivers received and read the reminders.

Client Attitudes Toward Technology

Clients’ attitudes towards technology were assessed via open-ended questions about the usefulness/helpfulness of each smartphone component (e.g., “What was the most helpful about the Daily Surveys?” and “What was the least useful part of the technology?”). Given the relatively high satisfaction level with the technology within the TE-HNC group (see Jones et al., 2014, Results), the current investigation focused on smartphone components perceived to be the least useful. Each case report highlights the smartphone component perceived to be least useful by the participating caregiver, and preliminarily explores links between attitudes and use. In addition, the discussion highlights themes, or similar reactions to each smartphone component shared by the majority of cases (i.e., at least 5), and explores the influence of these reactions on the correspondence between use and treatment outcomes.

Treatment Response

Treatment response was examined in two ways for the 7 TE-HNC completers only:

Pre- to postassessment change in disruptive behavior was assessed using the Eyberg Child Behavior Inventory (EBCI), a 36-item inventory for 2- to 16-year-olds (Eyberg & Pincus, 1999). The ECBI has two scales: (a) Intensity Scale, which measures the frequency (1 = never to 7 = always) with which the child engages in each of 36 behavioral problems (Range 36 to 252; ≥ 131 clinically significant); and (b) Problem Scale, which asks caregivers to indicate whether each of the 36 behaviors is “a problem for you” (yes or no; Range 0 to 36; ≥ 15 clinically significant). The ECBI is sensitive to BPT interventions (e.g., Eisenstaedt, Eyberg, McNeil, Newcomb, & Funderbuck, 1993; Nixon, Sweeny, Erickson, & Touyz, 2003; Webster-Stratton & Hammond, 1997) and has demonstrated internal consistency and test-retest reliability (Burns & Patterson, 1990; Eyberg & Robinson, 1983; Funderburk, Eyberg, Rich, & Behar, 2003; Robinson, Eyberg, & Ross, 1980). The alpha for the Intensity Scale was 0.85 and the Problem Scale was 0.76.

Efficiency of services was measured by tracking the number of weeks required for each family to complete the mastery-based HNC program. In order to also assess attendance consistency, weeks where the client canceled or did not show up for a session were also included in analyses.

Plan of Analysis

A case report is provided for the total analysis sample of 9 TE-HNC families. In contrast to prior between-group reports from this study, which focus on completers only (Jones et al., 2014), case reports for the current study include demographics, baseline problem behavior level on the ECBI (Eyberg & Pincus, 1999), and percentage of use of the smartphone components, regardless of whether the TE-HNC family completed the program or dropped–out (N = 9). Percentage of smartphone component use was compared to the average use of the other families (e.g., Case 1 was compared to the average of Cases 2–9). Then, analysis of potential links between smartphone technology use, treatment response, perceived usefulness, and service delivery efficiency is provided for completers only (n = 7).

Treatment response is characterized by comparing the pre-post treatment ECBI (Eyberg & Pincus, 1999) change score in the current study to the aggregated pre-post ECBI change scores published in prior studies of standard BPT programs (see Self-Brown et al., 2012; Ware et al., 2008 for meta-analyses). Using procedures described by others in pilot services research, including those in research and development of technology-enhanced service delivery approaches (e.g., Andreasen et al., 2005; Comer et al., 2014; Löwe, Unützer, Callahan, Perkins, & Kroenke, 2004), treatment response status was defined by: (a) Full treatment responders [i.e., ECBI Intensity and Problem scores were above the clinically significant level at baseline (i.e., Intensity Score at or above 131 and Problem Score at or above 15) and at postassessment had improvement scores at or above one standard deviation (SD = 19.52 Intensity, SD = 5.39 Problem) of the aggregated average change score of published studies (i.e., a 44 point improvement on the Intensity score and a 13 point improvement on the Problem score)]; (b) Partial treatment responders [i.e., ECBI Intensity and Problem scores within one standard deviation above the aggregated change score (i.e., scores fall between the comparison mean change score and one standard deviation of the score) or, change scores between 25–43 on the Intensity scale and change scores between 8–12 on the Problem scale]; and (c) Minimal treatment responders [i.e., ECBI Intensity and Problem scores below the mean change score for the comparison studies (i.e., Intensity scores < 25 and Problem scores < 8)].

Results

Case 1

Case 1 was a 4-year-old Caucasian female whose biological mother (age = 35, married) was the participating caregiver (see Table 1 for more detail on individual family demographics). At baseline, the mother reported her daughter’s disruptive behavior to be in the clinical range on both the ECBI Intensity Scale (Intensity score = 185; clinical cutoff = 131) and Problem Scale (Problem score = 28; clinical cutoff = 15) (see Table 2).

Table 2.

Eyberg Child Behavior Inventory Scores (N = 9)

Case	ECBI Intensity Score			ECBI Total Problem Score
	Pre	Post	Change	Pre	Post	Change
1	185	101	84	28	1	27
2	138	99	39	18	16	2
3	175	95	80	31	9	22
4	128	80	48	17	4	13
5	243	*	*	22	*	*
6	149	60	89	21	1	20
7	221	*	*	33	*	*
8	124	74	50	20	10	10
9	133	73	60	23	1	22

Note.

^*Denotes dropout.

As demonstrated in Table 3, this caregiver used the smartphone enhancements more than the other participants in the TE-HNC group (i.e., 76% compared to 65%). For three of the four smartphone enhancements, this caregiver’s use was above the average of the other TE-HNC participants (i.e., completion of the Daily Survey, Midweek Videoconference, and Weekly Video recording of Skills Practice). For example, the family completed a weekly Video Recording of Skills Practice and completed the Midweek Videoconference for every session week attended, whereas on average, other participants in the TE-HNC group completed and submitted a weekly Video Recording of Skills Practice about 59% of the time and completed the Midweek Videoconference about 91% of the time. For the fourth smartphone enhancement, viewing Skills Videos, however, this caregiver’s use was below the other cases (i.e., 38% compared to 55%). This caregiver at postassessment indicated that the Skills Videos were the least useful smartphone-enhancement out of the four components. This perception of lack of usefulness may be linked to this caregiver’s relatively lower use of this component.

Table 3.

Case Report Data for Completers (n = 7)

	C1	C2	C3	C4	C6	C8	C9	Average
Smartphone Use
Daily Survey	67%	72%	72%	41%	27%	62%	68%	58%
Mid-Week Video Conferencing	100%	89%	100%	83%	86%	100%	86%	92%
Weekly Video Recording	100%	50%	100%	50%	33%	50%	71%	65%
Viewing Skills Videos	38%	22%	100%	20%	13%	100%	74%	52%
Average Use	76%	58%	93%	49%	40%	78%	75%	67%
Clinically Significant Change	Y	N	Y	Y	Y	Y	Y
Intensity Scale Treatment Response
Full Response	X		X	X	X	X	X
Partial Response		X
Minimal Response
Problem Scale Treatment Response
Full Response	X		X		X	X	X
Partial Response				X
Minimal Response		X
Number of Weeks to Complete Program	7	11	8	8	14	8	9	9.29

Notes. C = Cases,% = The proportion of times a skill was completed per session, Y = Yes, N = No, X = Treatment response designation.

This family’s percentage of use corresponded with full treatment response [i.e., more than a 44 point improvement on the Intensity score and more than a 13 point improvement on the Problem score of the ECBI (Eyberg & Pincus, 1999)]. In addition, this family completed the mastery-based program in 7 weeks, while on average it took the other TE-HNC families 9.67 weeks to master program skills (see Table 3).

Case 2

Case 2 was a 6-year-old Caucasian male whose biological mother (age = 34, divorced) was the participating caregiver. At baseline, the mother reported disruptive behaviors slightly above the clinical range on both the Intensity and Problem Scales of the ECBI (Intensity Score =138; Problem score = 18).

Across all four smartphone enhancements, the caregiver’s average use of 58% was 11 percentage points lower than the other TE-HNC participants’ average (i.e., 69%). This caregiver’s use of the smartphone enhancements varied to some extent within and between skills. Specifically, the caregiver’s use of three of the four enhancements (i.e., Midweek Videoconference, weekly Video recording of Skills Practice, and viewing Skills Videos) was below the average use of those enhancements among other participants. For example, the caregiver watched the Skills Videos at a rate below the other TE-HNC participants across all five skills (i.e., 22% compared to 58%). This mother reported that the skills videos were the least useful component of TE-HNC: “Honestly, I never felt the need to watch the videos as a reminder.” The caregiver’s attitude about the usefulness of this component may have influenced her level of use of this component. In addition, although the average use of the fourth enhancement, completion of Daily Surveys, was above the group average, the caregiver’s use was inconsistent across the program. For example, the mother completed the Daily Surveys above the average completion rate for the first two skills in the program, Attends and Rewards (i.e., 88% compared to 48%); however, the survey completion rate appeared to decline as the program progressed (e.g., declined 14 percentage points between the skills Attends and Rewards).

This level of use corresponded with a partial treatment response on the ECBI Intensity scale (i.e., change scores between 25–43) and a minimal treatment response on the ECBI Problem scale (i.e., change scores < 8). This family was the only family of the seven completers that did not meet criteria for full treatment response on the ECBI Problem scale. In addition, this family required 11 weeks to complete the program, two weeks longer than the average among other TE-HNC families (9 weeks).

Case 3

Case 3 was a 6-year-old, Asian, Caucasian, and Native Hawaiian/Pacific Islander female whose biological mother (age = 29, married) was the participating caregiver. At baseline, the mother reported that her daughter exhibited problem behaviors in the clinical range indicated by ECBI Intensity and Problem scores of 175 and 31, respectively.

Across all four of the smartphone enhancements, this mother’s overall average use was higher than the average of the other TE-HNC participants (93% compared to 63%). Within the TE-HNC group, this caregiver was the only caregiver to use three of the four smartphone enhancements (i.e., complete the Midweek Videoconference, Video record Skills Practice, and view Skills Videos) every week between sessions (i.e., 100% of the time). In addition, her use of the Daily Surveys was above the average use of the other TE-HNC families (i.e., 72% compared to 56%). This mother, however, indicated that “if I had to pick [a least useful component], it would be the surveys.” This perception may reflect the relatively lower use of the Daily Surveys. Overall, her average smartphone technology use was above the group average, and this family had the highest rate of smartphone enhancement use. This was reflected in the caregiver’s overall impressions of the smartphone-enhancements: “Overall I loved the iPhone. It was great seeing the videos and having to be accountable each night for the skills that I practiced when doing the survey.”

Case 3’s level of use paralleled full treatment response on both the Intensity and Problem scales of the ECBI (i.e., more than a 44 point improvement on the Intensity score and more than a 13 point improvement on the Problem score). Notably, the change score on the Intensity scale was almost double the criteria for full treatment response (i.e., 80 point decrease in Intensity scale score). In addition, she completed the program in 8 weeks (other TE-HNC families’ average completion time = 9.5 weeks).

Case 4

Case 4 was a 4-year-old African American and Caucasian male whose biological father (age = 31, married) was the participating caregiver. The father reported his son’s problem behaviors to be below clinical range on the Intensity Scale (Intensity score = 128) and slightly above the clinical range on the Problem Scale (Problem score = 17).

Across all four of the smartphone enhancements, this family used the smartphone technology below the other families’ average (i.e., 49% compared to 70%). In particular, this family’s use of the Skills Videos was more than 30 percentage points lower than the other TE-HNC families (i.e., 20% compared to 58%). This difference in use may reflect this father’s reactions to the Skills Videos component of the smartphone. At postassessment he indicated, “Videos on the iPhone are small and do not lend themselves towards keeping your attention. This is especially true after repeated viewings.” Of the completers, this father was the only participant who did not endorse finding Watching Skills Videos as useful.

Case 4’s percentage of use paralleled partial treatment response on the ECBI Problem Scale (i.e., change scores between 8 and 12). Although this family did not meet criteria for full treatment response on one subscale of the ECBI, this level of use did not seem to influence clinically significant change or service delivery efficiency. Case 4 completed the program in 8 weeks, about one and a half weeks fewer than the average of the other participants’ completion time (i.e., 9.5 weeks).

Case 5

Case 5 was a 4-year-old African American female whose adoptive mother (age = 44, same sex partnership) was the participating caregiver. At baseline, the mother rated her daughter’s disruptive behavior in the clinical range on both the ECBI Intensity Scale (Intensity score = 243) and Problem Scale (Problem score = 22).

Case 5 only completed 4 weeks and dropped out of the program due to the hospitalization of another child in the family. During those 4 weeks, the family worked on the first two HNC skills, Attends and Rewards. They spent 1 week learning Attends and 3 weeks on Rewards. By session four only one other family, Case 7, had not progressed to the third HNC skill, Ignoring, and this family also withdrew from the study before completion. The Case 5 family only engaged with one of the four smartphone enhancements. The caregiver completed 50% of the possible Midweek Videoconferences. Not only was this percentage of use more than 40 percentage points below the TE-HNC completer group average, but it was also the lowest usage within the group for this smartphone enhancement.

Case 6

Case 6 was a 7-year-old Latino male whose biological mother (age = 47, divorced) was the participating caregiver. At baseline, the mother rated her son’s disruptive behavior problems in the clinical range on both scales of the ECBI (i.e., Intensity Score = 149; Problem score = 21).

This mother’s overall use of the four smartphone enhancements was below the average among other families (i.e., 40% compared to 72%). This caregiver’s use of the smartphone enhancements varied to some extent within and between skills. For the first 5 weeks as the three skills of Phase I of the program (i.e., Attends, Rewards, and Ignoring) were taught, the family did not complete any of the Daily Surveys (i.e., intended to summarize skill practice and progress for therapist, as well as to remind families of the importance of skill practice) and, therefore, did not report viewing the Skills Videos. Although the family did not use the technology during Phase I, the family used the smartphone enhancements and reported practicing above the average of the other families in the TE-HNC group during Phase II (i.e., 60% during Phase II compared to 52% during Phase I). For each enhancement, the mother’s usage was lower than the average of the other TE-HNC families. For example, this mother viewed Skills Videos 13% of time whereas on average the other TE-HNC families viewed Skills Videos 59% of the time. This caregiver did not complete the postassessment satisfaction survey, and therefore representative quotes about her reactions to the smartphone enhancements are not available.

This level of use did not seem to parallel treatment response. The family met criteria for full treatment response (i.e., more than a 44 point improvement on the Intensity score and more than a 13 point improvement on the Problem score). This relatively low percentage of use, however, seemed to correspond with low efficiency in service delivery. This family took 14 weeks to complete the program, longer than any other family in the TE-HNC group (i.e., Range 7 to 14; also see Table 3).

Case 7

Case 7 was a 4-year-old Caucasian female whose biological mother (age = 28, separated) was the participating caregiver. At baseline, the mother reported her daughter’s disruptive behavior in the clinical range on both the ECBI Intensity Scale (Intensity score = 221) and Problem Scale (Problem score = 33).

Case 7 dropped out of the study after session three. The caregiver was learning Rewards at the time of dropout. At the time Case 7 withdrew from the study, they were one of two families that had not progressed to the next HNC skill, Ignoring. The other family, Case 5, also withdrew from the study. Case 7 withdrew due to family stressors, including separation, Department of Social Services allegations, change in living situation, and an employment shift. During Attends, this family was using the smartphone enhancements more than the other TE-HNC families (i.e., 83% compared to 62%). The mother, however, did not use the smartphone technology between the second and third weeks. In addition, throughout the time enrolled in the program, the family completed 67% of the possible Midweek Videoconferences with the therapist. This was 25 percentage points below the other TE-HNC families’ levels of use (i.e., 92%).

Case 8

Case 8 was a 4-year-old Latino male whose biological mother (age = 32, divorced) was the participating caregiver. At baseline, the mother reported that her son exhibited disruptive behavior below the clinical range on ECBI Intensity Scale (Intensity Score = 124), but above the clinical range on the Problem Scale (Problem score = 20).

This family’s overall technology use was above the other TE-HNC participants (i.e., 78% compared to 65%). Their smartphone enhancement use was above average of the other families, except for Video Recording Skills Practice. The mother Video Recorded Skills Practice 50% of the time whereas on average other caregivers in the TE-HNC group Video Recorded Skills Practice 67% of the time. This mother expressed discomfort being videotaped: “I can be uncomfortable taping myself but after awhile I got more comfortable with it.” This discomfort may be linked to her lower use, particularly given that she was one of only two families who expressed concerns with this component. The mother, however, used two of the four components, viewing Skills Videos and completing the Midweek Videoconference with the therapist, between each session (i.e., 100% of the time).

This level of use corresponded with full treatment response (i.e., more than a 44 point improvement on the Intensity score; more than a 13 point improvement on the Problem score). In addition, they completed the program in 8 weeks (average completion time other TE-HNC families = 9.5 weeks), suggesting that this level of use corresponded with efficient service delivery.

Case 9

Case 9 was a 6-year-old African American female whose biological father (age = 37, married) was the participating caregiver. At baseline, the father reported that his daughter exhibited disruptive behaviors in the clinical range on both ECBI scales (Intensity Score = 133; Problem score = 23).

This family’s overall technology use exceeded the use of other TE-HNC families (i.e., 75% compared to 66%). The father’s smartphone enhancement use was above the other families’ average use on three of the four enhancements (i.e., completion of Daily Surveys, Video Recording of Skills Practice, and viewing Skills Videos). For the fourth enhancement, completing the Midweek Videoconference with the therapist, the father completed this 86% of the time, whereas on average the other families completed this 93% of the time. During the postassessment, this father indicated that he believed all of the smartphone enhancements were useful; however, he indicated that the Midweek Videoconference was least useful: “If I must give an answer then it would be the midweek call. I rarely had questions.” The lack of perceived utility may account for this family’s relatively lower use of this enhancement.

This level of use corresponded with full treatment response (i.e., more than a 44 point improvement on the Intensity score, more than a 13 point improvement on the Problem score). This level of use, however, did not seem to correspond with the number of weeks it took the family to complete the mastery-based program. Case 9 completed the program in about the average number of weeks for the group (i.e., 9 weeks).

Discussion

The current within-group case series extends previously reported between-group findings (Jones et al., 2014) by examining the correspondence of variability in caregiver use of and attitudes toward technology-enhanced services. It is important to state at the outset that there was relatively little variability in technology use, and even less variability in treatment outcomes across cases within the TE-HNC group. With regard to treatment outcome, the fact that the majority of families were full treatment responders on the ECBI Intensity (exception of Case 2) and Problem Scales (exception of Cases 2 and 4; Eyberg & Pincus, 1999) was expected to some extent, given that TE-HNC includes the full treatment components of HNC, the standard of care treatment for disruptive behavior disorders (McMahon & Forehand, 2003). In contrast, the relatively high level of smartphone component use within the TE-HNC group (i.e., 52% to 92% across all of the HNC skills, Attends, Rewards, Ignoring, Clear Instructions, Time-Out) was perhaps less expected; however, not entirely surprising given the promising between-group outcomes previously reported. That said, the relative lack of within-group variability may suggest participants in the TE-HNC group were unique in some way (e.g., families would be high engagers regardless of technology use); however, it is important to remember that random assignment to group in the pilot RCT should have decreased the likelihood of this group being unique. As such, the high levels of technology use warrant additional exploration of and validate excitement for technology-enhanced mental health service delivery (see Aguilera & Muench, 2012; Enock & McNally, 2013; Jones et al., 2013, for reviews).

Despite the aforementioned caveats, the case series design still provided a preliminary window into how variability in use of and attitudes toward technology may correspond with outcomes in technology-enhanced services. First, variability in use across components may elucidate aspects of functionality that influence client acceptability and use of the technology. The lowest levels of use (i.e., 52% and 58%) were obtained for the components families were asked to complete daily (i.e., Skills Videos and Daily Surveys, respectively). While most families used these components weekly between each session, clients rarely used the components daily. On the other hand, weekly components including both the Midweek Videoconference (i.e., 92%) and Video Recording Skills Practice (i.e., 67%), had relatively higher levels of use. Of note, the Midweek Videoconference was also initiated by the therapist, which may further explain its relatively high level of use.

In addition, case reports suggest that within-group variability of smartphone component use seemed to correspond with treatment outcomes, and in some cases completion of the program in fewer weeks. For example, in Case 1 the relatively consistent and high rate of smartphone enhancement use may account for the above average pre-post treatment change scores, and, in turn, full treatment response. In addition, the relatively inconsistent level of smartphone enhancement use for Case 6 may account for the decreased efficiency of service delivery. The dramatic increase in the use of the smartphone enhancements by Case 6 during Phase II relative to Phase I, however, may account for this family’s designation as a full treatment responder. In contrast, Case 2, the only case that did not meet full treatment response on either ECBI subscale (Eyberg & Pincus, 1999), used the smartphone enhancements relatively less than the other caregivers in the TE-HNC group. This comparative deficit in use may account for the extra weeks required to master the HNC skills and the minimal treatment response on the ECBI Problem scale (Eyberg & Pincus, 1999).

Given preliminary findings, both between-group (Jones et al., 2014) and within-group, suggesting correspondence between smartphone component use and treatment outcomes, it is essential to understand client factors related to technology use and uptake. While the literature suggests that many client factors (e.g., comfort with and exposure to technology) influence the use or uptake of technology-enhanced service delivery methods, client attitudes towards technology are considered a central context in which to understand use, and, in turn, treatment outcomes (e.g., Reed et al., 2014; Venkatesh et al., 2003; Waller & Gilbody, 2009). Across all cases, the smartphone component perceived to be the least useful by a given family was used the least by the same family. While there is research to suggest that the relationship between use and attitudes towards technology may be bidirectional (i.e., attitudes towards technology improve with increased exposure to the technology and vice versa; Reed et al., 2014), this finding highlights the importance of assessing clients’ attitudes towards each component of a technology-enhanced intervention. Moreover, findings highlight the importance of clinicians providing adequate rationale for each component and monitoring use to enhance the perceived usefulness and, in turn, potential impact, of technology-enhanced service delivery methods.

In addition, themes in client attitudes towards each component of the technology reveal functions of technology perceived to be most helpful: (a) Daily Surveys increased accountability (Case 3: “It was a way of not letting me out of doing Child’s Game”); (b) Midweek Videoconferencing enhanced communication and increased opportunities to problem solve with the therapist (Case 8: “It made communication with therapist easier”); (c) Video Recording of Skills Practice increased opportunities for therapist feedback, particularly in terms of skill generalization in different settings (Case 1: “It was nice to have the therapist see how it was really working at home and to give me suggestion or/and encouragement”); and (d) the Skills Video Series served as a reminder of skills learned and increased opportunities for skills modeling (Case 2: “The videos helped to remind me of things I was told during the meetings, but maybe forgot. Sometimes I would see something I did not see before or it would just make everything click.”). These themes preliminarily suggest that the intended function of the technology was achieved (i.e., increase engagement and adherence to interventions by facilitating efficient communication between clinicians and clients, and by increasing access to resources to help skills generalize between sessions). Although additional research is needed to explore this link further, it is possible that client attitudes towards technology account for the uptake and sustainability of technology-enhanced service delivery.

Consistent with the pilot and case report approach to the analyses, findings should be considered preliminary in light of study limitations. First, caution is warranted when generalizing findings beyond low-income families; however, it is important to note that the theory or hypotheses grounding this study were not developed to be specific to low-income families or to a specific technology. Rather, low-income families were targeted given that they are less likely to engage in BPT services, and smartphone technology was selected because it was more widely available to low-income families than other platforms (e.g., desktop computers, notebooks, tablets). Second, although some study variables were captured via the smartphone enhancements (e.g., Weekly Video Recording of Skills), others relied on caregiver (e.g., whether or not the caregiver watched a Skills Video) or therapist report (e.g., Midweek Videoconference participation), increasing the possibility of error or bias. Third, the pilot nature of these analyses precluded the capacity to examine the role of specific components of TE-HNC (e.g., Daily Surveys, Video recording Home Practice) on treatment response. Finally, this study did not examine client motivation; however, it is possible that motivation may account for both use of technology and boosted treatment outcomes.

Despite these limitations, the study also has several strengths. Consistent with calls to maximize the potential for knowledge generated from technology-enhanced services research (Riley et al., 2011; Rothwell, 2005; Wu et al., 2006), this study uses case reports to describe variability in caregiver smartphone enhancement use within the TE-HNC group. This approach is critical to better understanding the feasibility, usability, and potential impact of technology-enhanced BPT approaches for vulnerable, yet underserved families, with broader service-delivery implications for children’s mental health. In addition, efforts were made to examine a platform, smartphones, that was relevant and accessible to low-income families that are most in need of BPT and least likely to receive services. Such an approach is consistent with calls by leaders in mental health, as well as technology, to more explicitly focus on the design and development of technology to best meet the needs of the target population and particular presenting issues (Jones, 2014; Nelson et al., 2011; Swartz, 2014). Third, although the sample was too small to quantitatively examine the processes through which technology influenced treatment response, the case series approach helped elucidate the extent to which variability in use within TE-HNC may correspond with variability in therapeutic gains.

Finally, and perhaps most important, our goal with this research is to explore service delivery options that enhance the therapeutic outcomes of low-income families by increasing opportunities for assessment (e.g., Daily Surveys of skill practice and progress), modeling (e.g., Skills Video Series) and tailored feedback and support (e.g., therapist review of Video Recording of home practice) between sessions and in the context of daily stressors and challenges. Between-group analyses suggested the promise of technology enhancements relative to standard BPT; yet, acceptability and sustainability of such an approach depends on a better understanding of how caregivers respond to and interface with the technology. Importantly, our findings suggest that families used these technologyenhancements at a fairly high rate, yet patterns of variability suggested that those who used it more or more regularly might have benefited the most. Of course these findings need to be replicated with larger samples and more sophisticated methods; nevertheless, the findings hold promise for technology as a sustainable service-delivery vehicle for even the most difficult to engage families.