Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Nov 1.
Published in final edited form as: Psychol Serv. 2018 Nov;15(4):419–428. doi: 10.1037/ser0000193

Emergency Department Text Messaging for Adolescent Violence & Depression Prevention: A Pilot RCT

Megan L Ranney a, Sarah K Pittman a, Shira Dunsiger b, Kate Guthrie c, Anthony Spirito c, Edward Boyer d, Rebecca M Cunningham e
PMCID: PMC6600822  NIHMSID: NIHMS1032743  PMID: 30382737

Abstract

The purpose of this study was to evaluate feasibility and acceptability, obtain preliminary efficacy data, and evaluate predictors of improvement with iDOVE, a technology-augmented violence and depression prevention intervention for high-risk adolescents seen in the emergency department (ED). We conducted a pilot randomized controlled trial with 116 English speaking adolescents (ages 13 to 17), presenting to the ED for any reason, who reported past-year physical peer violence and current depressive symptoms. The CBT- and MI-based intervention consisted of a brief in-ED intervention session and 8 weeks of automated text-message daily mood queries and tailored responses. The control was a brief in-ED presentation and twice-weekly text messages on healthy behaviors. Follow-up was conducted at 8 and 16 weeks. Descriptive statistics, bivariate comparisons, mixed effects longitudinal regression models, and latent class models were calculated. iDOVE had high acceptability and feasibility, with 86% of eligible youth consenting (n=116), 95% completing 8-week follow-up, and 91% completing 16-week follow-up. High quantitative and qualitative satisfaction was reported by participants in both intervention and control groups. Comparing intervention to control, improved depressive symptoms (p=0.07) and physical peer violence (p=0.01) were observed among the more symptomatic youth in the intervention group (but no difference in symptoms between the full intervention and control groups). Latent class modeling showed that intervention responsiveness correlated with lower mood (measured through daily text-messages) at day 7 of the intervention. This randomized control trial of a technology-augmented intervention shows high feasibility and acceptability, and a promising signal of reduced violence among the highest-risk participants.

Keywords: Adolescent, Peer violence, Emergency department, Mobile health, Text messaging, Motivational Intervention, Cognitive behavioral therapy

Depressive symptoms and physical peer violence, defined as a physical fight with a peer, are epidemic and inter-related conditions among adolescents (Merikangas et al., 2010; Ranney, Walton, et al., 2013; Saluja et al., 2004). A strong theoretical basis exists to explain the bidirectional relationship between depression and exposure to peer violence. Risk factors for depression (e.g., poor problem-solving skills, misreading of social cues, and inability to regulate one’s emotional responses) predict involvement in peer violence (Hamilton et al., 2016; Shadur & Lejuez, 2015; Silk, Steinberg, & Morris, 2003). Conversely, peer violence may increase stress reactivity and reinforce maladaptive coping skills and negative world-views, thereby worsening depressive symptoms (Seiffge-Krenke, 2000). Of note, peer violence perpetration and victimization are highly overlapping, and are similarly associated with depressive symptoms(Malik, Sorenson, & Aneshensel, 1997; Ranney, Patena, et al., 2016).

The emergency department (ED) is a promising location for identifying and intervening with at-risk youth. Higher rates of peer violence (40–50%), depressive symptoms (20–40%) and other mental health issues are reported among youth seen in the ED, regardless of the reason for their ED visit, compared to youth in schools or pediatric clinics (Ranney, Patena, et al., 2016; Riese, Mello, Baird, Steele, & Ranney, 2015). Adolescents with depressive symptoms and a history of peer violence often use the ED as their primary source of care (Wilson & Klein, 2000). These youth are unlikely to access evidence-based preventive or therapeutic treatments due to both structural and attitudinal barriers (e.g., lack of transportation; stigma), and are likely to be missed by traditional clinic- or school-based interventions (Bohnert et al., 2015; Owens et al., 2002; Substance Abuse and Mental Health Services Administration, 2015). Moreover, both depression and violence are associated with low adherence to treatment (Pailler et al., 2009; Spirito, Boergers, Donaldson, Bishop, & Lewander, 2002). Multi-session in-person interventions, although effective for many adolescents, are therefore unlikely to be successful with this population (Spirito et al., 2002). Brief ED-based interventions are effective at reducing recurrent peer violence in youth with a history of alcohol use (Cunningham et al., 2012; Walton et al., 2010), and may have temporary effects on co-existing depressive symptoms (Ranney et al., 2017).

Text messaging is an innovative delivery mechanism for longitudinally engaging and delivering content to these high-risk adolescents. Although only 75% of adolescents (across all socioeconomic strata) have smartphones, ~95% of adolescents – including ED patients – have mobile phones with text-messaging (Lenhart, 2015; Ranney, Choo, Spirito, & Mello, 2013). Text-messaging interventions may be more effective than a one-time brief intervention in changing the trajectory of behavioral risk factors, such as substance use and smoking (Armanasco, Miller, Fjeldsoe, & Marshall, 2017; Hall, Cole-Lewis, & Bernhardt, 2015; Mason, Ola, Zaharakis, & Zhang, 2015). Additional advantages of text message interventions may include automated tailoring, low costs after the development phase, and circumvention of many common barriers to adolescents’ accessing preventive interventions (e.g., stigma, workforce, transportation) (Atienza & Patrick, 2011).

To our knowledge, no published study has examined the feasibility, acceptability, and potential efficacy of text-message-based interventions to prevent violence and reduce depressive symptoms among at-risk adolescents. A brief, in-person intervention in the ED, followed by an automated text-messaging program, may successfully engage at-risk adolescents who often lack access to alternative prevention modalities.

In this paper, we describe our main outcomes from a pilot randomized controlled trial of iDOVE (Ranney, Freeman, et al., 2016; Ranney et al., 2015) compared to a control text-message intervention on health and safety. We hypothesized that this intervention would be highly acceptable (defined as 65% enrollment rates and 80% agree/strong ratings on the Client Satisfaction Questionnaire (Attkisson & Zwick, 1982) and feasible (defined as 80% completion of both in-person and text-message sessions). Although not powered to detect efficacy, the secondary aim was to analyze changes in peer violence and depressive symptoms in iDOVE participants at 8 and 16 weeks, relative to an automated health-and-safety brief intervention and text message program. Finally, we examine the correlation between participants’ mood at the close of the study and their daily automated text-message-based mood ratings.

Methods

Procedure

Study Setting.

Recruitment for this randomized controlled trial occurred from February-December 2015 at a Level 1 trauma pediatric ED, in a Northeast urban city, that serves ~50,000 pediatric patients per year. Patients at this ED represent a heterogeneous population (~40% non-White, 30% Hispanic, 30% uninsured) of adolescents (ages 13–17) (Ranney, Patena, et al., 2016). Institutional Review Board approval and an NIH Certificate of Confidentiality were obtained prior to initiation of the study.

Participants.

A total of 116 participants (58 of whom received the iDOVE intervention, and 58 of whom received the control condition) were recruited. Participants were mostly female (58%), non-white (51%), non-LGB (76%), and low socioeconomic status (67%). The Intervention and Control groups were demographically similar. Mean baseline BDI-2 score was 13.5 (SD 9.3), and median baseline CTS-2 score (perpetration or victimization) was 2.00 (range 36). See Table 1 and Results, below, for more details.

Recruitment.

Research assistants (RAs) screened consecutively admitted patients to the ED all days of the week, 8am-11pm, based on a convenience sample of RA shifts adjusted for ED patient flow. Every patient meeting screening inclusion criteria (age 13–17 years, with English language proficiency and with a parent/guardian present and able to consent) and not meeting exclusion criteria (chief complaint of suicidality, psychosis, sexual assault, or child abuse; in police or child protective services’ custody; in need of emergency psychiatric care; unable to assent) during an RA shift was approached. Potentially eligible participants were identified by RAs from the electronic ED tracking log. After obtaining parental consent/adolescent assent, RAs administered a computerized screen for past-two-week depressive symptoms and past-year peer violence. Screening surveys were conducted on a touch-screen tablet using REDCap, (Research Electronic Data Capture; P. A. Harris et al., 2009), a secure, web-based application designed to support data capture for research studies.

Eligibility for the intervention was determined based on being classified as “at-risk” for future violence and depression, determined through participant self-report of the following: 1) Past-year peer (non-partner) physical violence (victimization or perpetration), using a modified version of the Revised Conflict Tactics Scale 2 (CTS2; Straus, Hamby, Boney-McCoy, & Sugarman, 1996). This validated measure (α=.89-.91), previously used by our team for ED-based peer violence prevention interventions (Cunningham et al., 2012; Ranney, Freeman, et al., 2016), asks about frequency of different types of physical victimization and perpetration (e.g., “In the past 12 months, how often have you punched or hit someone with something that could hurt?”); 2) Mild-to-moderate depressive symptoms, defined as a Patient Health Questionnaire-9 (PHQ-9) score of 5–19 (inclusive) (Kroenke, Spitzer, & Williams, 2001; Moriarty, Gilbody, McMillan, & Manea, 2015). A range of 5–19 accords with that used by other preventive intervention studies, including our own (Ranney, Freeman, et al., 2016). Finally, adolescents had to report possession of a cell phone with texting capability.

If adolescents met eligibility criteria for the full study based on the screen, a second consent/assent and a contact information form was completed. Participants also completed a baseline assessment prior to randomization (see Table 2 and Measures below).

Randomization.

After consent/assent and completion of the baseline assessment, participants were randomized into one of two groups, the iDOVE intervention condition or a healthy-lifestyle control condition. Randomization was carried out in REDCap based on a stratified randomization protocol, with randomly varying blocks of size 4 or 6. As both gender and baseline depressive symptoms are known moderators of treatment effect (Calvete & Cardenoso, 2005), randomization was stratified by these variables (assessed during the baseline assessment, prior to randomization). Specifically, we aimed to have a balanced number of participants in the intervention and control arms across stratum (sex; and baseline Beck Depression Inventory-2 [BDI] score <20 or ≥20)(Beck, Brown, & Steer, 1996). The BDI-2 is a widely used measurement with excellent validity and reliability in adolescents (α = .92–.93) (Osman, Barrios, Gutierrez, Williams, & Bailey, 2008; Osman, Kopper, Barrios, Gutierrez, & Bagge, 2004); it was administered as part of the baseline assessment (see Measures below, and Table 2). The BDI-2 score was used instead of the PHQ-9 score because of its greater validity and reliability(Osman et al., 2008), and because it was the primary outcome measure. BDI-2 cutoffs were chosen according to clinical cutoffs as well as the mean score in the unblinded one-arm pilot phase (Beck et al., 1996).

An analyst not involved in conducting the study generated the randomization table prior to initiation of the study using Stata SE 13 (StataCorp LP, College Station, TX). All study staff, with the exception of the interventionists, were blinded to the participant randomization group.

iDOVE Intervention.

As described elsewhere (Ranney, Freeman, et al., 2016), iDOVE consisted of a brief, in-person, computer-guided, intervention in the ED (~15–20 minutes) followed by an eight-week, bi-directional, automated text-message curriculum. The in-ED intervention was designed based on motivational interviewing (MI) and cognitive behavioral therapy (CBT), and iteratively developed based on participant feedback and expert guidance (Ranney et al., 2014; Ranney, Freeman, et al., 2016; Ranney et al., 2015; Thorsen, Patena, Morrow Guthrie, Spirito, & Ranney, 2016). The brief intervention provided feedback about physical peer violence and depressive symptoms, and encouraged goal-setting and reflection; this section was followed by a brief introduction to basic CBT concepts (e.g., the thoughts-feelings-actions triangle) and their connection to both peer violence and depressive symptoms; and closed with a brief introduction to the text messaging system. The in-ED intervention was delivered by the same trained research assistant who recruited the participant. The RAs were college graduates without prior mental health intervention training. All interventions were audio-recorded for fidelity.

After completing the in-ED brief intervention, iDOVE participants were enrolled into an automated, tailored text message content streams. Text message intervention content consisted of: 1) Daily mood queries; 2) Daily messages communicating core cognitive behavioral therapy concepts (cognitive restructuring, emotional regulation, and behavioral activation), tailored according to daily mood; 3) On-demand supportive messages, which could be pulled by texting the keywords “stressed,” “sad,” or “angry.” Text messages were delivered to participants’ own cell phones, at the time of day that the participants preferred, using an automated computerized system delivered by an outside company, Reify Health. If participants’ responses to text messages were outside of the expected parameters (i.e., the numbers 1–5, yes, no), an automatic reply text message was sent with information about crisis services.

Control Condition.

Participants randomized to the healthy lifestyle control group participated in a brief, attention-matched computer-guided in-ED session, which also took place during waiting periods in the ED stay. Topics included: healthy nutrition, home safety, recreational safety, transportation safety, hygiene, and an overview of the texting component. The control in-ED session was primarily informational; interventionists were specifically trained to not use MI and CBT skills. Control participants then received one-way text-message content twice a week for 8 weeks total, also through Reify Health, covering similar topics to the in-person control session.

Compensation.

All participants completing the screening survey received a small gift (~$1–2) such as a pen or pack of gum. Upon completion of the baseline assessment and in-ED intervention session, participants received a $25 gift card. Participants were compensated with a $40 gift card for completing the 8-week follow-up assessment, and an additional $50 gift card for completing the 16-week assessment. If participants completed the follow-up surveys online prior to the 8- and 16-week follow-up deadline, they received another $5 for each follow-up assessment. Finally, participants were reimbursed $10 every four weeks for the eight-week duration of the intervention ($20 total) to cover potential costs of receiving the text messages, and received an additional $5 for proactively informing the study of changes in contact information.

Crisis Management Protocol.

As approved by the Institutional Review Board, if at any point during screening or recruitment in the ED, participants disclosed suicidal ideation or severe depressive symptoms, the attending emergency physician was notified; if suicidal ideation or severe depressive symptoms were disclosed during follow-ups, or identified via text messaging, the study PI contacted the participant and their parent(s) and referred to outside services as needed.

Measures

Self-report screening, baseline, and follow-up (8-week and 16-week) assessments were completed by participants using REDCap, to minimize assessment bias (see Table 1).

Participants self-reported demographic variables during screening and baseline, including age, race, ethnicity, gender, status as a parent, their grade in school, and whether or not they received public assistance in accordance with questions from the National Study for Adolescent Health (K. Harris et al., 2003), and sexual orientation based on recommended best practices (Gender Identity in U.S. Surveillance Group, 2013).

Acceptability and feasibility of the intervention were determined using a combination of self-reported and objective measures. Enrollment and retention rates for both intervention and control groups were calculated. For the intervention group, we calculated the percent of daily mood queries receiving a reply, number of on-demand messages requested, and number of auto-replies sent. At the 8-week follow-up, a modified version of the self-report Client Satisfaction Questionnaire-8 (CSQ-8; Attkisson & Zwick, 1982) was administered. The CSQ-8 has good validity and reliability (Attkisson & Zwick, 1982). In addition, all participants were asked brief open-ended questions at 8-week follow up: “What was MOST helpful about iDOVE?” and “We welcome your comments and feedback on iDOVE.”

To determine ED intervention fidelity, the principal investigator coded 20% of intervention recordings. Adherence to specific content areas was outlined in the BI manual, with 80% as the minimum criterion. Overall competence was rated using the general therapeutic competence 6-item subscale of the Cognitive Therapy Rating Scale (CTRS) and an adapted version of the Motivational Interviewing Therapy Integrity scale (MITI), as piloted in our prior work (Moyers, Martin, Manuel, Hendrickson, & Miller, 2005; Ranney, Freeman, et al., 2016; Young & Beck, 1980).

Past-two-month violence was measured at the same time periods (baseline, 8 weeks, 16 weeks) using the modified physical subscale of CTS-2 (Straus et al., 1996), described under “Study Eligibility” above (α=.89-.91). Past-two-week depressive symptoms were measured at baseline, 8, and 16 weeks using the BDI-2 (Beck et al., 1996), as described above.

Analysis

Descriptive statistics were calculated to examine enrollment rates, participants’ ratings of the clarity and structure of the in-person and text-message sessions, and satisfaction. We also examined qualitative free-text responses from participants at the close of the trial (both intervention and control groups). The percentage of daily and on-demand use of the text-messaging system was calculated using data from the text-message system. Between-group differences in satisfaction, baseline demographics, peer violence, and depressive symptoms were assessed using t-tests for continuous variables and chi-squared tests for categorical variables.

We used a series of mixed effects longitudinal regression models to simultaneously test the effects of intervention on violence and depressive symptoms at 8-week and 16-week follow-up, controlling for baseline symptoms. Models included a random intercept to adjust for correlated responses over time within participants. Potential confounders were chosen a priori and included gender and BDI stratification. As a subsequent step, we conducted post-hoc comparisons stratified by baseline outcomes (e.g., baseline BDI<20 versus baseline BDI>=20 and baseline CTS 4 vs >=4, as per our a priori stratification variables). As there was significant variability in CTS scores, and transformation was less than successful at bringing the variable towards normality, we modeled the median as a measure of central tendency rather than the mean. Violence intervention effects were therefore estimated using quantile regression models, controlling for baseline and gender.

Finally, to better understand the pattern of daily mood amongst intervention participants, we fit a series of Latent Class Models to identify potential discrete classes of mood over time. Latent Class Models (LCM’s) assume that the outcomes (daily measure of mood, in this case) arise from a mixture over a finite number of latent classes (similar sets of responses that are objectively grouped into classes that can be interpreted to represent distinct patterns of the outcome). Although it is not necessary to make assumptions about the latent class distribution, we assumed that the number of classes is fixed and known. The final number of classes was chosen such that the Bayesian Information Criteria (BIC) was minimized. Trajectories of daily mood (self-reported via text-message query, as part of the iDOVE intervention) were mapped onto a class (most likely pattern to describe the changes over time) and then regressed mean BDI at follow-up on class.

All analyses were conducted using an intent-to-treat approach: all patients randomized to intervention were included in the analysis. As mixed effects models use a likelihood-based approach to estimation, all available data was used without directly imputing missing outcomes.

Analyses were conducted using SAS 9.3, with alpha set at 0.05 a priori. As this was a pilot study, and not powered for strict statistical hypothesis testing, we chose to additionally present effect sizes (where appropriate).

Results

Of 1,190 potential screening subjects presenting to the ED for any reason during from February-December 2015, 1063 (89.3%) consented to screening. Of these, 1031 were successfully screened (86.6%) and 142 (13.4%) met eligibility criteria for the full trial. Of those eligible for the full RCT, 116 (81.8%) consented. Follow-up rates were 95% (110/116) at 8 weeks, and 91% (106/116) at 16 weeks, with no differential attrition between groups (see Figure 1).

Mean in-ED intervention duration for the iDOVE group was 22 minutes (SD 4 minutes), versus mean 7 minutes (SD 2 minutes) for the control group. Of the 20% of iDOVE interventions that were rated, per protocol, for fidelity, mean adherence was 100%, mean CTRS was 35/36, and mean MITI was 4.9/5.

Among the intervention group, 56 (96%) responded to at least one of the daily text message queries (range 0–56 responses, mean 47 responses per participant [84% of days]). Twenty-two intervention participants (19%) requested on-demand support text-messages (mean number of requests per intervention participant = 1.5, range 0–26).

A total of 555 automatic messages containing crisis phone numbers were sent to 93 participants (53 intervention, 40 control, 80% of total participants), due to text message responses outside of the expected range. Almost all of these automatic response messages were due to participants’ texting phrases like “Thank you” or “OK” to the iDOVE system. Nine participants, all of whom were in the intervention group, received the automatic text messages due to sending a text message containing signs of distress (e.g., texting to the iDOVE system that “I’m sad”). Upon PI follow-up with these nine participants, none required referral to crisis services.

Almost all participants in both groups rated the program as overall “excellent” or “good” (mean CSQ-8 score (out of 40): Intervention 32.3 (SD 5.7), Control 31.2 (SD 5.5), t-test p=0.33). Intervention participants reported non-significantly higher satisfaction with iDOVE (90.9% saying they were “mostly satisfied” or “very satisfied”) compared with control participants (80%, p=0.105). They were non-significantly more likely to report that the study helped them deal with problems effectively (83.6% vs. 72.7%, p=0.166). The intervention and control groups were equally likely to rate quality of the text messages (85.5% vs 85.5%, p=1.00) and in-person session (87.3% vs. 80%, p=0.303) as excellent or good.

Qualitatively, both groups appreciated the positivity and advice in the text-messages. For example, a participant in the intervention group said: “it helped me to stay strong and keep going everyday! It gave me great advice that was really helpful”. Control group participants also reported liking the text messaging, e.g.: “I liked getting random text messages from iDOVE. Even if it was just a little fun face or a text telling me to wear my seatbelt in cars. I liked just having that notification”. The intervention group explicitly wanted the program to be longer: “I love iDOVE! I wish it could text me everyday forever”; no control group participants requested a longer intervention, although one said “I loved the experience and would love to continue”. Only intervention participants requested greater tailoring and/or the ability to interact with a human: e.g. “The responses were very general – [I’d want] more personal messages” and “You should have somebody who response [sic]”.

Intervention effects on physical peer violence and depressive symptoms

Unadjusted median baseline and follow-up peer violence and mean baseline and follow-up depressive symptoms are presented in Table 3.

Mixed effects longitudinal regression models (see Table 3) suggest that mean intervention effects on past-two-week depressive symptoms were non-significant (albeit in the expected direction, namely, intervention participants showed greater reductions in mean BDI compared to control) at 8 weeks and 16 weeks. Among those with baseline moderate depressive symptoms (BDI>=20), however, intervention effects were borderline significant at 8-week follow-up (mean difference=−7.48, SE=4.11, p=0.07, d=0.37), with continued, albeit attenuated, differences at 16 week follow-up.

The median past-two-month CTS-2 was 2 at baseline, 3 at 8 weeks and 2 at 16 weeks. Much like models of mean BDI over time, there were no significant intervention effects on median outcomes over time. However, subgroup analysis showed significant intervention effects on median CTS at 8 weeks amongst those with higher baseline violence (CTS>=4, as per our a priori definition) with a difference in median CTS at 8 weeks=−7.29, SE=2.62, p=0.01, d=0.46); just as for depressive symptoms, this difference was attenuated at 16-week follow-up (see Table 3).

Patterns of daily mood effects among the intervention group

Latent Class Models of daily text-message responses (by definition, obtained only from the intervention group) suggested that a 3-class model was best supported by the data; 48% of participants fell in class 1, 35% in class 2, and 17% in class 3. Mean daily scores over time, by class, are presented in Figure 2. Mean difference in daily mood scores between classes first reached significance at day 7 (chi-squared=7.12, p=.02).

Regression models showed that compared to class 2, class 1 and 3 had significantly higher mean BDI scores at 8 and 16 week follow-up (controlling for baseline values); in other words, greatest reductions in BDI were amongst those in class 2 (beta=−8.65, SE=4.79, p=0.01). No significant differences between the classes were observed for CTS-2 scores, nor were there significant demographic differences between the classes.

DISCUSSION

This pilot randomized controlled trial of a novel, two-part, technology-augmented violence and depression prevention intervention for at-risk youth shows high feasibility and acceptability, and makes major contributions to the science regarding adaptive technology-based interventions for adolescents. Although the study was not powered for efficacy, it shows promising trends for the subgroup of adolescents with higher levels of violence and depression at baseline. It provides promising data for future interventions, by defining, through latent class analysis, an objective measure (daily mood, measured by text-messaging) that may allow for in-the-moment adaptation of the text-message intervention.

Our data on feasibility and acceptability are extremely strong. iDOVE achieved its target recruitment well ahead of schedule, with 86% of eligible youth participating in the study, and demonstrated excellent retention among both intervention and control groups, with 95% completing 8-week follow-up, and 91% completing 16-week follow-up. In-ED intervention duration was, on average, 22 minutes – an amount of time that is achievable, even in the busy ED setting. Future work could translate the in-person intervention into one delivered by remote therapists, or exclusively on the computer, to further advance the disseminability and feasibility of the intervention.

iDOVE’s study population reported high-risk characteristics (e.g., low socioeconomic status, high number of minority youth). These populations are traditionally difficult to engage in longitudinal preventive behavioral interventions (Bisgaier & Rhodes, 2011; Dicola, Gaydos, Druss, & Cummings, 2013; Owens et al., 2002) – particularly when initiated in a non-traditional setting such as the ED (D’Onofrio et al., 2012). Yet 96% of intervention participants responded to at least one automated daily text-message, with participants actively engaging (as demonstrated by responses to daily SMS) on a mean of 86% of days. A high percent of intervention participants “pulled” additional content, as well: 19% requested additional support, and those who did request extra support were likely to request >1 message. These numbers surpass those in our formative development phase (Ranney, Freeman, et al., 2016), as well as those reported by other researchers (Robinson et al., 2006). The high engagement may reflect iDOVE’s use of best practices in mHealth intervention development, including a strong theoretical basis and extensive formative development work (Broderick et al., 2014; Head, Noar, Iannarino, & Grant Harrington, 2013; Yardley, Morrison, Bradbury, & Muller, 2015).

Participants were likely engaging at a high rate because they found the intervention, subjectively, helpful: iDOVE received extremely strong satisfaction ratings, was deemed easy to use, and was qualitatively described as something they “loved”. Indeed, preliminary analyses of the violence and depressive symptoms – which would be the main outcomes in an adequately powered trial – show a promising decrease in median physical violence, and a trend toward reduced depressive symptoms, among youth with more elevated symptoms at baseline. Our findings of greater improvements in more symptomatic youth are consistent with past prevention and intervention literature. Efficacy in the overall groups may also have been masked by the small sample size and by the large degree of variability in participant symptoms. Alternatively, the intervention may only be effective in the most symptomatic youth. Future studies should systematically examine intervention effects in both high- and low-violence youth, and should use lower-intensity comparison interventions.

Interestingly, there did not appear to be qualitative or quantitative differences in the feasibility or acceptability of the intervention and control conditions. Though the study was not powered to detect differences in clinical outcomes, it is notable that adolescents seemed to like both interventions equally and completed them both. The relatively high satisfaction and engagement in the control group may be due to the regular text message contact (Heron & Smyth, 2010).

We were able to use latent class modeling to identify distinct subgroups of intervention participants who show similar patterns of daily text-message-based mood ratings over the course of the intervention. These text-message mood patterns can identify which intervention participants were less likely to improve with the iDOVE intervention. These patterns also identify the critical time – 7 days – at which the divergence between improvers and non-improvers is evident on daily text-message assessments. This analysis therefore shows that text-based daily mood ratings are a well-operationalized, feasible marker of participants’ need for additional, “adaptive” or stepped intervention (Almirall & Chronis-Tuscano, 2016; Nahum-Shani et al., 2016). In future interventions, levels of care could be adaptively changed or increased for youth showing these signs of non-responsiveness, based on daily mood assessments, at 7 days.

Finally, qualitative feedback among participants in this RCT, in our pilot (Ranney, Freeman, et al., 2016), and in others’ studies (Gibson & Cartwright, 2014; Hanley, 2012; Mohr, Cuijpers, & Lehman, 2011), suggest that more personalized text messaging with a human counselor would be helpful for intervention participants. Texting with a live counselor is highly acceptable to youth, as illustrated by the exponential growth of Crisis Text Line (Aseltine, James, Schilling, & Glanovsky, 2007; Lublin, 2015). Use of human support is also associated with improvements in outcomes in internet-based CBT prevention programs (Mohr et al., 2013; Morrison, Yardley, Powell, & Michie, 2012; van Ballegooijen et al., 2014). Potential avenues for future improvement of the study may therefore include enhanced day-to-day tailoring through incorporation of human interaction.

LIMITATIONS

The primary limitation of this study is that it was not adequately powered to show efficacy. A larger, fully powered randomized controlled trial should be conducted to formally evaluate iDOVE’s efficacy in reducing peer violence and depressive symptoms among at-risk youth. An additional limitation is that the study enrolled during a convenience sample of shifts, at a single ED, excluding non-English-speaking adolescents; these limitations may introduce bias or reduce generalizability. However, we systematically approached all youth during shifts, recruited a very high proportion of eligible youth, and excluded only 10% of youth from screening due to being non-English speaking. Additionally, it is possible that our high participation and follow-up rates were due to our compensation, rather than due to the inherent likability of the intervention; we note, however, that our compensation is on par with that of other ED-based studies with at-risk adolescents, and that participants were not compensated until after follow-up was completed.

CONCLUSION

In conclusion, this pilot RCT of a novel brief in-ED intervention and longitudinal text messaging intervention showed high feasibility, high acceptability, and is promising for reducing violence and depressive symptoms among more symptomatic youth. It also demonstrates a distinct cut-off of daily text message mood at seven days, which correlates with lack of symptomatic improvement at follow-up. Our results may inform future adaptive mobile health interventions for this at-risk group, who currently have limited access to preventive services. Future studies should also determine efficacy in a larger, fully powered sample.

Acknowledgements of Support:

John Patena, Joshua Freeman, Gerianne Connell, Faith Birnbaum, and Cassandra Duarte for their assistance in the design and implementation of the intervention.

This study was funded by NIMH K23 MH095866 (Ranney), K24 HD062645 (Guthrie), and K24DA027109 (Boyer).

Appendix

TABLE 1:

Participant demographics (Presented as N (%) unless otherwise noted)

Intervention (n=58) Control (n=58)
Age (Mean[SD]) 14.83 (1.23) 15.05 (1.13)
Gender: Female 34 (58.62) 34 (58.62)
School Year
Grades 6–8 16 (27.59) 10 (17.24)
Grades 9–12 37 (63.79) 48 (82.76)
College 2 (3.45) 0 (0.00)
Not in school 3 (5.17) 0 (0.00)
Ethnicity: Hispanic 19 (32.76) 23 (39.66)
Race: Non-White 29 (50.00) 30 (51.72)
Sexual Orientation: Straight 43 (74.14) 46 (79.31)
Socio-economic status: Low 36 (62.07) 42 (72.41)
Baseline Violence:
   CTS-2 (mean[SD]) 5.07 (6.72) 2.69 (2.64)
Baseline Depressive Symptoms:
   PHQ-9 Score (Mean[SD]) 9.43 (3.82) 8.78 (3.49)
   BDI-II Score (Mean[SD]) 13.6 (8.6) 13.4 (10.1)
Open in a new tab

Table 2:

Schedule of participant assessments

Construct Measure Citation Screening Baseline 8-week follow-up 16-week follow-up
Depressive Symptoms PHQ-9 Kroenke et a., 2015 X
BDI-2 Beck et al., 1996 X X X
Peer Violence CTS-2 Straus et al., 1996 X X X
Participant Satisfaction CSQ-8 Attkisson & Zwick, 1982 X
Demographics Ad Health GENIUSS K. Harris et al., 2003; Gender Identity in U.S. Surveillance Group, 2013 X
Qualitative Feedback X
Open in a new tab

Figure 1.

Figure 1

Open in a new tab

Flow chart of recruitment, enrollment and follow-up of participants.

Table 3.

Unadjusted Mean/Median Scores (SE) over time by Group, and Stratified by Baseline

Unadjusted Intervention Score Unadjusted Control Score Adjusted difference c p-value
BDI Score a
Baseline 13.6 (8.6) 13.4 (10.1) 0.15 (1.70) 0.93
8w 14.8(12.4) 15.0 (11.9) −0.62 (1.73) 0.72
16w 12.5(12.8) 11.7 (11.0) 0.35 (1.75) 0.84
BDI Score where baseline >=20 a N=16 N=15
Baseline 24.5 (4.2) 27.9 (9.2) −3.48(3.84) 0.37
8w 19.9 (12.1) 27.2 (11.5) −7.48 (4.11) 0.07
16w 21.9 (14.9) 25.2 (10.0) −4.04 (4.05) 0.32
CTS Score b
Baseline 3.0 (36.0) 2.0 (9.0) 1.00(0.84) 0.23
8w 2.5 (29.0) 3.0 (15.0) −1.00(0.95) 0.29
16w 2.0 (29.0) 1.0 (32.0) 0.00(0.71) 1.00
CTS Score where baseline>=4 b N=24 N=18
Baseline 8.0 (32.0) 6.5 (5.0) 1.00(1.66) 0.55
8w 4.0 (27.0) 4.0 (14.0) * −7.29(2.62) 0.01 *
16w 2.0 (29.0) 5.0 (32.0) −3.00(3.30) 0.37
Open in a new tab
a

Means (Standard Deviations) presented for BDI scores

b

Median and range presented for CTS scores

c

For BDI, this is a mean adjusted difference. For CTS outcomes, this is an adjusted difference in medians.

*

p<.05 for between-group difference

Figure 2.

Figure 2

Open in a new tab

Latent Class Analysis of Daily Mood

REFERENCES

  1. Almirall D, & Chronis-Tuscano A (2016). Adaptive Interventions in Child and Adolescent Mental Health. J Clin Child Adolesc Psychol, 45(4), 383–395. doi: 10.1080/15374416.2016.1152555 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Armanasco AA, Miller YD, Fjeldsoe BS, & Marshall AL (2017). Preventive Health Behavior Change Text Message Interventions: A Meta-analysis. Am J Prev Med doi: 10.1016/j.amepre.2016.10.042 [DOI] [PubMed] [Google Scholar]
  3. Aseltine RH Jr., James A, Schilling EA, & Glanovsky J (2007). Evaluating the SOS suicide prevention program: a replication and extension. BMC Public Health, 7, 161. doi: 10.1186/1471-2458-7-161 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Atienza A, & Patrick K (2011). Mobile health: The killer app for cyberinfrastructure and consumer health. Am J Prev Med, 40(5 Suppl 2), S151–153. [DOI] [PubMed] [Google Scholar]
  5. Attkisson CC, & Zwick R (1982). The Client Satisfaction Questionnaire: Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann, 5(3), 233–237. [DOI] [PubMed] [Google Scholar]
  6. Beck AT, Brown G, & Steer RA (1996). Beck Depression Inventory II Manual San Antonio, TX: The Psychological Corporation. [Google Scholar]
  7. Bisgaier J, & Rhodes KV (2011). Auditing access to specialty care for children with public insurance. N Engl J Med, 364(24), 2324–2333. doi: 10.1056/NEJMsa1013285 [DOI] [PubMed] [Google Scholar]
  8. Bohnert KM, Walton MA, Ranney M, Bonar EE, Blow FC, Zimmerman MA, … Cunningham RM (2015). Understanding the service needs of assault-injured, drug-using youth presenting for care in an urban Emergency Department. Addict Behav, 41, 97–105. doi: 10.1016/j.addbeh.2014.09.019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Broderick J, Devine T, Langhans E, Lemerise AJ, Lier S, & Harris L (2014). Designing Health Literate Apps Retrieved from http://www.iom.edu/Global/Perspectives/2014/~/media/Files/Perspectives-Files/2014/Discussion-Papers/BPH-HealthLiterateApps.pdf
  10. Calvete E, & Cardenoso O (2005). Gender differences in cognitive vulnerability to depression and behavior problems in adolescents. J Abnorm Child Psychol, 33(2), 179–192. [DOI] [PubMed] [Google Scholar]
  11. Cunningham RM, Chermack ST, Zimmerman MA, Shope JT, Bingham CR, Blow FC, & Walton MA (2012). Brief motivational interviewing intervention for peer violence and alcohol use in teens: one-year follow-up. Pediatrics, 129(6), 1083–1090. doi: 10.1542/peds.2011-3419 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. D’Onofrio G, Fiellin DA, Pantalon MV, Chawarski MC, Owens PH, Degutis LC, … O’Connor PG (2012). A brief intervention reduces hazardous and harmful drinking in emergency department patients. Ann Emerg Med, 60(2), 181–192. doi: 10.1016/j.annemergmed.2012.02.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Dicola LA, Gaydos LM, Druss BG, & Cummings JR (2013). Health insurance and treatment of adolescents with co-occurring major depression and substance use disorders. J Am Acad Child Adolesc Psychiatry, 52(9), 953–960. doi: 10.1016/j.jaac.2013.06.012 [DOI] [PubMed] [Google Scholar]
  14. Gender Identity in U.S. Surveillance Group. (2013). Gender-Related Measures Overview Retrieved from Los Angeles, CA: http://williamsinstitute.law.ucla.edu/wp-content/uploads/GenIUSS-Gender-related-Question-Overview.pdf
  15. Gibson K, & Cartwright C (2014). Young people’s experiences of mobile phone text counselling: Balancing connection and control. Children and Youth Services Review, 43, 96–104. doi: 10.1016/j.childyouth.2014.05.010 [DOI] [Google Scholar]
  16. Hall AK, Cole-Lewis H, & Bernhardt JM (2015). Mobile text messaging for health: a systematic review of reviews. Annu Rev Public Health, 36, 393–415. doi: 10.1146/annurev-publhealth-031914-122855 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Hamilton JL, Kleiman EM, Rubenstein LM, Stange JP, Flynn M, Abramson LY, & Alloy LB (2016). Deficits in Emotional Clarity and Vulnerability to Peer Victimization and Internalizing Symptoms Among Early Adolescents. J Youth Adolesc, 45(1), 183–194. doi: 10.1007/s10964-015-0260-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Hanley T (2012). Understanding the online therapeutic alliance through the eyes of adolescent service users. Counselling and Psychotherapy Research, 12(1), 35–43. [Google Scholar]
  19. Harris K, Florey F, Tabor J, Bearman P, Jones J, & Udry J (2003). The national longitudinal study of adolescent health: Research design [WWW document] Retrieved from http://www.cpc.unc.edu/projects/addhealth/design.
  20. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, & Conde JG (2009). Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform, 42(2), 377–381. doi: 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Head KJ, Noar SM, Iannarino NT, & Grant Harrington N (2013). Efficacy of text messaging-based interventions for health promotion: A meta-analysis. Social Science & Medicine, 97(0), 41–48. doi: 10.1016/j.socscimed.2013.08.003 [DOI] [PubMed] [Google Scholar]
  22. Heron KE, & Smyth JM (2010). Ecological momentary interventions: Incorporating mobile technology into psychosocial and health behaviour treatments. Br J Health Psychol, 15(Pt 1), 1–39. doi: 10.1348/135910709X466063 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kroenke K, Spitzer RL, & Williams JB (2001). The PHQ-9: validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613. doi:jgi01114 [pii] [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Lenhart A (2015). Teens, Social Media & Technology Overview 2015 Retrieved from http://www.pewinternet.org/2015/04/09/teens-social-media-technology-2015/
  25. Lublin N (2015). Helping texters, saving lives… With big data. Paper presented at the White House Commemoration of World Mental Health Day, Washington, D.C. [Google Scholar]
  26. Malik S, Sorenson SB, & Aneshensel CS (1997). Community and dating violence among adolescents: Perpetration and victimization. J Adolesc Health, 21(5), 291–302. doi: 10.1016/S1054-139X(97)00143-2 [doi] [DOI] [PubMed] [Google Scholar]
  27. Mason M, Ola B, Zaharakis N, & Zhang J (2015). Text messaging interventions for adolescent and young adult substance use: a meta-analysis. Prev Sci, 16(2), 181–188. doi: 10.1007/s11121-014-0498-7 [DOI] [PubMed] [Google Scholar]
  28. Merikangas KR, He J, Burstein M, Swanson SA, Avenevoli S, Cui L, … Swendsen J (2010). Lifetime prevalence of mental disorders in U.S. adolescents: Results from the National Comorbidity Survey Replication-Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry, 49(10), 980–989. doi:DOI: 10.1016/j.jaac.2010.05.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Mohr DC, Cuijpers P, & Lehman K (2011). Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res, 13(1), e30. doi: 10.2196/jmir.1602 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Mohr DC, Duffecy J, Ho J, Kwasny M, Cai X, Burns MN, & Begale M (2013). A randomized controlled trial evaluating a manualized TeleCoaching protocol for improving adherence to a web-based intervention for the treatment of depression. PLoS One, 8(8), e70086. doi: 10.1371/journal.pone.0070086 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Moriarty AS, Gilbody S, McMillan D, & Manea L (2015). Screening and case finding for major depressive disorder using the Patient Health Questionnaire (PHQ-9): a meta-analysis. Gen Hosp Psychiatry, 37(6). doi: 10.1016/j.genhosppsych.2015.06.012 [DOI] [PubMed] [Google Scholar]
  32. Morrison LG, Yardley L, Powell J, & Michie S (2012). What design features are used in effective e-health interventions? A review using techniques from Critical Interpretive Synthesis. Telemed J E Health, 18(2), 137–144. doi: 10.1089/tmj.2011.0062 [DOI] [PubMed] [Google Scholar]
  33. Moyers TB, Martin T, Manuel JK, Hendrickson SML, & Miller WR (2005). Assessing competence in the use of motivational interviewing. J Subst Abuse Treat, 28(1), 19–26. doi: 10.1016/j.jsat.2004.11.001 [DOI] [PubMed] [Google Scholar]
  34. Nahum-Shani I, Smith SN, Spring BJ, Collins LM, Witkiewitz K, Tewari A, & Murphy SA (2016). Just-in-Time Adaptive Interventions (JITAIs) in Mobile Health: Key Components and Design Principles for Ongoing Health Behavior Support. Ann Behav Med doi: 10.1007/s12160-016-9830-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Osman A, Barrios FX, Gutierrez PM, Williams JE, & Bailey J (2008). Psychometric properties of the Beck Depression Inventory-II in nonclinical adolescent samples. J Clin Psychol, 64(1), 83–102. doi: 10.1002/jclp.20433 [DOI] [PubMed] [Google Scholar]
  36. Osman A, Kopper BA, Barrios F, Gutierrez PM, & Bagge CL (2004). Reliability and validity of the Beck depression inventory--II with adolescent psychiatric inpatients. Psychol Assess, 16(2), 120–132. doi: 10.1037/1040-3590.16.2.120 [DOI] [PubMed] [Google Scholar]
  37. Owens PL, Hoagwood K, Horwitz SM, Leaf PJ, Poduska JM, Kellam SG, & Ialongo NS (2002). Barriers to children’s mental health services. J Am Acad Child Adolesc Psychiatry, 41(6), 731–738. doi: 10.1097/00004583-200206000-00013 [DOI] [PubMed] [Google Scholar]
  38. Pailler ME, Cronholm PF, Barg FK, Wintersteen MB, Diamond GS, & Fein JA (2009). Patients’ and caregivers’ beliefs about depression screening and referral in the emergency department. Pediatr Emerg Care, 25(11), 721–727. [DOI] [PubMed] [Google Scholar]
  39. Ranney ML, Choo EK, Cunningham RM, Spirito A, Thorsen M, Mello MJ, & Morrow K (2014). Acceptability, language, and structure of text message-based behavioral interventions for high-risk adolescent females: a qualitative study. J Adolesc Health, 55(1), 33–40. doi: 10.1016/j.jadohealth.2013.12.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Ranney ML, Choo EK, Spirito A, & Mello MJ (2013). Adolescents’ preference for technology-based emergency department behavioral interventions: does it depend on risky behaviors? Pediatr Emerg Care, 29(4), 475–481. doi: 10.1097/PEC.0b013e31828a322f [DOI] [PubMed] [Google Scholar]
  41. Ranney ML, Freeman JR, Connell G, Spirito A, Boyer E, Walton M, … Cunningham RM (2016). A Depression Prevention Intervention for Adolescents in the Emergency Department. J Adolesc Health, 59(4). doi: 10.1016/j.jadohealth.2016.04.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Ranney ML, Goldstick J, Eisman A, Carter PM, Walton M, & Cunningham R (2017). Effects of a brief ED-based alcohol and violence intervention on depressive symptoms. Gen Hosp Psychiatry, IN PRESS [DOI] [PMC free article] [PubMed]
  43. Ranney ML, Patena JV, Nugent N, Spirito A, Boyer E, Zatzick D, & Cunningham R (2016). PTSD, cyberbullying and peer violence: prevalence and correlates among adolescent emergency department patients. General Hospital Psychiatry, 39, 32–38. doi: 10.1016/j.genhosppsych.2015.12.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Ranney ML, Thorsen M, Patena JP, Cunningham R, Boyer E, Walton M, … K, M. (2015). ‘You need to get them where they feel it’: Conflicting Perspectives on How to Maximize the Structure of Text-Message Psychological Interventions for Adolescents. Paper presented at the Proceedings of the 48th Annual Hawaii International Conference on System Sciences. http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=7070206&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D7070206 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Ranney ML, Walton M, Whiteside L, Epstein-Ngo Q, Patton R, Chermack S, … Cunningham RM (2013). Correlates of depressive symptoms among at-risk youth presenting to the emergency department. Gen Hosp Psychiatry, 35(5), 537–544. doi: 10.1016/j.genhosppsych.2013.05.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Riese A, Mello MJ, Baird J, Steele DW, & Ranney ML (2015). Prompting discussions of youth violence using electronic previsit questionnaires in primary care: a cluster randomized trial. Acad Pediatr, 15(3), 345–352. doi: 10.1016/j.acap.2015.01.005 [DOI] [PubMed] [Google Scholar]
  47. Robinson S, Perkins S, Bauer S, Hammond N, Treasure J, & Schmidt U (2006). Aftercare intervention through text messaging in the treatment of bulimia nervosa--feasibility pilot. Int J Eat Disord, 39(8), 633–638. doi: 10.1002/eat.20272 [DOI] [PubMed] [Google Scholar]
  48. Saluja G, Iachan R, Scheidt PC, Overpeck MD, Sun W, & Giedd JN (2004). Prevalence of and risk factors for depressive symptoms among young adolescents. Arch Pediatr Adolesc Med, 158(8), 760–765. [DOI] [PubMed] [Google Scholar]
  49. Seiffge-Krenke I (2000). Causal links between stressful events, coping style, and adolescent symptomatology. J Adolesc, 23(6), 675–691. doi: 10.1006/jado.2000.0352 [DOI] [PubMed] [Google Scholar]
  50. Shadur JM, & Lejuez CW (2015). Adolescent Substance Use and Comorbid Psychopathology: Emotion Regulation Deficits as a Transdiagnostic Risk Factor. Curr Addict Rep, 2(4), 354–363. doi: 10.1007/s40429-015-0070-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Silk JS, Steinberg L, & Morris AS (2003). Adolescents’ emotion regulation in daily life: Links to depressive symptoms and problem behavior. Child Dev, 74(6), 1869–1880. [DOI] [PubMed] [Google Scholar]
  52. Spirito A, Boergers J, Donaldson D, Bishop D, & Lewander W (2002). An intervention trial to improve adherence to community treatment by adolescents after a suicide attempt. J Am Acad Child Adolesc Psychiatry, 41(4), 435–442. [DOI] [PubMed] [Google Scholar]
  53. Straus MA, Hamby SL, Boney-McCoy S, & Sugarman DB (1996). The revised conflict tactics scales (CTS2): Development and preliminary psychometric data. J Fam Issues, 17(3), 283–316. [Google Scholar]
  54. Substance Abuse and Mental Health Services Administration. Behavioral Health Barometer: United States, 2015. HHS Publication No. SMA–16–Baro–2015 Rockville, MD: Substance Abuse and Mental Health Services Administration, 2015. [Google Scholar]
  55. Thorsen MM, Patena JV, Morrow Guthrie K, Spirito A, & Ranney ML (2016). Using High-Risk Adolescents’ Voices to Develop a Comprehensible Cognitive Behavioral Therapy-Based Text-Message Program. Behav Med, 1–11. doi: 10.1080/08964289.2016.1223597 [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. van Ballegooijen W, Cuijpers P, van Straten A, Karyotaki E, Andersson G, Smit JH, & Riper H (2014). Adherence to Internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis. PLoS One, 9(7), e100674. doi: 10.1371/journal.pone.0100674 [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Walton M, Chermack S, Shope J, Bingham C, Zimmerman M, Blow F, & Cunningham R (2010). Effects of a brief intervention for reducing violence and alcohol misuse among adolescents: A randomized trial. Jama, 304(5), 527–535. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Wilson KM, & Klein JD (2000). Adolescents who use the emergency department as their usual source of care. Arch Pediatr Adolesc Med, 154(4), 361–365. doi: 10.1001/archpedi.154.4.361 [DOI] [PubMed] [Google Scholar]
  59. Yardley L, Morrison L, Bradbury K, & Muller I (2015). The person-based approach to intervention development: application to digital health-related behavior change interventions. J Med Internet Res, 17(1), e30. doi: 10.2196/jmir.4055 [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Young JE, & Beck AT (1980). Cognitive Therapy Scale Rating Manual Philadelphia, PA: Center for Cognitive Therapy. [Google Scholar]

RESOURCES