“You need to get them where they feel it”: Conflicting Perspectives on How to Maximize the Structure of Text-Message Psychological Interventions for Adolescents

Abstract

Mobile psychological interventions are of growing interest, particularly for populations with little access to traditional mental health services. Optimum structural components of these interventions are unknown. In this study, twenty-one adolescents (age 13-17) with past two week depressive symptoms were recruited from the emergency department to participate in a semi-structured interview, to inform development of a text-message-based depression prevention intervention. Teens expressed conflict about intervention structure. Although trust and reliability were essential to sustain engagement, teens disagreed about how to best maintain reliability; whether the program should be “pushed” or “pulled”; and what the ideal degree of human interaction would be. These findings highlight the challenges in automating psychological interventions that are normally delivered face-to-face. Data indicate a broad desire for developing tailoring methods for system design (duration, frequency, and level of interactivity). The paper closes with thoughts about potential solutions to these structural issues for mobile psychological interventions.

1. Introduction

Mobile health, or “mHealth”, interventions are of growing interest in both the research and business world. Despite increasing evidence that mHealth interventions are feasible, few existing interventions have been evaluated to the point that their efficacy is known. Few interventions maximize the technological capabilities inherent in modern smartphones, which permit interventions using text, graphics, audio, and video. Few interventions are based on evidence or theory. And few are consistently adopted and used by the intended recipient.[1]

Text-message (short message service, SMS) mHealth interventions have the most evidence supporting their efficacy. Despite the ubiquity of SMS capability and patients' high level of interest in SMS interventions, few SMS interventions are used outside of studies.[2] One reason for lack of engagement and real-world efficacy may be the lack of personalization or tailoring.[3]

The effectiveness of individual tailoring is well-established for print and computer health interventions. [4] Tailoring is generally described as delivering an individualized message, based not just on population-level characteristics (e.g. ethnicity, gender) but also based on individual psychological profiles (e.g. degree of self-control).[5] Extensive literature guides the development of behavioral intervention messaging according to personality types and psychometrics.[6] These variables' applicability to mHealth behavioral interventions are, however, not yet clear. For example, recent studies suggest that preferences for SMS content are for the most part independent of traditional personality measures.[7]

Importantly, prior literature places tailoring in the context of content rather than structure. One of the unique aspects of mobile interventions is the infinite forms which tailoring can adopt. Tailoring need not be limited to content alone; we can also tailor the structure – namely, the time, duration, frequency, and type of interaction. Tailoring is likely to increase mHealth interventions' long-term acceptability; and without effective tailoring, well-intentioned interventions may rapidly become annoying pests. To better define the ways in which tailoring can be applied to mHealth interventions, inclusion of the patient/participant voice is critical.[8]

2. Mobile psychological interventions in at-risk populations

Mental health is one of the most common topics of mHealth interventions.[1] mHealth has been advocated as being a potential solution to isolation, stigma, and limited mental health workforce, both for patients with pre-syndromal symptoms of mental illness and with serious mental illness.[9] The degree to which an automated program can replace a real-time counselor, however, remains controversial. The issue is particularly acute for mental health, where the structure of a behavioral intervention is rarely as formulaic as that of the motivational interviewing that is used for smoking cessation and other successful SMS-based interventions.[10] The idea of structural tailoring is particularly salient, therefore, for guiding the burgeoning field of mobile psychological interventions.[11, 12]

Emergency departments (ED) are often the primary source of care for patients at high risk of mental illness. Not only are sequelae of mental illness the fastest growing reason for an ED visit,[13] but a large proportion of ED patients have undiagnosed mental illness[14]. The substantial barriers preventing access to resources for patients with serious mental illness are even greater for those with less serious conditions.[15] As part of our research into expanding mental health care, we have found that the vast majority of ED patients are interested in technological psychological interventions [16] to augment services begun in emergency department settings.

Few studies have examined the optimum structure of a mobile intervention for such patients.[11] As a first step toward so doing, this study aimed to generate hypotheses about essential structural variables for a mobile psychological interventions for at-risk adolescent patients. The recruitment setting was the emergency department, for the reasons cited above. As this was an exploratory, hypothesis-generating study, qualitative methods were used.[8]

3. Methodology

3.1. Study design and recruitment

This study recruited adolescents age 13-17 from a Level I trauma pediatric ED. The study site is the primary children's hospital of an urban city in the Northeast, serving 50,000 pediatric patients per year with a diverse population (40% Hispanic, 25% African-American, 40% publically insured). A consecutive sample of all adolescents age 13-17 presenting to the ED with any chief complaint were screened for participation. Trained research assistants recruited all days of the week from 9:00am to 10:00pm on a convenience sample of shifts weighted by patient volume between August 2013 and May 2014. Screening inclusion criteria included English-speaking, having a parent/guardian present to consent, and medically, physically, and mentally able to consent. Screening exclusion criteria included presenting with a chief complaint of suicidality, psychosis, sexual assault, or child abuse; or being in police or child protective services' custody. Verbal parent/guardian consent and verbal adolescent assent were obtained before participation in the screening survey. Participants completed the screening survey on a touch-screen tablet and were offered a small gift valued at US$2 (e.g. gum, nail polish) on completion of the survey.

Participants were eligible for interviews if they reported: 1) past two week depressive symptoms (using the Patient Health Questionnaire-9 [PHQ-9] score ≥ 5) and 2) past year peer violence (using the Conflict Tactics Scale 2^nd edition [CTS-2] score ≥ 1). The PHQ-9 is a validated 9-item measure which corresponds to clinical criteria for depression.[17] This study chose a cutoff score of PHQ-9 ≥ 5 to represent adolescents with depressive symptoms since the larger goal of the study is to develop a depression prevention intervention.[18] The revised version of the CTS-2, a 14-item construct to measure past year peer violence,[19] assesses experiences with peer violence, defined using a cutoff score of ≥ 1. A history of peer violence is a well-established predictor of depressive disorders.[20, 21]

If eligible for the interview, participants and their parent/guardian completed a written consent and assent process. Interviews were conducted at a time and place of the participant's choice (i.e. at study site or in the community). Participants were compensated US$25 through a gift card for the interview. Study procedures were approved by the participating hospital's Institutional Review Board.

3.2. Interview protocol

Interviews were conducted using a semi-structured interview guide conducted by a research assistant trained in qualitative interview facilitation. The majority of interviews (n=20) were conducted face to face, with the remainder (n=1) via telephone, and lasted between 60–90 minutes. All interviews were audio recorded and transcribed verbatim.

The overall goal of the interviews was to inform design of a longitudinal text-message-based depression prevention intervention. The semi-structured interview was specifically designed to obtain feedback regarding: 1) current use of mobile phones and texting, 2) current strategies for managing emotions, 3) potential in-ED intervention content and format, and 4) potential text-message intervention content and format. The interview guide was developed by an emergency physician with expertise in technology-based ED preventive interventions (MR), a child/adolescent psychologist (AS), and a psychologist with expertise in behavioral health and qualitative methodology (KM).

3.3. Data analysis

Thematic analysis was used to create an initial coding schema developed from a previous formative research study [22] and from the core interview topics (e.g. preference for tailoring, conflicting views on interactivity, and content of messages). The Principal Investigator (MR) reviewed initial transcripts to develop the coding structure further based on the data collected. The coding structure was iteratively developed by the Principal Investigator and two research assistants (MT, JP) until the team found no other themes for the purposes of the study goals. An audit trail was maintained to track coding decisions and analysis of the process. Each transcript was coded by at least two members of the research team independently, and all transcripts were double-coded to ensure accurate capturing.

Agreed-upon codes were entered into NVivo 10 (QSR International Pty Ltd., Doncaster, Victoria, Australia) a qualitative computer software program which helps to organize and link codes within electronic interview transcripts. Final themes were developed based on the research team collectively reviewing codes. Thematic saturation on structural variables was reached after 21 interviews.

4. Results

Three coherent thematic areas emerged regarding critical structural decisions for an SMS-based psychological intervention for adolescents.

4.1. Participant characteristics

Twenty-one adolescents participated. The sample had a mean age of 15.3 (SD 1.2) and 43% were male (n=9). Approximately half were non-white, and two-thirds received public assistance (n=14). Although the majority report having a regular source of primary care (n=19) only a third report receiving mental health services in the past six months (n=7). Participants' mean PHQ-9 score was 11.3 (SD 6.5) with little variance between males and females. Table 1 describes overall participant demographics; Table 2 describes individual participant characteristics to give context to the qualitative analysis.

Table 1. Participant demographics (N=21).

	N (%)
Age (mean, SD)	15.3 (1.2)
Male	9 (42.9)
White	11 (52.4)
Hispanic	9 (42.9)
Identifies as straight	18 (85.7)
Receives public assistance	14 (66.7)
Lives with biological parent(s)	19 (90.5)
Has a regular source of care	19 (90.5)
Received mental health service	7 (33.3)
Has current access to a cell phone	21 (100)
PHQ-9 score
Total [range: 5-27] (mean, SD)	11.3 (6.5)
Males [range: 5-25] (mean, SD)	10.8 (6.6)
Females [range: 5-27] (mean, SD)	11.8 (6.7)
CTS-2 score
Total [range: 1-36] (mean, SD)	11.0 (9.5)
Males [range: 1-36] (mean, SD)	14.6 (11.8)
Females [range: 2-22] (mean, SD)	8.4 (6.8)

SD=standard deviation

Table 2. Participant identifying characteristics.

	Age	Gender	PHQ-9
#1	16	M	25
#2	14	M	7
#3	13	F	6
#4	14	F	18
#5	15	F	11
#6	16	F	17
#7	14	M	10
#8	14	F	6
#9	15	M	5
#10	16	M	5
#11	16	F	15
#12	14	F	10
#13	15	M	11
#14	17	F	5
#15	16	F	6
#16	17	M	15
#17	16	M	14
#18	17	F	27
#19	16	M	5
#20	14	F	6
#21	16	F	14

PHQ-9 1–4 “minimal depressive symptoms”

PHQ-9 5–9 “mild depressive symptoms”

PHQ-9 10–14 “moderate depressive symptoms”

PHQ-9 15–19 “moderately severe depressive symptoms”

PHQ-9 20–27 “severe depressive symptoms”[23]

4.2. Balancing reliability and variety

An oft-debated issue within the realm of mHealth is how to deliver content without imposing or making participants disengage. Participants universally felt that variety in both content and schedule was critical for continued engagement in the program: “If it was, like, the same thing every day, I would probably get annoyed after a while” (#15). The frequency and type of delivery of messages was a critical component of maintaining this interest.

In describing what would keep their interest, participants articulated a need for establishing a “trust factor” (#9), saying that if the program wasn't structurally reliable, they would dismiss it. A critical element of this reliability and trustworthiness was regular delivery of the messages, “just to help them remember that you're there” (#12).

The participants identified a wide range of thresholds at which “reliability” would cross over from demonstrating trustworthiness into being annoying or an intrusion. As explained by one adolescent:

“a lotta people kinda get mad if something's done daily cuz they kinda just – they're used to seeing it, and it just kinda, like, passes over them. If it's more of an infrequent thing, but it's still, you know, it happens enough… then it comes back up as just that nice thing that is just there” (#16)

What that “enough” consisted of differed wildly. While one participant may feel that “daily [text messages] might be a little too much” (#19), requesting weekly interaction, another would say that “I'm a very active texter when it comes to these things” and ask for multiple messages a day (#5).

Indeed, a number of participants recognized that each teenager's need was different. They also acknowledged their needs might shift over time: “some people it might be an everyday thing where they feel upset, or some people might be a few days a week and then like, the weekend they're fine” (#3).

The best way to address these disparate opinions may be the ability to target message frequency according to participants' individual characteristics. Many participants acknowledged, however, that they weren't sure what the best frequency would be for them or for other teenagers: “I don't know, because there are a lot of different personalities” (#20). Indeed, there was a strong tension between participants identifying what they thought might work for them, and what their friends might want; they often urged the interviewer to ignore their own opinion, since “other teens” would likely feel differently. Moreover, there was no clear trend in preferences regarding frequency and scheduling according to classic population-level variables such as sex, age, degree of depressive symptoms, or prior mental health exposure. Neither strong social supports nor comfort with texting predicted preferences, either.

Surprisingly, a few participants spontaneously suggested that they would want to provide feedback to the program about its frequency and content, similar to how one would provide feedback to, say, Netflix, about the appropriateness of its DVD recommendations. This type of sophisticated algorithm may be a solution to the structural dilemma articulated above, even for participants without smartphones. However, as we all know, Pandora is not perfect.

4.3. Push versus pull

Teens also articulated vastly different opinions, often within the same interview, as to whether text messages should be “pushed” (delivered according to a pre-determined schedule), “pulled” (delivered on request), or a combination of the two. For instance, one participant said:

“it would just be nice, you know, just to get a message, just sayin' there's people out there. [RA: So you would want us – the program – to text you?] Yeah… Like, if, like… you guys say “how you're feeling”. Like, I'm feeling angry. And then just be given, you know, somethin' about that.” (#16)

But he then immediately contradicted himself: “Uh, no. It's gotta be a thing where [teens] can, like, initiate it by saying ‘I feel angry’…” and went on to say that he may not ever want to receive an unsolicited message from the program.

Other participants similarly contradicted themselves during the course of the interview as to how they would best prefer to have the intervention initiated. They explicitly acknowledged that although they might be annoyed by “pushed” messages, they recognized that they needed them, and that pushed messaging would help them when they didn't even know they needed help. Indeed, some participants suggested a “reminder”: “Maybe sometimes they'll forget, and they're, like, hanging out with their friends… [send them] a reminder” (#8). They also expressed concern about the ability to re-initiate an interaction if they needed more support: “if you already sent a text, and then some hours come by, and you want to text again, like would you be able to text back?… I think that will help” (#4).

Some participants suggested that one solution to this tension would be to identify participants' in-the-moment needs. Yet others were worried about how accurate their self-assessments would be: “you're always gonna feel like the other person's judging you if they rate your anger. You know what I mean?” (#21). Indeed, one participant expressed concern that anything requiring a response on his part would be too much: “Because I know that's one thing that does not help a stressful day is pulling information from someone” (#9).

One potential solution to this tension, as suggested by one not particularly technologically-savvy participant, would be a system which tracks one's usage patterns or texting language and determines when a participant shows signs of anger, sadness, or disengagement: “I don't know how possible or easy this is, but try to [have the system] imagine or understand what the person might be going through, and what might be a useful text message” (#19).

4.4. A robot, a pocket counselor, or both?

The third and most overarching major structural theme was participants' conflicting desires about the most appropriate level of interactivity and personalization of an SMS-based program. This theme interwove with the two prior themes, but extended far beyond them. Although no participant expressly said that they wanted NO interactivity, many expressed concerns about the degree of interactivity that would be appropriate, with a range from those who desired one pre-programmed message each week to those who explicitly asked for the program to be a “pocket counselor.” Participants in the latter camp explicitly acknowledged the tension between traditional therapy and an automated program:

“I don't know if there's this particular extension to the program, but I thought of, like, uh—if maybe they could go and actually talk [text] to a person about how they're feeling. Unless that's, like, too much like therapy…” (#19)

Again, many participants explicitly differentiated between their own needs/desires in a program and that of “everyone else.” They would suggest that the value in the “pocket counselor” approach was that it could be eminently personalized:

“Like, if you wanna help as many people as you can, you gotta keep it a little more personal. Cuz if it's not personal, people will lose interest. You need to get them where they feel it” (#21)

At the same time, they recognized the technical and personnel limitations inherent to an automated SMS-based depression prevention program: “I don't know about the logistics of being able to do this…” (#9). Epitomizing this conflict is the following quote, in which a participant debates to herself whether the automated nature of the program would bother her:

“I kind of feel like it's a robot. Like, I'm talking to myself… I can-if I never meet this person I'm texting, I'm basically texting myself, you know?…. But I don't think you should tell people it's a robot. Some people might actually care that it's – that it's a real person… I think I actually prefer it [though].” (#21)

Some participants suggested that their needs for more intensive interactions with the program could shift over time; the ability to have greater interactivity “on demand” was articulated by multiple participants, across the spectrum of demographic characteristics. Surprisingly, some of the participants with the lowest levels of depressive symptoms were most likely to request a “pocket counselor” type of program.

5. Discussion

This study highlights the challenges in automating psychological interventions that are normally delivered face-to-face. Our data indicate a broad desire for developing structural tailoring methods for mobile psychological interventions. Such like pervasive systems design not just for content,[24] but also for structure –even to the point of approximating an office visit to a real therapist.

Our observations highlight the necessity of extending the classic continuum in intervention content – from generic, non-assessment-based messages (like Text4Baby)[25] to highly individualized messages based on one-to-one assessments (like CrisisText)[26] – into intervention structure.[5] Certain topics (like, say, smoking cessation) may have a universal delivery mechanism that needs little structural tailoring. Alternatively, other topics, like mental health interventions, may demand that the mechanisms of intervention delivery be exquisitely personalized. Although a text-message intervention may facilitate such nuanced communication, it is likely that it cannot substitute for this high degree of personalization.

Realizing this desire for personalized tailoring will require an ability to identify a participant's momentary need for a certain type of message, to integrate participants' prior experiences and responses when providing content, and to respond with a unique intervention content and structure. The technological delivery mechanism may need to be re-conceptualized. Whether the complexity inherent in this degree of adaptive structural tailoring negates the potential value of an automated program remains to be determined. Three potential approaches to such tailoring are outlined below.

5.1. Personalization of structure based on predefined variables

One common mHealth approach to structural tailoring is for the user to self-determine some aspects of personalization or tailoring. Typically, this involves selecting the time that they prefer message delivery or permitting the user to “opt-out” of a message because it is inappropriately timed. Our participants felt, however, that a user would not be able to accurately specify some structural components. A growing literature from mHealth weight loss interventions demonstrates a tension between both what adolescents say they want, what they actually respond to, and what results in effective weight loss.[27, 28] Our study suggests similar cautions regarding mobile psychological interventions for adolescents.

Another approach would be to use the same technique by which content is tailored: to personalize the structure passively, based on pre-identified variables. Our study's findings, however, raise the question of which pre-set variables would most effectively guide structural tailoring. In this study, contradictions about frequency, initiation, and degree of interactivity were consistently articulated across the spectrum of potential variables for targeted messages (age, gender, level of depressive symptoms, prior exposure to mental health system). Preliminary analyses suggest that level of social support and preexisting coping skills are also independent of structural preferences. It would therefore be difficult to personalize the structure of an intervention based on common user characteristics.

From the point of view of intervention development, the lack of correlation between objectively measured mental illness symptoms and structural preferences was particularly surprising. As illustrated by the quotes, these findings cannot be explained by (for instance) anhedonia or isolation of the more-depressed participants.

As-yet-unidentified latent variables may therefore better predict the most engaging and acceptable structural elements for each individual participant. Further research would be required to define these latent variables.

5.2. An alternative approach to structural personalization of a mobile intervention

An alternative approach would be to re-insert computer-mediated human-to-human interaction into what is an increasingly automated (purely human-to-computer interaction) field. Indeed, a recent meta-analysis of web-based mental health interventions found that one of the primary predictors of engagement and efficacy was frequent computer-mediated interactions with a counselor.[29] Our study's findings extend this meta-analysis into mobile psychological interventions. Our data clearly shows that for many teens, it is this human-to-human interaction that is desired.

However, requiring a human being on the other end of the text-message intervention may negate the value of automation. Ideally, a mobile psychological intervention should mimic, say, Amazon's ability to identify what you should consume based on prior and current searches and purchases.[30] It should deliver content that reflects not just static personality and demographic characteristics, but also real-time assessments of risk. Commercial recommender systems like Amazon's are, however, often excessively narrow or skewed, and reflect a single structural mode of communication with the recipient. The question then becomes, both from a behavioral intervention as well as programming standpoint: how do we maximally tailor a structure for our patients based on predetermined algorithms? Or is there a universal structure that can mimic human-like interactivity?

Having a computer program mimic a live clinician's ability to modify a therapeutic relationship represents a considerable programming challenge. One solution might be to develop individual packets of code that seamlessly integrate. Selection by the recipient of their momentary state (or passive identification of their momentary state, using “reality mining”)[31, 32] could guide the program in terms of selecting the packet— with its embedded programming, interventions, and responsiveness.

Whether participants would actually access such personalized content is unknown. A personalized interaction may give them the “best” care if available and used; a purely automated message may be the easiest to deliver; but where is the “sweet spot” in the middle?[33] And how does one define this sweet spot for each individual user? How does this differ for preventive interventions, as described in this analysis, versus for treatment or behavior change interventions?

Mobile psychological interventions also have a unique risk. A poorly written protocol would, at best, decrease in engagement in the program.[34, 35] At worst, it could miss the underlying context and actually cause harm.[22, 27] A hybrid between automated (computer-to-human) and mediated (human-to-human) interactions may be an acceptable compromise.

5.3. The need for cross-talk

The development of new structural interventions will require multiple long-term projects that require collaboration between behavioral scientists, computer scientists, and engineers. Unfortunately, most interest in this line of work appears to reside with “start-up” computing companies whose strategic vision is highly mutable, a characteristic that blunts engagement in longitudinal research. Similarly, most computing consulting firms are simply too expensive to be funded under federal research budgets. A need exists for a group of programmers with behavioral science backgrounds who can provide longitudinal expertise at a reasonable cost.

One response to this issue would be the creation of federally funded computing “cores” along the lines of bioinformatics, genomic, or statistical cores that are common in biomedical research facilities. The establishment of a dedicated behavioral programming effort would allow long-term development of new behavioral interventions, as well as rigorous testing of structural tailoring variables or algorithms in non-randomized-controlled-trial formats.[36]

5.4. Limitations

The primary limitation of this study is that it relies on self-reported preferences. As discussed above, participants may not know what they want or need in an intervention. However, the consistency of these results suggests that improved tailoring of structural variables is, nonetheless, important for future mobile interventions. Regardless, empiric testing of these findings is needed.

The second large limitation is that this study used one group of adolescents with a particular constellation of baseline symptoms. The results cannot necessarily be extrapolated to mobile psychological interventions for adults, or to other behavioral intervention topics. However, our findings do correlate with those found in other fields (e.g. mobile weight-loss interventions) and with other populations (e.g., web-based mental health interventions for adults). Comparison with qualitative data from other populations would be worthwhile.

Finally, our study did not formally assess information processing styles, such as locus of control and need for cognition; however the lack of variation based on comprehension of Cognitive Behavioral Therapy content (unpublished data) suggests that these variables would not help to determine the most appropriate structure.

6. Conclusions

In an ideal world, mobile psychological interventions would be evidence-based, impeccably adapted in-the-moment, engaging, and effective. The advantages to such a program are innumerable: they may reach those who may not have insurance, transportation, or motivation to come to a physical appointment; they may surmount inadequate resources for patients with serious mental illness; and they may offer preventive services to those who don't even realize that they need prevention.

However, such mobile interventions must also reflect a real need for personalized structure. This qualitative study highlights the necessity of developing structural as well as content-based tailoring of mobile psychological interventions. This complex undertaking will likely increase engagement and effectiveness. Solving the structural tailoring quandary will require innovative thinking by computer scientists, behavioral theorists, and clinicians alike.