A text messaging intervention to support the mental health of young adults: User engagement and feedback from a field trial of an intervention prototype

Abstract

Background

Young adults have high rates of mental health conditions, but most do not want or cannot access treatment. By leveraging a medium that young adults routinely use, text messaging programs have potential to keep young adults engaged with content supporting self-management of mental health issues and can be delivered inexpensively at scale. We designed an intervention that imparts strategies for self-managing mental health symptoms through interactive text messaging dialogues and engages users through novelty and variety in strategies (from cognitive behavioral therapy, acceptance and commitment therapy, and positive psychology) and styles of interaction (e.g., prompts, peer stories, writing tasks).

Methods

The aim of this mixed-methods study was to pilot 1- and 2-week versions of an interactive text messaging intervention among young adults (ages 18–25), and to obtain feedback to guide intervention refinements. Young adults were recruited via a mental health advocacy website and snowball sampling at a North American University. We used Wizard-of-Oz methods in which study staff sent messages based on a detailed script. Transcripts of interviews were subject to qualitative analysis to identify aspects of the program that need improvements, and to gather participant perspectives on possible solutions.

Results

Forty-eight individuals ages 18–25 participated in the study (mean age: 22.0). 85 % responded to the program at least once. Among those who ever responded, they replied to messages on 85 % of days, and with engagement sustained over the study period. Participants endorsed the convenience of text messaging, the types of interactive dialogues, and the variety of content. They also identified needed improvements to message volume, scheduling, and content.

Conclusions

Young adults showed high levels of engagement and satisfaction with a texting program supporting mental health self-management. The program may be improved through refining personalization, timing, and message volume, and extending content to support use over a longer timeframe. If shown to be effective in randomized trials, this program has potential to help address a substantial treatment gap in young adults' mental health.

1. Introduction

Young people in the U.S. face a substantial mental health burden (Murthy, 2021). Approximately 17 % of young adults (i.e., those between the ages of 18 and 25) experienced major depression in the past year (Substance Abuse and Mental Health Services Administration (SAMHSA), 2020), and approximately 15 % experienced past-month anxiety symptoms (Goodwin et al., 2020). The COVID-19 pandemic has further exacerbated mental health challenges young adults face (Vahratian, 2021), leading to more daily stressors, disrupted life transitions, and increased isolation.

Despite high rates of mental health challenges, relatively few young people use formal mental health services. Young adults have the most disproportionate underutilization of mental health services of any adult age group, with less than half receiving any formal mental health service such as psychotherapy, counseling, or medication management (Substance Abuse and Mental Health Services Administration (SAMHSA), 2020). Young adults' limited use of these treatments reflects both structural and attitudinal barriers (Gulliver et al., 2010). Structural barriers include financial costs, provider shortages, and transportation challenges, whereas attitudinal barriers include stigma, preferences for self-management, and beliefs that one's mental health condition is not severe enough to warrant treatment or that treatment will not help (Gulliver et al., 2010; Mehra et al., 2021).

Digital mental health interventions (DMHIs) delivered via mobile phones and other digital devices offer an opportunity to support young adults who may not have access to, or may not want, formal mental health services. These interventions can be designed to be self-guided, and to involve brief engagement in one's free time and personal environment. Because of these and other factors, many young adults are more willing to use DMHIs than to seek in-person help (Liverpool et al., 2020). DMHIs are also efficacious, with effect sizes comparable to face-to-face therapy in clinical trials (Linardon et al., 2019).

However, until recently, little attention had been paid to designing evidence-based DMHIs that individuals will use outside of research trials. When accessed “in the wild,” the vast majority of DMHIs have had low engagement and high attrition (Baumel et al., 2019; Torous et al., 2020; Fleming et al., 2018). These challenges reflect factors such as reaching a less motivated population outside the trial context, and losing structure and support for tool use (Mohr et al., 2017). To ensure tools are convenient and useful in the context of day-to-day life, it is necessary to involve end-users across the design and development process (Mohr et al., 2017; Blandford et al., 2018; Borghouts et al., 2021; Poole, 2013). There have also been calls to employ more widely used technologies, such as text messaging (Willcox et al., 2019), messaging apps (Fitzpatrick et al., 2017), and social media (Dobias et al., 2022), so that interventions fit easily into users' daily routines and are more scalable.

In this paper, we address a prototype of a text messaging intervention designed to support young adults' self-management of depression and anxiety symptoms. Consistent with the Accelerated Creation to Sustainment (ACTS) framework for digital intervention development (Mohr et al., 2017), the design process was rooted in the perspectives of end-users with experiences of depression and/or anxiety, involving them in elicitation of design ideas and iterative evaluation of intervention prototypes (Kornfield et al., 2022; Kruzan et al., 2022; Meyerhoff et al., 2022a), including evaluating example messages and proposing revisions. This process led to the preliminary design of a text messaging program that provides variety in the psychological strategies users can learn about, and in the types of interactions users can have with the program (Meyerhoff et al., 2022b). Text messaging was selected as an intervention medium as it is one the most used forms of interpersonal communication across demographic groups (Gallup., 2014), and is accessible without internet-capable devices like smartphones or without consistent access to a data plan (Willcox et al., 2019).

Here we describe procedures for, and results from, a study of a prototype of the “Small Steps SMS” program. While the program is intended to be automated, this study used a Wizard-of-Oz approach (Dow et al., 2005; Tennent et al., 2018), a user-centered design method in which research staff play the part of the system by manually sending messages based on a detailed script. We piloted 1- and 2-week versions of the program and collected feedback through semi-structured interviews. We report, based on message logs, how much users engaged with the program and—drawing on qualitative data—aspects of the design that kept them engaged and aspects to refine. We discuss how our results can inform messaging-based DMHIs for young adults.

2. Materials and methods

2.1. Intervention description

Our text messaging intervention provides exposure to an array of evidence-based strategies for managing affective symptoms and dialogue types through which users can engage with the system, as described below.

The intervention supports learning about and applying psychological strategies from diverse theoretical frameworks that have evidence for supporting management of depression and anxiety. While a full-length automated program will support learning a large number of psychological strategies over 8 weeks (Meyerhoff et al., 2022b), four primary strategies were emphasized here to fit the brief timeframe. First, behavioral activation, a component of cognitive behavioral therapy (CBT), seeks to increase users' contact with rewarding activities. Second, self-compassion draws on the psychological flexibility model in acceptance and commitment therapy (ACT) (Yadavaia et al., 2014), and involves challenging rigid self-perceptions and increasing recognition of personal growth. Third, willingness, which also stems from ACT, supports approaching a full range of human experiences without selectively avoiding unpleasant ones. The fourth strategy, help-seeking, is essential to accessing and productively engaging with treatments and informal sources of support.

Each strategy is supported by a set of dialogues, falling into four types (Kornfield et al., 2022; Meyerhoff et al., 2022b). Dialogues are launched on different days of the intervention. Across all dialogue types, a subset of messages are interactive, with users able to enter free text or select from predefined response options (e.g., “reply with yes or no”), and with user replies determining the next system message (i.e., “branching logic”). Dialogues vary in how many opportunities they provide for a user to respond. Furthermore, user responses can open new response opportunities (e.g., follow-up questions). Fig. 1 shows example interactions with the four dialogue types.

Fig. 1.

Examples of (a) prompt, (b) story, (c) writing, and (d) modular dialogues.

First, prompt dialogues provide psychoeducation and guide the user in taking a small action that day in line with a psychological strategy (e.g., noticing self-critical thoughts, as a part of self-compassion). Users may be asked, through open-ended questions, to elaborate on their thoughts, plans, and ideas, but a response is not required to receive messages later in the day. In the afternoon, users receive a brief motivational reminder to engage in the suggested action. At the end of the day, they are asked if they used the strategy, followed by a tailored message of positive reinforcement (for those who used the strategy), or encouragement to try later (for those who did not use the strategy). Participants are asked on a subset of days if they wish to continue receiving content about the same strategy or switch to something new.

Second, story dialogues are first-person narratives that illustrate how a peer addressed a challenge. Stories were composed by research team members or crowdsourced from peers with mental health concerns, with authors asked to relay true experiences with common concerns that arise in depression or anxiety (e.g., feeling isolated or unmotivated), and how they applied a strategy to address it (Bhattacharjee et al., 2022). Stories are delivered across approximately 3–5 messages.

Third, writing dialogues seek to activate the “helper therapy” principle (Riessman, 1965), which posits that people can increase well-being through providing assistance. After responding to a writing prompt (e.g., “Please help us write a short message that might motivate someone who feels depressed. What would you say to help them to get through a bad day?”), follow-up questions give users options to share their message 1) with others using the program, and 2) with themselves later on if/when they are struggling. Since message composition can feel overwhelming during periods of low mood (Kornfield et al., 2020), users first rate their mood and energy, which leads them down two possible branches, either 1) composing a support message when their mood is high or not reported, or 2) receiving a support message when their mood is low (Bhattacharjee et al., 2023).

Finally, modular dialogues involve brief interactions including reflection questions, supportive messages, peer stories, or immediately actionable prompts. Whereas other dialogue types unfold across multiple contact points, messages in modular dialogues are delivered in minutes at a single contact point, and do not reference content earlier or later in the day. Some modular dialogues enable users to cycle through many such brief interactions, leveraging moments when users are motivated and able to engage.

2.2. Methods: evaluating a wizard-of-Oz system prototype

Participants were recruited through a mental health advocacy organization, Mental Health America (MHA), and snowball sampling. Participants from MHA had completed online self-screenings for depression or anxiety, with results showing at least moderate symptoms according to the Patient Health Questionnaire-9 (PHQ-9; ≥10) (Kroenke et al., 2001) or the General Anxiety Disorder-7 (GAD-7; ≥10) (Spitzer et al., 2006), respectively. Interested individuals followed a link presented alongside their results to complete a study-specific screener and were eligible if they were in the U.S., 18–25 years old, and owned a mobile phone. The snowball sampling method involved inviting young adults from study team members' personal networks at a North American university to participate and to pass on study information to others. Those recruited through snowball sampling were eligible if they were 18–25, resided in North America, and owned a mobile phone. The authors' Institutional Review Board approved the study. Participants provided informed consent and gave their phone numbers, time zones, and the earliest and latest times of day they would be willing to receive text messages.

Participants used a prototype of the program for one or two weeks. As with other early dialogue systems (Nordberg et al., 2019), we employed a Wizard-of-Oz method wherein researchers carry out expected actions of the system, allowing for feedback on the content and protocol before automating it. Participants were informed that pre-scripted messages would be sent by researchers. On each study day (except for two break days in the two-week version), users received a dialogue representing one of the four types (prompts, stories, writing, or modular). Dialogue schedules for these programs are represented in Supplementary Table 1. Research team members followed a script specifying when each dialogue was initially launched, the content and timing of each message within the dialogue, and how to respond to each possible user response (or non-response). Generally, if the system solicits a user message (e.g., asking a question of the user), response times for system replies acknowledging or responding to those user messages are as soon as possible. Where a user response is not expected or the user does not send one, timing of the next system message is specified in the script relative to the last system message (e.g., 2 min later) or relative to the dialogue start time (e.g., 5 h after the start of the dialogue). An example script is shown in Supplementary Table 2. The next day's dialogue is launched as scheduled regardless of whether the user finished responding to messages received the prior day. The Twilio web platform was used to send and receive messages, with team members pasting system messages from the script. Team members took shifts monitoring incoming messages. Incoming and outgoing messages were logged by Twilio alongside timestamps. Daily engagement was computed from message logs, defined as the percent of days on which a user sent at least one message to the system.

At the end of the pilot test, participants were invited to complete optional Zoom interviews (20 to 40 min each) with a research team member. They were asked to discuss aspects they liked, valued, or found engaging, and areas of potential refinement. After transcribing the interviews, we followed a thematic analysis approach (Braun et al., 2012), with two coders independently performing open coding before meeting to prioritize a subset of codes for further coding. Prioritized codes and their definitions were captured in a shared codebook which the coders applied using the qualitative analysis software Dedoose. In several rounds, the coders overlapped in coding the same transcripts and then discussed results to guide codebook revisions such as removing rare or peripheral codes, consolidating overlapping codes, and adjusting definitions to accommodate the data and resolve discrepancies. After discussions stopped yielding codebook refinements, the transcripts were divided between the coders for final codebook application.

3. Results

3.1. Participant characteristics

48 individuals participated in the study, with thirty-two recruited through MHA and 16 through snowball sampling. Participants were 66.7 % (32/48) female and 33.3 % (16/48) male. They identified their race as 37.5 % (18/48) Asian, 35.4 % (17/48) White, 10.4 % (5/48) Black or African American, 4.2 % (2/48) American Indian or Alaska Native, 6.3 % (3/48) multiracial, and 6.3 % (3/48) undisclosed. The mean age was 21.9 (SD = 2.5).

3.2. Quantitative engagement

36 participants used a one-week version of the program, and 12 used a two-week version. Of 48 participants, 7 (14.6 %) did not respond to any messages in the study, of whom only one responded to researcher outreach. Among the 41 participants who responded at least once to the program, the rate of daily engagement (percent of days where a user sent any response) was 85.4 % (280/328), which was similar for the one-week and two-week programs (83.3 % and 89.0 %). Engagement was sustained over the study, as shown in Fig. 2. All dialogue types received similar engagement, with 85.4 % (72/84) of prompt dialogues, 86.5 % (45/52) of story dialogues, 83.9 % (78/93) of writing dialogues, and 85.4 % (85/99) of modular dialogues receiving at least one response. As most dialogues included multiple opportunities for users to respond, Supplementary Fig. 1a and b represents response rates across all response opportunities on each study day of the 1- and 2-week prototypes.

Fig. 2.

a: Daily engagement with a 1-week system prototype (N = 30).

b: Daily engagement with a 2-week system prototype (N = 11).

3.3. Qualitative findings

Nineteen participants (labeled P1 through P19) completed interviews in which they discussed aspects of the design they endorsed, and areas for potential improvement.

3.3.1. Design elements supporting engagement

Participants identified several aspects of the program that contributed to their engagement and satisfaction. These included the convenience of text messaging, the dialogue types, the program's variety, and the “tone” of messaging.

Providing support for text messaging as an intervention modality, several participants noted that the program was convenient. P9 described, “text messages are easy to read, fast, and they are low commitment.” P3 similarly relayed, “I think it's really beneficial having it be… a text message program. And, like I mentioned, all the other ones I have are apps, so I don't like that. And I like just how it kept responding back to me.”

While individual preferences varied, the four dialogue types all received strong endorsements from a subset of participants. Participants reported that prompt dialogues helped them learn new strategies, and most relayed following at least some prompts. P5 described, “Thanks to these messages, I know how to practice the strategy. So, I could remember like the first day I received the messages about [behavioral activation] I started to do it right away. And it felt very rewarding.” P19 described that she missed her messages once they stopped: “I was just like, ‘Oh, I want more prompts.’ Because I'd been doing it every day, being proactive.”

Stories were also popular. P19 valued seeing that others had navigated similar issues: “I really like the stories the most because it made me feel less alone, like, ‘Oh, somebody else is also dealing’ – and also what she did to counteract that.” Participants reported that first-hand narratives normalized their mental health challenges and offered inspiration for how they might address them.

Others identified the writing dialogue as a favorite component. P18 recalled benefiting from composing a supportive message: “I thought this was really cute. I remember that day, I kind of needed to hear myself say that.” A number appreciated that these dialogues provided a low-effort way to help others, such as P8: “If my thinking can somehow have any meaningful contribution to others, I'm willing to share everything I am writing.” For those who received a peer support message, these could help convey that their challenges were common and manageable. P15 described, “Just if you're having a pretty rough day and then you get a message just kind of telling you, ‘Hey –...Today is a rough day but that doesn't mean it's going to be like this every day’ type of thing. So, it was nice to see it. Kind of like validation.”

As far as the modular approach, participants appreciated that it reinforced skills they learned earlier, allowing them to make immediate use of strategies without much background information. P1 described that modular dialogues helped by “just reminding people that, yes, you have read this, and you can use this knowledge to apply to your troubles in life.”

Users also liked the balance in content types, endorsing variation from day to day in the strategies emphasized and the interactions used to support them. This variety counteracted “the short attention span that we have in this age” (P6). Participants also endorsed the balance of system-selected content and explicit choices. P3 described “I liked how there were options,” relaying that it was easy to just say “no” to get something else “if I wanted a different activity or something that worked for me more.” P6 described that they had enjoyed opportunities to make choices but thought having the system select dialogues was also useful: “I think both of them are worth exploring. Like maybe for one week, you give a set of random messages, but for the other week, you could give people choices [of] which one you want to send.”

The program's “tone” was viewed favorably by most. Several participants found the program kind or agreeable. P10 commented, “I felt like it was very gentle and very friendly.” P9 connected the tone to a clinical or professional demeanor: “They were nice messages to receive. To have that calm feeling the same way that a therapist would speak calmly.” P14 described that, although the program was automated, it provided a “sense of human interaction.” They elaborated that, “When I received a message, ‘Good morning,’ I thought that was surprisingly intimate, and surprisingly nice.”

Overall, these findings suggest that dialogues were perceived to help in different ways, including learning self-management strategies, feeling connected to others, and self-reflecting. Users largely found these types of interactions complementary and appreciated that the system provided a mix of content, as well as offering choice.

3.3.2. Challenges and suggestions to improve engagement

Despite generally positive evaluations, participants reported various challenges they faced using the program, typically around message volume and timing, and the desire for further personalization.

Several participants had challenges with the high message volume. Receiving too many messages in a short period of time could exceed users' capacity to engage, or annoy them, especially when psychoeducation was provided in back-to-back messages. P9 recalled, “When the text messages came, I read them and then, like, I put my phone down. Two minutes later, I get another text, and two minutes more, then two minutes more. And so, it just became really distracting.” P6 described that this problem could be exacerbated if message sequencing became unclear. Similarly, P10 was frustrated to lose the opportunity to respond to earlier questions when the program moved on to other content: “The messages sort of piled up, so I wasn't able to answer previous problems.” Some of those who thought there were too many messages also found the messages too long or complex, such as P7: “That annoyed me. When I got off work, I was like, why is there a long ass text?”

Despite feedback suggesting we had too many messages or delivered them too quickly, there were exceptions where messages were perceived as too slow. Within interactive dialogues, a delay could sometimes disrupt the conversational experience. P9 described, “That's why the bot is so nice theoretically is that you don't have someone waiting on the other side… You really don't want to delay, right?” There were also latency issues related to story dialogues. For example, story dialogues were scripted such that stories were broken into 3–5 messages, with each message sent 30 min apart. Several participants wanted to make sense of stories more quickly or were confused about when stories were over. P15 described, “As much as I liked the story, it was hard for me to keep track of the story... because it was coming in intervals.” P18 described that sending stories over several time points could leave story characters in a somewhat hopeless state, without resolution, reflecting, “This was the one time when I didn't like that the messages are spread out.”

In addition to timing and volume, users highlighted frustration at receiving messages when they were busy or not in the mood. P9 described, “The difficult thing is that you don't really know my schedule, so you can't really figure out when I'm in a meeting, versus when I'm bored or writing something.” Unpredictable events also changed users' moods in ways that affected willingness to engage. For example, P7 described, “Let's say, if I just got my mark back for class. If it's a good mark and I received a text, I'll be happy to respond. But if I get a really bad mark then, like, I don't want to.” Similarly, P5 described that their engagement would depend on “whether I have something on my mind or not.” To address these issues, common suggestions were to allow users to set their own messaging schedules and to pause messages when unavailable or uninterested.

A few participants discussed psychological strategies they did not like, that needed clarification, or that created discomfort, although feedback was idiosyncratic. P7 thought help-seeking messages did not adequately acknowledge fundamental barriers: “Even if you tell me that I need to ask for help, like, I'm not gonna reach out to anyone, because there's other reasons…. Like talk to a therapist, I don't have time for that.” Other participants had negative reactions to willingness messages prompting them to approach experiences that are temporarily uncomfortable but that might allow them to pursue meaningful goals. P19 recalled that these messages “got me overthinking… I was just sitting there with, like, thoughts in my head.” P8 wanted more guidance on managing distressing thoughts: “There could be a bit more information, like how I am supposed to deal with my negative feelings.… in a more constructive way, to be able to use this time to do something meaningful instead of sitting there doing nothing?” While participants suggested revisions to the presentation of strategies, they also appreciated that the program moved on to different content quickly, allowing them to stay engaged and find strategies that resonated with them more.

4. Discussion

User-centered design processes are increasingly incorporated to attain direction and input from users across all stages of DMHI design and development (Borghouts et al., 2021; Doherty et al., 2012; Saleem et al., 2021). Consistent with this, our goal was to investigate young adults' engagement and experiences with a prototype of an interactive text messaging program for managing mental health concerns, and to identify design elements with potential to support or undermine engagement and satisfaction in daily life. This discussion explores what our findings suggest about factors relevant to engaging young adults, and directions for improving future versions of this and other DMHIs.

While the timeframe of this pilot study was brief (one and two weeks), we found promising engagement rates, exceeding observed rates for depression apps in trials (Torous et al., 2020) and real world settings (Baumel et al., 2019; Fleming et al., 2018). This may relate to the convenience of text messaging; rather than navigating to and opening a dedicated tool, users receive messages through a habitual communication channel, which helps overcome challenges such as impaired memory, low motivation, or high avoidance. These findings may also reflect the level of user involvement in our design process, encompassing formative work with young adults to understand their mental health needs and routines of self-management and technology use (Kruzan et al., 2022; Meyerhoff et al., 2022a), and gathering iterative feedback on content (Kornfield et al., 2022), which likely improved acceptability (Willoughby and Furberg, 2015). Findings suggest that the four dialogue types comprising the program were perceived to help in different ways. Overall, the variety of content within the program, including user-generated content, appears to have benefitted engagement. These findings are encouraging in light of challenges reported in developing mental health content for young people, reflecting factors like rejection of clinical labels, limited familiarity with treatment concepts, or ambivalence around their own preferences (Ranney et al., 2015; Smith et al., 2014).

This study also revealed aspects of the intervention design that can be improved to address issues with volume, timing, and content, and to further personalize the system. As in prior work on DMHIs for young people (Ranney et al., 2015), there was not always consensus in critical feedback. Some users found the message volume and timing appropriate, but others were unsatisfied. A few shared feedback about psychological strategies that did not fit their needs. These patterns suggest the value of personalization, or adapting the program to specific individuals' needs, preferences, and contexts. Personalization has been indicated as supporting engagement across a range of DMHIs (Borghouts et al., 2021). Below, we describe what our findings suggest as far as both general modifications and personalization.

Text messaging is integrated in daily communication practices and is an interrupting medium (e.g., with notifications, sounds, and vibrations). These properties allowed messages to reach users throughout the day, but also meant that messages could sometimes overwhelm, distract, or frustrate users. Several users described moments of dissatisfaction from receiving too many messages, suggesting a need to shorten lengthy explanations and exclude non-essential messages. While messages were occasionally perceived as coming too slowly, this was largely within interactive dialogues that were conversational, and when presenting stories. For such dialogues, further experimentation may be necessary to identify optimal latencies. For example, users could be sent dialogues with shorter versus longer latency, with ratings or engagement rates used to determine future content delivery. It may also be possible to determine the ideal latency for a user based on their own latency in responding, akin to matching speech rates (Manson et al., 2013).

Dissatisfaction with message volume also reflected users' availability and psychological states, suggesting that systems should adapt accordingly. Many users wanted options to pause messages, or to actively launch dialogues at a chosen time. Users also wanted personalized schedules, fitting delivery into time windows when they tend to be available and motivated. This could involve sending messages within user-selected windows (e.g., late morning, early afternoon), or testing responsiveness in these windows and then adapting delivery. Sensor-based measurements (e.g., location, movement) offer another potential avenue to understand users' contexts (Meyerhoff et al., 2021). For example, a user who is at home outside work hours and whose phone is unlocked may be likely to be available for a message. Other work has proposed linking messaging systems to users' digital calendars (Bhattacharjee et al., 2022).

Finally, some participants objected to the content of psychological strategies. Preferences varied but could be strongly held. Further work is needed to understand how to present strategies that are therapeutically beneficial but also challenging, which may include normalizing the difficulty experienced, or conveying why a challenging strategy is still worthwhile. Participants also valued having the ability to discontinue strategies they deemed unhelpful or irrelevant. This suggests the risk of creating interventions that are too strictly “tunneled,” meaning that the user must complete content on the topic before proceeding to the next topic, without an option to exit early. Instead, designers must find convenient ways to allow ongoing customization, while considering the potential burden of habitually requesting feedback. For example, a system might follow up on especially low ratings or engagement with additional queries to better understand their basis (e.g., not liking a psychological strategy, a timing or volume issue) and offering a way to opt out of problematic content. For preferences that are less strongly held, the program can collect ongoing ratings or engagement data to inform passive, gradual tailoring, without complete exclusion of a content type.

This study has several limitations. Our Wizard-of-Oz approach allowed us to gather rapid user feedback and monitor free-text responses so researchers could conduct participant outreach, if indicated. Yet, this approach likely impacted users' experience. For example, researchers could perform disambiguation of unclear messages (e.g., where a user replies “ok” instead of “yes”) and could respond when users replied out of sequence or changed their answers, which likely improved user experience. In addition, knowing researchers would read their messages may have changed how users responded. The approach also was subject to some fidelity issues, where team members made errors in following messaging scripts or sent messages later than scheduled due to conflicts. In addition, our results focus on one definition of engagement: responding at least once to a daily dialogue. This parallels common definitions of engagement used for DMHIs (e.g., any daily app use) (Baumel et al., 2019), and other interactive messaging systems (Psihogios et al., 2019). This could be considered a low threshold since participants often had multiple timepoints at which they could respond. On the other hand, participants can use the program in ways that are invisible to the system (e.g., acting on a suggestion without sending any response). Future work should consider additional metrics, including link clicks, response latency, sensor-based data, and subjective engagement measures which may capture invisible responses like reading, thinking about, and acting on messages. Additional limitations include that the qualitative feedback came only from those willing to provide interviews, and that the findings related to general favorability of the program will not necessarily translate into acceptance or use of the psychological strategies, or reduced symptoms.

These limitations suggest the need for efficacy trials of the system to assess effects on depression and anxiety symptoms. Furthermore, additional study designs may be considered to link specific content types to effects on symptoms, differentiating the helpfulness of each content type for depression versus anxiety. For example, participants may be exposed one-at-a-time to psychological strategies or dialogue types before reporting on depression and anxiety symptoms at the completion of each, or multiple versions of the system could be created that each focus on one strategy or dialogue type, with these versions then being compared.

5. Conclusion

User engagement is hypothesized to be essential for DMHIs to achieve effects on symptoms; yet engagement has been low in most DMHIs. Our findings suggest that an interactive text messaging program, developed through a user-centered design process, produced high and sustained engagement over one to two weeks. Qualitative findings suggest that user engagement reflects the convenience of texting and the program's interactivity and variety. Our findings also point to directions for improvement, including adjusting personalization, and message volume and timing. These findings support developing content for a full-length automated version of the intervention and testing its effects on sustained engagement and symptom reduction. If effective, this highly scalable intervention design can play a role in meeting the unmet mental health needs of young adults.

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:

David C. Mohr, PhD has accepted honoraria and consulting fees from Otsuka Pharmaceuticals, Optum Behavioral Health, Centerstone Research Institute, and the One Mind Foundation, royalties from Oxford Press, and has an ownership interest in Adaptive Health, Inc. Jonah Meyerhoff, PhD, has accepted consulting fees from Boehringer Ingelheim.

Appendix A. Supplementary data

Supplementary material