Investigating the Role of Context in the Delivery of Text Messages for Supporting Psychological Wellbeing

Abstract

Without a nuanced understanding of users’ perspectives and contexts, text messaging tools for supporting psychological wellbeing risk delivering interventions that are mismatched to users’ dynamic needs. We investigated the contextual factors that influence young adults’ day-to-day experiences when interacting with such tools. Through interviews and focus group discussions with 36 participants, we identified that people’s daily schedules and affective states were dominant factors that shape their messaging preferences. We developed two messaging dialogues centered around these factors, which we deployed to 42 participants to test and extend our initial understanding of users’ needs. Across both studies, participants provided diverse opinions of how they could be best supported by messages, particularly around when to engage users in more passive versus active ways. They also proposed ways of adjusting message length and content during periods of low mood. Our findings provide design implications and opportunities for context-aware mental health management systems.

1. INTRODUCTION

Digital mental health (DMH) tools enable users to access resources and strategies for managing their psychological wellbeing at their own convenience [9, 86]. However, a common concern about these tools is that they often deliver interventions that are mismatched to users’ needs at a particular time [31]. This is of particular concern for push-based tools that initiate interactions with the user, such as text messages and notifications. Push-based DMH tools can potentially support users at moments when they may not have the motivation or forethought to proactively engage, yet they risk being perceived as insensitive to the user’s current mental state or availability [31, 87]. These issues can cause frustration and eventually contribute to quitting or disengaging from digital interventions altogether [61, 94].

Incorporating information about a user’s context has the potential to overcome this fundamental challenge of disengagement with push-based tools, helping DMH systems deliver interventions that will be perceived as timely, appropriate, and relevant [48, 80, 88]. While context is a broad term that can have many interpretations [20, 22], HCI researchers generally acknowledge contexts by dividing them into several contextual factors (e.g., location, time of day, activity level). HCI researchers generally integrate context into their work by gathering data about dynamic situational factors and using them to inform the delivery of an intervention. These dynamic factors may be calculated (e.g., time of day) [6, 75], gathered through sensors and digital traces (e.g., location, movement, social proximity, engagement with the phone) [48, 80], or actively reported through brief assessments (e.g., mood, energy) [27]. For just-in-time adaptive interventions (JITAIs), these contextual factors are incorporated into algorithms and machine learning models that personalize the timing and content of an intervention [35, 87].

Although JITAIs have shown promise for sustaining engagement and motivating health behavior change in more personalized ways, HCI literature has argued that they are still far from accounting for the dynamic changes in people’s lives [118, 129]. This may be particularly true in a mental health context, where contextual factors like social interaction and movement can have complex and highly individualized relationships to a user’s mental state and need for intervention [63, 89, 106]. Furthermore, many DMH tools have historically been designed in a top-down process by experts without reflecting a nuanced understanding of users’ perspectives or preferences. When users have been involved in the design process, it has often been to the extent of content generation [53]; rarely have users been asked to give input on how they perceive contexts to shape their need for and receptivity to support as they go about their day-to-day lives, nor have they been asked to identify specific contextual data and decision rules that should underlie an intervention. Hence, literature recommends that to implement contextually-tailored systems, designers should carry out formative investigations with actual users to understand their needs and expectations and how these shift over time [22, 53].

Our research is motivated by this gap in the literature. We seek to understand users’ perspectives to inform the thoughtful selection of contextual factors in context-aware DMH tools and to define how dynamic factors might impact users’ receptivity towards different intervention approaches. We limit our investigation to text messaging systems — one of the most promising digital platforms for promoting psychological wellbeing. Relative to alternatives like mobile applications and online programs, text messaging systems are more ubiquitous and accessible to the general population [25]. There are a number of text messaging systems that help support psychological wellbeing [16, 50, 64, 84, 99], and a few of these adapt themselves based on factors like the time of day and users’ activity level [51, 94]. To inform the design of such JITAIs, we aim to answer the following research questions:

• RQ1: Which contextual factors are perceived to influence the user experience of a text messaging service for psychological wellbeing?

• RQ2: What specific elements of text messaging interventions need to be tailored to reflect the users’ dynamic contexts?

We focus our research on North American young adults aged 18–25. This age group is vulnerable to many mental health problems, exhibiting increasing rates of depression and anxiety over the past several years [46, 104]. Mobile phone usage is also extremely high in this population [44], as a recent survey has shown that nearly 100% of American young adults own a mobile phone [15]. Combined, these trends make DMH tools distributed over text messaging particularly relevant to this population.

Our investigation started with formative work, consisting of interviews and focus group sessions with 36 participants who were asked to discuss the contextual factors they thought would impact their engagement with a text messaging service for psychological wellbeing management. Participants anticipated that their daily schedule and affective state (i.e., an individual’s mood and energy level at a given moment) would be the most important contextual factors. They were divided in whether they would be more willing to receive messages in the morning versus the late afternoon or evening. However, they anticipated that messages calling for passive engagement, such as support messages or reflection prompts, could help them manage periods of low mood.

For a subsequent deployment study, we created two interactive text messaging dialogues that allowed us to explore the role of contextual factors: one dialogue centered around the impact of the recipient’s schedule and the other around the impact of the recipient’s affective state. We deployed these dialogues in a text messaging probe to 42 participants to observe how those contextual variables impact people’s experiences. Individual interviews with 20 participants confirmed several findings and provided additional nuances regarding users’ changing needs and preferences. They affirmed the importance of adjusting the timing of message delivery to maximize receptivity. Contrary to our formative work, however, participants in the deployment study also expressed interest in getting reminder messages during work or school hours as long as they did not demand an immediate response or action. There were some design tensions as well; some participants wanted fewer questions about their emotional state to more readily access the intervention content itself, while others wished for extended dialogues to express their current mood and energy level.

To summarize, our contributions include:

• The identification of key contextual variables that influence users’ experiences with a text messaging system aimed to promote psychological wellbeing,

• The identification of specific messaging elements that should adapt based on those contextual factors, and

• A set of design considerations for building context-aware DMH tools, such as the incorporation of various data streams to gather contextual information.

2. RELATED WORK

In this section, we first discuss how text messages have been used as a medium for promoting behavior change and mental wellness. We then describe past work related to interventions that are personalized according to the user’s context.

2.1. Text Messaging as a Medium for Promoting Behavior Change and Psychological Wellbeing

Text messaging services are now commonly used for promoting health behavior change [111]. These services have demonstrated success in supporting behavior change for various physical and mental health challenges [41, 42, 120, 133]. For example, one domain in which text messaging services have had significant impact is in reducing alcohol consumption among individuals with alcohol use disorder. Suffoletto et al. [120] lowered drinking among young adults by first prompting them to set a drinking limit and then delivering a combination of motivational messages, self-efficacy support, ecological momentary assessment (EMA) queries [113], and reminders to help them stay within their self-assigned limit. Glasner et al. [34] created an intervention to improve medication adherence and mitigate heavy drinking among adults with HIV and substance use disorders. A similar text messaging service was created by Liao et al. [69] to promote smoking abstinence among adult smokers. Their intervention resulted in lower cigarette consumption rate among participants, particularly among those who received messages more frequently (3–5 messages per day). Other areas where text messaging has been applied include weight management [21, 115], physical activity promotion [56, 85, 116], and patient engagement [95, 133], among many others.

There has been a recent proliferation of text messaging services for promoting psychological wellbeing [1, 16, 50, 61, 64, 84, 99, 119]. Such services can vary widely in the type of content they deliver and the outcome they aim to achieve. In terms of content, many services are centered around therapeutic approaches from clinical psychology, such as cognitive behavioral therapy (CBT) [132], dialectical behavior therapy (DBT) [71], acceptance and commitment therapy (ACT) [43], and motivational interviewing [45]. A text messaging system can also provide support by sending motivational quotes [47], recommending physical exercises [23], or describing how a peer has overcome similar challenges [6, 92]. Messages can also provide important reminders about healthcare services, treatment protocols, or appointment attendance [4]. Agyapong et al. [1] deployed a text messaging service to help people manage their mental wellness during the COVID-19 pandemic. The service sent daily supportive messages to people along with requests to reflect on their stress, anxiety, and depression level. The authors found that their messages were able to significantly reduce self-reported anxiety scores among users. Levin et al. [68] delivered messages ranging from psychoeducational texts, reminders, and EMA queries to help people with bipolar disorder and hypertension manage their symptoms. Arps et al. [2] were able to reduce depressive symptoms among adolescents by sending them daily gratitude messages. Lastly, researchers have explored the design of artificially intelligent chatbots that try to promote self-reflection by engaging users in human-like conversations [50, 59, 123].

Yet, as the next subsection explores, text messaging interventions may be even more successful in promoting mental wellness by acknowledging the dynamic user contexts. We now discuss how DMH tools have attempted to achieve such flexibility in the past and how our work contributes to this space.

2.2. Contextual Factors in DMH Tools

Context-aware computing is a sub-domain of ubiquitous computing and human-computer interaction that seeks to achieve a grand vision where “computation is embedded into the fabric of the world around us” [22, 82, 109]. However, the term context has been defined in many ways. For example, Dey [20] states the following: “Context is any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and application themselves.” On the other hand, Dourish [22] views context as a relational property among objects or activities. To operationalize context in a way that can be characterized by technologies, researchers often break down context into contextual factors such as location, time of day, and activity level [91].

In this work, we are particularly interested in understanding dynamic contextual factors that can change within a span of minutes, hours, or days. Furthermore, we are interested in how these factors may impact the user experience of a text messaging service for psychological wellbeing. This can be contrasted with examinations of relatively stable contexts within a user’s life, such as race, ethnicity, cultural background, or profession. Even well-motivated interventions can come across as irrelevant or inappropriate when they fail to adapt to the rapid, frequent, and unexpected changes in users’ lives [129], particularly since messages must compete for users’ time in the presence of other activities and issues that require their attention. Thus, delivering the wrong type of intervention or intervening at an inconvenient time will likely lead to content being ignored or creating a negative user experience [87], which can potentially cause users to quit using tools early [30, 66]. Among individuals with mental health concerns, primary reservations about using an automated messaging program related to the potential for messages to be intrusive or too generic [62], further suggesting the importance of better adapting messaging to users’ contexts.

2.2.1. The Role of Context in Mental Health.

Past literature in psychology suggests a large number of contexts of daily life that are linked to mental health conditions. For example, there is a body of literature that connects mental health with circadian rhythms [6, 28, 75, 128] — the physical, mental, and behavioral patterns that follow a 24-hour cycle [127]. Mental health conditions may involve distortions of the typical rhythms; as an example, individuals may find themselves waking up, going to bed, eating, and socializing at inconsistent times from day-to-day. These disruptions can lead to a sense of being ungrounded and worsen symptoms. Studies have also found that symptoms of depression and anxiety may manifest most acutely at certain times of the day, including the early morning [38] and nighttime [8, 122]. Mental health symptoms have also been linked to physical activity, movement, and social proximity, among other dynamic factors [5]. For instance, lower levels of depression have been observed when individuals are more physically active [49, 57, 121], spend more time outside of the house or visit particular types of locations [105], and engage socially with others [117]. Mental health conditions are also susceptible to specific stressors. For example, Brown et al. [10] reported that stress in early adolescents is majorly impacted by their homework load.

2.2.2. Just-in-time Adaptive Interventions.

Attempts to address contextual factors in the design of DMH tools have mostly come through the proliferation of just-in-time adaptive interventions (JITAIs) — “an intervention design aiming to provide the right type/amount of support, at the right time, by adapting to an individual’s changing internal and contextual state” [87]. In other words, such technologies leverage information about the user’s context to decide on the ideal time to deliver specific interventions. The contextual variables are often collected through sensors, self-reported scores, or prediction algorithms [58].

Several JITAIs have been developed in the recent past to promote healthy behavior and mental wellness [26, 32, 35, 36, 40, 60, 70, 97], drawing upon the diverse contextual factors described earlier. Ismail et al. [51] designed an adaptive text messaging app that used the user’s goals, current step count, and information about their surrounding environment to promote physical activity. Their results suggest that the app was more successful in breaking sedentary behavior compared to static reminder messages. Clarke et al. [17] explored the utility of a stress management tool that automatically sent the user a message whenever the sensors on their smartwatch detected an elevated heart rate. A more sophisticated version of this system by Howe et al. [48] adapted stress-reduction interventions not only based on changes in heart rate, but also on the time of day, the user’s facial expression, and the user’s volume of emails and calendar events. Another work by Paredes et al. [94] relied on user-reported depression scores and mobile phone sensor data (e.g., accelerometer, screen status) to deliver interventions.

While promising, the JITAIs described above have largely selected their contextual variables and set decision rules according to theories from psychology instead of being guided by an understanding of users’ own experiences or priorities. There is a rich history in HCI of seeking to understanding how a person’s context influences their receptivity to user experiences, especially to motivate prolonged use of a product [54, 67, 91, 124]. Interestingly, such approaches are not yet routinely applied to the design of JITAIs [53]. Therefore, existing JITAIs may not fully capture the nuances of users’ lives or how context is experienced to shape one’s specific needs from a technology.

Following the approach advocated by Kabir et al. [53], we address this gap in the literature with respect to adaptive DMH tools. We examine qualitative data from our formative study to identify the contextual variables participants thought would be most important in shaping their needs when using a text messaging service to support their psychological wellbeing. The subsequent deployment of a text-message probe further reveals how dynamic variables impact users’ experiences of receiving automated messaging and their receptivity towards different intervention approaches.

3. FORMATIVE STUDY DESIGN

We conducted our formative study with 36 individuals to understand which contextual factors they anticipated would be crucial to their interaction with a text messaging system for psychological wellbeing. We also used this opportunity to elicit people’s suggestions on how DMH tools can adapt to reflect dynamic changes in those factors. Below, we describe the protocol for this investigation. Research activities took place in two North American universities and were approved by both institutions’ Research Ethics Boards.

3.1. Participants

Recruitment for the individual interviews and focus group discussions was facilitated by Mental Health America (MHA), a community-based non-profit organization that is dedicated to promoting mental wellbeing in the United States. MHA hosts screening surveys that are widely used by young people seeking to better understand their mental health and connect to resources. People showing moderate levels of depression and anxiety according to the Patient Health Questionnaire-9 (PHQ-9) [65] and General Anxiety Disorder-7 (GAD-7) [72] (scores of 10 or higher) were invited to learn more about the project through a link that appeared with their screening survey scores. Potential participants completed an additional study-specific screening survey and were deemed eligible if they met all of the following criteria: (1) located in the United States; (2) between 18–25 years old, or 19–25 years old if they were in Nebraska; and (3) owned a mobile phone.

Of the 725 individuals who originally expressed interest in the study by completing the screening survey, 106 were deemed to be eligible. We invited 30 individuals (FP1–FP30) to take part in individual interviews, selecting participants such that the final sample would be representative of MHA’s audience with regards to mental health symptoms as well as demographic characteristics such as age, gender identity, and race. From these 30 participants, nine (FP1–FP9) also took part in focus group discussions; again, these participants reflected MHA’s audience. Focus group participants were invited to attend as many sessions as they wished upon receiving their initial invitation, and six participants attended more than one (range: 1 to 4). We conducted 5 focus groups in total. Facilitators were trained on strategies to encourage input from new and quieter group members, such as asking each member to write down their answer before going around to each member and giving them the chance to speak.

We were also interested in exploring multiple possible pathways of reaching young adults who would benefit from a DMH tool distributed over text messaging. The growing mental health concerns among university students [6, 114] motivated us to recruit 6 students (FP31–FP36) from a large North American University using a combination of snowball sampling [37] and word-of-mouth. The research team members invited students from an introductory programming course and an HCI research lab to participate in the study and encouraged them to pass on the invitations to their peers and colleagues. These participants were between 18–25 years old, and at the time of the study, they were studying computer science, cognitive science, or psychology. They did not need to meet any criteria regarding mental health symptoms in order to participate in the study.

Participants were recruited in a rolling manner, and we continued recruitment until we achieved data saturation from our semi-structured interviews and focus group sessions. We did not make attempts to have balanced groups across recruitment pathways or mental health status since drawing explicit comparisons between these two groups is beyond the scope of the work. Instead, we were more concerned with identifying important contextual factors that impact the text messaging experience across these groups. The mean age of our final participant cohort was 21.7 years old. They identified with multiple genders (30 women, 4 men, 1 non-binary, and 1 undisclosed) and racial groups (16 White, 10 Asian, 1 Black/African American, 1 American Indian or Alaskan Native, 4 mixed, and 4 undisclosed).

3.2. Study Procedure

We anticipated that some people might feel more comfortable discussing their mental health in a more private setting because of the topic’s sensitivity, whereas a group setting allows for people to build upon and respond to one another’s contributions to yield new insights and areas of convergence or divergence [33]. Therefore, we collected data both through individual interviews and focus group discussions. We first asked participants to share their experience of using DMH tools to generate a baseline of their prior experiences. We then asked participants to project how a text messaging service can better support them in managing their psychological wellbeing; in particular, we sought to collect data on ways that text messaging systems should adapt based on a user’s daily and moment-to-moment context. Interview questions were developed iteratively by two of the authors (RK and JM) as part of a broader effort to understand the needs and preferences for self-management of young adults who were experiencing mental health symptoms and were not interested in formal psychotherapy, but were interested in using DMH tools to manage their symptoms. This study reports on responses to questions in the interviews and focus group sessions that were targeted to understand the association between key contextual factors in participants’ lives and various elements of text messaging like content, volume, frequency, and suggested follow-up action. Questions included, but were not limited to:

• Are there factors or variables that change in your life and that would affect how receptive you would be to mental health-related text messages?

• How might those things change the type of text messages you would want to receive?

• Which of these factors do you think should be considered more important while text messaging programs are being designed, and why?

• What are some times when you would be more open to receiving messages? When would you be less open?

• Might you want different message frequency based on when it is more convenient for you to interact with them versus when it is less convenient? How?

The interviews were semi-structured in nature, allowing us to deviate from the interview script as needed to ask additional follow-up questions. We also provided participants with explanations and examples whenever necessary.

The individual interviews were conducted by one member of the research team via telephone or the Zoom videoconferencing platform. The focus groups were hosted on Zoom and proctored by two team members. The size of the focus groups ranged from 2–5 participants each. Individual interviews took 15–30 minutes, and focus group discussions lasted 60–75 minutes. All participants were compensated at a rate of $20 USD per hour.

3.3. Data Analysis

We followed a thematic analysis approach [18] to analyze the qualitative data. After transcribing interviews and focus group discussions, two members from the research group (referred to as “coders”) reviewed all transcripts to become familiar with the data. The coders then followed an open-coding process [55] to assign codes to segments of the data, and each developed a preliminary codebook on their own. This initial round generated roughly 50 codes, including “low mood”, “messages during the morning”, and “reflection during work”. The coders then engaged in several discussions to refine the code definitions, identify overlapping codes, and exclude codes that were not central to our research questions. Next, they applied their codebook to a subset of the data (10 interview transcripts and 1 focus group discussion) and met to refine the codebook further. This iterative process was repeated until the coders reached a consensus in their understanding of code definitions. Finally, they applied the final codebook to separate halves of the data. We focus below on several dominant themes that arose from this analysis, centering on participants’ daily schedule and their affective state.

3.4. Ethical Considerations

Our team members included faculty members and graduate students with training in human-computer interaction, clinical psychology, and cognitive science. Research on promoting psychological wellbeing can raise several ethical considerations, so we took measures to address these issues throughout our work. We informed participants at the beginning of the interviews and focus group discussions that they could skip any questions or stop the conversation altogether if they felt uncomfortable at any point. Interviewers were also trained to follow the Columbia-Suicide Risk Assessment protocol [96] if participants indicated thoughts of suicide or self-harm. They were also trained to deliver safety planning or refer participants to crisis services as needed. However, none of the aforementioned risks emerged during the study.

4. FORMATIVE STUDY FINDINGS

The participants in our formative study were typically frequent users of technologies for work, school, and socializing. While many reported previously using mental health smartphone apps, their usage was short-lived either due to forgetting or loss of motivation to engage. A few participants indicated using other DMH tools like emails or web-based services, but none indicated using an automated text messaging service for managing psychological wellbeing. Nevertheless, participants suggested that text messaging had the potential to better sustain their engagement than apps since they are already deeply immersed in text messaging with their friends, family, and colleagues.

Our focus groups and interviews revealed that two contextual factors dominated users’ thoughts regarding DMH tools: (1) daily schedule and (2) affective state. We elaborate below on why these were viewed as important in shaping receptivity and preferences for messaging.

4.1. Role of Daily Schedule in Shaping Message Receptivity and Preferences

Participants in our formative study repeatedly mentioned the importance of their daily schedule in relation to their ability to engage with text messages. However, we were given diverse responses when we asked them to share their specific preferred times for receiving messages. Participants generally indicated the early morning and late afternoon as suitable times for receiving messages, although some people were in strong favor of one versus the other. Those who believed that they would prefer to receive messages during the morning expressed that they have moments of time to themselves before leaving their home for work or school, which could not be said about other parts of the day that were more hectic and socially involved. A number of participants saw the morning period as an opportune moment for receiving an encouraging message that could uplift their mood and give them confidence to face the day. FP32 said,

“I prefer morning messages because they can be inspiring. It is the beginning of the whole day, and I need to have motivation to do the tasks. … If, in the morning, I get a message that gives me motivation, I will be happy for the rest of the day.”

(FP32)

In contrast, others were opposed to receiving messages in the morning. Participants like FP7 said that they generally wake up late and therefore have to hurry to get to class or work on time. Understandably, people in this situation would either delay reading or engaging with messages or ignore them altogether. Many of these same participants speculated that they would have more bandwidth to read and act upon messages in the afternoon or evening when they are less rushed.

Another reason that many individuals presumed that they would like messages later in the afternoon was that they felt the need for support after work hours. Some people reported experiencing loneliness and unhelpful thought patterns towards the end of their workday, leading them to overthink their perceived failures and shortcomings or to worry about the future, which could eventually lead to sleep troubles. The experience was captured in the following comment by FP16:

“I feel sending those messages out in the afternoon or closer to when people are going to bed is probably the best thing because, from my experience and from what other people have told me, when they kind of start heading of to sleep is when … I wish I had someone with me. Because when I wake up in the morning, if I’m not having the best day, I still have to get ready and then I go see my family and take my dog on a walk and those things kind of distract me. But when I’m on my way to start going to bed, it’s kind of like when I am alone more often.”

(FP16)

Participants expressed disinterest in receiving messages during the middle of the day, particularly during the workweek. They suspected that their attention would be preoccupied with their work or classes, so they would not get much time to process or act upon a message. In the event that a message had to be received during intense working hours, participants suggested that the message should be easy to read and should not prompt any action requiring significant effort.

4.2. Role of Affective States in Shaping Message Receptivity and Preferences

Participants unanimously acknowledged that their affective state — their mood and energy level at a given moment — is an important indicator of their receptivity towards text messages. While mood references the valence of one’s affective state, and energy references the level of arousal [108], participants in this study saw these dimensions as highly correlated, and sometimes used the terms ‘mood’ and ‘energy’ interchangeably.

Just as with their perspectives on the importance of their daily schedule, there were diverse suggestions regarding their preferred message content in different affective states. Some participants anticipated that they would only want to receive messages from text messaging services for psychological wellbeing when they are experiencing negative emotions, as reflected in FP1’s comment:

“You wouldn’t particularly want any messages when you’re doing well. But on days that are really tough, those messages would actually be pretty welcome, and you’d be pretty receptive to them.”

(FP1)

However, this sentiment was not unanimous since others saw value in receiving messages during periods of positive emotion. For example, some participants were open to receiving interactive cohesive dialogues during periods of high mood. FP15 felt that such dialogues would help them notice and sustain positive emotions for a longer time.

Participants were also willing to dedicate extra time towards explaining their emotions to a system so that it could deliver personalized messages targeted to their needs. FP5 commented:

“I would start of saying I’m feeling a certain way, like a mood. And then the system will text me back whatever advice they have regarding what I’m feeling.”

(FP5)

When asked about their preferred message content during low mood, some participants expressed interest in receiving simple check-in messages that would cater to their current feelings. Participants saw several benefits in these messages. For example, FP9 felt that check-in messages would actually prompt them to reflect on their feelings and emotions, making them more aware of their thought process. Another type of message that participants expected to be useful during periods of low mood were those that describe simple coping strategies. Several participants noted that during difficult times, they tend to focus on their own suffering and forget to appreciate the supportive people or helpful aspects in their lives, so P1 suggested that messages might introduce strategies related to gratitude. Participants also suggested that they would be willing to hear from others who are going through similar experiences. FP3 posited that a message from a peer might normalize the experiences of depression and even illustrate actionable strategies that could be useful in overcoming their struggle.

While recognizing the benefits of receiving messages during periods of low mood and energy, participants also conveyed that text messaging services should not overwhelm users by demanding a response or sending repeated reminders. Some of them posited that if they read a suggestion to partake in an activity requiring significant effort, there would be a low probability that they would act upon that suggestion immediately. In those moments, repeated prompts to engage with the messages could come across as overwhelming. Participants also advised against other message types that were perceived to demand high effort responses from users experiencing low mood, such as physical activity prompts or writing exercises (e.g., where the user is expected to compose a long free-text message). FP6 explained:

“If they’re having a hard time motivating or encouraging themselves, they might not feel like this is something they could do.”

(FP6)

They went on to describe that activities requiring moderate effort might be more appropriate “for someone who is like not [experiencing low mood] or has like a milder case”.

While most participants endorsed that mood and energy were strongly correlated, a few participants distinguished between the two constructs. For example, FP25 reflected on times when they felt tired but were not experiencing negative emotions. The cause for low energy in these cases was often busyness from other obligations.

Implications for Deployment Study:

Our formative study findings motivated the design of our subsequent deployment study. The two contextual factors we identified during our interviews and focus group discussions guided design of two message dialogues: one catering to daily schedule (Daily Schedule Dialogue) and the other to individual affective states (Affective State Dialogue). The message contents within each dialogue were also inspired by formative study participants’ opinions. The daily schedule dialogue asked participants to engage in brief activities or respond to reflective questions twice a day, enabling us to observe people’s reactions to both forms of message in the morning and afternoon. The affective state dialogue aimed to deliver messages based on participants’ mood and energy level. Participants received passive supportive texts if they were experiencing negative emotions; otherwise, they were asked to draft supportive texts for others. The following section explains our deployment study design in much more detail.

5. DEPLOYMENT STUDY DESIGN

Although our formative study helped us understand people’s expectations for adaptive DMH tools, prior literature has suggested that such investigations should be followed by deployments so that researchers can understand how those expectations translate to people’s real-world experiences [103, 112]. Hence, we conducted a deployment study to confirm, refute, and extend our findings from the formative work. We first describe the design of the probes we used to elicit feedback, and then we describe the protocol that was used to deploy and evaluate the probes.

5.1. Formation of Message Dialogues

We designed two separate dialogues to investigate the key contextual variables that emerged from our formative work. The first dialogue was designed to investigate people’s receptiveness to text messages at different times of day, and the second dialogue was designed to understand how people would react when text messages were tailored to their affective state. The messages in both dialogues were developed by the research team, which consisted of faculty members and graduate students in human-computer interaction, psychology, and cognitive science. The dialogues went through an iterative design process where the research team held multiple meetings to ensure that the messages reflected findings from the formative work. The dialogues are illustrated in Fig. 1 and Fig. 3; however, we provide a short description of each dialogue below.

Figure 1:

Example conversations in the morning and late afternoon within the daily schedule dialogue.

Figure 3:

Two example conversations under the affective state dialogue: (a) a conversation involving passive support reception, and (b) a conversation involving active writing.

5.1.1. Daily Schedule Dialogue.

Participants in our formative study expressed interest in receiving messages in the morning or in the late afternoon depending on their daily schedule and habits. To explore the importance of this factor, participants who engaged with this dialogue were sent two sequences of messages each day: one starting at 9:00 AM and another at 4:30 PM in their local timezone. We limited our protocol to these two time windows since the formative study participants were worried about getting overwhelmed by too many messages within the same day. Each sequence started with one or two messages designed to help people manage their stress or negative emotions. These messages were randomly selected from a message bank composed of two broad types of messages, examples of which are provided in Table 1:

Table 1:

Examples of messages used in the daily schedule dialogue.

Message Type	Examples
Brief Activities	Please find a comfortable seating position. First, exhale completely through your mouth, making a woosh sound. Then, close your mouth gently and inhale through your nose for 4 seconds, hold your breath for 7 seconds and exhale through your mouth for 8 seconds. Repeat this cycle 3 times.
	Reflecting gratefully on your day, right before you go to bed, can lead to better sleep. Set aside a few “gratitude” minutes right before you go to bed tonight.
Reflective Questions	Is there a small step you’ve taken recently that you didn’t give yourself enough credit for?
	What is one thing you can remove from your everyday schedule to create more space for rest and self-care?

• Brief activities: These messages prompted participants to engage in a small activity that could be completed within a few minutes, such as breathing or mindfulness exercises. Activities like these have been found to improve moods and reduce momentary stress [24, 94].

• Reflective questions: These messages asked participants to reflect on various elements of life, such as their career, relationships, and health. Doing so allowed participants to re-evaluate their thought patterns and find alternative perspectives in a structured manner [3].

One hour after the initial message(s), participants were asked whether they enjoyed reading the messages. If they said ‘yes’, they received an acknowledgement message that read, “We’re glad you liked it! Thank you for the feedback.”; if they said ‘no’, they received an acknowledgement message that read, “Sounds like this didn’t really work for you. Thank you for the feedback.” Participants who did not provide a response did not receive an acknowledgement message. This procedure was repeated each morning and afternoon irrespective of past engagement. Participants were not sent the same message sequence twice on the same day.

5.1.2. Affective State Dialogue.

Participants in our formative study also expressed interest in receiving messages that would allow them to express their emotional states and receive responsive content accordingly. Literature suggests that emotion labeling (i.e., “using a specific word to describe an emotion” [27]) can help people observe, express, and accept emotional responses without self-judgment and eventually help them engage in activities more mindfully [52, 110]. By being more aware of their psychological state, a person may also build a more nuanced understanding of how their feelings relate to their thoughts and behaviors, thereby motivating changes in behavior and thought patterns [132].

Motivated by these opportunities and benefits, we created a check-in message that assessed the user’s state of mind based on the circumplex model of emotions [108]. According to this model, which is illustrated in Fig. 2, human emotions are distributed in a two-dimensional space. The model’s horizontal axis encapsulates valence, which pertains to how positive or negative the user’s emotional state or mood is. The model’s vertical axis encapsulates arousal or energy level. For our check-in message, participants were asked to rate both their energy and mood levels as either ‘high’ or ‘low’. They were then asked to select the emotion that best represented their state of mind from a list that was dynamically generated according to their previous answers. For example, if someone reported a high mood and low energy, the list would contain emotional states corresponding to positive valence and low arousal: calm, relaxed, and content. Participants were also able to indicate that none of the emotions sufficiently represented their state of mind.

Figure 2:

An illustration of the circumplex model [108], which defines emotions along on two dimensions: mood (valence) and energy (arousal).

Participants in this dialogue received the check-in message at 9:30 AM in their local timezone. If the participant reached the point of the dialogue where they had selected a term to describe their current state, the system sent a message that acknowledged their emotions (e.g., “Sounds like you are feeling relaxed right now.”). If the participant could not find a suitable match from our list, the system acknowledged the shortcoming and provided them with a link to a website containing a larger set of emotions. Next, the dialogue asked participants to complete one of two activities that required different levels of involvement:

• Passive support reception: Participants received a short supportive message if they were experiencing an emotion associated with low mood or they could not find an emotion that reflected their state of mind. Each message was designed to provide social support, such as validation or encouragement. The messages also included a sentence at the beginning to explain the message source. Table 2 shows the four different message we used for this dialogue. At the end of this message sequence, the participant was asked whether they would like to receive the same message again in future.

• Active writing: Participants received this activity suggestion if they were experiencing an emotion with high mood The activity entailed drafting a text message to help someone else experiencing any negative emotion. Participants who saw this message were able to see example submissions provided by the research team. Once the participant finished writing their message, they were asked if they would be willing to share their message with others in future, and if they would like to see it when they themselves were in a low mood in the future.

Both sequences required responses at multiple points in order for the next message to be sent. If participants did not send a response at these junctions, the sequence was eventually dropped.

Table 2:

Examples of texts used for providing passive support in the affective state dialogue.

Message Type	Messages
Professional Message	Here is a message from one of our mental health professionals for people going through rough days: “Focusing on the good in life is a skill that gets better with practice.”
Peer Message	Here is a message from another person using the texting program: “There are still good things in you and good things in the world.”
General Message	Here is a message written for people having hard days: “There are still good things in you and good things in the world.”
Self-written Message	Here is a message you saved to send to yourself when you are having a rough day: [a supportive message written by the participant themselves in an earlier study day as part of a different dialogue]

5.2. Participants

We recruited 42 participants between the ages of 18 and 25 to engage with our probe; we refer to them as DP1–DP42. Participants were recruited using two distinct methods. The first group of participants (DP1–DP6 and DP30–DP42) were enrolled via a combination of snowball sampling and word of mouth, as in the formative study. These participants did not have to meet any inclusion criteria other than a general interest in testing a text message-based service for managing psychological wellbeing. The other group of participants (DP7–DP29) were recruited via targeted ads on MHA’s website, the same community-based non-profit organization that facilitated the formative study. These participants were recruited only if they reported meeting clinical cutoff scores for symptoms of depression or anxiety according to the PHQ-9 or GAD-7 (scores of 10 or higher). Five of these participants (DP7–DP11) also took part in our formative study.

As before, we recruited participants in a rolling manner until we reached data saturation. We were again not interested in drawing comparisons between sources of participants, resulting in slightly imbalanced group sizes. The mean age of our participants was 22.0 years old. Participants spanned two genders (29 women, 13 men) and multiple racial groups (17 Asian, 15 White, 4 African American, 2 American Indian or Alaskan Native, 2 mixed, and 2 undisclosed). At the time of the study, all participants were living in North America.

5.3. Study Procedure

Our investigation of contextual factors was part of a broader project exploring people’s engagement with text messages of diverse types (e.g., narrative messages, reminders, didactic lessons). Participants were recruited in several waves between September 2020–February 2021. They were recruited to receive daily messages for a total of 1–2 weeks depending on the number of daily dialogues that were being tested in the broader project at the time. This paper reports their experiences with a subset of dialogues centering on contextual factors. Those in the earlier waves (DP1–DP11) received the affective state dialogue on one day of the study, while the rest of the participants (DP12–DP42) received the daily schedule dialogue on one day of the study and the affective state dialogue on another.

We used Twilio, a message delivery platform, to send the messages. Research team members manually sent messages using a Wizard-of-Oz approach [90] so that unanticipated responses could be handled using human judgment (e.g., open-ended response to a close-ended question, spelling mistakes). Upon enrollment, participants were informed that their responses would be reviewed by research team members for this purpose. The team members were provided with a detailed script that contained instructions on the branches conversations should follow based on responses of the participants. After completing the study, participants were invited to take part in a semi-structured interview to provide their feedback on the two message dialogues and reflect upon how the contextual factors impacted their receptivity towards different messages. Our questions included, but were not limited to:

• How did you feel about being asked to rate both mood and energy, and then picking from a list of emotions? How would you improve upon this?

• Did you notice any differences in how responsive you were to messages depending on morning versus afternoon? Or based on your level of busyness?

• How did the message dialogues help you manage your mood and negative emotions?

The interviews took 10–30 minutes. They were conducted by one member of the research team via telephone or the Zoom teleconference platform. We did not compensate participants for engaging with our text messaging probe to ensure that the payment would not influence their interaction level, but participants who agreed to give interviews were compensated at a rate of $20 USD per hour.

5.4. Data Analysis

We analyzed participants’ responses to the message dialogues using mixed methods. Quantitatively, we report the response rates across all participants for both of the message dialogues. We calculate response rate as the number of response messages sent by a participant relative to the number of responses expected. Qualitatively, we analyze interview data using the same thematic analysis procedures [18] used in our formative work, albeit with a separate codebook.

5.5. Ethical Considerations

We informed participants at the beginning of the study that the messaging program was not intended to be a crisis service. We did not solicit suicide-related information from participants at any point during the study, but given the open-ended nature of text messaging, we anticipated the unlikely possibility that someone could express suicidal thoughts or other risk-related information while engaging with our probe. Hence, we developed several measures to ensure the safety of all participants. Participants were provided with the contact information of several crisis services (e.g., suicide hotlines, crisis text lines), and all text responses were reviewed on a daily basis by the research team. If any message indicated a risk of suicidal ideation or self-harm, team members were trained to reach out to the sender of that message and conduct the Columbia-Suicide Risk Assessment protocol [96], as previously described in Section 3.4. No such risks emerged during the study, and therefore, we did not need to conduct any follow-up assessments.

6. DEPLOYMENT STUDY FINDINGS

In this section, we first briefly describe the amount of engagement that participants had with the text message probes. We then report participants’ feedback on the individual dialogues, followed by insights that were revealed at the intersection of the underlying contextual factors.

6.1. Engagement Across Both Dialogues

It is difficult to measure engagement for a text messaging service like ours since people can choose to not respond to a dialogue despite having read and followed its suggestions. Hence, we refrain from making any statistical claims about participants’ preferences, but we report quantitative metrics of engagement here as it may relate to our qualitative observations.

For the daily schedule dialogue, the response rate across all participants was 72.6% (45/62), and 25 out of 31 participants responded at least once. The response rate was 67.7% (21/31) for the messages sent in the morning, and 77.4% (24/31) for the ones sent in the afternoon. For the affective state dialogue, the number of responses required to complete the dialogue varied depending on the branch they followed. The response rate across all participants was 80.3% (106/132), and 37 out of 42 participants provided response at least once. However, 19 participants fell short of providing three responses required to receive a passive supportive text or a suggestion to do active writing (in response to the questions about mood, energy, and discrete emotions).

6.2. Qualitative Feedback on Individual Message Dialogues

6.2.1. Daily Schedule Dialogue.

Our deployment study supported many of the findings from our formative work as far as how users’ daily schedule impacted their receptivity to text messages. Participants were again divided in their choice between the morning and afternoon being their preferred time for receiving messages, and these inclinations were both strong and personal. Preferences were often based on a confluence of factors like the user’s availability, their social situation, and the degree to which they were feeling contemplative or connected to their emotions at particular times of day. Those with preferences for morning messages also mentioned that the interventions gave them the chance to set the tone or plan for the coming day.

However, relative to participants from our formative study, participants who engaged directly with our probes provided a slightly different perspective regarding their receptiveness to messages during work or school hours. These opinions were typically formed when participants received a message in the morning but did not have time to interact with it until a few hours later in the day. After these experiences, participants proposed that certain messages could be well suited for busy periods of the day, provided they do not demand too much time or immediate effort. For example, some participants acknowledged that they often feel overworked, so they appreciated text messages that reminded them to take breaks or prioritize their mental health (e.g., “What is one thing you can remove from your everyday schedule to create more space for rest and self-care?”).

That being said, many deployment study participants echoed the challenges that were anticipated by the formative study participants regarding messages during busy periods. They commented on the pressure to maintain productivity and the difficulty of switching their attention during work hours, particularly when messages demanded some degree of physical or mental exertion. Participants like DP32 described that if they were asked to do an exercise or outdoor activity during busy hours, they would simply not be able to carry out the suggestion even if they found it appealing or potentially helpful. These high-effort suggestions would be postponed, forgotten, or ignored. Participants like DP38 advocated that messages should explicitly convey to users that it is reasonable for them to follow through with the suggestion later in the day:

“Don’t ask people ‘Do you want to do this quick exercise that you can do to help you relax?’. I might just say ‘no’ because I am doing something else, and now just because I say ‘no’, you’re not sending me any follow-up. But later on, when I am in the mood to do the exercise or whatever, I might do it.”

(DP38)

DP21 and DP29 provided similar opinions, stating that they often read messages immediately upon reception but acted on them hours later when it was more convenient. According to them, follow-up and reminder messages can increase the probability that people do the activities, but such messages should be sent in moderation to avoid irritating users.

Related to this, some participants suggested that messages should be strategically distributed throughout the day to provide sustained benefits and to maximize the likelihood of follow-through. DP34 suggested that even if a user prefers doing activities in the afternoon, it may be helpful to provide some background information earlier in the day to help them understand the rationale for proposing such behavior changes or to mentally prepare for future messages. DP34 also suggested that users who prefer doing activities in the morning may appreciate a follow-up message the following morning to serve as a reminder:

“If I were to wake up, and then I see the text message on my phone, it might not be the first thing that I start my day with. But then at 5 PM, if I get a question ‘What was your favorite part?’, then I’m like, ‘Oh right, the exercise! I should do that.’ … I think that being prompted later in the day, might act as another reminder to go and actually do the activity that was recommended.”

(DP34)

Although our findings generally support expanding the times of day we might consider sending messages, there were also important caveats. Specifically, a few participants suggested that messaging systems should not only try to send messages at users’ preferred times but also focus on minimizing the probability of sending messages at the most inconvenient times. They suspected that even receiving a single message during a highly inconvenient time (e.g., during an exam or an important meeting) would have a significantly negative impact on their engagement and retention. Thus, in addition to asking users about their preferred time for receiving messages upon enrollment, DP30 recommended that systems should ask users to report the times during which they do not want to receive any messages at all.

6.2.2. Affective State Dialogue.

Participants generally appreciated the messages that asked about and catered to their mood and energy level. Confirming the expectations of participants in the formative study, many people found the emotion labeling to be helpful, noting that the activity facilitated emotional awareness. DP1 said,

“Having to look at all the different emotions is like, ‘Oh, actually, I’m feeling like this?’ Or like, it kind of makes you separate what you’re feeling between like, is it sadness, or is it just anxiousness.”

(DP1)

Participants also felt that emotion labeling at the beginning of the dialogue gave them a sense of control over the type of content they received, which made the user experience feel more personal.

Some of the participants stated that they were unmotivated to even look at their messages while they were experiencing low mood and energy, but when they did, they appreciated the fact that the corresponding passive support messages were less demanding. Reflecting on a peer message, for example, DP8 felt that it “really resonated with how I was feeling.” During periods of high mood and energy, the writing activity was also well received since it could be used as a fun break from work and an opportunity to help others.

Although participants appreciated the ability to receive messages tailored to their mood and energy, we received contradictory opinions about the number of questions we sent to achieve this functionality. Some participants felt that the dialogue included too many back-and-forth questions to achieve emotion labeling. During situations when they were experiencing low energy, DP27 felt insufficiently motivated to answer so many questions and wanted to receive an activity suggestion more rapidly. On the other hand, participants like DP22 and DP29 were willing to answer even more questions. DP29 said,

“If I say something like my mood is high, I think that it would be good to maybe include a question that says “What contributed to your high mood today?” so that it can give the participant an opportunity to reflect on their day, see what’s contributing to that mood … If it’s a low mood, then they will fully check-in and identify the trigger for it so that they can make that connection.”

(DP29)

Continuing the feedback regarding the emotion labeling portion of the dialogue, DP30 and DP31 suggested that there should be more options for specifying their mood as they felt that the given options failed to cover how they were feeling. DP10 noted that a person can experience several emotions at the same time, so the dialogue should have allowed users to select multiple options. Finally, while we included a message to acknowledge the emotion that the user had selected (e.g., “Sounds like you are feeling angry right now”), a couple participants like DP32 felt that the system’s repetition of what user had just said came across as condescending and judgemental.

6.3. Associations Between Contextual Variables

While we created two separate dialogues to explore the importance of contextual factors independently, our findings revealed that participants frequently drew connections between their daily schedule and their affective state. They pointed out how their mood and energy levels fluctuate during different parts of the day. For example, DP4 noted that their energy level tends to be low in the morning just after waking up:

“If it’s early in the morning when I just get up from bed, I’m really irritated. Like, I don’t want to say anything and I don’t want to reply.”

(DP4)

Confirming initial concerns voiced during our formative study, participants occasionally experienced anxiety in the evening as they reflected on the workday that had passed. Several people also said that they experienced undue stress as they anticipated a busy schedule, whether it was at the start of the busy day or the preceding night. DP32 reflected on this connection:

“On Monday, I am super busy. So if the program knows in advance about my busy days, it could kind of prompt me before the next day … probably like in the evening. I usually feel very overwhelmed on Sunday night.”

(DP32)

People also shared that they required less support when they were surrounded by other people. DP34 mentioned that when they attended parties in the afternoon, they wanted to enjoy the time with their friends rather than be distracted by messages. In fact, they speculated that a message during those times could have even worsened their mood by reminding them of their negative emotions. Some people expressed that they did not want to be reminded of strategies for managing their negative emotions even while being surrounded by coworkers for similar reasons. However, in the era of remote work induced by the COVID-19 pandemic, the distinction between working and non-working hours became blurred, so people like DP30 who were working from home recognized the value of suggestions for mental health support during work hours.

7. DISCUSSION

In our discussion, we first summarize the key findings of our work and emphasize our contribution as it relates to adaptive DMH tools and context-aware computing. We then provide some design recommendations based on our findings, after which we conclude by enumerating some of the limitations of our work.

7.1. Key Insights

7.1.1. RQ1: Types of Contextual Factors That Were Identified.

Our work contributes to the long line of work on context-aware computing [6, 22, 63, 82, 98, 109] by recognizing the role of context dictating people’s experiences with digital interventions. We saw that people’s daily schedule and affective state are perceived to be important contextual factors that influence how they interact with a text messaging system for managing psychological wellbeing. These observations support findings from prior HCI literature in the digital mental health domain; for example, Bhattacharjee et al. [6] advocated accounting for users’ schedules to increase intervention relevance, while Kornfield et al. [63] suggested that DMH tools should bifurcate the user experience depending on whether they are experiencing low versus high mood and energy.

To extend this literature, we report diverse perspectives regarding how these factors impact one’s receptivity toward different intervention content. For example, some people felt that they would appreciate an encouraging and positive message in the beginning of the day; as the day progresses, they would be less likely to check their phone because of their daily commitments like work or school. Another group of participants expressed a desire for receiving messages after work hours since that was when they had free time or experienced loneliness and distressing thoughts that messaging could counteract. Past work has also found the need for support after work hours particularly for individuals who tend to deviate from the usual circadian cycle (e.g., going to sleep late at night and waking up late in the morning) [28, 128]. Extending Kornfield et al.’s commentary on affective state [63], participants across both studies generally expressed the need for support while experiencing low mood and energy, preferring more passive forms of support. While they were willing to complete an emotional labeling task in order to receive appropriate forms of support, participants had conflicting opinions about the number of interactive messages to which they would be willing to respond. Some found answering the three questions in our affective state dialogue overwhelming, while others wished for longer conversations to achieve further customization and allow for deeper self-reflection.

Extending the findings from our formative work, our deployment study revealed additional associations between people’s daily schedule and affective state, providing insights on how people experience the temporality of everyday life changes [98]. Several participants mentioned feeling stressed due to anticipating an upcoming busy schedule, while others shared that their emotions varied depending on whether they were alone or surrounded by people.

7.1.2. RQ2: Adjusting Text Messaging Based on Contextual Factors.

Throughout our investigations, participants identified several key dimensions of text messaging that may be adjusted in relation to the user’s context to support psychological wellbeing:

• Message volume: Past work has suggested that users have very different preferences about the volume of messages they wish to receive from a digital health intervention [7, 78]. Our findings build on this work by suggesting that, even for the same individual, tolerance for messages fluctuates widely based on context. Generally, participants did not appreciate interruptions during work hours. Too many message notifications might distract users from their tasks, creating annoyance and an overall negative user experience [6, 131]. However, these findings also suggest that follow-up messages and reminders are sometimes important. Our participants proposed strategies for strategically scheduling follow-up messages, such as sending an informational message in the morning to prepare them for an activity in the afternoon. Several people also advocated that messaging systems should be respectful of their decreased willingness to be responsive during periods of low mood and energy, so sending too many notifications during those times could be overwhelming.

• Required effort to engage: Literature on education and social media has shown that different digital activities can require varying degrees of cognitive effort [13, 39, 126]. For example, researchers have shown that browsing a news feed necessitates less effort than contributing to one. Our findings reveal that such a differentiation between passive versus active engagement also extends to automated text messaging programs [29], with some passive support messages not requiring deep engagement and other messages requiring users to respond or carry out activities. Furthermore, we find that the amount of effort a person is willing to exert depends upon their context. Although active engagement with content is generally more helpful for improving learning and wellbeing [12, 13, 125], our results suggest that users are often unwilling to exert significant effort during periods of low mood and energy. At such times, we observe that passive engagement can play a key role in maintaining a baseline connection to the support system without overwhelming users.

• Time sensitivity: Related to the required effort commanded by messages is the urgency with which the message conveys its suggestions [77]. Participants often objected when the system prompted a particular task they did not have time to complete; on the other hand, brief support messages and reminders were generally well received even when participants were busy since they demanded less task-switching. Busy users often delayed larger tasks tangential to their work, so they felt that an intervention would be more acceptable if it acknowledged and normalized delays (e.g., “Here is an exercise to try when you have time”). Such strategies may be particularly helpful during work hours or when the user’s schedule is unknown.

These findings can provide useful guidelines for selecting tailoring variables and creating decision rules in JITAIs for mental health management [53]. In particular, they suggest ways of adapting specific elements of text messaging (e.g., volume, required effort, time sensitivity) to account for the user’s context, some of which we propose in Section 7.2. By highlighting the personal and complex interaction between contextual factors and people’s intervention needs, our work demonstrates how designers can carry out formative investigation work with users to understand the subtle changes in their expectations regarding context-aware DMH tools.

7.2. Design Recommendations

Based on the design tensions we observed in both of our studies, we provide recommendations for how DMH tools designed to support psychological wellbeing can adapt their functionality based on contextual factors:

7.2.1. Gathering Detailed Schedule Information.

Despite the fact that our findings indicate the need for adapting messages according to people’s schedules, text messaging platforms are not ideally suited for gathering such information on a daily basis. A suboptimal way to accomplish this would be to send daily requests for people to answer questions about their schedule, whether as a single question that requires them to type out all of their activities or as a series of yes-or-no questions about their availability at different hours. This approach can either be cumbersome or insufficient for capturing the nuances of a person’s schedule [6]. Another way to accommodate people’s schedules is by allowing for on-demand messaging wherein users can request messages when they feel the need for support. However, we recognize that some people might lack the energy or forget to reach out for support as they experience negative emotions.

As an alternate approach, text messaging systems could be integrated with digital calendars, which have become a pervasive tool for organizing events in both personal and professional settings [14]. By showing the time periods when a person is available or unavailable, calendars can help text messaging systems better integrate themselves into a person’s schedule. For example, days with many meetings and fewer breaks have been found to be an indicator of increased workplace stress [48, 74]. A text messaging system might use calendar information to identify when a person is attending consecutive meetings or classes and remind them to take short breaks while also keeping the volume of messages acceptably low. Such integration could be particularly important for people who have unusual routines, such as those who work at night or weekends. Access to one’s digital calendar would also make it easier to build cohesive dialogues over the course of the day, seeding background information before meetings and commitments while sending reminders later in the day after these commitments are over. Regardless of how digital calendars are integrated into a text messaging system, careful consideration should be taken to ensure user privacy when such information is monitored.

7.2.2. Incorporating Sleep and Physical Activity Information.

Across both of our studies, people often indirectly talked about their sleep as an important factor on their ability to engage with messages. People who wake up late in the morning found themselves rushing to their classes or work and, as a result, they did not find time to notice morning messages. Some people also shared that they experience irrational thought patterns after their workday, leading them to have sleep troubles. Sleep is an important indicator of one’s mental state [75, 79, 100], making it relevant to one’s mood and energy. The same can be said for physical activity and movement [93, 107].

Mobile phone sensors are capable of tracking sleep and physical activity with moderate accuracy [19, 100]. By simply identifying when a person first opens their phone in the morning, a text messaging system could infer an upper bound for the user’s wakeup time. Comparing that time to the user’s typical sleep schedule, a text messaging system can decide whether they have sufficient time to read a message or if they are going to feel rushed while getting ready for the day. Conversely, detecting that the user is not asleep after their normal bedtime may indicate that they are having trouble falling asleep due to distressing thoughts. In such situations, a text messaging system could provide supportive messages and strategies for improving their sleep cycle. Mobile phone sensors like accelerometers, gyroscopes, and GPS sensors can also provide precise and accurate information about a user’s movement (e.g., walking, running, in vehicle, on bicycle) [80, 130]. Together with digital calendars, these sensors can be used to detect when an individual is available and to personalize the message content accordingly.

7.2.3. Mitigating Negative User Experiences.

JITAIs ideally fade into the background of a person’s day-to-day experiences, intervening only when necessary to initiate or sustain positive behavior changes [87]. Sensor-based technologies and algorithms for detecting the user’s context can occasionally be incorrect, leading to the delivery of interventions at inopportune moments. Literature on human-centered design suggests that users remember negative experiences more strongly and for a longer period than positive ones, ultimately having greater impact on their behavior [73]. Consistent with this finding, some of our deployment study participants suggested that systems should not only account for the most opportune moments for intervening but also inconvenient times for delivering an intervention. Participants informed us of several situations when they would not like to receive a message, such as mornings when they are rushing to prepare for the day or during exams or important meetings. Hence, text messaging applications should actively collect information about inconvenient times for content delivery using the aforementioned data sources (e.g., digital calendars and mobile phone sensors).

Negative user experiences may also be influenced by the content or language used in a given message. Participants in our studies occasionally raised concerns about the perceived urgency of our message suggestions, so future interventions may want to consider explicitly giving users the flexibility to follow through whenever it is most convenient for them. Doing so may result in participants delaying and forgetting about the messages altogether; hence, follow-up and reminder messages should be strategically placed. Giving users the option to snooze messages for a short period of time may also provide users with control and reduce feelings of being rushed or overwhelmed.

7.2.4. Balancing Demand with Mood and Energy.

Our findings suggest a few design tensions with regards to how messages should be constructed to support people experiencing low mood and energy. Participants across both studies generally said that they were not keen to respond to messages when they experience negative emotions with the exception of brief check-in messages. However, our affective state dialogue required users to answer three questions in order to receive such a personalized message, which was perceived as too demanding by a subset of participants. On the other hand, some people wanted the flexibility to express their emotions in a way that often required more than three questions. One way to address this tension could be by giving users an open-ended prompt where they can express their feelings with as much detail as they would like and leveraging sentiment analysis techniques to infer their affective state automatically. The length of the user’s response may also be an indicator of their availability, with shorter responses suggesting that the user is unlikely to respond to follow-up questions.

7.2.5. Facilitating Social Connection and Diffuse Sociality.

Our findings touched upon some social aspects of the text messaging experience. For example, some participants shared that they did not wish to engage with text messages when they were around other people they knew; however, they generally expressed willingness to connect with people experiencing similar problems to their own via text messaging platforms. We observed that when their mood was high, the participants from the deployment study appreciated drafting supportive texts to help others going through difficult times. When their mood was low, receiving messages could provide a sense of being less alone. Our findings are consistent with the need to support “diffuse sociality” [11] (i.e., the feeling of connection with others without direct interactions) in DMH interventions. Many individuals with depression prefer to engage with DMH tools independently [102], but may still benefit from building a sense of connection with others through opportunities for diffuse sociality.

Text messaging platforms can facilitate diffuse sociality in several ways beyond the peer-to-peer message exchange examined here. For example, users can engage with peers’ first-person narratives of recovery so that they can draw explicit connections between narrative characters and themselves [7]. In addition, guided chat could be applied to buttress more in-depth peer-to-peer support interactions. For example, multi-step structured prompts have provided guidance both for the sharing of personal experiences and for provision of personalized support [92]. Based on our findings, some of these strategies may come across as overwhelming during periods of low mood or high busyness as they require users to read and respond to a sequence of messages; however, strategically deploying them throughout the week at opportune moments can mitigate such concerns.

7.3. Limitations

Our text messaging probe was designed specifically to support young adults between the ages of 18 and 25. Although they spanned different ethnicities, our participants were living in North America at the time we ran our study. Hence, our findings should be situated within a particular cultural context. Although our findings uncovered daily schedule, mood, and energy level as critical contextual factors for DMH tools to consider, other issues like busyness and social proximity were occasionally surfaced. With a different population, these issues might have been even more prominent. Similar lines of work could also explore how the role of contextual factors may vary depending on individuals’ habits of phone usage.

We also recognize that our deployment study examined each contextual factor in relation to bespoke message dialogues with a handful of message varieties. Although we encouraged participants to comment on other types of messages they anticipated would be better suited to varied contexts, future work could further investigate alternative message categories.

Some people also reported that the options we provided during the emotion labeling portion of the affective state dialogue failed to cover their experiences. This could be due to our framing around the circumplex model, which has its limitations despite being widely used model in the literature [134]. Most notably, researchers have criticized the circumplex model for not relying upon clear theoretical guidelines in the mapping between its dimensions and various emotions [83, 101]. Future work could explore other models for operationalizing emotion labeling, such as Plutchik’s model [81] or the Pleasure-Arousal-Dominance model [76].

Finally, we note that what participants might appreciate most at a particular moment is not necessarily what will bring the greatest benefit regarding reducing symptoms. Therefore, DMH tools should be evaluated not only according to how well they can reach and engage users, but also according to how well they can reduce symptoms of depression and anxiety by producing new thoughts and behavior patterns.

8. CONCLUSION

Understanding how users view dynamic contextual factors in their lives is important to the design of DMH tools, as it enables the delivery of interventions that are well-matched to their needs. In this work, we investigated how such variables influence the reception of text messaging systems for supporting psychological wellbeing. Our formative work and deployment study revealed one’s daily schedule and affective state to be prominent contextual factors that shape how users receive and act upon messages. Participants across both studies felt that text messaging systems should adapt the message volume, time sensitivity, and required effort for the interventions according to those contextual factors. We hope that our work represents a major first step towards the development of user-centered, context-aware DMH tools.

CCS CONCEPTS.

• Human-centered computing → Empirical studies in HCI.