Skip to main content
NCBI home page
Internet Interventions logoLink to Internet Interventions
. 2023 Sep 28;34:100676. doi: 10.1016/j.invent.2023.100676

User experiences of college students using mental health applications to improve self-care: Implications for improving engagement

Ria Nagar 1, Heather D Quirk 1, Page L Anderson 1,⁎
PMCID: PMC10587513  PMID: 37867616

Abstract

The purpose of this study is to evaluate themes on ‘user experiences’ among college students (N = 265) enrolled in an upper-division Psychopathology course who were assigned a project in which they were instructed to identify a self-care goal, choose from a list of six mental health-focused mobile applications (apps) provided by the instructor, and use the app over the course of three weeks to support progress towards their goal. Prior literature on user experiences typically evaluates user reviews, or asks participants to reflect on past app use or anticipate future use. Students reported their experiences using the app during key decision points: app selection, while using the app, and at the conclusion of the assignment. Using thematic analysis, results identified seven central themes and eight subthemes pertaining to the content of the app (e.g., app features) and the context of using the app (e.g., classroom assignment). Content-wise students liked: 1) features with a strong evidence base, namely, thought diaries and guided meditations; 2) progress tracking, because it increased awareness of mood/stressors, motivated students to see improvement, and helped them stay on track. Students appreciated having 3) crisis support resources; 4) app interfaces that allowed for customization (poor app interfaces were sometimes cited as the reason for disengagement); and 5) apps that included varied, comprehensive resources such that it felt like a one-stop shop. In addition to the content of features and design interface, the context in which mental health apps are introduced and used is important. The remaining themes related to the context in which the app was used, including 6) preparation for app usage, such as reviewing scholarly/credible sources, and 7) social support from fellow students completing the same assignment. Future research should evaluate the ‘who, what, when, why, where, and how’ of app utilization during key decision points, such as initial app selection or subscription renewal, to better understand the impact of user experience on engagement.

Keywords: College students, User experience, Mobile mental health applications, Engagement, Self-care

Highlights

  • •

    Evaluated user experiences at key decision points - app selection, overcoming obstacles to app use, whether to stop using app.

  • •

    Preparing users for app use with research on self-care/mental health positively influences user attitudes.

  • •

    Identifying low perceived need and stigma as barriers to mental health services affects user experience.

  • •

    The content of apps - and the context in which apps are used - affect user experiences.

  • •

    Evaluating the ‘who what when why where – and how’ of app use provides insight on engagement.

1. Introduction

The importance of place, space, and design on user experience has been studied for thousands of years. Publications dating back to the 1960s describe the powerful ways engineering and psychology intersect in the science of design (Carroll, 1997). In the modern era, literature focuses on personal computers, mobile devices, and the applications (apps) that populate them to better understand and optimize the interaction between humans and digital mental health interventions (IQVIA, 2021; Hassenzahl and Tractinsky, 2006).

Digital mental health interventions are a potential means of addressing the mental health crisis among college students (Lattie et al., 2020). College counseling centers provide access to mental health services for college students, but they are under-resourced, have long waiting lists, and struggle to meet student demand (Byrne et al., 2021; Center for Collegiate Mental Health, 2020; Lattie et al., 2019, LeViness et al., 2018). Mental health apps are a promising avenue to address these problems because college students report high levels of technological literacy, smartphone ownership, and openness to mental health treatment (Kern et al., 2018; Melcher et al., 2022).

There is an important gap in the scientific literature on user experience among college students, although the only study to evaluate user experiences of college students with mental health apps designed for them showed promising results. That study, a single-arm pilot of IntelliCare, a suite of mental health apps for college students, reported that the suite was perceived as usable and engaging (Lattie et al., 2020). These findings are important because low engagement with mental health apps is widely recognized as a major challenge—often attributed to poor app usability and design, privacy concerns, and limited emergency resources (Torous et al., 2018) and limits mental health apps' ability to fulfill their promise to address the shortage of mental health services (Su and Anderson, 2022).

Empirical research shows that user experience metrics, such as ease of use and user interface, may be primary drivers of engagement (Han and Lee, 2018; Melcher et al., 2022). Public-facing guides to mental health apps, such as the One Mind PsyberGuide (https://onemindpsyberguide.org/), IQVIA's AppScript Score (https://www.appscript.net/score-details), and the ‘American Psychological Association’ App Advisor (https://www.psychiatry.org/psychiatrists/practice/mental-health-apps) also include information about user experience, suggesting that this is an important issue to consumers. This study examines the responses of college students to a classroom assignment in which they were instructed to identify a self-care goal, choose one of six mental health-focused apps provided by the instructor, and use the app over the course of three weeks to support progress towards their goal. The objective of this study is to examine college students' user experiences of mental health apps at key points of engagement – selecting an app, addressing obstacles to using the app, and deciding whether to continue to use the app after the assignment concluded.

2. Methods

2.1. Sample description

Students (N = 265) enrolled in an asynchronous, online upper-division psychology class during Fall 2020 and Spring 2021 semesters completed a Self-Care Project in which they used an app for three weeks to make progress on a self-identified self-care goal. Students' responses were de-identified prior to analyses, and demographic information for students in the classes is not available. As a proxy, demographic characteristics of the university's undergraduate study body are shown in Table 1.

Table 1.

Demographic characteristics of the Georgia State University Undergraduate Student Body.a

Demographics 2021 undergraduate enrollment N = 28,985 (%)
Race/ethnicity
 Black 42
 White 20
 Asian 15
 Hispanic 13
 Two or more races 6
 Non-resident alien 3
 Unknown 1
 American Indian/Alaska Native 0
 Native Hawaiian/Pacific Islander 0
Age
 24 & under 85
 25 & over 15
Gender
 Female 60
 Male 40
Open in a new tab
a

Note: sourced from https://nces.ed.gov. All categories and wording are copied directly from the source. They do not reflect the authors' preferences or choices.

2.2. Classroom assignment: Self-Care Project

Importantly, this assignment was not designed to answer a research question, but to support students during a stressful time – the COVID-19 pandemic (Quirk et al., 2023). At the beginning of the semester, students were divided into permanent groups of 10–11 members with whom they would interact to complete a Self-Care Project. The project required students to develop a goal for self-care, choose an app from a list provided by the instructor, and use the app for three weeks to make progress on their goal.

Prior to beginning the project, students were ‘cognitively prepared’ for the task via a description of a common barrier to seeking mental health services (low perceived need) and references to concepts they were learning in class (diathesis-stress model):

“Sometimes we are more willing to help others than to help ourselves. Did you know that research from the World Health Organization shows that most people who meet diagnostic criteria for a psychological disorder do not believe their problems are ‘bad enough’ to warrant addressing? The diathesis-stress model shows us that it is essential to take care of ourselves to reduce stress and improve coping in order to prevent psychological disorders from emerging.”

Then, students read the purpose of the assignment:

“The purpose of this assignment is for you to identify a self-care goal and to use an evidenced-based strategy to practice doing it (e.g., sleeping, eating, moving, having fun, feeling accomplished, getting social support, talking back to negative thoughts, cultivating gratitude, meditation).”

The purpose of the assignment was explicitly tied to a primary goal of the course - supporting student well-being:

“This assignment is designed promote your well-being by demonstrating the importance of self-care, using an app based on psychological science to practice self-care, and cultivating the habit of looking for scholarly/credible information based on psychological science when you have a question related to mental health and its treatment so that you can better help yourself or your loved ones.”

The Self-Care project was scaffolded by two discussion assignments, which gave students the opportunity to think, reflect, write, and receive feedback on their experiences—and maximized the likelihood that students would use the app for the full three weeks. The discussion assignments consisted of answering open-ended questions, incorporating and citing scholarly/credible information (e.g., peer reviewed articles), and replying to a groupmate's discussion post.

The first discussion assignment, in which students were asked to read an empirical article on self-care, was due the third week of class:

  • 1.

    “Identify the research you chose, describe the research in your own words, and explain how you found the research useful in thinking about self-care.”

  • 2.

    “What app did you choose and why did you choose it?”

The second discussion assignment focused on app use and progress. It was due the fifth week of class (presumably after using the app for two weeks):

  • 1.

    “What was it like to use the app to improve self-care?”

  • 2.

    “What obstacles did you encounter when trying to work your plan and how did you get around them (or not)?”

  • 3.

    “What progress (if any) did you see over time?”

The Self-Care Project also included questions about whether students would continue to use the app or continue to work on self-care after the assignment ended. These questions were due the sixth week of class:

  • 1.

    “Would you continue to use the app for self-care? Why or why not?”

  • 2.

    “Would you prioritize self-care (using an app or not), in the future? Why or why not?”

The full data set includes all questions and answers and is openly available from the Inter-university Consortium for Political and Social Research at doi:https://doi.org/10.3886/E182081V1. If students did not answer all questions, the responses to the questions they did complete were included in the data to be analyzed. Although the discussion questions were graded, the instructor explicitly stated that engagement with the app (e.g., frequency of app use) or success in meeting self-care goals would not be graded.

2.3. Application(s) description

Students were required to use one of six apps provided by the instructor. The apps were selected because they were available for no cost (or had a free trial), were available for iOS and Android devices, and either had evidence for efficacy or utilized tools with a strong evidence base. One app, The Safe Place, was included because, at the time, it was the only app specifically designed for the university's majority racial group—Black and African American students—and endorsed by the National Alliance for the Mentally Ill. The proportion of students selecting each app was as follows: Headspace (50 %), FearTools (21 %), MoodTools (15 %), The Safe Place (7 %), Intellicare (4 %), PTSD Coach (2 %), and Other (1 %). See Quirk et al. (2023) and Table 2 for further details.

Table 2.

Apps used for Self-Care Project.

Name Description Criteria Fulfilled
iOS and Android available No cost or trial period Scientific evidence for app’s efficacy Evidence-based features included Culturally tailored Citations of scientific evidence
MoodTools Evidence-based app that provides six CBT-informed tools to aid against clinical depression and negative moods. ☑ ☑ ☑ ☑ Su and Anderson (2022)
FearTools Evidence-based app designed with six CBT-informed tools to help users combat anxiety and stress. ☑ ☑ ☑
Intellicare A suite of apps that work together to help people deal with common causes of stress, depression, and anxiety. ☑ ☑ ☑ Mohr et al. (2017); Mohr et al. (2019); Graham et al. (2020)
PTSD Coach App designed by the VA for those who have, or may have, post-traumatic stress disorder (PTSD). Tools include relaxation skills, positive self-talk, and anger management. ☑ ☑ ☑ Bröcker et al., (2022); Hallenbeck et al. (2022); Owen et al. (2015)
The Safe Place App geared towards the Black community to bring more awareness, education, and hope about mental health and treatment-seeking challenges within this population. ☑ ☑ ☑
Headspace App that focuses on science-backed meditation and mindfulness tools to support mental health and well-being. ☑ ☑ ☑ Economides et al. (2018); Howells et al. (2016); Bostock et al. (2019); Kubo et al. (2018); Abbott et al. (2021)
Open in a new tab

This study was designated Non-Human Subjects research by the University's Institutional Review Board, H21192, after their determination that the research should be exempt from federal regulations as defined in 45 CFR 46 and that it falls within one or more of the eight exempt categories allowed by the institution and meets the organization's ethical standards. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

3. Theory/calculation

This study not only adds to the scant literature on user experiences among college studies, but also addresses some limitations of methods typically used to assess user experience of mental health apps. For example, user studies of mental health apps often ask participants to reflect on past app use or to anticipate what would be important to them if they were to use a mental health app (Palmer and Henderson, 2019; Kern et al., 2018; Melcher et al., 2022), which misses key information that could be collected by obtaining feedback in real time. Consumer reviews (e.g., in the iPhone's App Store or Android's Google Play) are another commonly used method to evaluate user experiences. A limitation of this approach is selection bias, because only a subset of users leaves reviews. In contrast, the classroom assignment approach asked students to reflect on their experiences while using an app for self-care during key decision points: app selection, while using the app, and at the conclusion of the assignment. Furthermore, students were prepared for app use by addressing common attitudinal barriers to the uptake and use of mental health services: low perceived need and stigma (Fleming et al., 2018). Low perceived need is addressed by the manner in which the assignment is introduced (“Did you know that research from the World Health Organization shows that a majority of people who meet diagnostic criteria for a psychological disorder do not believe their problems are ‘bad enough’ to warrant addressing?”) Stigma is minimized by requiring all students to use the app (whether they think they need to or not), for a non-stigmatizing task (assignment), framed using non-stigmatizing language (self-care), while learning about relevant content in a non-stigmatizing setting (classroom), and getting feedback in a non-stigmatizing manner (classmates that are required to respond) and from people with shared life experience and stressors (college students).

4. Results

4.1. Data analytic plan

All students' answers were de-identified and assigned a number by a teaching assistant who was not a part of the team evaluating this data. The data are fully anonymized and there is no record linking identifying information with the number assigned. Two research assistants organized the data in Excel sheets, question by question, and split up the data by question for individual review. A team of five undergraduate and graduate research assistants (including the two who organized the data) then coded all answers to their assigned questions. To ensure inter-rater reliability, the group reviewed each member's codes, reflected on reflexivity, and discussed observations. After five meetings, the team agreed upon major themes and subthemes regarding user experiences in totality and organized them into a clear hierarchy. This process was performed over the course of four months using a five-step thematic analysis procedure (Braun and Clarke, 2006).

4.2. Theme overview

Seven central themes and eight subthemes emerged regarding user experiences, app features, and the context in which the app was used. Students' responses showed they 1) had positive experiences using features with a strong evidence base, namely, thought diaries (1a) and guided meditations (1b); 2) liked using progress tracking because it increased awareness of their mood and stressors (2a), documented improvement (2b), and helped them stay on track via reminders (2c); 3) appreciated resources for crisis support; 4) found app interfaces important, including aspects they liked, such as customization (4a) and designs they didn't like, which were sometimes mentioned as reasons they would not continue to use the app (4b); and, 5) apps that felt like a one-stop shop. The remaining themes were related to the context of using an app as a part of class assignment, including that 6) preparation for app utilization was beneficial and the positive influence of incorporating scholarly/credible sources while reflecting on app use (6a) and 7) social support from fellow students completing the same assignment.

Under each theme and subtheme below, supporting quotes are provided. To find any quote in the full data set, use the participant ID number and the Excel sheet cell number we have provided after each quote. The full data set is openly available from the Inter-university Consortium for Political and Social Research at doi:https://doi.org/10.3886/E182081V1.

4.3. Features with a strong evidence base

Students wrote about two activities within apps that were particularly helpful and that also have a solid base in the empirical literature: thought diaries and guided meditation.

4.3.1. Thought diaries

“Being able to write my thoughts and feelings down in the app has really helped me with just helping clear my head and think properly, which I realized had a huge correlation with my anxiety…when I write my thoughts out on the app…it makes me understand myself, my flaws and helps me be a better version of myself.”

(P1040, L40)

“I also like that I have a journal on me at all times. I thought that journaling had to do with getting a notebook and a pencil, and finding somewhere quiet. I now can feel myself getting anxious, get my phone out, and write down my thoughts. ”

(P1139, L127)

One of the reasons students liked thought diaries is that it didn't require them to disclose personal information to another person.

“I sometimes don't feel comfortable talking to someone about my problems and situations because I don't want sympathy or for anyone to feel obligated that they have to speak on what I have going on. Fear Tools has allowed me to write out and express my feelings that only I am aware of in the ‘thought diary’.”

(P1065, L62)

Some students, however, reported that physical journals were preferable to digital thought diaries.

“I will be adapting its anxiety journal feature into my life by buying a real journal and filling it out…The anxiety journal has consistently been my favorite feature of Fear Tools, and I think it is better for me to create a real one that logs my thoughts.”

(P1107, L100)

“All that I found useful was the journal section, which I can do on physical paper. I could just continue to use the app for this, but I like the customization options on writing a tangible journal.”

(P2131, L280)

4.3.2. Guided meditations

“I enjoy Headspace in that the narrator tells you exactly what to do step by step, what to do if your mind wanders elsewhere.”

(P1115, L108)

“Guided meditation and this research article have taught me that compassion towards myself is significant… Taking a few minutes every day to reconnect with mindfulness and the present has helped me.”

(P1061, J58)

4.4. Progress tracking

Students liked tracking their progress.

“I love seeing the progress I make as the days go by. It keeps me on track and reminds me when I need to make time for myself…because I would certainly forget about my goals… Being able to have something that keeps me organized is a huge benefit with such a busy schedule I keep for myself.”

(P2132, L281)

“I also really appreciated the fact that it kept with your progress to see how you were doing or if you were accomplishing anything.”

(P1095, L89)

4.4.1. Increased awareness of mood and stressors

“The app has been beneficial in that it has provided me with a way to also track my symptoms and feelings if need be. These trackers ask questions about certain physical and mental symptoms that I would not have thought about myself.”

(P1000, cell L2)

“It has allowed me to really consider how intrusive my negatives thoughts were.”

(P2086, L235)

“The FearTools app helped me with this significantly, and by identifying my stressors, I was better able to stabilize my anxiety.”

(P1015, J16)

4.4.2. Seeing improvement

“Now looking back at all my diary entries, I can definitely see a huge improvement with my mood.”

(P1004, L6)

“I can log my daily activity and the before and after effect on my mood, which helps me keep track of the effect the activity had on me.”

(P1010, L11)

4.4.3. Reminders helped to stay on track

“I also liked the fact that it had reminders and it would let you know if you hadn't used the app for that day. I thought that was beneficial for those who have a lot on their plate, and they could forget to use the app that day.”

(P1095, L89)

4.5. Crisis support

Students reported the importance they felt around safety features included in the application.

“One of my favorite features on the app was the safety plan… the app develops a suicide safety plan to keep you safe and utilize emergency resources during a crisis.”

(P1057, J55)

4.6. App interface

Students' comments showed the importance of a good interface.

4.6.1. Ability to customize is positive

“I like how the breathing exercise looks because of the colors and shapes you can pick and change on the app. The customization feature the app has makes one feel a part of the app.”

(P1080, L75)

4.6.2. Poor user interface is negative

“It is not visually pleasing, and it does not have some of the features I want.”

(P1110, L103)

“I would not continue to use this APP mainly due to its interface.”

(P1152, L138)

“I was not a fan of the layout of the app. It seems to be childlike and overwhelming at time.”

(P2037, L186)

Students even cited poor user interface and lack of customization options as a reason for quitting the app.

“However, instead of keeping a virtual diary on my phone, I will be using a personalized notebook for journaling … If I decorate the book to my liking, it would motivate me further to take the time and maintain a diary.”

(P1003, L5)

4.7. Being a “one-stop shop”

Students liked the convenience of having a variety of activities and resources within one app,

“There are many apps adjacent to MoodTools, but the ability to switch between journaling and resources to cope with how I'm feeling is nice to have in a single app.”

(P1152, L138)

“The music, calm ambiance, serene environment, and soothing words are also all in one place. Headspace gives users all that they need in one location, which makes it easy to access moments of relaxation and mindfulness whenever it is needed.”

(P2029, L178)

“I also plan on continuing the use of this app because it lets me log my negative thoughts and record my alternative ‘helpful’ way of thinking all in one place.”

(P2023, L172)

4.8. Psychoeducation in preparation for using app is helpful

Students described how psychoeducation can be helpful.

“Considering my area of self-care was managing anxiety, the figures on anxiety disorders and their prevalence, provided by NAMI [National Alliance on Mental Illness] helped establish a frame of reference for the severity of the issue I am tackling, and how applicable my knowledge of self-care would be to others…NAMI's research also helped identify symptoms of anxiety and potential consequences.”

(P1076, J71)

4.8.1. Positive influence of scholarly/credible resources

For the assignment, students were required to read a peer reviewed article related to their area of self-care and reflect upon it in a discussion post. They indicated that this advance reading helped them prepare for the experience of app utilization.

“After coming across this article, it was clear to me how applicable this research was to my self-care… I realized if mindfulness works for healthcare workers in distress than it's bound to have some benefit for a full-time student.”

(P1009, cell J10)

4.9. Social support

“The best thing about the class I really loved were the discussion posts surrounding the self-care project, we not only had to write our thoughts on the app, but also elaborate them to our peers, which was great to me and it really stood out, even though this is an online class and we do not connect with our peers as much, the app's discussion posts very much helped me connect to most of my peers.”

(P1040, J40)

5. Discussion

Prior research on user experiences with mental health apps has typically asked participants what features they would/would not want in a potential app, or what they did/did not like after using an app (Almathami et al., 2020; Arigo et al., 2019; Larsen et al., 2019; Torous et al., 2018). This study used a different approach; information about user experience were gleaned from responses to questions asked in real time that corresponds to key decision points in using an app to support a self-care goal: deciding what app to choose, identifying obstacles to progress and how to deal with them, and considering whether to engage with the app when it is no longer required to do so.

Results of the study are both consistent and inconsistent with prior research. Students reported that they liked thought diaries, guided meditations, and progress tracking – features that have been identified as well-liked in prior research using surveys and objective user metrics (Fitzpatrick et al. (2017). Also consistent with prior research, students reported that customization was important (Luxton et al., 2011), poor user interface (e.g., ‘not visually pleasing, child-like’) was a reason to disengage from the app (Alqahtani and Orji, 2019), and ‘crisis support’ was appreciated (Huguet et al., 2016). Crisis support is a long-standing topic among clinicians and developers who appreciate the scalability of apps but are concerned about safety and liability (Esmaeilzadeh, 2020; Esmaeilzadeh et al., 2021; Luxton et al., 2011).

Inconsistent with prior research, students in this study liked psychoeducation, social support, and reminders. Perhaps it is not surprising that students enrolled in an upper level psychology course appreciate psychoeducation, whereas psychoeducation is rarely used in other contexts (Su and Anderson, 2022). Past research suggests that interaction with other app users – including peers - is viewed as problematic (Al-Asadi et al., 2014); students in this study may have viewed social support positively because it came from fellow students with the shared experiences of being in college, in the same class, and doing the same assignment. Finally, though reminders are perceived as burdensome in general (BinDhim et al., 2015), students who are learning organization and time management may rely more on reminders or find them more helpful.

These results highlight the importance of the context in which a mental health app is introduced to potential users. In this study, students were introduced to a mental health app in a familiar setting, for a familiar task, framed using wellness-oriented language (self-care), while engaging with relevant content and getting feedback in a familiar way from people with shared life experience and stressors. Requiring students to evaluate research on self-care and reflect on its relation to students' goals (via class discussion posts) seemed to help students ‘open up’ to the idea of using an app to improve self-care. Relative to the typical context in which college students access mental health services (e.g., in-person services at college counseling centers), this unique context (a class assignment that uses destigmatizing, wellness-oriented language) may have mitigated common barriers to mental health app uptake and engagement, including low perceived need and stigma (Fleming et al., 2018). Future research using experimental designs is needed to determine the extent to which these factors influence user experience of, engagement with, and ultimately benefit from mental health apps.

5.1. Limitations

There are important limitations of this study that should be addressed in future work. As this was a graded assignment, response bias—specifically social desirability bias—is likely. Even though students were told that progress on their goal or amount of app use would not affect their grade, some students may have felt pulled to respond positively. This concern is mitigated by the fact that some students reported that the app was not useful or that little progress was made on their self-care goal. Secondly, students were not asked to provide demographic information as part of their project answers, so examining responses according to demographic characteristics (e.g., sex, race, age) was not possible. This is problematic because many studies of psychological interventions do not report participant characteristics (Johnson and Anderson, 2016), even though many of these characteristics (e.g., race, gender) are well-known determinants of health (Miani et al., 2021; Paradies et al., 2015). Finally, psychological research has been criticized because of its reliance on WEIRD populations (White, Educated, Industrialized, Rich, and Democratic) as ‘standard’ subjects, which may be among the least representative populations (Henrich et al., 2010). This is problematic because digital mental health is seen as a tool to increase access to psychological service to those with low access (e.g., non-WEIRD populations). Similarly, results of this study may not generalize to non-college populations or college students not enrolled in a psychology class.

5.2. Future directions

Research on the context in which mental health apps are introduced to and used by people who could potentially benefit from it is sorely needed. Studies on user experience among communities of color is needed to create more culturally relevant and targeted interventions (Ellis et al., 2022). Historically, research on mental health interventions has not done a good job reporting the demographic characteristics of their participants (Medina et al., 2021), and it is imperative that future research on digital mental health interventions do so to evaluate their potential to improve mental health equity. Future research may use the results from this study (and others) to inform app development and marketing, as well as encourage user-centered design (Singh and Sagar, 2022). Finally, students sometimes shared how their experience with an app sparked ideas about ways to improve self-care without the app. Future research should evaluate the ‘indirect’ effects of interactions with mental health apps that could lead to benefits for college students (Torous et al., 2017); such research would benefit from theoretical models.

6. Conclusion

This work sheds light on the importance of the ‘who, what, when, why, where, and how’ in understanding user experiences. College students (who) found that features and functions of a mental health app (what) are important to make progress self-care (why) in the context of a time-limited (when) classroom assignment (where). Requiring students to both review empirical research on self-care and to include scholarly/credible sources in their reflections on app use in real time (how), indicated the positive role that engaging with science can play in preparing students to improve self-care using an app. Additionally, consulting scholarly/credible sources helped deepen college students' thinking about their experiences. Future research on contextual factors has the potential to enhance app engagement, encourage sustained app use (at least long enough to determine whether it is beneficial), and lead students to pursue resources and avenues outside the app to facilitate self-care and well-being.

Declaration of competing interest

Ria Nagar is funded by the HRSA GPE “Training in Integrated Pediatric Psychology Services” (TIPPS) training fellowship.

Heather Quirk is funded by the HRSA GPE “Training in Integrated Pediatric Psychology Services” (TIPPS) training fellowship.

References

  1. Abbott D.A., Anderson P.L., Lack C. Does using a mindfulness app reduce anxiety and worry? A randomized-controlled trial. J. Cogn. Psychother. 2021;36(2) doi: 10.1891/JCPSY-D-20-00058. [DOI] [PubMed] [Google Scholar]
  2. Al-Asadi A.M., Klein B., Meyer D. Posttreatment attrition and its predictors, attrition bias, and treatment efficacy of the anxiety online programs. J. Med. Internet Res. 2014;16(10) doi: 10.2196/jmir.3624. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Almathami H.K.Y., Win K.T., Vlahu-Gjorgievska E., Bontovics Á. Barriers and facilitators that influence the use of mobile health applications for self-management: a systematic review. J. Med. Syst. 2020;44(7):118. doi: 10.2196/16407. [DOI] [PubMed] [Google Scholar]
  4. Alqahtani F., Orji R. Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization. 2019. Usability issues in mental health applications; pp. 343–348. June. [DOI] [Google Scholar]
  5. Arigo D., Jake-Schoffman D.E., Wolin K., Beckjord E., Hekler E.B., Pagoto S.L. The history and future of digital health in the field of behavioral medicine. J. Behav. Med. 2019;42(1):67–83. doi: 10.1007/s10865-018-9966-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. BinDhim N.F., Hawkey A., Trevena L. A systematic review of quality assessment methods for smartphone health apps. Telemed. e-Health. 2015;21(2):97–104. doi: 10.1089/tmj.2014.0088. [DOI] [PubMed] [Google Scholar]
  7. Bostock S., Crosswell A.D., Prather A.A., Steptoe A. Mindfulness on-the-go: effects of a mindfulness meditation app on work stress and well-being. J. Occup. Health Psychol. 2019;24(1):127–138. doi: 10.1037/ocp0000118. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Braun V., Clarke V. Using thematic analysis in psychology. Qual. Res. Psychol. 2006;3(2):77–101. [Google Scholar]
  9. Bröcker E., Olff M., Suliman S., Kidd M., Mqaisi B., Greyvenstein L., Kilian S., Seedat S. A clinician-monitored 'PTSD Coach' intervention: findings from two pilot feasibility and acceptability studies in a resource-constrained setting. Eur. J. Psychotraumatol. 2022;13(2) doi: 10.1080/20008066.2022.2107359. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Byrne A., Barber R., Lim C.H. Impact of the COVID-19 pandemic – a mental health service perspective. Prog. Neurol. Psychiatry. 2021;25:27–33b. doi: 10.1002/pnp.708. [DOI] [Google Scholar]
  11. Carroll J.M. Human-computer interaction: psychology as a science of design. Annu. Rev. Psychol. 1997;48:61–83. doi: 10.1146/annurev.psych.48.1.61. https://home.cs.colorado.edu/~martin/Csci6402/Papers/carroll97.pdf [DOI] [PubMed] [Google Scholar]
  12. Center for Collegiate Mental Health 2019 Annual Report. 2020. https://eric.ed.gov/?id=ED602859
  13. Digital Health Trends 2021: Innovation, Evidence, Regulation, and Adoption. IQVIA; 2021. https://www.iqvia.com/-/media/iqvia/pdfs/institute-reports/digital-health-trends-2021/iqvia-institute-digital-health-trends-2021.pdf?&_=1652118184995 [Google Scholar]
  14. Economides M., Martman J., Bell M.J., Sanderson B. Improvements in stress, affect, and irritability following brief use of a mindfulness-based smartphone app: a randomized controlled trial. Mindfulness. 2018;9:1584–1593. doi: 10.1007/s12671-018-0905-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Ellis D.M., Draheim A.A., Anderson P.L. Culturally adapted digital mental health interventions for ethnic/racial minorities: a systematic review and meta-analysis. J. Consult. Clin. Psychol. 2022;90(10):717–733. doi: 10.1037/ccp0000759. [DOI] [PubMed] [Google Scholar]
  16. Esmaeilzadeh P. Use of AI-based tools for healthcare purposes: a survey study from consumers’ perspectives. BMC Med. Inform. Decis. Mak. 2020;20(1):1–19. doi: 10.1186/s12911-020-01191-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Esmaeilzadeh P., Mirzaei T., Dharanikota S. Patients’ perceptions toward human–artificial intelligence interaction in health care: experimental study. J. Med. Internet Res. 2021;23(11) doi: 10.2196/25856. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Fitzpatrick K.K., Darcy A., Vierhile M. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR Ment. Health. 2017;4(2) doi: 10.2196/mental.7785. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Fleming T., Bavin T., Lucassen M., Stasiak K., Hopkins S., Merry S. Beyond the trial: systematic review of real-world uptake and engagement with digital self-help interventions for depression, low mood, or anxiety. J. Med. Internet Res. 2018;20(6) doi: 10.2196/jmir.9275. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Graham A.K., Greene C.J., Kwasny M.J., Kaiser S.M., Liepnis P., Powell T., Mohr D.C. Coached mobile app platform for the treatment of depression and anxiety among primary care patients: a randomized clinical trial. JAMA Psychiatry. 2020;77(9):906–914. doi: 10.1001/jamapsychiatry.2020.101. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hallenbeck H.W., Jaworski B.K., Wielgosz J., Kuhn E., Ramsey K.M., Taylor K., Juhasz K., McGee-Vincent P., Mackintosh M.A., Owen J.E. PTSD Coach Version 3.1: a closer look at the reach, use, and potential impact of this updated mobile health app in the general public. JMIR Ment. Health. 2022;9(3) doi: 10.2196/34744. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Han M., Lee E. Effectiveness of mobile health application use to improve health behavior changes: a systematic review of randomized control trials. Healthc. Inform. Res. 2018;24(3):207–226. doi: 10.4258/hir.2018.24.3.207. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Hassenzahl M., Tractinsky N. User experience - a research agenda. Behav. Inform. Technol. 2006;25(2):91–97. doi: 10.1080/01449290500330331. [DOI] [Google Scholar]
  24. Henrich J., Heine S.J., Norenzayan A. The weirdest people in the world? Behav. Brain Sci. 2010;33(2–3):61–83. doi: 10.1017/S0140525X0999152X. discussion 83-135. Epub 2010 Jun 15. PMID: 20550733. [DOI] [PubMed] [Google Scholar]
  25. Howells A., Ivtzan I., Eiroa-Orosa F.J. Putting the 'app' in happiness: a randomised controlled trial of a smartphone-based mindfulness intervention to enhance well-being. J. Happiness Stud. 2016;17:163–185. doi: 10.1007/s10902-014-9589-1. [DOI] [Google Scholar]
  26. Huguet A., Rao S., McGrath P.J., Wozney L., Wheaton M., Conrod J., Rozario S. A systematic review of cognitive behavioral therapy and behavioral activation apps for depression. PLoS One. 2016;11(5) doi: 10.1371/journal.pone.0154248. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Johnson S., Anderson P.L. Don’t ask, don’t tell: a systematic review of participants participating in treatment outcome research for social anxiety disorder. Anxiety Stress Coping Int. J. 2016;29(6):589–605. doi: 10.1080/10615806.2016.1138289. [DOI] [PubMed] [Google Scholar]
  28. Kern A., Hong V., Song J., Lipson S.K., Eisenberg D. Mental health apps in a college setting: openness, usage, and attitudes. Mhealth. 2018;4:20. doi: 10.21037/mhealth.2018.06.01. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Kubo A., Altschuler A., Kurtovich E., Hendlish S., Laurent C.A., Kolevska T., Li Y., Avins A. A pilot mobile based mindfulness intervention for cancer patients and their informal caregivers. Mindfulness. 2018;9:1885–1894. doi: 10.1007/s12671-018-0931-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Larsen M.E., Huckvale K., Nicholas J., Torous J., Birrell L., Li E., Reda B. Using science to sell apps: evaluation of mental health app store quality claims. npj Digit. Med. 2019;2(1):18. doi: 10.1038/s41746-019-0088-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Lattie E.G., Lipson S.K., Eisenberg D. Technology and college student mental health: challenges and opportunities. Front. Psychiatry. 2019;10:246. doi: 10.3389/fpsyt.2019.00246. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Lattie E., Cohen K.A., Winquist N., Mohr D.C. Examining an app-based mental health self-care program, IntelliCare for college students: single-arm pilot study. JMIR Ment. Health. 2020;7(10) doi: 10.2196/21075. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. LeViness P., Bershad C., Gorman K., Braun L., Murray T. The Association for University and College Counseling Center Directors: Annual Survey 2018. 2018. https://www.aucccd.org/assets/documents/Survey/2018%20AUCCCD%20Survey-Public-June%2012-FINAL.pdf
  34. Luxton D.D., McCann R.A., Bush N.E., Mishkind M.C., Reger G.M. mHealth for mental health: integrating smartphone technology in behavioral healthcare. Prof. Psychol. Res. Pract. 2011;42:505–512. doi: 10.1037/a0024485. [DOI] [Google Scholar]
  35. Medina L.D., Torres S., Gioia A., Lopez A.O., Wang J., Cirino P.T. Reporting of demographic variables in neuropsychological research: an update of O’Bryant et al.’s trends in the current literature. J. Int. Neuropsychol. Soc. 2021;27(5):497–507. doi: 10.1017/S1355617720001083. [DOI] [PubMed] [Google Scholar]
  36. Melcher J., Camacho E., Lagan S., Torous J. College student engagement with mental health apps: analysis of barriers to sustained use. J. Am. Coll. Heal. 2022;70(6):1819–1825. doi: 10.1080/07448481.2020.1825225. [DOI] [PubMed] [Google Scholar]
  37. Miani C., Wandschneider L., Niemann J., Batram-Zantvoort S., Razum O. Measurement of gender as a social determinant of health in epidemiology—a scoping review. PLoS One. 2021;16(11) doi: 10.1371/journal.pone.0259223. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Mohr D.C., Tomasino K.N., Lattie E.G., Palac H.L., Kwasny M.J., Weingardt K., Karr C.J., Kaiser S.M., Rossom R.C., Bardsley L.R., Caccamo L., Stiles-Shields C., Schueller S.M. IntelliCare: an eclectic, skills-based app suite for the treatment of depression and anxiety. J. Med. Internet Res. 2017;19(1) doi: 10.2196/jmir.6645. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Mohr D.C., Schueller S.M., Tomasino K.N., Kaiser S.M., Alam N., Karr C., Vergara J.L., Gray E.L., Kwasny M.J., Lattie E.G. Comparison of the effects of coaching and receipt of app recommendations on depression, anxiety, and engagement in the Intellicare platform: factorial randomized controlled trial. J. Med. Internet Res. 2019;21(8) doi: 10.2196/13609. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Owen J.E., Jaworski B.K., Kuhn E., Makin-Byrd K.N., Ramsey K.M., Hoffman J.E. mHealth in the wild: using novel data to examine the reach, use, and impact of PTSD Coach. JMIR Ment. Health. 2015;2(1) doi: 10.2196/mental.3935. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Palmer K.M., Henderson S.G. College students’attitudes abou tmental health-related content in mobile apps. J. Technol. Behav. Sci. 2019;4(4):381–389. doi: 10.1007/s41347-019-00102-0.23. [DOI] [Google Scholar]
  42. Paradies Y., Ben J., Denson N., Elias A., Priest N., Pieterse A., Gupta A., Kelaher M., Gee G. Racism as a determinant of health: a systematic review and meta-analysis. PLoS One. 2015;10(9) doi: 10.1371/journal.pone.0138511. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Quirk H.D., Nagar R., Anderson P.L. A qualitative exploration of college students’ experiences using mobile apps to improve self-care during the COVID-19 pandemic. J. Am. Coll. Heal. 2023 doi: 10.1080/07448481.2023.2198033. [DOI] [PubMed] [Google Scholar]
  44. Singh S., Sagar R. Time to have effective regulation of the mental health apps market: maximize gains and minimize harms. Indian J. Psychol. Med. 2022;44(4):399–404. doi: 10.1177/02537176221082902. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Su L., Anderson P.L. User behavior of a publicly available, free-to-use, self-guided mHealth app for depression: observational study in a global sample. JMIR Formative Res. 2022;6(10) doi: 10.2196/35538. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Torous J., Roberts L.W., Valuckas K.P. The ethical use of mobile health technology in clinical psychiatry. J. Psychiatr. Pract. 2017;25(5):366–372. doi: 10.1097/NMD.0000000000000596. [DOI] [PubMed] [Google Scholar]
  47. Torous J., Nicholas J., Larsen M.E., Firth J., Christensen H. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evid. Based Ment. Health. 2018;21(3):116–119. doi: 10.1136/eb-2018-102891. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Internet Interventions are provided here courtesy of Elsevier

RESOURCES