Abstract
Background
The working population encounters unique work-related stressors. Despite these challenges, accessibility to mental healthcare remains limited. Digital technology-enabled mental wellness tools can offer much-needed access to mental healthcare. However, existing literature has given limited attention to their relevance and user engagement, particularly for the working population.
Aim
This study aims to assess user perceptions and feature utilisation of mindline at work, a nationally developed AI-enabled digital platform designed to improve mental wellness in the working population.
Methods
This study adopted a mixed-methods design comprising a survey (n=399) and semistructured interviews (n=40) with office-based working adults. Participants were asked to use mindline at work for 4 weeks. We collected data about utilisation of the platform features, intention for sustained use and perceptions of specific features.
Results
Participants under 5 years of work experience reported lower utilisation of multimedia resources but higher utilisation of emotion self-assessment tools and the AI chatbot compared with their counterparts (p<0.001). The platform received a moderate level of satisfaction (57%) and positive intention for sustained use (58%). Participants regarded mindline at work as an ‘essential’ safeguard against workplace stress, valuing its secure and non-judgmental space and user anonymity. However, they wanted greater institutional support for office workers’ mental wellness to enhance the uptake. The AI chatbot was perceived as useful for self-reflection and problem-solving, despite limited maturity.
Conclusion
Identifying the unique benefits of specific features for different segments of working adults can foster a personalised user experience and promote mental well-being. Increasing workplace awareness is essential for platform adoption.
Keywords: Artificial intelligence, Public health informatics, Online Systems, Public Health
WHAT IS ALREADY KNOWN ON THIS TOPIC
The working population faces unique work-related stress, yet their access to mental healthcare is limited.
Digital mental wellness platforms transform mental healthcare delivery, allowing for wider dissemination.
There is limited research on the relevance and user engagement of digital mental wellness platforms, particularly concerning the working population.
WHAT THIS STUDY ADDS
Digital bibliotherapy, carefully curated for adults in the workforce, can potentially mitigate stress by assisting them in tacking challenges and improving problem-solving skills.
Artificial intelligence (AI)-powered emotionally intelligent chatbot can facilitate self-reflection and problem-solving in the context of work-related issues.
There is a significant link between work experience and utilisation of features, underscoring the importance of tailoring features for different segments of working population.
Institutional support and workplace awareness are vital for the adoption of digital mental wellness platform and sustained user engagement.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Implementing AI-driven digital tools and personalised informatics tailored to the needs of diverse workforce may foster a proactive approach to stress reduction and enhance the overall mental well-being of working adults.
Introduction
There is a growing recognition of the public health challenges related to mental disorders. Globally, anxiety and depression are the leading causes of disability, affecting more than 450 million people.1 Of particular concern is the mental wellness of the working population. It has been recognised that the working population faces unique work-related stressors and challenges, which can lead to undiagnosed depression and work absenteeism.2 3 Mental health disorders result in significant economic costs due to lost productivity, estimated at around US$1 trillion annually. This figure could rise to $6 trillion by 2030.4
In addition to these existing challenges, the COVID-19 pandemic has exacerbated mental health issues among workers, leading to substantial disruptions in their work routines.5 This includes the sudden transition from traditional office-based work to remote work arrangements and the adoption of virtual teamwork.6,8 Additionally, workers have had to navigate the difficulties of social isolation, loneliness and looming fear of unemployment amidst the economic downturn.9 Furthermore, the added burden on working parents is even greater as they have to balance their jobs with homeschooling and caring for their children.10
Despite the intensified challenges experienced by the working population, accessibility to mental healthcare remains limited.11 12 According to a recent study, among working adults with symptoms of anxiety and depression, only 24% received necessary services and support.13 The ability to access adequate mental healthcare is often hindered by factors such as stigma, financial constraints and a shortage of qualified mental healthcare professionals.14 15 Therefore, there is a critical need for addressing these barriers and improving accessible mental wellness support for this population.
Digital technology-enabled mental wellness tools can provide much-needed access to mental healthcare.15 16 Mental wellness programmes delivered online (eHealth) and mobile applications (mHealth) transform mental healthcare delivery, allowing for wider dissemination.17 These digital tools hold great potential for overcoming some of the barriers to supporting access and reducing stigma, given their flexible, anonymous and cost-effective nature.18 Existing literature has identified the positive effects of digital mental wellness platforms on enhancing mental wellness and work-related outcomes.19 20
Although there is a plethora of research on the effectiveness of digital platforms in improving mental wellness outcomes, relatively limited attention has been devoted to their relevance and user engagement, particularly concerning the working population.20 A recent review suggests that many online resources do not necessarily lead to sustained use, especially those with minimal guidance and full automation.21 Furthermore, working adults may have distinct patterns of features usage that assist them in coping with work-related stresses22; however, utilisation of specific features in times of work-related stress has not been thoroughly explored. In essence, there exists a significant research gap in the literature concerning user experience, highlighting the need for substantial evidence in this area. Therefore, further investigations are necessary to evaluate the perceived value and quality of digital mental wellness platforms that are tailored specifically for the working population.
Therefore, this study aimed to evaluate user perceptions and utilisation of specific features of the mindline at work platform, focusing on its effectiveness in addressing the mental wellness needs of the working population
Methods
Study setting: a workplace mental wellness platform
In June 2021, in view of the pandemic-induced new normal of working arrangements, the Ministry of Health Office for Healthcare Transformation in Singapore introduced an artificial intelligence (AI) web-based mental wellness platform known as mindline at work (https://www.mindline.sg/workspace) to promote the adoption of good workplace habits. To achieve this, the e-platform strategically provides multiple levels of increasingly immersive interaction, starting with emotion self-assessment tools (eg, mood check-in and mental wellness questionnaire). By utilising a predefined algorithm, the assessment tools categorise users into different mental wellness levels according to the severity of their symptoms. Subsequently, it enables the e-platform to customise multimedia resources (eg, articles, exercise videos and comics) recommendations specific to the user’s needs. Finally, to close the loop, individuals seeking ‘human interaction’ can engage in free-form conversation and digital therapeutics exercises with Wysa in a safe and anonymous manner.23 24 Wysa is an emotionally intelligent AI-enabled conversational agent that leverages evidence-based cognitive behavioural therapy (CBT)-inspired techniques to provide a therapeutic virtual space for user-led conversation.25 Moreover, should the need arise, distressed individuals have the option to reach out to external helplines for immediate professional assistance. Figure 1 shows a snapshot of mindline at work.
Study design
This study adopted a sequential explanatory mixed-method design in which a cross-sectional survey (quantitative) and a semi-structured interview (qualitative) were conducted and analysed. The study findings and interpretations were triangulated narratively in the discussion.
Data collection
Cross-sectional survey
Participants were recruited through an email newsletter disseminated via the human resource departments of public agencies and private enterprises. Participants were eligible if they were1 21 years old and above,2 office workers of public agencies or private enterprises and3 employed full time for a minimum duration of 6 months within the workplace. Eligibility was assessed on completion of a Qualtrics online form. Consented participants were asked to use mindline at work and its various features at least three times a week for 4 weeks. After the 4-week period, an online survey link was sent to the participants between December 2022 and February 2023. A survey aimed at evaluating user perceptions and utilisation of features was developed based on pertinent insight from relevant literature.26 27 Survey items included a usage rating of the mindline at work features in times of work-related stress on a 7-point ordinal scale (1=most used, 7=least used) and a three-item question on general satisfaction, intention to continue using the platform and their willingness to recommend it to others on a 5-point Likert scale (1=strongly disagree, 5=strongly agree). The questionnaire was pretested with five working adults. Feedback was collected to assess the content validity of the question items, and modifications were subsequently made.
Semistructured interview
The survey participants had the option to provide their contact details at the end of the survey if they wished to take part in the interviews. Those who indicated interest were purposively sampled based on their age and years of working experience. An interview guide was developed based on the relevant literature28 29 and included questions about participants’ perceived usability and acceptability of mindline at work, and perceptions of various features and their impacts. The guide also included questions about participants’ sociodemographic characteristics. All interviews were conducted virtually over Zoom by two trained qualitative researchers (HG and XCL) between March and May 2023 and recorded with permission to facilitate transcription. Field notes and memos were taken to capture insights. Interviews lasted between 45 min and 75 min. Fieldnotes and memos were used to support reflexivity by enabling interviewers to reflect on their own roles, biases and experiences in the research process, thereby enriching the analysis with a deeper understanding of the influence of their personal perspectives. For rigour and transparency, we anchored our methodology according to the Consolidated Criteria for Reporting Qualitative Research checklist.30
Data analysis
Cross-sectional survey
Data on usage rating of specific features, satisfaction, intention to continue using the platform and willingness to recommend the platform to others were summarised using appropriate descriptive statistics—counts and percentages for categorical variables, means and SD for continuous variables. Fisher’s exact tests were used to assess the associations between participant characteristics and utilisation of features. If Fisher’s exact test results were statistically significant, Bonferroni’s correction was performed to identify which pair of utilisation of features differed in the characteristic. Spearman correlation test was performed to detect multicollinearity between variables. All statistical analyses were conducted using R V.4.2.4 in RStudio (V.2023.07.0+554), and a two-tailed test was used, with a significance level of p<0.05.
Semistructured interview
Data from semistructured interviews were transcribed verbatim. Six steps of thematic analysis suggested by Braun and Clark were undertaken to explore the perceived usability and acceptability of mindline at work as well as its associated features.31 Two researchers with expertise in qualitative research reviewed the data and independently generated initial codes before combining codes into categories and potential themes. Initially, we employed inductive coding to identify preliminary codes directly from the data, free from any preconceived categories. For example, initial codes such as ‘cushioning from stress’ and ‘providing calming effects’ were combined to form the subtheme ‘serving as an essential safeguard against workplace stress by redirecting the user’s focus’. This subtheme was then merged with another subtheme, ‘limited enduring effects on mental well-being without addressing the source problems’, to create the overarching theme concerning the ‘usability and potential impact’ of midline at work. This iterative process ensured a comprehensive and balanced analysis, capturing both new insights and established concepts. The themes were summarised and constantly reorganised and refined to ensure the best representation of the data. Discrepancies were resolved through consensus meetings involving all research team members. During these meetings, researchers discussed divergent codes and interpretations, systematically reviewing the data to understand the reasons behind the discrepancies. Through collaborative examination, researchers reached agreement by cross-referencing the data with the emerging themes. This process provided additional transparency and robustness to the analysis, ensuring that the final coding scheme accurately reflected the data while maintaining coherence with the research objectives. The iterative process of independent coding and consensus meetings continued until no new emergent themes were identified. NVivo V.12 was used to facilitate the data analysis.
Results
Participant characteristics
For the cross-sectional survey, 433 individuals initially consented to participate, but 34 withdrew during the 4-week study period. The final sample included 399 unique survey participants. Reasons for drop-out included lack of time, change of mind and being uncontactable during the study period. For semistructured interviews, out of 132 working adults who indicated their interest, 40 participated in semistructured interviews. Inductive thematic data saturation was achieved. The majority of survey participants were Chinese (92%), between the ages of 21 and 40 (76%), and women (70%). Work experience spanned from under 5 years to over 15 years. The majority of interview participants were Chinese (92%) and between the ages of 21 and 40 (77%). Table 1 shows the characteristics of the participants.
Table 1. Participant characteristics.
Survey (n=399) | Qualitative interview (n=40) | |
Characteristics | N (%) | N (%) |
Age (years) | ||
21–30 | 145 (36.3) | 10 (25.0) |
31–40 | 159 (39.9) | 17 (42.5) |
41–50 | 57 (14.3) | 6 (15.0) |
51–60 | 32 (8.0) | 5 (12.5) |
>60 | 6 (1.5) | 2 (5.0) |
Ethnicity | ||
Chinese | 368 (92.2) | 37 (92.5) |
Malay | 17 (4.3) | 1 (2.5) |
Indian | 8 (2.0) | 1 (2.5) |
Others | 6 (1.5) | 1 (2.5) |
Gender | ||
Female | 280 (70.2) | 29 (72.5) |
Male | 119 (29.8) | 11 (27.5) |
Years of work experience | ||
≤5 years | 127 (31.8) | 7 (17.5) |
6–10 years | 122 (30.6) | 11 (27.5) |
11–15 years | 59 (14.8) | 7 (17.5) |
>15 years | 91 (22.8) | 15 (37.5) |
Quantitative findings
Usage frequency of mindline at work features
The top feature employed by participants to maintain overall well-being during episodes of work-related stress was ‘reading an article’, which was reported by 28.4% of participants. Following closely behind was ‘watching a video’ reported by 18.1% of participants and ‘chatting with Wysa chatbot’ by 17.6% of participants. The least used feature was ‘contact an external helpline’, with only 2.3% of participants reporting its usage (figure 2).
Utilisation of features by participant characteristics
The features within mindline at work have been grouped into three distinct categories: multimedia resources, emotion self-assessments and AI chatbot (Wysa). Results indicated a significant association between years of work experience and the utilisation of certain feature groups. Bonferroni paired tests showed that multimedia resources have different distribution in years of working experience compared with the other two groups (p value=0.009 for emotion self-assessments; p value <0.001 for AI chatbot (Wysa)). However, no statistically significant difference was observed between emotion self-assessments and AI chatbot (Wysa) (p value=0.24). Participants with ≤5 years of work experience reported a lower likelihood of utilising multimedia resources but showed a higher likelihood of utilising emotion self-assessments and the AI chatbot (Wysa) compared with their counterparts with longer work experience (p<0.001) (online supplemental file 1). There was no statistically significant association between age (p value=0.137), ethnicity (p value=0.277), gender (p value=0.959) and the pattern of utilising particular feature groups (table 2). In addition, Spearman correlation analysis was conducted to evaluate multicollinearity between variables. There was a positive correlation between age and years of working experience (r=0.21, p value=<0.001).
Table 2. Utilisation of mindline at work features by participant characteristics.
Features | ||||
Variables | Multimedia resources* | Emotion self-assessments† | Wysa chatbot and external helpline | P value‡ |
Age (years) | .137 | |||
21–30 | 100 (69.0%) | 22 (15.2%) | 23 (15.8%) | |
31–40 | 93 (58.5%) | 23 (14.5%) | 43 (27.0%) | |
41–50 | 43 (75.4%) | 7 (12.3%) | 7 (12.3%) | |
51–60 | 22 (68.7%) | 4 (12.5%) | 6 (18.8%) | |
>60 | 4 (66.7%) | 2 (33.3%) | 0 (0%) | |
Ethnicity | .277 | |||
Chinese | 239 (69.4%) | 54 (14.7%) | 75 (15.9%) | |
Malay | 9 (53.0%) | 4 (23.5%) | 4 (23.5%) | |
Indian | 8 (100%) | 0 (0%) | 0 (0%) | |
Others | 6 (100%) | 0 (0%) | 0 (0%) | |
Gender | .959 | |||
Female | 185 (66.1%) | 40 (14.3%) | 55 (19.6%) | |
Male | 77 (64.7%) | 18 (15.1%) | 24 (20.2%) | |
Years of working experience | <0.001* | |||
≤5 years | 63 (49.6%) | 33 (26.0%) | 31 (24.4%) | |
6–10 years | 89 (73.0%) | 15 (12.3%) | 18 (14.8%) | |
11–15 years | 49 (83.1%) | 9 (15.3%) | 1 (1.7%) | |
>15 years | 70 (76.9%) | 13 (14.3%) | 8 (8.8%) |
Multimedia resources include articles, mindful exercise videos and comics.
Emotion self-assessments include mood check-in and mental wellness questionnaire.
P values were obtained from Fisher's exact tests.
User satisfaction and sustained engagement
A moderately high level of user satisfaction was observed, with 57.8% of participants responding positively (‘strongly agree’ and ‘somewhat agree’) about mindline at work and its features. More than half (54.3%) reported their intention to continue using the platform even after the study period ended, while 22.8% stated potential unwillingness to continue using the platform. Similarly, a substantial proportion of users (57.4%) indicated their willingness to recommend mindline at work to others (‘strongly agree’ and ‘somewhat agree’).
Qualitative findings
User perceptions of the mindline at work platform
Participants’ perceptions of the platform can be broadly categorised into three key themes:(1) usability and potential impact, (2) uptake and acceptability and (3) safety and trust (table 3). Under the theme of usability and potential impact, participants regarded mindline at work as an essential ‘safeguard’ against workplace stress. By offering curated CBT-inspired exercises, the platform assisted them in effectively managing their emotions by shifting their focus from stress and reframing their thoughts. However, participants also noted that mitigation of work-related stress achieved through mindline at work was temporary and lacked a lasting effect. Some participants described that they were trapped in a ‘toxic’ work environment, which continued to trigger the same negative emotions despite using the platform. They emphasised the importance of exploring alternative strategies, such as career coaching, which could bring about tangible changes in their work lives and lead to a more fulfilling and sustainable improvement in their mental well-being.
Table 3. User perceptions of mindline at work platform.
Theme | Subtheme | Illustrative quotes |
Usability and potential impact | Serving as an essential safeguard against workplace stress by redirecting the user’s focus | ‘Being in a highly stressful (work) environment, there is no way that I could get out of it as I'm (experiencing) constant stress. The only way I could have for mindline (at work) is to cushion the level of my stress. My mind is focused on the website when I go through the (therapy) sessions. Whatever in mind is taken off from that highly stressed environment.’Age group 51–60, F‘I think what helped from it was some therapies on reframing things to look at the work situations from a different angle. It calms (me) down and makes my mind clearer.’ Age group 21–30, M |
Limited enduring effects on mental well-being without addressing the source problems | ‘It does not really solve the main issue. Though resources provide temporary relief, I think it’s only good for short-term distraction. It doesn't give me anything tangible. It leaves me desiring for (solutions) that are more contextualized to my situation.’ Age group 31–41, M‘If I'm very stressed at work, instead of regulating my emotions, maybe this job is not for me. Long-term guidance like a career coach might help even more because sometimes, it might be best to just leave that toxic environment. No matter how much self-care you do, it will not change our environment. Self-care and regulating emotions will definitely help, but it is a short-term solution.’ Age group 31–40, F | |
Uptake and acceptability | Perceived need for more organisational support for mental well-being to promote uptake and a culture of self-care in the workplace | ‘Educating the top levels and moving as a whole organization to normalize self-care at work, whereby seeking (mental) help is valued and encouraged. If you are using mindline (at work) for a few minutes during work time, that doesn't mean you're slacking off, nor should you feel guilty for that. You're just trying to improve your mood so that you can work with better productivity.’ Age group 31–40, F‘But if I know my boss is using it, then I will be interested in using it too. Also, if my boss uses it during work time, then I will also feel less worried about using it at work.’ Age group 21–30, F |
Concerns about using the platform during working hours due to prevailing stigma on mental wellness | ‘I mean also gives a bad impression about me if my boss sees (me assessing) this kind of thing. I think it also leaves a notion that this staff is not coping well emotionally, so perhaps this person is not up to standard and allocated with less important tasks.’ Age group 21–30, M‘I guess people are still uncertain about showing too much (mental vulnerability). In a way, I guess people still view it differently, and it might affect your progression at work. So, I always use it at home.’ Age group 41–50, M | |
Safety and trust | Platform’s anonymity lowers the entry barrier and safeguards privacy, facilitating open communication | ‘Very good (of not having to create a user account). The whole idea of creating an account is like a barrier to a lot of things. People are apprehensive about joining because they worry that their email and particulars will be tracked. So, they can share anything under the roof without an account, and there’s no way to track back what the person said.’ Age group 21–30, M‘This is a chatbot. It doesn't ostracize you simply because it is anonymized. It is there to provide a listening (ear) without being judgmental. Good for participants to share safely and freely.’ Age group 41–50, M |
Endorsement by national-level agencies enhances trust | ‘I find it very secure. If I'm not mistaken, this is supported by the government. So, anything the government supports, I feel safe because I know that the government is working hard to secure user privacy.’ Age group>60, F |
A recurring narrative under the theme of uptake and acceptability was a sense of insufficient organisational support to encourage the adoption of mindline at work in the workplace. Multiple participants felt that openly showing ‘mental vulnerability’ at the workplace would not only place one under a negative spotlight but could even impact performance appraisal and career progression. Consequently, some participants were extremely cautious about using mindline at work in the workplace. A top-down strategy, where senior management actively endorses its use, was the most favoured approach to promote a culture of self-care and normalise help seeking. Participants stressed that when they observe their superiors using mental wellness resources, it could serve as a powerful endorsement for seeking help. This visible support from higher-ups could convey a clear message that prioritising mental well-being is not only encouraged but also valued within the organisation.
Finally, throughout the interviews, the importance of safety and trust in using mindline at work was highlighted. Most participants viewed the platform as a secure and non-judgemental space because it allowed users to remain anonymous. This anonymity encouraged users to express themselves openly, ultimately fostering a strong sense of safety. Participants appreciated that the platform did not require personal information, which significantly lowered the entry barrier for users. The participants’ trust in the mindline at work platform also stemmed from their belief that a government-level agency’s involvement indicates a high level of scrutiny and privacy protection.
Perceptions of specific features of mindline at work
Participants described the usefulness of specific features available in mindline at work. In accordance with the quantitative findings, we grouped them into multimedia resources, emotion self-assessments and the Wysa chatbot (table 4). By and large, multimedia resources had a potentially positive impact on mental wellness and were highly appreciated by users, who noted them to be ‘applicable’, ‘funny’ and ‘provided calming effects’. Not only were the topics relevant to commonly encountered work stressors but they also supported stress relief and relaxation. In particular, several participants highlighted the significance of ‘reading an article’ feature, expressing that it held the potential to offer direct solutions to the problems they currently face, thereby mitigating work stress. They emphasised that this feature allowed them to pinpoint articles specifically addressing their issues ‘simply by searching on the titles’. This targeted approach contrasted with other multimedia resources that offered more general stress-relaxation methods, without specifically addressing the issues in question. Meanwhile, regarding exercise videos, participants pointed out the impracticality of certain recommended exercises, especially those that necessitated additional space or equipment, as they might not be ‘appropriate’ for the workplace environment.
Table 4. Perceptions of specific features of mindline at work.
Features | Themes | Illustrative quotes |
Multimedia resources | Articles: a practical and valuable feature helping to navigate challenges in the workplace | ‘I like (the articles). They are specific and I always turn to them whenever I face any work-related problems. It is quick to locate the right article from the title. Sometimes after reading them, I have a better perspective on how to manage my problems actually. It makes me feel less nervous as I can manage my problems better!’ Age group 31–40, F‘In contrast to comic or exercise(videos)), I prefer this (reading article). If I found the right ones, it’s like equipping you with strategies to combat the issues head-on. Then problems solved or became better. The rest (of the features) are good but not so specific, more suited for present (moment) relaxation.’ Age group 41–50, F |
Exercise videos: stress relief aid but less suited for office settings | ‘The breathing exercise and the background music are very soothing. So, whenever I am feeling stressed or anxious, I watch the videos and follow the instructions on meditation or breathing exercises. It is very useful to help me to calm down and relax.’ Age group 31–40, F‘But some exercises like yoga might not be appropriate at the workplace. It may be a bit strenuous because I think you need to sit down and move your body and need to have a yoga mat. So, a bit tough to do in the office.’ Age group 41–50, M | |
Comics: Light-hearted in nature and uplifting spirits | ‘Comic was something that made me feel better. It’s short, sweet, and simple. You read something funny, (and) then you have a laugh. Laughter is the best medicine. I think it’s just something that lifts your mood up.’ Age group 31–40, F‘When I read comics, it really gives me the impression that, ok, don’t stress yourself too much, things will get better, and sometimes just take things lightly… Usually, I read it while I was taking breaks in between.’ Age group 31–40, M | |
Emotion self-assessments | Mood check-in: enhancing self-awareness and self-care, yet limited mood options impact user experience | ‘It invokes how I should treat myself and others when my mood is as such. I’ll start to think retrospectively, if I’m tired, how should I care for myself better? Also, I will be more mindful when I feel frustrated. I should not lash out at other people. I thought that was quite helpful.’ Age group 31–40, M‘When I look at all the six moods (options), I’m like, ok, I’m not this nor that. Then I will forget about that or skip (using this feature).’ Age group 51–60, F |
Mental wellness questionnaire: normalising help-seeking through burnout recognition | ‘If someone is not aware of some of the (depressive) signs that they should take note of, at least, this will be a good guide telling them that maybe you’re quite burnout, you probably might need help in that kind of situation.’ Age group 31–40, M | |
AI chatbot (Wysa) | Providing perspectives on users’ emotions for enhanced problem-solving and therapeutic alliance | ‘Wysa puts words into perspective, puts a framework to user experience, and helps users recognize that: Hey, you are not alone! It’s ok to feel like that, and it is ok to be like that. Helping users to feel that they are not alone may provide some relief and encourage them to confront their problems.’ Age group 41–50, F‘Chatting with the chatbot can help me identify (the) actual thoughts behind some of (my) feelings, and how I can break it down into smaller pieces and then address them better. It helps me identify things I can work on with (the help of) Wysa to help myself.’ Age group 31–41, F |
Safe and convenient emotional outlet without professional or colleague involvement | ‘Some people are slightly apprehensive about talking to a therapist. So, you can talk to Wysa, and maybe it could provide you (with) some simplistic solutions to solve your current conundrum at that point in time.’ Age group 31–40, M‘I don’t want to talk to my colleagues because I don’t want to add further to their (mental) burden, and I will feel bad for doing so. But when I talk to the chatbot, I feel like I don’t have to burden anyone. It’s quite convenient… I can use it anytime when I am feeling overwhelmed.’ Age group 41–50, F | |
Discontent over chatbot maturity and need for a greater human engagement | ‘Bearing in mind that the chatbot also has its own limitations. You know, it can never replace a human, like a counselor or a therapist. In that sense, I'm not sure how helpful it would be. Although this option is good, it is better to have signposting to human support especially for people who still need help after exhausting the chatbot option.’ Age group 21–30, M‘The lack of human presence also limits the understanding of the whole complex situation. Sometimes, when I talk to someone, I first want them to listen and understand my feelings instead of saying something very general, which sometimes Wysa does. Sometimes, it does not understand me. Maybe the database is still expanding?’ Age group 31–40, M |
Regarding the features related to emotion self-assessments, the mood check-in improved overall self-awareness and self-care practices. Participants reported being ‘more mindful’ when discovering one’s own mood state, leading to positive changes in their behaviour and interactions with others. However, some participants, although not many, expressed discontent with the limited options available for mood selection, which diminished their overall user experience. Regarding the mental wellness questionnaire, participants highlighted its significance in ‘normalising help-seeking behaviour’. They found that the questionnaire helped users recognise burnout as an early indicator and encouraged them to seek help when necessary.
Analysis revealed that most participants held a positive view of Wysa. They particularly appreciated its ability to offer additional perspectives and structures that deepened understanding of one’s underlying emotions. Participants stated that by engaging with Wysa, they gained valuable insights and alternative viewpoints that helped them to make sense of their feelings. This, in turn, strengthened the therapeutic bond as they were empowered to solve present issues. Furthermore, participants noted the convenience of accessing Wysa at any time, bypassing certain barriers such as geographic and time limitations. Not only did it eliminate the need to rely on colleagues or friends when seeking someone to talk to but it also reduced the feeling of being alone in one’s struggles. Taken together, Wysa’s expanded understanding and convenience appeared to promote self-reflection and problem-solving. However, several participants expressed concerns over the chatbot’s maturity to handle complex situations. As helpful as it was intended to be, the lack of human involvement created dissatisfaction as these participants believed that only a human professional could truly understand the sensitivity associated with mental wellness issues.
Discussion
This study offers a comprehensive understanding of user perceptions and utilisation of specific features of the nationally developed AI-enabled digital mental wellness platform mindline at work. Overall, participants have responded positively to mindline at work and its specific features. Particularly noteworthy are the customised resources on the platform, which effectively help users relax by promoting present-moment awareness and non-judgemental acceptance. This intentional focus allowed users to temporarily detach from work stressors, offering a sense of relaxation even in high-paced work environments. Moreover, through interaction with the platform, participants recognised the importance of managing distractions, which led them to prioritise self-care and establish boundaries to support their long-term mental well-being.32 These findings echo prior literature that demonstrates the effectiveness of digital platforms on mindfulness in promoting overall mental wellness.33 34
What is novel in our findings is the significant value of bibliotherapy, involving reading materials to support office-based working adults dealing with workplace issues. In the context of mindline at work, bibliotherapy is a thoughtful process that encompasses reading, reflection and engaging in discussions focused on workplace-related topics. This deliberate engagement brought about a cognitive shift among our participants, helping them effectively manage stress and cultivate workplace resilience.35 Furthermore, it is important to note that bibliotherapy in mindline at work is different from traditional literature reading in a structured setting.36 The selections in mindline at work are professionally curated, offering users a unique opportunity to pinpoint content directly relevant to their individual concerns. This feature is particularly beneficial as it self-directs users to familiarise themselves with pertinent knowledge and provides them with the tools necessary to confront their issues directly.37 By gaining insights and understanding through these articles, users can address challenges effectively, enhancing their problem-solving capabilities and, in turn, mitigating stress. Nevertheless, despite these advantages, a recent examination of 578 mental wellness platforms revealed a notable absence of targeted reading activities as a common feature.38 This underscores the untapped potential of bibliotherapy as a stress mitigation strategy particularly among working adults, making our study a valuable contribution to the field.
Concurrently, this study has addressed the lack of understanding in the literature regarding utilisation of specific features within digital mental wellness platforms among working adults. We observed that individuals with less than 5 years of work experience favoured more interactive features such as emotion self-assessments and the chatbot. It is plausible that these younger and less work-experienced users, often holding junior-level positions, are likely to experience heightened stress, including job insecurity, limited control and complex interpersonal relationships.39 As a result, they demonstrated a propensity for adopting proactive approaches, such as actively seeking help through frequent interactions with tools like chatbots to effectively manage their work-related issues. On the other hand, users with longer work experience reported a higher likelihood of using multimedia resources such as reading articles or watching videos. These individuals typically occupy more senior roles and may have accumulated enriched life experiences, leading to better coping mechanisms when faced with work-related stress.40 Hence, they tended to use static resources for general wellness improvement rather than seeking immediate relief during stressful situations. These observations emphasised the importance of thoughtful planning and feature development for workplace mental wellness platforms, taking into account the varying needs of users at different stages of their careers.
What distinguishes mindline at work from other general digital mental wellness platforms is the incorporation of an emotionally intelligent and empathy-driven AI chatbot (Wysa) with transdiagnostic capabilities, a unique feature that is not commonly found in many mental wellness platforms.41 In today’s fast-paced and competitive work environment, working adults often encounter a broad spectrum of mental wellness challenges. From our qualitative component of the study, we found that Wysa provided participants with a structured and constructive perspective, encouraging them to explore their own thoughts and emotions through dynamic dialogues. This seemed to foster an active approach to addressing their mental health concerns. This finding aligns with the existing literature, emphasising the potential of AI-enabled chatbots as valuable tools for promoting mental wellness and complementing traditional mental wellness services such as onsite wellness programmes and therapy sessions.42 43 Despite the evident benefits, participants in our study recognised the limitations of AI chatbots and expressed the importance of incorporating human coaching. Hence, the potential integration of human support through trained professionals could be considered to further enhance the platform’s efficacy, cultivating a stronger sense of connection and optimising the benefits of AI chatbots in promoting mental wellness.
Finally, in spite of consistent emphasis in the literature on the importance of continuous utilisation of mental wellness platforms,44 many digital mental wellness tools often suffer from poor rates of sustained engagement.45 46 In a recent review of the top 50 platforms designed for depression and anxiety, nearly 60% have no active monthly users.47 This lack of consistent engagement poses a significant challenge to the platform’s usability, as many mental wellness issues require continuous support and monitoring. When users do not regularly interact with the platforms, they may miss out on vital support, potentially leading to a disconnection from their therapy goals.48 Although our survey results showed that over half of the participants would continue using mindline at work, it remains crucial to explore various strategies aimed at user engagement and retention. Personalised support and diverse content offerings were found to be instrumental in addressing the problems associated with poor sustained use.49 50 Our qualitative findings also highlighted the importance of workplace encouragement for uptake and sustained use. Therefore, institutional efforts aimed at supporting workers’ mental well-being can further contribute to improving sustained use, leading to more meaningful outcomes for the working population.
Strengths and limitations
This study expands on existing theories on digital mental wellness and workplace stress, such as self-determination theory, which posits that personal level factors such as autonomy, competence and relatedness influence one’s intention to seek mental healthcare.51 52 At the same time, our findings underscore the critical role of organisational support, on top of personal level factors, in promoting the adoption and sustained use of digital wellness platforms. This reinforces the ongoing discourse emphasising the importance of environmental and sociocultural factors in promoting workplace mental health interventions.53
The strength of this study lies in its unique focus on user perspectives of a digital mental health platform among working adults, a population segment largely overlooked in the plethora of available mental wellness platforms.54 The findings highlight the necessity for personalised user experiences tailored to different segments of the workforce, such as younger employees preferring interactive features and more experienced workers favouring static resources. This indicates that future digital mental wellness platforms should incorporate adaptive algorithms to customise content based on user profiles. The study also uncovers both the benefits and constraints of an emotionally intelligent AI chatbot from the perspectives of working adults.29 While the findings underscore the value of integrating enhanced AI capabilities with human support, we recommend a hybrid approach where users can escalate from chatbot interactions to live sessions with mental health professionals for more holistic care. Moreover, the critical role of organisational endorsement suggests that developers should collaborate with companies to promote a supportive workplace culture, ensuring higher adoption and sustained engagement. These insights provide concrete recommendations for creating more effective, user-centred digital mental wellness solutions that address the specific needs and stressors of working adults. In addition, the mixed-methods design employed in our research facilitated the identification of avenues for enhancing a nationally initiated digital mental health platform and its features through user feedback. The insights gleaned may offer transferable lessons applicable beyond our specific context and may be particularly relevant to the field of workplace mental wellness using informatics and digital technology.
The study has a few limitations to consider. First, participants used the platform for only 4 weeks. Due to its anonymous nature, it was not possible to monitor full compliance, potentially affecting response accuracy. Although the majority of participants expressed their intention to sustain usage, the absence of a longitudinal follow-up study limits our ability to substantiate these claims. Another limitation of our study is the voluntary nature of participation, which may result in a non-representative sample due to self-selection bias. This could impact the generalisability of our findings, as those who chose to participate may differ systematically from those who did not. Finally, as baseline data were not collected, establishing a relationship between changes in well-being and the utilisation of specific features was not possible. Future research could consider adopting an intervention study with a larger sample, which might provide a more robust understanding of the impact of mindline at work on mental wellness among the working population.
Conclusion
This study examined the user perceptions and utilisation of specific features within an AI-enabled digital mental health platform, aimed at improving mental wellness among working adults. Our findings indicate that digital bibliotherapy holds great potential as a stress mitigation strategy, particularly among the working adult population. Additionally, an AI-powered emotionally intelligent chatbot proves useful in facilitating self-reflection and problem-solving in the context of work-related issues. There is a significant connection between the work experience and the use of platform features, underscoring the importance of tailoring features to accommodate diverse segments of the workforce. Finally, our study suggests that institutional support and workplace awareness are vital for the adoption of a digital mental wellness platform and sustained user engagement.
supplementary material
Acknowledgements
We would like to thank the participants for their support in this study.
Footnotes
Funding: This research was funded by the Ministry of Health Office for Healthcare Transformation (MOHT). The funder declared no commercial interest in this study.
Provenance and peer review: Not commissioned; externally peer-reviewed.
Patient consent for publication: Not applicable.
Ethics approval: This study involves human participants and was approved by NUS Institutional Review Board (NUS-IRB-2022-402). Participants gave informed consent to participate in the study before taking part.
Contributor Information
Sungwon Yoon, Email: sungwon.yoon@duke-nus.edu.sg.
Hendra Goh, Email: hendra.goh@u.nus.edu.
Xinyi Casuarine Low, Email: low.casuarine@u.duke.nus.edu.
Janice Huiqin Weng, Email: janice.weng@moht.com.sg.
Creighton Heaukulani, Email: creighton.heaukulani@moht.com.sg.
Data availability statement
Data are available upon reasonable request.
References
- 1.WHO . Geneva: World Health Organization; 2017. Depression and other common mental disorders: global health estimates. [Google Scholar]
- 2.Henderson M, Harvey SB, Overland S, et al. Work and common psychiatric disorders. J R Soc Med. 2011;104:198–207. doi: 10.1258/jrsm.2011.100231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Bond G, Lerner D, Drake R, et al. Washington, DC: US Department of Health and Human Services; 2019. Work-focused interventions for depression: final report. [Google Scholar]
- 4.The Lancet Global Health Mental health matters. Lancet Glob Health. 2020;8 doi: 10.1016/S2214-109X(20)30432-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Santomauro DF, Mantilla Herrera AM, Shadid J, et al. Global prevalence and burden of depressive and anxiety disorders in 204 countries and territories in 2020 due to the COVID-19 pandemic. The Lancet. 2021;398:1700–12. doi: 10.1016/S0140-6736(21)02143-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Yoon S, Goh H, Nadarajan GD, et al. Perceptions of Mobile Health Apps and Features to Support Psychosocial Well-being Among Frontline Health Care Workers Involved in the COVID-19 Pandemic Response: Qualitative Study. J Med Internet Res. 2021;23:e26282. doi: 10.2196/26282. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Yoon S, Goh H, Chan A, et al. Spillover Effects of COVID-19 on Essential Chronic Care and Ways to Foster Health System Resilience to Support Vulnerable Non-COVID Patients: A Multistakeholder Study. J Am Med Dir Assoc. 2022;23:7–14. doi: 10.1016/j.jamda.2021.11.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Yoon S, Hoe PS, Chan A, et al. Impact of COVID-19 on perceived wellbeing, self-management and views of novel modalities of care among medically vulnerable patients in Singapore. Chronic Illn. 2023;19:314–26. doi: 10.1177/17423953211067458. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kniffin KM, Narayanan J, Anseel F, et al. COVID-19 and the workplace: Implications, issues, and insights for future research and action. Am Psychol. 2021;76:63–77. doi: 10.1037/amp0000716. [DOI] [PubMed] [Google Scholar]
- 10.Calear AL, McCallum S, Morse AR, et al. Psychosocial impacts of home-schooling on parents and caregivers during the COVID-19 pandemic. BMC Public Health. 2022;22:119. doi: 10.1186/s12889-022-12532-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Demyttenaere K, Bruffaerts R, Posada-Villa J, et al. Prevalence, severity, and unmet need for treatment of mental disorders in the World Health Organization World Mental Health Surveys. JAMA. 2004;291:2581–90. doi: 10.1001/jama.291.21.2581. [DOI] [PubMed] [Google Scholar]
- 12.WHO . Geneva: World Health Organization; 2019. Mental health ATLAS 2020. [Google Scholar]
- 13.Chodavadia P, Teo I, Poremski D, et al. Prevalence and economic burden of depression and anxiety symptoms among Singaporean adults: results from a 2022 web panel. BMC Psychiatry. 2023;23:104. doi: 10.1186/s12888-023-04581-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Jorm AF, Patten SB, Brugha TS, et al. Has increased provision of treatment reduced the prevalence of common mental disorders? Review of the evidence from four countries. World Psychiatry. 2017;16:90–9. doi: 10.1002/wps.20388. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Wainberg ML, Scorza P, Shultz JM, et al. Challenges and Opportunities in Global Mental Health: a Research-to-Practice Perspective. Curr Psychiatry Rep. 2017;19:28. doi: 10.1007/s11920-017-0780-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Firth J, Torous J, Yung AR. Ecological momentary assessment and beyond: The rising interest in e-mental health research. J Psychiatr Res. 2016;80:3–4. doi: 10.1016/j.jpsychires.2016.05.002. [DOI] [PubMed] [Google Scholar]
- 17.Lattie EG, Stiles-Shields C, Graham AK. An overview of and recommendations for more accessible digital mental health services. Nat Rev Psychol . 2022;1:87–100. doi: 10.1038/s44159-021-00003-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Solomon D, Proudfoot J, Clarke J, et al. e-CBT (myCompass), Antidepressant Medication, and Face-to-Face Psychological Treatment for Depression in Australia: A Cost-Effectiveness Comparison. J Med Internet Res. 2015;17:e255. doi: 10.2196/jmir.4207. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Fu Z, Burger H, Arjadi R, et al. Effectiveness of digital psychological interventions for mental health problems in low-income and middle-income countries: a systematic review and meta-analysis. Lancet Psychiatry. 2020;7:851–64. doi: 10.1016/S2215-0366(20)30256-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Moe-Byrne T, Shepherd J, Merecz-Kot D, et al. Effectiveness of tailored digital health interventions for mental health at the workplace: A systematic review of randomised controlled trials. PLOS Dig Health. 2022;1:e0000123. doi: 10.1371/journal.pdig.0000123. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Tong F, Lederman R, D’Alfonso S, et al. Digital Therapeutic Alliance With Fully Automated Mental Health Smartphone Apps: A Narrative Review. Front Psychiatry. 2022;13:819623. doi: 10.3389/fpsyt.2022.819623. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Peters D, Deady M, Glozier N, et al. Worker Preferences for a Mental Health App Within Male-Dominated Industries: Participatory Study. JMIR Ment Health. 2018;5:e30. doi: 10.2196/mental.8999. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.MOHT Empowering people to improve their health and emotional wellbeing. 2020. https://www.moht.com.sg/mindline-sg Available.
- 24.Weng JH HY H, Tan C, Chang JK, et al. Mental Wellness Self-care in Singapore with mindline.sg: A Framework for the Development of A Digital Mental Health Platform for Behaviour Change. J Med Internet Res. 2024;26 doi: 10.2196/44443. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Beatty C, Malik T, Meheli S, et al. Evaluating the Therapeutic Alliance With A Free-Text CBT Conversational Agent (Wysa): A Mixed-Methods Study. Front Digit Health. 2022;4:847991. doi: 10.3389/fdgth.2022.847991. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Smail-Crevier R, Powers G, Noel C, et al. Health-Related Internet Usage and Design Feature Preference for E-Mental Health Programs Among Men and Women. J Med Internet Res. 2019;21:e11224. doi: 10.2196/11224. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Batterham PJ, Calear AL. Preferences for Internet-Based Mental Health Interventions in an Adult Online Sample: Findings From an Online Community Survey. JMIR Ment Health. 2017;4:e26. doi: 10.2196/mental.7722. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Malik T, Ambrose AJ, Sinha C. Evaluating User Feedback for an Artificial Intelligence-Enabled, Cognitive Behavioral Therapy-Based Mental Health App (Wysa): Qualitative Thematic Analysis. JMIR Hum Factors. 2022;9:e35668. doi: 10.2196/35668. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Inkster B, Sarda S, Subramanian V. An Empathy-Driven, Conversational Artificial Intelligence Agent (Wysa) for Digital Mental Well-Being: Real-World Data Evaluation Mixed-Methods Study. JMIR Mhealth Uhealth. 2018;6:e12106. doi: 10.2196/12106. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19:349–57. doi: 10.1093/intqhc/mzm042. [DOI] [PubMed] [Google Scholar]
- 31.Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3:77–101. doi: 10.1191/1478088706qp063oa. [DOI] [Google Scholar]
- 32.Shimazu A, Schaufeli WB. Does distraction facilitate problem-focused coping with job stress? A 1 year longitudinal study. J Behav Med. 2007;30:423–34. doi: 10.1007/s10865-007-9109-4. [DOI] [PubMed] [Google Scholar]
- 33.Huberty J, Puzia ME, Green J, et al. A mindfulness meditation mobile app improves depression and anxiety in adults with sleep disturbance: Analysis from A randomized controlled trial. Gen Hosp Psychiatry. 2021;73:30–7. doi: 10.1016/j.genhosppsych.2021.09.004. [DOI] [PubMed] [Google Scholar]
- 34.Can YS, Iles-Smith H, Chalabianloo N, et al. How to Relax in Stressful Situations: A Smart Stress Reduction System. Healthcare (Basel) 2020;8:100. doi: 10.3390/healthcare8020100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Monroy-Fraustro D, Maldonado-Castellanos I, Aboites-Molina M, et al. Bibliotherapy as A Non-pharmaceutical Intervention to Enhance Mental Health in Response to the COVID-19 Pandemic: A Mixed-Methods Systematic Review and Bioethical Meta-Analysis. Front Public Health. 2021;9:629872. doi: 10.3389/fpubh.2021.629872. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Bilich LL, Deane FP, Phipps AB, et al. Effectiveness of bibliotherapy self-help for depression with varying levels of telephone helpline support. Clin Psychol Psychother. 2008;15:61–74. doi: 10.1002/cpp.562. [DOI] [PubMed] [Google Scholar]
- 37.Sharma V, Sood A, Prasad K, et al. Bibliotherapy to decrease stress and anxiety and increase resilience and mindfulness: a pilot trial. Expl NY. 2014;10:248–52. doi: 10.1016/j.explore.2014.04.002. [DOI] [PubMed] [Google Scholar]
- 38.Camacho E, Cohen A, Torous J. Assessment of Mental Health Services Available Through Smartphone Apps. JAMA Netw Open . 2022;5:e2248784. doi: 10.1001/jamanetworkopen.2022.48784. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Hsu HC. Age Differences in Work Stress, Exhaustion, Well-Being, and Related Factors From an Ecological Perspective. Int J Environ Res Public Health. 2018;16:50. doi: 10.3390/ijerph16010050. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Aronsson G, Theorell T, Grape T, et al. A systematic review including meta-analysis of work environment and burnout symptoms. BMC Public Health. 2017;17:264. doi: 10.1186/s12889-017-4153-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Chandrashekar P. Do mental health mobile apps work: evidence and recommendations for designing high-efficacy mental health mobile apps. Mhealth. 2018;4:6. doi: 10.21037/mhealth.2018.03.02. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Darcy A, Daniels J, Salinger D, et al. Evidence of Human-Level Bonds Established With a Digital Conversational Agent: Cross-sectional, Retrospective Observational Study. JMIR Form Res . 2021;5:e27868. doi: 10.2196/27868. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Fitzpatrick KK, Darcy A, Vierhile M. Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using A Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR Ment Health. 2017;4:e19. doi: 10.2196/mental.7785. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Henson P, Wisniewski H, Hollis C, et al. Digital mental health apps and the therapeutic alliance: initial review. BJPsych Open. 2019;5:e15. doi: 10.1192/bjo.2018.86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Baumel A, Muench F, Edan S, et al. Objective User Engagement With Mental Health Apps: Systematic Search and Panel-Based Usage Analysis. J Med Internet Res. 2019;21:e14567. doi: 10.2196/14567. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Kaveladze BT, Wasil AR, Bunyi JB, et al. User Experience, Engagement, and Popularity in Mental Health Apps: Secondary Analysis of App Analytics and Expert App Reviews. JMIR Hum Factors. 2022;9:e30766. doi: 10.2196/30766. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Wasil AR, Gillespie S, Shingleton R, et al. Examining the Reach of Smartphone Apps for Depression and Anxiety. Am J Psychiatry. 2020;177:464–5. doi: 10.1176/appi.ajp.2019.19090905. [DOI] [PubMed] [Google Scholar]
- 48.Zhang R, Nicholas J, Knapp AA, et al. Clinically Meaningful Use of Mental Health Apps and its Effects on Depression: Mixed Methods Study. J Med Internet Res. 2019;21:e15644. doi: 10.2196/15644. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Connolly SL, Hogan TP, Shimada SL, et al. Leveraging Implementation Science to Understand Factors Influencing Sustained Use of Mental Health Apps: a Narrative Review. J Technol Behav Sci. 2021;6:184–96. doi: 10.1007/s41347-020-00165-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Zhang Y, Pratap A, Folarin AA, et al. Long-term participant retention and engagement patterns in an app and wearable-based multinational remote digital depression study. NPJ Digit Med. 2023;6:25. doi: 10.1038/s41746-023-00749-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Peters D, Calvo RA, Ryan RM. Designing for Motivation, Engagement and Wellbeing in Digital Experience. Front Psychol. 2018;9:797. doi: 10.3389/fpsyg.2018.00797. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Patrick H, Williams GC. Self-determination theory: its application to health behavior and complementarity with motivational interviewing. Int J Behav Nutr Phys Act. 2012;9:18. doi: 10.1186/1479-5868-9-18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Scheutzow J, Attoe C, Harwood J. Acceptability of Web-Based Mental Health Interventions in the Workplace: Systematic Review. JMIR Ment Health. 2022;9:e34655. doi: 10.2196/34655. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Hwang WJ, Jo HH. Impact of Mental Health on Wellness in Adult Workers. Front Public Health. 2021;9:743344. doi: 10.3389/fpubh.2021.743344. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
Data are available upon reasonable request.