Preliminary effectiveness of an evidence-based mobile application to promote resilience among working adults in Singapore and Hong Kong: Intensive longitudinal study

Abstract

Evidence-based mobile health (mHealth) applications on smartphones are a cost-effective way for employees to take proactive steps to improve well-being and performance. However, little is known about what sustains engagement on these applications and whether they could dynamically improve occupational outcomes such as resilience and mood. Using real-world data, this intensive longitudinal study examines (a) which employees would continually engage with a cognitive behavioural therapy-informed mHealth application (‘Intellect’); and (b) if daily engagement of ‘Intellect’ would relate to better occupational outcomes on the following day. A total of 515 working adults in Singapore and Hong Kong (M_age = 32.4, SD_age = 8.17) completed daily in-app items on mood and resilience components (i.e. sleep hours, sleep quality, physical activity, and stress levels). Our results revealed that employees with lower baseline resilience (β = −0.048, odds ratio (OR) = 0.953, p < 0.01), specifically poorer sleep quality (β = −0.212, OR = 0.809, p = 0.001) and/or higher stress levels (β = −0.255, OR = 0.775, p = 0.05), were more likely to resume engagement on the application. Among the 150 active users (i.e. ≥3 consecutive days of engagement) (M_age = 32.2, SD_age = 8.17), daily engagement predicted higher resilience (β = 0.122; 95% confidence interval (CI) 0.039–0.206), specifically lower stress levels (β = 0.018; 95% CI 0.004–0.032), higher physical activity (β = 0.079; 95% CI 0.032–0.126), and mood levels (β = 0.020; 95% CI 0.012–0.029) on the following day even after controlling for same-day outcomes. Our preliminary findings suggest that engaging with a mHealth application was associated with higher dynamic resilience and emotional well-being in employees.

Introduction

mHealth interventions for the workplace

Most organizations today implement reactive measures which provide immediate relief of workplace stress but not proactive care. ¹ Not only do some of these employee assistance programmes (EAPs) carry little to no effect^2,5 but they also tend to be costly and suffer from low utilization rates per annum. In Asia, less than 1% of over 10,000 employees proactively sought assistance from EAPs even when >80% of them were aware of these programmes. ⁶ Recent research has supported mobile health (mHealth) interventions as a proactive and preventative approach for employee's mental health. ⁷ Not only are they easily accessible to any employee with a smartphone but they also address minor stressors before they evolve into more severe mental health problems, thus offering interventions earlier than traditional methods. ⁸ Such mHealth applications (apps) also lend themselves to higher levels of personalization than traditional workplace well-being support. Through a digital pathway, employees can begin and complete the self-guided intervention at their own pace and choose the content that best applies to their current stressors.^8,10 Self-guided apps, consisting of short and simple tasks, can then be incorporated into employees’ daily schedules, creating more opportunities for the desired behaviour to occur. ¹¹ This benefits both the individual and the organization as a whole. ¹

Evidence base for mHealth interventions

Recent systematic reviews have also supported the efficacy of these digital apps for workplace well-being, namely employees’ psychological resilience and affective states. In a meta-analysis of 66 randomised controlled trials (RCTs) of smartphone interventions, Linardon et al. ¹² found that mHealth apps significantly outperformed active and waitlist control conditions on depressive symptoms (g = 0.28, n = 54), anxiety symptoms (g = 0.30, n = 39), and positive affect (g = 0.44, n = 6). These findings complemented an earlier meta-analysis of web-based RCTs delivered in occupational settings, which also derived small effect sizes on alleviation of negative affect symptoms (depression: g = 0.28; anxiety: g = 0.29). ¹³ Similarly, RCTs of computer-based (n = 14) and smartphone apps (n = 6) enhanced participants’ trait resilience with moderate to large effect sizes (g = 0.54–1.09). ¹⁴ Given that psychological resilience and emotions intertwine to influence work-related outcomes such as job satisfaction, performance, job search behaviours, and daily well-being, ¹⁵ continuous use of smartphone interventions is highly encouraged at this time where employers and employees are already keen to take proactive steps to maintain health and wellness. ¹⁶

Empirical gaps for mHealth interventions in the workplace

Lack of evidence base on real-time effectiveness

Despite the growing efficacy and interest in using smartphone interventions to improve occupational well-being, ¹⁷ empirical evidence for mHealth interventions in the workplace remains limited.^5,18,19 Many of these commercially available apps have also not been evaluated with robust scientific methods,^5,17 such as using ecological momentary assessments (EMAs) in the workplace. ²⁰ To our knowledge, few studies have reported dynamic associations between daily app engagement and employee's well-being. For example, the meta-analyses by Carolan et al. ¹³ and Linardon et al. ¹² did not examine any digital interventions with EMAs. Contrary to the large effects on trait resilience, internet interventions with pre–post assessments on dynamic resilience revealed much smaller effect sizes (g = 0.29). ¹⁴ Moreover, these web-based interventions administered retrospective assessments on a psychological outcome that varies dynamically, which introduces recall biases and further compromises on scientific accuracy. ²¹ Understanding the dynamic impact of mHealth apps such as potential in-the-moment improvements on resilience and affective states is important, as depleted levels of daily resilience are associated with turnover intentions and resignations ²² while daily negative emotions hindered employees’ engagement in more health- and work-promoting behaviours. ²³ Thus, re-evaluating workplace well-being using EMAs within a digital intervention is highly recommended.

Low mHealth engagement among employees

Engagement rates with mHealth interventions among employees are usually very low,^24,26 which in turn limits their capacity to sustain behavioural changes.^27,28 As many as 80% of mHealth users logged into the application once and did not return.^29,30 In addition, the median retention rates of 93 commercially available mHealth apps at 15 and 30 days of installation were 3.9% and 3.3%, respectively. ³¹ Some researchers have begun identifying key elements that promote employees’ engagement of mHealth apps. For example, interactive (i.e. two-way), peer support and embedded professional support features were found to be important in sustaining app engagement,^32,33 while the lack of well-designed in-app reminders, goal-setting and self-monitoring features, relevant feedback, and a reward system discouraged engagement. ³³ Despite this evidence, less attention is dedicated to evaluating a user's mental health needs that may incline or decline his/her engagement with a mHealth intervention. ³⁴ Furthermore, some inconsistencies in the mHealth literature are noteworthy. For example, high severity of negative emotions (e.g. feeling depressed or fatigued)^35,36 increased participants’ engagement motivation in needs assessment studies^37,38 but not in evaluation studies.^39,40 To our knowledge, the literature also has not assessed the lack of psychological resilience as a form of engagement motivation. Currently, only few mHealth studies examined the underlying motivations for continuous app engagement.^17,41 Strengthening this base cannot only help to improve employees’ acceptance of these technologies but also drive further personalization of mHealth content to an individual's needs and filter out other apps that provide ineffective content. ¹⁹

Operationalisation of resilience in the workplace

In occupational health research, ²² psychological resilience is defined as the ongoing process of responding adaptively (i.e. bouncing back) to daily or weekly work challenges such as competitive pressures, employers’ expectations, miscommunication, and organizational restructuring.^42,43 Till date, there has been little consensus on the constitution of dynamic resilience. Contemporary definitions of ‘dynamic’ resilience recognise a pattern of protective and promotive factors that individuals can monitor and build proactively. ⁴⁴ Individual trajectories of psychological characteristics, namely emotional regulation, ²² cognitive resources, ⁴⁵ or social support, ⁴⁶ were more often seen as constituents of dynamic resilience. Fewer studies attempted to construe and evaluate dynamic resilience in terms of physiology,^46,47 such as sleep–wake patterns (i.e. sleep quantity and quality) and physical activity which formerly influenced individual's responses to daily stressors.^48,50 Another organizationally relevant response but not yet considered as a component of dynamic resilience is an employee's perceived stress, ⁵¹ which directly influences many factors relating to daily psychological resilience (i.e. perceptions of self-control, coping self-efficacy, cognitive flexibility, and emotional regulation).^51,52 Together, it may be helpful to test for dynamic resilience incorporating other physiological aspects and the perceptual appraisal of stress. As detailed in the Methods section, we followed the practice of recent research^22,53,54 and developed a summative and composite resilience score with these factors.

Study objectives

This study has two objectives. Firstly, we examined the baseline outcomes that may motivate employees to continuously engage with an evidence-based mHealth app. In this scenario, continuous engagement is defined as whether the employee had returned to use the app on the second day. Next, using an intensive longitudinal design, this study evaluated whether day-to-day engagement with the app would consistently relate to better psychological outcomes (i.e. resilience and emotions) on the following day. The use of intensive assessments or EMAs allows us to conduct multilevel time-lagged analyses to evaluate both conditional (i.e. Is the intervention effective on a daily basis?) and differential effectiveness (i.e. Who benefits more from the intervention?) of an intervention while controlling for the temporal associations between day-to-day outcomes. ⁵⁵ Together, the main hypotheses of the study are as follows:

• Hypothesis (H1a): Lower resilience at baseline is related to higher likelihood of continuous app engagement.

• Hypothesis (H1b): Poorer affective states at baseline are related to lower likelihood of continuous app engagement.

• Hypothesis (H2a): Daily engagement with an evidence-based mHealth app is significantly associated with gains in next-day resilience, after controlling for the effects of same-day resilience.

• Hypothesis (H2b): Daily engagement with an evidence-based mHealth app is significantly associated with gains in next-day mood, after controlling for the effects of same-day mood.

In addition, we explored the individual resilience components that are significantly associated with an increased likelihood of continuous engagement (H1a), and those that would improve as a result of previous-day app engagement (H2a). Using multilevel analyses, we also explored interindividual differences to gain more insight into possible heterogeneity between individuals within the dynamic improvements (H2a and H2b).

Methods

Ethics

This study received an advisory review from the Advarra Institutional Review Board and was deemed to be of no more than minimal risk (protocol number: Pro00059554). Co-funding for recruitment procedures between ‘Intellect’ and an external industry partner was also informed to the Institutional Review Board. Informed consent was obtained from the final set of participants. All participants consented that their participation was anonymous, and that their de-identified data was collected in an aggregated manner.

Design

This study used an intensive, longitudinal within-subjects design to evaluate daily changes in engagement and well-being outcomes of working adult participants. Such a design involves repeated examination of the same participant to detect any behavioural or psychological changes that may occur across time. ⁵⁶ These participants engaged with the mHealth application, ‘Intellect’, anytime from January 2022 to June 2022. For each participant, we set the data collection window ranging from a day to a month since his/her start of using ‘Intellect’. The primary dependent variables measured included resilience and its components, and mood scores.

Recruitment

Social media posts and advertisements, each containing a link to the participant information sheet and consent form, were disseminated online throughout January 2022 to June 2022. A total of 22,932 users accessed the electronic link, while 1471 users provided consent by clicking on the ‘Agree’ button. These participants were then directed to an online survey hosted on Qualtrics, where they completed baseline measures on stress levels, physical activity (i.e. number of steps/day), sleep hours, perceived quality of sleep, mood, and demographic information such as age and gender. Thereafter, they were all guided to download the mHealth application ‘Intellect’ on their personal smartphones from the Apple App Store or Google Play Store.

Study participants

Upon obtaining the baseline data, participants were further screened for eligibility criteria which included being a working adult of aged 18 and older, residing in Singapore and Hong Kong, and able to understand or speak English. After screening, 336 participants were further excluded from the study as they did not meet at least one of the inclusion criteria. A total of 620 more participants chose not to continue with the study. The final sample of 515 participants was predominantly female (n = 322; 62.5%), from Singapore (n = 457; 88.7%), and with a mean age of 32.41 years (SD = 8.17, range = 18–58). The participant flowchart is illustrated in Figure 1. Participants were invited to engage with the ‘Intellect’ features and complete the in-app daily assessments of sleep hours, perceived quality of sleep, stress levels, physical activity, and mood levels over the course of a month. In-app reminders to engage with the application were not sent in order to prevent influence on adherence. All participants were also instructed to refrain from using any wellbeing mHealth apps other than the given mHealth app from enrolment until the end of the study. This minimised potential confounding effects. Participants did not receive any monetary reimbursement.

Figure 1.

Participants flowchart.

Intervention

Intellect's self-care features

Intellect is a consumer-based mental health application that gives users access to a variety of self-guided evidence-based features, of which some have been validated in previous randomised-controlled trials.^57,58 On the ‘Home’ tab, each participant can access any of the three self-help features: ‘Learning Paths’, ‘Rescue Sessions’, or ‘Guided Journals’. ‘Learning Paths’ educate participants with evidence-based content to gain insights into their mental health (Figure 2). One example is the ‘Stress Coping Learning Path’, where the programme provides psychoeducation on cognitive distortions related to stress and anxiety (e.g. catastrophizing) and then guides the user to reframe his/her thoughts. Being equipped with such knowledge and skills through psychoeducation has been shown to help users regulate their stress and negative emotions more effectively, even on mobile platforms. ⁵⁹ ‘Rescue Sessions’ are a mix of self-guided interventions that target specific themes of adversity, such as ‘procrastination’, ‘sleep’, ‘anger and frustration’, ‘burnout’, and ‘relationships’. These interventions are guided by principles of mindfulness, self-compassion, and cognitive behavioural therapy, which also improved mental health symptoms in other mHealth applications. ⁶⁰ Participants are encouraged to choose the rescue session deemed most relevant to their current struggles. Intellect's ‘Guided Journals’ consist of six themes, ‘gratitude’, ‘reflection’, ‘problem-solving’, ‘goal-setting’, ‘sleep’, and ‘self-affirmation’. Each journal prompts participants to write an entry relating to the theme. For example, the ‘gratitude’ journal reminds each user to recall something that s/he can be thankful for but does not usually notice. The nurturing of greater self-awareness and mood-tracking can facilitate discovery of other perspectives and lead participants to reframe difficult scenarios and be gentler to themselves. ⁶¹ Participants who prefer a standardised schedule for self-care can also access the ‘Daily’ tab, where they follow a list of daily exercises to enhance physical and mental well-being. Exercises within this daily ‘Toolkit’ are dedicated to three parts of the day. The morning section includes deep breathing exercises guided through therapist-led audio messages. Such brief interventions were previously effective in improving mood ⁶² and anxiety symptoms virtually.^63,64 The afternoon section provides grounding techniques that increase mindfulness to the users’ surroundings. Lastly, the evening section focuses on improving sleep through meditation, gently reflecting on any positive events that may have happened during the day. Images of these self-care features can be found in Appendix A.

Figure 2.

Example of a ‘Self-Esteem’ learning path accessible on the ‘Home’ tab.

Intellect's coaching services

Embedded between the two tabs lies the ‘Coach’ tab, which provides access to Intellect's professional coaching services (Figure 3). Intellect offers personalised, virtual coaching services to support each participant's personal and professional development. Participants gain access to several hundred coaches on the platform, and coach selection is optimised by a set of algorithms that make recommendations based on the participant's personality, motivations, and goals. Each coaching session is conducted primarily via a video with a professionally certified coach and lasts approximately 30–45 minutes. Although the majority of the communications occur via a video, the ‘Coaching’ tab also allows for text messaging and phone communications. This study provides up to two free coaching consultations to each participant.

Figure 3.

Intellect's behavioural health coaching accessible on the ‘Coach’ tab.

Baseline and daily measures

Participants were advised to track their psychological resilience and mood scores on a daily basis. Mood levels were rated using a mood slider located at the top of the ‘Home’ page, while the four items on daily resilience were accessed after participants clicked on the bottommost feature ‘Highlights of the Day’ (Appendix A). Table 1 details the operational definitions and the methods of data collection for all dependent variables.

Table 1.

Operational definitions and measures used for psychological resilience and mood.

Dependent variables		Operational definition	Measures used
Psychological resilience	Daily sleep hours	Actual sleep hours last night (self-reported)	Three-point slider: 1 = ‘Unhealthy’ = ≤5 hours or >10 hours of sleep’ 2 = ‘Healthy’ = ‘>5–7 hours of sleep’ 3 = ‘Optimal’ = ‘>7–8 hours of sleep’
	Perceived quality of sleep	Quality of sleep last night (self-reported)	11-point slider: 0 = ‘Terrible’ 10 = ‘Excellent
	Daily physical activity	Step-count throughout the day (self-reported)	Five-point slider: 1 = ‘<30 minutes’ = (<5000 steps/day), Sedentary 2 = ‘30 minutes–1 hour’ = (5000–7499 steps/day), Low Active 3 = ‘>1–1.5 hour’ = (7500–9999 steps/day), Somewhat Active 4 = ‘>1.5–2 hours’ = (10,000–12,499 steps/day), Active 5 = ‘>2 hours’ = (≥12,500 steps/day), Highly Active
	Daily stress level	Stress levels throughout the day (self-reported)	Four-point slider: 1 = ‘Low’ 2 = ‘Medium’ 3 = ‘High’ 4 = ‘Very high’
Mood levels		Mood levels throughout the day (self-reported)	Five-point slider: 0 = ‘Terrible’ 4 = ‘Fantastic’

Psychological resilience

We calculated a composite score for dynamic workplace resilience that comprises the sum of four components—‘Daily Sleep Hours’, ‘Perceived Quality of Sleep’, ‘Daily Physical Activity’, and ‘Daily Stress Levels’. The physiological and mental (i.e. stress) components were equally weighted to compute the overall score.

Daily sleep hours

Participants were asked to report their actual sleep hours last night. As suggested by previous meta-analyses linking sleep duration to workplace outcomes such as poorer organizational performance, adverse health, and all-cause mortality,^65,66 the item is rated on a three-point scale (1 = ‘Unhealthy’, 2 = ‘Healthy’, and 3 = ‘Optimal’). Highest scores were allocated to the Optimal sleep duration of ‘>7–8 hours’, lower scores to Healthy ‘>5–7 hours or 8–10 hours’, and Unhealthy ‘≤5 hours or >10 hours’.

Perceived quality of sleep

Participants’ quality of sleep last night was assessed using the Single-Item Sleep Quality Scale (SQS ⁶⁷ ; that was featured in daily interventions). ⁶⁸ This item is scored on an 11-point scale (0 = ‘Terrible’ and 10 = ‘Excellent’) with higher scores indicating better perceived quality of sleep.

Daily physical activity

Participants’ daily physical activity was measured by their self-reported step count throughout the day. Following Tudor-Locke and Bassett's ⁶⁹ and Tudor-Locke et al.’s^70,71 proposed step indices for daily physical activity in healthy adults (including working adults), this item is rated on a five-point scale accordingly (1 = ‘<30 minutes (<5000 steps/day), Sedentary’, 2 = ‘30 minutes–1 hour (5000–7499 steps/day), Low Active’, 3 = ‘>1–1.5 hours (7500–9999 steps/day), Somewhat Active’, 4 = ‘>1.5–2 hours (10,000–12,499 steps/day), Active’, and 5 = ‘>2 hours (≥12,500 steps/day), Highly Active’).^69,71

Perceived stress

Participants rated their stress levels on a self-developed stress slider. The scale was reverse scored (1 = ‘Very High’ and 4 = ‘Low’). Previously, we examined convergent validity of this self-developed item by separately recruiting a large sample of working professionals (n = 997) from the online Rakuten platform in an unpublished study. The Rakuten platform functions similarly to the Amazon Mechanical Turk in the United States (MTurk). We found significant positive correlations (r = 0.390, p < 0.001) and acceptable factor loadings (λ = 0.68) between the stress slider and Perceived Stress Scale (PSS-4) items. ⁷²

Mood levels

Similar to the stress slider, participants rated their daily mood scores on a self-developed five-point mood slider (0 = ‘Terrible’ and 4 = ‘Fantastic’). Higher total scores indicate a more positive mood. Previously, significant negative correlations between the mood slider and Patient-Health Questionnaire (PHQ-4) items (r = −0.501, p < 0.001) and high factor loadings (λ = 0.72) indicate good support for convergent validity. ⁷³

Statistical analyses

Descriptive statistics for the sample were calculated using IBM SPSS Statistics 29.0. ⁷⁴ To examine the first hypothesis, we first created a categorical variable that indicates presence of engagement (binary: yes/no) across the maximum 30-day period for each participant. As we were mainly interested in whether participants returned to use ‘Intellect’ on the second day, we only analysed the presence of second day engagement for each participant. Baseline (Day 1) values of resilience and its components, and mood levels were lagged using the LAG command on Mplus 8.2. ⁷⁵ A binary logistic regression model was conducted on Mplus 8.2 to examine if lagged, baseline values of psychological outcomes predicted Day 2 engagement. For the second hypothesis, we examined whether day-to-day engagement with ‘Intellect’ consistently predicts next-day increases in resilience and mood levels. Consecutive days of engagement are defined as continuous engagement starting from the first day. For within-individual longitudinal analyses, we focused only on those who completed at least three consecutive days of surveys as recommended by Trougakos et al. ⁷⁶ and da Motta Veiga and Gabriel ⁷⁷ , leaving us with a sample of 150 active users.^76,77 Multilevel random effect models were specified using the ANALYSIS = TWOLEVEL RANDOM command in Mplus 8.2. ⁷⁵ A random intercept was also included in all models. In each model, it was tested whether psychological outcomes at each day (t) could be predicted by engagement with ‘Intellect’ on the previous day (t − 1), controlling for the autocorrelated lagged values of the respective outcomes. To separate within-subject effects from between-subjects effects, we used the recommended approach of person-mean centreing of the predictor variables, by calculating daily within-person deviations from the person's mean.^56,78 This cancels out all between-subject differences, which then allows all regression coefficients to be interpreted as within-subject effects. The coefficients were allowed to differ across individuals by modelling them as random slopes, to take into account the possible heterogeneity of the effects among participants. We examined the significance of the random effects variances using maximum likelihood-ratio.

Results

After assessing the demographic characteristics of all our participants, we tested hypotheses H1a and H1b by using binary logistic regression models to examine the baseline variables that were significantly related to continuous app engagement. Before testing the remaining hypotheses on the sample of active users (i.e. ≥3 consecutive days of app engagement), Student’s t-tests and χ² analyses were conducted to examine systematic differences between active participants and participants who dropped out of ‘Intellect’. Intraclass correlation coefficients of the dependent variables were also assessed to confirm a multilevel structure of the data. Subsequently, we tested hypotheses H2a and H2b using multilevel, time-lagged statistical models.

Participants

The descriptive statistics for all 515 participants are shown in Table 2. Participants were predominantly from Singapore (n = 457; 88.7%), female (n = 322; 62.5%) middle-aged adults (M_age = 32.4, SD_age = 8.17). On average, each participant engaged with ‘Intellect’ for a consecutive 6.83 days since his/her first day (SD = 7.04, range = 1–28).

Table 2.

Descriptive statistics for demographics.

Variables	Participants (n = 515)
Age, M (SD)	32.41 (8.17)
Continuous engagement (Days), M (SD)	6.83 (7.04)
Gender, n (%)
Female	322 (62.5%)
Male	173 (33.6%)
Non-binary	9 (1.70%)
Others	6 (1.20%)
Country of residence, n (%)
Singapore	457 (88.7%)
Hong Kong	58 (11.3%)

Baseline variables on continuous engagement

To test hypotheses H1a and H1b, a binary logistic regression was performed to evaluate the effects of baseline variables on the likelihood of continuous engagement (Model 1, Table 3). Findings from the model suggest that lower baseline resilience (β = −0.048, odds ratio (OR) = 0.953, p < 0.01) significantly predicted continuous engagement with ‘Intellect’ on Day 2. This indicates that a unit increase in baseline resilience is associated with a 4.7% decrease in the odds of engaging with ‘Intellect’ on a second day, which is consistent with H1a. None of the other variables were statistically significant (mood: β = 0.110, OR = 1.12, p = 0.526, age: β = −0.008, OR = 0.993, p = 0.668, gender: β = 0.267, OR = 1.31, p = 0.273). The non-significant relationship between low mood and continuous app engagement contradicts H1b. As an exploratory analysis, a second binary logistic regression model was specifically conducted to test the effects of the four resilience components at baseline on continuous engagement. Participants with poorer baseline sleep quality (β = −0.212, OR = 0.809, p = 0.001) and/or higher baseline stress levels (β = −0.255, OR = 0.775, p = 0.05) were more likely to continuously engage with ‘Intellect’ after the first day (Model 2, Table 3). A unit increase in sleep quality and stress levels (i.e. reduced stress) is associated with a 19.1% and 22.5% decrease in the odds of using ‘Intellect’ for a second day.

Table 3.

Estimations of binary logistic regression analyses of continuous engagement with ‘Intellect’ as dependent variable.

Baseline variables	B	SE	p-Value	Odds ratio (OR) (95% CI)
Model 1
Age	−0.008	0.017	0.526	1.12 (0.839–1.49)
Gender	0.267	0.243	0.273	1.31 (0.926–0.981)
Mood	0.110	0.173	0.526	0.993 (0.964–1.02)
Total resilience	−0.048**	0.017	0.006	0.953** (0.926–0.981)
Model 2
Sleep hours	−0.045	1.30	0.728	0.956 (0.771–1.18)
Sleep quality	−0.212**	0.071	0.003	0.809** (0.719–0.910)
Stress levels	−0.255*	0.135	0.038	0.775* (0.621–0.967)
Physical activity	−0.016	0.027	0.553	0.984 (0.942–1.03)

Note: Bold values indicate statistical significance.

**p<0.01

*p<0.05

Attrition analysis

A total of 365 participants did not engage with ‘Intellect’ for at least three consecutive days. No significant differences (through t-test and χ²) were observed between these participants and the sample of 150 active users (≥3 consecutive days of engagement) in terms of age and gender (age: t = 0.432, p = 0.666; gender: χ² (3, 365) = 5.84, p = 0.119). Similar to the original sample, the majority of active users were middle-aged adults (M_age = 32.2, SD_age = 8.17), female (n = 99, 66.0%) and reside in Singapore (n = 136, 90.7%).

Intraclass correlation coefficients

Before testing the lagged models, we examined the between-person and within-person variance components of all variables used in the analysis. The intraclass correlation coefficients (ICCs) for mood and resilience were ρ = .39 and ρ = .37, respectively. This means that 40% and 38% of participants’ mood and resilience levels could be explained by between-person variations across the days of using ‘Intellect’, whereas 60% and 62% of the variance, respectively, could be explained by daily variations within the participant. Similarly, the ICCs for the resilience components (i.e. sleep hours, sleep quality, stress levels, and physical activity) were ρ = .24, ρ = .37, ρ = .41, and ρ = .39, respectively (Table 5). The proportions of total variance attributed to between-person were 24%, 37%, 41%, and 39%, while 76%, 63%, 59%, and 61%, respectively, were related to within-person variance. Together, these findings confirmed the multilevel structure of our data as sufficient variance could be explained at both the between- and within-person levels.^45,79

Table 5.

Random-effect variances from the multilevel random effect models.

Random-effect variances (Level 2 between-person)
Intercept	0.308 (0.060)***	25.1 (5.11)***	1.23 (0.215)***	11.8 (2.14)***	0.424 (0.094)***	2.29 (0.442)***
Engagementt − 1	Omitted	Omitted	Omitted	Omitted	Omitted	omitted
Moodt − 1	Omitted
Resiliencet − 1		Omitted
Stress levelst − 1			0.035 (0.017)*
P.A.t − 1				Omitted
Sleep hourst − 1					omitted
Sleep qualityt − 1						omitted
Random-effect Variances (Level 1 Within-person)
Residual	0.556 (0.033)***	62.9 (3.45)***	1.62 (0.092)***	19.5 (0.092)***	1.42 (0.076)***	4.28 (0.236)***
ICC	0.40	0.38	0.41	0.39	0.24	0.37

Note: Bold values indicate statistical significance.

Estimates are unstandardised coefficients (B). Between parentheses: standard error of the estimate (SE). Omitted: random effects omitted from the model because of ns. P.A.: physical activity; ICC: intraclass correlation coefficient based on the random intercept model.

*p < 0.05.

**p < 0.01.

***p < 0.001.

Day-to-day engagement and next-day outcomes

Consistent with hypotheses H2a and H2b, the fixed effects of the multilevel models indicated that daily engagement with ‘Intellect’ predicted some increases in next-day psychological well-being (Models 1–4, Table 4). The first and second models address our hypotheses, H2a and H2b, respectively. Models 1 and 2 revealed that engagement with ‘Intellect’ on a certain day were followed by higher levels of overall resilience and mood levels on the following day, respectively (resilience (t): β = 0.122; 95% CI 0.039–0.206; mood (t): β = 0.020; 95% confidence interval (CI) 0.012–0.029) even after controlling for lagged values of resilience and mood. As an exploratory analysis, random effects did not show significant variability between subjects in the effects of engagement on resilience or mood (all p > 0.05, Table 4), signalling that participants reported largely similar improvements in these outcomes. ⁸⁰ As these random effects were non-significant, they were removed from the model to improve model fit. ⁸¹ The rest of the models examined the fixed and random effects of engagement on the four components of resilience. For exploratory analyses, Models 3 and 4 separately revealed that engagement with ‘Intellect’ was followed by lower stress levels and higher physical activity on the next day (stress (t): β = 0.018; 95% CI 0.004–0.032; physical activity (t): β = 0.079; 95% CI 0.032–0.126). Only the autoregressive effects on stress levels showed significant variability between subjects. The variation around the stress (t − 1) slope was 0.035, with estimated slopes ranging between 0.006 and 0.063 in approximately 95% of the population (Table 5). This illustrates that the magnitude of the effect of same-day stress on following-day stress levels differed substantially between participants. The remaining random effects were non-significant and were removed from the models to improve model fit. Participants reported improvements of largely similar magnitude on next-day stress and physical activity levels (Table 5). In the final two exploratory models, we observed that the fixed and random effects of engagement on sleep hours and sleep quality were non-significant (all p > 0.05, Tables 4 and 5).

Table 4.

Fixed effects estimates from the multilevel random effect models.

Variable	Outcomes
	Mood (Model 1)	Resilience (Model 2)	Stress levels (Model 3)	Physical activity (Model 4)	Sleep hours (Model 5)	Sleep Quality (Model 6)
Fixed effects
Intercept	2.21 (0.281)***	79.2 (2.55)***	10.6 (0.483)***	16.1 (1.53)***	12.0 (0.682)***	9.03 (0.335)***
Engagementt − 1	0.020 (0.005)***	0.122 (0.051)*	0.018 (0.008)*	0.079 (0.029)**	−0.005 (0.007), ns	0.015 (0.013), ns
Moodt − 1	0.068 (0.038), ns
Resiliencet − 1		0.047 (0.035), ns
Stress levelst − 1			0.002 (0.049), ns
P.A.t − 1				−0.036 (0.039), ns
Sleep hourst − 1					−0.011 (0.036), ns
Sleep qualityt − 1						0.036 (0.035), ns

Note: Bold values indicate statistical significance.

Estimates are unstandardised coefficients (B). Between parentheses: standard error of the estimate (SE). Omitted: random effects omitted from the model because of ns. ns: non-significant; P.A.: physical activity.

*p < 0.05.

**p < 0.01.

***p < 0.001.

Discussion

To evaluate user engagement on a daily mHealth intervention, this study first examined if employees with lower psychological resilience at baseline were more likely to sustain engagement with the app. To evaluate its day-to-day effectiveness, this study then investigated whether continuous app engagement was related to following-day resilience and mood using an intensive longitudinal design.

Consistent with H1a, employees with lower baseline resilience were significantly more likely to continue using ‘Intellect’ on a second day relative to those with higher baseline resilience. Previous studies on military and community samples demonstrated that individuals with lower resilience were more motivated to seek help from external resources ⁸² and mental health professionals.^83,84 Our study extended these findings to mHealth interventions. In previous studies, individuals with higher daily resilience mostly preferred seeking social support from close contacts^82,85 These support our observation that participants with higher baseline resilience were less likely to continue seeking support from external resources, such as a mHealth intervention. Though, more research is needed to critically evaluate engagement behaviours within the profile of highly resilient employees in order to make a conclusion. On the resilience components, lower baseline ‘sleep quality’ and higher baseline ‘stress levels’ independently predicted higher odds to continuous engagement with ‘Intellect’ and vice versa. This resonates with a recent study, where college students with high stress were significantly more likely than those with low-moderate stress to have used and still be willing to use self-help resources. ³⁸ Likewise, employees with insomnia were more motivated to adopt and participate in mindfulness apps or use them in conjunction with sleep medication. ⁸⁶ Possibly, because current treatment options for poor sleep quality (e.g. pharmacological and behavioural therapies) were short of meeting the needs of the working population. ⁸⁷

Contrary to Hypothesis H1b, we did not find a significant relationship between low mood and app engagement. This finding seems to challenge recent evaluation studies that negative affect (i.e. anxiety and depression) discouraged long-term engagement on the mHealth platform.^39,40 Notably, some of these correlations were weak. ⁸⁸ Other studies have shown that negative emotions causally predict the use of certain categories of app features (e.g. entertainment) but not others (e.g. educational). ⁸⁹ Given that psychoeducation is an important element in all of Intellect's self-care features, it is plausible that negative emotions thus would not influence engagement behaviours on these features. It is likely that this relationship was confounded by extraneous variables. For example, as our participants have engaged with ‘Intellect’ minimally for a day, they may carry different beliefs about the app which may further influence their return of engagement. This study did not control for participants’ perceived ease-of-use and perceived usefulness of the mHealth app, both of which had previously hampered app uptake in the adult population. ⁸⁹ Henceforth, our results need to be interpreted with caution. Researchers are encouraged to further clarify the contextual influences on the bidirectional relationship between emotions and app usage. Concordant with the literature, the relationships between lower baseline ‘sleep hours’ and ‘physical activity’ and continuous engagement with ‘Intellect’ were also non-significant. In comparison to poor sleep quality, deficits in sleep duration were found to be less important reasons to why employees would seek help from mHealth apps. ⁸⁶ Participants in past studies who were conscious of their lack of physical activity were more inclined to monitor and improve their daily fitness with wearable technologies and fitness apps, ⁹⁰ and would not be as excited to use a mental health app to achieve these goals.

Hypotheses H2a and H2b were also supported. Daily engagement with ‘Intellect’ significantly predicted next-day increases in resilience and mood after controlling for the autocorrelated effects of daily resilience and daily mood. On the resilience components, our exploratory models revealed that daily ‘Intellect’ engagement was associated with reductions in next-day stress levels and increases in next-day physical activity. To our knowledge, this is the first study to demonstrate that day-to-day app engagement was related to positive changes in dynamic workplace resilience. Systematic reviews have reported the effectiveness of web-based and mHealth interventions on employees’ stress and affectivity,^13,91 but only a minority of them focused on workplace resilience. ⁹² Even fewer studies examined ways to improve resilience on a daily basis, which may be an important research gap as daily resilience demonstrated carryover effects on work behaviours.^92,93 Furthermore, resilience interventions implemented in current organizations were often lengthy and intensive. ⁹⁴ For example, most companies have recommended organizational training programmes or external resources (e.g. coworker/supervisor support) as interventions to improve resilience. ⁹³ This study provides early evidence that evidence-based mHealth app interventions such as ‘Intellect’ may help employees respond adaptively to the fast-paced and dynamic nature of daily work. Theoretically rooted in Intellect's self-guided features, mental health interventions that similarly incorporated elements of mindfulness, emotional regulation, and cognitive-behavioural skills have previously made significant improvements on employees’ mood and stress levels (see ⁹⁵ for a review). However, most of the previous studies relied on between-level methods where they aggregated data across participants in the sample and generalised their findings to the population. ⁵⁵ Such methods would not evaluate intraindividual dynamic processes, and thus derived inaccurate conclusions especially when mood and stress demonstrate considerable day-to-day variations within-individuals.^96,97 Recent studies have demonstrated similar improvements on same-day outcomes (e.g. Sianoja et al.). ⁹⁸ Our findings extend the scant literature by using across-day analyses, which strengthens the dynamic effects of mHealth intervention like ‘Intellect’ on next-day workplace well-being. In addition, it is intriguing that employees would report increases in step count across-days as these are not often observed after using a mental health app. Walsh et al. ⁹⁹ recently presented that behavioural change features such as ‘goal-setting’, ‘action-planning’, ‘self-monitoring’, and ‘positive feedback’ in a personalised mHealth app substantially improved daily step counts. ‘Intellect’ shares these identical features within ‘Learning Paths’, and such features were also guided through the ‘Professional Coaching’ services. We encourage future research on mediators between mHealth apps and physical activity. Daily usage of ‘Intellect’, however, was not related to better sleep indices across days perhaps due to the lack of sleep-directed features. Sleep disturbances typically require targeted interventions for improvement. ¹⁰⁰ Our final exploratory analyses indicated that there were no significant interindividual differences in the found effects. The small random effect variances indicated a lack of individual differences in the magnitude of the lagged effect of daily engagement with ‘Intellect’ on psychological well-being. There is little degree of heterogeneity in the intervention (i.e. fixed) effects, signalling that the active users reported largely similar improvements.

Limitations

Nearly 30% of the participants remained active on ‘Intellect’ for at least three consecutive days, which is many times more than the current median rate of app usage (i.e. 4%).^101,102 Despite the early assessment of the real-world efficacy and engagement of ‘Intellect’, this study has several limitations. First, this study did not utilise a randomised controlled design. We are unable to draw a causal link between daily engagement with ‘Intellect’ and the various outcome measures. This also limits the inferences that can be made regarding the efficacy of ‘Intellect’ as compared to other smartphone interventions. Future research may consider including an active attention control group and a waitlist control group assessed similarly on a day-to-day basis. Secondly, the study did not control for other features that may possibly increase uptake and acceptance of a mHealth intervention. Consequently, our findings that lower baseline resilience (i.e. poorer sleep quality and stress levels), but not negative emotions, motivated sustained engagement need to be interpreted with caution. In testing resilience and/or mental health statuses as a motivational factor to app engagement, hence future research may benefit from systematically controlling for users’ perceptions and impressions of the mHealth app (e.g. perceived usefulness, perceived ease of use, and perceived risks). ¹⁰³ Thirdly, given that this study did not utilise any validated measures of resilience or affective states, our findings need to be interpreted with caution. Though, to our knowledge, there are no validated measures that could assess resilience on a daily basis. ¹⁰⁴ Nonetheless, we encourage future researchers to replicate our findings using validated resilience instruments that can be adapted for daily use to increase internal validity (e.g. Martinez-Corts et al.). ⁴⁵ Fourthly, few other mHealth studies termed ‘app engagement’ slightly differ as ‘having active contact with the intervention content’.^105,106 Future studies may consider adopting this alternative interpretation and re-evaluate our findings on engagement motivation. Fifthly, our findings are not applicable to every possible aspect of dynamic resilience. Dynamic processes of resilience may consist of other factors above and beyond physiological and stress indices, such as biological (gene-expression patterns), cognitive (appraisal patterns), behavioural (coping styles and self-efficacy), emotional (positive and negative emotions), and social (social connections). ¹⁰⁷ An index that encompasses all of these measures may be a more accurate representation that determines if employees are able to cope dynamically with work demands. Nevertheless, our current resilience composition characterises possibly significant facets of workplace resilience (see Winwood et al. ¹⁰⁸ ). Data in these four components can also be collected passively through wearable devices, which would maximise feasibility and optimise accuracy. This might be an interesting avenue for future research and can minimise social desirability biases that may exist in our current daily self-reports. Sixthly, without mediation analyses, we were not able to specifically evaluate which features in ‘Intellect’ were associated with better psychological outcomes. It remains plausible that our participants were commonly engaged with a particular set of features on the app. Researchers are encouraged to systematically compare the effectiveness of the different categories of features (i.e. self-guided and professional coaching services) in order to further support the overall efficacy of modern mHealth platforms such as ‘Intellect’. Lastly, this study has limited external validity to the working population at large. The current sample consists mainly of female employees (62.5%) residing in Singapore (88.7%) and Hong Kong (12.3%). The transferability of our findings can be further strengthened by replicating our study with more diverse samples.

Practical implications

Contrary to the reactive nature of traditional workplace interventions, our primary findings showed that mHealth applications like ‘Intellect’ could potentially improve resilience and affective states on a day-to-day basis. Moreover, the sample was large and comprised working adults residing in Singapore and Hong Kong, suggesting that these preliminary effects can possibly generalise to the larger Asian workforce. The study supports the current paradigm shift towards preventive care, where organizations are increasingly encouraged to adopt low-intensity and cost-effective mobile interventions to promote positive mental health among employees.^109,110 Of course, certain psychological issues require additional support in the form of psychotherapy or in-person counselling.^111,112 Occupational psychologists agreed that the current best practice to maximise employees’ well-being is for organizations to subsidise end-to-end mental health care.^113,114 A balanced approach in responding end-to-end care needs would markedly reduce the cost burdens of existing organizations which mainly rely on poorly utilised EAPs alone. ¹¹⁵

Further implications can also be drawn from our secondary findings, where employees with lower resilience were significantly more likely to engage with a mHealth app. Following recent evidence on utilizing habit formation to sustain app engagement, future app developers may first consider incorporating resilience-building features into their well-being applications. App designers may then use resilience-related problems (i.e. higher stress levels and poorer sleep quality) as cues for employees to engage with these features. Provided that these features are effective, this would eventually generate a habit of mHealth app use through conditioning. ¹¹⁶

Conclusion

This study evaluated the engagement motivations and possible effectiveness of an evidence-based mHealth application, ‘Intellect’, in improving next-day resilience and mood among employees in Singapore and Hong Kong. Employees with lower baseline resilience, contributed by higher stress levels and lower sleep quality, were more likely to continue using ‘Intellect’ as compared to those with higher baseline resilience. With daily usage of ‘Intellect’, employees reported positive changes in mood and resilience (i.e. stress levels and physical activity) on the following day. Our findings are useful given the potential for scaling up such brief, easily accessible, low-cost, and effective mHealth interventions to many organizations which currently aim to extend support to their employees. This study provides preliminary evidence that a mHealth app can possibly improve daily occupational welfare, which can translate into significant improvements of public mental health at the population level.