Feasibility and efficacy of a digital resilience training: A pilot study of the strengths-based training RESIST

Abstract

Background

Work-related stress is a risk factor for a number of adverse health and work outcomes. Resilience trainings are a promising approach for adequately dealing with work stress and keeping employees mentally healthy. However, results of previous resilience trainings have been heterogeneous, ranging from null findings to large effects. Existing digital resilience interventions show a lack of consistency in terms of an underlying theoretical framework and methods used to foster resilience. Positive Appraisal Style Theory of Resilience offers an innovative conceptualization of resilience. Strengths-based cognitive behavioral therapy is a corresponding therapeutically method reflecting resilience as a resource-oriented process of dealing with stress. Based on this background, a new hybrid web-and app-based digital resilience intervention for employees named RESIST was developed.

Objective

The first aim of the study was to investigate the feasibility of the newly developed training RESIST regarding its usability, user behavior, user experience and motivation to use. Second, the study sought to explore preliminary effects of the intervention on reducing stress and enhancing resilience by conducting a pilot randomized controlled trial.

Methods

The feasibility study was conducted in three phases. First, the usability of the app was investigated in a pre-test with five participants using a thinking-aloud method. Second, the preliminary efficacy of the training was examined in a pilot randomized controlled trial. A sample of 30 employees were randomized either to receive the resilience training (n = 15) or to be member of a control group (n = 15). The primary outcome was measuring perceived stress. Secondary outcomes included measures of resilience and depressive symptoms. Third, semi-structured interviews were undertaken with six participants of the resilience training group on training content, motivation for use, and user experience.

Results

Overall, results indicate that RESIST can be a feasible training for resilience promotion and stress reduction with high user satisfaction. Analysis of covariance showed that, relative to controls, participants who received RESIST reported significantly lower stress scores at post-intervention (F1,27 = 16.91, p < 0.001; Cohen's d = 1.57; 95 % CI 0.71–2.43) than controls. Significant differences, with moderate-to-large effect sizes, were also detected for general resilience and various resilience factors.

Conclusions

Results are promising and provide hope that a hybrid web- and app-based resilience intervention based on strengths-based cognitive behavioral therapy can have a positive impact on dealing adequately with stress and improve resilience of employees.

Highlights

• •Strengths-based CBT is feasible for digital interventions and a potentially effective method for promoting resilience.
• •PASTOR might be a useful theoretical framework for developing resilience interventions.
• •Preliminary results showed a reduction in perceived stress as well as an increase in resilience-related outcomes.
• •The flexible combination of web- and app-based training elements might have contributed to positive effects on mental health.

1. Introduction

Work can contribute to a fulfilling and healthy life (Bono et al., 2013) but can also be a source of psychosocial stress (van der Molen et al., 2020). Work-related stress can be a risk factor for a variety of diseases including sleep complaints (Litwiller et al., 2017), depression (Rugulies et al., 2017) cardiovascular diseases (Dragano et al., 2017; Eddy et al., 2017) metabolic syndromes (Tenk et al., 2018), gastrointestinal issues (Nixon et al., 2011) and cancer (Yang et al., 2019). Moreover, work-related stress was found to be associated with lower employee productivity (Burton et al., 1999), higher rates of sickness absence (Mortensen et al., 2017; Götz et al., 2018) and an early retirement (Hintsa et al., 2015; Juvani et al., 2018; Mäcken, 2019). The promotion and strengthening of resilience particularly with respect to work-related stress is therefore an important endeavor.

Resilience is assumed to be of major importance for coping with stress (Ang et al., 2022). Resilience describes the phenomenon that some people maintain or quickly regain mental health despite exposure to severe psychological or physical adversity, for instance work-related stressors (Kalisch et al., 2015; Kalisch et al., 2017; Ayed et al., 2019). The concepts of resilience in the mental health literature are, however, very heterogeneous. This poses a problem for integrating research findings as conceptualizations are reflected both in the methods used to measure resilience and for the design of intervention methods. Roughly, two theoretical approaches can be distinguished: One, defining resilience as a relatively stable personality trait (Hu et al., 2015); a second, which understands resilience as a result of a dynamic process by which the individual succeeds in adapting to adverse circumstances and therefore remains healthy despite the adversity (Ayed et al., 2019; Kalisch et al., 2015; Kalisch et al., 2017). The latter approach opens the perspective to train resilience (Feder et al., 2019).

In an attempt to integrate existing approaches into one model, the Positive Appraisal Style Theory of Resilience (PASTOR) was developed. The theory aims to provide a common conceptual framework for the understanding resilience ranging from neuroscientific to epidemiological research and including the development of transdiagnostic interventions as well as the investigation their mechanisms of action (Kalisch et al., 2015; Kalisch et al., 2017). PASTOR highlights the idea that resilience manifests when the individual is exposed to stressors and defines a certain set of evidence-based and modifiable resilience factors that are assumed to unfold their protective effects on mental health through common cognitive processes. It is assumed that these common processes are characterized by a positive appraisal style, positive reappraisal and the inhibition of negative appraisals. Resilience factors with strongest evidence for positive mental health include self-efficacy, optimism und social support (Helmreich et al., 2017). Although PASTOR has the potential for theoretical and empirical advancement in resilience research that has often been criticized for weak or even no definitions (Díaz-García et al., 2021; Helmreich et al., 2017), most support for comes from laboratory or observational research yet. In terms of interventions, theory suggests that resilience interventions should target a few selected transdiagnostic relevant issues, as opposed to psychotherapy, which addresses multiple targets and is substantially longer in delivery. However, PASTOR does not specify concrete therapeutically methods to foster resilience and so far, it is unclear if the theory provides a useful basis for digital resilience interventions.

During the last 10 years an increasing number of resilience trainings has been developed and evaluated and several meta-analyses and narrative reviews have summarized the evidence for the effects of resilience trainings (face to face and digital trainings) for mostly working adults (Ang et al., 2022; Macedo et al., 2014; Robertson et al., 2015; Vanhove et al., 2016; Lehr et al., 2018; Linz et al., 2020). However, these studies showed mixed results ranging from null findings to moderate and large effects on resilience outcomes (Leppin et al., 2014; Vanhove et al., 2016; Liu et al., 2020; Ang et al., 2022).

The heterogeneity might be explained by a missing unified framework for resilience and as a consequence difference in the content of interventions, therapeutically techniques employed or study characteristics may emerge. For example, interventions differ with regard to the consideration of stressors. Intervention could be provided before, during, or after stressors occur, with the goal either to prepare, or to support during stressful times or to mitigating the consequences of stressors (Chmitorz et al., 2018a). Interventions employ different therapeutically techniques that were mostly developed within the context of psychotherapy such as cognitive restructuring (e. g. Abbott et al., 2009), problem-solving (e. g. Bekki et al., 2013), stress inoculation (e. g. Varker and Devilly, 2012) or acceptance and commitment therapy (e. g. Bond and Bunce, 2000). All were developed to reduce mental distress while it is unclear, if these techniques are also best suited to promote resilience (Padesky and Mooney, 2012). In their systematic review Chmitorz et al. (2018a) found that no consistent definition of resilience is used in interventions. Moreover, only a few are based on a theoretical model that focus on personal strengths such as the broaden and build theory (e. g. Pidgeon et al., 2014), while most did not specify the underlying theory (e. g. Abbott et al., 2009; Pidgeon et al., 2014; Sood et al., 2011). Interventions target a wide variety of factors ranging from emotion regulation, empathy, active coping to self-compassion (Abbott et al., 2009; Bekki et al., 2013; Pidgeon et al., 2014), while theoretical or empirical justification for the selected factors might not always based on strong evidence (Chmitorz et al., 2018a; Helmreich et al., 2017). Finally, resilience scales, specific resilience factors (e.g. self-efficacy), mental health outcomes (e. g. depression) or perceived stress were used as operationalizations of resilience and might contributed to the heterogenous picture of results regarding the efficacy of resilience interventions (Chmitorz et al., 2018a).

In the present study we based intervention development on the theoretical approach PASTOR proposed by Kalisch and colleagues (Kalisch et al., 2015) and results from meta-analyses and systematic reviews on important resilience factors, e. g., optimism, social support, positive emotions (including self-compassion and self-care), locus of control and self-efficacy (Stewart and Yuen, 2011; Khazanov and Ruscio, 2016; Gallagher et al., 2020; Wang et al., 2021).

We hypothesized that strengthening resilience factors with adequate and well-established therapeutic techniques, should foster resilience. Padesky and Mooney (2012) argued that building personal resources (i. e., resilience factors) need different cognitive-behavioral methods than those employed for reducing distress, e. g., as part of stress-management trainings. Thus, in the present study the strengths-based cognitive behavioral therapy (strengths-based CBT), developed by Padesky and Mooney (2012), was adopted to design an evidence-based resilience training. Strength-based CBT aims to raise the awareness of successfully employed strategies to cope with obstacles resulting in the establishment of a resource-orientated self-concept (Padesky and Mooney, 2012).

Finally, when developing a new intervention to foster resilience, the delivery format must be carefully considered, particularly with respect to the target group.

Many people have integrated the use of mobile devices into their daily life and mobile apps have become more attractive in working environment. The handy format, low-threshold use and 24–7 availability of mobile apps may offer the opportunity to use mobile apps to practice training-related skills even throughout a stressful workday (Marciniak et al., 2020; Balaskas et al., 2021).

Digital trainings promoting resilience have also been on the rise in recent years (Chmitorz et al., 2018a), with most evidence-based digital interventions being web-based, while only a few are designed for mobile devices (Heber et al., 2017; Phillips et al., 2019). Some interventions prefer a hybrid format combining web- and app-based elements (Heckendorf et al., 2019; Ebenfeld et al., 2020). This hybrid solution might resolve the dilemma that necessarily longer text passages for psychoeducational content or exercises that require writing or careful reflection may be more suitable for a web-based format, while small-stepped, repetitive exercises for daily training (e. g., small CBT based and mindfulness based exercises, diaries) may be more effectively delivered and also more compliantly completed by a mobile application (Linardon et al., 2019). Initial studies indeed showed some potential of combining web and mobile elements (Linardon et al., 2019). So far, there is only little evidence on digital resilience-enhancing interventions for employees in the context of work-related stress (Weber et al., 2019).

1.1. Aims of the study

The aim of the study is two-fold: First, we sought to investigate the feasibility of the newly developed web- and app-based resilience intervention RESIST regarding usability, user experience and motivation to use it (Craig et al., 2008). Second, we aimed to explore preliminary effects of the intervention on reducing stress and enhancing resilience in stressed employees by conducting a pilot randomized controlled trial.

The results of this pilot trial will provide important information about the potential of this new intervention and will inform whether it is worth conducting a larger randomized controlled trial proving its more generalizable efficacy.

2. Methods

2.1. Intervention

We based the development of RESIST on the resilience framework proposed by PASTOR (Kalisch et al., 2015) as well as on previous evidence for well-established resilience factors, i.e., self-efficacy, locus of control, optimism, positive emotions and social support (Khazanov and Ruscio, 2016; Gallagher et al., 2020; Wang et al., 2021; Stewart and Yuen, 2011).

PASTOR provided the theoretical framework for the development of the training, determining which resilience factors were to be strengthened. The resilience factors which were trained in RESIST (self-efficacy, locus of control, optimism, positive emotions and social support) have in common that they promote a positive appraisal style and thus further support the development of resilience (Veer et al., 2021). However, PASTOR did not offer practical guidance regarding the design of exercises to promote resilience factors. Strengths-based CBT (Padesky and Mooney, 2012) provided a new method of how those resilience factor could be trained, claiming that resources should best be promoted by resource-focused intervention techniques. Padesky and Mooney (2012) stated that exercises that were developed to reduce problems and deficits, e. g. cognitive restructuring in psychotherapy, are suboptimal for building up resilience and proposed a cognitive behavioral approach aiming to increase strengths.

RESIST consisted of a web-based training and an accompanying smartphone-based mobile app for daily exercises. The web-based part of the resilience training included six sessions, which require about 45–60 min each to be completed. Participants were expected to complete one session per week so that the web-based training lasts in total six weeks. As previous research has shown that including the offer of an eCoach appears to enhance efficacy and adherence to online interventions (Santarossa et al., 2018; Musiat et al., 2022) after each session, each participants received email-based feedback from an eCoach within 48 h. The eCoach gave advice on the exercises and answered the participants' questions about certain training components. For the feasibility study, the eCoach was a psychologist with a master's degree and a degree in systemic counseling. The web-based training and the mobile app were connected through a technical interface. Thus, specific mobile app content was synchronized with the web-based training and was made available to participants within their personal web-based resilience training sessions.

Session one included an introduction to how participants can use the training to strengthen their resilience. In addition, trainees received psychoeducation on the theoretical approach of PASTOR used in the training. The resilience factors self-efficacy, locus of control, optimism, social support and positive emotions (including self-compassion and self-care) were each promoted in sessions two to five of the web-based training and were trained daily within the app. Session six was a review of the content and exercises taught during the training and is finished by planning the transfer of the exercises learnt in the training to everyday life.

Table 1 provides an overview on the structure of the web- and app- based training program. RESIST followed the four steps of strengths-based CBT (Padesky and Mooney, 2012) in order to strengthen these resilience factors: 1) search for strengths, 2) construct a Personal Model of Resilience (PMRe) including metaphors of resilient behavior, 3) apply PMRe and 4) practice PMRe. The app focussed on step 1), the web component targeted steps 2), 3) and 4).

Table 1.

Key training elements.

	Step 1 Search for strengths	Step 2 Construct PMR	Step 3 + 4 Practice PMR
Strengths-Based-CBT	Moments of resilience •Collecting past experiences of resilience •Assign them to resilience factor	Resilient self-image •Developing positive self-image/metaphor based on selected moment of resilience from app	Resilience project •Selecting future challenge in need of resilience •Planning to apply PMRe to cope with challenge •Overcome challenge
Training components	App	Web-based training	Web-based training

Note. CBT = cognitive behavioral therapy; PMRe = personal model of resilience.

To apply the strengths-based CBT approach within the app and the web-based training, three key training elements (moments of resilience, resilient self-image and resilience project) were provided to the participants (see Table 1). First, within the app participants were asked to collect so-called “moments of resilience” (see Fig. 1a). These are moments in which participants succeeded in doing things well despite major or minor obstacles. Thus, participants were guided to direct their attention to successful experiences in stressful situations. Once a moment of resilience was discovered, it could be linked with a photo of the situation (step 1, see Table 1 and Fig. 1b). Second, the participant was encouraged to reflect on this moment of resilience within a daily review (see Fig. 1b), thus stimulating positive appraisals. In this review the participant could also assign the detected moment of resilience to certain resilience factors (see Appendix Fig. A1) and then identify resilient behavior or cognitions that were helpful to cope with these challenging situations within (see Fig. 1c). The moments found with the app were also shown through a link at the start of each session of the web-based training to continue working with them in the web-based exercise “resilient self-image”. This exercise is intended to build a personal model of resilience (PMRe, step 2, see Table 1) and has a central role in sessions 2–4 of the web-based training. In this exercise, participants developed a narrative metaphor to describe a situation in which they felt resilient and reflects his or her strengths in that situation well (e. g., “I felt resilient at that moment because I maintained an overview like a lighthouse.”). The participant could reinforce and internalize this metaphor as a mental imagery via an audio guided exercise. Both techniques, the narrative metaphors and the mental imagery, are supposed to make the PMRe memorable and increase the motivation to apply it (Stott, 2010). In addition, participants ideally use these metaphors and associated mental imagery to generate new ideas when new challenges arise (Padesky and Mooney, 2012). Third, in sessions 2–4 of the web-based training, a “resilience project” - applying the PMRe- was planned and the implementation was prepared by the participants to cope with a present challenge (step 3 and 4, see Table 1). Additional, small exercises to strengthen resilience factors, so-called power-ups, were provided within the app and web-based training to foster resilience factors (self-efficacy, locus of control, optimism, social support, positive emotions, see Appendix Fig. A2a and A2b.

Fig. 1.

Screenshots of the RESIST APP.

1a. displaying the app screen to collect moments of resilience; 1b. displaying the app screen to start the daily review; 1c. displaying the assignment of resilient behavior or cognitions to a detected moment of resilience.

2.2. Study design and time frame

The feasibility study was conducted in three phases from December 2018 to June 2019. In phase 1, the usability of the newly developed web-based and mobile training components was investigated in a pre-test. In phase 2, the preliminary efficacy of the training was investigated in a pilot randomized controlled trial. This was followed by phase 3 during which we conducted semi-structured interviews with participants of the intervention group regarding the training's content, the motivation for use and the user experience (see Fig. 2 for details). The Ethics Committee of Leuphana University of Lueneburg (EB-Antrag_201811-11-78-Lehr_Resilienz) approved this study. It was registered as a feasibility study in the open science framework register (osf.io/a6qh8/).

Fig. 2.

Study flow.

2.2.1. Phase 1

Usability testing was conducted with five employees. They received access to the first session of the web-based training and to the app for one week. Following this first training phase, they were invited to explore the app and to complete specified tasks (i.e., collecting a resilience moment, reflect upon experience via review) within the app while being observed by a researcher. The participants were asked to “think aloud” and provide a running commentary of their thoughts while performing the tasks (Ericsson and Simon, 1998). The thinking-aloud method was chosen to elicit real-time feedback and emotional responses (Ericsson and Simon, 1998). These five participants did not participate in the pilot randomized controlled trial in phase 2 and 3.

2.2.2. Phase 2

In the pilot randomized controlled trial (pilot RCT), participants (n = 30) were randomly assigned to either receive access to the web-based resilience training and mobile app (RESIST) with eCoach guidance or to the waitlist control group that received access to RESIST eight weeks later. The group had full access to routine occupational health care in the meantime. Primary and secondary outcomes were measured at baseline (T1) and post-intervention (eight weeks post randomization, T2) using validated questionnaires.

2.2.3. Phase 3

At the end of the training phase semi-structured interviews by telephone (n = 6) were conducted with participants that finished at least five sessions from the web-based training about their experiences within the training.

2.3. Recruitment and procedure

For phase 1 of the study employees of several German companies were contacted and asked if they would like to participate in the testing of a newly developed digital resilience training. Those interested in participating prospected the study leaders via email to express their interest and then received general information on the study procedure and data protection by email and were asked to give their consent. Once consent to the study had been given, participants (n = 5) immediately received access to the first training session of the online training and to the app.

Participants for phases 2 and 3 were recruited (a) via posts on Instagram and Facebook, (b) as an incentive to participate in another study on attitudes toward e-mental health programs and (c) by actively addressing working/employed students via information events at a German university. Interested employees registered for the study at https://www.geton-training.de/resilienz and had to provide an email address in order to receive detailed information about the study and a link to the online screening questionnaire. Afterwards, they were randomized to one of the two intervention arms (training group or waitlist control) using a computer-generated randomization list with a ratio of 1:1 and a block size of 2 (http://randomisation.net/). The randomization was allocated in a concealed way by a researcher in our department who was not otherwise involved in this study. Blinding to group allocation was not feasible. The participants were informed about the randomization outcome via email. Participants allocated to the training group had immediate access to the intervention. Participants of the intervention group that finished at least five sessions of the web-based training were interviewed via telephone on aspects of user experience. Participants in the waitlist control group received access to the training after their eight weeks assessment.

2.3.1. Inclusion and exclusion criteria

For phase 1 the only inclusion criterion laid in being employed and having stable access to the internet and a smartphone.

For phase 2 and 3 all registered individuals who were employed and had stable access to the internet were asked to complete the online screening and sign the informed consent form. To allow comparison with the results of a digital stress management training (Heber et al., 2016), we included individuals who experienced elevated levels of stress measured with the total sum score of the Perceived Stress Scale-10 ≥ 22 (Klein et al., 2016; Reis et al., 2019). Subjects that were receiving psychotherapy or were on a waiting list for psychotherapy or who took part in other mental health trainings or participants showing suicidal ideation (Beck Depression Inventory-II, item 9 answers with score > 1, “I would like to kill myself” or “I would kill myself if I had the chance.”) were excluded. People taking medication to cope with stress (e. g., valerian, St. John's wort) were not excluded but were requested to keep their medication constant during the study (see study flow, Fig. 2).

2.4. Primary and secondary measurements

2.4.1. Primary outcome

The primary outcome of the pilot RCT was perceived stress measured by the 10-item Perceived Stress Scale (PSS-10) with each item rated from 0 to 4 and a Cronbach's α = 0.84. Lower scores indicated lower perceived stress (Klein et al., 2016; Reis et al., 2019).

2.4.2. Secondary outcomes

Secondary outcomes included resilience-related and mental health-related measures. Resilience-related outcomes comprised resilience, measured by the German version of the 6-item Brief Resilience Scale (BRS) with items that are rated from 1 to 5, and Cronbach's α = 0.83 (Chmitorz et al., 2018b). The BRS measured resilience as a dynamic, changeable construct and could be classified as an intervention adequate instrument (Chmitorz et al., 2018b).

The resilience factor -efficacy was measured using the 3-items Short Scale for General Self-Efficacy Beliefs (ASKU-3) rated from 1 to 5 and a ω = 0.86 (Beierlein et al., 2012). Locus of control was measured with the internal-external control belief scale (I-E) that contained the 2-items subscales (Internal control belief ω = 0.71 and external control belief scale ω = 0.63). Each item was rated from 1 to 5 with higher scores indicating higher control belief (Kovaleva et al., 2012). The resilience factor optimism was measured with the German version of the revised version of the Life Orientation Test (LOT-R). The scale consists of 10 items that are rated from 0 to 4 with a Cronbach's α = 0.59 for the optimism subscale and a Cronbach's α = 0.69 for the pessimism subscale (Scheier et al., 1994; Glaesmer et al., 2008). The resilience factor social support was measured with subscales of the Berlin Social Support Scales (BSSS), including perceived available emotional social support subscale containing 4 items (α = 0.73), the perceived available instrumental social support subscale containing four items (α = 0.69), and the support seeking subscale with five items (α = 0.81). The item ratings ranged from 1 to 4 for all subscales (Schulz and Schwarzer, 2003). The German version of the Self-Compassion Scale Short Form (SCS-SF) was used to measure the resilience factor experienced self-care (α = 0.91). The rating scale ranges from 1 to 5 (Hupfeld and Ruffieux, 2011).

As measure of the mental health outcomes the level of depressive symptoms was assessed by the 20-item Center for Epidemiological Studies-Depression Scale (CES-D), each item rated from 0 to 3 and a Cronbach's α = 0.88 (Hautzinger et al., 2012). Scores above 23 indicated clinically relevant symptom severity (Hautzinger et al., 2012). The 18-item Brief Symptom Inventory was used to assess psychological distress including physical symptoms (Derogatis, 2017). The level of distress in the past week was rated using a 5-point Likert scale ranging from 0 (not at all) to 3 (severely).

Work-related characteristics included measures with the 16-item short version of the Effort-Reward-Imbalance questionnaire (Rödel et al., 2004; Siegrist et al., 2009). Demanding aspects of the work environment were assessed with three items of the effort subscale. The subscale work-related reward was measured with seven items and the subscale over-commitment was measured with six items. The rating scales range from 1 to 5 (Rödel et al., 2004; Siegrist et al., 2009). Furthermore, a ratio between effort (nominator) and reward (denominator) was formed to calculate the imbalance between costs and gains experienced at work (Lehr et al., 2010). According to Lehr and colleagues, a ratio > 0.715 represented an adverse workplace situation. Work ability was measured as a single-item score from the Work Ability Index; range 0–10 (Ahlstrom et al., 2010). Subjects' number of full days on sick leave (absenteeism) and number of full days with reduced efficiency at work while feeling ill (presenteeism) over the past four weeks were assessed with the German Version of the Trimbos/Institute of Medical Technology Assessment questionnaire for costs associated with psychiatric illness (Bouwmans et al., 2013).

Exposure to critical life events was collected retrospectively, using an adapted German version of a standard Life Events (LE) Checklist with 27 items (Canli et al., 2006). Participants indicated whether the event occurred within the previous three months. Occurrences of events were summed up in a life event sum score. Daily hassles were assessed using the Mainz Inventory of Microstressors (Chmitorz et al., 2020). Participants reported the number of days microstressors occurred (from a list of 58 Daily hassles) within the past seven days (range 0–58 × 7 days = 406).

2.5. Feasibility measures

2.5.1. User experience

To measure user experience of the training, the 10-item questionnaire short form of AttraktDIFF (Hassenzahl and Monk, 2010) was exploited. A global user-experience score was generated by summing mean scores from two user-experience subscales with four items each: pragmatic quality (PQ), hedonic quality-identity (HQ) and 2 single-item scales goodness and beauty.

At the end of each web-based trainings session participants were asked about the benefit and difficulty of completing the session using a self-developed scale with a range from 0 to 4 (“Was the session useful for you?”, “Was the session easy for you to pass through?”).

Satisfaction with the intervention was measured by the 8-tems Client Satisfaction Questionnaire (CSQ, item answer scale ranging from 0 to 4, α = 0.93) adapted to the online context (Boß et al., 2016).

Training adherence was measured by the number of sessions completed, self-assessed training time per session, number of moments of resilience, resilient self-image and power-ups collected.

In addition, observations through thinking aloud in phase 1 as well as semi-structured interviews in phase 3 were used to assess user experience.

2.6. Data analysis plan

All analyses are reported in compliance with the CONSORT-EHEALTH guidelines for improving and standardizing the report of web-based and mobile health interventions (Eysenbach, 2011) and the extension to pilot and feasibility trials (Eldridge et al., 2016).

2.7. Sample size calculation

Sample size calculation followed the recommendations of Bell and colleagues for feasibility studies (Bell et al., 2018): For a 80 % powered main trial with an expected medium effect size, the pilot should have 10 subjects per arm. Thus, the target sample size for the pilot randomized trial was 20 participants to detect an expected effect (for the primary outcome PSS-10) of d = 0.4 with a significance level of α = 0.05 and a power of 80 %. The effect of d = 0.4 represents 2.5 points differences between groups assuming a standard-deviation of 6.2 (Heber et al., 2016). Taking a dropout rate of 30 % into account, we attempted to recruit 30 participants in order to obtain a final n = 20.

2.8. Statistical analysis

Quantitative analyses were performed using IBM SPSS v. 26. No interim analyses for intervention efficacy were conducted. Reported p-values are two-sided, with the a priori threshold for statistical significance set at α = 0.05.

Analysis was performed in the complete-case population, including all randomized participants who provided data at both the pre- and post-assessment. The training group and wait-list group were compared at eight weeks (T2) using analysis of covariance (ANCOVA) with baseline levels of the primary and secondary outcomes as covariates. Between-group Cohen's d values were calculated using pooled estimated marginal means and standard deviations of the training group and waitlist control group (WLC) groups at eight weeks. All participants completed the baseline assessment. At 8 weeks (T2), 6.6 % (n = 2 in the training group) of the data were missing.

2.8.1. Missing data

All participants completed the baseline assessment. At eight weeks (T2), 13 % (n = 2 in the trainings group) of the data were missing. Additional sensitivity analyses with the last observation carried forward method were performed for the primary outcome to replace the missing data.

2.9. Qualitative analysis

Thematic analysis as described by Braun and Clarke (2006) was used to develop an understanding of the qualitative data from the conducted interviews in phase 3. Two authors independently reviewed and coded a subset of the transcripts and discussed and resolved any inconsistencies to arrive at a shared interpretation of the data. The first author coded the remaining transcripts. These were reviewed by the second author for inconsistencies.

3. Results

3.1. Phase 1

In general, the participants got along well with the app and were able to navigate through it and use the main functions without any difficulties. In addition, after completing the first session of the web-based training, the five participants understood the central content and training objectives. All five users used smartphones running on iOS.

Nevertheless, usability testing via the thinking aloud method revealed several modifiable technical bugs and user experience issues. There were general app usability issues: large login delays, password requirements appeared to be too strong, uncomfortable back- and forth navigation within the app, the home button was not usable, and difficulties emerged regarding the display of uploaded photos. Moreover, improvement of some intervention specific app functions was indicated. Participants had difficulties using the daily review function (e. g., self-reflecting about well-being and experienced annoyances). Two participants did not find this function. All participants found it difficult to answer the reflection questions embedded in the review. In addition, three participants found it difficult to rate the detected moments of resilience and to decide the extent in which these affected certain resilience strengths. Most difficulties were experienced in the use and implementation of the app exercises called “power-ups”. All participants had difficulties choosing the power-ups to train and did not realize that the power-ups were assigned to individual resilience factors. Users also found that the selected exercises were not adequately displayed. These usability bugs were fixed before starting study phase 2.

3.2. Phase 2

3.2.1. Participants

In study phase 2, a sample of 30 individuals were randomized to RESIST (n = 15) or the waitlist control group (n = 15) (Fig. 1). The majority of the study's participants (25/28; 89.3 %) had not undergone health training (e. g., yoga) in the last three months and had no experience with apps for health promotion with in the last three months. All participants (n = 30) reported working under adverse workplace conditions (ERI ratio > 0.715) at the start of training, indicating high stressor exposure. In terms of overcommitment, the participants in the trainings group and control group scored between norm values of healthy and clinically ill employees (Lehr et al., 2010) before start of the training. Overall, participants in both groups showed a moderately reduced workability as compared to lift-time best (intervention group M = 7.27; SD = 1.22; control group 7.00, SD = 1.46). In both groups more days of presenteeism then days of absenteeism were reported at the start of the training. In addition, on average, participants of the intervention group reported experiencing 70.33 (SD = 28.64) daily hassles in the past seven days and 7.87 (SD = 3.64) critical life events in the past three months. The participants of the control group reported an average of 82.40 (SD = 27.77) daily hassles in the past days and 7.53 (SD = 5.83) critical life events in the past three months.

Table 2.

Baseline characteristics in study phase 2 and 3.

	Total (n = 30)	RESIST (n = 15)	Controls (n = 15)
Sociodemographics
Females, n (%)	22 (73.3)	11 (73.3)	11 (73.3)
Married/partnership, n (%)	13 (43.3)	7 (46.7)	6 (40.0)
Age (years), mean (SD)	40.1 (12.9)	40.9 (15.8)	39.4 (9.7)
Working characteristics
Years of occupational experience, mean (SD)	14.6 (11.9)	15.2 (13.4)	14.1(10.6)
Full-time, n (%)	17 (56.6)	8 (53.3)	9 (60.0)
Education level
University degree, n (%)	19 (63.3)	11 (73.3)	8 (53.3)
High school diploma, n (%)	28 (93.3)	15 (100)	13 (86.7)
Clinically relevant depressive symptomsa
Yes, n (%)	4 (13.3)	1 (6.7)	3 (20)
No, n (%)	26 (86.7)	14 (93.3)	12 (80)
Stressor exposure before training
Adverse workplace situation (ERI >0.715), n (%)	30 (100 %)	15 (100 %)	15 (100 %)
Daily hassles, mean (SD)	76.4 (28.4)	70.3 (28.6)	82.4 (27.8)
Life events, mean (SD)	7.8 (4.8)	7.9 (3.6)	7.6 (5.8)

^aDepressive symptom score > 23; CES-D.

3.2.2. Primary and secondary outcome analysis

A significant group effect in the ANCOVA indicated lower scores on the PSS-10 for the RESIST study completer group at post-intervention as compared to the WLC (F1,27 = 16.91, p < 0.001). Compared to the control group, stress of the RESIST group was reduced on average by eight points. Large between-group effect sizes were observed at post-intervention (Cohen's d = 1.57; 95 % CI 0.71–2.43).

Sensitivity analysis with last observation carried forward method also showed a large between-group effect size at post-intervention (F1, 29 = 9.78, p = 0.004; Cohen's d = 1.15, 95 % CI 0.37–1.93). Perceived stress of the RESIST group was reduced on average by six points.

Chmitorz et al. (2018a) recommend that “resilience intervention studies should control for stressor-exposure in the statistical analysis. The number of micro- and macrostressors could be considered as covariates in an analysis of covariance for mental health” (p. 86). Therefore, another ANCOVA with daily hassles, critical life events and work-related effort as covariates was calculated for perceived stress and showed comparable results (F1, 27 = 18.51, p < 0.001; Cohen's d = 1.66, 95 % CI 0.79–2.53).

3.2.2.1. Depressive symptoms

No significant differences in reduction in depressive symptoms between study groups were found at post-intervention (d = 0.38, see Table 3).

Table 3.

Effects of RESIST group (study completers, n = 13) compared with the control group (n = 15) on primary and secondary outcomes at post-test.

Outcome and assessment point	RESIST (n = 13)	Controls (n = 15)	Differences between study conditions
	Observed mean (SD)	Observed mean (SD)	Estimated mean difference between group (95 % CI)a	Cohen's d (95 % CI)
Stress
Perceived stress scaleb
T1	23.61 (4.27)	25.13 (3.60)
T2	13.38 (5.24)	22.0 (5.38)	−7.79 (−11.68, −3.89)	1.57 (0.71; 2.43)
Depressive symptoms
Center for Epidemiological Studies-Depression Scaleb
T1	10.60 (4.19)	15.73 (6.13)
T2	7.08 (4.57)	11.20 (5.10)	−1.32 (−4.64, 2.00)	0.38 (−0.37; 1.13)
Psychological distress
Brief symptom inventory
General severity index
T1	0.72 (0.37)	0.83 (0.35)
T2	0.37 (0.32)	0.63 (0.28)	−0.21 (−0.42, −0.01)	0.80 (0.03, 1.57)
General resilience
Brief Resilience Scale
T1	15.84 (3.05)	15.07 (3.47)
T2	19.46 (3.92)	16.33 (3.68)	2.43 (0.09, 4.77)	0.81 (0.04; 1.59)
Optimism
Life Orientation Test-Revised
T1	14.08 (4.94)	12.00 (3.68)
T2	16.54 (3.45)	13.20 (2.96)	2.34 (0.41, 4.26)	0.96 (0.17;1.75)
Self-efficacy
General Self-Efficacy Beliefs
T1	10.08 (1.26)	9.60 (1.72)
T2	11.38 (1.39)	10.33 (3.11)	0.86 (−0.88, 2.61)	0.38 (−0.37; 1.13)
Locus of control
Internal-external control belief scale
Internal control belief
T1	3.57 (0.61)	2.83 (0.69)
T2	4.04 (0.59)	3.07 (0.56)	0.83 (0.33, 1.33)	1.36 (0.53; 2.19)
External control beliefb
T1	2.77 (0.64)	2.97 (0.69)
T2	2.38 (0.77)	2.80 (0.73)	−0.36 (−0.95, 0.22)	0.48 (−0.27; 1.24)
Positive emotions
Self-compassion scale
T1	2.54 (0.46)	2.76 (0.64)
T2	3.40 (0.61)	2.63 (0.54)	0.64 (0.26, 1.03)	1.33 (0.51; 2.16)
Social support
Berlin social support scales
Perceived Instrumental support
T1	14.08 (2.92)	12.47 (2.59)
T2	15.08 (4.73)	13.13 (2.21)	0.99 (−1.93, 3.93)	0.27 (−0.47; 1.02)
Perceived Emotional support
T1	14.54 (1.76)	13.53 (2.39)
T2	15.54 (1.12)	13.27 (2.31)	1.72 (0.61, 2.83)	1.22 (0.41, 2.04)
Support seeking
T1	15.23 (2.62)	12.07 (2.40)
T2	16.31 (2.56)	13.13 (2.67)	1.13 (−0.78, 3.03)	0.50 (−0.26; 1.25)

^aBetween group differences in means (95 % CI) were calculated using estimated marginal means.

^bLower scores indicate lower perceived stress/depressive symptoms/psychological distress/external control beliefs.

Psychological distress.

Significant differences were found on the overall scale general severity index between the groups (d = 0.80, see Table 3) at post-intervention.”

3.2.2.2. General resilience and resilience factors

Significant differences at post-intervention between groups were detected for general resilience and the resilience factors optimism, internal control belief, self-compassion and perceived emotional support with effect sizes ranging between d = 0.81 for general resilience and d = 1.36 for internal control (Table 3).

No significant differences between study groups were found for general self-efficacy, external control belief and the two social support subscales perceived instrumental support and support seeking at post-intervention (Table 3).

Work-related outcomes.

No between-group differences of absenteeism, presenteeism and work ability were observed at post-intervention (see Appendix Table A.1).

3.2.3. Feasibility outcome analysis

3.2.3.1. User experience

In the training group, 12 of 15 participants provided data at post-assessment about the use of RESIST. On average, the participants rated the user experience of the app with M = 4.6 (SD = 0.9), which is to be assessed as positive on a rating scale from 1 to 7 (Hassenzahl and Monk, 2010, Appendix Fig. A3).

Twelve participants replied that the overall satisfaction with the web- and app-based training was high (n = 12; M = 28.3, SD = 3.6), indicating that all were “satisfied in an overall, universal sense” with the training (item 7, answering “does totally apply” or “does partly apply to me”). Participants indicated that they received the kind of training they wanted (12/12), that the training met their needs (11/12), that they were satisfied with the amount of training they received (11/12), that the training has helped them to deal more effectively with their problems (11/12), and that they would use the training again if they needed to (11/12). Moreover, all 12 participants stated that they would recommend the training to a friend. Five participants reported their reason for dropping out of the intervention defined as dropping out before finishing session six; one participant remarked that the training did not suit him, one participant replied quitting due to technical problems (i.e., logging in), two participants noted not having enough time and one participant had not yet completed the training.

Participants rated the personal benefit of the single session of the web-based training on a scale from 0 (“not useful at all”) to 4 (“very useful”) between M = 3.1 (session 6) and M = 3.4 (session 3).

The level of difficulty of processing each session on a scale from 0 (“very easy”) to 4 (“very difficult”) was evaluated on average between M = 0.6 (session 6) and M = 1.6 (session 2).

Intervention Use.

On average, 4.3 sessions (SD = 2.0) were completed in the training group (session one by 14/15 participants, session 2 by 12/15 participants, session three by 12/15 participants, session four by 12/15 participants, session five by 9/15 participants, all sessions by 7/15 participants). Those who completed the training needed 48.0 days (SD = 15.2) on average. Most participants required on average 0.5 to 1 h for each session.

Most participants (8/12) reported that they used a combination of the web- and app-based training. Three participants only used the online training and one participant first used both, online training and app, and then switched to app only.

3.2.3.2. Moments of resilience

On average, participants collected 50.5 (SD = 85.6) moments of resilience (ranging from 0 to 300 collected moments of resilience) within the first eight weeks after randomization. The participants collected diverse moments of resilience, which they could assign to one of the trained resilience factors in an app review (see Appendix Table A.2).

3.2.3.3. Power-ups

On average, participants planned 5.7 (SD = 7.9) exercises and indicated 2.9 (SD = 4.1) of them as implemented over the course of the training.

3.2.3.4. Resilient self-image

Thirteen out of 15 participants were able to develop at least one verbal description of a resilience self-image (86.6 %). More than half of the participants (9/15; 60 %) succeeded in developing a resilience self-image as a metaphor (see Appendix Table A.3).

3.3. Phase 3

Telephone interviews about training experience were conducted with participants that finished at least five sessions of the web-based training. Of the nine participants who had completed at least five sessions, six participants were available to be interviewed. The interview was divided into five areas: access to the training, motivation to use, understanding of the training concept, user experience and usage behavior with different themes identified per area (see Appendix Table A.4–A.8).

3.3.1.1. Access to the training

Most of the interviewed participants had access to the training because they came across the study by accident via social media, i.e., Instagram post (n = 5). Only one participant stated that he specifically searched the internet for a way to increase resilience.

3.3.1.2. Motivation to use

The participants stated various motives as the reason for their participation. Three of them cited that they participated in the training because they felt suffering from psychological distress.

In addition, one participant expressed beginning training in hope of improving his well-being. Another motive for using training was the aim of personal development by increasing resilience and knowledge about it (n = 2).

3.3.1.3. Understanding of the training concept

Half of the interviewed participants said they thought the training helped them learn to better cope with stress due to a new appraisal style (n = 3). Furthermore, two participants indicated self-reflection as important training content and one participant mentioned focusing on the positive.

3.3.1.4. Usage behavior

The use of the training and training components varied between the interviewed participants. There were participants who primarily used the app (n = 2), others primarily used the web-based training (n = 2), and others used a combination of both on a regular basis (n = 2). One participant reported not completing the training. The reasons for this were: feeling helped already and having time pressure.

3.3.1.5. User experience

There was positive and negative feedback regarding the user experience of the web-based training. One part of the participants found the web-based training exhausting and time-consuming (n = 2). Another part found the web-based training component very important and stated that it was important for reflection and knowledge input (n = 2). Most of the participants rated the user experience of the app as an easy-to-use, easy-to-understand and efficient tool (n = 4). Nevertheless, another part found the app unhandy and had technical difficulties, especially with the power-ups (n = 2). All interviewed participants would recommend the training to others (n = 6).

4. Discussion

4.1. Principal results

This pilot study investigated the feasibility and preliminary efficacy of a newly developed intervention for resilience (RESIST), guided by an eCoach in a sample of employees with an elevated stress level. Therefore, a mixed-methods approach was employed focusing on the hybrid intervention concept of combining web- and app-based components, the application of the theoretical framework of the PASTOR model combined with evidence syntheses on resilience factors and the usage of methods for building resilience according to strengths-based CBT. Overall, results indicated that RESIST can be a feasible and effective training for resilience promotion and stress reduction with moderate to large effects.

4.2. Feasibility

In the first phase, a usability pre-test was conducted, as recommended by Craig et al. (2008). Results revealed several issues that could be solved before proceeding including login, navigation, photo upload and usage of daily review.

In the second quantitative phase and the third qualitative phase, feasibility of the training was investigated from various perspectives: First, user satisfaction with the training was high and comparable with another hybrid, web- and app-based mental health interventions to cope with panic disorder (Ebenfeld et al., 2020; Ebenfeld et al., 2021) and slightly higher as compared to a gratitude training (Heckendorf et al., 2019). Second, most participants completed five out of six training sessions and half of the participants completed the whole training. This is in line with adherence rates for a guided stress management training addressing a similar group of participants (Zarski et al., 2016). Third, results for standardized measures of user experience and attractiveness all fell into the positive range of the semantic differential (Hassenzahl and Monk, 2010). Fourth, in the third phase, motives for use and usage behavior were identified in semi-structured interviews. Most of the participants stated that they were motivated for participating because of suffering from psychological distress. This result is in line with a recent systematic review that suffering from mental stress often motivates participants to take part in trainings (Borghouts et al., 2021). Interestingly, some of the participants expressed the wish to develop themselves personally via training. This indicates that employees especially find it important to continuously develop themselves and to strengthen the ability to cope with stressful workdays (Bakker and van Woerkom, 2018).

Beyond these general aspects of feasibility, it was particularly interesting how participants perceived the aspects of the training directly related to the PASTOR model and the strengths-based CBT approach. The interviews revealed that participants understood the core idea of PASTOR of being resilient in relation to having a functional (positive) appraisal style. Those results indicate the participants understanding of the underlying theory and goals of the training. That might have contributed to a comparably high engagement with the intervention, an association that was previously found reported by Tursi et al. (2013).

For the strengths-based CBT approach, searching for personal strengths during daily life is crucial (Padesky and Mooney, 2012). To facilitate this step, the collection of moments of resilience, the app was developed. Participants on average collected 50 moments of resilience over the course of six weeks, indicating that participants used the app training regularly for identifying their strengths. The engagement in the app is comparable to the number of gratitude moments collected using the app of a hybrid web- and app based gratitude training (Heckendorf et al., 2019). The high number of moments of resilience collected is particularly noteworthy, as other studies reported low app usage (Linardon et al., 2019; Linardon and Fuller-Tyszkiewicz, 2020). Thus, regular app usage (e. g., collecting moments of resilience) might be particularly important for the efficacy of RESIST, as the narrative review of Linardon and Fuller-Tyszkiewicz (2020) revealed that greater engagement is associated with greater improvement in mental health.

The usage of narrative metaphors and mental imagery is another important characteristic of the strengths-based CBT approach. About 60 % of the participants were able to develop one or more metaphors to describe their resilient self-images. Metaphors often utilize rich and distinctive imagery, better memorable than words (Stott, 2010). When it comes to envisioning positive events, imagery is linked to a positive mood greater than thinking about positive events in words (Holmes and Mathews, 2010). Positive moods are empirically linked to an increase in emotional resources as well as to health promotion, well-being and resilience (Fredrickson, 2001). However, 40 % of the participants used a plain verbal description of the resilience self-image without developing a metaphor. Reasons could be found in personality differences in metaphoric thinking (Fetterman et al., 2016) and in the complexity of the task of finding a personally suiting metaphor that could need more intensive support from an eCoach (Ang et al., 2022).

From the perspective of intervention developers, we found the combination of PASTOR and strengths-based CBT very useful in finding a focus given the almost overwhelming number of approaches to designing resilience interventions. PASTOR provided guidance on the selection of resilience factors to be considered, strengths-based CBT provided concrete therapeutic techniques aimed at building personal strengths, and PASTOR in again helped to frame and focus all exercises in the intervention, as the theory emphasizes that everything should lead to a positive appraisal style, which is assumed to capture the common core of all resilience factors.

All participants stated that they initially used a combination of the web- and app-based training components and would recommend others to do so as well. This finding highlights the importance of both training components. Interestingly, some participants preferred using both components as the training program progressed, others the online training and some used primarily the app. Seen from the perspective of naïve users, not following the proposed combined usage could be regarded as a deficit in adherence, with the consequence of providing stronger external structure in a revised version of the intervention (Shorey and Chua, 2022). However, from the viewpoint of capable user, this flexible pattern of usage may indicate the competence of the user to make the best out of the online training by choosing those parts that lead to the best individual benefit and fit best to daily life (Borghouts et al., 2021). In line with the capable user perspective, Ang et al. (2022) argued that interventions should offer such flexibility to consider individual preferences in the usage of online interventions. As a consequence, participants should be actively encouraged by the program itself and the eCoaches to use the components of the intervention in their own way and thereby provide users with more control (Phillips et al., 2019). However, the eCoach should monitor the participant's stress development and in case participants are not satisfied with their progress they should be encouraged to make use of all the parts of the training.

4.3. Efficacy

Regarding the efficacy of RESIST, participants reported 40 % less perceived stress as compared to the stress level of controls, a reduction that could be considered as practically meaningful (Bauer-Staeb et al., 2021). The difference between groups of over eight points on the PSS-10 clearly exceeded the non-inferiority margin for stress reduction in the PSS-10 of two points as proposed by Boß and colleagues (Boß et al., 2021). The observed effect on stress reduction compared to controls (d = 1.6) was considerably higher than expected. In the meta-analysis of Ang and colleagues (Ang et al., 2022), the authors reported no significant stress reduction and a small effect (d = 0.1) on post-intervention of mainly self-help digital resilience trainings for various populations.

Regarding secondary outcomes, the effect on depressive symptoms (d = 0.3) was comparable with interventions targeting depressive symptoms directly in non-clinical (g = 0.4, Reins et al., 2021), but lower than in clinical samples (d = 0.5, Karyotaki et al., 2018).

With respect to resilience-related outcomes, significant and large increases in resilience were demonstrated in this pilot study (d = 0.8). This effect size was larger than in previous meta-analyses of non-digital (Leppin et al., 2014; Robertson et al., 2015; Vanhove et al., 2016; Liu et al., 2020) and digital resilience trainings (Ang et al., 2022). Ang et al. (2022) found moderate resilience enhancing effects (g = 0.5) for digital resilience trainings.

According to the PASTOR framework and the evidence synthesis at the time of training development the resilience factors self-efficacy, locus of control, optimism, social support and positive emotions were chosen and trained systematically in the intervention and assessed by one or more measures in this study. Overall, the chosen factors were modifiable, and effects compared favorably with existing research findings (Kunzler et al., 2020b; Kunzler et al., 2020a). In contrast to prior research (Kunzler et al., 2020b; Kunzler et al., 2020a; Blessin et al., 2022) effects on perceived emotional support were observed in the present study that could be explained by the fact that RESIST dedicated one complete session on this topic. As social support is an important protective factor (Schwarzer and Leppin, 1989) building social resources might need special attention in resilience interventions. Therefore, social support might not increase significantly in more general resilience interventions, which do not address this topic as extensively as RESIST.

Finally, the results can be discussed considering other interventions using strengths-based CBT. The effect on stress reduction found in the present trial was larger than effects observed in a previous study comparing a face-to-face strengths-based CBT intervention and a waitlist control group in a student sample (d = 0.4) (Victor et al., 2017). In a study comparing online strengths-based CBT with face-to-face strengths-based CBT and waitlist control group, significant differences between the face-to-face intervention and waitlist control group were found for reduction in depressive symptoms and increase in resilience. In contrast, no significant differences were found between online strengths-based CBT on the one hand and the face-to-face format and also waitlist control group on the other (Victor et al., 2018). These results suggest that, to date, there is little evidence regarding the feasibility for digitally delivered strengths-based CBT interventions. However, the present study provides initial indications.

To summarize, results of this pilot RCT indicate substantial effects on stress, resilience and the trained resilience factors. Although this pilot trial does not allow strong conclusions to explain the favorable effects of RESIST, some of the training features described above may have contributed to its effectiveness. First, engagement with the intervention was high and might have contributed to the effects (Linardon and Fuller-Tyszkiewicz, 2020). Second, participants' use of metaphors and associated imagery might be effective for increasing a positive mood (Holmes and Mathews, 2010). Third, the flexible usage of the web-based and app parts of the intervention seemed to be beneficial (Ang et al., 2022). Fourth, as various exercises within the intervention were complex, the support of an eCoach might have increased efficacy (Phillips et al., 2019). Finally, the chosen resilience factors based on the PASTOR framework might represent a balanced combination of factors that improve resilience (Ang et al., 2022).

4.4. Limitations

The following limitations should be considered when interpreting the results.

First, the goal of the study was to explore the range of potential training effects in order to design a large RCT which allows to generalize the efficacy of RESIST. As an association was found between small samples and inflated effects with regard to digital resilience trainings (Ang et al., 2022) as well as in other fields (Kühberger et al., 2014) the observed effects from this pilot study should not be overstated. They should be considered carefully as a first orientation in combination with evidence from other digital interventions and could be used to define an effect that is practically meaningful for the effect size estimation for a larger randomized trial (Bell et al., 2018; Cook et al., 2018).

Second, data were collected only at baseline and eight weeks after the start of training, which limits the evidence regarding long-term training effects. This may be sufficient for an initial evaluation of the training effect in the context of a pilot study. Future studies, however, should consider further follow-up measurement points to investigate if trainings effects can be sustained.

Third, the sample was highly educated with >60 % holding a university degree and >90 % holding a high-school diploma. Therefore, positive results on the understanding of the training concept and theory might not apply to samples with education levels representative for the general population. This is a general shortcoming of internet interventions, as comparably high education levels have been observed in previous trials on digital interventions (Späth et al., 2017).

4.5. Implications for the planning of a larger RCT and further development of the training

We identified several direct implications that should be considered in the design of a large RCT as well as in the further development of the training.

First, the encouraging large effects open various avenues for planning RCTs. In order to reach a larger population with a positively connotated mental health intervention a self-help version of RESIST that requires fewer personal resources could be a useful option for universal prevention. As the present study aimed to reach and included highly stressed employees in a universal prevention setting smaller individual effects are expected but a larger reach could compensate and make the intervention useful from a public health perspective (Behrendt et al., 2020; Ebert et al., 2021).

Secondly, in order to improve mental health promotion, it would be of interest to investigate how RESIST relates to already existing and widely available interventions i. e., self-help books for resilience. It could be argued that established self-help books represent the low-threshold prevention standard and should be considered as an active and real-life comparator to a digital resilience training in future trials (Ang et al., 2022).

Third, although this pilot study gives encouraging indications that RESIST could be effective in reducing stress und enhancing resilience, little is known about the actual process through which the training works. Investigating if RESIST unfolds its effect via the proposed resilience factors could contribute to the knowledge of underlying mechanism of interventions change and could guide further refinement of the intervention to make it more precise, efficient and effective (Holmes et al., 2018). Therefore, future RCTs should include larger samples and several measurement time points to ensure sufficient power for mediation analyses to examine the resilience factors used in RESIST as possible mediators of the training's efficacy (Holmes et al., 2018). Fourth, the apparent benefit of the flexible usage of the web- and app-based components is an important implication from the study. Demonstrating dose-response effects in larger trials for each component would strengthen the argument that each component causes a beneficial effect (Holmes et al., 2018). Establishing an interaction between both dose-response effects would indicate that participants benefit more when using both components. Moreover, in future trials, the highly flexible usage should among other things be even more emphasized during the eCoaches' communication.

Finally, it seems promising to use qualitative methods to investigate the usage of metaphors in depth. Larger trials should also test the assumption that participants using a metaphor for the resilient self-image as compared to participants using a plain verbal description of their resilient self-image achieve a greater increase in positive emotions (Holmes et al., 2007).

4.6. Conclusions

To summarize, this feasibility study suggests that the theoretical framework of PASTOR and therapeutic methods of strengths-based CBT provide a solid basis for the newly developed resilience intervention RESIST. From a technological perspective, the flexible combination of web- and app-based components was well accepted and might have contributed to positive effects on mental health. Results indicated that employees benefitted from the intervention to a large extent. They reported a reduction in perceived stress as well as an increase in resilience-related outcomes. This could be a crucial finding for organizations, as pleased workers can be more productive and are less likely to leave the company (Meyers et al., 2013).

Funding

This work was supported by the Federal Ministry of Eductaion and Research BMBF [02L14A212]. This publication was funded by the German Research Foundation (DFG).

CRediT authorship contribution statement

Dirk Lehr and Dörte Behrendt contributed to the design of the study. Dörte Behrendt led the development of the intervention content, Dirk Lehr, Michèle Wessa, and Angela M. Kunzler contributed to the development of the intervention. Dörte Behrendt prepared the outcome analyses. Dörte Behrendt drafted the first proof of the paper and integrated coauthor comments from Dirk Lehr, Leif Boß, Sandy Hannibal, Michèle Wessa, and Angela M. Kunzler and edits. All the authors contributed to the further writing of the paper and approved the final paper.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

Supplementary material