Skip to main content
NCBI home page
Elsevier - PMC COVID-19 Collection logoLink to Elsevier - PMC COVID-19 Collection
. 2022 May 27;311:604–613. doi: 10.1016/j.jad.2022.05.118

Cognitive training using a mobile app as a coping tool against COVID-19 distress: A crossover randomized controlled trial

Burcin Akin-Sari a,⁎, Mujgan Inozu b, A Bikem Haciomeroglu c, Ezgi Trak b, Damla Tufan b, Guy Doron d
PMCID: PMC9137239  PMID: 35636516

Abstract

Background

The COVID-19 pandemic has been suggested to constitute a broad base stressor with severe mental health consequences. mHealth applications are accessible self-help tools that can be used to reduce psychological distress during the pandemic. This randomized controlled trial evaluated the effects of mobile-based cognitive training exercises on COVID-19 related distress and maladaptive cognitions.

Methods

Following initial screening (n = 924), participants scoring 1 standard deviations above the mean of the COVID-19 Distress Scale were randomized into two groups. Participants in the immediate-app group (iApp; n = 25) started using the application at baseline (T0) for 12 days (from T0 to T1). Participants in the delayed-app group (dApp; n = 22) started using the mobile application at T1 (crossover) and used it for the following 12 days (T1 to T2).

Results

Intention to treat analyses indicated that the iApp group exhibited lower COVID-19 distress, lower depression, fewer intolerance of uncertainty and obsessive beliefs than the dApp group at T1. In addition, using the app for 12 consecutive days was associated with large effect-size reductions (Cohen's d ranging from 0.81 to 2.35) in COVID-19 distress and related maladaptive cognitions in the iApp group (from T0 to T1) and the dApp group (from T1 to T2). Moreover, these reductions were maintained at the follow-up.

Limitations

This study was a crossover trial with a relatively limited sample size and mainly female participants.

Conclusion

Our findings underscore the usefulness of brief, low-intensity, portable interventions in alleviating the negative effects of the pandemic on mental health.

Keywords: COVID-19, Pandemic, Distress, Obsessive beliefs, Mobile apps

1. Introduction

The Coronavirus disease (COVID-19) emerged in China in December 2019 and had a globally devastating impact. Data from January 2022 suggest about 520 million confirmed cases of COVID-19, including more than 6 million deaths (WHO, 2022). Waves of new COVID-19 variants have governments across the world implementing rigid health safety practices including lockdowns, strict social distancing practices, and quarantine (Murphy et al., 2020).

The COVID-19 pandemic has been associated with significant challenges in the fields of education, health, and the economy (Bambra et al., 2020; Daniel, 2020). The social, health and economic challenges associated with the COVID-19 pandemic have, therefore, been suggested to constitute a broad base stressor with severe mental health consequences (Cao et al., 2020; Wang et al., 2020; Xiang et al., 2020; Zvolensky et al., 2020). Indeed, up to 13.6% of US adults reported serious psychological distress in April 2020 (McGinty et al., 2020). This is a significant increase from the 3.9% of adults reporting such distress in 2018. Similarly, the estimated depression rate in 2021 has increased seven-fold when compared to depression rates reported in 2017 (Bueno-Notivol et al., 2021). Consistent with this, a recent study has shown that people in 2020 report feeling five times more hopeless during the pandemic than in pre-pandemic periods (Twenge and Joiner, 2020).

Increased levels of COVID-19 related distress have been linked with persistent emotional distress (Vintila et al., 2022), higher rates of psychopathology (Megalakaki et al., 2021), greater mental healthcare utilization (Roy et al., 2020), decreased quality of life (Khodami et al., 2021), and increased interference in functioning (Gallagher et al., 2020) compared with the general population. According to CBT models, distress related to fears of infection, contamination and illness escalates when individuals catastrophically appraise common psychological (e.g., thoughts, images, urges) and physiological phenomena (e.g., physical sensations) (Cisler et al., 2007; Salkovskis and Warwick, 2001). Such catastrophic appraisals are based on pre-existing maladaptive beliefs such threat overestimation, intolerance for uncertainty, likelihood and awfulness of illness and catastrophizing of bodily sensations (Obsessive Compulsive Cognitions Working Group [OCCWG], 2005; Salkovskis and Warwick, 2001). More recently, additional maladaptive beliefs about illness (e.g., the belief that thinking in certain way can prevent or cause illness; Bailey and Wells, 2015) have also been implicated in health anxiety in the context of the COVID-19 pandemic (Hassoulas et al., 2021).

CBT interventions often target maladaptive beliefs using psychoeducation, cognitive restructuring techniques, exposure and response prevention, behavioral experiments, and cognitive bias modification (Abramowitz, 2006; Beck, 1991; Teachman et al., 2014). Using these therapeutic strategies, clinicians help individuals find alternative explanations for their disturbing cognitions, re-evaluate maladaptive appraisals and beliefs, and reduce behaviors that negatively affect their functionality (Mohammadi and Cummings, 2020; Sanderson et al., 2020). However, high cost, lack of available trained professionals and stigma, however, hinder many individuals from attending CBT (Price et al., 2014; Rees and Maclaine, 2015). Increased demand for psychological treatments and COVID-19 related limitation (e.g., lockdowns and quarantines) have further reduced availability of face-to-face psychological support and increased the need for alternative treatment delivery modalities (Bambra et al., 2020; Murphy et al., 2020; Naeem et al., 2020).

Mobile health (mHealth) apps have been suggested to help overcome existing treatment barriers by providing low cost, accessible, continuously available and anonymous CBT-based interventions to anyone owning a smartphone (e.g., Gagnon et al., 2016; Linardon et al., 2019). Findings suggest that mHealth apps help individuals increase their self-monitoring skills, tolerance for stress, emotional awareness, and cognitive reappraisal skills (Firth et al., 2017a, Firth et al., 2017b; Marley and Farooq, 2015). Use of mHealth apps has also been shown to be associated with reductions in anxiety, depression, self-harm and negative body image (e.g., Ben-Zeev et al., 2021; Burns et al., 2011; Chandrashekar, 2018; Firth et al., 2017a; Melia et al., 2020; Rodgers et al., 2018).

Although there are more than 300,000 mHealth apps with an estimated 10,000 focusing on mental health available, most mental health mHealth apps lack empirical support (Wasil et al., 2019). Fewer still have been shown to reduce COVID- 19 related distress (e.g., Gordon et al., 2021; Mira et al., 2020; Moulaei et al., 2021; Sun et al., 2021). One mobile mental health platform with relatively robust empirical support including 6 published randomized controlled trials (RCTs) is GGtude (Aboody et al., 2020; Akin-Sari et al., 2022; Ben-Zeev et al., 2021; Cerea et al., 2020, Cerea et al., 2021; Roncero et al., 2019).

GGtude is a CBT-based mobile application platform comprising distinct modules targeting maladaptive beliefs associated with various psychological difficulties (e.g., OCD, depression, low self-esteem, body image distress). The ‘GGcov’ module assessed in this study is included in an app named: ‘GG OCD, anxiety and depression’. This module was designed to challenge beliefs (e.g., likelihood and awfulness of illness) and meta-beliefs (e.g., belief that thinking in certain way can prevent or cause illness) associated with COVID-19 related distress. Users of this module progressively complete three levels a day. Each daily practice includes 3–4 min of training (see The COVID-19 intervention below).

Several elements of the GGtude platform have been theorized to be implicated in the reduction of users' maladaptive cognitions and associated symptoms (Aboody et al., 2020; Ben-Zeev et al., 2021; Giraldo-O'Meara and Doron, 2021; Roncero et al., 2018, Roncero et al., 2019). These include 1) psychoeducation to increase users' motivation and provide knowledge of basic CBT principles, 2) daily categorization exercises to promote users' awareness of their inner monologue, 3) repeated exposure to adaptive self-statements to increase cognitive availability of such self-statements, and 4) concurrent priming of maladaptive beliefs and exposure to unanticipated competing appraisals to accelerate adjustive reflective processing. Together, these daily exercises increase cognitive availability and accessibility of adaptive cognitions over availability of maladaptive ones.

The objective of the present study was to extend previous findings by evaluating the associations between short, daily mobile delivered CBT-based training and reduction in COVID-19 cognitions and distress. Although previous findings assessed the effectiveness of cognitive training exercises on various psychopathological symptoms (e.g., Ben-Zeev et al., 2021; Cerea et al., 2020; Roncero et al., 2019), the effectiveness of such exercises on COVID-19 related distress has yet to be assessed. An additional aim of the study was to assess the effectiveness of this intervention in a Turkish sample of individuals showing high levels of COVID-19 related distress. Although the various module of the GGtude platform are available in English, Spanish, Italian and Hebrew, the COVID-19 distress module was not available in Turkish nor was the effectiveness of this module assessed in a Turkish sample.

To achieve this, we conducted a randomized controlled trial (RCT) with crossover design and evaluated pre- to post- changes in levels of COVID-19-related distress and dysfunctional beliefs. We hypothesized that at T1 the immediate-use App group (iApp; who used the app immediately after the baseline measurement) would show a statistically significant decrease in COVID-19 related distress symptoms, obsessive beliefs, level of intolerance of uncertainty, depression, anxiety, and stress levels compared to the delayed-use App group (dApp) (see Fig. 2). These decreases were expected to be maintained at 12-day follow-up. We also expected users in the dApp group to show significant reductions in all measurements following crossover (between T1 and T2).

Fig. 2.

Fig. 2

Open in a new tab

Graphs of the outcome measures across T0, T1, and T2 for iApp and dApp groups.

Note. iApp: immediate-use App; dApp: delayed-use App; CDS: COVID-19 Distress Scale; DASS Stress: Depression, Anxiety, Stress Scale-21 Stress Scale; PHQ-9: Patient Health Questionnaire-9; IUS: Intolerance of Uncertainty Scale; OBQ-44: Obsessive Beliefs Questionnaire-44.

2. Method

2.1. Participants

We conducted an online screening study to identify individuals with high COVID-19 distress. The study was announced via university mailing lists to university students, employees, and alumni as well as and via the university's social media accounts (Facebook and Twitter) to community members. Nine hundred twenty-four individuals (710 females, M age = 25.27, SD age = 9.67, age range between 18 and 63 years old) participated in the initial screening study and completed the demographic information form and the COVID-19 Distress Scale (CDS; Trak et al., 2021). The mean CDS score for the screening study was 2.77 (SD = 0.75). Participants who reported having been diagnosed with a mental disorder and continued to receive treatment were excluded from the sample (n = 93). The researchers then contacted via telephone or e-mail 86 participants who scored at least 1 SD above the mean CDS score (see Fig. 1 ), starting with the participant showing the highest scores. Only the participants who could be reached, agree to participate, had a smart mobile phone, and access the internet were included in the study. Forty-seven participants met these inclusion criteria and were randomly assigned to iApp and dApp groups. The iApp group consisted of 25 participants, and the dApp group included 22 participants (see Fig. 1).

Fig. 1.

Fig. 1

Open in a new tab

CONSORT flow diagram of the participants through the trial.

2.2. Design

To evaluate the efficacy of the app in reducing COVID-19 distress and related psychological outcomes, we conducted a randomized controlled trial with a crossover design (see Fig. 2 ). The iApp group began to use the app at T0 and continued to use it for the following 12 days (until T1). The dApp group on the waiting list started to use the app at T1 and used it for the next 12 days (until T2). After obtaining informed consent form, both groups completed self-report questionnaires at T0, T1, and T2.

2.3. Measures

Participants completed the demographic information form at T0 and all other measures at T0, T1, and T2.

2.3.1. Demographic information form

The demographic information form contained items assessing gender, age, educational characteristics, marital status, physical and mental health condition, diagnosis with COVID-19, loss of loved ones due to COVID-19, and time spent on mobile devices daily.

2.3.2. COVID-19 Distress Scale (CDS; Trak and Inozu, 2022)

The CDS is a 14-item self-report measure assessing COVID-19 related distress. Items are rated on a 5-point Likert scale. Examples of scale items include: “I think that any minor health issue I experience is due to the coronavirus.”, “I believe that people in my immediate circle are very likely to become infected with the coronavirus.” or “I constantly read coronavirus-related content on the internet/social media.” Higher scores indicate greater pandemic-related distress. The CDS demonstrated robust psychometric properties in a community sample of adults. In a study with 548 individuals, exploratory factor analysis suggested a three-factor structure including anxiety, threat perception, and hopelessness related to COVID-19. In another study with 626 individuals, the 3-factor model obtained in the exploratory factor analysis was tested with confirmatory factor analysis. Results of the confirmatory factor analysis indicated a good fit to the data: χ2/df = 2.917, CFI = 0.94, NFI = 0.92, RFI = 0.90, IFI = 0.95, TLI = 0.93, RMSEA = 0.057 (90% confidence interval: 0.050–0.064), SRMR = 0.060. The CDS scores were positively associated with general distress, health anxiety, and obsessive-compulsive tendencies, while they were negatively related to resilience and positive affect. In addition, CDS scores significantly predicted general distress and health anxiety over and above other mental health measures. The CDS had a Cronbach's alpha score of 0.87 and test-retest reliability of 0.79 over two weeks (N = 249). Cronbach's alpha values were 0.90 for the screening study (N = 924), 0.79 at T0, 0.92 at T1 and 0.91 at T2 (N = 46).

2.3.3. Depression Anxiety Stress Scale-21 (DASS-21; Lovibond and Lovibond, 1995)

DASS-21 is a self-report measure consisting of three 7-item scales assessing depression, anxiety, and stress. Items are rated on a 4-point Likert scale, and higher scores indicate higher levels of distress. The scale demonstrated good validity and reliability in clinical and non-clinical samples (Lee et al., 2019). Research indicated that the Turkish adaptation of the scale has adequate validity and reliability (Yildirim et al., 2018). In the current study, only the Stress Scale was used. Cronbach's alpha values for the Stress Scale were 0.82, 0.79, and 0.78 at T0, T1 and T2 (N = 46), respectively.

2.3.4. Patient Health Questionnaire-9 (PHQ-9; Kroenke et al., 2001)

PHQ-9 is a self-report measure assessing depression. The scale has 9 items rated on a 4-points Likert scale, and higher scores indicate higher depressive symptoms. The original version and the Turkish adaptation of PHQ-9 demonstrated excellent psychometric properties (Gulec et al., 2012; Kroenke et al., 2001). In the present study, the scale had a Cronbach's alpha value of 0.82 at T0, 0.86 at T1, and 0.85 at T2 (N = 46).

2.3.5. Intolerance of Uncertainty Scale (IUS; Carleton et al., 2007)

IUS is a 12-item self-report measure assessing the intolerance regarding the uncertainty of the occurrence of adverse events. Items are rated on a 5-point Likert scale, and higher scores correspond to greater intolerance of uncertainty. IUS has good psychometric qualities (Carleton et al., 2007). IUS's Turkish adaptation also has adequate validity and reliability (Saricam et al., 2014). The Cronbach's alpha value for the scale was 0.91 at T0, 0.93 at T1, and 0.94 at T2 (N = 46) in the current study.

2.3.6. Obsessive Beliefs Questionnaire (OBQ-44; OCCWG, 2005)

OBQ is a 44-item self-report measure assessing beliefs and appraisals related to obsessive-compulsive disorder. Items are rated on a 7-point Likert scale. OBQ has three subscales: responsibility and overestimation of threat, perfectionism, and intolerance of uncertainty, importance, and control of thoughts. The original and Turkish adaptations of the scale demonstrated excellent validity and reliability (Obsessive Compulsive Cognitions Working Group, 2001, Obsessive Compulsive Cognitions Working Group, 2003; Yorulmaz and Gencoz, 2008). The Cronbach's alpha value for the total scale score was 0.95 at T0, 0.97 at T1, and 0.97 at T2 (N = 46) in the current study.

2.3.7. The COVID-19 stress reduction intervention (GGcov)

The COVID-19 stress reduction module of the GGtude platform (GGcov) consists of short training exercises intended to help people increase availability and accessibility of functional self-statements that facilitate adaptive interpretations of thoughts, emotions, and events associated with COVID-19 distress. Users first go through a tutorial consisting of short psychoeducation on the meaning and importance of one's inner monologue, the aim of the mobile application (broaden one's inner monologue), the rationale behind the exercises, and instructions for doing the exercises. After the psychoeducation, users learn to respond to statements that challenge maladaptive beliefs underlying COVID-19 related distress (e.g., intolerance of uncertainty, overestimation of threat, likelihood and awfulness of illness, inadequacy of medical services, catastrophizing of bodily sensations and the body's inability to cope with illness) by swiping the statements down (pulling them towards themselves). In contrast, users learn to reject and dissociate from statements that are consistent with COVID-19 related maladaptive beliefs by swiping the statements up (pushing them away from themselves). In this way, the app trains users to identify and challenge negative cognitions and maladaptive beliefs related to COVID-19 related distress and expands users' functional inner monologue by providing self-statements associated with adaptive attitudes.

Participants using GGcov were asked to complete three levels per day (about 4 min of training a day) before going to sleep. Push notifications reminded users to complete their daily training each day. Three levels on GGcov make a set dedicated to a specific belief. Each set of beliefs is preempted by a short rationale for challenging this belief. Next, several adaptive or maladaptive sentences appear on the screen one by one. For instance, a level dealing with tolerating uncertainty may include self-statements such as “Feeling uncertain is a natural part of life.” and “Uncertainty is unbearable.” A level dealing with the likelihood of becoming ill may include self-statements such as “I am as strong as others.” and “I am more likely than others to get an illness”. Each level completed is followed by a short memory quiz to increase user attention during the training session.

2.4. Procedure

All expressions in the application were translated into Turkish by five clinical psychologists who are fluent in English. The suitability of the app to the Turkish language was then assessed in a pilot study. The sample of the pilot consisted of 20 participants between the ages of 18–35 years. The participants were asked to report the incomprehensible sentences and possible translation errors they noticed during the 12-day use period for the application. A small compensation fee (50 Turkish Liras) was given to the participants for their time.

In the randomized controlled trial, iApp and dApp groups responded to an online questionnaire package including a demographic information form, CDS, PHQ-9, DASS-21 Stress Scale, IUS, and OBQ-44 (T0). Then participants were randomized to the iApp or dApp groups. Randomization was carried out in a 1:1 ratio and based on a prespecified computer-generated randomization list generated on Randomizer.org. Group assignment was then performed via email using the next available number on the randomization list. After the assessment at T0, researchers made a 15-min video interview with the iApp group participants and informed them about the GGcov app and the intervention. After using the GGcov for 12 days, users in the iApp group responded to all T1 measures. They were then asked to cease using the application for the next 12 days, while the dApp group started using GGcov. The dApp group also completed three levels a day for the next 12 days (until T2). Both groups completed the self-report measures for the third time at T2. Participants were compensated 50 Turkish Liras (approximately 7$) for their participation upon completing the study. The study was conducted in accordance with the Declaration of Helsinki and approved by the Ethical Commission of the [blinded for review purposes] University. The current trial was designed to CONSORT-EHEALTH checklist and registered.

2.5. Statistical analysis

We investigated group differences in socio-demographic variables and outcome measures (CDS, DASS-21 Stress Scale, PHQ-9, IUS and OBQ-44) with t-tests and calculated Pearson product moment correlations for study measures at T0. To prevent the potential biased effect of randomized allocation and missing responses, we used an intent-to-treat approach and implemented a multiple imputation strategy to replace missing values (Hollis and Campbell, 1999). Multiple imputation is considered a suitable method of handling missing data in repeated measure designs and has been shown to have utility with levels of missing data greater than that observed in this study (Vallejo et al., 2011). Specifically, we used Multivariate Imputation with Chained Equations (Van Buuren and Groothuis-Oudshoorn, 2011; [MICE]). Multiple imputation methods follow 3 steps: a) Imputing – repeating over several iterations (i) as opposed to a single imputation. b) Analyzing – after each iteration the dataset completed is analyzed, leading to a distribution of i statistics, 1 per dataset. c) Pooling – the i results are pooled into one estimate. Multiple imputation therefore also has the added benefit of examining the variance in estimates over iterations, reflecting the degree of uncertainty over which value to impute (Lall, 2016). All participants starting a trial were represented with data at each follow-up time point with the Multiple Imputation procedure currently regarded as best practice. We conducted a 2 (Group: iApp, dApp) × 2 (Time: T0 and T1) repeated measures multiple analysis of variance (MANOVA) for five dependent variables to examine the impact of GGcov on outcome measures. We conducted two repeated measures MANOVAs to examine the differences across three time points for each group separately. All statistical analyses were conducted with IBM SPSS.

3. Results

3.1. Preliminary analysis

We compared iApp and dApp groups in terms of gender, age, education (years), CDS, DASS Stress, PHQ-9, IUS and OBQ-44 scores at baseline. Groups did not significantly differ from each other in terms of gender, age, years of education, and self-report measures (see Table 1 ).

Table 1.

Comparisons between the iApp and dApp groups in sociodemographic variables and outcome measures at baseline.

iApp group
 dApp group
 t(54)/x2 p
M (SD) M (SD)
Gender
(Female; Male)		 21 (77.8%) Female;
6 (22.2%) Male		 22 (81.5%) Female;
5 (18.5%) Male
Age 22.22 (7.11) 24.74 (8.51) −1.18 0.17
Education (years) 11.74 (1.58) 11.81 (2.24) −0.14 0.20
CDS 3.98 (0.50) 3.89 (0.46) 0.63 0.44
DASS Stress 1.56 (0.78) 1.70 (0.84) −0.65 0.42
PHQ-9 1.51 (0.63) 1.40 (0.61) 0.68 0.70
IUS 3.65 (0.80) 3.31 (0.79) 0.92 0.92
OBQ-44 4.08 (0.85) 4.08 (1.03) 0.29 0.29
Open in a new tab

Note. CDS: COVID-19 Distress Scale; DASS Stress: Depression, Anxiety, Stress Scale-21 Stress Scale; PHQ-9: Patient Health Questionnaire; IUS: Intolerance of Uncertainty Scale; OBQ-44: Obsessive Beliefs Questionnaire-44.

Pearson product-moment correlation coefficients indicated that individuals with higher levels of COVID-19 related distress also reported higher levels of stress, depression symptoms, intolerance of uncertainty and obsessive beliefs (Table 2 ).

Table 2.

Means, standard deviations and intercorrelations between study variables at T0 (N = 47).

Variables CDS DASS Stress PHQ-9 IUS M SD
CDS 3.94⁎ 0.48
DASS Stress 0.42⁎⁎ 1.63⁎⁎⁎ 0.81
PHQ-9 0.42⁎⁎ 0.67⁎⁎ 1.46 0.62
IUS 0.54⁎⁎ 0.52⁎⁎ 0.42⁎⁎ 3.48 0.81
OBQ-44 0.58⁎⁎ 0.38⁎⁎ 0.35⁎⁎ 0.53⁎⁎ 4.08 0.94
Open in a new tab

Note. CDS: COVID-19 Distress Scale; DASS Stress: Depression, Anxiety, Stress Scale-21 Stress Scale; PHQ-9: Patient Health Questionnaire; IUS: Intolerance of Uncertainty Scale; OBQ-44: Obsessive Beliefs Questionnaire-44.

⁎

p < .05.

⁎⁎

p < .01.

⁎⁎⁎

p < .001.

3.2. Between-group differences (iApp group versus dApp group)

Since the correlations between the study measures were higher than 0.30, to decrease the probability of type 1 error, we examined the effectiveness of GGcov in reducing COVID-19 distress, stress, depression, intolerance of uncertainty and obsessive beliefs scores with a 2 (Group: iApp, dApp) × 2 (Time: T0 and T1) repeated-measures MANOVA where group was treated as a between-subject factor and time was treated as a within-subject factor. We used a Bonferroni-corrected significance level of p < .01 (0.05/5 = 0.01).

The main effect of time (Wilks' λ = 0.45, F [5, 48] = 11.85, p = .00, Cohen's d = 2.21) and the interaction of group × time (Wilks' λ = 0.66, F [5, 48] = 4.97, p = .00, Cohen's d = 1.44) were statistically significant. The follow-up ANOVAs indicated a significant group × time interaction effect for CDS, PHQ-9, IUS and OBQ-44 (see Table 3 ). These results indicated that participants in the iApp group exhibited lower COVID-19 distress, lower depression symptoms, fewer intolerance of uncertainty beliefs and obsessive beliefs than the participants in dApp group on the second assessment at T1 (see Fig. 2).

Table 3.

Means, standard deviations and comparisons across T0 and T1 for iApp and dApp groups.

T0
 T1
 Time
 Group
 Time × Group
M (SD)
 M (SD)
 F(1,52) p d F(1,52) p d F(1,52) p d
iApp dApp iApp dApp
CDS 3.98 (0.50) 3.89 (0.46) 3.11 (0.74) 3.74 (0.70) 38.47 0.000 1.74 3.51 0.067 – 18.74 0.000 1.19
DASS Stress 1.56 (0.78) 1.70 (0.84) 1.40 (0.73) 1.86 (0.71) 0.00 0.977 – 2.54 0.117 – 2.87 0.096 –
PHQ-9 1.51 (0.63) 1.40 (0.61) 1.01 (0.60) 1.41 (0.64) 9.77 0.003 0.87 0.93 0.340 – 10.77 0.002 0.91
IUS 3.65 (0.80) 3.31 (0.79) 2.97 (0.86) 3.20 (0.79) 24.44 0.000 1.37 0.08 0.782 – 12.47 0.001 0.97
OBQ-44 4.08 (0.85) 4.08 (1.03) 3.60 (1.11) 4.16 (1.21) 3.98 0.051 – 1.06 0.309 – 8.08 0.006 0.77
Open in a new tab

Note. CDS: COVID-19 Distress Scale; DASS Stress: Depression, Anxiety, Stress Scale-21 Stress Scale; PHQ-9: Patient Health Questionnaire; IUS: Intolerance of Uncertainty Scale; OBQ-44: Obsessive Beliefs Questionnaire-44.

We conducted a second 2 (Group: iApp, dApp) × 2 (Time: T1 and T2) repeated-measures MANOVA where group was treated as a between-subject factor and time was treated as a within-subject factor to examine the effectiveness of GGcov in reducing COVID-19 distress, stress, depression, intolerance of uncertainty and obsessive beliefs in dApp group after using the mobile app. We used a Bonferroni-corrected significance level of p < .01 (0.05/5 = 0.01).

The main effect of time (Wilks' λ = 0.78, F [5, 48] = 2.71, p = .03, Cohen's d = 1.06) and the interaction of group × time (Wilks' λ = 0.66, F [5, 48] = 4.91, p = .00, Cohen's d = 1.44) were statistically significant. The follow-up ANOVAs indicated a significant group × time interaction effect for CDS, DASS Stress Scale, PHQ-9, and OBQ-44 (see Table 4 ). These results indicated that while participants in the dApp group exhibited higher COVID-19 distress, stress level, depression symptoms, and obsessive beliefs than the participants in iApp group at T1, two groups did not significantly differ in terms of these measures at T2 (see Fig. 2).

Table 4.

Means, standard deviations and comparisons across T1 and T2 for iApp and dApp groups.

T1
 T2
 Time
 Group
 Time × Group
M (SD)
 M (SD)
 F(1,52) p d F(1,52) p d F(1,52) p d
iApp dApp iApp dApp
CDS 3.11 (0.74) 3.74 (0.70) 3.07 (0.51) 3.05 (0.72) 12.22 0.001 0.97 4.06 0.049 0.55 9.43 0.003 0.84
DASS Stress 1.40 (0.73) 1.86 (0.71) 1.51 (0.74) 1.21 (0.80) 6.66 0.013 0.70 0.19 0.662 – 13.35 0.001 1.00
PHQ-9 1.01 (0.60) 1.41 (0.64) 1.17 (0.57) 1.03 (0.60) 1.50 0.227 – 0.95 0.335 – 9.14 0.004 0.84
IUS 2.97 (0.86) 3.20 (0.79) 2.89 (0.87) 2.66 (0.66) 5.47 0.023 0.67 0.00 0.986 – 2.91 0.094 –
OBQ-44 3.60 (1.11) 4.16 (1.21) 3.78 (0.97) 3.19 (0.98) 8.20 0.006 0.81 0.00 0.958 – 16.92 0.000 1.15
Open in a new tab

Note. CDS: COVID-19 Distress Scale; DASS Stress: Depression, Anxiety, Stress Scale-21 Stress Scale; PHQ-9: Patient Health Questionnaire; IUS: Intolerance of Uncertainty Scale; OBQ-44: Obsessive Beliefs Questionnaire-44.

3.3. iApp group within group effects and 12 days follow-up effect

In the iApp group, we expected pre-post reduction in COVID-19 distress, stress, depression symptoms, intolerance of uncertainty and obsessive beliefs as well as retention of these effects in the follow-up period. Therefore, we examined pre-to-final changes with a repeated-measures MANOVA with Bonferroni corrections between T0, T1 and T2. Mauchly's test of sphericity indicated that the assumption of sphericity was met for all measures. CDS, PHQ-9, IUS and OBQ-44 scores were significantly decreased from T0 to T1. In addition, there were no significant changes from T1 to T2 for CDS, PHQ-9, IUS and OBQ scores (see Table 5 ). These results indicated that the iApp group exhibited lower COVID-19 distress, lower depression symptoms, fewer intolerance of uncertainty beliefs and obsessive beliefs at T1 than T0 and the differences for COVID-19 distress, depression, intolerance of uncertainty, and obsessive belief levels were maintained in T2.

Table 5.

Means, standard deviations and comparisons across three time points for iApp and dApp group.

T0
M (SD)		 T1
M (SD)		 T2
M (SD)		 F1,38 p Cohen's d Post-hoc
CDS iApp 3.98 (0.50) 3.11 (0.74) 3.07 (0.51) 36.26 0.00 2.35 T0 vs T1 = p = .00
T0 vs T2 = p = .00
T1 vs T2 = p = 1.00
dApp 3.89 (0.46) 3.74 (0.70) 3.05 (0.72) 18.88 0.00 1.70 T0 vs T1 = p = .44
T0 vs T2 = p = .00
T1 vs T2 = p = .00
DASS Stress iApp 1.56 (0.78) 1.40 (0.73) 1.51 (0.74) 0.73 0.49 – T0 vs T1 = p = .53
T0 vs T2 = p = 1.00
T1 vs T2 = p = 1.00
dApp 1.70 (0.84) 1.86 (0.71) 1.21 (0.80) 8.47 0.00 1.15 T0 vs T1 = p = .90
T0 vs T2 = p = .06
T1 vs T2 = p = .00
PHQ-9 iApp 1.51 (0.63) 1.01 (0.60) 1.17 (0.57) 9.80 0.00 1.22 T0 vs T1 = p = .00
T0 vs T2 = p = .02
T1 vs T2 = p = .60
dApp 1.40 (0.61) 1.41 (0.64) 1.03 (0.60) 5.07 0.01 0.87 T0 vs T1 = p = 1.00
T0 vs T2 = p = .85
T1 vs T2 = p = .02
IUS iApp 3.65 (0.80) 2.97 (0.86) 2.89 (0.87) 17.49 0.00 1.63 T0 vs T1 = p = .00
T0 vs T2 = p = .00
T1 vs T2 = p = 1.00
dApp 3.31 (0.79) 3.20 (0.79) 2.66 (0.66) 8.19 0.00 1.12 T0 vs T1 = p = 1.00
T0 vs T2 = p = .00
T1 vs T2 = p = .05
OBQ-44 iApp 4.08 (0.85) 3.60 (1.11) 3.78 (0.97) 4.62 0.01 0.84 T0 vs T1 = p = .02
T0 vs T2 = p = .11
T1 vs T2 = p = 1.00
dApp 4.08 (1.03) 4.16 (1.21) 3.19 (0.98) 16.23 0.00 1.57 T0 vs T1 = p = 1.00
T0 vs T2 = p = .00
T1 vs T2 = p = .00
Open in a new tab

Note. CDS: COVID-19 Distress Scale; DASS: Depression, Anxiety, Stress Scale; PHQ-9: Patient Health Questionnaire; IUS: Intolerance of Uncertainty Scale; OBQ-44: Obsessive Beliefs Questionnaire-4.

3.4. dApp group within-group effects

In the dApp group, we expected that crossover (i.e. use of the app) would be associated with a significant decrease in COVID-19 distress, stress, health anxiety symptoms, intolerance of uncertainty and obsessive beliefs. Thus, we conducted a repeated-measures MANOVA with Bonferroni corrections between T0, T1 and T2. Mauchly's test of sphericity revealed that the assumption of sphericity had been violated for CDS (χ 2 [2] = 7.62, p = .02), IUS (χ 2 [2] = 10.43, p = .01) and OBQ-44 (χ 2 [2] = 10.23, p = .01). Therefore, we reported Greenhouse-Geisser corrected values for these variables. As expected there were no significant differences between T0 and T1 in related measures, within-group differences between T1 and T2 following the crossover revealed significant reductions in the CDS, DASS-21 Stress Scale, PHQ-9 and OBQ-44 scores (see Table 5). These findings suggest that while COVID-19 distress, level of stress, depression symptoms, and obsessive beliefs of dApp group did not change significantly from T0 to T1, they exhibited significantly lower COVID-19 distress, lower stress, lower depression symptoms, and fewer obsessive beliefs at T2 than T1.

4. Discussion

The widespread psychological distress in response to the pandemic has led mental health professionals to seek reliable information resources and effective interventions to reduce COVID-19 related distress (Kondylakis et al., 2020; O'Donnell et al., 2020; Reyes, 2020). Several advantages of mHealth apps including continuous availability, wide reach and low cost have made mobile delivered interventions a viable option. Building on previous findings supporting the GGtude platform for a variety psychological difficulty (e.g., Aboody et al., 2020; Cerea et al., 2020, Cerea et al., 2021; Roncero et al., 2019), in this study we evaluated the effectiveness of the COVID-19 stress reduction module of the GGtude platform (GGcov). This is the first RCT study to assess the efficacy of GGcov in a Turkish cohort showing heightened COVID-19 related distress.

Consistent with previous findings, we found that using GGcov was associated with reductions in targeted symptoms and cognitions relative to our delayed use comparison group. More specifically, our findings suggest large effect-size decreases in measures of COVID-19 distress, depression, anxiety and stress symptoms in the iApp group (between T0 and T1) that were replicated in the dApp group (following the crossover between T1 and T2).

Using GGcov was also associated with significant large effect-size reduction in COVID-19 related beliefs such as intolerance of uncertainty evaluated with IUS, and overestimation of threat and inflated responsibility measured by OBQ-44 in the iApp group (T1 to T2) and following the crossover in the dApp group (T1 to T2). Most of these reductions were maintained at follow-up (T1 to T2). Unexpectedly, following Bonferroni correction the iApp group did not show statistically significant reductions in stress scores compared to the dApp group between T0 and T1. This may have been the result of our limited sample size. Indeed, within-group analyses of the iApp and the dApp group showed large effect size reductions following GGcov use.

Pending further replication, our findings associating the GGcov use with reductions in COVID-19 related cognitions and symptoms support the proposal that targeting maladaptive cognitions associated with OCD and health anxiety may significantly reduce COVID-19 related distress. GGcov targets beliefs such intolerance of uncertainty, catastrophizing of illness and over-estimation of threat. Reducing maladaptive cognitions and underlying beliefs may have decreased catastrophic interpretations of daily events thereby decreasing COVID-19 related distress. Future studies may benefit from assessing the mediating role of obsessive beliefs in the reduction of such symptoms.

Although the findings from this randomized controlled study are encouraging, several limitations of the current research should be noted. Our sample size was limited, and our participants were mostly young women. Although being young and a female have been identified as risk factors for psychological distress during the pandemic (e.g., Horesh et al., 2020; Xiong et al., 2020), our sample may not be representative broader population. In addition, this study did not include an active control group. Future studies may benefit from including a comparison group using another app or using a different module of the same platform.

A relatively novel approach towards cognitive modification is called Approach-Avoidance Training (AAT). During AAT, people approach some stimuli with joystick (e.g., by pulling a joystick towards the body in response to low calorie food) and avoid from high calorie food by pushing a joystick away from the body. There are studies showing that such a motor-response training successfully modifies maladaptive cognitive mechanism underlying problematic behaviors (e.g. Mathew, 2019; Stice et al., 2016). Although the GGcov application seems to benefit from AAT principles as it contains motor responses that include pulling the statements towards themselves or pushing the statements away from themselves, there is no study comparing the effectiveness of GGcov and other mobile applications using the AAT technique. It is recommended to carry out future studies on different motor-response training.

Our study included a short follow up period. Although, previous studies have shown the effects of GGtude platform interventions remain for at least one month (e.g., Aboody et al., 2020; Ben-Zeev et al., 2021), our results were assessed at 12 days. Future studies would benefit assessing the effects of such intervention over a longer follow up periods. In addition, information regarding COVID-19 infection, vaccination, and pandemic related events such as job loss was not collected. Although our sample was randomized and the likelihood of such occurrences within the period of our study is relatively low, such factors may have influenced our outcome measures. Future studies undertaken during the COVID-19 crisis may benefit from collecting such information and assessing its impact on intervention outcomes.

4.1. Conclusion

According to the World Health Organization, Turkey has the lowest number of mental health specialists in the European Region per capita. While the average number of psychiatrists per 100,000 people in 28 countries in the European Union is 16.8, there are around 3.8 psychiatrists per 100,000 people in Turkey (Songur et al., 2017). During the COVID-19 pandemic, an overburdened health system together with imposed social distancing, fear of infection and economic difficulties have reduced accessibility to mental health treatment in Turkey and across the world. Online and mobile applications-based self-help interventions may help overcome these barriers and provide a more accessible, effective and low-cost alternative. Moreover, similar interventions have been found to increase resilience to mental health difficulties. mHealth interventions, therefore, may also be used as resilience promoting tools.

CRediT authorship contribution statement

Burcin Akin-Sari: Conceptualization, Methodology, Data curation, Writing – original draft, Writing – review & editing. Mujgan Inozu: Conceptualization, Methodology, Data curation, Writing – original draft, Writing – review & editing. A. Bikem Haciomeroglu: Conceptualization, Methodology, Data curation, Writing – original draft, Writing – review & editing. Ezgi Trak: Data collection, Data curation, Writing – original draft, Writing – review & editing. Damla Tufan: Data collection, Data curation, Writing – original draft, Writing – review & editing. Guy Doron: Conceptualization, Writing – review & editing.

Declaration of competing interest

Burcin Akin-Sari declares that she has no conflict of interest.

Mujgan Inozu declares that she has no conflict of interest.

A. Bikem Haciomeroglu declares that she has no conflict of interest.

Ezgi Trak declares that she has no conflict of interest.

Damla Tufan declares that she has no conflict of interest.

Guy Doron is one of the authors of the paper and a co-developer of GGcov. Guy Doron is also a co-founder of GGtude Ltd. GGcov is the subject of this evaluation and therefore has financial interest to GGtude Ltd.

Acknowledgments

Acknowledgements

We would like to thank Burcu Yıldırım, Şükriye Acar, Yasemin Erol, Güzide Memişçi and Caner Tülek for data collection and data entry, without whom this project would not have been possible.

We acknowledge funding support from The Scientific and Technological Research Council of Turkey (Project Number: 120K364) to cover the research project expenses.

Role of funding organizations

This work was supported by The Scientific and Technological Research Council of Turkey, Project Number: 120K364. The funders had no role in study design, data collection, data analysis and the preparation of the manuscript.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all participants included in the study.

Animal rights

No animal studies were carried out by the authors for this research.

References

  1. Aboody D., Siev J., Doron G. Building resilience to body image triggers using brief cognitive training on a mobile application: a randomized controlled trial. Behav. Res. Ther. 2020;134 doi: 10.1016/j.brat.2020.103723. [DOI] [PubMed] [Google Scholar]
  2. Abramowitz J.S. The psychological treatment of obsessive-compulsive disorder. Can. J. Psychiatry. 2006;51(7):407–416. doi: 10.1177/070674370605100702. [DOI] [PubMed] [Google Scholar]
  3. Akin-Sari B., Inozu M., Haciomeroglu A.B., Cekci B.C., Uzumcu E., Doron G. Cognitive Training via a mobile application to reduce obsessive-compulsive-related distress and cognitions during the COVID-19 outbreaks: a randomized controlled trial using a subclinical cohort. Behav. Ther. 2022 doi: 10.1016/j.beth.2021.12.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bailey R., Wells A. Metacognitive beliefs moderate the relationship between catastrophic misinterpretation and health anxiety. J. Anxiety Disord. 2015;34:8–14. doi: 10.1016/j.janxdis.2015.05.005. [DOI] [PubMed] [Google Scholar]
  5. Bambra C., Riordan R., Ford J., Matthews F. The COVID-19 pandemic and health inequalities. J. Epidemiol. Community Health. 2020;74(11):964–968. doi: 10.1136/jech-2020-2144. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Beck A.T. Cognitive therapy as the integrative therapy. J. Psychother. Integr. 1991;1(3):191–198. doi: 10.1037/h0101233. [DOI] [Google Scholar]
  7. Ben-Zeev D., Chander A., Tauscher J., Buck B., Nepal S., Campbell A., Doron G. A smartphone intervention for people with serious mental illness: fully remote randomized controlled trial of CORE. J. Med. Internet Res. 2021;23(11) doi: 10.2196/29201. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bueno-Notivol J., Gracia-García P., Olaya B., Lasheras I., López-Antón R., Santabárbara J. Prevalence of depression during the COVID-19 outbreak: a meta-analysis of community-based studies. Int. J. Clin. Health Psychol. 2021;21(1) doi: 10.1016/j.ijchp.2020.07.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Burns M.N., Begale M., Duffecy J., Gergle D., Karr C.J., Giangrande E., Mohr D.C. Harnessing context sensing to develop a mobile intervention for depression. J. Med. Internet Res. 2011;13(3) doi: 10.2196/jmir.1838. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cao W., Fang Z., Hou G., Han M., Xu X., Dong J., Zheng J. The psychological impact of the COVID-19 epidemic on college students in China. Psychiatry Res. 2020;287 doi: 10.1016/j.psychres.2020.112934. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Carleton R.N., Norton M.A., Asmundson G.J. Fearing the unknown: a short version of the Intolerance of Uncertainty Scale. J. Anxiety Disord. 2007;21(1):105–117. doi: 10.1016/j.janxdis.2006.03.014. [DOI] [PubMed] [Google Scholar]
  12. Cerea S., Ghisi M., Bottesi G., Carraro E., Broggio D., Doron G. Reaching reliable change using short, daily, cognitive training exercises delivered on a mobile application: the case of relationship obsessive compulsive disorder (ROCD) symptoms and cognitions in a subclinical cohort. J. Affect. Disord. 2020;276:775–787. doi: 10.1016/j.jad.2020.07.043. [DOI] [PubMed] [Google Scholar]
  13. Cerea S., Ghisi M., Bottesi G., Manoli T., Carraro E., Doron G. Cognitive behavioral training using a mobile application reduces body image related symptoms in high risk female university students: a randomized controlled study. Behav. Ther. 2021;52(1):170–182. doi: 10.1016/j.beth.2020.04.002. [DOI] [PubMed] [Google Scholar]
  14. Chandrashekar P. Do mental health mobile apps work: evidence and recommendations for designing high-efficacy mental health mobile apps. MHealth. 2018;4(6) doi: 10.21037/mhealth.2018.03.02. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Cisler J.M., Reardon J.M., Williams N.L., Lohr J.M. Anxiety sensitivity and disgust sensitivity interact to predict contamination fears. Pers. Individ. Differ. 2007;42(6):935–946. doi: 10.1016/j.paid.2006.09.004. [DOI] [Google Scholar]
  16. Daniel J. Education and the COVID-19 pandemic. Prospects. 2020;49(1):91–96. doi: 10.1007/s11125-020-09464-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Firth J., Torous J., Nicholas J., Carney R., Pratap A., Rosenbaum S., Sarris J. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry. 2017;16(3):287–298. doi: 10.1002/wps.20472. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Firth J., Torous J., Nicholas J., Carney R., Rosenbaum S., Sarris J. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J. Affect. Disord. 2017;218:15–22. doi: 10.1016/j.jad.2017.04.046. [DOI] [PubMed] [Google Scholar]
  19. Gagnon M.P., Ngangue P., Payne-Gagnon J., Desmartis M. M-health adoption by healthcare professionals: a systematic review. J. Am. Med. Inform. Assoc. 2016;23(1):212–220. doi: 10.1093/jamia/ocv052. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Gallagher M.W., Zvolensky M.J., Long L.J., Rogers A.H., Garey L. The impact of Covid-19 experiences and associated stress on anxiety, depression, and functional impairment in american adults. Cogn. Ther. Res. 2020;44(6):1043–1051. doi: 10.1007/s10608-020-10143-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Giraldo-O'Meara M., Doron G. Can self-esteem be improved using short daily training on mobile applications? Examining real world data of GG Self-esteem users. Clin. Psychol. 2021 doi: 10.1080/13284207.2021.1923126. [DOI] [Google Scholar]
  22. Gordon J.S., Sbarra D., Armin J., Pace T.W., Gniady C., Barraza Y. Use of a guided imagery mobile app (See Me Serene) to reduce COVID-19–related stress: pilot feasibility study. JMIR Form. Res. 2021;5(10) doi: 10.2196/32353. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Gulec M.Y., Gulec H., Simsek G., Turhan M., Sunbul E.A. Psychometric properties of the Turkish version of the patient health questionnaire–somatic, anxiety, and depressive symptoms. Compr. Psychiatry. 2012;53(5):623–629. doi: 10.1016/j.comppsych.2011.08.002. [DOI] [PubMed] [Google Scholar]
  24. Hassoulas A., Umla-Runge K., Zahid A., Adams O., Green M., Hassoulas A., Panayiotou E. Investigating the association between obsessive-compulsive disorder symptom subtypes and health anxiety as impacted by the COVID-19 pandemic: a cross-sectional study. Psychol. Rep. 2021;1–22 doi: 10.1177/00332941211040437. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Hollis S., Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ. 1999;319(7211):670–674. doi: 10.1136/bmj.319.7211.670. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Horesh D., Kapel Lev-Ari R., Hasson-Ohayon I. Risk factors for psychological distress during the COVID-19 pandemic in Israel: loneliness, age, gender, and health status play an important role. Br. J. Health Psychol. 2020;25(4):925–933. doi: 10.1111/bjhp.12455. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Khodami M.A., Seif M.H., Koochakzadeh R.S., Fathi R., Kaur H. Perceived stress, emotion regulation and quality of life during the COVID-19 outbreak: a multi-cultural online survey. Ann. Med. Psychol. 2021 doi: 10.1016/j.amp.2021.02.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Kondylakis H., Katehakis D.G., Kouroubali A., Logothetidis F., Triantafyllidis A., Kalamaras I., Votis K., Tzovaras D. COVID-19 mobile apps: a systematic review of the literature. J. Med. Internet Res. 2020;22(12) doi: 10.2196/23170. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Kroenke K., Spitzer R.L., Williams J.B. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 2001;16(9):606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Lall R. How multiple imputation makes a difference. Polit. Anal. 2016;24(4):414–433. doi: 10.1093/pan/mpw020. [DOI] [Google Scholar]
  31. Lee J., Lee E.H., Moon S.H. Systematic review of the measurement properties of the depression anxiety stress scales–21 by applying updated COSMIN methodology. Qual. Life Res. 2019;28(9):2325–2339. doi: 10.1007/s11136-019-02177-x. [DOI] [PubMed] [Google Scholar]
  32. Linardon J., Cuijpers P., Carlbring P., Messer M., Fuller-Tyszkiewicz M. The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry. 2019;18(3):325–336. doi: 10.1002/wps.20673. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Lovibond P.F., Lovibond S.H. The structure of negative emotional states: comparison of the depression anxiety stress scales (DASS) with the beck depression and anxiety inventories. Behav. Res. Ther. 1995;33(3):335–343. doi: 10.1016/0005-7967(94)00075-U. [DOI] [PubMed] [Google Scholar]
  34. Marley J., Farooq S. Mobile telephone apps in mental health practice: uses, opportunities and challenges. BJPsych Bull. 2015;39(6):288–290. doi: 10.1192/pb.bp.114.050005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Mathew A.S. Vol. 2224. 2019. Evaluating the Effect of Approach-avoidance Training on Action Tendencies for Individuals with Skin Picking Disorder.https://dc.uwm.edu/etd/2224 Theses and Dissertations. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. McGinty E.E., Presskreischer R., Anderson K.E., Han H., Barry C.L. Psychological distress and COVID-19–related stressors reported in a longitudinal cohort of US adults in April and July 2020. JAMA. 2020;324(24):2555–2557. doi: 10.1001/jama.2020.21231. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Megalakaki O., Kokou-Kpolou C.K., Vaudé J., Park S., Iorfa S.K., Cénat J.M., Derivois D. Does peritraumatic distress predict PTSD, depression and anxiety symptoms during and after COVID-19 lockdown in France? A prospective longitudinal study. J. Psychiatr. Res. 2021;137:81–88. doi: 10.1016/j.jpsychires.2021.02.035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Melia R., Francis K., Hickey E., Bogue J., Duggan J., O'Sullivan M., Young K. Mobile health technology interventions for suicide prevention: systematic review. JMIR Mhealth Uhealth. 2020;8(1) doi: 10.2196/12516. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Mira J.J., Vicente M.A., Lopez-Pineda A., Carrillo I., Guilabert M., Fernández C., Pérez-Jover V., Delgado J.M., Pérez-Pérez P., Vargas A.C., Astier-Peña M.P., Martínes-García O.B., Marco-Gómez B., Bouzán C.A. Preventing and addressing the stress reactions of health care workers caring for patients with COVID-19: development of a digital platform (Be+ Against COVID) JMIR Mhealth Uhealth. 2020;8(10) doi: 10.2196/21692. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Mohammadi A., Cummings J. How CBT can be protective against anxiety disorders during COVID-19 era. Iran. J. Psychiatry Behav. Sci. 2020;14(3) doi: 10.5812/ijpbs.106601. [DOI] [Google Scholar]
  41. Moulaei K., Sheikhtaheri A., Ghafaripour Z., Bahaadinbeigy K. The development and usability assessment of an mHealth application to encourage self-care in pregnant women against COVID-19. J. Healthc. Eng. 2021 doi: 10.1155/2021/9968451. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Murphy R., Calugi S., Cooper Z., Dalle Grave R. Challenges and opportunities for enhanced cognitive behaviour therapy (CBT-E) in light of COVID-19. Cogn. Behav. Ther. 2020;13:E14. doi: 10.1017/S1754470X20000161. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Naeem F., Irfan M., Javed A. Coping with covid-19: urgent need for building resilience through cognitive behaviour theraphy. Khyber Med. Univ. J. 2020;12(1):1–3. doi: 10.35845/kmj.2020.20194. [DOI] [Google Scholar]
  44. O’Donnell K.T., Dunbar M., Speelman D.L. Effectiveness of using a meditation app in reducing anxiety and improving well-being during the COVID-19 pandemic: a structured summary of a study protocol for a randomized controlled trial. Trials. 2020;21(1):1–2. doi: 10.1186/s13063-020-04935-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Obsessive Compulsive Cognitions Working Group Development and initial validation of the obsessive beliefs questionnaire and the interpretation of intrusions inventory. Behav. Res. Ther. 2001;39(8):987–1006. doi: 10.1016/S00057967(00)00085-1. [DOI] [PubMed] [Google Scholar]
  46. Obsessive Compulsive Cognitions Working Group Psychometric validation of the obsessive beliefs questionnaire and the interpretation of intrusions inventory: part I. Behav. Res. Ther. 2003;41(8):863–878. doi: 10.1016/S0005-7967(02)00099-2. [DOI] [PubMed] [Google Scholar]
  47. Obsessive Compulsive Cognitions Working Group Psychometric validation of the obsessive belief questionnaire and interpretation of intrusions inventory-part 2: factor analyses and testing of a brief version. Behav. Res. Ther. 2005;43(11):1527–1542. doi: 10.1016/j.brat.2004.07.010. [DOI] [PubMed] [Google Scholar]
  48. Price M., Yuen E.K., Goetter E.M., Herbert J.D., Forman E.M., Acierno R., Ruggiero K.J. mHealth: a mechanism to deliver more accessible, more effective mental health care. Clin. Psychol. Psychother. 2014;21(5):427–436. doi: 10.1002/cpp.1855. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Rees C.S., Maclaine E. A systematic review of videoconference-delivered psychological treatment for anxiety disorders. Aust. Psychol. 2015;50(4):259–264. doi: 10.1111/ap.12122. [DOI] [Google Scholar]
  50. Reyes A.T. A mindfulness mobile app for traumatized COVID-19 healthcare workers and recovered patients: a response to “The use of digital applications and COVID-19”. Community Ment. Health J. 2020;56(7):1204–1205. doi: 10.1007/s10597-020-00690-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Rodgers R.F., Donovan E., Cousineau T., Yates K., McGowan K., Cook E., Lowy A.S., Franko D.L. BodiMojo: efficacy of a mobile-based intervention in improving body image and self-compassion among adolescents. J. Youth Adolesc. 2018;47(7):1363–1372. doi: 10.1007/s10964-017-0804-3. [DOI] [PubMed] [Google Scholar]
  52. Roncero M., Belloch A., Doron G. A novel approach to challenging OCD related beliefs using a mobile-app: an exploratory study. J. Behav. Ther. Exp. Psychiatry. 2018;59:157–160. doi: 10.1016/j.jbtep.2018.01.008. [DOI] [PubMed] [Google Scholar]
  53. Roncero M., Belloch A., Doron G. Can brief, daily training using a mobile app help change maladaptive beliefs? Crossover randomized controlled trial. JMIR Mhealth Uhealth. 2019;7(2) doi: 10.2196/11443. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Roy D., Tripathy S., Kar S.K., Sharma N., Verma S.K., Kaushal V. Study of knowledge, attitude, anxiety & perceived mental healthcare need in Indian population during COVID-19 pandemic. Asian J. Psychiatr. 2020;51 doi: 10.1016/j.ajp.2020.102083. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Salkovskis P.M., Warwick H.M.C. In: Hypochondriasis: Modern Perspectives on an Ancient Malady. Starcevic V., Lipsitt D.R., editors. Oxford University Press; 2001. Meaning, misinterpretations, and medicine: a cognitive-behavioral approach to understanding health anxiety and hypochondriasis; pp. 202–222. [Google Scholar]
  56. Sanderson W.C., Arunagiri V., Funk A.P., Ginsburg K.L., Krychiw J.K., Limowski A.R., Olesnycky O.S., Stout Z. The nature and treatment of pandemic-related psychological distress. J. Contemp. Psychother. 2020;50(4):251–263. doi: 10.1007/s10879-020-09463-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Saricam H., Erguvan F.M., Akin A., Akca M.S. The Turkish short version of the intolerance of uncertainty (IUS-12) scale: the study of validity and reliability. Ressjournal. 2014;1(3):148–157. doi: 10.17121/ressjournal.109. [DOI] [Google Scholar]
  58. Songur C., Saylavci E., Kiran S. A comparative examionation of mental health services on Europe and Turkey. Soc. Sci. Stud. J. 2017;3(4):276–289. doi: 10.26449/sssj.36. [DOI] [Google Scholar]
  59. Stice E., Lawrence N.S., Kemps E., Veling H. Training motor responses to food: a novel treatment for obesity targeting implicit processes. Clin. Psychol. Rev. 2016;49:16–27. doi: 10.1016/j.cpr.2016.06.005. [DOI] [PubMed] [Google Scholar]
  60. Sun S., Lin D., Goldberg S., Shen Z., Chen P., Qiao S., Brewer J., Loucks E., Operario D. A mindfulness-based mobile health (mHealth) intervention among psychologically distressed university students in quarantine during the COVID-19 pandemic: a randomized controlled trial. J. Couns. Psychol. 2021 doi: 10.1037/cou0000568. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Teachman B.A., Beadel J.R., Steinman S.A. In: The Wiley Handbook of Anxiety Disorders, Vol. 1. Theory and Research; Vol. 2. Clinical Assessment and Treatment. Emmelkamp P., Ehring T., editors. Wiley Blackwell; 2014. Mechanisms of change in CBT treatment; pp. 824–839. [Google Scholar]
  62. Trak E., Uzumcu E., Inozu M., Hacıomeroglu A.B. Development and initial validation of COVID-19 Distress Scale. Int. J. Cogn. Ther. 2021 doi: 10.1002/ijop.12844. [DOI] [Google Scholar]
  63. Twenge J., Joiner T.E. Mental distress among U.S. adults during the COVID-19 pandemic. J. Clin. Psychol. 2020;76(12):2170–2182. doi: 10.1002/jclp.23064. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Vallejo G., Fernández M.P., Livacic-Rojas P.E., Tuero-Herrero E. Comparison of modern methods for analyzing repeated measures data with missing values. Multivar. Behav. Res. 2011;46(6):900–937. doi: 10.1080/00273171.2011.625320. [DOI] [PubMed] [Google Scholar]
  65. Van Buuren S., Groothuis-Oudshoorn K. mice: multivariate imputation by chained equations in R. J. Stat. Softw. 2011;45:1–67. doi: 10.18637/jss.v045.i03. [DOI] [Google Scholar]
  66. Vintila M., Tudorel O.I., Stefanut A., Ivanoff A., Bucur V. Emotional distress and coping strategies in COVID-19 anxiety. Curr. Psychol. 2022;1–10 doi: 10.1007/s12144-021-02690-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Wang C., Pan R., Wan X., Tan Y., Xu L., Ho C.S., Ho R.C. Immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (COVID-19) epidemic among the general population in China. Int. J. Environ. Res. Public Health. 2020;17(5):1729. doi: 10.3390/ijerph17051729. [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Wasil A.R., Venturo-Conerly K.E., Shingleton R.M., Weisz J.R. A review of popular smartphone apps for depression and anxiety: assessing the inclusion of evidence-based content. Behav. Res. Ther. 2019;123 doi: 10.1016/j.brat.2019.103498. [DOI] [PubMed] [Google Scholar]
  69. World Health Organization WHO Coronavirus (COVID-19) Dashboard. 2022. https://covid19.who.int/ accessed 18 May 2022.
  70. Xiang Y.T., Yang Y., Li W., Zhang L., Zhang Q., Cheung T., Ng C.H. Timely mental health care for the 2019 novel coronavirus outbreak is urgently needed. Lancet Psychiatry. 2020 doi: 10.1016/s2215-0366(20)30046-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  71. Xiong J., Lipsitz O., Nasri F., Lui L.M., Gill H., Phan L., Chen-Li D., Iacobucci M., Ho R., Majeed A., McIntyre R.S. Impact of COVID-19 pandemic on mental health in the general population: a systematic review. J. Affect. Disord. 2020;277:55–64. doi: 10.1016/j.jad.2020.08.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Yildirim A., Boysan M., Kefeli M.C. Psychometric properties of the Turkish version of the Depression Anxiety Stress Scale-21 (DASS-21) Br J Guid Counc. 2018;46(5):582–595. doi: 10.1080/03069885.2018.1442558. [DOI] [Google Scholar]
  73. Yorulmaz O., Gencoz T. Examination of Interpretations of Intrusions Inventory, Obsessive Beliefs Questionnaire and Thought Control Questionnaire Used For Evaluation of Obsessive-Compulsive Disorder Symptoms in Turkish Sample. Turkish Psychological Articles. 11(22) 2008. pp. 14–16. [Google Scholar]
  74. Zvolensky M.J., Garey L., Rogers A.H., Schmidt N.B., Vujanovic A.A., Storch E.A., Buckner J.D., Paulus D.J., Alfano C., Smits J.A.J., O'Cleirigh C. Psychological, addictive, and health behavior implications of the COVID-19 pandemic. Behav. Res. Ther. 2020;134 doi: 10.1016/j.brat.2020.103715. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of Affective Disorders are provided here courtesy of Elsevier

RESOURCES