Skip to main content
Internet Interventions logoLink to Internet Interventions
. 2022 Jul 13;29:100560. doi: 10.1016/j.invent.2022.100560

Enhancing mental health literacy in obsessive-compulsive disorder and reducing stigma via smartphone: A randomized controlled trial protocol

Antonio Chaves a, Sandra Arnáez a,b, Diana Castilla a,c, María Roncero a, Gemma García-Soriano a,
PMCID: PMC9305319  PMID: 35874968

Abstract

Background

Obsessive-compulsive disorder (OCD) is a disabling disorder that can be successfully treated. However, individuals with OCD do not seek or delay seeking treatment. This delay may be explained by poor mental health literacy and stigmatizing attitudes toward OCD in community. In order to work on these variables, a gamified mental health mobile application (app) called esTOCma has been developed. The purpose of this study is to describe the protocol for a study to test the efficacy of esTOCma, increasing mental health literacy and help-seeking intention, reducing the stigmatizing attitudes and social distance suffered by people with OCD, as well as the distress associated with obsessive-compulsive symptoms.

Methods

A randomized controlled trial with a crossover design with two conditions (immediate-use App group versus delayed-use App group) will be conducted on a non-clinical adult sample of the community of a minimum size of 200 participants. Participants in the immediate-use App group will start using the app at baseline until completion (10 days); whereas participants in the delayed-use App group will wait 10 days, and then start using the app until completion (10 days). The outcomes will be measured at four assessment points (baseline; 10 days from baseline; and 20 days from baseline; and after 3 months). The following instruments will be administered: Attribution Questionnaire, General Help-Seeking Questionnaire, Social Distance Scale, Mental Health Literacy, Psychoeducation Questionnaire, Social Desirability Scale, Single-Item Self-esteem Scale, and Obsessive-Compulsive Inventory-Revised.

Discussion

This protocol presents the first study to describe a randomized control trial of a mental health app focused on changing mental health literacy, stigmatizing attitudes, social distance and help-seeking intention associated with OCD. An app intervention of these characteristics is especially relevant nowadays as the COVID-19 pandemic has increased obsessive-compulsive symptoms and severity. An improvement in general knowledge about OCD and a reduction in stigma could be associated with earlier OCD detection and an increase in help-seeking intention, which could result in greater wellbeing. Moreover, normalizing intrusions and knowledge about the cognitive OCD model could serve as a protective variable in vulnerable individuals.

Trial registration

ClinicalTrials.gov identifier: NCT04777292. Registered February 23, 2021, https://clinicaltrials.gov/ct2/show/NCT04777292.

Keywords: OCD, RCT, Stigma, Mental health literacy, Help-seeking intention, App

Highlights

  • esTOCma is a gamified app-based intervention developed to reduce OCD help-seeking delay.

  • esTOCma aims to increase mental health literacy on OCD and reduce stigma.

  • esTOCma could promote earlier help-seeking in evidence-based treatments.

  • esTOCma's efficacy has been tested with a randomized controlled trial crossover design.

  • Apps could reduce the personal, social and financial costs associated with OCD.

1. Background

Obsessive-compulsive disorder (OCD) is a debilitating heterogeneous condition (Pinto et al., 2006) that affects approximately 2–2.3 % of the population (Ruscio et al., 2010). It is also considered one of the 10 leading causes of disability worldwide (World Health Organization, 1999) and is associated with high impairment in quality of life (Fontenelle et al., 2010) and considerable interference in daily living (Torres et al., 2006). This interference in OCD sufferers' lives has increased with the current COVID-19 pandemic and patients have perceived a worsening of their OCD symptoms (Alonso et al., 2021; Wheaton et al., 2021). COVID-19 effects seemed more negative on contamination (Alonso et al., 2021; Wheaton et al., 2021) and responsibility for harm symptoms (Wheaton et al., 2021) than on other OCD dimensions. Furthermore, OCD patients reported greater fears about COVID-19 than community participants (Alonso et al., 2021; Wheaton et al., 2021), as well as higher perceived anxiety and depression, and had more frequent suicide-related thoughts than community (Alonso et al., 2021). Fortunately, research shows that OCD can be treated successfully through cognitive-behavioral therapy and pharmacotherapy with serotonin reuptake inhibitors (Koran and Simpson, 2013; McKay et al., 2015; National Institute for Health and Care Excellence [NICE], 2005). However, there is a lengthy delay in seeking treatment and a significant proportion of OCD patients never ask for professional help (Albert et al., 2019; Belloch et al., 2009). This delay in seeking treatment is a serious public health problem that has considerable effects and costs for the individual, family, mental health system and society (National Institute for Health and Clinical Excellence, 2006).

Studies on OCD highlight two main factors behind the delayed treatment-seeking behavior among OCD patients. First, a poor mental health literacy (MHL) (i.e., misidentification of OCD symptoms, not knowing where to find help, OCD patients believing that they can manage their symptoms, and being ashamed to tell others about their symptoms or ask for treatment) in the disorder (García-Soriano et al., 2014; Hathorn et al., 2021; Stewart et al., 2019). And second, stigmatizing attitudes regarding OCD (Glazier et al., 2015; Ociskova et al., 2013). In this sense, stigma has been defined as the prejudice and discrimination that deprives individuals with mental illness of the opportunity to achieve their personal goals (Corrigan et al., 2015). It can be distinguished between public stigma, which occurs when the general population endorses prejudice and manifests discrimination toward people with mental illness, and self-stigma, which occurs when people with mental illness internalize stigma experience and suffer diminished self-esteem and self-efficacy (Corrigan and Watson, 2002). Regarding self-stigma, many OCD patients expect negative reactions from their environments and consider the concealment of symptoms as a viable strategy for dealing with OCD (Stengler-Wenzke et al., 2004), which encourages keeping OCD diagnosis in secret, known only by patients and relatives who know about the problem (Ociskova et al., 2013). Both MHL and stigma differ according to varying OCD content dimensions. In this regard, there are some obsessions, such as contamination or order, that are better recognized than others (García-Soriano and Roncero, 2016). Furthermore, different studies suggest that obsessions about violent/harm and sexual themes, in comparison with other contents, are associated with greater stigmatizing attitudes, higher social distance desire or greater perceived dangerousness (Ponzini and Steinman, 2021).

Accordingly, the implementation of evidence-based interventions and educational campaigns to increase community's mental health literacy could help to eliminate long delays in help-seeking and the personal, social and economic costs associated with this delay, such as a worst treatment response (Dell'Osso et al., 2010) and lower probabilities of remission of OCD symptoms (Eisen et al., 2010). It could also bring patients to evidence-based therapies and dismiss stigma (Chaves et al., 2021c; Hathorn et al., 2021). These campaigns should focus on increasing the recognition and understanding of the various dimensions of OCD, identifying evidence-based OCD therapies, improving access to mental health services and disregarding stigma associated with OCD (Chaves et al., 2021c; Hathorn et al., 2021). Improving mental health literacy on community population may be useful in increasing the public recognition of OCD as a problem, the knowledge about the benefits of empirically based treatments, and in reducing the barriers to seek for help. Additionally, improving community awareness of the universality of intrusive thoughts, and understanding the influence of dysfunctional appraisals and control strategies on the development/maintenance of clinical obsessions (Salkovskis, 1985) could potentially function as a protective mechanism for the development of OCD. In this sense, and as proposed by cognitive appraisal OCD models, research supports the mediating role of misinterpretations, emotional reactions, and control strategies in the escalation of intrusive thoughts to clinical obsessions (García-Soriano and Belloch, 2013; Llorens-Aguilar et al., 2021; Purdon and Clark, 1993; Radomsky et al., 2014).

Given the complicated mental health consequences we are experiencing with COVID-19 worldwide, it is all the more necessary to devote efforts to developing and validating tools and strategies that can help prevent OCD. However, intervention programs developed to increase MHL and reduce associated stigma are few and far between. Most of these interventions use psychoeducation (i.e. written information or an educational video) as the mechanism to increase OCD literacy and reduce stigma (Chaves et al., 2021c; Glazier and McGinn, 2015; Rees et al., 2014), with only one program (Gürbüz et al., 2020) using indirect contact (i.e. a video in which an OCD patient and a member of his family describe symptoms and problems associated with OCD). Results on OCD interventions show that brief psychoeducational interventions are useful in the short term to change dysfunctional OCD appraisals in community participants (Rees et al., 2014), increase knowledge about OCD, dismiss stigmatizing attitudes in teachers (Chaves et al., 2021c) and OCD misidentification in doctoral students in the mental health field (Glazier and McGinn, 2015). Moreover, Gürbüz et al. (2020) showed that their intervention, a video including psychoeducation and indirect contact, was able to change social distance and negative beliefs toward mental illness in community participants; and these results were maintained after 6 months.

Unfortunately, none of these proposals have benefited from the potential to develop Internet-based or app-based interventions. App-based interventions have the potential to deliver interventions to a large number of people: they are low-cost tools of great convenience for people who do not need to attend a workshop or therapy session, and they are available 24 h a day (Lattie et al., 2022; Thomas et al., 2014). Besides, serious games have been proposed in the literature as a new paradigm for increasing adherence to Internet Communication Technology (ICT)-based interventions (Fleming et al., 2016). Serious games can be defined as interventions with gaming elements as a primary method for achieving therapeutic goals (Fleming et al., 2015). Their use offers several advantages: providing immediate feedback; motivating users toward their goals; they are easy to use and inexpensive once developed; and they reach out to a vast segment of the population (Fitzgerald and Ratcliffe, 2020).

In this context, owing to the limited number of interventions centered on mental health literacy and stigma associated with OCD and the need for a tool that encourages early help-seeking, which in turn could serve as a tool to help prevent OCD, we have designed a gamified stand-alone intervention app called esTOCma. The name ‘esTOCma’ is a play on words in Spanish: estigma (in English, stigma) and TOC (in English, OCD). It is described in the Methods section. The development was based on evidence in the literature regarding: (1) the barriers associated with the delay in seeking treatment for OCD (García-Soriano et al., 2014; Glazier et al., 2015) thus offering information to increase participants' mental health literacy and reduce their potential stigmatizing attitudes associated with OCD; (2) the mechanisms used by the most effective programs to reduce mental health stigma (Griffiths et al., 2014; Morgan et al., 2018), thus using psychoeducation as well as contact and cognitive restructuring strategies; and (3) general principles about serious-game design (Connolly et al., 2012; Fleming et al., 2016), thus using quizzes, feedback after each activity, challenges and reward elements after each mission and the game are completed.

This paper aims to describe the protocol for a study that will test the efficacy of the version 1.0 of the mental health app esTOCma (esTOCma v1.0) in a community sample through a randomized control trial with a crossover design with two conditions: undertaking immediate-use or delayed-use of the intervention through the esTOCma mobile app. The delayed-use group will work as a waiting list comparison. We hypothesize that, after using esTOCma v1.0, participants will exhibit greater mental health literacy, an increase in OCD help-seeking intention in case of experiencing symptoms similar to those of OCD, fewer stigmatizing attitudes, a reduced desire for social distance associated with OCD, higher self-esteem and lower levels of distress associated with OCD symptoms. We also hypothesize that this change will continue to be significant in the follow-up assessments: after 10 days and 3 months from the post-intervention assessment.

2. Material and methods

2.1. Trial design

The esTOCma v1.0 app will be evaluated with a randomized control trial with a crossover design with two conditions: undertaking immediate-use (iApp) or delayed-use (dApp) of the intervention through the esTOCma mobile app. Participants have to read and agree with the participant information sheet and informed consent form before they can proceed with the app. If a participant does not agree, the app will ask the participant the reason for leaving the game, thanks them for their time and closes. Then, participants will be asked to complete sociodemographic data and indicate their relationship with OCD (non-clinical participant, OCD diagnosis, not diagnosed with OCD but with suspected symptoms). A minimum of 200 participants self-identified as non-clinical will be automatically randomized in the app either to the iApp group or the dApp group using a 1:1 allocation ratio and the standard function of the PHP language (app programming language). The participants in each group will then be randomized using a sampling without replacement method to one of six vignettes describing different obsessive-compulsive contents (i.e., aggression/harm; sexual; religious/blasphemous/immoral; contamination/washing; doubts/checking; superstition/symmetry/order). Randomization will be performed by the app. Most of the evaluation instruments will be answered in reference to the assigned vignette. The vignettes are based on real clinical cases (Belloch et al., 2011), and special care has been taken to write similar descriptions of the same length. All the patients described are called A, their sex is not specified, they are all of the same age (i.e., A is middle-aged), they all described symptoms that meet the diagnosis criteria for OCD following DSM-5, although the labels OCD, obsessions and compulsions are avoided. All the patients described a similar severity of the symptomatology with similar interference in social, familial and work areas, associated discomfort, and time consumed by obsessions and compulsions. In addition, all the vignettes use a similar structure: firstly, they give a description of the obsessions and compulsions; and secondly, they explain interference with and impairment of quality of life. The research team will be unable to influence any aspect of the randomization procedure and it will be a simple-blinded design for the assignment of each participant. Participants will then be asked to complete baseline questionnaires (T0). Participants in the iApp group will be allowed to start using the app immediately after completing T0 and until they complete the game. Ten days is the time recommended by the app to complete the game, as the recommendation is playing one mission per day, however, each person can progress according to their own needs. In order to avoid dropout, no penalty or restriction will be applied to participants not completing the app during the recommended time. Immediately after completing the app, they will be asked to complete the post-intervention assessment (T1), and then, 10 days later, they will be asked to complete a follow-up (T2). Participants in the dApp group will work as a waiting list comparison. Thus, they will not be allowed to use the app immediately after completing the baseline assessment (T0). Ten days after completing the baseline questionnaires, they will be asked to complete the baseline assessment (T1) again and then they will be allowed to play the game until they complete it. Afterwards, they will be asked to complete the post-intervention assessment (T2). Both groups will be asked to complete a follow up assessment at 3 months from post-intervention assessment. Fig. 1 shows the study design based on the Consolidated Standards of Reporting Trials (CONSORT) criteria. Thus, outcomes will be measured by self-report measures at four assessment points in the app (T0, T1, T2, T3). To remind participants to do one mission a day, a pop-up reminder will appear in their mobile at the end of the day if they have not completed the mission. If any of the assessments or missions are not completed, a maximum of two reminders will be sent through the app by e-mail on the second and fifth days after the event. If participants do not reply to any of the reminders sent and depending on their current status in the app and their progress through the missions, the research team will consider contacting participants by phone in order to encourage them to continue their participation with the app and avoid dropout. In case of participants finally not wanting to continue, a question about the reasons for leaving the game will be asked during the phone interview. The study follows the CONSORT criteria to describe, report and publish results. This procedure was approved by the Ethics Committee of Research in Humans of the Ethics Commission in Experimental Research of the University of Valencia (1276901). The trial is registered at clinicalstrials.gov as NCT04777292l.

Fig. 1.

Fig. 1

Design flowchart.

2.2. Sample and recruitment

Sample size has been estimated a priori using G-power 3.1 (Faul et al., 2007) based on previous studies using similar measures (Griffiths et al., 2014), and on a pilot study on the use of esTOCma (Chaves et al., 2021a). Power analyses revealed that 154 participants would give 80 % power at alpha = 0.05 to detect small effect size (0.18) in a repeated measure (4 times) between factors ANOVA analysis framework. Assuming an attrition rate of 30 %, the necessary sample increases to 200 participants.

Inclusion criteria will comprise being 18 years of age or older (there is no maximum age for participation in the study), having an Android or iOS mobile system with Internet access and self-reporting not having an OCD diagnosis nor having suspected symptoms of OCD. Exclusion criteria will be to show responses suspected of social desirability response bias based on the Spanish version of the Marlowe-Crowne Social Desirability Scale (Crowne and Marlowe, 1960). To encourage participants to complete the app, they will be given entries to a prize draw for a voucher to spend on Amazon (40 EUR).

Participants will be recruited through the social media and flyers. They will be invited to participate in a research study downloading the esTOCma v1.0 app for free from Google Play Store (Android) or Apple's App Store (iOS).

2.3. Intervention program

EsTOCma v1.0 is a serious game in which participants are asked to fight against the OCD stigma monster with their knowledge by accomplishing ten missions along a mountain road that finishes at the top, where esTOCma monster is located (see Fig. 2 and Appendix A for more detail). Participants have to free 10 characters who are prisoners of esTOCma monster, a creature that feeds on false beliefs and misinformation, preying on its victims from society's stigmatizing and dysfunctional beliefs about OCD. The app recommends completing one mission a day; however, each participant can choose the rate at which they complete the game.

Fig. 2.

Fig. 2

Characters and other visual elements of esTOCma v.1.

The missions are organized into three different intervention approaches: (1) Psychoeducation, that is the most common type of intervention for reducing stigma (Griffiths et al., 2014). It is focused on offering information about OCD and covers the most relevant areas of MHL; (2) indirect contact through 6 videos of approximately 2–3 min. Research suggests that indirect contact via video is an easier, more effective way of disseminating anti-stigma interventions than direct contact (Reinke et al., 2004). esTOCma videos involve three OCD-diagnosed patients from the Spanish national health system who talk about their own experience with OCD; and (3) cognitive restructuring to replace dysfunctional beliefs about OCD. This last intervention strategy has found to be effective in many anti-stigma interventions (Clement et al., 2015; Griffiths et al., 2014). Fig. 3 shows how each strategy intervention works in the app. See Appendix B for more detail for a description of each mission.

Fig. 3.

Fig. 3

esTOCma v.1 screenshots with examples of the different intervention approaches.

esTOCma final version (v1.0) was developed in five stages. Firstly, focus groups with PhD-level clinical psychologists and a usability expert were set up to complete the app specifications; secondly, a professional designer developed the graphic design of the app and created the characters and graphic elements of the game; thirdly, a computer engineer developed the app. Multiple prototypes were tested in pilot studies before the final version was built, and all stakeholder groups were involved throughout the design process. Finally, a beta version of esTOCma app was tested by a pilot group of participants from the community (Chaves et al., 2021a; Chaves et al., 2021b). The final version (v1.0) was developed taking into account the data of use, feasibility, usability and satisfaction testing results. The RCT will be conducted with the final version of esTOCma (called esTOCma v1.0.).

2.4. Measures

  • Sociodemographic variables will include sex, age, level of education, and profession. Participants will also be asked if they had previous mental health training/knowledge, have sought help with a mental health problem, or have had contact with people with mental disorders.
    • Primary outcome measures: stigma and social distance associated with OCD, help-seeking intention, and mental health literacy in OCD. These outcomes will be measured with the following questionnaires.
  • Attribution Questionnaire (AQ-27) (Corrigan et al., 2003). This measures stigma associated with mental disorders. The instrument consists of 27 items rated on a Likert-type scale ranging from 1 to 9. It includes 9 subscales: Blame, Anger, Pity, Help, Dangerousness, Fear, Avoidance, Segregation and Coercion. Questions are answered in relation to the assigned vignette (e.g., I would feel unsafe around A). Spanish version: Muñoz et al. (2015).

  • Social Distance Scale (SDS) (Link et al., 1987). This assesses an individual's willingness to interact with an individual with a mental disorder described in a vignette across seven different situations on a 4-point Likert Scale from 1 (definitely willing) to 4 (definitely unwilling). It includes 7 items and data will be calculated as a total score, with higher scores indicating a greater preference for social distance, and an internal reliability of 0.920.

  • General Help-Seeking Questionnaire (GHSQ) (Wilson and Deane, 2005). This measures the intention to seek help from different sources regarding the specific content described in the assigned vignette. Participants rate 10 items regarding their help-seeking intentions on a 7-point scale ranging from 1 (extremely unlikely) to 7 (extremely likely) for 10 different sources of help. The questionnaire includes a total score, and two subscales: formal sources and informal sources of help; higher scores indicate higher intention. The Spanish translation will be used (Pacheco del Castillo, 2018).

  • Mental Health Literacy Questionnaire (MHL). This was formulated for this study based on previous studies (Chaves et al., 2021c). It has two parts. Part 1 has 4 multiple-choice questions associated with the assigned vignette. Specifically: (1) problem recognition; (2) OCD identification; (3) perception of causality; and (4) effective treatment option. Part 2 has four multiple-choice questions related to their general knowledge of OCD: (1) identification of OCD as a mental disorder; (2) definition of obsession; (3) definition of a compulsion; and (4) role played by compulsions and other control strategies in the maintenance of obsessions. A total score will be calculated from the number of correct answers.
    • Secondary outcome measures: distress caused by OCD symptoms and self-esteem.
  • Obsessive-Compulsive Inventory-Revised (OCI-R; Foa et al., 2002). This will be used to assess distress caused by obsessive-compulsive symptoms. It is an 18-item self-report questionnaire rated on a 4-point scale from 1 (not at all) to 4 (extremely). The total score will be used. Spanish version: Fullana et al. (2005).

  • Single-Item Self-esteem Scale (SISE) (Robins et al., 2001). This consists of one single item for measuring self-esteem on a 5-point Likert scale ranging from 1 (not very true of me) to 5 (very true of me).

Additionally, participants will complete the Spanish short version of the Marlowe-Crowne Social Desirability Scale (Crowne and Marlowe, 1960) (MC-SDS) that has shown a good reliability with a Cronbach's 0.80 (Gutiérrez et al., 2016). It consists of 18 items where participants have to answer whether the items are true or false in relation to themselves. As suggested by Sanz et al. (2018), answers from participants scoring ≥14 are suspected of corresponding to high levels of social desirability and they will be excluded from the study, based on the criteria of the Spanish version of the MC-SDS. MC-SDS will be assessed after mission 5.

2.5. Data management

In relation to data security and protection, and following the main principles of Privacy by Design (Cavoukian, 2010), when participants register in the app, they are given an identification number (ID) automatically generated by the app. The data from the questionnaires and the game will be matched with the personal player ID. In this way, the database generated by the app will be anonymized and the app will only collect the mobile terminal ID. Participants complete their nicknames and contact information when they register in the app, but this information will be accessed only by the primary researcher of the study. The data is stored on a secure server. Appropriate measures have been taken to ensure complete confidentiality of the participants' personal data in this study in accordance with Personal Data Protection Regulation 3/2018, Regulation (EU) 2016/679 of the European Parliament and of the Council and the Data Protection Agency. Only members of the research team will have access to the data files.

2.6. Statistical analysis

Descriptive statistics will be calculated for the data obtained. To check that the iApp group and the dApp group do not previously differ in the demographic variables or the variables of interest, t-tests and χ2 tests will be calculated. To assess the effect of the esTOCma app in both study groups, a series of repeated analysis of variance measures with Bonferroni adjustments will be performed. Cohen's d effect size will be calculated. Only data from participants who start and finish the app will be analyzed, i.e., those who complete all the missions. All statistical analyses will be carried out with SPSS v26.

3. Results and discussion

This paper describes the protocol for analyzing the effectiveness of esTOCma v1.0 using a randomized control trial. esTOCma v1.0, a gamified app, offers an intervention on the public knowledge and stigmatizing attitudes toward mental health problems, specifically in OCD. It responds to the need to reduce long delays in seeking treatment for OCD and the personal, social and economic costs associated with the delay. Providing mental health knowledge to the general population could have a direct impact on earlier OCD detection, encourage patients to seek evidence-based help earlier, improve their quality of life, and act as a protective mechanism against the vulnerability of developing OCD. Furthermore, esTOCma takes into consideration the heterogeneity of OCD content both at evaluation and throughout the intervention program. This is very relevant since some obsessional contents, such as contamination—partly because of the attention given in the media—are much better known than others (McCarty et al., 2017). In addition, lesser-known contents are associated with higher rates of stigma and concealment (Ponzini and Steinman, 2021). esTOCma v1.0 also responds to the need for tools to help with the understanding and management of psychopathology in a context marked by an increase in psychopathological symptoms as a consequence of COVID-19, in which experts claim for empirically supported interventions in an easy-to-deliver format not requiring face-to-face contact (Wind et al., 2020).

esTOCma v1.0 includes mechanisms of action (i.e. psychoeducation, contact, cognitive restructuring) based on the literature on stigma programs (e.g. Morgan et al., 2018), taking into account the scarce OCD interventions developed until now (e.g. Gürbüz et al., 2020), and adding the potential of a gamified app format that makes it possible to reach a vast segment of the population that would not be accessible through a face-to-face format (Fitzgerald and Ratcliffe, 2020). In this sense, esTOCma v1.0 is a pioneer project as it is the first intervention program designed through an app with a serious-game format focused on increasing mental health literacy and reducing stigmatizing attitudes toward OCD sufferers. This paper also responds to the need for e-mental health applications with rigorous designs and analyzes its efficacy. In fact, studies using RCT to assess the efficacy of mental health apps are still a minority in general (Miralles et al., 2020) and in OCD (Lee et al., 2019) in particular.

Some limitations are also expected. Firstly, online interventions are often characterized by a low adherence rate and secondly, individuals who are more familiar with the use of the Internet may be more likely to participate in the study, which may result in self-selection bias (Wantland et al., 2004). Although dropouts have been considered in the sample estimation, efforts have been made in the app design to facilitate adherence, including daily pop-ups and reminders by e-mail. Furthermore, anonymous, online, self-reported data collection is beneficial when participants are asked to report on sensitive topics such as stigma, and it has a disinhibiting effect and reduces social desirability (Griffiths, 2012). Social desirability will be also controlled through a specific self-reporting instrument.

Besides esTOCma's relevance for the community, if esTOCma v1.0 proves effective, we intend to test its efficacy in specific samples. Firstly, we aim to use it in a clinical context with OCD patients and their families. OCD becomes chronic if not treated, has a high cost for individuals, and frequently reveals self-stigmatizing attitudes that prevent them from seeking help (Glazier et al., 2015). Moreover, among help-seekers, only a minority of individuals receive evidence-based care (Schwartz et al., 2013). esTOCma v1.0 could offer patients cognitive-behavioral psychoeducation or reinforce it in patients who are under treatment. It will make it possible for them to seek empirical-based treatments, and it could dismiss self-stigma, improving their quality of life. OCD has also a high cost for families, who frequently take part in compulsions, reassure patients, assume part of their responsibilities, try to conceal the disorder, and show stigmatizing attitudes (Ociskova et al., 2013; Stengler-Wenzke et al., 2004). esTOCma v1.0 could help them to understand their relatives, obtain guidance to deal with their symptoms and help them to receive effective treatment. Secondly, esTOCma v1.0 could be implemented as a school program to improve the early detection of OCD and promote early help-seeking in schoolteachers and their students. A special emphasis on children and adolescents, parents and teachers is necessary, as early onset is related to seeking treatment later on (Belloch et al., 2009; Stengler et al., 2013). Thirdly, other specific samples, such as mental and medical health professionals who have revealed low levels of literacy in OCD (Glazier et al., 2013; Glazier et al., 2015; Steinberg and Wetterneck, 2017), will be a relevant focus of study.

4. Conclusions

We expect the results of this study to have relevant social and clinical implications. Combatting the long delay in seeking professional help is one of the main goals of esTOCma v1.0. In this way, increasing OCD literacy and reducing the associated stigma in the community could help with early recognition and detection of the disorder, detect under-diagnosed OCD individuals in the community, give the community strategies to know why to look for, and where to find evidence-based treatments, to avoid stigmatizing attitudes toward people with OCD. Besides, normalizing non-clinical intrusions might also serve as a protective variable in vulnerable individuals. Therefore, the implementation of esTOCma v1.0 in the community could reduce the personal, social and financial costs associated with OCD.

Funding sources

This work was supported by Grant RTI2018-098349-B-I00 funded by MCIN/AEI/10.13039/501100011033 and by ERDF A way of making Europe. None of the funders had any role in the design, preparation, review, or approval of the manuscript or on the decision to submit the manuscript for publication.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

We would especially like to thank to the OCD-diagnosed volunteers for allowing us to use the interview videos in esTOCma and to Ro García Virgós for her art design contributions.

Contributor Information

Antonio Chaves, Email: antonio.chaves@uv.es.

Sandra Arnáez, Email: sandra.arnaez@uv.es.

Diana Castilla, Email: diana.castilla@uv.es.

María Roncero, Email: maria.roncero@uv.es.

Gemma García-Soriano, Email: gemma.garcia@uv.es.

Supplementary data

Appendix A

Description of the esTOCma v1.0.

mmc1.docx (25.2KB, docx)
Appendix B

Table A. Strategy intervention and description of esTOCma missions.

mmc2.docx (30.5KB, docx)

Supplementary data to this article can be found online at https://doi.org/10.1016/j.invent.2022.100560.

References

  1. Albert U., Barbaro F., Bramante S., Rosso G., De Ronchi D., Maina G. Duration of untreated illness and response to SRI treatment in obsessive-compulsive disorder. Eur. Psychiatry. 2019;58:19–26. doi: 10.1016/J.EURPSY.2019.01.017. [DOI] [PubMed] [Google Scholar]
  2. Alonso P., Bertolín S., Segalàs J., Tubío-Fungueiriño M., Real E., Mar-Barrutia L., Fernández-Prieto M., Carvalho S., Carracedo A., Menchón J.M. How is COVID-19 affecting patients with obsessive-compulsive disorder? A longitudinal study on the initial phase of the pandemic in a Spanish cohort. Eur. Psychiatry. 2021;64(1):1–9. doi: 10.1192/j.eurpsy.2021.2214. e45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Belloch A., Cabedo E., Carrió C. Alianza ed; Madrid: 2011. TOC: obsesiones y compulsiones: tratamiento cognitivo; p. 408. [Google Scholar]
  4. Belloch A., del Valle G., Morillo C., Carrió C., Cabedo E. To seek advice or not to seek advice about the problem : the help-seeking dilemma for obsessive-compulsive disorder. Social Psychiatry and Psychiatric Epidemiology. 2009;44:257–264. doi: 10.1007/s00127-008-0423-0. [DOI] [PubMed] [Google Scholar]
  5. Cavoukian A. Privacy by design. Identity Inf. Soc. 2010;3(2):1–12. [Google Scholar]
  6. Chaves A., Árnaez S., Belloch A., García-Soriano G. Resultados preliminares de un estudio exploratorio. VII Congreso Internacional En Contextos Psicológicos, Educativos y de La Salud. 2021. ¿Puede una aplicación móvil de e-salud disminuir el estigma asociado al Trastorno Obsesivo-Compulsivo y aumentar la intención de búsqueda de ayuda? [Google Scholar]
  7. Chaves A., Árnaez S., Castilla D., García-Soriano G. VII Congreso Internacional En Contextos Psicológicos, Educativos y de La Salud. 2021. Satisfacción y usabilidad de una app para luchar contra el estigma asociado al TOC y aumentar los conocimientos del mismo. [Google Scholar]
  8. Chaves A., Arnáez S., Roncero M., García-Soriano G. Teachers’ knowledge and stigmatizing attitudes associated with obsessive-compulsive disorder: effectiveness of a brief educational intervention. Front. Psych. 2021;12(June):1–10. doi: 10.3389/fpsyt.2021.677567. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Clement S., Schauman O., Graham T., Maggioni F., Evans-Lacko S., Bezborodovs N., Morgan C., Rüsch N., Brown J.S.L., Thornicroft G. What is the impact of mental health-related stigma on help-seeking? A systematic review of quantitative and qualitative studies. Psychol. Med. 2015;45(1):11–27. doi: 10.1017/S0033291714000129. [DOI] [PubMed] [Google Scholar]
  10. Connolly T.M., Boyle E.A., MacArthur E., Hainey T., Boyle J.M. A systematic literature review of empirical evidence on computer games and serious games. Comput. Educ. 2012;59(2):661–686. doi: 10.1016/j.compedu.2012.03.004. [DOI] [Google Scholar]
  11. Corrigan P., Markowitz F.E., Watson A., Rowan D., Corrigan P. An attribution model of public discrimination towards persons with mental illness. J. Health Soc. Behav. 2003;44(2):162–179. [PubMed] [Google Scholar]
  12. Corrigan P.W., Gause M., Michaels P.J., Buchholz B.A., Larson J.E. The California assessment of stigma change: a short battery to measure improvements in the public stigma of mental illness. Community Ment. Health J. 2015;51(6):635–640. doi: 10.1007/s10597-014-9797-5. [DOI] [PubMed] [Google Scholar]
  13. Corrigan P.W., Watson A.C. The paradox of self-stigma and mental illness. Clin. Psychol. Sci. Pract. 2002;9(1):35–53. doi: 10.1093/clipsy/9.1.35. [DOI] [Google Scholar]
  14. Crowne D.P., Marlowe D. A new scale of social desirability independent of psychopathology. J. Consult. Psychol. 1960;24(4):349–354. doi: 10.1037/h0047358. [DOI] [PubMed] [Google Scholar]
  15. Dell’Osso B., Buoli M., Hollander E., Altamura A.C. Duration of untreated illness as a predictor of treatment response and remission in obsessive – compulsive disorder. World J. Biol. Psychiatry. 2010;11:59–65. doi: 10.3109/15622970903418544. [DOI] [PubMed] [Google Scholar]
  16. Eisen J.L., Pinto A., Mancebo M.C., Dyck I.R., Orlando M.E., Rasmussen S.A. A 2-year prospective follow-up study of the course of obsessive-compulsive disorder. The Journal of Clinical Psychiatry. 2010;71(8):1033–1039. doi: 10.4088/JCP.08m04806blu. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Faul F., Erdfelder E., Lang A.-G., Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav. Res. Methods. 2007;39:175–191. doi: 10.3758/BF03193146. [DOI] [PubMed] [Google Scholar]
  18. Fitzgerald M., Ratcliffe G. Serious games, gamification, and serious mental illness: a scoping review. Psychiatr. Serv. 2020;71(2):170–183. doi: 10.1176/appi.ps.201800567. [DOI] [PubMed] [Google Scholar]
  19. Fleming T.M., Cheek C., Merry S.N., Thabrew H., Bridgman H., Stasiak K., Shepherd M., Perry Y., Hetrick S. Serious games for the treatment or prevention of depression: a systematic review. <span/><span>Rev. Psicopatol. Psicol. Clin</span>. 2015;19(3):227. doi: 10.5944/rppc.vol.19.num.3.2014.13904. [DOI] [Google Scholar]
  20. Fleming T.M., de Beurs D., Khazaal Y., Gaggioli A., Riva G., Botella C., Baños R.M., Aschieri F., Bavin L.M., Kleiboer A., Merry S., Lau H.M., Riper H. Maximizing the impact of E-therapy and serious gaming: time for a paradigm shift. Frontiers. Psychiatry. 2016;7(APR) doi: 10.3389/FPSYT.2016.00065/ABSTRACT. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Foa E.B., Huppert J.D., Leiberg S., Langner R., Kichic R., Hajcak G., Salkovskis P.M. The obsessive-compulsive inventory: Development and validation of a short version. Psychol. Assess. 2002;14:485–496. doi: 10.1037/1040-3590.14.4.485. [DOI] [PubMed] [Google Scholar]
  22. Fontenelle I.S., Fontenelle L.F., Borges M.C., Prazeres A.M., Rangé B.P., Mendlowicz M.V., Versiani M. Quality of life and symptom dimensions of patients with obsessive-compulsive disorder. Psychiatry Res. 2010;179(2):198–203. doi: 10.1016/j.psychres.2009.04.005. [DOI] [PubMed] [Google Scholar]
  23. Fullana M.A., Tortella-Feliu M., Caseras X., Andión O., Torrubia R., Mataix-Cols D. Psychometric properties of the spanish version of the obsessive – compulsive inventory — revised in a non-clinical sample. J. Anxiety Disord. 2005;19:893–903. doi: 10.1016/j.janxdis.2004.10.004. [DOI] [PubMed] [Google Scholar]
  24. García-Soriano G., Roncero M. Psychiatry Research, En Revisión. Vol. 250. 2016. What do spanish adolescents think about obsessive-compulsive disorder? Mental health literacy and stigma associated with symmetry/order and aggression-related symptoms. [DOI] [PubMed] [Google Scholar]
  25. García-Soriano G., Rufer M., Delsignore A., Weidt S. Factors associated with non-treatment or delayed treatment seeking in OCD sufferers: a review of the literature. Psychiatry Res. 2014;220(1–2) doi: 10.1016/j.psychres.2014.07.009. [DOI] [PubMed] [Google Scholar]
  26. García-Soriano G., Belloch A. Symptom dimensions in obsessive-compulsive disorder: differences in distress, interference, appraisals and neutralizing strategies. J. Behav. Ther. Exp. Psychiatry. 2013;44(4):441–448. doi: 10.1016/j.jbtep.2013.05.005. [DOI] [PubMed] [Google Scholar]
  27. Glazier K., Wetterneck C., Singh S., Williams M. Stigma and shame as barriers to treatment in obsessive-compulsive and related disorders. Depress. Anxiety. 2015;4(3):191. doi: 10.4191/2167-1044.1000191. [DOI] [Google Scholar]
  28. Glazier K., Calixte R.M., Rothschild R., Pinto A. High rates of OCD symptom misidentification by mental health professionals. Ann. Clin. Psychiatry. 2013;25(3):201–209. [PubMed] [Google Scholar]
  29. Glazier K., McGinn L.K. Non-contamination and non-symmetry OCD obsessions are commonly not recognized by clinical, counseling and school psychology doctoral students. J. Depress. Anxiety. 2015;04(03):1000190. doi: 10.4172/2167-1044.1000190. [DOI] [Google Scholar]
  30. Griffiths K.M., Carron-Arthur B., Parsons A., Reid R. Effectiveness of programs for reducing the stigma associated with mental disorders. A meta-analysis of randomized controlled trials. World Psychiatry. 2014;13(2):161–175. doi: 10.1002/wps.20129. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Griffiths M.D. The use of online methodologies in studying paraphilias - a review. J. Behav. Addict. 2012;1(4):143–150. doi: 10.1556/JBA.1.2012.4.1. [DOI] [PubMed] [Google Scholar]
  32. Gürbüz A.A., Yorulmaz O., Durna G. Reducing the social stigma associated with obsessive compulsive disorder: a controlled trial of an intervention program in a turkish community sample. J. Evid.-Based Psychother. 2020;20(2):101–120. doi: 10.24193/jebp.2020.2.14. [DOI] [Google Scholar]
  33. Gutiérrez S., Sanz J., Espinosa R., Gesteira C., García-Vera M.P. The marlowe-crowne social desirability scale: norms for the spanish general population and development of a short version. An. de Psicol. 2016;32(1):206–217. doi: 10.6018/analesps.32.1.185471. [DOI] [Google Scholar]
  34. Hathorn S.K., Lochner C., Stein D.J., Bantjes J. Help-seeking intention in obsessive-compulsive disorder: predictors and barriers in South Africa. Front. Psych. 2021;12(September):1–8. doi: 10.3389/fpsyt.2021.733773. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Koran L.M., Simpson H.B. APA Practice Guidelines, March, 1–22. 2013. Guideline watch (March 2013): Practice guideline for the treatment of patients with obsessive-compulsive disorder.http://psychiatryonline.org/content.aspx?bookid=28&sectionid=40634994 [Google Scholar]
  36. Lattie E.G., Cohen K.A., Hersch E., Williams K.D.A., Kruzan K.P., MacIver C., Hermes J., Maddi K., Kwasny M., Mohr D.C. Uptake and effectiveness of a self-guided mobile app platform for college student mental health. Internet Interv. 2022;27 doi: 10.1016/j.invent.2021.100493. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Lee E.B., Hoepfl C., Werner C., McIngvale E. A review of tech-based self-help treatment programs for obsessive-compulsive disorder. J. Obs.-Compuls. Relat. Disord. 2019;23(June) doi: 10.1016/j.jocrd.2019.100473. [DOI] [Google Scholar]
  38. Link B.G., Cullen F.T., Frank J., Wozniak J.F. The social rejection of former mental patients: understanding why labels matter. Am. J. Sociol. 1987;92(6):1461–1500. doi: 10.1086/228672. [DOI] [Google Scholar]
  39. Llorens-Aguilar S., García-Soriano G., Arnáez S., Aardema F., O’Connor K. Is context a crucial factor in distinguishing between intrusions and obsessions in patients with obsessive-compulsive disorder? J. Clin. Psychol. 2021;77(3):804–817. doi: 10.1002/jclp.23060. [DOI] [PubMed] [Google Scholar]
  40. McCarty R.J., Guzick A.G., Swan L.K., McNamara J.P.H. Stigma and recognition of different types of symptoms in OCD. Journal of Obsessive-Compulsive and Related Disorders. 2017;12(September 2016):64–70. doi: 10.1016/j.jocrd.2016.12.006. [DOI] [Google Scholar]
  41. McKay D., Sookman D., Neziroglu F., Wilhelm S., Stein D.J., Kyrios M., Matthews K., Veale D. Efficacy of cognitive-behavioral therapy for obsessive-compulsive disorder. Psychiatry Res. 2015;227(1):104–113. doi: 10.1016/j.psychres.2015.02.004. [DOI] [PubMed] [Google Scholar]
  42. Miralles I., Granell C., Díaz-Sanahuja L., van Woensel W., Bretón-López J., Mira A., Castilla D., Casteleyn S. Smartphone apps for the treatment of mental disorders: systematic review. JMIR MHealth and UHealth. 2020;8(4) doi: 10.2196/14897. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Morgan A.J., Reavley N.J., Ross A., Too L.S., Jorm A.F. Interventions to reduce stigma towards people with severe mental illness: systematic review and meta-analysis. J. Psychiatr. Res. 2018;103(March):120–133. doi: 10.1016/j.jpsychires.2018.05.017. [DOI] [PubMed] [Google Scholar]
  44. Muñoz M., Guillén A.I., Pérez-Santos E., Corrigan P.W. A structural equation modeling study of the Spanish mental illness stigma attribution questionnaire (AQ-27-E) Am. J. Orthopsychiatry. 2015;85(3):243–249. doi: 10.1037/ort0000059. [DOI] [PubMed] [Google Scholar]
  45. National Institute for Health and Care Excellence (NICE) National Institute for Health and Care Excellence (NICE); 2005. Obsessive-compulsive Disorder and Body Dysmorphic Disorder: Treatment.www.nice.org.uk/guidance/cg31 Issue November 2005. [Google Scholar]
  46. National Institute for Health and Clinical Excellence . The British Psychological Society and The Royal College of Psychiatrists; 2006. Obsessive-Compulsive Disorder: Core Interventions in the Treatment of Obsessive-Compulsive Disorder (The Britis) [PubMed] [Google Scholar]
  47. Ociskova M., Prasko J., Cerna M., Jelenova D., Kamaradova D., Latalova K., Sedlackova Z. Obsessive compulsive disorder and stigmatization. Act. Nerv. Super. Rediviva. 2013;55(1):19–26. [Google Scholar]
  48. Pacheco del Castillo L.A. Dominican college students’ experiences of distress, help-seeking and stigma. Diss. Abstr. Int. A Hum. Soc. Sci. 2018;78(11-A(E)) http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=psyc15&NEWS=N&AN=2017-36664-067 No-Specified. [Google Scholar]
  49. Pinto A., Mancebo M.C., Eisen J.L., Pagano M.E., Rasmussen S.A. The Brown longitudinal obsessive compulsive study: clinical features and symptoms of the sample at intake. J. Clin. Psych. 2006;67(5):703–711. doi: 10.4088/jcp.v67n0503. http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3272757&tool=pmcentrez&rendertype=abstract [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Ponzini G.T., Steinman S.A. A systematic review of public stigma attributes and obsessive–compulsive disorder symptom subtypes. Stigma Health. 2021 doi: 10.1037/sah0000310. [DOI] [Google Scholar]
  51. Purdon C.L., Clark D.A. Obsessive intrusive thoughts in nonclinical subjects. Part I. Content and relation with depressive, anxious and obsessional symptoms. Behav. Res. Ther. 1993;31(8):713–720. doi: 10.1016/0005-7967(93)90001-b. [DOI] [PubMed] [Google Scholar]
  52. Radomsky A.S., Alcolado G.M., Abramowitz J.S., Alonso P., Belloch A., Bouvard M., Clark D.A., Coles M.E., Doron G., Fernández-Álvarez H., Garcia-Soriano G., Ghisi M., Gomez B., Inozu M., Moulding R., Shams G., Sica C., Simos G., Wong W. Part 1—you can run but you can’t hide: intrusive thoughts on six continents. Journal of Obsessive-Compulsive and Related Disorders. 2014;3(3):269–279. doi: 10.1016/j.jocrd.2013.09.002. [DOI] [Google Scholar]
  53. Rees C.S., Austen T., Anderson R.A., Egan S.J. Can corrective information reduce negative appraisals of intrusive thoughts in a community sample? Behavioural and Cognitive Psychotherapy. 2014;42(4):502–507. doi: 10.1017/S1352465813000994. [DOI] [PubMed] [Google Scholar]
  54. Reinke R.R., Corrigan P.W., Leonhard C., Lundin R.K., Kubiak M.A. Examining two aspects of contact on the stigma of mental illness. J. Soc. Clin. Psychol. 2004;23(3):377–389. doi: 10.1521/jscp.23.3.377.35457. [DOI] [Google Scholar]
  55. Robins R.W., Hendin H.M., Trzesniewski K.H. Measuring global self-esteem: construct validation of a single-item measure and the Rosenberg self-esteem scale. Personal. Soc. Psychol. Bull. 2001;27(2):151–161. doi: 10.1177/0146167201272002. [DOI] [Google Scholar]
  56. Ruscio A.M., Stein D.J., Chiu W.T., Kessler R.C. The epidemiology of obsessive-compulsive disorder in the National Comorbidity Survey Replication. Mol. Psychiatry. 2010;15(1):53–63. doi: 10.1038/mp.2008.94.The. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Salkovskis P.M. Obsessional-compulsive problems: a cognitive-behavioural analysis. Behav. Res. Ther. 1985;23(5):571–583. doi: 10.1016/0005-7967(85)90105-6. [DOI] [PubMed] [Google Scholar]
  58. Sanz J., Navarro R., Fausor R., Altungy P., Gesteira C., Morán N., García-Vera M.P. La escala de deseabilidad social de Marlowe-Crowne como instrumento para la medida de la deseabilidad social, la sinceridad y otros constructos relacionados en psicología legal y forense. Piscopatología Clínica, Legal y Forense. 2018;18:112–133. [Google Scholar]
  59. Schwartz C., Schlegl S., Kuelz A.K., Voderholzer U. Treatment-seeking in OCD community cases and psychological treatment actually provided to treatment-seeking patients: a systematic review. J. Obs.-Compuls. Relat. Disord. 2013;2(4):448–456. doi: 10.1016/j.jocrd.2013.10.006. [DOI] [Google Scholar]
  60. Steinberg D.S., Wetterneck C.T. OCD taboo thoughts and stigmatizing attitudes in clinicians. Community Ment. Health J. 2017;53(3):275–280. doi: 10.1007/s10597-016-0055-x. [DOI] [PubMed] [Google Scholar]
  61. Stengler-Wenzke K., Trosbach J., Dietrich S., Angermeyer M.C. Coping strategies used by the relatives of people with obsessive – compulsive disorder. J. Adv. Nurs. 2004;48(1):35–42. doi: 10.1111/j.1365-2648.2004.03166.x. [DOI] [PubMed] [Google Scholar]
  62. Stengler K., Olbrich S., Heider D., Dietrich S., Riedel-Heller S., Jahn I. Mental health treatment seeking among patients with OCD: impact of age of onset. Soc. Psychiatry Psychiatr. Epidemiol. 2013;48(5):813–819. doi: 10.1007/s00127-012-0544-3. [DOI] [PubMed] [Google Scholar]
  63. Stewart E., Grunthal B., Collins L., Coles M. Public recognition and perceptions of obsessive compulsive disorder. Community Ment. Health J. 2019;55(1):74–82. doi: 10.1007/s10597-018-0323-z. [DOI] [PubMed] [Google Scholar]
  64. Thomas N., McLeod B., Jones N., Abbott J.A. Developing internet interventions to target the individual impact of stigma in health conditions. Internet Interv. 2014;2(3):351–358. doi: 10.1016/j.invent.2015.01.003. [DOI] [Google Scholar]
  65. Torres A.R., Prince M.J., Bebbington P.E., Bhugra D., Brugha T.S., Farrell M., Jenkins R., Lewis G., Meltzer H., Singleton N. Obsessive-compulsive disorder: prevalence, comorbidity, impact, and help-seeking in the British National Psychiatric Morbidity Survey of 2000. Am. J. Psychiatry. 2006;163(November):1978–1985. doi: 10.1176/ajp.2006.163.11.1978. [DOI] [PubMed] [Google Scholar]
  66. Wantland D.J., Portillo C.J., Holzemer W.L., Slaughter R., Mcghee E.M. The effectiveness of web-based vs. Non-web-based interventions: a meta-analysis of behavioral change outcomes. J. Med. Internet Res. 2004;6(4) doi: 10.2196/jmir.6.4.e40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Wheaton M.G., Ward H.E., Silber A., McIngvale E., Björgvinsson T. How is the COVID-19 pandemic affecting individuals with obsessive-compulsive disorder (OCD) symptoms? Journal of Anxiety Disorders. 2021;81 doi: 10.1016/J.JANXDIS.2021.102410. [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Wilson C.J., Deane F.P. Measuring help seeking intentions: properties of the general help seeking questionnaire. Can. J. Couns. 2005;39(1):15–28. [Google Scholar]
  69. Wind T.R., Rijkeboer M., Andersson G., Riper H. The COVID-19 pandemic: the ‘black swan’ for mental health care and a turning point for e-health. Internet Interv. 2020;20(March) doi: 10.1016/j.invent.2020.100317. [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. World Health Organization . WHO; 1999. The “Newly Defined” Burden of Mental Problems. Fact Sheets No. 217. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix A

Description of the esTOCma v1.0.

mmc1.docx (25.2KB, docx)
Appendix B

Table A. Strategy intervention and description of esTOCma missions.

mmc2.docx (30.5KB, docx)

Articles from Internet Interventions are provided here courtesy of Elsevier

RESOURCES