Abstract
Background
Major depressive disorder (MDD) is a highly prevalent mental disorder with large disease burden, high levels of relapse or persistence, and overall suboptimal outcomes of protocolized pharmacological and psychotherapeutic treatments. There is an urgent need to improve treatment effectiveness, possibly through systematic treatment personalization. In psychotherapeutic treatments this can be achieved by case conceptualization. To support this process, we developed the Therap-i module, which consists of personalized Experienced Sampling Methodology (ESM) and feedback. The Therap-i module is integrated into outpatient psychotherapeutic treatment as usual (TAU) for depression. The study aim is to investigate the efficacy of the Therap-i module in decreasing symptomatology in unresponsive or relapsing patients diagnosed with MDD. We hypothesize that the Therap-i module will contribute to TAU by i) decreasing depressive symptoms, and ii) improving general functioning, therapeutic working alliance, and illness perception. This paper provides details of the study rationale, aims, procedures, and a discussion on potential pitfalls and promises of the module.
Methods
Patients diagnosed with MDD (n = 100) will enrol in a pragmatic two-armed randomized controlled trial. Randomization is stratified according to the patient’s treatment resistance level assessed with the Dutch Method for quantification of Treatment Resistance in Depression (DM-TRD). All fill-out the Inventory of Depressive Symptomatology Self Report (IDS-SR), Outcome Questionnaire (OQ-45), Illness Perception Questionnaire Mental Health (IPQ-MH), and Work Alliance Inventory Self Report (WAI-SR). In the intervention arm, through close collaboration between patient, clinician, and researcher, a personalized ESM diary is developed based on the patient’s case conceptualization. During the ESM monitoring period (8 weeks, 5 assessments/day), patients receive feedback three times, which is discussed among the abovementioned three parties. Both patients and clinicians will evaluate the Therap-i module.
Results
Data collection is ongoing.
Discussion
This is the first study in which personalized ESM and feedback is integrated in outpatient psychotherapeutic TAU for depression. The labour intensive procedure and methodological pitfalls are anticipated challenges and were taken into account when designing the study. When hypotheses are confirmed, the Therap-i module may advance treatment for depression by providing insights into personalized patterns driving or perpetuating depressive complaints.
Trial registration
Trial NL7190 (NTR7381), registered prospectively 03-08-2018.
Supplementary Information
The online version contains supplementary material available at 10.1186/s12888-021-03123-3.
Keywords: Depression, Ecological momentary assessment (EMA), Mental health care, Personalized experience sampling method (ESM) monitoring and feedback, Pragmatic randomized controlled trial
Background
Major depressive disorder (MDD) is a highly prevalent mental disorder with large disease burden for patients, their relatives, and society [1, 2]. Due to its intermittent course, where remission is often followed by relapse [3] or persistence of depressive complaints, there is a large and growing need to improve treatment outcome. This is particularly the case for those who have been unresponsive, or insufficiently responsive, to stepwise protocolized pharmacological and psychotherapeutic treatments [4]. These patients are considered difficult-to-treat and further personalization of treatment is needed. Clinicians typically personalize psychological treatment through (re-)investment in a patientۥ s case conceptualization [5]. In case conceptualization patients and clinicians develop a holistic working theory about the patient’s psychopathology.
The core feature of case conceptualization is the active participation of both the patient and clinician. In close collaboration, both combine their unique expertise to describe and explain the patient’s psychopathology in a holistic theory, or case concept. In short, case conceptualization involves: i) describing the presenting patient’s psychopathology, ii) providing a working hypothesis about the psychological mechanisms that drive or maintain a patient’s psychopathology, iii) organizing and integrating patient information based on the judgment of the clinician, and iv) informing diagnosis and treatment [5]. However, although well-established, there is no gold standard for the methodology and theoretical basis of case conceptualization [5–7]. This results in a lack of consensus on the essential features of case conceptualization and low inter-rater reliability between case concepts of different clinicians [8, 9]. Contributing factors might be that a case concept is based on retrospective information and depends on the questions asked by a particular clinician. These issues can be addressed by systematic momentary monitoring in a patient’s normal daily-life and subsequently providing feedback on this information during a subsequent regular consult. A scientific method to obtain this goal is the experienced sampling method (ESM).
ESM involves repeated sampling of momentary affect, cognitions, and/or behavior, typically via a patient’s own smartphone [10]. Prolonged ESM monitoring (i.e., weeks, months) results in time-series data, which can reveal the dynamic course of psychopathology in individuals [11, 12]. ESM holds the promise to advance the case conceptualization process for notable reasons: i) ESM provides patients and clinicians access to momentary information, thereby reducing retrospective bias [10, 13, 14]; ii) ESM promotes the patients’ reflections and insights in his/her own psychopathology by intensive self-monitoring [15]; and iii) ESM derived time-series data can be summarized into personalized feedback reports with intuitive visualizations of the course of the data (e.g., [16]). Such feedback may be further improved through more sophisticated analyses on the time-series data. A method that may hold particular promise for case conceptualization are person-specific network models [17, 18]. Proof-of-principle studies showed that results from such network models may be used as a starting point for a dialog between the patient and clinician during regular consults (e.g. [16, 19, 20].
The efficacy of ESM-based feedback as a therapeutic tool in decreasing depressive symptoms in outpatients diagnosed with depression was first shown in a pioneer randomized controlled trial (RCT, n = 102 [21];). The treatment as usual (TAU) condition primarily consisted of pharmacotherapy, with less than 10% of the patients receiving additional psychotherapeutic treatment. In the experimental condition of this study, patients received TAU and monitored their momentary positive affect for 6 weeks. Every week a feedback report was generated from the collected time-series data by a clinical researcher, who discussed the feedback with the patient and his/her clinician. In another, more recent pragmatic RCT in patients indicated for treatment for depression (n = 161, [22, 23]), efficacy of ESM-based feedback added to pharmacological and/or psychotherapeutic TAU was not established. In this study, patients monitored their momentary positive and negative affect for 4 weeks. ESM monitoring started after the intake procedure while the patient was waiting for psychotherapeutic treatment, continued monitoring during treatment and weekly feedback was emailed to the patient. The fourth feedback report was discussed with the patient and a research assistant. The authors concluded that, while ESM-based feedback in depression treatment was highly appreciated by both patients and clinicians in their study, its promise to augment the efficacy of regular depression treatment was not met. However, they cannot rule out that it advanced other domains (e.g. acquire better self-insight), or provided a more efficient way of delivering care (i.e. more patients may be treated when combining face-to-face sessions with ESM compared to full face-to-face treatment). Besides notable differences between the two ESM studies outlined above [21, 23], they have in common that ESM was neither personalized nor fully integrated into patient’s psychotherapeutic treatment.
Interestingly, recent research findings from patients and clinicians stress the importance of personalizing ESM assessment and feedback [15]. Furthermore, Bos and colleagues recommended that ESM should be implemented by an interdisciplinary team of patients, clinicians, researchers, and information technology specialists. These recommendations are in line with the clinical process of case conceptualization outlined above. The present study follows these recommendations by implementing them in a new ESM-based intervention: the Therap-i module. In Therap-i fixed ESM items will be supplemented with personalized items to cover core elements of the case concept of each individual. Patients and their clinicians will collaborate, together with the researcher, in personalizing ESM and, discussing ESM-derived feedback results in feedback sessions to advancing the case conceptualization process. Therap-i feedback includes graphs on daily fluctuations of scores on the ESM items, their associations, and contextual information (e.g. notes on (un) pleasurable events, company and activities). Researchers will assist patients and their clinicians during the entire procedure. This approach may anchor the case concepts more robustly in the patient’s narrative during psychotherapeutic treatment.
Study aims
We aim to test the efficacy of the Therap-i module as a supportive tool in psychotherapeutic TAU in MDD patients, who have been insufficiently responsive to protocolized treatments for depression. We hypothesize that the Therap-i module is effective in these patients in i) decreasing depressive symptom severity (primary outcome), ii) increasing general functioning, iii) increasing the therapeutic working alliance, and iv) improving illness perception (more specifically, increased illness insight, increased personal control over the illness, increased control through treatment, and reduces emotional representation of the illness). After treatment, self-management strategies will be examined and the Therap-i module will be evaluated by both patients and clinicians with quantitative and qualitative instruments. This paper provides a detailed overview of the patient inclusion procedure, instruments, and ESM protocol used in the Therap-i study. Furthermore, potential pitfalls and promises of the module are discussed.
Methods
Participants
The Therap-i study is a pragmatic RCT to evaluate the efficacy of the Therap-i module as a tool to support psychological TAU in patients diagnosed with MDD. In the RCT a control group, which receives TAU, is compared with an experimental group, which receives ESM-supported TAU (i.e. the Therap-i module). The design of the Therap-i study is shown in Fig. 1. The trial was registered prospectively on August 3th in 2018 in the Dutch Trial Register (NTR7381, www.trailregister.nl). The study was approved by the Medical Ethical Committee of the University Medical Center Groningen (UMCG, METc2018/434). All patients provide written informed consent prior to participation. According to UMCG policies for studies with negligible risk, an UMCG study monitor was appointed and a monitoring plan was written to assure quality of research. All investigators working on the study have received training and are certified in study conduct, informed consent, and risk assessment (BROK® certified).
The in- and exclusion criteria are given in Table 1. We aim to recruit 100 outpatients diagnosed with MDD, who have been unresponsive, or insufficiently responsive, to current or past protocolized psychotherapeutic treatments. Patients are recruited from multiple mental health care institutions in the North of the Netherlands. Patients are eligible for participation if they are starting with outpatient psychotherapeutic treatment, or if they are currently in treatment and their clinician decides to re-invest in case conceptualization and to personalize treatment. No limitations are applied regarding TAU, although use of medication and psychological interventions will be systematically monitored. Patients are informed that participation is voluntary and that not participating will not affect their TAU. Patients will be rewarded with €30 for their participation.
Table 1.
Inclusion criteria | Exclusion criteria |
---|---|
- Meets a MDD diagnosis according to the Diagnostic Statistical Manual (DSM-V) within the past six month. - Has received a primary protocolled psychological intervention for depression, consisting of four months of (cognitive) behavioural therapy, interpersonal therapy or brief psychodynamic therapy, and which is evaluated as not effective (enough) to repeat or continue by the clinician responsible for the participant. - Receives, or is scheduled to receive, policlinic psychological treatment, with weekly one-on-one consults with a clinician for at least 8 weeks. - Aged between 18 and 65 years. |
- Meets a diagnosis of MDD with psychotic features, bipolar disorder, substance use disorder, schizophrenia spectrum and other psychotic disorder, neurocognitive disorder in the past six months. - Has a history of treatment with electroconvulsive therapy. - Insufficient mastery of the Dutch language. - Inability or unwillingness to use a smartphone. |
Procedure
Recruitment
Potential participants receive a brochure and a contact consent form from their treating or directing clinician. After consent for contact, researchers contact the patient to provide further information about the study. Eligibility also includes approval of the treating clinician in case a patient is contacted via a directing clinician. After 1 week to consider participation, the patient is contacted again to inquire about their willingness to participate in the study and if so, to conduct a short screening for study eligibility. Research Electronic Data Capture (REDCap) software [24, 25] is used to register all contacts with (potential) participants and to manage logistics of the research procedure. Details on data management are given below.
Screening
Prior to enrolment, patients are screened on psychiatric disorders that are listed in the in- and exclusion criteria with the Mini-Schedules for Clinical Assessment in Neuropsychiatry (mini-SCAN) and assessed on treatment resistance with the Dutch Method for quantification of Treatment Resistance in Depression (DM-TRD). The mini-SCAN is a validated semi-structured interview to assess psychiatric disorders [26]. Parts of the interview that are relevant to the diagnoses mentioned in the in- and exclusion criteria, are used to confirm their respective presence or absence. The DM-TRD is a questionnaire to assess the level of treatment resistance in depression [27], which is used for stratification in the study (further details given below).
Randomisation and stratification
Randomization is outsourced to the research support office of the UMCG (Data Management & Research IT) and programmed in ALEA FormsVision (www.aleaclinical.eu). Participants are randomly allocated to the experimental or control group (allocation ratio 1:1). Randomization is stratified according to the level of treatment resistance based on the DM-TRD (cut-off score >11, [27]). A random block size will be used with a minimum of 4 and a maximum of 8, resulting in blocks of 4, 6 or 8. After n = 25, it is evaluated whether the chosen cut-off for stratification is effective. An adjustment of the cut-off will be considered if the chosen stratification cut-off fails in filling the cells in the randomization blocks. Failure is defined as one stratum containing ≥75% of the included participants. If this is the case, the stratification cut-off score will be replaced with the average DM-TRD score calculated on the participants included until then. This evaluation process is an iterative process, which will be repeated after every n = 25 inclusions. Allocation concealment is guaranteed through the use of ALEA FormsVision, making the randomisation sequence unpredictable for investigators. The ultimate goal of this process is to equally distribute the level of treatment resistance of the included patients across the two study groups.
The Therap-i module
For participants in the experimental group, the Therap-i module is added as a supportive tool to their psychotherapeutic TAU. The Therap-i module involves; i) a resilience interview, ii) the development and initialisation of an ESM diary, iii) ESM monitoring, and iv) three feedback sessions.
First, in the resilience interview, we focus on assessing strengths of the patient. This aspect can be relatively underrepresented in clinical practice because of the primary focus on psychopathological complaints. Personal strengths and resilience factors are inquired by asking questions originating from solution-focused therapy [28], such as “How would you describe yourself as a person before you got sick?”, “Can you describe one or several uplifts in the last month?”
Second, ESM items are developed for each patient individually. Items cover five domains that are important for case conceptualization: affect, behavior, cognition, body, and context. The ESM diary contains items that are assessed multiple times per day and items that are assessed once per day (morning/evening). It includes 22 fixed items (Table S1.1) and up to about ten personalized items. The fixed items are part of the ESM item bank developed in our department for the PErsonalized Treatment by Real-time Assessment (PETRA; www.petrapsy.nl [29];) project. The number of personalized items is flexible, but patients will be advised to limit the total number of items to 30 in order to manage measurement strain [15, 30]. The personalized items are developed in the ESM development session (45 min). Patient, clinician, and researcher collaborate to identify important personal aspects that trigger, perpetuate or influence psychiatric complaints to create the personalized ESM items. The researcher provides suggestions based on information from the electronic patient health record and the resilience interview. The researcher checks whether identified aspects are covered by items in the PETRA item bank. If this is not the case, a new item will be created based on a set of criteria for high-quality ESM items given in the Supplementary Materials S2. After the first and second feedback session (explained in detail below), it is possible to add diary items if new insights were obtained. The session in which the ESM diary is personalized is immediately followed by a 30-min instruction session between the patient and researcher, which also includes a practice run.
Third, patients fill out their personalized ESM diary 5 times a day for 8 weeks. The personalized ESM diary is initiated in a web-based flexible interface on our secured server system Routine Outcome and QUality Assessment (RoQua, [31]). Timing of assessments will be personalized to fit natural wake-sleep rhythm; the first daily assessment will be 2–3 h after waking-up in the morning. The system is programmed to send text messages (beeps) with links to the online questionnaires to the patient’s smartphone. This yields a maximum of 280 data points for the personalized feedback. Patients will be called 48 h after starting with ESM monitoring for motivation and to ask whether they have questions or experience technical problems. Every other day, the frequency of filling out ESM diaries is monitored in RoQua. When more than three subsequent assessments are missed, patients will be contacted to check whether they experience problems. Patients are instructed to complete the ESM diaries as soon as possible after receiving the beep, preferably within 10 min, and that they have limited time to respond. When no response is received, patients will be reminded after 10 min, and the link will be closed after 30 min. During the ESM monitoring period the patient can contact the researcher 24/7 when needed.
Fourth, patients receive personalized feedback during a regular consult with their clinician and the researcher after 2, 4, and 8 weeks of ESM monitoring. Dynamic visualizations of ESM data are provided with in-house developed software for personalized feedback (in R, using packages RMarkdown and Shiny by L. von Klipstein [32, 33];). In the feedback sessions the focus is on combining quantitative information from ESM rating scales with qualitative (contextual) information from patient-entered text during monitoring. In line with previous feedback report procedures [21, 23], the reports contain increasingly rich information across the treatment period. Patients receive feedback on raw course ESM time-series data after 2 weeks, and after 4 and 8 weeks monitoring additionally feedback on ESM time-series derived network models is available. Detailed information on the feedback module is beyond the scope of the current design paper and described elsewhere (von Klipstein et al., in preparation). A log of the graphs discussed along with notes of the session will be saved as a pdf file and sent to the patient and clinician for reference.
Study parameters
Questionnaire assessments
Figure 1 shows the schedule of questionnaire assessments, and Table 2 provides the instruments used at each wave (T0-T7). All questionnaire were assessed in Dutch. All participants fill out questionnaires over a period of 8 months: before and during the first 8 weeks of their treatment period (T0-T3) and during a follow-up period (T4-T7). Questionnaire data are collected via our secured RoQua web application developed to monitor outcomes in health care and research [31]. Participants receive an email with a link to fill out the questionnaires. When necessary, participants receive a reminder to fill out the questionnaires by email after 2 and 4 days and a call by a research assistant after 4 days.
Table 2.
P | T0 | T1 | T2 | T3 | T4 | T5 | T6 | T7 | ||
---|---|---|---|---|---|---|---|---|---|---|
ESM period | ||||||||||
Screening/Stratification | ||||||||||
Mini-SCAN | psychiatric diagnoses | X | ||||||||
DM-TRD | treatment resistance | X | ||||||||
Baseline characteristics and others | ||||||||||
Questionnaire | demographics, medication, prior treatment | X | ||||||||
TAS-20 | alexithymia | X | ||||||||
LEIDS-RR | cognitive reactivity | X | ||||||||
Long-term difficulties questionnaire | long-term difficulties | X | ||||||||
DAS-A-17 | dysfunctional attitudes | X | ||||||||
CTQ-SF | childhood trauma | X | ||||||||
Happiness index | happiness | X | X | |||||||
Brugha | life events | X | ||||||||
Single item | medication switches | X | X | |||||||
Single item (clinician) | type of psychological intervention | X | ||||||||
Outcomes | ||||||||||
IDS-SR | depressive symptom | X | X | X | X | X | X | X | X | |
OQ-45 | psychol. functioning | X | X | X | X | X | X | X | X | |
WAI-SR | therapeutic alliancea | X | X | X | X | X | X | X | ||
IPQ-MH | illness perception | X | X | X | X | X | X | X | X | |
QSRD | self-management | X | ||||||||
Daily assessments | ||||||||||
ESMb | momentary affect and other factors | −5 times a day- | ||||||||
Actigraphy | physical activity | -continuous- | ||||||||
Evaluation Therap-i moduleb | ||||||||||
Questionnaire | participant evaluation | X | ||||||||
Questionnaire | clinician evaluation | X | ||||||||
Interview | participant evaluation | X |
Note: P, pre-baseline; T0, baseline measurement; T1 till T7, follow-up waves (see method section for details); CTQ-SF Childhood Trauma Questionnaire Short Form, DAS-A-17 Dysfunctional Attitude Scale form A, DM-TRD Dutch Method for quantification of Treatment Resistance in Depression, ESM Experienced Sampling Method, IDS-SR Inventory Depressive Symptomatology - Self-Report, IPQ-MH Illness Perception Questionnaire Mental Health, LEIDS-RR Leiden Index of Depression Sensitivity 2nd Revision, Mini-SCAN Mini-Schedules for Clinical Assessment in Neuropsychiatry, OQ-45 Outcome Questionnaire-45, QSRD Questionnaire Self-management in Recovery of Depression, TAS-20 Toronto Alexitymia Scale-20, WAI-SR Work Alliance Inventory Short form Revised. a, WAI-SR is assessed at baseline if the patient already knows their clinician (see method section for details); b = experimental group only
Outcomes are assessed at all waves with the: i) Inventory of Depressive Symptomatology Self-Report (IDS-SR) to assess depression symptom severity [34–36]; ii) the Outcome Questionnaire (OQ-45) to assess changes in psychosocial functioning within three domains, including symptom distress, interpersonal functioning and social role [37, 38]; iii) Working Alliance Inventory – Short form Revised (WAI-SR) to assess therapeutic alliance between a patient and their clinician [39, 40]; and iv) Illness Perception Questionnaire Mental Health (IPQ-MH) to assess a patients’ perception of their illness with four of its sub-scales, namely, personal control over the problem, control gained in treatment, coherence, and emotional representation [41].
Baseline measurement (T0) includes i) a questionnaire on sociodemographic characteristics (gender, age, education level, work- and living situation), medication use, and past psychotherapy; ii) the Toronto Alexithymia Scale (TAS-20) to assess alexithymia (a subclinical inability to describe emotions), which may influence a participant’s ability to fill out the ESM diaries and thereby possibly influencing the efficacy of the Therap-i module [42–44]; iii) the Leiden Index of Depression Sensitivity - second revision (LEIDS-RR, [45, 46] to assess cognitive reactivity to negative mood to index depression vulnerability; iv) the Long-term difficulties questionnaire (e.g. housing or financial problems), which may partly mediate the severity of depressive symptomatology or interfere with the recovery process [47]; v) the Dysfunctional Attitude Scale (form A) revised (DAS-A-17 [48]; to assess the presence and intensity of dysfunctional attitudes; vi) the Childhood Trauma Questionnaire – Short Form (CTQ-SF) to assess the severity of childhood maltreatment, which may partly mediate the severity and/or chronicity of depressive symptomatology [49]. Some additional measurements are performed including i) the Questionnaire of Self-management in the Recovery from Depression (QSRD) at T4 to assess self-management using a descriptive outcome measure [50]. The QRSD originates from qualitative research in a sample of depressed patients, ii) the Brugha recent life events questionnaire, enriched with positive life events, at T4 to assess important positive and negative life events [51, 52], iii) the happiness index at T0 and T4, which is a single item, used in a large Dutch sample before, to assess a global estimation of how a participant feels in general [53, 54], and iv) information regarding TAU is collected, including medication switches (T3 and T7), applied types of psychotherapeutic treatment (T3 filled out by the clinician), and the number of sessions and treatment length in minutes (T7) collected from the electronic patient health record.
Actigraphy assessment
All participants wear an actigraph device (MotionWatch8®) for a period of 8 weeks. Actigraphy measurement starts together with ESM monitoring in the experimental group and is timed to start about 10 days after baseline in the control group to approximately match this timing (also see Fig. 1). Actigraphy data will be used for calculation of day-time activity patterns and circadian rhythms. Patients are instructed to push the button on the actigraph when going to sleep and right after waking up.
Qualitative evaluation interview
In the experimental arm only, participants and clinicians will evaluate the Therap-i module. The participants’ evaluation questionnaires include experienced utility of ESM (content, practicality, strain), feedback reports (content, intelligibility, lay-out), and feedback sessions (content, intelligibility). The clinicians’ evaluation questionnaires include the experienced utility of the Therap-i module supporting TAU. Both questionnaires are in-house instruments. Via a semi-structured interview, qualitative evaluations of the Therap-i module are obtained from 20 patients. The interview covers five topics: i) overall evaluation of the Therap-i module, ii) evaluation of the separate components of the Therap-i module, including the resilience interview, filling out the ESM diaries, feedback rapports, and the collaborative discussion based on the feedback rapports with the clinician and researcher iii) obtained insights by patients on their depression and if and how these insights were used in daily life to better manage depressive complaints, iv) influences of the Therap-i module on the psychological treatment, and v) influences of the Therap-i module on the therapeutic alliance.
Data management
In the Therap-i study, the following types of data will be collected: questionnaire data, ESM data, logistic data, actigraphy data, interview data, personal data and work documents. Questionnaire and ESM-data are collected via the RoQua server [31]. RoQua is a secure web application which makes it possible to measure and monitor outcomes in health care and research. Logistic data will be collected via REDCap (Research Electronic Data Capture; https://www.project-redcap.org). REDCap is a secure web application for building and managing databases. Data collected in REDCap are automatically backed-up on a daily basis on the servers of the Universitair Medisch Centrum Groningen (UMCG). Logistic data concerns all data collected during the execution of research activities. For each procedural step, a digital questionnaire and checklist is built to support data capture and planning of research activities. Actigraphy data will be collected via CamNtech’s MotionWatch and downloaded via the software MotionWare. Interview data originate from the semi-structured interview as part of the qualitative evaluation of the Therap-i module. Personal data concern identifiable data, such as name, date of birth, (email) address, and telephone number. Work documents concern documents that are used in order to create the electronic diary or which contain feedback information discussed during a consult with the patient and therapist. All files are stored on a secured drive on the servers of the UMCG, which is accessible for authorized members of the research team only. Interviews are digitally recorded and the audio files will be transcribed verbatim. Work documents are scanned and stored as pdf-files. Paper versions are stored in a locked cabinet in our department. When the study closes, REDCap will be frozen after verification, all queries have been handled and the close-out visit of the monitor. Data files containing raw data are stored in a password protected folder on the secured drive. These data files as well as the actigraph, interview and personal data files will be locked for editing and contain the date of downloading/saving in their name. After publication of the main results of the study, the processed, pseudonymised data will be made available for re-use via a data transfer agreement.
Statistical methods
Sample size calculation and withdrawal
We performed a sample size calculation for our main hypothesis that adding the Therap-i module to TAU will lead to greater reduction of depressive symptoms (as assessed with the IDS-SR) compared to TAU (G*Power 3.1, F test for repeated measures, within-between interactions [55]. Our power analysis is based on the effect size found in a previous study by Kramer and colleagues [21]. In their pioneer RCT, the authors showed that systematic intensive self-monitoring and personalized feedback on contextualized patterns of positive affect through ESM could provide an effective add-on tool to routine clinical care. IDS-SR scores for the experimental group showed an initial 3-point drop 7 weeks after baseline. While this effect became larger at later measurements, we chose the 7-week measurement as a reference point because it was the smallest post-intervention effect and therefore provided a conservative basis for the power analysis. Given a range of standard deviations of 10.0–10.5, the decrease in IDS-SR scores translates to an effect size of 0.14–0.15 (Cohen’s f). With a sample size of 80 (40 per group), an alpha of 0.05 and an intraclass correlation of 0.5 (between the pre and post repeated measures), and 4 measurement occasions (T0-T3), we have an 85–90% power to detect such an effect. To achieve a realistic estimation of the required sample size, one needs to further account for dropout. In a study from our department that is closely related to the proposed study [22, 23], dropout in the intervention period was 20%. Accordingly, to ensure that we will have at least 40 participants in each treatment arm, we will enrol 50 participants per study arm (total n = 100).
Patients can withdraw from the study at any time for any reason if they wish to do so without any consequences. In the case of withdrawal, attempts will be made to follow-up with the patient to establish cause of withdrawal, and to collect qualitative data regarding experience of participation. All data, including those from withdrawn patients (unless they request for their data to be deleted), will be included in the final analysis. If a patient withdraws from the study prematurely without providing valuable data, a replacement participant will be sought if resources allow. In case patients withdraw before randomization, they will be replaced. After randomization they will not be replaced.
Efficacy of the Therap-i module
The following analysis strategy will be applied to both primary (IDS-SR) and secondary outcomes (OQ-45, IPQ-MH subscales, and WAI-SR). The data have a two-level hierarchical structure, because multiple assessments are clustered within subjects. Therefore, multilevel regression analyses will be used to estimate fixed effects for time, treatment allocation, and their interaction, as well as random intercepts and a random slope for time. In this way, we can examine the effect of time on outcomes (across multiple outcome measurements), the effect of group (2 levels: experimental or control group), and the interaction between time and group separately. Thresholds for statistical significance will be set at p < 0.05.
Users evaluation of the Therap-i module
The evaluations of the Therap-i module by patients and clinicians will be assessed in two ways: quantitatively, and qualitatively through semi-structured interviews in a subsample of patients (n = 20). The quantitative data will be presented using descriptive statistics (means and standard deviations). The qualitative analyses will be performed according to the validated Qualitative Analysis Guide of Leuven (QUAGOL) procedure [56].
Discussion
The ongoing global digitalisation influences the possibilities to monitor and manage our health and behaviours, and holds potential to improve outpatient mental healthcare. While routine outcome monitoring is being used to evaluate and compare treatment outcomes and process variables [57], the use of more frequent repeated measurements to assess psychopathological complaints has been limited. Currently existing monitoring routines are not fully taking advantage of the potential of these repeated measurements to obtain a fine-grained view on triggers and perpetuating factors of psychopathology. ESM offers the possibility to bring relevant information from outpatients’ daily life into regular consultations, enhancing a systematic personalized characterization of a patient’s psychopathology. In order to fulfil its promise to support precision psychiatry, ESM monitoring needs to be implemented in clinical practice, become part of the treatment process, and allow for personalization [58]. Until now, personalized ESM monitoring during psychotherapeutic treatment has not been researched as a supportive tool and with our pragmatic RCT study we aim to fill this gap. The unique contribution of our study is the integration of personalized ESM into regular outpatient treatment for depression. Quantitative and qualitative outcome variables will be combined to examine the added value of the Therap-i module. Specifically, the use of qualitative methods complements interests in clinical practice, since recovery and empowerment of patients are concepts which are difficult to grasp with quantitative measures. Data collection has started and will continue for at least another year. In short, the intent of the Therap-i module is to support the case conceptualisation process by increasing insight in a patient’s complaints and motivating change in behaviour through informed treatment strategies. As such the module may advance treatment efficacy for depression and the quality and duration of recovery from depression.
The following potential challenges should be mentioned. First, when interpreting our results, we have to take into account the selection bias of who was willing to enrol in the study. Not all patients and clinicians will be able, or are motivated, to invest time and effort needed to successfully participate. However, enrolment will show an accurate impression of the level of adoption of the module in this particular sample in clinical practice. Second, the study tests a combination of potentially active components in the Therap-i module: focus on resilience (via the resilience interview), ESM monitoring, the feedback rapport, and the collaborative discussion of the feedback results. Clear conclusions about the active ingredients are not warranted and more research will be needed for detailed insight into this matter. However, through our, quantitative and specifically qualitative evaluation of the Therap-i module, we gather information on these separate components to aid further testing and development. Third, the trial is “pragmatic” because it is designed to fit into regular clinical practice, which involves some flexibility in the procedure (e.g., different types and dose of TAU, different centers and therapists, a range of different comorbidities). The resulting heterogeneity may reduce statistical power and hide a potential effect of the Therap-i module. However, the fact that the trial is pragmatic also is a testament to the possibility to implement the module in clinical practice. Fourth, in case we find evidence that the Therap-i module is effective and a valuable contribution to the treatment of MDD patients, the findings cannot be generalized to other patient groups without additional study. Fifth, future large-scale clinical implementation of the Therap-i module would likely differ from this study because ideally patients and clinicians are able to implement the module independently without involvement of a researcher. To this end, therapists need to be trained and intuitive software needs to be developed.1
Despite these limitations, the Therap-i study will provide valuable insights into the promise of personalized ESM and ESM-based feedback in clinical practice. Currently, it is acknowledged that severe depressive problems are seldom fully cured as residual complaints often remain and that those affected have life-long vulnerabilities for relapse [59]. For this reason, the ultimate ambition behind our Therap-i module is to contribute to putting patients “back into the driver’s seat” and relieving them from the feeling that their depression is overpowering their lives.
Supplementary Information
Acknowledgements
Not applicable.
Abbreviations
- CTQ-SF
Childhood Trauma Questionnaire Short Form
- DAS-A-17
Dysfunctional Attitude Scale (form A) revised
- DM-TRD
Dutch Method for quantification of Treatment Resistance in Depression
- EMA
Ecological Momentary Assessment
- ESM
Experienced Sampling Methodology
- IDS-SR
Inventory of Depressive Symptomatology Self Report
- IPQ-MH
Illness Perception Questionnaire Mental Health
- LEIDS-RR
Leiden Index of Depression Sensitivity - second revision
- MDD
Major Depressive Disorder
- METc
Medical Ethical Committee
- Mini-SCAN
Mini-Schedules for Clinical Assessment in Neuropsychology
- OQ-45
Outcome Questionnaire
- PETRA
PErsonalized Treatment by Real-time Assessment
- QSRD
Questionnaire of Self-management in the Recovery from Depression
- RCT
Randomized Controlled Trial
- REDCap
Research Electronic Data Capture software
- RoQua
Routine Outcome and QUality Assessment
- TAS-20
Toronto Alexithymia Scale
- TAU
Treatment As Usual
- UMCG
University Medical Center Groningen
- WAI-SR
Working Alliance Inventory – Short form Revised
Authors’ contributions
HR, DvdV and MNS initiated, and HR, LvK, DvdV and MNS designed the Therap-i study. HR drafted the manuscript and LvK, DvdV, RS and MNS critically reviewed and commented on the manuscript. All authors contributed to the manuscript and approved the final version of the manuscript.
Funding
The research described in this publication was supported by grants awarded to HR from the charitable foundation” Stichting tot Steun VCVGZ” (grantno 239) and the “Innovatiefonds De Friesland” (grantno DS81). The MotionWatch actigraphy devices used in the Therap-i study are kindly provided by the iLab of the department of Psychiatry of the UMCG (www.ilab-psychiatry.nl). The funding sources had no role in the design of the study or in writing the manuscript.
Availability of data and materials
Data collection is ongoing therefore data and materials are not available. As such, there are currently neither publications containing the results of this study already published nor manuscripts in which results are described submitted to any journal.
Ethics approval and consent to participate
Ethical approval has been obtained from the Medische Ethische Toetsingscommissie UMCG (METc2018/434). All patients provide written informed consent prior to participation. All staff working on the study have received training in study conduct, informed consent and risk assessment. The informed consent process will ensure that participants understand the nature of the study, and the data being collected.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Footnotes
To meet this challenge, our group is currently developing the PErsonalized Treatment by Real-time Assessment (PETRA, petrapsy.nl) tool.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Wittchen HU, Jacobi F, Rehm J, Gustavsson A, Svensson M, Jonsson B, et al. The size and burden of mental disorders and other disorders of the brain in Europe 2010. Eur Neuropsychopharmacol. 2011;21(9):655–679. doi: 10.1016/j.euroneuro.2011.07.018. [DOI] [PubMed] [Google Scholar]
- 2.Olesen J, Gustavsson A, Svensson M, Wittchen HU, Jonsson B. CDBE2010 study group, et al. the economic cost of brain disorders in Europe. Eur J Neurol. 2012;19(1):155–162. doi: 10.1111/j.1468-1331.2011.03590.x. [DOI] [PubMed] [Google Scholar]
- 3.Kessler RC, De Jonge P, Shahly V, Van Loo HM, Wang PSE, Wilcox MA. Epidemiology of depression. In: Gotlib IH, Hammen CL, editors. Handbook of depression. 3. New York, NY, US: Guilford Press; 2014. pp. 7–24. [Google Scholar]
- 4.Hofmann SG, Asnaani A, Vonk IJJ, Sawyer AT, Fang A. The efficacy of cognitive behavioral therapy: a review of meta-analyses. Cognit Ther Res. 2012;36(5):427–440. doi: 10.1007/s10608-012-9476-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kuyken W, Padesky CA, Dudley R. Collaborative case conceptualization: working effectively with clients in cognitive-behavioral therapy. New York: Guilford Press; 2009. [Google Scholar]
- 6.Johnstone L, Dallos R. Introduction to formulation. In: Johnstone L, Dallos R, editors. Formulation in psychology and psychotherapy. 2. New York: Routledge; 2014. [Google Scholar]
- 7.Sperry L, Gudeman JE, Blackwell B, Faulkner LR. Psychiatric case formulations. Arlington: American Psychiatric Association; 1992. [Google Scholar]
- 8.Flitcroft A, James IA, Freeston M, Wood-Mitchell A. Determining what is important in a good formulation. J Cogn Psychother. 2007;35(3):325–333. doi: 10.1017/S135246580600350X. [DOI] [Google Scholar]
- 9.Persons JB, Mooney KA, Padesky CA. Interrater relability of cognitive-behavioral case formulations. Cognit Ther Res. 1995;19(1):21–34. doi: 10.1007/BF02229674. [DOI] [Google Scholar]
- 10.Myin-Germeys I, Kasanova Z, Vaessen T, Vachon H, Kirtley O, Viechtbauer W, et al. Experience sampling methodology in mental health research: new insights and technical developments. World Psychiatry. 2018;17(2):123–132. doi: 10.1002/wps.20513. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Wichers M. The dynamic nature of depression: a new micro-level perspective of mental disorder that meets current challenges. Psychol Med. 2014;44(7):1349–1360. doi: 10.1017/S0033291713001979. [DOI] [PubMed] [Google Scholar]
- 12.Piccirillo ML, Beck ED, Rodebaugh TL. A clinician's primer for idiographic reserach: considerations and recommandations. Behav Ther. 2019;50(5):938–951. doi: 10.1016/j.beth.2019.02.002. [DOI] [PubMed] [Google Scholar]
- 13.Stone A, Shiffman S. Ecological momentary assessment (EMA) in behavioral medicine. Ann Behav Med. 1994;15:199–202. doi: 10.1093/abm/16.3.199. [DOI] [Google Scholar]
- 14.Mehl MR, Conner TS. Handbook of research methods for studying daily life. New York: The Guilford Press; 2012. [Google Scholar]
- 15.Bos FM, Snippe E, Bruggeman R, Wichers M, van der Krieke L. Insights of patients and clinicians on the promise of the experience sampling method for psychiatric care. Psychiatr Serv. 2019;70:983–991. doi: 10.1176/appi.ps.201900050. [DOI] [PubMed] [Google Scholar]
- 16.Kroeze R, Van der Veen DC, Servaas MN, Bastiaansen JA, Oude Voshaar R, Borsboom D, et al. Personalized feedback on symptom dynamics of psychopathology: a proof-of-principle study. JPOR. 2017;3(1):1–10. doi: 10.17505/jpor.2017.01. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.von Klipstein L, Riese H, van der Veen DC, Servaas MN, Schoevers RA. Using person-specific networks in psychotherapy: challenges, limitations, and how we could use them anyway. BMC Med. 2020;18(1):345. doi: 10.1186/s12916-020-01818-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Bringmann LF, Vissers N, Wichers M, Geschwind N, Kuppens P, Peeters F, et al. A network approach to psychopathology: new insights into clinical longitudinal data. PLoS One. 2013;8(4):e60188. doi: 10.1371/journal.pone.0060188. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Groot PC. Patients can diagnose too: how continuous self-assessment aids diagnosis of, and recovery from, depression. J Ment Health. 2010;19(4):352–362. doi: 10.3109/09638237.2010.494188. [DOI] [PubMed] [Google Scholar]
- 20.Bak M, Drukker M, Hasmi L, van Os J. An n=1 clinical network analysis of symptoms and treatment in psychosis. PLoS One. 2016;11(9):e0162811. doi: 10.1371/journal.pone.0162811. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Kramer I, Simons CJ, Hartmann JA, Menne-Lothmann C, Viechtbauer W, Peeters F, et al. A therapeutic application of the experience sampling method in the treatment of depression: a randomized controlled trial. World Psychiatry. 2014;13(1):68–77. doi: 10.1002/wps.20090. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Bastiaansen JA, Meurs M, Stelwagen R, Wunderink L, Schoevers RA, Wichers M, et al. Self-monitoring and personalized feedback based on the experiencing sampling method as a tool to boost depression treatment: a protocol of a pragmatic randomized controlled trial (ZELF-i) BMC Psychiatry. 2018;18(1):276. doi: 10.1186/s12888-018-1847-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Bastiaansen JA, Ornee DA, Meurs M, Oldehinkel AJ. An evaluation of the efficacy of two add-on ecological momentary intervention modules for depression in a pragmatic randomized controlled trial (ZELF-i) Psychol Med. 2020;14:1–10. doi: 10.1017/S0033291720004845. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–381. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O'Neal L, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform. 2019;95:103208. doi: 10.1016/j.jbi.2019.103208. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Nienhuis FJ, van de Willige G, Rijnders CA, de Jonge P, Wiersma D. Validity of a short clinical interview for psychiatric diagnosis: the mini-SCAN. Br J Psychiatry. 2010;196(1):64–68. doi: 10.1192/bjp.bp.109.066563. [DOI] [PubMed] [Google Scholar]
- 27.Peeters FP, Ruhe HG, Wichers M, Abidi L, Kaub K, van der Lande HJ, et al. The Dutch measure for quantification of treatment resistance in depression (DM-TRD): an extension of the Maudsley staging method. J Affect Disord. 2016;205:365–371. doi: 10.1016/j.jad.2016.08.019. [DOI] [PubMed] [Google Scholar]
- 28.de Jong P, Berg IK. Interviewing for solutions. 4th edition ed. Cengage Learning Company: Wadsworth; 2012. [Google Scholar]
- 29.Bos FM, von Klipstein L, Emerencia AC, Veermans E, Wichers M, Riese H. PErsonalized Treatment Real-time Assessment (PETRA): A flexible interface for personalized diaries0020in psychiatry. 2021; Manuscript in preparation.
- 30.van Genugten CR, Schuurmans J, Lamers F, Riese H, Penninx BW, Schoevers RA, et al. Experienced burden of and adherence to smartphone-based ecological momentary assessment in persons with affective disorders. J Clin Med. 2020;9(2):322. doi: 10.3390/jcm9020322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Sytema S, Van der Krieke L. Routine outcome monitoring: a tool to improve the quality of mental health care? In: Thornicroft G, Ruggeri M, Goldberg D, editors. Improving mental health care: the global challenge. 1. Chichester: Wiley; 2013. pp. 246–263. [Google Scholar]
- 32.Allaire J, Xie Y, McPherson J, Luraschi J, Ushey K, Atkins A, et al. rmarkdown: Dynamic documents for R: R package version 2.5; 2020. https://github.com/rstudio/rmarkdown
- 33.Chang W, Cheng J, Allaire JJ, Xie Y, McPherson J. Shiny: Web Application Framework for R: R package version 1.3.2; 2019. https://CRAN.R-project.org/package=shiny
- 34.Rush AJ, Gullion CM, Basco MR, Jarrett RB, Trivedi MH. The inventory of depressive symptomatology (IDS): psychometric properties. Psychol Med. 1996;26(3):477–486. doi: 10.1017/S0033291700035558. [DOI] [PubMed] [Google Scholar]
- 35.Nolen WN, Dingemans PMAJ. Meetinstrumenten bij stemmingsstoornissen (instruments for measuring mood disorders) Tijdschrift voor Psychiatrie. 2004;46:681–686. [Google Scholar]
- 36.Schulte-van Maaren YW, Carlier IV, Zitman FG, van Hemert AM, de Waal MW, van der Does AJ, et al. Reference values for major depression questionnaires: the Leiden routine outcome monitoring study. J Affect Disord. 2013;149(1–3):342–349. doi: 10.1016/j.jad.2013.02.009. [DOI] [PubMed] [Google Scholar]
- 37.Lambert MJ, Burlingame GM, Umphress V, Hansen NB, Vermeersch DA, Clouse GC, et al. The reliability and validity of the outcome questionnaire. Clin Psychol Psychother. 1996;3:249–258. doi: 10.1002/(SICI)1099-0879(199612)3:4<249::AID-CPP106>3.0.CO;2-S. [DOI] [Google Scholar]
- 38.de Jong K, Nugter MA, Polak MG, Wagenborg JEA, Spinhoven P, Heiser WJ. The outcome questionnaire (OQ-45) in a Dutch population: a cross-cultural validation. Clin Psychol Psychother. 2007;14(4):288–301. doi: 10.1002/cpp.529. [DOI] [Google Scholar]
- 39.Horvath AO, Greenberg LS. Development and validation of the working Alliance inventory. J Couns Psychol. 1989;36(2):223–233. doi: 10.1037/0022-0167.36.2.223. [DOI] [Google Scholar]
- 40.Vervaeke G, Vertommen H. De Werkalliantievragenlijst (WAV) Gedragstherapie. 1996;29:139–144. [Google Scholar]
- 41.Witteman C, Bolks L, Hutschemaekers G. Development of the illness perception questionnaire mental health. J Ment Health. 2011;20(2):115–125. doi: 10.3109/09638237.2010.507685. [DOI] [PubMed] [Google Scholar]
- 42.Bagby RM, Parker JD, Taylor GJ. The twenty-item Toronto alexithymia scale--I. item selection and cross-validation of the factor structure. J Psychosom Res. 1994;38(1):23–32. doi: 10.1016/0022-3999(94)90005-1. [DOI] [PubMed] [Google Scholar]
- 43.Bagby RM, Taylor GJ, Parker JD. The twenty-item Toronto alexithymia scale--II. Convergent, discriminant, and concurrent validity. J Psychosom Res. 1994;38(1):33–40. doi: 10.1016/0022-3999(94)90006-X. [DOI] [PubMed] [Google Scholar]
- 44.Taylor GJ, Bagby RM, Parker JD. The 20-item Toronto alexithymia scale. IV. Reliability and factorial validity in different languages and cultures. J Psychosom Res. 2003;55(3):277–283. doi: 10.1016/S0022-3999(02)00601-3. [DOI] [PubMed] [Google Scholar]
- 45.van der Does W. Cognitive reactivity to sad mood: structure and validity of a new measure. Behav Res Ther. 2002;40(1):105–120. doi: 10.1016/S0005-7967(00)00111-X. [DOI] [PubMed] [Google Scholar]
- 46.Solis E, Antypa N, Conijn JM, Kelderman H, Van der Does W. Psychometric properties of the Leiden index of depression sensitivity (LEIDS) Psychol Assess. 2017;29(2):158–171. doi: 10.1037/pas0000326. [DOI] [PubMed] [Google Scholar]
- 47.Oldehinkel AJ, Rosmalen JGM, Buitelaar JK, Hoek HW, Ormel J, Raven D, et al. Cohort profile update: The TRacking Adolescents' Individual Lives Survey (TRAILS). Int J Epidemiol. 2015;44(1):76–76n. 10.1093/ije/dyu225. PMID: 25431468. PMCID: PMC4339762. [DOI] [PMC free article] [PubMed]
- 48.De Graaf LE, Roelofs J, Huibers MJ. Measuring dysfunctional attitudes in the general population: the dysfunctional attitude scale (form a) revised. Cognit Ther Res. 2009;33(4):345–355. doi: 10.1007/s10608-009-9229-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Thombs BD, Bernstein DP, Lobbestael J, Arntz A. A validation study of the Dutch childhood trauma questionnaire-short form: factor structure, reliability, and known-groups validity. Child Abuse Negl. 2009;33(8):518–523. doi: 10.1016/j.chiabu.2009.03.001. [DOI] [PubMed] [Google Scholar]
- 50.van Grieken RA, Kirkenier AC, Koeter MW, Nabitz UW, Schene AH. Patients' perspective on self-management in the recovery from depression. Health Expect. 2015;18(5):1339–1348. doi: 10.1111/hex.12112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Brugha T, Bebbington P, Tennant C, Hurry J. The list of threatening experiences: a subset of 12 life event categories with considerable long-term contextual threat. Psychol Med. 1985;15(1):189–194. doi: 10.1017/S003329170002105X. [DOI] [PubMed] [Google Scholar]
- 52.Spinhoven P, Elzinga BM, Hovens JG, Roelofs K, van Oppen P, Zitman FG, et al. Positive and negative life events and personality traits in predicting course of depression and anxiety. Acta Psychiatr Scand. 2011;124(6):462–473. doi: 10.1111/j.1600-0447.2011.01753.x. [DOI] [PubMed] [Google Scholar]
- 53.Abdel-Khalek AM. Measuring happiness with a single-item scale. Soc Behav Pers. 2006;34(2):139–150. doi: 10.2224/sbp.2006.34.2.139. [DOI] [Google Scholar]
- 54.Bos EH, Snippe E, de Jonge P, Jeronimus BF. Preserving subjective wellbeing in the face of psychopathology: buffering effects of personal strengths and resources. PLoS One. 2016;11(3):e0150867. doi: 10.1371/journal.pone.0150867. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Faul F, Erdfelder E, Lang AG, Buchner A. G*power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39(2):175–191. doi: 10.3758/BF03193146. [DOI] [PubMed] [Google Scholar]
- 56.Dierckx de Casterle B, Gastmans C, Bryon E, Denier Y. QUAGOL: A guide for qualitative data analysis. In J Nurs Stud. 2012;49(3):360–371. doi: 10.1016/j.ijnurstu.2011.09.012. [DOI] [PubMed] [Google Scholar]
- 57.Lambert MJ, Whipple JL, Kleinstäuber M. Collecting and delivering progress feedback: a meta-analysis of routine outcome monitoring. Psychotherapy. 2018;55(4):520–537. doi: 10.1037/pst0000167. [DOI] [PubMed] [Google Scholar]
- 58.Spinhoven P, Cuijpers P, Hollon S. Cognitive-behavioural therapy and personalized treatment: an introduction to the special issue. Behav Res Ther. 2020;129:103595. doi: 10.1016/j.brat.2020.103595. [DOI] [PubMed] [Google Scholar]
- 59.van Os J, Guloksuz S, Vijn TW, Hafkenscheid A, Delespaul P. The evidence-based group-level symptom-reduction model as the organizing principle for mental health care: time for change? World Psychiatry. 2019;18(1):88–96. doi: 10.1002/wps.20609. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
Data collection is ongoing therefore data and materials are not available. As such, there are currently neither publications containing the results of this study already published nor manuscripts in which results are described submitted to any journal.