Skip to main content
Elsevier - PMC COVID-19 Collection logoLink to Elsevier - PMC COVID-19 Collection
. 2022 May 5. Online ahead of print. doi: 10.1016/j.cbpra.2022.04.002

Efficacy and Acceptability of Remote Cognitive Behavioral Therapy for Patients With Major Depressive Disorder in Japanese Clinical Settings: A Case Series☆☆

Waka Nogami *, Atsuo Nakagawa †,‡,, Noriko Kato §,, Yohei Sasaki #, Taishiro Kishimoto ♠,, Masaru Horikoshi c, Masaru Mimura *
PMCID: PMC9417319  PMID: 36059862

Abstract

Remote cognitive and behavioral therapy (CBT) via videoconference has been garnering attention as a means of improving access to CBT for depression, in particular during the coronavirus disease 2019 pandemic. However, there is a lack of evidence supporting its implementation in Japanese clinical settings. This case series aimed to establish preliminary evidence of whether remote CBT can be an effective therapy for major depression in Japanese clinical settings. Five patients who met the diagnostic criteria for major depressive disorder were enrolled and underwent remote CBT via videoconference and face-to-face assessment interviews. The results showed that remote CBT via videoconference improved depressive symptoms, enabling a relatively high level of patient satisfaction and working alliance. Moreover, detailed feedback from our patients showed that continuous monitoring was preferable for increasing treatment engagement. Further research is warranted to test the efficacy and acceptability of remote CBT via videoconference for treating major depression.

Keywords: teletherapy, cognitive behavioral therapy, depression, videoconference

Abbreviations: BDI, Beck Depression Inventory; CBT, Cognitive behavioral therapy; COVID-19, coronavirus disease 2019; CSQ-8, Client Satisfaction Questionnaire; DSM, Diagnostic and Statistical Manual of Mental Disorders; M.I.N.I., Mini International Neuropsychiatric Interview; SCID-I/P for DSM-IV-TR, Structured Clinical Interview for DSM-IV-TR Axis I Disorders-Patient Edition; SRS, Session Rating Scale


Major depression is a common mental disorder in Japan and in other high-income countries (Bromet et al., 2011) and can lead to significant distress for individuals, have a great impact on quality of life, morbidity, and mortality, and place an enormous burden on society (Kessler et al., 2005, GBD, 2018, Hasin et al., 2005). Moreover, depression has been shown to be the leading cause of sick leave in Japan, followed by neoplasms (Nishiura et al., 2017). Although pharmacotherapy is the mainstay of depression treatment, only one-third of patients remit with initial antidepressants (Souery et al., 1999, Trivedi et al., 2006, Rush et al., 2006, Malhi and Mann, 2018). Therefore, there is a dire need to improve treatment strategies for major depression.

Cognitive behavioral therapy (CBT), the most studied form of psychotherapy based on Beck’s cognitive theory (Beck, 1976), is an effective treatment for depression (National Institute for Health and Care Excellence, 2009, Gelenberg et al., 2010). Previous meta-analysis has shown that both pharmacotherapy and CBT have comparable effects on major depression (Cuijpers et al., 2013). In addition, evidence shows that the combination of CBT and pharmacotherapy is more effective than pharmacotherapy alone (Cuijpers et al., 2009, Cuijpers et al., 2013). Furthermore, CBT is effective for patients with treatment-resistant depression (Nakagawa et al., 2017, Wiles et al., 2013, Wiles et al., 2016). However, in practice, one of the disadvantages of CBT is the availability of the therapy due to the limited number of trained therapists (Shafran et al., 2009). Therefore, improving the delivery efficiency of CBT, such as by group, remote, and online format, is vital for enhancing service provision (Berryhill et al., 2019, Karyotaki et al., 2021, López-López et al., 2019, Mohr et al., 2008, Shafran et al., 2021).

Research evidence has shown that teletherapy conducted via telephone or videoconference appears to be effective and has various advantages (Abbasy, 2020, Smith et al., 2020). Clinical trials have demonstrated the effectiveness of CBT delivered via telephone (Castro et al., 2020, Mohr et al., 2008, Simon et al., 2004). Recently, there has been a surge in the number of trials for CBT via videoconference for depression (Giovanetti et al., 2022, Griffiths et al., 2006, Lichstein et al., 2013, Manchanda and McLaren, 1998, Ong et al., 2020, Scogin et al., 2018, Stubbings et al., 2013). Griffiths et al. (2006) reported that 6–8 sessions of remote CBT significantly improved symptoms among outpatients with major depression and anxiety disorders. Furthermore, in a small randomized clinical trial, Stubbings et al. (2013) reported no significant differences in efficacy between remote and face-to-face CBT among patients with clinical depression and anxiety. In addition, no difference was reported in the level of working alliance and patient satisfaction. Thus, remote CBT via videoconference for patients with depression may be a promising delivery form of CBT.

In Japan, CBT has drawn attention from clinicians, academics, and the public as a treatment option for depression over the last decade. Since 2010, face-to-face CBT for outpatients with major depression has been covered by the National Health Insurance Scheme (Ono et al., 2011). However, the dissemination of CBT has been a challenge, as there is a marked regional difference in the number of implemented cases (https://remhrad.jp/index_data). One of the obstacles to implementing face-to-face CBT within the National Health Insurance Scheme is low insurance reimbursement, set to JPY4,800 (approximately 45 USD) per session, and a maximum of 16 sessions (typically on a weekly basis). Moreover, it is limited by allowing only physicians or trained nurses as the therapist (Hayashi et al., 2020). On the other hand, telemedicine blended with face-to-face visits has been covered by National Health Insurance Scheme since 2018, but it only covers a single visit per month. Thus, the current framework of insurance coverage for telemedicine visits is not feasible for delivering CBT sessions on a weekly basis (Supplementary Table 1).

The coronavirus disease 2019 (COVID-19) pandemic has had a significant impact on psychiatric care linked to commuting restrictions due to lockdown and the need for physical distancing in an interview room. Teletherapy is expected to be a contagion-safe format of delivery for stable mental health care during the COVID-19 global pandemic. However, there is no study that examined the feasibility of remote CBT for major depression in Japan. Therefore, there is a need to clarify an optimal implementation strategy for CBT in Japanese clinical settings. Thus, in this single case series, we aimed to determine whether remote CBT may be an effective therapy for major depression in Japanese clinical settings. Here, we present a case series of five patients with major depression who were offered remote CBT blended with face-to-face assessment interviews and evaluated the feasibility of implementing the treatment in terms of its efficacy and acceptability, as well as the quality of treatment, including patient satisfaction, working alliance, and patient feedback.

Method

Patients

The patients were referred to our CBT research clinic by their treating psychiatrist. To be considered for the case series, patients needed to meet the following inclusion criteria: (a) meeting the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) criteria (American Psychiatric Association, 1994) for major depressive disorder, (b) depressive symptoms with minimum score of 14 on the Beck Depression Inventory (BDI-II) (Beck et al., 1996), and (c) available to receive at least eight sessions of CBT. Patients were excluded if they had a history of bipolar disorder, psychosis, drug or alcohol dependence within the previous year, or severe or unstable organic brain damage.

Written informed consent was obtained from each patient prior to study entry. This study was approved by the Research Ethics Committee and was conducted in accordance with the relevant laws and institutional guidelines.

Procedure

Prior to enrollment, all patients underwent a face-to-face initial assessment interview conducted by a trained psychiatrist or psychologist. A Structured Clinical Interview for DSM-IV-TR Axis I Disorders-Patient Edition (SCID-I/P for DSM-IV-TR; First et al., 2005) was administered to confirm the depression diagnosis. We used SCID-I/P for DSM-IV-TR because the Japanese version of DSM-5 was not yet available at the study initiation. In addition, a Mini International Neuropsychiatric Interview (M.I.N.I.) (Sheehan et al., 1998, Otsubo et al., 2005) was administered to diagnose any co-morbid conditions. Sociodemographic information including age, gender, marital status, educational attainment, occupational history, and treatment history was obtained.

CBT Treatment

Therapists followed the CBT treatment manual for depression (Supplementary Table 2) (available from the Japanese Ministry of Health, Labour and Welfare website: http://www.mhlw.go.jp/bunya/shougaihoken/kokoro/dl/01.pdf), which is based on Beck’s treatment manual (Beck, 1979) with some adaptations for Japanese cultural characteristics, such as an enhanced emphasis on interpersonal relationships and considering the family as an essential part of treatment. Therapists were encouraged to revisit the necessary sections if deemed clinically appropriate. The patients typically received 16 weekly sessions, with up to 4 additional sessions when appropriate (Fujisawa et al., 2010). In two cases, the web-based CBT tool, Kokoro-no-skill-up-training (Supplementary Table 3) (available at the website: https://www.cbtjp.net) (Nakao et al., 2018), was partially provided as educational material for learning about depression, the rationale of CBT, and basic CBT skills. To ensure treatment fidelity, all therapists received 1-hour on-site group supervision, which included intensive review and detailed feedback following each session from a skilled CBT supervisor (corresponding author).

CBT Delivery Format and Medication Visit

Two delivery formats were used. In the first format, a videoconference system was used to conduct CBT sessions between our clinic and other medical facilities. Thus, the patients visited the hospital during each session. In the other format, a videoconference system was used to connect between the patients’ home and our clinic; thus, patients mainly received the sessions at home.

Patients regularly visited their treating psychiatrists approximately every 2 to 4 weeks and continued their antidepressant medication. The regular visit consisted of medication management along with education regarding medication and dosage schedules, review of adverse effects, and supportive guidance.

Assessments and Measures

All patients underwent assessments that consisted of the following measures: The baseline assessment was completed within 2 weeks before the start of CBT, the midpoint assessment was completed after eight sessions, and the posttreatment assessment was arranged within 4 weeks of therapy completion.

Beck Depression Inventory (BDI-Ⅱ; Beck et al., 1996)

The BDI-II is a 21-item self-administered rating scale developed to measure the severity of depression in adults and adolescents. Higher scores represent greater depression severity (range, 0–63), and minimal, mild, moderate, and severe symptom severity ranges have been specified. The BDI-II was administered at baseline, at the midpoint of treatment, and at posttreatment.

Client Satisfaction Questionnaire (CSQ-8; Larsen et al., 1979)

The Japanese version of the CSQ-8 (CSQ-8J; Tachimori & Ito, 1999), which is an 8-item brief scale, was used to assess the level of patient satisfaction with the therapy. Each item is scored on a 4-point Likert scale with eight different questions (range, 8–32). Higher scores indicate greater satisfaction. The CSQ-8J was administered at baseline and posttreatment.

Session Rating Scale (SRS; Duncan et al., 2003)

The SRS is a 4-item visual analogue instrument (range, 0–40) that was used to assess the working alliance. Specifically, the patient was asked to use a sliding scale ranging from 0 to 10 points on each scale to assess key dimensions of the therapeutic relationship: the relevance of the goals and topics, respect and understanding, patient-therapist fit, and overall alliance. The SRS was administered after each session.

Patient feedback

Open-ended interviews lasting 40–50 minutes were conducted with patients during or following the completion of therapy in face-to-face settings to ask patients for details about their experience with remote CBT. Interviews were conducted by a trained psychiatrist within 4 weeks of completion. These interviews consisted of a series of open-ended questions, such as “Could you tell me something about your experience with the remote CBT?” to capture the unique features of the treatment experience by encouraging patients to talk about their general experience with the treatment. In addition, to evaluate the specific aspects of technical or privacy concerns and psychological resistance, the following questions were asked: “Could you tell me if you had any concerns about the technical or privacy aspects of remote CBT?” and “Could you tell me if you had any negative psychological feelings about receiving remote CBT?”

Results

Five patients were enrolled and underwent CBT. All patients met the DSM-IV criteria for major depressive disorder and had no other comorbid DSM-IV Axis I diagnoses confirmed with M.I.N.I. The sociodemographic and clinical characteristics of the patients at baseline are shown in Table 1 . Patients 1 and 2 received a total of 20 sessions by remote CBT; patients 3, 4, and 5 received a total of 16 sessions by remote CBT. Patients 1, 2, 4, and 5 were prescribed the same dose of antidepressants throughout the CBT program, and patient 3 was not prescribed any antidepressant. Patients 4 and 5 partially used an online CBT tool. Patients 1, 2, 3, and 4 received the sessions via a videoconference system between our clinic (patient site) and another medical facility (therapist site), and patient 5 received sessions via a videoconference system between his home and our clinic (therapist site) to maintain physical distancing restrictions due to the COVID-19 pandemic.

Table 1.

Five Patients’ Sociodemographic Characteristics at Baseline

Age Gender Education, y Work status Marital status No. of depressive episodes Duration of index depressive episode, y Duration of current depressive episode, mo Used antidepressant Experience using videoconference
Patient 1 45 Female 16 Housewife Married 3 4 5 Escitaroplam 10 mg Some
Patient 2 24 Male 16 On sick leave Unmarried 1 1 5 Miltazapine 45 mg Often
Patient 3 33 Male 15 Unemployed Unmarried 4 10 2 No antidepressant Some
Patient 4 45 Female 14 On sick leave Unmarried 2 17 9 Duloxetine 20 mg Little
Patient 5 40 Male 20 On sick leave Married 3 17 2 Escitaroplam 10 mg Very often

‘Experience using videoconference’ was assessed on a 5-point Likert scale (none, little, some, often, and very often). y = years; mo = months.

Patient Overview

Patient 1 was a 45-year-old female who engaged in freelance work, living with her husband and two children. She had increased anxiety and depressive mood due to the stress of child-rearing and a conflicting marital relationship. She received remote CBT, which helped in sorting out her problem, and engaged in behavioral activation such as talking with friends to assuage her problems because she appeared to have difficulty discussing problems with her husband. Following the completion of CBT, her depressive symptoms improved, as measured by BDI-II scores (from 36 to 11), and treatment was terminated after a few months.

Patient 2 was a 24-year-old male who was employed by an informative technology company. He was placed on a leave of absence from work because stress had built up in his relationship with his boss and he had long working hours after his transfer. He received remote CBT, in which he worked on problem-solving, decided to resign from his job, and underwent mainly behavioral activation. His depressive symptoms on BDI-II scores decreased from 30 to 14, but there remained residual symptoms and he avoided returning to work; therefore, additional CBT treatment focusing on rumination was conducted. He continued to receive antidepressants from a nearby clinic after completing two sets of CBT programs.

Patient 3 was a 33-year-old male who had spina bifida complicated with auditory memory retention. He worked as a clinical laboratory technician and resigned from his job because he kept being reprimanded for his work performance. He started a job search, which depressed him as it reminded him of the stress he had experienced at his previous job. He came to our clinic seeking CBT, and the treating psychiatrist had judged that CBT treatment was more appropriate than medication. He started remote CBT, but his symptoms did not change because he had difficulty understanding the session with his therapist through verbal communication due to auditory memory retention in the first half of therapy. The delivery of the therapy was modified, and he especially learned assertion skills to express his emotions and wishes when stress builds up in his work. After he completed CBT, his depressive symptoms slightly decreased from 26 to 19 on the BDI-II score. He continued treatment with his treating psychiatrist and resumed his job search.

Patient 4 was a 44-year-old female who worked for a company as an administrative assistant. Her stress had built up from her marriage and work, so she took a leave of absence from work at 43 years of age. She was referred to our clinic and received remote CBT. The sessions focused on anxiety about returning to work and interpersonal problems, and she learned behavioral activation and cognitive restructuring. After she completed CBT, her depressive scores on BDI-II decreased from 53 to 40. She continued medication with her treating psychiatrist and returned to work 6 months after the sessions.

Patient 5 was a 40-year-old male working at a company as a system engineer. Following the COVID-19 pandemic, he was forced to introduce remote work. The unfamiliar work style and the increase in work hours caused his depression, and he took a leave of absence. He was referred to our clinic and received remote CBT at home. He returned to work at the beginning of therapy, and the therapy focused on reducing the workload and hours as well as cognitive restructuring. After the completion of CBT, his depressive symptoms on BDI-II decreased from 29 to 14. The patient continued to receive medication management at a nearby clinic.

Treatment Outcomes

Remote CBT treatment improved depressive symptoms in all patients, as measured by BDI-II scores. The mean score changed from 34.80 (range 26 to 53) at baseline to 19.69 (range 11 to 40) posttreatment. (Cohen’s d effect size 1.51) (Figure 1 ). At the posttreatment assessment, three out of five patients met the criteria for treatment response (baseline score reduction of 50% or more). No severe adverse events, such as hospitalization or suicide attempts, were identified. The level of patient satisfaction according to the CSQ-8J ranged from 19 to 32 at posttreatment, and the mean score increased at the posttreatment assessment (Figure 2 ). Four patients rated the level of working alliance (measured using the SRS) as relatively high throughout the entire treatment; however, patient 3 reported that it improved in later sessions when a better delivery method was devised. The mean score at the last session for all patients was 38.64 (Figure 3 ).

Fig. 1.

Fig. 1

Changes in depressive symptoms based on Beck Depression Inventory-II scores over time.Note. BDI-II scores at baseline, middle, and posttreatment. The mean score changed from 34.80 (SD 9.66) at baseline to 19.60 (SD 10.52) posttreatment. BDI-II = Beck Depression Inventory Second Edition; SD = standard deviation.

Fig. 2.

Fig. 2

Changes in patient satisfaction based on the CSQ-8J.Note. CSQ-8J scores at baseline and posttreatment. The mean score changed from 23.00 (SD 3.58) at baseline to 27.60 (SD 4.84) at posttreatment. CSQ-8J = The Client Satisfaction Questionnaire 8 item Japanese version; SD = standard deviation.

Fig. 3.

Fig. 3

Level of working alliance based on the Session Ratings Scale at each session.Note. Remote CBT was administered to all patients. The mean Session Rating Scale score for all patients at the last session was 38.64 (SD 1.56). SRS = Session Rating Scale; SD = standard deviation; no. = number.

Patient Feedback

Overall Feedback on the Treatment

Most patients were generally satisfied with the treatment and regarded it positively. They felt that it was helpful and, in particular, the patients were highly content with the accessibility of the treatment: they could receive CBT from a distance, and the patient who received therapy at home did not have to visit a clinic. Below are some excerpts of patient feedback.

“I was satisfied with the treatment. I thought it was great that I could get treatment from a specialist in a distant location.“ ”I didn't have to come to the hospital weekly, so it reduced my fatigue and physical burden. Also, it was easy to make a schedule with work and CBT.”

Patient 3 mentioned that he found it challenging to communicate in the remote sessions after he started the therapy because he had difficulty with auditory memory. However, therapy was started without thoroughly explaining the way to share the materials during the sessions, so he was frustrated with the sessions and discussed with the therapist this miscommunication. Their notes were frequently shared on the screen, and this improved understanding and enhanced communication. He eventually mentioned that this modification made the sessions run more smoothly. Further, patient 5, who was more familiar with information technology, introduced his personal documentation during the session, such as his mood chart and problem analysis chart in Excel files, and mentioned that this made the sessions progress more smoothly and helped him understand the CBT skills better.

“I was engaged with the session using tools that I am familiar with, such as Excel files and memos, which helped me explain my situation and understand the session better. It was easier to work through the treatment than in the face-to-face sessions I received before.” “I was able to work on the session using graphs which I made about the change of daily mood, which is still useful after I finish therapy.”

Furthermore, patients cited an initial face-to-face assessment as a positive factor that provided more comfort with remote therapy.

“I had a face-to-face interview before the therapy and was explained the diagnosis and treatment, which put me at ease to start remote therapy.” “Since I had a face-to-face interview beforehand, I was familiar with the therapist and felt comfortable to start the remote therapy.”

Technical and Privacy Settings

Patients who were familiar with information technology were not very concerned with the technical and privacy issues of using videoconference. They were more interested in remote therapy, irrespective of the potential privacy risk. On the other hand, some patients mentioned technical difficulties and privacy concerns as negative aspects of the treatment, and they showed more concern at the initial assessment. However, they stated that they no longer cared about these concerns, in particular after the first session and as the sessions progressed.

“Before starting therapy, I was very concerned about the technical and privacy issues, such as whether there would be problems with the connection or audio, whether there would be other people outside the screen, or whether I would be recorded without asking. However, after the first session, I was able to work without any worries and focus on the sessions.”

Psychological Reaction Toward the Remote Settings

Some patients stated that they were resistant to videoconference before starting therapy. This is because they were concerned that they might feel separated from their therapists or express their emotions less than face-to-face, so they could not focus on their sessions. However, these concerns diminished, especially after the first session and as the sessions progressed. Some of the patients also mentioned that the initial face-to-face assessment was particularly positive, which provided more comfort with remote therapy. One patient who received therapy at home reacted positively and could work in his usual environment, making it easier for him to deal with stress in his daily life.

“I was worried about whether I would be able to communicate with my therapist like as face-to-face sessions, but there was no problem. I got used to the remote therapy after a few sessions and did not feel any difference from face-to-face.” “I was able to talk about problems in my daily life in the therapy as I received at home. I think it was good to have a combination of the treatment for depression by a psychiatrist and CBT by a therapist that deals with topics close to my daily life.”

Discussion

In this study, we sought to conduct a preliminary case series exploration of whether remote CBT is a potentially effective and acceptable treatment for major depression in Japanese clinical settings. We conducted more than 16 sessions of remote CBT on five patients who were in their 20s to 40s, on sick leave or unemployed, and continued the same antidepressant medication without changing the dose during the course of CBT. Favorable outcomes were demonstrated with regard to improvements in depression and patient satisfaction. In addition, the working alliance level, as measured with the SRS, was rated as high in four out of five cases, and the remaining patient rated it as high in later sessions after modifying the delivery methods, suggesting that remote CBT for major depression is feasible in Japanese clinical settings.

With regard to depression outcomes, all five patients showed clinical improvement based on the BDI-II score changes. Our effect size of 1.51 estimated with the BDI-II is comparable with the result of that reported by Griffiths (d = 1.18), who conducted a single-arm study of remote CBT for major depression and anxiety disorders (Griffiths et al., 2006). Furthermore, the treatment was well-tolerated in our study. All patients received more than 16 CBT sessions, which was better than a previous CBT study which had a 20% dropout rate (Stubbings et al., 2013). Notably, no serious adverse events were observed in our study. The level of overall patient satisfaction (as measured with the CSQ-8J) was relatively high at posttreatment, which agrees with the results of a prior study conducted among depressed veterans (Luxton et al., 2016). Further, the level of working alliance (as measured using the SRS score) at the last session for all five patients was satisfactory (mean score = 38.63, SD = 1.75), and exceeded the cut-off score of 36 (Duncan et al., 2003).

Through feedback, our patients expressed that they anticipated advantages of delivering therapy remotely, such as improved accessibility of CBT and reduced travel time and physical burden, which concurs with previous reports (Bleyel et al., 2020, Etzelmueller et al., 2018). Although some patients expressed concerns about remote delivery format, however, these concerns diminished as the sessions progressed. Other patients attested that the initial assessment visit in face-to-face format provided more reassurance for the remote therapy, which is also in line with previous studies (Etzelmueller et al., 2018, Lichstein et al., 2013, Swinton et al., 2009, Wilhelmsen et al., 2013). Accordingly, combining face-to-face sessions with remote sessions may improve patient satisfaction.

It is recommended that if the score is lower than the cut-off score on the SRS, therapists should ask patients for any comments or concerns about therapy and address them in therapy (Duncan et al., 2003, Miller et al., 2015). In our delivery format, we reviewed the SRS score at every session and had a discussion with the patient when the satisfaction score was low; the therapy was then tuned to enhance engagement. For example, the level of working alliance improved after tuning the therapy to cultivate a better understanding in patient 3 in our study. Periodic monitoring sessions using the SRS may have enriched the therapeutic relationship and improved depressive symptoms. Clinical guidelines for depression management recommend routine session monitoring and the quality evaluation of the patient-therapist relationship (American Psychological Association, Presidential Task Force on Evidence-Based Practice, 2006). Moreover, measurement-based care implementation leads to better treatment outcomes (Scott and Lewis, 2015, Trivedi et al., 2006).

Enhancing patient satisfaction may lead to higher acceptability of the treatment. Dunlop et al. (Dunlop et al., 2017) reported that the preferred treatment of the patient may not necessarily improve treatment outcomes but lead to higher retention rate. Further, literature shows that remote therapy is sufficiently satisfactory (Fletcher et al., 2018, Ruskin et al., 2004, Stubbings et al., 2013). Indeed, all five patients in our study achieved a high retention rate and sufficient satisfaction of the treatment. It is noteworthy that Kenter et al. (2015) asserted that simply adding a remote format to routine practice does not automatically lead to improved depression treatment. It is also necessary to reconsider the current National Health Insurance Scheme to enhance remote therapy in Japanese clinical settings. More research is needed to explore a better implementation strategy for delivering remote CBT.

Conclusion

In conclusion, quantitative and qualitative data from the current preliminary study indicate that remote CBT delivered via videoconference improves depressive symptoms and attains a relatively high level of patient satisfaction and working alliance. These results suggest that remote CBT is potentially feasible in Japanese clinical settings for treating major depression. Although there is little evidence regarding the optimal implementation strategy for remote CBT for major depression, the results of our study imply that it is preferable to continuously monitor and assess the level of working alliance, which may boost treatment engagement. Further research is warranted to investigate the acceptability of remote CBT more rigorously via videoconference for treating major depression.

Supplementary data to this article can be found online at https://doi.org/10.1016/j.cbpra.2022.04.002.

The following are the Supplementary data to this article:

Appendix A
mmc1.docx (14.6KB, docx)
mmc2.docx (16.8KB, docx)
mmc3.docx (926.3KB, docx)

Footnotes

This study was funded by Pfizer Health Research Foundation, the Inogashira Hospital Grants for Psychiatry Research, the Intramural Research Grant (grant No. 30-2) for Neurological and Psychiatric Disorders, Japan Agency for Medical Research and Development (AMED grant No. 16lk1010023h0001), and the MGH-SAFER grant. The funders had no role in the design and conduct of the study, collection, management, analysis, and interpretation of the data, preparation, review, or approval of the manuscript, and decision to submit the manuscript for publication.

☆☆

Dr. Nakagawa has received royalties from Igaku-Shoin and Kongo-Shuppan publishers for CBT textbooks. Dr. Kishimoto has received research grant from MICIN. Dr. Horikoshi has received royalties from Igaku-Shoin publisher for CBT textbooks. All the other authors declare that they have no conflicts of interest.

We would like to thank Dr. Dai Mitsuda for performing the study therapy, and Mses. Sayaka Odagiri and Machiko Kajiwara for assistance with the study management. We also would like to thank Editage for English language editing.

References

  1. Abbasy S. Importance and Advantages of Telepsychiatry in Mental Health During COVID 19 Pandemic. Psychology and Psychotherapy Research Study. 2020;4(2) doi: 10.31031/PPRS.2020.04.000585. [DOI] [Google Scholar]
  2. American Psychiatric Association . 4th ed., rev. ed. Author; 1994. Diagnostic and statistical manual of mental disorders. [Google Scholar]
  3. American Psychological Association, Presidential Task Force on Evidence-Based Practice Evidence-based practice in psychology. American Psychologist. 2006;61(4):271–285. doi: 10.1037/0003-066X.61.4.271. [DOI] [PubMed] [Google Scholar]
  4. Beck A.T. International Universities Press; 1976. Cognitive therapy and the emotional disorders.https://ci.nii.ac.jp/ncid/BA06899306 [Google Scholar]
  5. Beck A.T. Guilford Press; 1979. Cognitive Therapy of Depression.https://ci.nii.ac.jp/naid/10030012013/en/ [Google Scholar]
  6. Beck A.T., Steer R.A., Brown G.K. 2nd ed. The Psychological Corporation; 1996. The Beck Depression Inventory. [Google Scholar]
  7. Berryhill M.B., Culmer N., Williams N., Halli-Tierney A., Betancourt A., Roberts H., King M. Videoconferencing Psychotherapy and Depression: A Systematic Review. Telemedicine and e-Health. 2019;25(6):435–446. doi: 10.1089/tmj.2018.0058. [DOI] [PubMed] [Google Scholar]
  8. Bleyel C., Hoffmann M., Wensing M., Hartmann M., Friederich H.C., Haun M.W. Patients' Perspective on Mental Health Specialist Video Consultations in Primary Care: Qualitative Preimplementation Study of Anticipated Benefits and Barriers. Journal of Medical Internet Research. 2020;22(4):e17330. doi: 10.2196/17330. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Bromet E., Andrade L.H., Hwang I., Sampson N.A., Alonso J., de Girolamo G., de Graaf R., Demyttenaere K., Hu C., Iwata N., Karam A.N., Kaur J., Kostyuchenko S., Lépine J.P., Levinson D., Matschinger H., Mora M.E., Browne M.O., Posada-Villa J., Viana M.C., Williams D.R., Kessler R.C. Cross-national epidemiology of DSM-IV major depressive episode. BMC Medicine. 2011;9:90. doi: 10.1186/1741-7015-9-90. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Castro A., Gili M., Ricci-Cabello I., Roca M., Gilbody S., Perez-Ara M.Á., Seguí A., McMillan D. Effectiveness and adherence of telephone-administered psychotherapy for depression: A systematic review and meta-analysis. Journal of Affective Disorders. 2020;260:514–526. doi: 10.1016/j.jad.2019.09.023. [DOI] [PubMed] [Google Scholar]
  11. Cuijpers P., Berking M., Andersson G., Quigley L., Kleiboer A., Dobson K.S. A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Canadian Journal of Psychiatry. 2013;58(7):376–385. doi: 10.1177/070674371305800702. [DOI] [PubMed] [Google Scholar]
  12. Cuijpers P., Dekker J., Hollon S.D., Andersson G. Adding psychotherapy to pharmacotherapy in the treatment of depressive disorders in adults: A meta-analysis. Journal of Clinical Psychiatry. 2009;70(9):1219–1229. doi: 10.4088/JCP.09r05021. [DOI] [PubMed] [Google Scholar]
  13. Duncan B.L., Miller S.D., Sparks J.A., Claud D.A., Reynolds L.R., Brown J., Johnson L.D. The Session Rating Scale: Preliminary psychometric properties of a “working” alliance measure. Journal of Brief Therapy. 2003;3(1):3–12. [Google Scholar]
  14. Dunlop B.W., Kelly M.E., Aponte-Rivera V., Mletzko-Crowe T., Kinkead B., Ritchie J.C., Nemeroff C.B., Craighead W.E., Mayberg H.S. Effects of Patient Preferences on Outcomes in the Predictors of Remission in Depression to Individual and Combined Treatments (PReDICT) Study. American Journal of Psychiatry. 2017;174(6):546–556. doi: 10.1176/appi.ajp.2016.16050517. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Etzelmueller A., Radkovsky A., Hannig W., Berking M., Ebert D.D. Patient's experience with blended video- and internet based cognitive behavioural therapy service in routine care. Internet Interventions. 2018;12:165–175. doi: 10.1016/j.invent.2018.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. First M.B., Spitzer R.L., Gibbon M., Williams J.B. Biometrics Research Department, Columbia University; New York, NY: 2005. Structured clinical interview for DSM-IV-TR Axis I disorders: patient edition. [Google Scholar]
  17. Fletcher T.L., Hogan J.B., Keegan F., Davis M.L., Wassef M., Day S., Lindsay J.A. Recent advances in delivering mental health treatment via video to home. Current Psychiatry Reports. 2018;20(8):56. doi: 10.1007/s11920-018-0922-y. [DOI] [PubMed] [Google Scholar]
  18. Fujisawa D., Nakagawa A., Tajima M., Sado M., Kikuchi T., Hanaoka M., Ono Y. Jun 7). Cognitive behavioral therapy for depression among adults in Japanese clinical settings: A single-group study. BMC Research Notes. 2010;3:160. doi: 10.1186/1756-0500-3-160. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Gelenberg A.J., Freeman M., Markowitz J., Rosenbaum J., Thase M., Trivedi M., Van Rhoads R. American Psychiatric Association practice guidelines for the treatment of patients with major depressive disorder. American Journal of Psychiatry. 2010;167(Suppl 10):9–118. [Google Scholar]
  20. Giovanetti A.K., Punt S.E.W., Nelson E.L., Ilardi S.S. Teletherapy versus in-person psychotherapy for depression: A meta-analysis of randomized controlled trials. Telemedicine and e-Health. 2022 doi: 10.1089/tmj.2021.0294. epub ahead of print. [DOI] [PubMed] [Google Scholar]
  21. GBD 2017 Disease and Injury Incidence and Prevalence Collaborators Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: A systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2018;392(10159):1789–1858. doi: 10.1016/s0140-6736(18)32279-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Griffiths L., Blignault I., Yellowlees P. Telemedicine as a means of delivering cognitive-behavioural therapy to rural and remote mental health clients. Journal of Telemedicine and Telecare. 2006;12(3):136–140. doi: 10.1258/135763306776738567. [DOI] [PubMed] [Google Scholar]
  23. Hasin D.S., Goodwin R.D., Stinson F.S., Grant B.F. Epidemiology of major depressive disorder: Results from the National Epidemiologic Survey on Alcoholism and Related Conditions. Archives of General Psychiatry. 2005;62(10):1097–1106. doi: 10.1001/archpsyc.62.10.1097. [DOI] [PubMed] [Google Scholar]
  24. Hayashi Y., Yoshinaga N., Sasaki Y., Tanoue H., Yoshimura K., Kadowaki Y., Arimura Y., Yanagita T., Ishida Y. How was cognitive behavioural therapy for mood disorder implemented in Japan? A retrospective observational study using the nationwide claims database from FY2010 to FY2015. BMJ Open. 2020;10(5):e033365. doi: 10.1136/bmjopen-2019-033365. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Karyotaki E., Efthimiou O., Miguel C., Bermpohl F.M.G., Furukawa T.A., Cuijpers P., Individual Patient Data Meta-Analyses for Depression C., Riper H., Patel V., Mira A., Gemmil A.W., Yeung A.S., Lange A., Williams A.D., Mackinnon A., Geraedts A., van Straten A., Meyer B., Bjorkelund C., Knaevelsrud C., Beevers C.G., Botella C., Strunk D.R., Mohr D.C., Ebert D.D., Kessler D., Richards D., Littlewood E., Forsell E., Feng F., Wang F., Andersson G., Hadjistavropoulos H., Christensen H., Ezawa I.D., Choi I., Rosso I.M., Klein J.P., Shumake J., Garcia-Campayo J., Milgrom J., Smith J., Montero-Marin J., Newby J.M., Breton-Lopez J., Schneider J., Vernmark K., Bucker L., Sheeber L.B., Warmerdam L., Farrer L., Heinrich M., Huibers M.J.H., Kivi M., Kraepelien M., Forand N.R., Pugh N., Lindefors N., Lintvedt O., Zagorscak P., Carlbring P., Phillips R., Johansson R., Kessler R.C., Brabyn S., Perini S., Rauch S.L., Gilbody S., Moritz S., Berger T., Pop V., Kaldo V., Spek V., Forsell Y. Internet-Based Cognitive Behavioral Therapy for Depression: A Systematic Review and Individual Patient Data Network Meta-analysis. JAMA Psychiatry. 2021;78(4):361–371. doi: 10.1001/jamapsychiatry.2020.4364. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Kenter R.M., van de Ven P.M., Cuijpers P., Koole G., Niamat S., Gerrits R.S., Willems M., van Straten A. Costs and effects of Internet cognitive behavioral treatment blended with face-to-face treatment: Results from a naturalistic study. Internet Interventions. 2015;2(1):77–83. [Google Scholar]
  27. Kessler R.C., Chiu W.T., Demler O., Merikangas K.R., Walters E.E. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry. 2005;62(6):617–627. doi: 10.1001/archpsyc.62.6.617. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Larsen D.L., Attkisson C.C., Hargreaves W.A., Nguyen T.D. Assessment of client/patient satisfaction: Development of a general scale. Evaluation and Program Planning. 1979;2(3):197–207. doi: 10.1016/0149-7189(79)90094-6. [DOI] [PubMed] [Google Scholar]
  29. Lichstein K.L., Scogin F., Thomas S.J., DiNapoli E.A., Dillon H.R., McFadden A. Telehealth cognitive behavior therapy for co-occurring insomnia and depression symptoms in older adults. Journal of Clinical Psychology. 2013;69(10):1056–1065. doi: 10.1002/jclp.22030. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. López-López J.A., Davies S.R., Caldwell D.M., Churchill R., Peters T.J., Tallon D., Dawson S., Wu Q., Li J., Taylor A., Lewis G., Kessler D.S., Wiles N., Welton N.J. The process and delivery of CBT for depression in adults: A systematic review and network meta-analysis. Psychological Medicine. 2019;49(12):1937–1947. doi: 10.1017/S003329171900120X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Luxton D.D., Pruitt L.D., Wagner A., Smolenski D.J., Jenkins-Guarnieri M.A., Gahm G. Home-based telebehavioral health for U.S. military personnel and veterans with depression: A randomized controlled trial. Journal of Consulting and Clinical Psychology. 2016;84(11):923–934. doi: 10.1037/ccp0000135. [DOI] [PubMed] [Google Scholar]
  32. Malhi G.S., Mann J.J. Depression. Lancet. 2018;392(10161):2299–2312. doi: 10.1016/s0140-6736(18)31948-2. [DOI] [PubMed] [Google Scholar]
  33. Manchanda M., McLaren P. Cognitive behaviour therapy via interactive video. Journal of Telemedicine and Telecare. 1998;4(Suppl 1):53–55. doi: 10.1258/1357633981931452. [DOI] [PubMed] [Google Scholar]
  34. Miller S.D., Hubble M.A., Chow D., Seidel J. Beyond measures and monitoring: Realizing the potential of feedback-informed treatment. Psychotherapy. 2015;52(4):449–457. doi: 10.1037/pst0000031. [DOI] [PubMed] [Google Scholar]
  35. Mohr D.C., Vella L., Hart S., Heckman T., Simon G. The Effect of Telephone-Administered Psychotherapy on Symptoms of Depression and Attrition: A Meta-Analysis. Clinical Psychology. 2008;15(3):243–253. doi: 10.1111/j.1468-2850.2008.00134.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Nakagawa A., Mitsuda D., Sado M., Abe T., Fujisawa D., Kikuchi T., Iwashita S., Mimura M., Ono Y. Effectiveness of Supplementary Cognitive-Behavioral Therapy for Pharmacotherapy-Resistant Depression: A Randomized Controlled Trial. Journal of Clinical Psychiatry. 2017;78(8):1126–1135. doi: 10.4088/JCP.15m10511. [DOI] [PubMed] [Google Scholar]
  37. Nakao S., Nakagawa A., Oguchi Y., Mitsuda D., Kato N., Nakagawa Y., Tamura N., Kudo Y., Abe T., Hiyama M., Iwashita S., Ono Y., Mimura M. Web-Based Cognitive Behavioral Therapy Blended With Face-to-Face Sessions for Major Depression: Randomized Controlled Trial. Journal of Medical Internet Research. 2018;20(9):e10743. doi: 10.2196/10743. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. National Institute for Health and Care Excellence (2009). Clinical Guidelines: Depression in adults: recognition and management. https://www.nice.org.uk/guidance/cg90. [PubMed]
  39. Nishiura C., Nanri A., Kashino I., Hori A., Kinugawa C., Endo M., Kato N., Tomizawa A., Uehara A., Yamamoto M., Nakagawa T., Yamamoto S., Honda T., Imai T., Okino A., Miyamoto T., Sasaki N., Tomita K., Nagahama S., Kochi T., Eguchi M., Okazaki H., Murakami T., Shimizu C., Shimizu M., Kabe I., Mizoue T., Sone T., Dohi S. Age-, sex-, and diagnosis-specific incidence rate of medically certified long-term sick leave among private sector employees: The Japan Epidemiology Collaboration on Occupational Health (J-ECOH) study. Journal of Epidemiology. 2017;27(12):590–595. doi: 10.1016/j.je.2017.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Ong J.C., Dawson S.C., Mundt J.M., Moore C. Developing a cognitive behavioral therapy for hypersomnia using telehealth: A feasibility study. Journal of Clinical Sleep Medicine. 2020;16(12):2047–2062. doi: 10.5664/jcsm.8750. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Ono Y., Furukawa T.A., Shimizu E., Okamoto Y., Nakagawa A., Fujisawa D., Nakagawa A., Ishii T., Nakajima S. Current status of research on cognitive therapy/cognitive behavior therapy in Japan. Psychiatry and Clinical Neurosciences. 2011;65(2):121–129. doi: 10.1111/j.1440-1819.2010.02182.x. [DOI] [PubMed] [Google Scholar]
  42. Otsubo T., Tanaka K., Koda R., Shinoda J., Sano N., Tanaka S., Aoyama H., Mimura M., Kamijima K. Reliability and validity of Japanese version of the Mini-International Neuropsychiatric Interview. Psychiatry and Clinical Neurosciences. 2005;59(5):517–526. doi: 10.1111/j.1440-1819.2005.01408.x. [DOI] [PubMed] [Google Scholar]
  43. Rush A.J., Trivedi M.H., Wisniewski S.R., Nierenberg A.A., Stewart J.W., Warden D., Niederehe G., Thase M.E., Lavori P.W., Lebowitz B.D., McGrath P.J., Rosenbaum J.F., Sackeim H.A., Kupfer D.J., Luther J., Fava M. Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: A STAR*D report. American Journal of Psychiatry. 2006;163(11):1905–1917. doi: 10.1176/ajp.2006.163.11.1905. [DOI] [PubMed] [Google Scholar]
  44. Ruskin P.E., Silver-Aylaian M., Kling M.A., Reed S.A., Bradham D.D., Hebel J.R., Barrett D., Knowles F., 3rd, Hauser P. Treatment outcomes in depression: Comparison of remote treatment through telepsychiatry to in-person treatment. American Journal of Psychiatry. 2004;161(8):1471–1476. doi: 10.1176/appi.ajp.161.8.1471. [DOI] [PubMed] [Google Scholar]
  45. Scogin F., Lichstein K., DiNapoli E.A., Woosley J., Thomas S.J., LaRocca M.A., Byers H.D., Mieskowski L., Parker C.P., Yang X., Parton J., McFadden A., Geyer J.D. Effects of Integrated Telehealth-Delivered Cognitive-Behavioral Therapy for Depression and Insomnia in Rural Older Adults. Journal of Psychotherapy Integration. 2018;28(3):292–309. doi: 10.1037/int0000121. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Scott K., Lewis C.C. Using Measurement-Based Care to Enhance Any Treatment. Cognitive Behavioral and Practice. 2015;22(1):49–59. doi: 10.1016/j.cbpra.2014.01.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Shafran R., Clark D.M., Fairburn C.G., Arntz A., Barlow D.H., Ehlers A., Freeston M., Garety P.A., Hollon S.D., Ost L.G., Salkovskis P.M., Williams J.M., Wilson G. Mind the gap: Improving the dissemination of CBT. Behaviour Research and Therapy. 2009;47(11):902–909. doi: 10.1016/j.brat.2009.07.003. [DOI] [PubMed] [Google Scholar]
  48. Shafran R., Myles-Hooton P., Bennett S., Ost L.G. The concept and definition of low intensity cognitive behaviour therapy. Behaviour Research and Therapy. 2021;138:103803. doi: 10.1016/j.brat.2021.103803. [DOI] [PubMed] [Google Scholar]
  49. Sheehan D.V., Lecrubier Y., Sheehan K.H., Amorim P., Janavs J., Weiller E., Hergueta T., Baker R., Dunbar G.C. The Mini-International Neuropsychiatric Interview (M.I.N.I.): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry. 1998;59(Suppl 20):22–33. [PubMed] [Google Scholar]
  50. Simon G.E., Ludman E.J., Tutty S., Operskalski B., Von Korff M. Telephone psychotherapy and telephone care management for primary care patients starting antidepressant treatment: A randomized controlled trial. JAMA. 2004;292(8):935–942. doi: 10.1001/jama.292.8.935. [DOI] [PubMed] [Google Scholar]
  51. Smith K., Ostinelli E., Macdonald O., Cipriani A. COVID-19 and Telepsychiatry: Development of Evidence-Based Guidance for Clinicians. JMIR Mental Health. 2020;7(8):e21108. doi: 10.2196/21108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Souery D., Amsterdam J., de Montigny C., Lecrubier Y., Montgomery S., Lipp O., Racagni G., Zohar J., Mendlewicz J. Treatment resistant depression: Methodological overview and operational criteria. European Neuropsychopharmacology. 1999;9(1–2):83–91. doi: 10.1016/s0924-977x(98)00004-2. [DOI] [PubMed] [Google Scholar]
  53. Stubbings D.R., Rees C.S., Roberts L.D., Kane R.T. Comparing in-person to videoconference-based cognitive behavioral therapy for mood and anxiety disorders: Randomized controlled trial. Journal of Medical Internet Research. 2013;15(11):e258. doi: 10.2196/jmir.2564. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Swinton J.J., Robinson W.D., Bischoff R.J. Telehealth and rural depression: Physician and patient perspectives. Families, Systems, & Health. 2009;27(2):172–182. doi: 10.1037/a0016014. [DOI] [PubMed] [Google Scholar]
  55. Tachimori H., Ito H. Reliability and validity of the Japanese version of Client Satisfaction Questionnaire. Seishin Igaku (Clinical Psychiatry) 1999;41(7):711–717. [Google Scholar]
  56. Trivedi M.H., Rush A.J., Wisniewski S.R., Nierenberg A.A., Warden D., Ritz L., Norquist G., Howland R.H., Lebowitz B., McGrath P.J., Shores-Wilson K., Biggs M.M., Balasubramani G.K., Fava M. Evaluation of outcomes with citalopram for depression using measurement-based care in STAR*D: Implications for clinical practice. American Journal of Psychiatry. 2006;163(1):28–40. doi: 10.1176/appi.ajp.163.1.28. [DOI] [PubMed] [Google Scholar]
  57. Wiles N., Thomas L., Abel A., Ridgway N., Turner N., Campbell J., Garland A., Hollinghurst S., Jerrom B., Kessler D., Kuyken W., Morrison J., Turner K., Williams C., Peters T., Lewis G. Cognitive behavioural therapy as an adjunct to pharmacotherapy for primary care based patients with treatment resistant depression: Results of the CoBalT randomised controlled trial. Lancet. 2013;381(9864):375–384. doi: 10.1016/s0140-6736(12)61552-9. [DOI] [PubMed] [Google Scholar]
  58. Wiles N.J., Thomas L., Turner N., Garfield K., Kounali D., Campbell J., Kessler D., Kuyken W., Lewis G., Morrison J., Williams C., Peters T.J., Hollinghurst S. Long-term effectiveness and cost-effectiveness of cognitive behavioural therapy as an adjunct to pharmacotherapy for treatment-resistant depression in primary care: Follow-up of the CoBalT randomised controlled trial. Lancet Psychiatry. 2016;3(2):137–144. doi: 10.1016/s2215-0366(15)00495-2. [DOI] [PubMed] [Google Scholar]
  59. Wilhelmsen M., Lillevoll K., Risør M.B., Høifødt R., Johansen M.L., Waterloo K., Eisemann M., Kolstrup N. Motivation to persist with internet-based cognitive behavioural treatment using blended care: A qualitative study. BMC Psychiatry. 2013;13:296. doi: 10.1186/1471-244x-13-296. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix A
mmc1.docx (14.6KB, docx)
mmc2.docx (16.8KB, docx)
mmc3.docx (926.3KB, docx)

Articles from Cognitive and Behavioral Practice are provided here courtesy of Elsevier

RESOURCES