Telehealth treatment of patients with major depressive disorder during the COVID-19 pandemic: Comparative safety, patient satisfaction, and effectiveness to prepandemic in-person treatment

Abstract

Background

The COVID-19 pandemic impelled a transition from in-person to telehealth psychiatric treatment. There are no studies of partial hospital telehealth treatment for major depressive disorder (MDD). In the present report from the Rhode Island Methods to Improve Diagnostic Assessment and Services (MIDAS) project, we compared the effectiveness of partial hospital care of patients with MDD treated virtually versus in-person.

Methods

Outcome was compared in 294 patients who were treated virtually from May 2020 to December 2021 to 542 patients who were treated in the in-person partial program in the 2 years prior to the pandemic. Patients completed self-administered measures of patient satisfaction, symptoms, coping ability, functioning, and general well-being.

Results

In both the in-person and telehealth groups, patients with MDD were highly satisfied with treatment and reported a significant reduction in symptoms from admission to discharge. Both groups also reported a significant improvement in positive mental health, general well-being, coping ability, and functioning. A large effect size of treatment was found in both treatment groups. Contrary to our hypothesis, the small differences in outcome favored the telehealth-treated patients. The length of stay and the likelihood of staying in treatment until completion were significantly greater in the virtually treated patients.

Limitations

The treatment groups were ascertained sequentially, and telehealth treatment was initiated after the COVID-19 pandemic began. Outcome assessment was limited to a self-administered questionnaire.

Conclusions

In an intensive acute care setting, delivering treatment to patients with MDD using a virtual, telehealth platform was as effective as treating patients in-person.

1. Introduction

Depression is one of the leading causes of psychosocial morbidity worldwide (Liu et al., 2020) and is responsible for excess mortality (Laursen et al., 2016). Depression is one of the most frequently treated disorders in primary care (Finley et al., 2018), and the most frequently diagnosed disorder treated in outpatient psychiatric practice (Zimmerman et al., 2008).

The COVID-19 pandemic has been depressogenic (Bueno-Notivol et al., 2020; Jia et al., 2022; Kessler et al., 2022; Morin et al., 2021; Salari et al., 2020; Stephenson et al., 2022). Major disruptions in lifestyle due to social isolation, job loss, financial strain, and deaths of neighbors, family and friends are potential contributors to the increased levels of depression due to the pandemic. As one of the core elements of psychotherapeutic approaches towards treating depression is behavioral activation and increased social contact (Forbes, 2020; Nagy et al., 2020), the psychosocial limitations imposed by COVID-19 might make it more difficult to treat depression during the pandemic.

The pandemic prompted recommendations for social distancing and other safety measures resulting in a rapid transition from in-person to telehealth behavioral health visits (Montoya et al., 2022; Wright and Caudill, 2020). Even before the pandemic, telehealth services for mental health treatment had already been recognized as a cost-effective way to increase accessibility to evidence-based treatments (Gros et al., 2013; Ralston et al., 2019). Reviews of the research literature suggest that telehealth treatment is generally acceptable, feasible, and comparable to in-person mental health services in improving symptoms of psychiatric disorders (Drago et al., 2016; Shigekawa et al., 2018). In addition to similar rates of improved clinical outcomes, an equally strong therapeutic alliance can be developed with telehealth visits as with in-person therapy (Simpson and Reid, 2014).

It is uncertain how long the COVID-19 pandemic will last. It is also uncertain what role telehealth treatment will continue to play in the delivery of ambulatory behavioral health treatment. While some states have mandated an expansion of telehealth services and required private payers to continue to reimburse telehealth services at the same level as in-person treatment, other states have already rescinded, or allowed to expire, emergency orders that required equivalent telehealth reimbursements. The ongoing and future reimbursement for telehealth services is likely to depend, in part, on research determining whether telehealth treatment is as safe and effective as in-person treatment.

Several studies have found that the treatment of depression with synchronous telehealth methods to be equally effective as in-person treatment (Choi et al., 2012; Egede et al., 2015; Luxton et al., 2016; Mohr et al., 2012; Ruskin et al., 2004). There are, however, several limitations to this literature. Many studies excluded patients with suicidal ideation or recent suicide attempts (Choi et al., 2012; Egede et al., 2015; Luxton et al., 2016; Mohr et al., 2012). Some studies limited the age range of the patients (Choi et al., 2012; Egede et al., 2015; Luxton et al., 2016). One study was of elderly, low income, home bound patients in which the in-person treatment was delivered in the patients' home, and the telehealth treatment began with a single in-person session (Choi et al., 2014). Three studies were conducted in Veterans Affairs medical centers; therefore, the sample composition was predominantly male (Egede et al., 2015; Luxton et al., 2016; Ruskin et al., 2004). All studies provided protocol-driven, manualized psychotherapy with a fixed number of sessions; medication, if prescribed, was at a stable dose before study entry and was not changed during the study. Thus, the studies to date comparing telehealth and in-person treatment of depression deviate in many ways from how depression is treated in usual clinical practice.

Furthermore, all research comparing the effectiveness of synchronous telehealth and in-person treatment of depression has been conducted in the context of outpatient, individual treatment settings. There is a paucity of research, in general, assessing the comparative efficacy of telehealth and in-person delivery in partial hospital and other intensive treatment settings. In partial hospital and intensive outpatient settings the level of severity and the risk of self-harm and suicidal behavior is generally greater than in outpatient practice thereby raising concerns as to whether telehealth treatment could be provided while maintaining patient safety. This is particularly important in the treatment of patients with depression where suicidal behavior is a concern.

Because of the COVID-19 pandemic, most treatment in ambulatory behavioral health settings has transitioned to a virtual format due to public health recommendations and legal guidelines for social distancing (Lewnard and Lo, 2020; Wright and Caudill, 2020). While this has impelled clinicians across settings to quickly adapt and make significant changes to the structure of their service delivery, partial hospital programs (PHPs) and intensive outpatient treatment programs confronted distinct concerns and obstacles (Hom et al., 2020; Inchausti et al., 2020). For instance, in working virtually with acutely and severely ill psychiatric patients who require a higher level of care than usual outpatient treatment, enhanced appropriate risk management is critical. Furthermore, for group therapy-based programs, additional considerations regarding privacy, confidentiality, and technological limitations are needed.

In the Rhode Island Methods to Improve Diagnostic Assessment and Services (MIDAS) project we previously examined the effectiveness of our in-person partial hospital treatment program utilizing an Acceptance and Commitment Therapy (ACT) treatment model, a well-established, inherently transdiagnostic, behavior therapy (Morgan et al., 2020). ACT is a third wave behavioral therapy treatment that has been demonstrated to be of benefit for patients with depression (Coto-Lesmes et al., 2020; Twohig and Levin, 2017; Washburn et al., 2021). In our transition to a completely telehealth-based program as a response to the COVID-19 pandemic, we continued to evaluate the effectiveness of treatment in our PHP in a diagnostically heterogeneous sample (Zimmerman et al., 2021). We did not alter the inclusion and exclusion criteria to the program upon transitioning from in-person to telehealth treatment. Certainly, we were concerned about patient safety, and we instituted procedures in the telehealth program to minimize risk. The addition of risk management strategies, while essential, can be perceived as intrusive and burdensome and reduce patient satisfaction.

In the present report from the MIDAS project, we focus on patients with major depressive disorder (MDD) who were treated in our PHP program. We compare the safety, effectiveness, and patient satisfaction with PHP services delivered via telehealth to in-person PHP treatment provided to patients treated prior to the COVID-19 outbreak. Because of the depressogenic nature of the COVID-19 epidemic we speculated that treatment might be less effective. If telehealth treatment during the pandemic was not demonstrably different in effectiveness as in-person treatment delivered prior to the pandemic, this would strengthen the evidence base for the safety and effectiveness of telehealth treatment of depression.

2. Methods

2.1. Setting

The Rhode Island Hospital Adult PHP (RIH PHP) is an acute care setting serving a range of presenting concerns referred from various clinical settings. A multidisciplinary team of psychiatrists, psychologists, clinical social workers, postdoctoral fellows, and doctoral level graduate student therapists delivered the treatment. All intake assessments, individual therapy and psychiatry visits, and group therapy sessions were conducted virtually using HIPAA-compliant real-time audio and visual computer-based communication using the Zoom virtual platform, business account version.

A minority of the patients in the PHP were interviewed by a diagnostic rater who administered the Structured Clinical Interview for DSM-IV (SCID) (First et al., 1997) and the borderline personality disorder section of the Structured Interview for DSM-IV Personality (SIDP-IV) (Pfohl et al., 1997). Most patients who presented for treatment were not evaluated with the semi-structured interviews because of a lack of available interviewers but were instead diagnosed by board-certified psychiatrists.

We previously described in detail our adaption of the program from in-person treatment to a virtual format (Zimmerman et al., 2021). In brief, the inclusion and exclusion criteria for admission to the PHP were not changed when we transitioned from the in-person to the virtual program. The therapeutic orientation of the RIH PHP is based on ACT and related evidence-based psychotherapy techniques (e.g. CBT, dialectical behavior therapy) delivered consistent with ACT principles (Morgan et al., 2020). The length of treatment is flexible, based on patients' symptoms, functioning and engagement in treatment. Patients meet with a therapist and psychiatrist daily or nearly every day for individual sessions, as well as attend multiple group therapy sessions. The content and structure of the telehealth group therapy sessions was consistent with the in-person treatment (Morgan et al., 2020).

All elements of PHP treatment, including the intake assessment, individual therapy, psychiatrist meeting, and group therapy sessions were conducted virtually using real-time audio and visual computer-based communication using Zoom as a platform for telehealth. Additional safety procedures were implemented to address the unique challenges of delivering treatment via telehealth in an acute care setting in which patients frequently present with safety concerns, including suicidal ideation, self-injurious behavior, and aggressive ideation and behavior. To address the challenge of tracking patient attendance and location in a virtual program, a daily check-in procedure was implemented in which patients initiate a Zoom call with support and administrative staff who record their attendance. A program attendance record is sent to the full multidisciplinary team along with daily updated email and physical address information for each patient. As patients are not physically present in the program and thus cannot be accompanied throughout the process of transferring to inpatient or emergency room care, this information was necessary to send emergency support to patients' residences in instances of worsening suicidal ideation or safety concerns. Patients were also required to identify an emergency contact support person and to submit release of information paperwork upon beginning the program.

Additionally, in the virtual program, a therapist assumed a clinical and technical oversight role. This person monitored all group sessions and was available at all times throughout program hours to respond by phone or Zoom visit to urgent patient needs, including needs for urgent clinical support outside of individual sessions, assistance with troubleshooting technical problems, and assistance in connecting patients to their individual providers.

2.2. Measures

After completion of the initial evaluation by the psychiatrist, the patients were asked to complete the Clinically Useful Patient Satisfaction Scale (CUPSS) (Zimmerman et al., 2017). The partial hospital version of the CUPSS includes an item assessing overall global satisfaction with the initial evaluation (Please rate your overall level of satisfaction with your initial visit with your doctor. 0 = not at all satisfied; 1 = slightly satisfied; 2 = moderately satisfied; 3 = very satisfied; 4 = extremely satisfied) and an item assessing expectation of improvement in the program (After the evaluation I was more hopeful I would get better. 0 = definitely not; 1 = probably not; 2 = not sure; 3 = probably yes; 4 = definitely yes). On the day of discharge from the program, the patients completed a satisfaction scale on which they rated their overall satisfaction with treatment (Please rate your overall level of satisfaction with the program: 0 = not at all satisfied; 1 = slightly satisfied; 2 = moderately satisfied; 3 = very satisfied; 4 = extremely satisfied), whether they would recommend the program to a friend or family, and their overall level of improvement (Compared to how you were feeling when you first started the program, at the time of ending do you feel: 0 = no better; 1 = slightly better; 2 = moderately better; 3 = a lot better; 4 = very much better).

The primary outcome measure in the present study was the depression subscale of a modified version of the Remission from Depression Questionnaire (RDQ-M) (Zimmerman et al., 2014). In contrast to most measures that assess only symptom presence during the past week or two, the RDQ assesses a broader array of features reported by patients as relevant to determining treatment outcome–symptoms, functioning, coping ability/stress tolerance, positive mental health, and general well-being/life satisfaction. The domains covered on the RDQ were based on a literature review, our previous study of patients' ratings of the relative importance of 16 factors in determining remission from depression (Zimmerman et al., 2006), and two focus groups. The depression subscale includes 14 items assessing the DSM-5 symptom criteria of MDD. As previously reported, the depression subscale had high internal consistency and test-retest reliability (Zimmerman et al., 2013). We modified the RDQ to accommodate use with patients with varied diagnoses as well as patients with multiple psychiatric disorders. Nineteen items were added to the original 41-item scale. The modified 60-item measure included 25 symptom items, 5 coping ability/stress tolerance items, 12 positive mental health items, 10 functioning items and 8 general well-being/life-satisfaction items. The time frame of the measure is the past week. The items are rated on a 3-point rating scale (not at all or rarely true; sometimes true; often or almost always true). The items are scored 0, 1, and 2 with higher scores indicating more severe symptomatology, better coping ability, more positive mental health, better functioning, and greater well-being. The internal consistency (Cronbach's alpha) of the RDQ-M subscales at discharge in PHP patients was high in both the in-person and telehealth samples (symptom scale 0.94 and 0.94; coping/stress tolerance subscale 0.75 and 0.74; positive mental health subscale 0.93 and 0.94; functioning subscale 0.86 and 0.88; well-being/life satisfaction; subscale 0.92 and 0.93).

As part of our usual clinical procedure the patients were asked to complete the RDQ-M at admission and discharge. During the in-person program, the RDQ-M was handed to the patients by their treating clinicians. In the virtual program, patients were sent links to complete the scales online. The Rhode Island Hospital institutional review committee approved the research protocol, and all patients provided informed consent to allow us to use the information collected for research purposes. Consent in the in-person program was obtained on a paper consent form, whereas in the virtual program it was obtained on an electronically signed form.

2.3. Data analysis

We used t-tests to compare the telehealth and in-person groups on continuously distributed variables and chi-square statistics to compare categorical variables. For each of the RDQ-M subscales, paired t-tests were used to compare follow-up scores to baseline values, and effect sizes (Cohen's d) were computed. Consistent with prior recommendations, an effect size of 0.2 was considered small, 0.5 medium, and 0.8 large (Cohen, 1988). Pre-post change scores were used to compare the amount of change from admission to discharge on the RDQ-M subscales between the in-person and telehealth groups.

3. Results

3.1. Patient characteristics

During an 18-month period from May 1, 2020, to December 1, 2021, 294 patients with a principal diagnosis of MDD were treated for the first time in the RIH PHP. Patients who had been previously treated in the program were not included in our analyses. The sample included patients who dropped out during the course of treatment as this was one of the outcomes of interest.

During the 18 months from May 1, 2018, to December 1, 2019, 542 patients were treated for the first time in the program and had a principal diagnosis of MDD. The in-person and telehealth-treated groups were similar in age, gender, race, and marital status (Table 1 ). Significantly more patients in the telehealth group had graduated from a 4-year college.

Table 1.

Demographic characteristics of partial hospital patients with major depressive disorder treated in-person or in a telehealth format.

	In-Person (n = 542)		Telehealth (n = 294)		χ2	p level
Gender, % (n):					1.95	n.s.
Male	27.5	(149)	23.3	(68)
Female	69.4	(376)	74.0	(216)
Transgender or non-binary	3.1	(17)	2.7	(8)
Race, % (n):					6.80	n.s.
White	66.9	(362)	71.5	(208)
Hispanic	13.1	(71)	14.4	(42)
Black	7.8	(42)	7.2	(21)
Asian	3.5	(19)	1.7	(5)
Other	8.7	(47)	5.1	(15)
Education, % (n):					15.96	.001
Less than high school graduate	7.0	(38)	2.7	(7)
High school diploma or GED	60.2	(325)	51.7	(135)
4-year college degree	32.8	(177)	45.6	(119)
Marital status, % (n):					5.09	n.s.
Married	23.8	(129)	29.5	(86)
Living together	13.5	(73)	12.3	(36)
Widowed	3.7	(20)	2.1	(6)
Separated	3.1	(17)	2.1	(6)
Divorced	14.4	(78)	14.4	(42)
Never married	41.4	(224)	39.7	(116)
Agea, M (SD):	39.07	(15.26)	38.27	(14.17)	t = 0.732	.47

Data missing Gender: 2 telehealth; Race: 1 in-person, 3 telehealth; Education: 2 in-person, 33 telehealth; Marital Status: 1 in-person, 2 telehealth.

n.s. indicates not significant.

^aAge was compared by t-test.

About half of the patients in the telehealth and in-person groups were referred to the PHP by outpatient mental health clinicians (54.8 % vs. 47.8 %, X ² = 3.71, NS). Less than 20 % of the patients in the telehealth and in-person cohorts were referred from inpatient psychiatric units (12.9 % vs. 18.1 %, X ² = 3.72, NS), and >10 % were referred from emergency services (11.6 % vs. 16.4 %, X ² = 3.58, NS).

The patients treated by telehealth were twice as likely to have been interviewed with the SCID and SIDP-IV (62.3 % vs 33.1 %, X ² = 64.68, p < .01). When examining diagnostic frequencies, we limited our analyses to the patients interviewed with semi-structured interviews. Most patients were diagnosed with a comorbid disorder. The mean number of current disorders was significantly higher in the telehealth group (3.6 ± 1.5 vs. 3.2 ± 1.7, t = −2.12, p < .05). The patients in the telehealth cohort were significantly more often diagnosed with generalized anxiety disorder and borderline personality disorder (Table 2 ).

Table 2.

Current diagnoses of partial hospital patients treated in-person or in a telehealth format.

	In-person (n = 187)		Telehealth (n = 185)		χ2	p level
Anxiety disorders, % (n):
Panic disorder	11.2	(21)	12.4	(23)	0.13	.72
Panic disorder with agoraphobia	12.3	(23)	11.9	(22)	0.02	.90
Agoraphobia without panic	2.7	(5)	1.1	(2)	1.28	.26
Social anxiety disorder	31.0	(58)	27.0	(50)	0.79	.40
Specific phobia	5.3	(10)	5.4	(10)	0.00	.98
Posttraumatic stress disorder	25.7	(48)	33.5	(62)	2.75	.10
Generalized anxiety disorder	53.5	(100)	64.3	(119)	4.52	.03
Obsessive-compulsive disorder	5.3	(10)	5.9	(11)	0.06	.80
Body dysmorphic disorder	3.2	(6)	2.7	(5)	0.08	.77
Other anxiety disorder	11.2	(21)	15.1	(28)	1.24	.27
Substance use disorders, % (n):
Alcohol abuse/dependence	10.7	(20)	10.8	(20)	0.00	.97
Drug abuse/dependence	7.5	(14)	13.5	(25)	3.60	.06
Borderline personality disorder, % (n):	14.4	(27)	25.9	(48)	7.65	.006
Any eating disorder, % (n):	8.6	(16)	9.7	(18)	0.15	.70
Any somatoform disorder, % (n):	0.5	(1)	1.1	(2)	0.35	.56
Any impulse control disorder, % (n)a	9.6	(18)	10.3	(19)	0.04	.84
Any adjustment disorder, % (n):	1.6	(3)	1.1	(2)	0.19	.66

3.2. Patient satisfaction

The completion rate on the CUPSS after the initial evaluation by the psychiatrist was significantly lower in the telehealth cohort (55.1 % vs. 76.4 %, X ² = 40.29, p < .01). We compared the demographic and diagnostic characteristics of the patients who did and did not complete the CUPSS. In the telehealth sample, those who did not complete the initial satisfaction scale were more likely to have a diagnosis of alcohol use disorder (16.7 % vs. 6.5 %, X ² = 4.80, p < .05). In the in-person sample individuals who did not complete the initial satisfaction scale were more likely to have a diagnosis of drug use disorder (14.9 % vs. 5.0 %, X ² = 4.97, p < .05). There were no other significant differences between the groups who did and did not complete the CUPSS.

Significantly more patients in the in-person sample indicated that they were very or extremely satisfied with the initial evaluation (87.9 % vs. 76.6 %, X ² = 13.18, p < .01). The majority of patients in both the telehealth and the in-person groups were hopeful that they would get better (82.4 % vs. 76.4 %, X ² = 1.61, NS). At the end of treatment, about 90 % of the patients in the telehealth and in-person groups reported being very or extremely satisfied with their treatment (87.7 % vs. 90.5 %, X ² = 0.89, NS). More than 90 % of the patients treated in both formats indicated that they would recommend the treatment program to a friend or family member (93.9 % vs. 97.8 %, X ² = 0.63, NS).

3.3. Program completion

The average number of days attending the program was significantly higher in the telehealth program (14.5 ± 8.2 vs. 8.8 ± 4.9, t = 12.25, p < .01). When limiting this analysis to patients who had completed treatment, the average length of stay remained significantly greater in the telehealth program (16.4 ± 7.9 vs. 10.7 ± 4.4, t = 11.03, p < .01). The average number of days missed while in treatment was low in both the telehealth and in-person programs (1.8 ± 2.5 vs. 1.6 ± 1.9, t = 0.92, NS).

A significantly higher proportion of patients completed treatment in the telehealth program (77.5 % vs. 67.2 %, X ² = 9.57, p < .01). Consistent with this, a significantly higher percentage of patients were discharged from the in-person program due to nonattendance (13.4 % vs. 7.3 %, X ² = 7.0, p < .01). Few patients in the telehealth and in-person programs required hospitalization (1.0 % vs. 2.3 %, X ² = 1.56, NS). Likewise, only a small number of patients receiving telehealth and in-person treatment discontinued treatment because of dissatisfaction with the program (3.5 % vs. 3.2 %, X ² = 0.04, NS). No patients attempted or completed suicide during their treatment in the program.

3.4. Treatment outcome

A significantly higher percentage of in-person patients completed the RDQ-M at admission (89.9 % vs. 75.8 %, X ² = 29.67, p < .01) and at discharge (64.9 % vs. 55.2 %, X ² = 7.65, p < .01). Complete outcome data was available for a higher percentage of in-person patients (61.8 % vs. 42.8 %, X ² = 28.13, p < .01). In the telehealth group, patients with complete outcome data were more likely to have social anxiety disorder (36.9 % vs. 18.8 %, X ² = 7.61, p < .01).

There were no significant differences between groups on the RDQ-M subscales at admission. For both the telehealth and in-person programs the patients significantly improved from admission to discharge on each of the RDQ-M subscales, with large effect sizes found for most of the subscales (Table 3 ). Change scores from admission to discharge were significantly greater in the telehealth group for the depression (−10.0 ± 6.8 vs. -8.3 ± 6.1, t = 2.74, p < .01) and anxiety (−3.8 ± 3.2 vs. -3.2 ± 3.0, t = 2.0, p < .05) subscales. On the discharge satisfaction scale, the majority of patients in the telehealth and in-person groups indicated that they were a lot or very much better at discharge (73.6 % vs. 75.0 %, X ² = 0.80, NS).

Table 3.

Admission and discharge scores on Remission from Depression Questionnaire Modified (RDQ-M) subscales for patients with major depressive disorder treated in the partial hospital in-person or via telehealth.

RDQ-M subscale	Admission M (SD)	Discharge M (SD)	Paired t-test	Effect size (Cohen's d)
In-person group (n = 338)
Total symptoms subscale	31.7 (8.1)	17.6 (10.5)	t = 24.8, p < .01	1.35
Depression	18.2 (4.5)	10.0 (6.0)	t = 24.9, p < .01	1.35
Anxiety	7.4 (2.5)	4.2 (2.8)	t = 19.7, p < .01	1.07
Anger	3.0 (2.0)	1.3 (1.6)	t = 16.2, p < .01	0.88
Physical pain	3.2 (2.0)	2.1 (2.0)	t = 9.6, p < .01	0.52
Positive mental health	5.9 (5.0)	13.2 (6.2)	t = −20.4, p < .01	1.11
Functioning	7.6 (4.1)	12.5 (4.8)	t = −17.1, p < .01	0.93
Coping skills	2.8 (2.1)	5.6 (2.5)	t = −18.2, p < .01	1.00
Well-being	3.3 (3.3)	8.5 (4.5)	t = −20.0, p < .01	1.08
Telehealth group (n = 127)
Symptoms	33.0 (7.7)	16.0 (10.6)	t = 16.0, p < .01	1.41
Depression	18.9 (4.1)	8.8 (5.9)	t = 16.7, p < .01	1.48
Anxiety	7.6 (2.4)	3.7 (2.8)	t = 13.3, p < .01	1.18
Anger	3.3 (1.9)	1.4 (1.8)	t = 10.0, p < .01	0.88
Physical pain	3.2 (1.9)	2.1 (1.8)	t = 6.5, p < .01	0.58
Positive mental health	5.1 (3.7)	13.3 (6.2)	t = −13.1, p < .01	1.17
Functioning	7.8 (4.0)	12.6 (4.7)	t = −10.3, p < .01	0.91
Coping skills	2.8 (2.0)	6.2 (2.5)	t = −11.6, p < .01	1.06
Well-being	3.1 (2.9)	8.7 (4.5)	t = −13.5, p < .01	1.19

There were no significant differences between groups for scores on the RDQ-M subscales at admission.

Change scores from admission to discharge were significantly greater in the telehealth group for depression (10.0 ± 6.8 vs. 8.3 ± 6.1, t = 2.74, p = 0. < 01) and anxiety (3.8 ± 3.2 vs. 3.2 ± 3.0, t = 2.0, p = 0. < 05) subscales.

We also examined the 2 items on the RDQ-M that assessed suicidal ideation. From admission to discharge there was a significant reduction in the percentage of patients reporting death wishes (telehealth: 66.1 % vs. 14.2 %, X ² = 7.50, p < .01; in-person: 60.4 % vs. 29.0 %, X ² = 50.0, p < .001) and suicidal ideation during the past week (telehealth: 41.7 % vs. 10.2 %, X ² = 11.0, p < .001; in-person: 47.3 % vs. 19.2 %, X ² = 44.86, p < .001). At admission there was no difference between the telehealth and in-person groups in the frequency of patients reporting death wishes or suicidal ideation. At discharge half as many patients in the telehealth group continued to report death wishes (14.2 % vs. 29.0 %, X ² = 10.83, p < .001) and suicidal ideation (10.2 % vs. 19.2 %, X ² = 5.35, p < .05).

4. Discussion

In an intensive acute care setting consisting of daily group and individual therapy sessions as well medication treatment, providing care to patients with MDD using a virtual, telehealth platform was as effective as treating patients in-person. For both methods of delivering treatment, patients were largely satisfied with the initial diagnostic evaluation. Though fewer patients were satisfied with initial evaluation when it was conducted virtually, equal numbers were hopeful at admission that treatment would be beneficial. In both the in-person and telehealth groups there was a significant reduction in depressive symptoms and suicidality from admission to discharge, a significant reduction in symptoms of anxiety, anger, and pain, as well as improvement in functioning, coping ability, positive mental health, and general well-being. A large effect size of treatment was found in both treatment groups. Contrary to our hypothesis, the small differences in outcome favored the telehealth-treated patients. The length of stay and likelihood of staying in treatment until completion was greater in the telehealth treated patients.

In advance of the transition to the telehealth platform, our group discussed concerns about treating patients with MDD because of the associated suicide risk. In general, to qualify for partial hospital level of care patients need to be significantly functionally impaired, have failed to progress in outpatient treatment, and/or be at risk for self-harm. More than one-quarter of the patients were referred from inpatient units or emergency rooms. Thus, patients in a partial hospital program tend to be more severely and chronically ill than patients treated as outpatients. As we described in the Methods, we adopted procedural safeguards to reduce risk and attend to crises should they arise during the treatment day. Despite the severity and acuity of the patients' symptoms, no patient attempted suicide during the study. Moreover, the safeguards that were implemented did not reduce overall patient satisfaction with treatment.

Many studies have demonstrated that treatment delivered with synchronous audio and video transmission is as effective as in-person treatment; however, little research has examined telehealth treatment for patients requiring PHP level of care. Moreover, no prior study of PHP treatment has examined the acceptability, safety, and effectiveness of telehealth to treat patients with MDD. Unlike outpatient telehealth studies of individual therapy of MDD, many of the patients in the PHP reported suicidal ideation at admission to the program because the presence of suicidal ideation did not exclude patients from treatment. Precautions were taken to ensure that emergencies could be addressed in the virtually treated patients. Because a PHP is essentially an outpatient treatment setting, albeit more intensive than usual outpatient treatment in terms of the frequency of visits (5 days per week) and the duration of each visit (6 h per day), it is routine to assess risk and conduct safety planning interventions. We did not refuse admission of suicidal patients to our PHP, whether conducted virtually or in-person, unless a high level of intent was judged to be present whereupon the patient was referred for inpatient care. In fact, a small percentage of patients in both treatment formats were referred for inpatient admission though there was no significant difference between the formats in this regard. Likewise, there was no significant difference between the formats in the percentage of patients reporting suicidal thoughts at admission.

The present study was not a randomized controlled study comparing in-person and virtual PHP treatment. We transitioned to the virtual platform because of the COVID-19 pandemic, and we therefore examined the effectiveness of treatment in sequentially recruited cohorts. The only variable we controlled for was the time of year the patient was admitted to the PHP. Fortunately, there were few differences between the patient groups in demographic characteristics, comorbid psychiatric diagnoses, and baseline scores on the outcome measure. Of course, a randomized, controlled trial is the gold standard clinical trial design; however, it would be very costly to do such a study because it would require the doubling of clinical staff needed to run two simultaneous PHPs.

We were more successful collecting data when the patients were treated in-person. Handing paper-and-pencil questionnaires directly to patients by the treating clinician likely enhanced completion rates when compared to sending electronic links to surveys to be completed by patients online. We observed few differences in the demographic and clinical characteristics of the patients who did and did not complete the various measures. Social anxiety disorder was associated with the completion of outcome scales in the telehealth but not the in-person group. We are not aware of prior studies suggesting that social anxiety impacts the collection of outcome data.

In contrast to the lower level of cooperation in completing the questionnaires by the patients treated via telehealth, treatment participation was higher in telehealth-treated patients. Consistent with research in outpatient mental health clinics which found a lower “no show” rate for telehealth visits during the pandemic compared to in-person visits scheduled before the pandemic (Mishkind et al., 2021) we found that the patients in the virtual program more frequently completed treatment and more patients were discharged from in-person treatment due to nonattendance. We would hypothesize that nonattendance was greater in the in-person program because of transportation issues and difficulty waking up and being sufficiently motivated to arrive on time to the program.

Patients in the virtual program were treated for more days. The longer duration of treatment and greater completion rate in the telehealth group may have been artefacts of the COVID-19 pandemic. COVID-19 inspired social distancing recommendations increased social isolation and for some patients, attendance in the PHP was a primary source of social engagement. This may have increased some patients' desire to stay in the virtual program for a longer amount of time. The pandemic affected some patients' job status, with some having been furloughed or laid off—these patients were less pressured to be discharged in order to return to work. Early in the pandemic, health insurance companies' utilization review procedures were suspended thereby reducing pressure to discharge patients sooner than clinicians would have liked. An indirect contributor to the longer duration of treatment and greater treatment completion rate in the telehealth patients was the elimination of patient travel. Clinicians may have been more hesitant to discharge patients with some suicidal ideation in the telehealth program and thus kept them longer until the suicidal ideation more completely resolved. Patients treated virtually were diagnosed with more comorbid diagnoses and significantly more often diagnosed with borderline personality disorder and perhaps this resulted in longer duration of treatment because of lower effectiveness of treatment. Post hoc analyses did not find that in the pre-COVID in-person sample the number of diagnoses was associated with treatment completion. Finally, it is also possible that the response to treatment was slower with the telehealth format, and this resulted in a longer duration of treatment.

A limitation of the study is that outcome was only evaluated with self-administered questionnaires, and we did not include clinician rating scales. However, previous research from the MIDAS project found that the effect size of treatment was similar when based on self-report scales and clinician-administered measures (Zimmerman et al., 2018).

A limitation of comparing treatment effectiveness in sequentially treated cohorts is that circumstances unrelated to treatment efficacy could impact treatment outcomes. We adopted the telehealth format out of necessity due to the COVID-19 pandemic. Our uncertainty about the effectiveness and safety of virtual treatment during the pandemic motivated the study. The cohorts thus differ in two ways—how treatment was delivered and the altered social zeitgeist due to the pandemic. The pandemic has had a negative impact on the mental health of the general population (Bueno-Notivol et al., 2020; Jia et al., 2022; Kessler et al., 2022; Morin et al., 2021; Salari et al., 2020; Stephenson et al., 2022), as well as psychiatric patients (Dalkner et al., 2022; Fleischmann et al., 2021; Lewis et al., 2022). We speculated that the impact of the pandemic on social engagement, employment, education, and parental responsibilities would impede achieving positive treatment outcomes in the telehealth group. Fortunately, there was no evidence of inferior outcome in the telehealth cohort despite being treated while dealing with the COVID-19 pandemic. Nonetheless, to enhance confidence that a telehealth PHP is as effective as in-person treatment for MDD it will be important to compare treatment formats when pandemic-related issues have subsided.

Finally, a few words about the future. We are hopeful that pandemic will end in the not-too-distant future. However, we are unsure of how low a level of community spread will be required before returning to an in-person program largely based on group therapy. Several patients whom we have treated virtually have commented that they would not have presented for in-person treatment even if there was no pandemic. Some of these patients had medical illnesses that made in-person treatment attendance more difficult to manage. For some patients, limited transportation options made in-person treatment more difficult. Thus, we hope that telehealth partial hospital treatment is here to stay. Of course, decisions about how care is delivered in the future likely will be determined by insurance reimbursement. Hopefully, regulations will be adopted requiring equal compensation for treatment (Zimmerman, 2022). In the absence of such regulations, we fear that insurance companies will eliminate coverage for telehealth treatment thereby reducing access. If this occurs, it will be done despite considerable evidence that telehealth behavioral treatment is as effective as in-person care.

Role of funding source

None.

CRediT authorship contribution statement

Mark Zimmerman designed the study, wrote the first draft of the manuscript, and directed the data analysis. Catherine D'Avanzato contributed to the writing of the first draft and reviewed the draft of the entire manuscript and provided feedback that was incorporated into the final submission. Brittany King managed and analyzed the data, conducted the evaluations, and reviewed the draft of the manuscript and provided feedback to Dr. Zimmerman that was incorporated into the final submission.

Conflict of interest

None.