Treatment Outcomes of an Adolescent Intensive Outpatient Program for Depressed and Suicidal Youth

Abstract

Objective

Intensive outpatient programs (IOPs) provide an intermediate level of care for high-risk youth and were required to rapidly pivot to telehealth services amid the COVID-19 pandemic. This article aimed to compare symptom outcomes among adolescents treated in IOPs before vs during the pandemic and during the latter period to examine outcomes among adolescents who attended in-person vs via telehealth IOPs.

Method

Depressed adolescents recommended for IOPs between December 2014 and April 2022 (N = 1,152) completed validated self-reports throughout treatment. Patients treated during the pandemic completed treatment in person, via telehealth, or a combination of both. Mixed-effects models were used to examine changes in symptoms over time in treatment.

Results

Compared with pre-pandemic patients (n = 828), adolescents who presented for treatment after the pandemic onset (n = 324) reported increased depression, anxiety, and nonsuicidal self-injury (ps ≤. 006) at intake. Adolescents treated in IOPs (n = 855) demonstrated significant improvement over each week in treatment (ps < .001) across measures of depression (b = −0.79, 95% CI [−0.88, −0.71]), suicidal ideation and behavior (odds ratio = 0.59, 95% CI [0.55, 0.62]), and nonsuicidal self-injury (odds ratio = 0.51, 95% CI [0.46, 0.56]). Significant interactions between time and patient cohort indicated that the slopes of improvement for all outcomes were steeper during the pandemic (ps < .001). There were no differences in improvement of depression, suicidal thoughts and behaviors, or nonsuicidal self-injury for in-person vs telehealth treatment during the pandemic.

Conclusion

These findings indicate similar treatment response to both IOP treatment modalities for suicidal adolescents. Future research should discern factors leading to faster response after the pandemic onset.

Plain language summary

This study examined treatment response in an intensive outpatient program (IOP) for depressed and suicidal adolescents before and during the COVID-19 pandemic, and compared response based on IOP delivery modality (ie, in-person vs telehealth). Depressed and suicidal adolescents presenting for IOP after the pandemic onset reported more severe symptoms at IOP intake than those who presented prior to the pandemic; however, their symptoms improved more quickly. Youth showed similar response to treatment regardless of whether IOP was delivered in person, via telehealth, or in combination. Clinicians providing treatment for depressed and suicidal youth may consider community-based IOPs that incorporate evidence-based programming delivered either in person or virtually as a promising treatment option for those in need of more intensive treatment, hospital diversion and/or step-down care.

Adolescent depression and suicide continue to be urgent public health concerns despite decades of advocacy, research, and intervention. Recent epidemiological data indicate that the rates of adolescents endorsing persistent sadness or hopelessness and seriously considering suicide have increased from 2011 to 2021.¹ Moreover, emergency department visits for suspected adolescent suicide attempts are also increasing, with visits in early 2021 more than 50% higher compared with the same time period in early 2019, suggesting increases in emergency resource utilization.² Despite a mounting need for emergency mental health services, adolescent psychiatry inpatient beds remain limited, especially following the onset of the COVID-19 pandemic.³ Therefore, outpatient services designed to divert youth in crisis from hospitalization are crucial to mitigate the growing burden on an already constrained emergency mental health care system.⁴

Intensive outpatient programs (IOPs) are an intermediate level of care between weekly outpatient care and hospitalization that offer an alternative for patients who require more support than traditional outpatient behavioral health services, but either do not meet criteria for an inpatient admission or are unable to access services due to limited space. IOPs typically comprise group and individual therapy and psychiatric services totaling at least 9 hours of treatment per week in an outpatient setting. Thus, IOP treatment visits are longer in duration and more frequent than the typical outpatient structure of a single appointment once a week but still allow patients to reengage in typical activities (eg, school, work). As such, IOPs can serve as a much-needed step-down from higher levels of care or as a hospital diversion for depressed and suicidal youth,⁵ serving to treat and prevent worsening symptoms and reduce the need for costly inpatient services. Step-down services are particularly critical in the months immediately following discharge from inpatient care, during which time youth are at significantly increased risk for suicide and rehospitalization^6,7 and also less likely to engage in traditional outpatient services.⁸

Importantly, use of IOP services is associated with decreased emergency department volumes, health system costs, and hospitalizations.⁴ Yet, research on the effectiveness of such programs in reducing youth suicide risk is more limited. Many IOPs integrate and adapt efficacious treatments for adolescent depression and suicidality designed to be delivered in standard weekly outpatient settings.9, 10, 11 Thus, the specific benefits and challenges of implementation of efficacious treatments in more intensive care settings remain obscure.¹² The few studies that have documented acceptability and feasibility of in-person IOPs informed by dialectical behavior therapy (DBT) and cognitive-behavioral therapy (CBT) for depressed and suicidal youth show significant reductions in anxiety, suicidal ideation and attempt, nonsuicidal self-injury (NSSI), and depressive symptoms over the course of the IOP.^5,13, 14, 15 Therefore, preliminary data support both cost-effectiveness and symptom improvement among at-risk youth receiving IOP level of care.

The onset of the COVID-19 pandemic in March 2020 brought about new challenges for youth suicide prevention—most notably, a prompt shift from provision of in-person to telehealth services.¹⁶ Although telehealth offers clear benefits, including improving reach and accessibility, it also poses challenges for clinicians such as ensuring patient confidentiality and adapting in-person evidence-based curricula for remote implementation.¹⁷ Furthermore, implementation of an IOP via telehealth requires careful consideration of safety risks associated with treating depressed and suicidal youth in need of IOP level of care. Studies of youth attending a nationwide remote-only DBT- and CBT-informed IOP have demonstrated high attendance rates and significant reductions in depression, suicidal ideation, NSSI, and pediatric emergency visits following discharge.^18,19 Although these results are promising, it remains unclear how outcomes from telehealth-delivered IOPs compare with in-person IOPs for at-risk youth. To date, no studies have compared treatment outcomes from in-person vs telehealth-delivered IOPs for depressed and suicidal youth. Such information may help inform treatment recommendations for at-risk youth.

The Services for Teens At-Risk (STAR) clinic established DBT- and CBT-informed IOP services for depressed and suicidal adolescents in 2007; in 2014, the clinic implemented electronic data collection weekly throughout IOPs. During the initial onset of the COVID-19 pandemic, IOPs pivoted from in-person to telehealth sessions, with the clinic later offering both in-person and telehealth options once COVID-19 cases declined, all with minimal changes in IOPs. Considering the abrupt transition in delivery formats, the STAR clinic IOP data provide a unique opportunity to compare IOP treatment outcomes for depressed and suicidal youth when delivered in-person vs via telehealth. We thus aimed to examine treatment outcomes among at-risk adolescents attending IOPs before vs during the COVID-19 pandemic and when delivered in-person vs telehealth. Enhanced understanding of trajectories of symptom response to IOPs by delivery modality holds promise to inform approaches to optimizing outcomes for high-risk youth.

Method

Procedure

The STAR clinic provides a specialty IOP for acutely depressed and suicidal teens. Referral sources for the STAR IOP include self-referral, community outpatient providers, primary care physicians, emergency services, and inpatient psychiatric hospitals. Master’s-level clinicians conduct a semistructured clinical intake to determine psychiatric diagnoses, evaluate suicide risk and collaboratively create a safety plan; these data inform treatment recommendations and level of care. Eligibility criteria for IOP admission include recurrent suicidal ideation, recent suicidal behaviors, persistent self-injury, and moderate functional impairment as determined by a clinical team including a master’s-level clinician, a program supervisor, and a psychiatric attending physician. The recommendation and participation in an IOP were contingent on many factors including availability and family preference.

The STAR IOP is informed by CBT and DBT evidence-based approaches for treating youth suicide and depression.^5,20 Group therapy, led by master’s-level therapists, occurs 3 days per week for 3 hours each day and focuses on building skills to disrupt mood and anxiety symptoms.²⁰ Adolescents step out of group therapy to attend weekly 1-hour individual therapy visits with a master’s-level clinician and 30-minute medication management visits with a child and adolescent psychiatrist for a total of 9 hours of treatment weekly. Patients’ caregivers are invited to attend weekly medication management visits and weekly parent psychoeducation groups. Family therapy sessions are scheduled as needed.

The IOP curriculum comprises 18 rotating modules and incorporates a variety of skills from CBT and DBT (see Table S1, available online, for more information on IOP curricula).^5,20, 21, 22, 23, 24 Group leaders incorporate individualized goal setting and mindfulness skills in all groups and provide educational worksheets at the start of each group. In January 2020, the clinic therapists were trained in the youth version of Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TransS-C-Youth), and 2 IOP modules incorporating strategies adapted from this intervention were added to the curriculum rotation.²⁵

IOP Delivery via Telehealth

Once stay-at-home orders in the region were issued in March 2020, clinic staff created procedures and protocols for delivering IOP content via telehealth using the same schedule and structure as in-person treatment. Patients were required to turn their camera on at least once during each visit to confirm attendance, and group leaders encouraged them to keep their cameras on throughout treatment to confirm attendance, privacy, and attention. Group leaders shared curriculum handouts via e-mail and engaged patients in the content through screen sharing during group therapy. Clinicians implemented various games to increase engagement and virtual social interactions between group members, such as charades, trivia, or “show and tell” where group members shared a special item in their home and its story.

In fall 2021, local schools returned to in-person instruction; IOP treatment followed suit by reestablishing in-person treatment. Beginning October 4, 2021, patients and their families chose which treatment delivery they preferred. Personal illness or regional increase in COVID-19 cases necessitated occasional need for a combination approach (ie, both in-person and telehealth visits). The clinic delivered all IOP treatment via telehealth from December 29, 2021, to February 24, 2022, due to a surge in COVID-19 transmission in the region. From February 25, 2022 onward, patients and their families were again able to choose the in-person or telehealth IOP.

Measurement-Based Care

The STAR clinic began electronic data collection in 2014. All STAR IOP patients complete a battery of validated self-report measures assessing depression, anxiety, NSSI, suicidal ideation, and suicidal behaviors. The self-report battery is administered at intake and weekly thereafter as part of standard measurement-based care (MBC) treatment at STAR.²⁶ MBC data collection allows providers to identify trends in treatment response, collaborate efficiently with other treatment team members, and identify when other clinical recommendations are needed. In a 2018 survey, most STAR clinicians reported MBC as beneficial in directing patients’ care, attaining positive outcomes, and motivating patients (see Victor et al.²⁶ for further details on the STAR MBC protocol).

All self-report measures selected have proven reliable and valid for the age range of the IOP patients. In an effort to avoid response fatigue or abandonment, we opted for brief measures that are completed in a matter of minutes. The measures are widely used and therefore a resource in communicating with providers both inside and outside the clinic. However, the sensitivity in response to acute treatment is unknown given that the time frame was adapted to fit the weekly structure of IOP. Yet, the measures are available in the public domain, and use was not contingent on budgetary restrictions.

IOP patients complete self-report batteries at the beginning of the first IOP group session each week. All MBC questionnaires were adapted to ask about symptoms in the past week. IOP patients absent at the first group of the week complete the battery at their next attended group that week. Patients reporting NSSI, suicidal ideation, suicidal ideation with plan, or suicide attempt meet individually with a master’s-level clinician to assess risk and review and bolster their safety plan. Youth attending IOP in-person complete questionnaires on an electronic tablet; youth attending via telehealth access questionnaires by following a hyperlink sent to their e-mail address.

Measures

Psychiatric Diagnoses

Trained master’s-level clinicians assessed all patients in a semistructured clinical interview and provided diagnoses based on DSM-5.²⁷ Diagnoses were confirmed through a treatment team approach consisting of a case presentation to the program supervisor and psychiatric evaluation by a psychiatrist.

Self-Report Symptom Outcomes

The Short Mood and Feelings Questionnaire (SMFQ) is a 13-item questionnaire with satisfactory psychometric properties including diagnostic accuracy in assessing depression symptoms.^28,29 Total scores range from 0 to 26 with higher scores indicating greater depression severity. Patients completed a modified version of the Ask Suicide-Screening Questions (ASQ) inventory.³⁰ Modifications for STAR IOP include prompts for the presence of passive death wish/recurrent thoughts (question 1), nonspecific suicidal ideation (question 2), suicidal ideation with plan (question 3), suicide attempt (question 4), or NSSI (question 5) in the prior week. Most severe suicidal ideation and behavior was quantified using the most severe of ASQ items 1 through 4 endorsed (0 = no item, 1 = wish you were dead, 2 = thought of killing self, 3 = had a suicidal plan, 4 = made suicide attempt). Patients completed the 5-item Screen for Child Anxiety Related Disorder (SCARED); this short version displays similar psychometrics to the full 41-item SCARED in screening for child anxiety disorders.^31,32 Total scores on the SCARED short version range from 0 to 10 with higher scores indicating greater anxiety severity.

Participants

The total sample included 1,152 adolescents (76.9% female at birth, mean [SD] age = 15.80 [1.45] years) who completed an initial clinical assessment and were recommended for STAR IOP between December 2014 and April 2022, 828 before COVID-19 in-person (pre-COVID cohort; between December 2014 and March 16, 2020) and 324 during COVID-19 (during-COVID cohort; between March 16, 2020, and April 2022). There were 25 adolescents excluded from analyses due to initial disruptions in treatment delivery while the clinic developed and implemented the IOP telehealth protocol.

The majority of the patients in this report (n = 1,117 [97%]) provided informed consent to participate in a research registry allowing their MBC data to be used for research purposes. STAR adopted an honest broker system for use of MBC data in January 2022 (pertinent to n = 35 [3%] patients). Both procedures were approved by the University of Pittsburgh Institutional Review Board.

Adolescents who had at least 2 clinic visits (ie, initial clinical assessment plus 1 IOP visit) with complete self-report data over a period of 2 to 9 weeks were included in the treatment analyses, resulting in 855 adolescents (77.66% female sex at birth, mean [SD] age = 15.79 [1.43] years). Any IOP visits with self-report data ≥28 days beyond the prior visit were excluded, as this likely represented notable absences or adjunctive/intermediate treatment interventions. Similarly, data after 9 weeks of IOP treatment were excluded, as patients receiving treatment longer than this period suggested unusual conditions prolonging treatment (n = 21). Self-reports had to be completed within 2 weeks of the last self-report entry to be considered punctual. For patients who engaged in more than 1 course of IOP treatment, only the first treatment epoch was included in analyses.

Analyses

All analyses were conducted using R and Stata 18.0.^33,34 We present descriptive statistics to characterize intake and pre-COVID and during-COVID cohort samples; in instances when data were not available for all patients, percentages were calculated out of the available sample. Differences in sociodemographic and clinical characteristics and treatment engagement between patient cohorts before and during the COVID-19 pandemic were assessed using t and χ² tests. We conducted linear (SMFQ, SCARED), ordered logistic (highest of ASQ items 1-4), and logistic (ASQ item 5) mixed-effects models to examine symptom trajectories over weeks in the IOP. To compare treatment trajectories over time among the pre-COVID and during-COVID patient cohorts and treatment modalities in the during-COVID patient cohort only, we included an interaction term for treatment cohort (pre = 0, during = 1) or treatment modality (telehealth = 1, in-person = 2, combination = 3) by time. In all instances, we included a random intercept for subject and a random slope for weeks in treatment, as indicated by likelihood ratio tests, and covaried for age and sex assigned at birth. Lastly, we conducted sensitivity analyses controlling for values of each outcome variable at intake. We also dichotomized assessment measures using clinically significant cutoffs (SMFQ >11, SCARED >2) and conducted McNemar tests for paired proportions to compare patients at the start and end of treatment.

Results

Intent to Treat: Sociodemographic and Clinical Characteristics

Sociodemographic and clinical characteristics of patients referred to IOP who completed at least 1 assessment (N = 1,152, intent to treat) are presented in Table 1. Most patients were assigned female sex at birth (n = 886 [76.91%]) and identified as non-Hispanic (n = 1,088 [96.61%]) and White (n = 928 [80.56%]). Comorbidity was the norm, with 74.39% (n = 857) of patients presenting with 2 or more psychiatric disorders; nearly all patients presented for treatment with a depressive disorder (n = 1,069 [92.80%]), and most had an anxiety disorder (n = 783 [67.97%]). At intake, most patients scored in the clinical range on the SMFQ (n = 944 [82.73%]) and the SCARED (n = 888 [78.86%]). Moreover, greater than 75% of patients indicated having recent nonspecific suicidal thoughts (n = 862 [75.75%]), with 33.66% (n = 383) endorsing thoughts of suicide with a plan and 15.91% (n = 181) endorsing a suicide attempt in the month before intake.

Table 1.

Sociodemographic and Clinical Characteristics of Intent-to-Treat Sample at Intake

Variable	Overall sample (N = 1,152)		Pre-COVID (n = 828)		During-COVID (n = 324)		Statistic	p
Sociodemographic characteristics	Mean ± SD	(minimum-maximum)	Mean ± SD	(minimum-maximum)	Mean ± SD	(minimum-maximum)
Age, y	15.80 ± 1.45	(12.15-18.97) [n = 1147]	15.83 ± 1.43	(12.15-18.97) [n = 823]	15.71 ± 1.47	(12.99-18.64)	t1145 = 1.26	.209
	n	(%)	n	(%)	n	(%)
Female sex at birth	886	(76.91)	626	(75.60)	260	(80.25)	χ21 = 2.57	.109
Hispanic ethnicity	38	(3.39) [n = 1,121]	25	(3.14) [n = 797]	13	(4.01)	χ21 = 0.31	.581
Race							χ24 = 4.07	.397
Asian	26	(2.26)	21	(2.54)	5	(1.54)
Black	124	(10.76)	93	(11.23)	31	(9.57)
Multiracial	36	(3.13)	24	(2.90)	12	(3.70)
Undisclosed or unknown	38	(3.30)	16	(1.93)	22	(6.79)
White	928	(80.56)	674	(81.40)	254	(78.40)
Clinical presentation
Psychiatric diagnoses
Depression	1,069	(92.80)	760	(91.79)	309	(95.37)	χ21 = 3.95	.047
Bipolar	53	(4.60)	42	(5.07)	11	(3.40)	χ21 = 1.14	.287
Anxietya	783	(67.97)	541	(65.34)	242	(74.69)	χ21 = 8.93	.003
ADHD	88	(7.67)	60	(7.25)	28	(8.64)	χ21 = 0.46	.498
Eating	47	(4.08)	32	(3.86)	15	(4.63)	χ21 = 0.18	.671
PTSD	78	(6.80)	55	(6.64)	23	(7.10)	χ21 = 0.02	.883
Otherb	45	(3.92)	36	(4.35)	9	(2.78)	χ21 = 1.14	.286
Comorbidity diagnoses							χ22 = 14.97	<.001
0-1	295	(25.61)	237	(28.62)	58	(17.90)
2-3	781	(67.80)	535	(64.61)	246	(75.93)
≥4	76	(6.60)	56	(6.76)	20	(6.17)
Medication use	673	(58.57) [n = 1,149]	459	(55.50) [n = 827]	214	(66.46) [n = 322]	χ21 = 11.02	<.001
Self-report measures
ASQ		[n = 1,138]		[n = 816]		[n = 322]
Q1: Recurrent thoughts	949	(83.39)	675	(82.72)	274	(85.09)	χ21 = 0.77	.379
Q2: Thoughts of killing self	862	(75.75)	613	(75.12)	249	(77.33)	χ21 = 0.50	.481
Q3: Suicidal plan	383	(33.66)	272	(33.33)	111	(34.47)	χ21 = 0.09	.767
Q4: Suicide attempt	181	(15.91)	137	(16.79)	44	(13.66)	χ21 = 1.46	.227
Q5: NSSI	578	(50.79)	393	(48.16)	185	(57.45)	χ21 = 7.61	.006
Most severe ideation/behavior							χ24 = 5.60	.23
No ideation/behavior	135	(11.86)	103	(12.62)	32	(9.94)
Recurrent thoughts	127	(11.16)	92	(11.27)	35	(10.87)
Thoughts of killing self	430	(37.79)	306	(37.50)	124	(38.51)
Suicidal plan	265	(23.29)	178	(21.81)	87	(27.02)
Suicide attempt	181	(15.91)	137	(16.79)	44	(13.66)
	Mean ± SD	(minimum-maximum	Mean ± SD	(minimum-maximum	Mean ± SD	(minimum-maximum
SMFQ	17.10 ± 5.71	(0-26) [n = 1,141]	16.66 ± 5.77	(0-26) [n = 819]	18.24 ± 5.42	(0-26) [n = 322]	t1139 = −4.24	<.001
SCARED	4.72 ± 2.45	(0-10) [n = 1,126]	4.57 ± 2.49	(0-10) [n = 809]	5.09 ±2.32	(0-10) [n = 317]	t1124 = −3.20	.001

Note: For variables with missing data, sample sizes are listed in brackets. Percentages are calculated out of the number of individuals with complete measure data.

ADHD = attention-deficit/hyperactivity disorder; ASQ = Ask Suicide-Screening Questions; NSSI = nonsuicidal self-injury; PTSD = posttraumatic stress disorder; Q1-Q5 = question 1–question 5; SCARED = Screen for Child Anxiety Relative Disorders; SMFQ = Short Mood and Feelings Questionnaire.

^aAnxiety disorders category includes patients who met criteria for ≥1 of the following diagnoses: separation anxiety disorder, social anxiety disorder, specific phobia, panic disorder, generalized anxiety disorder, and obsessive-compulsive disorder.

^bOther disorders category includes patients who met criteria for ≥1 of the following diagnoses: psychosis, oppositional defiance disorder, substance use disorder, adjustment disorder, Tourette’s disorder, borderline personality disorder, and autism spectrum disorder. There were no more than 15 patients with any one of the listed diagnoses.

The intent-to-treat patients before the onset of the COVID-19 pandemic (n = 828) did not differ in sex at birth, age, ethnicity, or race compared with patients presenting for treatment during the pandemic (n = 324) (all ps ≥ .109). Compared with the pre-COVID cohort, during-COVID patients had a greater prevalence of depressive disorders (95.37% vs 91.79%, χ²₁ = 3.95, p = .047), anxiety disorders (74.69% vs 65.34%, χ²₁ = 8.93, p = .003), co-occurring psychiatric disorders (82.10% vs 71.38% with 2 or more psychiatric disorders, χ²₂ = 14.97, p < .001), and reported psychotropic medication usage (66.05% vs 55.43%, χ²₁ = 11.02, p < .001) at intake. Patients presenting during the pandemic also reported greater depression severity (SMFQ: 18.24 ±5.42 vs 16.66 ± 5.77, t₁₁₃₉ = −4.24, p < .001) and anxiety severity (SCARED: 5.09 ± 2.32 vs 4.57 ± 2.49, t₁₁₂₄ = −3.20, p = .001) at intake, with more during-COVID patients meeting clinical cutoffs for these measures (SMFQ: 87.89% vs 80.71%, χ²₁ = 7.85, p = .005; SCARED: 85.49% vs 76.27%, χ²₁ = 11.07, p < .001). During-COVID patients also reported greater prevalence of NSSI (57.45% vs 48.16%, χ²₁ = 7.61, p = .006). There were no differences in rates of suicidal thoughts or attempts or most severe ideation/behavior (all ps ≥ .227) or other clinical characteristics (all ps ≥ .286). See Table 1 for more details.

Treatment Sample: Sociodemographic and Clinical Characteristics

The treatment sample (ie, ≥2 assessments) included 855 youth with a mean (SD) of 6.19 (1.79) assessments (range 2–14) up to a 9-week period in IOP. Sociodemographic and clinical characteristics of the treatment sample are presented in Table S2, available online. Treated adolescents had a lower rate of eating disorders (2.81% vs 7.74%, χ²₁ = 12.50, p < .001) and posttraumatic stress disorder (5.61% vs 10.10%, χ²₁ = 6.34, p = .012) at intake compared with adolescents who had fewer than 2 total assessments (n = 297). Treated patients also reported greater depression severity (SMFQ: 17.53 ± 5.34 vs 15.87 ± 6.54, t_435.93 = 3.91, p < .001), anxiety severity (SCARED: 4.84 ± 2.43 vs 4.37 ± 2.49, t₁₁₂₄ = 2.79, p = .005), recurrent thoughts (ASQ Q1: 84.95% vs 78.91%, χ²₁ = 5.32, p = .021), and thoughts of killing self (ASQ Q2: 77.37% vs 71.88%, χ²₁ = 4.35, p = .037) at intake. There were no differences between the treated sample and adolescents with missing data in all other sociodemographic and clinical characteristics (ps < .167). Overall, patients treated in an IOP showed good rates of self-report completion with 71% (n = 607) not missing any self-reports, 26% (n = 222) missing 1, and 3% (n = 26) missing 2 or 3.

Compared with patients treated in an IOP before the pandemic (n = 605), patients who were treated in an IOP during the pandemic (n = 250) demonstrated clinical differences similar to those described for the intent-to-treat sample except for most severe ideation/behavior (χ²₄ = 12.54, p = .014). However, most severe ideation/behavior was greater among patients treated during COVID-19 (χ²₄ = 12.54, p = .014); compared with patients treated before the onset of COVID-19, more patients treated during COVID-19 endorsed suicidal plan (28.63% vs 21.14%) and fewer endorsed no ideation/behavior (6.05% vs 11.24%) as their highest level of ideation/behavior (Table S2, available online). Patients treated during COVID-19 had fewer total assessments over the course of the defined IOP treatment epoch compared with patients treated before COVID-19 (5.77 ± 1.82 vs 6.36 ± 1.75, t₈₅₃ = 4.47, p < .001). Given pandemic restrictions, patients during the pandemic engaged in treatment via different modalities with 90 (36%) completing the IOP via telehealth, 24 (9.6%) completing it in-person, and 136 (54.4%) completing it via a combination of telehealth and in-person sessions. There was a higher rate of missed self-reports in the during-COVID cohort compared with the pre-COVID cohort (44.8% vs 22.5%, χ²₁ = 42.80, p < .001).

Treatment Trajectories

There were significant effects of time for all self-report outcome measures. Symptoms of depression (b = −.79, 95% CI [−0.88, −0.71], t_676.24 = −17.87) and anxiety (b = −0.14, 95% CI [−0.16, −0.11], t_703.51 = −10.75) significantly decreased over weeks spent in the IOP (ps < .001). Most severe suicidal ideation and behavior (OR = 0.59, 95% CI [0.55, 0.62], z = −18.09) and odds of NSSI (OR = 0.51, 95% CI [0.46, 0.56], z = −13.06) also decreased over time (ps ≤ .001). Results did not change when covarying for intake values of each outcome measure. McNemar tests also indicated a significant decrease in the rates of clinically significant symptoms between the first and last visit (ps < .001) (Table 2) in the overall, pre-COVID, and during-COVID samples.

Table 2.

McNemar Tests Comparing Clinically Significant Scores at First and Last Visits

Outcome	Variable	Overalla					Pre-COVIDb					During-COVIDc
		First		Last		χ21	First		Last		χ21	First		Last		χ21
		n	(%)	n	(%)		n	(%)	n	(%)		n	(%)	n	(%)
Depression	MFQ >11	707	(85.2)	459	(55.3)	199.69	489	(83.4)	335	(57.2)	116.25	218	(89.3)	124	(50.8)	84.96
Anxiety	SCARED >2	655	(80.2)	548	(67.1)	63.96	448	(77.5)	387	(67.0)	30.75	207	(86.6)	161	(67.4)	36.48
Recurrent thoughts	ASQ: Q1	702	(85.0)	447	(53.3)	209.28	487	(83.7)	338	(58.1)	105.22	215	(88.1)	102	(41.8)	109.14
Suicidal ideation	ASQ: Q2	639	(77.4)	302	(36.6)	274.99	446	(76.6)	240	(41.2)	153.75	193	(79.1)	62	(25.4)	125.26
Suicidal plan	ASQ: Q3	284 (34.4)	(34.4)	73	(8.8)	158.44	197	(33.8)	63	(10.8)	90.69	87	(35.7)	10	(4.1)	71.43
Suicide attempt	ASQ: Q4	137	(16.6)	21	(2.5)	90.92	105	(18.2)	19	(3.3)	65.82	31	(12.7)	2	(0.8)	25.48
NSSI	ASQ: Q5	422	(51.1)	142	(17.2)	222.73	283	(48.6)	116	(19.9)	122.86	139	(57.0)	26	(10.7)	102.15

Note: McNemar Test of matched proportions. Only patients with score at both first and last visit included. All tests were significant at the p < .001 level.

ASQ = Ask Suicide-Screening Questions; MFQ = Short Mood and Feelings Questionnaire; NSSI = nonsuicidal self-injury; Q1-Q5 = question 1–question 5; SCARED = Screen for Child Anxiety Relative Disorders.

^aMFQ: n = 830; SCARED: n = 817; ASQ: n = 826.

^bMFQ: n = 586; SCARED: n = 578; ASQ: n = 582.

^cMFQ: n = 244; scared: n = 239; ASQ: n = 24

Comparisons in Treatment Trajectories Between Pre-COVID and During-COVID Patient Cohorts

There were interaction effects of weeks in treatment by patient cohort for all symptom outcomes (ps < .001). The during-COVID cohort demonstrated steeper decreases in depression (cohort × time: b = −.42, 95% CI [−0.61, −0.23], t_679.43 = −4.34) (Figure 1A), anxiety (cohort × time: b = −0.10, 95% CI [−0.15, −0.04], t_701.96 = −3.55) (Figure1B), most severe suicidal ideation and behavior (cohort × time: OR = 0.88, 95% CI [0.83, 0.93], z = −4.40), and NSSI (cohort × time: OR = 0.67, 95% CI [0.58, 0.79], z = −4.99) (Figure 3A) compared with the pre-COVID cohort (Figure 1, Figure 2, Figure 3). Simple slope analyses revealed significant reduction in symptoms over time for both cohorts for all outcomes (ps < .001). Results did not change when covarying for intake values of each outcome measure.

Figure 1.

Treatment Trajectories of Depression and Anxiety

Note:Predicted values of depression and anxiety over weeks in treatment by (A, B) patient cohort in all patients and (C, D) treatment modality in During-COVID patients only. Predicted values were estimated using linear mixed-effects modeling. Shading indicates 95% CI. IOP = intensive outpatient program; SCARED = Screen for Child Anxiety Related Disorders; SMFQ = Short Mood and Feelings Questionnaire.

Figure 3.

Predicted Odds of Nonsuicidal Self-Injury (NSSI) Over Time

Note:Predicted odds of NSSI over weeks in treatment (A) by patient cohort in all patients and (B) by treatment modality in During-COVID patients only. Predicted odds were estimated using logistic mixed effects modeling. Shading indicates 95% CI. ASQ = Ask Suicide-Screening Questions; IOP = intensive outpatient program.

Figure 2.

Predicted Odds of Suicidal Thoughts and Behaviors (STB) Over Time

Note:Predicted odds of STB over weeks in treatment (A) in all patients regardless of cohort, (B) by patient cohort, and (C) by treatment modality in During-COVID patients only. Predicted values were estimated using ordinal logistic mixed-effects modeling. Shading indicates 95% CI. IOP = intensive outpatient program.

Comparisons in Treatment Trajectories Across Modalities Among Patients During COVID-19

There were no effects of treatment modality (ie, in-person, telehealth, or combination) on the decrease of depression (modality × time: χ²₂ = 1.31, p =.520) (Figure 1C), most severe ideation (modality × time: χ²₂ = 2.03, p = .36), or NSSI (modality × time: χ²₂ = 0.09, p = .96) (Figure 3B) over time for the patients treated during COVID-19. There was a marginal effect of treatment modality on the decrease in anxiety (modality × time: χ²₂ = 5.57, p = .062) (Figure 1D) whereby patients who completed the IOP using a combination format improved at a steeper rate compared with patients who completed IOP virtually (virtual vs combination × time: b = −.12, 95% CI [−0.22, −0.02], t_208.73 = −2.31, p = .022). Simple slope analyses revealed significant negative effects of time for all treatment modalities for all outcomes (ps ≤ .001).

Discussion

Our study aimed to understand response to IOP treatment for depressed adolescents at-risk of suicidal behaviors, with a particular focus on comparing treatment trajectories after the onset of the COVID-19 pandemic and related shifts in modality of treatment delivery. Overall, self-report data indicate clinically significant depression and anxiety upon IOP initiation, with significant improvement over the course of IOP treatment. Similarly, suicidal thoughts and behaviors and NSSI improved with IOP treatment. When comparing clinical characteristics of patients before and during COVID-19, those presenting for IOP treatment after the start of the pandemic were diagnosed with higher occurrence of depression, anxiety, and co-occurring diagnoses as well as self-reported depression, anxiety, and NSSI. Whereas our results found significant improvement in outcomes across depression, anxiety, most severe suicidal ideation, and NSSI, patients treated during the pandemic appeared to respond faster to treatment than patients treated before the pandemic. These results did not change when covarying for baseline severity. There was no effect of treatment modality on depression, most severe ideation, or NSSI, although there was a nonsignificant effect on reduction of anxiety for patients receiving combination treatment as opposed to telehealth only.

Patients’ presenting concerns and subsequent improvement in clinical symptoms with the IOP across both in-person and telehealth modalities are in line with prior findings. A previous report before the pandemic suggests that in-person IOP is an acceptable treatment for at-risk youth,⁵ whereas findings during the pandemic support improvement in symptoms when IOP is delivered via telehealth.^18,19 In our study, at-risk adolescents displayed similar improvements in clinical symptoms with IOP regardless of whether patients attended the IOP in-person, virtually, or a combination of both.

At intake, we found patients treated in the STAR IOP during COVID-19 had higher rates of depression and anxiety disorders and greater likelihood of having comorbid diagnoses than patients treated before the pandemic. Studies on mental health during the pandemic also suggest that adolescent mental health worsened amid pandemic restrictions.35, 36, 37 Yet, patients responded to IOP treatment faster during COVID-19 than patients treated before the pandemic, even after accounting for intake severity. This improvement was consistent across treatment modalities (ie, treatment during COVID-19 included in-person, telehealth, or combination). These results conflict with a previous qualitative analysis of outpatient treatment studies conducted before the pandemic in which baseline depression severity predicted poorer treatment outcome.³⁸ It is possible that the higher level of care offered in IOPs for youth with more severe depressive symptoms accounts for the differential treatment response. These results also are consistent with regression to the mean.

The more rapid rate of response to IOP during COVID-19 may be understood by curriculum updates and schedule changes of adolescents implemented in response to the pandemic. The addition of sleep health curriculum to the STAR IOP in January 2020, just months before the shift to telehealth services, could possibly have influenced symptom improvement.^39,40 In addition to receiving sleep health information, adolescents adjusted to remote academic classes and treatment, which eliminated travel time to school and IOP, respectively, thus having more flexibility around sleep-wake times. Thus, patients treated during COVID-19 may have been able to implement sleep health strategies taught in the IOP more effectively during this period, resulting in accelerated treatment response. For some youth, shelter-in-place mandates were also accompanied by decreased academic expectations and negative or anxiety-provoking in-person peer interactions, thereby limiting additional factors contributing to maintenance of adolescent depression and anxiety symptoms and facilitating treatment response.^41,42 Future investigation of these factors in relation to the accelerated treatment response may provide insight to further enhance IOP treatment.

The pandemic quickly shifted many treatment programs, including STAR IOP, into less familiar treatment modalities. Our retrospective analyses provide support for modifications, including both telehealth and combination of in-person and telehealth modalities for delivering IOP, as both were associated with improvements in depressive and anxiety symptoms as well as suicidal thoughts and behaviors and NSSI in our at-risk youth. In fact, data indicate accelerated IOP treatment response during COVID-19. We did not covary for comorbidity, as comorbidity was the norm. We acknowledge the limitation of a community-based sample and challenge of examining treatment effects on specific psychiatric disorders. Further explorations for possible explanations of this result could enhance treatment for severely depressed patients with comorbid diagnoses, a population that has historically displayed poor outcomes.³⁸

These data were collected at a community clinical program for treatment-seeking youth. We did not have access to insurance status for patients and could not explore differences between youth with public vs private insurance. Furthermore, patients were mostly White and female at birth, limiting conclusions regarding treatment acceptability and response among more diverse samples. A similar study of an in-person youth suicide prevention IOP reported similar sociodemographics, with 86.3% White and 79.9% female at birth.⁵ A telehealth-only IOP study¹⁸ reported that only a small percentage of patients identified as male (6.8%) and did not report race or ethnic identity. Another study found that overall adolescents in a Yale New Haven Psychiatric Hospital IOP during the pandemic displayed increased telehealth attendance, though this was not observed for Medicaid-insured youth, Black youth, or Hispanic/Latinx youth.⁴³ Identification of ways to make suicide prevention treatment, and particularly more intensive treatments such as IOPs, acceptable and feasible for a more inclusive demographic should be prioritized, especially considering that the rate of suicide is increasing the fastest for Black male youth.⁴⁴

Our findings should be considered in light of study limitations, including reliance on self-reported measures of outcomes and lack of longer-term outcomes in the months following IOP completion. Analysis of clinician-rated outcome measures could contribute depth to our understanding of treatment effectiveness. Also, combined medication and CBT for depressed adolescents has been found to be more favorable than CBT alone, suggesting that medication can enhance psychotherapy effectiveness and vice versa.⁴⁵ However, we did not gather data on prescription of or adherence to psychiatric medication over the course of IOP treatment. We did find that significantly more patients presenting for treatment during COVID-19 who were referred for IOP level of care were already prescribed psychiatric medications at clinic intake. Thus, future studies may examine the contribution of psychiatric medications (type, dosage, adherence) to treatment response trajectories among youth in IOP.

Overall, IOP treatment is associated with improvements in depression and anxiety and reductions in suicidal thoughts and behaviors and NSSI. Positive outcomes were maintained when the program urgently adapted to telehealth delivery, and patients appeared to have responded at a faster rate during the COVID-19 pandemic regardless of IOP delivery modality. Future directions include examining how other factors such as sleep health, school stress, and medication may enhance or limit treatment access and participation and rate of improvement throughout treatment.

CRediT authorship contribution statement

Kelsey M. Bero: Writing – review & editing, Writing – original draft, Supervision, Project administration, Investigation, Data curation. Giana I. Teresi: Writing – review & editing, Writing – original draft, Visualization, Formal analysis. Giovanna Porta: Writing – review & editing, Visualization, Formal analysis, Data curation. Kimberly Poling: Writing – review & editing, Supervision, Project administration, Investigation. David A. Brent: Writing – review & editing, Methodology, Investigation, Funding acquisition, Conceptualization. Tina R. Goldstein: Writing – review & editing, Supervision, Methodology, Investigation, Funding acquisition, Conceptualization.