Measurement-based care in an adolescent partial hospitalization program: Exploring treatment progress and predictors using the top problems assessment

Abstract

This study used measurement-based care (MBC) to examine patient improvement and trajectories of change during treatment in an adolescent partial hospitalization program (PHP). The current study also explored whether youth trajectories in PHP varied based on demographic variables, symptom severity at intake, or presence of an anxiety or depressive disorder diagnosis. Participants included 124 youth (aged 13-18) with primarily anxiety and depression diagnoses who received care in a PHP. The Top Problems Assessment (TPA), an idiographic measure of patients’ chief complaints, was administered weekly alongside depression and anxiety symptom questionnaires to incorporate the consumer perspective. Results demonstrated that youth significantly benefitted from PHP treatment across all measures. The rate of TPA improvement slowed down over time whereas the rates of decline for the anxiety and depression questionnaires did not change statistically. Anxiety disorder status and more severe baseline anxiety symptoms were also independently associated with slower rates of improvement on TPA, but not on the symptom measures. Overall, these results suggest youth improved from treatment in this PHP, and that changes in youth-reported top problems may be more prominent at the beginning of treatment. Moreover, providers may want to consider treatment modifications, or perhaps monitor progress more closely, for youth who present with higher anxiety severity at PHP admission. This study demonstrates how incorporating MBC practices and personalized assessment in a PHP can enable exploration of who benefits most, under what conditions, and when.

Youth partial hospitalization programs (PHPs) are intermediate level of care settings that bridge the gap between inpatient and outpatient treatments. They provide intensive, highly structured, interdisciplinary behavioral health services to a patient population with significant psychopathology that warrants a higher level of care than traditional weekly outpatient therapy, but less intensive care than an inpatient unit. Youth in PHPs receive intensive multimodal day programming and return home at the end of each partial day. PHPs and other intensive treatment models are being increasingly utilized, in part due to the shift towards ambulatory treatment as the cost of inpatient mental health care rises (Clarke et al., 2022; Kiser et al., 1995; Leffler & Frazier, 2022; Robinson et al., 1999).

Effectiveness of PHPs

Generating a consensus about the effectiveness of PHPs has been complicated by the fact that there is heterogeneity in the patient population, interventions offered, and outcomes measured within these programs. Some PHPs treat a wide range of presenting populations, whereas others are designed for specific conditions (Kiser et al., 1995). As such, PHPs can vary widely on the interventions provided, such as cognitive behavioral therapy, dialectical behavioral therapy, and behavioral parental training (Leffler, 2020; Mochrie et al., 2020; Neuhaus et al., 2007; Sommerhalder et al., 2021).

In an effort to establish procedures to evaluate the effectiveness of PHPs more routinely, the Joint Commission implemented new standards in 2018 requiring healthcare organizations to demonstrate the effectiveness of their programs by incorporating formal assessments in the form of standardized instruments with well-established psychometric properties, such as the Patient Health Questionnaire-9 or Columbia Suicide Severity Rating Scale, more systematically (Commission, 2018; Posner et al., 2011; Spitzer et al., 1999). To date, even though the literature is still growing, research suggests that children and adolescents treated in PHPs show statistically significant improvements in symptom presentation, behavioral control, and levels of psychosocial functioning, compared to baseline (Kiser et al., 1991, 1996; Lavender et al., 2022; Leffler, 2020; Leffler et al., 2017, 2020; Leffler & D’Angelo, 2020; Leffler & Frazier, 2022; Lenz et al., 2014; Robinson et al., 1999; Sommerhalder et al., 2021; Thatte et al., 2013). Although the incorporation of control groups and randomized controlled trial methods would provide even stronger support, these initial studies demonstrate preliminary evidence for this level of care. Furthermore, follow-up studies indicate that treatment gains are maintained as long as one year after discharge, suggesting that the therapeutic benefits of PHPs are sustained even after youth return to their natural environments (Kiser et al., 1996; Thatte et al., 2013).

Moving Beyond Pre-Post Design Methodology

To date, the majority of the evidence supporting PHPs has relied on pre-post design methods where patients complete assessments at the start and end of an episode of care (Granello et al., 1999; Kotsopoulos et al., 1996; Lenz et al., 2014; Lenz & Del Conte, 2018). Although this methodology enables evaluation of treatment outcomes, assessing only at the start and end of treatment limits the ability to detect timing of treatment gains and evaluation of factors that might impact the rate of improvement. Moreover, pre-post design methods prevent the identification of cases in real time that are not responding well to treatment, and may lead to missed opportunities for treatment teams to make clinically-informed changes to the treatment plan. Assessing patients at more frequent intervals during a PHP admission has the potential to tailor treatment including earlier identification of treatment non-response.

Benefits of Measurement-Based Care

Over the last few decades, there has been a call to incorporate measurement-based care (MBC) into clinical settings (Jensen-Doss et al., 2020). MBC is the process of collecting data using formal measures to evaluate patient progress during their treatment, and then using this data to guide clinical decision making with patients (Fortney et al., 2017; Lavender et al., 2022; Lewis et al., 2019). MBC offers benefits not only at the patient level (Bickman et al., 2011; Gondek et al., 2016), but also for the providers and organizations implementing these measures. By allowing patients to be more regularly involved in their care, patients are more likely to better understand their condition, become cognizant of symptom deterioration, and improve communication with their provider (Fortney et al., 2017; Valenstein et al., 2009). Patients’ visualization of their data may also lead to heightened recognition of symptom improvement, increased feelings of hopefulness and adherence to treatment (Fortney et al., 2017; Leffler et al., 2017, 2020; Zimmerman & McGlinchey, 2008). In addition, monitoring patient outcomes can help guide treatment implementation, provide global information about the effectiveness of treatment, and allow opportunities to tailor treatment on an individual level (Lavender et al., 2022; Valenstein et al., 2009). On the health care level, treatment response measured through aggregate MBC data could inform accreditation organizations and payers about the value of the mental health services provided (Fortney et al., 2017; Scott & Lewis, 2015).

PHPs are an opportune setting for the implementation of MBC. In a PHP, patients are seen multiple times per week, making them easier to assess than in traditional weekly outpatient therapy models (Lavender et al., 2022). Typical length of stay (LOS) can be between 5 and 30 days in PHPs, but longer-term programs can last over 30 days (Kiser et al., 1995; Leffler et al., 2022). Early detection of cases that are decompensating can be key in acute PHPs given their compressed time frame. Irrespective of LOS, MBC practices can provide valuable information about the trajectories of improvement, which can help optimize cost effectiveness of these programs.

Although studies incorporating implementation of MBC demonstrate improved patient outcomes (Lambert et al., 2002; Lavender et al., 2022), these studies have mainly been conducted with adults in traditional outpatient clinics. Lavender et al. (2022) is one of the first papers describing the process of incorporating MBC into a youth PHP. Lavender and colleagues (2022) found that over two-thirds of the providers felt that MBC improved the quality of patient care and their interactions with patients and patients’ families. In addition, a majority of providers strongly agreed with the rationale of incorporating MBC in their treatment, and reported that it was not burdensome (Lavender et al., 2022). This study adds support to the notion that MBC is perceived by PHP staff as a valuable and acceptable practice.

Adding the Consumer Perspective to Progress Monitoring

Historically, symptom measures have been used to assess outcomes in PHPs (Kiser et al., 1996; Kotsopoulos et al., 1996; Lenz et al., 2014). While useful, it is also valuable to understand the problems patients would like to address in treatment from the consumer’s perspective and to assess progress on these chief complaints. Because symptom measures alone do not often capture those specific concerns, it is critical to find measures that efficiently identify client-nominated problems and assess them over the course of treatment. One such measure is the Top Problems Assessment (TPA), an idiographic, consumer-driven assessment in which a diagnostician guides patients and their caregivers to nominate up to three problems they are most concerned about, and then rate the severity of each problem on a 0 to 10 scale, with higher ratings indicating greater problem severity (Weisz et al., 2011). The design of the TPA creates both qualitative and quantitative data that enables evaluation of the type of problem as well as progress on mitigating each problem.

The TPA has multiple benefits that can augment existing symptom measures. It is free, brief, easy to administer, and may help providers build rapport with their patients by creating a working alliance to target patient-identified problems (Milgram et al., 2022). Previous findings with the TPA in outpatient and research settings suggest that this measure is psychometrically sound, with acceptable test-retest reliability, discriminant validity, convergent validity, and sensitivity to treatment change (Fitzpatrick et al., 2021; Milgram et al., 2022; Weisz et al., 2011). This instrument can also capture the concerns of a complex patient population, adding specificity to symptoms that would be missed in commonly used symptom measures. Given that this measure is not bound to specific diagnostic presentations, the TPA can be used across PHPs despite differences in the populations treated. If used across clinics within a system of care or even across different systems, this measure can help make direct comparisons between treatment programs and among systems. In an era where shared decision making with clients is valued and has been shown to increase patient satisfaction and participation in treatment (Langer & Jensen-Doss, 2018; Loh et al., 2007), tools like the TPA that can help characterize the population entering services, aid in treatment decisions, enable programs to evaluate impact on consumer priorities, and has the capacity to foster comparisons across programs, are invaluable.

Despite the potential utility of the TPA, this assessment has not been used widely with youth in acute psychiatric care settings. Chiu and colleagues (2022) adapted the TPA to a self-administered format for an adolescent PHP sample and coded the types of problems nominated by youth and their parents at admission. This study found that the TPA is a feasibly implemented measure within an adolescent PHP setting that can complement traditional symptom measures used in assessment. The top problems that were nominated by youth and caregivers were generally consistent with the most common diagnostic presentations seen in that PHP, namely diffuse anxiety and depression (Chiu, Desai, et al., 2022). This study found that approximately one-third of youth-caregiver pairs did not match on any target problem upon admission to a PHP (Chiu, Desai, et al., 2022), highlighting that youth and caregivers sometimes disagree on perceived reasons for seeking treatment. Although this study was the first to incorporate the TPA in a PHP setting, it solely examined the types of problems nominated by consumers and did not include results of TPA severity ratings as a measure of treatment progress.

Predictors of Treatment Response

As research grows on the effectiveness of PHP programs, it is valuable to study the impact of patient characteristics and clinical variables on treatment response, as current literature has varying conclusions. Studies from research outpatient clinics show that symptom severity at admission is associated with poorer outcomes.. For example, higher baseline anxiety symptom severity and poorer family functioning were consistent predictors of poorer anxiety outcomes across treatment conditions in a multi-site randomized controlled trial comparing CBT for youth anxiety against medication only and their combination (Compton et al., 2014). Several additional studies examining the effectiveness of CBT for OCD also found that higher baseline anxiety and OCD symptom severity predicted poorer response to treatment, worse remission rates and poorer symptom reduction severity based on CGI-Severity and CY-BOCS scales (Rech et al., 2020; Rudy et al., 2014; Torp & Skarphedinsson, 2017). Conversely, some studies conclude that less severe symptoms are related to poorer outcomes. As an example, the effectiveness of CBT for OCD found that lower baseline symptom severity predicted less progress in treatment (Krompinger et al., 2017). Last, studies have found no relationship between patient baseline characteristics and treatment outcomes. One study using the TPA to assess the benefits of the Unified Protocol among youth ages 6 to 18 years old with emotional disorders in a research clinic found that younger age was associated with greater child TPA improvement, but was not associated with child-reported measures of depression or anxiety symptoms (Milgram et al., 2022). Milgram and colleagues (2022) also found that symptom severity and other demographic variables did not significantly influence treatment benefit. Another study examining the effects of CBT for generalized anxiety disorder with youth had similar findings, with no effects related to age, gender, race, ethnicity, and baseline anxiety symptom severity (Bradford et al., 2011).

Predictors of Treatment Response in PHPs

Few studies have explored predictors of treatment response in PHPs and day treatment programs, and the literature has mostly focused on patient demographics and symptom severity. The studies that have explored symptom severity as a predictor have produced mixed results. Some findings suggest that youth who have less severe symptoms at the start of treatment fare worse. As an example, worse treatment outcomes were associated with lower levels of baseline externalizing symptom severity in a youth psychiatric day program (Balvardi et al., 2022). Conversely, some findings suggest that youth who have less severe symptoms fare better. A study examining behavioral improvements in an adolescent psychiatric day program found that higher levels of baseline symptom severity led to worse treatment outcomes (Milin et al., 2000). Although the direction of findings seems equivocal, symptom severity appears to be implicated as an important factor related to outcomes. For this reason, it is important to include symptom severity as a predictor. To our knowledge, the studies that have explored whether patient characteristics and demographic variables are related to treatment response status in PHPs have found non-significant effects (Balvardi et al., 2022; Kiser et al., 1996).

Study Aims

The main aim of the current paper was to assess whether patients improved on self-reported top problems during treatment in an adolescent PHP, and to evaluate whether trajectories of change on top problems were consistent with changes in self-reported symptoms of anxiety and depression. For this aim, weekly administrations of the TPA were used to measure treatment benefits from the consumer perspective. The secondary aim of this study was to determine if youth trajectories in the PHP varied based on demographic variables, symptom severity at intake, or presence of an anxiety or depressive disorder diagnosis.

Method

Adolescent Partial Hospital Program (PHP) Description

The adolescent PHP is an intermediate level of care setting for youth ages 13 to 18 years old that serves as a step-down from an inpatient hospitalization or as a strategy to divert an inpatient hospitalization. The primary goals in PHPs are safety, stabilization, generalization of skills, psychotropic medication evaluation, and linkage to aftercare services. The multidisciplinary team of providers consists of one psychiatrist, one psychologist, two social workers, several psychosocial rehabilitation counselors (i.e., creative art therapists, mental health counselors, creative arts therapists and occupational therapists with master’s degrees), and trainees in child and adolescent psychiatry, psychology and social work. At any time, the maximum census is approximately 10 patients. The average LOS is approximately two to three weeks and the majority of youth access services through insurance (Chiu, Falk, et al., 2022). At the time of this study, the PHP was fully in-person and operated on weekdays from 8:30am to 2:45pm. Patients receive two hours of daily school instruction with a State Department of Education certified teacher followed by group-based programming with a supervised lunch break. Each day, participants receive four 45-minute groups daily, which include skills groups run by social workers, psychologists, and child psychiatry residents, along with art, music, and drama therapy delivered by psychosocial rehabilitation specialists. Skills groups focus on cognitive behavioral skills (i.e., relaxation, cognitive restructuring, behavioral activation, goal setting) and skills drawn from dialectical behavioral therapy (i.e., distress tolerance, interpersonal effectiveness). Skills groups begin with an overview of the group rules, followed by homework review, an overview of the skill, in vivo practice, recap of the skill, and assignment of practice. Individual sessions always incorporate a safety assessment along with the focus of the session, which is personalized based on patient-specific treatment goals and progress. The treatment team meets before and after program each day to discuss patient care. Individual, family, and medication management sessions occur before or after program hours, or as pull-out sessions from an existing group. Family sessions occur at least once per week and individual sessions take place generally every day of the program. For a more detailed description of the program and a sample schedule from the PHP used in the current study, see Chiu, Falk, et al. (2022). More than three-fourths (78.09%) of patients admitted into the PHP access services through private medical insurance, 20.67% through Medicaid coverage, and a small minority, 1.23%, pay privately.

Procedures

Study procedures were approved by the academic medical center Institutional Review Board and conducted in accordance with ethical standards. All youth who were admitted to a PHP located in a metropolitan academic medical center in the Northeast United States between September 9th, 2016 and March 5^th, 2020 with at least one English-speaking caregiver, were invited to participate in this study. Youth and their caregivers were approached for the study during their treatment. Unless the youth was 18 years old, youth participants were included in the study only when both the caregiver signed consent and the youth provided assent to participate.

All youth and their caregivers were given a battery of questionnaires on the day of admission as a routine part of standard clinical care. Families completed these questionnaires on-site using paper and pencil. As part of the program, youth completed a subset of the questionnaire battery on a weekly basis during their PHP treatment episode. Measures were completed weekly, as opposed to selecting a more frequent assessment schedule, to be mindful of the burden on both patients and the treatment team, and to ensure that it was feasible with the limited research assistant support time available. A research assistant was available to answer participant questions about the measures. Results from these weekly questionnaires were scored in real time and results were plotted on a clinical dashboard; a visual tool that visually graphs the patients’ weekly measures (Chorpita et al., 2008). The clinical dashboard was presented at the next team meeting during which scores were reviewed and treatment plans were discussed. Treatment providers were instructed to share the clinical dashboard results with patients and their families, and to collaboratively discuss results in the context of the patient’s treatment plan and progress. When participants consented to participate in the current study, intake and weekly follow-up questionnaires, as well as clinical information from the electronic medical record, were retained and used for study purposes. The current study used completed measures from the intake and weekly follow-up assessments to explore the study aims. No incentives were given for participation.

A total of 360 unique families were admitted to the PHP between September 9th, 2016 and March 5^th, 2020. Out of this total, 172 (47.78%) did not supply consent for participation in the study. Of those that did not provide consent, 76 (48.19%) youth left the program before signing both the youth assent and caregiver consent, 14 youth (8.14%) had caregivers that did not speak English and therefore could not provide consent, and 16 youth (9.3%) actively declined to participate in the study due to privacy concerns. We do not have data on why the remaining 93 (54.07%) youth did not provide consent. A total of 188 youth consented for the study. Sixty four of these youth were excluded from analyses because they only had one assessment data point available, leaving a total of 124 youth that were included in analyses for the current study.

Demographics

Age, sex, ethnicity, race, and parent education were collected as part of a background questionnaire completed by youth participants and caregivers at intake or from the electronic medical record.

Youth Diagnostic Information

Clinician-rated diagnoses.

Diagnostic data were extracted from the participant’s discharge summary in the electronic medical record by postdoctoral psychology fellows who reviewed charts every three months as part of routine clinical reporting of cases in the PHP. Diagnoses therefore reflect the treatment team’s diagnostic impressions at the end of the participant’s stay in the program, which were informed by the team’s initial clinical assessment, individual and family sessions, collateral information and direct observations of the youth during the course of treatment within the PHP. Following data extraction, diagnoses were coded into diagnostic categories following the classification system used in the Diagnostic and Statistical Manual of Mental Disorders – 5^th Edition (American Psychiatric Association, 2013). Dichotomous variables were created to permit evaluation of whether the presence of an anxiety disorder or a depressive disorder diagnosis would predict treatment progress. Youth were grouped into those with an anxiety diagnosis and those without an anxiety diagnosis. Youth were also grouped into those with a diagnosis of major depressive disorder (MDD) and those without MDD.

Client-Nominated Top Problems

Youth- and caregiver-reported top problems were collected at intake using a modified version of the Top Problems Assessment (TPA; Weisz et al., 2011) that was adapted to a self-administered format. At intake, parents and youth were asked separately to identify up to three top problems they would like addressed in treatment. In contrast to procedures used in previous studies (Chorpita et al., 2017; Hoffman & Chu, 2015; Weisz et al., 2011), a diagnostician did not assist with the identification of top problems in the current study. Instead, youth and their caregivers were instructed to identify and write in their own chief concerns using free text. After nominating top problems, youth and their caregivers provided a rating of how severe each problem has been over the past week on a scale from 0 (not a problem at all) to 10 (a huge problem). During their treatment in the PHP, youth supplied weekly follow up ratings on how severe their nominated problems were using the same 0 to 10 scale. For a description of the types of problems nominated by youth and their caregivers as well as caregiver-youth agreement on these top problems, please see Chiu and colleagues (2022). The current study used only the youth-reported top problem ratings from the intake and the weekly follow-up assessments. Caregiver follow-up data was not collected due to logistical challenges. Specifically, the same caregiver was not always dropping off and picking up the adolescent at the beginning and end of the program day. Not having the same caregiver reliably available presented a challenge for data collection since data was collected via paper and pencil and research funds were not available to support calling caregivers weekly to obtain follow up scores.

Anxiety

The Screen for Child Anxiety Related Emotions Disorders, Child Version (SCARED-C).

The SCARED-C (Birmaher et al., 1999) is a 41-item self-report measure used to screen for anxiety symptoms in children. Items are scored on a 3-point Likert-type scale from 0 (not true or hardly every true) to 3 (very true or often true). The measure yields a composite total anxiety score as well as subscales for panic disorder, diffuse anxiety, separation anxiety, social phobia and school avoidance. Higher scores indicate greater impairment and symptom severity. Composite total anxiety scores of greater than 25 suggest the presence of a youth anxiety disorder. The SCARED-C has demonstrated good internal consistency (α = 0.7-0.9), test-retest reliability (p = 0.6-0.9), and discriminant validity (Birmaher et al., 1997; Muris et al., 2001). The current study used the SCARED-C composite score from the intake and weekly follow up assessments.

Depression

Patient Health Questionnaire – 9 (PHQ-9).

The PHQ-9 is a nine-item self-report measure used to screen for symptoms of depression (Spitzer et al., 1999). Youth rate how often they experience each symptom of depression over the past two weeks on a scale from 0 (not at all) to 3 (nearly every day). Total scores can be used as an indicator of symptom severity and a dichotomous scoring algorithm can indicate a probable diagnosis of major depression. The PHQ-9 is widely used in clinical settings with excellent reliability (α = 0.89) good validity, and sensitivity and specificity with adolescent samples (Allgaier et al., 2012; Borghero et al., 2018; De Los Reyes & Kazdin, 2005; Kroenke et al., 2001; Richardson et al., 2010). A total score of 11 is recommended as a clinical cutoff (Borghero et al., 2018; Richardson et al., 2010). PHQ-9 total scores from the intake and weekly follow-up assessments were used in the present study.

Analytic Plan

The first research questions involved exploring whether youth improved during their time in the PHP and assessing whether youth improved at a similar rate during the course of their PHP admission. To address this aim, we constructed separate linear mixed models (LMMs) to analyze whether youth improved on three outcome measures: TPA, PHQ-9 and SCARED-C respectively. Analyses included all youth who consented to participate in the study. For each outcome, we fit subject-level random intercept linear mixed effect model: controlling for both linear and quadratic time as fixed effects to examine rate of change.

The second aim was to explore predictors of improvement. Specifically, we examined whether youth trajectories on the outcome measures above varied based on demographic variables, symptom severity at intake or presence of MDD or anxiety diagnoses. To evaluate this aim, we constructed LMMs controlling for linear time, each individual covariate, and their two-way interaction with time. Each of the models included a subject-level random intercept with time as continuous and fixed effects varied by outcome measures. For modeling TPA score, we included fixed effects for time measured in weeks since intake assessment, gender, SCARED-C at intake, PHQ-9 at intake, having an anxiety disorder diagnosis, having a MDD diagnosis, and two-way interactions: time x SCARED-C at intake and time x anxiety diagnosis. For modeling SCARED-C score, we included fixed effects for time, gender, TPA at intake, PHQ-9 at intake, anxiety disorder diagnosis, and a MDD diagnosis. For modeling PHQ-9 score, we included fixed effects for time, SCARED-C assessment at intake, TPA at intake, anxiety disorder diagnosis, MDD diagnosis, and the following two-way interactions: time x SCARED-C at intake and time x TPA at intake.

We used the Akaike information criterion (AIC) to assist with selecting the appropriate statistical model. We chose the model that demonstrated the minimum AIC because a lower AIC value indicates better quality of fit. We selected the model that yielded the lowest AIC. Analyses were conducted using R 3.6.3 (2020-02-29). All statistical tests were 2-sided, performed at an overall 5% level of significance.

To assess the potential impact of missing data, we also ran analyses using participants who had complete data for all four timepoints (intake, week 1, week 2, and week 3). Results with this dataset were consistent with our findings below in the results section.

Results

Participant Characteristics

Table 1 shows the sample characteristics for the 124 youth who were admitted to an adolescent PHP located in a northeast metropolitan hospital. Participants ranged in age from 13 to 18 years (M = 15.5 years, SD = 1.41). Over half of the youth participants (64.5%) reported that their sex assigned at birth was female. Youth in this sample represented a broad range of diagnoses, including depressive disorders (71.4%), anxiety disorders (72.2%), attention-deficit/hyperactivity disorder (11.1%), obsessive-compulsive disorder (11.1%), bipolar disorder (4.8%), and other diagnoses present in less than 4% of the sample. More than two-thirds (76.2%) had more than one diagnosis; 76.2% had two diagnoses, 33.3% had three diagnoses and 10.3% had four diagnoses.

Table 1.

Patients Characteristics at Baseline

	n	%
Gender
Female	80	64.5%
Ethnicity
Hispanic/Latino/Spanish	30	24.2%
Not Hispanic/Latino/Spanish	89	71.8%
Unknown	1	0.8%
Missing	4	3.2%
Race
American Indian / Native American	2	1.6%
Asian	13	10.5%
Black / African American	11	8.9%
Multi-racial	14	11.3%
White / Caucasian	83	66.9%
Missing	1	0.8%
Parental Highest Education Level
No High School	2	1.6%
High School or GED	4	3.2%
College or Technical School	36	29.0%
Post Graduate Degree	50	40.3%
Missing	32	25.8%
Anxiety and Depression Diagnoses
Anxiety Diagnosis	89	71.8%
MDD Diagnosis	90	72.6%

Note. GED = General Education Diploma. MDD = Major Depressive Disorder.

See Table 2 for descriptive statistics on the Top Problems, PHQ-9, and SCARED-C by timepoint.

Table 2.

Top Problems, PHQ-9, and SCARED-C by Timepoint

	Intake	Week 1	Week 2	Week 3
TPA score
Mean (SD)	7.66 (1.58)	6.63 (2.19)	6.13 (2.37)	6.05 (2.39)
Min, Max	2.00, 10.0	0.00, 10.0	0.00, 10.0	0.00, 10.0
Missing	10 (8.1%)	9 (7.3%)	22 (17.7%)	63 (50.8%)
PHQ-9
Mean (SD)	15.2 (6.60)	13.2 (7.01)	12.7 (7.23)	12.3 (7.03)
Min, Max	0.00, 27.0	0.00, 27.0	0.00, 27.0	0.00, 27.0
Missing	9 (7.3%)	6 (4.8%)	22 (17.7%)	63 (50.8%)
SCARED-C
Mean (SD)	38.4 (14.9)	36.7 (15.7)	36.5 (17.5)	37.2 (20.7)
Min, Max	1.00, 77.0	1.00, 77.0	0.00, 82.0	0.00, 82.0
Missing	8 (6.5%)	8 (6.5%)	21 (16.9%)	67 (54.0%)

Note. N = 124. Percentages for calculating missing data used total n of 124 as the denominator. Data is considered missing for a variety of reasons: patient was discharged, patient left program against medical advice, or data was not collected even though patient was still enrolled in the PHP.

Aim 1: Do Youth Improve During PHP Treatment Based on Patient Reported TPA and Are the Findings Consistent with Self-reported Changes in their Anxiety and Depression Symptoms?

See Figure 1 for model estimates of TPA, SCARED-C and PHQ-9 over time during PHP treatment. To assess improvement in the TPA, a linear mixed effect model controlling for the linear and quadratic effects of time was fitted with a patient-level random intercept. Both the linear (−1.17, p = 0.00) and quadratic terms (0.19, p = 0.02) were statistically significant. These results suggest that youth improved on the TPA during PHP admission. On average, youth participants decreased by 1.80 points on the TPA from the start of treatment to the end. The rate of improvement on the TPA slowed down over time. To assess improvement on the symptom measures, linear mixed effect models with the linear and quadratic effects of time fit with a patient-level random intercept found that youth improved on both the SCARED-C (−2.22, p = 0.02) and the PHQ-9 (−1.89, p = 0.00) over time in the PHP. On average, youth participants decreased 4.52 points on the SCARED-C, and 3.57 points on the PHQ-9 from the start of treatment to week 3. However, unlike the TPA trajectory that slowed down over time, the SCARED-C (0.24, p = 0.45) and the PHQ-9 (0.23, p = 0.21) rates of decline did not change statistically over the course of treatment.

Figure 1. Model Estimates of TPA, SCARED-C and PHQ-9 Over Time During PHP.

Note. PHP = Partial Hospitalization Program. Measures were collected at admission and every subsequent week in treatment. Week 0 = PHP Admission, Week 1 = follow up assessment at 1 week, Week 2 = follow up assessment at week 2, Week 3 = follow up assessment at week 3.

Aim 2: Do Youth Trajectories Vary Based on Demographic Variables, Symptom Severity at Intake and Presence of MDD Or Anxiety Diagnosis?

To understand what factors were related to change in TPA, SCARED-C and PHQ-9 scores, we used LMM. To evaluate this aim, we constructed LMMs controlling for linear time, individual covariates and two-way interactions. Each of the models included a subject-level random intercept with time as continuous and fixed effects varied by outcome measures. For selecting variables with different effects on outcome over time, we fit separate LMM controlling for time, variable, and time x variable interaction. For each of the demographic variables, we fit separate mixed effect models with fixed effects for time, time x variable, and variable and patient-level random intercept. See Table 3 for the final model results.

Table 3.

Linear Mixed Models With Subject Level Random Intercepts For TPA, SCARED-C and PHQ-9

	Parameter	Estimate	Std. Error	df	t	p
TPA
	Male	−0.39	0.33	103.90	−1.17	0.24
	Week	−1.43	0.26	255.70	−5.52	0.00
	PHQ-9at intake	0.06	0.03	100.00	2.16	0.03
	SCARED-Cat intake	0.03	0.01	161.50	2.02	0.04
	Anxiety Dx	−0.10	0.41	187.80	−0.24	0.81
	MDD Dx	0.09	0.35	100.60	0.25	0.80
	Week x SCARED-C	0.01	0.01	251.60	2.10	0.04
	Week x Anxiety Dx	0.44	0.19	251.30	2.34	0.02
SCARED-C
	TPA at intake	2.04	0.83	101.54	2.46	0.02
	PHQ at intake	1.06	0.20	100.44	5.25	0.00
	Week	−1.63	0.34	233.11	−4.86	0.00
	Anxiety Dx	2.10	2.68	100.85	0.79	0.43
	MDD Dx	0.93	2.67	100.48	0.35	0.73
	Male	8.02	2.54	101.06	3.16	0.00
PHQ-9
	Male	0.63	1.06	107.89	0.59	0.55
	TPA at intake	0.90	0.36	161.83	2.45	0.02
	SCARED-C at intake	0.23	0.04	150.49	5.57	0.00
	Anxiety Dx	−0.69	1.07	107.68	−0.65	0.52
	MDD Dx	2.14	1.05	106.87	2.05	0.04
	Week	0.02	1.03	254.26	0.02	0.98
	TPA at intake x Week	−0.16	0.14	254.49	−1.10	0.27
	SCARED-C at intake x Week	0.00	0.01	248.98	0.00	1.00

Note. Week is in reference to the participant’s time in PHP treatment measured in weeks from intake. MDD = Major Depressive Disorder. Dx = Diagnosis. Anxiety and MDD diagnoses were extracted from the discharge summary report.

Top Problems as the Outcome

Time was still significantly associated with the decline of TPA during treatment after controlling for gender, symptom severity and presence of MDD or anxiety disorder at intake. Gender did not have a significant effect on TPA trajectory change and was not found to significantly differ at intake (F = 1.38, p = 0.24). The presence of MDD was also not significantly associated with TPA change. Higher PHQ-9 at intake was associated with higher TPA at intake but did not affect the rate of decline on the TPA. Having an anxiety disorder, however, slowed TPA improvement significantly (F = 5.48; p = 0.02). On average, having an anxiety disorder was related to a 44% slower rate of TPA improvement for every week in treatment with a 95% CI (0.07 to 0.81). Having a higher SCARED-C score at intake was both associated with having a higher TPA score at intake and was related to a significantly slower rate of TPA improvement (F = 4.14; p = 0.04). On average, the rate of decline slowed by 5 percent for every five-point increase on the SCARED-C intake score. See Figure 2 to see predicted TPA over time between youth within one standard deviation of the average SCARED-C score, with and without an anxiety diagnosis.

Figure 2. Predicted TPA Trajectory by Anxiety Diagnosis and Severity.

Note. Scores shown with and without an anxiety diagnosis and within one standard deviation of mean SCARED-C score (mean +/− sd: 23.55 and 53.31), separately. Week 0 = Admission, Week 1 = follow up assessment at 1 week, Week 2 = follow up assessment at week 2, Week 3 = follow up assessment at week 3.

SCARED-C as the Outcome

SCARED-C score changed at the same rate across various levels of clinical and demographic variables. Unlike the TPA pattern of results, having an anxiety disorder diagnosis did not affect the rate of change on the SCARED-C.

PHQ-9 as the Outcome

PHQ-9 score changed at the same rate across various levels of clinical and demographic variables. Neither having an anxiety disorder diagnosis nor a higher SCARED-C intake score had an effect on the rate of change on the PHQ-9 during treatment in the PHP.

Discussion

The purpose of this study was to assess whether patients endorsed improvement on top problem severity from intake to discharge using the TPA. This study also aimed to evaluate whether findings with the TPA were consistent with self-reported changes in symptoms, and explore whether trajectories of change varied based on a variety of factors. The TPA was administered weekly alongside standardized questionnaires to measure treatment benefits from the consumer perspective. While prior studies have substantiated the validity of the TPA and its sensitivity to detect change during treatment in community clinic and school settings (Weisz et al., 2011) and while receiving transdiagnostic treatment of emotional disorders within a research clinic (Milgram et al., 2022), the TPA has not been used with youth in intermediary levels of care to evaluate treatment benefit. In addition to assessing trajectories of change during treatment, prior studies have found associations between baseline characteristics, such as the type of mental disorder, severity of mental illness, gender, and age, and treatment outcomes (Balvardi et al., 2022; Compton et al., 2014; Granello et al., 2000; Krompinger et al., 2017; Leffler et al., 2017; Rech et al., 2020; Rudy et al., 2014; Torp & Skarphedinsson, 2017). Therefore, the secondary aim of this paper was to determine if treatment progress varied based on demographic variables, symptom severity at intake, or presence of an anxiety or depressive disorder diagnosis.

In this study, the TPA was selected in our implementation of measurement-based care for a variety of reasons. Unlike traditional measures of treatment outcome, this measure personalizes assessment by capturing priorities from the patient perspective, which can meaningfully augment measures of symptoms and patient functioning that are already commonly used by adding detail to problems that are identified generically in symptom measures. For example, a patient can elaborate on anxiety symptoms by indicating the context in which it appears (i.e., “anxiety about schoolwork”). This questionnaire is free, quick and easy to administer. Given the capacity of the TPA to capture a broad array of chief patient concerns, this measure can also be used in almost any treatment setting, thereby enabling comparisons between programs despite differing LOS, patient populations treated, and interventions provided. Prior studies have already compared TPA results across sites using benchmarking analyses (Cheron et al., 2022).

In the present study, youth experienced statistically significant changes across all study measures during their PHP admission. Patients decreased 1.802 points on the TPA, 4.523 points on the SCARED-C, and 3.565 points on the PHQ-9 from the start of treatment to week 3. Adolescents in our sample shifted from the moderately severe level of depression symptoms (score between 15 and 19) to the moderate level of depression symptoms (score between 10 and 14) on the PHQ-9 (Kroenke et al., 2010). Collectively, these data indicate that youth improve during PHP treatment on the chief concerns they nominate at the time of admission as well as on anxiety and depression symptom severity. Despite improving, it is important to note that scores on the PHQ-9 and SCARED-C at week 3 were still elevated above both the clinical cutoff of 10 for the PHQ-9 (Kroenke et al., 2010), and the score of 12 on the SCARED-C that has predicted remission in a prior study (Caporino et al., 2017).

This pattern of results is similar to findings from prior research on youth PHPs that have demonstrated that this level of care can be beneficial for youth across a variety of diagnostic presentations (Granello et al., 2000; Kiser et al., 1995; Lavender et al., 2022; Leffler et al., 2017; Lenz et al., 2014; Thatte et al., 2013). These findings are also similar to results from previous studies indicating that different types of therapy modalities can be beneficial, including CBT (Neuhaus et al., 2007), DBT (Leffler, 2020; Mochrie et al., 2020), and behavioral parent management (Sommerhalder et al., 2021). With respect to magnitude of change, the effect size for estimated TPA change after controlling for intake PHQ, SCARED-C, anxiety and MDD diagnostic status was 0.74 in the current study. The effect size was 0.58 for estimated PHQ-9 change after controlling for intake TPA, SCARED-C, and anxiety and MDD diagnosis. Moreover, the effect size for estimated SCARED-C change from admission to week 3 was 0.43 after controlling for intake PHQ, TPA, and anxiety and MDD diagnosis. These effect sizes, which hover generally between moderate to large effects, are comparable to prior studies in youth PHPs. Lenz et al. (2014) found that the estimated effect sizes for anxiety and depression were 0.55 and 0.77, respectively, indicative of medium to large effect sizes. Granello et al. (2000) found that the estimated effect size for anxiety was 0.87, with additional symptomatology such as psychotic behavior and conduct disorder ranging from 0.58 to 0.92.

When we looked at trajectories during the course of PHP treatment, we found that the rate of TPA slowed down over time whereas the rates of decline in the SCARED-C and PHQ-9 did not change statistically over the course of treatment. In the treatment literature, sudden gains refer to a phenomenon whereby patients experience rapid, sizeable changes observed between treatment sessions (Aderka et al., 2012; Shalom & Aderka, 2020; Tang & DeRubeis, 1999). In a study exploring sudden gains among children receiving services in community mental health settings, Dour et al. (2013) found that 20–42% of participants in their sample experienced at least one sudden gain during treatment on the TPA and that most sudden gains occurred early on in treatment. Similar to Dour et al.’s findings, a steeper TPA slope at the start of treatment (between admission and week 1) in our study sample suggests that youth might also experience early treatment sudden gains in intermediate level of care settings such as PHPs. In light of discoveries that sudden gains may have an effect on long-term symptom improvement (Dour et al., 2013), a future direction is to adopt Dour et al.’s criteria for sudden gain calculation, and explore whether sudden gains are indeed experienced by a subset of youth in PHP settings, whether these gains may also be related to long-term outcomes after receipt of PHP treatment, to evaluate predictors of sudden gains, and to see if these findings generalize to other samples of youth treated in intermediate levels of care. This work might help us better understand who does best under which conditions and which factors contribute to early improvement, which can ultimately inform ways to optimize treatment and increase the efficiency of treatment.

For our second aim, we found that having an anxiety disorder and more severe baseline anxiety symptoms were independently associated with slower rates of improvement on the TPA. Specifically, youth with an anxiety disorder in our sample experienced a 44% slower rate of TPA improvement for every week in treatment. Additionally, the rate of decline on the TPA slowed by 5 percent for every five-point increase on the SCARED-C intake score. Although the literature has been mixed, this study adds additional support to studies that have found that higher anxiety and OCD symptom severity at intake predicts poorer response to treatment (Compton et al., 2014; Milin et al., 2000; Rech et al., 2020; Rudy et al., 2014; Torp & Skarphedinsson, 2017). Some of these studies have speculated that patients with higher baseline symptoms may require modified or more intensive treatment, and clinicians may decide to allocate more therapeutic resources to such patients to augment treatment benefit (Compton et al., 2014; Rech et al., 2020). Our findings signal that patients in a PHP setting with more pronounced anxiety symptomatology at admission might also require more intensive programming as well as regular assessment of how they are responding in real time so treatment adjustments can be made as clinically indicated.

SCARED-C and PHQ-9 scores changed at the same rate irrespective of anxiety disorder status. Additionally, the rate of improvement over time on the PHQ-9 was not affected by SCARED-C severity at intake. Taken together, these results suggest that TPA might be capturing a construct that is distinct from the symptom scales used in this study. Indeed, the TPA is a measure of chief complaints, and patients are asked to self-nominate their top three perceived problems that they want to address in treatment. In a post hoc review of the top problems from a similar sample of youth within the same adolescent PHP, 68.7% of the top problems nominated did not fall into the diffuse anxiety or depression codes, suggesting that the TPA does indeed measure more than just anxiety and depression (for study, see Chiu, Desai, et al., 2022). In addition, other studies using the TPA have reported results on its construct validity indicating that it captures constructs different from symptoms alone (Weisz et al., 2011).

When we explored other predictors in our study, no other demographic or clinical variables appeared to significantly impact the trajectory of treatment. This is consistent with prior research that has failed to find predictors that robustly impact treatment response in PHPs (Balvardi et al., 2022; Kiser et al., 1996).

There are notable strengths to this work. This study offers an example of how MBC can be used to expand our extant knowledge about PHP effectiveness. This is the first study to explore trajectories of change within an adolescent PHP using MBC. In light of mixed findings about whether MBC benefits observed in adults extend to youth (Bergman et al., 2018) and calls to add to the literature on MBC in intensive treatment settings (Childs & Connors, 2022), the current work helps to fill a gap in the literature on MBC applications among youth requiring higher levels of care. By collecting weekly information on treatment benefits, we were able to explore when benefits occurred during the course of PHP treatment and whether factors impacted patient trajectories. This work would not have been possible with pre-post designs used commonly in the evaluation of PHPs (Granello et al., 1999, 2000; Kotsopoulos et al., 1996; Lenz et al., 2014; Lenz & Del Conte, 2018).

To our knowledge, this is also the first study to personalize assessment of treatment benefit in a youth PHP by using an idiographic, consumer-informed measure of top problems. This study had a fairly large sample size (n =124) for a PHP considering the frequency of data collection with weekly follow-ups during treatment. By augmenting symptom questionnaires with the TPA, this study was able to show that patterns of change differed between a personalized measure of chief complaints and symptom scales. Our findings add to the literature on how the TPA likely captures a different dimension of patient progress from that of depression and anxiety symptom severity. These results demonstrate that routinely monitoring how youth are progressing on their chief complaints can add value above and beyond self-reported symptom change. Although this remains to be explored, it is possible that tracking progress on problems deemed important from the consumer perspective can help tailor treatment to promote patient engagement, increase investment in treatment and influence other factors that can lead to improved outcomes.

Our findings should be interpreted in the light of several limitations. First, this is an uncontrolled study. Without a control group and random assignment, we cannot be sure that improvement is attributable to treatment in the PHP. We also cannot draw conclusions that youth benefit more relative to a comparison condition, as it is possible that youth may spontaneously improve on their own without PHP treatment. It is also possible that our PHP may be less effective than a program that was not studied. Nevertheless, controlled studies in acute levels of psychiatry care are rare, in part because it is unethical to withhold treatment and difficult to conduct studies of this kind with an acute population without significant funding. The temporal nature of the change provides some justification for relating improvement to PHP treatment. Using youth as the only informant represents another limitation of the study. In light of research that parent and child reports are often discrepant (Achenbach et al., 1987; De Los Reyes, 2011; De Los Reyes et al., 2023; Lagattuta et al., 2012; Michels et al., 2013; Sequeira et al., 2020), results need to be interpreted with caution. Indeed, findings would have been strengthened had we incorporated parent or clinician report. Unfortunately, neither parent nor clinician data on treatment progress was collected or available. Unlike some studies that have looked at long-term follow up with respect to symptoms, functioning, and use of mental health resources and services after discharge (Kiser et al., 1991; Sommerhalder et al., 2021; Thatte et al., 2013), this study also did not include follow up of patients after discharge from the PHP. As a result, we do not know if patients sustained gains after exiting the PHP. A future direction might be to conduct a systematic chart review of our patients to assess the long-term effect of treatment in the PHP.

At the start of this project, the process of collecting this data was not as standardized and consent rates were low. Many families had privacy concerns and some patients were discharged against medical advice before the research staff could follow up. However, as the study progressed, we were able to implement procedures to increase consent rates. As examples, research consent was made part of the routine intake process for all families, families were approached multiple times throughout the youth’s stay to follow up on consent, and time was carved out in the PHP schedule for adolescents to complete weekly assessments supervised by a research assistant. Collectively, these changes improved our consent rates and completeness of data collection.

Given the variability in populations treated and interventions offered across PHPs, it is not evident that findings from the current study will generalize to other PHPs. Future research should aim to replicate this study in other PHPs and other intermediate levels of care settings. We believe adding the TPA to a battery is an efficient way to personalize assessment in PHPs. This measure, given its capacity to be used across diverse patient populations and settings, is an ideal measure for a level of care that is known to vary widely in terms of the interventions provided, the LOS, and the clientele treated. In addition to collecting ratings from patients, it will be important to also include caregiver and clinician ratings of top problems during treatment to evaluate whether patterns of change are similarly observed across informants. This is particularly needed given the finding that one third of youth-caregiver pairs do not agree on top problems at the start of PHP treatment (Chiu, Desai, et al., 2022). Although this was beyond the scope of the current project, future directions should also collect measures from stakeholders (i.e., patients, caregivers, treatment providers, administration) on the acceptability and feasibility of implementing MBC practices to add to the growing literature on integrating MBC within intermediate levels of care. Additionally, future work might explicitly test the benefits of adding MBC to existing intensive programs. For those interested in a strength-based approach to assessment, one consideration is to adapt the TPA measure to assess consumer-nominated strengths. Helping clients to focus on their strengths, rather than their top problems, may serve to have multiple benefits, including orienting clients to focus on what is good, highlighting areas for providers that can be strengthened, and empowering clients to remember traits that they inherently possess that will help them overcome challenges.

Future research should prioritize collecting data routinely during treatment, not just at admission and discharge, to enable evaluation of patient trajectories and examination of who benefits most (and least) from treatment. Regular assessments can also improve clinical decision making if progress data can be fed back to patients and their clinical teams so that they can make decisions in real time (Lavender et al., 2022). Indeed, this PHP attempted to do so by providing clinical dashboards summarizing the results of questionnaires each week to the clinical team – Chorpita and colleagues (2008) describe the clinical dashboard in more detail, share an example of the visual tool, and also provide an overview on how this tool can be used in shared decision making. For groups interested in incorporating MBC into intensive care settings, a roadmap for MBC implementation is available that offers guidance organized by an implementation framework (Childs & Connors, 2022).

In summary, the current study adds to the literature supporting the effectiveness of adolescent PHPs. This study is the first to establish that youth who receive PHP treatment improve on their self-nominated chief complaints as well as on symptom scales. Our study demonstrates that youth improved more rapidly on their top problems at the beginning of PHP treatment, whereas the rate of improvement stayed steady on measures of anxiety and depression symptoms across time. Additionally, having an anxiety disorder and higher severity baseline anxiety symptoms were independently related to slower rates of improvement on the TPA, but not on the SCARED-C or PHQ-9. This work highlights the benefits of augmenting symptom scales with idiographic, consumer-nominated problems to personalize assessment. The current work also provides promising support for the incorporation of MBC in evaluating the effectiveness of youth PHPs, and its potential to help answer important questions about who benefits most, under which conditions, and when.