Community Study of Outcome Monitoring for Emotional Disorders in Teens (COMET): A Comparative Effectiveness Trial of a Transdiagnostic Treatment and a Measurement Feedback System

Abstract

Emotional disorders, encompassing a range of anxiety and depressive disorders, are the most prevalent and comorbid psychiatric disorders in adolescence. Unfortunately, evidence-based psychosocial therapies typically focus on single disorders, are rarely adopted by community mental health center clinicians, and effect sizes are modest. This article describes the protocol for a comparative effectiveness study of two novel interventions designed to address these challenges. The first intervention is a transdiagnostic treatment (the Unified Protocol for Transdiagnostic Treatment of Emotional Disorders in Adolescents, UP-A), a promising new approach that uses a small number of common strategies to treat a broad range of emotional disorders, and their underlying shared emotional vulnerabilities. The second intervention is a standardized measurement feedback system, the Youth Outcomes Questionnaire (YOQ), designed to improve clinical decision making using weekly symptom and relational data. The three study arms are treatment as usual (TAU), TAU plus the YOQ (TAU+), and UP-A (used in combination with the YOQ). The primary aims of the study are to (1) compare the effects of the UP-A and TAU+ to TAU in community mental health clinics, (2) to isolate the effects of measurement and feedback by comparing the UP-A and TAU+ condition, and (3) to examine the mechanisms of action of both interventions. Design considerations and study methods are provided to inform future effectiveness research.

Introduction

Emotional disorders, encompassing anxiety and depressive disorders, are the most prevalent and comorbid psychiatric disorders in adolescence¹. They are chronic, impairing, and share common vulnerabilities^2–8. Researchers have developed and tested dozens of disorder-specific psychosocial treatments for emotional disorders⁹. However, the effectiveness of these evidenced-based treatments (EBTs) in community mental health clinics (CMHCs) has been disappointing. A recent meta-analysis concluded that, although EBTs typically outperformed treatment as usual (TAU), the average effect was small (d = .30 for emotional disorders; ⁹. In addition, the adoption of EBTs into CMHCs has been slow¹⁰.

Several characteristics of EBTs might decrease their “real-world” effectiveness and uptake. Many EBTs are designed to address single diagnoses, whereas comorbidity is common in CMHCs¹¹. A related problem is the training burden clinicians face to become competent in multiple single-diagnosis EBTs¹². Clinicians often perceive EBTs as too rigid for personalized treatment^13,14; recent data suggest that flexible treatments are more appealing to clinicians¹⁵ and more effective than TAU and traditional EBTs¹⁶.

A recent innovation that addresses many of these limitations is transdiagnostic treatments, which target multiple problems within a single conceptual framework^5,17,18. Transdiagnostic approaches may be particularly relevant for adolescents, whose high comorbidity rates and shifting symptom profiles complicate the typical treatment approach¹⁹. The Unified Protocol for Transdiagnostic Treatment of Emotional Disorders in Adolescents (UP-A; ²⁰ applies emotion-focused intervention strategies to a broad range of internalizing symptoms. It has demonstrated efficacy in a research setting²¹; a logical next step is to examine its effectiveness in CMHCs.

Traditionally, effectiveness studies have compared the EBT of interest only to TAU. However, interpretation of these studies can be challenging because the EBT clinicians receive regular feedback through standardized assessment and supervision. If treatment is going poorly, the clinician is often informed of this so that treatment can be adjusted. Some EBTs also make explicit use of session-by-session assessment of symptom severity to inform clinical care²². If EBTs outperform TAU, it can therefore be difficult to determine the degree to which these differences are due to the treatment techniques, or to the “confounding effects” of increased monitoring and feedback to clinicians. Disentangling these factors has high scientific value and important implications for dissemination and implementation of EBTs.

One strategy to control for these effects is the use of standardized measurement feedback systems (MFSs). MFSs consist of assessment tools, often part of an online system, to regularly track the processes (e.g., therapeutic alliance) and outcomes (e.g., symptom improvement) of therapy, with clinician reports summarizing the results²³. Extensive research with adults suggests that using an MFS can increase therapy success rates^24,25. Preliminary evidence also suggests benefits for youth^26,27, although effects vary by organization²⁸. Because MFSs are designed to facilitate clinical decision-making across a clinician’s entire caseload, they also share many of the same advantages of transdiagnostic treatments.

Given the disadvantages of single disorder EBTs and the need for novel designs focused on disentangling the confounding effects of assessment and feedback, our team designed the Community Study of Outcome Monitoring for Emotional Disorders in Teens (COMET).

Study Design and Aims

COMET is funded by National Institute of Mental Health (R01MH106536 & R01 MH106657). COMET is a two-site randomized controlled effectiveness trial focused on two interventions, the UP-A, and the Youth Outcomes Questionnaire²⁹ (YOQ), a MFS tracking youth symptoms and therapy alliance. The study will compare three conditions: (1) TAU alone; (2) TAU plus the YOQ (TAU+); and (3) UP-A plus the YOQ (UP-A). The Institutional Review Boards at University of Miami and University of Connecticut School of Medicine have approved all study procedures and the study is registered at ClinicalTrials.gov (NCT02567266).

Study Aims

The primary study aims are:

Aim 1: To compare the effectiveness of UP-A and TAU+ to TAU.

Aim 1 will test whether adolescents in the UP-A and TAU+ conditions demonstrate greater clinical improvements in anxiety and depression than those receiving TAU. We hypothesize that adolescents randomized to UP-A and TAU+ conditions will demonstrate greater clinical improvements relative to TAU.

Aim 2: To isolate the effects of evidenced-based measurement and feedback.

Aim 2 will examine the relative effectiveness of the UP-A condition to the TAU+ condition. We hypothesize treatment outcomes will be better in the UP-A condition than in TAU+.

Aim 3: To examine mechanisms theoretically associated with UP-A and YOQ.

We hypothesize that differences in outcomes between the UP-A and the other two conditions will be mediated by changes in: 1) Distress Tolerance and 2) Emotional Avoidance³⁰. We further hypothesize that, among participants in the UP-A and TAU+ conditions, treatment outcomes will be better for participants whose therapists: 1) rate the YOQ results as more credible, 2) view the YOQ reports more frequently, and 3) discuss the reports with them in session more frequently. Moreover, we hypothesize that differences in outcomes between TAU and the two treatments using the MFS (UP-A and TAU+) will be mediated by differences in: 1) therapeutic alliance and 2) therapy engagement.

Exploratory Aim.

Based on the literature³¹ and domains outlined by Burns et al.³² and Jensen et al.,³³ a number of potential predictors and moderators will be examined (gender, age, domain/severity of illness, psychiatric comorbidity, medication usage).

Participants

Adolescent participants will include 222 adolescents (111 per site) with elevated symptoms of anxiety or depression, as indicated by the presence of an anxiety, depressive, obsessive-compulsive or adjustment disorder with anxiety and/or mood specifier. Participants are drawn from the referral pool of clients from participating CMHCs.

Youth inclusion criteria are: 1) between the ages of 12 and 18 with clinically significant symptoms of anxiety or depression, as defined by a Clinical Severity Rating (CSR) of 4 or higher on any DSM-5 anxiety, obsessive-compulsive or depressive disorder, including adjustment disorders, as determined by the Anxiety Disorders Interview Schedule for the DSM-5, Child Version, Child and Parent Report Forms (ADIS-5-C/P³⁴), 2) deemed eligible and appropriate for outpatient services by one of the study clinics, 3) living with a legal guardian at least 50% time who is willing to attend treatment sessions, and 4) and has a caregiver who is able to complete all study procedures in English or in Spanish. Youth exclusion criteria include: 1) receiving concurrent psychosocial interventions, 2) suicidal behavior that warrants a higher level of care than routine outpatient treatment, and 3) other indicators that would make the UP-A contra-indicated (e.g., significant substance abuse; IQ < 80).

Clinician participants are employees or trainees working at the participating clinics. Inclusion criteria are: 1) at least part-time employee or completing an approved practicum or internship of at least one year, and 2) able to speak, read and understand English.

Procedures

COMET is being conducted in 14 clinics (six in South Florida and eight in Connecticut). Clinicians are recruited through agency administrators, and then go through a study consent procedure and complete baseline measures before being randomly assigned to deliver one of the three study conditions. Clinicians are randomized in blocks of three within each agency, using a random number generator (www.randomization.com). If an agency has three clinicians who speak Spanish, these clinicians are randomized within a separate block. These procedures ensure that each agency has clinicians in all three conditions and, if the agency will be enrolling Spanish-speaking adolescents, that Spanish-speaking clinicians are available in all three conditions. Each clinician will deliver only one of the study conditions to minimize contamination between conditions. Clinicians randomized to the experimental conditions then receive training and consultation in the UP-A, and/or YOQ, and all clinicians see study cases as part of their regular caseload. Clinicians are asked to audio record sessions, complete a session report every session, complete measures at the 8- and 16-week point for each study case, and are also asked to repeat some of the baseline measures after they complete their first study case. Depending on the agency, clinicians are either reimbursed for training and consultation time through their regular agency procedures or directly by the study. In addition, all clinicians receive $60 at the end of each case for completing the study measures.

While specific procedures vary by clinic, families typically receive information about the study at the initial phone contact, or after completing the clinic’s intake assessment. If eligible for services at the clinic and interested in participating in the study, adolescents and their caregiver sign assent/consent and take part in a baseline study assessment. Eligible adolescents are then randomized to one of the three study conditions and assigned to the appropriate therapist at their clinic. Randomization lists (in blocks of 3) were generated by the study statistician using www.sealedenvelope.com/simple-randomiser/v1/lists. Randomization lists were stratified by agency, whether or not the adolescent has a depression diagnosis, whether or not the adolescent is taking medication, and whether they need an English or Spanish-speaking clinician. Research assessments then take place 8 weeks after treatment initiation (estimated to be mid-treatment), 16 weeks after treatment initiation (post treatment; estimated to correspond with the average length of UP-A treatment), and 7 months after treatment initiation (a 3-month follow-up).

Study Conditions.

Unified Protocol for Transdiagnostic Treatment of Emotional Disorders in Adolescents(UP-A; ³⁵.

The UP-A³⁵ is a transdiagnostic approach to treating adolescents with emotional disorders. The UP-A targets core dysfunctions (e.g., neuroticism; ^36,37 believed to characterize a wide range of emotional disorders. The UP-A’s intervention techniques are formatted to allow application to various intense emotions (e.g., fear, sadness, anger) and problematic emotional behaviors (e.g., avoidance, aggression, etc.), making it an inherently flexible approach that can be personalized to various settings and clinical presentations. There are eight primary modules in this treatment and one additional parent module administered in weekly sessions. Treatment techniques include motivational enhancement, psychoeducation, functional assessment of emotional experiences, emotion-focused behavioral experiments (i.e., behavioral activation and/or exposure), antecedent cognitive reappraisal, problem-solving strategies, and awareness and mindfulness strategies. Parenting strategies center on modification of “emotional parenting behaviors” (over-control, inconsistency in discipline, criticism, and parental modeling of distress). Using these techniques, clinicians are encouraged to focus on a primary goal of reducing youth and parent distress while experiencing strong negative affect, thereby promoting more adaptive, approach-oriented behavioral choices²¹. Clinicians assigned to the UP-A condition also use the YOQ²⁹, as described below.

Treatment as Usual + YOQ (TAU+).

Clinicians in the TAU+ condition use their “as usual” therapy practices, but also use the YOQ-30-TA²⁹ during every session. The YOQ²⁹ is a youth version of the adult Outcomes Questionnaires (OQ)³⁸. The OQ system is the most extensively tested MFS for adults; numerous randomized controlled trials (RCTs) have found that using the OQ to provide therapists feedback about symptoms and alliance, particularly when combined with alerts about clients who are not “on track” to obtain good outcomes, leads to increased engagement (i.e., longer treatment) and improved client outcomes³⁹. The YOQ measures have not been tested in an RCT, but they have been found to be sensitive to change⁴⁰ and several studies support the accuracy of the YOQ alert system in identifying cases at risk for treatment failure in TAU settings^41–45.

YOQ-30-TA consists of parent- and youth-report measures of symptoms (30 items) and alliance (4 and 5 items in the parent and youth versions, respectively), administered each session on a tablet. The YOQ online system then generates reports to provide clinicians with systematic feedback about client progress, flagging “critical items” that have been endorsed (e.g., suicidality, substance use), highlighting concerns about therapeutic alliance, presenting graphs of ratings over time, and providing empirically-derived “alerts” when clients are failing to progress or exhibiting deterioration. Clinicians are trained to use this feedback to modify treatment as needed and to share it with families as appropriate.

Little formal data exist regarding mechanisms of change in measurement and feedback⁴⁶. However, theoretical models posit that MFSs lead to better outcomes because they impact: 1) therapists’ behavior (i.e., therapists who view feedback data, view them as credible and change therapeutic approaches when needed)^47–49, 2) therapeutic alliance⁵⁰, and 3) therapy engagement^51,52.

Treatment as Usual (TAU).

Clinicians assigned to the TAU condition are instructed to use whatever treatment methods and outcome monitoring strategies they typically use with adolescents with internalizing disorders. Neither the South Florida nor Connecticut clinics in the current study routinely use any MFS.

Training and Consultation

Training for the UP-A consists of a 12-hour workshop focused on didactics, skill demonstrations, and role-plays, followed by ongoing weekly one-hour consultation calls. On consultation calls, UP-A consultants review weekly administration of session content, provide psychoeducation about core intervention techniques and UP-A materials (e.g., adolescent workbook), problem-solve barriers to intervention delivery and provide opportunities for both consultant-directed and clinician-directed rehearsal and role-play of UP-A strategies. In addition, UP-A consultants monitor YOQ administration and review feedback reports. A study listserv is used to circulate summaries of issues discussed on calls and allows clinicians to post general questions about UP-A and to read responses to their and others’ questions. In addition, 20% of UP-A tapes are reviewed as they become available to assure high quality UP-A treatment delivery, with dimensions of both competency and adherence rated.

Training in the YOQ consists of a four-hour workshop to orient clinicians to the YOQ measures and how to use the YOQ online platform, followed by weekly 30-minute consultation calls for the clinicians in the TAU+ condition. On calls, consultants help clinicians brainstorm how to fit the YOQ into their treatment setting and workflow, and review and discuss the interpretation of YOQ feedback reports. The YOQ data system allows the consultants to monitor whether clinicians are completing forms with clients. If clinicians are not engaging in these activities, consultants use the consultation calls to discuss strategies for increasing adherence.

Clinicians in the TAU condition do not receive any training beyond an orientation to study procedures. Clinicians in all conditions continue to receive their standard clinic supervision to address clinical and administrative issues.

Measures

Measures were selected based on the goal of balancing objectivity with practicality, given that study assessments take place in a variety of CMHCs. Table 1 lists all study measures, including who completes them and when they are administered. All youth- and caregiver-report measures are administered in English or Spanish.

Table 1:

COMET Measures

Measure	Reporter	Every Session	Baseline	8 Weeks	16 Weeks	3 month follow-up
Screening Measures
Family Background and Medical History Form	Y, P		X
Service Utilization Assessment	P		X		X	X
Columbia-Suicide Severity Rating Scale (C-SSRS, 53	IE		X		X	X
CRAFFT Screening Test 54	IE		X		X	X
Barriers to Treatment Participation Scale – Expectancies 55	P		X
Independent Evaluator-Rated Measures of Primary Outcomes
ADIS-5-C/P34	Y, P, IE		X		X	X
CGI-Severity and Improvement (CGI-S and CGI-I)56,57	IE		X		X	X
Children’s Global Assessment Scale (CGAS, 58	IE		X		X	X
Rating Scales of Adolescent Symptoms
SCARED/SCARED-P59	Y, P		X	X	X	X
Mood and Feelings Questionnaire-Self and Parent Reports (MFQ)60,61	Y, P		X	X	X	X
SDQ-Self & Parent Reports62	Y, P		X	X	X	X
YOQ 30 TA and YOQ 30 SR TA29	Y, P	X1	X	X	X	X
Rating Scales of Caregiver Psychopathology and Emotion Regulation
Generalized Anxiety Disorder-7 (GAD-7)63	P		X	X	X	X
Patient Health Questionnaire-9 (PHQ-9)64	P		X	X	X	X
Coping with Children’s Negative Emotions Scales (CCNES)65	P		X	X	X	X
Egna Minnen Betraffande Uppfostran – Short Form (S-EMBU)66	Y, P		X		X	X
Measures of Potential UP-A Mechanisms
Behavioral Indicator of Resiliency to Distress (BIRD)67	Y		X	X	X	X
Distress Tolerance Scale (DTS)68	Y, P		X	X	X	X
Emotional Avoidance Strategy Inventory for Adolescents (EASI-A)69	Y		X	X	X	X
Avoidance Hierarchy	Y, P, IE		X	X	X	X
Measures of Potential YOQ Mechanisms
Session Report Form (Measures therapist behavior)	C
Working Alliance Inventory (WAI)70,85	Y,C
Session attendance and completion assessed via chart review
Measures of Clinician Characteristics
Evidence-Based Practice Attitude Scale (EBPAS 71)	C		X2		X3
Attitudes Toward Standardized Assessment Scales-Monitoring and  Feedback (ASA-MF72)	C		X2		X3
Monitoring and Feedback Attitudes Scale (MFA 72)	C		X2		X3
TCU Organizational Climate Form (TCU-ORC)73	C		X2

Note: ADIS-5-C/P = Anxiety Disorders Interview Schedule for the DSM-5, Child Version, Child and Parent Report Forms; CGI = Clinical Global Impressions; SCARED = Screen for Child Anxiety Related Emotional Disorders-Youth and Parent Reports; SDQ = Strengths and Difficulties Questionnaire; YOQ 30 TA and YOQ 30 SR TA = Youth Outcome Questionnaire-30 Therapeutic Alliance, parent and self-report versions. 1. TAU+ and UP-A Conditions only; 2. Administered to the clinician before they start treating their first study case; 3. Administered to the clinician after they complete their first study case.

Screening & Background Measures.

Several measures are administered at baseline to screen for exclusion criteria and to characterize the sample. Two measures developed for this study, the Family Background and Medical History Form and the Service Utilization Assessment, are used to gather demographic and clinical history information and to assess for current service utilization. The Service Utilization Assessment is re-administered at the post-treatment and follow-up assessments to gather data regarding service use during the study. The Columbia-Suicide Severity Rating Scale (C-SSRS, ⁵³)⁵⁶ and the CRAFFT Screening Test⁵⁴ are Independent Evaluator (IE)-administered screeners for suicidal ideation and behavior, and for problematic substance use. Finally, the Barriers to Treatment Participation Scale –Expectancies⁵⁵ is administered at baseline to characterize factors that might interfere with treatment engagement.

Independent Evaluator (IE)-Rated Measures of Primary Outcomes.

The study’s primary outcome measures are IE ratings on the Clinical Global Impression-Severity (CGI-S)⁵⁶, CGI-Improvement (CGI-I)⁵⁷, and Children’s Global Assessment Scale (CGAS)⁵⁸. These ratings will be generated after the IE interviews the parent and youth with the ADIS-5-C/P³⁴. The CGI-S is a 7-point clinician rating of symptom severity. Severity ratings range from 1 (no illness) to 7 (extremely severe). The CGI-I is a 7-point rating of treatment response anchored by 1 (very much improved) and 7 (very much worse). The CGAS is a widely used 100-point rating scale used to measure global functional impairment.

Secondary Outcome Measures.

Secondary outcomes include several rating scales, all with parallel youth and caregiver versions. Anxiety symptoms will be measured via the Screen for Child Anxiety Related Emotional Disorders (SCARED)⁵⁹ and depression symptoms via the Mood and Feelings Questionnaire (MFQ)^60,61. The Strengths and Difficulties Questionnaire⁶² is used as a broadband measure to measure total symptom severity and comorbid symptoms, including conduct problems, hyperactivity/inattention, and peer relationship problems. Finally, the YOQ is administered to youth in all three conditions at the research assessments.

Caregiver Psychopathology and Emotion Regulation Measures.

To allow for examination of caregiver-related outcomes or potential moderators/mechanisms of treatment effects, caregivers complete measures of their anxiety and depression (Generalized Anxiety Disorder-7⁶³, Patient Health Questionnaire-9⁶⁴), ability to cope with their child’s negative emotions (Coping with Children’s Negative Emotions Scales⁶⁵), and parenting style (Egna Minnen Betraffande Uppfostran – Short Form⁶⁶).

Measures of UP-A Mechanisms.

Measures of UP-A mechanisms include those that assess distress and time to discontinuation during a frustrating task (Behavioral Indicator of Resiliency to Distress⁶⁷), ability to withstand uncomfortable emotions (Distress Tolerance Scale⁶⁸), emotional avoidance strategies (Emotional Avoidance Strategies Inventory –Adolescent Version⁶⁹), and parent, child and IE reports of frequency and intensity of avoidance behavior (Avoidance Hierarchy) across major study time points. These mechanisms measures were chosen for their links to the theory of change articulated for the Unified Protocols (UP) by Barlow and colleagues³⁰, which posits that UP facilitates change in the intensity of negative affect and emotional disorder symptoms centrally via extinction of distress during the experience of strong emotion (as measured here by change in distress tolerance) and reduction of avoidance.

Measures of Measurement & Feedback Mechanisms.

Therapeutic alliance will be measured via youth- and therapist-report on the Working Alliance Inventory⁷⁰ and via youth and parent-report on the YOQ. Engagement data (i.e., attendance and treatment dropout) will be collected via medical record review. Therapist behavior mechanisms (i.e., credibility of the reports and plans to change the treatment plan in response to feedback) will be assessed weekly in the session report form therapists are asked to complete every session.

Measures of Clinician Characteristics.

To characterize the clinician sample and examine potential differences in clinicians and/or sites, clinicians complete several measures at baseline, including the Evidence-Based Practice Attitude Scale (EBPAS⁷¹; measures clinician attitudes toward EBTs), the Attitudes Toward Standardized Assessment Scales-Monitoring and Feedback (ASA-MF⁷²; measures attitudes toward standardized progress measures), the Monitoring and Feedback Attitudes Scale (MFA⁷²; measures attitudes toward utilizing progress data in clinical decision-making and treatment), and the TCU Organizational Climate Form (TCU-ORC⁷³; measures organizational climate). The EBPAS, ASA-MF, and MFA are all administered again after clinicians compete their first cases, to examine clinician-level effects of learning these new evidence-based practices.

Assessment of Treatment Differentiation and Fidelity.

All treatment sessions across conditions are audiotaped; these tapes are used to monitor ongoing fidelity for UP-A (see Training and Consultation, above), to quantify treatment adherence in the UP-A and TAU+ conditions, to assure treatment differentiation, and to document TAU characteristics. Two coding systems are applied to these recordings. The UP-A Adherence and Competence Checklist²⁰ is a measure of UP-A adherence and competency developed for the original UP-A RCT²¹. Trained coders apply this system to 20% of UP-A session tapes on an ongoing basis to monitor fidelity. In addition, at the end of the project, an IE will code 20% of UP-A tapes to generate data regarding overall fidelity levels in the trial. In addition, the Treatment Adherence, Content, and Competence Checklist⁷⁴ will be completed by an IE while reviewing 20% of audiotapes of treatment sessions from all three treatment conditions to assure treatment differentiation and to characterize TAU. This measure contains three subscales covering: 1) treatment content, 2) session components, and 3) nonspecific factors. In addition, an item will be added to the measure to document whether clinicians are discussing feedback reports from the YOQ with their clients. Data regarding other aspects of YOQ fidelity (i.e., administering measures, reviewing reports) will be gathered directly through the YOQ system. To ensure that data are available for all participants, tapes will be randomly sampled within participants.

IE Training

Training and certification of IEs includes 1) reading published articles on clinical interviewing and the diagnostic interview, 2) a review of study measures, and 3) “matching” videotaped gold-standard cases to ensure inter-rater agreement on the ADIS-5-C/P, CGAS, and CGIs. Specifically, IEs must obtain 100% inter-rater agreement (defined as matching on the presence of the primary and secondary diagnosis and assigning a score within one point on the CSR, CGI-S, and within the decile on the CGAS) on a minimum of three interviews. New IEs are then observed during their first live interview and approved by the site PI and the study IE supervisor who oversees IE training for both sites. That PIs hold weekly cross-site conference calls and IEs receive ongoing supervision at each site by the study PIs. To ensure cross-site consistency, all assessments are taped and 20% of interviews per year are blindly reviewed by the study IE supervisor or certified IE at the Uonn site to assess inter-rater reliability and rater drift on the ADIS-5-C/P, CGAS, and CGI-S/I. If an IE is unable to meet the above criteria for certification, the site PI (or experienced and certified IE) engages in remediation/retraining by viewing and discussing recorded or live tapes.

Data Analysis Plan

We will use multilevel modeling (MLM) with a logistic linking function to examine treatment effects on the dichotomous outcome (“treatment response”, defined as CGI-I of 1 or 2). Additional IE-rated (e.g., CGI-S, CSR) and questionnaire-based (e.g., SCARED, MFQ) outcomes are all continuous measures and will be analyzed using traditional MLM. Repeated assessments over time will be nested within individuals, who will be nested within therapist. Since inaccurate statistical models can lead to misleading results⁷⁵, various growth curve models (linear, quadratic, logarithmic, hyperbolic, and piecewise) will be examined to determine the model that best fits the data (based on AIC and BIC information criteria). Multivariate MLM will be used where it is deemed appropriate.

For Aim 1 (To compare the effectiveness of UP-A and TAU+ to TAU), differences between the three treatment groups (UP-A, TAU+, and TAU) will be tested by including two dummy coded treatment variables as predictors of the growth curve parameters (intercept and slopes). For Aim 2 (To isolate the effects of evidenced-based measurement and feedback), the same models will be applied to the data, but with the dummy coded treatment variable contrasting the UP-A and TAU+ conditions.

For Aim 3 (To examine mechanisms theoretically associated with UP-A and YOQ), some analyses will focus on mediators of treatment effects and others on predictors of outcomes. Mediators will be examined by simultaneously adding them to the growth curves as time varying covariates (TVCs). Since TVCs conflate between-subjects and within-subjects effects, all TVCs will be disaggregated into their between-subjects effects and within-subjects effects before adding them to the analyses (e.g., ⁷⁶). Moderated mediation will also be explored. Significance of mediated pathways will be determined using the distribution of products test⁷⁷. Predictors of outcomes will be analyzed by entering them as level 2 predictors of the growth curves. Finally, our exploratory aim, which will focus on potential predictors and moderators of outcomes, will be investigated by entering candidate moderators, and interaction terms between them and treatment group, as predictors of the growth curves.

Power Analysis.

Monte Carlo simulation analyses indicated that the study is powered at >.90 to detect a medium effect size difference on our primary outcomes. Our Monte Carlo simulation indicated power greater than .56 to detect small to medium effect sizes on our dichotomous outcome variable (response rate), and power >.80 to detect such differences on our continuous outcome measures. Monte Carlo studies further show that we have power greater than .90 to detect a mediated pathway if the “a” path and the “b” path are medium effect sizes.

Discussion

Transdiagnostic approaches like the UP-A and YOQ hold great potential for improving outcomes for adolescents with anxiety and depression. Effectiveness trials like COMET are an essential next step to understanding whether they are robust to the challenges of routine clinical settings and ready for wide-scale dissemination. This trial will also make an important contribution by disentangling the effects of evidence-based assessment and EBT.

Despite the potential impact of this study, there are some limitations to the approach. First, the UP-A has only been tested in one previous trial comparing it to a waitlist control group. As such, we considered whether an additional efficacy trial comparing the UP-A to a waitlist or single disorder EBT would be more appropriate. However, the UP-A waitlist control RCT exhibited impressive effect sizes (d > 2 for some measures at post)²¹ and other versions of the unified protocols have demonstrated equivalence with established EBTs for anxiety^78,79. As such, adding a fourth arm was judged either unethical (in the case of a wait list control), to be a poor use of resources, or inappropriate clinically for this comorbid population (in the case of a single disorder EBT comparison). Moreover, Shirk⁸⁰ argues that promising evidence-based approaches such as the UP-A should be “fast-tracked” into general practice via systematic effectiveness trials, given predominant evidence that TAU is often not effective for the average clinic client^81,82.

Second, the study does not employ a fully factorial design, where UP-A was also used without the YOQ. However, as discussed in the introduction, the research assessment and supervision needed to implement the UP-A serves a very similar role to the YOQ; it was not clear that there would be differentiation between “UP-A alone” and “UP-A + YOQ” conditions.

Third, the three conditions do not receive equivalent doses of supervision/consultation, as the TAU group receives standard agency supervision, the TAU+ group standard supervision plus half an hour of consultation, and the UP-A standard supervision plus an hour of consultation. We considered approaches to control for the confounding effects of added consultation. However, an increase in quantity of consultation might not yield increased quality in terms of feedback to clinicians. Also, changing the nature of the TAU supervision could decrease the generalizability of study findings, as the trial would no longer be comparing the study interventions to truly representative TAU. Fortunately, the inclusion of mechanism measures in the study will allow us to determine whether any differences between conditions are related to the theorized intervention mechanisms or to other non-specific factors.

Finally, randomizing clients and therapists to condition within clinics raises the possibility of treatment contamination across conditions. We therefore considered randomizing clinics to condition, rather than randomizing clients and therapists within clinics. The decision to randomize within clinics was made because the diversity of client populations across sites would make it difficult to interpret whether differences in outcomes were due to study conditions or to characteristics of the clinics themselves. Our power analyses also indicated that randomization within clinics yielded substantially more statistical power than randomizing clinics. In addition, given the intensity of training and supervision required to change clinician treatment practices, we considered it unlikely that having peers who are using a new treatment might have a substantive impact on clinician practices. Indeed, previous studies using this design have not found evidence of cross-condition contamination^16,83,84.

In conclusion, the current paper proposes an innovative approach for conducting comparative effectiveness trials on transdiagnostic approaches for emotional disorders in CMHCs. Due to the study being conducted at CMHCs rather than an academic research clinic, our goal was to balance strong methodology with feasibility and generalizability in order to address the current issues surrounding most evidence-based therapies. The results of this investigation will have important implications for treating diverse youth in CMHCs, by demonstrating whether UP-A and YOQ approaches are more effective than current approaches to treating adolescent emotional disorder symptoms (Aims 1 and 2). Additionally, the current study will increase our understanding of mechanisms of treatment change in transdiagnostic interventions (Aim 3). Finally, this trial makes an important methodological contribution by disentangling the effects of evidence-based treatment from those of enhanced monitoring and feedback. While analyses regarding our primary aims may start to untangle the complex issues of what or how much evidence-based assessment and/or treatment should be targeted for dissemination to improve outcomes for youth with emotional disorders, the study of these phenomenon will no doubt also prompt further research into the complexities of collaboratively working with community clinics and therapists to understand the answers to such challenging questions in the future.