Effects of High vs. Low Intensity Clinician Training on Implementation of Family-Focused Therapy for Youth with Mood and Psychotic Disorders

Abstract

The implementation of evidence-based psychotherapies often requires significant commitments of time and expense from mental health providers. Psychotherapy protocols with rapid and efficient training and supervision requirements may have higher levels of uptake in publicly-funded clinics. Family-focused therapy (FFT) is a 4-month, 12-session treatment for bipolar and psychosis patients consisting of psychoeducation, communication training, and problem-solving skills training. In a pilot randomized trial, we compared two methods of training community clinicians in FFT: (a) high intensity, consisting of a 6-hour in-person didactic workshop followed by telephone supervision for every session with training cases; or (b) low intensity training (n=23), consisting of a 4-hour online workshop covering the same material as the in-person workshop followed by telephone supervision after every third session with training cases. Of 47 clinician participants, 18 (11 randomly assigned to high intensity, 7 to low) enrolled 34 patients with mood or psychotic disorders (mean age 16.5±2.0 years; 44.1% female) in an FFT implementation phase. Expert supervisors rated clinicians’ fidelity to the FFT manual based on taped family sessions. We detected no differences in fidelity scores between clinicians in the two training conditions, nor did patients treated by clinicians in high versus low intensity training differ in end-of-treatment depression or mania symptoms. Levels of parent/offspring conflict improved in both conditions. Although based on a pilot study, the results suggest that low intensity training of community clinicians in FFT is feasible and can result in rapid achievement of fidelity benchmarks without apparent loss of treatment efficacy.

INTRODUCTION

Despite substantial investment by the National Institutes of Health (NIH) in developing evidence-based psychotherapies (EBPs) for persons with severe mental illness, many promising treatments have been inconsistently translated, disseminated, and implemented in community care settings (CMHCs) (Glasgow et al., 2012; McHugh & Barlow, 2010). To address this gap between evidence and practice, the NIH is emphasizing studies of dissemination and implementation of EBPs with patients who are treated in public mental health settings.

A recent qualitative study of clinicians’ and health administrators’ attitudes toward training and implementation of EBPs clarifies some of the reasons for this gap (Chung, Mikesell, & Miklowitz, 2014). Clinicians working in publicly-funded CMHCs struggle to balance substantial clinical and administrative demands (e.g., seeing a prescribed number of patients each week) with delivering EBPs with fidelity to the respective treatment manuals. Many clinicians feel that existing EBPs are inflexible and do not recognize the heterogeneity of patient populations seen in CMHCs. Mental health administrators are concerned with the amount of training and supervision time required by EBP protocols. Overall, treatment protocols that offer practitioners a balance between structure and flexibility, and for which training and supervision requirements are less time-intensive may have higher levels of uptake in CMHCs (Chung et al., 2014).

Family-focused therapy (FFT), an outpatient protocol for adult or adolescent patients with mood, psychosis, or psychosis-risk syndromes, lends itself well to community dissemination because of its balance of structure with flexibility. FFT is given in combination with pharmacological treatment and aims to enhance symptom stabilization after an acute episode of mood or psychotic illness. It consists of sessions in which patients and family members learn about the nature, causes, and treatment of the patient’s disorder, how to identify and intervene with early signs of recurrences, and learn skills to constructively communicate and solve family problems related to the post-episode period. In randomized efficacy trials in bipolar disorder, FFT plus pharmacotherapy has been found to be more effective than supportive care plus pharmacotherapy in hastening episode recovery and reducing rates of recurrence over 1–2 years (Miklowitz, George, Richards, Simoneau, & Suddath, 2003; Rea et al., 2003; Miklowitz, Richards, et al., 2003; Miklowitz et al., 2007; Miklowitz et al., 2008; Miklowitz, Schneck, et al., 2014; Miklowitz, Efthimiou, et al., 2020). In three trials, FFT was found to be associated with a more favorable trajectory of symptoms in youths who were at clinical and familial risk for bipolar disorder (Miklowitz et al., 2013; Miklowitz, Schneck, et al., 2020) or psychosis (Miklowitz, O’Brien et al., 2014; Worthington et al., 2020).

Despite its record in university-based trials, FFT has not been widely disseminated in publicly-funded community clinics (Miklowitz & Chung, 2016). In the present study, we explored the feasibility of implementing FFT in high-volume CMHCs through comparing brief online versus in-person clinician training, followed by low or high frequency telephone supervision from experts. Clinicians enrolled CMHC adolescent or young adult patients with mood or psychotic spectrum disorders in a 4-month, 12 session FFT protocol. We opted to focus the study on adolescents and young adults in the early stages of illness who were likely to be in close contact with their families.

The primary outcome was clinicians’ fidelity to the FFT manual, defined as adherence to the content of the psychoeducation, communication, and problem-solving modules, and competence (i.e., skill and style of delivery) in administering the modules (Waltz, Addis, Koerner & Jacobson, 1993; Marvin, Miklowitz, O’Brien, and Cannon, 2016). Fidelity, we reasoned, could be achieved equally by clinicians in both training intensities. Secondarily, we hypothesized that patients treated by clinicians who received low-intensity training would have equivalent post-treatment symptom outcomes and family functioning compared to patients whose clinicians received high intensity training. Finally, we explored factors that were associated with clinicians’ engagement and fidelity in implementing FFT in their respective settings.

METHODS

This study reports results from a pilot randomized trial funded under the National Institute of Mental Heath’s R21/R33 funding mechanism, “Exploratory Clinical Trials of Novel Interventions for Mental Disorders.” The focus of this mechanism is on the feasibility, acceptability, and standardization of novel treatment and study measurement methods to inform the design of large-scale effectiveness trials.

Engaging clinicians in the pilot randomized trial

We engaged four outpatient CMHCs in Los Angeles County. Harbor-UCLA Outpatient Mental Health Clinic (Torrance, CA), San Fernando Mental Health Center (Granada Hills, CA), and Long Beach Mental Health Services (Long Beach, CA) are directly operated by the Los Angeles County Department of Mental Health (LAC DMH). Didi Hirsch Community Mental Health Centers (6 clinics in Los Angeles county) are funded by a contract from LAC DMH. The sites were chosen because of their large populations of severely ill patients; the racial, ethnic, and socioeconomic diversity of patients and providers; and their enthusiasm for the study. The study was reviewed and continuously approved by the University of California, Los Angeles’s (UCLA) institutional review board and the LAC DMH research committee.

The principal investigator (PI; DJM) described the study in clinicians’ weekly team meetings at the respective sites. For clinicians expressing interest, dates were set for training workshops. The PI explained to clinicians that they should only sign up for the study if they were willing to be randomly assigned to a high-intensity (in-person workshop) or a low-intensity (online workshop) training, after which they would be expected to enroll training cases and receive teleconference supervision.

Providers from partner CMHCs were eligible for the study if they (a) provided direct mental health care for youth with or at risk for bipolar or psychotic disorders, and (b) were licensed to practice mental health care in the state of California, or were trainees who were eligible to provide services under a licensed provider. Written consent from providers was obtained in-person prior to participation. Study investigators made clear that there would be no adverse consequences to clinicians for not participating. Within each CMHC, a UCLA research coordinator assigned each clinician a training condition, using a 1:1 split based on a computerized random number table. Clinicians and expert supervisors did not know who was assigned to which condition until the clinicians arrived at UCLA for the live or online training.

Training procedures

The high and low intensity training and supervision procedures are summarized in Table 1. Clinicians in both trainings learned how to: (1) introduce the program and engage all family members; (2) provide information to patients and families about the nature, causes, and treatment of mood or psychotic disorders and assist them in developing a relapse prevention plan; (3) administer communication skills training, in which family members and patients are coached to use active listening, deliver positive or negative feedback to each other, communicate clearly, or ask for changes in each other’s behavior; and (4) assist families in identifying, defining, and generating solutions to family problems (e.g., getting to places on time, fulfilling household duties).

Table 1.

High-intensity vs. Low-intensity Clinician Training

	High-intensity training	Hours	Low-intensity training	Hours
Initial Workshop	In-person training with lecture (4 hrs.), video examples (1 hr), and interactions with workshop director and other providers (1 hr.)	6	Online webinar with lecture (4 hrs), without video examples or interactions with director or other trainees	4
Workshop content	Techniques of psychoeducation, communication training, and problem-solving skill training		Same content via webinar
Post-workshop case supervision	30 min weekly by phone for 12 sessions over 18 weeks	6	30 min after every third session for 12 sessions over 18 weeks	2
Total hours		12		6

The trainings differed in two respects: whether the initial workshop was in-person (high intensity) or a pre-recorded online webinar (low intensity) and whether 30-min. telephone supervision occurred after each FFT session (high intensity) or every third FFT session (low intensity). The design of the high intensity condition reflected input from providers and administrators regarding the didactic and supervision parameters of prior EBP trainings, which usually consisted of a one-day workshop followed by weekly supervision for the first training case. Providers explained that once they had achieved minimum fidelity requirements, supervision was typically tapered to once every 3–4 sessions. This frequency of supervision was chosen for the low intensity condition.

Training began with a brief introduction by the study’s PI. Then, in the high-intensity condition, trainees attended a 6-hour live seminar held in a UCLA conference room, consisting of 4 hours of lecture with slides, videotaped examples of family sessions (1 hour), and small group role-plays and question/answer exchanges (1 hour). In the low intensity condition, trainees were instructed to view a 4-hour pre-recorded online training consisting of the same slides and lecture content as the in-person workshop. However, participants could not ask questions during the online training, were not provided with videos of sample sessions, and were not given opportunities to role-play the techniques with other trainees. At the end of both workshops, the PI explained the upcoming study and eligibility criteria for training cases.

Patient participants

The trainees who wished to continue in the trial identified, in consultation with their CMHC teams, one or more patients who would meet the study’s eligibility criteria (below) and provided the patient and/or parents with a study flyer. Clinicians explained to patients (13–25 years) and their parents or other caregivers that they were under no obligation to participate and that the clinician would continue to treat them regardless of whether they chose to do so. If patients signed HIPAA release forms, research staff members contacted them to arrange an initial visit. During this visit, staff members described the study procedures in full and asked patients and family members (parents, stepparents, legal guardians, or siblings with whom the patient lived) to sign university-approved informed assent or consent documents to participate in study assessments and treatment. For adult patients (≥ 18 years), we allowed spouses or significant others to participate in assessments and treatment. Because there were few Spanish-speaking research personnel, we excluded patients who spoke only Spanish, although we did allow for Spanish-speaking parents or grandparents if the enrolled patient spoke English.

A research staff member administered the MINI International Neuropsychiatric Interview (MINI) for children or adults (Sheehan et al., 1998; Sheehan, 2016) to the patient and, for those under age 18, at least one parent about the youth, with diagnoses based on a consensus of youth and parent reports. Based on MINI ratings, participants had to meet lifetime Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV; American Psychiatric Association, 2000) criteria for: (1) bipolar disorder, type I or II; (2) schizophrenic or schizoaffective disorder; (3) major depressive disorder; or (4) other specified bipolar or depressive mood disorder (Birmaher et al., 2009); or psychotic spectrum disorder (onset or worsening in past 12 months of attenuated positive symptoms below threshold for a DSM-IV psychotic disorder; Cannon et al., 2008)). Excluded were participants with a DSM-IV substance or alcohol abuse disorder, autism spectrum disorder, or other neurological disorders that might explain the presence of mood or psychotic symptoms.

Family treatment and clinician fidelity

Once patients were deemed study-eligible, clinicians from both intensity trainings offered families 12 sessions of FFT (8 weekly and 4 biweekly sessions) in the relevant outpatient clinic. Families were unaware of whether the clinician had been assigned to high-intensity or low-intensity training. Clinicians audiotaped the 50-min sessions and uploaded them to a secure server, where they were accessed by expert supervisors from the UCLA team. Clinicians in the high intensity condition received 30-minute supervisory telephone conferences from the PI or another expert supervisor after each of the 12 FFT sessions. Clinicians in low intensity training received 30-minute supervisory telephone conferences after sessions 3, 6, 9 and 12 (Table 1).

Regardless of training condition, expert raters made fidelity ratings from digital audiotapes of sessions 1, 3, 6, 9, and 12 using the Therapy Competence and Adherence Scales, Revised (TCAS-R; Marvin et al., 2016). The TCAS-R consists of 10 Likert-type items rated from 1 (poor] to 7 (excellent) covering clinicians’ adherence to specific elements of treatment (e.g., providing accurate psychoeducation, coaching of communication or problem-solving skill exercises, assigning homework) and competence (skill) in providing this content (e.g., building rapport, structuring and pacing sessions appropriately). Raters also applied an overall (1–7) fidelity rating for each session. Interrater reliability on the TCAS-R between pairs of raters was 0.71 (intraclass r) calculated across the scales (Marvin et al., 2016).

We calculated four factor subscores for each TCAS-R-rated FFT session, based on a factor analytic evaluation of 947 TCAS-R tape ratings from prior randomized trials (Miklowitz & Chung, 2016). These four factors were conceptualized as ‘fidelity components,’ or markers of therapist adherence and skill that may activate mechanisms of change in patients and families. The first factor, quality of communication training, was based on ratings of the clinician’s ability to conduct role-plays and coach family members to use speaking and listening skills. The second factor, directiveness, is based on TCAS items relevant to keeping the session focused and appropriately paced so that its major didactic goals are met. The third component, psychoeducational stance, refers to the degree to which clinicians deliver accurate illness information while also addressing family members’ reactions to the material. Finally, problem-solving orientation is the sum of TCAS items relevant to assisting the family in defining and generating solutions to specific problems. Scores on these four factors were computed for each session in which the relevant TCAS-R items were rated.

Outcome evaluations of participants

Outcome evaluations were conducted with patients at a pre-treatment baseline, the end of treatment (4 months), and when possible, at 12 months. At each time point, patients completed the Patient Health Questionnaire–9 (PHQ-9; Kroenke, Spitzer, Williams, & Löwe, 2010), a standardized measure of depressive symptoms. A trained interviewer who was unaware of clinicians’ training conditions rated patients on the Young Mania Rating Scale (YMRS; Young, Biggs, Ziegler, & Meyer, 1978), covering the prior week. To measure the effects of treatment on interactions within the family, patients and parents were asked to complete the Conflict Behavior Questionnaire (CBQ; Prinz, Foster, Kent, & O’Leary, 1979) at each time point.

Statistical analyses

To evaluate whether the random assignment resulted in balanced groups, we compared clinicians assigned to low- or high-intensity training on demographic variables (age, gender), degree status, and years of experience using χ² and t-tests. Next, we compared patients whose clinicians were assigned to high- vs. low intensity training on diagnoses, age, gender, and baseline symptom severity. We also examined whether clinicians in the two conditions were equally likely to take on a training case (χ² test).

For the primary analyses, we conducted two sets of analyses. First, using univariate analyses of variance, we compared clinicians who received high- and low-intensity training on mean TCAS-R overall scores (scaled from 1– 7) and fidelity component scores, calculated across all sessions for their initial training cases. Sensitivity analyses examined whether fidelity scores differed when considering mean TCAS-R adherence items (e.g., content of psychoeducation) or competence items (e.g., rapport, pacing). Next, using repeated measure mixed regression models, we examined whether fidelity scores varied with phase of treatment (psychoeducation, communication training, or problem-solving) or its interaction with training condition. All of the regression models included random effects for clinicians and patients (nested within clinicians). Because there were no effects of clinic site or training by site interactions, we present the results with site terms removed.

As a final check on fidelity differences between the two training conditions, we determined how many clinicians met operational criteria for quality administration of one full study treatment. Quality administration meant that the clinician (a) completed at least 9 of the scheduled 12 FFT sessions, (b) had a mean TCAS-R score (calculated across sessions) of 4.50 or higher on the 1–7 point overall fidelity scale; and (c) had no sessions with TCAS–R items rated less than 4 (where 4 indicated good fidelity).

Fidelity component scores tended to be positively skewed. Thus, we computed Spearman rank-order correlations for the relationships between changes in fidelity component scores and changes in patients’ symptom scores (PHQ-9 and YMRS) from pre-treatment to post-treatment (4 months). In mixed effect regression models, we examined training condition and study visit (baseline, 4 months) as main and interactive predictors of PHQ-9 and YMRS scores. Lastly, we examined the effects of training condition, study visit and their interaction on patient- and parent-reported Conflict Behavior Questionnaire scores.

RESULTS

Participants

The 47 clinician participants were randomly assigned to high-intensity training (n = 24) or low-intensity training (n = 23) (see Figure 1 and Table 2). Clinicians were diverse in terms of sex (70% female), ethnicity (34% Hispanic), and race (23% Asian, 13% African American, 57% Caucasian, 6% mixed race). A total of 18 clinicians had doctoral or medical degrees (38.3%) and 28 (59.6%) had master’s degrees (1 did not report degree status). There were no differences between the low- and high-intensity groups on these clinician variables (Table 2).

Figure 1.

CONSORT Diagram

Table 2.

Clinician Characteristics (N=47)

	Live Training – High Intensity (n = 24)		Webinar Training – Low Intensity (n = 23)
Characteristics	Mean	SD	Mean	SD	P
Age (years)	35.8	8.1	33.7	4.6	0.28
Years in occupation	5.7	5.5	5.2	2.9	0.68
	N	%	N	%	P
Female	19	79.1	14	60.9	0.21
Race					0.70
Caucasian	6	25.0	5	21.7
Asian	4	16.7	7	30.4
Black or African American	3	12.5	3	13.0
Hispanic or Latino	9	37.5	7	30.4
Other	2	8.3	1	4.3
Education Background1					0.21
Doctorate	7	29.2	11	47.8
Master’s	17	70.8	11	47.8
Training
CBT or other behavioral therapy	24	100.0	23	100.0	1.0
Interpersonal therapy	10	41.7	13	56.5	0.38
Experience in family consultation					0.61
None	1	4.2	1	4.3
1 year or less	4	16.7	6	26.1
2 years – 5 years	12	50.9	8	34.8
6 years – 10 years	4	16.7	6	26.1
More than 10 years	3	12.5	1	4.3
Licensed	11	45.8	12	52.2	0.77
Site					0.79
Harbor/UCLA	10	41.7	10	43.5
Didi Hirsch Mental Health	8	33.3	9	39.1
San Fernando Mental Health Ctr.	3	12.5	1	4.3
Long Beach Mental Health Services	3	12.5	3	13.0
Participated in implementation phase 2	11	45.8	7	30.4	0.28
	Mean	SD	Mean	SD	P
Age, yrs.	33.8	7.4	31.7	3.9	0.11
Years in occupation	3.78	2.53	3.86	1.21	0.94
	N	%	N	%	P
Female, no. (%)	8	72.7	4	57.1	0.49
Experience in family consultation					0.73
≤ 5 years	7	63.6%	5	71.4%
> 5 years	4	36.4%	2	28.6%
Licensed	5	45.5	4	57.1	0.63

¹One clinician did not report this variable.

²One clinician in low-intensity training enrolled a patient who did not attend any sessions.

Feasibility of training and treatment implementation

Following the training day, 18 of the 47 clinicians (36.2%) enrolled at least one patient/family as a training case; 7 of these clinicians had been assigned to low-intensity and 11 to high-intensity training (χ²(1)=1.18, p=0.28). One clinician enrolled a patient who withdrew before the first session, and did not enroll any additional participants. Eight of the remaining 17 clinicians enrolled two or more cases. Enrolling more than one training case was unrelated to training condition (χ²(1)=0.01, p=.91). Clinicians who enrolled at least one training case reported that, in their current practices, they conducted an average of 13.7±12.1 hours/week of family therapy, whereas clinicians who did not enroll training cases conducted an average of 4.8±3.4 hours/week (F[1,46]=13.79, p=.0006). Additionally, clinicians who enrolled training cases had fewer years of occupational experience than those who did not (F[1,42]=4.32, p=0.04). None of the other clinician variables (Table 2) were associated with enrolling a training case.

Clinical and demographic characteristics of the 34 enrolled patients (24 with clinicians in high intensity and 10 with clinicians in low intensity training) are described in Table 3. The average age of patients was 16.5 ± 2.0 years; 15 (44.1%) were girls, 16 (47.1%) were Hispanic, 11 (32.4%) were African American, and 3 (8.8%) reported another racial heritage. The majority of relatives (27 of 49, 55.1%) were mothers. Two families had English-speaking adolescents and one or more Spanish-speaking parents or grandparents. In both of these cases, a clinician delivered the primary interventions in English, with translations given as needed for Spanish-speaking members. Handouts were also translated where appropriate.

Table 3.

Patient Demographics and Illness Characteristics (N = 34)

	Mean	SD
Age, years	16.5	2.0
Average days in study for patients with follow-up data	277	134.3
Patient Health Questionnaire, at baseline
High-intensity group	10.6	7.6
Low-intensity group	7.7	6.9
Young Mania Rating Scale, at baseline
High-intensity group	9.7	6.5
Low-intensity group	11.4	6.6
	N	%
Female gender	15	44.1
Race
Caucasian	20	58.9
African American	11	32.3
Asian	2	5.9
Biracial	1	2.9
Hispanic ethnicity	16	47.1
Primary diagnosis
Schizophrenia	8	23.5
Schizoaffective disorder	3	8.8
Bipolar I disorder	7	20.6
Major depressive disorder with psychotic features	5	14.7
Major depressive disorder without psychotic features	4	11.8
Other specified psychotic spectrum disorder	3	8.8
Other specified mood disorder	3	8.8
Unknown	1	2.9
Participating relatives (n = 49)
Mothers/stepmothers	27	55.1
Fathers/stepfathers	7	14.3
Grandmothers	1	2.0
Sisters	7	14.3
Brothers	6	12.2
Male significant other	1	2.0

Most of the patients were in full or partial symptomatic remission at baseline, with no differences between diagnostic groups in PHQ-9 or YMRS severity scores (Table 3). There were no differences between the two clinician training conditions on any of the patient or family variables in Table 3.

Treatment and study completion

Of the 34 patients, 8 did not attend any FFT sessions (7 with clinicians in high-intensity and 1 with a clinician in low-intensity training; Figure 1). There was no difference between the two training conditions in number of sessions attended by patients (F[1,24]=2.34, p=0.14; Figure 1). Fifteen patients (9 in high, 6 in low) completed all 12 sessions (χ²(1) = 1.45, p=.23). A total of 22 of the 34 patients (64.7%) completed the post-treatment (4 month) follow-up assessments, with no difference in frequency across conditions (9 in low intensity, 13 in high; χ²(1)=2.09, p =0.15). Because only 11 patients completed 12-month visits, the analyses of symptom change were limited to the 22 patients with baseline and post-treatment assessments.

Training condition and FFT fidelity scores

Clinicians in the high-intensity training received approximately 12 hours of training and supervision for their first completed FFT case (Table 1) whereas those who received the low-intensity training received 6 hours. Were these differences in time commitment associated with differences in levels of clinician fidelity?

Based on 119 session ratings of the 17 clinicians who treated at least one patient/family, overall TCAS fidelity scores (scaled from 1 to 7) were consistently between 4 (competent) and 6 (very good) (grand mean = 5.1±0.9). In a mixed effect regression model, we detected no differences between clinicians in the two training conditions on overall fidelity (F[1,94]=0.08, p=.78; Table 4). Of the 17 clinicians, 5 of 11 (45.5%) in high intensity training met criteria for quality administration of at least one full treatment, whereas 5 of 6 (83.3%) in low intensity training met these criteria, a nonsignificant difference (χ²(1)=2.29, p=0.13). Further, clinicians in the high- and low-intensity groups were not distinguishable on any of the fidelity component scores (Table 4), nor on the TCAS-R items measuring competence vs. adherence. There were no effects of patient age, gender, diagnosis, or baseline symptom severity on clinicians’ fidelity scores.

Table 4:

Training Assignment and Fidelity Scores

Fidelity Variable	High-intensity Training (11 Clinicians)			Low-intensity Training (7 Clinicians)			Cohen’s d1	95% CI
	No. Sessions	M	SD	No. Sessions	M	SD	0.23	0.05 to 0.38
Overall Fidelity	81	5.1	1.0	39	5.0	1.2	0.37	0.13 to 0.70
Quality of Psychoeducation	81	5.2	1.0	39	5.2	0.7	0.06	−0.11 to 0.21
Directiveness	81	5.3	0.8	39	5.0	1.0	0.27	0.08 to 0.40
Quality of Communication Training	42	5.0	1.1	19	4.6	1.1	0.37	0.13 to 0.70
Quality of Problem-Solving Training	27	4.9	1.2	7	4.6	1.4	0.29	−0.06 to 0.80

¹Calculated using pooled standard deviations.

Note: All fidelity scores range from 1 (no evidence of fidelity) to 7 (excellent fidelity) on the Therapy Competence and Adherence Scales, Revised. Component scores are based on averages of the individual scale items (all ranging from 1 to 7) composing the component.

In exploratory mixed effect models, we examined whether fidelity scores varied during different phases of treatment. We observed no effects of training condition, treatment phase, or interactions between training condition and phase on overall TCAS-R scores or fidelity component scores. When constraining the mixed models to individual clinicians’ first training cases (61 available session ratings), there were no detectable differences between training conditions or condition by treatment phase interactions on overall fidelity or fidelity component scores (for all comparisons, p > 0.10).

Effects of clinician training on patients’ symptoms and family conflict

There were no effects of training condition or interactions between training condition and study visit (pre-treatment, post-treatment) on PHQ-9 depression or YMRS scores (Tables 5a, 5b). PHQ-9 scores did not change significantly from pre- to post-treatment (estimate −0.52, 95% CI, −3.22 to 2.16), although reductions in YMRS scores approached significance (estimate = −2.38, 95% CI −5.89 to 1.14; F[1,22]=4.1, p=0.05).

Table 5a.

Patients’ PHQ-9 Depression Scores

Training Condition	Baseline N	Baseline mean	SD	4-month N	4-month mean	SD
Low Intensity	10	9.2	8.1	9	6.6	5.3
High Intensity	23	10.5	6.6	13	8.8	5.0

Ranges for the PHQ-9 are from 0 to 27. Scores of 5–9 indicate mild, 10–14 moderate, 15–19 moderately severe, and ≥ 20 severe depression. Results of mixed effect regression model comparing training conditions, with patients nested within clinicians, indicated no main effect of training condition, F[1, 21] = 0.11, p = 0.75; study visit, F[1,21] = 2.06, p = 0.17; or training condition by study visit interaction, F[1,21] = 0.00, p = 0.99.

Table 5b.

Patients’ YMRS Mania Scores

Training Condition	Baseline N	Baseline mean	SD	4-month N	4-month mean	SD
Low Intensity	10	11.4	6.6	9	7.3	6.6
High Intensity	24	9.5	6.3	13	7.2	6.3

YMRS scores can range from 0 to 60. Scores < 12 indicate minimal or no mania; 12–19 indicate hypomania or mild symptoms; scores of 20–25 indicate mild but fully syndromal mania, 26–37 moderate mania, and 38–60, severe mania. Results of a mixed effect regression model comparing training conditions, with patients nested within clinicians, indicated no differences between training conditions, F[1,22] = 0.57, p = 0.46; an overall reduction in scores from baseline to 4 months, F[1,22] = 4.1, p = 0.05; and no interaction of training condition with study visit, F[1,22] = 0.3, p = 0.61.

No relationships emerged between fidelity scores and changes in PHQ-9 or YMRS scores over the course of treatment. However, two TCAS-R items – effectiveness in assigning and following up on homework practices (Spearman’s rho = −0.58, p=.009) and session command (rho = −0.59, p=0.008) – were associated with decreasing YMRS scores. Total patient-rated Conflict Behavior Questionnaire (CBQ) scores dropped from pre- to post-treatment (est. = −2.29, 95% CI −4.08 to −0.49, p=.008; Table 5c), as did parent-rated CBQ scores (est. = −2.88, −5.76 to −0.0; p=0.02; Table 5d), suggesting less family conflict over the course of treatment. There were no effects of training condition and no training condition by visit interactions on changes in patient- or parent CBQ scores.

Table 5c.

Patient-Rated Conflict Behavior Questionnaire Scores

Training Condition	N at Baseline	Baseline mean	SD	N at 4 months	4 months mean	SD
Low Intensity	8	5.1	4.8	8	2.8	2.8
High Intensity	17	7.8	5.2	13	6.1	5.2

Ranges for this scale are 0 to 20, with no established cutoffs. A mixed effect regression model comparing training conditions, with patients nested within clinicians, indicated no effect of training condition, (F[1,19] = 0.89, p = 0.36; an overall reduction in family conflict from baseline to 4 months, F[1,19] = 8.68, p = .008; and no interaction of training condition with study visit (F[1,19] = 0.02, p = 0.88).

Table 5d.

Parent-Rated Conflict Behavior Questionnaire Scores

Training Condition	N at Baseline	Baseline mean	SD	N at 4 months	4 months mean	SD
Low Intensity	8	8.6	7.2	8	5.5	6.4
High Intensity	17	10.0	6.1	12	7.8	5.3

Mixed effect regression model comparing training conditions, with parent reporters nested within clinicians: training condition, (F[1,18] = 0.13, p = 0.72; study visit, F[1,18] = 6.25, p = .02; interaction of training condition with study visit (F[1,18] = 0.00, p = 0.99).

DISCUSSION

The training, supervision, and fidelity requirements of many EBPs can be barriers to their implementation in community settings. In this pilot randomized trial, we compared two levels of training intensity on the fidelity of community practitioners to FFT for adolescents and young adults with mood or psychotic spectrum disorders. Our findings suggest that disseminating FFT (i.e., training clinicians and recruiting, treating, and following patients) is feasible using standardized methods of provider education and supervision. We conclude that lower intensity training in FFT (4-hour online training, supervision after every 3 sessions) is acceptable to clinicians and associated with levels of fidelity that are comparable to those achieved by clinicians who attend a 6 hour workshop followed by higher intensity (i.e., weekly) case supervision.

Results both converge and diverge from findings of a study of online versus in-person clinician training in CBT for youth anxiety (Beidas, Edmunds, Marcus, & Kendall, 2012). Similar to our findings, there was no significant effect of training modality on therapy adherence or skill from pre- to post-training. However, this previous study found that higher doses of supervision following training were associated with higher fidelity scores at a 3-month follow-up. The present study did not examine post-training adherence to FFT training, and it is possible that differences in treatment fidelity would have emerged later.

The current study found that patients whose clinicians received low-intensity training completed a comparable number of FFT sessions and had equivalent symptomatic outcomes to patients whose clinicians received high-intensity training. Families in both conditions reported comparable decreases in conflict. Independent of training intensity, clinicians who had greater command over sessions (e.g., re-focused sessions when they were derailed) and regularly assigned and followed up on homework practices had patients who showed greater improvement in mania/hypomania symptoms. Possibly, clinicians’ efforts to structure and direct family therapy sessions may be less challenging with patients who show greater symptom stabilization during treatment. A limitation of the study design is that it did not allow us to evaluate the causal relationship between fidelity elements and improvement in patients’ symptoms or family environments.

Providers in urban, safety net CMHCs often have large caseloads and have to limit their training and supervision hours unless these hours are reimbursable. Many of the clinicians in this study were already engaged in training in other EBPs. Clinicians could decide whether they wanted to take on one or more FFT training cases, and it is notable that only 18 of 47 did so, independent of training intensity. Nonetheless, clinicians who regularly provided family therapy – and those who were earlier in their career trajectories - were more likely to take on an FFT training case. Community agencies may want to focus FFT training opportunities on preselected clinicians who are familiar with family therapy and express a commitment to learning new approaches. In order to incentivize the uptake of FFT or similar evidence-based approaches in community care agencies, state Medicaid regulators (and in California, the Mental Health Services Oversight Commission) may need to assess whether such approaches improve clinical outcomes or social outcomes (e.g. justice system involvement, homelessness, school performance) or decrease utilization of high cost service such as emergency service or inpatient hospitalizations (Chung et al., 2014).

Limitations

In an era where priority is placed on distance learning, there is likely to be greater endorsement of online and “on-demand” provider training and supervision. Although this study cannot identify which elements of provider training are most beneficial in assuring implementation fidelity, it appears that trainee/trainer interaction in an in-person workshop does not offer clear advantages over a standardized pre-recorded online format. Further, relatively infrequent check-ins for supervision may be adequate for clinicians’ initial provision of FFT. Because we conceptualized this study as a pilot trial, we cannot answer more nuanced questions about whether the effects of training endure beyond training. Future trials may be able to determine the conditions under which clinicians continue to offer FFT to patients well after the training period, or whether fidelity of implementation degrades over time as clinicians learn other EBPs.

The study was not designed to determine the clinical effectiveness of FFT in community settings. Doing so would have required enrolling patients who were more variable in baseline symptom states, as well as randomizing a subgroup of patients to usual care without FFT. Given the limited sample size, we were underpowered to examine interactions between clinician training and patient factors (e.g., diagnosis, symptom severity, comorbid disorders) or other treatment variables (e.g., medication regimens, receiving other forms of therapy) on patients’ improvement or clinicians’ fidelity scores. These are critical issues to pursue in future trials.

Conclusions

Our results are promising in suggesting that low-intensity FFT training is feasible and associated with high levels of fidelity in implementation. Although training in any EBP will require release time for clinicians, this initial investment in FFT by community care agencies may pay off in the form of better long-term outcomes among young patients with mood and psychotic disorders (Miklowitz & Chung, 2016). Future studies should explore whether clinicians are able to implement FFT (or other EBPs) effectively when supervision is offered in digital format (i.e., through email or chatroom correspondence) or by the provision of comprehensive training manuals with video examples illustrating the broad array of challenges encountered by community clinicians (Fairburn and Patel, 2017).