Reducing Preschool Behavior Problems in an Urban Mental Health Clinic: A Pragmatic, Non-Inferiority Trial

Abstract

Objective:

This pragmatic, randomized, non-inferiority trial compared the effectiveness and cost of group-based parent management training with mastery-based individual coaching parent management training in a low-income, predominantly African American sample.

Method:

Parents seeking treatment for their 2- to 5-year-old children’s behavior problems in an urban fee-for-service child mental health clinic were randomized to the Chicago Parent Program (CPP; n = 81) or Parent–Child Interaction Therapy (PCIT; n = 80). Consent followed clinic intake and diagnostic assessment and parent management training was delivered by clinicians employed at the clinic. Primary outcome measures were externalizing child behavior problems, assessed at baseline and postintervention follow-up, using the Child Behavior Checklist (CBCL) and average per-participant treatment cost.

Results:

Data from 158 parents were analyzed. Most were mothers (75.9%), African American (70.3%), and economically disadvantaged (98.7% Medicaid insured). Of children, 58.2% were boys, and mean age was 3.6 years (SD 1.03). Based on CBCL scores, behavior problems improved in the 2 conditions (Cohen d = 0.57 for CPP and 0.50 for PCIT). CPP was not inferior to PCIT (90% CI −1.58 to 4.22) at follow-up, even after controlling for differences in treatment length (90% CI −1.63 to 4.87). Average per-participant treatment cost was higher for PCIT (mean $2,151) than for CPP (mean $1,413, 95% CI −1,304 to 2212170).

Conclusion:

For parents of young children living in urban poverty, CPP is not inferior to PCIT for decreasing child behavior problems. CPP requires less time to complete and costs a third less than PCIT.

Parent management training (PMT) is a well-established approach for treating child behavior problems^1–3 and the first line of treatment for preschoolers with behavior problems.⁴ However, PMT has been consistently less effective for children from economically disadvantaged families.^5–7 The most common explanation for these differential effects by income level relate to difficulties parents might have incorporating the new strategies, persisting with treatment, and struggling with multiple adversities associated with poverty.^5,8 Acceptability of treatment models also might affect uptake and benefit; a meta-analysis found that low-income children whose parents received individually administered PMT benefitted more than those who received group-based PMT.⁶ Although cultural adaptations to existing PMT programs have been studied, there is limited evidence that these adaptations are more effective and acceptable than their original versions for low-income families, many of whom are from underserved racial and ethnic populations.^9–12

The decreased benefit of PMT for families living in poverty is an important problem because it contributes to widening mental health disparities by income, race, and ethnicity.^13–15 In the United States, children living in poverty are significantly more likely to be African American or Hispanic, raised by a single parent, report at least 1 adverse childhood experience, and have behavior or conduct problems.¹⁶ Untreated or undertreated child behavior problems are frustrating and demoralizing for children and families and costly to society because they contribute to a cascade of health and social problems affecting children’s long-term health, well-being, and prospects for economic self-sufficiency.¹⁷

This pragmatic randomized controlled trial compared the effectiveness and cost of 2 evidence-based PMT programs in a sample of low-income, predominantly African American parents seeking treatment for their preschool children’s behavior problems. The Chicago Parent Program (CPP), a 12-session parent group-based PMT, was compared with Parent–Child Interaction Therapy (PCIT), an individual mastery-based parent–child coaching PMT. These 2 programs are manual based, teach the same core behavioral strategies, assign “homework” for practicing the skills between sessions, and have been shown to improve parenting skills and decrease behavior problems in young children 2 to 5 years old.^7,18 However, the programs differ in their delivery models, treatment duration, and clinician training requirements, which have implications for their cost, effectiveness, and acceptability to parents.^19,20

PCIT was developed in the late 1960s and is considered to be among the most efficacious PMT programs available for young children with externalizing behavior problems.²¹ The CPP^22–24 was developed in 2002 with an advisory board of low-income African American and Latino parents to address the gap in evidence-based PMT programs for parents of color raising young children in low-income communities. The primary study hypothesis was that CPP would not be inferior to PCIT for decreasing behavior problems and would cost less when offered to low-income urban families seeking treatment for their children’s behavior problems. To inform practice, this pragmatic non-inferiority trial was conducted in an urban fee-for-service mental health clinic using clinicians employed at the clinic who had been trained in 1 of these 2 PMT programs. Details of the protocol are published elsewhere.¹⁹

As previously reported,²⁰ we found no significant differences in PMT initiation or completion rates by treatment condition. Of those randomized to CPP, 63.3% initiated treatment and 32.0% of those families completed CPP. Of those randomized to PCIT, 60.8% initiated treatment and 31.3% of those families completed PCIT. CPP took less time to complete and parents with higher baseline depression scores were less likely to drop out of CPP than out of PCIT. The present study builds on these findings by comparing the effectiveness of these 2 PMT programs for decreasing externalizing child behavior problems and their costs. Our goal was to inform treatment decisions for urban child mental health clinics serving predominantly low-income children of color presenting with externalizing behavior problems.

METHOD

Participants

Parents of young children were recruited from an urban clinic after intake and the clinic’s standard diagnostic assessment protocol (ie, Diagnostic Infant Preschool Assessment, a comprehensive psychosocial, psychiatric, medical, and developmental assessment) was conducted by a psychologist or social worker. Parents met inclusion criteria if they were the legal guardian of a 2- to 5-year-old child presenting with behavior problems, spoke English, were willing to be randomized to PCIT or CPP, and did not have an illness or cognitive disability that would interfere with their ability to fully participate in PMT. Consistent with eligibility guidelines for CPP and PCIT, children who were actively psychotic; suicidal; diagnosed with autism or pervasive developmental disorder; or had a congenital, genetic, or sensory anomaly affecting their ability to participate in PMT were excluded from the study and referred for other services. Study consent was conducted by the project manager who was a clinician at the site. Randomization occurred in blocks of 10, allowing time to enroll a full group of 10 CPP participants while those randomized to PCIT received treatment. Once consented, participants were informed of their PMT assignment but not the study hypotheses. Mean length of time from consent to treatment initiation was 44.4 days (SD 32.22) for CPP and 68.9 days (SD 51.41) for PCIT. Overall wait time from phone intake to the first scheduled treatment session was 112.3 days (SD 71.73) for CPP and 139.6 days (SD 76.67) for PCIT.

Interventions

The 2 PMT treatments being compared, CPP and PCIT, were previously described and validated against treatment-as-usual conditions.^22,25,26 Several important features distinguish these 2 PMT approaches. The most important differences relate to the delivery format and treatment duration.²⁰ PCIT is delivered individually in 1-hour sessions to a parent–child dyad by a trained clinician who coaches the parent from behind a 1-way mirror (the parent wears a Bluetooth device). Treatment progresses at the rate with which parents can master each new skill. In contrast, CPP is delivered in 12 2-hour weekly sessions in groups of approximately 10 parents led by 2 trained clinicians. Food is provided and children are not present in the groups. Parents learn vicariously by watching and discussing video vignettes of parent–child models engaged in common situations typical of families with young children. CPP modules focus on helping parents clarify their child-rearing goals and values, learning positive attention and limit-setting strategies to achieve their goals, stress management techniques, and problem-solving skills. Make-up sessions were not provided to parents who missed CPP sessions.

All PMT sessions were conducted by licensed, graduate-prepared clinicians employed at the clinic who were previously trained in 1 of the 2 treatments. Participants were compensated for completing research assessments; however, consistent with a pragmatic trial design, no additional resources beyond what was normally provided by the clinic were included in the trial. Clinicians only implemented CPP or PCIT and parents only participated in the PMT treatment to which they had been randomly assigned. If needed, parents also could receive individual therapy sessions for support or crisis intervention. Reasons for additional sessions were concerns related to the child’s health or mental health (70.0%); housing, school, insurance, or system problems (53.0%); parent mental health (52.4%); and safety concerns (10.7%). These additional sessions were included in treatment costs.

Four clinicians implemented 8 CPP groups. To assess CPP fidelity, clinicians submitted audio recordings of all CPP groups to an independent monitor and 20% were randomly selected for quality checks.²⁷ CPP fidelity was high (mean adherence score 85%, SD 1.0; mean quality score 2.7, possible range 1.0–3.0, SD 0.24).

Four clinicians led all PCIT sessions and submitted video recordings of their sessions to PCIT International for independent fidelity assessments. Although the fidelity data were not made available to the study team, all 4 clinicians became certified PCIT providers during the course of this study, signifying high fidelity.

Assessments and Procedures

The primary outcome variable, externalizing child behavior problems, was assessed using the preschool version of the Child Behavior Checklist (CBCL). The CBCL is a well-established, standardized parent-report measure used to assess problem behavior in children referred for mental health treatment.²⁸ Its validity across racial, ethnic, and income groups has been demonstrated.²⁹ Participants were assessed at baseline, after discharge, and 4 months after discharge. This study reports treatment effects only for the CBCL externalizing scores, from baseline to the first postdischarge assessment.

To assess level of psychosocial risk, symptoms of parent depression were assessed at baseline using the Center for Epidemiologic Studies Depression Scale–Revised (CESDR).³⁰ This 20-item measure designed for the general population assesses the extent to which respondents have experienced a range of depressive symptoms during the previous 2 weeks. We previously reported that parents with higher baseline CESDR scores were more likely to drop out of PCIT but not out of CPP.²⁰ Therefore, baseline CESDR scores were controlled in analyses of PMT effects on child behavior outcomes.

We measured the direct costs (ie, direct monetary expenditures) and indirect costs (ie, additional resource costs) to the provider and participants associated with participation in CPP compared with PCIT, from baseline to postdischarge follow-up. PMT provider costs included the dollar value of the amounts of time different clinicians spent in PMT sessions and the additional time clinicians needed outside PMT sessions to complete paperwork and additional follow-up for each participant. Time spent in these activities was reported by clinicians during in-person interviews completed by research project staff. Dollar-valued time costs were constructed by multiplying clinicians’ hours spent in specific activities by an appropriate hourly wage plus imputed dollar values for employer payroll tax and fringe benefit costs. Provider expenses for other mental health services participants received during the study (eg, individual therapy, medication management) were captured from the provider’s administrative billing data. Other provider costs included the imputed costs of space needed for delivery of PMT sessions and cost of supplies and other clinical, administrative, and travel supports. Space costs were calculated using internal accounting information on clinic building costs per square foot and allocated proportionally to treatments based on PMT session time. Other costs were obtained from project expense records and accounting information from the provider organization. Participants’ costs included the costs of foregone paid worktime and out-of-pocket costs for transportation and child care. All costs are expressed in constant 2016 dollars.

Data Analytic Strategy

Only parents who attended at least 1 PMT session completed postdischarge follow-up assessments because there was no reason to expect the effects of no PMT exposure would differ by condition. Thirty-two (40.5%) participants enrolled in PCIT and 29 (36.7%) enrolled in CPP never attended the PMT. Of those who started PMT, 4 (5.1%) PCIT participants and 3 CPP (3.8%) participants were lost to follow-up. Therefore, 43% of follow-up CBCL missing values were imputed for this analysis. Multiple imputations, 5 imputations with a fully conditional specification method, were used to perform an intent-to-treat analysis comparing child behavior problems and treatment costs for all parents randomized to treatments. The imputation method included indicator variables for CPP blocks to account for a possible intra-cluster correlation within a block.

Originally we intended to recruit 210 parents with 95% power to detect a non-inferiority margin of ¹/₂-SD difference in postdischarge CBCL scores, a difference determined to represent a clinically significant difference (margin). However, changes in the clinic population led to a smaller pool of eligible families. A revised target sample size was estimated based on achieving at least 85% power to detect a ¹/₂-SD difference in CBCL scores at discharge, a difference determined to be clinically meaningful. A minimum sample size of 154 (n = 77 per condition) would be sufficient to detect a clinically meaningful non-inferiority margin of [ −5, 5] in child behavior problems (ie, 1 SD = 10 on CBCL).

Although CPP is a group-based intervention, analyses using intra-cluster correlation showed no significant correlation among CPP participants within a clustered group on behavior change scores (estimated intra-cluster correlation −0.07, p .908). Therefore, CPP data were analyzed at the individual level as independent units of analyses. Baseline characteristics were tested for balance between conditions using 2-sided t tests (or χ² test, depending on variable type) at an α value of 0.05. To compare mean CBCL scores by condition, we conducted 1-sided independent-sample t tests (α = 0.05), equivalently a 90% 2-sided CI, or a 95% 1-sided confidence bound, at postdischarge follow-up and for change in CBCL scores from baseline to postdischarge follow-up. Effect sizes were estimated within and between groups using Cohen d for unadjusted comparison and Cohen f² for adjusted analyses.

Differences in PMT duration contributed to wide variability in the length of time from baseline to postdischarge follow-up for PCIT (mean 389 days, SD 226.2) and CPP (mean 157 days, SD 56.3; p < .001), creating significant differences in child maturation rates by condition. Adjusted analyses controlling for number of days from baseline to postdischarge follow-up were run using general linear models. Separate adjusted analyses controlling for parents’ baseline depression on CBCL scores were conducted using similar methods. We used SAS 9.4 (BASE and STAT) for Windows 9.4 (SAS Institute, Cary, NC) for all data management and statistical analyses.

RESULTS

Participant Flow

From January 2, 2012 to August 1, 2016, 691 parents of 2-to 5-year-old children presenting with behavior problems were screened from intake records (Figure 1). Final eligibility was determined after diagnostic assessment. Of the 353 who met eligibility criteria, 126 (35.7%) were triaged by the diagnostic clinician to a different trauma-specific treatment based on the child’s primary presenting problem (eg, recent exposure to sexual abuse). The remaining 161 parent–child dyads were approached for consent; all agreed to participate. After randomization, 2 CPP parents1 withdrew and did not complete baseline assessments. Participants were assigned to the next available clinician trained in CPP or PCIT, who contacted the parent to schedule the first appointment. In accord with clinic protocol, all parents met individually with their assigned clinician to discuss the treatment plan before starting PMT. One PCIT participant did not complete a baseline CBCL and was excluded from analyses, leaving a final analyzable sample of 158 parents (n = 79 per condition).

FIGURE 1.

Participant Flow

Note: CBCL = Child Behavior Checklist; CPP = Chicago Parent Program; PCIT = Parent–Child Interaction Therapy; PDD pervasive developmental disorder.

Baseline Characteristics

Parent and child baseline characteristics are listed in Tables 1 and 2, respectively. There were no significant differences (p > .05) by condition in parent or child demographic characteristics, parents’ baseline depression scores, or child diagnoses. Most parents were the target child’s mother (75.9%), were African American (70.3%), had a high school diploma or less (58.2%), and reported annual incomes lower than $20,000 (72.8%). Nearly all children (98.7%) were insured by Medicaid and some (12.7% of CPP and 8.9% of PCIT) received psychotropic medication.

TABLE 1.

Parent Demographic Characteristics and Baseline Depression by Condition (N = 158)

Characteristics	CPP (n = 79)	PCIT (n = 79)	p
Relationship to target child, n (%)			.348a
Mother	59 (74.7)	61 (77.2)
Father	8 (10.1)	5 (6.3)
Foster parent	3 (3.8)	0 (0.0)
Grandmother	5 (6.3)	7 (8.9)
Aunt	1 (1.3)	4 (5.1)
Other	3 (3.8)	2 (2.5)
Parent ethnicity, n (%)			.443a
Non-Hispanic	74 (93.7)	77 (97.5)
Hispanic or Latino	5 (6.3)	2 (2.5)
Parent race/ethnicity, n (%)			.128a
Black or African American, non-Hispanic	53 (67.1)	58 (72.5)
White, non-Hispanic	16 (20.3)	18 (22.8)
Other	10 (12.7)	3 (3.8)
Highest level of education, n (%)			.197a
High school/GED or less	48 (60.8)	44 (55.7)
Associate/vocational degree or some college	28 (35.4)	26 (32.9)
Bachelor’s or graduate degree	3 (3.8)	9 (11.4)
Current employment status, n (%)			.930a
Working	24 (30.4)	27 (34.2)
In school only	2 (2.5)	3 (3.8)
Not working	49 (62.0)	45 (57.0)
Other	4 (5.1)	4 (5.1)
Marital status, n (%)			.366a
Married	17 (21.5)	13 (16.5)
Unmarried	43 (54.4)	42 (53.2)
Unmarried, living with partner	17 (21.5)	17 (21.5)
Other	2 (2.5)	7 (8.9)
Annual household income, n (%)			.149a
<$20,000	60 (75.9)	55 (69.6)
$20,000–$40,000	13 (16.5)	10 (12.7)
>$40,000	6 (7.6)	14 (17.7)
Baseline depression (CESDR), mean (SD)	17.2 (16.02)	15.8 (14.28)	.555b

Note: CESDR = Center for Epidemiologic Studies Depression Scale–Revised; CPP = Chicago Parent Program; GED = General Educational Development; PCIT = Parent–Child Interaction Therapy.

^aBy χ², or Fisher exact test as appropriate.

^bBy t test; all 2-tailed tests.

TABLE 2.

Child Characteristics by Condition (N = 158)

Child Characteristics	CPP (n = 79)	PCIT (n = 79)	p
Sex, n (%)			.197
Boys	42 (53.2)	50 (63.3)
Girls	37 (46.8)	29 (36.7)
Age (y), mean (SD)	3.6 (1.08)	3.7 (0.98)	.489
Primary diagnosis, n (%)			.893
ADHD	11 (13.9)	9 (11.4)
Adjustment disorder	43 (54.4)	44 (55.7)
Disruptive behavior disorder	9 (11.4)	11 (13.9)
Oppositional defiant disorder/conduct disorder	9 (11.4)	8 (10.1)
Posttraumatic stress disorder	6 (7.6)	4 (5.1)
Separation anxiety disorder	1 (1.3)	3 (3.8)

Note: The data were analyzed using t test, χ² test, or Fisher exact test as appropriate (all 2-tailed tests). ADHD = attention-deficit/hyperactivity disorder; CPP = Chicago Parent Program; PCIT = Parent–Child Interaction Therapy.

Comparison of Child Behavior Problems

Baseline CBCL mean scores were not significantly different between CPP (mean 68.5, SD 12.4) and PCIT (mean 65.7, SD 11.5; Table 3). CBCL scores decreased from baseline to postdischarge follow-up in the 2 conditions, with similar decreases for CPP (mean −7.2, SD 8.91; Cohen d = 0.57) and PCIT (mean −5.7, SD 10.57; Cohen d = 0.50; 90% CI −4.31 to 1.29). After adjusting for differences in child maturation on decreases in CBCL means, CPP was found to be non-inferior to PCIT (90% CI 1.63 to 4.87). After adjusting for parent depression on decreases in CBCL means, CPP was found to be non-inferior to PCIT (90% CI −1.94 to 3.89).

TABLE 3.

Mean Comparisons for Child Behavior Checklist (CBCL) Scores by Condition (N = 158)

	CPP (n = 79)		PCIT (n = 79)		Mean Difference (CPP vs. PCIT)	90% CI
CBCL Score	Mean	SD	Mean	SD
Baseline	68.5	12.35	65.7	11.47	2.84	−0.91 to 6.58
Postdischarge follow-up CBCL	61.4	10.42	60.0	10.01	1.32	−1.58 to 4.22
Postdischarge CBCL adjusted for child maturation					1.62	−1.63 to 4.87
Postdischarge CBCL adjusted for parent depression					0.97	−1.94 to 3.89
Change from baseline to postdischarge follow-up	−7.2	8.91	−5.7	10.57	−1.51	−4.31 to 1.29
Change from baseline to postdischarge follow-up adjusted for child maturation					1.62	−1.63 to 4.87
Change from baseline to postdischarge follow-up adjusted for parent depression					0.98	−1.94 to 3.89

Note: The p value for comparing conditions on mean CBCL externalizing score at baseline was .137, with 95% CI for baseline comparison only because this comparison was not tested for non-inferiority; all other CIs are 90% for non-inferiority. Means are for standardized T scores on the CBCL externalizing scale. CPP = Chicago Parent Program; PCIT = Parent–Child Interaction Therapy.

After comparing CBCL means at postdischarge follow-up only, CPP was found to be non-inferior to PCIT for child behavior problems (90% CI −1.58 to 4.22; Cohen d = 0.14). Non-inferiority was maintained when CBCL scores were adjusted for child maturation (−1.63 to 4.87; Cohen f² = 0.01) and parent depression (−1.94 to 3.89; Cohen f² = 0.03; 90% CI). As exploratory analyses, post hoc comparisons of CBCL subscale T scores (emotionally reactive, attention problems, and aggressive behavior sub-scales) were conducted, detecting no significant differences by treatrment arm. Table S1, available online, presents a comparison of CBCL externalizing T score means (SDs) based on per-protocol, single-imputation, and imputation analyses.

To assess whether behavioral improvements were clinically meaningful, we compared the number of children whose CBCL scores remained unchanged, improved, or worsened from baseline to postdischarge follow-up using CBCL cut-points established for clinical, borderline, and normal ranges (Table 4).²⁸ Using these categories, 43 (54.4%) CPP and 48 (60.8%) PCIT children had CBCL scores in the normal range at baseline and at postdischarge follow-up. Ten (12.7%) CPP and 6 (7.6%) PCIT children with scores in the borderline or clinical range at baseline remained unchanged at postdischarge follow-up. Of the remaining children, 24 (30.4%) in CPP and 22 (27.9%) in PCIT improved their clinical status from the clinical or borderline range at baseline to the normal range at post-discharge follow-up. Scores worsened for 2 CPP (2.5%) and 3 PCIT (3.8%) children whose scores changed from the normal to borderline range or the borderline to clinical range; none worsened from the normal range at baseline to the clinical range after discharge.

TABLE 4.

Change in Child Behavior Clinical Status From Baseline to Postdischarge Follow-up by Condition (N = 158)

Change Pattern	Clinical Status Change From Baseline → Follow-Up	CPP (n = 79)		PCIT (n = 79)		p
		n	%	n	%
Clinical status remained in normal range	Normal → Normal	43	54.4	48	60.8	.896a
Clinical status did not improve after treatment	Borderline → Borderline	9	11.4	6	7.6
	Clinical → Clinical	1	1.3	0	0
Clinical status improved after treatment	Clinical → Borderline	3	3.8	1	1.3
	Clinical → Normal	2	2.5	2	2.5
	Borderline → Normal	19	24.1	19	24.1
Clinical status got worse after treatment	Normal → Clinical	0	0	0	0
	Normal → Borderline	2	2.5	2	2.5
	Borderline → Clinical	0	0	1	1.3

Note: CPP = Chicago Parent Program; PCIT = Parent–Child Interaction Therapy.

^aBy Fisher exact test.

Comparison of Costs

Mean provider and participant costs of the 2 interventions from baseline to postdischarge follow-up are presented in Table 5. Average PMT session attendance was 7.5 (SD 3.8) 2-hour sessions for CPP and 17.3 (SD 14.1) 1-hour sessions for PCIT. Total PMT costs for CPP were approximately half those for PCIT sessions ($850 for CPP versus $1,670 for PCIT; z = 3.97, p < .001). However, the use of additional non-PMT mental health treatment services was nominally greater in CPP compared with PCIT. Thus, the average overall mean treatment costs of CPP were $1,413 per participant, whereas the overall mean treatment costs of PCIT were $2,151 per participant, for a difference of −$738 (or −34.3%; 95% CI −1,304 to −170; z = −2.55, p = .011) favoring CPP.

TABLE 5.

Average Treatment Costs per Participant by Parent Management Training (PMT) Condition (in 2016 dollars)

	CPP (n = 79)	PCIT (n = 79)	Difference	95% CI		z	p
				LL	UL
Total treatment cost	1,413	2,151	−738	−1,304	−170	−2.55	.011
Total PMT costs	850	1,670	−820	−1,233	−407	−3.87	<.001
Provider costs	811	1,620	−809	−1,208	−409	−3.97	<.001
Parenting session clinician time (non-MD)	225	634	−409	−558	−261	−5.41	<.001
Clinician time before and after PMT (preparation, documentation)	233	419	−186	−291	−80	−3.43	.001
PMT space and supplies	94	33	61	39	83	5.45	<.001
Clinical, administrative, and parent travel supporta	259	534	−275	−406	−144	−4.11	<.001
Participant costsb	39	50	−11	−25	3	−1.55	.121
Lost worktime	29	26	3	−6	12	0.68	.498
Travel, out-of-pocket costs	4	10	−6	−9	−4	−5.00	<.001
Child care	6	14	−8	−11	−4	−4.25	<.001
Additional non-PMT mental health treatmentc	563	481	82	−81	246	0.99	.321
Psychotherapy (non-MD)	444	324	120	−1	242	1.94	.052
Psychiatrist services (MD)	119	157	−38	−81	6	−1.69	.091

Note: CPP = Chicago Parent Program; LL = lower limit; MD = medical doctor; PCIT = Parent–Child Interaction Therapy; UL = upper limit.

^aThese provider cost estimates include only the direct costs of the 2 PMT interventions for clinical, administrative, and travel support expenses (ie, for travel subsidies clinic provides to clients living within 20-mile radius).

^bThese participant costs include the direct costs of travel to and from PMT sessions. However, participants might have incurred additional travel- and employment-related costs when traveling to and from non-PMT mental health appointments. Additional participant costs incurred as a result of attending additional non-PMT mental health care appointments are not included in our cost estimates. This approach assumes that participants in the 2 groups had similar costs associated with attending outpatient mental health appointments.

^cProvider might have incurred additional clinical and administrative expenses related to children’s participation in outpatient mental health treatments. These administrative and travel costs were netted out of cost estimates based on the assumption that the administrative and travel costs of additional non-PMT therapy sessions and psychiatrist services are approximately the same in the 2 PMT conditions. The measured difference in costs for outpatient mental health encounters during the period of intervention was not statistically significant and might be inconsequential. “Child therapy” services in the CPP group cost $120 more per child ($444 per child in CPP versus $324 per child in PCIT; z = 1.94, p = .05), whereas psychiatrist encounters in the CPP group cost $38 less per child ($119 per child in CPP versus $157 per child in PCIT; z= −1.67, p= .09). These differences could be underestimated because they might not include intervention-related indirect cost differences for administration or travel support.

Startup costs (eg, training, licensing fees) were obtained but not included in these treatment costs. Total per-clinician startup costs were $2,700 for CPP and $9,775 for PCIT (supplemental materials comparing startup costs available upon request).

DISCUSSION

CPP does not appear to be inferior to PCIT for decreasing preschool child behavior problems in low-income and pre-dominantly African American families. These results held true even after controlling for differences in PMT treatment length.

Per-participant cost for CPP alone was nearly 50% less than the cost for PCIT ($849 versus $1,669). The lower cost for CPP is likely due to the fact that the intervention is delivered in groups led by 2 clinicians and constrained to 12 2-hour sessions, whereas PCIT is delivered individually to parent–child dyads in 1-hour sessions by 1 clinician and continues for as long as it takes the parent to master the new skills. Although the difference was not statistically significant, parents randomized to CPP received more non-PMT mental health treatment services than did those randomized to PCIT. With non-PMT mental health treatment included, average participant treatment cost for CPP was approximately one-third less than the cost for PCIT. Group formats can limit the amount of attention parents receive for managing individual issues, which can contribute to the additional treatment costs. In this clinic, additional sessions could be initiated by the parent or clinician, so it is unclear whether parents actively sought the additional help or clinicians perceived the need for additional check-in appointments with families engaged in group treatments.

The lower CPP cost suggests that CPP might be a cost-effective strategy as a first line of PMT for behavior problems, followed by PCIT if additional treatment is still needed. Future research is needed to assess the benefit of a 2-stage PMT model for low-income urban families.

There are a number of important study limitations. First, this trial was conducted at a single fee-for-service clinic located in the US mid-Atlantic region, qualities that could affect treatment costs and generalizability. Future research using multiple sites and regions would enhance generalizability. Second, we used only 1 parent-reported indicator of treatment outcome, namely child behavior problems. Although the CBCL is one of the most frequently used measures for assessing PMT effectiveness,^31,32 testing multiple outcomes from multiple informants beyond discharge would expand our knowledge on how these treatments compare for low-income urban families. Third, it was not possible to blind treatment assignment, potentially leading to bias. Notably, no significant differences in baseline demographic or mental health variables were found, suggesting that assignments to condition were unbiased.

All children presented with behavior problems at enrollment and received a psychiatric diagnosis. However, only 40% scored in the borderline or clinical range on the CBCL at baseline. Given the pragmatic design,³³ we recruited all parents of preschoolers whose primary complaint was child behavior problems and without comorbidities contraindicated for these PMT treatments (eg, autism, psychosis). Although our design was consistent with usual care, the limited number of children with CBCL scores in the clinical range likely contributed to floor effects.

It is important to note that child behavior problems improved in the 2 treatment groups and that decreases inCBCL scores were moderately strong (effect size range 0.50–0.57). More than 68% of children entering treatment with CBCL scores in the borderline or clinical range showed clinically meaningful improvements, including 8 of the 9 children with CBCL scores in the clinical range at baseline.

Another limitation was the substantial delay to treatment start, which likely contributed to the 61 (38.6%) participants never initiating PMT.³⁴ Average wait time from intake to the first scheduled PMT session was 112 to 140 days. Consistent with the pragmatic trial design, we did not augment clinic resources to shorten wait times or locate parents with disconnected phones. Expanding clinical capacity and shortening treatment delays would likely have decreased this attrition.

Of those who initiated PMT, 39% did not complete treatment (ie, did not attend ≥10 CPP sessions or reach PCIT skill mastery). Although this attrition rate is consistent with other PMT studies conducted in economically disadvantaged populations struggling with psychosocial adversities,²⁰ future research needs to address strategies for boosting PMT engagement and retention in these highly vulnerable families.

This is the first randomized pragmatic trial comparing the effectiveness and cost of 2 evidence-based PMT programs in a highly vulnerable population of families exposed to urban poverty and psychosocial adversity. The results suggest that, compared with PCIT, a program considered a “gold standard” PMT program, CPP is an effective and less expensive treatment option for decreasing behavior problems in young children living in urban poverty.

Supplementary Material

Supplementary Table S1.

TABLE S1 Mean Comparisons for Child Behavior Checklist (CBCL) Scores by Condition With Different Imputation Methods