Effects of primary care provider characteristics on changes in behavioral health delivery during a collaborative care trial

Abstract

Objective:

Pediatric primary care providers (PCPs) are increasingly expected to deliver behavioral health (BH) services, yet PCP characteristics that facilitate or hinder BH service delivery are poorly understood. This study examined how PCP characteristics and collaborative care participation influenced changes in BH-related effort and competency over time.

Methods:

Pediatric PCPs (N = 74) participating in a cluster randomized trial (8 practices) of a collaborative care intervention for disruptive behavior problems completed self-report measures at 0, 6, 12, and 18 months. Latent growth curve models tested the impact of PCP characteristics (i.e., age, gender, negative BH beliefs, BH burden, BH competency) on changes in identification/treatment of disruptive behavior disorders (DBDs) and competency over the course of the trial.

Results:

Participation in collaborative care was associated with increases in identification/treatment, with no evidence that PCP characteristics moderated changes in identification/treatment. For competency, however, older PCPs (>50 years) in collaborative care exhibited steep increases over time, while older PCPs in the comparison condition exhibited steep decreases, suggesting differential benefits of collaborative care participation by PCP age. In both conditions, PCPs with more negative BH beliefs reported less identification/treatment over time. Baseline competency was positively associated with identification/treatment and associations weakened over time. Gender and perceived burden had little impact.

Conclusions:

PCP characteristics are associated with changes in PCPs’ BH-related effort and competency over time. Participation in a collaborative care model appears to be especially beneficial for older PCPs. Implementation of collaborative care can promote growth in BH-related effort and competency for PCPs.

Youth behavioral health (BH) problems commonly present in primary care.¹ Recent efforts to improve BH care have focused on the integration of physical and BH care, which involves collaboration between PCPs and BH specialists as well as greater PCP responsibility for management of mild-to-moderate BH concerns.^2–4 Although pediatric PCPs are increasingly expected to deliver BH services, PCPs acknowledge barriers to BH care, including time constraints, limited BH resources, the need for additional skills, and competing clinical demands.^5–8 Accordingly, shifting from an approach emphasizing identification and external referral to an integrated approach involving ongoing management of BH concerns within primary care is potentially challenging for already overburdened PCPs.

Collaborative care is a fully integrated team-based model^4,9–11 that has been shown to improve BH outcomes in pediatric populations (d = 0.63).² PCPs participating in collaborative care show improvement in their delivery of BH care.¹² However, there is considerable variability in PCPs’ willingness and capacity to identify and treat BH problems,^5,6 and few rigorous studies have examined the role of PCP characteristics in the delivery of integrated care. Existing research has found that PCP characteristics (e.g., female gender, specialized BH training, positive attitudes) may promote BH service delivery.^13–16 In a large sample of pediatric PCPs, negative beliefs about psychosocial problems were associated with less self-reported identification and treatment of BH problems.¹⁷ Similarly, perceptions that BH care is particularly burdensome have been associated with less identification and treatment.^16–20 However, other studies find conflicting results or no associations between PCP characteristics and BH service delivery.^{13,18,21–23} Associations between PCP characteristics and BH-related effort and competency have not yet been examined longitudinally, and little is known about how PCP characteristics may influence changes in BH-related effort and competency as PCPs adjust to working within a collaborative care model. Understanding how PCP characteristics affect BH delivery over time and whether these associations differ for PCPs participating in collaborative care can inform efforts to increase BH integration.

Current Study

This study sought to identify individual characteristics of pediatric PCPs associated with changes in BH-related effort and competency over time. We utilized data from a cluster-randomized trial of a collaborative care intervention for children’s behavior problems, findings from which showed that PCPs participating in collaborative care improved their delivery of BH care.¹² Details of the clinical trial design, methods, and results are published elsewhere.¹²

For the current study, we selected five of the most widely examined provider-level variables (age, gender, negative beliefs about BH, burden associated with BH, and perceived competency in BH treatment) and tested how they impact the frequency with which PCPs identify and treat disruptive behavior disorders (DBDs) over time. We also tested the impact of provider-level variables (age, gender, negative BH beliefs, BH burden) on changes in competency over time. To our knowledge, this is the first study to use longitudinal data to test how PCP characteristics predict or moderate changes in PCPs’ BH-related effort and competency over time.

Methods

Participants and Procedures

Participants were PCPs (N = 74) from pediatric primary care practices (N = 8) randomized to Doctor Office Collaborative Care (DOCC; n = 31) or a comparison condition without a fully integrated model of care (Enhanced Usual Care [EUC]; n = 43). DOCC practices were provided a BH specialist (master’s level) as a care manager, and PCPs in DOCC received specialized training in the identification of BH problems, access to evidence-based pharmacological treatment guidelines, assistance from the care manager with BH assessment and management, and access to psychiatric consultation. EUC practices were provided a care manager who completed an initial BH assessment, provided feedback and brief psychoeducation to the family, and facilitated specialty BH care referrals. Integrated, on-site specialty BH services were neither provided nor made available to EUC practices. Participants were primarily White (89%) and female (57%), ranging in age from 29 to 69 years. More detailed demographics are available elsewhere.¹² There were no demographic differences between DOCC and EUC PCPs. All participants provided informed consent. The study was approved by the University of Pittsburgh institutional review board.

Measures

PCPs completed measures at baseline (prior to condition assignment and training), 6, 12, and 18 months. Measures included demographics (e.g., age, gender), negative beliefs about BH treatment (Physician Belief Scale¹⁷; 8 items, α = 0.66), perceived burden associated with providing BH treatment (Physician Belief Scale¹⁷; 6 items, α = 0.64), frequency of identifying/treating DBDs (MH-SKIP survey¹²; 2 items, r = 0.45-0.62 across timepoints), and perceived competence in treating DBDs (MH-SKIP survey¹²; r = 0.70-0.87 across timepoints). Mean scores were used for analyses. See Kolko and colleagues for additional details.¹²

Data Analyses

To test if PCP characteristics predicted or moderated PCPs’ BH-related effort and competency, latent growth curve modeling (LGM) was conducted using the nlme package in R version 3.5.1.²⁴ LGM examines longitudinal change by estimating fixed effects for predictors of the intercept and slope (change over time) of an outcome and random effects that account for individual variation in intercepts and slopes of an outcome. We centered time at the final time point, so that intercept values represent those at study end. Missing data were sparse and considered missing at random; one participant had missing data across self-report measures, and an additional five did not indicate age. Consistent with recommendations,²⁵ maximum likelihood estimation was used to address missingness.

The model for identification/treatment included time-invariant fixed effects of intervention condition (0 = EUC, 1 = DOCC), PCP age, PCP gender (male = 0; female = 1), baseline negative beliefs, baseline burden, and baseline competency. The model for competency included time-invariant fixed effects of intervention condition, PCP age, PCP gender, baseline negative beliefs, and baseline burden. All models included random effects for intercept, time, and their covariance. Linear change was the best fitting growth trajectory for both outcomes. Full models included main effects and all possible two-way and three-way interactions between time, condition, and PCP characteristics. For parsimony, interactions with p > .10 were trimmed from the final models (except for lower-order components of higher-order interactions), which significantly improved fit for all models. Interactions were probed with simple slopes testing using the reghelper package in R.

Results

Preliminary models containing condition, time (linear), and the interaction of condition and time are as follows: Identification/treatment significantly decreased over time for PCPs in EUC (b = −.15, SE(b) = .05, p < .01) and significantly increased over time for PCPs in DOCC (b = . 12, SE(b) = .06, p < .05). Competency significantly decreased over time for PCPs in EUC (b = −.15, SE(b) = .04, p < .001) and significantly increased over time for PCPs in DOCC (b = .15, SE(b) = .05, p < .01).

Differential Benefits of Collaborative Care Participation by PCP Characteristics

Results of final models for identification/treatment and competency are presented in Table 1. A three-way interaction indicates that a PCP characteristic is associated with differential effects of DOCC participation. A two-way interaction between condition and time (in the absence of a three-way interaction) indicates that intervention condition moderates change over time, but these changes are not impacted by PCP characteristics.

Table 1.

Latent Growth Curve Models Predicting Outcomes

Outcome	Identification/Treatment				Competency
	Estimate	(SE)	95% CI		Estimate	(SE)	95% CI
			Lower	Upper			Lower	Upper
Fixed Effects
Intercept	1.99	.73**	.54	3.43	2.63	.85*	.94	4.32
Time	.28	.23	−.18	.73	.03	.27	−.48	.53
Condition (0 = control, 1 = DOCC)	1.62	.51**	.60	2.64	−.49	1.07	−2.63	1.64
Age	.02	.01+	−.001	.04	−.003	.02	−.04	.03
Gender (0 = male, 1 = female)	.04	.21	−.37	.46	.15	.22	−.29	.58
Negative Beliefs	.10	.04*	−.18	−.02	−.03	.04	−.10	.05
Perceived Burden	−.01	.02	−.05	.04	−.04	.02+	−.09	.002
Competency	.35	.11**	.14	.57	-	-	-	-
Time x Condition	.16	.07*	.03	.30	−.45	.34	−1.13	.22
Time x Age	.01	.004*	.001	.01	−.006	.005	−.02	.005
Time x Gender	−.12	.07+	−.26	.02	.13	.07+	−.01	.27
Time x Negative Beliefs	−.04	.01**	−.06	−.02
Time x Competency	−.12	04**	−.19	−.04
Condition x Perceived Burden	−.06	.03+	−.12	.002
Time x Condition x Age					.01	.001*	.003	.03
Random Effects
Intercept variance	.61		.45	.82	.62		.50	.78
Slope variance	.10		.03	.33	.01		<.001	<.001
Covariance	.90		−.87	1.00	.01		<.001	.65
Residual	.51		.45	.58	.56		.50	.62
Model Fit
AIC	536.49				580.42
BIC	597.49				634.79
Log Likelihood	−250.24				−274.21
Conditional R2	.71				.62

Note.

⁺p <.10;

^*p < .05;

^**p < .01;

DOCC = Doctor-Office Collaborative Care (collaborative care intervention)

For identification/treatment, there were no significant three-way interactions, indicating the benefits of DOCC participation did not differ by PCP characteristics. There was a significant two-way interaction between condition and time, indicating that improvements in identification/treatment were found for DOCC but not EUC, regardless of PCP characteristics (Figure 1A). PCPs in EUC did not significantly change in identification/treatment over time (p = .23), while those in DOCC significantly increased in identification/treatment over time (p < .05).

Figure 1.

Interactions involving intervention condition and time.

A: Interaction between condition and time predicting identification/treatment.

B: Interaction between age, condition, and time predicting competency. BL = Baseline.

For competency, there was a significant three-way interaction between age, condition, and time indicating that PCP age moderated the association between DOCC participation and changes in competency over time (Figure 1B). In EUC, there was decline in competency that grew stronger with age (age 30 years, b = −.15; age 50 years, b = −.26). In DOCC, age was unrelated to change in competency over time for providers younger than 50 years (bs < −.10, ns). However, for providers age 50 years and older, there was an increase in competency that grew stronger with age (age 50 years, b = .12; age 60 years, b = .23; ps < .05).

Effects of PCP Characteristics on Change over Time

Several interactions were found between PCP characteristics and time, indicating that PCP characteristics influence BH delivery over time regardless of intervention condition. For identification/treatment, significant interactions were found between age and time, negative beliefs and time, and competency and time (Figure 2). Age was not significantly associated with identification/treatment at the first three waves (ps > .15) but was marginally positively associated with identification/treatment by the 18-month wave (p = .07). Negative beliefs were not significantly associated with identification/treatment at baseline and 6-month waves (ps > .43) but were marginally negatively associated at the 12-month wave (p = .06) and significantly associated with less identification/treatment by the 18-month wave (p < .05). Baseline competency was positively associated with identification/treatment across waves (ps < .01), but associations weakened over time. Lastly, marginal interactions between time and gender with no consistent pattern of findings were found for both identification/treatment and competency (not shown).

Figure 2:

Interactions between PCP characteristics and time predicting identification/treatment.

A: Interaction between age and time predicting identification/treatment.

B: Interaction between negative beliefs and time predicting identification/treatment.

C: Interaction between competency and time predicting identification/treatment.

Differential Effects of PCP Characteristics by Intervention Condition

Only one interaction was found that did not include time (not shown). There was a marginal interaction between perceived burden and condition (p = .06). In EUC, burden and identification/treatment were not associated (p = .77), but in DOCC, lower burden significantly predicted greater identification/treatment of BH concerns (p = .02).

Discussion

This study examined PCP characteristics associated with changes in BH-related effort and competency within the context of a cluster-randomized trial of a collaborative care intervention for pediatric DBDs. We tested the influence of PCP age, gender, negative beliefs about BH, burden associated with BH, and perceived BH competency on DBD identification/treatment and competency across conditions and over time. Results provide additional support for collaborative care as a means of effectively translating specialty pediatric BH expertise to the primary care setting.

We found two significant interactions involving both condition and time. PCPs participating in collaborative care exhibited greater increases in the identification/treatment of DBDs over time than PCPs in the comparison condition, consistent with previous results from this trial.¹² There were no three-way interactions for identification/treatment, suggesting that the benefits of participation in collaborative care on identification/treatment did not differ by PCP characteristics. The lack of differences by PCP characteristics may be due in part to the presence of a care manager who provided assessments and referrals in EUC practices, potentially reducing variability in identification/treatment.

Results for competency suggested differential benefits of collaborative care participation by PCP age. In the comparison condition, providers exhibited a decline in competency over time that grew stronger with greater age. In collaborative care, however, providers younger than 50 years exhibited no meaningful change in competency, while those 50 years and older exhibited increases in competency over time. These findings suggest that older PCPs (>50 years) may benefit the most from the supports provided in the collaborative care model, perhaps because they received less exposure to BH assessment and intervention during residency training. Encouragingly, the educational and experiential opportunities offered by participation in collaborative care (e.g., additional training, consultation on specific cases) appear to help older PCPs change practice behaviors to better address pediatric BH concerns.

There were several interactions of time with PCP characteristics, suggesting that PCP characteristics have some influence on BH delivery over time regardless of participation in collaborative care. PCP age was minimally associated with identification/treatment, with older age associated with marginally greater identification/treatment at the final timepoint. Self-reported competency at baseline was positively associated with identification/treatment of DBDs, with associations weakening over time. These results are consistent with prior research findings that more confident and knowledgeable PCPs are more likely to deliver BH services.^13,16

Surprisingly, baseline negative beliefs were not associated with the frequency of identification/treatment at baseline. However, this association became stronger over time; providers with more negative beliefs at baseline reported significantly less frequent identification/treatment 18 months later. It may be that the beginning of the trial brought greater attention to BH concerns that temporarily weakened the association between PCP negative beliefs and identification/treatment. This may explain why associations between negative beliefs and BH-related effort and competency have been found in survey data,¹⁷ but not in the context of a clinical trial.^13,18 Efforts to reduce negative beliefs about BH among PCPs may lead to improvements in BH-related effort and competency.

There was little evidence for gender as a predictor of changes in BH-related effort and competency. Findings for the impact of perceived burden were also relatively weak. In contrast with prior work finding negative associations between burden and BH-related effort and competency,^16,18–20 burden was not directly associated with identification/treatment in this study. However, burden was marginally associated with less competency over time and the inclusion of competency in the models may have diminished the impact of perceived burden. The interaction between condition and burden marginally predicted identification/treatment, such that burden was not associated with identification/treatment in the comparison group, but lower perceived burden at baseline was associated with more identification/treatment for PCPs in the intervention group. These findings suggest that the supports provided by the collaborative care model are necessary for low perceived burden to translate into greater BH-related effort and that attempts to reduce PCPs’ perceived burden alone are unlikely to increase BH-related effort and competency.

A key strength of this study is the longitudinal examination of change. The overall tendency for PCPs in the comparison condition to decline in effort and competency over time, despite the presence of a care manager, is concerning and suggests that additional training and active ongoing collaboration are crucial to supporting PCPs in BH integration. Limitations include the use of PCP self-report and broad questions to assess identification/treatment and competency. PCPs may not accurately assess the frequency with which they engage in identification/treatment and may differ in what they consider identification (e.g., using evidence-based screening tools, asking a general question about behavior) and treatment (e.g., ongoing monitoring, psychoeducation, prescribing). Future work could gather data from electronic health records and/or observations to more accurately measure the frequency of identification/treatment and objectively assess competency.

In summary, our findings indicate that expansion of collaborative care models may benefit PCPs by promoting growth in BH-related effort and competency, particularly for older PCPs who may have received less training in managing BH concerns. Large collaborative care trials are a unique opportunity to assess changes in PCP attitudes, effort, and competency over time. Next steps in this line of research may include examining mediators of collaborative care effects on PCPs’ effort and competency, assessing the impact of organizational culture and climate, and examining how changes in PCPs’ behavior are associated with patient outcomes. Widespread implementation of collaborative care models may promote growth in PCPs’ BH-related effort and competency and ultimately improve patient BH outcomes.^2,4,10

What’s New:

Primary care provider (PCP) characteristics are associated with changes in PCPs’ BH-related effort and competency over time. Participation in a collaborative care model appears to be especially beneficial for older PCPs.