Aspects of mental health communication skills training that predict parent and child outcomes in pediatric primary care

Abstract

Objectives

Training in communication can change clinician behaviors, but brief training may function by altering attitudes rather than teaching new skills. We used data from a trial of mental health training for office-based primary care to determine indicators of uptake that predicted parent and child outcomes.

Methods

Clinicians (n=50) were randomized to be controls or receive training. Uptake was determined comparing pre- and post-training visits with standardized patients (SPs) coded for skills and patient centeredness. Clinical outcomes were assessed by recruiting and following 403 children/youth ages 5-16 making visits to participants. At six months, change in mental health was assessed by parent and youth reports using the Strengths and Difficulties Questionnaire.

Results

Trained clinicians used more agenda setting, time, and anger management skills than controls and showed increased patient centeredness toward SP parents, but not adolescents. Increased patient-centeredness toward parents predicted improvement in child/youth symptoms and functioning (rated by parents), and improvement in youth-rated symptoms. Increased skills alone were not associated with improvement, but patients of clinicians above the mean for both skill and patient-centeredness change improved most.

Conclusions and implications

Targeting overall style and skills may produce the most clinical impact. Longer training may be needed to teach unfamiliar skills.

1. Introduction

Many though not all studies find that training can alter medical providers’ communication with patients in a variety of clinical contexts [1-4]. Finding corresponding impact on clinical outcomes has been more difficult [5], but changes have been found in patient satisfaction [6, 7] psychosocial distress [8], and child functioning [9]. Outcomes do seem to be related to active versus passive involvement in the training [10], and some authors have speculated [11, 12] that rather than imparting specific skills, training changes provider attitudes toward patients in general or toward a particular clinical situation. Providers then, in turn, may not so much use new skills as change their overall way of interacting with patients, building better relationships that then result in better clinical outcomes [13, 14].

We designed and tested a communications skills training program for pediatric primary care providers to improve children’s emotional and behavioral problems [9]. The skills taught were chosen to enable providers to increase patient expectations for positive outcomes, reach agreement on the nature of problems and desired treatment, and influence behavior change. The training also sought to improve providers’ expectations surrounding mental health care by providing an approach consistent with their existing pediatric knowledge and by teaching skills directly applicable to concerns about the length of mental health-related visits. In our initial intention to treat analysis, parents seeing trained providers had greater reductions in distress than parents seeing control providers, and the functioning of minority children seeing trained providers improved more than that of minority children seeing control providers.

Because any provider training uses scarce trainer and clinician resources, we wanted to better understand how the training may have related to clinical outcomes. We hypothesized that training could relate to outcomes in either or both of two ways. First, clinicians might use the skills that we taught, which had been chosen based on evidence for their effectiveness in primary care and psychotherapy [15-20]. Second, clinicians might respond to the training’s overall theme of the feasibility of engagement with mental health problems with a change their style of interaction with patients with mental health problems. Specifically, they might adopt a more patient-centered form of interaction, engaging in more psychosocial interaction and showing more empathy with and responsiveness to psychosocial distress [21, 22]. Because our training was brief (only a few hours), we hypothesized that providers with some prior exposure to mental health-related skills might demonstrate greater training uptake because they already had some underlying familiarity with the material. We also hypothesized that providers with more positive attitudes toward mental health care would have greater uptake.

2. Methods

2.1 Populations

Details of the study recruitment and training have been previously published [9]. Data were collected from 15 primary care offices in Baltimore, MD, Washington, DC, and rural New York. All sites served patients with a mix of insurance types. At each site we attempted to recruit all full-time providers routinely involved in pediatric primary care. Participating providers were randomized to receive the interactive training (described below) or to serve as controls. Control providers received a written manual covering the same background and skills contained in the training, but took part only in pre- and post-training assessments.

We sought to increase provider expectations by teaching skills for visit time management and by pointing out parallels between standard pediatric diagnostic and treatment principles and mental health care. We targeted provider and patient expectations by teaching skills for working with parent and child hopelessness and anger. Skills for building a patient-provider relationship included mutual agenda setting and joint formulation of problems. Skills for influencing behavior involved techniques for developing and presenting advice and managing resistance. These skills were drawn from manuals describing patient-centered care [15], family therapy [16], solution-focused cognitive therapy [17, 18], motivational enhancement [19], and family engagement [20].

Training was delivered in three cycles spaced about three weeks apart [23]. Each cycle began with a 60-minute small group discussion, held on-site during work hours and led by a child psychiatrist. The discussion used a slide set containing background material, written and video examples of skills, and programmed pauses for interaction. Audiotapes of the discussions were made to monitor the psychiatrist’s fidelity to planned content. Immediately after each discussion, providers practiced skills in a 10-minute standardized patient visit. Videos of these visits were returned to providers for guided self-assessment [24]. Total time for training was approximately four hours.¹

Provider population

Of 69 providers approached for participation, 58 agreed (84%); 27 became controls and 31 were trained. Although all 31 assigned to the training group completed the training (attended all three sessions and the associated standardized patient visits, and had patients recruited for assessment of clinical outcomes), four did not complete the follow-up skill up-take assessment described below. Four of the 27 controls also did not complete the follow-up skill assessment. Thus, the results reported here are for 50 providers (Table 1). Those who did not complete the follow-up skill assessment were more likely to be male (88% versus 34%, p=.004), from specialties other than pediatrics (88% versus 32%, p=.003), and less likely to have been at their practice site more than two years (13% versus 54%, p=.029). They did not differ in scores on either of the attitude measures described below.

Table 1.

Provider characteristics

Characteristics	Trained (N=27)	Control (N=23)	All (N=50)
Years since receiving degree (mean, SD)	15.3 (9.0)	15.9 (12.1)	15.6 (10.4)
Gender (%) female	74%	57%	66%
Specialty (%)
Family practice	26%	35%	30%
Pediatrics	74%	61%	68%
Other	--	4%	2%
Degree (%)
MD/DO	82%	78%	80%
Nurse practitioner	19%	17%	18%
Physician’s assistant	--	4%	2%
Additional training in child:
Behavior (% yes)	22%	22%	22%
Development (% yes)	7%	4%	6%
Counseling (% yes)	11%	8%	8%
PBS burden scale (mean, SD)*	16.8 (3.6)	15.8 (3.7)	16.8 (3.6)
PBS belief and feeling scale (mean, SD)**	13.8 (3.1)	15.0 (3.4)	14.4 (3.3)
Comfort with treatment scale (mean, SD)***	23.1 (6.6)	20.0 (3.6)	21.7 (5.6)
Greater than two years in practice at study site (%)	59%	47%	54%

^*Physicians’ Belief Scale, burden subscale. Possible range 6-30, higher scores indicate more negative feelings toward psychosocial issues

^**Physicians’ Belief Scale, belief and feeling subscale. Possible range 6-40, higher scores indicating more negative feelings toward psychosocial issues

^***Comfort with treatment scale. Possible range 5-40, higher scores indicate greater confidence in treating child and adolescent mental health problems

Providers had practiced an average of 15.6 years since obtaining their degree (range 1-40, median 14); 54% had been at their site for more than two years and 60% were female. Most (68%) were trained in pediatrics or family practice (30%). Most (80%) were physicians, 18% were nurse practitioners, and one was a physician’s assistant. About a quarter (22%) had additional training in child behavior, 6% in development, and 8% in counseling. There were no statistically significant differences between trained and control providers in any of these characteristics (Table 1).

Parents and children/youth

Because the goal of the trial was to assess the impact of training on mental health outcomes, we wanted to assemble of cohort of children representative of those routinely seen in primary care with emotional and behavioral problems. To do this, study staff first systematically approached parents in the office waiting areas of participating providers with the goal of recruiting 10 children for each. Children were eligible if they were between 5 -16 years old, scheduled to see a participating provider, and reported pain as 4 or less on an analog scale of 1-10 (1 = no pain). One child per family was randomly selected if more than one was eligible. Of a total of 875 families approached, 28 (3%) were not seeing a participating provider that day, and 39 (4%) declined, were ineligible, or agreed but did not complete the screening described in the next paragraph.

Second, Children (n=808) were screened using the Strengths and Difficulties Questionnaire (SDQ), a brief survey of child emotional and behavioral symptoms with good sensitivity and specificity compared to more elaborate instruments [25]. We chose the SDQ because it was available in both English and Spanish and because of a readily available algorithm for increasing accuracy by combing ratings from more than one observer [26]. For children 5-10, parents and teachers completed the SDQ; for youth 11-16, parent and youth responses were collected. The algorithm classifies children as being probably, possibly, or not likely to have a mental disorder. We followed all children and youth in the probable and possible categories (n=337, 42%), and a sample of those SDQ “not likely” who were subsequently identified by their provider as having mental health problems (n=130, 16%), for a total of 467. The SDQ prevalence rates for both probable (“clinical”) and “possible” (threshold) conditions were similar to those found in a primary care study using the Diagnostic Interview Schedule for Children (DISC) [27]. Of the 467 children, 49 parents (10%) declined further participation or could not be contacted after their index visit, leaving a sample of 418 cared for by all of the providers who had initially consented to be in the study.

For the 50 providers included in this training evaluation, 403 children and youth were in the follow-up cohort. Of these, 344 (85%) completed the 6-month follow-up. At the time of recruiting, 117 (29%) were rated as probably having an mental disorder, 145 (36%) as possible, and 141 (35%) as unlikely. About half (187, 46%) were female. The average age was 10.4 years. Most were White (217, 53%) or African-American (115, 28%). The others were Latino (51, 13%) or other/undisclosed. These children and youth were re-assessed, using the SDQ, 6 months following their enrollment. At that time parents of all children and youth provided an SDQ rating and youth 11-16 also rated themselves. Thus, for clinical outcome measures, we were able to use the post-pre difference in the parent rating on the SDQ for all children and youth, and the post-pre difference in youth self-ratings for those 11-16.

2.2 Measures

Prior to randomization, providers reported on their training background (degree, specialty, prior training in child behavior, development, or counseling) and length of time in their current practice site. They completed two instruments related to their attitudes toward treating mental health problems.

The 14-item Physician Belief Scale (PBS) [28, 29],measures attitudes toward treating psychosocial problems. Higher scores reflect more negative attitudes and are inversely correlated with patients’ disclosure of psychosocial information [29, 30]. The PBS has two subscales: the belief subscale includes items relating to providers’ feelings of competence to address psychosocial problems and their beliefs about patients’ desires to discuss them; the burden subscale includes items about the impact of psychosocial problems on overall workload, competing demands, and available time.

The 11-item Provider Confidence Scale (PCS) [31] measured three areas of confidence in managing mental health problems in primary care. Four items assessed confidence managing a school-age child with emotional problems, behavior problems, hyperactivity or short attention, or trouble getting along with friends. Four items assessed confidence managing an adolescent with either of the same problems. Three items assessed confidence counseling a child’s parent. Possible scores range from 11 to 55, with higher scores indicating more confidence.

Training uptake was measured by assessing participants prior to randomization and then about 4 weeks after the intervention group completed training. At both of these times, participants completed a 10-minute video-recorded interview with a standardized patient parent and teenager. The standardized patients’ script was designed to present six “challenges” that would provide opportunities to use the skills taught in the training. Standardized patients anticipated a greeting to both the parent and teen, presented a range of concerns that required elicitation and setting an agenda, disagreed with each other during the visit, implicitly or explicitly asked for advice, expressed hopelessness or frustration, and at some point either the teen or parents was resistant or negative in response to the participating provider’s actions or suggestions. After their initial training and practice with the script, standardized patient fidelity was measured by coding participants’ videos for the presence of the challenges (this was part of the coding that allowed intervention participants’ self-assessment). For the final assessment video, standardized patients presented three of the challenges 100% of the time; disagreements were presented in 26/27 visits; hopelessness in 26/27, and resistance in 21/27.

Coders blinded to provider training assignment assessed the videos for:

1. Skills from the training. Skills were grouped into 7 clusters (Table 2). Two clusters were presented in the first training session, three in the second, and two in the third. Each cluster contained several skills (range 4-9). During the training these skills might have been mentioned as interchangeable (for example, different ways of responding to a patient’s reluctance to accept advice) or as likely to be used jointly (for example, in the course of setting the visit agenda, first using an open-ended question and then probing for additional concerns). Coders worked from descriptions of the skills as they had been presented to trainees. A trainee received credit for using an individual skill if he or she used it once or more in the course of a standardized patient visit. Twenty of the 100 SP visits coded for the study (one pre- and one post-training for each participant) were double coded.. Agreement on the use of individual skills ranged from 63% to 100%, with higher rates of agreement generally observed for more commonly-used skills.

2. Patient centeredness: Patient-centeredness is a philosophy of general medical care that includes attention to patients’ psychosocial needs, facilitating disclosure of patient concerns, building a patient-provider partnership, and actively facilitating patient involvement in decisions [22]. Patient centeredness has been measured in several ways, and has been associated with several positive patient outcomes [21]. As in our own past work [32], we measured providers’ patient-centeredness using variables from the Roter Interactional Analysis System (RIAS) [33]. RIAS analysis of medical visits classifies each speaker’s utterances into one of several exclusive categories including information giving, question asking, empathy, and partnership building (ie, talk that facilitates mutual conversation). We operationalized patient centeredness by summing the RIAS categories of provider talk in medical and psychosocial information giving, asking psychosocial questions, rapport building (empathy and reassurance), and partnership building. Coder reliability was monitored by duplicate coding of a random sample of 20 videos assessed by a second coder throughout the coding period. Correlation coefficients for individual RIAS categories ranged from .71 to .99.

Table 2.

Crude and adjusted changes in the use of individual skills

Skill cluster	Description and (# of skills in cluster)	Immediate uptake¶		Mean number used at baseline	Mean number used at follow up	Trained/control difference in uptake adjusted for baseline**
Time management	Manage rambling or long list of concerns, set priorities (4)	12/27	Trained Control	.48 .44	.78* .13	.54 (p=.002)
Problem solving	Techniques from solution-focused brief therapy (7)	21/27	Trained Control	0 0	0 0	−.04
Managing anger	Techniques from family therapy: black/white statements, restating concerns, anger at clinician (6)	18/27	Trained Control	.11 .13	.59* .17	.36 (p=.05)
Agenda setting	Elicit concerns, engage child and parent, promote turn-taking (5)	27/27	Trained Control	3.4 3.0	3.7* 3.0	.85 (p=.006)
Problem formulation	Exploring family ideas, seeking common language, prioritizing (7)	20/27	Trained Control	.42* .78	.85 .96	.22
Advice giving	Asking for permission, offering choices, exploring preferences (7)	22/27	Trained Control	.10 .22	.39 .48	.09
Managing resistance	Not pushing back, agreeing, asking permission to provide more information (9)	22/27	Trained Control	.55 .67	.78 1.3	−.43

^*crude difference significant (t-test) between trained and control

^**difference in mean change in usage per visit attributed to training, adjusted for participant’s baseline usage

^¶Number of trainees using at least one skill from the cluster in standardized patient practice visit immediately following training session

2.3 Statistics

Analyses look at differences from pre to post assessment, adjusted for baseline. We use simple linear regression for measures of provider uptake, and GEE regression for the relationship between provider uptake and clinical outcomes (because each participating provider cared for more than one patient).

To understand if there are participant characteristics associated with greater uptake of the training, we initially used stratified analyses looking at the impact of training for different strata of participant characteristics (for example, for men and women). We then confirmed apparent differences by looking for significant interaction terms including training/control assignment and the characteristic thought to vary training uptake.

2.4 Human subjects

Parents provided written consent and youth age 11 to 16 years provided assent. Providers gave written consent for participation. IRB approval was obtained from the Johns Hopkins School of Public Health and the IRB’s of three other institutions associated with study sites.

3. Results

3.1. Uptake of skills taught in training

Demonstration immediately after skill presentation

There was variation among the 7 skill clusters in trainees’ ability to demonstrate at least one component immediately after the skills had been presented (Column 3, Table 1). Time management skills were the least likely to be demonstrated immediately (about 44% of trainees did so); in contrast, agenda setting skills were demonstrated by all of the trainees. The modal number of clusters demonstrated immediately after learning was 5 (average 5.2), with a range of 3-7.

Skill use at post-training assessment compared to baseline

In the baseline standardized patient visit, providers assigned to the training and control conditions differed significantly only in their use of problem formulation skills: control providers used these skills more frequently, on average .78 times per standardized patient visit compared to .42 times per visit among trained providers (Table 1). At follow-up, providers receiving the full training used skills from three of the 7 clusters more frequently than did controls: time management, managing anger, and agenda setting. These differences were statistically significant when examined as a crude difference between frequencies and when adjusted for the individual participant’s baseline usage.

3.2. Change in patient-centeredness

There were no statistically significant differences between trained and control providers’ overall patient-centeredness toward mothers or teens at baseline (Tables 3a and 3b). Overall, patient-centered talk to parents and teens tended to decrease from the baseline to the follow-up visit; regression coefficients indicating time were negative and significant. The exception was that, for talk to parents, trained providers maintained their level of patient-centeredness from baseline to follow-up, so that the net effect was statistically significant greater patient centeredness to parents at follow-up compared to controls.

Table 3a.

Change in provider patient centered utterances to SP parent

Type of talk		Mean number of utterances/visit at baseline	Mean number of utterances/visit at follow-up	Mean difference in change from baseline to follow-up, trained versus control!
Total patient- centeredness	Trained Control	54.1 59.2	53.4 42.4	20.4 (p=.015)
Rapport	Trained Control	11.2 13.2	12.5* 8.34	4.82 (p=.005)
Psychosocial information	Trained Control	11.1 13.0	15.6 10.3	6.13 (p=.058)
Medical information	Trained Control	5.2 5.7	5.2 5.3	0.06 (p=.98)
Psychosocial questions	Trained Control	7.8 6.7	8.6* 4.3	4.04 (p=.069)
Partnership	Trained Control	21.1 20.0	20.3 14.2	5.4 (p=.084)

Table 3b.

Change in provider patient-centered utterance to SP teen

Type of talk		Mean number of utterances/visit at baseline	Mean number of utterances/visit at follow-up	Mean difference in change from baseline to follow-up, trained versus control!
Total patient- centeredness	Trained Control	83.9 84.4	56.9 63.9	−11.8 (p=.12)
Rapport	Trained Control	9.0 9.7	7.8 6.6	1.41 (p=.33)
Psychosocial information	Trained Control	13.7 10.9	10.7 11.3	−.86 (p=.75)
Medical information	Trained Control	4.0 3.7	2.7 3.0	−.34 (p=.69)
Psychosocial questions	Trained Control	23.0 22.8	12.5* 19.9	−.74 (p=.001)
Partnership	Trained Control	30.3 36.7	17.9 23.1	−4.14 (p=.22)

^!adjusted for baseline

^*Trained control difference significant at p<.05

When we unpack the patient centeredness components toward the SP parent (Table 3a), the one for which there is significant increase in the trained compared to controls is rapport (p=.005). Psychosocial information giving also tended to improve relative to controls (p=.058), as did partnership-building statements (p=.084) and psychosocial question-asking (p=.069).

Training did not have a significant impact on patient centeredness toward the SP teen (Table 3b), with the exception that trained providers came to ask about one fewer psychosocial questions per visit compared to control providers. Overall, however, there was a trend that was in the opposite direction seen in patient centeredness toward parents – relative to control trained providers became less patient centered toward teens and more patient centered toward parents.

3.3. Trainee characteristics, skill uptake, and change in patient-centeredness

None of the measured provider characteristics predicted skill uptake overall, but training in pediatrics (versus family medicine) was related to changes in the uptake of skills for anger management and advice giving. For these two skill areas there were significant interactions between trained/control status and having a background in pediatrics. That is, the impact of training was different for those with pediatric versus family medicine backgrounds. For advice giving the coefficient for the interaction was positive (.76, p=.037), suggesting that pediatricians were more likely to learn the skills, given exposure to training. For anger management, the coefficient was negative (−.90, p=.016) suggesting that pediatricians were less likely to learn skills in this cluster.

Three provider characteristics were related to change in patient-centeredness over the course of the study. When examining change in patient centeredness toward mothers, there were significant interactions between trained/control status and having a background in pediatrics (versus family medicine) (coefficient for interaction term=37.7, p=.036). That is, training impact on patient centeredness was greater for those with pediatric versus family medicine training. There were also significant interactions for prior behavioral, developmental, or counseling traininng (46.5, p=.013), and scoring above the group mean on the PBS burden subscale (34.7, p=.038). However, scores on the PBS belief scale and the PCS did not show significant interactions with training, although there was a trend for greater uptake among those with PCS scores that were above the group mean (more comfortable with mental health care).

3.4. Markers of training uptake as predictors of clinical outcomes

Uptake of training, as measured by increased patient-centeredness toward the standardized parent (there was no significant change toward the standardized youth), was associated with greater improvement in parent-rated symptoms and function, and with youth self-rated symptoms (though not function) (Table 4). Uptake measured by use of anger management or agenda setting skills was not related to any of the clinical outcomes. However, uptake as measured by use of time management skills had a negative impact on youth self-rated symptoms. That is, uptake of time management skills was related to an increase in youth self-rated symptoms.

Table 4.

Parent and youth-rated change in symptoms and function as a function of provider change in skills and patient centeredness during training

Training uptake measure*	Parent-rated change in child and youth symptoms**	Parent-rated change in child and youth function**	Youth self- rated change in symptoms**	Youth self-rated change in function**
Anger management skills	−1.15 (−2.5 to .24)	.19 (−.28 to .66)	−.30 (−2.4 to 1.8)	−.20 (−.73 to .32)
Agenda setting skills	.36 (−1.11 to 1.83)	−.38 (−.88 to .13)	−1.2 (−3.2 to .79)	−.05 (−.56 to .44)
Time management skills	.80 (−.62 to 2.2)	.22 (−.25 to .69)	3.2 (1.1 to 5.3)	.18 (−.36 to .71)
Patient- centeredness to SP parent	− 1.7 (−3.0 to −.40)	−.46 (−.89 to −.03)	− 2.3 (−4.2 to −.28)	.07 (−.43 to .56)

^*skill uptake measures dichotomized to increased use of skill by at least one instance versus did not increase use; patient-centeredness dichotomized on mean increase for all study providers

^**this coefficient (with 95% confidence limits) represents the change attributable to the provider having a change in the skill or patient centeredness measure above the mean for all participating providers. This is adjusted for the child/youth baseline level of symptoms or function for clustering of patients within clinicians. Bold entries in the table are statistically significant at the p<.05 level. N= 344 for parent ratings and 149 for youth self-ratings.

Table 5 shows the impact of training uptake on clinical outcomes when change in skill acquisition (across all skills introduced in the training program) and change in patient centeredness are taken together. Each column in the table shows a linear regression including change in skill use, change in patient centeredness, and their interaction, adjusted for child/youth baseline clinical data and for provider baseline skills and patient centeredness. There are significant interaction effects predicting improvement in parent-rated symptoms and function and youth-rated symptoms; the interaction coefficient for improvement in youth-rated function also approaches significance. When the interaction term is present neither skill acquisition nor increase in patient centeredness to parents is individually associated with clinical improvement. In fact, when youth self-ratings are the outcome, taken alone uptake of skills is associated with worsening of symptoms.

Table 5.

Interaction of skill and patient centeredness change on parent and youth-rated outcomes.

Training uptake measure	Parent-rated change in child and youth symptoms*	Parent-rated change in child and youth function*	Youth self-rated change in symptoms*	Youth self-rated change in function*
Provider above mean for overall skill uptake	1.27 (−.59 to 3.12)	.60 (−.02 to 1.21)	4.62 (1.64 to 7.58)	.55 (−.22 to 1.32
Provider above mean for increase in patient- centeredness to parent	.66 (−1.40 to 2.74)	.34 (−.35 to 1.03)	3.79 (.42 to 7.15)	.81 (−.08 to 1.70)
Interaction term: above mean for both skills and patient- centeredness	− 3.90 (−6.50 to −1.23)	− 1.29 (−2.17 to −.40)	− 10.13 (−14.52 to −5.73)	−1.13 (−2.30 to .03)

^*coefficients (with 95% confidence limits) from linear regression including all three terms (skill change, patient centeredness change, and their interaction). Coefficients adjusted for the child/youth baseline level of symptoms or function, for baseline provider skills and patient centeredness, and for clustering of patients within clinicians. Bold entries in the table are statistically significant at the p<.05 level. N= 344 for parent ratings and 149 for youth self-ratings.

4. Discussion and conclusion

4.1 Discussion

The results of these analyses support the conclusions of our initial intent to treat analyses: we can identify changes in provider behavior that are more common in the trained than the control providers, and those changes, in turn, are related to improvement in child clinical outcomes by both parent ratings and youth self-ratings. Training seemed to influence use of specific skills and overall interaction style, and it was the change in both of these taken together that was most powerfully associated with clinical outcomes. Change in interaction style (patient centeredness to parents), taken alone, had a significant impact on outcomes, while change in skills, taken alone, did not. This result suggests that training to impact mental health ideally needs to focus on both overall interaction style and attitudes as well as on specific skills intended to have a therapeutic effect. This is consistent with recent reports from adult primary care showing that a communication style that builds a working relationship with patients strongly influences the success of specific treatment targeting depression [34, 35].

There are other possibilities for why skill uptake alone was unrelated to positive outcomes. One is that the training may have been too brief for participants to learn to use skills effectively. Demonstrating the skills with standardized patients (corresponding to the level in Miller’s [36] competency pyramid of “showing how”) does not necessarily imply that clinicians are able to use a skill in a real clinical setting (“does,” the pinnacle of Miller’s pyramid). One study of training in motivational interviewing, for example, found that clinicians could readily demonstrate use of component techniques after a brief training, but this didn’t correlate with their ability to use them in a clinically effective way [37]. A second possibility is that the baseline and follow-up standardized patient assessment itself was simply too brief (10 minutes) to allow trainees to demonstrate their mastery of all seven skill areas (and thus the result is an artifact of our method of outcome assessment). Even though we tried to present situations that would elicit use of all seven skill clusters, in a brief visit trainees may chose different goals that mandate use of different sets of skills. Better assessments might involve either multiple simulated visits, or analysis of multiple visits with real patients. Both of these tactics are used to assess primary care clinicians’ competency in exams administered by the UK’s Royal College of General Practitioners [38].

The results leave us with the question of why some skills were learned while others were not. The skills that trainees seemed to retain to the follow-up assessment and use more frequently than controls (setting the agenda, managing anger, and managing time) might be seen as those that were either already familiar or of most interest to providers caring for mental health problems in the context of a busy practice. They also are skills that potentially relate more to things the trainee can control – that is, the conduct of the visit – versus skills that relate to the parent’s or child/youth’s behavior. Other studies have found that clinicians seem to learn only a subset of what is presented in communication training programs, and that introductions and agenda-setting skills are more readily learned than those that require more complex interactions [39, 40]. Simply extending training time is not likely to be a remedy, since even relatively lengthy studies have failed to find impact on trainee behavior [2], but a change in feedback procedures might be what is needed. We had evidence that not all of our trainees could reproduce skills in a standardized patient visit immediately after brief training sessions. If we could rate these practice visits in real time, we could offer immediate remediation and at least assure that all participants had been able to perform skills from each cluster at least once.

There is a note of caution to be drawn from the one significant relationship between an individual skill use and a clinical outcome: use of time management techniques was associated with a relative worsening of youth self-rated symptoms. A study based on feedback from adult cancer patients [41] suggested that it is possible to convey a sense of adequate time and attention even in short, structured visits, but that this requires combining time management with the ability to convey a sense of undivided, personal attention for as long as the visit lasts. Many studies have found that pediatricians and, in other countries, generalist physicians, converse more with parents than with children in the course of pediatric visits [42], and, in our study, trained providers increased their patient-centeredness to parents but not youth. Thus the time management skills may have been benign or helpful with parents, but could have been negatively experienced by youth. This finding may also speak to the essential nature of using skills in the context of a more patient centered overall style. Training seemed to shift patient centeredness more toward parents, perhaps at the expense of emotional attention devoted to youth. In practice this effect could be offset by time that providers spend seeing youth individually, but it serves as an alert that providers need skills to engage both parents and youth simultaneously when they are being seen together. On the other hand, change in patient centeredness toward parents was positively associated with youth outcomes, even when rated by youth alone. This result deserves replication and further exploration. Youth improvement could be the result of changes in parent mood or behavior, or youth could be helped by observing a more positive relationship between their parent and their provider.

4.2 Conclusions and practice implications

Our results support the effectiveness of brief training to increase mental health capacity in pediatric primary care. Training targeting both overall interactional style and specific skills seems likely to produce the most positive clinical impact. However, more focused training may be required to teach unfamiliar skills, including improving interactions with children and youth themselves. Longer or different training may be necessary for providers without prior exposure to concepts of development, behavior, and counseling.