Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Jan 8.
Published in final edited form as: J Behav Health Serv Res. 2013 Jan;40(1):20–35. doi: 10.1007/s11414-012-9292-0

Mental Health Communications Skills Training for Medical Assistants in Pediatric Primary Care

Jonathan D Brown 1, Lawrence S Wissow 2, Benjamin L Cook 3, Shaina Longway 4, Emily Caffery 5, Chris Pefaure 6
PMCID: PMC4287209  NIHMSID: NIHMS415223  PMID: 23070564

Abstract

Paraprofessional medical assistants (MAs) could help to promote pediatric primary care as a source of mental health services, particularly among patient populations who receive disparate mental health care. This project piloted a brief training to enhance the ability of MAs to have therapeutic encounters with Latino families who have mental health concerns in pediatric primary care. The evaluation of the pilot found that MAs were able to master most of the skills taught during the training, which improved their ability to have patient-centered encounters with families during standardized patient visits coded with the Roter Interaction Analysis System. Parents interviewed one and six months following the training were more than twice as willing as parents interviewed one month before the training to discuss mental health concerns with MAs and they had better perceptions of their interactions with MAs (all p < 0.01) even after controlling for a range of patient and visit characteristics. Before training, 10.2% of parents discussed a mental health concern with the MA but not the physician; this never happened six months after training. This pilot provides preliminary evidence that training MAs holds potential to supplement other educational and organizational interventions aimed at improving mental health services in pediatric primary care but further research is necessary to test this type of training in other settings and among different patient populations.

Keywords: medical assistant, paraprofessional, training, skills, mental health, primary care, pediatric, communication, patient-centered, Roter Interaction Analysis System, Latino, patient activation

Introduction

Interventions to improve treatment of mental health problems in pediatric primary care range from relatively brief communication training for primary care providers (PCPs)1 to collaborative care models that attempt to better integrate mental health services within primary care settings.23 These interventions have sought to overcome the well-documented barriers to receiving mental health services in pediatric primary care, including the burden that PCPs associate with treating mental health problems,4 the lack of mental health training and treatment skills among PCPs,5 and the fragmentation of primary care and specialty mental health services.6

Addressing the mental health needs of Latino families in pediatric primary care may be especially challenging given language barriers and cultural differences in the way mental health problems are perceived and expressed.7 Latinos who are recent immigrants may be unfamiliar with the U.S. health care system and unsure of whether primary care is an appropriate venue to discuss mental health problems.89 Nonetheless, some studies suggest that Latino parents are generally receptive to receiving advice about their child’s mental health or social functioning in primary care.10 In addition, at least one mental health communications training intervention for PCPs found that Latino (and African-American) children had more substantial gains in functioning after visiting a trained PCP compared with their non-Latino Caucasian counterparts, suggesting that trained PCPs may have gained skills to overcome some of the language or cultural challenges associated with communication about mental health.1 As primary care practices, health plans, and policymakers continue to look for ways to enhance the mental health treatment capacity of primary care with limited resources, particularly in public clinics and other settings that serve large uninsured Latino populations, there may be opportunities to think about ways to enhance the role of other staff in the clinic.

Although they are the most common clinical support staff in primary care and community health clinics,11 paraprofessional medical assistants (MAs) have not played a major role in interventions designed to improve mental health services in these settings. Rather, these interventions tend to focus on enhancing the skills of PCPs, implementing screening, co-locating mental health professionals, and/or changing practice workflows to facilitate delivering or coordinating mental health care. The failure of past interventions to capitalize on the skills of MAs may be an oversight, considering that patients develop therapeutic relationships not only with PCPs but also with other clinical and support staff 1213—relationships that increase the effectiveness of specific interventions14 and may encourage patients to disclose mental health concerns and engage in care. Furthermore, despite evidence from adult and pediatric settings that a team-based approach may best facilitate the effective treatment of mental health problems in primary care,1516 there has been little consideration of how MAs function as part of the primary care team delivering mental health services. Findings ways to enhance the mental health treatment skills of MAs may have particular advantages in clinics where PCPs are culturally dissimilar from their patients but MAs share a common language or cultural heritage. For example, in home visiting programs and hospitals, paraprofessionals have functioned as “cultural bridges” between patients and providers in situations where differences in race, ethnicity, or language impede care.1718

In primary care, MAs are typically responsible for escorting patients to the examination room, orienting patients to the visit, measuring vital signs, gathering health information, and identifying the reason for the visit. They may also perform procedures, such as administering screening forms, giving immunizations, or collecting specimens. In these roles, MAs can “activate” patients to participate in the visit when helping them formulate their concerns and set the agenda for the visit. They can also encourage the patient to seek help from the PCP (for example, when a patient is uncertain whether he or she should discuss a problem) and may help patients feel optimistic that treatment will help. 1920

Nonetheless, MAs face several challenges to having therapeutic encounters with patients who have mental health problems. During their short visits with patients, the competing demands of physical health problems and maintaining productivity may impede the ability of MAs to respond to mental health concerns in an empathetic and supportive manner that encourages patients to discuss such concerns with their PCP.2123 MAs may also lack the time, skills, and confidence to encourage the disclosure of mental health problems and efficiently gather information about the problem.

This project piloted communication training for MAs in pediatric primary care to examine the feasibility of enhancing their skills. The training was piloted in a clinic that served a primarily Latino population to address the aforementioned challenges to delivering mental health services in primary care for this population and because a previous training for PCPs demonstrated positive results for Latino famlies.1 In addition, there is some evidence that Latinos have lower rates of patient activation than their non-Latino counterparts9 and that interventions designed to enhance patient activation could be particularly beneficial for engaging Latino populations in care and reducing disparities.2425 Thus, piloting the training in this setting not only yielded data to understand whether the training changed MAs’ behaviors and parents’ perceptions of care but also provided an opportunity to gather data to understand how the training could be tailored to a population who receives disparate mental health care.9,26

Following the example of other “complex interventions,”2728 the project began by refining the goals for the training and identifying contextual factors that could affect the design and outcomes of the training. The evaluation of the pilot not only gathered data on the potential effectiveness of the training—that is, whether it improved parents’ willingness to discuss mental health concerns with MAs or PCPs—but also gathered information on how the training functioned to create such outcomes by examining which skills the MAs learned and applied in their encounters with families and whether the training improved parents’ perceptions of their interactions with MAs. Thus, the evaluation sought to answer the following questions: (1) To what extent can MAs master the skills taught during the training; (2) Do patients’ perceptions of MAs and their willingness to discuss mental health concerns with MAs change improve after the training; and (3) Are patients more likely to discuss mental health concerns with PCPs after the training? The evaluation was intended to gather preliminary evidence about the feasibility of the training to inform refining the training for a large-scale trial and potential applications in other settings.

Methods

Setting

The training was developed in collaboration with a Federally Qualified Health Center (FQHC) that primarily serves Latino families in Washington, DC. The center provides a range of health and social services to adults, youths, and children at two urban sites and one suburban site. The training was conducted at the pediatric practice at the larger of the two urban sites, because the center administrators selected this site as having the greatest need for training.

To recruit MAs, nursing and mental health supervisory staff at the center helped arrange a series of meetings between the authors and MAs, during which the study goals, voluntary nature of participation, and intention not to share training outcomes with center management in a way that could identify specific participants were discussed.

Development of Training

The Gateway Provider Model (GPM) served as the guiding conceptual framework for the training.29 The GPM posits that characteristics of communities, service systems, providers, and patients interact in a complex and dynamic manner to influence whether PCPs identify and treat mental health problems. A key component of the GPM is the removal of barriers to the flow of information from patient to provider (as well as between providers) to facilitate the identification and treatment of mental health problems. Thus, the training was initially conceived as a mechanism to bolster the communication skills of MAs given their integral role in the flow of information - they serve a dual purpose of activating the patient to disclose mental health concerns and relay those concerns to the PCP. The training was intended to enhance the skills of MAs based on the hypothesis that the use of those skills would improve patients’ perceptions of the MAs as caring and empathetic, increasing their willingness to discuss mental health concerns with both MAs and PCPs, and ultimately facilitate the identification and treatment of mental health problems.

To understand how the GPM applied to this particular clinic and community, and inform the content and format of the training, the project began with a series of interviews with center leadership and four focus groups: one with the participating MAs (n = 7), one with parents (n = 8), and two with youth (n = 15) who had used services at the center. Center staff not otherwise involved in the study recruited parents and youth for the focus groups; all provided written informed consent and received $20 gift cards as honoraria. A child psychiatrist conducted the parent focus group in Spanish and the teen groups in English.

Interviews with center administrators sought to identify barriers to having therapeutic encounters with patients who have mental health problems and strategies MAs could use to overcome them. Focus groups with MAs sought to understand the challenges they encountered responding to patients with mental health problems, while focus groups with parents and teens explored the kinds of mental health problems for which they might consider seeking help at the center, then discussed experiences there that facilitated or hindered such care.

The initial content for training was based on a previously developed program designed to improve mental health-related interaction skills of pediatric PCPs.1 The training included teaching a set of skills drawn from patient activation interventions,3031 motivational enhancement,32 family therapy.33 We used the focus group and administrator interview data to map and adapt the original training content to the MAs’ core job functions (Table 1). Some components of the training addressed the unique needs of Latino patient populations. For example, the training component that covered following up on parent or youth/child hints that there might be a mental health problem included discussion of the words and phrases that Latino immigrant populations might use to describe their stresses and concerns. There was also discussion about how Latino patients viewed mental health treatment, common sources of mental health concerns in the local Latino community (which included a high rate of poverty) and the places families might find acceptable to seek mental health treatment. Finally, there was discussion about the need for privacy and confidentiality regarding mental health concerns within the local Latino community.

Table 1.

Content of Training

MA Job Function Communication Skills Targeted During Training
Greeting patients Using patient names and appropriate body language (for example, handshake, eye contact, smiling)
Orienting patients to the visit Explaining role of MA and next steps in visit
Identifying patients’ concerns (those of both parent and child) Asking open-ended questions to gather information about health and psychosocial problems and checking with both parent and child for “anything else?”
Managing rambling or numerous patient concerns Helping patients focus on current visit and prioritizing concerns
Following up on hints or clues of mental health problems Probing body language (for example, acting withdrawn) or ambiguous or leading statements (for example, “it’s nothing”).
Managing disagreement or anger between patients (for example, between mother and youth) Acknowledging concern, avoiding taking sides, and focusing on agenda for current visit
Responding to coerced patients (for example, patients forced to seek care by school) Avoiding vilifying the coercive force, empathizing with patient, and emphasizing the independent role of the clinic as helper
Setting agenda for visit with PCP Repeating list of concerns and getting agreement on the agenda from everyone present
Encouraging discussion of concerns with PCP Prompting patient to discuss health and mental health concerns with PCP and reiterating that concerns are appropriate for visit

Training was delivered in three one-hour lunchtime sessions spaced evenly over six weeks, during which MAs participated in group discussions, watched video examples of interactions between MAs and patients, and practiced their skills through role play. So as to be replicable, the sessions were structured by embedding the examples and cues for discussion in a set of “slides” that outlined the topics to be covered. A child psychiatrist facilitated each session. MAs were given pocket-sized reminder cards that prompted them to use the skills taught during the sessions.

Evaluation of Training

Given the lack of previous communication trainings for MAs, we followed the model of development of complex interventions,2728 by trying to develop a study that could most efficiently help us identify the key processes and outcomes that might need to be observed in a future trial. After discussion with site administrators, we decided that the most acceptable design that would meet this goal was a sequential cohort pre-post design, carried out at a single site in which three cohorts of families would be recruited to assess whether the training had any positive or negative impact on patients: one cohort was recruited the month immediately before the training, a second cohort was recruited the month immediately following the final training session, and the third cohort was recruited six months after the final training session. The study was intended to understand what skills MAs could learn during the training and whether parents would respond. Randomization of MAs to receive the training or delaying training to any of the MAs was not possible because the MAs work in such close collaboration with each other that contamination was inevitable. It was not possible to recruit a comparison group of families within the clinic because all MAs received the training at the same time and therefore all families interacted with the trained MAs following the training.

Bilingual (English and Spanish) research assistants approached every parent in the clinic waiting area to assess his or her eligibility and interest in participating in the study. To be eligible for the study, children had to range in age from 18 months to 16 years, and the parent/youth could not report child/youth pain of 5 or greater on a 1–10 scale to avoid burdening those in severe pain. Parents provided written consent, and youth ages 11–16 years provided written assent. Each participating parent received a $20 gift card as an honorarium.

All study forms were available in Spanish and English and administered orally if parents had difficulty reading. Before the visit, the parent completed a brief questionnaire reporting the reason for the visit and family demographic information. The parent also completed the Patient Health Questionnaire-9 (PHQ-9),34 which has been validated in Spanish,3536 to report his or her depressive symptoms and severity.

Parents of children ages 3–16 years completed the Strengths and Difficulties Questionnaire (SDQ)37 to measure mental health difficulties during the past six months. This measures, which has been widely used among Spanish-speaking populations,3839 yields an overall “total difficulties” score. Because the SDQ has not been widely used or validated for younger children, parents of children ages 18–36 months completed the Mental Health Inventory (MHI), a brief set of items from the Child Behavior Checklist (CBCL) specifically designed for such young children in the U.S. National Health Interview Survey.40 Both these instruments were scored using their standard algorithms. Based on U.S. norms, an SDQ total difficulties score of 12 or higher and a score of 3 or higher on the MHI indicated moderate to high symptoms.4142

Following the visit, the parent reported time spent with the MA (less than 5 minutes, 5–10 minutes, or more than 10 minutes) and completed a questionnaire indicating his or her willingness to discuss several health and mental health topics with the MA. The first series of questions, developed specifically for this project, asked, “How much do you agree that the medical assistant should talk with you about (1) why you are visiting the clinic today; (2) your child’s health; (3) how your child is doing in school; (4) your child’s feelings or behaviors; (5) your own health; (6) stress that you are feeling; (7) problems in your family; (8) questions about your child’s treatment?” Parents responded to each item using a five-point Likert scale ranging from “strongly disagree” to “strongly agree.” The second series of questions, derived from an evaluation of a communication training for PCPs,1 asked the parent to report whether the visit with the MA or PCP included discussion about the reason for the visit, the child’s behaviors or feelings, school performance, getting along with other children, parental stresses or strains, parental health, and family problems. Finally, parents answered the question, “How much do you agree that the medical assistant (1) asked inappropriate questions; (2) acted bossy; (3) made you feel uncomfortable; (4) didn’t understand why you came to the clinic?” using a five-point Likert scale that ranged from “strongly disagree” to “strongly agree.”1

Meetings with MAs, PCPs, and administrators were conducted six months following the training to debrief on their perceptions of the usefulness of the training and its impact on the clinic.

Uptake of Skills

Repeated observations of “visits” in which MAs performed their usual clinical tasks with two standardized patients (SP) were used to measure the uptake of skills. These visits were incorporated into the training as, respectively, a baseline session one month before the training began, opportunities to practice skills immediately after the first and third training sessions, and then a final SP visit about four months later. Following the example of previous communication trials that used SP visits,1 for each SP visit, MAs took a break between seeing real patients to come to an examination room in the clinic where they were briefed by a study team member who introduced them to the SPs and gave them a mock patient record on which to take notes. Each SP visit involved a mother and teenage daughter who followed scripts designed to elicit use of the skills taught during the training. The visits were video recorded both for analysis and so that MAs could receive a copy for their own review. The SPs were prepared by rehearsing their script with a psychiatrist and another researcher. Two research assistants coded the videos to assess whether or not MAs demonstrated the skills. The research assistants practiced with videos from a prior training of PCPs to become proficient with the coding scheme. The videos were assigned random numbers so coders were blinded to the sequence of visits. Coders worked independently, then compared their coding and watched the videos together to resolve any discrepancies. Disagreements were resolved by the child psychiatrist.

The videos were also coded to assess MAs’ patient-centeredness—a philosophy of care that includes attending to patients’ psychosocial needs, encouraging disclosure of concerns, building a patient-provider partnership, and actively facilitating patient involvement in decisions.43 Patient-centeredness, which is associated with positive patient outcomes, has been measured in several ways.44 The videos in this project were coded using the Roter Interaction Analysis System (RIAS),4546 which classifies each speaker’s utterances into one of several exclusive categories. We measured patient-centeredness by summing several RIAS categories: provider talk in medical and psychosocial information giving; asking psychosocial questions; showing empathy; giving reassurance; and partnership building (that is, talk that facilitates conversation). A single coder analyzed all the videos after training to reliability on a different set of videos collected during a previous study.47

Analysis

Chi-square and t-tests were conducted to examine bivariate relationship; paired t-tests were used to examine changes in patient-centeredness over time. Because responses were skewed and not distributed normally, responses to questions regarding parent willingness to discuss mental health and other psychosocial concerns with MAs were dichotomized into “strongly agreed” versus all others while questions regarding parent perceptions of the MAs were dichotomized into “strongly disagree” versus all others.

Because different instrumentation was used to gather information about child mental health symptoms, we constructed a variable that identified if the child/youth had moderate to high symptoms on either the MHI or SDQ. We constructed another to indicate if the parent discussed any one of the five mental health topics with the MA or PCP.

Logistic regression was used to model the odds that a parent strongly agreed the MA should discuss a specific topic as a function of the cohort in which the data were collected and covariates that were hypothesized to potentially influence the outcome, including the number of years the family received services at the clinic (first visit, less than one year, one to five years, or more than five years), reason for the visit (well-child, acute/sick care, or other), child age and gender, PHQ score, moderate to high child/youth mental health symptoms, child pain, parent education (less than high school, high school, or more than high school) and age, number of years living in the United States, and whether there was any discussion of mental health with the MA.

Logistic regression was also used to model the odds that the parent strongly disagreed the MA asked inappropriate questions, acted bossy, made the parent feel uncomfortable, or did not understand the reason for the visit as a function of the same variables.

Human Subjects

The Johns Hopkins School of Public Health Institutional Review Board approved the instrumentation, recruitment, and data collection procedures.

Results

Participants

All seven MAs agreed to participate. All were Latino and native Spanish speakers; two were male and five female. They had worked at the clinic for an average of 2.5 years (SD = 1.8). None had training in mental health, child behavior, or child development. One was a physician in his native country but was not licensed in the United States.

The baseline cohort of families consisted of 136 parent-child pairs recruited one month before training. The second cohort included 137 pairs and the final cohort 123 pairs. Table 2 summarizes health and demographic characteristics of parents and children/youth. None of these characteristics differed among the cohorts. Parents’ average age was 32 years (SD = 7.8) and they resided in the United States for an average of 10.6 years (SD = 6.14); most were mothers.

Table 2.

Family Demographic and Mental Health Characteristics

Characteristics Number Percent
Child Age
 10–35 months 80 20.2
 3–6 years 209 52.8
 7–10 years 57 14.4
 11–16 years 50 12.6
Child Gender
 Male 200 50.5
 Female 196 49.5
Child Ethnicity
 Hispanic/Latino 389 98.2
 African not Hispanic/Latino 7 1.76
 Other 3 0.75
Child Country of Birth
 United States 358 90.4
 El Salvador 21 5.3
 Other 17 4.3
Language Child Speaks at Home
 Spanish 256 64.8
 English 63 16.0
 Both Spanish and English 71 17.9
 Other 5 1.3
Child scored medium to High Mental Health Symptoms (SDQ or MHI) 143 36.1
Parent/Guardian Relationship
 Mother 348 87.9
 Father 35 8.8
 Grandparent or other relative 13 3.3
Parent Country of Birth
 United States 17 4.3
 El Salvador 240 60.7
 Other Latin American country 133 33.5
 Other 6 1.5
Parent Education
 Less than high school 161 41.3
 High school 54 13.9
 More than high school 175 44.8
Parent Medium to High Depressive Symptoms (PHQ) 130 32.8
Financial Difficulty
 More than enough money 2 .5
 Enough money 139 35.5
 Not enough money 251 64.0
Number of Years Coming to Clinic
 First visit 13 3.3
 Less than 1 year 19 4.8
 1–5 years 176 44.4
 More than 5 years 188 47.5
 Total Number of families recruited 396 100.0

Uptake of Skills

MAs changed their communication style from the pre-training interaction with SPs compared with immediately following the final training session. The total number of patient-centered utterances per visit increased from an average of 38.1 at baseline (SD = 13.7) to 45.0 after the final training (SD = 12.9, p = .035) (Table 3). However, at the six-month assessment, the average fell back to baseline levels (37.3, SD = 5.1).

Table 3.

Uptake of Skills and Change in Patient-Centeredness

Skill or Interaction One Week Before First Training Session Immediately After Final Training Session Six Months After First Training Session*
Any verbal greeting 2/7 2/7 3/6
Any orienting statement 7/7 6/7 6/6
Any open-ended opening 7/7 7/7 6/6
Any checking for additional concerns 3/7 5/7 5/6
Any discussion of agenda 1/7 5/7 2/6
Any encouraging statements 4/7 7/7 4/6
Total patient-centered utterances/visit (mean and SD)** 38.1 (13.7) 45.0 (12.9) 37.3 (5.1)
*

Only six of the seven trainees were able to participate in the final SP visit.

**

Difference between one week before first training session and immediately after final training session statistically significant at p =.035

MAs changed their use of some skills taught but not others (Table 3). Given the small number of trainees, we did not attempt to apply statistical tests. All MAs used orienting statements (stating their role, saying what would happen next) and started with open-ended questions at baseline, and they continued both at follow-up. Only two practiced specific verbal greetings (using the parent’s or child’s name) at baseline; this did not improve. Checking for additional concerns, reviewing the agenda for the visit, and making encouraging statements all improved from baseline, but the last two became less common at follow-up.

Parent Communication with Medical Assistants

More parents recruited one and six months after the training were willing to discuss mental health problems compared with parents recruited before the training (Figure 1). For example, before the training only 67% of parents strongly agreed that they should discuss their child’s feelings or behaviors with the MA compared with 91% of parents directly after the training and 85% of parents six months after the training. These findings were consistent after controlling for covariates (Table 4). Parents with moderate to high PHQ scores were more than twice as likely to say MAs should discuss parental stress. Those with more than a high school education were less likely than those with less than a high school education to agree MAs should discuss mental health.

Figure 1.

Figure 1

Proportion of Parents Who Strongly Agreed that MA Should Discuss Topic

Table 4.

Logistic Regression of Parent Willingness to Discuss Topic with Medical Assistant

Strongly Agreed that MA Should Discuss Child’s Feelings or Behaviors
Strongly Agreed that MA Should Discuss Parental Stress
Strongly Agreed that MA Should Discuss Family Problems
Odds Ratio 95% CI Odds Ratio 95% CI Odds Ratio 95% CI
One month after training 5.46*** 2.26–13.21 2.33* 1.17–4.63 2.16** 1.20–3.86
Six months after training 2.09* 1.01–4.35 2.11* 1.06–4.16 1.51 0.84–2.69
Years visiting clinic
 1–5 years 0.47 0.10–2.14 2.20 0.47–4.46 1.05 0.39–2.75
 More than 5 years 0.59 0.12–2.81 1.45 0.69–7.01 1.07 0.37–2.56
Reason for visit
 Acute/sick visit 0.91 0.43–1.91 0.72 0.38–1.38 1.32 0.85–2.25
 Other 0.40 0.10–1.63 0.19 0.05–1.01 0.95 0.31–3.10
Child age 0.90 0.81–1.01 1.04 0.95–1.13 1.05 0.98–1.13
Female child 0.81 0.42–1.55 1.28 0.73–2.18 1.12 0.71–1.78
Moderate/high PHQ score 1.27 0.60–2.71 2.29** 1.19–4.39 1.41 0.84–2.36
Moderate/high SDQ or MHI score 0.76 0.38–1.55 0.79 0.42–1.47 0.90 0.56–1.54
Child pain (5 or higher) 1.08 0.85–1.37 1.24 0.09–1.53 1.03 0.87–1.23
Discussed psychosocial topic with MA 2.96** 1.29–6.77 3.01*** 1.53–5.88 1.85* 1.09–3.01
Parent education level
 At least high school 0.89 0.29–2.64 0.41* 0.17–1.01 0.86 0.39–1.72
 More than high school 0.48* 0.23–0.97 0.24*** 0.13–.469 0.55* 0.32–.881
Years in US 1.06 0.99–1.13 0.96 0.92–1.01 0.98 0.94–1.03
Parent age 1.00 0.95 –1.05 0.99 0.96–1.03 1.02 0.98–1.05
*

p < 0.05

**

p < 0.01

***

p < 0.001

Parents had more positive perceptions of MAs following the training. After controlling for the same covariates shown in Table 4, those recruited in the month after the training had 1.81 higher odds (95% CI: 1.01–3.27) of strongly disagreeing that the MA asked inappropriate questions, 2.11 higher odds (95% CI: 1.12–3.91) of strongly disagreeing that the MA acted bossy, 2.22 higher odds (95% CI: 1.34–4.32) of strongly disagreeing that the MA made them feel uncomfortable, and 3.24 higher odds (95% CI: 1.65–6.25) of strongly disagreeing that the MA did not understand the reason for the visit. All but one of these findings (“made me feel uncomfortable”) were sustained six months following the training.

Roughly 30% of parents in each cohort discussed at least one mental health topic with the MA, which did not change after the training. However, before the training, 10.2% of parents discussed mental health with the MA but not the PCP, compared with 5.8% in the month following the training, but this never happened six months after the training.

The training did not increase visit length as reported by the parent. Before the training, 27.8% (n=32) of parents spent more than 10 minutes with the MA, 43.5% (n=50) 5–10 minutes, and 28.7% (n=33) less than 5 minutes. In the month following the training, 12.6% (n=15) spent more than 10 minutes with the MA, 62.2% (n=74) 5–10 minutes, and 25.2% (n=30) less than 5 minutes (p=0.01); the finding was similar six months after the initial training. Thus, after the training fewer parents spent more than 10 minutes with the MA.

Staff Feedback on Training

During meetings with staff six months after the training, MAs perceived they were better at helping families feel comfortable during the visit and encouraging them to discuss their mental health problems, though they also believed some parents preferred to discuss mental health with PCPs because MAs lacked “status.”

MAs reported using more open-ended questions after the training and feeling more prepared to respond to parents who were angry or stressed during the visit. They said the training prompted them to promote other center services, and they felt empowered to suggest parents talk to a social worker as well as the PCP. However, they felt that productivity demands and the difficulty of finding private space in which to engage families limited their ability to use their new skills.

Although MAs found the training engaging, they would have liked more time for practice and direct feedback on their work. They thought new staff should receive the training but did not wish to become trainers, in part because none wanted to be viewed as superior to others.

PCPs did not perceive a change in MAs’ behavior and reported no impact, positive or negative, of the training on their interactions with MAs or patients. PCPs did not feel that there had been a change in the number of patients identified as having mental health problems. They supported training for the MAs but underscored a requirement that it not slow the pace of visits or reduce practice efficiency.

Discussion

A brief communication skills training for MAs was associated with improvements in parents’ perceptions of care and their willingness to discuss mental health concerns. After the training fewer parents reported that they had told an MA about a concern but then did not discuss it with the PCP. We have some evidence that these shifts in attitudes resulted from changes in MAs’ interactions with parents. In the SP assessments, MAs increased their patient-centeredness, voiced more encouragement, and more often asked families to consider their agenda for the visit. MAs were receptive to the training and reported improvement in their abilities to respond to patients with mental health problems and function as part of a team to encourage patients to utilize other resources at the center. MAs were able to apply their skills without lengthening visits or disrupting patient care. PCPs were supportive of the training and did not report an increase in the volume of patients with mental health problems following the training, suggesting that MAs were not overburdening PCPs with additional work.

Though to our knowledge this is the first study that aimed to teach mental health-related communication skills to MAs, it is one of many studies that have found that brief training can have lasting impact on the way that clinicians interact with patients.4849 The results of the training are consistent with those of a prior trial involving PCPs, where a similar training (which also consisted of three one-hour sessions followed by practice with standardized patients) increased patient-centeredness among trainees compared to control providers who received only written material.1,50 The need for communication skills training has been advocated across the span of medical education, and for a other office staff in primary care practices.51

There are several limitations to this pilot study. The results of a quasi-experimental study at a single site must be considered preliminary. Compared with those in other studies, a high proportion of parents reported at baseline that they discussed mental health during the visit.4 A ceiling effect may have occurred in the ability to increase the proportion of visits including discussion of mental health within this particular clinic. Future efforts should examine how similar training functions in different practice structures and among different racial and ethnic groups.

Although these findings are promising, in the absence of a comparison group, we cannot rule out that some external factors could account for the improvements in parents’ perceptions of care and willingness to discuss mental health. However, no organizational changes or competing educational interventions occurred, and, notably, several of our findings follow a pattern in which the outcome improves directly following the initial training and then dissipates but remains significantly better six months later (changes also consistent with the decay in patient-centeredness and skill use found at four months after training). In fact, the lack of organizational change–in particular the ongoing pressure for productivity and the inability to resolve the dilemma of not having private space in which MAs could ascertain patient concerns -- is a likely contributor to the fall-off in use of skills. Though it would certainly be desirable to have ongoing sessions reviewing the skills for which there was less uptake, the dissemination literature emphasizes that clinician behavioral change requires organizational support and, in many cases, organizational changes.52 In addition, the clinic may benefit from some means of institutionalizing this sort of patient-MA interaction either through continued education or incorporating these communication skills into MAs’ core competencies.

Implications for Behavioral Health

Training MAs could be a relatively easy-to-implement component of educational and organizational interventions aimed at improving primary care mental health services.3 MAs could receive training as part of the implementation of collaborative care or other efforts to integrate behavioral and physical health services. Such training would be consistent with the medical home model of delivering primary care, which emphasize the importance of a team-based approach to care that capitalize on the resources of practices to address the health and mental health needs of patients.5354 The transformation of primary care practices into fully functioning medical homes is a core component of the Patient Protection and Affordable Care Act of 2010 (ACA), which provides financial incentives, training, and other forms of support for practices to become medical homes. In addition, expansion of Medicaid eligibility requirements and the establishment of health insurance exchanges as a result of the ACA are expected to extend health care coverage to millions of Americans with mental health problems55–individuals who are most likely to seek help in primary care given the chronic shortage of mental health specialists.56 Facing the increased strain of treating more patients with mental health problems, primary care practices need practical strategies to best utilize their staff resources and enhance the functioning of the primary care team to address mental health concerns. Training MAs could be one component of such strategies and could complement efforts of the Substance Abuse and Mental Health Services Administration (SAMHSA) and Agency for Healthcare Research and Quality (AHRQ) to integrate mental health and primary care services. This pilot study suggests that MAs may be well positioned to help promote the discussion of mental health problems with PCPs and help patients feel comfortable with the idea of primary care as a source of mental health care but further work is needed to understand the other roles that MAs could play in the actual delivery of care. Further, given the range of educational backgrounds and levels of training among MAs, primary care practices will need to navigate the appropriate scope of practice for MAs.

The training is practical–it required about three hours of MA time, was delivered at times that MAs were available, and provided MAs with an opportunity to share their experiences, learn from each other, and model behaviors. The use of SPs in this type of training has become common, and the short (approximately five-minute) SP visits with MAs are an efficient way for MAs to practice their skills without disrupting work.

Nonetheless, the study identified some threats to scalability and avenues for future research. Threats to scalability included the location where MAs interacted with patients: a nursing station where privacy was not assured and the overall pace of work challenged the ability to discuss sensitive topics. Consistent with MAs’ concerns about the lack of their social status with patients, more educated parents were slightly less willing to talk with them about mental health. Finally, MAs worried that becoming trainers would create status differences among themselves.

Although parents’ willingness to discuss every type of mental health concern improved after training, roughly 40% remained uncertain MAs should discuss family problems, and about 25% did not strongly agree they should discuss parental stress. Other studies have suggested mothers are more satisfied with pediatric visits during which the PCP addresses maternal stress,57 but some parents might have reservations about discussing such problems with MAs. These concerns should be considered in future trainings and may have been specific to this clinic or population.

This pilot focused on pediatric primary care but replication of this training in adult primary care and other community health care settings would provide opportunities to understand how the training could be adapted to fit the context and needs of a particular site or community. Implementation of the training in diverse sites may allow for quasi-experimental comparisons of outcomes and be useful to understand how this type of training can be integrated with other models of care and improve outcomes. There are plans to test this type of MA training through a multi-arm randomized trial that would involve multiple clinics and examine the benefits of combining the MA training with PCP communication training and changes in practice workflows. Future evaluations of the training would examine possible changes in referrals, billing patterns, and health outcomes. There are also tentative considerations of incorporating the training into states’ efforts to enhance mental health services in primary care for children and consider whether MAs could play other roles in the delivery of care, including care monitoring and follow-up with families.

As providers, payers, and policymakers look for efficient strategies to improve the delivery of mental health services in primary care, opportunities may exist for MAs to acquire skills and take a more substantial role in delivering care. Further research should build on this preliminary work to test the training in other settings and among different patient populations and identify other skills MAs could learn to promote primary care as a source of mental health treatment and more effectively function as part of the primary care team.

Acknowledgments

National Institute of Mental Health grant R21MH083625 supported this work. We appreciate the participation of the families, medical assistants, and other clinic staff.

Footnotes

Conflict of interest: The authors have no conflict of interest associated with the publication of this manuscript.

Contributor Information

Jonathan D. Brown, Email: jbrown@mathematica-mpr.com, Mathematica Policy Research, Inc. 1100 First Street, 12th floor, Washington, DC 20002, Phone: 202-264-3446, Fax: 202-863-1763.

Lawrence S. Wissow, Email: lwissow@jhsph.edu, Department of Health, Behavior, and Society, Johns Hopkins University Bloomberg School of Public Health. 624 N. Broadway Room 703 Baltimore, MD 21205 Phone: 410-614-1243, Fax: 410-955-7241.

Benjamin L. Cook, Email: bcook@charesearch.org, Cambridge Health Alliance, Center for Multicultural Mental Health Research, 120 Beacon Street, Somerville, MA 02143, Phone: 6717-503-8449, Fax: 617-503-8430.

Shaina Longway, Email: slongway@jhsph.edu, Department of Health, Behavior, and Society, Johns Hopkins University Bloomberg School of Public Health. 624 N. Broadway Room 703 Baltimore, MD 21205 Phone: 410-614-1243, Fax: 410-955-7241.

Emily Caffery, Email: ecaffery@mathematica-mpr.com, Mathematica Policy Research, Inc., 220 East Huron Street, Suite 300, Ann Arbor, MI, 48104, Phone: 734-794-1126, Fax: 734-794-0241.

Chris Pefaure, Email: cpefaure@mathematica-mpr.com, Mathematica Policy Research, Inc., 1100 First Street, 12th floor, Washington, DC 20002, Washington, DC 20024, Phone: 202-484-9220, Fax: 202-863-1763.

References

  • 1.Wissow LS, Gadomski A, Roter D, et al. Improving child and parent mental health in primary care: a cluster-randomized trial of communication skills training. Pediatrics. 2008;121:266–275. doi: 10.1542/peds.2007-0418. [DOI] [PubMed] [Google Scholar]
  • 2.Asarnow JR, Jaycox LH, Duan N, et al. Effectiveness of a quality improvement intervention for adolescent depression in primary care clinics: a randomized controlled trial. Journal of the American Medical Association. 2005;293(3):311–319. doi: 10.1001/jama.293.3.311. [DOI] [PubMed] [Google Scholar]
  • 3.Asarnow JR, Jaycox LH, Anderson M. Depression among youth in primary care models for delivering mental health services. Child Adolescent Psychiatry Clinics of North America. 2002;11(3):477–97. doi: 10.1016/s1056-4993(02)00006-8. [DOI] [PubMed] [Google Scholar]
  • 4.Brown JD, Riley AW, Wissow LS. Identification of youth psychosocial problems during pediatric primary care visits. Administration & Policy in Mental Health. 2007;34:269–281. doi: 10.1007/s10488-006-0106-7. [DOI] [PubMed] [Google Scholar]
  • 5.Leaf PJ, Owens PL, Leventhal JM, et al. Pediatricians’ training and identification and management of psychosocial problems. Clinical Pediatrics. 2004;43(4):355–365. doi: 10.1177/000992280404300407. [DOI] [PubMed] [Google Scholar]
  • 6.Kelleher KJ, Stevens J. Evolution of child mental health services in primary care. Academic Pediatrics. 2009;9(1):7–14. doi: 10.1016/j.acap.2008.11.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Clayman ML, Wissow LS. Pediatric residents’ response to ambiguous words about child discipline and behavior. Patient Education & Counseling. 2004;55:16–21. doi: 10.1016/S0738-3991(03)00245-3. [DOI] [PubMed] [Google Scholar]
  • 8.Guarnaccia P, Martinex I, Acosta H. Mental health in the Hispanic immigrant community. Journal of Immigrant & Refugee Services. 2005;3:21–46. [Google Scholar]
  • 9.Cunningham PJ, Hibbard J, Gibbons CB. Raising low ‘patient activation’ rates among Hispanic immigrants may equal expanded coverage in reducing access disparities. Health Affairs. 2011;30(10):1888–1894. doi: 10.1377/hlthaff.2009.0805. [DOI] [PubMed] [Google Scholar]
  • 10.Brown JD, Wissow LS, Zachary C, Cook BL. Receiving advice about child mental health from a primary care provider: African American and Hispanic parent attitudes. Medical Care. 2007;45(11):1076–1082. doi: 10.1097/MLR.0b013e31812da7fd. [DOI] [PubMed] [Google Scholar]
  • 11.Chapman S, Marks A, Chan M. The increasing role of medical assistance in small primary care physician practice: Key issues and policy implications. Center for the Health Professions, University of California; San Francisco: 2010. [Google Scholar]
  • 12.Ware NC, Tugenberg T, Dickey B, et al. An ethnographic study of the meaning of continuity of care in mental health services. Psychiatric Services. 1999;50:395–400. doi: 10.1176/ps.50.3.395. [DOI] [PubMed] [Google Scholar]
  • 13.Pulido R, Monari M, Rossi N. Institutional therapeutic alliance and its relationship with outcomes in a psychiatric day hospital program. Archieves of Psychiatric Nursing. 2008;22:277–287. doi: 10.1016/j.apnu.2007.07.003. [DOI] [PubMed] [Google Scholar]
  • 14.van Os TW, van den Brink RH, Tiemens BG, et al. Communicative skills of general practitioners augment the effectiveness of guideline-based depression treatment. Journal of Affective Disorders. 2005;84:43–51. doi: 10.1016/j.jad.2004.09.005. [DOI] [PubMed] [Google Scholar]
  • 15.Butler M, Kane RL, McAlpine D, et al. Integration of mental health/substance abuse and primary care. Rockville, MD: Agency for Healthcare Research and Quality; 2008. AHRQ Publication No. 09–E003. [Google Scholar]
  • 16.Bower P, Gilbody S, Richards D, et al. Collaborative care for depression in primary care. Making sense of a complex intervention: systematic review and meta-regression. British Journal of Psychiatry. 2006;189:484–493. doi: 10.1192/bjp.bp.106.023655. [DOI] [PubMed] [Google Scholar]
  • 17.Olds DL, Robinson J, Pettitt L, et al. Effects of home visits by paraprofessionals and by nurses: age 4 follow-up results of a randomized trial. Pediatrics. 2004;114:1560–1568. doi: 10.1542/peds.2004-0961. [DOI] [PubMed] [Google Scholar]
  • 18.Derose KP. Networks of care: how Latina immigrants find their way to and through a county hospital. Journal of Immigrant Health. 2000;2:79–87. doi: 10.1023/A:1009533901752. [DOI] [PubMed] [Google Scholar]
  • 19.Ferrer RL, Mody-Bailey P, Jaen CR, et al. A medical assistant-based program to promote healthy behaviors in primary care. Annals of Family Medicine. 2009;7:504–512. doi: 10.1370/afm.1059. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Hibbard JH, Cunningham PJ. How engaged are consumers in their health and health care, and why does it matter? Research Briefs. 2008;(8):1–9. [PubMed] [Google Scholar]
  • 21.Brown JD, Wissow LS, Riley AW. Physician and patient characteristics associated with discussion of psychosocial health during pediatric primary care visits. Clinical Pediatrics. 2007;46:812–820. doi: 10.1177/0009922807304144. [DOI] [PubMed] [Google Scholar]
  • 22.Furedi J, Rozsa S, Zambori J, et al. The role of symptoms in the recognition of mental health disorders in primary care. Psychosomatics. 2003;44:402–406. doi: 10.1176/appi.psy.44.5.402. [DOI] [PubMed] [Google Scholar]
  • 23.Epner JE, Levenberg PB, Schoeny ME. Primary care providers’ responsiveness to health-risk behaviors reported by adolescent patients. Archives of Pediatric & Adolescent Medicine. 1998;152:774–780. doi: 10.1001/archpedi.152.8.774. [DOI] [PubMed] [Google Scholar]
  • 24.Alegria M, Sribney W, Perez D, et al. The role of patient activation on patient-provider communication and quality of care for US and foreign born Latino patients. Journal of General Internal Medicine. 2009;24 (Suppl 3):534–541. doi: 10.1007/s11606-009-1074-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Cortes DE, Mulvaney-Day N, Fortuna L, et al. Patient--provider communication: understanding the role of patient activation for Latinos in mental health treatment. Health Education & Behavior. 2009;36:138–154. doi: 10.1177/1090198108314618. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Vega WA, Lopez SR. Prioirty issues in Latino mental health services research. Mental Health Services Research. 2001;3(4):189–200. doi: 10.1023/a:1013125030718. [DOI] [PubMed] [Google Scholar]
  • 27.Rowlands G, Sims J, Kerry S. A lesson learnt: the importance of modeling in randomized controlled trials for complex interventions in primary care. Family Practice. 2005;22:132–139. doi: 10.1093/fampra/cmh704. [DOI] [PubMed] [Google Scholar]
  • 28.Campbell NC, Murray E, Darbyshire J, et al. Designing and evaluating complex interventions to improve health care. British Medical Journal. 2007;334:455–459. doi: 10.1136/bmj.39108.379965.BE. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Stiffman AR, Pescosolido B, Cabassa LJ. Building a model to understand youth service access: the gateway provider model. Mental Health Services Ressearch. 2004;6(4):189–198. doi: 10.1023/b:mhsr.0000044745.09952.33. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Greenfield S, Kaplan S, Ware JE., Jr Expanding patient involvement in care. Effects on patient outcomes. Annals of Internal Med. 1985;102:520–528. doi: 10.7326/0003-4819-102-4-520. [DOI] [PubMed] [Google Scholar]
  • 31.Kaplan SH, Greenfield S, Ware JE., Jr Assessing the effects of physician-patient interactions on the outcomes of chronic disease. Medical Care. 1989;27:S110–27. doi: 10.1097/00005650-198903001-00010. [DOI] [PubMed] [Google Scholar]
  • 32.Miller WR, Rollnick S. Motivational Interviewing: Preparing People to Change Addictive Behavior. New York, NY: Guilford Press; 1991. [Google Scholar]
  • 33.Allmond BW, Jr, Tanner JL, Gofman HF. The Family Is the Patient: Using Family Interviews in Children’s Medical Care. 2. Baltimore, MD: Williams & Wilkins; 1999. [Google Scholar]
  • 34.Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. Journal of the American Medical Association. 1999;282:1737–1744. doi: 10.1001/jama.282.18.1737. [DOI] [PubMed] [Google Scholar]
  • 35.Wulsin L, Somoza E, Heck J. The Feasibility of Using the Spanish PHQ-9 to Screen for Depression in Primary Care in Honduras. Primary Care Companion to the Journal of Clinical Psychiatry. 2002;4:191–195. doi: 10.4088/pcc.v04n0504. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Diez-Quevedo C, Rangil T, Sanchez-Planell L, et al. Validation and utility of the Patient Health Questionnaire in diagnosing mental disorders in 1003 general hospital Spanish inpatients. Psychosomatic Med. 2001;63:679–686. doi: 10.1097/00006842-200107000-00021. [DOI] [PubMed] [Google Scholar]
  • 37.Goodman R. The extended version of the Strengths and Difficulties Questionnaire as a guide to child psychiatric caseness and consequent burden. Journal of Child Psychology & Psychiatry. 1999;40:791–799. [PubMed] [Google Scholar]
  • 38.Achenbach TM, Becker A, Dopfner M, et al. Multicultural assessment of child and adolescent psychopathology with ASEBA and SDQ instruments: research findings, applications, and future directions. Journal of Child Psychology & Psychiatry. 2008;49:251–275. doi: 10.1111/j.1469-7610.2007.01867.x. [DOI] [PubMed] [Google Scholar]
  • 39.Woerner W, Fleitlich-Bilyk B, Martinussen R, et al. The Strengths and Difficulties Questionnaire overseas: evaluations and applications of the SDQ beyond Europe. European Child and Adolescent Psychiatry. 2004;13(Suppl 2):II47–54. doi: 10.1007/s00787-004-2008-0. [DOI] [PubMed] [Google Scholar]
  • 40.NHIS English and Spanish Questionnaires. 2008 http://www.cdc.gov/nchs/nhis/quest_data_related_1997_forward.htm.
  • 41.Bourdon KH, Goodman R, Rae DS, et al. The Strengths and Difficulties Questionnaire: U.S. normative data and psychometric properties. Journal of the American Academy of Child and Adolescent Psychiatry. 2005;44:557–564. doi: 10.1097/01.chi.0000159157.57075.c8. [DOI] [PubMed] [Google Scholar]
  • 42.Pastor PN, Duran CR. NHIS Mental health indicator: does it identify 2–3 year olds with emotional/behavioral problems?. Presentation for the CDC-SAMSHA Children’s Mental Health Seminar; March 17, 2010. [Google Scholar]
  • 43.Levinson W, Lesser CS, Epstein RM. Developing physician communication skills for patient-centered care. Health Affairs. 2010;29:1310–1318. doi: 10.1377/hlthaff.2009.0450. [DOI] [PubMed] [Google Scholar]
  • 44.Mead N, Bower P. Measuring patient-centredness: a comparison of three observation-based instruments. Patient Education & Counseling. 2000;39:71–80. doi: 10.1016/s0738-3991(99)00092-0. [DOI] [PubMed] [Google Scholar]
  • 45.Roter DL, Hall JA, Kern DE, Barker LR, Cole KA, Roca RP. Improving physicians’ interviewing skills and reducing patients’ emotional distress. A randomized clinical trial. Archives of Internal Medicine. 1995;155(17):1877–1884. [PubMed] [Google Scholar]
  • 46.Wissow LS, Roter DL, Wilson ME. Pediatrician interview style and mothers’ disclosure of psychosocial issues. Pediatrics. 1994;93:289–295. [PubMed] [Google Scholar]
  • 47.Roter DL, Stewart M, Putnam SM, et al. Communication patterns of primary care physicians. Journal of the American Medical Association. 1997;277:350–356. [PubMed] [Google Scholar]
  • 48.Fallowfield L, Jenkins V, Farewell V, Solis-Trapala I. Enduring impact of communication skills training: Results of a 12-month follow-up. British Journal of Cancer. 2003;89(8):1445–1449. doi: 10.1038/sj.bjc.6601309. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Finset A, Ekeberg O, Eide H, Aspegren K. Long term benefits of communication skills training for cancer doctors. Psychooncology. 2003;12(7):686–93. doi: 10.1002/pon.691. [DOI] [PubMed] [Google Scholar]
  • 50.Wissow L, Gadomski A, Roter D, et al. Aspects of mental health communication skills training that predict parent and child outcomes in pediatric primary care. Patient Education & Counseling. 2011;82:226–232. doi: 10.1016/j.pec.2010.03.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Brown JD, Wissow LS. Rethinking the mental health treatment skills of primary care staff: A framework for training and research. Administration and Policy in Mental Health. 2011 doi: 10.1007/s10488-011-0373-9. [DOI] [PubMed] [Google Scholar]
  • 52.Beidas RS, Kendall PC. Training therapists in evidence-based practice: A critical review of studies from a systems contextual perspective. Clinical Psychology Review. 2010;17(1):1–30. doi: 10.1111/j.1468-2850.2009.01187.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Croghan Thomas, Brown Jonathan D. Report prepared for the Agency for Healthcare Research and Quality. Washington, DC: Mathematica Policy Research; Mar 9, 2010. Integrating Mental Health in the Patient- Centered Medical Home. [Google Scholar]
  • 54.Kelleher KJ, Stevens J. Evolution of child mental health services in primary care. Academic Pediatrics. 2009;9:7–14. doi: 10.1016/j.acap.2008.11.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Donohue J, Garfield R, Lave J. The impact of expanded health insurance coverage on individuals with mental illnesses and substance use disorders [White paper] Office of the Assistant Secretary for Planning and Evaluation, Department of Health and Human Services; 2010. [Google Scholar]
  • 56.Ellis AR, Konrad TR, Thomas KC, Morrissey JP. County-level estimates of mental health professional supply in the United States. Psychiatric Services. 2009;60(10):1315–1322. doi: 10.1176/ps.2009.60.10.1315. [DOI] [PubMed] [Google Scholar]
  • 57.Brown JD, Wissow LS. Discussion of maternal stress during pediatric primary care visits. Ambulatory Pediatrics. 2008;8:368–374. doi: 10.1016/j.ambp.2008.08.004. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES