Training Pediatric Residents in a Primary Care Clinic to Help Address Psychosocial Problems and Prevent Child Maltreatment

Abstract

OBJECTIVE

The objectives of this study were to determine whether 1) residents trained in the SEEK (A Safe Environment for Every Kid) model would report improved attitudes, knowledge, comfort, competence, and practice regarding screening for psychosocial risk factors (parental depression, parental substance abuse, intimate partner violence, stress, corporal punishment, and food insecurity); 2) intervention residents would be more likely to screen for and assess those risk factors; and 3) families seen by intervention residents would report improved satisfaction with their child’s doctor compared to families receiving standard care from control residents.

METHODS

Pediatric residents in a university-based pediatrics continuity clinic were enrolled onto a randomized controlled trial of the SEEK model. The model included resident training about psychosocial risk factors, a Parent Screening Questionnaire, and a study social worker. Outcome measures included: 1) residents’ baseline, 6-month, and 18-month posttraining surveys, 2) medical record review, and 3) parents’ satisfaction regarding doctor-parent interaction.

RESULTS

Ninety-five residents participated. In 4 of 6 risk areas, intervention residents scored higher on the self-assessment compared to control subjects, with sustained improvement at 18 months. Intervention residents were more likely than control subjects to screen and assess parents for targeted risk factors. Parents seen by intervention residents responded favorably regarding interactions with their doctor.

CONCLUSIONS

The SEEK model helped residents become more comfortable and competent in screening for and addressing psychosocial risk factors. The benefits were sustained. Parents viewed the intervention doctors favorably. The model shows promise as a way of helping address major psychosocial problems in pediatric primary care.

SCREENING FOR PSYCHOSOCIAL risk factors is an increasingly important aspect of the pediatrician’s role.¹ The American Academy of Pediatrics programs such as Practicing Safety and Bright Futures suggest that physicians should discuss and monitor for risk factors such as intimate partner violence (IPV), family stress, maternal depression, and effective disciplinary strategies.^2–4 Maryland, as well as other states, has incorporated family psychosocial assessments in their EPSDT (Early Periodic Screening, Diagnosis, Treatment) schedules.^5,6 Screening for these risk factors may not only improve child outcomes, but may also help prevent child maltreatment, which remains a pervasive problem.⁷

Physicians have multiple opportunities to screen for psychosocial risk factors during child health supervision visits. However, surveys suggest there are gaps in physician’s knowledge and skills, as well as discomfort with addressing these issues.^8–11 Residents who learn how to address family psychosocial problems may be more likely to screen for those issues when they are in practice.^11,12 Pediatric residency, and in particular, the continuity experience, is a unique opportunity for faculty to influence the attitudes and behaviors of future pediatricians. It is also an opportunity for future physicians to apply new skills in identifying and addressing psychosocial issues.

To address these gaps in knowledge, skills and comfort, the SEEK project (A Safe Environment for Every Kid) was developed to assess whether pediatric residents, with special training, could assess and address targeted psychosocial factors that affect children’s health, development and safety in the context of primary care.⁷ The current study evaluates one component of the SEEK project, focusing on the effectiveness of the model on residents’ thinking and behavior, and parents’ perceptions of their child’s pediatrician.

The study had 3 hypotheses. The first was that implementation of the SEEK model would improve pediatric residents’ attitudes, knowledge, comfort, perceived competence and practice regarding targeted psychosocial risk factors (parental depression, substance abuse, IPV, stress, corporal punishment and food insecurity) The second hypothesis was that intervention residents would screen parents more often, and would be more likely to identify and further assess parental problems, compared to control residents. Third, parents bringing children to see the intervention residents would report improved satisfaction compared to families receiving standard care from control residents.

Methods

Setting

The study took place in the primary care continuity clinics of a medium-sized, inner-city pediatric practice, serving children 0–18 years of age. The clinic had approximately 9,000 registered children, with 14,000 visits per year. Eighty percent of the families received Medicaid insurance.

Sample

The pediatric residency program had about 60 residents. The study sample consisted of the categorical pediatric and combined medicine-pediatrics residents who provided care in the continuity clinic. These residents were assigned to a specific clinic day, and attended a clinic session once per week throughout their training, providing primary pediatric care to a panel of patients. All residents were invited to participate, and all agreed. In the first year of the study, all residents, regardless of their year of training, were recruited and participated. In subsequent years, interns were recruited, while previously trained residents remained in the study.

Procedures

The university’s Institutional Review Board approved the study protocol; informed consent was obtained from all participants. Continuity clinic days were randomized to either intervention or control by coin toss. Residents were included in either the intervention or control group on the basis of their assigned clinic day. Residents’ assignments are made in such a way that the number of residents from each training year is balanced across clinics. Attending physicians monitored all resident decisions. The study began in the summer of 2002. In the first wave, all 52 residents, regardless of training level, were recruited into the study. In subsequent years, new interns, as well as upper level residents who transferred into the residency program, were included. In the 3 subsequent waves, 12, 13, and 18 residents were included in the study, respectively.

Parents who brought their child (0–5 years) for a child health supervision (CHS) visit with a resident were recruited to evaluate the SEEK model. Inclusion criteria were English-speaking parents who did not have another child in the study. Exclusion criteria were children in foster care. Parents who agreed to participate in the evaluation were given an appointment to complete a computerized study protocol in a nearby study office within 2 weeks. The evaluation included a measure of parents’ satisfaction with their child’s physician. The evaluation was repeated 6 months later. At the time of the evaluations, the residents had been in the study for variable time periods. Therefore the first evaluation was not a true baseline, but represents an early outcome. The flow chart describing recruitment of families has been published.⁷

The SEEK Model

Parent Screening Questionnaire

The Parent Screening Questionnaire (PSQ) is a 20-item yes/no screen for the 6 targeted psychosocial risk factors noted above. It has adequate stability and validity.^13–16 The PSQ was given to all parents bringing their child for a CHS visit to an intervention clinic. PSQ responses were reviewed by the resident during the visit. Completing the PSQ was optional.

Training

Intervention residents received 8 hours of training in small group discussion sessions, conducted by an interdisciplinary faculty including pediatricians, a social worker and a psychologist. Clinic preceptors were invited, but attendance was variable. Training focused on 6 psychosocial risk factors: parental depression, parental substance abuse, IPV, stress, corporal punishment, and food insecurity. Residents learned about how these issues affect children’s health, development and safety. The sessions focused on how to briefly assess a positive screen, and how to initially address identified problems. The training was repeated each year for new residents. One-hour booster sessions were held every 6 months for the intervention group. These were interactive and involved discussions of illustrative cases as well as role plays. Control residents received no special training.

Resident and Parent Handouts

Each resident received reference materials, including key information, on a laminated card, and/or electronically for a personal digital assistant (PDA). Residents had binders with parent handouts for each targeted problem. The handouts contained advice and phone numbers for community resources. They were user friendly, at a fourth-grade reading level.

Social Worker

A project social worker, involved in the training, was available to assist intervention residents and families. She developed a directory of community resources. Control residents had access to the clinic human services worker, who functioned similarly to the social worker, but did not have access to the SEEK training or materials.

Outcome Measures

Physician Questionnaire

The Physician Questionnaire (PQ) was developed to evaluate the effect of SEEK on the residents. Residents in both the intervention and control groups completed the PQ at baseline, and 6 and 18 months later. The questionnaire was mailed to residents who had graduated from the residency program.

The PQ has 5 vignettes. Each vignette is followed by 7 to 14 statements assessing residents’ knowledge, attitudes, comfort level, perceived competence, and practice concerning each of the targeted risk factors. For each statement, residents responded on a 5-point Likert scale (strongly agree to strongly disagree). The PQ was modified after pilot testing in 2 other pediatric residency programs to ensure clarity of vignettes and statements, and variability in responses. An example of a vignette is: You are seeing ZD for his 2-year checkup. His mother, Ms D, says everything is fine. No problems are detected, although you notice she seems more quiet than usual. There is a modest decline in ZD’s weight and age from the 40th to the 25th percentile since the visit 6 months ago.

Items in the PQ were grouped conceptually into 6 psychosocial problem scales (eg, Substance Abuse, Depression) (Table 1). For example, “I’m comfortable talking to parents about adult depression” was included in the Depression scale. Cronbach alpha values¹⁷ were adequate for most scales: Depression (α = .57), IPV (α = .73), Stress (α = .72), and Corporal Punishment (α = .65). Mean scores were computed for each scale. For Substance Abuse and Food Insecurity, individual items were analyzed because the scale α values were low.

Table 1.

Sample Questions From the Pediatrician Questionnaire

Psychosocial Problem Scale	Sample Questions from Scale	Cronbach Alpha
Parental Depression	I know how to assess depression in a parent. Screening is unnecessary; I can usually detect this. A few questions can effectively screen for depression.	.57
IPV	I feel uncomfortable asking about the possibility of domestic violence. I’m not sure what to do if domestic violence turned out to be a problem. I’m concerned that asking about domestic violence might irritate the mother.	.73
Parental Substance Abuse	I‘d feel comfortable asking the mother if she used drugs or alcohol. I don’t usually ask parents about drug or alcohol use.	.53*
Corporal Punishment	Spanking is an acceptable method of disciplining, even if I disapprove. I feel comfortable asking about discipline practices. If a parent believes in physical punishment, there’s little I can do.	.65
Stress/Social Support	I usually ask families about their social support during well child care visits. I feel comfortable asking about sources of financial support. There isn’t enough time in a routine visit for me to ask about the mother’s own problems.	.72
Food Insecurity	If necessary, I know ways to get the family more food. Asking the mother about their food supply feels intrusive. I know how to assess whether the family has enough food.	.44*

^*As a result of low alpha values, items from Parental Substance Abuse and Food Insecurity were analyzed separately

The PQ also included basic demographic information such as gender, year of training, and type of training program (categorical pediatrics vs pediatrics/medicine combined program). Residents were asked to estimate the number of prior hours of formal child maltreatment and IPV training, and the number of cases of child maltreatment and of IPV they had helped manage.

Children’s Medical Chart Review

Medical records of children from recruited families were reviewed toward the end of the study to objectively determine physician practice in addressing the risk factors. A standardized data abstraction tool with a set of decision rules was developed to abstract key data from all CHS visits in each child’s medical record, while minimizing potential bias by the medical student research assistant doing this task. She could not, however, be blinded to study group assignment as the PSQs were only included in the intervention group records. Information gathered from charts included documentation of screening for each psychosocial problem (including, but not limited to, presence of the PSQ in the chart) and documentation of any further assessment and/or action for identified problems. Any notation regarding screening or assessment of the psychosocial problem was accepted as adequate documentation.

Parent Satisfaction With Child’s Physician

Parents recruited for the evaluation of the SEEK model rated their satisfaction with the resident caring for their child. The 19-item Patient-Doctor Interaction Scale (PDIS) measures patient satisfaction; validity and reliability are good.¹⁸ The PDIS was modified for use in a pediatric setting by referencing the child rather than an adult patient. In addition, some language was simplified to a fourth-grade reading level. The 5-point Likert scale was changed to a yes/no response set as a result of limited variability in responses. Individual scores could range from 0 to 18, with higher scores indicating greater satisfaction.

Statistical Analysis

The main independent variable was group assignment to intervention or control. Repeated measure analyses of variance (ANOVA) were used to control for the effect of group, time, and their interaction on the differences between the intervention and control groups on the Physician Questionnaire scores. For the 2 scales with low alphas (Substance Abuse 0.53 and Food Insecurity 0.44) repeated measure ANOVAs were run examining key individual items. For the Medical Chart Review data, chi square tests were used to compare the proportion of children in each group whose families were screened for each of the targeted problems as documented. The mean scores on the PDIS were compared by group by Student’s t test.

Results

The sample consisted of 95 residents (50 intervention and 45 control) (Fig. 1). Three residents did not complete the PQ at baseline, but completed subsequent questionnaires. One resident only completed the pretest. Ninety-one completed questionnaires at 6 months; 84 (88%) did so at 18 months. There were no group differences in baseline characteristics, or in prior experience addressing child maltreatment and IPV (Table 2).

Figure 1.

Chart of resident randomization and retention.

Table 2.

Baseline Characteristics of the Intervention and Control Group Residents^*

Characteristic	Intervention			Control
	Mean (SD) or Median	n	%	Mean (SD) or Median	n	%
N		49			43
Residency program
Pediatrics		35	71		35	81
Pediatrics/medicine		14	29		8	19
Female		36	74		31	72
Age, y	28.7 (3.3)			28.4 (3.5)
CM-IPV prior training (h)*	7.0 (8.2)			5.7 (4.6)
No. of prior CM cases	2			2
No. of prior IPV cases	0			0

CM = child maltreatment; IPV = intimate partner violence.

^*One intervention and 2 control residents had missing baseline data. There were no significant differences between groups.

Demographic characteristics of families in the intervention (N = 308) and control (N = 250) groups were similar. Most respondents were unemployed single mothers, with a mean age of 25 years. Most children were African American, receiving Medicaid. Intervention group children were younger (median 6 vs 8 months, P = .03) and their families had fewer children compared to control subjects (2.2 vs 2.5, P = .04). The demographic characteristics of the families were similar to that of the overall clinic population.⁷

Physician Questionnaire

On the basis of PQ self-report, intervention group residents improved more than control subjects on 3 psychosocial problem scales: Depression, IPV, and Stress (Fig. 2). This improvement was sustained over 18 months (P < .01, P = .03, P = .04, respectively). Regarding Corporal Punishment, however, control residents improved more than intervention residents. For Food Insecurity, 2 of the 3 individual items showed sustained improvement at 18 months among intervention residents compared to control subjects. They were: “If necessary, I know ways to get the family more food” (P = .01) and “I know how to assess whether the family has enough food” (P = .01). Scores on items regarding substance abuse did not differ between the groups.

Figure 2.

Mean Physician Questionnaire scores for intervention and control residents over 3 time points. Scores reflect resident self-assessment on knowledge, attitudes, comfort, competence, and practice regarding screening for each of 4 psychosocial problems.

Medical Chart Review

Before the training, there was no difference in rates of documented screening between the 2 groups, with very little screening taking place (Table 3). For example, before the study, only 16% of intervention group charts and 12% of control group charts included any screening for maternal depression during CHS visits. After the training, and with the use of the PSQ in the intervention clinics, intervention residents were far more likely than control subjects to screen parents for the targeted risk factors. For example, among intervention residents, the rate of depression screening increased to 88%; control subjects remained low at 16% (P < .001). In general, if the screen was positive, residents in both groups documented further assessment and took some action. However, the number of families who were assessed by the residents as having a problem that required further intervention was low, particularly in the control groups. This very low rate of problems precluded statistical comparisons. For example, control residents identified 2 mothers with depressive symptoms in the period before training, and 9 mothers in the period after. Intervention residents identified 2 mothers with depressive symptoms before and 48 mothers after the training. If a problem seemed likely, residents almost always took some action, such as recommending further evaluation by the study social worker or referral to community services.

Table 3.

Percentage of Parents Screened for Targeted Problems at Least Once, Before and During the Study, as Documented in the Children’s Medical Charts^*

Risk Factor	Treatment Group	Preintervention n/Total (% screened)	P Value	During the Intervention n/Total (% screened)	P Value
Maternal Depression	Intervention	11/69 (16%)	.501	260/296 (88%)	<.001
	Control	8/67 (12%)		39/243 (16%)
IPV	Intervention	11/69 (16%)	.045†	255/296 (86%)	<.001
	Control	3/67 (4%)		29/243 (12%)
Parental Substance Abuse	Intervention	14/69 (20%)	.412	258/296 (87%)	<.001
	Control	10/67 (15%)		41/243 (17%)
Corporal Punishment	Intervention	10/69 (14%)	.316	260/296 (88%)	<.001
	Control	6/67 (9%)		53/243 (22%)
Stress/Social Support	Intervention	28/69 (41%)	.749	275/296 (93%)	<.001
	Control	29/67 (43%)		119/243 (49%)
Food Insecurity	Intervention	10/69 (14%)	.077†	260/296 (88%)	<.001
	Control	3/67 (4%)		29/243 (12%)

IPV = intimate partner violence.

^*Chi-square tests were used to examine differences between groups, unless otherwise specified.

^†Fisher’s exact test was used because cell counts were <5.

Modified Patient-Doctor Interaction Scale (PDIS)

Five hundred fifty-eight parents were recruited; 429 parents (238 of 308, 77%, on control days and 191 of 250, 76%, on intervention days) completed the PDIS. The PDIS showed that parents were mostly satisfied with their last interaction with their child’s doctor. However, at the first study evaluation, parents of children seen by intervention doctors were more satisfied with their child’s doctor compared to those seen by control doctors (17.4 vs 16.9; P < .01). Group differences were not found, however, at the 6-month evaluation.

Discussion

This study demonstrates that the SEEK model improved residents’ level of comfort, perceived competence, and practice behavior regarding prevalent psychosocial problems that are risk factors for child maltreatment. Indeed, the improvement in the areas of depression, IPV, stress, and food insecurity remained significant even 18 months after the initial training, despite the fact that long-term retention of educational material is difficult to achieve.¹⁹ In addition to the training, use of the PSQ, the booster sessions, and the clinical application of the SEEK materials may have helped sustain the improvements. It is encouraging that improvements were found in 3 different outcome measures from 3 sources: resident self-report, medical record abstraction, and initial parental report of satisfaction. The SEEK model appears to have been very effective. It offers promise that residents can be equipped to address psychosocial problems that often impair parental and family functioning. However, SEEK training for corporal punishment and substance use may need revision; they did not improve resident skills in addressing these topics.

When problems are overt, or a parent brings a problem to a physician’s attention, most providers will intervene. However, problems are often well masked. Thus, screening for psychosocial risk factors, such as those chosen for this study, is recommended by Bright Futures.³ These concerns are not only potential precursors to child maltreatment, but they also affect family functioning, parenting, and children’s development and health.20 The Parent Screening Questionnaire encourages parents to disclose psychosocial problems, and prompts physicians to address positive responses.

The SEEK model is innovative in that it extends the focus of child maltreatment training for pediatricians from identification and reporting toward prevention.^21,22 Until recently, most resident training programs paid inadequate attention to addressing the family and community risk factors that jeopardize children’s health and development, and that may lead to child maltreatment.^23,24 Although some programs have focused on family risk factors, often the focus is narrowed to just one psychosocial problem.^24,25 In contrast, the PSQ allowed residents to screen for several key risk factors. This is valuable given the ever-increasing number of topics that are considered to be the responsibility of the primary care provider, despite time constraints.^8,26

Emphasizing their role in prevention is important for pediatric trainees.^4,27 Residents are especially open to new ideas and skills during this formative phase, whereas physicians already in practice may be less likely to embrace new approaches.²⁸ At least one other study by Garg and colleagues has shown that training residents to address family psychosocial concerns can be effective.29 Training during residency should provide residents with valuable skills that they can bring to their practice of pediatrics, and should enhance the functioning of families and the care of children.

We hypothesized that satisfaction with health care would improve when physicians attended to the needs of not just the child, but also those of parents and the family.^30–32 Although the level of satisfaction between the parent groups was initially statistically significant, the difference in scores was small. Importantly, almost all parents rated their doctor highly, a common finding in such research.³³ We had some concern that parents might find personal questions too intrusive in the context of a pediatric visit. The small difference in parental satisfaction between groups suggests that there was no such negative effect. Perhaps, parents viewed the approach as an expression of care and concern, rather than as an intrusion. Anecdotally, we are unaware of parental complaints about the SEEK model, perhaps because they had the option of refusing to answer questions on the PSQ.

Limitations

When conducting research in a clinical teaching facility, ensuring internal validity can be challenging. Control residents may have been aware of the intervention in the other clinic sessions, despite our request that intervention residents not share study information. On rare occasions, residents were moved to other clinic days, potentially sharing information. Make-up booster sessions for absent residents were not offered, leading to variation in training “dose.” Residents may have participated in SEEK for a variable duration at the time the parents completed the first protocol. Preceptors for the intervention groups experienced variable exposure to the workshops. All of these problems would have biased our results toward finding no significant differences between groups. Another limitation is that resident self-reports on their practice may be inflated. However, the chart review provided a more objective assessment and confirms that change in screening practice was real. A caveat is that control residents may have screened parents appropriately, but may not have documented those conversations.

We did not have data on the length of the relationship between the family and their child’s pediatrician, a factor that could affect their satisfaction. Therefore, we were unable to account for this as a possible confounder. However, the randomization process should have minimized differences in the length of physician-patient relationship between the 2 groups. Generalizability of the findings may not apply to lower risk communities, private practice, or other settings.

We did not detect any differences in the rate of follow-up for identified problems between intervention and control groups. Although differences in screening rates could have been solely the result of use of the PSQ, differences in the rates of further addressing psychosocial problems would have provided stronger support for the need for both screening and resident training. However, the residents’ changes in perceived competence and comfort in addressing psychosocial problems suggest that the program’s effectiveness is not the result of the PSQ alone.

Conclusions

The American Academy of Pediatrics Task Force on the Family reached 2 important conclusions.²⁰ First, the trajectory of children’s physical and emotional health, including cognitive and social functioning, is strongly affected by the environment in which they grow up, including family functioning. Second, pediatricians should screen for problems that may impact children’s future well-being, and support and nurture families. The SEEK model appears to meet this goal while acknowledging the limitations of practice, especially time constraints. In the SEEK model residents gained confidence and became more comfortable with these sensitive topics and altered their practice, with sustained improvement up to 18 months. Results may last beyond the residency training years. The SEEK model appears to be a promising approach for addressing major psychosocial problems facing many families. It should be replicated in other pediatric settings. If effective, it could have far-reaching benefits for many children and families.

WHAT’S NEW.

The SEEK model incorporates psychosocial risk factor screening into practice. Residents report sustained improvement in their ability to screen and identify problems. Families are more likely to be screened. This model shows promise to meet the need for training in this area.