Evaluation of a Women's Safe Shelter Experience to Teach Internal Medicine Residents About Intimate Partner Violence

Abstract

Although intimate partner violence (IPV) remains a major public health problem, physicians often fail to screen female patients. Reported IPV training approaches suffer from weak study designs and limited outcome assessments. We hypothesized that an educational experience for residents at a women's safe shelter would have significantly greater impact on IPV competencies, screening, and care for victims than a workshop seminar alone. In a pre-post randomized controlled trial, we compared residents exposed to the workshop seminar alone (controls) to residents exposed to these methods plus an experience at a women's safe shelter (cases). Competencies were assessed by written questionnaire and included knowledge, skills, attitudes, resource awareness, and screening behaviors. Of the 36 residents in the trial, 22 (61%) completed both pre- and postquestionnaires. Compared to controls, cases showed significantly greater pre-post improvement in the knowledge composite subscale. There were no significant differences between cases and controls in the subscales of skills, attitudes, or resource awareness. Cases increased their self-reported screening frequency but this did not differ significantly from the controls. Enhancing traditional IPV curriculum with a women's safe shelter educational experience may result in small improvements in residents' knowledge about IPV.

Intimate partner violence (IPV) remains a major public health problem, resulting in high rates of morbidity and mortality for women.^1–7 Health care providers and trainees infrequently ask patients about IPV.^8–15 Reasons given for not screening include perceptions that female patients do not want to be asked, a belief that asking will have no impact if the woman will not leave, a lack of training/understanding of what to do once the information is received, lack of confidence about how to ask, fear of offending, and legal implications.^16–27

Many different curricula have been developed to train health care providers in screening and intervention for IPV.^28–36 There are minimal outcome data on efficacy of screening and interventions for IPV, however, to guide primary care physicians.^37,38 Multiple recent reviews of this literature have revealed major deficiencies, including weak study designs and minimal or no efficacy evaluations.^38–45 The U.S. National Research Council and the Institute of Medicine have recently called for more rigorous evaluation of training and education of health care providers and trainees about recognition and treatment of women who have experienced IPV.^38,39,46,47

Residency programs commonly offer lectures, seminars, videotapes, or role plays about IPV. We developed a curriculum that supplements these instructional strategies with an experience at a women's safe shelter, hypothesizing that directly interacting with victims would have a significant impact on residents, thereby improving IPV competencies, screening, and care for victims. In a randomized controlled trial, we evaluated the impact of the shelter experience on residents' knowledge, self-reported skills, attitudes, resource awareness, and screening behaviors concerning IPV. We isolated this effect by providing control residents with the identical curriculum with the exception of the shelter experience.

METHODS

Subjects and Setting

We studied residents in a university-based primary care Internal Medicine residency program. In the study year, this program had 69 residents (61 categorical, 8 preliminary), of whom 10% were Internal Medicine graduates, 42% minorities, and 43% female. The residents within the program rotate between 2 community hospitals and 1 large tertiary care university hospital.

Residents were eligible for the study if assigned to the outpatient ambulatory block rotation during academic year 2001–2002. Eligible residents were randomized using a random numbers table to either the workshop seminar alone (controls) or the workshop seminar plus shelter experience (cases).

Educational Interventions

Workshop Seminar (All Participants)

The workshop seminar included a didactic session presenting background information about IPV, including definition, prevalence, risk factors, and specific instruction on “how to” screen, diagnose, and intervene. The seminar also included presentation of a video (see description below), a breakout session discussing the video, and finally, a role play. The sessions lasted approximately 90 min, with approximately 10–15 residents in attendance per session. The seminar was presented and facilitated by a faculty member with specific expertise and interest in IPV.

The video, “Voices of Survivors,” described in more detail in a previously published paper,⁴⁸ was shown to all residents. This video was selected because it was developed using analysis of domestic violence patient experiences and patients' descriptions of their experiences focusing on what they wanted physicians to understand and do about IPV. The video consists of still photographs with a soundtrack of interview excerpts grouped by themes that emerged from previous qualitative research. The documentary aims to improve physicians' awareness and empathy about IPV, to increase detection, and to reinforce responses that survivors find helpful. A companion guide is available that supplements each identified theme with traditional aspects of IPV training, such as statistics, screening questions, clinical indicators, and documentation guidelines. The video has a running time of approximately 30 minutes.

After viewing the video, a small group discussion was facilitated by the faculty presenter, reinforcing themes discussed by the women in the video, including perceived and real barriers for entering or being identified by the health care system as a victim of IPV. In addition, a list of local resources was distributed to each resident and also posted within the continuity clinic settings.

The final experience for the group was a role play including 3 participants (a patient reluctant to discuss violence in her relationship, her overprotective partner, and the physician). The goal of the role play was to expose the residents to the experience of identifying and applying the appropriate wording to screen a patient for IPV as well as how to handle the paramount concerns of privacy and confidentiality within the interaction.

Women's Safe Shelter Experience (Case Group Only)

Case residents had 1 visit to a local women's safe shelter in addition to attending the workshop seminar. During this visit, they attended a weekly evening meeting of IPV survivors, which lasted approximately 2–3 hours. The residents were paired with a “buddy” resident of the opposite gender to increase comfort for both the residents and the women at the shelter. The study investigators attended several weekly meetings to assess the setting and content of the meetings as well as to establish a rapport and level of trust with the shelter's administration prior to sending the residents. We addressed a concern about complete confidentiality of both the content and the location of the meetings by personally explaining the importance of this to each resident and having each resident sign a confidentiality agreement. We also emphasized to the residents and the shelter staff that the objective of the experience was primarily to act as a listener and not to assume the role of providing non–IPV-related medical advice.

The focus of the evening meetings was to introduce new or recent arrivals to the group by having them “tell their stories.” Experienced shelter staff counselors facilitated this process. We specifically asked the staff to facilitate discussion emphasizing women's experiences with their own doctors as well as experiences with the health care system in general, with respect to IPV. Finally, as a means of conveying to the residents the full impact of IPV on a woman's life and health, we asked the staff to facilitate the women's stories, focusing on the impact of being a survivor of IPV on their life, work, health, children, and home.

To express appreciation for allowing our residents to participate in the group sessions, the case residents and faculty provided separate evening sessions where a variety of health care issues important to women were presented. This experience also contributed to the residency programs' training in community health education. These sessions were held after completion of the posttest, so were not part of the intervention.

Outcome Assessment

We administered a written survey to all residents immediately before and again 6–12 months after the intervention. The 43-item survey included 10 original questions and 33 adapted from 3 previously reported instruments.^49–52 The majority of survey items were adapted from the Health Care Provider Survey for Domestic Violence,⁵¹ a validated, Likert-scale provider survey. All 3 of these surveys demonstrated content validity and responsiveness to IPV educational interventions^49–51; 2 showed high test-retest reliability^49,51; and 1 showed adequate internal consistency.⁵¹ To confirm our survey's content validity, we circulated it to experts in IPV, who agreed that it included the most important constructs of adult IPV.

The survey included 33 statements with Likert scale responses (with anchors 1=strongly disagree, 5=strongly agree) to assess residents' knowledge (7 statements), self-reported skills (4 statements), attitudes (18 statements), and local resource awareness (4 statements) about IPV. Sample statements for each subscale of the instrument are presented in Table 1. To minimize the influence of “social desirability” in resident responses, we included statements with desirable, ideal, or factually correct answers in both extremes of the Likert scale. The “desirable” response for the knowledge, skills, and resource awareness domains was 5 or strongly agree. For the attitude statements, the “desirable” response was 1 or strongly disagree.

Table 1.

Sample Items for Subscales of Instrument

Subscales	Sample Items
Knowledge	“Women are at greatest risk of injury when they leave a relationship.”
	“Women who are pregnant are at increased risk for IPV.”
Attitudes	“The victim has often done something to bring about violence in the relationship.”
	“It is demeaning to patients to question them about abuse.”
Skills	“If I find a patient who is a victim I don't know what to do.”
	“I feel confident that I know how to ask about IPV.”
Resource awareness	“I am easily able to refer my patients to local safe shelters.”
	“I have ready access to information and resources regarding IPV in my clinic.”

IPV, intimate partner violence.

In addition, residents reported their screening behaviors for all women and those with “red flag” presentations (headache, pelvic pain, depression, anxiety, and gastrointestinal symptoms) on a 5-point Likert scale with anchors of never and always. We also included statements about medical record documentation of IPV for potential future use of this method to assess behavior change. Finally, the survey included questions about residents' demographic characteristics, prior IPV training, and personal experience with victims of IPV.

The surveys were distributed and collected by a nonstudy staff member to ensure anonymity. Unique ID numbers were assigned to all participants to log and keep track of the pre- and postsurveys.

Analysis

Standard frequencies and means were calculated for individual variables. Because of the small sample size and lack of a normal distribution, we used nonparametric tests for comparisons. For all Likert scale statements, we compared the mean pre-post changes between the control and case groups using the Mann-Whitney statistic by individual statement. Because the analysis of this many individual statements may be subject to multiple comparison error, we calculated composite mean subscores for resident responses for each of the 4 domains (knowledge, skills, attitudes, resource awareness). Statements included within each domain are presented in Table 1 and Appendix I (available online at: http://www.blackwellpublishing.com/products/journals/suppmat/jgi/jgi40207/jgi40207sm.htm). We then compared the pre- versus postsubscores for the cases and controls separately using the Wilcoxon signed rank test for related samples. Finally, we compared the pre- to postchanges for the cases versus the controls using the Mann-Whitney statistic for unrelated samples. Self-reported screening, documentation, and personal experience with IPV items were analyzed using the χ² statistic. An intention to treat analysis was utilized as 1 individual randomized to the case group was unable to attend the shelter experience.

Human Subjects

The study protocol was approved by the Institutional Review Boards of the participating institutions. Informed consent was obtained from the residents in the trial as well as assurance of confidentiality maintenance about the shelter experience, including the location of the shelter. Authors had full access to all of the data in the study, and accepted full responsibility for the integrity of the data and the accuracy of the data analysis.

RESULTS

Thirty-six residents participated in this study; 16 (44%) were male and 20 (56%) were female. Twenty-two (61%) completed both the pre- and postsurveys and were included in this analysis. The mean duration between administration of the pre- and postsurveys was 7.5 months (range 6–12 months).

Table 2 presents demographics of responders compared to nonresponders. Responders did not differ from nonresponders in terms of age, postgraduate year, ethnicity, or gender distribution. Overall, the demographics of the participants in the study (approximately half of the total number of residents within the program) were not significantly different from the distribution within our residency program (data not shown).

Table 2.

Demographics of Responders and Nonresponders

	Responders N=22	Nonresponders N=14	P Value
Age (mean + SD)	31.1±2.8	30.5±2.7	N/S
PGY, N (%)
I	6 (27)	4 (29)	N/S
II	9 (41)	6 (43)	N/S
III	7 (32)	4 (29)	N/S
Gender, N (%)
Female	12 (55)	7 (50)	N/S
Male	10 (45)	7 (50)	N/S

N/S=P≥.05.

PGY, postgraduate year; N/S, not significant.

Table 3 presents the pre- and postcomposite subscores and the pre-post change for cases versus controls for the 4 subscales within the instrument (knowledge, skills, attitudes, resource awareness). Of note, while all residents improved in terms of knowledge, compared to controls, case residents showed a significantly greater pre-post improvement in the knowledge subscale. There were no significant differences between cases and controls for pre-post change in the skill, attitude, or resource awareness composite subscales.

Table 3.

Composite Subscores

Domain	Group	Composite Subscores			P Value*
		Pretest	Posttest	Change
Knowledge	Cases	3.8	4.5	0.69	.04
	Controls	3.8	4.2	0.40
Skills	Cases	3.2	4.0	0.78	.3
	Controls	3.4	4.0	0.61
Attitudes	Cases	1.9	2.0	0.67	.9
	Controls	1.7	1.8	0.67
Resource awareness	Cases	3.1	3.7	0.60	.8
	Controls	3.1	3.7	0.42

^*P value for comparison of pre-to-post change in cases versus controls.

Before the educational intervention, 65% of the study residents had ever identified a patient who was a victim of IPV and 10% had ever identified a patient who was a perpetrator of IPV. This proportion increased post intervention to 85% and 35% (victim and perpetrator, respectively) and was not significantly different for cases and controls (data not shown).

With respect to the residents' self-reported frequency of screening for IPV in women with “red flag” presentations (e.g., headache, pelvic pain, chronic abdominal pain), cases showed an improved pre to post increase in screening compared to controls, but this trend did not achieve statistical significance.

Regarding self-reported medical record documentation of IPV, we found that at baseline, 70% of controls and 80% of cases always or sometimes documented, with no residents in either group stating they never documented. After the intervention, 100% of both the controls and cases stated that they always or sometimes documented. This change was not significantly different for cases and controls. We also included an open-ended section asking residents why they did or did not document discussion about IPV in the medical record. Examples of responses included: “I document if positive but rarely if negative”; “I have had patients who were victims, now I always document”; “Sometimes the patient will ask me not to document”; “I don't feel comfortable documenting because it might not be related to the patients' presenting problem.”

Finally, the residents who were randomized to the shelter rated the experience very positively. In fact, many of the residents who were not included in this group later requested this experience as an elective.

DISCUSSION

Overall, the results from this randomized controlled trial reveal that the addition of an educational experience at a women's safe shelter may have a modest impact on Internal Medicine residents' knowledge about IPV compared to an instructional approach that included a workshop seminar (didactic, video, role play). In contrast, the enhanced curriculum had no additional effect on residents' self-reported skills, attitudes, or resource awareness in composite subscale analysis. Individual statement analysis revealed small significant differences between cases and controls for a minority of statements within the knowledge, skills, and resource awareness domains. There were also no differences in self-reported screening behaviors between cases and controls.

Reasons for failure to identify other significant differences in this study may be due to the high preintervention responses for both cases and controls, creating a ceiling effect, which may have prevented the detection of more substantial changes. Furthermore, controls received a substantial educational intervention, which might further eclipse differences between groups. Because we identified a significant pre-post change only within the knowledge subscale, the educational significance is unclear given that multiple areas of competency are necessary for effective and compassionate care of IPV survivors.

The lack of additional impact of the shelter experience on residents' attitudes and screening behaviors is disappointing but not unexpected, because these areas are more difficult to influence. Although failure to detect change in the attitude domain is counterintuitive given our hypothesis about the impact of the shelter intervention, this may be secondary to less reliability in measurement and/or less sensitivity in detecting change within this construct.⁵¹ In addition, one could speculate that the shelter experience motivated residents to become more aware of objective issues (knowledge, resource awareness) but did not impact long-standing attitudes. Finally, the shelter experience may not add much to the video in terms of influence on attitudes, as both feature real victims “telling their stories.” We included the videotape because of its similar content to the shelter experience (hearing real women's stories), the fact that the women in the tape focused on what they wanted to teach physicians, and its significantly greater ease of administration than development and implementation of a shelter experience for the residents. Perhaps our largely negative results suggest that hearing real women's stories is the “active” ingredient irrespective of educational format chosen (e.g., video vs shelter).

Our study contributes to the existing literature in IPV education by virtue of the innovative educational intervention and rigorous evaluation via a randomized controlled trial. Although limited, some previous studies have also demonstrated improvements in IPV knowledge, attitudes, and skills of physician trainees after an educational intervention. In a study focusing on third-year medical students, Jonassen et al.⁴⁰ demonstrated sustainable improvements in knowledge, attitudes, and skills after a 2- and 3.5-day curricular intervention that included multiple educational formats (e.g., lectures, films, role play, etc.). After a half-day workshop (lecture, video, panel discussion, role play), Kripke et al.²⁹ were able to show an improvement in knowledge, skills, and attitudes of internal medicine residents, but no change in rates of documentation of IPV detection up to 6 months after the intervention.

Our study is limited by the small sample size and concomitant limited power to detect differences. Nonetheless, we were still able to detect significant pre-post change within the domain of knowledge. Study participants were from only 1 residency program and a higher proportion of women were enrolled in this study than within the total residency program, potentially limiting generalizability. Another limitation includes the use of self-reported outcomes (for skills, attitudes, resource awareness, and screening behaviors), which are subject to recall bias and social desirability. As the average follow-up survey was administered approximately 8 months after the intervention, it is difficult to speculate whether the change we observed would be maintained without further educational reinforcement. In addition, our survey instrument was adapted from previously reported instruments, which could potentially affect measurement validity and reliability. Finally, our failure to detect a change in the items within the attitude domain may be related to lack of sensitivity to change in some of these items, in combination with a ceiling effect on the preintervention scores.

In conclusion, compared to a workshop seminar alone, the addition of an educational experience at a women's safe shelter may have a small incremental impact on residents' knowledge about IPV. It is unclear from this study whether the identified small statistical differences have real educational significance. Future research, including more objective outcome measures (direct observation of screening behaviors), is necessary to validate these results. This research should address what aspect of educational interventions for IPV, including incorporating women's personal stories, impacts most significantly on residents' screening behaviors in the clinic setting.