Effectiveness of an Evidence-Based Quality Improvement Approach to Cultural Competence Training: The Veterans Affairs’ Caring for Women Veterans Program

Abstract

Introduction:

Although providing culturally sensitive healthcare is vitally important, there is little consensus regarding the most effective strategy for implementing cultural competence trainings in the healthcare setting. Evidence-based quality improvement (EBQI), which involves adapting evidence-based practices to meet local needs, may improve uptake and effectiveness of a variety of healthcare innovations. Yet, to our knowledge EBQI has not yet been applied to cultural competence training. To evaluate whether EBQI could enhance the impact of an evidence-based training intended to improve Veterans Affairs (VA) healthcare staff gender sensitivity and knowledge (Caring for Women Veterans; CWV), we compared the reach and effectiveness of EBQI delivery versus standard web-based implementation strategies of CWV and assessed barriers and facilitators to EBQI implementation.

Method:

Workgroups at four diverse VA healthcare sites were randomized to either an EBQI or standard web-based implementation condition (SI). All EBQI sites selected a group-based implementation strategy. Employees (N=84) completed pre- and post-training assessments of gender sensitivity and knowledge, and focus groups/interviews were conducted with leadership and staff before and after implementation.

Results:

Reach of CWV was greater in the EBQI condition versus the SI condition. Whereas both gender sensitivity and knowledge improved in the EBQI condition, only gender sensitivity improved in the SI condition. Qualitative analyses revealed that the EBQI approach was well-received, although a number of barriers were identified.

Conclusions:

Findings suggest that EBQI can enhance the uptake and effectiveness of employee trainings. However, the decision to pursue EBQI must be informed by a consideration of available resources.

The importance of providing culturally sensitive healthcare to patients cannot be overstated, as a lack of cultural sensitivity may contribute to health disparities for vulnerable and underserved populations¹. Culturally competent healthcare involves providing dynamic and continuous care informed by knowledge of the heritage, attitudes, and behaviors of the patients receiving care^2,3. The widespread recognition of the importance of providing culturally competent healthcare has been accompanied by the development of trainings intended to enhance healthcare professional’s cultural competence⁴. However, there is little consensus regarding how best to effectively implement these trainings¹.

Recently, evidence-based quality improvement (EBQI) has been proposed as an approach that can improve the uptake of evidence-based programs and practices into routine practice⁵. EBQI uses bottom-up engagement of local site personnel to adapt the evidence-base to the local context through consensus development, group decision-making among researchers, key stakeholders, and local-level providers and staff^6,7. A key tenet of EBQI is that local hospital leadership and clinical staff are best equipped to determine how to improve outcomes within their healthcare system^6,8. Thus, EBQI engages managers and staff in adapting evidence to local contexts, embedding an efficacious training in a large organizational change initiative with multi-level involvement and support. EBQI also has the potential to enhance the effectiveness of evidence-based innovations by tailoring them to the specific needs and preferences of local site staff. Indeed, experts in implementation research suggest that all other things being equal, quality improvement approaches that adapt evidence-based innovations to align more closely to local realities and preferences should be also more effective than efforts to enhance fidelity to established evidence-based protocols⁹. The majority of EBQI efforts to date have focused on evidence-based clinical practices, such as care delivery of depression treatment⁵ and implementation of patient aligned care teams¹⁰. Given the success of these clinically-oriented EBQI-efforts, EBQI may also be a fruitful strategy for improving uptake and effectiveness of evidence-based practices that focus on cultural competence trainings.

Although most of the literature on cultural competence in the health care setting has focused on racial/ethnic disparities^11–13, a key aspect of cultural competence in the Department of Veterans Affairs (VA) healthcare system relates to the care of female veterans. Not only are women a minority patient population within the VA, but evidence indicates that female veterans have unique health-care needs that have historically received less attention in the VA healthcare system (e.g., reproductive health services^14,15). Despite substantial efforts to improve the care of female veterans over the last several decades, there is continued evidence of disparities in outcomes for female and male patients^16,17 and research suggests that some VA healthcare staff are less accustomed to caring for this unique patient population^18,19.

Based on findings from a large multi-site study which identified a number of areas for improvement ins VA employee gender sensitivity and gender knowledge,¹⁸ Vogt and colleagues^20,21 developed a web-based training to target these identified deficits. Clinical trial findings suggest that this training, referred to as Caring for Women Veterans (CWV; available to VA employees at http://www.tms.va.govcourse#15876; See Supplemental Material) is an efficacious tool for enhancing VA employee gender sensitivity and knowledge¹⁹.

To date, implementation of CWV has been primarily limited to online delivery. Although web-based approaches to trainings have benefits, including ease of use, flexibility, and broad-based accessibility, empirical evidence for their effectiveness is limited²², and employees often report that they view these trainings as a waste of time or irrelevant to their jobs²³. Moreover, when these trainings are optional rather than mandatory they may have limited reach. As of March 2016, the CWV training has been completed by approximately 10% of VHA employees (D. Vogt, personal communication March 4^th, 2016), suggesting the need for enhanced implementation of this evidence-based training.

Therefore, the primary aim of the current study was to compare the reach and effectiveness of EBQI delivery of CWV compared to standard web-based delivery. Based on evidence for EBQI implementation of other healthcare innovations ^5,10, as well as research documenting the critical role that leadership support can play in the success of health professions educational efforts^24–27, we hypothesized that EBQI delivery of CWV would have greater reach and effectiveness compared to standard web-based delivery. A secondary aim was to identify barriers to and facilitators of EBQI delivery of CWV from the perspective of site leaders and staff.

Method

Study Design and Setting

The design for this multi-method study involved a group-level randomized trial and took place at four geographically diverse VA facilities in the VA Women’s Health Practice-Based Research Network (WH-PBRN)²⁸. Based on local priorities, expert panels at each site selected eight workgroups (i.e., selected specialty and primary care clinics) eligible for the study. Workgroups at each site were randomized to an EBQI implementation condition or a standard web-based implementation (SI) condition. The research team randomized eight workgroups at each site: three workgroups to SI and five workgroups to EBQI. Key stakeholders at all four sites also elected to target their emergency departments (EDs) for the EBQI condition. In total, 673 employees were invited to participate in the EBQI implementation of CWV and 320 employees were invited to complete the standard web-based implementation of CWV.

Reach was evaluated based on the number of participants in the study who reported completing CWV in each implementation condition. Effectiveness was evaluated by comparing the gender sensitivity and knowledge of employees before (T1) and after (T2) implementation. Additionally, we conducted pre- and post-implementation interviews and focus groups to identify barriers to and facilitators of EBQI delivery of CWV. The study was approved by the VA Central Institutional Review Board. Figure 1 graphically presents the overall study timeline and procedures.

Figure 1.

Graphical Representation Timeline and Study Procedures for the Caring for Women Veterans Implementation Study

Notes. Shaded areas indicate non-research activities; Non-shaded areas are research activities. T1= pre-implementation; T2 = post-implementation; EBQI = evidence-based quality improvement condition; SI = “Standard Implementation” condition.

Training Implementation

CWV program.

CWV is a 30-minute evidence-based training designed to improve gender awareness among VA healthcare workers. It includes five modules that address beliefs about women veterans’ use of medical care, knowledge regarding women’s use of VA healthcare services, sensitivity to women’s privacy and safety concerns, knowledge about the history of women in the military and at VA, and sensitivity to the impact of care-giving responsibilities on women’s healthcare use^20,21. This web-based training uses a multimedia presentation style and includes video scenarios and interactive question-and-answer segments.

Standard Implementation (SI).

The SI implementation was modeled after the standard approach within VHA to rolling out similar trainings. A VA leader sent an informational email about the training opportunity that contained a link to the training. Employee trainings then took place over the course of approximately one month. At Site 2 only, SI employees who completed the training received continuing education credits (CEUs).

EBQI Implementation.

A key component of EBQI is actively engaging the local context in adapting the implementation strategy. Thus, following recommendations for EBQI⁵, expert panel discussions were conducted with site leaders and key stakeholders (e.g., facility and women’s health leadership) to tailor the CWV delivery to local site needs and preferences. Table 1 presents a description of how key stakeholders at each site decided to adapt the CWV training to their local context. Sites 1, 2, and 3 decided to modify the online CWV training to a facilitated group format, where a facilitator with expertise in women’s health led workgroups through the training. Site 4 chose a training model that involved a workgroup “primer” in-person orientation that focused on optimizing and improving health care for women Veterans. Primer participants received a brief description of the training, along with a handout that summarized its key points, and were then encouraged to complete the training individually online.

Table 1.

Site Specific EBQI Adaptations to Caring for Women Veterans Training

	SITE 1	SITE 2	SITE 3	SITE 4
How will training be completed?	Facilitated groups	Facilitated groups	Facilitated groups	Individually, after attending a group introductory “primer” session
Who will facilitate the training?	Women Veterans Program Manager	Women Veterans Program Manager	Women Veterans Program Manager	Women Veterans Program Manager
Who will serve as a co-facilitator?	Member of the workgroup being trained	Member of the workgroup being trained	Local Site Research Principal Investigator	None
Will leadership endorse the training?	Yes, via email from facility leadership to clinic leaders	Yes, via public service announcement video	Yes; Director and Chief of Staff attended and introduced every training; Support to supervisors to allow attendance.	Yes, via email to clinic leaders with mention of Chief of Staff (COS) support (with COS copied on email)
Will the training be mandatory?	Expected, not mandatory	Optional, encouraged	Optional, encouraged	Optional, encouraged
How will employees be informed of the training?	Clinic leadership	Clinic leadership	Email from hospital director	Flyer/email
Employee recognition for completing training?	None	Continuing education credits	Signed completion certificate from hospital director	Signed completion certificate from hospital director
Other components to include?	• 2-minute video about women veterans health care shown at the beginning of each training session • Women’s health posters and flyers to increase awareness of women veterans • Authorized release time to attend training	Workgroup action plans	Authorized release time to attend training	Women’s health brochures

Each site’s EBQI plan supplemented CWV with additional components, which are described in Table 1. In accordance with principles of EBQI⁵, the local research team provided support to the local clinical implementation teams throughout the process (e.g., orienting facilitators to the training material, problem-solving logistical issues).

Study Procedures

Survey component.

Before implementation (T1), all employees in randomized workgroups received a personalized e-mail invitation to participate in the survey that contained a unique identification code and the link to research information sheet. Participants who agreed to the terms described in the research information sheet were directed to the survey. Those who declined were directed out of the survey and not contacted again. Those who agreed to participate received reminder emails at two weeks and four days before the survey closed (30 days later).

Among those invited, 156 individuals who received the invitation completed the T1 survey (N_EBQI = 109, N_SI = 46). Table 2 contains demographics of the T1 survey sample by condition.

Table 2.

Demographic Characteristics of the Survey Sample at Time 1

		EBQI		SI
Age		N	%	N	%
	20–30 years	5	4.6	2	4.3
	31–40 years	22	20.2	4	8.7
	41–50 years	33	30.3	14	30.4
	51–60 years	36	33.0	16	34.8
	Greater than 60 years	13	11.9	10	21.7
Gender
	Female	81	74.3	31	67.4
	Male	24	22.0	15	32.6
Race
	Asian	15	13.8	3	6.5
	Hispanic	7	6.4	2	4.3
	Black	9	8.3	5	10.9
	Pacific	0	0.0	1	2.2
	White	70	64.2	34	73.9
	Multi-racial	7	6.4	1	2.2
Education
	Some college	12	11.0	10	21.7
	Vocational or technical training	11	10.1	5	10.9
	Four-year college graduate	17	15.6	6	13.0
	Some graduate or professional school	5	4.6	1	2.2
	Graduate or professional degree	63	57.8	24	52.2
Type of patient contact
	Direct health care	86	78.9	33	71.7
	Ancillary care/healthcare support	22	20.2	13	28.3
Amount of daily patient contact
	0 hours	0	0.0	1	2.2
	1–2 hours	3	2.8	4	8.7
	3–4 hours	15	13.8	4	8.7
	5–6 hours	25	22.9	13	28.3
	7–8 hours	47	43.1	21	45.7
	9+ hours	18	16.5	2	4.3
Frequency of contact with female patients
	Never	0	0.0	2	4.3
	1–2 times a month	23	21.1	9	19.6
	1–2 times a week	28	25.7	11	23.9
	Several times a week	22	20.2	11	23.9
	Almost daily	21	19.3	8	17.4
	Daily	15	13.8	4	8.7
Years of VA employment
	0–2 years	16	14.7	9	19.6
	3–5 years	34	31.2	19	41.3
	6–10 years	25	22.9	9	19.6
	11–15 years	11	10.1	1	2.2
	More than 15 years	23	21.1	8	17.4
Completed training*
	Yes	47	74.6	10	40.0
	No	16	25.4	15	60.0

Note.

^*Numbers reported for “completed training” are from those participants who completed Survey 2; EBQI = Evidence-based quality improvement condition; SI= Standard implementation condition; VA= Department of Veterans Affairs.

T1 survey participants received T2 survey invitations approximately one month after trainings were completed (or 30 days after T1 in the SI condition). The T2 survey procedure and content were identical to the T1 survey, with one additional question about whether participants had completed the CWV training. Eighty-five participants completed the T2 survey (N_EBQI = 61, N_SI = 24).

In addition to sociodemographic questions, the survey included two scales from the GAI-VA²⁹, a validated gender awareness measure developed for use with VA staff. The 29-item gender sensitivity scale assesses sensitivity to the healthcare needs of female patients (e.g., “Women Veterans should have access to care by experts in women’s health.”). Responses are recorded using a 5-point response format (1 = strongly disagree, 5 = strongly agree) and summed for a total score. The gender knowledge scale represents a sum of correct responses to 18 multiple-choice questions addressing knowledge about women Veterans and their use of VA care (e.g., “Of all the patients using VA medical centers, what percentage are women?”).

Leadership interviews.

Up to three key stakeholders or others involved in delivering the EBQI training were invited to participate in T1 and T2 implementation interviews at each site. Most participants who completed T1 interviews were also contacted for T2 interviews. However, alternative interviewees were identified when they could offer a more comprehensive perspective. Seven individuals completed T1 interviews and 10 completed T2 interviews.

Potential interviewees were contacted via email and/or telephone to request participation in a 30-minute semi-structured telephone interview. Potential participants received a research information sheet via email and an interview was scheduled. T1 interviews assessed site characteristics that might impact implementation and expectations for and impressions of the training process. T2 interviews addressed barriers to and facilitators of the training, reactions to the training, and perceived impact of the training on the care of women Veterans at the participant’s facility.

Focus Groups with Staff.

At each site, a subset of staff from the EBQI condition received email invitations to participate in T1 and/or T2 focus groups to discuss their expectations and/or experiences with the training, as well as perceived barriers to and facilitators of implementation. Focus groups ranged in size from 2–12 participants, and lasted approximately one hour. At the beginning of the focus group, participants completed written informed consent and signed confidentiality agreements. Moderators followed a focus group interview guide that included questions addressing the concept of gender sensitivity, the training process, and the content of the CWV training. All focus groups were digitally recorded.

Each site held two T1 and two T2 focus groups. In general, clinicians and staff attended separate focus groups. Because of recruitment and scheduling complexities, one of Site 3’s T2 focus groups included only one person and was excluded from analysis, one of Site 4’s T2 focus groups was composed of a mix of providers and staff, and Site 1 held a third T2 focus group with a mix of clinicians and staff. Across sites, 51 employees participated in the eight T1 focus groups; 33 employees participated in the eight T2 focus groups.

Data Analyses

Quantitative data analyses included manipulation checks (e.g., t-tests, chi-square tests) to ensure that there were no significant differences between conditions at T1. Data from the T1 and T2 surveys were analyzed with 2 (Condition: EBQI v SI) × 2 (Time: T1 v. T2) repeated measures analyses of variance (RMANOVAs). Given limited power to detect significant interactions in this study, we also computed paired-samples t-tests for sensitivity and knowledge for each condition separately. For all mean comparisons we calculated Pearson’s r as a measure of effect size for all mean comparisons, with an r =.10 interpreted as a small effect, r = 0.30 considered a medium effect, and r = 0.50 considered a large effect³⁰.

Qualitative data were analyzed using a rapid analytic approach³¹. Main topics (“domains”) were drawn from the interview and focus group guides and a summary template was developed. Three team members [AF, BD, SWS] used the template to summarize the same three transcripts to ensure that the domains were identifiable in the data and that there was consistency across team members in capturing the domains. Once consistency of summary content was established, transcripts were divided up across the team and summarized using the template. Bullet points from the summary templates were then placed into a matrix to analyze the depth and breadth of information for each domain³². Aggregated preliminary results were presented to site leadership to facilitate cross-checking³³ (i.e., confirming credibility of the preliminary analysis). Subsequent to this initial process, transcripts were analyzed by [AH] using ATLAS.ti, with top-level codes corresponding to the previously identified domains and sub-codes and emergent codes developed after thorough review of main content areas.

Results

Reach and Effectiveness

75% of those in the EBQI implementation condition and 40% of those in the SI implementation condition reported completing the training at T2, indicating that reach was higher in the EBQI condition.

Initial analyses confirmed that there were no significant differences between conditions at T1, or between those who completed both surveys versus only the T1 survey on sociodemographic variables. Results of subsequent repeated measures ANOVAs revealed that overall, scores on gender sensitivity significantly increased from T1 (M = 118.42, SD = 11.01) to T2 (M = 121.29, SD = 10.82; F(1,82) = 8.34, p =.005, r =.30). In addition, SI participants reported slightly higher gender sensitivity (M = 120.44, SD = 11.75) compared to the EBQI condition (M=117.74, SD = 11.27) across time, although this effect was only marginally significant (F(1,82) = 2.99, p =.09, r = .19). Finally, the condition by time interaction was not statistically significant (F(1,82) = .05, p =.83, r =.03), indicating that EBQI did not improve gender sensitivity significantly more than SI.

Just as for gender sensitivity, there was a small increase in gender knowledge from T1 (M = 13.64, SD = 2.55) to T2 (M = 14.14, SD = 2.26; F(1,81) = 2.49, p =.12, r =.17), although this effect did not achieve statistical significance. Gender knowledge also did not differ for participants in EBQI (M = 14.13, SD = 2.29) and SI (M = 13.65, SD = 2.95) conditions across time (F(1,81) = 0.97, p =.33, r =.11). Finally, EBQI did not improve gender knowledge significantly more than SI, as the condition by time interaction was not statistically significant (F(1, 81) = .85, p = .36, r = .10).

Because power was a concern for testing interactions (i.e., the extent to which changes in gender sensitivity and knowledge differed for EBQI and SI conditions), and because graphs of the interactions suggested there may be important differences across conditions (see Figure 2), we also computed paired-samples t-tests for each condition separately. In the EBQI condition, both gender sensitivity and gender knowledge significantly increased from T1 to T2 (sensitivity: t (60) = 2.38, p =.02, r =.20); knowledge: t (58) = 2.84, p =.01, r =.22). For the SI condition, gender sensitivity significantly increased from T1 to T2, (t (22) = 2.39, p = .03, r = .31) but the effect for gender knowledge did not achieve statistical significance (t (23) = .29, p = .78, r =.11).

Figure 2.

Changes in Gender Sensitivity and Knowledge over Time

Identifying Barriers to and Facilitators of EBQI Implementation

Qualitative analyses from interviews and focus groups revealed that there were a variety of logistical challenges at the sites using the group training format. For example, audio-visual equipment sometimes failed, space was not optimal at some sites, and scheduling trainings for busy providers and staff was challenging. A number of barriers to the EBQI approach were identified, including the time and effort required of both research and clinical staff and the complexities inherent in adapting a training initially developed for individual completion to a group setting.

At the same time, the group training format was consistently identified as a strength of the training experience because it provided the opportunity for in-depth discussion of unique concerns for female patients in a predominately male-populated healthcare setting; this was especially valued in mixed gender groups (some of which included women veteran employees). In some cases, the group-based trainings fostered discussion of systems-level changes that could make clinical units more welcoming to women. Furthermore, the group facilitators were consistently perceived to have been effective due to their “insider” status as members of the local workgroup. Some participants suggested having women veterans as facilitators in future trainings.

Leadership support was viewed as a critical facilitator of the training. Each site took its own unique approach to conveying such support. For example, at one site, top facility leaders attended and actively participated in every training. At other sites, having protected time to attend the training affirmed leadership support, and made the training feel “more official.”

Most participants expressed that the training was well-received, and a number of participants reported especially appreciating the facts and figures provided in the training. Overall, the content was perceived to be appropriate and informative, with some suggestions for enhancements related to other gender issues such as transgender health.

When queried about the impact of the training on the healthcare environment, many participants noted that they or other staff were already sensitive to the unique healthcare needs of female patients. Several remarked that their training session was conducted among individuals who were already highly gender-sensitive (e.g., women’s and mental health providers), so the training was “preaching to the choir.” Some indicated that there was insufficient time to observe a change in organizational culture. However, even those who did not observe a major impact said that the training was a good “refresher” on women Veterans’ health and gender-sensitive care, and some individuals described ways the training had affected or would affect their care of patients. For example, reflecting on a recent contact with an actual patient, this provider felt that she would have managed the patient differently had she gone through the training prior to seeing the patient:

“…I actually did have an encounter with a female veteran patient before this training. And I didn’t realize that a lot of them deal with sexual trauma in the past. And so that patient kind of surprised me, because she was very sensitive. And when we went through this training and then I sort of have a better understanding as to why they are that way. And I think its given me a better awareness as to better deal with these patient in the future…”

Participants uniformly suggested that the CWV training be made readily available, and that the training was needed for other staff (e.g., front-line staff such as clerks and lab technicians)—and even patients—in the facilities. As one participant noted, “I can’t think of anyone who doesn’t need this training.” Most participants felt that the training should be mandatory and available either in group trainings or online, similar to other VA trainings; some thought it should also be available for patients, running on a continuous video feed in waiting rooms.

Discussion

In traditional implementation efforts, emphasis is typically placed on maintaining fidelity to evidence-based practices. With EBQI, however, researchers and local staff work in partnership to adapt the evidence-based practice to the needs of the local context. In the present study, we compared a standard implementation of the CWV training to an EBQI implementation approach. Study findings suggest that the reach and effectiveness of CWV were greater for the EBQI implementation as compared to the standard web-based implementation, although improvements in staff sensitivity were also observed for the standard implementation. These findings extend prior efficacy research by documenting the effectiveness of CWV outside the more strictly controlled nature of the laboratory setting¹⁷, and suggest that local-level adaptations to the implementation of CWV did not negatively impact its effectiveness. Coupled with interview results indicating that the EBQI implementation of CWV was generally well-received, these findings highlight the added benefit that an EBQI approach may have in the implementation of cultural competence trainings such as CWV. Nevertheless, future research that examines longer-term effects on employee gender awareness, as well as studies that examine outcomes related to patient care are needed to further evaluate the impact of these different implementation strategies.

Importantly, all four sites chose to modify the CWV training to a facilitated group format as part of the EBQI process (with a slight variation in approach at the fourth site). Group-based collaborative learning allows for a diversity of perspectives, deeper reflection, and increased feedback among participants, all of which can facilitate learning³⁴. Combined with the leadership support that serves as a backbone of EBQI-based strategies implementing healthcare innovations and which is an essential component of successful health professions educational efforts^23–26, we expected and found that gender knowledge improved more in the EBQI condition than in the SI condition.

It is noteworthy, however, that the changes in gender sensitivity and knowledge we observed in the EBQI condition were modest (effect size range .20 –.22). Additionally, qualitative interviews revealed that there were important challenges related to the time and resources required for EBQI implementation, as well as the added challenge of adapting a training designed for individual completion to a facilitated group format. This begs the question of whether the costs of an EBQI approach to the CWV training outweigh the potential benefits. An EBQI approach to implementing CWV and other cultural competence trainings may be less likely to succeed in settings where there are fewer resources and where leadership support for the implementation is limited. For these settings, a web-based training implementation strategy, which was also shown to enhance gender sensitivity in the current study, may be the best alternative to implementing staff trainings like CWV.

A particularly interesting finding was that although there was room for improvement in VA employee’s gender sensitivity and knowledge (as reflected by increases in both from T1 to T2), most employees perceived that they are already sufficiently sensitive to the concerns of this patient population. This finding is consistent with findings from another study that identified providers’ beliefs that they were already culturally competent as an important obstacle to cultural competence education programs³⁵.

Several limitations to the present study should be noted. The response rate for the T1 survey was poor in both conditions, resulting in a smaller sample size than originally planned. Additionally, we initially planned to conduct hierarchical linear modeling analyses (HLM) that would take workgroup and site cluster effects into account. However, as both the number of workgroups (n = 32) and the average number of employees per workgroup (M = 4.84, with several workgroups with only 1 participant) were small, we were significantly underpowered for HLM analyses. Although we were not able to use HLM, the consideration of effect size estimates, along with the statistical significance of findings, nonetheless enhances confidence in the study findings.

Two sites experienced difficulties with recruitment for the T2 focus groups, likely due to participants being very busy healthcare professionals. As a consequence, several focus groups only included two people and one focus group had to be removed from analysis because it included only one attendee. Another limitation was that a number of focus group participants were from clinics (e.g., women’s health, mental health) with staff who are typically more familiar with issues relevant to the treatment of female Veterans, which may have contributed to the “preaching to the choir” theme that emerged in the qualitative results. Finally, we were only able to collect data on reach from participants who completed the survey, representing only a small percentage of people who were invited to complete either the EBQI or standard web-based implementation. Although we have information regarding the number of individuals who completed the EBQI training (67% of those invited), we were not able to track participation among individuals who completed the web-based training in the SI implementation condition.

Despite these limitations, to our knowledge, the current study represents the first effort to evaluate and compare an EBQI versus standard web-based strategy for delivering employee training in the health-care setting. The finding that EBQI delivery of the CWV training was successful in enhancing staff gender sensitivity and knowledge highlights the value of EBQI as a strategy that may increase the impact of evidence-based trainings and suggests a useful model for efforts to implement other employee trainings in both the VA and other healthcare settings.

LESSONS FOR PRACTICE.

1. Implementation strategies driven by evidence-based quality improvement (EBQI) can increase the impact of trainings for healthcare employees.

2. The decision to purse EBQI must be informed by a consideration of resources available to support its implementation.