Effects of an evidence-based practice intensive on competency: a pre/post domain-level program evaluation

Abstract

Background

Nurses struggle to engage meaningfully in all aspects of the evidence-based practice (EBP) process. This is due to a variety of barriers at the organizational and interpersonal level. Providing education can not only increase knowledge, but also potentially address other aspects of competency, such as attitudes and behaviors, as well as provides demonstrable organizational support. The aim of this initiative was to increase nurses’ competency in the EBP process by implementing and evaluating a cohorted, hands-on educational program.

Methods

Subject matter experts created and facilitated a cohorted EBP intensive course that included didactic work and the completion of an EBP project, from problem identification to dissemination. A pre/post course evaluation was used to examine the effects of the course on nurses’ EBP values, knowledge, and implementation behaviors. Other project outcomes, such as impact on future research, publications, and presentations were also tracked.

Results

As of 2020, four cohorts had been held with a total of 53 participants across 5 and the 6 hospitals in the health system. There were no significant differences between pre/post value and implementation scores; however, there was statistically significant improvement in perceived knowledge scores with a large effect size. The cohorts produced 5 conference presentations, 5 published manuscripts, and provided the background for 3 multi-site research studies linked to Magnet designation.

Conclusion

Participants demonstrated statistically significant improvement in perceived knowledge, and non-statistically significant decreases in perceived value and increases in implementation. There were no differences in domains as compared to years of experience. This evaluation underscores the importance of addressing all domains of competency, and the dual focus on skill development and real-world application demonstrates the potential for education programs to simultaneously build capacity and advance the profession’s evidence base while acknowledging the need for systemic, sustainable supports.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s12912-026-05017-9.

Background

The term evidence-based practice (EBP) refers to the “problem-solving approach to clinical decision-making that integrates the best available scientific evidence with the best available experiential evidence and encourages critical thinking in the judicious application of such evidence to the care of individual patients, a patient population, or a system” [1, p. 6]. This should be a structured process guided by established models or frameworks to ask questions, gather and appraise evidence, apply the findings, and measure the change [2, 3]. While the ability for nurses and allied healthcare professionals to engage meaningfully in these aspects of the EBP process, is a global expectation, there is a well-documented deficit in their ability to do so [2, 4, 5]. This is due to a variety of barriers, at both the individual (e.g. attitudes towards EBP, education background, EBP knowledge, and experience) and organizational levels (e.g. resources, organizational support) [6–8]. Addressing these barriers can increase healthcare professionals’ ability to engage in EBP to improve patient, staff, and organizational outcomes [7, 9].

Evidence highlights that the ability to conduct EBP projects is multi-faceted, and simply increasing knowledge is not sufficient to increase involvement in EBP work. Providing organizational support, while addressing and assessing the dynamic domains of competency (knowledge, skill/implementation, and attitude/value) offers a fuller picture of the complicated interconnections between what healthcare professionals know, what they can do, and how they feel about it [10]. This interplay ultimately reflects the true ability to improve patient care and outcomes and share findings. Many valid and reliable tools exist to measure these elements of competency, including the EBP Questionnaire (EBPQ), Developing EBP Questionnaire (DEBPQ), Evidence-Based Practice Implementation Scale, Evidence-Based Practice Beliefs Scale, and the Quick-EBP-Value, Implementation, Knowledge (VIK) instrument [11–14].

Despite the many factors influencing EBP in nursing and allied health, research has demonstrated providing education and facilitation can address deficits in knowledge, skills, attitude, confidence, and behavior, including dissemination [4, 9, 15, 16]. Specifically, face-to-face trainings with multiple components (e.g. lecture, discussion, and hands-on practice) have been shown to be particularly effective in positively influencing knowledge/skills, attitudes, and behavior [15, 17]. Additionally, cohort-based education, defined as placing learners into a unified group that progresses through a program together and share experiences while completing group milestones promotes a shared learning community that allows participants to collaborate and use their diverse personal and professional perspectives to learn from each other, think critically, and solve problems [8, 18, 19]. These also serve as professional support and networking opportunities to create ongoing connections beyond the limits of the educational offering. Each of these principles is key in the context of EBP competency development. This community aspect is particularly impactful in team-based healthcare environments that require subject matter expertise, critical thinking, clinical reasoning, collaboration, communication, and organizational support [1, 18]. Cohort-based education in the academic setting has been shown to have better educational outcomes and student retention than more traditional approaches but has limited evidence in the field of nursing professional development in the clinical setting [18].

While education as a remedy for limited knowledge or skill is not new, this paper illustrates how a cohort-based approach influences competency. It offers a replicable model and highlights the complex interplay between nurses’ perceptions, skills, and knowledge that shapes how effectively they can be upskilled in this foundational area.

Aim:

The aim of this initiative was to increase nurses’ competence in the EBP process, including dissemination, by implementing and evaluating a cohort-based, hands-on, EBP intensive (EBP-I) education program. Additional goals included addressing priority clinical issues identified by leadership, and sharing project findings to benefit healthcare as a whole.

Methods

Program evaluation design

A pre/post course evaluation was conducted to assess the EBP-I educational program. The evaluation examined participants perceived value, knowledge and implementation behaviors before and after the educational component of the program. Because survey responses were not linked across time points, the data were analyzed as independent samples. The evaluation included item-level and domain-level (value, knowledge, implementation) analyses of survey responses, as well as descriptive tracking of EBP project completion, dissemination activities, and reported practice changes. The EBP-I program was delivered to four cohorts comprising 53 initial participants. Each cohort completed the same core educational content and project requirements. Data from the four cohorts were combined and analyzed in aggregate rather than compared by cohort.

Participants and setting

The program took place in a 6-hospital health system in the United States, including academic and community hospitals in the Mid-Atlantic region and Florida. Participants included nurses and allied health professionals from diverse clinical specialties and professional roles from 5 of the affiliated hospitals. Participants were largely cohorted by affiliate or physical proximity.

Local nursing and organizational leaders identified prospective participants who had demonstrated an interest in EBP, clinical inquiry, or addressing a priority practice issue. Participation in the EBP-I program was supported by local leadership and system-level subject matter experts (SMEs).

Intervention

Participants completed a cohort-style, 5-day EBP-I (40 h total) run by EBP SMEs using the Johns Hopkins Evidence-based Practice Model for Nurses and Healthcare Professionals, 4th edition [20]. The program combined didactic instruction with the development of an EBP project addressing a priority topic identified in collaboration with organizational and local leadership. See Fig. 1 for a map of course material and associated project deliverables. Interested participants could also elect to complete an extension that included a mentored writing experience with the goal of publishing a peer-reviewed manuscript and presenting findings at a conference. Project topics are detailed in Table 1.

Fig. 1.

Course map and associated deliverables

Table 1.

Topics and outcomes by cohort

Cohort	EBP question	Outcomes
1 (n = 10)	What are best practices for promoting safe patient care by hospital-based traveling staff?	-1 conference presentation [21] -1 published manuscript [26]
2 (n = 10)	What are best practices for implementing k-cards in the healthcare setting?	-1 conference presentation [22] -1 published manuscript [27]
3 (n = 8)	What are best practices for preventing violence against healthcare workers by patients identified as at risk for aggression?	-1 conference presentation [23] -1 published manuscript [29] -Used a background for 1 multi-site research study
4 (n = 25)	What are best practices for implementing virtual nursing?	-2 conference presentations [24, 25] -2 published manuscripts [28, 30] -Used as background for 2  multi-site research studies

Measures

Perceived value, implementation, and knowledge (VIK) of EBP were assessed using a multi-item instrument, the Quick EBP-VIK survey, with demonstrated validity (intraclass correlation coefficients [ICCs] = 0.43–0.80) and internal consistency reliability (Cronbach’s α > 0.70) [14]. The tool contains three domains with Likert-scale responses- (1) perception of value (5 items), (2) reported implementation behavior (7 items), and (3) perception of knowledge (7 items) [14]. For this course evaluation, the instrument demonstrated acceptable to excellent reliability (Values α = 0.63 raw; 0.73 standardized; Knowledge α = 0.93; Implementation α = 0.82). See supplemental material for specific instrument questions. The survey also collected years of professional healthcare experience to describe program participants’ characteristics and assess whether perceived EBP value, knowledge, and implementation varied by time in practice. Years of experience were treated as an indicator of professional healthcare tenure, rather than a direct measure of seniority, professional maturity, or EBP expertise. The variable was categorized into five groups for exploratory between-group analyses. These analyses were intended to identify whether participants at different career stages entered the program with different EBP-related perceptions or behaviors, which could inform future participant selection, course tailoring, and program support.

Data collection

Participants completed the surveys via Qualtrics or Microsoft Forms. Links were sent via email to participants to complete the survey prior to the first class. Follow-up surveys were completed at the end of the second didactic day in person or via email.

Project outcomes, including dissemination efforts and associated practice changes were tracked for each cohort and recorded in an Excel spreadsheet.

Data analysis

All statistical analyses were conducted using IBM SPSS Statistics, Version 29.0 (IBM Corp., 2022). Descriptive statistics were used to summarize participant characteristics, survey responses, and project outcomes. Data from the four cohorts were analyzed in aggregate. Because the pre- and post-program surveys were anonymous and could not be linked at the participant level, comparisons between timepoints were conducted using independent-samples analyses.

Reliability

Internal consistency reliability for each domain was assessed using Cronbach’s alpha. Inter-item correlations were examined to evaluate internal structure and domain cohesion.

Descriptive and correlation analyses

Descriptive statistics were calculated for all individual items and domain (value, implementation, and knowledge) scores. Item-level relationships and domain structure were examined using a full inter-item Pearson correlation matrix across value, implementation, and knowledge items. Because several individual items violated assumptions of normality, nonparametric methods were selected for item-level pre–post comparisons.

Domain scores were calculated using a two-step mean-of-means approach. First, item-level Likert responses within each conceptual subscale were averaged to generate subscale scores. Second, domain-level scores for value, implementation and knowledge were computed as the means of their respective subscale means, ensuring equal weighting of each subscale regardless of the number of items. Domain scores retained the original Likert-scale metric and were interpretable as average levels of agreement or frequency.

Descriptive statistics, including means, standard deviations, medians, and interquartile ranges, were calculated for each domain by pre- and post-implementation group. Distributional assumptions were assessed using visual inspection of histograms and Shapiro–Wilk tests, which indicated non-normal distributions for several domain scores.

Given the use of independent pre- and post-implementation samples and non-normal distributions, group differences in domain scores were examined using Mann–Whitney U tests. This nonparametric test compares the relative distributions of scores between groups based on ranked data and does not assume normality. For each domain, scores from both groups were pooled and ranked, and mean ranks were calculated to summarize the average relative position of each group within the combined distribution, with higher mean ranks indicating generally higher domain scores.

Effect size (r) was calculated for statistically significant Mann–Whitney U results using the formula r = Z / √N, where N represents the total number of observations across groups. Statistical significance was defined as p < .05 (two-tailed).

To evaluate whether EBP domains varied by years of nursing experience, one-way analyses of variance (ANOVA) were conducted with value, implementation and knowledge domain scores as dependent variables and years of experience as the independent categorical variable. Assumptions of normality, homogeneity of variance, and independence were assessed. Effect sizes for the ANOVA results were reported using eta squared (η²) Missing data were handled using listwise deletion.

Per institutional policy, ethical review by the Institutional Review Board was not required as the goal of the data collection and analysis was continuous program improvement. Participation in the evaluation was voluntary.

Results

As of 2025, four cohorts have been held with a total of 53 participants across 5 hospitals (50 nurses, 2 pharmacists, 1 respiratory therapist) with a broad distribution of experience (see Table 2). Two participants declined the pre-program assessment, resulting in 51 pre-assessment responses. Of these, 42 completed the post-program assessment, representing an 82.4% follow-up response rate and 17.6% survey attrition. There was no attrition from the educational program itself. Descriptive analyses indicated high baseline value domain scores that remained stable from pre-implementation (M = 3.95, SD = 0.50) to post-implementation (M = 3.87, SD = 0.61). Knowledge domain scores increased substantially from pre-implementation (M = 2.59, SD = 0.61) to post-implementation (M = 3.23, SD = 0.54). Implementation domain scores showed a modest increase from pre-implementation (M = 2.03, SD = 0.66) to post-implementation (M = 2.12, SD = 0.55).

Table 2.

Years of participant experience

Year Range	Frequency (percent)
0–3	12 (23.5%)
4–7	8 (15.7%)
8–11	10 (19.6%)
12–15	7 (13.7%)
> 16	14 (27.5%)
Total	51 (100%)

There were no significant differences between pre/post value and implementation scores; however, there was statistically significant improvement in perceived knowledge scores with a large effect size. See Table 3 for pre- and post-implementation comparison of value, implementation, and knowledge domain scores.

Table 3.

Pre- and post-implementation comparison of value, knowledge, and implementation domain scores

Domain	Group	n	M	SD	Mean rank	Mann–Whitney U	Z	p
Value	Pre	51	3.95	0.50	47.17	1062.50	-0.07	0.947
	Post	42	3.87	0.61	46.80
Knowledge	Pre	51	2.59	0.61	35.17	467.50	-4.68	< 0.001
	Post	42	3.23	0.54	61.37
Implementation	Pre	51	2.03	0.66	44.21	928.50	-1.10	0.270
	Post	42	2.12	0.55	50.39

Note. Domain scores represent mean Likert-scale values calculated using a mean-of-means approach. Due to non-normal distributions and independent samples, group differences were examined using Mann–Whitney U tests. Mean ranks reflect the average relative position of scores within the combined pre- and post-implementation distributions

There were no statistically significant differences across domains based on years of experience (Value: F(4, 46) = 0.45, p = .776, η² = 0.04; Implementation: F(4, 46) = 0.17, p = .952, η² = 0.02; Knowledge: F(4, 46) = 0.52, p = .719, η² = 0.04). Years of experience accounted for less than 5% of the variance in any domain, suggesting EBP competency is not strongly associated with practice tenure. Overall, the EBP projects completed by each cohort resulted in 5 conference presentations [21–25], 5 published manuscripts [26–30], and served as the background for 3 multi-site research studies used to support Magnet designation. See Table 1 for project outcomes by cohort and topic.

Discussion

This program evaluation found statistically significant improvement in participant perceived knowledge, as well as non-statistically significant decreases in perceived value and increases in implementation. There were no demonstrable differences across the three domains as compared to years of experience. Each project had a measurable outcome, including publications, presentations, and identification of gaps in the literature to inform future research studies. This dual focus on skill development and real-world application demonstrates the potential for education programs to simultaneously build elements of capacity while advancing the profession’s evidence base.

Value

Lack of change in value scores may be reflective of the purposeful approach to recruiting cohort members. Leadership from each affiliate selected nurses and allied healthcare providers who had a demonstrated interest in EBP and likely had an inherently higher value of EBP before the program began (3.95 on a Likert scale of 1- strongly disagree to 5- strongly agree) which is consistent with others’ findings that perceptions of the value of EBP is high [5]. Program organizers should recognize cohort-based learning has the ability to build upon existing buy-in, or lack thereof, through community-building and group coalescence around popular ideas. This principle can engender support or sew disengagement [18]. Prevailing attitudes should be monitored carefully.

This study’s finding that years of experience was not associated with perceived EBP value aligns with some prior studies, however, the literature is mixed [31–33]. Some studies suggest more favorable EBP attitudes among less experienced clinicians [34], while others report stronger EBP attitudes among more experienced practitioners [35, 36]. This variation may be due to the recent exposure newer nurses have to EBP or the relative agency more experienced nurses feel with their ability to enact best practices and navigate organizational systems– meaning Value may stem from various sources. This evaluation adds to the mixed evidence on the impact of years of experience on perceived value of EBP which further highlights the need to target all nursing demographics as none are clearly more favorable than others.

Implementation

Consistent scores for implementation were to be expected due to the short time separating the pre- and post-surveys. While participants completed some of the listed tasks as a part of their cohort experience, there was likely not sufficient time to complete sufficient tasks to bump a participant from one Likert bracket to another. Beyond the limited time frame, many studies have shown that although nurses view EBP favorably (value), they are limited in their ability to carry it out (implementation) [5, 34, 37]. This is due to many factors and simply addressing knowledge and value is not sufficient. Our evaluation demonstrates that despite the academic and practical promise of cohort-based programs, organizational support, organizational readiness, and resources must be in place for inner factors to become actionable [7, 18, 34, 38]. Beyond investing in education, organizations must have enduring, accessible supports for nurses to engage in EBP in a meaningful way. This could include mentorship programs, paid project time, physical space and equipment, ongoing professional development, and integration of EBP culture into existing infrastructures, such as shared governance and Magnet designation [39].

Knowledge

The observed improvement in knowledge scores is to be expected, because both intuitively, and as reflected in the literature, education is a proven way to increase learning [8, 15, 18]. However, much like other published evidence, the changes seen within this evaluation’s cohorts are subjective, self-reported, and may not be reflective of actual knowledge [7, 8, 15, 38]. Subjective knowledge assessments are particularly challenging because pre-tests can be influenced by the Dunning-Kruger effect, the cognitive bias that occurs when experts underestimate their expertise and beginners overestimate their abilities—essentially, they don’t know what they don’t know [40]. Theoretically, this means that pre/post knowledge scores could remain the same because of the inverse relationship between actual and perceived knowledge. As people learn more, they also recognize the true depth of a subject, and although they know more, they also rate themselves more modestly. This well-documented phenomenon makes it difficult to ascertain the true pre-knowledge of a participant and should call into question the utility of a knowledge self-assessment without a correlated objective assessment, or additional opportunities for self-reflection or feedback [41–43].

Outcomes

Each EBP project had a measurable impact on the science of nursing and the state of the literature on a given topic but was not measured in terms of patient or staff level outcomes. Although each project was selected by local leadership, the individuals within the cohorts represented various departments which made implementation of best practices difficult across settings and populations. Using dissemination as an EBP project outcome has been used in other EBP upskilling initiatives, such as the Center for Clinical Research and Scholarship at Rash University or the Evidence-Based Mentorship Program at Boston Children’s Hospital, who had similar successes [16, 44].

Collectively, these findings indicate that the intervention successfully addressed knowledge, but that sustained implementation strategies may be necessary to support consumption and translation of evidence into routine clinical practice. Regardless of changes in individual participants, concrete project outcomes indicate demonstrable success in the establishment of best practices as a whole to address real-world clinical problems.

Limitations

There are several limitations to this study that are common among this small-scale program evaluation approach. Selection bias may limit generalizability because participants were identified by local leadership based on demonstrated interest in EBP or clinical inquiry. Overall, there were only 53 participants. Fifty-one of these participants were represented in the pre-data, but the response rate fell in the post-group, representing only 79% of participants. This inconsistency, as well as the unmatched t-test, makes comparisons between groups less rigorous. Because pre- and post-program responses were unmatched, differences between timepoints cannot be attributed solely to the EBP-I program and may reflect differences in the respondents who completed each assessment. There was also insufficient data to conduct a meaningful analysis of open-ended participant feedback to conduct a more in-depth evaluation of the course itself.

Another limitation is the short time between the pre and pos tsurveys. The post survey was administered at the end of the didactic content without longitudinal follow up, but enduring changes in value, implementation, and knowledge are what truly changes practice. Therefore, the findings reflect only immediate changes in perceived EBP knowledge and cannot determine whether these gains were retained, translated into EBP behaviors, or integrated into clinical practice over time. Additionally, all the values were self-reported and may not be reflective of true differences. This can be particularly problematic for assessment of knowledge acquisition because people frequently have difficulty accurately quantifying this domain.

Finally, while the evaluation tracked project outcomes, it did not assess any changes to patient, staff, or organizational metrics. This makes it difficult to establish the ultimate impact of the training and its goal to improve healthcare.

Conclusion

Globally, there is a gap between EBP expectations and abilities. This targeted, cohort-based program aimed to strengthen local EBP competency, address an organizational priority using an EBP approach, and contribute to the evidence base by intentionally mitigating common barriers to EBP engagement, including limited knowledge, time, mentorship, and resources while providing engaging learning communities. This evaluation offers a replicable approach while underscoring the importance of assessing and addressing competency as a multidimensional construct and the limitations of relying solely on subjective measures of program impact. Although the findings are promising, more rigorous evaluation is needed, including longitudinal follow-up and objective assessments of knowledge, implementation outcomes, and patient-level metrics.

Supplementary Information

Below is the link to the electronic supplementary material.

Abbreviations

EBP

Evidence-based practice

VIK

Value, Implementation, Knowledge

Author contributions

MW and HW made substantial contributions to the project, data acquisition, interpretation of the data, and drafting the manuscript. HW completed the statistical analysis. All authors reviewed and approved the manuscript.

Funding

This work did not receive research funding.

Data availability

All data supporting the findings of this program evaluation are available either within the paper and its Supplementary Information, or upon request.

Declarations

Ethics approval and consent to participate

Not applicable. Per institutional policy, ethical review by the Johns Hopkins Institutional Review Board was not required as the goal of the data collection and analysis was continuous program improvement. Because the project did not constitute human subjects research per institutional definition, the Declaration of Helsinki was not applicable. Informed consent was not required because the project was considered continuous improvement. Participation in the evaluation was voluntary.

Consent for publication

Not applicable. All material is owned by the authors, and no permission is required.

Competing interests

The authors declare no competing interests.