Evaluation of an evidence-based practice mentorship programme in a paediatric quaternary care setting

Abstract

Background

Evidence-based practice (EBP) is essential for clinical decision-making, improving care, reducing costs and achieving optimal patient outcomes. The Evidence-based Practice Mentorship Program (EBPMP) is a flexible, self-directed programme whereby participants carry out EBP projects guided by expert mentors.

Aims

To evaluate EBPMP effectiveness and participant experience.

Methods

To evaluate effectiveness, as measured by changes in EBP value, knowledge and implementation, participants completed pre- and post-EBPMP Quick-EBP-Value, Implementation and Knowledge (VIK) surveys. To understand participants’ experiences individual and group interviews were conducted at the end of the programme and analysed using qualitative content analysis.

Results

Most participants were over 50 years old, Caucasian, inpatient staff nurses, baccalaureate prepared, with over 11 years’ experience. Statistically significant improvements were observed in the post Quick-EBP-VIK knowledge and implementation domains. Individual and group participant interviews revealed four categories of importance to the experience: 1. perceived benefits of EBP, 2. time as a barrier to EBP, 3. desire for more cohort interaction and 4. positive mentee–mentor experience.

Conclusions

EBPMP can improve participants’ knowledge and implementation of EBP in an environment that values EBP; however, opportunities exist to implement programme modifications that address barriers identified by participants including project time and increased participant interactions.

Introduction

The Evidence-based Practice Mentorship Program (EBPMP) was implemented within a 404-bed academic free-standing quaternary care children’s hospital with Magnet® designation in the northeast United States (Schuler et al., 2020). The goal of the EBPMP was to promote value, facilitate implementation and increase evidence-based practice (EBP) knowledge to advance the science of nursing practice. Thirteen nurses navigated a one-year programme guided by a workbook, online educational modules and EBP expert nurse mentors to support the participants’ scholarship. This paper describes evaluation of the EBPMP using Quick-EBP-Value, Implementation and Knowledge (VIK) survey and through individual and group interviews.

Background

EBP is a problem-solving process integrating best evidence, clinical expertise and patient preferences and values to inform clinical decision-making (Fineout-Overholt et al., 2005; Melnyk et al., 2012). Nurses’ implementation of EBP has been associated with reduced costs, higher quality care and improved patient outcomes (Melnyk and Fineout-Overholt, 2019). EBP has also been associated with increased job satisfaction and nursing retention, which can contribute to institutional achievement of Magnet® designation as awarded by the American Nurses Credentialing Center (Melnyk, 2007; Melnyk et al., 2005). Nurses who participate in EBP programmes feel empowered to question and improve practice (Christenbery et al., 2016). The Institute of Medicine’s 2020 goal that 90% of all care decisions are evidence-based has encouraged healthcare institutions to foster a culture of EBP through educational initiatives designed to promote nurses’ EBP knowledge and skill (Breckenridge-Sproat et al., 2015; Christenbery et al., 2016; Melnyk et al., 2005).

Despite its known benefits, barriers to implementing EBP persist and include inadequate time, leadership support, organisational awareness, access to evidence, and EBP knowledge and skill (Melnyk, 2016; Melnyk and Fineout-Overholt, 2019). A national survey of 1015 nurses, in 2012, found that only half (53%) of respondents believed EBP was consistently applied within their institution; 34% believed nursing peers were utilising evidence-based care; 35% reported having access to EBP mentors; and 68% desired access to institutional resources (Melnyk et al., 2012). These findings renewed the need for healthcare organisations to seek innovative solutions that address individual and organisational barriers that preclude nurses from integrating evidence into practice.

According to the Advancing Research and Clinical practice through close Collaboration model, EBP mentors are essential to building and sustaining a culture of EBP by supporting nurses and clinicians at the point-of-care (Breckenridge-Sproat et al., 2015; Melnyk, 2012). Due to the essential role of mentors, educational initiatives have focused on utilising mentorship as part of a multifaceted approach to conducting EBP scholarship (Chan et al., 2020; Spiva et al., 2017; Wallen et al., 2010).

In 2010, Wallen et al. demonstrated the effectiveness of an EBP mentorship programme in a 234-bed research hospital. This programme included a two-day workshop and offered ongoing EBP project support and intermittent mentorship-building sessions over a seven-month period. A pre- and post-intervention comparison of quantitative data demonstrated an increased effect on programme participants’ belief in institutional culture and readiness for EBP, utility and implementation of EBP as well as job satisfaction and group cohesion (Wallen et al., 2010). These findings were consistent with those of Spiva et al. (2017), who implemented a nurse mentor training programme in a five-hospital integrated non-profit healthcare system. Nurse leaders and clinical educators were trained as EBP mentors through a three-part programme. Online educational modules, didactic sessions and interactive webinars with follow-up activities were used to reinforce knowledge and skills. Using the Evidence-based Nursing Questionnaire (EBNQ) and investigator-developed Confidence Scale, pre and post survey of 66 mentors revealed statistically significant improvements in their perceptions of knowledge, attitude, skill, confidence level and organisational readiness for EBP. In addition to the mentor training programme, bedside nurses were offered four 30-minute online educational modules to provide foundational knowledge. Responding to the EBNQ, Barriers to Research Utilisation Scale, EBP Nurse Leadership and EBP Work Environment Scale, the 367 nurses who completed the modules reported perceived improvements in EBP attitude, skill, knowledge, barriers, work environment and nurse leadership (Spiva et al., 2017).

Chan et al. (2020) used a similar approach to develop frontline EBP champions within a 1600-bed tertiary hospital in Singapore. Nine nurses participated in a 12-month research and EBP mentorship programme consisting of didactic and in-person skill building sessions. These nine nurses, then serving as EBP champions, co-led EBP projects on the wards with their mentors. To evaluate the programme, the nine EBP champions and their ward peers were surveyed using the Evidence-based Practice Questionnaire (EBPQ). Pre (n = 197) and post (n = 194) programme, EBPQ survey comparisons indicated significant increases in knowledge, attitude and EBP practice (Chan et al., 2020).

Problem summary

EBP is a problem-solving process that informs clinical decision-making, improves care, reduces costs and achieves optimal patient outcomes. Emerging literature demonstrates that nurses value EBP and mentorship can be an effective mechanism in the development of nurses’ EBP knowledge and implementation; although, given its importance, there are still relatively few published papers describing programmes that can help engage direct care nurses in the process. This paper describes findings from a programme evaluation following implementation of an organically developed EBPMP at a single paediatric academic medical centre.

EBPMP

In 2012, an institution-wide survey indicated institutional readiness for EBP and revealed opportunities for improvement in the areas of EBP knowledge and implementation (Paul et al., 2016). In response, nursing executive leadership called upon the institution’s EBP committee to address these identified gaps. The EBPMP was designed as a self-directed programme to guide nurse participants as they carried out EBP projects that addressed important clinical practice questions. Throughout the 12-month programme, participants were mentored through the EBP process and were exposed to many EBP resources available from within and outside the institution.

A workbook (Table 1) guided participants through the steps of the EBP process: develop a practice question, conduct a search strategy for relevant evidence, critically evaluate and synthesise findings. The step-by-step instructional workbook included a list of recommended readings and 10 web-based educational modules to build participants’ foundational knowledge. Individual modules were designed to take no longer than 30 minutes to complete and topics correlated to the associated step. Participants reviewed modules on their own timeframe and submitted them to the project coordinator upon completion. Each participant was aligned with an EBP expert nurse mentor based on clinical area of expertise and accessibility within the institution, to help guide and support them through the process. Clinical nurse specialists, advanced practice nurses, doctoral prepared nurse leaders and EBP committee members who completed an EBP immersion experience in 2008 served as programme mentors. The primary goal of the programme was for nurse mentees to develop and present a professional poster of their findings at the institution’s annual nurse’s week celebration. Presentation signified programme completion, at which point they were encouraged to disseminate abstracts externally and serve as an EBP resource to others in their clinical environment and throughout the institution.

Table 1.

Evidence-based Practice Mentorship Program workbook 2016.

Step	Goal(s)	Recommended reading (example)
1: Evidence-based practice overview	Discuss programme participation, support and expectations with nurse manager or director; select a topic(s) of interest for a clinical practice question; submit topic(s) of interest to the programme coordinator Participate in open dialog with evidence-based practice mentor about areas of interest and development of a clinical practice question; discuss establishing a timeline, completing meeting and activity logs, accessing online modules and completing programme goals	Schaffer M, Sandau K and Diedrick L (2013) Evidence-based practice models for organisational change: Overview and practical applications. Journal of Advanced Nursing 69(5): 1197–1209. DOI:10.1111/j.1365-2648.2012.06122.x.
2: Forming a clinical practice question	Review clinical practice question format and feasibility; develop a search strategy and timeline with evidence-based practice mentor to conduct a literature search for topic of interest Present clinical practice question for feedback from the evidence-based practice committee	Echevarria I and Walker S (2014) To make your case, start with a PICOT question. Nursing 44(2): 18-19. DOI: 10.1097/01.NURSE.0000442594.00242.f9.
3: Collecting the evidence	Meet with institutional librarian to learn how to perform a literature search; perform a literature search and collect evidence that will help answer the clinical practice question; organise evidence by date, name, and type Discuss use of internal and external sources with evidence-based practice mentor to complete collection of evidence	Caldwell PH, Bennett T and Mellis C (2012) Easy guide to searching for evidence for the busy clinician. Journal Of Paediatrics & Child Health 48(12): 1095-1100. DOI:10.1111/j.1440-1754.2012.02503.x.
4: Critical appraisal of the evidence	Review critical appraisal of the evidence with evidence-based practice mentor; select critical appraisal tool; populate evidence table	Fineout-Overholt E, Melnyk BM, Stillwell SB and Williamson KM (2010) Evidence-based practice, step by step: Critical appraisal of the evidence: Part II: Digging deeper–examining the “keeper” studies. The American Journal of Nursing 110(9): 4–148. DOI:10.1097/01.NAJ.0000388264.49427.f9.
5: Hierarchy of evidence	Rank evidence according to standardised ranking system; discuss strengths and weaknesses of evidence with evidence-based practice mentor to formulate clinical practice recommendation	Melnyk BM (2014) Speeding the translation of research into evidence-based practice and conducting projects that impact healthcare. Quality, patient outcomes and costs: The ‘So What’ outcome factors. Worldviews on Evidence-Based Nursing 11(1): 1–4 4p. DOI:10.1111/ wvn.12025.
6: Translating evidence into practice	Discuss project implications including clinical practice or process changes; develop a plan for translating the recommendation into practice Present findings to the evidence-based practice committee and nurse manager or director in a professional presentation; discuss process of receiving and integrating professional feedback	Aronson L (2015) Story as evidence, evidence as story. The Journal of the American Medical Association 314(2): 125-126. DOI:10.1001/jama.2015.3930.
7: Dissemination	Receive evidence-based practice committee evaluation and feedback; discuss plan and timeline for implementing and integrating evidence-based practice outcomes into nursing practice Internally disseminate evidence-based practice project at annual Nurses Week Celebration and among unit-based nurses; examine venues to externally disseminate	Pitkänen A, Alanen S, Rantanen A, Kaunonen M, Aalto P (2015) Enhancing nurses participation in implementing evidence-based practice. Journal for Nurses in Professional Development 21(2): E1-E5. DOI: 10.1097/NND.0000000000000161.

Purpose and aims

The purpose of this programme evaluation was to examine effectiveness of the EBPMP, as well as participant experience with mentors, programme progression and materials. Specific aims were to evaluate programme effectiveness through pre-post changes in nurse participants’ EBP value, knowledge and implementation as measured by Quick-EBP-VIK; and explore participants’ experience with the EBPMP through individual and group interviews conducted near the end of the year-long programme.

Methods

Design

This project employed a mixed methods study design using both quantitative (Quick-EBP-VIK survey) and qualitative (individual and group interviews) methods.

Sample

Staff nurses were recommended for programme participation by their direct supervisor or nurse manager. The pilot cohort consisted of 13 paediatric staff nurses from inpatient and ambulatory settings. To be included in this study’s convenience sample, participants had to be members of the first EBPMP cohort and employees of the hospital.

Setting

The setting was a 404-bed academic free-standing quaternary children’s hospital in the northeast United States that delivers highly specialised medical care to local, regional, national and international paediatric patients of all ages. Since 2008, the institution has been recognised for excellence in nursing care through designation as a Magnet® hospital by the American Nurses Credentialing Center (2020).

Recruitment

Recruitment for individual and group interviews occurred through secure email communication towards the end of the year-long programme. The email contained study information including how to schedule the interviews. Two reminder emails were sent to bolster recruitment.

Data collection

Quick-EBP-VIK survey

The Quick-EBP-VIK survey was administered via Qualtrics® online survey platform in April 2016, at the start of the EBPMP. The survey was administered again after the programme ended, in May 2017. Quick-EBP-VIK is a 19-item instrument measuring nurses’ value, knowledge and implementation of EBP (Paul et al., 2016). Questions are based on a five-point Likert-type scale from 1 to 5 (1 – strongly disagree, 2 – disagree, 3 – neutral, 4 – agree, 5 – strongly agree). The Quick-EBP-VIK underwent comprehensive testing for psychometric properties (Connor et al., 2017). Test and re-test response rates of over 30% were reported with intra-class correlation coefficients ranging between 0.43 and 0.80 for the 19 items. Cronbach’s alpha coefficients were greater than 0.7 for items within each of the three domains, demonstrating internal consistency. Of note, Quick-EBP-VIK question 5, ‘EBP is not important to my nursing practice’, was the only statement requiring reverse coding on a 5–1 Likert scale (as opposed to 1–5 scale) (Connor et al., 2017). Quick-EBP-VIK has subsequently been translated into a Chinese version with similar psychometric testing results (Zhou et al., 2018). In addition to the 19 base items, an additional 11 survey items were developed to elicit participants’ demographic information such as clinical setting, work schedule, level of education, years of nursing experience and previous experience with EBP.

Individual and group interviews

To understand mentees’ perception of the programme and their experience throughout the learning process, qualitative investigation was undertaken through individual and group interviews. Questions were open-ended with no right or wrong responses. Participants were reminded that they were the expert and their reflections about the experience were most relevant and important. Examples of interview questions included:

1. Please share with me your experience working with your mentor.

2. As you reflect over the past few months, what has been your experience progressing through the EBPMP?

3. In what ways have or haven’t EBPMP materials (i.e. workbook and online modules) been helpful? What about other learning opportunities?

Interviews were conducted in private conference rooms by two nurses, a PhD-prepared nurse and a doctoral student, both trained in qualitative research methodology and interviewing techniques, and without any supervisory responsibilities for the participants. Written, informed consent was obtained at the beginning of each session. The 60-minute interviews were audio recorded and transcribed verbatim by a Collaborative Institutional Training Initiative programme certified research team member for data analysis. Identifying information, including names or specific clinical settings, were omitted to ensure confidentiality.

Ethical considerations

The hospital and academic institution’s human subjects’ boards reviewed and approved this study (IRB-P00023767). Data were de-identified and reported in aggregate to ensure participant confidentiality.

Data analysis

Descriptive statistics were used to characterise the EBPMP pilot cohort and analyse survey data. Aggregate pre- and post-intervention means were calculated for each survey question and examined for equality of variances using a p-value of <0.05. Questions with equal variance were compared via t-test to examine differences in value, knowledge and implementation of EBP. Questions without equal variance were compared via Satterthwaite t-test. A p-value of <0.05 was indicative of a statistically significant finding. All data were analysed using SAS platform version 9.4.

Qualitative analysis of individual and group interviews was performed by the principle investigator, two key members of the institution’s EBP committee, the interview facilitators. Using conventional content analysis (Graneheim and Lundman, 2004), coding was performed individually, then collaboratively, to achieve agreement of codes and to enhance criticality and integrity. The team maintained an audit trail of findings and decisions for transparency. To maintain rigour, the research team ensured that codes remained close to the data and were not altered by interpretation or inferred possibilities from their experiences. First level coding of the transcripts consisted of identifying words, comments or phrases participants emphasised, repeated or stressed to convey importance. First level codes were then compared across transcripts and discussed among team members to resolve any discrepancies. After ensuring that all first level codes were addressed, the team collaborated on clustering similar first level codes and creating second level codes that were more inclusive. The second level codes were relabelled and defined as categories. The categories were then reviewed to determine whether they consisted of subcategories or could be combined into patterns. Each category was independent and could not be combined, but when taken together, they provided an informative and helpful evaluation of the programme according to the EBPMP pilot cohort’s experience.

Results

EBPMP pilot cohort demographics, nursing characteristics and project outcomes

The majority of participants were 50 years old (46%), Caucasian (100%) and baccalaureate prepared (84%) (Table 2). More than half of participants (54%) had not received previous formal EBP training. One participant withdrew from the programme and was excluded from the post-EBPMP survey and interview. Of the 12 participants who completed the programme, seven found evidence to support a practice change while five did not. These five participants concluded that current practice was supported by evidence or more research was needed before making a practice change. Figure 1 provides an example of a completed EBP project from the pilot cohort.

Table 2.

Evidence-based Practice Mentorship Program cohort characteristics (N = 13).

Characteristics	Frequency	Percentage
Age (years)
Less than 30	3	23
30–49	4	31
50–65	6	46
Ethnicity
White or Caucasian	13	100
Nursing level a
Staff Nurse I	5	38
Staff Nurse II	7	54
Staff Nurse III	1	8
Highest nursing degree
Associates	1	8
Baccalaureate	11	84
Masters	1	8
Nursing experience (years)
1–10	4	31
11–20	3	23
21–30	4	31
31–35	2	15
Clinical setting
Main campus, Inpatient	7	54
Main campus, Ambulatory	1	8
Main campus, Procedural Unit	3	23
Satellite campus, Procedural Unit	2	15
Area of expertise
Medical-Surgical	3	23
Critical Care/Intensive Care Unit	3	23
Perioperative	2	15
Emergency Department	1	8
Cardiovascular	1	8
Other	3	23
Typical work schedule
Day shift only	10	77
Night shift only	1	8
Rotating schedule	2	15
Previous formal EBP training
Yes	6	46
No	7	54

^aThe institution has a professional advancement ladder for staff nurses. As nurses achieve professional goals, they advance from level I to level III.

EBP: evidence-based practice.

Figure 1.

Evidence-based practice project example.

Quick-EBP-VIK survey

Thirteen participants completed the pre-EBPMP survey and 12 completed the post-EBPMP survey. p-values for t-test comparison of pre and post means are shown in Table 3. While no statistically significant differences were noted within the value domain, all items within the knowledge domain demonstrated statistically significant improvements post-EBPMP. Within the implementation domain, nurse mentees demonstrated statistically significant improvements in the frequency of literature searches performed, frequency of critical appraisals of literature search for evidence, frequency of having shared the EBP process knowledge, and frequency of having used EBP results to propose change post-EBPMP.

Table 3.

Comparison of Quick-EBP-VIK pre- post-Evidence-based Practice Mentorship Program participation.

Items	Pre-EBPMP n = 13 Mean (SD)	Post-EBPMP n = 12 Mean (SD)	Difference in means (95% CI)	p-value
Value
EBP enables me to provide highest quality care	4.31 (1.11)	4.00 (1.04)	0.31 (–0.59, 1.20)	0.483
Environment values change in practice based on EBP	4.15 (0.80)	4.25 (0.45)	–0.10 (–0.64, 0.45)	0.718
Nursing leadership values EBP	4.23 (0.83)	4.33 (0.49)	–0.10 (–0.67, 0.47)	0.714
I can change nursing practice by using EBP	4.00 (0.71)	4.00 (0.60)	0 (–0.55, 0.55)	1
EBP is not important to my nursing practice a	4.54 (0.52)	4.00 (1.04)	–0.54 (–1.25, 0.17)	0.127
Knowledge
Regarding steps of EBP	2.31 (0.75)	3.67 (0.65)	–1.36 (–1.94, –0.78)	<0.001
How to form a PICO question	2.15 (0.90)	3.75 (0.62)	–1.59 (–2.24, –0.95)	<0.001
About ranking system for the hierarchy of evidence	2.38 (0.87)	3.33 (0.49)	–0.95 (–1.54, –0.36)	0.003
Performing a literature search using an online database	2.46 (0.97)	3.67 (0.49)	–1.20 (–1.85, –0.56)	<0.001
Critically appraising a systematic review	2.45 (0.93)b	3.25 (0.45)	–0.80 (–1.42, –0.17)	0.016
Critically appraising a qualitative research study	2.31 (0.95)	3.25 (0.45)	–0.94 (–1.56, –0.32)	0.005
Critically appraising a quantitative research study	2.31 (0.95)	3.17 (0.39)	–0.86 (–1.46, –0.25)	0.008
Implementation
Frequency of literature searches performed	1.23 (0.44)	2.83 (1.11)	–1.60 (–2.29, –0.91)	<0.001
Frequency of critical appraisals of literature search evidence	1.15 (0.38)	2.08 (0.29)	–0.93 (–1.20, –0.65)	<0.001
Frequency of having performed steps of EBP	2.15 (1.46)	3.08 (1.00)	–0.93 (–1.97, 0.12)	0.079
Frequency of having developed a PICO question	1.92 (1.32)	2.67 (1.15)	–0.74 (–1.77, 0.29)	0.149
Frequency of having shared the EBP process knowledge	1.62 (0.96)	2.67 (0.89)	–1.05 (–1.82, –0.28)	0.009
Frequency of having used EBP results to propose change	1.23 (0.44)	1.82 (0.40) b	–0.59 (–0.95, –0.23)	0.003
Frequency of EBP results having resulted in a change	1.23 (0.44)	1.42 (0.51)	–0.19 (–0.58, 0.21)	0.340

^aThis statement requires reverse coding on a 5–1 Likert scale (as opposed to 1–5 scale) as described by Connor et al. (2017).

^bn = 11.

EBP: evidence-based practice; VIK: value, implementation and knowledge; EBPMP: Evidence-based Practice Mentorship Program; CI: confidence interval; PICO: Population, Intervention, Comparison, Outcome.

Individual and group interviews

Half of the programme participants (n = 6) consented to participate in the interviews. Data from four individual and one group interview revealed four categories of importance to the experience: 1. perceived benefits of EBP, 2. time as a barrier to EBP, 3. desire for more cohort interaction, 4. positive mentee–mentor experience.

Perceived benefits of EBP

Nurse mentees felt empowered and enthusiastic to begin their EBP journey following a presentation by an international EBP expert at the beginning of the year-long programme. They acknowledged that the EBPMP provided a means to answer important clinical practice questions. Nurses felt the programme provided a positive experience that instilled knowledge of EBP.

‘I think it’s a great programme, and I, um, I see a lot of value in it. As far as answering questions about why we do these things that we do. And really having the evidence to support them’ (Participant 6).

‘The programme made it less overwhelming. It helped me put my ideas forward into motion’ (Participant 5).

‘…as nurses I think you see something, and I think you need to take responsibility and ownership… if you want to change it’ (Participant 4).

They admitted that being invested in the topic affected the overall experience.

‘You got to pick a subject you are passionate about, and it’s really going to help your kids. It’s so worth doing. You have to find that love and passion’ (Participant 5).

Time as a barrier to EBP

Nurse participants were given varying amounts of protected, indirect time away from their clinical responsibilities. Nevertheless, sometimes nurse mentees were asked to forgo their protected, indirect time for EBPMP because patient acuity or volume necessitated their presence in the clinical areas.

‘If it did get busy, I was still technically assigned and would have to go to that area and work…’ (Participant 3).

When nurse mentees did receive time away from the clinical areas, some found it difficult to detach themselves from clinical duties and concentrate on their EBP projects. These barriers led to use of personal time to complete the EBP steps.

‘I didn’t mind doing it at home, but I had no resources. So, if I was doing it here, I could run upstairs, [and] see if my mentor was around. I could run over to the library’ (Participant 1).

Nurse mentees also discussed that EBP steps 5 and 6 involving literature critique and hierarchical categorisation were the most time-consuming and difficult elements of the curriculum, resulting in a slower pace of programme progression. As stated by one mentee,

‘I felt like it was a little over my head. I’m finally through critiquing all my articles. And I feel like I’ve learned a ton’ (Participant 2).

While another commented,

‘…the time it took with the articles and putting [them] into the tables and trying to figure out terminology and what type of article it was… the time it would take to get me through that step was unexpected’ (Participant 3).

Desire for more cohort interaction

Nurse mentees expressed a desire for more cohort interaction. Although the programme was self-directed with mentor guidance, nurse mentees wished for increased interaction with their peers to compare progress, share ideas and discuss projects and challenges.

‘I didn’t feel connected to the [other] mentees at all, but I think that there was some great work being done’ (Participant 6).

Further probing revealed several recommendations for programme change. These included dedicated cohort meeting times such as didactic lectures, interactive webinars and sharing works-in-progress sessions.

‘It would have been really helpful to talk to other people doing their projects. Or even… be able to get together with them or email back and forth and be like, oh how are you doing this or where are you with this?’ (Participant 3)

The use of an online, private social media application was implemented, though participants did not find it helpful for cohort interaction. Additionally, mentees were invited to attend the monthly EBP committee meetings; however, they did not find them an effective modality for increasing their knowledge of EBP; the focus was centred on group evaluation of individual projects.

‘I don’t understand, don’t really understand what that meeting is about. It’s not helping me move forward with my project’ (Participant 1).

Positive mentee–mentor experience

Nurse mentees perceived their mentors as supportive and they appreciated their availability and willingness to share their EBP knowledge and practice insights.

‘My mentor was a very memorable person, quite eloquent, really understands the process and how to dive deep into the more theoretical aspects of EBP’ (Participant 6).

Mentees valued the individual attention and requested feedback early and often.

‘She was so supportive, and she didn’t hover, but she made sure that we were working together, you know’ (Participant 5).

They all agreed that direction and guidance were most necessary during the EBP steps involving literature review, article critique and hierarchical categorisation.

‘I was able to meet with her in person about that [putting articles into tables] and she looked over what I had done so far so that was really helpful to get feedback, to know I wasn’t doing it completely wrong before I finished doing it’ (Participant 3).

Nurse mentees who were interested in becoming mentors themselves requested ongoing support from their mentor.

‘I would still need a lot of guidance because I’m not at that stage where I can do it all on my own’ (Participant 4).

Discussion

This programme evaluation revealed a number of important findings contributing to the evolving body of literature on EBP mentorship programmes and prompted several changes to the EBPMP. According to the pre/post Quick-EBP-VIK survey findings, participation in the EBPMP increased nurse mentees’ knowledge and implementation of EBP, demonstrating its effectiveness as an educational approach; however, it did not increase mean scores within the value domain. Survey item responses within the implementation domain showed that nurse mentees ‘shared the EBP process knowledge’ and ‘used EBP results to propose change’ more often following the programme. These findings indicated that participants were sharing what they learned, from the programme and their projects, with their peers. This aligns with previous evidence showing that graduates of an EBP Scholars Program continue applying EBP skills, initiating new projects and mentoring others one year beyond programme completion (Royer et al., 2018) and is an important step to building an institutional culture of EBP. It was not surprising to the investigative team that the mean value scores did not change significantly; the EBPMP participants were highly motivated nurses who voluntarily participated and demonstrated a desire to improve patient care, and the mean value scores were already 4 or above, leaving little room for improvement in that domain.

Interestingly, mentees reported no change when responding to the implementation domain survey item ‘frequency of EBP results having resulted in a change’. This suggested that while the EBPMP was beneficial for mentees to evaluate and synthesise evidence, and to disseminate clinical practice recommendations, the programme lacked a process to guide participants through implementing a practice change. As highlighted by Tucker (2019), institutions must utilise EBP implementation models that promote uptake of findings from EBP initiatives to bridge the gap between knowledge and evidence-based care. Likewise, recommendations from EBP initiatives must be integrated into clinical pathways for easy adoption by bedside clinicians at the point of care (Stevens, 2012).

The EBPMP leadership responded to this programmatic need by developing a process for implementation and measurement of practice changes. This has been accomplished through several established structures within the institution including unit-based quality improvement groups, the Nursing Science Fellowship and the Nursing Practice Committee (DeGrazia et al., 2019). Yet, another interpretation of these findings is that not all EBP projects lead to practice change recommendations. Several participants found that evidence supported current practice while others discovered there was not enough literature to support a practice change. In these situations, a pre- to post-EBPMP change in measurement for this survey item would not be expected.

While previous mentorship programmes identified EBP mentors as beneficial for institutional readiness and support from nursing leadership (Spiva et al., 2017; Wallen et al., 2010), results from this study specifically identified where mentors are the most beneficial in the EBP process. Steps involving literature critique and hierarchical categorisation were the most time consuming and difficult elements of the programme. Despite this finding, post-EBPMP survey and interview results demonstrated nurse participants’ increased knowledge and improved their abilities to perform these steps. Programme participants identified their mentors as being most needed and helpful while learning and performing these complex steps of the EBP process, adding to the body of literature demonstrating the benefit of EBP mentors.

Though findings of this programme evaluation indicated that EBPMP increased participants’ knowledge and implementation of EBP, only four mentees from the pilot cohort continued in the mentor role. This may reflect the fact that conducting EBP is a skill acquired through experience and may not be mastered through a one-time learning opportunity. As such, the EBPMP leadership looked for additional ways to foster development of mentors over time. Programme modifications included graduates undertaking additional peer mentorship opportunities, sponsoring future EBP projects and implementing smaller unit based EBP working groups. Graduates now have many opportunities to hone their EBP skills by co-mentoring future EBPMP participants.

Through the qualitative interview participants indicated that time was a barrier to EBP. To address time as a barrier, a new process was developed for nurses to apply for programme entry. Through this new process applicants meet with their nurse managers or directors and develop a plan that includes protected time away from patient care to consistently work on their EBP projects throughout the 12-month programme. The application process also provides applicants with the ability to be paired with another mentee to carry out one EBP project under guidance from a single mentor. This 2:1 alignment has facilitated optimum utilisation of a limited mentor pool and allowed participants to share ideas and responsibilities during critical appraisal, which was notably the most difficult and time-consuming step. Additionally, to facilitate cohort interaction and support participants through critical appraisal, formal workshops for literature critique and hierarchical categorisation were integrated. Regularly scheduled works-in-progress sessions were established to facilitate mentee knowledge sharing, communication and collaboration.

Limitations

Three main limitations of this study are small sample size, lack of participant diversity, and group interviews. The study was powered such that improvements in mean scores between pre- and post-EBPMP surveys of 0.8 SD units had 80% statistical power. Thus, only large effects were expected to achieve statistical significance. Smaller improvements (e.g. increases of 0.5 SD units between the pre- and post-EBPMP surveys) would not likely have achieved statistical significance but can be clinically meaningful. Results should be interpreted accordingly. Another perceived limitation was our heterogeneous sample. Most participants were experienced, bachelor prepared nurses who had received some formal EBP training. What remains unknown is whether novice nurses, or those with no formal training, would have achieved the same outcomes as observed with this pilot cohort. Last, the inclusion of group interviews was a limitation. Group interviews have the ability to influence group dynamics and participants’ answers (Parker and Tritter, 2006). However, the majority of interviews in this study were conducted with single individuals.

Conclusions

Findings from this study suggest that EBPMP can improve participants’ knowledge and implementation of EBP, in an environment that values EBP. This study adds to the emerging body of literature demonstrating the effectiveness of a hospital-based programme designed to meet the needs of direct care nurses. Results were useful in making programme modifications designed to improve the participant experience. Ongoing evaluation of the participant experience is needed to determine whether further programme modifications are warranted.

Key points for policy, practice and/or research

• The Evidence-based Practice Mentorship Program improved participants’ knowledge and implementation of evidence-based practice.

• Nurse mentees perceived their mentors as supportive and appreciated their availability and willingness to share knowledge and practice insights.

• The programme can benefit direct care nurses within an institution that values evidence-based practice.

• Works-in-progress sessions were established to facilitate mentee knowledge sharing, communication and collaboration addressing the need for more participant interactions.

Biography

Ethan Schuler, Critical Care Paediatric Nurse Practitioner, Chair of the Evidence-based Practice Committee and member of the Nursing Executive Committee for Research and Inquiry at Boston Children’s Hospital.

Sandra Mott, Distinguished and celebrated nurse who has led the advancement of paediatric nursing science through her many roles as clinician, educator, researcher, leader, mentor, consultant, nurse scientist and author.

Peter W Forbes, Senior biostatistician in the Clinical Research Center at Boston Children’s Hospital providing biostatistical support for several departments including Psychiatry, Nursing and Haematology.

Alexis Schmid, Paediatric emergency nurse and global health fellow at Boston Children’s Hospital, with research interests in the health needs of children in humanitarian crises and development of emergency care systems.

Carole Atkinson, Advanced Practice Nurse at Boston Children’s Hospital for over four decades, while also precepting, mentoring, educating and leading as past chair of the evidence-based practice committee.

Michele DeGrazia, Director of Nursing Research, Neonatal Intensive Care Unit, Nurse Executive Committee for Research and Inquiry co-chair, Boston Children’s Hospital and Assistant Professor of Paediatrics, Harvard Medical School.

Michele DeGrazia, Director of Nursing Research, Neonatal Intensive Care Unit, Nurse Executive Committee for Research and Inquiry co-chair, Boston Children's Hospital and Assistant Professor of Pediatrics, Harvard Medical School.