Refining knowledge, attitude and practice of evidence-based medicine (EBM) among pharmacy students for professional challenges

Abstract

Practicing evidence based medicine (EBM) is a professional need for the future clinical pharmacist in UAE and around the world. An attempt was made to evaluate pharmacy student’s knowledge, attitude and proficiency in the practice of EBM. A within-subject study design with pre and post survey and skill test were conducted using case based practice of EBM through a validated questionnaire. The results were tabulated and there was a statistically significant increase in pharmacy students’ perceived ability to go through steps of EBM, namely: formulating PICO questions (95.3%), searching for evidence (97%), appraising the evidence (81%), understanding statistics (78.1%), and applying evidence at point of care (81.2%). In this study, workshops and (Problem Based Learning) PBLs were used as a module of EBM teaching and practices, which has been shown to be an effective educational method in terms of improving students’ skills, knowledge and attitude toward EBM. Incorporating hands on experience, PBLs will become an impetus for developing EBM skills and critical appraisal of research evidence alongside routine clinical practice. This integration would constitute the cornerstone in lifting EBM in UAE up to the needed standards and would enable pharmacy students to become efficient pharmacists that rely on evidence in their health practice.

1. Introduction

Evidence based medicine (EBM) was established as a method for guiding health care providers including clinical pharmacists throughout the patient care process (Ilic and Forbes, 2010). It refers to the appropriate procedure of using and evaluating information to make sounding clinical decisions in the field of practice (Bernstein, 2004). Advancement of medication knowledge in conjunction with huge increase in relevant literatures has given rise to clinical researches that are variable in quality and clinical significance. This has led to the emergence of evidence-based medicine (EBM) as the new paradigm for clinical practice (Alahdab et al., 2012). It involves integrating individual clinical expertise with the best available external clinical evidence and use of individual patient values and preferences in making clinical decisions about patient care (Haynes et al., 2002).

Assuming the importance of continuous professional development in clinical pharmacy, EBM is necessary for all therapy related aspects (1994). In an attempt to teach undergraduates the way of selecting and using the best available evidence when dealing with their patients in order to meet their related needs, EBM principles have been progressively introduced in medical curricula, mainly in the USA (Aiyer et al., 2002). There is an increasing focus on real-time practice of EBM skills in medical education programs (Health Professions Education: A Bridge to Quality, 2003; Alahdab et al., 2012; Swing, 2002).

Therefore, many studies have been conducted to evaluate medical students’ awareness and their attitudes toward EBM in the general practice (Al-Almaie and Al-Baghli, 2004). However, effectiveness of teaching and practice EBM skills should be evaluated and assessed among pharmacy students as they are becoming members of future health care professionals.

We aimed at this study to assess students’ knowledge, attitudes and skills toward EBM practice by enrolling them in an intensive didactic course and problem based learning for 2 weeks constitutively followed by skill test. Moreover, this study aimed to explore the feasibility of initiating and introducing interactive and real-time approach in learning and developing EBM skills.

2. Study design and participants

A within-subject study design with pre and post-course particulars was collected using a validated questionnaire to comprehend the transformation in participant’s knowledge, attitude and skills related to EBM. A test was also administered at the beginning and at the end of the EBM workshop and PBL series. This instrument was used to assess the skills that the learners had gained during the sessions. About 64 undergraduate pharmacy fourth year students at the Dubai Pharmacy College (DPC) participated voluntarily in the study and a verbal consent was taken from all participants. The Dubai Pharmacy College is the first Pharmacy institute in the Gulf region which was established in 1992. The DPC attracts students from all UAE as well as from neighboring countries such as Kuwait, Qatar, Bahrain and Oman.

2.1. Pre and post workshop survey

An EBM course was prearranged by a Clinical Pharmacist expert, who had a professional experience in practicing evidence based medicine. The EBM course comprised a series of interactive large and small group seminars and a series of PBL sessions. The curriculum outline of the course included the following:

• 1.The concept of EBM and how to use it in clinical practice.
• 2.Their exposure to the PICO trend (P: population or patient; I: intervention; C: comparison; and O: outcome) in deriving highly specific clinical questions from clinical problem.
• 3.Identify strategies to search for evidence-based medical literature, including useful web sites.
• 4.Review key factors that should be appraised with systematic reviews, clinical practice guidelines, and individual RCTs.
• 5.Discuss how EBM principles can be applied to evaluating new drugs.

2.2. Real-time practice skill test assessment

A series of PBLs were conducted and the students were given simulated case scenarios both at the beginning and at the end of the didactic courses. The pre-test and post-test were administered on the first day and the last day of the PBL series, respectively and assessed multiple competencies, including formulating a clinical question, using effective strategies to identify the best clinical literature to answer the question, and analyzing the relevance and validity of the retrieved article. A faculty member evaluated the test answers using a competency-based instrument developed to measure critical appraisal skills.

2.3. Data analysis

All data were entered into a Microsoft Excel spreadsheet and exported for analyses using the statistical software IBM SPSS v20.0. ANCOVA test was used for comparing categorical variables as multiple comparisons were made and probability values of p < 0.001 and 0.01 were taken to denote statistical significance. Standard approaches to summarize questionnaire data were used, including frequencies and descriptive summaries for categorical data, and means, ranges, standard deviations for numerical data.

3. Results

All sixty four senior pharmacy students who attended the workshop completed the questionnaires (response rate of 100%). Basic characteristics of participants are described in Table 1.

Table 1.

Participant characteristics.

Variable		N	%
Gender	Female	64	100
Previous EBM exposure	Read about or attend a lecture	63	98.4
	Had a formal EBM training	1	1.6
Previous research exposure	Participated in 1 or more research activity	30	46.9
	No previous research exposure	34	53.1
Frequency of online search for evidence	⩾Once per week	19	29.7
	<Once per week or Not at all	45	70.4
First source sought for evidence (before the course)	Google	55	85.9
	Up to date	1	1.6
	PubMed	5	7.8
	The Cochrane library	1	1.6
	Other	2	3.1
First source sought for evidence (after the course)	Google	6	9.4
	Up to date	4	6.2
	PubMed	51	79.7
	The Cochrane library	0	0
	Other	2	3.1

The assessment of the participants’ previous exposure to any training in evidence-based medicine showed that 63 students (98.4%) had read about it and one student (1.6%) attended a lecture on EBM.

Majority of the students (85.9%) reported that the first source sought for evidence before the course was Google, this percentage has been dropped 9.4%. The interest toward the use of more specialized databases has been increased dramatically particularly for PubMed as shown in Table 1.

As part of our assessment of the efficacy of the learning module used in our EBM course, we aimed to measure the increase in self-reported understanding and knowledge of different concepts of EBM. Thus, our questionnaires included a set of questions about the self-reported ability of pharmacy students to go through the steps of EBM in both pre- and post-course questionnaires. Table 2 summarizes this perceived knowledge before and after the course.

Table 2.

Perceived knowledge in EBM before and after the course.

EBM knowledge components	Easy/relatively easy		Difficult/relatively difficult
	Pre (%)	Post (%)	Pre (%)	Post (%)
Formulating PICO	1(1.6)	62(95.3)⁎⁎	63(98.4)	2(3.2)
Skills of searching online	44(68.7)	62(97)⁎	20(68.7)	1(1.6)
Evaluating study design	25(39.1)	52(81)⁎⁎	39(69.7)	11(17.2)
Assessing bias	13(20.3)	49(76)⁎⁎	51(79.7)	14(21.9)
Sample size calculations	34(53.1)	57(90)⁎	30(46.9)	6(9.3)
Handling statistics	26(40.6)	50(78.1)⁎	8(59.4)	13(20)
Applying evidence at clinical point of care	16(25)	52(81.2)⁎⁎	48(75)	11(17.2)

ANCOVA significant difference.

^⁎⁎p value < 0.001.

^⁎p value < 0.01.

Students’ self-perceived ability of formulating PICO questions showed that only 1.6% of the students thought they could do so easily or relatively easily before taking the course, compared to 95.3% of students after the course, an impressive statistically significant increase of 93.7% (p < 0.001). When it came to the students’ skill of searching online for relevant evidence, 68.7% reported that this was easy or relatively easy before the course, compared to 97% after the course, an increase of 28.3% (p-value < 0.01). Questions about skills in evaluating studies for strengths and weaknesses before and after the course showed an impressive increase in self-confidence. For instance, students’ self-assessment of this ability to assess the suitability of study designs easily or relatively easily had increased by 41.9% (p-value < 0.001).

Regarding the students’ confidence in assessing biases in research studies, 20.3% of students before the course, compared to 76% after the course thought it was easy or relatively easy, an increase in confidence of 55.7% (p-value < 0.001) is reported.

When asked about sample size calculations prior to the course, students thought themselves as being able to easily or relatively easily decide if a particular sample size was suitable for a study, which increased by 36.9% upon finishing the course (p-value < 0.01). In regard to statistics, a usually difficult area for students, only 40.6% before the course reported that they could handle statistics of research easily or relatively easily, but after taking our course the percentage increased to 78.1%, a 37.5% increase (p-value < 0.01). When the students were asked about their abilities in applying the evidence at the clinical point of care, 25% before the course, compared to 81.2% after the course, thought this was easy or relatively easy, an increase of 56.1% (p-value < 0.001). The mean increase in self-reported knowledge and confidence in EBM concepts mentioned above suggests that the students tend to overestimate their skills in EBM after completing the course.

3.1. Attitudes and perception toward EBM before and after the course

Students’ answers to questions probing their attitudes toward EBM before and after the course showed that the course corrected many of their previous false impressions they had about EBM. Details of the students’ response are summarized in Table 3. When asked if EBM has a weak effect on the practice of pharmacy, there was no significant change in the student perception in this regard. A slight change in students’ perceptions toward the importance of clinical experience and the patient desires over EBM was noticed. When asked if EBM was merely a passing fashion and that it would disappear soon, 11% of the students agreed with this statement before the course, and 18.7% of them agreed with it after taking the course.

Table 3.

Attitudes and perception toward EBM before and after the course.

EBM attitude component	Total agreement		Total disagreement
	Pre	Post	Pre	Post
EBM has a weak effect on the practice of pharmacy	19(29.7%)	18(28.1%)	45(70.3%)	45(70.3%)
Clinical experience is more important than evidence	44(68.7%)	33(51.6%)	20(31.3%)	30(46.9%)
Patient desires are more important than medical evidence	34(53.1%)	31(48.4%)	30(46.8%)	32(50%)
EBM is merely a passing fashion and that it would disappear soon	7(11%)	12(18.7%)	57(89.1%)	51(79.7)
Systematic reviews important in decision-making	51(79.6%)	56(87.4%)	13(20.3%)	7(10.9%)

A slight change in pharmacy students’ attitudes toward EBM was also evident in the question they were asked about the importance of systematic reviews in clinical decision-making, with 79.6% of them thinking it was essential in EBM before taking the course and a full 87.4% after.

3.2. Pre- and post-test

Before and after a series of PBL sessions students were given case studies to apply EBM skills and students were evaluated through a pre-test and then post-test. The case studies were graded on 5 different items pertaining to EBM criteria with a possible score range of 0–26 points. The mean scores for both tests are reported in Table 4. In pre-test and post-test evaluation it was found that the student skills to find out their clinical question, logically arose from the case study and it was clear and focused. Therefore there was no significant difference in their scores, and they scored 3.64 and 3.78 in pre and post-test respectively out of 4 points. Similarly, their skills to justify the research methodology and study design were accurate and they scored 2.32 and 2.78 in pre and post-test respectively out of 4 points. The students scored low in EBM components like search strategy, critical appraisal of the evidence and appropriateness of evidence with 3.26, 2.76 and 3.42, respectively in pre-test out of 6 from each component. Therefore there was a significant difference in the score for these components with p values <0.01 and <0.001.

Table 4.

Comparison of pre- and post-tests.

EBM attitude component	Total Score	Total agreement
		Pre	Post
Logical research question	4	3.64	3.78
Search strategy	6	3.26	5.85⁎
Research design	4	2.32	2.78
Critical appraisal	6	2.76	4.45⁎⁎
Appropriateness of Evidence	6	3.42	5.76⁎⁎
	26	15.4	22.61⁎⁎

ANCOVA significant difference.

^⁎⁎p value < 0.001.

^⁎p value < 0.01.

4. Discussion

This study was designed to measure how effective it would be to expose undergraduate pharmacy students to EBM principles and practice, in terms of developing a positive attitude toward EBM and improving EBM knowledge and skills. There are different modules of teaching EBM to students as reported in the EBM literature; these modules range from workshops, morning reports and journal clubs, to the systems-based method, the problem based method, or the integration in the curriculum (Green and Ellis, 1997; Meats et al., 2009; Reilly and Lemon, 1997; Tamim et al., 2009). In our study, we used workshops as a module of EBM teaching, which has been shown to be an effective educational method in terms of improving students’ skills (Cartwright et al., 2002; Taheri et al., 2008).

The pre- and post-course results of our study indicate that our EBM workshops positively changed undergraduate pharmacy students’ beliefs and attitudes toward EBM and improved their perceived knowledge and skills about EBM. Additionally, students have shown a great interest and desire to incorporate teaching of EBM in their undergraduate curricula. Teaching EBM in UAE is hindered by several challenges. One of them is the need of ‘role models’ for the students to see EBM being applied on the wards by a variety of senior pharmacists and the need to promote EBM ‘culture’ in the clinical setting. Changing the behavior of such pharmacists by urging them to use EBM methods more frequently will have a significant impact on their junior pharmacists as well as the students, which in turn will have a direct effect on improving the quality of care in the future.

The areas in which the students scored the lowest on the skill test were the search strategies, appraisal of literature and appropriateness of the evidence. Actually, students were not exposed to pharmacy informatics before giving the EBM course, and thus it was a weakness in the student’s approach in finding the drug information and accordingly, they preferred Google as a first source of information instead of PubMed or other authenticated scientific websites. However, after giving EBM course to the students, their approach changed and about 90.6% of them used PubMed, Up-to-date, a premier evidence-based clinical decision support resource. As a result of this study, new course of “Pharmaceutical Informatics” was included in B. Pharm curriculum.

On the other hand, students’ knowledge on handling statistics was not remarkably improved. Only 78% of the students felt easy or relatively easy in handling statistics after the course. Since the students had just basic statistic course during their first year of B. Pharm program, it was difficult for them to recall those skills during this study. Anyway after this study, the curriculum was changed and Biostatistics course was introduced with advanced contents to the students in final year.

Even though gains were small in these areas, students might be expected to improve with cumulative clinical exposure and a more integrated, longitudinal EBM curriculum.

It is sometimes difficult to convey the importance and impact of EBM to pharmacy students, hence the need and importance of conducting foundation courses that can familiarize students with the key concepts and motivate them to learn and practice EBM. The teaching and practice of EBM obviously need certain resources. Rapid and convenient access to valid and relevant evidence on a portable computing device has been shown to improve learning in evidence-based medicine, increase current and future use of evidence, and boost students’ confidence in clinical decision making (Leung et al., 2003).

The main limitation of our study is the students’ self-assessment of their attitudes, rather than direct measurement of behavior, and thus it is not known whether those self-reported change was translated into actual changes in behavior. This study also lacks data on whether improvements in attitudes were sustained over time. Other limitations included small sample size and inability to provide internet access for all students to search for evidence.

Alternatively, to teach the students online search, we asked them to build a search strategy, and we applied the search strategy on a big screen with internet access.

5. Conclusions

To date, EBM skills have not been traditionally covered in undergraduate or pharmacy students in UAE. The road to a better EBM reality in UAE starts with teaching EBM in pharmacy school and developing the proper environment to facilitate transforming current medical education and practice to an evidence-based standard in UAE. Based on our study, integrating EBM teaching in undergraduate B. Pharm curriculum in UAE would be feasible and might imply a better EBM practice in the future. This integration would constitute the cornerstone in lifting EBM in UAE up to the needed standards and would enable pharmacy students to become efficient pharmacists that rely on evidence in their health practice.