Knowledge and Attitudes of Oman Medical Specialty Board Residents towards Evidence-Based Medicine

Abstract

Objective

This study aims to evaluate the knowledge and attitudes of Oman Medical Specialty Board (OMSB) residents towards Evidence-Based Medicine (EBM).

Methods

This cross sectional study was conducted on all OMSB residents through a self-administered online questionnaire between October 2012 and March 2013. An electronic survey was designed to identify and determine residents' knowledge and attitudes toward the use of EBM.

Results

The survey was completed by 93 (21%) OMSB residents, 76 (82%) of whom took part in continuing education courses and 50 (54%) belonged to professional practice-oriented organizations. On average, the residents were reportedly involved in patient care for approximately 70% (Standard Deviation [SD] 17%) of their time, while 14% (SD 12%) participated in research activities. The results showed that 53 respondents (57%) were competent users of medical search engines compared to 23 residents (25%) who rated their skills as neutral. Sixteen percent of the respondents strongly agreed and 46% only agreed that the facility supports the use of current research in practice. Fourteen percent strongly agreed and fifty-three percent only agreed that the foundation of EBM is part of OMSB academic preparation. On the other hand, 17% of the respondents thought that insufficient time is always a barrier against EBM, while another 27% perceived insufficient time as a usual barrier. The lack of information resources was reported to always be a barrier in 11% of the respondents while 32% thought that it usually acts as a barrier.

Conclusion

Time constraints and skills in EBM were found to be the two major obstacles. This study was, however, limited by the low response rate of the survey; thus larger studies with a previously validated questionnaire should be conducted in the future.

Introduction

Evidence-based medicine (EBM) is "the process of systematically reviewing, appraising and using clinical research findings to aid the delivery of optimum clinical care to patients".¹ The formal introduction of EBM into medical school curricula and medical boards is important for the vast majority of physicians. The primary method of building up their expertise in EBM largely depends on educational activities during their postgraduate training.² The need and promotion of better patient care has contributed towards the growing trend of teaching and implementing the use of EBM in the past decade.^3-5 A review of residents’ performance has suggested that learning how to practice EBM and adopting evidence-based practice protocols developed by respected organizations can keep them ahead of medical advances and help to enhance their clinical performances.⁶ EBM practice also supports decision-making shared with users, which is already favored within the medical community as the ideal form of decision-making.^7-9 Many studies have been conducted worldwide to investigate the attitudes and knowledge of different health professionals including physicians towards EBM.

To the best of our knowledge, no similar studies have been carried out in Oman. Therefore the goal of this study is to evaluate the knowledge and attitudes of Oman Medical Specialty Board (OMSB) residents towards EBM. The results of this study would be the basis for similar future comparative studies and to formulate suggestions to incorporate a wider application of EBM in practice.

Methods

This cross-sectional study was conducted on OMSB residents of different specialties in Oman. The self-administered questionnaire was submitted to all residents through their OMSB e-mails, and the survey was administered during a study period from October 2012 to March 2013. The survey was designed as an electronic survey using Google documents and all information were automatically backed up on Google servers after being completed and submitted by the residents. The survey was sent to the residents four times as reminders and to encourage more residents to participate in the study. This study was approved by the OMSB administration and data confidentiality was assured.

The questionnaire was designed to incorporate four main sections. The first section included personal and professional characteristics of the residents: age, gender, medical specialty, postgraduate level, participation in continuing education courses, belonging to professional practice-oriented organizations, number of calls and patients seen per month, total work time spent by residents in different activities such as patient care, search and education as well as familiarity with medical research engines. The second section addressed the residents’ knowledge pertaining to the following: access to current research, access to relevant databases and internet at work and at home, facility supports use of current research in practice, foundations for EBM as part of academic preparation and receiving formal training in search strategies and in critical appraisal. This section was used to measure the extent of residents' confidence towards critically reviewing professional literature and finding relevant research to answer clinical questions. It was also used to determine their understanding of the relative risk, absolute risk, systematic review, odds ratio, meta-analysis, confidence interval, heterogeneity, publication bias, p value, randomization, concealment of allocation, intention of therapy, random error, systemic error, likelihood ratio and multivariable regression. This section was additionally used to identify any barriers which may affect the use of EBM, for instance: insufficient time, lack of information, resources, poor ability to critically appraise literature, lack of research skills, lack of ability to generalize it to patients, inability to apply findings to patients, lack of understanding of statistical analysis, lack of collective support among colleagues and lack of interest. The third section of the survey focused on the residents’ attitudes towards EBM in terms of: the importance of EBM in practice, interest to use EBM, improve quality of patient care and making decisions in management of patients. The final section of the questionnaire was composed of 9 items used to evaluate the residents’ skills in using EBM.

The statistical program SPSS 20 (SPSS for Windows, Rel. 20.0.0. 2011. Armonk, NY: SPSS Inc) was used to perform data and statistical analyses. Continuous variables were presented as means, standard deviation (SD) and ranges. Responses to different questions were grouped according to their domain and presented as frequencies and percentages, and plotted as bar charts. The internal consistency was quantified as a measure of reliability for the questionnaire using Cronbach’s α. Three methods of validation of the questionnaire were used; these included face, content and construct validity. The questionnaire was reviewed and approved by all investigators for face validity. While for the content validity, extensive literature review was done and a number of previous studies were retrieved and contents were reviewed in order to be considered for inclusion in the current questionnaire.

Five hypotheses were formulated to test the knowledge domain of the questionnaire in testing the construct validity. These five constructs stated that senior residents, internal medicine and pediatric residents, residents involved in research, residents who learned research and residents who practice EBM will score higher in the knowledge domain of the questionnaire. The different responses to different questions in the knowledge domain were added towards a total score which was compared between the two different groups using the Student t test or Mann-Whitney test if the distribution was not normal. All comparisons were performed using 2-sided tests and an alpha error threshold of 0.05.

Results

In this study, only 93 (21%) of the total 450 residents at OMSB completed the questionnaire. The study participants were composed of 46% males. Tables 1 presents the breakdown of demographic data of the respondents according to their level and specialty, as well other demographics. Of all the respondents, 76 (82%) were undertaking continuing education courses, while 50 (54%) were part of professional practice-oriented organizations. On average, 71 residents (77%) had approximately 4 to 5 calls per month compared to 14 residents (15%) and 8 residents (9%) who reported to have had 3 and 6 or more calls per month, respectively. The responses also indicate that 42 (45%) residents attended to 10 or more patients per month, while 37 residents (40%) reported to have seen 6-9 patients per month, and 14 (45%) residents reported to have seen around 3 patients per month. Residents dedicated approximately 70% (SD: 17%) of their time towards patient care and around 14% (SD: 12%) towards research activities. Fifty-three (57%) residents reported to be familiar with medical search engines compared to 23 (25%) who reported to be neutral with regards to this particular question.

Table 1. Demographic characteristics of the studied sample of OMSB residents (n=93).

Variable	Frequency	Percent
Gender
Female	50	54
Male	43	46
Age (years)
20-25	6	7
25-29	61	65
≥30	26	28
Specialty
Anesthesia	7	8
Biochemistry	6	7
Dermatology	9	10
EMERGENCY	5	5
Family Medicine	20	22
Hematology	3	3
Internal Medicine	25	27
Microbiology	2	2
OB & GYN	1	1
Ophthalmology	1	1
Orthopedic	1	1
Pediatrics	4	4
Psychiatry	5	5
Radiology	3	3
Surgery	1	1
Level
R* 1**	35	38
R2**	22	24
R3**	8	9
R4**	17	18
R5**	7	7
R6**	4	4
Participation in continuing education courses
No	17	18
Yes	76	82
Belong to professional practice-oriented organizations
No	43	46
Yes	50	54
Number of calls per month
3	14	15
4-5	71	76
6	8	9
Number of patients per month
5	14	15
6-9	37	40
10	42	45
Total work time spent by residents in activities
Patients care
Mean	70	-
Median	70	-
Std.Deviation	17	-
Range	89	-
Research
Mean	14	-
Median	10	-
Std.Deviation	12	-
Range	50	-
Teaching
Mean	16	-
Median	10	-
Std.Deviation	12	-
Range	70	-
Familiar with medical research engines
Strongly Agree	11	12
Agree	42	45
Neutral	23	25
Disagree	11	12
Strongly Disagree	6	6

* R: Resident; ** 1-6: years in training program

Knowledge

In terms of the knowledge domain, it was observed that 16% of the respondents strongly agreed with the notion that "their facility supports the use of current research in practice", while 46% only agreed with this notion. On the other hand, the results revealed that only 14% of the study participants strongly agreed with the idea that "the foundation for EBM is part of OMSB academic preparation", however, 53% only agreed with that idea. More than 50% of the respondents attested to have had received formal training in both search strategies and critical appraisal. Moreover, 53% agreed that they were indeed confident to find relevant researches to answer any clinical question, and 28% only agreed with the notion that "they are confident to critically review professional literature". Fig. 1 presents the distribution of responses related to training and mastering of EBM.

Figure 1.

Training and mastering EBM. (Q= Question)

The results also show that only 53% of the respondents reported to have had access to current research, while a relatively higher proportion of the respondents reported to have had access to relevant databases and internet at home (80%) than at work (68%). The distribution of responses pertaining to the awareness and availability of EBM showed that 65% of the respondents indicated that "practice guidelines are available for topics related to their practice" and 69% asserted themselves as being able to access practice guidelines online. However, it was apparent from the results of this survey that only 77% of the respondents indicated that they were "aware that practice guidelines were available online". The residents’ self-evaluated knowledge of specific terms associated with EBM is described in Fig. 2.

Figure 2.

Self-reported knowledge of specific terms.

Barrier, attitude, EBM activities and skills

The section of the survey pertaining to barriers, attitude, EBM activities and skills revealed that 17% of respondents viewed "insufficient time" as always being a barrier against EBM, while 27% considered it to be a usual barrier against EBM. Another aspect which was perceived to always be a barrier against EBM included "lack of information resources" which was reported by 11% of the respondents, while 32% thought that "lack of information resources" usually acts as a barrier against EBM (Fig. 3). Other important factors which were perceived to act as barriers against EBM included "lack of understanding of statistical analysis" and "lack of research skills" which were reported by 30% and 39% of the respondents, respectively.

Figure 3.

Self-reported barriers to EBM.

The distribution of responses focusing on the residents’ attitudes towards EBM is presented in Fig. 4 which shows that the respondents generally held positive attitudes towards EBM, with the majority contending that they strongly agreed and agreed in the following dimensions: that EBM is necessary for the practice of patient management (51% and 41%, respectively); literature and research findings are useful in practice (46% and 43%, respectively); the need to increase the use of EBM in daily practice (63% and 32%, respectively); interested in learning or improving skills to incorporate EBM (62% and 33%, respectively); and that EBM improves the quality of patient care (70% and 24%, respectively). Moreover, 29% of the respondents disagreed with the view that the adoption of EBM places an unreasonable demand on patient management, while a further 29% of the respondents were neutral in view of this implication. On the other hand, the responses indicate that 28% of the respondents agreed that strong evidence is lacking to support most of the interventions, whereas 24% disagreed with the notion in question and 34% of the respondents were neutral.

Figure 4.

Self-reported attitudes about EBM.

The results also suggest that the majority of respondents reported a practice of reading fewer than two articles per month (83%), 57% stated using professional literatures in the process of clinical decisions between 2-5 times per month, with only 15% making clinical decisions more than 5 times per month. The data expresses that 48% of the respondents performed database searches between 2-5 times per month compared to 23% who reported to perform database searches more than 5 times per month.

In highlighting the self-reported skills of OMSB residents in EBM (Fig. 5), the data indicates that 52% of the respondents agreed and 12% strongly agreed with the assertion of seeking practice guidelines for areas pertaining to their practice. In terms of the use of guidelines in practice, 43% of the respondents agreed with this notion compared with only 10% who strongly agreed with the implication and 31% who were neutral in their stance. Generally, more than half of the respondents (54%) were neutral in response to the question as to whether they are able to incorporate patient preferences with practice guidelines or not, compared with 25% who agreed and 10% who disagreed.

Figure 5.

Self-reported skills.

Reliability and validity

Cronbach’s α was estimated at 0.86, which indicates a good reliability index for internal consistency. We demonstrated the validity of three out of the five constructs. The results were consistent with the first construct that "senior residents do better in understanding medical terms than junior residents" (p=0.002). The difference in the total score between internal medicine and pediatric residents on one side and other specialties on the other in understanding the same medical terms was not statistically significant (p=0.520), which is not consistent with the preformed construct. In addition, the results of the study were not consistent with the third construct that implied the residents involved in research would score better (p=0.171). However, residents who reported to have learnt research methods scored higher than those who had not (p=0.002), which is consistent with our construct. Moreover, the results revealed that residents who practiced EBM scored higher than those who did not (p=0.002), again consistent with our construct.

Discussion

This is the first paper to evaluate knowledge and attitudes of OMSB residents towards EBM. The overall participants' response was positive, despite the fact that the present survey demonstrated that a sizable proportion of residents lack adequate knowledge of basic concepts of EBM. The majority of the study participants continue to use traditional sources of knowledge and, in spite of access, they do not use evidence-based sources sufficiently. Nowadays, there is a worldwide plethora of books, workshops and courses on how to practice and teach EBM. Electronic and online databases of systemic reviews and summaries of EBM are increasingly becoming available for all,^10-12 but many OMSB residents do not make use of these resources in clinical decision-making as per our study. Recent studies have suggested that most physicians still depend heavily on the opinion of colleagues or seniors when making management decisions.^13-16

The responses varied in terms of the residents’ attitudes towards the statement implying that the adoption of EBM places an unreasonable demand on patient management and strong evidence is lacking to support most of their interventions. The study participants exhibited very low activity in research with a mean of 14%, compared to patient care and teaching, which scored 70% and 16%, respectively. That can be explained by the fact that the residents are in a training program focusing more on patient care. Also, these numbers may not be representative due to the low response rate. We found that a large proportion of residents reported not being confident in critically reviewing professional literature. Lack of good formal training and practicing role models may explain such finding,¹⁷ nonetheless, this is unlikely in a postgraduate and university settings. Incorporating EBM training into the half days and regular journal clubs in the residents’ curriculum would likely improve the residents’ confidence in critical appraisal.^18-21 Brent et al.²¹ concluded in their study that faculty development programs for community should feature EBM and the curriculum should be planned in a helpful way to be used and taught. Despite the importance of EBM, the majority of physicians are still not fully aware about its values as found by Nadira et al.²²

Understanding the concepts and methods commonly used in EBM is a key element in practicing EBM. A large proportion of residents indicated that they do not understand multivariable regression. This may be acceptable for difficult concepts for a junior resident but the expectation from a resident close to graduation is to master these concepts. In our study cohort, senior residents gave better responses, which may reflect more EBM related activities towards later years of training. In general, it is still below expectations if we consider the notion that most researchers agree with the importance of EBM and its effect on intervention management of patients.^22-24 Different guidelines may have been available at different hospitals in which the residents were rotating. It is advised that all guidelines should be evidence based and should be written in a clear and concise manner.²⁵ Many reasons could lead to underutilization such as the lack of well-organized library which is easily accessible to all the residents without long waiting queues. Also, the lack of internet access in some of the affiliated hospitals could be another limitation of practicing EBM.

Lack of understanding of statistical analysis was a major barrier to EBM and that could be due to the lack of EBM training courses and workshops which was also perceived to be a barrier to practicing EBM by the residents. The most appropriate way to move from opinion-based medicine to EBM is through focusing on training in critical appraisal of research. Lack of emphasis on EBM was a major problem which has been revealed in other studies where the lack of familiarity with EBM was a demotivating factor to practice.^26,27 Lack of technical skills to appraise studies was shown to be a factor in a similar study.²⁸ Thus, it has been suggested that changes in practice as a result of understanding EBM occur more frequently when barriers to change are addressed and resources deployed to help the learners.²⁹ Lack of resources, like internet access and library as mentioned above, as well as appraisal skills with lack of time are all seen to be key factors in delaying improvement and implementing EBM; however, it could also be as a result of the high frequency of call duties, program activities such as exams, presentations and other activities. Lack of workshops, EBM courses and residency program curriculum which encourage residents to practice EBM is a barrier in practicing and understanding EBM and applying it in patient management.^1,21 Similarly to our study, Al Shawwa advised that workshops should match ambition and should possess high standards to attract clinical faculty members and to improve their ability. Also, workshops should be interactive and focus on certain specific problems faced by them.³⁰

The major limitation in our study is the small sample size and the low response rate of only 21%. Attempts were made to improve the response rate by sending multiple reminders but this helped only marginally. The low response rate may have affected the generalizability of the results particularly in view of the fact that the responses may differ between subjects who respond and those who do not. The other limitation was the use of a new questionnaire that has not been previously formally validated. However, we were able to validate it in our resident population. We were also able to demonstrate the face validity by having all investigators review the questionnaire, including two instructors who teach statistics and evidence-based medicine. We also looked into many other questionnaires addressing similar interests and incorporated some items and domains into our study questionnaire. Unfortunately, this resulted in a very long questionnaire which may have contributed to the low response rate. No factor analysis was performed for this study but it is planned as a separate project to produce a validated and relatively shorter questionnaire for future use by others. Two of our constructs could not be validated; however, the results were consistent with the other three constructs. In addition, another limitation may have been the fact that residents tend to over-report their use of journals, online databases and practicing EBM or they chose the "neutral" response, perhaps related to their tendency toward taking a neutral stance. The study population, however, covered all OMSB specialties which may be considered to be a strength of the study.

Conclusion

Evaluation of EBM should be carried out on a regular basis to evaluate the effectiveness of the teaching modules and to modify the curriculum accordingly. Suggestions for future research include studies to examine the actual processes through which evidence is gathered, synthesized and applied by clinicians across various specialties. Also, we recommend encouraging the use of EBM by taking further steps to improve access, utilization of internet-based evidence databases and address the barriers.