Evidence-Based Practice Knowledge, Perceptions, and Behavior: A Multi-Institutional, Cross-Sectional Study of a Population of U.S. Dental Students

Abstract

The aim of this study was to help inform faculty and curriculum leaders in academic dental institutions about the knowledge, skills, perceptions, and behavior of an institutionally diverse population of dental students with respect to evidence-based practice (EBP). A survey utilizing the validated Knowledge, Attitudes, Access, and Confidence Evaluation instrument developed by Hendricson et al. was conducted in 2012 with fourth-year dental students at seven geographically dispersed U.S. dental schools. The survey addressed elements of EBP knowledge, attitudes toward EBP, behavior in accessing evidence, and perceptions of competence in statistical analysis. A total of 138 students from the seven schools participated. A slight majority of these students correctly responded to the knowledge of critical appraisal questions. While the students demonstrated positive attitudes about EBP, they did not report high levels of confidence in their critical appraisal skills. The findings also showed that the students accessed various sources of evidence with differing frequencies. The most frequently accessed resources were colleagues, the Internet (excluding Cochrane Database of Systematic Reviews), and textbooks. The results of this study help to identify areas for improvement in EBP education in order to advance dental students’ preparation to become evidence-based practitioners.

A 2013 Institute of Medicine report on health care in the U.S. concluded that “available knowledge is too rarely applied to improve the care experience,”¹ even when evidence-based treatments for prevention and management are known^2,3 and widely disseminated. While most dentists believe they practice evidence-based dentistry (EBD), the reality is that dentistry lags behind many medical professions in fostering the uptake of research-informed treatments.^4-6 Prevailing surgical care models, for example, are in stark contrast to the American Dental Association's (ADA) wide dissemination of clinical practice guidelines for the use of fluoride and pit-and-fissure sealants.^7-9 Resistance to the uptake of evidence-based clinical care in preventive measures suggests dissemination of best practices are lacking in approach, negated by outside forces (economic or otherwise), or merely not sufficient to change clinician behavior.⁵

Previous studies have shown that access to information alone appears to have little impact on practitioners’ performance.^10,11 In fact, Bonetti et al. found no association between knowledge and behavior.¹² Academic dental institutions’ customary use of passive learning environments, typified by memorization and repetition of isolated facts, not only fails to develop students’ ability to become critical thinkers and lifelong learners, but also fails to prepare them to address the future needs of their communities and practices.¹³ If knowledge is an essential but insufficient precursor to adoption of an evidence-based approach to dentistry, what can we, as dental educators, do in order to facilitate the uptake of available evidence to improve patient outcomes?

To answer this question, we need to first understand the current knowledge, skills, perceptions, and behavior of dental students with respect to evidence-based practice (EBP). Such understanding will help us to determine if our current teaching of EBP is effective and, if not, how we can have a greater impact on the attitudes and behaviors of dental students. While the current dental accreditation standards require the implementation of EBP training by dental schools, this mandate focuses on the “acquisition, assessment, and implementation of scientific evidence in the practice of dentistry.”¹⁴ However, the impact of predoctoral students’ “education in EBP on their behavior as practitioners is largely unknown.”¹⁵

To increase our understanding of dental students’ attitudes about and practice of EBP, we conducted a survey of randomly selected dental students enrolled in the final year of the predoctoral program at seven U.S. dental schools using the Knowledge, Attitudes, Access, and Confidence Evaluation instrument¹⁶ to provide a view of four domains that together describe EBP competence. To our knowledge, this study is the first to administer this instrument on a broad scale. The aim of the study was to gain insight regarding the knowledge, skills, perceptions, and behavior of a population of dental students with diverse curricular integration of EBP. We hypothesized that these students had experienced widespread exposure to an evidence-based philosophy of practice and would demonstrate varying levels of knowledge related to critical appraisal, perceived value, and implementation.

Methods

Seven U.S. dental schools that are diverse in geography and curricular integration of EBP were invited to participate in the study: University of Pittsburgh, University of Iowa, Virginia Commonwealth University, University of Colorado, University of Missouri-Kansas City, Loma Linda University, and Nova Southeastern University. Participation was coordinated with a dental faculty member at each institution familiar with the purpose and procedures of the study and responsible for gaining approval from its Institutional Review Board (IRB). Approved IRB protocols are as follows: University of Pittsburgh PRO11050029, University of Iowa 201107798, Virginia Commonwealth University HM14104, University of Colorado 11-1679, and Nova Southeastern University GG2012-4. The University of Missouri-Kansas City and Loma Linda University IRBs deemed this research project exempt from IRB approval. Data use agreements were completed with each institution. The selection of dental schools for this study was purposeful with institutions varying in longevity and philosophy of instruction.

The validated Knowledge, Attitudes, Access, and Confidence Evaluation (KACE) instrument developed by Hendricson et al. is designed to measure the outcomes of training in EBP in the context of dentistry.¹⁶ The KACE instrument has four subscales corresponding to the domains of knowledge of EBP principles, attitudes regarding EBP, behavior in accessing evidence, and confidence in critical appraisal. The KACE instrument is distinctive since it is tailored to a dental context. Hendricson et al. found that the instrument has excellent consistency, capacity to detect differences between individuals with different training or experience, and test-retest reliability. The KACE instrument is provider-focused and offers valuable insight regarding dental students’ perspectives related to EBP. We chose this instrument for our study to capture a high-quality, cross-sectional view of EBP competence in this population of students enrolled in their final predoctoral year (survey is available from the corresponding author or with the Hendricson et al. article).

The KACE instrument is comprised of a total of 35 items in four categories. Unlike other instruments that assess EBP, the knowledge section of the KACE places its questions in a dental context. The ten knowledge questions are designed with one best response for each, including the option of “I don't know.” Respondents’ attitudes about EBP are measured with ten qualitative statements, on which respondents indicate level of agreement on a five-point rating scale. To indicate their behavior in accessing evidence, respondents use a five-point scale to rank access frequency of nine health care resources. To measure their confidence in critical appraisal, respondents use a five-point scale to rate their confidence regarding a list of six components in a published research report.

In 2012, dental students enrolled in the final year of the predoctoral program at each of the seven dental schools were randomly selected and recruited by email. Requests for participation outlined the nature of the research project and provided instructions for completion of the KACE questionnaire through SurveyMonkey. Responses were anonymous with no follow-up contact. Completion of the survey served as informed consent. The process was incentivized by offering $5 Amazon.com gift cards. The cards were awarded through a mechanism separate from the submission of responses to preserve anonymity. Initial email contact invited 20 or more randomly selected students at each school to participate in the study. The random selection methods, which varied by institution, were informed and/or approved by the institutions’ review boards. Rolling enrollment continued at each institution until 18-20 participants fully completed the survey. Additional students were recruited by email as necessary to achieve 18-20 complete surveys from each institution. Data from incomplete surveys were not collected.

Each of the four domains was scored in accordance with the type of questions on each. Each of the knowledge domain's ten multiple-choice questions was graded, and the number of correct answers was determined for each participant with possible values ranging from 0 to 10. On the attitudes domain's ten statements (to which respondents indicated level of agreement), responses were summed for each participant; possible values ranged from 10 to 50, with 50 being most desirable. On the behavior domain (on which respondents indicated access frequency of each of nine evidence sources), responses were summed for each participant; possible values ranged from 9 to 45, with 45 being most desirable. In the confidence domain, respondents’ rankings of their confidence in critical appraisal of six components of a published research report were summed for each participant; possible values ranged from 6 to 30, with 30 being most desirable.

Survey data from SurveyMonkey were compiled for each of the seven institutions. Subscale scores and descriptive statistics, frequency distributions, and graphics were generated using SAS 9.3 (Cary, NC, USA).

Results

A total of 138 dental students in the seven dental schools completed the survey. Six of the seven institutions had 20 complete surveys; one institution had 18 complete surveys. Data were available on 35 questions for these participants.

The mean score of correct responses to knowledge of critical appraisal questions was 5.62 out of a possible 10 (Table 1). Responses regarding attitudes toward EBP were relatively positive (median=37, range 23 to 49). These students accessed various sources of evidence with differing frequencies, and the respondents reported below average confidence in the majority of critical appraisal skills in the KACE (median=18, range 10 to 30).

Table 1.

KACE subscale scores for dental students in study (N=138)

Subscale	Mean	Std Dev	Median	Possible Range	Minimum	Maximum
Knowledge	5.62	2.03	6	0 to 10	0	10
Attitudes	36.96	4.88	37	10 to 50	23	49
Accessing evidence	23.97	4.87	24	9 to 45	9	36
Confidence	18.21	4.02	18	6 to 30	10	30

Knowledge of Critical Appraisal

Searching for evidence, ranking strength of evidence, and judging quality of evidence

A common framework for guiding the search for evidence is defining an information need in terms of the patient, problem, or population, intervention, comparison, and outcome (PICO).¹⁷ In this study, 52.9% of the respondents did not select the statement best defining a PICO model (Table 2). Among these respondents, 44.2% selected “I don’t know.” However, in the context of conducting a search for evidence utilizing PubMed, 66.7% correctly identified the least productive search strategy. When asked to rank the strength of diverse examples of research evidence paired with qualitative statements, 58.0% correctly identified expert opinion as the lowest form of evidence. The respondents demonstrated less discernment in judging the quality of the dental literature, with only 42.8% correctly recognizing a Cochrane review of an oral health topic as the highest level of quality among the sources listed.

Table 2.

Respondents’ knowledge of critical appraisal, by response to each EBP concept (N=138)

EBP Concept	Correct Response	Incorrect Response	“I Don't Know”
1. Ranking of evidence	58.0%	39.1%	2.9%
2. Judging the quality of dental literature	42.8%	54.3%	2.9%
3. PubMed search strategy	66.7%	19.5%	13.8%
4. Number of subjects in a clinical trial	64.5%	31.1%	4.4%
5. PICO	47.1%	8.7%	44.2%
6. Research study scenario	73.9%	21.7%	4.4%
7. Meta-analysis	68.1%	21.8%	10.1%
8. Study design	31.9%	61.6%	6.5%
9. Sensitivity and specificity	65.2%	23.9%	10.9%
10. Prevalence and incidence	44.2%	46.4%	9.4%

Note: Boldface indicates highest percentage for each question.

Understanding of research design

Among the respondents, 47% incorrectly selected a randomized clinical trial as the study design most appropriate to evaluate the efficacy of a new diagnostic device for assessment of oral health problems. This selection was followed in frequency (31.9%) by the correct response of a blind comparison with a gold standard. The majority of respondents (64.5%) were able to accurately associate the number of subjects in a clinical trial with its influence on treatment effects. When provided with a short scenario of a research study, a significant proportion (73.9%) were able to correctly identify factors in the research study that contributed to erroneous results.

Statistical analysis

Meta-analysis was correctly identified by the majority (68.1%) of respondents as a statistical process that quantitatively pools the results of several research studies. In addition, the majority (65.2%) correctly identified statements related to test sensitivity and specificity. A lesser degree of understanding regarding disease prevalence and incidence was evident, with 46.4% of the respondents selecting incorrect statements related to these principles.

Attitudes Regarding EBP and Accessing Evidence

Students in this study overwhelmingly agreed that EBP should be an integral part of the dental school curriculum (agreed 52.9% plus strongly agreed 35.5%), and the majority viewed EBP as a routine part of professional growth as a dentist (agreed 58.0% plus strongly agreed 15.9%) (Table 3). Compared to one year ago, most respondents reported increased support of EBP principles (agreed 51.4% plus strongly agreed 18.1%), as well as an increased belief in the future value of EBP (agreed 57.2% plus strongly agreed 23.9%). Reported belief in the impact of EBD on the way an individual learns varied. However, the majority of respondents reported personal appreciation of the advantages in practicing EBP (agreed 59.4% plus strongly agreed 27.5%) and reported belief that EBP improves the quality of patient care (agreed 58.7% plus strongly agreed 25.4%). While almost half of the respondents expressed difficulty practicing EBD (agreed 37.0% plus strongly agreed 10.9%), just over half reported utilization of EBP was routinely feasible when providing care for patients in the dental school clinic (agreed 42.8% plus strongly agreed 9.4%). Only slightly over half of the students in this study disagreed with the statement “EBP is ‘cookbook’ dental care that disregards clinical experience in providing the best treatment for patients” (disagreed 37.0% plus strongly disagreed 14.5%).

Table 3.

Respondents’ attitudes about evidence-based practice (EBP), by level of agreement with each item (N=138)

Item	Strongly Disagree	Disagree	Uncertain	Agree	Strongly Agree
1. I now believe that evidence-based practice will be more valuable in my future practice as a dentist than I did one year ago.	0	6.5%	12.3%	57.2%	23.9%
2. I personally appreciate the advantages of practicing evidence-based patient care.	0	0.7%	12.3%	59.4%	27.5%
3. EBP should be an integral part of the dental school curriculum.	0.7%	1.4%	9.4%	52.9%	35.5%
4. I support EBP principles more than I did one year ago.	0	10.1%	20.3%	51.4%	18.1%
5. EBP is a routine part of my professional growth as a dentist.	0.7%	8.0%	17.4%	58.0%	15.9%
6. The practice of evidence-based dentistry has changed the way I learn.	1.4%	15.2%	29.7%	42.0%	11.6%
7. It has been difficult for me to practice evidence-based dentistry in the past year.	1.4%	28.3%	22.5%	37.0%	10.9%
8. EBP is “cookbook” dental care that disregards clinical experience in providing the best treatment for patients.	14.5%	37.0%	28.3%	15.2%	5.1%
9. It is feasible to use EBP routinely when providing care for patients in the dental school clinic.	2.9%	19.6%	25.4%	42.8%	9.4%
10. EBP improves the quality of dental patient care.	0	0.7%	15.2%	58.7%	25.4%

Note: Boldface indicates highest percentage for each question. Percentages may not total 100% because of rounding.

The most frequently accessed resources for dental evidence were reported to be colleagues (often 32.6%, very frequently 44.2%), the Internet (often 34.8%, very frequently 18.1%), and textbooks (often 39.1%, very frequently 9.4%) (Table 4). Resources reported as being infrequently accessed were databases of critically appraised topics (rarely 21.7%, never 68.1%), podcasts and web conferences (rarely 24.6%, never 59.4%), and the Cochrane Database of Systematic Reviews (rarely 31.9%, never 37.0%).

Table 4.

Respondents’ access of sources of evidence, by frequency of each (N=138)

Source	Never	Rarely	Occasionally	Often	Very Frequently
1. Colleagues: other dentists or health care providers	0.7%	5.1%	17.4%	32.6%	44.2%
2. Textbooks	2.9%	10.1%	38.4%	39.1%	9.4%
3. The Internet (excluding Cochrane reviews)	2.9%	7.2%	37.0%	34.8%	18.1%
4. Original research papers published in peer-reviewed journals	8.7%	28.3%	39.9%	19.6%	3.6%
5. Cochrane Database of Systematic Reviews	37.0%	31.9%	20.3%	9.4%	1.4%
6. Journal of Evidence-Based Dental Practice, Evidence-Based Dentistry, Journal of Evidence-Based Medicine	18.8%	22.5%	39.9%	15.9%	2.9%
7. Continuing education courses/workshops	30.4%	25.4%	31.2%	10.9%	2.2%
8. Podcasts and web conferences (webinars)	59.4%	24.6%	13.0%	2.9%	0.0%
9. Databases of critically appraised topics (CATs)	68.1%	21.7%	8.0%	1.4%	0.7%

Note: Boldface indicates highest percentage for each question. Percentages may not total 100% because of rounding.

Assessment of Confidence in Critical Appraisal Skills

Questions regarding confidence in appraisal of diverse aspects of a published research report yielded varied responses, with the majority of respondents reporting moderate confidence and few selections of “not at all confident” or “very confident” (Table 5). These students reported highest confidence in appraising the generalizability of findings (moderately confident 52.2%, confident 30.4%, very confident 5.1%), the overall value of a published research report (moderately confident 51.4%, confident 29.0%, very confident 5.1%), and the adequacy of the sample size (moderately confident 47.1%, confident 23.9%, very confident 5.1%). The students reported the least amount of confidence in the appropriate use of statistical tests (not confident 39.1%, not at all confident 15.2%).

Table 5.

Respondents’ confidence in their critical appraisal skills, by level of confidence in each (N=138)

Type of Critical Appraisal	Not at All Confident	Not Confident	Moderately Confident	Confident	Very Confident
1. Appropriateness of the study design	0.7%	21.7%	51.4%	22.5%	3.6%
2. Bias in the study design or data analysis	2.2%	23.9%	44.9%	24.6%	4.3%
3. Adequacy of the sample size	0.7%	23.2%	47.1%	23.9%	5.1%
4. Generalizability of the findings	0.0%	12.3%	52.2%	30.4%	5.1%
5. Appropriate use of statistical tests	15.2%	39.1%	31.9%	10.9%	2.9%
6. Overall value of the research report	0.0%	14.5%	51.4%	29.0%	5.1%

Note: Boldface indicates highest percentage for each question. Percentages may not total 100% because of rounding.

Discussion

Dental students in their final year of predoctoral education provide valuable insights as U.S. dental schools strive to advance training in EBP. Unfortunately, the students in this study appeared to have low levels of working knowledge of relevant fundamental EBP principles. Although the respondents appeared to value EBP, they seemed to lack knowledge of study design, the rigors of various publication types, and statistical methodology—challenging their ability to critically appraise the dental literature. Information gained from this study can help inform educational programming and faculty development efforts.

Knowledge of Critical Appraisal

The KACE knowledge of critical appraisal questions are diverse in content and require understanding as well as application of knowledge at a level of complexity appropriate for predoctoral dental students.¹⁶ The results of the knowledge questions in this study should be interpreted with caution as the questions are limited in breadth and do not address facets of hypothesis generation, diagnosis, screening, or prognosis. The collective response of students surveyed in this study suggests a small majority had fundamental knowledge of pertinent EBP principles. The highest level of correct responses, in descending order, were interpretation of study results to a provided scenario, identification of the correct description of a meta-analysis, and understanding strategies for productive PubMed searches.

The KACE knowledge of study design for new diagnostic devices had the majority of incorrect responses. Knowledge of study design is critical to navigating the myriad of technological advances and explosion of associated research. The majority of respondents selected a randomized clinical trial as the study design most appropriate to evaluate the efficacy of a new diagnostic device for assessment of oral health problems. This finding suggests the need for clarification of research design selection in the instruction of EBD. Richards et al. explain that the type of study design should be tailored to the research question.¹⁸ Prominent diagrams of study designs feature randomized controlled trials at the highest level of research designs, which may deceive students. The appropriate indications for the use of randomized controlled trials, as well as the limitations of randomized controlled trials, may not be readily evident to predoctoral students.

Further introduction and evaluation of the various forms of published evidence may prove valuable in predoctoral EBP instruction. The majority of our respondents did not identify a Cochrane review as the highest level of evidence, rather selecting a peer-reviewed, published case series article. This response may suggest a lack of familiarity or understanding of published sources of evidence, the hierarchy of levels of evidence, or the rigors of Cochrane systematic reviews.

EBP Attitudes, Accessing Evidence, and Confidence in Critical Appraisal

These students’ overall responses to statements regarding attitudes about evidence-based practice demonstrated a value and appreciation of this approach to practice. The students were most likely to agree or strongly agree that EBP should be an integral part of the dental school curriculum. The majority reported an appreciation of the advantages of EBP and the belief that EBP improves the quality of dental care. This generation of dental graduates is perhaps the first to widely view EBP as a routine component of professional growth. This change may have been influenced by the current emphasis on EBD instruction by the Commission on Dental Accreditation (CODA) Accreditation Standards for Dental Education Programs.¹⁴ The students in our study showed greater appreciation for the value of EBP than dental school graduates in earlier decades.¹⁵

The respondents were most uncertain about the impact of EBD on the way in which they learn. It may be that students entering dental school have established learning patterns as adult learners. Students may rely on these established patterns in mastering dental school content. It may also be that principles of EBD instruction, self-inquiry, interpretation of findings, and application to clinical problems are not pervasive in dental education, making it difficult for students to acknowledge a substantive change to their learning.

Challenges encountered by dental practitioners in the implementation of EBP have long been documented. Such challenges include information overload, lack of awareness of existing evidence, personal disagreement with existing evidence, and patient preferences.^15,19-27 Among our respondents, 47% reported difficulty in practicing EBD in the past year. It is possible that our respondents perceived the same challenges as those reported previously. However, the response to this survey item may have been influenced by the condition under which students render patient care. The treatment and services students provide are always under the direction of a licensed dentist, so students may not perceive themselves as practicing EBD when final decisions are being made by the supervising dentist. This explanation seems to fit with the responses given to the survey items on feasible use of EBP in the dental school clinic and personal appreciation for the advantages of EBP. The majority of respondents agreed or strongly agreed with these items, suggesting that these students appreciated the benefits of EBP and believed clinics to be conducive to EBP.

One of the prominent challenges in transferring research evidence into the provision of health care services is development of the ability to locate and utilize high-quality evidence. Doing so is essential for confronting the enormous volume of scientific information, which cannot be reviewed in its entirety. Predoctoral dental students in this study reported accessing dental evidence most frequently from colleagues, the Internet (excluding Cochrane reviews), and textbooks. These findings could indicate deficiencies in student understanding of appropriate levels of evidence or could be attributed to obstacles associated with accessing evidence. Given the age of most dental students, they are likely to access the Internet for any type of information.²⁸ However, it takes time, effort, and a skill set to wade through an ever-expanding knowledge base, so students may prefer to seek the opinions of others. This preference is not unlike experienced physicians and dentists who have reported preferring to consult with colleagues when searching for information or making clinical decisions.^29-34 Nevertheless, relying on colleagues is in contrast to current educational models on EBD. To address this challenge, U.S. dental schools could develop easily accessible online resources that foster EBP and model the application of EBP to actual patient care in dental school clinics, both of which may require a focus on faculty development.

While self-reported data regarding ability or skill may not be accurate, our respondents’ lower confidence in components of critical appraisal skills mirrors published reports of medical, dental, and allied health care practitioners,^15,35-37 as well as our impressions as dental educators with experience across institutions. The students’ low self-reported confidence in this study and the results regarding aspects of statistical concepts suggest additional statistical instruction may be needed in predoctoral dental education. Given strides to advance EBP instruction in dental schools, this finding calls into question the preparedness of predoctoral students in foundational statistical knowledge prior to entering dental school as well as their level of exposure during predoctoral training.

Limitations

This research study provides information regarding the knowledge, perceptions, and behavior of an institutionally diverse population of predoctoral students regarding an evidence-based philosophy of practice. While the study suggests an appreciation for the value of EBP among these dental students, there are inherent limitations that should be considered when interpreting the results. The data were based on approximately 20 dental students’ opinions at each of the seven participating institutions. Data were collected and analyzed only from surveys with all fields completed. The number of students who were contacted to participate or who only partially completed the survey is unknown. This study represents a fraction of potential respondents: all U.S. dental students enrolled in the final year of the predoctoral program. However, the samples are probably representative of each school because of random selection.

The type of institutional EBP initiatives, institutional support, faculty expertise, curricula, and instruction in the schools in this study is also unknown, so the students’ responses cannot be linked to any particular methodology, content, or school emphasis. In addition, the academic and research backgrounds of the students were unknown. Finally, the small sample size in each school means there was a lack of statistical power to compare results between and among schools. For these reasons, we chose not to present results by institution. Further research evaluating the domains specified in this study and their relationship to institutional initiatives supporting the practice of EBD, faculty expertise in EBD, and curricular content, delivery, and assessment should be pursued.

Conclusion

This study assessed the awareness of EBP knowledge, perceptions, and behavior of a population of institutionally diverse predoctoral students in their final academic year preceding graduation from U.S. dental schools. The results supported our hypothesis of widespread exposure to evidence-based philosophies of practice among these students. The students demonstrated low levels of knowledge of the critically appraised questions and positive attitudes towards evidence-based philosophies, with significant differences in access to evidence and confidence in evidence-based skills. This work is an important first contribution to the larger effort of examining and shaping dental curricula for the successful education of dentists competent in EBP. It is essential that academic leaders determine whether current EBP teaching methodologies are effective and whether they influence the knowledge, attitudes, and behaviors of dental students. Dental schools should strive to advance students’ EBP knowledge and skill sets, thereby providing them with the capability to positively impact their level of expertise and future practice.