Abstract
Introduction:
The levels of evidence (LOE) of dental education texts is critical from the educational point of view. The present study aimed to evaluate the levels of evidence of references used in three clinical chapters in the textbook Pathways of the Pulp.
Material & Method:
The references of three clinical chapters in the text book Pathways of the Pulp were assessed in five of its editions. The levels of evidence were ranked according to study type and the Oxford scale from 0 to 5. The chi-square test was used to compare the level of evidence between different editions of the "Retreatment," "Trauma," and "Surgery" chapters.
Results:
A total of 3656 references were reviewed and analyzed from the "Trauma" (928 references), "Re-treatment" (1906 references), and "Surgery" (822 references) chapters in the 1998, 2002, 2006, 2011, and 2016 editions. The percentage of the LOE 0 (no evidence) was high (>60%) in all three chapters in all editions (P<0.001). The levels of evidence had the same distribution in all editions (P=0.871). The LOE of the "Re-treatment" (P=0.044) and "Surgery" (P<0.001) chapters changed in some editions.
Conclusion:
The majority of references in the three clinical chapters of the book are low-level evidence. Encouragement policies for researchers to conduct studies with high LOE are necessary.
Key Words: Endodontics, Evidence-Based Dentistry, Level of Evidence
Introduction
In the 1990s, a new process called evidence-based medicine (EBM) was introduced to review scientific evidence in medical sciences [1]. Recently, many endodontists have highlighted the role and importance of EBM in clinical decision-making [2-6]. This phenomenon was a systematic approach to summarizing the large volume of texts and articles that health professionals needed to share their experiences with others [7]. Evidence-based medicine, described in 1996 by Sackett, is tool for identifying for the best accessible scientific evidence to choose the most suitable treatment for every patient [1, 8]. The American Dental Association (ADA) has defined evidence-based dentistry (EBD) as "an approach to oral health care that requires the judicious integration of systematic assessment of clinically relevant scientific evidence, relating to the patient's treatment needs and preference" [9].
Dental treatment planning and management, in fields such as endodontics, should be based on high-level evidence for optimum effect, efficacy, and treatment. The existence and availability of high-quality clinical studies in clinical evidence-based decision-making are essential [10, 11]. The Oxford Center for Evidence-based Medicine defined an LOE scoring system in 2009 for the first time. The highest LOE belongs to RCT. The lowermost LOE pertains to expert opinion [12].
In the last decade, significant efforts have been made to assess the levels of evidence in endodontics. Torabinejad assessed the clinical studies related to root canal therapy failure and success and allocated levels of evidence to these articles. Results indicated that few high-level articles had been published in the last four decades pertinent to the failure and success of root canal therapy [13, 14]. Asgary et al. searched for endodontic studies from Iran published in PubMed-indexed journals. The results indicated that the levels of evidence of studies in this field was insufficient [15]. Mead et al. published a literature review for clinical articles pertinent to endodontic surgery [16]. The results indicated that published endodontic articles do not have high levels of evidence. Most of the literature is comprised of low-level case studies.
Shafiei and Shahravan evaluated the levels of evidence in two endodontic journals in 2012. They reported no significant differences concerning the published articles in the two journals. These studies are of paramount importance to published journals that present an upper level of evidence to manage clinical problems [17].
The textbook Pathways of the Pulp is one of the essential references in the education of undergraduate and postgraduate dental students. It has been translated into more than 14 languages. Based on our research, the levels of evidence of references in medical books and endodontic books have not yet been evaluated. Therefore, this study aimed to assess all references in three clinical chapters ("Surgery", "Re-treatment", and "Trauma") in each of the five consecutive editions of the textbook Pathways of the Pulp published in 1998, 2002, 2006, 2011, and 2016.
Materials and Methods
In this study, the levels of evidence assessment method consisted of four main stages:
1. References of all three chapters of the textbook were collected. Then the full texts of the articles were acquired.
2. Two independent dental researchers evaluated the levels of evidence of the articles separately. The level ranking was carried out according to a ranking system (Table 1) [18, 19], which is a revision of the LOE ranking of the Australian National Health and Medical Research Council [20], the Center for Evidence-Based Medicine [21], and the American Society of Plastic Surgeons [22].
Table 1.
Level of evidence (LOE)
| Level | Type of evidence |
|---|---|
| 1 | High-quality RCTs, or SRs |
| 2 | Non-randomized clinical trials, or prospective cohorts, or SRs |
| 3 | Retrospective cohorts, or case-control studies, or SR |
| 4 | Case-series or cross-sectional studies |
| 5 | Case reports or expert opinions |
RCT: Randomized Controlled Trials; SR: Systemic Review
3. In cases where the researchers' results differed, the exact LOE was determined by a third researcher experienced in evidence-based dentistry.
Data extraction included chapter name, reference number, chapter number, study type, and level of evidence. The final statistical analysis excluded papers whose full texts could not be found. The Friedman test was used to compare the frequency distribution of evidence levels of a chapter across different editions, and the Kruskal-Wallis test was used to compare the evidence levels of the chapters in each edition. The statistical significance was set at 5%. The ggstatplot package of R (Version 4.2) software was used for statistical analysis [23].
Results
A total of 3656 articles were reviewed and analyzed. The maximum percentage of references that were not found was 3.3%, and all references in the last two editions were found and reviewed (Table 2).
Table 2.
Total reference analysis
| Edition | Group | Full text not found | Total |
|---|---|---|---|
| 1998 | Re-treatment | 1(1.2%) | 86 |
| Surgery | 1(1.1%) | 92 | |
| Trauma | 5(3%) | 165 | |
| Total | 7(2%) | 343 | |
| 2002 | Re-treatment | 3(2.8%) | 106 |
| Surgery | 0(0%) | 84 | |
| Trauma | 6(3.3%) | 180 | |
| Total | 9(2.4%) | 370 | |
| 2006 | Re-treatment | 0(0%) | 215 |
| Surgery | 0(0%) | 541 | |
| Trauma | 1(0.7%) | 134 | |
| Total | 1(0.1%) | 890 | |
| 2011 | Re-treatment | 0(0%) | 244 |
| Surgery | 0(0%) | 579 | |
| Trauma | 0(0%) | 166 | |
| Total | 0(0%) | 989 | |
| 2016 | Re-treatment | 0(0%) | 277 |
| Surgery | 0(0%) | 610 | |
| Trauma | 0(0%) | 177 | |
| Total | 0(0%) | 1064 |
The distribution of levels of evidence was significantly different (P < 0.001) in all editions. The levels of evidence in the "Trauma" chapter were higher than in the "Re-treatment" and "Surgery" chapters in the 1998 (P<0.001, P=0.003) and 2002 editions (P<0.001, P<0.001). In the 2006 and 2011 editions, the levels of evidence of the "Trauma" chapter were higher than the "Re-treatment" chapter (P<0.001, P<0.014), and in the 2016 edition, the levels of evidence in the "Trauma" and "Surgery" chapters were higher than the "Re-treatment" chapter (P<0.001, P=0.002).
The data showed that at least 60% of the references for the "Trama" chapter in all editions had zero evidence levels (P < 0.001). The levels of evidence had the same distribution in all editions (P=0.701) (Figure 1). In the "Re-treatment" chapter, except for the 2011 edition, where the percentage of references with zero level of evidence was 76%, this percentage was higher than 80% (P=0.044) (Figure 2). In the "Surgery" chapter, the percentage of references with zero evidence level decreased from 89% in the 1998 edition to 67% in the 2016 edition (P<0.001) (Figure 3).
Figure 1.
Level of evidence of "Trauma" Chapters in 5 editions. The levels of evidence had the same distribution in all editions
Figure 2.
Level of evidence of "Re-treatment" Chapters in 5 editions. The levels of evidence of the "Re-treatment" chapters changed in some editions
Figure 3.
Level of evidence of "Surgery" Chapters in 5th editions. The percentage of references with zero evidence level decreased in new editions (from 89% in the 1998 edition to 67% in the 2016 edition)
Discussion
We evaluated this issue in three clinical chapters ("Trauma," "Surgery," and "Re-treatment") in the textbook Pathways of the Pulp for the first time. The LOE of 0 decreased slightly in later editions; however, it was 60% at its lowest. Considering the concept of LOE, addressing clinical questions requires high-level studies, including systematic reviews and clinical trials. Therefore, the number of high-level studies was expected to have increased in more recent editions.
Two explanations can be proposed for the high percentage of references with zero levels of evidence in the three clinical chapters of Pathways of the Pulp. In the first and more likely case, the results with a high level of evidence in endodontics literature are few, for which some evidence will be presented. The second hypothesis is that the results with high levels of evidence are either unpublished or, if published, the book's authors did not feel the need to use them. The likelihood of clinical trials with insignificant results being published by journals is low, which is commonly referred to as publication bias. However, to determine if there were any articles with high levels of evidence that the authors did not use, it is necessary to conduct a systematic review of the book's specific topics. However, the book's authors are well-known endodontic scholars, and such a hypothesis is unlikely.
In line with our results evaluations of endodontic journals, including the Journal of Endodontics, International Endodontic Journal, and Iranian Endodontic Journal, showed that most of the published results had zero levels of evidence; however, the evidence levels slowly increased over time [17, 24]. Therefore, the high percentage of references with low levels of evidence in the three chapters can be considered as the consequence of the small number of published studies with high levels of evidence.
One of the main concerns in endodontics is that most research is cross-sectional or in vitro. A study by Torabinejad et al. conducted on different endodontic treatment approaches in endodontic journals showed that few high-evidence projects had been published in these journals [25] as many studies were cross-sectional (LOE of 4), which is consistent with Hui et al. in orthodontics [12]. Most studies in the dental materials field are related to substances and their behaviors; accordingly, most studies are in vitro or on laboratory media that do not have the LOE criteria. Thus, animal studies could occasionally be carried out in the clinical practice before necessary clinical studies [17].
It is critical to understand why many published results have low levels of evidence. Patients do not participate in clinical trials easily, samples in animal studies are rarely lost during research, side effects and compensation are not a problem for animal researchers, and animal studies are much less expensive. In addition, researchers are evaluated in academic systems based on the number of articles they have published, the ranking of the journals they have published in, the grants they have received, and other activities. As a result, there is no distinction between a researcher who has conducted animal and one who has conducted human studies. Thus, hard-working researchers risk statistically non-significant results that are difficult to publish in high-profile journals if they conduct human studies. Therefore, few in vitro studies move on to the clinical trial phase.
Our evaluation showed that the percentage of non-evidence articles (LOE of 0) was noticeably higher than articles with high levels of evidence (LOE of 1) in all studied chapters in the five editions of the textbook. Therefore, only a few published studies in this textbook can answer the clinical questions of practicing dentists or endodontists.
Different types of studies might be necessary for various fields of dentistry. The restorative material field is one of the most creative fields in dental science and restorative procedures seem to be the most commonly performed dental procedures. In our study, 70.1% of studies in the "Surgery" chapters were no-evidence (LOE of 0). We can conclude that, for example, animal studies among non-evidence articles were necessary before clinical trials. The rapid development and advances in endodontic equipment and technology justify the high number of animal and laboratory articles. However, it is not usually easy to convince patients to participate in trials of new treatment approaches based on the outcomes of animal studies [25, 26].
Conclusion
The levels of evidence in the book's three clinical chapters were very low, indicating a significant gap between clinical questions and the answers provided by research findings. Because it is much more difficult to conduct studies with high-level evidence (clinical trials or cohort studies) than other studies, policies are required to encourage researchers to conduct these types of studies.
Conflict of Interest:
‘None declared’.
References
- 1.Perrier A. Evidence-based medicine and critical care. J Schweizerische Medizinische Wochenschrift. 1999;129(43):1572–82. [PubMed] [Google Scholar]
- 2.Asgary S, Motazedian HR, Parirokh M, Eghbal MJ, Kheirieh S. Twenty Years of Research on Mineral Trioxide Aggregate: A Scientometric Report. Iran Endod J. 2013;8(2):84. [PMC free article] [PubMed] [Google Scholar]
- 3.Eghbal MJ, Davari Ardakani N, Asgary S. A Scientometric Study of PubMed-Indexed Endodontic Articles: A Comparison between Iran and Other Regional Countries. Iran Endod J. 2012;7(2):56–9. [PMC free article] [PubMed] [Google Scholar]
- 4.Eghbal MJ, Torabzadeh H. Review of High Level Endodontic Research in PubMed Index Journals from Iran. Iran Endod J. 2012;7(3):109–14. [PMC free article] [PubMed] [Google Scholar]
- 5.Kakooei S, Mostafavi M, Parirokh M, Asgary S. Indexing of Iranian Publications in Well-known Endodontic Textbooks: A Scientometric Analysis. Iran Endod J. 2016;11(3):157–63. doi: 10.7508/iej.2016.03.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Shahravan A, Haghdoost AA. Endodontic Epidemiology. Iran Endod J. 2014;9(2):98–108. [PMC free article] [PubMed] [Google Scholar]
- 7.Chiappelli F. Evidence-Based Dentistry: Two Decades and Beyond. J Evid Based Dent Pract. 2019;19(1):7–16. doi: 10.1016/j.jebdp.2018.05.001. [DOI] [PubMed] [Google Scholar]
- 8.Sackett DL, Rosenberg WM, Gray JM, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ; 1996 doi: 10.1136/bmj.312.7023.71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.London SD, Chamut S, Fontelo P, Iafolla T, Dye BA. Assessment of the Quality of Current American Dental Association Clinical Practice Guidelines. JDR Clin Trans Res. 2022:23800844221083563. doi: 10.1177/23800844221083563. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Haapasalo M. Level of evidence in endodontics: what does it mean? Endod Topics. 2016;34(1):30–41. [Google Scholar]
- 11.Kakooei S, Mostafavi M, Parirokh M, Asgary S. Indexing of Iranian Publications in Well-known Endodontic Textbooks: A Scientometric Analysis. Iran Endod J. 2016;11(3):157–63. doi: 10.7508/iej.2016.03.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Hui J, Han Z, Geng G, Yan W, Shao P. The 100 top-cited articles in orthodontics from 1975 to 2011. Angle Orthod. 2013;83(3):491–9. doi: 10.2319/040512-284.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Torabinejad M KD, Machnick TK, Ismail A, Newton CW. Levels of evidence for the outcome of nonsurgical endodontic treatment. J Endod. 2005;31(9):637–46. doi: 10.1097/01.don.0000153593.64951.14. [DOI] [PubMed] [Google Scholar]
- 14.Kontakiotis EG, Filippatos CG, Agrafioti A. Levels of evidence for the outcome of regenerative endodontic therapy. J Endod. 2014;40(8):1045–53. doi: 10.1016/j.joen.2014.03.013. [DOI] [PubMed] [Google Scholar]
- 15.Asgary S, Sabbagh S, Eghbal M. Published endodontic articles in PubMed-indexed journals from Iran. Iran Endod J. 2012;7(1):1. [PMC free article] [PubMed] [Google Scholar]
- 16.Mead C J-NS, Mego ME, Nash B, Torabinejad M. Levels of evidence for the outcome of endodontic surgery. J Endod. 2005;31(1):19–24. doi: 10.1097/01.don.0000133158.35394.8a. [DOI] [PubMed] [Google Scholar]
- 17.Shafiei L, Shahravan A. The level of evidence in two leading endodontic journals. Iran Endod J. 2013;8(1):18. [PMC free article] [PubMed] [Google Scholar]
- 18.Sheikhi M, Shahravan A. The level of evidence of published articles on orthodontics in PubMed journals from Iran during 2000-2015. J Oral Health Oral Epidemiol. 2016;5(4):210–4. [Google Scholar]
- 19.Asgary S. Mineral Trioxide Aggregate and Evidence-Based Practice. In: Camilleri J, editor. Mineral Trioxide Aggregate in Dentistry: From Preparation to Application. Berlin, Heidelberg: Springer Berlin Heidelberg; 2014. pp. 173–99. [Google Scholar]
- 20.Council. NHaMR. A guide to the development, evaluation and implementation of clinical practice guidelines. Commonwealth of Australia: National Health and Medical Research Council. 1999. Available from: http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cp30.pdf.
- 21.Council NHaMR. A Guide to the Development, Implementation and Evaluation of Clinical Practice Guidelines. PsycEXTRA Dataset: American Psychological Association (APA); 1999. [Google Scholar]
- 22.Sullivan D CK, Eaves FF 3rd, Rohrich RJ. The level of evidence pyramid: indicating levels of evidence in Plastic and Reconstructive Surgery articles. Plast Reconstr Surg. 2011;128(1):311–4. doi: 10.1097/PRS.0b013e3182195826. [DOI] [PubMed] [Google Scholar]
- 23.Patil I. Visualizations with statistical details: The 'ggstatsplot' approach. J Open Source Software. 2021;6:61. [Google Scholar]
- 24.Rezvaninejad R, Shahravan A. The level of evidence of articles published in Iranian Endodontic Journal in 3 years (2007, 2012 and 2013) Chin J Dent Res. 2016;5(3):161–5. [Google Scholar]
- 25.Torabinejad M, Bahjri K. Essential elements of evidenced-based endodontics: steps involved in conducting clinical research. J Endod. 2005;31(8):563–9. doi: 10.1097/01.don.0000164137.28104.2f. [DOI] [PubMed] [Google Scholar]
- 26.Lai TY, Leung GM, Wong VW, Lam RF, Cheng AC, Lam DS. How evidence-based are publications in clinical ophthalmic journals? Invest Ophthalmol Vis Sci. 2006;47(5):1831–8. doi: 10.1167/iovs.05-0915. [DOI] [PubMed] [Google Scholar]