Summary:
Levels of evidence (LOE) aid in the critical appraisal of evidence by ranking studies based on limitation of its design. Analyzing LOE provides insight into application of evidence-based medicine. The aim of this study is to determine if the quality of evidence in plastic surgery research has improved over the past 10 years. Systematic review of research published in Plastics and Reconstructive Surgery journal over the years, 10-year period (2008, 2013, 2018), was performed. LOE for each article was determined using the American Society of Plastic Surgeons (ASPS) guidelines. Each level was calculated as percentage of publications per year and compared yearly and between different topics. Eight hundred eighty-four studies were included in the final analysis. The LOE of the research improved over the study period. Level 4 evidence was the most frequent published (50.6%, 447/884), with a decline from 63.2% in 2008 to 41.3% in 2018. Level 1 evidence improved each year and accounted for 2.1% of all research in 2018. Aesthetic surgery was the most frequent published topic with upper limb research demonstrating an 18.5% increase in high-quality evidence over the study period. Increased awareness of evidence-based medicine has improved the quality of plastic surgery research over the past decade. It is vital this continues to provide gold standard patient care.
INTRODUCTION
Advances in technology, improved understanding of disease pathogenesis, and superior interventions have enhanced plastic surgery patients’ outcomes over the past decade.1,2 To support these advancements, up to date research endeavors are mandatory. The resultant increase in studies has generated an overwhelming amount of evidence. To assist clinicians in critical appraisal of this evidence, a conceptual tool known as evidence-based medicine (EBM) was developed.3
Levels of evidence (LOE) is the foundation of EBM. It is a hierarchical appraisal system which grades research (levels 1–5) based on inherent limitations of study methodology.4 LOE enables clinicians to rapidly appraise evidence before translating into clinical practice.5 It is also a reliable method of evaluating the quality of evidence published. High-quality research is a prerequisite in maintaining optimal patient care. The aim of this study is to determine if the quality of evidence in plastic surgery research has improved over the past 10 years.
METHODOLOGY
A systematic review of published research articles was performed in Plastic and Reconstructive Surgery (PRS) journal. Articles were selected from this journal, as it is the highest impact factor plastic surgery journal, publishing on a wide variety of plastic surgery topics. To evaluate any trends, articles were initially reviewed from 3 years, covering a 10-year period (2008, 2013, 2018). Editorials, letters, announcements, reflections, book reviews, Continuing Medical Education (CME) articles were excluded from this study. Review articles and laboratory studies (animal, cadaver, basic science) were included in the initial review but excluded from the final analysis, as no LOE can be allocated to these studies.
Each clinical article was allocated an LOE based upon published American Society of Plastic Surgeons (ASPS) guidelines.6 First, the research aim was broadly divided into 3 categories: therapeutic, risk, and diagnostic. Within these categories, evidence was ranked from 1 to 5 mirroring the hierarchal research pyramid. This ranking is based upon the probability that the research design has reduced the potential bias. Highest quality evidence (level 1) is produced from randomized control trials or systematic reviews/meta-analysis of these. Articles of limited study design with biases, such as expert opinions, are ranked the lowest (level 5).
Two authors (C.M.S., C.W.J.) independently evaluated published research articles. Discrepancies in the assignment of LOE were discussed with the senior author (S.M.C.). Information obtained from the articles included year of publication, topic, study design, and LOE. Each LOE was expressed as percentage of the overall publications that year and to the other years. Study design was also assessed. Further analysis on the different topics published in the journal was performed. The percentage of higher-level evidence (levels 1 and 2) was calculated for each topic and compared over the 10-year period.
RESULTS
Two thousand six hundred sixty-four articles were published in the PRS journal in the years 2008, 2013, 2018. By applying the inclusion criteria, a total of 1,369 articles were reviewed. Review articles and laboratory studies accounted for 14.7% and 20.7% of the remaining articles. These were excluded from the final analysis as they are not part of LOE hierarchy. A LOE rank was applied to 884 articles (2008 = 313, 2013 = 291, 2018 = 280) (Fig. 1).
Fig. 1.
Flow diagram of study methodology. CME indicates continuing medical education; LOE, levels of evidence.
Therapeutic studies were the most frequent research aim, accounting for 83.6% of all research. Level 4 evidence was the greatest level published across the years (50.6%, 447/884). There was a decline in the percentage of level 4 evidence from 63.2% in 2008 to 41.3% in 2018. Twelve publications accounted for level 1 evidence, with 11 of these published in the past 5 years (Table 1).
Table 1.
Percentage of Each Level of Evidence Published per Year
| Levels of Evidence | 2008 | 2013 | 2018 |
|---|---|---|---|
| 1 | 0.3 | 1.7 | 2.1 |
| 2 | 6.3 | 11.3 | 13.6 |
| 3 | 19.6 | 33.3 | 34.5 |
| 4 | 63.2 | 45.5 | 41.7 |
| 5 | 11.3 | 8.5 | 7.9 |
Case series was the most common study design (36.4%, 498/1,369). This study designed decreased over time from 48.3% in 2008 to 25.5% in 2018. Case–control studies increased from 9.7 % to 19.9% (Table 2).
Table 2.
Evaluation of Each Study Methodology Used in Research, per Year
| 2008 n (%) |
2013 n (%) |
2018 n (%) |
|
|---|---|---|---|
| Systematic review/meta analysis | 5 (1.1) | 21 (4.6) | 15 (3.2) |
| Randomized control trials | 8 (1.7) | 5 (1.1) | 18 (3.8) |
| Cohort study | 4 (0.9) | 10 (2.2) | 8(1.7) |
| Case–control | 44 (9.7) | 75 (16.7) | 93 (19.9) |
| Case series | 219 (48.3) | 156 (34.7) | 123 (26.3) |
| Case report | 10 (2.2) | 9 (2.0) | 9 (1.2) |
| Expert opinion | 25 (5.5) | 15 (1.1) | 12 (2.1) |
| Review article | 45 (9.3) | 61 (15.8) | 99 (22.2) |
| Laboratory study | 96 (21.1) | 97 (21.6) | 90 (19.3) |
| n = 453 | n = 449 | n = 467 |
Aesthetic (21.6%), breast (17.3%), craniofacial (4.2%), and upper limb surgery were the most frequently published topics in PRS journal. The percentage of higher LOE (levels 1 and 2) published in these topics over the years is seen in Table 3. The largest percentage increase over the 10-year period was demonstrated by upper limb surgery (18.5%).
Table 3.
Percentage Comparison of High-quality Evidence (Levels 1 and 2) per Plastic Surgery Topic, per Year
| Topics | 2008 | 2013 | 2018 |
|---|---|---|---|
| Aesthetics | 6.3 | 6.2 | 15.3 |
| Breast | 19.2 | 13.7 | 18.7 |
| Craniofacial | 1.5 | 8.3 | 8.5 |
| Upper limb | 11.5 | 11.1 | 25 |
DISCUSSION
The application of EBM involves merging individual clinical experience with the best scientific evidence.7 The interrogation of EBM into plastic surgery practice has been limited.8 By evaluating trends in LOE of published research, the utilization of EBM principles can be measured. This study has shown that the LOE in plastic surgery research has improved over the past decade. There has been a growth in levels 1, 2, and 3 evidence, with a reduction in the publication in lower-quality evidence. In 2018, high-quality evidence (levels 1 and 2) accounted for 15.7% of all plastic surgery research. This was marginally lower than orthopedic literature (21.6%),9 but higher than neurosurgical (10.3%)10 and maxillofacial research (2%).11 Within the PRS journal, upper limb and aesthetics surgery demonstrated the largest increase in high-quality evidence over the 10-year period.
Case series are the backbone of surgical research. By evaluating a similar group of patients undergoing a common intervention, this study design replicates everyday surgical practice.12 The absence of a control group justifiably ranks this design at the lower end of the evidence pyramid. Despite this, case series are vital. They may be the only feasible and ethical study methodology obtainable, as seen with craniofacial surgery.13 In our study, craniofacial research accounted for the lowest percentage publication (8.4%) of high-quality evidence, with no improvement over the past 5 years. The rarity of craniofacial pathology coupled with a small number of patients makes it difficult to produce higher quality research. Case–control studies are an upgrade from case series, with the addition of a control group significantly reducing study bias.14 Case –control studies increased in our study period. In a specialty where obtaining high-quality evidence is challenging, the evolution from case series to case–control studies is an important indicator of EBM application.
The concept of LOE was originally described 50 years ago.15 Yet, its application in plastic surgery research has been underwhelming,16 with lack of awareness a probable reason.17 To overcome this, PRS journal, in 2011, made it mandatory for authors submitting manuscripts to attach an LOE rating. This is then displayed as a small pyramidal graphic on the abstract page, providing immediate context for the reader. This editorial policy could account for the greater increase in LOE between 2008 and 2013, in comparison to the past 5 years of this study. Other journals have a similar requirement, but the LOE is allocated by the editorial board, out of fear of authors over infiltrating their own research.18 However, good interobserver and intraobserver reliability has been reported when grading LOE.19 By placing the responsibility of LOE ranking with the submitting author, knowledge of EBM has improved along with the quality of evidence published.
Footnotes
Published online 30 September 2019
Disclosure: The authors have no financial interest to declare in relation to the content of this article.
REFERENCES
- 1.Grunwald T, Krummel T, Sherman R. Advanced technologies in plastic surgery: how new innovations can improve our training and practice. Plast Reconstr Surg. 2004;114:1556–1567. [DOI] [PubMed] [Google Scholar]
- 2.Hernandez-Boussard T, McDonald KM, Rhoads KF, et al. Patient safety in plastic surgery: identifying areas for quality improvement efforts. Ann Plast Surg. 2015;74:597–602. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Sackett DL, Rosenberg WM, Gray JA, et al. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312:71–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Evans D. Hierarchy of evidence: a framework for ranking evidence evaluating healthcare interventions. J Clin Nurs. 2003;12:77–84. [DOI] [PubMed] [Google Scholar]
- 5.Poolman RW, Kerkhoffs GM, Struijs PA, et al. Don’t be misled by the orthopedic literature: tips for critical appraisal. Acta Orthop. 2007;78:162–171. [DOI] [PubMed] [Google Scholar]
- 6.ASPS Evidence Rating Scales. Available at: https://www.plasticsurgery.org/documents/medical-professionals/health-policy/evidence-practice/ASPS-Rating-Scale-March-2011.pdf. Accessed June 25, 2019.
- 7.Sackett DL, Richardson WS, Rosenberg W, et al. Evidence-Based Medicine: How to Practice and Teach. 20002nd ed Edinburgh, Scotland: Churchill-Livingstone. [Google Scholar]
- 8.Burns PB, Rohrich RJ, Chung KC. The levels of evidence and their role in evidence-based medicine. Plast Reconstr Surg. 2011;128:305–310. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Little Z, Newman S, Dodds A, et al. Increase in quality and quantity of orthopaedic studies from 2002 to 2012. J Orthop Surg (Hong Kong). 2015;23:375–378. [DOI] [PubMed] [Google Scholar]
- 10.Yarascavitch BA, Chuback JE, Almenawer SA, et al. Levels of evidence in the neurosurgical literature: more tribulations than trials. Neurosurgery. 2012;71:1131–1137; discussion 1137. [DOI] [PubMed] [Google Scholar]
- 11.Lau SL, Samman N. Levels of evidence and journal impact factor in oral and maxillofacial surgery. Int J Oral Maxillofac Surg. 2007;36:1–5. [DOI] [PubMed] [Google Scholar]
- 12.Coroneos CJ, Ignacy TA, Thoma A. Designing and reporting case series in plastic surgery. Plast Reconstr Surg. 2011;128:361e–368e. [DOI] [PubMed] [Google Scholar]
- 13.Drolet BC, Lorenzi NM. Registries and evidence-based medicine in craniofacial and plastic surgery. J Craniofac Surg. 2012;23:301–303. [DOI] [PubMed] [Google Scholar]
- 14.Song JW, Chung KC. Observational studies: cohort and case-control studies. Plast Reconstr Surg. 2010;126:2234–2242. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.The periodic health examination. Canadian task force on the periodic health examination. Can Med Assoc J. 1979;121:1193–254. [PMC free article] [PubMed] [Google Scholar]
- 16.Loiselle F, Mahabir RC, Harrop AR. Levels of evidence in plastic surgery research over 20 years. Plast Reconstr Surg. 2008;121:207e–211e. [DOI] [PubMed] [Google Scholar]
- 17.Grol R, Wensing M. What drives change? Barriers to and incentives for achieving evidence-based practice. Med J Aust. 2004;180(Suppl 6):S57–S60. [DOI] [PubMed] [Google Scholar]
- 18.Stewart MG. Reporting levels of evidence. Laryngoscope. 2014;124:358. [DOI] [PubMed] [Google Scholar]
- 19.Cashin MS, Kelley SP, Douziech JR, et al. The levels of evidence in pediatric orthopaedic journals: where are we now? J Pediatr Orthop. 2011;31:721–725. [DOI] [PubMed] [Google Scholar]