An Analysis of the Top-cited Articles in Anesthesiology Education Literature

Abstract

Background:

In the same way that impact factor is calculated for journals, the number of citations an article receives can indicate its influence or value to a particular field. This study was designed to identify the most frequently cited articles in anesthesiology education to yield insight into which articles have been most useful for researchers in ongoing research and publication.

Methods:

The Web of Science database was searched to capture the top-cited articles in anesthesiology education both in anesthesiology and nonanesthesiology journals. Results were sorted by the most frequently cited. The top 40 cited articles were identified. Articles were included if they (1) related to anesthesiology or included anesthesiologists as subjects and (2) were related to the education of current or future anesthesiologists. The full text was analyzed, and themes were identified.

Results:

There was a total of 2923 citations of articles in anesthesiology journals and 924 citations of articles in nonanesthesiology journals. Thirty-two of 40 articles (80%) were research studies. Twenty-four of 40 (60%) were about teaching methods. Twenty-five of 40 (63%) focused on simulation, and 31 of 40 (78%) had residents as the subjects. Twenty-eight of 40 (70%) articles were about either case management (15) or learning procedures (13).

Conclusions:

This study identifies the most widely cited articles in anesthesiology education. Common themes included procedural learning, interventional research study designs, simulation, and studies involving residents as subjects. This article may be a resource to anesthesiology education researchers to identify what articles are widely cited by other researchers.

Introduction

Like the field of medicine itself, there is a move in medical education to base educational practices on the best available evidence.¹ While groups like the Best Evidence Medical Education (BEME) Collaboration are working toward making best evidence available through systematic reviews, the types of research that education researchers find useful in a particular field may or may not be reflected in these systematic reviews.¹ Identifying articles most often cited in a particular field of medical education may provide important insights on which publications have been useful for education researchers and what studies have had the most impact on ongoing publications.

In Emergency Medicine, a review of the top-cited articles in the education literature was conducted in 2016.² The authors hoped the publication would serve as a resource for educators and researchers to identify trends that may be used to guide further research and publishing efforts in the field.² Munzer et al performed their research in response to an overall evaluation of top-cited articles in medical education by Azer in 2015.³ This article attempts to extend this model of analysis to the field of anesthesiology.

This study was designed to identify and analyze the content of the top 40 most frequently cited articles in anesthesiology education, both in anesthesiology journals and nonanesthesiology journals, to identify which articles have been most useful for researchers in ongoing research and publication.

Materials and Methods

Search Strategy

The Web of Science (Clarivate Analytics) database was searched in March 2017 to capture the top-cited articles in anesthesiology education published in anesthesiology and nonanesthesiology journals. The identification of articles, which was based on a modified version of the search methods used in Munzer et al, consisted of 2 independent searches.² To identify articles in anesthesiology journals, the first step was to identify peer-reviewed anesthesiology-focused journals. Twenty-eight journals were identified in Journal Citation Reports (Clarivate Analytics) and approved by the authors. Next, this set of 28 peer-reviewed anesthesiology-focused journals was searched with a set of education-related terms to create the list of articles for anesthesiology journals. To identify articles in nonanesthesiology journals, the second search combined anesthesiology terminology with the same set of education terms, and excluded articles published in the previously identified 28 anesthesiology journals. Results in both searches were sorted from most to least frequently cited. Non-English studies were excluded. No date restrictions were included. See Appendix 1 for the complete, reproducible searches.

After sorting the results, 2 authors (LZ and EP) independently reviewed the abstracts of each article retrieved through the searches and assessed whether or not it met the inclusion criteria. They each reviewed abstracts until both identified 60 articles, 30 from the anesthesiology journal search and 30 from the nonanesthesiology search. An article was included if it was (1) related to anesthesiology or included anesthesiologists as research subjects and (2) was related to the education of current or future anesthesiologists. Education topics were defined to include assessment and evaluation of learners, description and evaluation of educational activities, teaching methods, and educational design. Education topics were chosen based on similar prior research.^2,3 Articles were excluded if they did not directly relate to methods of education or were about nursing education or other subjects focused on allied health professionals. Articles were included if both authors found the article to meet the inclusion criteria. Any disagreements were discussed, and a third author reviewed the article to decide if it should be included. Though the goal was to identify the top 20 articles in anesthesiology and nonanesthesiology journals, the 2 authors originally identified 30 articles each, in anticipation that some articles deemed to meet the inclusion criteria by one author would be determined not to meet the inclusion criteria by the other author and the third author reviewer.

Article Review

Authors were split into pairs. Each author pair was assigned to review the full text of 10 articles, 5 from the anesthesiology journal list and 5 from the nonanesthesiology journal list. Each author was asked to independently review and catalog the following information: (1) article name, (2) first author, (3) source journal, (4) year of publication, (5) number of citations, (6) funding source (if applicable), (7) journal impact factor, (8) journal discipline, (9) article type(s), (10) educational content, (11) subjects, (12) research method (if applicable), and (13) topic of article/additional notes. Each pair discussed the collected information for each article. If they were unable to reach consensus in any category, a third author reviewed the full text of the article and rendered a decision.

All authors met either in person or through conference call to discuss the findings, themes, and categories. Figure 1 shows the methodology for article selection and review.

Figure 1.

Methodology for article selection and review.

Statistical Analysis

Pearson correlation coefficients were calculated using SAS v. 9.4 (SAS Institute, Cary, NC) to determine if the age of the article or impact factor of the journal were correlated to the number of citations.

This manuscript adheres to the applicable PRISMA guidelines. Since there were no human subjects, the authors did not need IRB approval.

Results

The 20 articles from anesthesiology journals^4–23 and 20 from non-anesthesiology journals^24–43 are included in Tables 1 and 2, respectively. There was a total of 2923 citations of articles in anesthesiology journals and 924 citations of articles in nonanesthesiology journals. The average impact factor for the anesthesiology journals was 4.649, and the average for nonanesthesiology journals was 3.092.

Table 1.

Most Cited Education Articles from Anesthesiology Journals

Table 2.

Most Cited Education Articles from Other (Nonanesthesiology) Journals

Table 1.

Most Cited Education Articles from Anesthesiology Journals

Table 2.

Most Cited Education Articles from Other (Nonanesthesiology) Journals

The top-cited articles were published in 7 anesthesiology journals (Table 3). Anesthesiology had the most articles from the top-cited list with 10. For nonanesthesiology journals, 12 journals were represented. Medical Education and Academic Medicine had the most articles, each with 4.

Table 3.

List of Journals and the Number of Articles per Journal

The article that received the most citations in anesthesiology journals was “Anaesthetists' Non-Technical Skills (ANTS): evaluation of a behavioural marker system” by Fletcher et al, published in the British Journal of Anaesthesia in 2003 and was cited 347 times.⁴ The article cited the most in nonanesthesiology journals was “Anesthesia crisis resource-management training -teaching anesthesiologists to handle critical incidents” by Howard et al, published in Aviation Space and Environmental Medicine in 1992 and cited 290 times.²⁴

Of all the articles included in both lists, 32 out of 40 articles (80%) were research studies. Only 1 article of 20 (5%) from the anesthesiology list was classified as a description of a curriculum with associated data, while for the articles in the nonanesthesiology list, 1 of 20 (5%) was a guideline; 2 of 20 (10%) were review articles; and 3 of 20 articles (15%) were descriptions of curriculum with associated data. One of 40 articles did not fit into our preset categories and was categorized as a descriptive article as it explained the use of a flight simulator for anesthesiology training.³⁰ For funding sources, 16 of the 40 articles (40%), 8 from both lists, disclosed foundation or government external support. Two disclosed internal support from departments.

The characteristics of the articles from both anesthesiology and nonanesthesiology lists are shown in Table 4. Looking solely at the 20 articles from the anesthesiology journals, half focused on learning procedures, such as direct laryngoscopy, regional anesthesiology procedures, and placing invasive lines. Fourteen of 20 (70%) included simulation, and 17 of 20 (85%) articles included residents as subjects. Most articles focused on teaching methods, with 13 of the 20 (65%) articles focusing on how to teach learners.

Table 4.

Characteristics for Articles from Anesthesiology and Nonanesthesiology Journals

Similarly, for articles from nonanesthesiology journals, 11 of 20 (55%) focused on teaching methods. Fourteen of 20 (70%) included residents, and 11 of 20 (55%) included simulation. There was a greater variety of topics in the nonanesthesiology journal group with a total of 8 general topics covered. Forty-five percent (9 of 20) focused on case management or general practice.

Based on the Pearson correlation coefficient (r) for both anesthesiology and nonanesthesiology journals, neither year of publication nor journal impact factor are correlated with the number of citations that an article received. For nonanesthesiology journals, the correlation between number of citations and year of publication was r = −0.24, P = .294 and the correlation between number of citations and journal impact factor was r = −0.24, P = .302. For anesthesiology journals, the correlation between number of citations and year of publication was r = −0.26, P = .266 and the correlation between number of citations and journal impact factor was r = 0.18, P = .442.

Discussion

The purpose of this study is to identify and examine the characteristics of the most cited articles in anesthesiology education. This work serves as a resource for education researchers to access and review the most cited articles, examine characteristics of these articles, recognize trends, and better understand which journals have published highly cited work in anesthesiology education.

The review uncovered some trends in the top-cited anesthesiology education publications. Content that focused on learning clinical skills or medical procedures either through simulation or direct patient care comprise a large portion of articles from both lists. This may be because these studies are easier to design and execute, but it also may reflect the prioritizing of patient safety with regard to learning medical procedures.

The most frequently cited educational methodology was simulation; in particular, the first use of a simulator in anesthesiology, while published in 1987, was one of the most widely cited articles. The articles that were research studies showed simulation being used to study a variety of different topics, like training in nontechnical skills, crisis resource management, learning procedures, and general case management. Researchers may rely on simulation to assess teaching methods and learners because of the ability to control the complex nature of the perioperative environment.

Another identifiable trend was the lack of systematic reviews or other article types that synthesize the existing evidence in the literature. This might indicate that articles like systematic reviews, education guidelines, or general articles about learning theory might not receive as much attention. These types of articles are critical because they synthesize the existing evidence and are able to guide evidence-based practice and future research.⁴⁴Thirty-one of 40 (78%) most-cited articles included residents as the subjects. This demonstrates a clear focus on graduate medical education, which may highlight an opportunity for researchers in undergraduate and continuing medical education. Changes in medical school curricula across the nation including earlier exposure to clinical specialties may provide greater opportunity to see how earlier exposure to the practice of anesthesiology complements basic medical training.⁴⁵ In addition, with the increased emphasis on Maintenance of Certification, investigating how to help physicians in lifelong learning is another opportunity for research.⁴⁶Articles in the anesthesiology journals were cited more than the articles in the nonanesthesiology journals. This is in contrast to the findings in Munzer et al who showed that articles from nonemergency medicine journals received more citations.²

This may be because the journal, Academic Emergency Medicine, focuses on education in emergency medicine. Without an indexed journal dedicated to anesthesiology education prior to the Journal of Education in Perioperative Medicine, it is possible that researchers may have had to look for journals outside of anesthesiology to publish their work.

Limitations

There are limitations to this study. First, the reason or in what capacity these works were cited is unknown since the original articles in which these works were cited were not reviewed. Frequency of citations might not be the best method for determining influence. Furthermore, some articles might lead to wide dissemination and implementation into practice, but may not lend themselves to further study and therefore would be less likely to be cited in future work.

Also, 1 database was used for the searches. The Web of Science is currently the only database that allows sorting by number of citations and the citation numbers are considered accurate. In addition, only articles written in English were included, which further limited the scope. If other databases were used and other languages included, there might have been other articles or the rank order might have changed.

Even though there were at least 2 authors who reviewed all articles in the original search results, there was subjectivity to the creation and application of our inclusion criteria. This may have resulted in some articles being excluded although others might consider them educational in scope.

Lastly, 1 article with a guideline for teaching transesophageal echocardiography was printed in both anesthesiology and nonanesthesiology journals.²⁷ Only the article in the nonanesthesiology journal, Journal of The American Society of Echocardiography, was included because the article did not have enough citations to be included in the anesthesiology top-cited list. In addition, only the number of citations the article received in the nonanesthesiology journal was used for sorting purposes. Since this article and any other article printed in 2 separate journals could be cited from either journal, that could influence the ordering and inclusion of publications in this study.

Conclusion

This study identifies and examines the characteristics of the most widely cited articles in anesthesiology education both in anesthesiology journals and nonanesthesiology journals. Common themes included procedural learning, interventional research study designs, simulation, and studies involving residents as subjects. This article may be a resource to anesthesiology education researchers to identify what articles are widely cited by other researchers.

Appendix

Appendix 1

Anesthesiology Education Articles

The Web of Science (Clarivate Analytics) database was searched in March 2017 to capture the top cited articles in anesthesiology education published in anesthesiology and non-anesthesiology journals. The identification of studies, which was based on a modified version of the search methods used in Munzer et al., consisted of two independent searches. The first search crossed a set of 28 peer-reviewed anesthesiology-focused journals, which were identified in Journal Citation Reports (Clarivate Analytics) and approved by the author group, with a set of education-related terms. The second search combined anesthesiology terminology with the education concept block, and excluded articles published in the previously identified set of 28 anesthesiology journals. Results in both searches were sorted by the most frequently cited. Non-English studies were excluded. No date restrictions were included, but the results prioritize older studies that have had time to be cited frequently. See Appendix 1 for the complete, reproducible searches.

Searches

Anesthesiology education articles in anesthesiology journals

Anesthesiology education articles in non-anesthesiology journals