Bibliometric analysis of the 3-year trends (2018–2021) in literature on artificial intelligence in ophthalmology and vision sciences

Abstract

Objectives

The objective of this analysis is to present a current view of the field of ophthalmology and vision research and artificial intelligence (AI) from topical and geographical perspectives. This will clarify the direction of the field in the future and aid clinicians in adapting to new technological developments.

Methods

A comprehensive search of four different databases was conducted. Statistical and bibliometric analysis were done to characterise the literature. Softwares used included the R Studio bibliometrix package, and VOSviewer.

Results

A total of 3939 articles were included in the final bibliometric analysis. Diabetic retinopathy (391, 6% of the top 100 keywords) was the most frequently occurring indexed keyword by a large margin. The highest impact literature was produced by the least populated countries and in those countries who collaborate internationally. This was confirmed via a hypothesis test where no correlation was found between gross number of published articles and average number of citations (p value=0.866, r=0.038), while graphing ratio of international collaboration against average citations produced a positive correlation (r=0.283). Majority of publications were found to be concentrated in journals specialising in vision and computer science, with this category of journals having the highest number of publications per journal (18.00 publications/journal), though they represented a small proportion of the total journals (<1%).

Conclusion

This study provides a unique characterisation of the literature at the intersection of AI and ophthalmology and presents correlations between article impact and geography, in addition to summarising popular research topics.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Bibliometric analysis as a method of characterising research in a field has become increasingly popular in recent years. Some bibliometric analyses on the body of ophthalmological literature have been published in specialised areas, as well as a small number in the intersection of artificial intelligence (AI) and ophthalmology.

WHAT THIS STUDY ADDS

• This study will provide a more recent and comprehensive profile of the intersection of AI and ophthalmology than previous studies, as well as examining a broader range of subspecialties and data sources.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• A better understanding of the existing literature on AI will provide insight into the growing influence of AI on ophthalmology, and will allow medical researchers and academics to anticipate emerging areas of research and allocate funds more effectively.

Introduction

Coined over 60 years ago by McCarthy and Minsky, the term artificial intelligence (AI) refers to the ability of a computer system to complete complex tasks normally requiring human abilities.¹ The popularity of this idea has grown in medicine in recent years as there is great potential for the increase in the efficiency of medical systems via AI, particularly in the areas of visual processing for diagnosis and determination of treatment pathways. To date, AI has been applied to ophthalmology with great efficacy in diagnosis of common diseases such as diabetic retinopathy, retinopathy of prematurity, glaucoma and macular degeneration.² A review by Grzybowski et al suggested that recent diagnostic software for diabetic retinopathy demonstrated a sensitivity of 87.0% and a specificity of 96.8%.³

Bibliometric analysis as a method of characterising research in a field has become increasingly popular in recent years.⁴ Previously published bibliometric analyses in ophthalmology and intersecting fields include an analysis on uveal melanoma literature, and keratoconus.^{5 6} In particular, the growing use of AI in ophthalmology has been profiled by AlRyalat et al who performed a comparative bibliometric analysis between the fields of glaucoma research and AI.⁷ Boudry et al have also demonstrated the growth of AI in the field of ophthalmology over several decades between 1966 and 2019.⁸

Here, we aim to provide a bibliometric profile of the intersection of ophthalmology and AI. Our study complements previous studies in this area by examining a more recent timeframe (2018 to August 2021) across a broader range of data sources and all subspecialties in ophthalmology. A better understanding of the existing literature on AI will provide insight into the growing influence and importance of AI on the field of ophthalmology. This will allow medical researchers and academics to anticipate emerging areas of research and allocate funds more effectively, to seek out research partners and institutions with common interests, and will allow the medical community to adapt to new technologies and integrate them into the future model of patient care.

Methods

This is a bibliometric analysis of articles relating to AI technology and ophthalmology and vision research. A detailed review of the bibliometric analysis study methods is reported elsewhere.⁹ The protocol for this study was also prospectively registered on Open Science Framework registry (https://doi.org/10.17605/OSF.IO/BZ9YJ).

Search strategy

A comprehensive search was conducted in Web of Science, Scopus, Dimensions and Cochrane from 1 January 2018 up to 4 August 2021. These specific databases were chosen as they encompass a wide selection of journals and articles pertaining to the selected topics and are compatible with a wide variety of bibliometric analytic softwares.^{10 11} A 3-year timeline for the citation analysis was chosen with regard to the feasibility of analyses as well as its focused overview of the latest and most relevant technology in AI and ophthalmology. Search strategy keywords were carefully selected from relevant literature and online medical and computer science glossaries to ensure only relevant documents were analysed. No language or study design restrictions were placed on the search strategy. The details of the search query are provided in online supplemental file 1.

Screening

All citations were uploaded to the DistillerSR software and deduplicated.¹² Following de-duplication, all articles were screened by title and abstract by a single reviewer for relevance. More information on the methods of extraction and data-cleaning processes are included in online supplemental file 2. Only articles directly pertaining to the field of ophthalmology and AI were included, and given that each article had to meet certain search criteria to be included in the preliminary dataset, articles passing the screening either clearly fell within the scope of ophthalmology and AI or did not.

Analytic methods

Several analytic methods were applied to this dataset to elucidate the present focus of the field and its future direction. Preliminary analyses were applied to the dataset using RStudio to obtain the number of articles and mean number of citations per year. Then charts displaying the most popular journals and countries and their gross publications were produced. Journals were categorised by topic and then an analysis was conducted using Excel. The journals contained in the dataset were categorised as belonging to medicine (M), vision (V), computer science (CS), engineering (E), artificial intelligence (AI) and general science (G). Journals belonging to both medicine and computer science were labelled as intersectional (I). A metric measuring average publications per journal, and by extension the significance of that journal in the field, was calculated by summing all the articles and then dividing by the number of journals in that category. This value corresponds to the average number of articles per journal in that category.

The international distribution of the publications was analysed. The raw number of publications per country was extracted along with the number of mean citations in the literature for each country. The countries were ranked by the number of publications, the number of citations to those publications and the average number of citations per publication based on the principal investigator. A statistical analysis was performed on the dataset to investigate if a statistically significant correlation existed between gross number of publications by a country and their average number of citations.

The data including the countries, their total number of articles published, and their average citations was exported, and a citation network was created using the VOSviewer software. A statistical analysis comparing countries by their published output and its average citations was performed. This was done via a Spearman rank correlation test. The null hypothesis (H₀) was that there is no correlation between the number of publications produced by a country and the average number of citations received by those publications (ie, that the value of r is 0). Further, single country publication (a ratio representative of the proportion of total publications with intra-national collaborations) and multiple country publication (MCP, the proportion of total publications with international collaborators) ratios were used to investigate the linkage between international collaboration and rate of citation. Average citations by country were graphed against MCP to see if correlation between the two variables could be established.

Author keywords were extracted, and a co-occurrence map was created with all words with a minimum of five connections to others. A link between words is established if two keywords are listed in conjunction by more than one author. The number of occurrences of each keyword was represented by the size of the nodes.

Results

From the initial search, 5917 articles were obtained from Dimensions, 5771 from Scopus, 3717 from Web of Science and 136 from Cochrane. Following deduplication, and screening, 3939 articles were included in the analysis, with 433 articles collected from 2018, 697 articles from 2019, 1416 from 2020 and 1393 from 2021.

The number of journals and articles in each discrete category is summarised in table 1. The highest number of articles were categorised as medicine, with computer science being second and vision being a close third. No journals were categorised as specialising in vision and AI, while only two journals were categorised as specialising in vision and computer science. Vision and computer science had the highest average number of publications (18.00 publications/journal), although it accounted for less than 1% of the total journals. The second highest average number of publications was in the vision (V) category, with 11.36 publications/journal. General medical journals (M), while accounted for the highest number of journals, had only 2.73 publications/journal whereas medical and computer science journals had an average of 8.19 publications/journal. The top three journals were Translational Vision Sciences and Technology, categorised as vision with 133 articles; Scientific Reports categorised as general science with 129 articles; and IEEE Access categorised as engineering with 120 articles. Below, we present the top five articles from IEEE Access, the engineering journal with the greatest number of publications, to exemplify the growing popularity of the field of ophthalmology and AI outside of medicine.

Table 1.

Number of journals and articles in each category

Category	Journals (n)	Articles (n)
Medicine (M)	371	949
Vision (V)	128	1454
Computer science (CS)	141	446
Engineering (E)	49	182
Artificial intelligence (AI)	47	124
General (G) nature, science, etc	120	306
Intersection of CS and medicine (I)	95	667

Based on corresponding authors’ affiliations, China (946, 25%) and the USA (719, 19%) produced the most number of publications overall (table 2). The rest of the publications came from a wide range of countries in Europe and Asia, with no country (aside from India) accounting for more than 5% of the total number of publications (figure 1). Austria had the highest average article citations, collaborated with authors from nine different countries, had 42 articles by corresponding authors, and 138 total publications. China collaborated with 17 distinct countries, had 946 articles by corresponding authors and had 2911 total publications (figure 2). When comparing countries by their published output and average citations, the findings did not reveal a significant correlation (p value=0.866, r=0.038). This suggests that there is no statistically significant correlation between gross amount of literature published by a country and average number of articles citations for that country, which is a surrogate metric for literature quality.

Table 2.

Countries ranked in order of most publications, accompanied by citation data

Publication rank	Citation rank	Average article citations	Corresponding author’s country	Publications	Total citations	Average article citations
							1	2	11	China	946	7769	8.21
2	1	6	USA	719	8108	11.28
3	4	21	India	367	1894	5.16
4	6	16	Korea	178	1190	6.69
5	3	3	UK	150	2254	15.03
6	8	13	Japan	134	998	7.45
7	9	10	Germany	106	871	8.22
8	11	14	Spain	106	747	7.05
9	5	2	Singapore	95	1460	15.37
10	7	5	Australia	94	1116	11.87
11	14	23	Turkey	85	372	4.38
12	13	20	Italy	82	466	5.68
13	10	4	Canada	52	772	14.85
14	15	17	Brazil	51	329	6.45
15	17	19	France	51	311	6.10
16	18	18	Iran	47	291	6.19
17	12	1	Austria	42	742	17.67
18	19	12	Pakistan	34	259	7.62
19	21	15	Saudi Arabia	34	235	6.91
20	16	8	Netherlands	33	312	9.46
21	23	22	Egypt	26	127	4.89
22	20	7	Switzerland	26	252	9.69
25	22	9	Portugal	24	226	9.42

Figure 1.

Breakdown of percentage of total number of publications identified based on the country of the corresponding author.

Figure 2.

Countries were clustered via unique colours representing the average number of citations for that country. Purple countries had the highest average citations (>12), light blue countries had between 8 and 12 average citations, light green countries had between 4 and 8, and red countries had the fewest, between 0 and 4. The sizes of the country names indicate their gross number of publications, the larger the label being correlated with the total number of publications for that country. Links between countries indicate which tend to collaborate, and the thickness of the linkage corresponds to the strength of the connection. Countries which collaborate on many papers will have a thicker connecting line. Links between countries are only displayed if there has been a minimum of five collaborative publications.

Austria had a higher MCP/total fraction, at 0.4762, as compared with China, which had an MCP/total fraction of 0.243. Plotting countries by their average citations per publication against their proportion of international collaborations yielded a weakly positive correlation coefficient of R²=0.283 (figure 3). This suggests that there is association between number of international collaborators and global popularity of literature.

Figure 3.

A plot depicting countries by their average citations per publication against their proportion of international collaborations.

The top five most frequent indexed keywords included ‘deep learning’ (677, 11%), ‘diabetic retinopathy’ (391, 6%), ‘machine learning’ (364,6%), ‘artificial intelligence’ (332, 5%) and ‘optical coherence tomography’ (311, 5%, figure 4). Diabetic retinopathy was the most frequently occurring ophthalmological disease by a margin of 291 occurrences (5% of the top 100 occurrences), with ‘age-related macular degeneration’ being the next most frequently occurring ophthalmic disease at only 100 occurrences.

Figure 4.

A co-occurrence network showing the top 20 keywords among all listed author keywords. Larger nodes correspond to a higher number of occurrences of that keyword, thicker connections indicate a higher frequency of two keywords being listed together.

Discussion

We conducted a bibliometric analysis of the intersection of ophthalmology and AI between January 2018 and August 2021. Many aspects of the dataset were analysed in order to gain both quantitative and qualitative insights. In particular, investigation into countries of publication and their correlation (or lack thereof) with literature quality was performed, and it was found that smaller countries tended to produce more highly cited literature. There was a direct correlation between country population and gross quantity of published literature. Furthermore, countries with more international collaboration tended to have higher average article citations. With respect to research topics, the most common application of the AI technology to ophthalmology tended to be in diagnostic imaging.

Our findings suggested that the field of ophthalmology and AI has been growing at an exponential rate as predicted by Lotka’s law until 2020 when the scientific production dropped sharply.¹³ The authors hypothesise that there are two main reasons for this finding. First, it is likely that SARS-CoV-2 affected scientific production in the field of ophthalmology and AI as the broad scientific community shifted to focus on developing a body of research on the novel virus. Second, articles were only collected up to August 2021, and had the articles been collected up to December it is predicted that the growth rate of the field would have increased rather than decreased, though likely not with the same increase in rate as in previous years.

It was noted in our analysis that China and the USA collectively account for over 40% of the literature in the dataset. This is not surprising in consideration of the population size and large number of research institutions in both countries. Within the dataset there is an over-representation in the advanced economies of Southeast Asia, where Japan, Korea and Singapore accounted for more research in this field than the UK and Germany combined.

Popular AI ranking indices have consistently placed the USA and China at the top of research, development and implementation of new AI technologies over the past 5 years, with Japan and Korea ranking in the top 10.^{14 15} According to the Stanford AI index, in 2021, East Asia accounted for 26.7% of all published academic articles pertaining to AI globally, while the USA accounted for 14.0%.^{14 15} Further, global AI publications have seen a steep growth curve recently, with total international journal publications having increased 2.5 times since 2015. This rapid growth is seen in conjunction with an exponential increase in AI patent filings globally, with a compound annual growth rate of 76.9% between 2015 and 2021.¹⁶ As more research is published, more innovation is spurred, while new technology promotes new research, in a positive and fast accelerating feedback loop. In 2021, China held the greatest number of AI patent filings, while the USA had the most granted patents as a percentage of the world total filed and granted patents.¹⁶

We have used the number of citations as a measurement of literature impact. Previous studies have suggested that the correlation between citation numbers and value of scientific knowledge and influence is not perfect, and citations might also be influenced by factors such as author prominence and randomness.¹⁷ Although, there are important factors that should be considered when using number of citations as an absolute measure of literature quality,¹⁷ the large size of our data set may give an accurate overall picture of global impact.¹⁸ Our findings showed no statistically significant correlation between the gross number of publications for a country and mean number of citations. This result indicates that while China and the USA may produce nearly half of the articles in this field, they do not also attract the most citations. Our findings suggested that research from countries such as Austria, had the most citations per publication and high proportional international collaboration than China. It is well-established for scientometric characteristics that collaboration between institutions, in particular internationally, tends to produce research that is cited more frequently than less-collaborative work.¹⁹ As such China and the USA, although produce most publications they tend to collaborate less with institutions in other countries. The reasons behind this effect are multi-faceted and beyond the scope of this paper. Besides the cultural and geographic factors that would limit their international connections, both China and the USA have many universities within their own borders with whom to collaborate. In contrast, the high impact of smaller countries such as Singapore and Austria are surrounded by many other countries to collaborate with and have some of the highest citations-per-publication alongside a high proportion of MCPs.

We noted that the most collaborative countries, as well as those with the highest average citation impact, tend to be smaller countries in Europe with the exception of Singapore. As an Asian city-state with a British colonial heritage, Singapore’s cultural-linguistic connections both to Europe and to South-East Asia enable it to have the second-highest citations-per-paper of all the countries in this survey, showing how collaborations are more important than size. We also found that while China is the most productive country, it lags behind the only other country of comparable output (the USA) which tends to have more international collaborations. This is corroborated by two popular AI index reports, which find that while China leads the USA in gross publications, the USA ‘leads on the most significant research into cutting-edge developments’.^14–16

From the co-occurrence network created diabetic retinopathy is most connected with the terms ‘deep learning’, ‘machine learning’ and ‘artificial intelligence’. Further, other popular terms relate to types of diagnostic imaging, such as ‘optical coherence tomography’ and ‘image segmentation’. This implies that the focus of the field is on applications of AI to diagnosis, and creation of algorithms for automating diagnosis and triage of ophthalmic diseases. Many medical fields follow a progression of care model, where diagnosis is the first step, followed by prognostication, development and administration of treatment protocols, and surgical management if necessary. As such, new technology may begin to develop first in the areas of need, in the case of the field of ophthalmology this is diagnosis and triage. Additionally, there is more cost and resource associated with research in robotics than computer research.²⁰

Conclusion

This paper presents an in-depth bibliometric analysis of literature in the field of ophthalmology and AI. Articles were collected from a wide variety of sources over a 3-year time period in order to gain a detailed perspective on the current state of the technology and its future trajectory. We have characterised the field via both qualitative and quantitative methods. We have investigated trends in topics in the field, and which varieties of research are currently gaining the most traction and may have practical application in the near future. We have determined that the USA and China together produce the highest volume of research, though they have among the lowest rates of international collaboration, while smaller countries with high rates of international collaboration such as Singapore and Austria produce the most cited research. Increasing international collaborations may be an effective way for geographic areas which are behind in this field to strengthen their body of research in AI and ophthalmology. Encouraging researchers to provide open source access to research, particularly to newly developed code for AI algorithms, can aid in increasing participation and collaboration from previously dormant countries. These findings will aid the ophthalmology medical and research community in adapting their practices to the changing landscape of vision care.

Data availability statement

Data are available upon reasonable request. The data is available upon request sent to the corresponding author.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

Ethics approval from our Institutional Review Board was not required as this is a review of published studies and does not involve human subjects.