Skip to main content
NCBI home page
JMIR Medical Education logoLink to JMIR Medical Education
. 2024 Dec 11;10:e58165. doi: 10.2196/58165

Topics and Trends of Health Informatics Education Research: Scientometric Analysis

Qing Han 1,✉
Editor: Blake Lesselroth
Reviewed by: Junhui Wang, Mohammad Sharif Zami
PMCID: PMC11669873  PMID: 39661981

Abstract

Background

Academic and educational institutions are making significant contributions toward training health informatics professionals. As research in health informatics education (HIE) continues to grow, it is useful to have a clearer understanding of this research field.

Objective

This study aims to comprehensively explore the research topics and trends of HIE from 2014 to 2023. Specifically, it aims to explore (1) the trends of annual articles, (2) the prolific countries/regions, institutions, and publication sources, (3) the scientific collaborations of countries/regions and institutions, and (4) the major research themes and their developmental tendencies.

Methods

Using publications in Web of Science Core Collection, a scientometric analysis of 575 articles related to the field of HIE was conducted. The structural topic model was used to identify topics discussed in the literature and to reveal the topic structure and evolutionary trends of HIE research.

Results

Research interest in HIE has clearly increased from 2014 to 2023, and is continually expanding. The United States was found to be the most prolific country in this field. Harvard University was found to be the leading institution with the highest publication productivity. Journal of Medical Internet Research, Journal of The American Medical Informatics Association, and Applied Clinical Informatics were the top 3 journals with the highest articles in this field. Countries/regions and institutions having higher levels of international collaboration were more impactful. Research on HIE could be modeled into 7 topics related to the following areas: clinical (130/575, 22.6%), mobile application (123/575, 21.4%), consumer (99/575, 17.2%), teaching (61/575, 10.6%), public health (56/575, 9.7%), discipline (55/575, 9.6%), and nursing (51/575, 8.9%). The results clearly indicate the unique foci for each year, depicting the process of development for health informatics research.

Conclusions

This is believed to be the first scientometric analysis exploring the research topics and trends in HIE. This study provides useful insights and implications, and the findings could be used as a guide for HIE contributors.

Keywords: health informatics education, scientometric analysis, structural topic model, health informatics, medical informatics, medical education

Introduction

Health informatics is at the crossroad of information science, computer science, data science, medicine, and health care, with great improvements in a wide range of health-related areas, including nursing, health care, public health, clinical decision, and biomedicine [1]. There is a growing need for professionals to be able to address health informatics issues in health-related fields through the design, development, implementation, evaluation, and use of innovative information technologies and applications.

Consequently, many academic and educational institutions are making significant contributions toward training health informatics professionals by offering educational programs or courses in the field [2]. The nature of health informatics attracts both health care and IT professionals as specialists in a highly interdisciplinary field. Health care professionals, on the other hand, need to grasp the opportunities and limitations that technology can bring to health care [3]. There are several ways to enter health informatics education (HIE) in different countries around the world, with the most common pathway being postgraduation following the completion of a bachelor’s degree in health informatics or a related degree in health care, life sciences, cognitive science, computer science and IT, health information management, and others, with associated work experience [4-10].

Due to its multidisciplinary foundations and diverse application areas, HIE has captured the interest of researchers from numerous areas. Engagement from these researchers has been beneficial to the field as it has led to the development of a rich research literature in many journals. This has resulted in the vast and flourishing development of this field, with the research community demonstrating significant growth and a constant increase in the scientific literature. As research in HIE continues to grow, it is essential to have a clearer understanding of its status, trends, and topics across the research field for researchers and participants.

This study explores the topic structure and evolutionary trends of HIE research by conducting a topic-based scientometric analysis of scientific articles concerning HIE from 2014 to 2023. Specifically, scientometric analysis is an effective way to evaluate research characteristics and frontiers in a specific research field, with the application of mathematical and statistical methodologies [11]. In particular, it can be employed to obtain a better understanding of what has been investigated in the past and further make predictions about what will happen in the future. Such analysis can depict the trends and distributions of the literature involving scientific impacts and productivity [12]. Meanwhile, topic modeling is a dimension reduction technology to extract hidden themes within a textual dataset, which detects term co-occurrence and assigns terms to the hidden themes [13]. It has been widely applied in scientific research trend analysis and in the identification of emerging topics within a particular research area [14,15]. Although scientometric and topic models have been popularly implemented in a variety of scientific fields, they have not been applied in the research field of HIE. Thus, this study contributes to deepening the understanding of HIE and provides valuable references and recommendations for researchers and potential contributors, focusing on the following questions:

  1. What research topics were the HIE community interested in?

  2. How did such research topics evolve over time?

  3. Which countries/regions and institutions were the major contributors to HIE?

  4. What were the scientific collaborations between countries/regions and institutions?

  5. What were the main research concerns of prolific contributors?

Methods

Analysis Framework

The scientometric analysis framework of this paper is shown in Figure 1, which is mainly divided into 4 parts: data collection, data preprocessing, topic modeling, and results analysis.

Figure 1.

Figure 1

Open in a new tab

The scientometric analysis framework of health informatics education research.

Data Collection

HIE-related scientific articles from 2014 to 2023 were retrieved from Web of Science Core Collection (WoSCC), using the search strategy “Topic=health informatics education, Year Published=2014-2023, Language=English, Document Type=Article.” The search was conducted using the following query: TS = (health informatics education) AND PY = (2014-2023) AND LA = (English) AND DT = (Article).

Only WoSCC was used to retrieve articles owing to its comprehensive collection of multidisciplinary academic journals, its higher quality than other databases, and its status as an authoritative source for citation information, making it a primary scientific database and the most preferred destination for researchers from all domains [16,17]. The search was restricted to this term and did not include other terms related to health informatics, such as medical informatics, nursing informatics, or consumer health informatics, as this search strategy has been shown to be effective for scientometric analysis of health informatics [18,19]. In addition, only research articles were included as they tend to make more original contributions when compared to other types of publications [20]. As a result, a total of 577 articles were retrieved.

It is commonly agreed that titles, abstracts, and article keywords are suitable for conceptual reviews because they usually represent the noteworthy content of articles [21,22]. In particular, abstracts are able to present summaries of articles in terms of research aims and problems, as well as major findings [23,24]. Therefore, in this study, the primary materials of topic modeling were the title, keywords, and abstract of each article. Of the articles collected, 2 without abstracts were excluded, and 575 articles with abstract information were selected for topic modeling.

Data Preprocessing

To improve data quality before conducting topic modeling, preprocessing was performed, which involved the following 3 aspects.

First, the names of countries/regions, institutions, and researchers were extracted from the authors’ addresses, which were further preprocessed to ensure unified expressions. Terms were extracted from the titles, keywords, and abstracts, and were consolidated when they had multiple spellings.

Second, numbers, punctuations, symbols, and stop words (eg, “me,” “I,” “or,” “him,” “a,” and “they”) were deleted to enhance consistency and reduce computational load [25]. Punctuations were removed, and all the remaining terms were converted to lowercase.

Third, following the previous work [20], the weights 0.4, 0.4, and 0.2 were assigned to terms extracted from titles, keywords, and abstracts, respectively. Since in topic modeling, the estimations of document-topic and topic-term distributions are based on the document-term distribution, which is frequency-based, the assignment of weights was performed by multiplying the frequency by weight. For example, for each document, if f1wi, f2wi, and f3wi denote the frequencies of the word wi in the keywords, title, and abstract, respectively, then the weighted frequency of the word wi in the document is denoted by 0.4×f1wi+0.4×f2wi+0.2×f3wi.

Data Analysis

The major research topics within the 575 research articles in relation to HIE were uncovered by using the structural topic model (STM) [26]. The STM belongs to the class of topic models seeking to discover latent thematic clusters within a collection of texts, and is an extension of the latent Dirichlet allocation (LDA) model [27] and the correlated topic model (CTM) [28]. Traditional topic modeling techniques, such as LDA, assume that topics are discrete and independent. However, in many cases, topics may not be independent, and documents may have complex relationships between topics. The STM addresses these limitations by incorporating structural information into the topic modeling process. Besides identifying topics, the STM estimates the prevalence of the found topics by incorporating document-level covariates (eg, time of publication or number of authors).

A diagram of the STM, adapted from a recent study using the STM to analyze smart health care [29], is presented in Figure 2. The process of topic modeling is to estimate the θ and β based on the observed term W, which represent document topic and topic word distributions, respectively, where n∈{1,2,…,N} denotes terms covering a document, k∈{1,2,…,K} is the topic, d∈{1,2,…,D} represents the document number, and z represents the underlying topic allocation of each term.

Figure 2.

Figure 2

Open in a new tab

A diagram of the structural topic model.

During topic modeling, the decision of a value indicating the number of topics is a priority. Previous work has demonstrated that an exclusive reliance on statistical measures could result in a less meaningful model parameter choice [30]. Thus, following the suggestion of previous studies [31], 10 candidate models were adopted with the numbers of topics ranging from 4 to 15. Two domain experts, who have taught health informatics for more than 10 years, independently compared the outputs of the 10 models by evaluating the representative terms and articles to select the best-fitted model based on the following criteria adopted in previous work [32]: (1) Representative terms in each topic could form a meaningful topic together; (2) The top representative articles for each topic were in close relation to the topic; (3) There was limited overlap between topics within a topic model; and (4) All obvious research topics in HIE were included. In this study, both the experts chose the 7-topic model. The output of the model involved topic content and topic prevalence. The 7-topic model showed good semantic consistency within topics and exclusivity between topics.

Topics were labeled manually according to the inspection of the highest ranking 12 terms and 20 articles that consisted of each topic. The labels were individually examined by the 2 domain experts, who then reached an agreement through discussion. Specifically, they first examined the most representative terms for each topic and interpreted the discriminating terms within topics based on their semantic meanings. Then, they examined a sample of representative articles for each of the topics. Finally, they compared the labeling results and discussed the inconsistent labels to unify the results.

In addition to the STM, other methods and tools have also been adopted in this study. On the one hand, several scientometric indicators, namely, article count, citation count, and average citations per article, were used to evaluate the performance of involved publication sources, subjects, institutions, and countries/regions. On the other hand, scientific collaborations among countries/regions and institutions were explored by social network analysis using VOSviewer. VOSviewer generated collaborative networks by representing countries/regions and institutions as nodes, and the collaborative relationships were reflected using lines. The node size indicated the number of articles in which 2 linked countries/regions or institutions collaborated, and colors reflected the countries/regions or institutions.

Results

Annual Trends of Publications

The annual trends of the article counts are depicted in Figure 3. Research in HIE has been growing during the studied period, particularly since 2020, reaching 84 articles in 2023.

Figure 3.

Figure 3

Open in a new tab

Number of articles published over time.

Although there were fluctuations in the data, it is reasonable to conclude that HIE has received a growing interest from the academic community. Therefore, it is highly likely that the amount of HIE research will continue to increase in the near future.

Prolific Countries/Regions and Institutions

In the analyses concerning countries/regions and institutions, all the countries/regions and institutions contributing to each article were included in the data, and the most prolific ones were evaluated using 3 scientometric indicators, namely, article count, citation count, and average citations per article.

A total of 70 countries/regions contributed to the 575 articles, and the top 10 countries/regions ranked by the article count are shown in Table 1. United States had 326 articles, followed by Canada (50 articles) and Australia (45 articles). These top 3 countries were also ranked among the top 3 in terms of citation count, with 4705, 673, and 514 citations, respectively.

Table 1.

Top 10 countries/regions ranked by article count.

Country/region Article count Citation count (ranking) Average citations per article (ranking) Web of Science category (article count)
United States 326 4705 (1) 14.43 (2)

  • Medical Informatics (130)

  • Health Care Sciences Services (115)
Canada 50 673 (2) 13.46 (3)

  • Medical Informatics (19)

  • Health Care Sciences Services (14)
Australia 45 514 (3) 11.42 (6)

  • Medical Informatics (14)

  • Health Care Sciences Services (13)
England 33 369 (5) 11.18 (7)

  • Medical Informatics (9)

  • Health Care Sciences Services (9)
Germany 28 493 (4) 17.61 (1)

  • Medical Informatics (17)

  • Health Care Sciences Services (15)
Netherlands 25 327 (6) 13.08 (4)

  • Medical Informatics (17)

  • Health Care Sciences Services (15)
China 23 289 (7) 12.57 (5)

  • Medical Informatics (8)

  • Health Care Sciences Services (4)
Brazil 20 168 (8) 8.40 (10)

  • Nursing (9)

  • Medical Informatics (5)
Sweden 17 153 (9) 9.00 (9)

  • Health Care Sciences Services (9)

  • Medical Informatics (8)
Spain 13 145 (10) 11.15 (8)

  • Medical Informatics (6)

  • Health Care Sciences Services (6)
Open in a new tab

Regarding average citations per article, the top 3 countries/regions included Germany (17.61), United States (14.43), and Canada (13.46). Although Germany had fewer articles (n=28) as compared to the top 3 countries, their articles had higher quality as indicated by the average citations per article. By considering all the abovementioned indicators, the performance of Germany is worth highlighting, indicating its great contribution to the research field of HIE.

The 575 articles concerning HIE were contributed by 200 institutions, among which the top 10 are shown in Table 2. Among the listed 10 institutions, 9 are from the United States, which again indicates the dominant position and active role of the United States in the field of HIE. Harvard University was the most prolific institution with 33 articles, followed by University of Texas System (28 articles) and University System of Ohio (24 articles).

Table 2.

Top 10 institutions ranked by article count.

Institution Country Article count Citation count (ranking) Average citations per article (ranking) Web of Science category (article count)
Harvard University United States 33 321 (4) 9.73 (9)

  • Health Care Sciences Services (21)

  • Medical Informatics (15)

  • Computer Science Information System (9)
University of Texas System United States 28 274 (7) 9.79 (8)

  • Medical Informatics (15)

  • Health Care Sciences Services (11)

  • Computer Science Information System (7)
University System of Ohio United States 24 317 (5) 13.21 (5)

  • Medical Informatics (7)

  • Nursing (7)

  • Computer Science Interdisciplinary Applications (5)
Harvard Medical School United States 20 252 (8) 12.60 (6)

  • Health Care Sciences Services (12)

  • Medical Informatics (6)

  • Computer Science Information System (5)
Indiana University System United States 20 327 (3) 16.35 (3)

  • Medical Informatics (11)

  • Health Care Sciences Services (10)

  • Computer Science Information System (7)
University of California System United States 20 521 (1) 26.05 (1)

  • Health Care Sciences Services (12)

  • Medical Informatics (11)

  • Computer Science Interdisciplinary Applications (7)
University of Minnesota System United States 20 394 (2) 19.70 (2)

  • Health Care Sciences Services (9)

  • Medical Informatics (8)

  • Health Policy Services (5)
University of Pennsylvania United States 19 226 (9) 11.89 (7)

  • Medical Informatics (8)

  • Health Care Sciences Services (6)

  • Computer Science Information System (5)
State University System of Florida United States 18 282 (6) 15.67 (4)

  • Medical Informatics (10)

  • Health Care Sciences Services (8)

  • Computer Science Interdisciplinary Applications (6)
University of Toronto Canada 18 146 (10) 8.11 (10)

  • Medical Informatics (7)

  • Health Care Sciences Services (6)

  • Nursing (4)
Open in a new tab

Regarding citation count, the top 3 institutions were University of California System (521 citations), University of Minnesota System (394 citations), and Indiana University System (327 citations). These top 3 institutions were also ranked among the top 3 in terms of the average citations per article, with values of 26.05, 19.7, and 16.35, respectively.

By considering all these indicators, the performance of University of California System is worth highlighting, indicating its great contribution to the field of HIE. If we put Harvard University and Harvard Medical School together, we get an institution with the most influence regarding the indicators of article count and citation count.

Top Publication Sources and Subjects

The 575 articles concerning HIE were contributed by 197 publication sources, among which the top 20 with a minimum article count of 5 are shown in Table 3, which together contributed to 51.3% (295/575) of the total studied articles. Journal of Medical Internet Research was the most prolific in the field with 42 articles, followed by Journal of the American Medical Informatics Association (37 articles), Applied Clinical Informatics (30 articles), and BMJ Open (28 articles). Regarding citation count, the top 3 journals were Journal of Medical Internet Research (1213 citations), Journal of the American Medical Informatics Association (442 citations), and Academic Medicine (328 citations). Regarding average citations per article, the top 3 journals were Academic Medicine (41.00), Journal of Medical Internet Research (28.88), and JMIR mHealth and uHealth (25.56). By considering all these indicators, the performance of Journal of Medical Internet Research is worth highlighting, indicating its significant contribution to the research field of HIE.

Table 3.

Top 20 publication sources ranked by article count.

Publication source Article
count		 Citation count (ranking) Average citations per article (ranking) Web of Science category (article count)
Journal of Medical Internet Research 42 1213 (1) 28.88 (2)

  • Health Care Sciences Services (42)

  • Medical Informatics (42)
Journal of The American Medical Informatics Association 37 442 (2) 11.95 (7)

  • Computer Science Information Systems (37)

  • Computer Science Interdisciplinary Applications (37)

  • Health Care Sciences Services (37)

  • Information Science Library Science (37)

  • Medical Informatics (37)
Applied Clinical Informatics 30 245 (5) 8.17 (12)

  • Medical Informatics (30)
BMJ Open 28 177 (8) 6.32 (14)

  • Medicine General Internal (28)
CIN: Computers Informatics Nursing 27 280 (4) 10.37 (10)

  • Computer Science Interdisciplinary Applications (27)

  • Medical Informatics (27)

  • Nursing (27)
International Journal of Medical Informatics 17 67 (14) 3.94 (17)

  • Computer Science Information Systems (17)

  • Health Care Sciences Services (17)

  • Medical Informatics (17)
BMC Medical Informatics and Decision Making 15 142 (9) 9.47 (11)

  • Medical Informatics (15)
Nurse Education Today 10 216 (7) 21.60 (4)

  • Education Scientific Disciplines (10)

  • Nursing (10)
JMIR Medical Informatics 9 26 (17) 2.89 (19)

  • Medical Informatics (9)
JMIR mHealth and uHealth 9 230 (6) 25.56 (3)

  • Health Care Sciences Services (9)

  • Medical Informatics (9)
Methods of Information in Medicine 9 98 (12) 10.89 (8)

  • Computer Science Information Systems (9)

  • Health Care Sciences Services (9)

  • Medical Informatics (9)
Academic Medicine 8 328 (3) 41.00 (1)

  • Education Scientific Disciplines (8)

  • Health Care Sciences Services (8)
International Journal of Environmental Research and Public Health 8 84 (13) 10.50 (9)

  • Environmental Sciences (8)

  • Public Environmental Occupational Health (8)
Journal of Interprofessional Care 8 62 (15) 7.75 (13)

  • Health Care Sciences Services (8)

  • Health Policy Services (8)
BMC Medical Education 7 23 (19) 3.29 (18)

  • Education Educational Research (7)

  • Education Scientific Disciplines (7)
Health Informatics Journal 7 40 (16) 5.71 (15)

  • Health Care Sciences Services (7)

  • Medical Informatics (7)
Informatics for Health Social Care 7 104 (11) 14.86 (6)

  • Health Care Sciences Services (7)

  • Medical Informatics (7)
Frontiers in Public Health 6 108 (10) 18.00 (5)

  • Public Environmental Occupational Health (6)
Journal of Nursing Education 6 24 (18) 4.00 (16)

  • Nursing (6)
Healthcare 5 13 (20) 2.60 (20)

  • Health Care Sciences Services (5)

  • Health Policy Services (5)
Open in a new tab

The 575 articles concerning HIE were distributed in 80 Web of Science categories, among which the top 20 with a minimum article count of 7 are shown in Table 4. Medical Informatics was the most relevant category in this field with 229 articles, followed by Health Care Sciences Services (189 articles) and Nursing (103 articles). The latter 2 categories also ranked among the top 2 from the perspective of citation count. Regarding average citations per article, the top 3 categories were Dentistry Oral Surgery Medicine (40.00), Education Scientific Disciplines (18.42), and Health Care Sciences Services (15.87). Although the article count of Dentistry Oral Surgery Medicine was only 7, its citation count was relatively high. That is because an article titled “Artificial Intelligence in Dentistry: Chances and Challenges” received widespread attention. This article reviewed artificial intelligence (AI) in dentistry and suggested that dental education needs to accompany the introduction of clinical AI solutions by fostering digital literacy in the future dental workforce.

Table 4.

Top 20 Web of Science categories ranked by article count.

Web of Science category Article count Citation count (ranking) Average citations per article (ranking)
Medical Informatics 229 371 (9) 1.62 (20)
Health Care Sciences Services 189 3000 (1) 15.87 (3)
Nursing 103 1326 (2) 12.87 (7)
Computer Science Interdisciplinary Applications 71 873 (3) 12.30 (8)
Computer Science Information Systems 70 697 (5) 9.96 (12)
Medicine General Internal 60 517 (8) 8.62 (14)
Public Environmental Occupational Health 46 593 (6) 12.89 (5)
Information Science Library Science 41 528 (7) 12.88 (6)
Education Scientific Disciplines 38 700 (4) 18.42 (2)
Health Policy Services 30 348 (10) 11.60 (15)
Pharmacology Pharmacy 14 221 (12) 15.79 (4)
Environmental Sciences 12 89 (15) 7.42 (16)
Medicine Research Experimental 10 95 (14) 9.50 (13)
Pediatrics 10 73 (18) 7.30 (17)
Radiology Nuclear Medicine Medical Imaging 10 121 (13) 12.10 (9)
Education Educational Research 8 26 (20) 3.25 (19)
Multidisciplinary Sciences 8 41 (19) 5.13 (18)
Oncology 8 88 (16) 11.00 (10)
Primary Health Care 8 85 (17) 10.63 (11)
Dentistry Oral Surgery Medicine 7 280 (11) 40.00 (1)
Open in a new tab

Scientific Collaboration

The macro picture of collaboration among countries in the research field of HIE from 2014 to 2023 is shown in Figure 4, indicating that related research in countries has benefited from international cooperation.

Figure 4.

Figure 4

Open in a new tab

A macro-level picture of collaboration among countries (world map).

The collaborative scientific research networks among countries/regions and institutions are visualized in Figures 5 and 6, respectively. In these collaborative networks, countries/regions and institutions are represented by nodes, and collaborations between countries/regions or institutions are represented by edges. The size of each node indicates the number of articles from each country/region or institution, with a larger node being associated with a larger number of articles. Additionally, the color of a node is determined by the cluster to which the node belongs, and the lines between nodes represent collaborative relationships. The distance between 2 nodes in the visualization approximately indicates the relatedness of the countries/regions or institutions in terms of collaborative links. In general, the closer 2 countries/regions or institutions are located to each other, the stronger their relatedness.

Figure 5.

Figure 5

Open in a new tab

Collaboration network of countries/regions with a minimum article count of 2.

Figure 6.

Figure 6

Open in a new tab

Collaboration network of institutions with a minimum article count of 2.

For country/region collaborations, the minimum number of articles of a country/region was set to 2, and 50 countries met the threshold out of the 72 countries. For each of the 50 countries, the total strength of the co-authorship links with other countries was calculated. The results of the top 10 countries/regions are shown in Table 5. Of these 10 countries/regions, United States, England, and Canada collaborated with the most countries/regions, with 138, 85, and 82 collaborations, respectively.

Table 5.

Top 10 countries/regions ranked by link strength.

Country/region Total link strength Article count
United States 138 326
England 85 33
Canada 82 50
Netherlands 76 25
Australia 75 45
Germany 67 28
Sweden 59 17
Greece 54 10
Norway 52 9
Spain 52 13
Open in a new tab

For institution collaborations, the minimum number of articles of an institution was set to 2, and 36 institutions met the threshold out of the 1203 institutions. For each of the 36 institutions, the total strength of the co-authorship links with other institutions was calculated. The results of the top 10 institutions are shown in Table 6. Of these 10 institutions, Vanderbilt University, Harvard Medical School, and University of Washington collaborated with the most institutions, with 31, 28, and 27 collaborations, respectively.

Table 6.

Top 10 institutions ranked by link strength.

Institution Total link strength Article count
Vanderbilt University 31 18
Harvard Medical School 28 20
University of Washington 27 16
Oregon Health Science University 25 11
Columbia University 23 16
Johns Hopkins University 22 13
University of Texas Health Science Center Houston 22 13
University of Minnesota System 20 20
Indiana University System 18 20
University of Utah 18 14
Open in a new tab

Major Research Topics and Trends

Table 7 shows the 7-topic STM results with representative terms under 4 different metrics, topic proportion within the whole corpus, and suggested topic labels. Among the 4 metrics, “Highest Prob” means terms with the highest probability, “FREX” weights words by their overall frequency and how exclusive they are to the topic, “Lift” weights words through division by their frequency in other topics, and “Score” divides the log frequency of the word in the topic by the log frequency of the word in other topics [26]. Topic proportion refers to the ratio of the number of articles on each topic to all articles. The results showed that Clinical Informatics Education (130/575, 22.6%) was the most researched topic among authors of HIE studies, followed by Mobile Application in Health Education & Healthcare (123/575, 21.4%) and Consumer Health Informatics Education (99/575, 17.2%). Additionally, a synopsis for each topic label has been provided to illustrate its meaning in the last column of Table 7.

Table 7.

The 7-topic structural topic model results with topic labels, representative terms under 3 different metrics, and topic proportions.

Topics Topic labels Representative termsa Topic (N=575), n (%) Synopsis of topic labels
Topic 1 Clinical Informatics Education

  • Highest Prob: care, clinical, system, record, provide, use, improve, health, medical, ehr, hospital, decision

  • FREX: clinical, care, ehr, record, system, hospital, decision, provide, order, improve, set, plan

  • Lift: ehr, clinical, decision, order, plan, care, record, hospital, system, efficient, team, point

  • Score: ehr, clinical, record, care, hospital, system, decision, medical, order, provide, improve, train

 130 (22.6) Integrating clinical informatics into the full spectrum of medical education
Topic 2 Mobile Application in Health Education & Healthcare

  • Highest Prob: study, use, group, evaluate, education, assess, disease, application, result, mobile, factor, health

  • FREX: mobile, disease, group, application, score, risk, evaluate, among, factor, age, random, assess

  • Lift: mobile, random, score, lower, average, risk, disease, overall, mean, age, application, measure

  • Score: mobile, disease, random, risk, score, age, factor, study, lower, group, application, test

 123 (21.4) Mobile apps are effective learning tools for medical education
Topic 3 Consumer Health Informatics Education

  • Highest Prob: health, information, use, online, access, social, user, study, engage, person, consumer, report

  • FREX: social, online, user, access, information, consumer, person, engage, use, search, report, active

  • Lift: consumer, social, user, online, person, access, engage, information, search, util, evid, source

  • Score: consumer, information, health, user, social, online, access, person, use, engage, search, active

 99 (17.2) Health informatics from multiple consumer or patient perspectives, including health literacy and consumer education
Topic 4 Professional Teaching of Biomedical and Health Informatics

  • Highest Prob: health, education, competency, domain, skill, manage, requirement, public, need, new, core, advance

  • FREX: domain, skill, requirement, competency, core, expert, advance, new, manage, gap, role, education

  • Lift: domain, core, define, requirement, competency, skill, expert, advance, gap, employ, relevant, role

  • Score: domain, competency, core, health, education, skill, public, expert, requirement, advance, new, define

 61 (10.6) Competencies of health informatics for health care professionals and others working in this field
Topic 5 Public Health Informatics Education & Digital Health

  • Highest Prob: health, medical, digit, education, learn, train, need, population, public, future, school, teach

  • FREX: digit, learn, train, population, medical, teach, future, school, need, health, global, service

  • Lift: digit, learn, teach, global, population, train, rapid, innovate, school, future, organ, achieve

  • Score: digit, medical, learn, train, health, population, public, school, teach, need, education, active

 56 (9.7) Courses on digital health to train public health professionals
Topic 6 Discipline of Biomedical and Health Informatics

  • Highest Prob: data, science, medical, course, education, public, nation, integrate, year, include, model, intern

  • FREX: science, data, course, nation, intern, field, year, computer, term, paper, center, topic

  • Lift: science, course, data, intern, field, term, topic, nation, paper, computer, center, article

  • Score: science, data, course, medical, public, school, nation, field, topic, intern, computer, integrate

 55 (9.6) Practices in teaching and learning in the discipline of biomedical and health informatics
Topic 7 Nursing Informatics Education

  • Highest Prob: nursing, competency, education, use, study, information, experience, record, integrate, assess, clinical, valid

  • FREX: nursing, competency, experience, valid, purpose, prepare, integrate, innovate, scale, critical, record, study

  • Lift: nursing, prepare, competency, scale, purpose, valid, innovate, experience, guide, unit, ability, critical

  • Score: nursing, competency, record, information, study, innovate, clinical, integrate, academic, prepare, scale, experience

 51 (8.9) Digital competencies for nurses and nursing care in this digital age
Open in a new tab

a“Highest Prob” means terms with the highest probability, “FREX” weights words by their overall frequency and how exclusive they are to the topic, “Lift” weights words through division by their frequency in other topics, and “Score” divides the log frequency of the word in the topic by the log frequency of the word in other topics.

Figure 7 shows the trends of the annual proportions of detected topics, which suggest that HIE research has become more diverse over time, with many different issues being of concern. The temporal dynamics of the distributions of all topics clearly indicate the unique foci for each year and demonstrate how the popularity of each topic has changed relative to other topics over time, depicting the process of development for health informatics research. For example, researchers showed continuous interest in Clinical Informatics Education (Topic 1) and Discipline of Biomedical and Health Informatics (Topic 6). In 2018, the predominant foci of the community were mainly related to Professional Teaching of Biomedical and Health Informatics (Topic 4). Years 2017 and 2022 witnessed a dramatic interest in issues concerning Mobile Application in Health Education & Healthcare (Topic 2). Some topics with high proportions at earlier time points showed reductions at the latest time point, such as Consumer Health Informatics Education (Topic 3) and Nursing Informatics Education (Topic 7). Since the year 2019, scholars in the field of HIE began to show interest in Public Health Informatics Education & Digital Health (Topic 5).

Figure 7.

Figure 7

Open in a new tab

Annual topic proportion within the whole corpus for 7 topics.

Discussion

Principal Findings

This study conducted a scientometric analysis on research of HIE from 2014 to 2023. Over the last decade, research on HIE has appeared to be increasing steadily. This field has been progressively developing and maturing, and has received a growing interest from the academic community. It is currently experiencing a phase of rapid growth.

The United States has made substantial contributions to the growth of publications in this research field. Additionally, Canada, Australia, England, Germany, China, Brazil, Sweden, and Spain demonstrated a high interest as well. Harvard University and Harvard Medical School represented the most influential institutions, and the most vigorous institutions included University of Texas System, University System of Ohio, Indiana University System, and University of California System. Research on HIE has been predominantly favored by journals related to health informatics, as indicated by article counts and citation counts. Journal of Medical Internet Research, Journal of The American Medical Informatics Association, and Applied Clinical Informatics were found to be the most influential and productive publication sources in terms of average citations per article and article counts. Collaboration among universities in the United States, England, and Canada was found to be high. Collaboration was identified to be mainly concentrated in the Americas, Europe, and Asia, with limited international collaboration from developing countries, suggesting that international collaboration in this research field needs to be strengthened.

Topic modeling analysis over time identified the knowledge structure and development trends in this research field. Seven topics were identified using R package STM, and representative terms were provided for each topic under 4 different metrics. Clinical Informatics Education was found to be the most researched topic among authors of HIE studies, followed by the topics of Mobile Application in Health Education & Healthcare and Consumer Health Informatics Education. Additionally, the temporal dynamics of the distributions of all topics have been presented, which demonstrated how the popularity of each topic has changed relative to other topics over time and thus depicted the process of development for health informatics research.

Implications of the 7 Topics

For a profound understanding of the identified research topics of HIE, the most representative research works for each topic were analyzed.

Topic 1: Clinical Informatics Education

Education in clinical informatics is important for not only medical students but also other clinical staff at all professional levels of education. Integrating clinical informatics into the full spectrum of medical education is a vital step required for the understanding and practice of modern medicine. Education in clinical informatics is present throughout the spectrum of formal medical education, extending from high school to postgraduate training. Mai et al [33] presented a model of a clinical informatics curriculum that has been in place at the Children’s Hospital of Philadelphia. This educational model can support resident involvement in hospital-wide informatics efforts. Similar models to promote resident engagement in clinical informatics across training programs have the potential to accelerate health care improvements and develop the clinical informaticians of the future.

Topic 2: Mobile Application in Health Education & Healthcare

Current evidence suggests an important knowledge gain among health science students and professionals with mobile devices (m-learning). Matos Lamenha-Lins et al [34] conducted a study to verify whether a mobile app was a good learning tool for improving dental students’ knowledge. Mobile apps are effective learning tools that have a significant role in transferring information and knowledge to clinicians and nurses. The use of mobile technologies to improve health outcomes or mobile health is rapidly evolving. Garner et al [35] demonstrated that a mobile health app was effective in improving diabetes health literacy. Nezamdoust et al [36] performed a study to identify the factors affecting nurses’ use of practical health-related mobile apps in education and patient interaction based on the combined technology acceptance model (TAM) and diffusion of innovation (DOI).

Topic 3: Consumer Health Informatics Education

The American Medical Informatics Association defines consumer health informatics (CHI) as a field devoted to informatics from multiple consumers or patient views and which includes patient-focused informatics, health literacy, and consumer education [37]. Scholars mainly conducted research from 3 aspects, including consumer health needs [38], the effect of health education materials [39,40], and consumer health behavior and literacy [41,42].

Topic 4: Professional Teaching of Biomedical and Health Informatics

Health informatics has acquired the attributes of a profession, such as a well-defined set of competencies, a certification of fitness to practice, shared professional identity, life-long commitment, and a code of ethics. Research on this topic involves the following: health informatics curricula at different education levels [43,44], essential skills and competencies for health care professionals and others working in the field of health informatics to support the development and implementation of health informatics in health care [10,45,46], and the quality of teaching and training with delivered HIE programs [47-49].

Topic 5: Public Health Informatics Education & Digital Health Education

A generally accepted definition of public health informatics is as follows: “the systematic application of information, computer science, and technology to public health practice, research, and learning” [50]. Public health has a strong historical basis in handling health information, and much of public health activity continues to involve collecting, analyzing, and disseminating information about the health of the public. Moreover, public health is inherently interdisciplinary, and the sciences underlying public health encompass a set of skills and knowledge that are recognized to be at the core of health informatics [51]. Meanwhile, digital health has been recognized as an innovative health solution that enables public health action to stop the rapid spread of a virus during a pandemic [52]. Digital health is a key driver in addressing public health challenges and achieving sustainable health systems and universal health coverage [53]. Therefore, there is an urgent need to analyze how to establish systematic courses on digital health in medical schools to train the next generation of doctors and public health managers in the integration of health theory and digital technology [54].

Topic 6: Discipline of Biomedical and Health Informatics

Health informatics became a professional discipline nearly 2 decades ago [55]. Being an interdisciplinary discipline according to its definition, health informatics draws upon different disciplines. Professionals entering the field have diverse backgrounds, and HIE at various levels needs to support a range of work roles. To help inform programs on practices in teaching and learning, a survey of master’s programs in health and biomedical informatics in the United States was conducted to determine the national landscape of culminating experiences, including capstone projects, research theses, internships, and practicums [56]. Bellazzi et al [57] provided a description of the fundamental design principles of 3 medical informatics education programs and explored some aspects of the teaching modules, highlighting their positive aspects.

Topic 7: Nursing Informatics Education

Nursing informatics is a subcomponent of health informatics pertaining to nurses and nursing care [58]. Nurses need digital competencies to integrate new technologies in their professional activities. The foundations for these digital competencies are acquired during their vocational training/undergraduate education and ongoing training, and are deepened in their workplaces [59]. Most research involving this topic was focused on improving competencies for nursing in the digital age [60,61].

Comparison With the Literature

As IT is applied widely in health care and medical education, some studies have examined the evolution of the intellectual and conceptual structure in the field of HIE. For example, Monkman et al [62] conducted a scoping review to identify health informatics competencies from research articles and grey literature originating from professional associations, and proposed a health informatics core competencies framework that could be used to inform Canadian health informatics programs to ensure graduates would be prepared for careers in health informatics. To outline the history of medical informatics education in China, Hu et al [63] systematically analyzed the status of medical informatics education at different academic levels and suggested reasonable strategies for the further development of the field in China. Asiri [64] explored the challenges that face HIE and training in the Kingdom of Saudi Arabia by examining the state of health IT education, with a focus on HIE. Jeon et al [65] presented the status of nursing informatics education and the content covered in nursing informatics courses in Korea, and concluded that a greater focus was needed on training faculty and developing the courses. Similarly, Masic et al [66] investigated the status of medical informatics education in Bosnia and Herzegovina, and Europe.

Although these studies made invaluable contributions to the understanding of HIE, they tended to focus on specific regions or disciplines and did not provide a global overview of research in HIE. Additionally, they did not use any topic modeling methods to reveal topic structure and evolutionary trends of this research field, and thus, the methodologies employed in these studies had some limitations. Meanwhile, there was a scarcity of comprehensive scientometric analysis in this research field. Therefore, this study provided a topic modeling–based scientometric analysis of scientific articles concerning HIE, and it analyzed aspects such as productive countries and institutions, scientific collaborations, and topics and trends of HIE research.

Limitations

The main limitation of this study was that only 1 database was searched to identify studies for inclusion, meaning that potentially relevant studies could have been overlooked. Although WoSCC is a widely used academic database globally, other medical-oriented databases may provide more significant academic and practical information. Subsequent studies should conduct searches in commonly used databases, such as PubMed, Scopus, IEEE, and Google Scholar, to ensure a more comprehensive analysis. Another constraint was that only journal articles were retrieved for analysis, excluding other relevant publication types in this field, such as reviews, book chapters, and conference proceedings. This could potentially affect the summarization and generalization of the results.

As AI has covered most of the research topics in health informatics, academic research on the intersection of AI and HIE has emerged in recent years. For example, Paranjape et al [67] summarized the state of medical education and recommended a framework to include AI education. Yang et al [68] examined the impact of an AI system tutoring course on clinical training and found an increased improvement in the expert-led AI tutoring group. However, AI did not appear as a representative term in any of these topics. The main reason may be that the time span chosen for this study was long at 10 years, and AI did not dominate during this period. Overall, the application of AI in HIE should be taken seriously, and formal training for clinical and health informatics programs should consist of AI technology and its applications.

Conclusion

The field of HIE is increasingly promising and active, with academic products flourishing. This study presents the results and findings from a topic-driven scientometric analysis of academic research articles concerning HIE published from 2014 to 2023. This study not only highlights the major research concerns and issues among scholars but also identifies the tendencies of article and citation counts; identifies major countries/regions, institutions, publication sources, and subjects; and outlines academic collaboration relationships. The topic-based scientometric analysis contributed to HIE by providing a comprehensive overview of its community. The contributions of this study with regard to HIE have been summarized.

First, it helps researchers, policymakers, and practitioners better understand the historical, extant, and upcoming structure of research in HIE. The exploration of major topics and their trends is helpful for the identification and comparison of current and potential scientific strength. Research governors or funding agencies can optimize research policies to promote potential competitive research areas and enhance scientific collaborations in specific research areas in order to bolster scientific and educational activities. Second, findings regarding prolific countries/regions and institutions can assist scholars in sharing research, conducting collaborations, and identifying important actors from whom to learn. This is valuable for the research community of HIE in terms of understanding its research status and trends, especially by translating the findings into directions for future research. Third, this study provides a clear outline of HIE, which could help interested educators as well as researchers who wish to continue working in this area. The results of prolific publication sources and subjects provide suggestions about sources to which scholars can submit their work and will help scholars to better position their research in the field.

As it explores the structures and research topics within the area of HIE, this study serves as a starting point for further in-depth research into HIE. The research scope of HIE is relatively broad, and this study is expected to provide editors and conference organizers with ways to identify gaps in the field, leading them to create special issues or conference sessions about topics that are generally not co-occurring, which could create potential new synergies in HIE.

Acknowledgments

This work was supported by Zhejiang Provincial Philosophy and Social Sciences Planning Project (reference number: 22JCXK11YB) and Zhejiang Provincial Education Sciences Planning Project (reference number: 2023SCG422).

Abbreviations

AI

artificial intelligence

HIE

health informatics education

LDA

latent Dirichlet allocation

STM

structural topic model

WoSCC

Web of Science Core Collection

Data Availability

The data and analysis code are available upon reasonable request.

Footnotes

Conflicts of Interest: None declared.

References

  • 1.Bichel-Findlay J, Koch S, Mantas J, Abdul SS, Al-Shorbaji N, Ammenwerth E, Baum A, Borycki EM, Demiris G, Hasman A, Hersh W, Hovenga E, Huebner UH, Huesing ES, Kushniruk A, Hwa Lee K, Lehmann CU, Lillehaug S, Marin HF, Marschollek M, Martin-Sanchez F, Merolli M, Nishimwe A, Saranto K, Sent D, Shachak A, Udayasankaran JG, Were MC, Wright G. Recommendations of the International Medical Informatics Association (IMIA) on education in biomedical and health informatics: second revision. Int J Med Inform. 2023 Feb;170:104908. doi: 10.1016/j.ijmedinf.2022.104908.S1386-5056(22)00222-2 [DOI] [PubMed] [Google Scholar]
  • 2.Amlung J, Huth H, Cullen T, Sequist T. Modernizing health information technology: lessons from healthcare delivery systems. JAMIA Open. 2020 Oct;3(3):369–377. doi: 10.1093/jamiaopen/ooaa027. https://europepmc.org/abstract/MED/33215072 .ooaa027 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Friedman C, Wyatt J, Ash J. Evaluation Methods in Biomedical and Health Informatics. Cham: Springer; 2022. [Google Scholar]
  • 4.Hersh W. A stimulus to define informatics and health information technology. BMC Med Inform Decis Mak. 2009 May 15;9:24. doi: 10.1186/1472-6947-9-24. https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/1472-6947-9-24 .1472-6947-9-24 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Kushniruk A, Lau F, Borycki E, Protti D. The School of Health Information Science at the University of Victoria: towards an integrative model for health informatics education and research. Yearb Med Inform. 2018 Mar 07;15(01):159–165. doi: 10.1055/s-0038-1638477. [DOI] [PubMed] [Google Scholar]
  • 6.Kolokathi A, Hasman A, Chronaki C, Madsen I, Moen A, Randell R, Mantas J. Education in biomedical and health informatics: a European perspective. Stud Health Technol Inform. 2019 Aug 21;264:1951–1952. doi: 10.3233/SHTI190729.SHTI190729 [DOI] [PubMed] [Google Scholar]
  • 7.Mantas J. The European educational programs in biomedical and health informatics. Stud Health Technol Inform. 2020 Jun 26;272:482–483. doi: 10.3233/SHTI200600.SHTI200600 [DOI] [PubMed] [Google Scholar]
  • 8.Alshammari R, Al-Shorbaji N, Ahmed Z, Al-Hamadan Z, Alshammari A, El Hassan O, Alhuwail D, Al Salmi S, Al-Jafar E, Ghazal H. Middle East and North African Health Informatics Association (MENAHIA): a new opportunity for the region. Yearb Med Inform. 2019 Apr 25;28(01):292–296. doi: 10.1055/s-0039-1677896. [DOI] [Google Scholar]
  • 9.Murray PJ, Betts HJ, Wright G, Tshayingca-Mashiya NV. Health informatics education and capacity building in Eastern Cape Province, South Africa. Yearb Med Inform. 2018 Mar 07;18(01):158–163. doi: 10.1055/s-0038-1638656. [DOI] [PubMed] [Google Scholar]
  • 10.Were MC, Gong W, Balirwa P, Balugaba BE, Yeung A, Pierce L, Ingles D, Kim Y, Lee HJ, Shepherd BE. Comparative analysis of competency coverage within accredited master's in health informatics programs in the East African region. J Am Med Inform Assoc. 2021 Aug 13;28(9):1843–1848. doi: 10.1093/jamia/ocab075. https://europepmc.org/abstract/MED/34151967 .6307173 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Chen X, Xie H. A structural topic modeling-based bibliometric study of sentiment analysis literature. Cogn Comput. 2020 Jul 31;12(6):1097–1129. doi: 10.1007/s12559-020-09745-1. [DOI] [Google Scholar]
  • 12.Chen X, Wang S, Tang Y, Hao T. A bibliometric analysis of event detection in social media. OIR. 2019 Feb 11;43(1):29–52. doi: 10.1108/oir-03-2018-0068. [DOI] [Google Scholar]
  • 13.Lester CA, Wang M, Vydiswaran VGV. Describing the patient experience from Yelp reviews of community pharmacies. J Am Pharm Assoc (2003) 2019;59(3):349–355. doi: 10.1016/j.japh.2019.02.004.S1544-3191(19)30074-3 [DOI] [PubMed] [Google Scholar]
  • 14.Chen X, Zou D, Xie H. Fifty years of British Journal of Educational Technology: A topic modeling based bibliometric perspective. Brit J Educational Tech. 2020 Feb 05;51(3):692–708. doi: 10.1111/bjet.12907. [DOI] [Google Scholar]
  • 15.Hao T, Chen X, Li G, Yan J. A bibliometric analysis of text mining in medical research. Soft Comput. 2018 Sep 6;22(23):7875–7892. doi: 10.1007/s00500-018-3511-4. [DOI] [Google Scholar]
  • 16.Falagas ME, Pitsouni EI, Malietzis GA, Pappas G. Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses. FASEB J. 2008 Feb;22(2):338–42. doi: 10.1096/fj.07-9492LSF.fj.07-9492LSF [DOI] [PubMed] [Google Scholar]
  • 17.Wang Q, Waltman L. Large-scale analysis of the accuracy of the journal classification systems of Web of Science and Scopus. Journal of Informetrics. 2016 May;10(2):347–364. doi: 10.1016/j.joi.2016.02.003. [DOI] [Google Scholar]
  • 18.Saheb T, Saheb M. Analyzing and visualizing knowledge structures of health informatics from 1974 to 2018: a bibliometric and social network analysis. Healthc Inform Res. 2019 Apr;25(2):61–72. doi: 10.4258/hir.2019.25.2.61. https://europepmc.org/abstract/MED/31131140 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Zou LX, Sun L. Visualization analysis of health informatics research from 2001 to 2018. Current Science. 2020;118(5):714–721. https://www.jstor.org/stable/27226351 . [Google Scholar]
  • 20.Chen X, Xie H, Wang FL, Liu Z, Xu J, Hao T. A bibliometric analysis of natural language processing in medical research. BMC Med Inform Decis Mak. 2018 Mar 22;18(Suppl 1):14. doi: 10.1186/s12911-018-0594-x. https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-018-0594-x .10.1186/s12911-018-0594-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Cretchley J, Rooney D, Gallois C. Mapping a 40-year history with Leximancer: themes and concepts in the Journal of Cross-Cultural Psychology. Journal of Cross-Cultural Psychology. 2010 Apr 14;41(3):318–328. doi: 10.1177/0022022110366105. [DOI] [Google Scholar]
  • 22.Zhong S, Geng Y, Liu W, Gao C, Chen W. A bibliometric review on natural resource accounting during 1995–2014. Journal of Cleaner Production. 2016 Dec;139:122–132. doi: 10.1016/j.jclepro.2016.08.039. [DOI] [Google Scholar]
  • 23.Lee K, Jung H, Song M. Subject–method topic network analysis in communication studies. Scientometrics. 2016 Sep 30;109(3):1761–1787. doi: 10.1007/s11192-016-2135-7. [DOI] [Google Scholar]
  • 24.Yan E. Research dynamics, impact, and dissemination: A topic‐level analysis. Asso for Info Science & Tech. 2015 Jun;66(11):2357–2372. doi: 10.1002/asi.23324. [DOI] [Google Scholar]
  • 25.Hoffman M, Blei D, Bach F. Online learning for Latent Dirichlet Allocation. NIPS'10: Proceedings of the 23rd International Conference on Neural Information Processing Systems - Volume 1; 23rd International Conference on Neural Information Processing Systems; December 6-9, 2010; Vancouver, British Columbia, Canada. 2010. [DOI] [Google Scholar]
  • 26.Roberts ME, Stewart BM, Tingley D. stm: An R package for structural topic models. J. Stat. Soft. 2019;91(2):1–40. doi: 10.18637/jss.v091.i02. [DOI] [Google Scholar]
  • 27.Blei D, Ng A, Jordan M. Latent dirichlet allocation. NIPS'01: Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic; 14th International Conference on Neural Information Processing Systems: Natural and Synthetic; December 3-8, 2001; Vancouver, British Columbia, Canada. 2001. [DOI] [Google Scholar]
  • 28.Blei D, Lafferty J. Correlated topic models. NIPS'05: Proceedings of the 18th International Conference on Neural Information Processing Systems; 18th International Conference on Neural Information Processing Systems; December 5-8, 2005; Vancouver, British Columbia, Canada. 2005. [DOI] [Google Scholar]
  • 29.Chen X, Xie H, Li Z, Cheng G, Leng M, Wang FL. Information fusion and artificial intelligence for smart healthcare: a bibliometric study. Information Processing & Management. 2023 Jan;60(1):103113. doi: 10.1016/j.ipm.2022.103113. [DOI] [Google Scholar]
  • 30.Levy KEC, Franklin M. Driving regulation. Social Science Computer Review. 2013 Dec 02;32(2):182–194. doi: 10.1177/0894439313506847. [DOI] [Google Scholar]
  • 31.Tvinnereim E, Fløttum K. Explaining topic prevalence in answers to open-ended survey questions about climate change. Nature Clim Change. 2015 Jun 1;5(8):744–747. doi: 10.1038/nclimate2663. [DOI] [Google Scholar]
  • 32.Chen X, Zou D, Cheng G, Xie H. Detecting latent topics and trends in educational technologies over four decades using structural topic modeling: A retrospective of all volumes of Computers and Education. Computers & Education. 2020 Jul;151:103855. doi: 10.1016/j.compedu.2020.103855. [DOI] [Google Scholar]
  • 33.Mai MV, Luo BT, Orenstein EW, Luberti AA. A model for clinical informatics education for residents: addressing an unmet need. Appl Clin Inform. 2018 Apr;9(2):261–267. doi: 10.1055/s-0038-1641735. https://europepmc.org/abstract/MED/29669389 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Matos Lamenha-Lins R, Maria de Carvelho Pugliesi D, José Camello de Lima F, Regina Oliveira Moreira A, Gonçalves Correia de Leite de Marcelos P, Dos Santos VE. Mobile application as a learning tool for improving dental students' knowledge regarding dental trauma. Eur J Dent Educ. 2022 Nov;26(4):700–706. doi: 10.1111/eje.12750. [DOI] [PubMed] [Google Scholar]
  • 35.Garner SL, Young P, Fendt M, Koch H, George CE, Hitchcock J, Green G, Kulaba P. Effectiveness of a culturally responsive mHealth gaming application to improve diabetes health literacy in India: a randomized controlled trial. Comput Inform Nurs. 2023 Oct 01;41(10):796–804. doi: 10.1097/CIN.0000000000001009.00024665-202310000-00010 [DOI] [PubMed] [Google Scholar]
  • 36.Nezamdoust S, Abdekhoda M, Rahmani A. Determinant factors in adopting mobile health application in healthcare by nurses. BMC Med Inform Decis Mak. 2022 Feb 22;22(1):47. doi: 10.1186/s12911-022-01784-y. https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-022-01784-y .10.1186/s12911-022-01784-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Informatics: Research and Practice. AMIA - American Medical Informatics Association. [2024-02-05]. https://www.amia.org/about-amia/why-informatics/informatics-research-and-practice .
  • 38.Pian W, Khoo CS, Chi J. Automatic classification of users' health information need context: logistic regression analysis of mouse-click and eye-tracker data. J Med Internet Res. 2017 Dec 21;19(12):e424. doi: 10.2196/jmir.8354. https://www.jmir.org/2017/12/e424/ v19i12e424 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Kraschnewski JL, Chuang CH, Poole ES, Peyton T, Blubaugh I, Pauli J, Feher A, Reddy M. Paging "Dr. Google": does technology fill the gap created by the prenatal care visit structure? Qualitative focus group study with pregnant women. J Med Internet Res. 2014 Jun 03;16(6):e147. doi: 10.2196/jmir.3385. https://www.jmir.org/2014/6/e147/ v16i6e147 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Vamos CA, Merrell L, Detman L, Louis J, Daley E. Exploring women's experiences in accessing, understanding, appraising, and applying health information during pregnancy. J Midwifery Womens Health. 2019 Jul;64(4):472–480. doi: 10.1111/jmwh.12965. [DOI] [PubMed] [Google Scholar]
  • 41.Wang W, Sun R, Mulvehill AM, Gilson CC, Huang LL. Handling internet-based health information: improving health information web site literacy among undergraduate nursing students. J Nurs Educ. 2017 Feb 01;56(2):110–114. doi: 10.3928/01484834-20170123-08. [DOI] [PubMed] [Google Scholar]
  • 42.Lim HM, Wong SS, Yip KC, Chang FWS, Chin AJZ, Teo CH, Abdullah A, Ng CJ. Online health information-seeking behaviour of patients attending a primary care clinic in Malaysia: a cross-sectional study. Fam Pract. 2022 Jan 19;39(1):38–45. doi: 10.1093/fampra/cmab099.6356361 [DOI] [PubMed] [Google Scholar]
  • 43.Jidkov L, Alexander M, Bark P, Williams JG, Kay J, Taylor P, Hemingway H, Banerjee A. Health informatics competencies in postgraduate medical education and training in the UK: a mixed methods study. BMJ Open. 2019 Mar 30;9(3):e025460. doi: 10.1136/bmjopen-2018-025460. https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=30928942 .bmjopen-2018-025460 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Berner ES, Dorsey AD, Garrie RL, Qu H. Assessment-based health informatics curriculum improvement. J Am Med Inform Assoc. 2016 Jul;23(4):813–8. doi: 10.1093/jamia/ocw073. https://europepmc.org/abstract/MED/27274021 .ocw073 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Valenta AL, Berner ES, Boren SA, Deckard GJ, Eldredge C, Fridsma DB, Gadd C, Gong Y, Johnson T, Jones J, Manos EL, Phillips KT, Roderer NK, Rosendale D, Turner AM, Tusch G, Williamson JJ, Johnson SB. AMIA Board White Paper: AMIA 2017 core competencies for applied health informatics education at the master's degree level. J Am Med Inform Assoc. 2018 Dec 01;25(12):1657–1668. doi: 10.1093/jamia/ocy132. https://europepmc.org/abstract/MED/30371862 .5145365 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Sapci AH, Sapci HA. Teaching hands-on informatics skills to future health informaticians: a competency framework proposal and analysis of health care informatics curricula. JMIR Med Inform. 2020 Jan 21;8(1):e15748. doi: 10.2196/15748. https://medinform.jmir.org/2020/1/e15748/ v8i1e15748 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Alzghaibi H. The gap between bachelor's degree graduates in health informatics and employer needs in Saudi Arabia. BMC Med Educ. 2023 Jun 26;23(1):475. doi: 10.1186/s12909-023-04442-7. https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-023-04442-7 .10.1186/s12909-023-04442-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.McLane TM, Hoyt R, Hodge C, Weinfurter E, Reardon EE, Monsen KA. What industry wants: an empirical analysis of health informatics job postings. Appl Clin Inform. 2021 Mar 31;12(2):285–292. doi: 10.1055/s-0041-1726423. http://www.thieme-connect.com/DOI/DOI?10.1055/s-0041-1726423 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Hovenga E, Grain H. Learning, training and teaching of health informatics and its evidence for informaticians and clinical practice. Stud Health Technol Inform. 2016;222:336–54. [PubMed] [Google Scholar]
  • 50.Yasnoff WA, O'Carroll PW, Koo D, Linkins RW, Kilbourne EM. Public health informatics: improving and transforming public health in the information age. J Public Health Manag Pract. 2000 Nov;6(6):67–75. doi: 10.1097/00124784-200006060-00010. [DOI] [PubMed] [Google Scholar]
  • 51.Gamache R, Kharrazi H, Weiner JP. Public and population health informatics: the bridging of big data to benefit communities. Yearb Med Inform. 2018 Aug;27(1):199–206. doi: 10.1055/s-0038-1667081. http://www.thieme-connect.com/DOI/DOI?10.1055/s-0038-1667081 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Crawford A, Serhal E. Digital health equity and COVID-19: the innovation curve cannot reinforce the social gradient of health. J Med Internet Res. 2020 Jun 02;22(6):e19361. doi: 10.2196/19361. https://www.jmir.org/2020/6/e19361/ v22i6e19361 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Global strategy on digital health 2020-2025. World Health Organization. 2021. [2024-12-01]. https://www.who.int/publications/i/item/9789240020924 .
  • 54.Poncette A, Glauert DL, Mosch L, Braune K, Balzer F, Back D. Undergraduate medical competencies in digital health and curricular module development: mixed methods study. J Med Internet Res. 2020 Oct 29;22(10):e22161. doi: 10.2196/22161. https://www.jmir.org/2020/10/e22161/ v22i10e22161 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Roberts J. Developing health informatics as a recognised professional domain supporting clinical and health management activity. World Hosp Health Serv. 2006;42(4):38–40. [PubMed] [Google Scholar]
  • 56.Cox SM, Johnson SB, Shiu E, Boren S. The national landscape of culminating experiences in master's programs in health and biomedical informatics. J Am Med Inform Assoc. 2021 Mar 18;28(4):856–861. doi: 10.1093/jamia/ocaa348. https://europepmc.org/abstract/MED/33596593 .6141520 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Bellazzi R, Cecconi M, Costantino ML, Veltri P. Bioengineering and medical informatics education in MD programs: perspectives from three Italian experiences. Int J Med Inform. 2023 Apr;172:105002. doi: 10.1016/j.ijmedinf.2023.105002. https://linkinghub.elsevier.com/retrieve/pii/S1386-5056(23)00019-9 .S1386-5056(23)00019-9 [DOI] [PubMed] [Google Scholar]
  • 58.Moen A, Mæland Knudsen L. Nursing informatics: decades of contribution to health informatics. Healthc Inform Res. 2013 Jun;19(2):86–92. doi: 10.4258/hir.2013.19.2.86. https://europepmc.org/abstract/MED/23882413 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Risling T. Educating the nurses of 2025: Technology trends of the next decade. Nurse Educ Pract. 2017 Jan;22:89–92. doi: 10.1016/j.nepr.2016.12.007.S1471-5953(16)30274-8 [DOI] [PubMed] [Google Scholar]
  • 60.Jouparinejad S, Foroughameri G, Khajouei R, Farokhzadian J. Improving the informatics competency of critical care nurses: results of an interventional study in the southeast of Iran. BMC Med Inform Decis Mak. 2020 Sep 11;20(1):220. doi: 10.1186/s12911-020-01244-5. https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-020-01244-5 .10.1186/s12911-020-01244-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Egbert N, Thye J, Hackl WO, Müller-Staub M, Ammenwerth E, Hübner U. Competencies for nursing in a digital world. Methodology, results, and use of the DACH-recommendations for nursing informatics core competency areas in Austria, Germany, and Switzerland. Inform Health Soc Care. 2019;44(4):351–375. doi: 10.1080/17538157.2018.1497635. [DOI] [PubMed] [Google Scholar]
  • 62.Monkman H, Mir S, Borycki EM, Courtney KL, Bond J, Kushniruk AW. Updating professional competencies in health informatics: A scoping review and consultation with subject matter experts. Int J Med Inform. 2023 Feb;170:104969. doi: 10.1016/j.ijmedinf.2022.104969.S1386-5056(22)00283-0 [DOI] [PubMed] [Google Scholar]
  • 63.Hu D, Sun Z, Li H. An overview of medical informatics education in China. Int J Med Inform. 2013 May;82(5):448–66. doi: 10.1016/j.ijmedinf.2012.04.011.S1386-5056(12)00086-X [DOI] [PubMed] [Google Scholar]
  • 64.Asiri HA. Challenges of the health informatics education in the Kingdom of Saudi Arabia: What stands in our way? Journal of Health Informatics in Developing Countries. 2014;8(1):114. https://www.jhidc.org/index.php/jhidc/article/view/114 . [Google Scholar]
  • 65.Jeon E, Kim J, Park H, Lee J, Kim J, Jin M, Ahn S, Jun J, Song H, On J, Jung H, Hong YJ, Yim S. Current status of nursing informatics education in Korea. Healthc Inform Res. 2016 Apr;22(2):142–50. doi: 10.4258/hir.2016.22.2.142. https://europepmc.org/abstract/MED/27200224 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Masic I, Pandza H. Medical informatics education - past, today and future. ejbi. 2018;14(2):40–45. doi: 10.24105/ejbi.2018.14.2.8. [DOI] [Google Scholar]
  • 67.Paranjape K, Schinkel M, Nannan Panday R, Car J, Nanayakkara P. Introducing artificial intelligence training in medical education. JMIR Med Educ. 2019 Dec 03;5(2):e16048. doi: 10.2196/16048. https://mededu.jmir.org/2019/2/e16048/ v5i2e16048 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Yang Y, Shulruf B. Expert-led and artificial intelligence (AI) system-assisted tutoring course increase confidence of Chinese medical interns on suturing and ligature skills: prospective pilot study. J Educ Eval Health Prof. 2019;16:7. doi: 10.3352/jeehp.2019.16.7. https://europepmc.org/abstract/MED/30986892 .jeehp.2019.16.7 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The data and analysis code are available upon reasonable request.

Articles from JMIR Medical Education are provided here courtesy of JMIR Publications Inc.

RESOURCES