The Societal Impact of the Papers Published in Blended Learning in Medicine on Social Network Sites: An Altmetrics Study

Abstract

Background:

Nowadays, blended learning in medicine (BLM) has gained the attention of most experts as an invaluable approach to improving the quality of medical education. The level of attention to articles in this field on social networks is substantial. This study aimed to study the effectiveness of published articles in blended learning, indexed in Scopus and Web of Science databases between 2013 and 2022, from an altmetrics perspective.

Methods:

The research is descriptive-analytical, with a scientometrics approach (using the Altmetrics index). The population includes all the articles on blended learning in medicine, indexed in Scopus and Web of Science databases, two well-known citation databases worldwide. Data were extracted using the Altmetrics bookmarklet tool and analyzed with descriptive statistics methods in Excel software.

Results:

Out of 1327 articles, 136 articles (10.25%) did not have a digital object identifier (DOI) or PMID number. Mendeley, X (previously Twitter), and Dimensions were the most widely used social networks in blended learning. The United States, the United Kingdom, and Australia had the highest number of tweets in blended learning in medicine.

Conclusion:

The number of articles with altmetrics indicators, categorized by publication year, demonstrates an improvement in the familiarity and use of social media by blended learning researchers in medicine. Blended learning researchers are advised to carefully select reputable journals - preferably with DOI - to increase the visibility and attention to their articles on social media.

Introduction

Blended learning is recognized as a combination of in-person (face-to-face) or direct classroom teaching methods and online methods (e-learning) in formal education (1). In its simplest definition, blended learning’s goal is to combine face-to-face and online environments by providing rich environments and using a simple online content repository to support face-to-face classes, leading to better engagement in learning and flexible learning experiences (2). Blended learning methods have been widely accepted in educational networks worldwide, providing more engagement (3). Numerous research studies have been conducted in this field, demonstrating that these blended approaches encourage learning and individual collaboration (4) and create more communication channels among learners (5). Furthermore, research in this area shows that blended learning classes provide a unique environment where the level of learner participation can be analyzed (6).

To assess the value of research in this field and the quality of the conducted research besides the outcomes derived from these studies, the effectiveness of such research should be evaluated through cost-benefit analysis to ensure investors that their capital has been properly spent (7). From this perspective, the quality of research requires scholarly evaluations. To this end, citation indexes were introduced in the 1960s by the Institute for Scientific Information (https://www.webofscience.com/wos/woscc/basic-search) to assess the usefulness of journal articles and have brought about significant changes in research evaluations based on citation analyses. However, there is no doubt that citation analyses have limitations as data analysis is time-consuming and slow (7). Moreover, citation analyses are limited in measuring the impact of publications from the authors’ perspective (8). In this regard, most researchers have referred to the application of altmetrics, social web-based indicators, as a solution to the above problem, indicators that reflect the impact of scientific articles (7).

The movement of altmetrics aims to collect the invisible impacts of scientific publications, both old and new, based on crowdsourced data gathered from social networks such as blogs, microblogs, social bookmarking tools, and reference management tools (8). One of the tools used in bibliometrics is Altmetric Bookmarklet, which ranks among the latest altmetrics tools and represents a novel research area in assessing research impact in the Web 2.0 or social web environment to provide a new index in social networking platforms. By this, downloads of the reviewed articles, attention to the articles, mentions in social media, news outlets, blogs, and generally in web 2.0-based services are measured. One of the less explored areas related to social networks is medical education, particularly blended learning.

Therefore, we aimed to investigate the status of published articles by researchers in BLM, indexed in Scopus and Web of Science databases between 2013 and 2022, from an altmetrics perspective.

Methods

The present study was a practical descriptive-analytical research conducted using the scientific survey method and applying the Altmetrics Bookmarklet tool. It was used to examine the effectiveness of articles in the field of BLM, due to the credibility and importance of this tool. The extensive use of previous research in the field of altmetrics by Altmetrics Bookmarklet may indicate the importance and credibility of this tool. By this, downloads of the examined articles, attention to the papers, and mentions of them on social media, news media, blogs, and generally web-based services were measured. Altmetrics Bookmarklet gathers relevant data about journal articles from news, blog posts, tweets, and posts related to research articles, and based on the information provided by each of these data sources, it assigns a score.

Therefore, by using the Altmetrics Bookmarklet tool, all references to Blended Learning articles in medicine, as well as the publication of videos, texts, or writings related to them, and relevant markers on each of the previously mentioned websites and sources, were studied and examined over a ten-year period. Different scores for these articles were obtained. The sum of these scores represents an overall Altmetrics Score, which indicates the level of sharing, attention, and use of these articles on social media.

In fact, the assigned Altmetrics Score represents the quantity and quality of attention- a document has received on social media. Only articles with Digital Object Identifier (DOI) and PMID or other standard identifiers were eligible for examination by Altmetrics Bookmarklet.

The present research community included scientific articles in the field of Blended Learning in medicine that have been indexed in Scopus and Web of Science databases during the time period from 2013 to 2022. Scopus and Web of Science databases were used to collect the data due to their importance among researchers in various fields. The search strategy was developed by entering the following terms in the field of the Article title, Abstract, Keywords in Scopus, and the Topic field in WoS on 22 Apr 2023. The search strategy was used as follows:

(“blended learning” OR “technology-mediated instruction” OR “technology mediated instruction” OR “web-enhanced indtruction” OR “mixed-mode instruction” OR “mixed mode instruction” OR “mixed-mode learning” OR “mixed mode learning” OR “hybrid learning” OR “personalized learning” OR “differentiated instruction” OR “blended education” OR “mixed learning” OR “blended courses” OR “combined education” OR “combined learning” OR “integrated learning”) AND medic*

To remove duplicate articles in these two databases, the search output from each database was imported into the EndNote software, and the “Discard duplicate” option was selected. Based on this, 1191 articles were retrieved from the Scopus database and 769 documents from the Web of Science database. After removing duplicate articles (633 papers), 1327 articles were obtained. Then, each of the 1542 research articles in the field of Blended Learning in Medicine that were indexed in these two databases was individually retrieved and examined using the “altmetric it” extension, previously added to the bookmarks section of the browser. If the article had an Altmetric score, it was displayed, and by clicking on this score, its details could be viewed. Finally, the obtained data were imported into Excel software for further analysis.

Results

The results of the examination of research articles in the field of BLM indexed in Scopus and Web of Science databases showed that out of a total of 1327 articles, 136 articles (10.25%) did not have a digital object identifier (DOI) or PMID number, making it impossible to examine them using the bookmarklet tool Altmetric. Among the articles with a DOI (1191), 558 records (42.05%) were mentioned in social networks, while 633 articles (47.70%) were not mentioned in any social networks.

According to Table 1, the presence of research articles in the field of blended learning in medicine has increased on social networks during 2017–2022, while it has fluctuated from 2013 to 2016.

Table 1:

Articles with Altmetric scores in the field of BLM, categorized by time period

Year	Number of articles	Articles with altmetrics score	Total altmetrics score	Average altmetrics score
2013	64	30	179	5.96
2014	76	26	142	5.46
2015	70	37	174	4.70
2016	92	49	414	8.45
2017	77	35	213	6.09
2018	123	57	377	6.61
2019	147	63	474	7.52
2020	164	68	392	5.76
2021	221	83	379	4.57
2022	293	110	835	7.59
2013–2022	1327	558	3576	6.41

Furthermore, the average altmetric score each year was 6.41. Among these, the highest average altmetric score was for scientific outputs in blended learning in medicine in 2016, and the lowest average score was for 2014.

According to Fig. 1, Mendeley, as a reference management tool, was the most widely used platform. Five hundred and fifty-two articles by researchers in BLM were mentioned 56595 times on Mendeley.

Fig. 1:

Distribution of social networks sharing articles with Altmetrics scores in the field of BLM

15.55% of all tweeters of articles related to BLM are from the United States, thus the United States has the highest percentage of tweet contributions in this field. Following the United States, the United Kingdom ranks second with 15.14% and Australia ranks third with 4.97% in terms of the number of tweet contributions in the field of BLM.

Furthermore, in 40.17% of the tweets, it was not possible to determine the geographical location of the senders due to incomplete profile information.

Table 2 indicates that 44.03% of the tweet authors discussing Blended Learning in Medicine research articles are Members of the Public. Therefore, ordinary individuals have the highest share of tweeting about research articles in this field. Following that, Scientists account for 26.46%, Practitioners (doctors, other healthcare professionals) for 23.38%, and Science communicators (journalists, bloggers, editors) for 6.13% in terms of the amount of tweeting about Blended Learning articles in Medicine.

Table 2:

Distribution of tweet articles in the field of Blended Learning in Medicine, based on the roles of the senders

Rank	Tweeters	Number of Tweets	Percentage of Tweeters
1	Member of Public	1343	44.03
2	Scientist	807	26.46
3	Practitioners (doctors, other healthcare professionals)	713	23.38
4	Science communicators (journalists, bloggers, editors)	187	6.13

0.14% of the references of research articles by Blended Learning researchers in the field of medicine are related to the United Kingdom. Therefore, the UK has allocated the highest percentage of article references in this field to itself. After the United Kingdom, the United States with 0.12%, and Malaysia with 0.04% are in the next ranks in terms of the percentage of article references in BLM.

In 99.29% of the references of research articles in BLM, it was not possible to determine the geographical location of the referrers in Mnedeley due to incomplete profile information.

Table 3 indicates that 10.87% of the total citation authors of research papers in the field of BLM in Mendeley are Master’s students. Therefore, Master’s students have the highest share in the citation of research papers in these fields in Mendeley. Following that, Bachelor’s students have 8.07%, Ph.D. students have 6.75%, and researchers have 6.15% in terms of the number of citations by researchers in the field of BLM in Mendeley.

Table 3:

Distribution of citations of scientific articles in the field of BLM in Mendley, based on roles of readers

Rank	Readers by professional status	Number of Readers	Rank Percentage of Readers	Rank	Readers by professional status	Number of Readers	Rank Percentage of Readers
1	Master Student	6157	10.87	7	Postgraduate Student	649	1.15
2	Bachelor Student	4568	8.07	8	Associate Professor	424	0.75
3	Ph.D. Student	3825	6.75	9	Professor	175	0.31
4	Researcher	3484	6.15	10	Librarian	130	0.23
5	Lecturer	3019	5.33	11	Senior Lecturer	161	0.28
6	Doctoral Student	2180	3.85	12	Unknown/Other/Unspecified	31863	56.26

Furthermore, in 56.26% of Mendley’s citations, it was not possible to determine the role of the citation authors due to incomplete profile information provided by the senders.

The findings of Table 4 show that 19.874% of the research articles referring to the field of BLM in Mendeley are related to the subject area of Medicine and Dentistry. Therefore, researchers and professionals in the field of Medicine and Dentistry have the highest share in referencing research articles on Blended Learning in Medicine in Mendeley. After that, the subject areas of Nursing & Health Professions with 8.893%, and Social Science with 8.674% rank next in terms of the number of references to BLM in Mendeley.

Table 4:

Distribution of citations of scientific articles in the field of BLM in Mendeley, categorized by topic

Rank	Readers by discipline	Number of Readers	Percentage of Readers	Rank	Readers by discipline	Number of Readers	Percentage of Readers
1	Medicine & Dentistry	11731	19.874	16	Sports & Recreations	43	0.073
2	Nursing & Health Professions	5249	8.893	17	Mathematics	41	0.069
3	Social Sci.	5120	8.674	18	Physics & Astronomy	35	0.059
4	Computer Science	2092	3.544	19	Economics, Econometrics & Finance	24	0.041
5	Psychology	1424	2.412	20	Environmental Sci.	18	0.030
6	Business, Management & Accounting	898	1.521	21	Immunology & Microbiology	15	0.025
7	Agricultural & Biological Sci.	641	1.086	22	Chemistry	12	0.020
8	Engineering	638	1.081	23	Chemical Engineering	5	0.008
9	Arts & Humanities	433	0.734		Materials Sci.	5	0.008
10	Pharmacology, Toxicology & Pharmaceutical Sci.	164	0.278	24	Design	2	0.003
11	Biochemistry, Genetics & Molecular Biology	144	0.244		Energy	2	0.003
12	Linguistics	105	0.178	25	Earth & planetary	1	0.002
13	Neuroscience	74	0.125		Decision S.	1	0.002
14	Philosophy	46	0.078	26	Unknown/Other	30020	50.858
15	Veterinary Sci. & Veterinary Medicine	44	0.075

Additionally, in 58.858% of the references in Mendeley, it was not possible to determine the subject areas of the referrers due to incomplete profile information provided by the senders.

Discussion

According to the information from the Altmetrics institution, a study of research articles in the field of Blended Learning in medicine published between 2013 and 2022 in Scopus and Web of Science databases shows that 10.25% of the papers were without a DOI or standard markings. Therefore, it was not possible to examine them using the Altmetrics bookmarking tool. More than half of these articles were presented papers at various international conferences. These articles were usually published in journals, in which either no indicator was assigned to the articles or the assigned marking was incorrect. Additionally, the number of articles in this field with a DOI or standard marking has increased between 2013 and 2022. The use of a digital identifier is substantial because it serves as an identification card for an article, thus increasing the credibility of its sources. Among these, 42.05% of the papers with a DOI have received attention on social networks. The reasons for this can be attributed to the sufficient regard of the authors of these articles to create a scientific profile on social networks and consequently increase the attention to their articles. Another reason is the proper understanding of these researchers on knowledge translation (expressing specialized knowledge in a simple and understandable language) in social networks, though it cannot be stated with certainty.

Alongside the increase in the number of research articles on Blended Learning in medicine with DOI or any standard tagging during the years 2013 to 2022, the presence of articles in this field on social networks has also grown, since this upward trend has been accompanied by fluctuations from 2013 to 2016. The increase in the presence of scientific productions on social media has been reported in some studies (9,10). It can be due to researchers’ interest in increasing the visibility and impact of their research in the scientific community, which also allows them to access scientific advancements. In addition, universities requiring researchers to register their research identifiers in university systems and incorporate them into national university evaluations, as well as the increasing importance of webometrics at the global level, are other factors contributing to the increased sharing of scientific productions by researchers on social networks

Nonetheless, in general, it indicates an improvement in familiarity and use of social media by researchers. The average Altmetric score for articles each year was determined to be 6.41. Among them, the highest average Altmetric score was for research articles in Blended Learning in Medicine in 2013, and the lowest average was for the year 2022, which can be attributed to the short time interval from the publication of the relevant article. The previous study has also shown that with increasing publication age, the attention to them in Altmetrics tools has also increased (11). Investigating various social media platforms shows that Mendeley, X, Dimensions, Facebook, News outlets, Blogs, Policy sources, Google Plus, Wikipedia, video uploaders, CiteULike, peer review sites, Redditor, and research highlight platforms are the sharing tools for research articles in the field of BLM during the years 2013–2022. X and Mendeley have the highest statistics and are more important than other social media platforms. The importance of these two social media platforms has also been mentioned in previous research (12–17). Furthermore, the importance of using X compared to other media has been highlighted in some previous research (18–20). In general, the coverage of altmetrics sources has become a fundamental challenge for its use in research evaluation. One of these challenges is the difference in research fields. This leads to variety in their popularity on different platforms. Some applied areas that relate to everyday life issues may result in virtual activities and research sharing on social networks (19). Therefore, the popularity of Blended Learning publications on X may be due to the interest of platform members in this field.

By examining the tweets of scientific articles in this field, the sending of tweets by other interested researchers, scientific journals, and educational institutions was confirmed in addition to the authors themselves (21). Furthermore, the scope and subject area covered by article-sharing tools can also affect the presence of researchers and articles from various fields. Some platforms with specialized audiences may be used less than tools with a broader audience. The geographical distribution study of tweets on scientific articles in this field showed that 15.55% of the total tweets on scientific papers belonged to the United States, 15.14% to the United Kingdom, and 4.97% to Australia. Previous studies have acknowledged the contribution of the United States and the United Kingdom to tweeting scientific articles (15–16 and 20). The scientific community extension, the availability of the research budget, and perhaps a greater inclination towards disseminating research findings may be among the reasons for this issue (22). In 40.17% of the tweets, the geographical location of the tweet senders was impossible due to incomplete profile information. Furthermore, ‘Member of Public’ (44.03%) had the highest share in tweeting research articles in these fields. The reason for such a high share may be attributed, on the one hand, to the presence of prominent users who have identified themselves as ordinary users in their X profiles regardless of their level of education, and on the other hand, to the interest of individuals in following research related to Blended Learning in Medicine. Furthermore, this can be seen as a potential of scientific tools to disseminate scientific information to the public by transforming specialized knowledge into understandable knowledge for the general population, making practical knowledge available to individuals in society.

Upon analyzing the references of scientific articles in the field of BLM in Mendeley, there was a high interest of students from different academic levels in collaborating with researchers to study and share scientific papers in the field of BLM. This result indicates the interest of students of different educational levels in collaborating with researchers to study and share scientific articles in the field of BLM (20, 23, 24).

Altmetrics is a new method that measures scientific texts in social media. The prominent indicators in altmetrics have gained popularity as quality assessment indicators for various types of research. These indicators can be used as complementary to traditional indicators. Therefore, researchers are striving to share their work on social networks. Additionally, altmetrics indicators lead to results in a short period. Most altmetrics-related studies belong to recent years, and alternative metrics are in their early stages, requiring further research. A thematic review of articles in BLM showed that (23) the subject areas with the highest number of citations in the Mendeley were Medicine and Dentistry, followed by Nursing and Health Professions. After Medicine and Dentistry, previous research in the field of medical sciences (medicine and para-medicine) was identified as one of the subject areas with the highest presence on social media (24). As researchers in medicine and dentistry are constantly seeking new therapeutic approaches and methods, they are interested in publishing their research through social media. Furthermore, through this platform, research in this field will reach the general public. On the other hand, the research field examines BLM, it is expected that researchers in the medical field will be more active in citing research outcomes in this area compared to other researchers.

Conclusion

Fluctuations in publishing articles and research reports by experts in Scopus and Web of Science databases have been noticeable during the years under study. However, these productions have generally experienced natural and acceptable growth. Additionally, the number of articles with altmetrics indicators, categorized by publication year, demonstrates an improvement in the familiarity and use of social media by Blended Learning researchers in medicine. Blended Learning researchers in medicine are advised to select carefully reputable journals - preferably with DOI - to increase the attention to their articles on social media. It originates from the fact that individuals tend to rely more on networks and social media platforms for obtaining information compared to scientific databases.

Journalism Ethics considerations

Ethical issues (Including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc.) have been completely observed by the authors.