Bibliometrics and altmetrics in the context of traditional, complementary, and integrative medicine

Abstract

This educational article explores the utility of bibliometrics and altmetrics in evaluating traditional, complementary, and integrative medicine (TCIM) research. We introduce the concept of bibliometrics and altmetrics, provide an overview of the opportunities and challenges in using these analytical techniques, and highlight their future potential in TCIM research. Bibliometrics, based on publication and citation information, offer insights into TCIM research activity, output, scholarly influence, field structure, and collaboration practices. Altmetrics, including social media mentions, downloads, and online bookmarks, can capture the broader societal engagement with TCIM research beyond traditional academic circles. We discuss challenges and opportunities in utilizing these metrics effectively, such as addressing biases, incorporating cultural nuances, and exploring emerging trends. Additionally, we issue a call to action for researchers, policymakers, and practitioners to collaborate and leverage bibliometric and altmetric data to advance evidence-based healthcare practices to promote integrative approaches to health and wellness. By embracing a multidimensional approach to research evaluation, stakeholders can harness the potential of bibliometrics and altmetrics to improve TCIM research and healthcare delivery worldwide.

1. Introduction

Bibliometrics is the quantitative analysis of research outputs and citations and can be used as the basis for measuring the activity, impact, collaboration, and structure of academic entities, such as researchers, institutions, fields, countries, or the entire global academic system. Altmetrics, on the other hand, provide alternative metrics beyond traditional citation-based measures, incorporating online interactions to gauge the broader societal impact of research outputs.

The World Health Organization (WHO) defines traditional medicine as “…the sum of the knowledge, skills and practices based on the theories, beliefs and experiences indigenous to different cultures, whether explicable or not, used in the maintenance of health and the prevention, diagnosis, improvement or treatment of physical and mental illness.”.¹ The US National Center for Complementary and Integrative Health (NCCIH) defines “complementary medicine” as a non-mainstream approach “used together with conventional medicine”, and “integrative medicine” as that which “brings conventional and complementary approaches together in a coordinated way”.² Given this scope, complementary and integrative medicine specifically requires a relationship with conventional medicine, limiting classifications of therapies not directly associated with standard healthcare. While acknowledging that differences in definitions exist,³ for the purpose of this article we will refer to this group of therapies as traditional, complementary, and integrative medicine (TCIM).

TCIM encompasses diverse practices that include, but are not limited to, acupuncture, herbal treatments, yoga, and mindfulness. They can also refer to whole medical systems that have evolved over time in different cultures and parts of the world, such as Ayurvedic medicine or traditional Chinese medicine. Publications of bibliometric research, has rapidly grown in TCIM journals, with a particularly high volume of articles from countries including China, the United States, and Germany.⁴ A recent scoping review by Ng et al.⁵ which aimed to examine the characteristics of bibliometric studies within TCIM literature, has provided a broader understanding of research trends and methodological approaches in this field. This review analysed 286 bibliometric studies on CAIM published between 1995 and 2023, with most appearing in the past five years and half originating from China. While all studies used performance analysis and many applied science mapping—most commonly co-word and co-authorship techniques, there was considerable methodological variability, emphasizing the need for standardized approaches and broader use of metrics and data sources in future research.⁵ To our knowledge, only one altmetric analysis has been conducted to date specific to the overall TCIM field.⁶ This study analysed 62,278 TCIM research outputs from the Altmetric.com database, spanning from 1925 to 2024, with over 497,000 online mentions, primarily on X (formerly Twitter), Facebook, and news outlets. Most outputs were published in a small set of journals, with open-access articles receiving higher average attention scores, particularly those under bronze and green OA types. Although fewer than 15 % of outputs had high Altmetric attention scores (≥10), these tended to focus on pain, mental health, and therapies like acupuncture and plant extracts, with online popularity not always aligned with citation counts.⁶

As TCIM continues to gain popularity worldwide, patients and practitioners alike are recognizing its potential to enhance conventional treatments, fostering a holistic approach to healthcare. For the purposes of this article, we will use the term TCIM to encompass this range of integrative, traditional, and complementary therapies that together support a comprehensive view of health and wellness.

Bibliometrics and altmetrics provide valuable tools for assessing the impact, visibility, and societal engagement of TCIM research.⁷ By leveraging these metrics, researchers, policymakers, and practitioners can gain insights into the dissemination of TCIM knowledge, evaluate the impact of TCIM research, and inform evidence-based decision-making in TCIM policy and practice. Understanding the role of bibliometrics and altmetrics in TCIM research facilitates interdisciplinary collaboration, knowledge exchange, and the integration of diverse perspectives and methodologies into research evaluation frameworks.

Bibliometric and altmetric analyses are under-utilized in TCIM research. As such, this educational article provides an introduction of bibliometrics and altmetrics to TCIM researchers and explores how these metrics can be applied in the context of furthering TCIM research by enhancing understanding of the field. By examining how these metrics can be utilized to analyze the value of TCIM publications, we provide insights into the advantages, disadvantages, and value of bibliometrics and altmetrics in assessing the visibility and impact of TCIM research. Overall, we aim to enhance understanding and value of the potential of these metrics among TCIM researchers to advance TCIM scholarship and inform evidence-based healthcare practices. Table 1 provides a summary of examples and applications of bibliometric and altmetric indicators for assessing traditional, complementary, and integrative medicine research.

Table 1.

Examples and applications of bibliometric and altmetric indicators for assessing traditional, complementary, and integrative medicine research.

Type of Indicator	Examples of Indicator	What it Captures	Example of Uses
Bibliometric Indicators
Publication-based indicators	Publication counts, shares of output in a field, country or geographical region	Level of academic output	Measuring the change in an institution's output of TCIM publications over time
Citation-based indicators	Entities’ average citations, shares of highly-cited publications	Level of impact in the academic community	Determining the most impactful institutions based on their share of highly-cited outputs
Author-based indicators	Counts or shares of publications that are collaborations	Level of collaboration	Identifying the authors, institutions, or countries an entity often publishes with
Structural indicators	Analyses of co-citation, co-author, or co-occurrence of key words	Cognitive structure of a field or system	Mapping the TCIM field into sub-fields based on methodological or topical orientations
Altmetric Indicators
Viewed	Number of views of a publication on the journal’s website	Extent of low-level engagement with an output	Comparing the average visibility of different journals
Saved	Number of times an output was saved to a reference manager	Level of relevance and potential level of citation impact	Attaining an early approximation of potential citations
Discussed	Number and content of comments about research outputs	Level and content of conversations about the output	Identifying the topics of discussion amongst X users about TCIM research outputs
Recommended	Number of recommendations of a publication	Level of endorsement of an output	Capturing the transfer of information into mediums for general audiences, e.g., Wikipedia

2. Understanding bibliometrics

2.1. Definition and scope

Bibliometrics is a quantitative approach for analyzing the visibility and impact of scholarly publications within academia. Bibliometric methods involve the measurement and evaluation of various aspects of publications, such as citations, authorship patterns, and impact metrics. Using these quantifications, bibliometrics provides insights into the dissemination, impact, and structure of research outputs, facilitating the understanding of scholarly communication dynamics and knowledge dissemination processes.

Bibliometrics extends beyond merely counting citations; it involves sophisticated analyses to reveal patterns and trends in scholarly literature.⁸ Bibliometric techniques enable the identification of seminal works, prolific authors, influential journals, and emerging research topics, and facilitate the assessment of research impact at various levels, including individual researchers, institutions, and disciplines. Essentially, bibliometrics is a valuable tool for quantifying the impact and visibility of scholarly publications, which enables informed decision-making, resource allocation, and strategic planning in academia and by funding agencies.

2.2. Key bibliometric indicators

Bibliometrics utilizes a suite of indicators to provide valuable insights into research activity, visibility, collaboration, structures, and the reception of work in academic circles. Bibliometric indicators can be divided into four categories based on their unit of focus: publication-based indicators, which gauge the activity of an entity (such as an individual researcher, a department, an institution, or a country); citation-based indicators, which capture the reception or acceptance of an entity’s output within academia; author-based indicators, which examine the composition of authorship teams and collaborations; and structural indicators, which define the structure of an entity, such as the most central topics and authors.

2.2.1. Publication-based indicators

The foundation of bibliometric analyses is research output, and particularly journal articles.⁹ A fundamental bibliometric indicator is the count or mean number of articles produced by an entity in a specified period of time, which provides insight into its level of activity. The percentage of an entity’s publications in relation to a higher aggregate may also be insightful, such as a country’s share of global publications, or a researcher’s share of total publications in a field. However, consideration must be given to entities’ sizes, such as institutions’ number of research staff or countries’ levels of research expenditure, when making comparisons to ensure meaningful interpretations.¹⁰ Publication-based indicators can thus provide insight into the activity or productivity level of entities and their trends over time.

2.2.2. Citation-based indicators

2.2.2.1. Citation counts and averages

A citation is a reference from one academic output to another, typically to reflect that information was obtained from this source.⁹ Citation counts are a fundamental metric in bibliometrics, representing the number of times a particular output has been cited by other publications. Citations offer a quantitative proxy measure of the influence and relevance of a publication, with high citation counts interpreted as the publication being of particular significance to the academic community.¹¹ Analyses often examine the raw or mean citation counts of an entity’s publications in a particular timeframe, and in comparison with other relevant entities.¹⁰ Notably, citations require time to accrue to levels that represent their long-term impact. Consequently, citation-based indicators should be calculated only after a sufficient period of time has elapsed for citations to amass, which is typically a number of years.¹²

2.2.2.2. Highly cited and uncited publications

The percentage of an entity’s publications that are in the top 10 % most highly cited publications from a particular field and year (known as the PPtop10 or Excellence Rate) is also a common measure of high performance and impact.¹⁰ A PPtop10 score above or below 10 % indicates the entity performed better or worse than expected, respectively. Conversely, the number and percentage of an entity’s publications that remain uncited after a particular period of time might provide insight into less impactful outputs and topics.

2.2.2.3. Journal impact factor and its equivalents

The Journal Impact Factor (JIF) is a citation-based indicator that gauges the impact of journals. JIF is the name of this indicator in the Web of Science (WoS), while CiteScore in Scopus¹³ and the SCImago Journal Rank indicator¹⁴ are calculated similarly. Essentially, these indicators measure the average number of citations received by articles published in a specific journal within a particular time frame, usually two or three years.¹⁰ These metrics reflect the journal's scholarly impact, often influencing researchers’ publishing decisions and academic recognition. However, these indicators have several limitations, including a dependence on the citation and publication practices of the journal’s field, and susceptibility to gaming by publishers.⁹ Importantly, these impact measures are applicable only to journals and are not suitable for individual- or institution-level analyses.

2.2.2.4. H-index

The h-index is a composite bibliometric indicator that considers both the levels of output and impact of an entity. Derived from the entity’s number of publications and the number of citations received by each publication, the h-index is the h number of publications with at least h citations.¹⁰ For instance, an author with 6 published articles with 0, 2, 4, 5, 9, and 12 citations, respectively, would have an h-index of 4, as 4 publications have at least 4 citations. The h-index captures both the quantity and impact of an entity's research output, providing a more comprehensive assessment than citation counts alone.⁹ However, as citations accrue over time, the h-index favours researchers with longer careers and so researchers’ “scientific ages” should be considered. Adjusted alternatives such as the m-index, defined as the h-index divided by the number of years since the scientist's first publication,¹⁵ are available, but used less frequently. Additional considerations regarding the h-index are discussed by Mingers & Leydesdorff.⁹

2.3. Considerations regarding citation-based indicators

When using citation-based indicators, it is important to recognise particular characteristics of citation data. For instance, there are many reasons why a publication might be cited and this context is not reflected in citation counts.⁹ Further, citation practices may vary substantially between disciplines, time periods, document types, open and closed access types, and between data sources due to the types of materials covered and cited.^9,10 Consequently, adjustments must first be applied to citation data to “normalize” these differences and facilitate meaningful comparisons between entities.¹⁶ Mingers and Leydesdorff⁹ provide a detailed discussion of normalization methods and considerations.

2.4. Author-based indicators

These indicators focus on aspects of a publication’s author(s). Usually, the underlying concept of interest is the level of collaboration in research outputs. Key indicators pertain to the percentage of an entity’s publications that are collaborations with other researchers, institutions, or countries. Author-based analyses can also be used to identify an entity’s key collaboration partners in particular topics and fields, and the effect of collaboration on an entity’s citations. Author-based indicators can reveal key partnerships and assist researchers and institutions in developing collaboration strategies.

2.5. Structural indicators

Many of the aforementioned indicators center on analysing an entity’s performance. In contrast, structural indicators enable analysts to uncover the cognitive structure of a field or system. Typically, these analyses focus on determining and mapping which topics, concepts, and actors are most central to a field.

The premise behind structural indicators is that citations, authors, or mentions of particular words that co-occur are likely to have similar characteristics, and analyzing these co-occurrences at a large scale can reveal underlying structures.¹¹ Co-citation analysis examines the relationships between cited and citing articles to identify themes and sub-areas of a field. Similarly, connections between co-authors can reveal both geographical structures and cognitive structures, e.g., groups of authors using particular theoretical frameworks or methodologies.¹¹ Examining co-occurrences of key words between publications can also reveal relations, such as the application of a specific method in two different fields, or identify emerging directions of research.¹¹

Structural indicators are useful for analyzing the networks and structures of a system. These indicators can complement performance-oriented indicators to provide insight into the structural features underpinning entities’ activity, collaboration, and impact. In combining an array of bibliometric indicators, analysts can generate a comprehensive understanding of entities’ structure and performance, facilitating evidence-based decision-making.

2.6. Key bibliometric data sources

Bibliometric data sources consist of large-scale collections of academic outputs, especially content from journals and conference proceedings.¹⁷ These databases record a variety of metadata about outputs, their authors, the publishing source, and citations between documents. These data form the basis of bibliometric indicators of activity, impact, collaboration, and structure. Evidently then, the content and accuracy of these databases strongly influences the indicators they produce.

Currently, the key bibliometric databases available with broad field coverage are WoS, Scopus, OpenAlex, and Dimensions. Databases specific to medicine include MEDLINE and EMBASE, however, a multitude of databases specific to TCIM also exist.^18,19 There are a range of differences between these sources. However, one prominent aspect is that WoS and Scopus are proprietary databases offered by Clarivate and Elsevier, respectively, on a subscription basis. Both of these providers utilize content selection boards to curate the sources indexed in their databases based on an array of criteria.^20,21 In contrast, OpenAlex and Dimensions are freely available data sources that prioritize inclusivity, with little or no selection process for the sources covered.^22,23

As a result of their indexation policies, the databases vary substantially in their size and focus. WoS indexes approximately 92 million items,²⁴ Scopus includes more than 97.3 million items,²⁵ and Dimensions and OpenAlex are substantially larger, with 137 million²⁶ and 243 million items,²⁷ respectively. Similarly, WoS and Scopus focus on more impactful journals and thus mostly index internationally-oriented, English-language journals.¹⁰ Dimensions and OpenAlex have greater diversity in content than WoS and Scopus. However, they also have more issues with incomplete or inaccurate metadata.^17,26,27 Analysts should carefully choose a database to ensure it fits their purpose and adequately captures the relevant output, so that results are accurate and reliable.

2.7. The potential for application of bibliometrics in TCIM research

The application of bibliometrics in TCIM research could offer valuable insights into the scholarly landscape of this interdisciplinary field. Bibliometric analyses would enable researchers, policymakers, and healthcare professionals to systematically evaluate the contributions of TCIM literature to the broader academic discourse and healthcare practices.

Bibliometrics could facilitate the identification of key publications, influential authors, and impactful journals within the TCIM domain. By analyzing citation patterns and publication trends, researchers could identify seminal works that have significantly shaped the development of TCIM research and practice. Moreover, bibliometric techniques would allow for the mapping of collaborative networks and interdisciplinary connections within TCIM, highlighting the cross-pollination of ideas and expertise across diverse disciplines. This has been shown in previously published bibliometric analyses of TCIM research.^4,28, 29, 30, 31

Furthermore, bibliometrics could aid in assessing the impact of TCIM research on healthcare policies, clinical guidelines, and patient outcomes.32, 33, 34, 35 Citation analyses could provide quantitative measures of the extent to which TCIM publications are referenced in the broader healthcare literature. Such insights are invaluable for policymakers and practitioners seeking evidence-based strategies to integrate TCIM approaches into mainstream healthcare systems.

Moreover, bibliometrics could inform strategic decision-making processes in TCIM resource allocation. By quantifying research activity and impact, funding agencies could prioritize investments in areas of TCIM research that demonstrate high visibility, scholarly influence, and potential for societal impact. Additionally, bibliometric analyses could help identify research gaps and emerging areas of interest within TCIM, guiding future research agendas and interdisciplinary collaborations.

Overall, the application of bibliometrics in TCIM research would facilitate a deeper understanding of the field's scholarly landscape, impact, and contributions to healthcare innovation. By leveraging bibliometric techniques, stakeholders could harness the full potential of TCIM research to advance evidence-based healthcare practices, promote interdisciplinary collaboration, and improve patient outcomes.

2.8. Limitations of bibliometrics

While bibliometrics provides valuable tools for assessing scholarly publications, its application in the context of TCIM research is not without limitations. These limitations stem from the unique characteristics of TCIM literature and the challenges inherent in capturing its full impact through quantitative measures alone. A significant limitation of bibliometrics in TCIM research is the incomplete coverage in bibliometric data sources of TCIM literature. TCIM encompasses a diverse range of healthcare practices, various therapies, research outputs, dissemination channels, and cultural contexts. However, bibliographic data sources such as WoS and Scopus prioritize indexation of journal articles and conference papers in internationally oriented, English-language sources.¹⁰ Consequently, non-traditional TCIM research output, such as community-based interventions, and patient-centered outcomes research, or publications in non-English languages, from certain regions or cultures, or that appear in non-traditional journals or as closed access may be overlooked in these databases. Analyses that exclude this research might produce biased assessments of the authors and institutions contributing to the field, the topics examined, and of the level of activity, impact, collaboration, and structure of TCIM research overall.

How a bibliometric analysis is conducted is critical in order to realize its potential and practical application for integrative medicine research.³⁶ To produce high-quality data there needs to be a rigorous research design together with consideration given to managing structural differences in the system, such as varying citation practices.¹¹ The methodologies outlined by Gan et al³⁶ and Donthu et al¹¹ provide some suggestions and highlight problems that may be encountered, such as the lack of in-depth analysis of content and the diversified subject content.

Additionally, bibliometrics may struggle to capture the societal impact and patient-centered outcomes of TCIM research. Many TCIM interventions focus on holistic approaches to health and well-being, emphasizing patient empowerment, self-care, and quality of life improvements. These outcomes may not always be reflected in citation-based metrics, which predominantly measure academic impact within scholarly circles.

While bibliometrics offers valuable insights into TCIM research, its application is subject to limitations. Overcoming these limitations requires a nuanced understanding of the unique characteristics of TCIM literature and the bibliometric data available, and the adoption of complementary approaches to fully capture TCIM research.

3. Exploring altmetrics

3.1. Definition and utility

Altmetrics, a portmanteau of “alternative metrics”, was introduced as a contemporary approach to evaluating the impact and reach of scholarly outputs beyond traditional citation-based metrics. Unlike bibliometrics, which predominantly rely on citations to gauge research impact, altmetrics encompass a diverse array of non-traditional indicators derived from various digital platforms to reflect the broader societal engagement with research outputs.³⁷

The utility of altmetrics lies in their ability to provide a broader and more nuanced understanding of societal impact in today's digital age. While bibliometrics offer valuable insights into scholarly influence in academic circles, altmetrics extend the evaluation framework to encompass wider societal interactions with research findings. By tapping into online activities, altmetrics capture the immediate and dynamic dissemination of research across diverse audiences, including academics, practitioners, policymakers, and the general public.

3.2. Types of altmetric indicators

Altmetrics can be classified based on the type and level of engagement they capture^38,39:

3.2.1. Viewed

Representing the lowest level of engagement with an article, views reflect the number of times an output is accessed from online repositories or publisher websites. Views may indicate the relevance and usage of research findings by both academic and non-academic audiences, including researchers, practitioners, and other stakeholders.

3.2.2. Saved

Saved metrics track the number of times an output is bookmarked on reference management platforms such as Mendeley or Zotero. These bookmarks serve as indicators of research relevance, highlighting articles or resources of particular interest to the scholarly community. Also, the number of bookmarks on Mendeley tends to correlate with citations,⁴⁰ so online bookmark metrics may also provide insights into the long-term citation potential of research outputs.

3.2.3. Discussed

These metrics track the online conversations surrounding research on platforms such as X, Facebook, and YouTube.³⁹ Discussion metrics provide near-immediate feedback on the reception and dissemination of research, offering insights into the level of public interest and societal impact of the research, and identify influential stakeholders in online communities. Discussions may also contain text regarding readers’ opinions about the output, facilitating qualitative analyses to examine why readers engage with the work, in addition to metrics about the level of engagement.

3.2.4. Recommended

Another level of altmetric engagement with an output is a recommendation, in which individuals endorse the output or refer it to others.³⁹ ResearchGate allows recommendations, and coverage of academic output by mainstream news outlets or inclusion on a Wikipedia page may also be considered a recommendation by altmetric data providers. Recommendations thus identify research outputs of particular relevance to an academic or non-academic community via endorsement.

By capturing these diverse types of activity, researchers gain a more comprehensive understanding of the visibility and impact of their research across digital platforms. Altmetrics complement traditional bibliometric measures by offering insights into the real-time dissemination and societal relevance of scholarly research, facilitating broader conversations and collaborations in and beyond the academic community.

3.3. Key altmetric data sources

Altmetric aggregators, such as Altmetric.com, PlumX, and ImpactStory, are usually applied for examining altmetric data, as they collect data for a large sample of outputs and sources simultaneously. These services monitor multiple online sources for mentions of outputs and aggregate them into metrics for specific entities.⁴¹ PlumX and Altmetric.com’s data are integrated into the Scopus and Dimensions bibliometric databases, respectively, and altmetric attention is reported for each article. Researchers may freely access Altmetric.com’s data for bibliometric research purposes,⁴² and subscribers to Elsevier may access PlumX data. The most pertinent differences between altmetric aggregators are the sources and types of information they include and their coverage of academic outputs. Analysts should consider these differences (e.g.,^41,43) when selecting an aggregator to ensure the resultant indicators are valid and reliable for their purposes.

3.4. The potential role of altmetrics in assessing TCIM research impact

Altmetrics could play a pivotal role in evaluating the impact of TCIM research by providing insights into its broader societal engagement and relevance. In TCIM, where diverse healthcare practices and cultural traditions intersect, altmetrics offer a valuable tool for capturing the multifaceted impact of research outputs beyond traditional bibliometric measures, as has been previously shown.⁶

One key role of altmetrics in assessing TCIM research impact is their ability to capture near-real-time engagement and dissemination across digital platforms.⁴⁴ This speed enables nearly immediate insights into the visibility and uptake of TCIM research, enabling researchers to identify emerging trends, measure the effectiveness of knowledge translation efforts, and engage with broader audiences.

Moreover, altmetrics enable researchers and stakeholders to track the uptake and utilization of TCIM research outputs in diverse contexts and by diverse stakeholders. By analyzing altmetrics, researchers can assess the practical relevance and accessibility of TCIM publications, identifying areas of high interest and demand within the healthcare community. Online bookmarks offer insights into the long-term citation potential of TCIM research,⁴⁰ highlighting articles or resources that resonate with researchers and practitioners.

Altmetrics also facilitate interdisciplinary collaboration and knowledge dissemination within the TCIM community. By tracking social media mentions and online discussions, researchers can identify influential stakeholders, fostering dialogue and collaboration across different disciplines and cultural contexts. Altmetrics serve as a catalyst for interdisciplinary exchange, promoting the integration of diverse perspectives and expertise in TCIM research and practice.

Altmetrics complement traditional bibliometric measures by offering a more holistic assessment of TCIM research impact. Altmetrics provide insights into the public perception and reception of research findings via broader societal engagement, highlighting their relevance and applicability beyond academic circles. As such, by integrating altmetrics into research evaluation frameworks, stakeholders gain a comprehensive understanding of the significance and relevance of TCIM research in addressing contemporary healthcare challenges and advancing evidence-based practice.

3.5. Limitations of altmetrics

Altmetrics have distinct strengths for assessing research impact, yet they also have limitations that analysts must consider. Altmetrics rely on data from online sources, which may be subject to biases and limitations. For instance, platforms like X and Facebook may not be representative of all demographic groups or cultural contexts, leading to skewed perceptions of research impact. Similarly, while altmetrics may be well-suited for assessing the impact of research outputs in fields with high public relevance or interest, such as healthcare and environmental science, they may be less informative in disciplines with smaller or more specialized audiences, such as mathematics or physics (e.g.,⁴⁵). In the field of critical care medicine it was found that despite the existence of a positive correlation between traditional bibliographic metrics and altmetrics, no connection was observed between the Altmetric Attention Score (Altmetric.com’s aggregate measure of altmetric attention) and future citation counts.⁴⁶ The suitability of altmetrics in measuring engagement in a particular field must therefore be considered.

Further, altmetrics provide quantitative measures of research impact but may lack the contextual information necessary to interpret these metrics accurately. For example, a high number of social media mentions may reflect controversy about research findings, rather than positive engagement.⁹ Similarly, altmetrics are susceptible to gaming by individuals invested in inflating the attention an output receives.⁹ While citations may also be gamed,⁴⁷ altmetrics are more readily influenced due to the ease with which one may generate social media attention in comparison to citations. Researchers must thus interpret altmetrics in conjunction with qualitative assessments to understand their true significance.

In summary, altmetrics offer valuable insights into research impact and engagement beyond traditional bibliometric measures. However, researchers must be aware of the limitations of altmetrics and interpret these metrics cautiously, taking into account contextual factors and disciplinary differences to derive meaningful conclusions about research impact.

4. Challenges and future directions

4.1. Emerging trends and innovations in bibliometric and altmetric methodologies

The field of bibliometrics and altmetrics is witnessing rapid advancements and innovative approaches that are reshaping the way research output is evaluated and measured. These emerging trends hold promise for enhancing the accuracy, granularity, and relevance of bibliometric and altmetric analyses in the context of TCIM research.

One notable trend is the integration of machine learning (ML) algorithms and natural language processing (NLP) techniques into bibliometric and altmetric methodologies. ML algorithms can analyze vast amounts of data to identify patterns, trends, and relationships in scholarly literature. This enables researchers to uncover insights and associations that may not otherwise be apparent. Similarly, NLP techniques can extract semantic meaning from text data, enabling researchers to analyze and interpret the content of research publications, social media discussions, and other online sources in more nuanced ways. For instance, ML and NLP have been used to understand the context of citations (e.g.,⁴⁸), which is crucial for determining the underlying cause of articles’ impact.⁴⁹ Advancements in ML and NLP thus offer many opportunities for analyzing large-scale bibliometric data to provide greater insight.

Additionally, innovations in data visualization and interactive dashboards are transforming the way bibliometric and altmetric data are presented and communicated to stakeholders. Advanced visualization techniques enable researchers to convey complex analyses in intuitive and accessible formats, enhancing the usability and interpretability of research assessments. Interactive dashboards allow users to explore bibliometric and altmetric data dynamically, enabling them to uncover insights and trends that may not be apparent through static reports or presentations.

In summary, emerging trends and innovations in bibliometric and altmetric methodologies offer exciting opportunities to advance research impact assessment in TCIM and beyond. By leveraging ML and NLP and data visualization techniques, researchers can gain deeper insights into the dissemination, visibility, and societal impact of TCIM research, ultimately contributing to evidence-based healthcare practices and improved patient outcomes.

4.2. Potential impact of bibliometrics and altmetrics on TCIM policy and practice

Bibliometrics and altmetrics have the potential to exert a profound influence on policy development, clinical practice, and healthcare delivery in the realm of TCIM. These metrics offer valuable insights into the dissemination, visibility, and societal impact of TCIM research. When combined with qualitative assessments, these indicators can thereby inform evidence-based decision-making and shaping healthcare policies and practices.

One potential impact of bibliometrics and altmetrics on TCIM policy is their ability to inform funding priorities and resource allocation. By responsibly quantifying research impact and productivity (e.g.,⁵⁰), these metrics assist policymakers and funding agencies in identifying areas of research excellence, emerging trends, and knowledge gaps in the TCIM field.⁵¹ This enables targeted investments in research initiatives that demonstrate high visibility, scholarly influence, and potential for societal impact, ultimately advancing scientific inquiry and innovation in TCIM.

Moreover, bibliometrics and altmetrics may contribute to the development of evidence-based clinical guidelines and best practices in TCIM.⁵² By analyzing citation patterns, publication trends, and social media discussions, researchers can identify seminal works, influential authors, and emerging research topics within TCIM. This knowledge can inform the development of clinical guidelines (e.g.,⁵³) that integrate evidence-based TCIM interventions with conventional medical practices, promoting safe, effective, and patient-centered care.

Additionally, bibliometrics and altmetrics play a crucial role in enhancing healthcare quality and patient outcomes in TCIM practice settings. By tracking research impact and societal engagement indicators, healthcare practitioners can stay abreast of the latest developments in TCIM, enabling them to make informed decisions about interventions.

Furthermore, bibliometrics and altmetrics can influence regulatory frameworks and accreditation standards in TCIM practice. By demonstrating the impact and relevance of TCIM research through quantitative metrics, stakeholders can advocate for policy changes that promote the integration of TCIM approaches into mainstream healthcare systems. This includes initiatives to expand insurance coverage for TCIM services, enhance practitioner training and education, and ensure compliance with ethical and safety standards.

In summary, bibliometrics and altmetrics have the potential to drive transformative changes in TCIM policy and practice by providing objective measures of research impact, visibility, and societal engagement. By leveraging these metrics, stakeholders can advocate for evidence-based policies, enhance clinical decision-making, and improve healthcare quality and access in TCIM settings, ultimately advancing integrative approaches to health and wellness.

5. Call to action for researchers, policymakers, and practitioners to leverage these metrics effectively

As the field of TCIM continues to evolve, there are benefits open to researchers, policymakers, and practitioners who harness the power of bibliometrics and altmetrics effectively. To maximize the potential of these metrics, we issue a call to action to all stakeholders involved in TCIM:

Researchers:

• •Embrace a multidimensional approach: Incorporate both bibliometric and altmetric analyses into research evaluations to capture the full spectrum of research impact.
• •Foster interdisciplinary collaborations: Collaborate with colleagues from diverse disciplines and cultural backgrounds to generate holistic insights into TCIM research.
• •Promote transparency and reproducibility: Adhere to best practices to ensure the reliability and validity of bibliometric and altmetric findings by following the guidance provided by the Leiden Manifesto⁵⁴ and the Guidance List for the repOrting of Bibliometric AnaLyses (GLOBAL)55, 56, 57

Policymakers:

• •Utilize evidence-based metrics: Incorporate bibliometric and altmetric data into policy decision-making processes, following the guidance of the Declaration on Research Assessment⁵⁸ and Coalition for Advancing Research Assessment,⁵⁹ to inform funding priorities, resource allocation, and regulatory frameworks in TCIM.
• •Support research infrastructure: Invest in the development of bibliometric and altmetric databases and tools to enhance the accessibility, fairness, and usability of research impact metrics for policymakers and stakeholders.
• •Foster collaboration and knowledge exchange: Facilitate partnerships between researchers, policymakers, and healthcare practitioners to translate research findings into policy solutions that promote integrative healthcare approaches.

Practitioners:

• •Stay informed: In addition to meta-analyses, systematic reviews, and clinical guidelines, utilize bibliometric and altmetric analyses to stay abreast of the latest research findings and trends in TCIM to inform clinical decision-making and practice.
• •Engage with research communities: Participate in interdisciplinary research networks and knowledge exchange platforms to contribute insights from clinical practice to TCIM research evaluation efforts.
• •Advocate for evidence-based practice: Advocate for the integration of evidence-based TCIM interventions into mainstream healthcare systems, leveraging bibliometric and altmetric data to demonstrate the impact and relevance of TCIM approaches.

By collectively embracing these actions, researchers, policymakers, and practitioners can harness the transformative potential of bibliometrics and altmetrics to advance evidence-based healthcare practices, improve patient outcomes, and promote integrative approaches to health and wellness in diverse cultural contexts.

6. Conclusion

In conclusion, this educational article examined the potential role of bibliometrics and altmetrics in evaluating TCIM research, emphasizing their utility in understanding its impact, visibility, and structure. Bibliometric methods assess research activity and scholarly influence, while altmetrics capture broader societal engagement. We highlighted future opportunities to leverage these metrics for advancing evidence-based healthcare, improving patient outcomes, and promoting integrative approaches.

CRediT authorship contribution statement

Jeremy Y. Ng: Conceptualization, Investigation, Methodology, Project administration, Writing – original draft, Writing – review & editing. Dimity Stephen: Investigation, Methodology, Writing – original draft, Writing – review & editing. Jianping Liu: Investigation, Methodology, Writing – review & editing. Thomas Ostermann: Investigation, Methodology, Writing – review & editing. Nicola Robinson: Investigation, Methodology, Writing – review & editing. Holger Cramer: Investigation, Methodology, Writing – review & editing.

Declaration of competing interest

JYN and DS are researchers involved in the Guidance List for the repOrting of Bibliometric AnaLyses (GLOBAL) project mentioned in this article. All other authors declare that they have no competing interests.