Social Media Influence Does Not Reflect Scholarly or Clinical Activity in Real Life

Abstract

Background

Social media has become a major source of communication in medicine. We aimed to understand the relationship between physicians’ social media influence and their scholarly and clinical activity.

Methods

We identified attending, US electrophysiologists on Twitter. We compared physician Twitter activity to (a) scholarly publication record (h-index) and (b) clinical volume according to CMS. The ratio of observed vs. expected Twitter followers (obs/exp) was calculated based on each scholarly (K-index) and clinical activity.

Results

We identified 284 physicians, with mean Twitter age of 5.0 (SD 3.1) years and median 568 followers (25^th, 75^th: 195, 1146). They had a median 34.5 peer-reviewed papers (25^th, 75^th: 14, 105), 401 citations (25^th, 75^th: 102, 1677), and h-index 9 (25^th, 75^th: 4, 19.8). The median K-index was 0.4 (25^th, 75^th: 0.15, 1.0), ranging 0.0008 – 29.2. The median EP procedures was 77 (25^th, 75^th: 0, 160) and E&M visits 264 (25^th, 75^th: 59, 516) in 2017. The top 1% electrophysiologists for followers accounted for 20% of all followers, 17% of status updates, had a mean h-index of 6 (vs. 15 for others, p=0.3), and accounted for 1% of procedural and E&M volumes. They had a mean K-index of 21 (vs. 0.77 for others, p<0.0001), and clinical obs/exp follower ratio of 17.9 and 18.1 for procedures and E&M (p<0.001 each, vs. others [0.81 for each]).

Conclusions

Electrophysiologists are active on Twitter, with modest influence often representative of scholarly and clinical activity. However, the most influential physicians appear to have relatively modest scholarly and clinical activity.

Graphical Abstract

Introduction

Social media has quickly permeated many aspects of daily life, often with great impact.¹ Science, technology, and medicine are no exception.^{2, 3} Researchers, clinicians, trainees, patients, and the lay public have taken to social media as a tool to gather, disseminate, and discuss relevant topics in health and medicine. Journals actively quantify and promote alt metrics that account for social media impact of articles, such as sharing, liking, and commenting.^{4, 5} One of the most popular platforms is Twitter, where medical users can share content, links, and/or media for relevant scholarly material, promote an upcoming conference, show cases, or discuss a pertinent discovery in an entirely open forum.^{6, 7}

A major advantage of these platforms is easy and real-time, open access: content can be shared immediately and broadly, for community discussion and group problem-solving. Clinicians can post with relevant hashtags and seek input from world experts on complicated diseases, perhaps improving care for patients. Conversely, physicians who historically did not participate in dissemination through academic publications can now share their knowledge and clinical expertise. While this accessibility of the social media platform can be appealing, it can also be a limitation – one cannot easily filter by topic, person, or quality. Users may not easily separate fact from fiction. To date, it has been difficult to identify reliable, expert Twitter clinicians based on scholarly and/or clinical activity, the two primary means of gaining expertise in clinical science.

We aimed to examine a specific subset of medical Twitter users as a microcosm of medicine in general, clinical cardiac electrophysiologists (EPs), in order to understand the relationship between social media impact versus scholarly and clinical activity. The objectives of this analysis were, (1) to describe the real-life characteristics of EPs on Twitter, (2) to examine scholarly productivity compared with Twitter influence, and (3) to examine clinical activity compared with Twitter influence.

Methods

Because of the sensitive nature of the data collected for this study, requests to access the dataset from qualified researchers may be sent to the corresponding author.

User identification and Demographic data

Clinical cardiac EPs on Twitter were identified as follows: we examined users of popular EP-related hashtags (e.g., “#epeeps”), and subsequently reviewed friends and followers of identified EP clinicians; known EPs were also identified and further expanded by reviewing other followers and active interactions among groups of known EPs. In order to define a discrete subset of clinicians, whose scholarly and clinical productivity could be accurately assessed, we restricted the cohort to only US, attending-level physicians (MD degree, or equivalent). Employment with a clinical organization was not required (e.g., physicians working for commercial entities were included). Users were verified as EPs if their primary professional activity was described as electrophysiology and/or they were board-certified in clinical cardiac electrophysiology (see below).

Demographic data was gathered from professional websites (i.e. institutional/practice web profiles), and included institution, institution type, clinical activity, and location. Academic institution type was defined based on a direct and active association with a university, medical school, or relevant training programs. Board certification status was collected by matching names in the American Board of Internal Medicine webpage; however, no certification was required to be included in this cohort or analysis.

Twitter Data

Twitter data was collected via the Twitter public application programming interface (API).⁸ All data available was collected on each user; the primary data elements used in the present analysis included screen name, description, account start date, number of status updates (i.e., “Tweets”), number of followers, number of friends (follows). Twitter user data are updated as of December 16, 2019.

Scholarly Productivity

Analysis of scholarly productivity was limited to users who could be identified in the Web of Science database (Clarivate Analytics), by searching author name. Their identity was further verified using the institution location used during publications. User h-indices, publication numbers, and citation counts were recorded. All Web of Science data were collected between August and November 2019. The h-index was used as the primary metric of scholarly activity, as (1) primarily clinical users were targeted with few basic scientists, (2) it incorporates both publication numbers and citation numbers, as a way of adjusting across types of publications and impact.

Clinical Activity

Clinical activity was determined by identifying users’ unique, individual National Provider Indicator (NPI) number based on full name and location, via the National Plan and Provider Enumeration System (NPPES) database. Only users with an identifiable, individual NPI were included in the analysis of clinical volume; if NPI number was ambiguous (multiple names) or unavailable, they were excluded. User billing information was then derived from the Centers for Medicare and Medicaid Services (CMS) Provider Utilization and Payment Data for the most recent year available (2017). Clinical volume in cardiac electrophysiology was defined by Current Procedural Terminology (CPT) billing codes for electrophysiology studies and ablations (93618, 93619, 93620, 93621, 93622, 93623, 93624, 93613, 93650, 93651, 93653, 93654, 93655, 93656, 93657), as well as permanent cardiac device implants (33206, 33207, 33208, 33212, 33213, 33214, 33215, 33216, 33217, 33218, 33220, 33221, 33222, 33223, 33224, 33225, 33226, 33227, 33228, 33229, 33230, 33231, 33240, 33249). Clinical activity was also assessed using evaluation and management (E&M) encounter codes (99211, 99212, 99213, 99214, 99215, 99201, 99202, 99203, 99204, 99205). However, any records which are derived from 10 or fewer beneficiaries are excluded from the Medicare Provider Utilization and Payment Data (to protect beneficiary privacy), and thus are not included in these counts.

In order to understand the overall, US EP physician population, the NPPES dataset was limited to individual providers, who indicated clinical cardiac electrophysiology as at least one taxonomy (code 207RC0001X), and who reported physician credentials (MD degree, or equivalent).

Statistical Analysis

Categorical variables are summarized as number (percentage), and continuous variables summarized as mean (standard deviation; SD) or median (25^th, 75^th quartiles), where appropriate. Univariate comparisons were performed with Chi-squared for categorical variables and ANOVA for continuous variables. Categorical variables are summarized as number (percentage) and univariate comparisons were performed with Chi-squared. A two-sided p-value of <0.05 was considered significant.

In order to assess relationship between social media influence and scholarly or clinical influence (respectively), for each user, we calculated ratios of observed to expected number of Twitter followers based on (1) scholarly record (h-indices), (2) EP procedure volumes, and (3) evaluation and management volumes, similar to metrics described previously.^{9, 10} In short, a linear regression analysis was performed based on h-index or clinical volume versus number of followers. An expected number of Twitter followers was then calculated based on the regression. The ratio was then calculated as the observed versus expected number of Twitter followers based on h-index (previously described as the Kardashian-index, or K-index),^{9, 10} procedural volumes, or clinic volumes, respectively. Given skew in these parameters, we repeated this analysis using linear regression on the log-transformed values, instead (h-indices, clinical volumes, twitter followers).

All data cleaning and analyses were performed using R (Version 3.6.1), RStudio (Version 1.1.463), with packages specifically geared to such analyses, including RTweet for access to the Twitter public API.^{8, 11–13} Given this was an observational analysis of publicly-available data and did not involve patients, Institutional Review Board approval was not pursued.

Results

A cohort of 284 US, attending-level clinical cardiac EPs on Twitter were identified for analysis (Table 1). For reference, this represents nearly 10% of physicians with electrophysiology taxonomy in the NPPES, individual NPI dataset (n=2952). The overwhelming majority was male (90.5%); the mean age of Twitter accounts was 5.0 years (SD 3.1), and the number of followers ranged from 1–33,755 (median 568; 25^th, 75^th; 195, 1146). The median number of total user status updates (Tweets) was 246 (range: 0–58,491; 25^th, 75^th; 54, 897), with a median of 5.3 updates per 30 days (range 0–578; 25^th, 75^th; 1.3, 16.6). The 284 EPs had a total of 372,009 followers and 497,586 status updates.

Table 1.

Baseline Characteristics

	Overall (n=284)
Male	256 (90.5)
Decade of Initial ABIM Internal Medicine Board Certification
1960s	1 (0.4)
1970s	2 (0.8)
1980s	18 (6.9)
1990s	45 (17.2)
2000s	131 (50.0)
2010s	65 (24.8)
ABIM Board Certified in Clinical Cardiac Electrophysiology	238 (83.8)
Institution Type
Academic	136 (47.9)
Commercial	4 (1.4)
Government	2 (0.7)
Hybrid	11 (3.9)
Private	131 (46.1)
Academic Rank
Instructor	3 (3.1)
Assistant Professor	33 (34.0)
Associate Professor	30 (30.9)
Professor	31 (32.0)
Twitter Profile
Age of Twitter Account, years	5.0 (3.1)
Followers, median (25th, 75th)	568 (195, 1146)
Friends (Follows), median (25th, 75th)	225 (92, 440.2)
Total Status Updates, median (25th, 75th)	246 (54, 897)
Status updates per 30 days on Twitter, median (25th, 75th)	5.3 (1.3, 16.6)

Values are presented as n (%) or mean (standard deviation), unless otherwise noted.

ABIM: American Board of Internal Medicine

Scholarly Activity

Of the total, a subset of 270 EPs on Twitter were also identified in Web of Science. These clinical cardiac EPs had relatively high scholarly productivity (Table 2). The median (25^th, 75^th) number of publications and citations were 35 (14, 105) and 401 (102, 1677), respectively. The median h-index was 9 (25^th, 75^th: 4, 20). The K-index (ratio of observed vs. expected Twitter followers based on scholarly activity [h-index]) ranged from 0.0008–29.2, with a median of 0.4 (25^th, 75^th: 0.15, 1.0; Supplemental Material, Figure S1).

Table 2.

Scholarly and clinical activity of EP physicians on Twitter.

Scholarly Data, n=270
Number of Publications	35 (14, 105)
Number of Citations	401 (102, 1677)
H-Index	9 (4–20)
CMS Clinical Activity (2017), n=273
Number of Procedures	77 (0, 160)
Number of Ablations for Atrial Fibrillation	0 (0, 22)
Number of clinical encounters (evaluation and management)	264 (59, 516)

Values are presented as median (25^th, 75^th), unless otherwise noted.

CMS: Centers for Medicare and Medicaid Services

Clinical Activity

A subset of 273 EPs on Twitter had identifiable, individual NPIs. This group performed a median of 77 (25^th, 75^th: 0, 160) EP procedures, and accounted for a median of 264 (25^th, 75^th: 59, 516) evaluation and management encounters, according to CMS in 2017 (Table 2). Ratios of observed vs. expected followers based on procedures and E&M visits were 0.4 for each (25^th, 75^th: 0.15, 0.9 for each), ranging 0.0007–22.6 and 0.0007–24.6, respectively (Supplemental Material, Figure S2).

Top Social Media Influencers

In order to understand the most influential EPs on social media, we conducted additional analyzes of those in the top 1% for numbers of followers. They accounted for 20% of all followers (73,077/372,009), 17% of all status updates (84,463/497,586), and had a mean h-index of 6 (vs. 15 for others, p=0.3; Figure 1). They accounted for 1% of each procedural (332/31053) and E&M volumes (992/93066; Figure 2; Figure 3). They had a mean K-index of 21 (vs. 0.77 for others, p<0.0001), and obs/exp followers based on clinical volume of 17.9 and 18.1 for procedures and E&M (p<0.001 each, vs. others [0.81 for each]). Histograms of the obs/exp follower ratios, based on h-index and clinical volumes are shown in the Supplemental Material, Figures S3A–C.

Figure 1:

Twitter followers (bars) versus h-indices (dots) for individual EP physicians on Twitter. Arrow: Top 1% for followers.

Figure 2:

Twitter followers (bars) versus clinical volume in CMS 2017 (red: procedures, green: evaluation and management), for individual EP physicians on Twitter. Arrow: Top 1% for followers. CMS: Centers for Medicare and Medicaid Services

Figure 3.

Cumulative numbers of Twitter followers compared with cumulative CMS 2017 EP procedures and E&M visits. CMS: Centers for Medicare and Medicaid Services; E&M: evaluation and management

In exploratory analysis using log-transformed values, this analysis improved the residual errors in linear regression models (Supplemental Material, Figures S4 and S5), however, yielded expected numbers of followers which were difficult to interpret meaningfully. Therefore, these were not used as the primary analysis.

Discussion

In these analyses we are the first to provide a detailed characterization of a clinical subspecialty community on Twitter, through linking with real-life identities, board certification, scholarly productivity, and clinical activity. Using a subspecialty characterized by both medical and procedural interventions provides insight into the use of Twitter across medicine as a whole, and adds substantially to other, commentary-only and survey-based descriptions of clinicians and social media.^{2, 14, 15} There are several major conclusions from these data: (1) we are easily able to identify nearly 10% of NPI-registered EP physicians on Twitter, a substantial proportion of a subspecialty; (2) there is significant variability in social media activity and influence across clinicians; and (3) while the majority of clinicians maintain modest social media, scholarly, and clinical presence, there are outliers where social medial influence and activity appear out of proportion to scholarly and/or clinical activity – two principle means of achieving expertise in clinical science. These findings are important and provide insight into the heterogeneity of physicians that influence social media in medicine and science and may be a microcosm of medicine in general.

The scholarly activity among this group was prolific – the number of papers, number of citations, and distribution of h-indices likely reflects a bias towards Twitter involvement by academically-oriented physicians, and also the subspecialty’s continued pursuit of scientific investigation. With the pace and volume of new research being published, Twitter is an ideal platform for sharing and discussing important scientific discoveries. The limited length, ability to attach media, and forum for public commentary all give users an opportunity to identify and disseminate important findings quickly and discuss them. There are several examples of Twitter communities using these public discussions to correct errors and encourage data reproducibility across fields.¹⁴ We found that overall, social media influence largely correlates with h-index – most EPs had a K-index (observed/expected ratio) of less than 1 (75^th percentile). However, there were some significant outliers with ratios up to 29. These data suggest some of the most active and prolific social media EPs are not those with the greatest scholarly impact (as traditionally measured). These findings have associated benefits and challenges – ideally novel findings, observations, and techniques are distributed broadly across physicians active on social media and can be communicated quickly, yet in a critical manner, in order to positively impact patient care. However, on the other hand, it is not clear that those with the most social media influence are the most active clinicians or scientists, either.

As a group of physicians in academic or private settings, collaboration on challenging clinical cases and/or promulgating contemporary clinical practice could promote better treatment and outcomes for patients, as a form of crowd-sourcing care.^15,16 For example, recent advancements in cardiac pacing within the specialized conduction system were promoted through the hashtag “#dontdisthehis” which was shown to correlate with increased usage of a specific clinical device.² Generally, we found that the social media influence of EPs reflected their clinical volume and experience. However, similar to scholarly activity, there were significant outliers with obs/exp ratio of followers, based on clinical volume, greater than 20 (despite 75% percentile of 0.9 for both procedures and E&M activity). We showed the top 1% of physicians for Twitter followers accounted for a modest amount total EP procedures among CMS patients. Again, these data suggest that EPs with the greatest social media impact are not necessarily those that are most clinically active. And while this discordance may not reflect the quality of contributions from these users, it may influence the interpretation of such interactions by those many followers, including other professionals and lay persons.

Strengths & Limitations

The 10% of EP physicians we identified is a sizeable group of the subspecialty, which provides very valuable insights as we consider EP as a microcosm for medicine in general. In fact, >90% of EPs on Twitter were male and while the most recent HRS survey report does not indicate respondents’ gender, 91% of EP physicians in the NPPES are also male.¹⁶ However, clinicians on Twitter may not appear to be entirely representative of a subspecialty as a whole. Three-quarters of EPs on Twitter received initial internal medicine board certification after 2000, and thus are less than aged 50, whereas the mean age of HRS respondents was 50 – younger physicians are over-represented on Twitter. Another potential limitation in this study was the lack of a central, verified repository of EP physicians – this is a large, dynamic, and minimally-regulated platform. However, our final cohort represents one of the most well-defined clinical subspecialty cohorts from social media.

There are limitations to using Web of Science for assessment of scholarly productivity, as not all citations may be captured, some may be misattributed, and indices such as h-index do not distinguish publication types (original peer-reviewed science versus editorials or other types of publications) nor do they provide insight into publication quality or scientific impact.¹⁷ Using h-index as a surrogate for scholarly productivity may bias results towards researchers with more publications and with publications more likely to be cited (e.g., guidelines statements). Nevertheless, it can provide a broad measure of overall scholarly impact, particularly across a group of scientists likely to publish in similar settings (e.g., clinical research in EP). There are also limitations to using CMS clinical data: (1) the most recent available dataset is 2017, (2) it does not reflect clinical care provided to non-CMS patients (e.g., private insurance, veterans administration, etc.), and (3) there is no indication of clinical complexity of individual cases. However, given the well-described aging of the patient population, particularly within cardiovascular disease, a large proportion of EP patients fall into CMS-qualifying age groups. Lastly, some users included may not have any clinical activity recorded because they are primarily employed in restricted clinical setting (e.g., the Veterans’ Affairs) or outside clinical medicine (e.g., with a commercial entity). Nevertheless, we felt it appropriate to include all clinicians, as the discourse among such users on Twitter, is primarily clinical in nature.

Conclusions

Social media has quickly assimilated into modern day medicine through a variety of routes. In this study we analyzed a group of specialists as a microcosm of medicine in general to understand how they use social media in context of scholarly productivity and clinical volume. We found that for many of these users, Twitter following is roughly correlated with scholarly and/or clinical activity. However, there are significant outliers with large social media influence, yet modest scholarly productivity and/or pedestrian clinical volume. These data should inform our future utilization and interpretation of social media in science and medicine.

What Is Known?

• Social media has become a major source of communication in medicine.

• Much of the discourse among physicians on social media includes clinical cases/challenges and/or interpretation of clinical investigation.

What the Study Adds?

• We identified attending, US electrophysiologists on Twitter, and characterized them according to their social media impact, scholarly publication record (h-index), and clinical volume according to the Centers for Medicare and Medicaid Services.

• We found that large social media following is not particularly reflective of high scholarly or clinical productivity, by the measures used.

• Future social media platforms may benefit from providing additional user characteristics, that may help inform interpretation of investigative or clinical opinions.

Nonstandard Abbreviations and Acronyms

EPs

electrophysiologists

API

application programming interface

NPI

National Provider Indicator

NPPES

National Plan and Provider Enumeration System

CMS

Centers for Medicare and Medicaid Services

CPT

Current Procedural Terminology