Benchmarking Scholarly Activity by Pharmacists Who Are Fellows Within the American College of Critical Care Medicine

OBJECTIVES:

Quantify scholarly activity by pharmacists who are Fellows within the American College of Critical Care Medicine and to develop a potential publication benchmark for fellowship application.

DESIGN:

Review of the Scopus and PubMed online citation databases.

SETTING:

None.

PATIENTS:

None.

INTERVENTIONS:

None.

MEASUREMENTS AND MAIN RESULTS:

Pharmacists designated Fellow of Critical Care Medicine (FCCM) were identified in January 2021 by the Society of Critical Care Medicine. Pharmacists designated Master of Critical Care Medicine (MCCM), without an active license, or were not identified in either online citation database were excluded. Practice setting characteristics were obtained from the American Hospital Association including country, state, geographic region, number of staffed beds, and hospital designation. Two online citation databases (Scopus and PubMed) were queried in February 2021, and year of first publication, total publications, citations, and Hirsch index were recorded. Of the 152 pharmacists designated FCCM, 138 (91%) were evaluable. Reasons for exclusion included MCCM designation (n = 7; 5%), lack of data in either online citation database (n = 4; 3%), and no active pharmacist license (n = 3; 2%). Most pharmacists were practicing in the Southern geographic region of the United States (n = 62; 45%) and at an academic medical center (n = 116; 84%). The median year of FCCM convocation was 2016 (2012–2019) and of the first publication was 2007 (2002–2011). After removing duplicates, 4,488 unique publications were identified. The median number of publications per individual pharmacist was 20 (9–43) with 10 (5–19) between the year of their first publication and FCCM convocation. Most scholarly activity was in the form of original research (n = 3,173; 71%) or reviews (n = 795; 18%). Individual pharmacists have 244 (99–661) citations and an h-index of 8 (5–13).

CONCLUSIONS:

Pharmacists designated FCCM have maintained a high level of scholarly activity. Pharmacists pursuing fellowship may use these data as a benchmark for fulfilling aspects of the core area of scholarly activities related to critical care medicine prior to application.

Pharmacists have participated in the care of critically ill patients since the 1960s (1). Proliferation of ICUs necessitated advanced training for pharmacists and culminated with the establishment of the first Critical Care Pharmacy Residency at The Ohio State University in 1981 (2). The number of American Society of Health-System Pharmacists accredited Critical Care Pharmacy Residencies is now 168 and trails only Ambulatory Care Residencies at 198 (3). Pharmacists were officially recognized within the Society of Critical Care Medicine (SCCM) in 1989 with the formation of the Clinical Pharmacy and Pharmacology (CPP) section (4) but were active within SCCM prior to that date. Pharmacists currently account for 1,600 (10%) of the SCCM’s approximate 16,000 members (5).

Designation as a Fellow of Critical Care Medicine (FCCM) in the SCCM’s American College of Critical Care Medicine (ACCM) is a goal for many pharmacists, as is becoming a Board Certified Critical Care Pharmacist. Acceptance into the College requires significant contributions and/or achievements in three core areas: 1) program development; 2) state, national, and/or international engagement; and 3) scholarly activities related to critical care (5). The SCCM lists nine methods for fulfilling the scholarly activities criterion including peer-reviewed original publications related to critical care (5).

There has been no quantification of scholarly activity by pharmacists designated FCCM. Benchmarking scholarly activity is important because the 2020 multiorganizational Position Paper on Critical Care Pharmacy Services stated pharmacists as key investigators are essential at level I centers (continuous comprehensive critical care) and desirable at level II (conditional comprehensive critical care) and III centers (initial stabilization) (6). Accordingly, the objective of this study was to quantify scholarly activity by pharmacists designated FCCM and to develop a benchmark for prospective applicants.

MATERIALS AND METHODS

Pharmacist Identification

Pharmacists designated FCCM from inception through 2020 convocation year were identified in January 2021 using a list provided by the SCCM. Year of convocation was provided as well. Pharmacists designated Master of Critical Care Medicine (MCCM) were not included because their scholarly productivity inflated the data despite representing a small percentage of the cohort (n = 7; 5%). Pharmacists without an active pharmacist license listed on a State Board of Pharmacy website or those without data in either citation database (Scopus and PubMed) were also excluded.

Online Citation Databases

Scopus is an online, source-neutral citation database established in 2004 by Elsevier (7). The “Authors” search function was used to identify each Fellow. The last and first names were entered into the search field, and results populated. Profiles were reviewed based on author name, affiliation, city, and country. Finally, each author’s citation list was manually reviewed to ensure publication dates, journals, and subject matter were appropriate.

PubMed is an online citation database established in 1996 by the National Center for Biotechnology Information at the U.S. National Library of Medicine at the National Institutes of Health (8). The “Advanced Search Builder” was used to identify each Fellow. The query box term was set to Author, and the Fellow’s name was entered. Publication titles were reviewed until a definitive publication by the Fellow was identified. Next, the publication page in PubMed was opened, and the author’s affiliations were reviewed. Once confirmed the author was correct, the name was selected via hyperlink brining you to their computed author sort order. According to the U.S. National Library of Medicine at the National Institutes of Health, the computed author sort is “…a ranking algorithm when users click the author search link. Because an author may share the same name with other authors, the objective is to display more relevant results by disambiguating common author names” (9). Each author’s citation list was manually reviewed to ensure publication dates, journals, and subject matter were appropriate.

Demographic Data

Current demographic data were collected including country, state and geographic region of practice site, number of staffed beds, and hospital designation (i.e., academic or community). Current practice site was determined using the SCCM Member Directory or an online search. The number of staffed beds and hospital designations was obtained from the American Hospital Association’s 2018 Annual Survey (10). Region within the United States was defined according to the U.S. Census Bureau.

Scholarly Activity Data

Scholarly activity was collected in February 2021 including year of first publication, total number of publications, citations, and Hirsch index (h-index, from Scopus only). The h-index measures scholarly output and performance in a numerical manner. An h-index of 10 means an author has 10 publications with at least 10 citations. Article type was obtained from Scopus’ citation export function and extrapolated to duplicate PubMed publications, when possible. A manual review of publication type was used for PubMed publications not identified in Scopus. Article types included original research, review, letter or note, editorial, conference article (e.g., executive sessions, summits, and consensus meeting proceedings), book or book chapter, or erratum. Published conference abstracts were not included in analyses, and the author could appear any place on the author list.

Statistical Analysis

Continuous variables are reported as median (interquartile range [IQR], 1–3) and proportions as number (%). Data are descriptive, and no statistical testing was conducted. Scopus and PubMed were queried separately, and then, duplicate publications were removed for analyses. Results from Scopus and PubMed are reported as unique data and then aggregated. This was done as a way to validate the findings from a single online citation database in an effort to be as accurate and inclusive as possible. These lists were then combined, and then, duplicates were removed for the aggregate analyses. Results from the aggregate analyses are the final data from which conclusions were drawn.

The number of publications between the first publication and the FCCM convocation year was used to determine a potential benchmark for pharmacists considering fellowship. Publications from the year of convocation were not included because convocation typically occurs in January or February. Pharmacist licensure year as the index date was not possible because many Boards of Pharmacy had limited online data.

RESULTS

Of the 152 pharmacists designated FCCM, 138 (91%) were evaluable. Reasons for exclusion included MCCM designation (n = 7; 5%), lack of data in either online citation database (n = 4; 3%), and no active pharmacist license (n = 3; 2%). Most pharmacists were practicing in the Southern geographic region of the United States (n = 62; 45%) with just four (3%) practicing outside of the United States (Table 1). The majority of pharmacists’ practice site was an academic medical center (n = 116; 84%) with 726 (466–926) staffed beds. The median year of FCCM convocation was 2016 (2012–2019) (Table 2).

TABLE 1.

Demographic Information and Practice Site Characteristics for Pharmacists Designated Fellow of Critical Care Medicine Through 2020

	Aggregate (n = 138)	Scopus (n = 134)	PubMed (n = 137)
Geographic location, n (%)
South (United States)	62 (45)	61 (46)	62 (45)
Midwest (United States)	39 (28)	37 (28)	38 (28)
Northeast (United States)	23 (17)	22 (16)	23 (17)
West (United States)	10 (7)	10 (7)	10 (7)
Canada	2 (1)	2 (1)	2 (1)
Middle East	2 (1)	2 (1)	2 (1)
Hospital type, n (%)a
Academic	116 (84)	113 (84)	116 (84)
Community	10 (7)	9 (7)	9 (7)
Other	12 (8)	12 (9)	12 (8)
Staffed bedsa	726 (466–926)	727 (466–926)	726 (462–932)

Continuous data are reported as median (interquartile range) and proportions as number (%).

^aHospital type and staffed beds were unavailable for some pharmacists working in academia or industry. Data were obtained from the American Hospital Association’s 2018 Annual Survey.

TABLE 2.

Scholarly Activity by Pharmacists Designated Fellow of Critical Care Medicine Through Convocation Year 2020

	Aggregate (n = 138)	Scopus (n = 134)	PubMed (n = 137)
Year of convocation	2016 (2012–2019)	2016 (2012–2019)	2016 (2012–2019)
Year of first publication	2007 (2002–2011)	2007 (2001–2010)	2007 (2002–2012)
Years from first publication to convocation	9 (6–12)	9 (6–12)	8 (6–11)
Publications per pharmacist	20 (9–43)	20 (10–44)	19 (8–41)
Publications per yeara	2 (1–3)	2 (1–3)	1 (1–3)
Publications before convocation	10 (5–19)	11 (6–22)	9 (4–16)
Publication typeb	n = 4,488	n = 3,752	n = 3,377
Original research	3,173 (71)	2,642 (70)	2,518 (75)
Review	795 (18)	675 (18)	562 (17)
Letter	344 (8)	280 (7)	227 (7)
Editorial	89 (2)	77 (2)	54 (2)
Erratum	52 (1)	52 (1)	5 (1)
Book/book chapter	22 (1)	17 (1)	0 (0)
Conference article	13 (1)	9 (1)	11 (1)
Citations per pharmacistc	244 (99–661)	244 (99–661)	NR
Hirsch index	8 (5–13)	8 (5–13)	NR

NR = not reported.

Continuous data are reported as median (interquartile range) and proportions as number (%).

^aCalculated by dividing the total number publications by years since first publication.

^bPercentages may not add up to 100% due to rounding.

^cCitation information is not available in PubMed.

A total of 3,988 publications were identified in PubMed. After removing duplicates (n = 611; 15%), 3,377 publications were evaluable (Table 2). The majority were original research (n = 2,518; 75%), followed by reviews (n = 562; 17%) and letters to the editor (n = 227; 7%). The oldest publication was from 1985 and the most recent in 2021. Using quartiles for publication year, 937 publications occurred between 1985 and 2010 (n = 25 yr; IQR, 1), which increased to 775 publications between 2019 and 2021 (n = 2 yr; IQR, 4). The minimum number of publications was one, and the maximum was 184. Using PubMed data, the median number of publications prior to convocation was 9 (4–16).

A total of 4,436 publications were identified in Scopus. After removing duplicates (n = 684; 15%), 3,752 publications were evaluable (Table 2). The majority were original research (n = 2,642; 70%), followed by reviews (n = 675; 18%) and letters to the editor (n = 280; 7%). The oldest publication was from 1984 and the most recent in 2021. Using quartiles for publication year, 942 publications occurred between 1984 and 2008 (25 yr period), which increased to 763 publications between 2019 and 2021 (2 yr period). The minimum number of publications per individual pharmacist was 1, and the maximum was 198. Using Scopus data, the median number of publications prior to convocation was 11 (6–22).

A total of 7,129 publications from PubMed and Scopus were combined producing an aggregated list. After removing duplicates (n = 2,641; 37%), 4,488 publications were evaluable. The majority were original research (n = 3,173; 71%), followed by reviews (n = 795; 18%) and letters to the editor (n = 344; 8%). The oldest publication was from 1984 and the most recent in 2021. Using quartiles for publication year, 1,128 publications occurred between 1984 and 2008 (24 yr period), which increased to 972 publications between 2019 and 2021 (2 yr period). The minimum number of publications was 1, and the maximum was 198. Using aggregate data, the median number of publications prior to convocation was 10 (5–19).

Pharmacists designated FCCM have published 7,129 times since 1984 without considering duplicate publications between pharmacists. They have collaborated with at least one other pharmacist on 2,641 (37%) publications. When examining citations, the median number was 244 (99–661) per pharmacist, and all pharmacists designated FCCM have amassed 78,897 citations. Finally, the median h-index was 8 (5–13), meaning they have eight publications with at least eight citations.

DISCUSSION

This was the first quantification of scholarly activity by pharmacists designated FCCM within the ACCM. Collectively, pharmacists have produced 7,129 publications resulting in 78,897 citations to date. On an individual level, pharmacists have 20 (9–43) publications with 244 (99–661) citations and an h-index of 8 (5–13). Publication of 10 (5–19) articles prior to fellowship application may be considered a benchmark for aspiring pharmacist fellows. When considering the IQR, this number may fluctuate between 5 and 19 for half of pharmacists.

Some heterogeneity was observed across the citation databases. Three pharmacists could not be identified in Scopus. Minor differences were observed in publications per pharmacist, publications before convocation, and publication types. Aggregating data from two sources likely limited the influence of heterogeneity on the outcomes reported. Other online citation databases were considered, including Google Scholar, Research Gate, and Web of Science, but were not ultimately included. Google Scholar contained data on just 24% of pharmacists, which was lowest among the databases. Web of Science had data on almost all pharmacists (97%), but the number of profiles claimed by authors was just 6%, which is an important component of the database. Research Gate had data on approximately 50% of pharmacists but was not included because it may function as a social media platform aimed at connecting researchers.

These data may be used for benchmarking pharmacist scholarly activity within the SCCM and other societies, application for fellowship, or by the SCCM’s CPP’s Research and Scholarship Committee who may consider tracking such metrics in the future. For example, pharmacist candidates for FCCM may see current FCCM are producing two to three publications per year, and if the SCCM recommends application after 5 years of practice, they may target 10 publications prior to application. Similarly, the CPP Research and Scholarship Committee may target pharmacists practicing internationally, in the Northeast or West, and at community hospitals for their Mentor-Mentee Program, as these groups were less represented.

In 2017, Mayer et al (11) described publication productivity differences between 1,922 male and female urologists using Google Scholar and Scopus. The median h-index was 10.3 (0–112) and was higher for males than for females (11.7 vs 6.3, p < 0.001). Kokulu et al (12) examined emergency medicine physician publications in emergency medicine journals using the Science Citation Index Expanded database. Collectively, emergency medicine physicians had an h-index of 80, and an individual level h-index of 12.8 in a separate publication. Wilkes et al (13) published a bibliometric analysis of 315 neurosurgeons in Great Britain and Ireland. The median h-index was 6 and was higher in professors and those with an additional degree. Finally, Pakpoor et al (14) published a study of gender differences in neurology authorship in three neurology journals. A significant increase in the percentage of female authorship was demonstrated (p < 0.01).

Our data are the first to describe scholarly activity in a select cohort of critical care pharmacists. Pharmacists who are Fellows have a lower h-index than urologists in the United States or emergency medicine physicians in the United States and Canada (8 vs 10.3 vs 12.8) and were most comparable to neurosurgeons from Great Britain and Ireland (8 vs 6). Most prior reports used the h-index to quantify scholarly activity. Our data are more granular and included total citations and publication type. Additionally, we used multiple reputable online citation databases (Scopus and PubMed) that reduced the likelihood of over- or underestimating the results.

This study has limitations. Heterogeneity existed across the databases, and many of them rely on active engagement to ensure publication lists are accurate. Similarly, each database has been scrutinized for their limitations, which are outside the scope of this article. However, the use of two databases enabled us to validate our findings and offer a greater degree of transparency. This report does not describe the quality of publications as measured by changes in thinking or practice. Although challenging to define, surrogates including citations and h-index were used. Examining “publication productivity” is an evolving paradigm with an increase in the number of articles describing it in PubMed from two in 2000 to 28 in 2020. Hospital practice site may not have been the site for all publications, which is a moving variable that was challenging to capture. Total scholarly productivity may be an underestimate because we were unable to capture gray literature or nonindexed publications. Future evaluations should focus on journal impact factor, place on the author list, and pharmacy subspecialty (e.g., medical, surgical, cardiac, cardiothoracic surgery, or neurologic critical care). The repository created for the present study may serve as a foundation for these efforts, which may be done in conjunction with the SCCM and the CPP Section. A manual review of article titles was completed separately by both authors to ensure each publication was appropriate. This may have introduced subjectivity. Finally, the author search function in both online citation databases is limited by an inability to account for name changes, spelling differences, and structure of reporting by various journals.

CONCLUSIONS

Pharmacists designated FCCM have maintained a high level of scholarly activity over the past 40 years. Pharmacists who are candidates for FCCM may consider these data as a benchmark for aspects of the core area of scholarly activities related to critical care.