We would like to respond to the article entitled: “Validity and reliability of a systematic database search strategy to identify publications resulting from pharmacy residency research projects.”1
We agree with the authors’ novel approach to confirm publication via a survey and to identify resident publications via a systematic approach to validate the utility of the database. The authors suggest that this approach can be used to compare publication success between residency programs; additionally, this information can be used for marketing purposes for the individual residency program.2 One limitation noted by the authors was the use of only PubMed and EMBASE as the search engines used to identify resident publications in the study and suggest other search engines, such as International Pharmaceutical Abstracts (IPA) and Google Scholar, may be used as alternatives.1
We aim to describe the methods and results of our project in which we assessed the utility of five different search engines to identify resident publications. Briefly, we identified the publication rate of abstracts presented at the 2003 (n=171), 2005 (n=221), and 2007 (n=273) Great Lakes Pharmacy Resident Conference (GLPRC) was 11.4% (n=76).3 We used a similar search strategy to Kwak et al1 to identify resident publication using the search engines 1) Scopus, 2) IPA, and 3) MEDLINE (PubMed).
For this project, we used information from the abstracts, which were previously confirmed for publications (n=76), to identify the proportion of these publications found in each of the following search engines: 1) Scopus, 2) IPA, 3) MEDLINE (PubMed), 4) EMBASE, and 5) Google Scholar. Two investigators identified published research projects (SMV, PMS) and any disagreement between investigators was reviewed for consensus. A kappa statistic was calculated to determine inter-rater reliability for each search engine. The proportion of published research was compared for each of the five search engines; additionally, all pairs of search engines were assessed to determine the proportion published using combinations of searches.
The proportions of published works were identified in the following search engines: Scopus (90.8%), IPA (75%), MEDLINE (PubMed) (80.3%), EMBASE (88.2%), and Google Scholar (90.8%) (Table 1). Additionally, a higher proportion of non-pharmacy journals were identified using the search strategy compared to pharmacy journals (Table 1) in each individual search engine. After assessing various combinations, only Scopus plus IPA or Google Scholar plus IPA identified all 76 publications (Table 2).
Table 1.
Proportion of Publications Identified per Search Engine
| Search Engine: n(%) |
Total Identified (n=76) |
Kappa Statistic | Pharmacy Journal (n=63) |
Non-Pharmacy Journal (n=13) |
|---|---|---|---|---|
| Scopus | 69 (90.8) | 0.916 | 56 (88.9) | 13 (100) |
| IPA | 57 (75.0) | 0.581 | 56 (88.9) | 12 (92.3) |
| MEDLINE (PubMed) | 61 (80.3) | 0.921 | 49 (77.8) | 12 (92.3) |
| EMBASE | 67 (88.2) | 0.647 | 55 (87.3) | 12 (92.3) |
| Google Scholar | 69 (90.8) | 0.801 | 56 (88.9) | 13 (100) |
IPA = International Pharmaceutical Abstracts
Table 2.
Proportion of Publications Identified per Search Engine Pair
| n(%) | Scopus | IPA | MEDLINE (PubMed) |
EMBASE | Google Scholar |
|---|---|---|---|---|---|
| Scopus | --- | 76 (100) | 69 (90.8) | 69 (90.8) | 69 (90.8) |
| IPA | --- | 74 (97.4) | 75 (98.7) | 76 (100) | |
| MEDLINE (PubMed) | --- | 67 (88.2) | 69 (90.7) | ||
| EMBASE | --- | 69 (90.7) | |||
| Google Scholar | --- |
IPA = International Pharmaceutical Abstracts
Based on our results, no single search engine was able to identify all publications. All nine potential pairs were assessed. Only the combination use of either Scopus and IPA or Google Scholar and IPA resulted in identifying all 76 publications. The use of MEDLINE (PubMed) and EMBASE resulted in the poorest combination of search engines as it only identified 88.2% (n=67) of publications. Kwak et al used the combination of MEDLINE (PubMed) and EMBASE with great success; however, the utility of this combination may differ in a general population of residency programs.1 Residents of certain geographic areas may submit to journals indexed to different search engines which may have contributed to the differences between our study and Kwak et al, as we assessed abstracts from the Great Lakes region. Additionally, confirmed publications in our previous search strategy used Scopus, IPA, and MEDLINE (PubMed) and not a validated measure which may have influenced our results.
In conclusion, we affirm that a validated measure approach, as used by Kwak et al, is the gold standard for confirming the utility of various search engines; however, it may be difficult to replicate in a larger, generalized population of residents.1 The use of a single search engine is likely not adequate to identify all published residents’ studies. Regional differences may exist in the use of search engines, as the combination of MEDLINE (PubMed) and EMBASE resulted in a high sensitivity in Kwak et al;1 however, use of either Scopus and IPA or Google Scholar and IPA resulted in identifying all publications previously identified in GLPRC abstracts.
Acknowledgments
Dr. Vouri is a research consultant with Avanir Pharmaceuticals, Inc. This study was supported by Washington University Institute of Clinical and Translational Sciences grant UL1 TR000448 from the National Center for Advancing Translational Sciences.
Footnotes
Dr. Stranges has no conflicts of interest to disclose.
Ms. Nissen has no conflicts of interest to disclose.
Contributor Information
Scott Martin Vouri, St. Louis College of Pharmacy, 4588 Parkview Place, St. Louis, MO 63104, scott.vouri@stlcop.edu.
Paul M. Stranges, University of Illinois at Chicago, Department of Pharmacy Practice (MC 886), College of Pharmacy, 833 South Wood Street, Chicago, IL 60612, pmstrang@uic.edu.
Jill Nissen, St. Louis College of Pharmacy, 4588 Parkview Place, St. Louis, MO 63104, jill.nissen@stlcop.edu.
REFERENCES
- 1.Kwak N, Swan JT, Thompson-Moore N, Liebl MG. Validity and Reliability of a Systematic Database Search Strategy to Identify Publications Resulting From Pharmacy Residency Research Projects. J Pharm Pract. 2015 doi: 10.1177/0897190014566316. [DOI] [PubMed] [Google Scholar]
- 2.Vouri SM, Stranges PM, Burke JM, Micek S, Pitlick MK, Wenger P. The importance of research during pharmacy residency training. Curr Pharm Teach Learn. 2015;7(6) doi: 10.1016/j.cptl.2015.08.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Stranges PM, Vouri SM, Bergfeld F, Crain M, Jindal N, Landrum M, Lindauer S, Mueller Z, Reich A, Thomas L. Pharmacy resident publication success: factors of success based on abstracts from a regional meeting. Pharmacotherapy. 2014;34(10):e192. [Google Scholar]