Skip to main content
PMC home page
Journal of the American Medical Informatics Association : JAMIA logoLink to Journal of the American Medical Informatics Association : JAMIA
. 2013 Dec;20(e2):e198–e205. doi: 10.1136/amiajnl-2013-002429

Recent trends in biomedical informatics: a study based on JAMIA articles

Xiaoqian Jiang 1, Krystal Tse 1, Shuang Wang 1, Son Doan 1, Hyeoneui Kim 1, Lucila Ohno-Machado 1
PMCID: PMC3861936  PMID: 24214018

Abstract

In a growing interdisciplinary field like biomedical informatics, information dissemination and citation trends are changing rapidly due to many factors. To understand these factors better, we analyzed the evolution of the number of articles per major biomedical informatics topic, download/online view frequencies, and citation patterns (using Web of Science) for articles published from 2009 to 2012 in JAMIA. The number of articles published in JAMIA increased significantly from 2009 to 2012, and there were some topic differences in the last 4 years. Medical Record Systems, Algorithms, and Methods are topic categories that are growing fast in several publications. We observed a significant correlation between download frequencies and the number of citations per month since publication for a given article. Earlier free availability of articles to non-subscribers was associated with a higher number of downloads and showed a trend towards a higher number of citations. This trend will need to be verified as more data accumulate in coming years.

Introduction

As the field of biomedical informatics continues to evolve and grow into subspecialties, it is important to track the literature generated in the field and to understand which topics attract the most attention by readers and by authors who use and cite particular articles in their own work. Universities are increasingly making use of bibliometrics to evaluate academic progress, and some already require that candidates include the total number of citations to their articles, the impact factor for each publication, the author's h-index (or equivalent), in addition to an account of the author's contribution to each article.1 2 Some institutions provide guidance as to which journals should be considered first tiered, second tiered, etc.

Following the pattern of academic institutions that attempt to measure productivity with bibliometrics, some funding agencies have also established guidelines for progress reports in high visibility programs. The measures include quantity and quality of publications, with a proxy for the latter represented in the form of journal impact factors. To respond to this increasing demand for quantification of academic production, web services to track citations and h-indices, such as the Institute for Scientific Information's (ISI) Web of Knowledge,3 Scopus,4 and Google Scholar5 have been created. Some journals also provide the number of article views and/or downloads per month (DPM). Even though, as pointed out by several authors, these measures are imperfect,6–9 they may provide useful clues to define trends in biomedical informatics and to start taking into account how citation in subspecialized fields varies. Although the number of downloads and citations is not the only way to measure information dissemination and scientific impact, they are relatively easy to track.

It is well known that the citation rates of journals from different disciplines vary significantly due to many factors such as the average number of authors, average number of citations in an article, size of the community, and so on.10–12 However, little is known about how citation rates vary within biomedical informatics topics. JAMIA is a generalist journal in biomedical informatics. To understand the publications that readers consider interesting and cite-worthy, we expanded our initial analysis of citation rates by topic and free access status done in late 2011 (which covered only articles published from 2009 to 2010),13 to articles published in 2011 and 2012. The distribution of citations for JAMIA peaks between 3 and 4 years, so our analyses for 2009–11 are reasonably accurate, whereas those for 2012 are still very preliminary. We present here an analysis that shows the yearly rates of publication and average citations for JAMIA articles according to selected MeSH topics in the past 4 years (2009–12) to understand whether different subfields of biomedical informatics have different citation distributions and how they are changing over time. In this analysis, we take into account the fact that the number of articles with free immediate access when published online or in print has increased rapidly in this period due to the option for authors to select an open access model. We report the associations among downloads, number of citations, and free access status.

Materials and methods

In July 2013 we collected the number of yearly citations to JAMIA articles published between 2009 and 2012 from the ISI Web of Science database. We removed non-peer reviewed materials such as highlights, editorials, letters, and American Medical Informatics Association (AMIA) messages, which left us with a total of 564 articles to analyze. These articles described research and applications, brief communications, perspectives, reviews, and case reports. Note that JAMIA had slightly different article category names and several article subcategories before 2011 (for example, model formulation, synthesis of research).

We recorded dates for the online and in-print publication in a regular issue of the journal, or in a special online issue when applicable. We recorded whether the article was open (that is, freely available immediately at online release, before inclusion in a specific journal issue), free (that is, freely available when it was included in a print or online journal issue), or embargoed (that is, freely available to non-subscribers only after 1 year of embargo from the date of publication in an issue of the journal). We also recorded the number of downloads for abstracts, full-text in html, and full-text in PDF from the JAMIA web site (http://www.jamia.org).

We utilized the same methodology first described by Kim et al13 to calculate the number of citations per month (CPM) from the date at which the article became available online, and calculated DPM using the number of full-text online views plus the number of PDF downloads. For articles in which no date of online publication was available, the date of publication in a journal issue was used. We calculated p values for comparisons that were sufficiently powered. With an effect size (Δ) of 0.8 in CPM, the required sample size was 25 (for each group) for a desired statistical power (1-β) of 0.8 at a significance level (α) of 0.05 using a two-sample t-test.

The nine most frequent topic categories, which were also employed in our 2011 article,13 were used to classify the articles. Two authors independently selected the most relevant topic for an article, by retrieving MeSH terms from PubMed, when available, and reading the titles and abstracts. The κ interrater agreement was 0.8916. Another author served as an arbitrator when there was a conflict, selecting a topic from the two proposed topics (there were 75 disagreements). Supplementary appendix A (available online only) shows the types of disagreements. This final selection was then used for the analysis.

Results

The topics in JAMIA are constantly changing. The visual trends for the nine most frequent MeSH topics are plotted in figure 1 to show the changes in the past 4 years. The figure shows the total number of articles in each of the categories (right vertical axis), and the breakdown of the average number of CPM (left vertical axis). A topic for which the block of bars is displayed in the group 201X in the X-axis shows the average number of CPM since publication for articles published each year between 2009 and 201X, with one bar representing each year, as indicated in the legend.

Figure 1.

Figure 1

Open in a new tab

Breakdown of number of accepted articles (line plot, right axis), average number of citations for articles published each year from 2009 to 2012 (bar plot, left axis), grouped by year in which the citation appeared (X-axis). (A) and (B)–(J) show the breakdown of citations for all articles and for articles in nine different categories, respectively. Note that the scale for the right axis, which represents the number of published articles, varies depending on the topic, while the left axis is fixed. ACPA, average number of citations per article; NLP, natural language processing.

Medical Record Systems was the most frequently assigned topic (124/573), while Prevention and Control represented the smallest group (24/573). Medical Informatics included articles related to biomedical and health informatics education, as well as some policy and other articles that could not be well classified into the other categories. As shown in figure 1, Medical Record Systems, Algorithms, and Methods are the fast growing categories in terms of accepted articles, and they also show an increasing trend in citations. Natural Language Processing, Medical Informatics, Information Storage and Retrieval, and Methods have a consistent pattern of increased citations. Topics related to computer science, such as Algorithms and User–Computer Interaction tend to have relatively fewer citations than those related to healthcare.

The correlations between DPM and CPM are significant in all years—that is, 2009 (p<0.01), 2010 (p<0.01), 2011 (p<0.01) and 2012 (p=0.06)—at a 0.1 threshold. These results indicate a consistent trend between downloads and citations in general. The CPM versus DPM plots are illustrated in figure 2, in which we also show the linear regression model for each year.

Figure 2.

Figure 2

Open in a new tab

Downloads per month (DPM) versus citations per month for all articles in the past 4 years. DPM include full-text online views and PDF downloads.

Figure 3 shows the boxplots displaying CPM for open access (freely available on publication online), free (free access on publication in a journal issue), and embargoed articles (for 2009 there were only embargoed articles, and for 2010 there was a small number of free access articles but no open access option).

Figure 3.

Figure 3

Open in a new tab

Boxplots for citations per month displaying citation trends from 2009 to 2012. (A) Boxplots of open access plus free articles and embargoed articles, and p values for comparisons that are powered to detect significant differences (β=0.2). (B) Comparisons among open access (that is, free on online publication), free (that is, free on publication in a journal issue), and embargoed articles for 2011 and 2012. Note that 2009 did not have open access or free articles, and 2010 did not have an open access option. Analyses for 2012 are preliminary given the relatively short time from publication to the time all data were collected in mid-2013.

Figure 3 also shows a number of comparisons based on the free availability of papers from 2010 to 2012. In figure 3A, we combined open and free access articles into one category (O+F), then compared to embargoed articles. O+F articles received significantly higher CPM in 2011 when tested with a two-tailed t-test (p=0.0036). This trend continued in the preliminary analysis for 2012, and the difference was statistically significant (p=0.0410). The free (typically either editor's choice or AMIA-endorsed) articles received higher CPM than the embargoed articles in the year 2011 (analysis not sufficiently powered given only 21 free articles) (see figure 3B). The open access articles in 2012 also had a higher CPM than the embargoed articles in the same year, but the difference was not statistically significant (p=0.3600).

As the embargoed articles are made freely available after 1 year of publication, there could be a stronger effect in citations accrued in the short term (that is, in the same year of publication and the subsequent year). We thus compared the mean number of citations (for two consecutive years including the year of publication) for freely available and embargoed articles published in 2010 and 2011 (see figure 4). For 2012, there was incomplete information as the total number of citations for 2013 was not yet available. Results show that this difference was significant for 2011 (p=0.0034), the only year for which the test had sufficient power.

Figure 4.

Figure 4

Open in a new tab

Citations (in two consecutive years including year of publication) for articles published in 2010 and 2011. The difference between freely available (free in 2010 or open access plus free in 2011) and embargoed articles was evaluated using a two-sample t-test. The test was not sufficiently powered for comparisons of 2010 articles.

Table 1 lists the most cited 2009–12 papers for two consecutive years starting with the one in which they were published in a journal issue and gives an idea of topics and types of articles that attracted most short-term citations. Table 2 lists the most downloaded papers for the entire period.

Table 1.

Most cited articles when considering two consecutive years including the year of publication in a journal issue

Rank* Title Citations Type
Most cited 2009 articles in 2009–10
1 A randomized trial comparing telemedicine case management with usual care in older, ethnically diverse, medically underserved patients with diabetes mellitus: 5-year results of the ideatel study14 12 Research paper
2 Tiering drug–drug interaction alerts by severity increases compliance rates15 12 Viewpoint paper
3 What evidence supports the use of computerized alerts and prompts to improve clinicians’ prescribing behavior?16 11 Review paper
4 Active computerized pharmacovigilance using natural language processing, statistics, and electronic health records: a feasibility study17 10 Research paper
5 Does computerized provider order entry reduce prescribing errors for hospital inpatients? a systematic review18 9 Review paper
6 Recognizing obesity and comorbidities in sparse data19 9 Viewpoint paper
7 Core content for the subspecialty of clinical informatics20 8 AMIA board white paper
8 Community-wide implementation of health information technology: the Massachusetts ehealth collaborative experience21 8 Case report
9 Program requirements for fellowship education in the subspecialty of clinical informatics22 7 AMIA board white paper
10 Electronic support for public health: validated case finding and reporting for notifiable diseases using electronic medical data23 7 Viewpoint paper
Most cited 2010 articles in 2010–11
1 Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications24 17 Application of information technology
1 Extracting medication information from clinical text25 17 Viewpoint paper (FREE)
3 An overview of MetaMap: historical perspective and recent advances26 17 Synthesis of research
4 MedEx: a medication information extraction system for clinical narratives27 17 Application of information technology
5 Leveraging informatics for genetic studies: use of the electronic medical record to enable a genome-wide association study of peripheral arterial disease28 15 Research paper
6 Health information technology: fallacies and sober realities29 14 Viewpoint paper
7 Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2)30 14 Model formulation (FREE)
7 Evaluating re-identification risks with respect to the HIPAA privacy rule31 14 Research paper
9 The impact of computerized provider order entry on medication errors in a multispecialty group practice32 12 Research paper (FREE)
10 Evaluation of a generalizable approach to clinical information retrieval using the automated retrieval console (ARC)33 9 Application of information technology
Most cited 2011 articles in 2011–12
1 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text34 23 Perspective
2 Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings35 20 Review (FREE)
3 Factors motivating and affecting health information exchange usage36 15 Research and applications
4 Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 Health Policy Meeting37 15 Perspective
5 Translational bioinformatics: linking knowledge across biological and clinical realms38 13 Perspective (OPEN)
6 Data from clinical notes: a perspective on the tension between structure and flexible documentation39 13 Perspective
7 Anaphoric relations in the clinical narrative: corpus creation40 11 Research and applications
8 Social disparities in internet patient portal use in diabetes: evidence that the digital divide extends beyond access41 11 Case report (FREE)
9 Ability of pharmacy clinical decision-support software to alert users about clinically important drug–drug interactions42 10 Research and applications
10 Factors influencing alert acceptance: a novel approach for predicting the success of clinical decision support43 10 Research and applications
Most cited 2012 articles in 2012 to June 2013
1 Use of diverse electronic medical record systems to identify genetic risk for type 2 diabetes within a genome-wide association study44 15 Research and applications
2 iDASH: integrating data for analysis, anonymization, and sharing45 10 Brief communication
3 A novel signal detection algorithm for identifying hidden drug–drug interactions in adverse event reports46 10 Research and applications (FREE)
4 The National Center for Biomedical Ontology47 9 Brief communication
5 The dangerous decade48 7 Perspective
6 The financial impact of health information exchange on emergency department care49 7 Research and applications (OPEN)
7 High-priority drug–drug interactions for use in electronic health records50 7 Research and applications
8 A translational engine at the national scale: informatics for integrating biology and the bedside51 6 Brief communication
9 A systematic review of the psychological literature on interruption and its patient safety implications52 6 Review
10 Portability of an algorithm to identify rheumatoid arthritis in electronic health records53 6 Research and applications (FREE)
Open in a new tab

*Ties are broken by date of publication, with most recent articles ranking higher.

Table 2.

Most downloads* from 2009 to 2012

Rank Title PDF downloads plus full-text online views Article type (availability) Year published in a journal issue
1 Transmitting and processing electronic prescriptions: experiences of physician practices and pharmacies54 10218 Research and applications (OPEN) 2012
2 Definition of biomedical informatics and specification of core competencies for graduate education in the discipline55 9166 AMIA Board white paper (FREE) 2012
3 The impact of computerized provider order entry on medication errors in a multispecialty group practice32 8985 Research paper (FREE) 2010
4 The financial impact of health information exchange on emergency department care49 7397 Research and applications (OPEN) 2012
5 The SMART Platform: early experience enabling substitutable applications for electronic health records56 7177 Research and applications (OPEN) 2012
6 The impact of the electronic medical record on structure, process, and outcomes within primary care: a systematic review of the evidence57 6660 Review (FREE) 2011
7 Shifts in the architecture of the Nationwide Health Information Network58 6621 Perspective (OPEN) 2012
8 The inadvertent disclosure of personal health information through peer-to-peer file sharing programs59 6400 Research paper 2010
9 Health information technology: fallacies and sober realities29 5734 Viewpoint paper 2010
10 Health-information exchange: why are we doing it, and what are we doing?60 5591 Perspective (FREE) 2011
Open in a new tab

*Note that the total number of downloads is calculated as the sum of the full text access and PDF downloads, and does not include abstract access.

Discussion

JAMIA has published several special focus issues in the past 2 years, and the scope of the journal was expanded to include translational bioinformatics,38 61–74 economic evaluations of health information technology,49 75–80 and brief communications of large national initiatives for sharing of tools and data.45 47 51 81–86 It is too early to know whether these topics will result in higher, the same, or lower numbers of citations than other topics. Some topics appear to have an increasing number of citations in the past few years, and the distribution of articles across the topics also appears to be changing.

There are several limitations in the topic analysis. First, only one topic was allowed per article. We could have chosen two or more per article. Fixing the number of topics per article was considered to be important so an article that was assigned several topics would not have a greater influence in the citation count than an article that was assigned fewer topics. By assigning just the main topic to an article, the arbitration was also much simpler, but may also have introduced bias.

The patterns of citations for different topics vary considerably. For a small but highly varied field such as biomedical informatics, in which some researchers are doing research more closely aligned with computer science than medicine, some form of weighting should be developed to account for the fact that the average number of citations to a computer science topic is smaller than the average number of citations to a health science topic.87

The analyses displayed for 2012 are very preliminary, as the number of CPM is still expected to increase. Citations were collected mid-year in 2013, and are significantly lower than the total number of citations expected in 2013. The number of downloads may serve as a proxy for upcoming citations in these years. However, this proxy has limitations. Although correlations between DPM and CPM are all statistically significant, it is easy to see in figure 2 that some highly downloaded articles have relatively few citations and vice versa. It is possible that some articles appeal to many readers, but only some of these appeal both to readers and authors. That some articles are highly cited but have relatively few downloads is harder to explain, but it is possible that authors are citing articles after only reading their abstracts online, or that they are using a paper copy of the journal. More research is needed to determine patterns of articles for which DPM and CPM are not correlated.

An important trend towards an increased number of online full-text views and downloads was observed. Free downloads and online views were not always associated with increased citations, but may be important for the dissemination of biomedical informatics to journal non-subscribers. Some influential articles may not result in a large number of citations, but may have a significant impact on the dissemination of knowledge into clinical practice and public policy.88 It is possible that this is happening with some JAMIA articles. The number of downloads is very high for recent free or open access articles. However, it is too early to assess the impact of these articles in clinical practice and health policy. In addition, the assessment of download frequencies can be biased (for example, it is possible to generate a large number of downloads to any particular article by developing automated programs to access those articles) and hence it is difficult to use download frequency in isolation as a measure of academic productivity and impact. In addition, we do not report here on PubMed Central download frequencies, which are likely to constitute a large portion of downloads.89 It is interesting to note that freely available 2012 articles dominate the list of most downloaded articles (that is, five were from 2012, and eight were freely available on publication in a journal issue). We will follow the trend of citations for those articles in the upcoming years to see whether they result in an overall higher number of citations than articles that were not freely available on publication. As, according to Davis,90 social stratification may result in most citations originating from authors who are based in institutions that have access to subscription journals, it is possible that citation rates will not be associated with free access status.

The analyses presented here show trends in the field that emphasize association but not causation. They have important limitations. There are unavoidable biases in terms of the analyses involving free articles. These are typically AMIA communications endorsed by its board of directors, and usually reporting on the results of workshops, or consensus statements, and JAMIA editor's choice articles, which represent embargoed articles that the editor considered important to be freely disseminated in the short term. Therefore, it could be the case that this designation was the main driver for higher exposure and higher citation rates. In addition, many of these articles had corresponding freely available webinars that attracted thousands of viewers,91 and some had news media or social media coverage.92 Similarly, some authors may have elected the open access option depending on the importance they placed on short-term dissemination. This could result in ‘self-selection’ of the authors’ best work, which might have been more highly cited anyway. In a retrospective study like this, it is impossible to eliminate these potential biases, but they do not invalidate the observation that freely accessible articles generate a larger number of online full-text views and downloads, and that they have a tendency to be more cited.

Conclusion

The rates of publication and citation vary within biomedical informatics topics. Our analyses showed that, for the JAMIA articles studied, the average monthly number of online full-text views plus article downloads (DPM) was highly correlated with the average number of CPM, and hence it may potentially serve as a short-term surrogate for the latter. We also showed an overall tendency for a higher number of citations to JAMIA articles that are freely available at the time of publication. Citation differences between free and embargoed articles were significantly higher for the former in 2011 (p=0.0096).

Supplementary Material

Web appendix

Acknowledgments

The authors would like to thank Betsy Humphreys and Enrico Coiera for suggestions and edits that resulted in an improved manuscript. They appreciated the helpful discussions with members from the Division of Biomedical Informatics (DBMI) at UC San Diego, and in particular they would like to thank Michael Walker and Jihoon Kim for guidance on the selection of statistical tests.

Footnotes

Contributors: XJ conducted most of the analysis, assigned a topic to each article, produced most figures, and wrote portions of the text. KT collected the citation data, assigned a topic for each article, performed preliminary data analysis and wrote portions of this manuscript. SW assisted with data analyses and figures. SD assisted with data collection and topic assignment. HK participated in topic assignment and methods, as well as selection of references. LO-M designed and provided oversight for this study, and wrote major portions of the manuscript. All authors participated in editing the manuscript.

Funding: This work was funded in part by NIH grant U54HL108460.

Competing interests: None.

Provenance and peer review: Commissioned; internally peer reviewed.

Data sharing statement: Some data are not available due to license terms from Thomson Reuters ISI.

References

  • 1.Feinberg School of Medicine (Northwestern University) Information Guide for Appointments, Promotion, and Tenure. http://www.feinberg.northwestern.edu/faculty-staff/fao/docs/faculty/Information-Guide-for-APT.pdf (accessed 23 Sep 2013)
  • 2.Dennis AR, Valacich JS, Fuller MA, et al. Research standards for promotion and tenure in information systems. Mis Quarterly 2006;30:1–12 [Google Scholar]
  • 3.ISI Web of Knowledge. http://wokinfo.com/ (accessed 23 Sep 2013)
  • 4. Scopus. http://www.scopus.com/home.url (accessed 23 Sep 2013) [Google Scholar]
  • 5. Google scholar. http://scholar.google.com (accessed 23 Sep 2013) [Google Scholar]
  • 6.Seglen PO. Citation rates and journal impact factors are not suitable for evaluation of research. Acta Orthopaedica 1998;69:224–29 [DOI] [PubMed] [Google Scholar]
  • 7.Austin A. Reliability of citation counts in judgments on promotion, tenure, and status. Arizona Law Review 1993;35:829–40 [Google Scholar]
  • 8.Bollen J, Van de Sompel H, Smith JA, et al. Toward alternative metrics of journal impact: a comparison of download and citation data. Inf Process Manag 2005;41:1419–40 [Google Scholar]
  • 9.Jeang K-T. Impact factor, H index, peer comparisons, and Retrovirology: is it time to individualize citation metrics? Retrovirology 2007;4:42. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Bhandari M, Busse J, Devereaux P, et al. Factors associated with citation rates in the orthopedic literature. Can J Surg 2007;50:119–23 [PMC free article] [PubMed] [Google Scholar]
  • 11.MacCallum CJ, Parthasarathy H. Open access increases citation rate. PLoS Biol 2006;4:e176. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Filion KB, Pless IB. Factors related to the frequency of citation of epidemiologic publications. Epidemiol Perspect innov 2008;5:3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Kim HE, Jiang X, Kim J, et al. Trends in biomedical informatics: most cited topics from recent years. J Am Med Inform Assoc 2011;18(Suppl. 1):i166–70 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Shea S, Weinstock RS, Teresi JA, et al. A randomized trial comparing telemedicine case management with usual care in older, ethnically diverse, medically underserved patients with diabetes mellitus: 5 year results of the IDEATel study. J Am Med Inform Assoc 2009;16:446–56 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Paterno MD, Maviglia SM, Gorman PN, et al. Tiering drug–drug interaction alerts by severity increases compliance rates. J Am Med Inform Assoc 2009;16:40–6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Schedlbauer A, Prasad V, Mulvaney C, et al. What evidence supports the use of computerized alerts and prompts to improve clinicians’ prescribing behavior? J Am Med Inform Assoc 2009;16:531–8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Wang X, Hripcsak G, Markatou M, et al. Active computerized pharmacovigilance using natural language processing, statistics, and electronic health records: a feasibility study. J Am Med Inform Assoc 2009;16:328–37 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Reckmann MH, Westbrook JI, Koh Y, et al. Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review. J Am Med Inform Assoc 2009;16:613–23 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Uzuner O. Recognizing obesity and comorbidities in sparse data. J Am Med Inform Assoc 2009;16:561–70 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Gardner RM, Overhage JM, Steen EB, et al. Core content for the subspecialty of clinical informatics. J Am Med Inform Assoc 2009;16:153–7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Goroll AH, Simon SR, Tripathi M, et al. Community-wide implementation of health information technology: the Massachusetts eHealth collaborative experience. J Am Med Inform Assoc 2009;16:132–9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Safran C, Shabot MM, Munger BS, et al. Program requirements for fellowship education in the subspecialty of clinical informatics. J Am Med Inform Assoc 2009;16:158–66 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Lazarus R, Klompas M, Campion FX, et al. Electronic support for public health: validated case finding and reporting for notifiable diseases using electronic medical data. J Am Med Inform Assoc 2009;16:18–24 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Savova GK, Masanz JJ, Ogren PV, et al. Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications. J Am Med Inform Assoc 2010;17:507–13 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Uzuner O, Solti I, Cadag E. Extracting medication information from clinical text. J Am Med Inform Assoc 2010;17:514–18 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Aronson AR, Lang FM. An overview of MetaMap: historical perspective and recent advances. J Am Med Inform Assoc 2010;17:229–36 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Xu H, Stenner SP, Doan S, et al. MedEx: a medication information extraction system for clinical narratives. J Am Med Inform Assoc 2010;17:19–24 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Kullo IJ, Fan J, Pathak J, et al. Leveraging informatics for genetic studies: use of the electronic medical record to enable a genome-wide association study of peripheral arterial disease. J Am Med Inform Assoc 2010;17:568–74 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Karsh BT, Weinger MB, Abbott PA, et al. Health information technology: fallacies and sober realities. J Am Med Inform Assoc 2010;17:617–23 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Murphy SN, Weber G, Mendis M, et al. Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2). J Am Med Inform Assoc 2010;17:124–30 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Benitez K, Malin B. Evaluating re-identification risks with respect to the HIPAA privacy rule. J Am Med Inform Assoc 2010;17:169–77 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Devine EB, Hansen RN, Wilson-Norton JL, et al. The impact of computerized provider order entry on medication errors in a multispecialty group practice. J Am Med Inform Assoc 2010;17:78–84 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.D'Avolio LW, Nguyen TM, Farwell WR, et al. Evaluation of a generalizable approach to clinical information retrieval using the automated retrieval console (ARC). J Am Med Inform Assoc 2010;17:375–82 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Uzuner O, South BR, Shen S, et al. 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text. J Am Med Inform Assoc 2011;18:552–6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Jaspers MW, Smeulers M, Vermeulen H, et al. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc 2011;18:327–34 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Vest JR, Zhao H, Jasperson J, et al. Factors motivating and affecting health information exchange usage. J Am Med Inform Assoc 2011;18:143–9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Bloomrosen M, Starren J, Lorenzi NM, et al. Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 Health Policy Meeting. J Am Med Inform Assoc 2011;18:82–90 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Sarkar IN, Butte AJ, Lussier YA, et al. Translational bioinformatics: linking knowledge across biological and clinical realms. J Am Med Inform Assoc 2011;18:354–7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Rosenbloom ST, Denny JC, Xu H, et al. Data from clinical notes: a perspective on the tension between structure and flexible documentation. J Am Med Inform Assoc 2011;18:181–6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Savova GK, Chapman WW, Zheng J, et al. Anaphoric relations in the clinical narrative: corpus creation. J Am Med Inform Assoc 2011;18:459–65 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Sarkar U, Karter AJ, Liu JY, et al. Social disparities in internet patient portal use in diabetes: evidence that the digital divide extends beyond access. J Am Med Inform Assoc 2011;18:318–21 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Saverno KR, Hines LE, Warholak TL, et al. Ability of pharmacy clinical decision-support software to alert users about clinically important drug–drug interactions. J Am Med Inform Assoc 2011;18:32–7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Seidling HM, Phansalkar S, Seger DL, et al. Factors influencing alert acceptance: a novel approach for predicting the success of clinical decision support. J Am Med Inform Assoc 2011;18:479–84 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Kho AN, Hayes MG, Rasmussen-Torvik L, et al. Use of diverse electronic medical record systems to identify genetic risk for type 2 diabetes within a genome-wide association study. J Am Med Inform Assoc 2012;19:212–18 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Ohno-Machado L, Bafna V, Boxwala AA, et al. iDASH: integrating data for analysis, anonymization, and sharing. J Am Med Inform Assoc 2012;19:196–201 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Tatonetti NP, Fernald GH, Altman RB. A novel signal detection algorithm for identifying hidden drug–drug interactions in adverse event reports. J Am Med Inform Assoc 2012;19:79–85 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Musen MA, Noy NF, Shah NH, et al. The National Center for Biomedical Ontology. J Am Med Inform Assoc 2012;19:190–5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Coiera E, Aarts J, Kulikowski C. The dangerous decade. J Am Med Inform Assoc 2012;19:2–5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Frisse ME, Johnson KB, Nian H, et al. The financial impact of health information exchange on emergency department care. J Am Med Inform Assoc 2012;19:328–33 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Phansalkar S, Desai AA, Bell D, et al. High-priority drug–drug interactions for use in electronic health records. J Am Med Inform Assoc 2012;19:735–43 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Kohane IS, Churchill SE, Murphy SN. A translational engine at the national scale: informatics for integrating biology and the bedside. J Am Med Inform Assoc 2012;19:181–5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Li SY, Magrabi F, Coiera E. A systematic review of the psychological literature on interruption and its patient safety implications. J Am Med Inform Assoc 2012;19:6–12 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Carroll RJ, Thompson WK, Eyler AE, et al. Portability of an algorithm to identify rheumatoid arthritis in electronic health records. J Am Med Inform Assoc 2012;19:e162–9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54. doi: 10.1136/amiajnl-2011-000515. Grossman JM, Cross DA, Boukus ER, et al. Transmitting and processing electronic prescriptions: experiences of physician practices and pharmacies. J Am Med Inform Assoc 2012;19:353–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55. doi: 10.1136/amiajnl-2012-001053. Kulikowski CA, Shortliffe EH, Currie LM, et al. AMIA Board white paper: definition of biomedical informatics and specification of core competencies for graduate education in the discipline. J Am Med Inform Assoc 2012;19:931–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56. doi: 10.1136/amiajnl-2011-000622. Mandl KD, Mandel JC, Murphy SN, et al. The SMART Platform: early experience enabling substitutable applications for electronic health records. J Am Med Inform Assoc 2012;19:597–603. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57. doi: 10.1136/amiajnl-2010-000019. Holroyd-Leduc JM, Lorenzetti D, Straus SE, et al. The impact of the electronic medical record on structure, process, and outcomes within primary care: a systematic review of the evidence. J Am Med Inform Assoc 2011;18:732–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58. doi: 10.1136/amiajnl-2011-000442. Lenert L, Sundwall D, Lenert ME. Shifts in the architecture of the Nationwide Health Information Network. J Am Med Inform Assoc 2012;19:498–502. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59. doi: 10.1136/jamia.2009.000232. El Emam K, Neri E, Jonker E, et al. The inadvertent disclosure of personal health information through peer-to-peer file sharing programs. J Am Med Inform Assoc 2010;17:148–58. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60. doi: 10.1136/amiajnl-2010-000021. Kuperman GJ. Health-information exchange: why are we doing it, and what are we doing? J Am Med Inform Assoc 2011;18:678–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Altman RB, Miller KS. 2010 translational bioinformatics year in review. J Am Med Inform Assoc 2011;18:358–66 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Wei W, Visweswaran S, Cooper GF. The application of naive Bayes model averaging to predict Alzheimer's disease from genome-wide data. J Am Med Inform Assoc 2011;18:370–5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Butte AJ, Shah NH. Computationally translating molecular discoveries into tools for medicine: translational bioinformatics articles now featured in JAMIA. J Am Med Inform Assoc 2011;18:352–3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Crockett DK, Lyon E, Williams MS, et al. Utility of gene-specific algorithms for predicting pathogenicity of uncertain gene variants. J Am Med Inform Assoc 2012;19:207–11 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Cantor MN. Translational informatics: an industry perspective. J Am Med Inform Assoc 2012;19:153–5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Chiu SY, Chen LS, Yen AM, et al. Population-based proband-oriented pedigree information system: application to hypertension with population-based screening data (KCIS No. 25). J Am Med Inform Assoc 2012;19:102–10 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Morgan AA, Pyrgos VJ, Nadeau KC, et al. Multiplex meta-analysis of RNA expression to identify genes with variants associated with immune dysfunction. J Am Med Inform Assoc 2012;19:284–8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Sarkar IN. A vector space model approach to identify genetically related diseases. J Am Med Inform Assoc 2012;19:249–54 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.Liu X, Liu ZP, Zhao XM, et al. Identifying disease genes and module biomarkers by differential interactions. J Am Med Inform Assoc 2012;19:241–8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Lussier YA, Stadler WM, Chen JL. Advantages of genomic complexity: bioinformatics opportunities in microRNA cancer signatures. J Am Med Inform Assoc 2012;19:156–60 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Buchanan CC, Torstenson ES, Bush WS, et al. A comparison of cataloged variation between International HapMap Consortium and 1000 Genomes Project data. J Am Med Inform Assoc 2012;19:289–94 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72.Peterson TA, Nehrt NL, Park D, et al. Incorporating molecular and functional context into the analysis and prioritization of human variants associated with cancer. J Am Med Inform Assoc 2012;19:275–83 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.Regan K, Wang K, Doughty E, et al. Translating Mendelian and complex inheritance of Alzheimer's disease genes for predicting unique personal genome variants. J Am Med Inform Assoc 2012;19:306–16 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.Wen Z, Liu ZP, Liu Z, et al. An integrated approach to identify causal network modules of complex diseases with application to colorectal cancer. J Am Med Inform Assoc 2013;20:659–67 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 75.Zlabek JA, Wickus JW, Mathiason MA. Early cost and safety benefits of an inpatient electronic health record. J Am Med Inform Assoc 2011;18:169–72 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 76.O'Reilly D, Tarride JE, Goeree R, et al. The economics of health information technology in medication management: a systematic review of economic evaluations. J Am Med Inform Assoc 2012;19:423–38 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77.O'Reilly D, Holbrook A, Blackhouse G, et al. Cost-effectiveness of a shared computerized decision support system for diabetes linked to electronic medical records. J Am Med Inform Assoc 2012;19:341–5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 78.Subramanian S, Hoover S, Wagner JL, et al. Immediate financial impact of computerized clinical decision support for long-term care residents with renal insufficiency: a case study. J Am Med Inform Assoc 2012;19: 439–42 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 79.Payne TH, Bates DW, Berner ES, et al. Healthcare information technology and economics. J Am Med Inform Assoc 2013;20:212–17 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 80.Sridhar S, Brennan PF, Wright SJ, et al. Optimizing financial effects of HIE: a multi-party linear programming approach. J Am Med Inform Assoc 2012;19:1082–8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 81.Athey BD, Cavalcoli JD, Jagadish HV, et al. The NIH National Center for Integrative Biomedical Informatics (NCIBI). J Am Med Inform Assoc 2012;19:166–70 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 82.Delp SL, Ku JP, Pande VS, et al. Simbios: an NIH national center for physics-based simulation of biological structures. J Am Med Inform Assoc 2012;19:186–9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 83.Kapur T, Pieper S, Whitaker R, et al. The National Alliance for Medical Image Computing, a roadmap initiative to build a free and open source software infrastructure for translational research in medical image analysis. J Am Med Inform Assoc 2012;19:176–80 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 84.Toga AW, Dinov ID, Thompson PM, et al. The Center for Computational Biology: resources, achievements, and challenges. J Am Med Inform Assoc 2012;19:202–6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 85.Floratos A, Honig B, Pe'er D, et al. Using systems and structure biology tools to dissect cellular phenotypes. J Am Med Inform Assoc 2012;19:171–5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 86.Berg JM. National Centers for Biomedical Computing: from the BISTI report to the future. J Am Med Inform Assoc 2012;19:151–2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 87.Radicchi F, Fortunato S, Castellano C. Universality of citation distributions: toward an objective measure of scientific impact. Proc Natl Acad Sci USA 2008;105:17268–72 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 88.Barbour V. How to stir up trouble... while riding a rollercoaster. PLoS Med 2013;10:e1001520. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 89.Davis PM. Public accessibility of biomedical articles from PubMed Central reduces journal readership—retrospective cohort analysis. FASEB J 2013;27:2536–41 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 90.Davis PM. Open access, readership, citations: a randomized controlled trial of scientific journal publishing. FASEB J 2011;25:2129–34 [DOI] [PubMed] [Google Scholar]
  • 91.UCSD DBMI. Journal Club. http://dbmi.ucsd.edu/display/DBMI/UCSD-iDASH+Journal+Club
  • 92.Thelwall M, Haustein S, Lariviere V, et al. Do altmetrics work? Twitter and ten other social web services. PLoS ONE 2013;8:e64841. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Web appendix
amiajnl-2013-002429-s1.pdf (24.2KB, pdf)

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of Oxford University Press

RESOURCES