Summary
Objective: To summarize recent research and to propose a selection of best papers published in 2016 in the field of Clinical Information Systems (CIS).
Method: The query used to retrieve the articles for the CIS section of the 2016 edition of the IMIA Yearbook of Medical Informatics was reused. It again aimed at identifying relevant publications in the field of CIS from PubMed and Web of Science and comprised search terms from the Medical Subject Headings (MeSH) catalog as well as additional free text search terms. The retrieved articles were categorized in a multi-pass review carried out by the two section editors. The final selection of candidate papers was then peer-reviewed by Yearbook editors and external reviewers. Based on the review results, the best papers were then chosen at the selection meeting with the IMIA Yearbook editorial board. Text mining, term co-occurrence mapping, and topic modelling techniques were used to get an overview on the content of the retrieved articles.
Results: The query was carried out in mid-January 2017, yielding a consolidated result set of 2,190 articles published in 921 different journals. Out of them, 14 papers were nominated as candidate best papers and three of them were finally selected as the best papers of the CIS field. The content analysis of the articles revealed the broad spectrum of topics covered by CIS research.
Conclusions: The CIS field is multi-dimensional and complex. It is hard to draw a well-defined outline between CIS and other domains or other sections of the IMIA Yearbook. The trends observed in the previous years are progressing. Clinical information systems are more than just sociotechnical systems for data collection, processing, exchange, presentation, and archiving. They are the backbone of a complex, trans-institutional information logistics process.
Keywords: Medical informatics, International Medical Informatics Association, Yearbook, clinical information systems
Introduction
In 2017, we reused the optimized query that we developed for the Clinical Information Systems (CIS) section of the 2016 edition of the IMIA Yearbook of Medical Informatics. The number of retrieved papers increased by nearly 20 percent as compared to the previous year. The trends that we observed in the previous years are continuing.
About the Paper Selection
The process described in [ 1 ] was again applied to select the candidate papers for the CIS section. Primarily, descriptors from the Medical Subject Headings (MeSH) thesaurus were used as search terms to cover the major topics of the CIS section: hospital records , health information exchange , health and clinical information systems and meaningful use , among others. In addition, as the indexing with MeSH descriptors may occur with a lag of several months after the inclusion of an article in PubMed, free text search terms covering the same topics were used to search within the title and abstract fields of PubMed ([TIAB] operator).
As in the previous years, free-text search was restricted to citations labeled with status “ publisher ” or “ inprocess ”, which signifies the lack of MeSH-indexing. The query was further restricted to journal articles published in 2016, in English language, and including an abstract ( 2016[DP] AND “english”[LA] AND “journal article”[PT] AND “has abstract” ). Topics from other IMIA Yearbook sections were excluded as far as possible with appropriate MeSH and free text terms. Articles retrieved for the 2016 IMIA Yearbook were likewise excluded. Additional exclusions by publication type were added to restrict the query to original research articles and reviews. The free-text query was additionally reformatted for Web of Science® (from Thomson Reuters) with an additional restriction to the subject area “Medical Informatics” and the exclusion of articles which already had been found in PubMed. The full queries are available upon request from the corresponding authors.
The queries were carried out in mid-January 2017. The search result set comprised 2,478 papers. Articles which had been published already in 2015 but showed up in the result set (n=208) were removed. Although classified as journal articles by PubMed, all conference papers published in Studies in Health Technology and Informatics (n=80) were also excluded, yielding a combined result set from PubMed and Web of Science of 2,190 articles. The resulting articles had been published in 921 different journals. Figure 1 depicts the top ten journals with the highest numbers of resulting articles.
Fig. 1.
Number of retrieved articles for Top-10 journals
The query results were loaded into the BibReview software [ 1 ] for a multi-pass review carried out separately by the two section editors (TG, WOH). Due to the high number of retrieved articles, the result set was bisected for the first pass in which ineligible articles were excluded based on their titles and/or abstracts by one section editor. All remaining articles were then assessed by both section editors based on abstracts and/or full-texts as required. The individual results of each section editor were merged after each assessment cycle. Articles which had been selected as possible candidate papers were discussed and added to a list upon mutual acceptance. Articles tagged with “pending” or “conflicting” status were jointly re-assessed in a total of three passes, yielding a list of 16 candidate best papers. This preliminary list was then reviewed by the Yearbook editors who checked if any articles had also been selected for other sections. After discussions, two papers were removed from the CIS candidate paper list because they had been nominated in other sections: a very interesting qualitative study by Cresswell and colleagues on Workarounds to hospital electronic prescribing systems [ 2 ] and an also very interesting contribution by Hravnak and colleagues on artifacts in non-invasive vital sign high-frequency data and implications for mining big data [ 3 ] but artifact in stored non-invasive vital sign (VS (finally this paper was selected as one of the best papers for the Sensor, Signal and Imaging Informatics section).
The 14 remaining papers were then peer-reviewed by Yearbook editors and external reviewers. During the selection meeting held on April 21 st 2017 in Paris, three papers were finally selected as best papers for the CIS section ( Table 1 ). A content summary of these three best CIS papers can be found in the appendix of this synopsis.
Table 1.
Best paper selection of articles for the IMIA Yearbook of Medical Informatics 2017 in the section ‘Clinical Information Systems’. The articles are listed in alphabetical order of the first author’s surname.
| Section Clinical Information Systems |
|---|
|
Findings and Trends: Clinical Information Systems 2016
As in the previous years, we strived to get a comprehensive overview of the content of the retrieved papers and to check if our query was reliable and valid. Last year [ 4 ] we implemented a text mining and bibliometric network visualizing approach [ 5 ] using a freely available tool, VOSviewer [ 6 ], to summarize the content of titles and abstracts of the articles in our CIS result set. As the results were very informative, we retained the approach, but decided to conduct discrete analyses for the titles and abstracts to get a better separation of their contents. Figures 2 and 3 depict the resulting co-occurrence maps of the top-100 terms from the titles and of the most relevant terms (top 60 percent, n =354) from the abstracts of the 2,190 papers of this year’s CIS result set.
Fig. 2.
Clustered co-occurrence map of the most relevant terms (top-100) from the titles of the 2,190 papers in the 2017 CIS query result set. Node size corresponds to the frequency of the terms (binary count, once per paper). Edges indicate co-occurrence and distance of nodes corresponds to the association strength of the terms within the titles (only top 300 of 1,444 total edges are shown). Colors represent the six different clusters.
Fig. 3.
Clustered co-occurrence map of the most relevant terms (top 60 percent, n=354) from the abstracts of the 2,190 papers in the 2017 CIS query result set. Only terms found in at least ten different papers were included in the analysis. Node size corresponds to the frequency of the terms (binary count, once per paper). Edges indicate co-occurrence and distance of nodes corresponds to the association strength of the terms within the texts (only top 1,200 of 23,524 edges are shown). Colors represent the five different clusters.
The analysis of the titles revealed six different clusters with five evident hubs ( Figure 2 ): patient , system , use , study and analysis . The analysis of the abstracts resulted in five clusters with many similarities to the results found last year. The biggest cluster (in red on bottom right, Figure 3 ) dealing with the CIS core domain clinical/health information systems and electronic medical/patient/health/personal records in all forms and flavors as well as other health-IT applications , their functionalities , requirements and challenges , including meaningful use and health information exchange amongst others, was found again. The second cluster (in green on the left, Figure 3 ) again contained terms related to the context of the articles (study types or designs, study population, inclusion or exclusion criteria, study objectives as well as outcomes or other relevant context factors). Another cluster (in blue on top left, Figure 3 ) with location-related terms ( e.g. geographic information system, area, environmental factor, distance, distribution ) showed up as seen in the last year. A small cluster (in pink, on bottom Figure 3 ) again mainly contained terms related to adverse events , their detection, reporting, and prevention .
What we could observe in more clarity than in the previous year was a very prominent cluster (in yellow, top right Figure 3 ) dealing with behavioral aspects in CIS usage and research ( behavior, intention, perception, understanding, awareness, influence, adherence, communication ).
In order to round out our overview, we conducted an additional analysis and applied a topic modelling and visualization approach [ 7 , 8 ] a web-based interactive visualization of topics estimated using Latent Dirichlet Allocation that is built using a combination of R and D3. Our visualization provides a global view of the topics (and how they differ from each other to the text corpora of the papers’ abstracts. After data cleaning and preparation steps (e.g. filtering of punctuation and stop words, stemming of all words), latent topics were identified based on co-occurring words in the texts. These topics were then visualized and can be explored using an interactive, browser-based user interface. Figure 4 depicts a screenshot of this user-interface (for interactive exploration of the 30 most prominent topics found in the abstracts of the 2017 CIS query result, the user interface is available at https://iig.umit.at/yearbook17 ).
Fig. 4.
Screenshot of the user-interface for the interactive exploration of the topics found in the abstracts of the 2017 CIS section query result set (online available at https://iig.umit.at/yearbook17 ). In the screenshot topic number 1 – Electronic Health Record (EHR) adoption – is depicted. On the left side, the spatial distribution of the single topics is displayed on an intertopic distance map. The right side depicts the top-30 most relevant terms in the selected topic.
The results of our analyses show that clinical information systems research covers a wide variety of topics. In the whole spectrum of these topics, we found a fair proportion of high quality, highly interesting, and well-written articles. Out of them, three papers highlighting aspects of particular importance for the CIS research were finally selected as best papers. The first one, a very interesting contribution by Anderson and colleagues on applying EHR phenotyping for Type 2 Diabetes mellitus prescreening [ 9 ], perfectly reflects the actual Yearbook’s special topic by demonstrating the tremendous potential impact that intelligent reuse of EHR data could bring to the public health sector.
The second of the 2017 CIS best papers is dedicated to the users of clinical information systems and highlights the importance of the “last mile” in modern information logistics processes – data presentation and visualization [ 10 ]. Badgeley and colleagues conceptualized and implemented an extensible visualization framework to create dashboards for longitudinal health and wellness data.
The third best paper in the 2017 CIS section is one of the finest examples of comprehensive lifelong clinical information system evaluation that we have ever seen [ 11 ].Three periods were considered corresponding to 4, 8 and over 10 years after the first CIS deployment in 2000, respectively. Hadji and colleagues remind us on the importance of taking care of system quality and user satisfaction. The introduction section of this article was one of our favorites and we would like to recommend its reading to anyone who wants to get an introduction to IT evaluation models in a nutshell.
Although not selected as best papers, the remaining eleven candidate best papers were just as much interesting. Moen and colleagues, for example, compared automatic summarization methods for clinical free text notes and demonstrated the potential feasibility of such methods [ 12 ]. Reducing information overload and redundancies would be of great practical value for actual clinical care. Perhaps we will read about successful implementations in real-world clinical settings in coming IMIA Yearbook editions. The paper from Hosseini and colleagues also contributed to this field and presented a novel prototype for continuity of care document de-duplication, consolidation, and management [ 13 ].
Bazemore and colleagues picked up an old idea from the 1940s – enrichment of health record data with external (e.g. socioeconomic or environmental) information – transformed it to current situations, and proposed a novel geocoded approach to the integration of the social determinants of health into EHRs [ 14 ]. Such an approach could be highly beneficial for patient care as well as for secondary use purposes.
However, one of the most important aspects for any secondary use purpose is data quality. Johnson and colleagues applied an ontology-based assessment process to EHR data and determined its usefulness in characterizing data quality [ 15 ]. Automating data quality assessment processes using such a methodology could enable sharing of data quality metrics and lead to improved validity, transparency, and reproducibility of EHR data reuse-based research.
One of the rare examples of a prospective, randomized trial in the CIS domain was presented by Jamieson and colleagues who assessed the impact of an electronic clinical documentation system on the quality of admission notes in a general medical unit [ 16 ]. The paper by Attaallah and colleagues [ 17 ] in contrast is representing a multitude of papers providing evidence for achievable efficiency increases due to CIS.
Two other candidate best papers pointed in an important future direction for the CIS field, towards interoperable health Apps and smart health services [ 18 ]. Demski and colleagues showed that open data models can support such developments in various ways. A variety of such new and future developments will be based on the SMART on FHIR platform. Mandel and colleagues [ 19 ] provided a good insight into this topic.
Two other candidate best papers were representative for quite some research that focuses on an increasingly important stakeholder in the CIS domain: the patient. Carberry and colleagues presented an interesting contribution in the field of patient-centered care coordination [ 20 ]. Wilcox with colleagues explored the design and usefulness of patient-facing tools supporting inpatient medication management and tracking [ 21 ]. They concluded that tools designed to meet patients’ needs can play an important role in fostering patient participation on their own care and patient-provider communication.
Sometimes, communication can be harmful. Especially when content or data are communicated unintentionally. Sites and Pianykh explored the security of medical image archives by a worldwide scanning for unprotected DICOM servers [ 22 ]. Although their methodology may be considered disputable because it may have posed risk to productive DICOM services, the results should raise our awareness regarding security of clinical information systems.
Conclusions and Outlook
Once more, the results of our analyses show that the clinical information system domain is multi-dimensional and very complex. It is hard to draw a well-defined outline between CIS and other domains or other sections of the IMIA Yearbook. The trends that we observed in the last years are progressing. Clinical information systems are more than just sociotechnical systems for data collection, processing, exchange, presentation, and storage. They are the backbone of a very complex, trans-institutional information logistics process.
Appendix: Content Summaries of Selected Best Papers for the IMIA Yearbook 2017 Section “Clinical Information Systems”
Anderson AE, Kerr WT, Thames A, Li T, Xiao J, Cohen MS Electronic health record phenotyping improves detection and screening of type 2 diabetes in the general United States population: a cross-sectional, unselected, retrospective study J Biomed Inform 2016;60;162-8
A large fraction of type 2 diabetes mellitus (DM2) patients in the United States today remain undiagnosed. The selection of patients for screening currently relies on only a subset of known risk factors. In this paper, Anderson et al . carried out a cross-sectional, retrospective study to determine whether more comprehensive pre-screening criteria could be achieved using phenotyping of electronic health records (EHRs).
Multivariate logistic regression was used on a de-identified EHR dataset containing 9,948 US patients to predict current DM2. A “full EHR” dataset utilizing 298 features was compared with a limited “EHR DX” model excluding medications as well as to a “conventional” model containing only the risk factors used in current diabetes risk models (smoking status, sex, age, body mass index, hypertensive status). Out-of-sample validation was carried out using a random forests probabilistic model.
The EHR-based phenotypes performed significantly better than the conventional risk score, using both the logistic regression as well as the random forests model. The authors conclude that, even though EHR data is often incomplete and recorded unsystematically, it could be leveraged to cost-effectively identify patients with active, untreated, DM2 for further screening.
Badgeley MA, Shameer K, Glocksberg BS, Tomlinson MS, Levin MA, McCormick PJ, Kasarskis A, Reich DL, Dudley JT EHDViz: clinical dashboard development using open-source technologies BMJ Open 2016;6;e010579
While an increasing amount of longitudinal health data is being captured in electronic health records (EHRs), presentation is often restricted to tabular and textual views which are not conducive to recognizing trends (e.g. disease progression) in the dataset. Additional time-series data is increasingly being generated through mobile patient monitoring and quantified-self devices.
In this article, Badgeley et al . describe the concept and implementation of the extensible visualization framework “EHDViz” to create dashboards for longitudinal health and wellness data. The platform uses open source components like the R statistics package to ingest biomedical data sources and provides an intuitive user interface for interactively selecting relevant data items for static or near-realtime visualization. Data from multiple sources are merged and normalized, and it is possible to integrate further processing steps (e.g. calculation of risk scores) into the pipeline. Using simulated datasets, the authors provide several prototype dashboards to demonstrate a visualization of multi-source data for a single patient as well as a multi-patient visualization with interactive selection of EHR data items.
The authors conclude that the framework can support the rapid design and implementation of visualization dashboards, which can be applied in clinical care and research as well as teaching use cases.
Hadji B, Martin G, Dupuis I, Campoy E, Degoulet P 14 Years longitudinal evaluation of clinical information systems acceptance: The HEGP case Int J Med Inform 2016;86;20-9
Active and meaningful use as well as user satisfaction are important success factors for the continuing success of a clinical information system (CIS) implementation. In this article, Hadji et al . describe a longitudinal evaluation of these aspects at the Georges Pompidou University Hospital in France. The long observation period of 14 years provided a unique opportunity to assess the evolution of CIS acceptance during early (4 years), late (8 years), and very late (>10 years) stages of CIS use.
The survey instrument covered items from multiple established acceptance models (TAM2, UTAUT, ISSM, ECM, ITPAM), considering six acceptance dimensions: CIS quality, facilitating conditions, perceived usefulness, confirmation of expectations, use, and global satisfaction. Relationships between these constructs were tested using multiple regression testing and structural equation modeling.
CIS acceptance increased over the study period. Perceived usefulness initially was lower in medical staff than in non-medical staff. While global satisfaction initially appeared to be determined by CIS use, CIS quality, and perceived usefulness, the association with CIS use disappeared over time.
The authors conclude that acceptance dimensions change significantly over time and that models should be adapted to the according phase of CIS use. They postulate that the decrease of the relationship between CIS use and satisfaction over time could be interpreted as a maturity indicator of a CIS project.
Acknowledgements
We would like to acknowledge the support of Martina Hutter, Lina Soualmia, Adrien Ugon, Brigitte Séroussi and the whole Yearbook editorial team as well as the numerous reviewers that contributed to the selection process of the best papers. Special thanks go to our colleague Lukas Huber who implemented the topic modelling approach.
Footnotes
Equal Contribution
References
- 1.Lamy JB, Séroussi B, Griffon N, Kerdelhué G, Jaulent MC, Bouaud J. Toward a formalization of the process to select IMIA Yearbook best papers. Methods Inf Med. 2015;54(02):135–44. doi: 10.3414/ME14-01-0031. [DOI] [PubMed] [Google Scholar]
- 2.Cresswell KM, Mozaffar H, Lee L, Williams R, Sheikh A.Workarounds to hospital electronic prescribing systems: a qualitative study in English hospitals. BMJ Qual Saf 2016(April):bmjqs-2015-005149. [DOI] [PubMed]
- 3.Hravnak M, Chen L, Dubrawski A, Bose E, Clermont G, Pinsky MR. Real alerts and artifact classification in archived multi-signal vital sign monitoring data: implications for mining big data. J Clin Monit Comput. 2016;30(06):875–88. doi: 10.1007/s10877-015-9788-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Hackl WO, Ganslandt T. New Problems - New Solutions : A Never Ending Story Findings from the Clinical Information Systems Perspective for 2015 Findings and Trends : Clinical. Yearb Med Inform. 2016:146–51. doi: 10.15265/IY-2016-054. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Waltman L, van Eck NJ, Noyons ECM.A unified approach to mapping and clustering of bibliometric networks J Informetr [Internet] 2010[cited 2017 May 26];0404629–35.Available from: http://www.sciencedirect.com/science/article/pii/S1751157710000660. [Google Scholar]
- 6.van Eck NJ, Waltman L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics. 2010;84(02):523–38. doi: 10.1007/s11192-009-0146-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Sievert C, Shirley K.LDAvis: A method for visualizing and interpreting topics. Proc Work Interact Lang Learn Vis Interfaces [Internet] 201463–70.Available from:http://www.aclweb.org/anthology/W/W14/W14-3110.
- 8.Chuang J, Manning CD, Heer J.Termite. Proc Int Work Conf Adv Vis Interfaces - AVI’12 [Internet] 201274Available from:http://dl.acm.org/citation.cfm?doid=2254556.2254572.
- 9.Anderson AE, Kerr WT, Thames A, Li T, Xiao J, Cohen MS. Electronic health record phenotyping improves detection and screening of type 2 diabetes in the general United States population: A cross-sectional, unselected, retrospective study. J Biomed Inform. 2016;60:162–8. doi: 10.1016/j.jbi.2015.12.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Badgeley MA, Shameer K, Glicksberg BS, Tomlinson MS, Levin MA, McCormick PJ et al. EHDViz: clinical dashboard development using open-source technologies. BMJ Open. 2016;06(03):e010579.. doi: 10.1136/bmjopen-2015-010579. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Hadji B, Martin G, Dupuis I, Campoy E, Degoulet P. 14 Years longitudinal evaluation of clinical information systems acceptance: The HEGP case. Int J Med Inform. 2016;86:20–9. doi: 10.1016/j.ijmedinf.2015.11.016. [DOI] [PubMed] [Google Scholar]
- 12.Moen H, Peltonen LM, Heimonen J, Airola A, Pahikkala T, Salakoski T et al. Comparison of automatic summarisation methods for clinical free text notes. Artif Intell Med. 2016;67:25–37. doi: 10.1016/j.artmed.2016.01.003. [DOI] [PubMed] [Google Scholar]
- 13.Hosseini M, Meade J, Schnitzius J, Dixon BE. Consolidating CCDs from multiple data sources: A modular approach. J Am Med Inform Assoc. 2016;23(02):317–23. doi: 10.1093/jamia/ocv084. [DOI] [PubMed] [Google Scholar]
- 14.Bazemore AW, Cottrell EK, Gold R, Hughes LS, Phillips RL, Angier H et al. “Community vital signs”: Incorporating geocoded social determinants into electronic records to promote patient and population health. J Am Med Inform Assoc. 2016;23(02):407–12. doi: 10.1093/jamia/ocv088. [DOI] [PubMed] [Google Scholar]
- 15.Johnson SG, Speedie S, Simon G, Kumar V, Westra BL, Johnson SG. Application of an Ontology for Characterizing Data Quality for a Secondary Use of EHR Data. Appl Clin Inf0rm. 2016;07(07):69–88. doi: 10.4338/ACI-2015-08-RA-0107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Jamieson T, Ailon J, Chien V, Mourad O. An electronic documentation system improves the quality of admission notes: A randomized trial. J Am Med Inform Assoc. 2017;24(01):123–9. doi: 10.1093/jamia/ocw064. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Attaallah AF, Elzamzamy OM, Phelps AL, Ranganthan P, Vallejo MC. Increasing operating room efficiency through electronic medical record analysis. J Perioper Pract. 2016;26(05):106–13. doi: 10.1177/175045891602600503. [DOI] [PubMed] [Google Scholar]
- 18.Demski H, Garde S, Hildebrand C. Open data models for smart health interconnected applications: the example of openEHR. BMC Med Inform Decis Mak. 2016;16(01):137. doi: 10.1186/s12911-016-0376-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Mandel JC, Kreda DA, Mandl KD, Kohane IS, Ramoni RB. SMART on FHIR: A standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc. 2016;23(05):899–908. doi: 10.1093/jamia/ocv189. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Carberry KE, Landman Z, Xie M, Feeley T, Henderson J, Fraser C. Incorporating longitudinal pediatric patient-centered outcome measurement into the clinical workflow using a commercial electronic health record: A step toward increasing value for the patient. J Am Med Inform Assoc. 2016;23(01):88–93. doi: 10.1093/jamia/ocv125. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Wilcox L, Woollen J, Prey J, Restaino S, Bakken S, Feiner S et al. Interactive tools for inpatient medication tracking: A multi-phase study with cardiothoracic surgery patients. J Am Med Inform Assoc. 2016;23(01):144–58. doi: 10.1093/jamia/ocv160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Stites M, Pianykh OS. How secure is your radiology department? Mapping digital radiology adoption and security worldwide. Am J Roentgenol. 2016;206(04):797–804. doi: 10.2214/AJR.15.15283. [DOI] [PubMed] [Google Scholar]