Facilitating Biomedical Researchers’ Interrogation of Electronic Health Record Data: Ideas from Outside of Biomedical Informatics

Abstract

Electronic health records (EHR) are a vital data resource for research uses, including cohort identification, phenotyping, pharmacovigilance, and public health surveillance. To realize the promise of EHR data for accelerating clinical research, it is imperative to enable efficient and autonomous EHR data interrogation by end users such as biomedical researchers. This paper surveys state-of-art approaches and key methodological considerations to this purpose. We adapted a previously published conceptual framework for interactive information retrieval, which defines three entities: user, channel, and source, by elaborating on channels for query formulation in the context of facilitating end users to interrogate EHR data. We show the current progress in biomedical informatics mainly lies in support for query execution and information modeling, primarily due to emphases on infrastructure development for data integration and data access via self-service query tools, but has neglected user support needed during iteratively query formulation processes, which can be costly and error-prone. In contrast, the information science literature has offered elaborate theories and methods for user modeling and query formulation support. The two bodies of literature are complementary, implying opportunities for cross-disciplinary idea exchange. On this basis, we outline the directions for future informatics research to improve our understanding of user needs and requirements for facilitating autonomous interrogation of EHR data by biomedical researchers. We suggest that cross-disciplinary translational research between biomedical informatics and information science can benefit our research in facilitating efficient data access in life sciences.

Graphical abstract

1 INTRODUCTION

Biomedical research has long benefited from a valuable and cost-effective data source: patient health records [1]. For example, the Apgar Scale [2] and the Goldman multifactorial index of cardiac risks [3] were both derived from analyses of patient health records. With the increasingly pervasive adoption of electronic health record (EHR) worldwide [4], many have recognized the rich clinical data increasingly made available by EHRs as a promising data resource for accelerating medical knowledge discovery [5] and for enabling comparative effectiveness research [6–10]. Subsequently, the demand for reusing EHR data for research among biomedical researchers has been rising rapidly [11–15]. Assisting biomedical researchers to interrogate EHR data has been a vital mission for the biomedical informatics research community. However, this task faces significant human and technological barriers [10, 16, 17]. Current data captured by EHRs are not optimized for secondary uses beyond clinical care or administration-centered documentation practices so that many institutions employ intermediating data analysts to retrieve EHR data for biomedical researchers, with varying degrees of assistance from self-service query tools. The use of intermediaries may not scale to large data networks such as the clinical data research networks (CDRNs) as part of the PCORnet [18] established by the Patient Centered Outcomes Research Institute [19]. For example, the heterogeneity of data representations across institutions and the complex, idiosyncratic local data collection processes that often remain “black boxes” to intermediaries are serious barriers facing users of the data contained in PCORnet. To contain cost for involved expensive operations, many institutions have to charge clinician scientists for reusing such data collected during patient care for research. Meanwhile, self-service query support is still at its early stage of development and may not support sophisticated data queries [20, 21].

By identifying and reviewing existing theories and best practices for EHR data interrogation, we aim to inform the design of next-generation EHR data interrogation aids that directly facilitate biomedical researchers to autonomously retrieve and reuse this data for clinical and translational research. Towards this goal, this paper contributes a literature review on this topic. We summarized existing approaches, identified research gaps, and recommended research priorities. Although this review focuses on EHR data, the knowledge gained may generalize to interactive end-user data interrogation for other reusable health data resources.

2 METHODS

2.1 Development of a Conceptual Framework for Interactive Data Retrieval

An information retrieval process addresses information needs using a sequence of tasks [22]. The complexity of the task sequence is dependent on the information retriever’s a priori knowledge of the information need, the information retrieval process stipulated by data owners, and the complexities of each of the tasks used to complete the information retrieval process [23–27]. Many models have been developed for characterizing the information retrieval process or for investigating how information systems enable users during this process [28–37]. For example, the berry-picking model [28] and the sense-making model [30] focus on how the user iteratively refines his or her information needs based on their conceptualizations of the information space. Among all existing models, only one developed by Bystrom and Jarvelin explicitly defined three entities that influence the complexities of an information retrieval process: user, channel, and source, to characterize the information retrieval process [24]. The user entity focuses on the user’s profiles, communication styles, and knowledge of data. The source concerns data representations towards optimal data retrieval efficiency. The channel masks the complexities of the source and translates user information needs to data representations. Therefore, source is the container of information and channel guides efficient navigation of the source. We adopted this conceptual framework to organize the literature for interactive EHR data retrieval.

In this paper, we survey related methods and theories in the context of EHR data retrieval for secondary use by end users who are unfamiliar with the data, such as biomedical researchers and clinician scientists. Since this paper aims to augment end users with improved query formulation, we focus primarily on effort supporting the user and the channel, while briefly describe existing efforts on the source. We co-opted the constructs of user, channel, and source combining them with the concepts of query formulation and query execution, as shown in Figure 1. For example, a researcher may want to identify an institution’s mortality rate among its patients undergoing coronary artery bypass. Query formulation transforms vague data requests (e.g., “adult patients younger than 75 years old with coronary artery bypass surgery last year”) into contextualized data requests consisting of specific EHR data elements (e.g., “patient DOB, Current Fiscal Year, billing code for coronary artery bypass billed in this current fiscal year”). The step of query execution further translates this query from contextualized data elements into executable database queries consisting of disparate data types and represented by local terminologies using The Structure Query Language (SQL).

Figure 1.

A conceptual framework for interactive data retrieval.

2.2 Literature Search

We adopted the post-positivist model for research [38]. We iteratively searched for related work published between 2009 and 2013 allowing us to treat all experience as data. Following this model, we searched beyond the field of biomedical informatics or clinical research informatics that were obvious to be relevant and included the literature in informatics and computer and information science. Also, we categorized all included citations by their focus on user, source, and channel so that significant amounts of qualitative information were categorized to produce quantitative information to help us draw the big picture and deduce evidence gaps.

We seeded our search with 29 articles proposed by the senior author (CW) [19, 23, 24, 28, 29, 39–62]. Additionally, citation searches within these articles provided an additional 45 references [16, 22, 25–27, 30–34, 37, 63–102]. These 74 articles served as the basis for the development of the search query. With our initial search query, we iteratively searched and reviewed the identified articles, incorporated new search keywords as they emerged, revised our search string and article inclusion/exclusion criteria iteratively according to their relevance as determined by manual review. We surveyed both the information science literature (i.e., http://dl.acm.org) and biomedical informatics literature (i.e., MEDLINE). We limited our search to the main journal citation databases, the ACM Digital Library, and Medline, for the respective fields of information science and biomedical sciences because we feel this will provide a representative sample for our topic.

Figure 2 is a flow chart that highlights the final search strings for PubMed and ACM databases and the inclusion and exclusion criteria for selecting articles for this review. The first author generated the final search string and reviewed the title and abstracts of the returned articles; articles containing any of the exclusion criteria were removed from the pool. Next, the first author iteratively reviewed and annotated the 125 included articles using the conceptual framework developed in Section 2.1. For each annotation, the first author wrote a summary and justification paragraph. After annotation, the first author reviewed the summary and justification paragraphs for each set of articles against the conceptual framework components and derived themes within these sections. For the source, the major themes identified were EHR data modeling (how data is structured and the standards used to store data elements) and warehousing (dedicated silos of data for secondary use). For the User, the major themes identified were Information Need (defining the complexity of the need) and user modeling (understanding the user attributes and the information seeking strategies used). For the channel, the major themes identified were query formulation (the process of defining an information need) and execution (the process of translating an information need into an executable database query). Table 1 organizes the articles according to our conceptual framework. As shown in Table 1, more work fell onto user modeling, human intermediaries, and reference interview in the field of information science than in the field of biomedical informatics. In the following sections, we will synthesize the major themes from each discipline and compare and contrast their ideas from difference sources.

Figure 2.

The search strings and article selection flowchart (*Articles could be classified into multiple categories as some content spans multiple categories).

Table 1.

The distribution of relevant topics in two bodies of literature

	Biomedical Informatics	Information Science
SOURCE
EHR Data Modeling	[8, 12, 41, 46, 48, 52, 70, 71, 90, 103–109]
EHR Data Warehousing	[40, 97, 110]
USER
Information Need
Information Need Complexity	[50, 53, 72, 111–113]	[24, 60, 92]
User Modeling
Information Seeking Processes	[54, 59, 77, 84]	[22–24, 26, 36, 57, 67]
User Cognitive Styles	[17, 55]	[23, 25, 28–37, 63, 85, 86, 91]
CHANNEL
Task 1: Query Formulation
Concept Representation	[17, 27, 46, 51, 62, 70, 71, 98, 105, 114–119]	[100]
Characterization of Data integrity	[17, 114, 120, 121]	[26]
Query Templates	[47, 58, 72, 75, 80, 81]	[122]
Self-Service Query Tools	[20, 41, 51, 52, 73, 82,88, 97, 101, 113, 123– 128]	[42, 93, 129–133]
Human Intermediaries	[1, 45, 134, 135]	[39, 44, 61, 83, 89, 94, 95]
Reference Interview		[39, 43, 56, 64–66, 68, 69, 74, 79, 87, 94, 96, 102]
Task 2: Query Execution
Phenotyping	[49, 62, 76, 78, 99, 117, 136–138]

3 SOURCE

The barriers to task-based data access in life sciences fall into two categories: (1) human factors (e.g., a user lacking a correct conceptualization of task complexities); (2) system factors (e.g., technology limitations in the existing systems such as data heterogeneity and fragmentation) [17]. We presented example known barriers and corresponding recommended solutions in Table 2. Human factors relate to the user, while system factors relate to the metadata of the source, or in this instance, the lack of metadata concerning known and underlying EHR data quality issues.

Table 2.

Known themes regarding EHR data access barriers and solutions

Level	Barriers	Solutions
Human factors	Known and latent EHR data quality issues [17, 120, 121]	Transparent reporting of data limitations for intended uses [26, 114]
System factors	System interoperability[108]; Real time analytics [8, 12]; Data fragmentation[108]; Data heterogeneity [121]	Streamlining data access workflow[71] [52]; Data warehousing [49, 78]; Data modeling and integration strategies (i.e. The Observational Medical Outcomes Partnership Common Data Model) [46] [48, 109]

In the context of this study, we discuss data warehousing at the institutional level rather than at the state or national level, although the same principles may apply to both. Data warehousing has been a focus in the clinical research informatics community for overcoming technical barriers to data access by providing efficient access to integrated EHR data, which can be centralized or federated. The centralized infrastructure [40, 41, 52] avoids data heterogeneity but can introduce challenges for incorporating new or latent data elements over time because each update affects the entire database, hence not being able to scale easily. In contrast, the federated architecture is flexible [90, 103–105] for allowing autonomous data control and growth over time. It allows data representation heterogeneity [70, 106, 107]. It also enables the leveraging of distributed local expertise for data modeling and data quality control and supports geographic distribution by multiple stakeholders. The national center for education statistics has summarized the tradeoffs between centralized and federated data repositories [139]. Briefly, centralized systems ease data governance, increase data retrieval performance, provide uniform data for efficient data mining, entail a high-cost burden for ensuring data currency and completeness, and are harder to scale for evolving data needs and different data access workflows [140]. Many institutions have centralized data repositories such as STRIDE [110]. The most widely used platform, i2b2 and its SHRINE architecture, supports distributed and federated data repository [97]. The federated architecture represents an established model for most large data networks such as PCORnet [18, 141].

4 USER

4.1 Information Need Complexity

One component of user modeling is understanding the complexity of information need, which depends on the diversities in the use context [72], variations in information seeking behaviors[24], and heterogeneity in languages used to express the information need [92]. Others have proposed a categorical scale of data need complexity by measuring the amount of work required to accomplished the task for satisfying the information need [60, 86]. Structures have been defined to characterize a complex information need, including a problem statement, an event of interest, a comparison event (if necessary), and potential effects of the event of interest [50]. Unfortunately, little is known about the data needs of biomedical researchers [111–113]. The very few studies available have largely focused on identifying sets of the major data elements needed to facilitate research in particular medical domains of interest. Cimino et al. leveraged these key data elements needed by researchers to inform the development of a user-centric query tool [113]. What has been lacking includes a thorough understanding of user preferences and search behaviors, as well as communication patterns between biomedical researchers and query analysts for clarifying data needs iteratively.

4.2 User Cognitive Styles

Five user characteristics influence a user’s information seeking tactics:

1. Phases of mental model of information –confusion, doubt, threat, hypothesis testing, assessing, and reconstructing [63]

2. Levels of need–visceral, conscious, formal, and compromised [39]

3. Levels of specificity–new problem, new situation, experiential needs, and well understood situation [23]

4. Expression–questions connections, and commands gap [23, 39]

5. Mood–invitational, and indicative [63]

Kuhlthau provides a great amalgamation of these user characteristics in her theoretical foundation of the information seeking process [67], which was well supported by Vakkari's review [57]. Additionally, the information seeking process has been modeled within the biomedical literature. Mendonca et al. [54] and Hung et al. [59] have provided models for the biomedical literature information seeking process, which propose to aid user’s search strategies through well-structured clinical queries and by leveraging the knowledge of human search experts, respectively.

User cognitive styles shape information seeking processes [36]. Many describe these styles along two orthogonal axes: analytic and descriptive. The analytic cognitive style captures an active approach to information seeking in which conceptual level questioning is used to resolve information need, whereas the descriptive cognitive style represents a passive approach, where concentration on the most detailed level of the subject matter is used to resolve an information need. User cognitive styles are either passive (high descriptive and low analytic with attention to detailed questions) or active (high analytic and low descriptive questioning). The active styles represent more effective and efficient search strategies than passive styles [37].

A user’s domain knowledge and technical knowledge are both associated with their cognitive styles and effective search strategies [17, 50, 91]. The users’ cognitive styles can be differentiated [85, 86] by search tactics. Users of varying cognitive styles have different sense making strategies or processes. Studies of cognitive styles offer a more generalizable mechanism to stratify users and to predict individual information seeking styles. Cognitive science allows characterization of the demand characteristics of a particular task and to focus on the appropriate problem dimensions [55]. Although user cognition during EHR data interrogation is rarely studied in the biomedical informatics literature, especially for facilitating EHR data interrogation, such studies are much needed. Cognitive studies of users can enable user centered EHR data interrogation designs aiming to improve the user experience.

5 CHANNEL

The complexity of a data source is multidimensional, including heterogeneous semantic representations [46, 70, 71], opaque data integrity, complex time expressions [98, 100], and fragmented knowledge of logical data constructs [17, 27, 114]. A channel enables users to navigate data despite complexities by providing users with an abstract mechanism to interact with data sources during query formulation or query execution.

5.1 Query Formulation

The query formulation component facilitates the iterative interaction between the user and the source to formulate a query in a user’s language. Hripcsak et al. investigated two EHR data retrieval channels, AccessMed and Query by Review, and found neither achieved adequate performance, indicating the difficulty of query formulation for EHR data [51]. Next we review common aids for query formulation and related execution challenges: i.e., self-serving query tools, query templates, as well as human intermediaries.

5.1.1 Human intermediaries

Human intermediaries are often employed to formulate user queries to ensure feasibility and precision [44, 45, 83, 89, 134]. Intermediaries usually have received formal training and possess a deep understanding of work culture and technical skills for data querying [1, 61, 95, 135]. The biomedical literature provides scant knowledge regarding how human intermediaries operate in the biomedical information rich domain. Information science has extensively studied human intermediaries, most notably, librarians and their development of the reference interview technique [39, 94]. Reference interviews elicit tacit user needs, specify vague queries, narrow overly broad questions, and suggest further dimensions of the information need that the user may not have expressed but are logically related to the user-stated objective. It enables a skillful interrogation process widely adopted by librarians for converting vague and general data request provided by users into specific data queries expressed using user language [39, 56, 65, 66, 68, 69, 87, 94, 96].

Elicitation strategies used in reference interviews have been explored to improve negotiation of information needs [43, 64, 74, 79]. Specifically, interrogation strategies are primarily developed to obtain the user’s objective surrounding the information need. When users are aware of the reference interview’s purpose, they are willing to provide additional information on objective and intent. In a related study, Lin et al. analyzed need negotiations and extracted a taxonomy of clarification questions that fit within a set of six classes [102]. The taxonomy applied in the context of EHR data interrogation is illustrated in Table 3, with example clarification questions. Our results imply that in the context of interactive EHR data retrieval, the reference interview may provide human intermediaries with a more efficient workflow to best extract a non-vague description of the data need from the user.

Table 3.

A Taxonomy of Clarification Questions Utilized During Need Negotiation

Question Type	Goal	EHR Data Clarification Examples
Relevance threshold	To establish the user’s relevance threshold, a mapping of a continuous scale into a binary decision	What is the A1c threshold for the diabetes patients that you are looking for?
Ambiguity in conjoined facets	To establish the relationship type between conjoined facets	Do you only want patients that have both a diagnosis of diabetes and hypertension or patients with a diabetes diagnosis with or without a hypertension diagnosis?
Example concept	To ensure an identified facet is relevant to the user’s information need	Would patients without Diabetes diagnoses, but taking a medication for diabetes be of interest to you?
Closely related or subset concept	To expand or narrow the scope of the user’s information need on the specificity of a particular facet	Do you want patients with both types of Diabetes? Type 1 and 2?
Related topical aspects	To establish the user’s interest in facets that are conceptually related, but not directly requested	Does it matter what treatment protocol the diabetic patients are on?
Acceptability of summaries	To define both the user’s expectation of the information retrieved	Do you just want to know how many patients fit your criteria?

5.1.2 Query Templates

Templates are another effective technique for expressing standards-based structured data needs free of ambiguity and vagueness [77, 84]. A query template provides an organizational structure for the user to describe their information need in a non-vague structure [80]. Templates have been developed to access clinical data [47] and medical literature [58, 72, 75, 80]. The Patient, Intervention, Control/Comparison, and Outcome (PICO) framework is extensively used to explore the medical literature for relevant resources [81, 122]. Currently, there is no well-accepted standard template based on community consensus. Instead, many medical institutions require users to complete data requests using free text.

5.1.3 Self-service query tools

Since human intermediaries are expensive and time-consuming, self-service query aids have been pursued in many institutions in recent years [41, 51, 73, 97, 101, 113, 123, 125, 127, 128]. Some are form-based, while others support queries in natural language [42, 51]. Visual query formulation is a recent trend and is expected to reduce user cognitive load by presenting information intuitively to the user [129]. The Informatics for Integrating Biology and the Bedside (i2b2) project represents the most widely adopted self-service EHR data retrieval system. The system’s terminology explorer and query builder allow the user to search the terminology for applicable terms and build cohorts using a frame system with Boolean constraints [52, 82, 88, 97, 113, 123–128]. Deshmukh et al. studied the various types of data requests these applications were able to resolve. The study suggested that i2b2 facilitated relatively simple, cohort identification queries. They also acknowledged that the majority of requests they studied were “simple” queries [20]. These reports indicate that the majority of self-service tools for EHR data support a limited scope of data specification. Additionally, these tools place the burden to identify the correct terms associated with a particular medical concept on the user.. Many complex data request require not just simple constraints, e.g. “all patients diagnosed with diabetes between May and July of 2012”, but complex relations between data elements, e.g. “all patients with their first recorded diagnosis of diabetes between May and July of 2012, and all lab glucose tests after their diagnosis and before the start of treatment.” For these complex temporal queries, temporal query tools can be used to visualize raw data or concepts over absolute and relative temporal timelines [93, 101, 130–133]. Though experimental, these tools offer a solution to a complex problem of temporal specification and visualization of EHR data. Meanwhile, significant EHR data processing and transforming are needed for these systems to work appropriately.

5.2 Query Execution

The query execution component focuses on the conversion of a user query into an executable database query by mapping medical concepts specified by the user to the EHR data elements that define that concept. The Electronic Medical Records and Genomics (eMERGE) consortium [117] has studied the problem of phenotyping disease concepts through enumerable data elements within the EHR. The EMERGE consortium has shown that each disease phenotype contains significant heterogeneity, underlying elements representing nested Boolean logic, complex temporality and ubiquitous ICD-9 codes [117, 136]. As of 2013, the group has validated 13 phenotypes [62]. Although the temporal nature of EHR data was considered only in some of the eMERGE phenotypes, temporal abstraction is an important technique for EHR phenotyping. Post et al. have established the PROTEMP method, which allows for the abstraction of temporal data events [99]. Additionally, Shahar’s framework on temporal abstraction has described promising methods for formally representing temporal patterns [76, 137, 138].

6 DISCUSSION

Interactive EHR data retrieval involves complex interactions among users, sources, and channels. The healthcare industry has heavily invested in infrastructures for data integration. To maximize the return on investment and to use these resources to advance medicine, we need to make such data accessible to biomedical researchers for various computing needs.

Self-service query tools have not fully addressed user needs and hence make human intermediaries indispensable in many institutions. These intermediaries utilize a needs- negotiation process with the user. Barriers facing this process include the lack of medical and technical knowledge by the intermediary and the user, respectively. Bridging these knowledge gaps for the intermediary and user may engender efficient communication. Furthermore, a user often presents a vague understanding and description of their information need. Intermediaries may benefit from a standardized structure through which requests can be organized, which may reduce the ambiguity of the request and allow the intermediary to focus on other tasks, i.e. query execution.

Relatively little is known regarding biomedical researcher’s cognitive styles and information seeking strategies. Table 4 lists key knowledge gaps and potential recommendations for bridging those gaps. Additional exploratory studies are needed to bridge the knowledge gaps concerning how biomedical researchers interrogate EHR data and what their barriers are. We can invest more effort on interactive information retrieval that augments not only the source but also the user.

Table 4.

The knowledge gaps and recommendations for advancing EHR interrogation

Aspects	Knowledge Gaps	Recommendations
User	Lack of measure of information need complexity Lack of knowledge of how the cognitive styles of medical researchers affect the information seeking process	Develop metrics for measuring EHR information need complexities Conduct qualitative studies of the information seeking processes of multidisciplinary medical researchers and their barriers to clinical data access
Query Formulation Channel	Lack of formalized structure for the medical researcher to express their information need Poor understanding of the data need- negotiation process performed by data intermediaries	Investigate other formal structures used for document retrieval, i.e. PICO framework Support reference interview for EHR data interrogation

Information-seeking models explain the sub-optimal outcomes resulting from current methods used for EHR data interrogation. The granularity of data that biomedical researchers are seeking adds more complexity to existing information seeking models. Additional understanding of process-oriented EHR data access by biomedical researchers is needed. To this end, we extracted from the literature three promising concepts to aid in the construction of an ideal process-oriented EHR data interrogation model.

First, both information science and biomedical informatics have established the important role semantics play in optimal information retrieval. For example, the PICO framework is an excellent user aid, which helps organize and express the information need of clinicians. In the context of EHR data interrogation, the PICO framework can be potentially a good starting point for supporting the expression of biomedical researchers’ EHR data need.

Second, the complexity of information need shapes information search tactics. A metric for complexity assessment of the information need can optimize resource allocation while resolving the user’s information need. For example, complex data requests would be directed to an informatician, whereas simple request would be facilitated through existing self-service query tools. A standardized method for EHR data need complexity assessment can further enable global resource optimization.

Third, the established reference interview has effectively helped librarians to clarify user needs. Informaticians provide a similar role as librarians but lack a guideline on how to conduct reference interview for EHR data. Experience is the only way for informaticians to gain insights and expertise for this task. To increase the efficiency and effectiveness of the EHR data needs negotiation, an EHR-based reference interview, conducted by an informatician, may aid the query formulation process in the translation of vague EHR data requests into specific data queries. More studies are needed in this area to enable reference interview for EHR data.

Our study has two major limitations. First, we developed our search criteria based on preselected topics. This method may be biased towards self-selected topics and leave out topics relevant to this review but not searchable by the query derived from the pre-selected topics. Nevertheless, we used an established method to identify a focused topic set and believe this review is representative of what is available in the literature. Second, our focus on the most recent four years of literature may have excluded seminal articles in the field from the past; however, we believe our exhaustive citation search should have largely mitigated this problem.

7 CONCLUSIONS

This review surveys the methodological considerations for interactive EHR data interrogation. We have identified knowledge gaps and research opportunities for advancing EHR data interrogation. Our results show that support for reference interview for EHR data is a promising direction for improving user autonomy for biomedical researchers during EHR data interrogation. More user understanding is needed to enable such support cost-effectively. We suggest that cross-disciplinary translational research between biomedical informatics and information science is needed to apply theories and techniques from information science to facilitate efficient end user data interrogation in life sciences.

• There is no standard complexity measure to assess EHR data requests

• How a researcher’s cognitive style affects EHR data seeking is unknown

• No formalized structure guiding researchers to express their EHR data need

• The need-negotiation process performed informaticians is poorly understood