Identified themes of interactive visualizations overlayed onto EHR data: an example of improving birth center operating room efficiency

Abstract

Objective

While electronic health record (EHR) systems store copious amounts of patient data, aggregating those data across patients can be challenging. Visual analytic tools that integrate with EHR systems allow clinicians to gain better insight and understanding into clinical care and management. We report on our experience building Tableau-based visualizations and integrating them into our EHR system.

Materials and Methods

Visual analytic tools were created as part of 12 clinician-initiated quality improvement projects. We built the visual analytic tools in Tableau and linked it within our EPIC environment. We identified 5 visual themes that spanned the various projects. To illustrate these themes, we choose 1 exemplary project which aimed to improve obstetric operating room efficiency.

Results

Across our 12 projects, we identified 5 visual themes that are integral to project success: scheduling & optimization (in 11/12 projects); provider assessment (10/12); executive assessment (8/12); patient outcomes (7/12); and control and goal charts (2/12).

Discussion

Many visualizations share common themes. Identification of these themes has allowed our internal team to be more efficient and directed in developing visualizations for future projects.

Conclusion

Organizing visual analytics into themes can allow informatics teams to more efficiently provide visual products to clinical collaborators.

INTRODUCTION

Electronic health record (EHR) systems allow for the storage and organization of vast amounts of health data. While front-end interfaces allow clinicians to access data on individual patients, aggregating data across groups of patients is more challenging. Though modern EHR systems provide tools for interactive dashboards, their functionality is often limited to simple data structures and clinical cohorts. In the larger data science field, there has been a growth of interactive graphics tools such as R-Shiny, Plotly, and Tableau. These tools allow users to point-and-click their way through visualizations of the data. One can change, for example, the underlying cohort, the time scale, or the outcomes of interest. Such tools are valuable for allowing interested parties, particularly non-data scientists, to develop a greater understanding of patterns that underlie the data.¹ Tableau, a software tool for creating data visualizations, has been used in retail and government sectors for many years; however, only recently has Tableau been introduced into the healthcare field.^2–6 One of the strengths of Tableau is that, unlike the other tools, it has direct functionality into the EPIC EHR environment. This allows a developer to leverage Tableau’s querying tools to gather EHR information and produce visualizations that allow a user (ie, clinician) to interact with the data.

Our team has used Tableau to build 12 interactive visual analytics from extracted EHR data. In order to better streamline processes within our group, we retrospectively assessed our dashboards, identifying common visual displays. After reviewing our projects, we identified 5 visual analytic themes. In this paper we illustrate these themes by working through a specific use case—improving obstetric operating room (OR) efficiency at the Duke Birthing Center. Overall, identification of these themes has helped further development by our group.

MATERIALS AND METHODS

Work environment

In response to the need for high-quality analytics, the multispecialty physician network at Duke University Health System—the Private Diagnostic Clinics (PDC)—formed the PDC Outcomes Research Team (PORT). PORT is a cross-functional group tasked with delivering EHR data and analytics for quality improvement needs. The analytic team consists of business analysts, data warehouse developers, business intelligence developers, and biostatisticians. This team works closely with the each project’s clinical collaborator.

Description of Tableau

Tableau is a business intelligence and visual analytics software tool that allows users to transform datasets into interactive charts and dashboards, thereby providing clinical insights. Developers can place the visuals on a customized refresh schedule. Clinicians log in to a secure Tableau project folder on Duke’s network, and, through Duke’s partnership with Tableau, any authorized user can be granted access to a project. The level of access granted to these users is determined through a collaboration between the PORT team and the principal investigator. We provide details on how we integrate Tableau with the EPIC environment in the Supplementary Material.

Theme identification & case study

PORT has developed 12 Tableau dashboards to date. To help us more efficiently work with clinical teams, we sought to identify visual themes. To identify these themes, we inspected our visualizations and grouped them into categories that shared similar underlying analytic architecture. We shared these groupings with our prior clinical collaborators, who assisted in labeling the themes based on how they used them. This resulted in 5 key visual themes.

To illustrate these themes, we focus on 1 specific project, Project BENEFIT. Project BENEFIT is an initiative at the Duke Birthing Center to enhance patient outcomes and experience through quality, safety, and efficiency initiatives. One goal of this project is to improve obstetric OR efficiency by targeting on time starts of scheduled cesarean deliveries and meeting time goals for unscheduled cesarean deliveries. PORT was tasked with developing a system to monitor and analyze metrics associated with OR utilization. In addition to time metrics of surgical events and patient flow through the center, we interlaced the cohort with age, race, ethnicity, marital status, address, and insurance status, in addition to case staffing, to understand if patient demographic characteristics or personnel were associated with our outcomes of interest. Ultimately our goal was to quantify the delay and determine potential factors that may be associated with longer delays.

Data scrambling

Owing to the quality improvement nature of the work, we scrambled all data. Therefore, while in production we use “real” data, the displayed results are not reflective of actual patient data. This work was declared exempt as quality improvement by our institution’s internal review board.

RESULTS

After an internal review of the 12 projects, we identified 5 critical themes (Table 1). While no project used every theme, each used at least 3.

Table 1.

Five identified Tableau themes

Theme	Goal	Number of projects (of 12 total)
Scheduling & optimization	Track degree of utilization over time	11
Provider assessment	Compare performance of difference providers	10
Executive summary	Magnitude of measure over time	8
Patient outcomes	Monitor patient outcomes	7
Control & target charts	Identify outliers and acceptable variance	2 (In development for other project)

Scheduling and optimization

Scheduling and optimization reports identify specified utilization targets over time. They are the most common display, featured in 11 of the 12 projects. In Figure 1, “Decision to Incision” time for unscheduled cases and in-room time for scheduled cases were key metrics to track. We displayed the goal decision to incision time for each case type (emergent, urgent, and unscheduled). If the case met the time metric, it appears green in the pie chart. If not, it is dark red. Additionally, these goals by case type show up on the control chart grid. The user can see both if the case is within acceptable variance of the mean, as well as if it was within the target goal. All cases above this target goal are shown in the detail grid. Similarly, we created pie charts and control charts to display the proportion of scheduled cases that started on time and the median delay times (data not shown).

Figure 1.

The Scheduling & Optimization theme is represented here by tracking decision to incision time by case classification. Each classification (urgent, unscheduled, emergent) has a different goal, and utilization rates over time are shown. Users can slice and dice these elements and see by surgeon, surgeon group, room, and time period what percent of their cases met the utilization goal. The outliers of these utilization goal times are provided in the detail grid.

Provider assessment

Often there is an interest in comparing outcomes across providers (10/12). This allows the users to look at variability across performance indicators. We compared on time starts for scheduled cases by provider (Supplementary Material Figure 1). A percent of total cases is shown to give a more normalized and fair magnitude of how these time groupings distribute. Based on feedback from providers, they aim to have most of their cases show up in the green section of the line graph. In the birthing center project, all providers have the ability to see other provider’s metrics but are blinded to names. In other projects, provider-specific metrics are private, and we apply appropriate filters to the displays.

Executive cohort analysis

The executive cohort analysis (8/12) typically serves as the landing page for the dashboard and shows the magnitude or intensity of measures over time. This included anesthesia start-to-finish metrics, scheduled vs start times of cesarean deliveries, or decision to incision time values. Figure 2 shows case classification tracked from August 2018 forward. Part of the project was ensuring that the front staff correctly classified cases. From the top left chart, one can see that over 90% of all cases now have a valid case classification. In addition, staff type filters were dropped in to see how the distribution of these case classifications varied by surgeon, by anesthesiologist, and primary circulator. Some basic demographic filters are also dropped in to see if there is any variance by profile. This dashboard also includes filters for demographics, such as race and financial class (insurance status).

Figure 2.

Executive Case Classification demonstrates the Executive Summary theme by showing the magnitude of Caesarian sections over time by OB level of case (urgent, emergent, scheduled, and unscheduled). Cases can be selected by time period, basic patient demographics, as well as staffing type. Most Executive Summary themes are shown at a patient level, but this is an example where the surgical case is the level of granularity.

Patient outcomes

For projects that had prespecified outcome goals (7/12), we included tailored displays to allow for simple internal evaluation and monitoring. Traditional metrics, such as the amount of return-to-OR cases, recurrent emergency department visits, 30-day readmissions, and referrals are tracked in these analyses by department and provider. A provider can log in to see a blended average for outcome rates by department or by broader categorizations of individual providers who may work on similar type cases for example. We also included patient reported outcomes such as distress and pain levels for cancer patients, and the MD Anderson Symptom Inventory score for thyroid patients. We did not need to track patient reported outcomes for the Project Birthing Center project. However, we provide an example from another project (Supplementary Material Figure 2).

Control and target charts

Our group recently integrated the use of control and target charts into projects (2/12) and have found them very useful. Control charts help identify outliers and acceptable variance for a specific value or dimension over time. Target charts allow a user to dynamically change a target metric. When adjusting this metric, the proportion of circles that are blue demonstrates cases that fall within the chosen target. (Supplementary Material Figure 3) A control chart is used to identify the acceptable range of time stamp metrics for cesarean sections such as anesthesia start-to-finish time, decision to incision time, and in room to out of room time. (Figure 3) More importantly, the control chart showed the outliers for each of these metrics, and the detail display below, for further investigation. This chart links to a separate dashboard where the staff on the case are shown, as well as all other time stamp metrics to see if being an outlier in 1 metric triggered being an outlier in other metrics. The size and color of the circle indicate cases that are outside of the acceptable standard deviation of the mean.

Figure 3.

Demonstrating the Control and Goal Theme, this variance chart takes surgical and anesthesia time stamps for surgical cases within range of the standard deviation. Cases that fall outside of the deviation are highlighted in red. The purpose of the chart allows providers to quickly see the detail of surgical case outliers, as well as see the acceptable range of values for chosen OR time stamp metrics.

DISCUSSION

Since its inception, the PORT team has provided visual analytics to 12 quality improvement projects. After reviewing our projects, we identified 5 common themes. Identifying these themes has been particularly useful for us internally. We are involved in large projects with short time frames, and reusability is key. With these themes, we can present templates that are easily customizable and can be tailored to the specific needs of each project early in development. Feedback from clinicians has been positive; the ability to interact with data on their own schedule is appreciated.

Not all projects use every theme, though each has used at least 3 of them. The most common displays are the scheduling & optimization and provider assessment. The frequency of these visuals highlights the quality improvement nature of this work. These visuals provide the means for providers to assess how care is being provisioned.

While overall Tableau has been a benefit for our institution, we have encountered some challenges. Many of these are technical in nature regarding our linking with EPIC. For example, while we are accessing multiple tables and data sources from the EHR, we do not access live data. The most recent data available is from 24 hours previously. We do this to aid performance and ensure a minimum performance for the end user. If a provider is filtering in a dashboard and it takes more than 5–10 seconds, that visualization needs to be optimized. Additionally, while Tableau has the functionality to drill down to the individual patient level, clinicians are not able to link directly to a table from the EPIC application. This functionality is technically available in EPIC but not in the instance that Duke uses (Maestro Care). Therefore, Tableau is better suited as tool for monitoring trends and changes overtime and not for managing individual patient care.

It is important to note that our work is only representative of the qualitative experience of our academic medical center. Moreover, much of this work at our institution is actively evolving. The use of Tableau has now expanded beyond the PORT Team and is now shared across wider Duke user groups. With this growing Tableau user group at Duke, we come together biweekly to share design and best practice ideas, showcase current Tableau project work, and provide walk through guides and recordings on how to build certain chart types and techniques in Tableau. We anticipate as users apply Tableau to a larger array of questions, some of these themes may evolve. We only have our own experience to use as an example, however, we think these themes have a more universal applicability and can aid in the development of visual analytic projects for other healthcare institutions.

CONCLUSIONS

We present our institution’s experience of visual analytic tools and integrating them with our EHR system via Tableau. We identified 5 visual themes that span across multiple projects. Organizing our prior work in these themes has assisted our team in further developing more high-quality tools.

FUNDING

This work was supported by the Private Diagnostic Clinics.

AUTHOR CONTRIBUTIONS

AS and TT wrote the manuscript, and ESR and CG provided clinical insights and editorial comments.

JG, GB, and WL worked on visual analytics. EGP provided editorial comments. BAG designed and edited the article.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

CONFLICT OF INTEREST STATEMENT

None declared.