From Testing to Decision-Making: A Data-Driven Analytics COVID-19 Response

Abstract

In March 2020, NorthShore University Health System laboratories mobilized to develop and validate polymerase chain reaction based testing for detection of SARS-CoV-2. Using laboratory data, NorthShore University Health System created the Data Coronavirus Analytics Research Team to track activities affected by SARS-CoV-2 across the organization. Operational leaders used data insights and predictions from Data Coronavirus Analytics Research Team to redeploy critical care resources across the hospital system, and real-time data were used daily to make adjustments to staffing and supply decisions. Geographical data were used to triage patients to other hospitals in our system when COVID-19 detected pavilions were at capacity. Additionally, one of the consequences of COVID-19 was the inability for patients to receive elective care leading to extended periods of pain and uncertainty about a disease or treatment. After shutting down elective surgeries beginning in March of 2020, NorthShore University Health System set a recovery goal to achieve 80% of our historical volumes by October 1, 2020. Using the Data Coronavirus Analytics Research Team, our operational and clinical teams were able to achieve 89% of our historical volumes a month ahead of schedule, allowing rapid recovery of surgical volume and financial stability. The Data Coronavirus Analytics Research Team also was used to demonstrate that the accelerated recovery period had no negative impact with regard to iatrogenic COVID-19 infection and did not result in increased deep vein thrombosis, pulmonary embolisms, or cerebrovascular accident. These achievements demonstrate how a coordinated and transparent data-driven effort that was built upon a robust laboratory testing capability was essential to the operational response and recovery from the COVID-19 crisis.

Introduction

The coronavirus pandemic of 2019 (COVID-19) is caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). COVID-19 was first detected in Wuhan, China, in December, 2019, and spread globally during the early part of 2020. As of mid-November 2020, about 55 million cases have been confirmed worldwide, with nearly 12 million cases in the United States.¹ The United States now has the most confirmed cases and confirmed deaths worldwide. One of the keys to slowing the spread of this disease is widespread testing so that patients can be quickly identified and isolated.² The publication of the first viral sequence in mid-January made the design of polymerase chain reaction (PCR) assays for SARS-CoV-2 possible.³

Covid-19 represents an unprecedented health care challenge at many levels, the most obvious of which is direct patient care. A number of downstream operational and research activities that support patient care are also crucial to enable clinicians to provide services. Analytics support allows tracking and trending of COVID-19 as well as making mathematical predictions about spread of the virus. This support is crucial to the medical professionals in infectious disease and pathology who rely on accurate data to make evidence-based decisions. In the short time span since the beginning of the pandemic, the majority documented analytics efforts have focused on global tracking and visualization of data.^4,5 Particular focus was given to interactivity and design methods such as usability, ease of use, and data visualization.^6,7 Some of the COVID spread data visualizations were also reported at the country and regional geographic levels.^8,9

While global surveillance of the disease is useful and necessary, clinicians must make on-the-spot decisions at the community and academic medical centers which they serve, and thus need local data. Comprehensive testing is necessary to achieve a data-driven response to COVID-19, yet it is insufficient without the enrichment and integration with the entirety of the clinical record. Thus, integrating data from multiple sources and applying robust rules and definitions into a single source of truth is critical for actionable analytics.¹⁰ Incorporating multiple information sources into a cohesive presentation also helps with pandemic vulnerability analysis¹¹ and utilization of data intelligence methods to predict trends such as emergence of the new hotspots of infection and its peaks.¹² A critical part of this data puzzle is an effective use of real-time clinical data that is mined from the electronic health record (EHR) and integrated into an analytical COVID-19 dashboard. Integrating real-time data is a formidable challenge requiring sophisticated technical capability,¹³ but dashboards that can combine relevant national, regional, and local data, as well as deliver information about the hospital/system with trends and prediction are invaluable support mechanisms for clinicians.

Business analytics employs the techniques, technologies, systems, methodologies, and applications for analysis of a vast amount of data and helps organizations better understand their operations.¹⁴ Clinical analytics focuses specifically on addressing health care organization needs such as quality and performance management as well as medical specialty focused analyses. Simply recognizing the need to build sophisticated analytics in support of evidence-based clinical decision-making is not sufficient to stand up a complex project grounded in rich data, complex computer tools, and the ability to collect and utilize real time data. Furthermore, the introduction of a new and robust novel disease testing apparatus brings multifaceted business and clinical challenges that require great depths in data management and analytics competencies to properly ingest, enrich, and integrate in order to make the data usable and trusted. The ability to respond to crises with on-demand analytics relies on existing maturity attained through years of talent building, data enrichment, business process improvement, and technical sophistication.¹⁵ Maturity models are health care industry standards designed to serve as guidelines for organizations to build their analytics capabilities, measure the level of development, and compare how well they stack up against others in the industry and the established industry standards of analytics.¹⁶ Some of the most trusted and established models of analytics success measurement are designed by the Health Information Management and Systems Society (HIMSS) that categorize and rank technology sophistication of the health care organizations¹⁷ based on 3 models: electronic medical record adoption model (EMRAM), outpatient electronic medical record adoption model (OEMRAM), and adoption model for analytics maturity (AMAM).¹⁸

NorthShore University Health System (NSUHS) is 1 of only 6 organizations in the world that has achieved the highest level in all 3 of these maturity models, thus displaying formidable technical capability in the processes of collecting, storing, and analyzing patient data. This effort results from years of investment into systems and human talent and has prepared the organization to supply cutting edge analytics support to clinicians. Prior to 2020, NorthShore has used clinical modeling capability for disease surveillance,¹⁹ cardiovascular risk assessment,^20,21 and diabetes care.²² In the face of the COVID-19 pandemic, our organization was properly positioned to develop a comprehensive COVID-19 tracking, data visualization, and analytical dashboard to support our clinicians.

This article describes how NorthShore achieved and capitalized upon its analytical maturity and worked in conjunction with the Pathology department to support clinicians in their critical roles as leaders and caretakers during the COVID-19 pandemic. We discuss the complex manufacturing-like processes required to take a key raw material, which is COVID-19 in-house testing, apply other raw materials from the clinical record and other sources, and deliver a finished product that supports data-driven decision-making. Community care and academic health care providers alike can use our methods to develop their own in-house or vendor partnership–based analytic solutions.

Methods

NorthShore University Health System is a 5-hospital integrated health care delivery system serving patients at over 140 locations across the Chicago metropolitan area. NorthShore University Health System utilizes SoftLab, which is integrated with the EHR, Epic, which serves all of NorthShore’s ambulatory and 4 of the 5 hospital-based populations.

Early in January of 2020, NSUHS clinical laboratories mobilized to develop and validate PCR-based testing for SARS-CoV-2. The NorthShore SARs-CoV-2 assay is a real-time reverse transcriptase (RT)-PCR assay based on the Center for Disease Control and Prevention (CDC) designed and published protocol (February 04, 2020) using primers and probe from IDT (Integrated DNA Technologies) and performed on a Roche LC480II instrument. Analytic specificity was established at approximately 5 viral genomes per RT-PCR reaction with no cross reactivity with other respiratory viral or bacterial pathogens. Clinical sensitivity and specificity was determined using contrived patient specimens across a range of viral loads, but focusing near our limit of detection (40 positive and 20 negative). Clinical samples were also tested in parallel with the Illinois Department of public health (IDPH), with 100% concordance. Our in-house assay was launched on March 12, 2020. We were among the earliest hospital laboratories in the United States and the first in Illinois to perform SARs-CoV-2 testing.

As the pandemic continued, testing was expanded by the addition of other analytic platforms: Abbott Molecular received EUA clearance on March 20 for a higher throughput COVID assay on their m2000 platform, and approval was also obtained for a higher throughput assay on the Alinity M instrument in May, further increasing our testing capacity to about 2000 per day. We also utilize rapid PCR testing platforms for SARS-CoV-2 including Cepheid GeneXpert, BD Max, and Roche cobas Liat systems.

NorthShore University Health System admitted the first positive COVID-19 patient on March 9, 2020. On that same day while the organization was mobilizing an emergency response, our analytics leadership created the Data Coronavirus Analytics Research Team (Data CART) in an effort to accumulate, analyze, and integrate the wide variety of data generated surrounding COVID-19. Composed of analytics leaders across clinical, operational, and health information technology (HIT), the goal of the Data CART was to deploy a COVID-19 analytics data hub to summarize key metrics and geographic analytics, provide real-time patient lists and reports, to be integrated into our electronic medical record, to support clinical and operations workflows, and to forecast future hospitalizations, intensive care unit (ICU) usage, and personal protective equipment (PPE) inventory. Through our experience in data analytics, we knew that without a comprehensive data engineering strategy and governance, we would not be able to fully realize the potential of our robust testing capabilities.

The foundation of the Data CART infrastructure was our laboratory SARS-CoV-2 testing data. COVID-19 lab orders are entered into our Epic EHR and transmitted to our laboratory information system Softlab via a 2-way interface. Once the specimen is collected and resulted in Softlab that information is transmitted back to the EHR. All laboratory data is stored as transactions in Epic’s production Caché database and is made immediately available in Epic’s Hyperspace application (see Figure 1). At this stage, the testing data is a raw material representing a potential value to the organization. NorthShore University Health System leveraged Epic’s Registry platform to create a real-time COVID-19 registry which is updated hourly. This is the primary engine where the raw laboratory elements are combined with the entire patient clinical record in order to create the enriched COVID-relevant information and knowledge required to develop our analytical tools and also support COVID-19 information in the EHR like banners and other clinical decision support. There have been over 90 enriched information elements created (and counting) for this specific registry ranging in complexity from patient identifiers to algorithms that determine the relevant hospitalization and SARS-CoV-2-positive active status within a particular time frame. Without this core manufacturing engine, the data are just numbers without context or any real business or clinical application. Furthermore, by performing the analytical tasks upstream in an Epic registry, we align analytical tools and decision guiding metrics with the point-of-care workflows to use a single source of truth. For example, this ensures that the same patient who is indicated as SARS-CoV-2-positive and active by our algorithm feeds the banner for COVID-19 positive status at the point-of-care as well as our positive rate metric in our Data CART dashboard.

Figure 1.

The data flow architecture. Laboratory data, originating in SoftLab, flows in to the Epic EHR where clinical and business rules are applied to transform the data in to meaningful information and metrics inside of the COVID registry, which feeds analytics and reporting within the EHR and then in to our Enterprise Data Warehouse (EDW) where further business and clinical rules and external data integration occurs. The primary analytics tools are fed from the COVID data mart in the EDW. HER indicates electronic health record.

Our analytical applications do not live in our EHR environment. The registry information is sent to Epic’s Clarity reporting database where the NSUHS Enterprise Data Warehouse (EDW) pulls information into a customized COVID-19 Data Mart where further data enrichment and additional merging with non-Epic external data occurs (eg, external claims data, census information, state [Illinois], and national COVID-19 statistics, etc). This final layer serves as our analytics data hub for a system-wide dashboard, daily reports, ad-hoc analyses, a research registry, the United States Department of Health & Human Services audit submission protocols,²³ and feeds our advanced analytics predictive tools. This entire end-to-end analytical processing updates every hour. Prior to this technical implementation, the data in our EDW was only updated each night (Figure 1).

Results

Data Coronavirus Analytics Research Team Dashboard

The Data CART dashboard has served NSUHS as the primary data hub for all key metrics and analyses supporting the COVID-19 response. The main screen (Figure 2) is updated every hour and includes the key metrics used to give our COVID-19 leadership an active view of the conditions on the ground. The main focus of the metrics is the total number of patients tested, total number of positive tests, the percentage of tests that are positive, the number of patients under investigation, and hospital census statistics. The trends show both our daily positive cases and a rolling 7-day positivity rate. All metrics can be filtered for a specified lookback period or specific population (NorthShore legacy hospitals, the newly acquired Swedish Hospital, symptomatic, and asymptomatic populations).

Figure 2.

Data CART dashboard landing page. Numbers updated in real time (hourly refresh) as depicted in Figure 1. The overall testing metrics on the first row are uploaded to a corporate-wide employee website, increasing transparency of information. Hospitalization, intensive care unit (ICU), and ventilation censuses allow operations to understand available capacity in real time. Trending data at the bottom are aggregated over rolling 7-day periods to account for weekday/weekend variation. Data were collected from the internal Epic electronic medical record system (EMR) and visualized in Tableau 2019.2. Data CART indicates Data Coronavirus Analytics Research Team.

The next most commonly used view is the hospital drill down view. This view displays trending hospitalizations, ICU admissions, and ventilations by census day which can be filtered to display 3 different views across these populations (Figure 3).

Figure 3.

Data CART trends in hospital census for all hospitalized, intensive care unit (ICU), and ventilated patients. These data use the enriched EDW source (Figure 1) to allow more precise reporting, which also feed predictive models to anticipate inpatient volume surges and allow a smooth reallocation of resources. Data were collected from the internal Epic electronic medical record system (EMR) and visualized in Tableau 2019.2. Data CART indicates Data Coronavirus Analytics Research Team; EDW, Enterprise Data Warehouse.

Since the tool went live in late March, the dashboard has been accessed nearly 20 000 times by our operational leaders and staff across the organization. In total, we have created 16 views with access to 245 users across the system. Use of the tool is comprised of leaders across HIT, Pathology, Emergency Medicine, Infection Control, and Hospital/Medical Group Operations. The information is then disseminated across the organization during daily COVID-19 operations and clinical central command calls. The metrics are also ported to the NSUHS intranet website (“pulse”) where all employees can access the information.

Geographic Surveillance and Community Outreach

In addition to the descriptive metrics, we also produce geographic data to be able to identify where clusters of activity are occurring across our region (Figure 4). NorthShore University Health System uses this information to proactively reach out to community leaders (aldermen, religious leaders, fire chiefs, mayoral office staff, and other community leaders) and share the information. An externally available tool developed in 2014 called What’s Going Around¹⁹ is made available to community leaders (school superintendents, religious leaders, community alderman, etc) for them to track the disease in their communities (Figure 5). Additionally, geographical data was used to determine whether patients deemed “persons under investigation” could be kept in their admitting pavilion when the COVID-19 dedicated pavilions were overflowing by determining if their home communities were experiencing lower than average infection rates.

Figure 4.

Data CART spatial toolset. A, Positivity % increase over past 14 days grouped by city. Shows rising hotspots based on geographical information entered by users. B, Positivity rate trend by city. Data grouped by city and week to display % positive tests. C, Positive Patient Ratio—Patient’s Home Zip. Shading represents positive % within a customizable time frame for all tests conducted within a zip code. Combined, these views offer different perspectives on geographical trends which inform staffing decisions by location. Data were collected from the internal Epic electronic medical record system (EMR) and visualized in Tableau 2019.2. Data CART indicates Data Coronavirus Analytics Research Team.

Figure 5.

What’s Going Around Application. Statistical Kriging¹⁹ processes smooth the map to create a rounded heat map effect. This application is available to the general public, allowing patients to participate in the decision-making process. Data were collected from the internal Epic electronic medical record system (EMR) and visualized using R. http://analytics.northshore.org/WGA/Home/Detail?condition=Covid

COVID Recovery of Surgical Volumes

At the peak of the Covid response, the periop team needed to weigh multiple variables in an effort to restart surgery safely. Aside from logical variables like PPE, hospital bed census for Covid+ patients was critical. With certified registered nurse anesthetists, post-anesthesia care unit nurses, and operating rooms (OR) nurses in direct caregiver roles in the Covid ICU and on the inpatient floors, knowing how many residual resources were available to safety run ORs was paramount. Monitoring the data coming from the dashboards and predictive models reassured the team that we could open the incremental rooms as the census dropped and we were able to relieve those people that traditionally worked in periop to go their “home” positions. Additionally, at the onset of the Covid response, system leaders decided to cease surgical activity at Glenbrook (our Covid pavilion) immediately and Skokie (our Orthopedic/Spine hospital) soon thereafter. This required the periop team to consolidate surgery to the remaining 2 hospital campuses. As the census rose, discussion about where Covid+ overflow patients would go became increasingly real. Using the census and projection data, we were able to take an empirical approach to the scenario planning. Without granular, real-time data it’s quite possible, we would have closed additional ORs in anticipation of continued Covid+ census growth. Availability of real data avoided relying on anecdote and prevented a decision that would have had major operational and financial consequences. Additional OR closure and postponement of cases would have caused a long wait, with further delays as we worked through the backlog before getting other surgical activity restarted. Consequently, NSUHS set a recovery goal to achieve 80% of our historical volumes by October 1, 2020. Using the data made available to our operational and clinical teams, we were able to achieve 89% of our historical volumes a month ahead of schedule, which allowed our patients to get the care they had been waiting for and our organization to quickly head toward financial recovery and stability (Table 1 and Figure 6).

Table 1.

Recovery by Service Line for Both September Year-Over-Year and an Annualized Fiscal Year Comparison.*

Case count
Service	Oct-19	Nov-19	Dec-19	Jan-20	Feb-20	Mar-20	Apr-20	May-20	June-20	July-20	Aug-20	Sep-20	YTD	Month over month			Current FY Ann. vs Previous FY final
														Sep-19	VS LY	%Var	FY20 (Ann)	FY19	Var	%Var
Cardiology	25	18	12	23	22	18	9	16	14	19	17	24	217	18	6	33%	217	197	20	10%
Cardiovascular	50	25	42	33	31	22	5	22	25	30	26	25	336	38	−13	−34%	336	474	−138	−29%
Dentistry	5	2	5	11	3	2	3	3	2	5	4	2	47	3	−1	−33%	47	54	−7	−13%
ENT	172	172	180	174	188	111	10	44	94	142	111	108	1506	136	−28	−21%	1506	2107	−601	−29%
Family /Sports medicine	0	0	0	0	0	0	0	0	0	0	0	0	0	0		0	0	28	−28	−100%
General	619	570	650	603	548	362	104	249	522	584	537	523	5871	551	−28	−5%	5871	6903	−1032	−15%
Gynecology	346	279	322	306	279	186	43	151	165	216	231	231	2755	241	−10	−4%	2755	3385	−630	−19%
Neurosurgery	138	115	129	105	104	86	46	85	110	123	117	141	1299	94	47	50%	1299	1358	−59	−4%
Ophthalmology	541	555	534	503	461	250	13	52	273	344	360	308	4194	489	−181	−37%	4194	5714	−1520	−27%
Orthopedic surgery	927	795	867	835	753	481	107	563	851	824	812	831	8646	770	61	8%	8646	9892	−1246	−13%
Pain service	70	81	58	77	69	41	0	20	60	53	43	56	628	61	−5	−8%	628	799	−171	−21%
Pediatric gastroenterology	57	56	54	74	50	34	0	36	30	57	54	53	555	48	5	10%	555	695	−140	−20%
Plastics	119	115	141	93	80	61	11	41	78	92	99	96	1026	91	5	5%	1026	1441	−415	−29%
Podiatry	47	60	37	48	32	34	3	29	35	36	33	43	437	46	−3	−7%	437	700	−263	−38%
Pulmonology	3	5	4	5	4	5	0	2	3	7	8	6	52	4	2	50%	52	31	21	68%
Thoracic	29	18	12	17	21	22	6	14	15	26	21	19	220	23	−4	−17%	220	276	−56	−20%
Urology	301	247	266	287	273	197	88	123	217	222	259	260	2740	255	5	2%	2740	3140	−400	−13%
Vascular surgery	65	61	51	63	55	56	34	39	61	59	61	67	672	56	11	20%	672	729	−57	−8%
Case total	3514	3174	3364	3257	2973	1968	482	1489	2555	2839	2793	2793	31201	2924	−131	−4%	31201	37923	−6722	−18%

Abbreviation: %Var, % variance; ENT, ear, nose, & throat (Otorhinolaryngology); FY, fiscal year; Var, variance; VS LY: versus last year; YTD: year-to-date.

* Note: NSUHS fiscal year starts October of each year. Data were collected from the internal Epic electronic medical record system (EMR) and summarized in Excel 2016.

Figure 6.

July-September 2020 (2nd bar from left) shows an 88.9% return to normal operational case volumes compared to entire FY2019 average monthly baseline and 90% compared to same year. October 2020 shows a 96% recovery compared to baseline. Data were collected from the internal Epic electronic medical record system (EMR) and visualized in Table 1.

The Data CART was also used to show that this recovery plan showed no negative impact with regard to iatrogenic COVID-19 infection or increases in deep vein thrombosis, pulmonary embolisms, or stroke. These achievements demonstrate how a coordinated and transparent data-driven effort that was built upon a robust laboratory testing capability was essential to the operational response and recovery from the COVID-19 crisis (Figures 7 and 8).

Figure 7.

Sepsis events comparing FY21 to FY20 Oct-Feb (Fiscal Year runs Oct-Sep) showing no significant difference pre and post COVID. Data were collected from the internal Epic electronic medical record system (EMR) and visualized in Cognos Analytics 11.

Figure 8.

Pulmonary Embolisms/DVT events comparing FY21 to FY20 Oct-Feb (Fiscal Year runs Oct-Sep) showing no significant difference pre and post COVID. Data were collected from the internal Epic electronic medical record system (EMR) and visualized in Cognos Analytics 11. DVT indicates deep vein thrombosis.

Discussion

Research supports the notion that business intelligence maturity enables hospital agility.²⁴ When it comes to pandemic tracking, a mixture of analytical capability and real-time data tracking creates a powerful combination that enables organizations to track their performance, challenges, and plan next steps.²⁵ Data-driven decision-making systems rely on the richness and reliability of the data inputs as raw materials, though, to achieve success. A robust in-house testing capability that is fed in to existing analytical systems provides the key ingredient to achieve a pandemic response. Using analytical maturity developed prior to the COVID-19 pandemic, NSUHS was capable of developing a sophisticated tool to empower its clinicians to make timely effective decisions. This tool, along with enriched data that supports all COVID-19 projects at NSUHS, is reusable not only to track whatever global health care challenges that may be ahead but also to power extensive research activity around COVID-19. Therefore, projects directed at fighting the COVID-19 pandemic contribute to further analytical maturity of the organization, rather than an isolated investment to solve one specific challenge, even though they originate out of this specific need. Consequently, the organization will be analytically prepared to use its advanced technology capabilities to serve operational, research, and technical goals related to public health crises and beyond, thus increasing the size and meaningful application of its toolbox that can be translated to many future advantages to come. We demonstrate how a novel and robust testing apparatus, when properly integrated with other rich data elements and enriched in to robust analytical tools, supports a timely return of health care delivery. NorthShore University Health System also demonstrated its ability to blend real-time, analytical, and predictive capabilities in a single technology offering that provides immediate points of access to all centralized data by clinicians and operations, as well as capability of taking advantage of its extensive clinical data resources on demand in a cohesive way. An important limitation of our tools is they are restricted by the populations tested by NSUHS. While NSUHS’ testing capacity was extensive it still is a sample of the entire population in the region. A fully integrated and interoperable intersystem data infrastructure would dramatically improve the tools and capabilities for health care stakeholders.