A Business Analytics Software Tool for Monitoring and Predicting Radiology Throughput Performance

Abstract

Business analytics (BA) is increasingly being utilised by radiology departments to analyse and present data. It encompasses statistical analysis, forecasting and predictive modelling and is used as an umbrella term for decision support and business intelligence systems. The primary aim of this study was to determine whether utilising BA technologies could contribute towards improved decision support and resource management within radiology departments. A set of information technology requirements were identified with key stakeholders, and a prototype BA software tool was designed, developed and implemented. A qualitative evaluation of the tool was carried out through a series of semi-structured interviews with key stakeholders. Feedback was collated, and emergent themes were identified. The results indicated that BA software applications can provide visibility of radiology performance data across all time horizons. The study demonstrated that the tool could potentially assist with improving operational efficiencies and management of radiology resources.

Introduction

It is imperative that operational efficiencies are maximised throughout our hospitals. This is especially relevant to diagnostic medical imaging departments where extended wait times can have a significant impact on determining a patient’s definitive diagnosis and treatment [1]. A primary cause for the buildup of patient wait times in radiology departments is a lack of understanding of the mismatch between capacity and demand resulting in inefficient management of radiology resources and inadequate capacity planning [2]. Despite significant volumes of data being available within the radiology department at the study site, it had proven difficult to leverage these data assets to deliver information to key decision makers in a meaningful way. The ability to visualise historic data, in addition to forecasting and modelling of future demand and capacity data, could potentially allow better anticipation of forthcoming variations and allow radiology departments to plan appropriately.

Business analytics (BA) software provides a mechanism to methodically explore and visualise an organisation’s data. It encompasses statistical analysis, predictive modelling and forecasting and is often used as an umbrella term for business intelligence and decision support systems [3]. The ability to discover meaningful patterns and identify signals and trends within datasets enables the extraction and visualisation of powerful insights from an organisation’s data assets. Data-driven decision making is increasingly associated with improved productivity and performance levels within organisations [4]. Traditional methods of summarising, viewing and reporting of data are rapidly being replaced by advanced analytics, a major disruptive innovation.

This proof of concept study hypothesised that a computerised approach to demand and capacity management, utilising BA technologies, should lead to improved decision support and more effective resource management within radiology departments. To facilitate this, it was proposed to implement and evaluate a prototype BA software tool to display a set of radiology key performance indicators (KPIs) as well as providing functionality that allowed for the forecasting and modelling of future demand and capacity data through user-defined predictive scenarios.

State of the Art

The two primary benefits of implementing a BA solution are as follows: (1) transparency and visibility of information and (2) fact-based decision support [5]. Access to and visibility of accurate and timely information gives end users the necessary knowledge to inform strategic decision making within an organisation. This potential is already being realised within radiology departments.

In the USA, a recently implemented radiology digital dashboard to support radiologist workflow has led to a significant decrease of medical image diagnostic reporting turnaround times of the order of 24 % [6]. Further, BA functionality has been used to display average turnaround times from report transcription to signoff as well as the volume of reports completed per individual radiologist [7]. Visibility of departmental workflow and associated metric data can help management identify bottlenecks in real-time, in instances where average wait times are exceeding performance targets [7].

BA software tools have also been used to extract and aggregate KPI data from a range of underlying clinical information systems reportedly leading to an improvement in management decision support, productivity, departmental performance and quality of radiology services [5]. The visualisation of radiology KPIs enables radiology staff to make more informed and accurate decisions, to improve patient care, to deliver better efficiencies to referring clinicians and to improve cost-effectiveness [8].

Access to up to date KPI information can also help to identify activities that are impacting departmental work processes, quality of service and patient satisfaction levels [9]. The ability to anticipate demand and to plan capacity accordingly has already been demonstrated to be successful within the healthcare sector [10–12]. Accordingly, while BA functionality is currently being utilised within radiology departments as a driver for decision support based on past performance [5, 6, 13], there is little evidence of the utilisation of future predictive analysis to drive decision making.

Methods

Requirements Gathering

In-depth semi-structured interviews were conducted at the study site with the clinical director of radiology, the radiology business manager and a senior medical physicist to determine a set of user requirements to inform the building of the prototype software tool. The interview questions focused on two areas as follows: (1) current KPI requirements and (2) forecasting and predictive analysis requirements. A number of prototype screens were developed in order to assist and inform user interface requirements and to act as probes to extract additional information during the requirements gathering phase. Interview transcription data was subsequently utilised to document prototype requirements and inform the software build; this has been detailed elsewhere [14].

System Design

An analytics tool was implemented to provide extract, transform and load (ETL) data transformation functionality as well as online analytical processing (OLAP) selection filtering and drill up/drill down capability. Prevedello provides a detailed description of this functionality [13]. Qlikview [15], a commercially available analytics software tool, was selected for the purposes of this study.

In order to deliver the necessary predictive scenario and forecasting functionality, a bespoke web application was developed, and the Qlikview functionality was embedded within this application. A key requirement was that the application would be mobile-enabled to facilitate access by clinicians on the hospital wards. The bespoke web application implemented the Microsoft ASP.NET MVC 5 open source model-view-controller web development framework facilitating mobile access. As a result, the application was accessible across all smart devices (tablets and phones) and was IOS, Android and Windows compatible (Fig. 1). By implementing a standalone web application, this also allowed for system access across a range of web browsers including Internet Explorer, Chrome and Firefox.

Fig. 1.

Smart device access to the RPM application

The application implemented two Qlikview models, each developed independently, one model displayed a dashboard of KPI data and a second displayed a dashboard of predictive data. Each model was developed within the Qlikview application and deployed to the Qlikview server as a URL that could subsequently be accessed from within the prototype application.

The tool utilised a combined dataset from the electronic patient record (EPR), radiology information system (RIS) and picture archiving and communications system (PACS). Magnetic resonance imaging (MRI) data was extracted for the period 2010 to 2014, anonymised, and converted to Excel format before being subsequently imported, aggregated and visualised within Qlikview. A relatively small data set was utilised and consisted of the data fields highlighted in Table 1.

Table 1.

Data set layout

Data field	Description
MRN	Unique patient identifier
Order ID	Unique order identifier
Modality	MRI, CT, ultrasound
Encounter type	Inpatient, outpatient, emergency
Exam type	Specific exam type performed, e.g. brain general, pelvis general
Order specialty code	Specialty placing order, e.g. radiology, urology, vascular
Creation date	The initial date the order is raised
Complete date	Date of exam (scan) completed
Report date	Date exam results are finalised by a radiologist
Cancel date	Date order cancelled

An overview of the prototype design is shown in Fig. 2. The orders.xlsx file is the Excel data extract and was used to populate both the KPI and the predictive analysis digital dashboards. To generate the KPI dashboard, the orders.xlsx file was imported into the Qlikview KPI model via a script which sorted and aggregated the data. The KPI data was then visualised on a data dashboard with multiple tabs displaying various associated KPI data. Similarly, for the predictive process, the orders.xlsx file was processed for historic data. This data was then imported, and a forecast algorithm was executed which generated individual forecasts for each user-defined predictive scenario.

Fig. 2.

Application design overview

Forecasts were calculated at a week number level based on forecast demand plus available capacity. Forecast demand was determined based on previous years demand plus a demand uplift percentage. Available capacity was calculated based on the number of hours that current imaging devices were available. Average scan times for each modality were then used to determine how many scans could be completed based on available capacity. The predictive scenarios then allowed the modelling of various demand and capacity adjustments to the base forecast. A detailed description of the forecast algorithm is available elsewhere [14]. Once generated, the forecast data was then imported into Qlikview via a script which visualised this data on the predictive analysis dashboard. Again a number of tabs were available to display associated forecast data.

Testing the Process

To test the KPI dashboard, a verification dataset of 300 MRI orders representing a cross section of exam types, encounter types and scan dates was selected. Each of the various KPIs selected for display was individually tested for accuracy, and particular attention was paid to non-standard functionality such as the exploding of the radiology source data into point in time snapshots for the patient wait list. Parallel testing in conjunction with data verification provided a double validation of the application’s data outputs. This was an essential component in stakeholders achieving the necessary confidence levels in the accuracy and data integrity of the application.

In order to test the forecast generation, five predictive scenarios were created with various demand and capacity adjustments entered against each. A forecast was executed, and 15 separate forecasts were generated as expected, one for each encounter type for each scenario. Adjustments for each forecast were then verified. For each week of the forecast, expected forecast adjustment values were compared to actual values to ensure accuracy. Once the forecast data was verified as being correct, the graph data was validated to ensure that the information was being visualised correctly.

A further test was conducted on the forecast data using a real-life scenario. The study site had introduced an additional MRI device during April of 2013, and data was available on the impact of this on patient wait times. A predictive scenario was created for a new MRI device duplicating the additional capacity, and the system was reset to January 2013. A forecast was then run for the 12 months of 2013, and the MRI predictive scenario data was compared to the actual live data. A deviation of just under 8 % was observed between the two data sets. This was deemed to be within acceptable limits and indicative of the general trend for patient wait times.

On completion of the prototype validation, the application was made available to radiology staff for a period of four weeks. Semi-structured qualitative evaluation interviews were then conducted with key stakeholders. The interview data collected was transcribed and analysed, and emergent themes were identified.

Results

Radiology Performance Manager

Radiology Performance Manager (RPM) was the implemented prototype software tool for monitoring and predicting radiology throughput performance. The system can be logically broken down into two modules. The first module allowed for the import and analysis of historical and current KPI data from back end hospital systems. The second module implemented the application’s predictive analysis modelling and visualisation functionality. The application home page could be accessed from any internet browser. A dropdown menu provided a clear and hierarchical view of all the main sections accessible from the home page as well as the various subsections contained within them.

KPI Dashboard

The KPI dashboard was designed based on a set of previously defined radiology KPIs [16] as well as accommodating those used in other healthcare organisations [8, 17]. Additional KPIs identified during the requirements gathering exercise were also implemented. Qlikview was utilised to implement a dashboard to visualise the KPI data. The KPI data dashboard had a number of tabs, each tab containing a collective set of associated KPIs. Figure 3 displays four of the screens included on the dashboard. Screen 1 included summary KPI details such as previous weeks order metrics, waiting list data as well as turnaround times (TATs) year to date. Screen 2 displayed radiology metrics including orders raised, patients scanned, patients reported and orders deleted. Drill up and drill down capability was also provided to year, quarter, month, week and day number level. Screen 3 displayed Median TATs including median exam TAT time, median report TAT and total median order TAT. Screen 4 visualised a graph displaying percentile wait times. The 90th percentile was always displayed (blue line), and a slider was implemented to allow further selection of the various wait time percentiles (red bars) for comparative analysis; this ranged from the 5th through to the 95th percentile. Patient wait list and order cancellation analysis screens (not shown) were also included on the dashboard.

Fig. 3.

Selection of screens from the KPI digital dashboard

All KPI data could be further filtered using the various selection criteria such as modality, dates (day, week, month, quarter or year), encounter type, order location and exam type. It was also possible to filter data by inpatient, outpatient or emergency. User selection of the various filters, or combinations of filters, resulted in the dashboard dynamically re-visualising the selected data in real-time.

The forecasting functionality within the application was implemented through the use of predictive scenarios. This predictive scenario functionality allowed for any combination of demand or capacity adjustment data to be entered against each user-created scenario down to week number level. Scenarios could be created for once-off analysis or retained to be used again within the application. The forecast algorithm then applied these scenarios to the forecast data and visualised the impact via the predictive analysis digital dashboard. This unique functionality provided for powerful user-defined predictive modelling and analysis of future forecasted radiology data. Maintenance of the applications predictive scenarios was provided by bespoke functionality (Fig. 4).

Fig. 4.

Predictive scenario maintenance, a user-defined ‘What-If’ scenario allowing entry of demand and capacity adjustments

Predictive Scenarios

The predictive scenario maintenance function allowed for the creation of user-definable ‘What-If’ scenarios for each modality at week number level. Once a scenario was created, then demand and capacity adjustments could be entered for specific weeks. Some examples of these adjustments could include an increase/decrease in number of devices available for the scenarios modality, an increase/decrease in number of radiographer’s hours available or an increase/decrease in average scan times. There was also a facility to model demand data such as allocation of additional scans or outsourcing of scans to other locations. Any combination of adjustments could be entered for each scenario. Once entered, the forecast data generated for each scenario could then be visualised using a number of graphs on the predictive analysis dashboard.

Predictive Analysis Dashboard

The predictive analysis dashboard again used Qlikview to visualise the predictive data across multiple tabs on the dashboard. Figure 5 illustrates the dashboard which consisted of four screens each displaying a unique set of predictive data.

Fig. 5.

Predictive analysis dashboard

Screen 1 visualised forecast backlog based on the various scenarios defined using the predictive scenario maintenance function (Fig. 4). All scenarios were displayed against a ‘business as usual’ forecast (blue line) which displays the baseline forecast, i.e. the expected forecast for the next 12 months assuming demand and capacity remained as expected. A drop down list box allowed for the selection of a specific predictive scenario (red line) for comparison analysis. The scenario selected (red line) in screen 1 is shown to model an additional 150 outpatient scans being placed in July; this is displayed against the business as usual scenario. The impact of this scenario is visualised graphically through a divergence between the two scenarios in July. As can be seen, the additional scan orders result in an increase in departmental backlog through to the end of the forecast year. This is a very straight forward example; however, more complex scenarios can be modelled such as additional orders being placed in July and a device being out of commission for two weeks in August. Multiple combinations of adjustments are permitted per scenario.

Screen 2 displayed forecasted demand for a selected predictive scenario with demand data visualised at month number level. Screen 3 visualised the forecasted patient waiting list to week number level for a selected scenario. Screen 4 displayed the optimum mix allocation for a selected scenario, i.e. the recommended optimum departmental inpatient/outpatient capacity allocation in order to meet forecasted demand. Optimum mix allocation data was displayed at week number level.

Again, OLAP functionality within the BA graphing tool provided selection and filtering of the data with drill up and drill down functionality to week or month level. A toggle button on each screen allowed forecast data to be displayed in either hours or number of scans, and selections were also provided by encounter type and modality.

Application Evaluation

Visibility of existing patient wait times was acknowledged as being a significant step forward in improving overall wait times. Optimum mix analysis of inpatient and outpatient capacity was also found to be especially useful in determining whether department capacity was being allocated based on real demand rather than expected demand. The application also enabled predictive modelling and analysis of average scan times which previously was not available. What was deduced from this was the significant impact that a change in scan time can have on backlog. Even very small changes were shown to have a major impact on wait times.

All interviewees were in agreement that the prototype tool could provide improved decision support. In terms of the historic KPI data, it was noted that, through visualisation of scan volumes, the tool gave advance warnings of where problems may be occurring. It was also acknowledged that the predictive scenario forecasting functionality was very useful, providing proactive decision support based on future forecasted data. The ability to display future demand was useful for planning. It was also felt that visibility of future forecasted demand was particularly useful for management of radiology resources, offering the ability to visualise scenarios a number of months into the future, thus offering a mechanism for flagging potential capacity issues. The order cancellation functionality also offered the ability to identify sources of increased cancellations and, as a result, feedback to departments with elevated cancellation levels. Whilst the tool does not solve these problems, by visualising the data, it does allow management to ask the necessary questions.

Improved staff utilisation was identified as a potential benefit. It was noted that the prototype’s interactive dashboard provided significant functionality enhancements and had a more efficient workflow than the current MATLAB tool. It was acknowledged that the prototype application was very efficient at organising and extracting data quickly. For smaller departmental projects, the ability to drill down easily through the data was a significant time saver. It was felt if the prototype was implemented and there was a level of confidence in the data being provided, then resulting time saved could be better utilised towards improved quality of care. This also had the potential to provide significant financial savings within the radiology department. The ability to analyse and track inpatient MRI turnaround times over a period of time provided a mechanism to assist with inpatient length of stay that was not previously available. This was acknowledged as having significant potential to contribute towards improved patient care and satisfaction levels. Furthermore, reduced inpatient turnaround times would also provide cost savings to the hospital. The medical imaging department is now planning to import ultrasound and CT data into the prototype tool and perform similar studies.

The feedback amongst all evaluation interviewees was that the prototype was generally easy to use. It was noted that the dashboard interface was intuitive and that very little user training was required to become familiar with all aspects of the tool. Furthermore, it was noted that the tool had the potential to be rolled out to other departments that were supply and demand based within the hospital. Similarly, it was acknowledged that predictive analysis functionality could allow modelling of future forecast data to assist with decision support across the hospital.

Discussion

Analytic applications are increasingly being implemented across the healthcare sector and have the potential to fundamentally shape the future of medicine and care delivery [18]. In order for radiology managers to make accurate, timely and fact-based decisions, it is important that they have access to relevant data at any point in time. Dashboards can deliver this information reliably and in an understandable and accessible format [8]. In this study, we demonstrate that BA software tools combined with bespoke software applications can provide visibility of radiology data across all time horizons. Historic KPI data provides retrospective analysis which can be used to inform and create predictive scenarios. These scenarios can then be utilised to generate and visualise future predictive demand and capacity data.

The predictive functionality could be useful to provide early warnings to management. Some examples cited included a time lag between a scan being completed and reported as well as a buildup of unreported studies. The latter example had already occurred at a major healthcare organisation in 2010 [19], and it was noted the prototype tool could highlight and alert these anomalies very quickly thus averting a potential crisis. Further, disaster planning, in the case where demand far exceeds supply, may be a key area where the application could be utilised [18]. In the case of lengthy periods of device downtime, the visualisation of the impact on backlog would offer a support tool for planning. Similarly, where a site has taken responsibility for managing part or all of another hospitals diagnostic imaging during hospital closure, the ability to predict the impact of additional scans being placed on a department would offer valuable management information.

BA applications are widely used across many business sectors; however, they are comparatively new within radiology and especially those delivering more advanced predictive functionality [20]. Golfarelli and Rizzi [21] discuss the potential benefit of performing ‘What-If’ simulation analysis on various datasets. To the author’s knowledge, no BA application has been designed for radiology which incorporates such future predictive analysis. Through the RPM application, complex scenarios can be created over a 12-month forecast window with combinations of demand and capacity adjustments permitted down to week number level. This allows a level of forecast generation not previously available.

Mobile radiology data access was also identified as a significant functionality enhancement. Radiology staff have a high adoption rate of mobile technology [22]. By implementing a mobile-enabled application, it was possible to provide access to important KPI data throughout the department at all times. In addition to this, clinicians could also access radiology waiting list data through various smart devices to determine current wait times for each modality. This provided a level of visibility that could assist clinician decision making on the wards without the need to contact the medical imaging department.

This study also highlighted the significant potential benefit of reducing average scan times for each modality, which has previously been identified in the literature [6]. This data is currently calculated manually based on numbers of patient’s scanned and available capacity. Once scan times can be calculated accurately, there is significant scope for improving the time taken for each of the various steps during the scan process. This data would also provide more accurate forecasting capability.

Analytics technologies are being rolled out extensively across the private sector in recognition of the improvements delivered in operational efficiencies and decision support. As a consequence, significant returns on investment are being realised [23]. This study has highlighted the ways in which BA technologies can also deliver these benefits within the healthcare domain. BA is a disruptive innovation; such innovations tend to sneak in from below through individuals keen to introduce immediate improvements for their departments. This bottom-up rollout of technology very often leads to widespread adoption. While disruptive innovations have the potential to introduce fundamental changes within the healthcare sector, it is imperative that these innovations are encouraged, supported and rewarded. Hence, national policy needs to open up to disruptive technologies and business models that challenge the status quo and have the potential to improve the quality of healthcare delivery nationwide.

It is further imperative that small isolated ICT solutions adhere to overall hospital strategy and are compliant with necessary standards. It is equally important that these systems are rolled out to multiple locations within the organisation as soon as problems are solved. This ensures that immediate improvements in one department are made available to multiple departments in order to deliver maximum impact.

Limitations to the research should also be noted. The system evaluation was based on the feedback of three people. Ideally, a larger cohort would have participated in the evaluation phase of the prototype application. Similarly, a longer evaluation phase would have permitted an assessment of the clinical impact to determine whether empirical data in the form of KPIs was improving over time. It should also be acknowledged that although the application used a live dataset, it was not deployed in a live environment. A further study is required to determine the actual benefit delivered by the application over a period of time. In order to conduct a study of this nature, it would be necessary to have the prototype system in use for a sufficient period of time. Quantitative measures of wait times pre- and post-implementation would help identify evidence of benefit. Similarly, a more detailed quantitative study comparing forecasted patient backlog to actual backlog across a range of predictive scenarios would help determine accuracy and provide evidence of benefit of the predictive analysis functionality.

Conclusion

In this study, we designed and developed a proof of concept standalone bespoke web application that utilises embedded BA functionality in order to display past performance and future predicted KPIs for radiology. The study has demonstrated the numerous potential benefits of implementing BA technologies within medical imaging departments. Novel predictive scenario functionality was implemented to allow user-defined ‘What-If’ scenario modelling of forecast data. Subsequent evaluation of the tool demonstrated the potential of BA applications to deliver improved management of patient wait times and improved operational efficiencies. Future work is required to determine the actual benefit delivered by the application over a period of time.

Investment in multiple hospital software systems to streamline existing processes has led to a significant increase in the volume of available data; however, despite this, healthcare agencies remain predominantly data-rich and information-poor. The focus must now shift to the leveraging of these data assets in order to analyse and present key information to decision makers in a meaningful way.

BA software tools deliver such functionality, enabling healthcare organisations to gain foresight and insight that can transform medical data into clinical knowledge. Access to this information can assist with more effective decision making, thus driving improved performance and clinical effectiveness.