A proof of concept for epidemiological research using structured reporting with pulmonary embolism as a use case

Abstract

Objective:

This paper studies the possibilities of an integrated IT-based workflow for epidemiological research in pulmonary embolism (PE) using freely available tools and structured reporting (SR).

Methods:

We included a total of 521 consecutive cases which had been referred to the radiology department for CT pulmonary angiography with suspected PE. Free-text reports were transformed into structured reports using a freely available IHE Management of Radiology Report Templates-compliant reporting platform. D-dimer values were retrieved from the hospitals laboratory results system. All information was stored in the platform’s database and visualized using freely available tools. For further analysis, we directly accessed the platform’s database with an advanced analytics tool (RapidMiner).

Results:

Results: We were able to develop an integrated workflow for epidemiological statistics from reports obtained in clinical routine. The report data allowed for automated calculation of epidemiological parameters. Prevalence of PE was 27.6%. The mean age in patients with and without PE did not differ (62.8 years and 62.0 years, respectively, p = 0.987). As expected, there was a significant difference in mean D-dimer values (10.13 and 3.12 mg l−1 fibrinogen equivalent units, respectively, p < 0.001). Conclusion: SR can make data obtained from clinical routine more accessible. Designing practical workflows is feasible using freely available tools and allows for the calculation of epidemiological statistics on a near realtime basis. Therefore, radiologists should push for the implementation of SR in clinical routine.

Summary sentence:

Implementing practical workflows that allow for the calculation of epidemiological statistics using SR and freely available tools is easily feasible.

Advances in knowledge:

Theoretical benefits of SR have long been discussed, but practical implementation demonstrating those benefits has been lacking. Here, we present a first experience providing proof that SR will make data from clinical routine more accessible.

Introduction

When considering referrals from emergency departments, CT pulmonary angiography (CTPA) for suspected pulmonary embolism (PE) is one of the most frequent radiological examinations. PE is a potentially life-threatening condition and the third most common acute cardiovascular disorder.¹ CTPA has been well-established as a mainstay in the diagnostic algorithms for PE.² However, there is concern that compliance to guidelines may be low and that CTPA is overutilized.^3–5 Prediction models could help to determine in which cases CTPA should or should not be considered, but potentially require a very large number of cases to be validated. Hence, it would be favorable if all cases from clinical routine could automatically be included in such efforts.

Traditional free-text reports are not easily accessible for further automated analysis. Therefore, data from clinical routine can often not be used to build prediction models or to compute epidemiological parameters such as the prevalence of a certain condition. Structured reporting (SR) could help solving this challenge and has recently been deemed the “future” or even the “nuclear fusion reactor” of radiology. It could be a very promising way to enhance radiological result reporting.^{6, 7} However, the implementation of SR into the daily workflow of radiologists has not been accomplished yet,^{8, 9} even though there is clear evidence for the benefits of SR.^10–14 Most major radiological societies have published recommendations promoting the adoption of SR into clinical practice.^15–17

With the recently published trial implementation of the IHE Management of Radiology Report Templates (MRRT) profile, the technical basis for SR has been well defined.¹⁸ Although at this point, integration of MRRT-compliant templates into solutions provided by major vendors is scarce, prototype applications have been described.¹⁹

In order to analyze the data collected from report databases or other clinical data repositories, specific software packages are needed. In order to enable easy access to the data for users with varying levels of expertise, visual, code-free tools could be a suitable solution. There are multiple tools available that fulfill these requirements, and technology in the field of big data and machine learning is progressing at great speed.²⁰ Also, machine learning techniques are becoming more and more common in medical research and may lead to new insights and a better understanding of prediction models.^21–24

In our work, we therefore aimed at exploring the potential of SR in radiology to make clinical and radiological report data accessible for further automated analysis. We present a proof-of-concept for a scalable workflow, where epidemiological and other parameters are calculated using only freely available tools in a well-defined clinical scenario.

Methods and materials

Patient selection

As a trial cohort to demonstrate the feasibility of our proposed workflow, we included all consecutive patients that were referred to the radiology department for CTPA with suspected PE from the hospital’s emergency departments between January 2013 and January 2015. Patient’s D-dimer values determined prior to the CTPA were obtained from the hospitals laboratory results system, while conventional written radiological reports were retrieved from the department’s radiology information system.

Structured reporting

We developed a dedicated report template for CTPA (Figure 1). This HTML5-template was developed according to the IHE’s MRRT profile¹⁸ and then imported into an open-source reporting platform, where it could be used to create structured reports. The results of these reports were then stored in the corresponding database tables. The reporting platform uses standard web-technology such as PHP, MySQL and JavaScript as well as some external libraries for Digital Imaging and Communications in Medicine communication and has already been described in the literature.¹⁹

Figure 1. .

Screen capture of the template used for reporting of pulmonary embolism. The template was written in IHE-MRRT-compliant HTML5 and then imported into our reporting platform.

The reporting platform was connected to the Picture Archiving and Communication System (PACS), so that the user could start reporting using the respective template by using a simple keyboard shortcut from within the PACS viewer. The platform queried the PACS for details on the patient and the study, allowing for the structured report to be stored and linked to the correct patient and study.

An independent user transferred the conventional written reports to the SR template without reviewing the corresponding imaging study.

Data analysis

Having the values of the structured reports stored in the platform’s dedicated MySQL-database provided for an easy access on a near real-time basis. To this end, we developed some simple access methods and integrated them into the reporting platform. A simple PHP-script calculated the current prevalence of PE across all patients for which a report had been created with the respective template. The prevalence was then displayed to the user upon accessing the corresponding webpage of the platform (Figure 2) . In order to give users the option of carrying out more in-depth and graphical analyses of the reports stored in the database, we integrated a freely available dimensional charting library (https://dc-js.github.io/dc.js/) (Figure 3).

Figure 2. .

Screen capture of the results produced by the script calculating the prevalence of pulmonary embolism amongst all reports and means for different parameters.

Figure 3. .

 Screen capture of the graphical analysis inside the structured reporting platform. Users were able to view results in greater detail for specific subpopulations by selecting either ranges of values for continuous variables or categories from the pie charts.

To allow for even more refined and complex analysis and a scalable platform, which would also be able to perform analyses with big data approaches, we accessed the MySQL report database directly with RapidMiner Studio 7.3 (RapidMiner, Cambridge, MA, USA). RapidMiner has been identified as one of the leading applications in advanced analytics.²⁵ It offers an easy-to-use graphical user interface, wiki-based contextual help, and is freely available on the company’s website (https://rapidminer.com/). As we focused on laying out an example workflow, we did not try to incorporate more complex machine learning methods, but rather focused on simple correlations and operations to manipulate the data and export aggregated results (Figure 4).

Figure 4. .

Example process for analysis of the report data in RapidMiner.

RapidMiner also provided an Excel-Output function, that allowed for aggregated results to be imported into SPSS v. 22 (IBM, Armonk, NY), where statistical tests were performed. Mann–Whitney U test was used to compare distributions between groups, diagnostic performance was calculated using the receiver operating characteristic-analysis tool.

Results

Structured reporting

In total, 521 consecutive examinations were performed in the observed period and included in our work. All studies had a written report; D-dimer values were missing in 15 cases. The report template was successfully developed and imported into the reporting platform. No problems were encountered while using the template to convert the conventional written reports into structured reports. All reports mentioned the presence or absence of PE, as well as the location and extent of the emboli and the presence or absence of right heart strain. All written reports were converted into corresponding structured reports and stored in the dedicated database. Due to the retrospective design of the study, no measures on reporting performance (e.g. time to finalized report) were taken.

Data analysis

In order to illustrate the workflow steps needed to enable further analysis of the reports stored in the platform’s database, we created an example RapidMiner process (Figure 4). First, the database connection is specified. Thereafter, MySQL-queries are used to retrieve the necessary data. As patient data and report data are stored in separate tables but using consistent IDs, joining operations were needed to build a data table where the correct patient data is assigned to the respective reports. After these preparations, unneeded attributes were removed and data was piped to an Excel-Output as well as to some steps required to calculate the means for D-dimer and age as well as a correlation matrix.

The Excel-Output was configured to only contain information on D-dimer values, presence or absence of PE and right heart strain, as well as age and Digital Imaging and Communications in Medicine Study Instance UID.

The overall prevalence of PE was 27.6%. The mean age was not significantly different for patients with PE (62.8 years) and without PE (62.0 years) (p = 0.987). However, as expected, there were significant differences in mean D-dimer values across both groups [10.13 vs 3.12 mg l⁻¹ fibrinogen equivalent units, p < 0.001]. The usually applied D-dimer cut-off value of below 0.50 mg l⁻¹ fibrinogen equivalent units yielded a negative-predictive value of 100% with a sensitivity of 100% and a specificity of 5.5%. Of those cases with confirmed PE, right heart strain was suspected in 29.2%.

All of these statistics were also visible and calculated in near real-time on the platform. Each newly generated report led to updated calculations and updated charts on the visualization page (Figure 3).

The correlation matrix revealed similar results as the statistical analysis (Figure 5). There was no correlation between age and any other parameter, whereas the presence of PE was correlated with the D-dimer value and the presence of right heart strain.

Figure 5. .

Correlation matrix for various parameters as outputted in RapidMiner.

Discussion

Today, radiology plays a key role in the work-up of most patients, not only with suspected PE, but also with a variety of other medical conditions. It is, therefore, a central junction at which clinical data converge and, together with the results of the imaging study, could be made accessible for further analysis. In our work, we present a scalable workflow where epidemiological and other parameters could be calculated using only freely available tools in an exemplified scenario of PE. We hope this work encourages radiologists to push for the implementation of SR in clinical routine, as its benefits become more and more evident and technology more and more available.

With respect to PE, multiple attempts have been made to improve the performance of prediction models and scores, which then, in turn, could help improve patient safety and management, avoid unnecessary radiation exposure and reduce healthcare costs. Today, two of the most important predictors considered for PE are D-dimer levels and Wells’ criteria.^26–28 Post-test mortality risk prediction is also being examined, mostly focusing on the Pulmonary Embolism Severity Score and the assessment of right ventricular dysfunction.^{29, 30} A significant number of these studies has been performed retrospectively, and included only a limited number of patients. However, in order to increase the performance of prediction models, the largest possible number of patients should be included—ideally, all patients examined in clinical routine.^{2, 31}

The conventional prose-like reports do not allow for easy access to the information on whether or not a certain pathology is present. This holds especially true when considering very large numbers of reports, where automated data extraction is needed. Data extraction from radiological reports, can either be performed using natural language processing (NLP) to analyze the conventional free-text reports produced by the radiologists or by means of SR. Although NLP tools have substantially improved over the past years, the a priori approach using SR seems easier to implement and integrate than the a posteriori approach using NLP.³² Moreover, SR provides the radiologist with a set of relevant items to be reported for the study being read. Even though SR does not alter the clinical management of patients with PE, there is considerable evidence that reports generated in a structured and standardized way contain more relevant information and help clinicians to make more confident decisions for further patient management.^{13, 14,33} Free-text reports, on the other hand, often do not contain certain potentially relevant information. Therefore, NLP systems, even as powerful ones as IBM’s Watson, would miss those aspects. Our method fulfills these requirements while also considering the fact that all major radiological societies promote the use of SR in clinical routine.

IHE-MRRT-compliant templates allow for easy modifications to tailor the templates to the local needs of different institutions. However, as long as the identifiers of the individual template fields remain consistent, data from different institutions should be easily comparable. Another method could be the implementation of RADLEX-terms in the template, which the MRRT profile explicitly encourages.^{18, 34}

Our work has some notable limitations. First of all, the IHE MRRT profile is still “trial implementation” and could potentially be significantly modified in the process of publication. Also, it needs to be noted that the reporting platform used in our work is still a prototype and not completely ready for widespread use in clinical routine. However, once major vendors provide commercial solutions for SR, it can be expected that the same possibilities for analysis of the report data will be available. Secondly, our work did not aim at providing an improved prediction model for patients with PE but was intended to provide a proof of concept for the proposed workflow. Although it seems highly likely that developing and improving prediction models could be facilitated by the use of SR, further studies are needed to prove this. Thirdly, we did not evaluate the potential implications of SR on clinical routine. There is some controversy in the literature with regards to the impact of SR on time needed to finalize the radiological report. While some studies suggest that more time is needed to finalize a report using SR, others showed no significant difference.^{35, 36}

The proposed workflow uses only freely available software and should easily scale to handle larger amount of data. In our example, we included the D-dimer values into the radiological reporting template, but a direct integration of data sources from other databases should also be possible. A wider range of variables integrated in the analysis could potentially help discovering new parameters that allow for a more accurate prediction of the presence or absence of PE and of the patient’s outcome. Such a setup would transform clinical routine in radiology into a sort of ongoing data collection and make retrospective analyses much easier than today, thus enabling radiology to play a key role in helping to improve patient care.

For such possibilities to be available in clinical routine radiologists should on one hand urge vendors to provide practical solutions able to integrate vendor-neutral reporting templates into their radiology information system and PACS platforms. On the other hand, radiologists and radiological societies should continue their efforts to collect and provide quality assured reporting templates that could define standard of care and help encourage vendors to develop corresponding tools.

Take-Home Points

• The integration of a complex workflow for epidemiological research into clinical practice is feasible using freely available tools and SR.

• Such workflow allows for near real-time calculation of epidemiological statistics and correlations.

• Aggregated results can be exported and would potentially allow for comparisons across department borders.

• Radiologists should push to implement SR in clinical routine.