Abstract
Growth in the field of medical imaging research has revealed a need for larger volume and variety in available data. This need could be met using curated clinically acquired data, but the process for getting this data from the scanners to the scientists is complex and lengthy. We present a manifest-driven modular Extract, Transform, and Load (ETL) process named Locutus designed to appropriately handle difficulties present in the process of reusing clinically acquired medical imaging data. The design of Locutus was based on four foundational assumptions about medical data, research data, and communication. All parts of a workflow must communicate with each other and be adaptable to unique data delivery requests. In addition, the workflow must be robust to possible errors and uncertainties in clinically-acquired data, which may require human intervention to resolve. With these assumptions in mind,Locutus presents a five-phase workflow for downloading, deidentifying, and delivering unique requests for imaging data. The phases include initialization, data preparation, extraction of data from the research server to a pre-deidentification data warehouse, transformation into deidentified space, and loading into post-deidentification data warehouse. To date, this workflow has been used to process 32,962 imaging accessions for research use. This number is expected to grow as technical challenges are addressed and the role of humans is expected to shift from frequent intervention to regular monitoring.
Introduction
The artificial intelligence revolution has put the need for access to “big data” at the forefront of medical science. In medical imaging, most big data studies are performed on a limited number of publicly available research quality datasets. While these datasets are valuable and highly useful, a wider variety of data is needed.1 There remains a vast amount of untapped data stored by the radiology departments of hospital systems, and previous work has demonstrated the value in curating clinically acquired data for research purposes.2 - 4
The process of reusing clinically acquired data for research uses is no small task. Generally, it starts with a request for data from a given health-service providing department, involves review and often removal of any information that could be used to identify the patient (per Institutional Review Board protocols), and ends with the researcher receiving a subset of their requested data. With structured electronic health record data, the removal of patient health information (PHI) is relatively straightforward, and requests have the potential to be fulfilled quickly. The ease of PHI removal decreases as the complexity of the individual data points increases: Semi-structured or unstructured text and medical imaging data are much more difficult to deidentify.
Focusing on medical imaging, researchers often undergo lengthy processes to request clinically acquired scans from the radiology department’s picture archiving and communications system (PACS). The radiology department’s primary service tier is for clinicians providing patient care; fulfilling research requests is a secondary priority. Department members are only able to download scans for researchers after addressing any clinical responsibilities. Once the scans have been downloaded from the PACS, in the form of Digital Imaging and Communications in Medicine (DICOM) files, they often must be stripped of all PHI before delivery to researchers. This description is an oversimplification of the many manual steps often involved in deidentifying clinically acquired medical images. Existing approaches do not scale well with even small increases in demand, let alone in the age of big data.
The purpose of this paper is to describe the infrastructure developed in one pediatric health system to facilitate the flow of DICOM files through a high throughput metadata and pixel deidentification workflow from the radiology PACS to medical imaging researchers. We introduce Locutus, a manifest-driven, modular ETL tool. Locutus was developed to facilitate communication with every necessary party, to integrate applicable software tools during the ETL process, and to deliver deidentified DICOM data in a timely manner to medical imaging researchers without imposing high load on clinical resources.
Methods
Locutus was designed with four foundational assumptions in mind:
All parts of a workflow must communicate with each other.
Each data delivery request is composed of a unique set of accessions and may have unique deidentification needs.
Clinically acquired data is not guaranteed to be completely error free.
A human must be kept in the loop to deal with uncertainties.
In the remainder of this paper, we reference these assumptions to demonstrate their importance in designing Locutus and in how Locutus operates.
These assumptions broaden the scope of Locutus from a simple platform-agnostic Extract, Transform, and Load (ETL) pipeline to a multistage workflow. Before Locutus can begin handling a medical imaging request, a great deal of preparation and communication occurs between the Locutus team, the radiology team, and the research team. The first two phases within the Locutus workflow consist of configuring the settings for later phases to use for the data ETL and confirming all data requested by the research team is available. The latter three phases of the workflow are where the extraction of identified DICOM files, transformation from the identified space to the deidentified space, and loading the deidentified DICOMs to the target location takes place. An overview of these phases and how they fit in interactions between different people in different departments can be seen in Figure 1. In the remainder of this section, we describe these phases in more detail.
Figure 1.
Overview of the Locutus workflow. The purpose of the workflow is to deliver a set of deidentified scans requested from the radiology department picture archiving and communications systems (PACS) to the researcher. Coordination between the research team, the radiology team, and the Locutus team enables Locutus to process a set of scans that are first copied from the Clinical PACS to a Research PACS. Locutus loads the delivery-specific system settings from the manifest in Phase 1, prepares to pull the requested scans from the Research PACS in Phase 2, extracts the requested scans to an identified DICOM dataset in Phase 3, transforms the identified data into a deidentified space in Phase 4, and loads the deidentified data into a deidentified DICOM dataset in Phase 5. Locutus itself can be conceptualized as a water wheel, where the data flowing through it is a collection of individual water molecules. The buckets next to Phases 3-5 represent the interim versions of each DICOM file as it flows through Locutus.
Pre-Locutus: Request Preparation. Before Locutus is engaged, several preparations must be made, as illustrated outside of the Locutus box in Figure 1. First, the research team must submit an imaging request to the radiology department and the Locutus team. The request manifest requires a minimum viable amount of information needed to find the accessions for the requested data in the clinical PACS. This information must include the list of “accession numbers” (identifiers corresponding to the desired scan sessions). Additional information such as year of scan, patient age at scan, and patient sex are also included if possible. Since Locutus assumes that no clinically acquired data is guaranteed to be completely error-free, these additional pieces of information can be used to resolve erroneous DICOM accessions.
Once the radiology team receives the radiology request, they locate the scans in the clinical PACS. To prevent excess demand on the clinical PACS by downloading data for research purposes, the radiology team sends the requested data to a parallel research-facing PACS, where each accession is assigned a universally unique identifier (UUID)5. The sending of data from the clinical PACS to the research PACS is unidirectional: the research PACS only contains data that has been shared with it by the radiology team and cannot access data stored in the clinical PACS. When the radiology team has delivered all available requested data to the research PACS, they notify the Locutus team of the update.
The Locutus team runs a “staging” process to index the information about the latest DICOM data delivered to the research PACS. This stager index consists of several tables that link the UUIDs for each DICOM accession with a subset of PHI-free DICOM metadata tags. The stager index serves as the bridge between how the research PACS views DICOM accessions and how Locutus plans to process each individual DICOM file. Later in Phase 3, Locutus will use the stager to look up the correct UUIDs on the research PACS associated with each accession number in the request manifest.
Phase 1: Initialize Locutus. While easy to overlook, the foundational assumptions that governed the design of Locutus place great value on the initialization phase. When the Locutus team confirms with the radiology team that all available clinical PACS data from the radiology request is available in the research PACS, they begin the first phase of Locutus’ operation. Locutus first validates the file format of the request’s corresponding input manifest and version. Then, Locutus configures the modules for each phase based on the needs of the request. Finally, Locutus checks that connections are available to the stager index, the research PACS, and all external modules used in later Locutus phases.
Phase 2: Prepare to process data. Once the connections between Locutus’s modules, the research PACS, and the stager index are stable, Locutus must check the state of the stager index against its own internal index of DICOM data changes. There are three possible outcomes for the comparison of these two indices for each accession in the manifest: (1) new DICOM data has been pushed to the research PACS that does not yet exist in the Locutus index; (2) changes have been made to DICOM data on the research PACS that previously existed in the Locutus index; or (3) the research PACS and the Locutus index are consistent with each other. For the two former options, updated stager data must be migrated into the Locutus index before proceeding. In keeping with the four fundamental assumptions, the two indices are compared a second time and the results are verified by a Locutus team member.
Locutus then creates an internal state machine representation of the accession numbers in the request manifest. If there are any accessions included in the manifest but not documented in the stager, Locutus warns the Locutus team member of the missing data or unexpected situations (e.g., multiple UUIDs for single accessions). The Locutus team member coordinates with the radiology team to address any missing data or inconsistencies, then restarts the Locutus workflow at Phase 1. Once Locutus has confirmed that the most recent changes to the requested DICOM files have been pulled into the internal index, the Locutus team member initiates the next phase.
Phase 3: Extract data from the research server into a pre-deidentification data warehouse. At the start of this phase, Locutus examines the accessions included in its input manifest. For each accession number in its manifest, Locutus transfers the DICOM files from the research PACS to a local, pre-deidentification data warehouse. This “pre-deid” warehouse is for staging and internal Locutus use only: it is never delivered directly to the researchers. At the end of Phase 3, a phase-wide sweep is performed to determine the state of the data in the pre-deidentification warehouse. A summary of the data availability is tabulated and shared with the Locutus team member who initiated the workflow for the manifest. The Locutus team member restarts and resolves any stalled or pending file transfers and reruns the process to summarize the state of the pre-deidentification warehouse data. The Phase 3 is considered complete when all accessions in the input manifest are confirmed to be present in the pre-deidentification warehouse.
Phase 4: Transform data into deidentified space. The nature of medical imaging simultaneously makes this transformation step easier and more difficult. An overview of this phase can be seen in Figure 2. The DICOM standard includes a large number of patient- and scan-related metadata fields. The primary challenge at this stage in the deidentification process are in determining the balance between removing enough DICOM tags to minimize the risk of potential PHI leakage and preserving any DICOM tags required for specific modalities or scan vendors.
Figure 2.
Schematic of Locutus Phase 4: Transform data into deidentified space. The process of removing patient health information from DICOM files requires several levels of deidentification. In addition to metadata filtering using the DICOM tag remove and keep lists, one of several pixel-level deidentification software modules may be used to redact burned-in pixel-based PHI.
General medical imaging deidentification workflows have a strict keep list of DICOM tags and redact any field not included in that list. Many of these metadata tags are easily identifiable as containing protected patient health information (PHI) such as “PatientName”, “ReferringPhysician”, and “PatientAddress”. Locutus can include these and other related tags in a universal “remove list” applied to all DICOM data for all research requests; however, this strict approach is not nuanced enough to be applicable to all types of medical imaging. Different imaging modalities use DICOM tags to store information about the protocol used to acquire a scan. A strict approach can strip PHI-free scan metadata vital for research analyses. To mitigate problems due to metadata redaction in modality specific analyses, Locutus allows researchers to build a custom DICOM tag keep list with the Locutus team. During Phase 4, the approved custom keep list is checked against the universal remove list, and any requested keep list tags not present in the remove list are preserved in the metadata.
DICOM metadata redaction removes PHI from potentially messy, but still structured parts of medical images. In addition to PHI in the DICOM tags, there is often patient or provider information embedded in the pixels of the image data itself. Burned-in information most often occurs in 2D scans such as ultrasounds or x-rays in the black border around the imaged tissue. It can also occur in volumetric imaging data: 3D scans of the head by their very nature include the patient’s face, and in extremely rare cases, a scan may capture the serial number of a medical implant. To deal with burned-in PHI, Locutus must work with one of several commercially available software modules that are able to detect and remove alphanumeric characters embedded in imaging data. Two popular commercially available embedded text removal tools are Google Cloud Health API’ and AWS HealthImaging’. Additionally, several research groups have developed tools for removing facial features from 3D medical images8; these tools can be set up as modules on local or cloud computing resources. The exact pixel-level deidentifier software module requires careful consideration of the type of images being processed as well as the time and monetary costs associated with each module.
There are several commercially available cloud-based software tools that can be used to meet the need for pixel redaction.6- 7 These tools require the user to upload a scan containing PHI to a HIPAA compliant server, run selected deidentification processes on the scan, and push the deidentified scan back to the user. From Locutus’s perspective, the key limitation of these cloud-based tools is the need for two transfers of each file: one upload and one download. Any changes to network status or network configurations that allow remote commercial servers to connect to local institutional servers can halt the flow of DICOM data. For a large request manifest, network changes can cause many accessions to stall during the deidentification phase, so we have invested significant enhancements around helping to ensure a robust workflow.
During Phase 4, Locutus performs phase sweeps of any accessions in its manifest to monitor the state of the deidentification processes. It compiles a status table containing information about the progress of each accession. These statuses are shared with Locutus team members. The Locutus team reviews the summaries to identify which scans were successfully deidentified and which scans were stalled in the deidentification pipeline. The Locutus team members can then determine the cause of stalled deidentifications and modify or restart the processes.
Phase 5: Load deidentified data into a post-deidentification data warehouse. Regardless of the specific software tool used to deidentify the DICOM data, the deidentified data is loaded into an internal, post-deidentification data warehouse. The summary of the delivered data is shared with the Locutus team member who began the workflow for the manifest, who may then have some project-specific post-Locutus steps to prepare the data for delivery, then notifies the research team that their deidentified data is ready for use.
The code for our implementation of Locutus will be posted in a publicly available GitHub pending review by the Office of Technology Transfer.
Results
As of January 2025, Locutus has facilitated the deidentification and delivery of over 32,9’2 clinically acquired medical imaging scans. The cumulative number of scans that have been processed via Locutus since its initial deployment can be seen in Figure 3. In its lifespan, the number of scans Locutus can process has increased from several hundred scans every three to six months to over a thousand per month. Considering the cost of different types of research scans, for instance $500-$1500 for a 1-hour NMI scan in most research settings (not to mention the cost of patient recruitment and remuneration), the use of deidentified medical images allows for large studies at a fraction of the cost.
Figure 3.
The number of patient scan sessions processed via Locutus at time of submission of the present paper. A) The barplot shows the discrete number of accessions processed by Locutus at each date. Accessions refer to identifiers that correspond to patient scan sessions. B) The blue line shows the cumulative number of accessions processed by Loctus. As Locutus has increased in scale, it has facilitated the deidentification and delivery of 32,962 clinically acquired scan sessions.
Two specific research teams were followed up with regarding the quality of the deidentification. The first team performed a semi-supervised examination of a subset of 3,840 neurological MRI scans for PHI. No PHI was reported to exist in the DICOM headers by the investigating data analysts. In fact, 2,214 of those scans were found to be missing non-PHI DICOM metadata required for modality specific analyses, resulting in modification of that project’s Locutus keep list to retain this metadata. The second team randomly performed a cursory examination of approximately 800 of 1,110 thoracic CT scans with 132 scans undergoing detailed manual examination, finding no evidence of PHI.
Discussion
Recent growth in the field of medical imaging research has revealed the need for a wider variety of imaging data sources. Previous work has demonstrated the potential value of clinically acquired data for research applications2 - 4. The process of cleaning clinical data for use in research is not “one size fits all” and can consume a great deal of time and manual effort. We have developed a manifest driven medical imaging ETL workflow named Locutus based on four foundational assumptions: each request for imaging data is unique, communication between phases of a workflow is vital, clinically acquired data cannot be guaranteed to be error-free, and human oversight is essential.
Locutus was designed to navigate a complex clinical and research infrastructure, as efficiently as possible, while obeying the foundational assumptions above. Locutus first establishes an internal understanding of the state of all available data. Locutus then coordinates with the research PACS to update its scan index before extracting DICOM data containing PHI, deidentifying the DICOM data at both the metadata and pixel levels, and loading the deidentified DICOM data into the configured target data store for any further post-Locutus project-specific steps, and ultimately delivery to the requesting researcher team. The Locutus workflow has already resulted in the delivery of over 32,000 deidentified clinically acquired scans to researchers. As interfaces between component software modules are refined, this number is expected to increase exponentially.
Workflows such as Locutus face numerous challenges based on the nature of clinically acquired medical imaging data. A great deal of consideration is given to the best way to balance the need for patient privacy and the need for analysis pipeline-specific metadata preservation. Both needs are important, and the priority for a healthcare center is to ensure patient data is protected. Related works highlight the importance of preventing PHI leakage10. Locutus errs on the side of protecting PHI through the use of a strict default metadata tag remove list in combination with custom keep lists. The custom keep lists allow the Locutus team to take advantage of requesting researcher domain expertise by permitting the preservation of non-PHI DICOM tags that are required for downstream analyses. Additional concerns about patient privacy and the use of cloud-based tools9 are addressed by performing DICOM metadata deidentification prior to the use of any cloud-based software for pixel-level redaction.
Another challenge is the difference between how clinical data is theoretically entered, stored, and made available and how it is actually entered, stored, and made available in practice. An example of this limitation can be seen in the identification of a false “one-to-one” assumption. In the early stages of Locutus, the demand was relatively minimal and it was assumed that for every accession Locutus internally indexed, there would be one and only one UUID present in the research PACS. Closer inspection revealed that there are two situations that can result in a single accession having multiple UUIDs in the research PACS. In one scenario, a single accession number may be split across multiple UUIDs within the clinical PACS itself. In the second, an accession can be sent from the clinical PACS to the research PACS more than once: in these situations, the additional copies of the same accession will have different UUIDs. Both of these scenarios can be managed using the Locutus workflow. Furthermore, once the Locutus team works with the radiology team to identify which scenario is the root cause of the multiple UUIDs, they can remediate the issue at the source (whether in the clinical PACS or the research PACS).
Many of the challenges in implementing Locutus have been technical in nature and are the main focus of planned future work. Limited disc space can be addressed relatively cheaply but requires support and approval of other departments. Network availability and the amount of time needed to upload and download data to commercially available cloud computing tools in Phase 4 pose larger challenges9. These could be addressed through the development of software modules that could be run in-house without the need to transfer data for pixel redaction. Currently, a on-premises (on-prem) deidentification process can be used for certain manifests where the pixel-based deidentification of the is determined to be unnecessary. Expanding the functionality of this on-prem module to include reliable pixel-based identification would reduce the network load and processing time in Phase 4. Though they would not impact the internal functionality of Locutus, changes from one PACS platform to another may require changes to how images are loaded from the research PACS into the stager index.
These areas for improvement are not limited strictly to the Locutus workflow. In the preparation phase, tools could be developed to aid the radiology team in their manual efforts needed to deliver clinical images to the research PACS, or to aid research teams in the process of cohort identification. For instance, exposing a copy of the metadata of images stored in the clinical PACS would enable research teams to construct more informed radiology requests that result in fewer downstream technical issues. These types of advancements would enable the radiology team to recycle more clinically acquired medical images for research purposes, more quickly, without impact on clinical operations.
Conclusions
The need for tools to facilitate the sharing of clinically acquired data with researchers is becoming more and more apparent. There are extended challenges when the clinical data of interest is medical imaging data. We described a manifest driven ETL workflow focused on facilitating communication between parties involved in the process and protecting patient information while accounting for nuances in clinical data as well as different research needs. While the amount of time and the effort needed to develop a stable and sustainable ETL workflow can be a significant initial commitment for any organization, Locutus demonstrates the significant time and money savings that can result in over a period of even just a few years.
Acknowledgements
The authors would like to gratefully acknowledge the contributions of Alex Felmeister, who developed the earliest prototype of Locutus using a Jupyter notebook to anonymize a set of scans physically delivered via compact disc and inspired this entire project; Jeff Miller, who developed the open-source ‘dicom_anon’ module utilized in both in the original Jupyter notebook and in subsequent Locutus DICOM deidentification modules; and Juan Giarrizzo, who championed the efforts to extend Locutus DICOM deidentification modules’ capability by integrating with commercially available cloud-based platforms. This work would not be possible without the oversight of the Arcus data privacy analyst Dianna Reuter. This work was funded by NIMH R01MH134896 (PI, AAB) and by the Children’s Hospital of Philadelphia Research Institute.
Disclosures
AAB has received consulting income from Octave Biosciences and holds equity in Centile Biosciences.
Figures & Tables
References
- 1.Morris MA, Saboury B, Burkett B, Gao J, Siegel EL. Reinventing radiology: Big data and the future of medical imaging. J Thorac Imaging. 2018 Jan;33(1):4–16. doi: 10.1097/RTI.0000000000000311. doi: 10.1097/RTI.0000000000000311. PMID: 29252898. [DOI] [PubMed] [Google Scholar]
- 2.Schabdach JM, Schmitt JE, Sotardi S, Vossough A, Andronikou S, Roberts TP, Huang H, Padmanabhan V, Ortiz-Rosa A, Gardner M, Covitz S, Bedford SA, Mandal AS, Chaiyachati BH, White SR, Bullmore E, Bethlehem RAI, Shinohara RT, Billot B, Iglesias JE, Ghosh S, Gur RE, Satterthwaite TD, Roalf D, Seidlitz J, Alexander-Bloch A. Lifespan Brain Chart Consortium. Brain growth charts for quantitative analysis of pediatric clinical brain MRI scans with limited imaging pathology. Radiology. 2023 Oct;309(1):e230096. doi: 10.1148/radiol.230096. doi: 10.1148/radiol.230096. PMID: 37906015; PMCID: PMC10623207. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Ertl-Wagner BB, Pai V. Broadening the scope of normal control images in pediatric neuroimaging-and possibly beyond. Radiology. 2023 Oct;309(1):e232598. doi: 10.1148/radiol.232598. doi: 10.1148/radiol.232598. PMID: 37906004. [DOI] [PubMed] [Google Scholar]
- 4.Amin J, Anjum MA, Gul N, Sharif M, Kadry S. Clinically acquired new challenging dataset for brain SOL segmentation: AJBDS-2023. Data Brief. 2023 Dec 7;52:109915. doi: 10.1016/j.dib.2023.109915. doi: 10.1016/j.dib.2023.109915. PMID: 38229924; PMCID: PMC10790026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Davis K, Peabody B, Leach P. Universally Unique Identifiers (UUIDS) RFC 9562. May 2024 doi: https://doi.org/10.17487/RFC9562 . [Google Scholar]
- 6.Google Cloud De-identify DICOM files. Google Cloud. https://cloud.google.com/healthcare-api/docs/how-tos/dicom-deidentify . Accessed September 10, 2024. [Google Scholar]
- 7.Amazon Web Services AWS HealthImaging features. Amazon Web Services. https://aws.amazon.com/healthimaging/features/ Accessed September 10, 2024. [Google Scholar]
- 8.Schwarz CG, Kremers WK, Wiste HJ, Gunter JL, Vemuri P, Spychalla AJ, Kantarci K, Schultz AP, Sperling RA, Knopman DS, Petersen RC, Jack CR Jr. Alzheimer’s Disease Neuroimaging Initiative. Changing the face of neuroimaging research: Comparing a new MRI de-facing technique with popular alternatives. Neuroimage. 2021 May 1;231:117845. doi: 10.1016/j.neuroimage.2021.117845. doi: 10.1016/j.neuroimage.2021.117845. Epub 2021 Feb 11. PMID: 33582276; PMCID: PMC8154695. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Javid K, Luqman S. Cloud-Based Medical Imaging: Enhancing Healthcare Diagnosis and Treatment. 2023 https://www.researchgate.net/publication/372826216_Cloud-Based_Medical_Imaging_Enhancing_He althcare_Diagnosis_and_Treatment. Accessed January 14, 2025. [Google Scholar]
- 10.Mesterhazy J, Olson G, Datta S. High performance on-demand de-identification of a petabyte-scale medical imaging data lake. arXiv preprint. 2020 Aug 4 https://doi.org/10.48550/arXiv.2008.01827. [Google Scholar]