The RSNA Abdominal Traumatic Injury CT (RATIC) Dataset

Abstract

Supplemental material is available for this article.

Summary

The RSNA Abdominal Traumatic Injury CT (ie, RATIC) dataset contains 4274 abdominal CT studies with annotations related to traumatic injuries and is available at https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection and https://imaging.rsna.org/dataset/5.

Key Points

• ■ The RSNA Abdominal Traumatic Injury CT (ie, RATIC) dataset is the largest publicly available adult abdominal traumatic injury CT dataset, with contributions from 23 institutions across 14 countries and six continents.

• ■ The dataset consists of medical images, segmentations, and image-level annotations, which were generated by subspecialist radiologists from the American Society of Emergency Radiology and the Society of Abdominal Radiology.

• ■ This dataset was used for the Radiological Society of North America 2023 Abdominal Trauma Detection competition and is made freely available to the research community for noncommercial use.

Introduction

Trauma is the most common cause of fatal injuries in the United States among individuals younger than 45 years. Globally, an estimated 6 million individuals die of traumatic injuries each year (1). Early accurate diagnosis and grading of traumatic injuries is critical in guiding clinical management and improving patient outcomes. CT plays a central role in the initial evaluation of hemodynamically stable patients (2,3). For blunt and penetrating abdominal trauma, the American Association for the Surgery of Trauma (AAST) organ injury grading system is the most well-recognized system for grading solid organ injuries (4,5) and is critical in triaging patients for surgery, minimally invasive intervention, or conservative management (6).

While the AAST grading system is an important guide for assessing solid organ injury, rapid interpretation of trauma studies is challenging given the large number of images to review and the potential for subtle findings. Diagnostic errors in the interpretation of trauma are common (7), and there is high interrater variability in the AAST grading system (8,9). Furthermore, the large variation in protocols used at different hospitals, including a single portal venous phase, multiphasic imaging, and split bolus approaches (10), can further complicate this task.

Automated assessment of traumatic abdominal injuries is an excellent use case for artificial intelligence (AI) algorithms, given the potential to prioritize studies that may require more expedient interpretations, as well as to augment radiologist accuracy and efficiency, which may be particularly valuable in areas where subspecialists are in short supply. Recent work on AI-based assessment of abdominal trauma includes studies on automated detection of splenic (11–14) and liver (15) injury, hemoperitoneum (16), and pneumoperitoneum (17). However, prior studies have typically been limited in scope to single organs and single institutions, limiting generalizability into clinical practice. Thus, there is a need for large multi-institutional publicly available annotated abdominal trauma datasets to address this challenge.

The Radiological Society of North America (RSNA) collaborated with the American Society of Emergency Radiology (ASER) and the Society of Abdominal Radiology (SAR) to curate a large, publicly available expert-labeled dataset of abdominal CT images for traumatic injuries focusing on injuries to the liver, spleen, kidneys, bowel, and mesentery and active extravasation. This dataset was used for the RSNA 2023 Abdominal Trauma Detection competition, which attracted 1500 competitors from around the world to develop innovative machine learning (ML) models that detect traumatic injuries at abdominal CT.

Dataset Curation and Annotation

Figure 1 shows a flowchart of the RSNA Abdominal Traumatic Injury CT (RATIC) dataset curation and annotation process, with a detailed description provided in Appendix S1. In brief, sites provided initial labels for the presence of different traumatic injuries according to clinical reports. Radiologist annotators recruited from the ASER and SAR then independently annotated solid organ injury grades and locations of bowel and mesenteric injuries and active extravasation. The annotator pool consisted of 43 attending radiologists (32 academic, four private practice, six hybrid, and one government), with 10.2 years ± 6.9 (SD) of experience as an attending, and two fellows from 38 different institutions. Annotator subspecialties consisted of 19 abdominal radiologists, 21 emergency radiologists, four general radiologists, and one interventional radiologist. Annotators annotated an average of 144 cases ± 123. Reference standard labels for the grading of each solid organ injury were established using majority voting among three different annotators and divided into low- (AAST I–III) and high-grade (IV and V) injury groups. In the event of label disagreement among annotators, a member of the organizing committee acted as an adjudicator. Image-level labels for bowel and mesenteric injuries and active extravasation were based on the consensus of different annotators. Voxelwise segmentations (Fig 2) were manually corrected after training an nnU-Net (18) on the TotalSegmentator dataset (19), focusing only on the organs being evaluated in the challenge: liver, spleen, left kidney, right kidney, and bowel (representing a combination of esophagus, stomach, duodenum, small bowel, and colon).

Figure 1:

Summary of the data curation and annotation process. * = bowel and mesenteric injuries were reviewed by two annotators. DICOM = Digital Imaging and Communications in Medicine.

Figure 2:

Example of abdominal organ segmentation, with each color representing different organs. (A) Axial CT DICOM image demonstrates a splenic laceration (arrow). (B) Image illustrates the segmentations for the liver (red), spleen (green), left kidney (blue), and gastrointestinal tract (brown) in the axial plane. (C) Image shows segmentation masks overlaying the corresponding CT image. (D) Image shows segmentation masks overlaying the corresponding organs on a reconstructed coronal CT DICOM image. DICOM = Digital Imaging and Communications in Medicine.

Dataset Description and Usage

The RATIC dataset is composed of CT scans of the abdomen and pelvis in 4274 adult (≥18 years) patients, with a total of 6481 image series from 23 institutions across 14 countries and six continents. A detailed breakdown of patient demographics and injuries across the different institutions is provided in Table 1. The demographic and case-level composition of the competition training and test set is presented in Table 2. The breakdown of injury severity of solid organs for the training set is found in Table 3.

Table 1:

Distribution of Positive and Negative Cases for Abdominal Injury with Breakdown of Injury Class per Institution

Table 2:

Distribution of Demographic and Case-level Breakdown for Abdominal Injury across Dataset Training and Test Subsets

Table 3:

Distribution of Injury Grades for Training Portion of Dataset

CT images are in Digital Imaging and Communications in Medicine (ie, DICOM) format. Study-level injury annotations and demographic information are provided in four comma-separated value files. The train_2024.csv file contains information about the presence of traumatic abdominal injuries (liver, kidney, spleen, bowel, and mesentery and active extravasation) for each patient. The image_level_labels_2024.csv file provides image-level labels for bowel and mesenteric injuries and active extravasation. The train_series_meta.csv file contains information regarding the phase of imaging and anatomic coverage of each CT series. The train_demographics_2024.csv file contains information about patient demographics. Pixel-level segmentations of abdominal organs are provided in Neuroimaging Informatics Technology Initiative (ie, NIfTI) format for a subset of 206 series from the training set. Data are available at https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection and https://imaging.rsna.org/dataset/5.

Discussion

We curated a large, high-quality dataset of abdominal trauma CT studies, with contributions from 23 institutions in 14 countries and six continents. This represents the largest and most diverse publicly available dataset of abdominal trauma CT scans. This dataset provides annotations relating to injuries of the liver, spleen, kidneys, bowel, and mesentery, as well as active extravasation. This rich dataset has further utility for investigators, as other injuries such as hematomas, fractures, and lower thoracic injuries are present within the dataset but were not explicitly annotated due to the challenge timeline and the intent of the challenge to focus on critical findings of the highest clinical importance for patients with trauma.

We chose broad inclusion criteria for the CT scans in the dataset. Our initial survey of potential contributing sites showed great variety in the protocols used for imaging patients with abdominal trauma. In fact, some institutions had multiple protocols and selected a protocol based on the severity of the trauma. Aspects that varied across protocols included the parts of the body that were imaged, phases of imaging, and image thickness. Stringent inclusion criteria that limited the dataset to only scans with a single homogeneous protocol (eg, thin-section, multiphasic CT scans of the abdomen and pelvis) would severely constrain the size and potential generalizability of this dataset. For this reason, we widened the inclusion criteria to facilitate a larger and more diverse dataset that could then be used to train more robust ML models. Biphasic (arterial and portal venous), split bolus, and portal venous phase protocols were considered acceptable.

Participating sites were asked to enrich the dataset with representative injuries given the relatively low prevalence of traumatic abdominal injuries at CT encountered in clinical practice. Despite this request, the number of cases with injuries submitted was lower than the organizing committee had anticipated. Addressing class imbalances in curated datasets is particularly important in improving ML model robustness and reducing bias (20,21). An explicit effort was made by the organizing committee to reduce potential biases in the dataset by considering factors such as sex, age, injuries, and contributing site when assigning scans to the training, public test, and private test datasets.

A challenge we faced in curating this dataset was the dramatic differences in z-axis coverage in the included CT scans. For example, some sites imaged from the skull vertex to feet, while many sites limited imaging to the abdomen and pelvis. To reduce the size of the dataset and to help ML model training by reducing the search space, we decided to limit scans to the abdomen and pelvis, using an upper bound of the mid heart and lower bound of the proximal femurs through an automated pipeline (22), and manually reviewed the processed scans.

Similar to prior challenges, we aimed to maximize use of the data and ensure high quality labels while not overburdening annotators. Contributing sites prelabeled submitted scans with information extracted from the clinical report that allowed annotators to focus on the abnormal scans. We considered a variety of annotation strategies that ranged from study- to pixel-level annotations. Our experience with the cervical spine fracture detection challenge (23) showed that the prizewinning models relied on study-level annotations and segmentations rather than bounding boxes (24). In addition, recent work has shown that strongly supervised models trained on slice-level labels from the RSNA Brain CT Hemorrhage dataset labels (25) do not outperform weakly supervised models trained on study-level labels (26). Pixel-level annotations of injuries, including bounding boxes, would be a time-consuming task with likely poor reproducibility, as abdominal injuries can be quite complex, with ill-defined borders. We settled on providing segmentations of the relevant abdominal organ systems to assist with localization and organ labels at the study level. Image-level labels were provided for bowel and mesenteric injuries and active extravasation, as these injuries can be subtle, manifest in variable anatomic locations, and manifest on a limited number of images.

Individual annotators were assigned a single organ system to annotate rather than providing annotations for multiple organ systems on their assigned CT scans. The organizing committee felt this would improve the efficiency of the annotation process and label quality by allowing an annotator to focus on a single task and AAST injury grading scale. The annotators provided granular labels using the AAST grading scale for solid organ injuries. Due to the well-documented issues with interrater agreement in the grading of solid organ injuries with AAST (9) and to help model training, AAST injury grades I–III were classified as low-grade injuries, while grades IV and V were classified as high-grade injuries. This grouping of injury grades still provides more information than a binary label for injury and reflects many clinical practices, as patients with grade IV and V injuries are more likely to undergo surgery or endovascular treatment (4–6). Rather than assigning a fixed number of CT scans for annotation, we utilized the crowdsourcing mode on the annotation platform. This allowed annotators to label as many cases as they wanted, with the public scoreboard providing motivation.

Each solid organ injury label was annotated independently by three radiologists, and the final reference standard labels were established by majority. In scenarios where all three annotators assigned different gradings (ie, no injury, low grade, and high grade), a member of the organizing committee adjudicated the case and assigned the final reference standard label. We felt that this approach would improve annotation quality by generating labels with better interrater agreement and avoiding the problem of poor-quality annotation in a single annotation scheme. With an approach that relies on a single annotator per scan, it can be difficult to detect poor annotators following completion of a set of training cases.

There are several limitations of this dataset. Reference standard labels for the grading of solid organ injuries were established through best of three majority voting. While this represents an improvement over a single annotator, there are inherent issues with AAST grading as a result of interrater variability. We recognize the absence of delayed phase imaging as a limitation because it forms part of the AAST imaging criteria for grade II–IV renal injuries in terms of collecting system injuries. Delayed phase imaging was not included because it was not part of the routine protocol for most contributing sites, and we were concerned that its inclusion in cases with renal injuries would bias models, potentially through spurious associations, rather than truly detecting collecting system injuries. Finally, reference standard labels for solid organ injuries, bowel and mesenteric injuries, and active extravasation were made using a web-based annotation platform, which is limited when compared with real-world clinical practice with access to high-resolution monitors, multiplanar thin-section imaging, clinical information, and prior imaging examinations.

In summary, the RATIC dataset represents the largest and most geographically diverse, publicly available expert-annotated dataset of abdominal traumatic injury CT studies. With the release of this dataset, we hope to facilitate research and development in ML and abdominal trauma that can lead to improved patient care and outcomes. This dataset is made freely available to all researchers for noncommercial use.

Keywords: Trauma, Spleen, Liver, Kidney, Large Bowel, Small Bowel, CT