Toward automated interpretable AAST grading for blunt splenic injury

Abstract

Background

The American Association for the Surgery of Trauma (AAST) splenic organ injury scale (OIS) is the most frequently used CT-based grading system for blunt splenic trauma. However, reported inter-rater agreement is modest, and an algorithm that objectively automates grading based on transparent and verifiable criteria could serve as a high-trust diagnostic aid.

Purpose

To pilot the development of an automated interpretable multi-stage deep learning-based system to predict AAST grade from admission trauma CT.

Methods

Our pipeline includes 4 parts: (1) automated splenic localization, (2) Faster R-CNN-based detection of pseudoaneurysms (PSA) and active bleeds (AB), (3) nnU-Net segmentation and quantification of splenic parenchymal disruption (SPD), and (4) a directed graph that infers AAST grades from detection and segmentation results. Training and validation is performed on a dataset of adult patients (age ≥ 18) with voxelwise labeling, consensus AAST grading, and hemorrhage-related outcome data (n = 174).

Results

AAST classification agreement (weighted κ) between automated and consensus AAST grades was substantial (0.79). High-grade (IV and V) injuries were predicted with accuracy, positive predictive value, and negative predictive value of 92%, 95%, and 89%. The area under the curve for predicting hemorrhage control intervention was comparable between expert consensus and automated AAST grading (0.83 vs 0.88). The mean combined inference time for the pipeline was 96.9 s.

Conclusions

The results of our method were rapid and verifiable, with high agreement between automated and expert consensus grades. Diagnosis of high-grade lesions and prediction of hemorrhage control intervention produced accurate results in adult patients.

Introduction

Splenic injury is the most common solid organ injury in adult blunt abdominal trauma [1, 2]. It is routinely evaluated on admission CT when abdominal trauma is suspected, both in stable patients and in those with hemodynamic compromise that demonstrate transient response to fluid resuscitation [3]. In 2018, the American Association for the Surgery of Trauma (AAST) Patient Assessment Committee (PAC) introduced an updated AAST splenic organ injury scale (OIS) for treatment decision-making based on admission two-phase abdominopelvic CT examination. While AAST grading has existed as a research tool for decades, the recent update reflects an attempt to operationalize this grading system for point-of-care use and standardize management practices. Treatment options vary by injury severity, including routine observation for low-grade injuries, and urgent angioembolization or splenectomy to control hemorrhage in high-grade injuries. Patient selection is critical as hemorrhage control interventions require mustering limited resources and staff and are not without the potential for both short- and long-term morbidity, including from catheter-related complications and an increased lifetime risk of overwhelming post-splenectomy infection (OPSI) [4, 5]. Clinical decision-making should be rapid and based on objective criteria, as the spleen is a highly vascular organ, and severe splenic injury can potentially lead to exsanguination, multi-organ system failure, and death [6].

CT-based AAST grading enjoys widespread adoption among trauma surgeons; however, in a survey of AAST member practices, only 45% of respondents reported routine use of AAST grading for blunt splenic trauma by radiologists at their institutions [7]. Even in an ideal circumstance of ubiquitous radiologist adoption and reporting, classification systems are prone to variability in the perceived grades among readers with varying experience and specialization, and reported agreement for the splenic AAST OIS has been modest under research conditions [8, 9]. In practice, radiologists are subject to shifting circumstances in their clinical environment with respect to study volume, reading room distractions, and fatigue-related performance degradation, such as from circadian rhythm disruptions after multiple consecutive night shifts [10–14]. Furthermore, admission trauma CT interpretation is time consuming. Among expert trauma radiologists, interpretation turnaround times for severely injured patients commonly exceed 20–30 min [10].

Automated AAST grading could potentially provide a rapid, objective, and accurate second-reader capability, but there has been limited automation research involving the individual CT features of splenic injury, and, to our knowledge, no work has described automated AAST grading to date [15–19].

Black box methods are prone to spurious causal inference [20] and are unaccountable to end users as the reasoning used to arrive at a decision cannot be verified. For an intelligent system to be considered responsible, ethical, and trustworthy—a requirement in the high-stakes setting of trauma care—it must at a minimum include a layer of explainability to ensure that the decisions made are justifiable [21, 22]. Since AAST grading is a multi-stage process, the intermediate steps of an automated method should be interpretable to end users, giving them agency to verify individual model assumptions should they choose to do so. Interpretability involves the provision of packets of symbolic information that are semantically similar to the common-sense causal reasoning that would be used by an expert to arrive at a decision—in this case, at a given splenic injury grade [23, 24].

To this end, we leverage transparent deep learning approaches that are based on clinical grading standards to develop a novel automated multi-stage deep learning (DL) method that predicts the AAST splenic OIS using the most salient features of the grading system, namely, active bleeding, pseudoaneurysm, and splenic parenchymal disruption [25].

Materials and methods

Datasets

The work was conducted as part of an IRB-approved study and utilized two deidentified single institution datasets. The primary clinical dataset is previously described [3] and consists of 174 dual-phase trauma CT scans from consecutively selected adult patients (age ≥ 18) collected between 2017 and 2019 and archived at 1.5–3-mm section thickness, with voxelwise labeling of pseudoaneurysm (PSA), active bleed (AB), and splenic parenchymal disruption (SPD). PSAs in this context are vascular injuries contained by splenic parenchyma with densities similar to or slightly higher than the blood pool [1]. Foci of AB extend beyond the splenic parenchyma and typically increase in size on the portal venous phase. The portal venous phase is optimal for the delineation of SPD and AB, whereas PSA is best detected on the arterial phase [26, 27].

All studies in this dataset had accompanying AAST consensus grading by three expert trauma radiologists and outcome data including whether patients underwent angioembolization or splenectomy. Foci of active bleed and pseudoaneurysm occupy a small fraction of an abdominal CT volume when present. To address AB and PSA class imbalance, the dataset was augmented with labeled dual-phase CTs from a second existing blunt splenic injury dataset with 68 consecutively selected patients who underwent splenic hemorrhage control intervention and had AB, PSA, or both on CT between 2007 and 2016 [28] (ABPSA dataset). A third dataset with the subset of 41 labeled splenic normals from the medical segmentation decathlon challenge [29] (SMSD dataset) was employed to develop an initial automated localization step of injured spleens.

Summary of the AAST splenic organ injury scale and clinical relevance

All patients with intraperitoneal AB receive a grade of V in this system, while any patient with PSA but no AB receives a grade of IV [25]. This assignment is irrespective of the size and number of splenic vascular lesions. High-grade (IV and V) injuries are considered to necessitate angioembolization (AE) for hemorrhage control at a minimum, and surgeons may opt instead for early splenectomy. Rates of failure for attempted splenic salvage for high-grade injuries are historically high, ranging from 20% to over 60% [5, 30–33], but are improved with the liberal use of angioembolization [31]. The surgeon’s judgment and institution-specific guidelines play an important role in the choice of hemorrhage control intervention [32]. In patients without splenic vascular injury, the AAST splenic OIS grade is primarily determined by the extent of visually estimated SPD [25, 34] using diameter measurement cut-offs established using rules of thumb in the original 1994 AAST classification [34]. Management of grade 3 injuries (with greater than approximately 3 cm estimated SPD depth but less than 25% parenchymal involvement) is highly variable. At many institutions, these injuries are considered low grade and are routinely managed conservatively [32]; however, some investigators report improved salvage rates with routine angiographic screening followed by AE if a vascular injury is seen on the image intensifier [5, 35], and still others recommend the routine use of AE as a precautionary measure [36]. Variability in practice patterns lies in the potential for missed small or subtle vascular injury on CT due to variable scan timing and transient vessel thrombosis or spasm [36]. Low-grade (grade I and II) injuries have less than approximately 3 cm SPD depth. Conservative management is considered the standard of care for low-grade injuries across institutions [32, 33].

Automated splenic injury grading pipeline: overview

The complete pipeline for our proposed automated AAST OIS grade prediction method is shown in Fig. 1. The pipeline begins with an automated 3D cropping step aimed at (a) reducing irrelevant background which could otherwise contribute to false positive results and (b) increasing the proportion of positive voxels in the data given small target volumes of AB, PSA, and SPD. 2D Faster R-CNN [37] is then applied to the detection of AB on portal venous phase axial slices and PSA on arterial axial slices, leveraging the optimal phase for the detection of each feature [26, 27]. SPD is segmented using nnU-Net [38] and quantified using voxel counting. Vascular injury detections and SPD volumes are then fed into a hierarchical rules-based system to derive the predicted AAST grade.

Fig. 1.

Overall pipeline of the proposed automatic splenic AAST grading algorithm. Splenic localization is first performed, which crops irrelevant slices on abdominopelvic CT cranial and caudal to the spleen and neighboring soft tissue that may harbor foci of active bleeding. Active bleeding is detected on portal venous CT scans, and pseudoaneurysm is detected on arterial CT scans by Faster RCNN on localized axial sections. SPD is segmented by nn-UNet, and volume is calculated. Active bleeding and pseudoaneurysm detection and SPD volume is then fed into a directed graph for AAST grading prediction

Step 1: automated splenic localization

A semi-supervised method using the noisy student algorithm [39] was employed to derive whole-organ label masks for injured spleens. The method initially utilizes a 3D U-Net trained on the external medical segmentation decathlon challenge (SMSD) “teacher” splenic normal labels to generate pseudo labels in the in-house clinical and ABPSA datasets. The labels and pseudo labels are used to create improved spleen segmentations in an iterative process. The 3D U-Net predicted segmentations from a given iteration are used as ground truth training cases in the next iteration, resulting in gradual segmentation refinement. We used the MONAI platform (40) to construct the 3D U-Net [41], with a learning rate of 1e-4. All training was performed using an RTX 3090 NVIDIA GPU with 24 GB of RAM. Training involved six iterations with 600 epochs per iteration. Subsequently, the splenic volume is dilated by 30 voxels to include relevant perisplenic soft tissue structures, and axial images above and below the dilated volume are excluded. Following this pre-processing step, visual inspection of each CT study indicated that all foci of vascular injury and SPD were contained within the cropped range and there were no failures.

Step 2: vascular lesion detection

Following splenic localization, 2D Faster R-CNN [37], a two-stage object detection network, was used to detect PSA on axial arterial phase images and AB on axial portal venous phase images. The network first extracts image features and generates region proposals. Second, it fine-tunes the box proposal size and location and classifies each proposal. We used ResNeXt-101 with Feature Pyramid Network (FPN) as the backbone given prior best performance in the COCO object detection task [42, 43]. Training was augmented using the non-overlapping ABPSA dataset, within which each CT study includes at least one focus of pseudoaneurysm or active bleed. Faster R-CNN was trained in five-fold cross-validation, splitting the combined dataset evenly into 5 independent subsets to avoid data leak, and using each fold for validation and the remaining 4 folds for training. Thresholds were selected to achieve the highest possible sensitivity. Faster R-CNN was implemented in the detectron2 platform [44], with the following parameters: learning rate of 0.02, a 10 × decay at 15,000 iterations, and a total of 30,000 training iterations.

Step 3: splenic parenchymal disruption segmentation and quantification

We applied nnU-Net [38] to segment splenic parenchymal disruption (SPD) due to its state-of-the-art performance across a large variety of segmentation tasks. nnU-Net trains four models in five-fold cross-validation (2D U-Net, low-resolution 3D U-Net, high-resolution 3D U-Net, and a low and high-resolution cascaded 3D U-Net) and determines the best-performing model or ensemble of models for inference. Design choices including hyperparameter selection and pre-processing steps are made automatically from specific dataset properties known as the dataset and pipeline fingerprint [38]. Voxel counting is then applied to automated label masks to determine laceration volumes.

Step 4: AAST grade determination

The automated detections and segmentation-derived volumes are directly applied to a hierarchical AAST OIS-based system of rules (Fig. 1). The directed graph starts from the highest-grade decision, proceeding toward the lowest grade in a manner similar to how AAST grading is employed in clinical practice (Table 1). First, if AB is detected, the patient receives a grade of V. If no AB is detected, but PSA is detected, the patient receives a grade of IV. If no vascular injury is present, the grade is determined by the extent of splenic parenchymal disruption. In our method, grades are stratified by the automated SPD volume in place of visual estimation of injury depth using a logistic regression-derived cut-off. Since low-grade lesions are managed conservatively, we combined grades I and II into a single “low grade” class. A laceration volume of 14 mL optimally discriminated between low-grade (I and II) and grade III lesions.

Table 1.

Comparison of the 2018 spleen AAST splenic organ injury scale (OIS) grading criteria and lesions in our method selected using available evidence and expert knowledge

	AAST grading criteria	Lesions in our system
Grade I	Subcapsular hematoma < 10% surface area Parenchymal laceration < 1 cm depth Capsular tear1	SPD < 14 mL
Grade II	Subcapsular hematoma2 10–50% surface area Intraparenchymal hematoma < 5 cm Parenchymal laceration 1–3 cm
Grade III	Subcapsular hematoma > 50% surface area Ruptured subcapsular/intraparenchymal hematoma3 ≥ 5 cm Parenchymal laceration4 > 3 cm depth	SPD ≥ 14 mLwith no PSA or AB
Grade IV	Any injury in the presence of a contained splenic vascular injury5 Parenchymal laceration producing > 25% devascularization6 Active bleeding confined within splenic capsule7	PSA
Grade V	Any injury in the presence of a splenic vascular injury with active bleeding extended beyond the spleen into the peritoneum8	AB

Capsular tear¹ and subcapsular hematoma² are ignored in our simplified system (see “Discussion”). Laceration and intraparenchymal hematoma^3,4 are grouped as splenic parenchymal disruption. Contained splenic vascular injury⁵ is synonymous with pseudoaneurysm on CT. In our dataset, only two patients with ground truth consensus grade of IV had laceration with no detected pseudoaneurysm, precluding derivation of a data-driven cut-off differentiating grade III and IV lesions by SPD volume⁶. Current literature does not differentiate between active bleeding confined to or extending beyond the capsule^7,8

Abbreviations: SPD- splenic parenchymal disruption; PSA- pseudoaneurysm; AB- active bleeding

Results

Descriptive statistics for the clinical dataset are provided in Table 2. The weighted Cohen’s κ between automated and consensus grades in the clinical dataset of 174 patients was 0.79.

Table 2.

Ground truth descriptive statistics of the clinical splenic injury dataset

	n (%)
Total	174 (100)
Active bleeding	28 (16)
Pseudoaneurysm	54 (31)
PSD ≥ 1 cm3	93 (53)
Expert consensus grade
Grade V	18 (10)
Grade IV	50 (29)
Grade III	15 (9)
Grades I & II	91 (52)

All CT studies in the ABPSA dataset (n = 68ven) had pseudoaneurysm (n = 31, 46%), active bleed (n = 54, 80%), or both (n = 17, 25%).

Using radiologist expert consensus grading as the reference standard, diagnosis of high-grade (IV and V) splenic injuries—those that require urgent hemorrhage control intervention for splenic salvage [33, 45]—was achieved with an accuracy, sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV) of 92%, 93%, 92%, 95%, and 89%. The proposed method correctly identified almost all high-grade injuries. Only 1 of 69 patients with high-grade (IV and V) injuries was underestimated as a grade III injury. 9 grade III injuries were overestimated as high grade. All patients classified as grades I and II by radiologists (n = 91) were correctly predicted as low-grade injuries using our method. This indicates that among patients who would normally be managed conservatively, there were no false positive severe injuries. In subanalysis of high-grade injuries, 19 out of 51 consensus grade IV patients are overestimated by our method as grade V, but only 2 out of 18 grade V patients are underestimated as grade IV. The area under the curve (AUC) for predicting a composite outcome of intervention with angioembolization or splenectomy was 0.83 for automated grades, comparable to an AUC of 0.88 for consensus grading.

Performance of detection and segmentation tasks

For AB detection in the clinical and ABPSA datasets, Faster R-CNN achieved an AUC, accuracy, sensitivity, specificity, NPV, and PPV of 0.84, 88%, 82%, 91%, 91%, and 83% (Table 3). For PSA, Faster R-CNN achieved an AUC, accuracy, sensitivity, specificity, NPV, and PPV of 0.79, 83%, 91%, 78%, 94%, and 71%. Examples of AB and PSA box detections are shown in Fig. 2.

Table 3.

Faster RCNN detection results for active bleeding and pseudoaneurysm

	AUC	Accuracy	Sensitivity	Specificity	PPV	NPV
Active bleeding	0.84	88%	82%	91%	91%	83%
Pseudoaneurysm	0.79	83%	91%	78%	94%	71%

Fig. 2.

Pseudoaneurysm (“psa,” part A) and active bleed (“ab,” part B) box detection on arterial and portal venous phase images, respectively. Detection is achieved using Faster R-CNN with a very deep (ResNeXt-101 with FPN) backbone. Numbers shown refer to probability of correct detection as a fraction of 1

A review of AB false positive detections revealed that in 13 of 19 patients, the detection network misclassified pseudoaneurysm as active bleeding on the portal venous phase, owing to lingering pseudoaneurysm blush which otherwise characteristically washes out and is inconspicuous on this phase [1, 26]. This could be attributable to imperfect CT timing using a descending thoracic aorta ROI trigger threshold or variability in cardiac output between patients [3]. Nevertheless, box proposals of these lesions provide transparent results which would allow radiologists, interventionalists, or surgeons to reject detections they disagree with.

Patients in the clinical dataset had labeled laceration with a range of volumes between 0.1 and 255.1 mL (median volume: 1.8 mL). All patients with < 1 mL had low-grade injuries. Automated segmentations in patients with ≥ 1 mL SPD and no vascular injury that would take absolute priority in injury grading had a volume similarity (VS) index of 0.68, and dice similarity coefficient of 0.54 with respect to manual labels, corresponding with high-saliency visual results that conformed to the margins of laceration (Fig. 3). Pearson’s r between manual and automated volumes was 0.89 (“excellent” range).

Fig. 3.

Splenic parenchymal disruption (SPD) segmentation/quantitative visualization for a range of volumes. On regression, an optimal cut-off of 14 mL distinguished between low-grade (I and II) and grade III lesions

Mean inference times for our method included 1.5 s for automated splenic localization, 4.2 s for Faster R-CNN, and 91.2 s for ensembled nnU-Net, for a total of 96.9 s.

Discussion

The AAST splenic organ injury scale is often used to guide surgical management decisions. High-grade (AAST IV and V) lesions typically require angioembolization or splenectomy for hemorrhage control [31–33, 45, 46]. Low-grade (AAST I and II) lesions are routinely managed conservatively [32, 33]. Management of grade III lesions remains variable and institution dependent [5, 35, 36]. AAST grading is limited by modest interobserver agreement, inconsistent reporting, and the long interpretation and reporting times of admission trauma CT [7–10]. An interpretable automated system could augment objective decision-making as a second-reader diagnostic aid, producing verifiable visual results that could be accepted or rejected by the end user.

To date, we are not aware of previous attempts to automate AAST splenic OIS grading, either with black box or interpretable methods. Few studies report automated methods for the detection or segmentation of individual features relevant to the grading of splenic injury. Several works describe whole-spleen segmentation in trauma patients using semi-automated [15] and automated methods, such as with 3D active shape contours and probabilistic atlases [16, 17]; however, whole-spleen volumes are of unknown clinical import in trauma. Other work describes black-box detection but not quantification of splenic parenchymal disruption using a random forest method and a convolutional neural network with a long short-term memory (LSTM) model [18]. One group examined detection of active bleed but not pseudoaneurysm using a hand-crafted feature engineering-based method. Of 30 splenic injury subjects, 4 had active bleeding. The method had a detection accuracy and PPV of only 73% and 33%, respectively [19]. In more recent work using deep learning segmentation methods, automated liver parenchymal disruption volumes predicted angiopositivity on subsequent conventional angiography [47]. A variety of robust methods have emerged using DL for non-trivial hemorrhage-related tasks, including hemoperitoneum, extraperitoneal pelvic hematoma, and hemothorax quantitative visualization [48–51]. Additional DL-related work in the spleen has demonstrated the feasibility of splenic vascular injury segmentation, however, without an initial detection step or the ability to differentiate between pseudoaneurysm and active bleeding [28]. Quantification of vascular injury burden is presently not included in the AAST OIS framework, and detection of vascular injury is made in a binary fashion on the patient level.

In the present work, we pilot the development of an interpretable automated AAST splenic grade prediction pipeline using the dual-phase imaging protocol currently recommended by the AAST Patient Assessment Committee [25]. Dual-phase imaging is optimized for the delineation of splenic disruption and detection of active bleeding on the portal venous phase and detection of pseudoaneurysm on the arterial phase [26, 27].

Our pipeline begins with robust splenic localization, followed by detection of pseudoaneurysm and active bleed using Faster R-CNN using a RexNeXt-101 backbone with FPN and nnU-Net segmentation of splenic parenchymal disruption. The detections and splenic volumes are then fed into an intuitive rules-based system guided by major concepts of the AAST OIS combined with consideration of clinical evidence and expert knowledge. Using the Landis and Koch scheme [52], we achieved substantial agreement with expert consensus ground truth AAST grading (weighted κ of 0.79) and a 92% accuracy for predicting high-grade (IV and V) lesions. The AUC of automated grades for predicting a composite outcome of angioembolization or splenectomy was comparable to the prediction of the same outcome using expert consensus AAST grading. The method is much more rapid in inference compared with reported admission trauma CT interpretation times.

For active bleed detection, Faster R-CNN achieved an accuracy, NPV, and PPV of 88%, 91%, and 83%, and for pseudoaneurysm detection, an accuracy, NPV, and PPV of 83%, 94%, and 71%. nnU-Net results demonstrated reasonably high DSC and volume similarity for a task involving small target volumes, with excellent Pearson correlation between manual and automated volumes (0.89) and highquality visual results. A threshold of 14 mL optimally distinguished between low-grade (I and II) and grade III lesions.

Our study had some limitations. In our clinical dataset, we were not able to determine a threshold distinguishing between grade III and the subset of grade IV patients without detected pseudoaneurysms as there were only two such patients, both with volumes that overlapped with the volume distribution of grade III injuries. A larger multi-institutional dataset will contain more such patients and likely allow determination of additional cut-offs for grade IV and V injuries. The 2018 AAST OIS is not without controversy. Despite the recommendation of dual-phase scanning, this protocol is not widely adopted at present. Some high-volume level I trauma centers perform scanning of the abdomen in the portal venous phase only or use a single-phase split bolus protocol [53–55]. Additional features included in the AAST OIS were selected using heuristics without strong evidence. For example, the estimated size of subcapsular hematoma is included, although our review of the literature yielded few studies supporting this as a univariate predictor of outcome [56], and we are not aware of studies showing that this feature is independently predictive when accounting for SPD and vascular injury. Laceration and intraparenchymal hematoma (blood pooled within the interstices of a laceration) are typically indistinguishable and grouped as splenic parenchymal injury or disruption in scientific works and clinical practice [26, 57]. The 2018 AAST discriminates between intra- and extraperitoneal active bleeding even though the extensive literature on vascular lesions does not differentiate between active bleeding confined to or extending beyond the capsule into the peritoneal cavity [26, 27, 58, 59]. Additionally, the AAST OIS currently includes capsular tear for discriminating between grade I and II lesions [25]. The capsule is microscopic and not directly visible on CT. Capsular laceration is only implied by the presence of SPD, and this feature may be redundant. By including only those features in our simplified system supported by strong evidence [26, 32, 45, 58, 60], we managed to achieve substantial agreement with human expert grading. The internal and clinical validity of each feature of the 2018 AAST splenic OIS is an active area of investigation by the American Society of Emergency Radiology (ASER) Splenic Trauma Expert Panel [61], and we will consider inclusion of additional features as dictated by its ongoing conclusions and further emergence of scientific evidence. Finally, our method is trained and validated using only adult patients, and future work will ultimately need to include pediatric trauma victims.

Other future avenues of investigation may include collaboration with participants of the ASER panel, which has curated a large, as yet unlabeled, multicenter CT dataset with studies performed using a variety of protocols on a wide range of scanner makes and models. The method could be refined, retrained, and tested on a hold-out sample. A simulated deployment study comparing multileader agreement and diagnostic performance with and without our interpretable method as a diagnostic support system is also planned.

In conclusion, in this single-center pilot study, we developed a rapid interpretable automated method for grading splenic injury using the most salient features of the AAST splenic OIS. The method achieved high agreement with and accuracy compared to consensus expert AAST grading in cross-validation. Prediction of hemorrhage control intervention was comparable between automated and consensus grading. Future avenues include scaling to a larger dataset, conducting a simulated deployment study, and assessing user acceptance.