The University of California San Francisco Adult Longitudinal Post-Treatment Diffuse Glioma MRI Dataset

Abstract

Supplemental material is available for this article.

Summary

The University of California San Francisco Adult Longitudinal Post-Treatment Diffuse Glioma MRI dataset is a publicly available annotated dataset featuring multimodal brain MRI scans from 298 patients with diffuse gliomas taken at two consecutive follow-ups (596 scans total), with corresponding clinical history and expert voxelwise annotations.

Key Points

• ■ Included segmentation masks delineate tumor subregions of enhancing tissue (ET), surrounding nonenhancing fluid-attenuated inversion recovery hyperintensity (SNFH), nonenhancing tumor core (NETC), and resection cavity (RC).

• ■ Longitudinal change annotations delineate change over time in SNFH and ET tumor subregions.

• ■ Preliminary training data provide expected baseline performance for future automated segmentation algorithms.

Introduction

Diffuse gliomas comprise a subclass of molecularly diverse brain tumors thought to arise from heterogeneous populations of neuroglial progenitor cells (1–3). Glioblastomas account for nearly three-quarters of all diffuse gliomas and carry with them dismal prognoses, with estimated 2-year survival rates of approximately 27% (1,4). Maximal safe surgical resection remains the mainstay of treatment, often in conjunction with radiation, chemotherapy, or both (2,5–7). Assessment of posttreatment gliomas is a challenging task, with MRI being the preferred imaging modality for longitudinal evaluation (6–9).

Advances in the field of artificial intelligence have been used to detect, segment, classify, and prognosticate a variety of neoplastic processes (10–13). However, implementation of clinically applicable machine learning studies has been limited by the lack of large, longitudinal, publicly accessible annotated datasets. In recent years, multiple initiatives, such as The Cancer Imaging Archive, the Multimodal Brain Tumor Segmentation Challenge, and the Radiological Society of North America AI Challenges have emerged in an attempt to support researchers seeking larger sample sizes to perform increasingly complex artificial intelligence investigations (8,14–17). Nevertheless, given the large amount of work required to collect, organize, process, and annotate datasets, as well as privacy concerns, currently available datasets remain relatively limited in clinical scope. In particular, the Multimodal Brain Tumor Segmentation Challenge datasets are limited only to pretreatment scans, despite the fact that most imaging in clinical practice is done for longitudinal follow-up during which posttreatment changes often take on highly varied appearances (8,9,18–21). To the best of our knowledge, none of the currently publicly available posttreatment glioma datasets contain expert voxelwise segmentations, though the more recently released LUMIERE (Longitudinal Glioblastoma MRI with Expert RANO Evaluation) dataset contains expert Response Assessment in Neuro-Oncology criteria (22).

To further support large-scale machine learning validations, we hereby report the public release of the University of California San Francisco Adult Longitudinal Post-Treatment Diffuse Glioma (UCSF-ALPTDG) MRI dataset. This expertly annotated, anonymized dataset consists of multimodal three-dimensional (3D) MR images of diffuse gliomas at two consecutive posttreatment time points and is readily available for training and validation of novel and existing artificial intelligence workflows.

Materials and Methods

Study Patients

This retrospective, single-center, Health Insurance Portability and Accountability Act–compliant study was undertaken with approval from the UCSF Institutional Review Board; a waiver for informed consent was granted. The sample consists of data from a prior study (8) whereby 298 patients diagnosed with diffuse glioma and with two consecutive imaging time points were retrospectively identified by software query of our institutional imaging archives (mPower; Nuance Communications), with 62 patients overlapping with a previously released dataset of preoperative diffuse glioma MRI scans (14). Of the total patients, 59% (177 of 298) were male and 41% (121 of 298) female, and the mean age of overall cohort was 51.9 years ± 14.0 (SD). As with our study sample, a slight male predominance has previously been observed with diffuse gliomas, with a 1.3:1.0 male-to-female ratio (23). According to World Health Organization 2021 criteria (24), diagnoses included 58 (19%) isocitrate dehydrogenase (IDH)–mutant astrocytomas, nine (3%) IDH–wild-type astrocytomas, 62 (21%) oligodendrogliomas, and 80 (27%) glioblastomas (Table 1). The study was limited to patients who underwent posttreatment surveillance MRI examinations within our university hospital network between January 2018 and December 2019. In patients for whom multiple consecutive follow-up time points were available, two consecutive time points with interval changes were selected for inclusion.

Table 1:

Patient Demographics and Diffuse Glioma Types

Image Acquisition and Preprocessing

All MRI scans were acquired as part of standard-of-care imaging with one of four Discovery 750 3.0-T (GE HealthCare) scanners. Typical acquisition parameters were as follows: T1 pre- and postcontrast: axial 3D inversion-recovery spoiled gradient-echo T1 (repetition time, echo time, and inversion time, 6, 2.3, and 450 msec, respectively; flip angle, 12 degrees; section thickness, 1.0 mm; matrix, 256 × 256; field of view, 25.6; number of excitations, one); T2: sagittal 3D fast spin echo (repetition time and echo time, 2200 and 100 msec; section thickness, 1.2 mm; matrix, 256 × 256; field of view, 25.6 cm; number of excitations, one); and T2 fluid-attenuated inversion recovery (FLAIR): coronal 3D fast spin echo (repetition time, echo time, and inversion time, 5700, 115, and 1650 msec, respectively; section thickness, 1.2 mm; matrix, 256 × 256; field of view, 25.6 cm; number of excitations, one). Gadoterate (Dotarem; Guerbet) at a dose of 0.2 mL/kg was used as the gadolinium-based contrast agent.

Two consecutive posttreatment scans were included for each patient, for a total of 596 scans. Preprocessing procedures included image de-identification and conversion to the Neuroimaging Informatics Technology Initiative file format, skull stripping, N4 bias correction, coregistration to the T1 postcontrast image obtained at the second posttreatment time point, and registration to the SRI24 atlas (8).

Reference Standard Voxelwise and Categorical Annotations

Voxelwise segmentations were performed using ITK-Snap software, version 4.0.0-alpha.3 (www.itksnap.org) (25) with a semiautomated, iterative procedure followed by manual revisions to delineate tumor subregions (8,21) (Figure). A previously developed 3D nnU-Net architecture was used to provide baseline subregion segmentations (8), which was retrained a second time after 100 cases were revised. The automated segmentations were each reviewed for accuracy and manually corrected as needed by one of three expert neuroradiologists (E.C. with 2 years, A.M.R. with 4 years, and J.D.R. with 4 years of experience in neuro-oncologic imaging, respectively); the individual who annotated each case is specified in Table S1. The four subregions consisted of enhancing tissue (ET), surrounding nonenhancing FLAIR hyperintensity (SNFH), nonenhancing tumor core (NETC), and resection cavity (RC). The RC class is new compared with the previously published study on this dataset (8), and all annotations were revised according to a new annotation protocol. Given the posttreatment nature of the scans, any T2 and FLAIR signal abnormalities, including radiation-related hyperintensity, gliosis, edema, and nonenhancing tumor, were included in the SNFH class. Any areas of thick or nodular enhancement were included in the ET class, though typical treatment-related thin linear enhancement along and within RCs and along the dura was not included in the ET class. The RC class consisted of both recent and chronic RCs. Chronic RCs, which are typically older than 3–6 months, were considered those with signal intensity isointense to cerebrospinal fluid. More recent RCs often contained air, blood, and/or proteinaceous materials and otherwise exhibited variable signal characteristics. The NETC class consisted of necrotic and nonenhancing tissue surrounded by ET and not otherwise clearly represented by a prior RC.

Representative examples of brain MRI and tumor segmentations in (A) a 55-year-old female patient with glioblastoma and (B) a 54-year-old female patient with glioblastoma. Subtraction images show representative segmentations of longitudinal (A) increases and (B) decreases in surrounding nonenhancing fluid-attenuated inversion recovery (FLAIR) hyperintensity (SNFH) and enhancing tissue (ET) in each patient, respectively. T1ce = T1 contrast-enhanced, RC = resection cavity, NETC = nonenhancing tumor core.

Separate voxelwise annotations were created to delineate regions that exhibited volumetric changes across the two time points for the SNFH and ET tissue classes. For SNFH, FLAIR sequences acquired at each posttreatment time point were subtracted from each other to guide voxelwise annotations of increasing or decreasing signal intensity. For ET, T1 postcontrast minus T1 precontrast subtraction images were first generated for each individual time point; subsequently, these newly generated enhancement subtraction images were then subtracted from each other (ie, [T1 postcontrast_Time2 – T1 precontrast_Time2] – [T1 postcontrast_Time1 – T1 precontrast_Time1]) to highlight areas of increasing or decreasing signal intensity. Compared with our previously published study (8), these longitudinal SNFH and ET change segmentations were revised to include mixed longitudinal changes (ie, areas of both increases and decreases in signal across time points in the same patient).

Volumetric changes in SNFH tissue and ET between the two time points were categorized as increased, decreased, unchanged, or a combination of increased and decreased based on volumes extracted from the final, expert neuroradiologist, manually corrected longitudinal change segmentations.

nnU-Net Convolutional Neural Network for Baseline Segmentation Performance

To provide baseline segmentation performance of this dataset, we used the nnU-Net (26), a self-configuring, 3D fully convolutional network. Networks for the four-class tissue segmentations and longitudinal changes were trained on a random set of 248 patients and tested on the remaining 50 patients. Performance was evaluated using Dice scores, volume similarities, and Hausdorff distances. In all experiments, default settings for the 3dfullres network were used with an automatically selected patch size of 128 × 128 × 128. Training was performed on a GeForce RTX 3090 graphics processing unit, version 11.2 (NVIDIA; 24 GB memory) for 1000 epochs using a combination of cross-entropy and Dice loss function (1:1). The code used for the nnU-Net implementation and pretrained networks is available at https://github.com/rachitsaluja/UCSF-ALPTDG-benchmarks.

Results

Demographics, Pathology, Treatment History, and Tumor Volumetric Information

The full scope of patient demographics, pathologic diagnoses, treatment history, tumor volumetric characteristics, and posttreatment changes over time in our cohort are reported in Tables 1 and 2, with complete information for each scan available in Table S1. Accompanying complete clinical data include genetic mutation statuses (eg, IDH, O⁶-methylguanine-DNA methyltransferase [MGMT], 1p19q, and ATRX); comprehensive treatment histories, including chemotherapeutic regimens; and temporal landmarks such as days from first surgery to first scan, time interval between scans, days from first scan to progression (based on expert multidisciplinary consensus in consideration of both radiologic and clinical factors), and days from first scan to death (Table S1).

Table 2:

Tumor Statistics

Briefly summarized, our dataset was composed of 27% (80 of 298) glioblastomas, 21% (62 of 298) oligodendrogliomas, and 19% (58 of 298) IDH-mutant astrocytomas. Forty-one percent (123 of 298) of patients were positive for IDH mutation status, 28% (82 of 298) were positive for O⁶-MGMT methylation status, and 21% (62 of 298) were positive for 1p19q codeletion status (Table 1). The median time between surgery (or MRI diagnosis if surgery was not pursued) and first scan was 766 days. The median time between scans was 65 days. A total of 21% (62 of 298) of patients had undergone gross total resections. The percentages of scans with ET, SNFH, NETC, and RC were 63% (373 of 596), 100% (596 of 596), 33% (198 of 596), and 90% (536 of 596), respectively (Table 2). There were 34% (101 of 298) with an increase in ET volume and 10% (30 of 298) with a decrease in ET volume between the time points. Likewise, 38% (112 of 298) showed an increase in SNFH, while 12% (37 of 298) demonstrated a decrease in SNFH between the evaluated time points.

Baseline Segmentation Performance with nnU-Net

Table 3 provides segmentation performance metrics in the test set, which include mean and median Dice score, volume similarities, and Hausdorff distances for the four subregions at individual time points and for longitudinal changes in SNFH and ET across time points.

Table 3:

nnU-Net Baseline Segmentation Performance Metrics

Data Availability

The UCSF-ALPTDG dataset is hosted on the UCSF Center for Intelligent Imaging Datasets for Medical Imaging repository and is readily available for public, noncommercial use after agreeing to a data use agreement (https://imagingdatasets.ucsf.edu/dataset/2).

Discussion

Assessment of posttreatment gliomas remains a diagnostic challenge in neuro-oncologic imaging given the complex, heterogeneous appearance of treatment-related changes and variability among interpreting radiologists in classifying tissue subregions (8,20,27). Accurate longitudinal volumetric assessment of residual ET has the potential to reduce physician burden and errors in measurement while additionally generating novel, clinically relevant metrics for radiologists, neuro-oncologists, radiation oncologists, and neurosurgeons involved in the care of these patients (8,28). Given the substantial effort required to annotate subtle longitudinal changes in the posttreatment setting, the release of the UCSF-ALPTDG dataset, which includes expert voxelwise segmentations of tumor subregions and longitudinal changes, will further support future studies seeking to validate automated assessment protocols of longitudinal posttherapeutic changes in these highly morbid tumors.

This dataset has several limitations. It is derived from a single institution, protocol, and scanner model and field strength, which may limit generalizability. Additional heterogeneous, multisite data would be helpful to train a more generalizable algorithm. Although all expert annotations were curated by subspecialty trained neuroradiology attending physicians, only a single reference standard voxelwise annotation is available for each study, thus limiting assessments of interrater reliability. The different tissue classes are based on conventional structural MRI characteristics and thus combine both viable tumor and posttreatment changes. Changes in volumes of SNFH tissue and ET classes are often used as a first step in the radiologic evaluation. Of note, a recent study used longitudinal volumes of ET after laser interstitial thermal therapy to predict outcomes (29), demonstrating that volumetric changes may be helpful on their own. The second step of assessing whether these changes are related residual or recurrent tumor or treatment-related changes would be a separate determination that would require appropriate clinical information, which often remains unknown until repeat surgery. In addition, future work using more advanced imaging modalities, such as perfusion, diffusion, and/or spectroscopy, in conjunction with clinical assessments may be helpful in distinguishing treatment-related changes from progressive disease. We also recognize that due to unpredictable effects of varying acquisition parameters on signal characteristics, longitudinal changes over time were quantified solely based on volumetric change and did not account for changes in signal intensity. Furthermore, because all imaging was performed as per standard-of-care clinical workflows, the timing between follow-up scans was largely dictated by clinical factors and tumor pathology (6). This heterogeneity in follow-up intervals between scans may also limit comparisons between patients.

Publicly available annotated biomedical imaging repositories support researchers in developing effective, sophisticated machine learning algorithms (30,31). We hope that the release of the UCSF-ALPTDG dataset will bolster efforts to develop artificial intelligence techniques for longitudinal disease monitoring in this high-risk patient population for which few existing publicly available datasets feature detailed annotations of posttreatment changes (22). Disseminating this dataset should facilitate the development of robust techniques that may eventually allow for the integration of automated quantitative analyses into prospective imaging workflows. Specific use cases may include training of a more generalizable nnU-Net segmentation architecture from multisite data, prediction of survival, progression and specific areas of recurrence and infiltrative tumor, and extraction of radiomics metrics to serve as potential quantitative imaging biomarkers to predict outcomes and response to different treatments.

Keywords: Deep Learning, Artificial Intelligence, Diffuse Glioma, Neuro-Oncology, Resection Cavity