QdMRI: A system for comprehensive analysis of thoracic dynamics via dynamic MRI

Abstract

Quantitative thoracic dynamic magnetic resonance imaging (QdMRI), a recently developed technique, provides a potential solution for evaluating treatment effects in thoracic insufficiency syndrome (TIS). In this paper, we integrate all related algorithms and modules during our work from the past 10 years on TIS into one system, named QdMRI, to address the following questions: (1) How to effectively acquire dynamic images? For many TIS patients, subjects are unable to cooperate with breathing instructions during image acquisition. Image acquisition can only be implemented under free-breathing conditions, and it is not feasible to use a surrogate device for tracing breathing signals. (2) How to assess the thoracic structures from the acquired image, such as lungs, left and right, separately? (3) How to depict the dynamics of thoracic structures due to respiration motion? (4) How to use the structural and functional information for the quantitative evaluation of surgical TIS treatment and for the design of the surgery plan? The QdMRI system includes 4 major modules: dynamic MRI (dMRI) acquisition, 4D image construction, image segmentation (from 4D image), and visualization of segmentation results, dynamic measurements, and comparisons of measurements from TIS patients with those from normal children. Scanning/image acquisition time for one subject is ~20 minutes, 4D image construction time is ~5 minutes, image segmentation of lungs via deep learning is 70 seconds for all time points (with the average DICE 0.96 in healthy children), and measurement computation time is 2 seconds.

1. Introduction

Thoracic insufficiency syndrome (TIS) is a serious childhood disorder with the inability of the thorax to support normal respiration or lung growth [1, 2]. Much of the morbidity in untreated TIS is due to progressive restrictive respiration, leading to respiratory failure and increased risk of early mortality [3]. Surgical management of TIS can substantially limit respiratory decline. Vertical Expandable Prosthetic Titanium Rib (VEPTR), an FDA-approved TIS treatment approach, was designed to adjust the thoracic bone structure to facilitate lung growth and further improve respiration [4]. Image-based approaches for TIS treatment evaluation and surgery planning suffer from the challenges of image acquisition and analysis as follows: (1) How to effectively acquire dynamic images that capture 3D structural and dynamic information? For many TIS patients, subjects are unable to cooperate with breathing instructions during image acquisition. Image acquisition can only be implemented during free-breathing, and it is not feasible to use a surrogate device for tracing breathing signals. (2) How to assess the thoracic structures from the acquired image, such as lungs, left and right, separately? (3) How to depict the dynamics of thoracic structures due to the respiration motion? (4) How to use the structural and functional information for the quantitative evaluation of surgical TIS treatment and for the design of the surgery plan?

Quantitative thoracic dynamic magnetic resonance imaging (QdMRI), a recently developed technique [5–8], provides a potential solution to the above questions. For pediatric patients with TIS, computed tomography (CT) or hyperpolarized gas magnetic resonance imaging (MRI) have not been widely utilized in clinical practice due to the heavy radiation concerns and difficulties in clinical implementation with specialized equipment, respectively. QdMRI is a purely image-based approach that is designed to utilize images acquired under free tidal breathing conditions, making it non-invasive and practical for implementation in clinical practice. QdMRI allows one to build one 4D image within one breathing cycle, and thoracic structures can be segmented from the 4D image by applying deep learning based automatic segmentation approaches, such that it is feasible to make measurements of properties such as tidal volume and tissue parenchymal properties for quantitative analysis. By combining it with our existing open-source software, CAVASS (computer-assisted visualization and analysis software system, http://www.mipg.upenn.edu/Vnews/mipg_software.html), QdMRI can generate visualizable 3D surface renditions for each 3D volume in one breathing cycle. QdMRI has been utilized to depict changes in regional thoracic dynamic function for TIS patients [7] and healthy children [8] via assessment of volumes at end of inspiration (EI) and end of expiration (EE) for each lung, hemi-diaphragm, and chest wall during one free-breathing cycle. QdMRI has also been applied to study the relationships between thoracic component tidal volumes and spinal curve type for TIS patients [9]. We have recently also utilized QdMRI to quantify respiratory function in patients with early onset scoliosis (EOS) and to assess the effects of placement of rib-based anchors in terms of impairment of chest wall motion [10, 11]. As such, we believe that this system will be useful for clinical and research application purposes, and as far as we know, no such system exists. The purpose of this paper is to introduce our software system from the image processing aspect, namely a quantitative dynamic MRI based software system for the comprehensive analysis of regional and global thoracic dynamics.

2. Materials and methods

A simplified architectural diagram of the QdMRI system is shown in Figure 1, which includes 4 major modules: dynamic MRI acquisition, 4D image construction, image segmentation (from 4D image) and visualization of segmentation results, dynamic measurements, and comparisons of measurements from TIS patients with those from normal children to assess patient status pre-operatively or change post-operatively.

Figure 1.

A schematic representation of the QdMRI system.

Image data:

88 dynamic MR images through the thorax and upper abdomen were obtained from 44 TIS patients before and after surgery among a cohort of 61 TIS patients. TIS patients with insufficient clinical or radiographic data were excluded. We also collected 124 dynamic MR images from normal children with the following exclusion criteria: (i) history of thoracic surgery; (ii) history of asthma or other lung disease; (iii) respiratory tract illness within the last 30 days; and (iv) history of scoliosis or other congenital skeletal abnormality. We included normal children in our study so that we can quantify and describe changes in normal thoracic dynamics during childhood maturation via QdMRI and to create a normative reference database to help assess patients with TIS.

QdMRI system modules:

1. Dynamic MR image acquisition: The thoracic dMRI protocol was performed as follows: 3T MRI scanner (Verio, Siemens, Erlangen, Germany), True-FISP (bright-blood) sequence, TR=3.82 ms, TE=1.91 ms, voxel size ~1×1×6 mm³, 320×320 matrix, bandwidth 258 Hz, and flip angle 76°. For each sagittal location across the thorax, 80 slices were obtained over several tidal breathing cycles at ~480 ms/slice. The image acquisition module provides scans (in DICOM format) for use by the 4D image construction module. A sparse dMRI imaging approach is also investigated, where we sample fewer image signals in both time and space but without significant loss of the ability to depict or quantify thoracic dynamics [12]. The scanning time can be reduced from 45 minutes for the above full scan to 15–20 minutes per subject for the sparse approach.

2. 4D Image construction: Supposing that the dynamic MR image from step (a) covers ~35 locations in the sagittal plane, with 80 sampling slices in temporal space for each location, we would have 2800 (35×80) 2D sampled slices for one subject. A subset (around 175–315) of all spatio-temporally sampled 2D MRI slices of the dynamic thorax are chosen to construct one representative 4D image comprising one full breathing cycle for each individual subject. One cycle might have 5 to 9 time points/ respiratory phases. The QdMRI system includes our recently developed optical-flux-based 4D image construction approach [6], which is a purely image-based and fully automated algorithm. Overall tested data sets and cycles for all subjects, the temporal disorderliness of this method is less than 0.1, and the objective non-smoothness factor is less than 1-pixel unit. The input for this module is DICOM format files, and the output is one 4D image. More details of 4D image construction can be found in [6].

3. Lung segmentation from 4D image: A 2D U-Net [13] type deep learning (DL) network was implemented to segment each lung in the constructed and intensity corrected/standardized 4D image [14,15]. Intensity standardization, which has been recently referred to as image harmonization [16] or image normalization [17], allows MR intensity values to attain tissue-specific numeric meaning. Previous research has shown that these operations are vital for quantitative analysis of MR images [15]. In this study, calibration parameters for intensity standardization were estimated based on normal children data set and were utilized to standardize both normal-subject and patient data sets. More details of intensity standardization can be found in [15].

   The input for the lung segmentation module is a 4D image and the output is binary masks of each lung at EE and EI. The lung segmentation network includes a localization neural network to catch the ROI (region of interest) of the lung using a 2D bounding box, and a delineation neural network to detect the boundary of the lung [18]. The cross-entropy and L1 loss functions were combined together to define the loss function in the localization network, and a false positive and false negative plus DICE loss function [19] was used to train the lung segmentation network. More details can be found in another one of our SPIE conference papers [18].

4. Dynamic measurements: Based on lung segmentation from the 4D image, the QdMRI system can output the volumes of the lungs at EE and EI, and chest wall and hemi-diaphragm excursion volumes from EE to EI can be derived [7, 8]. Furthermore, lung standardized signal intensity measurements can be performed for lung parenchymal analysis [20]. As a side-product, the respiratory rates for individual patient breathing cycles can also be estimated.

5. Statistical analysis: Using volumetric and intensity-based measures from TIS patients (before and after surgery) and normal children obtained via QdMRI, one can compare TIS patients to normal children to obtain regional and global quantitative markers of the deviation of thoracic respiratory function from normal. For example, the Mahalanobis distance may be used to evaluate the difference of measurements from TIS patients before and after surgery, or to evaluate the difference of measurements from TIS patients with those of healthy children [21].

3. Results

Figure 2 shows the graphic user interface (GUI) of the QdMRI system, with the input of dMRI DICOM files. There are display windows to show the slice at a specific location in a single image or in a cine/continuous mode. After clicking the button of “Construct” or “Segment”, another separate window will show the progress of image construction and segmentation.

Figure 2.

Graphic User Interface of QdMRI system with one subject dMRI scan as input.

The QdMRI system can generate lung segmentation results and 3D surface renditions for a subject at EE and EI as shown in Figure 3. The 3D surface rendition can be visualized via the open-source software, CAVASS [22], or can be saved in STL file format and then utilized by other 3D graphic software programs. Table 1 displays some representative QdMRI volumetric parameters of the left and right lungs, left and right lung tidal volumes, and left and right excursion volumes of chest wall and hemi-diaphragm in a healthy 8.2 year old female and with body mass index (BMI) 23.5 kg/m², including the symmetry measure of α(RLtv, LLtv) = (RLtv − LLtv) / (RLtv + LLtv). We also report chest wall and diaphragm tidal excursion volumes as well as the symmetry between left and right components for chest wall and diaphragm.

Figure 3.

Top row: Lung segmentation and 3D surface rendition results at end expiration (EE) and end inspiration (EI) for a normal child. Bottom row: 3D renditions of the surfaces at EE and EI before surgery (left) and after surgery (right) for a TIS patient of comparable age and same gender.

Table 1.

Representative QdMRI parameters from the healthy subject shown in Figure 3.

	LL	RL	LCW	RCW	LD	RD
EE	307.684 cc	448.061 cc	-	-	-	-
EI	378.401 cc	553.343 cc	-	-	-	-
Tidal volume (tv)	70.716 cc	105.282 cc	16.002 cc	28.116 cc	41.492 cc	68.411 cc
Bilateral tv (Bxtv)	159.510 cc		44.118 cc		109.903 cc
%Lxtv, %Rxtv	43.14%	56.86%	36.27%	63.73%	37.75%	62.25%
α(Rx, Lx)	0.15		0.27		0.24

EE: end expiration, EI: end inspiration.

LL: left lung, RL: right lung, LCW: left chest wall, RCW: right chest wall, LD: left hemi-diaphragm, RD: right hemi-diaphragm.

%Lxtv = Lxtv/Bxtv, %Rxtv = Rxtv/Bxtv with x= L (lung), CW (chest wall), or D (hemi-diaphragm).

α(Rx, Lx) = (Rx − Lx) / (Rx + Lx), with x= L (lung), CW (chest wall), or D (hemi-diaphragm).

Our current platform for the QdMRI system runs on a computer with i9 CPU, 64 GBytes CPU RAM, Ubuntu 18.04 with Nvidia RTX2080, CUDA 10.2, and TensorFlow 2.0.0. The time cost for the whole procedure is as follows: The scanning time/image acquisition for one subject is around 15–20 minutes, 4D construction is around 5 minutes, segmentation via deep learning is 70 seconds for all time points, and measurements take around 2 seconds. We designed image construction in a parallel mode with multithread programming on more CPUs to further reduce the image construction time. Of course, higher level GPUs can also save time in image segmentation. In the current version of QdMRI, the auto-segmentation for lungs in normal children performs well with an average DICE of 0.96±0.01. Lung auto-segmentation in TIS patients is more challenging with an average DICE of 0.87±0.06, which is still being improved.

4. Conclusions

In this paper, we introduce QdMRI as a software system from the image processing aspect. The system provides a comprehensive analysis of global and regional thoracic dynamics by including image acquisition, 4D image construction, 4D segmentation, and quantitative measurements for both TIS patients and normal children. Such a system does not currently exist for the analysis of pediatric or adult patient thoracic dynamics.

Although lung segmentation in TIS patients is still challenging due to thoracic deformity and low MR image quality, we obtained acceptable results and are developing a minimally interactive deep learning approach for this purpose. Once we finish building a normative pediatric QdMRI database across ages 6–18 years for both boys and girls, the data sets and parameters will be available to quantitatively assess regional lung function during childhood maturation. The QdMRI system can be utilized for scoliosis-related applications not only in children but also in adults, as well as in numerous other scenarios where a dynamic 4D image analysis technique is needed. Furthermore, it will enable new opportunities to better understand TIS and other conditions that affect thoracic function in individual patients, as well as the effects of therapeutic interventions, both in children and in adults.