Physics-Informed In-Silico Dynamic Computed Tomography of Human Lungs: Generation, Evaluation, and Refinement

Abstract

Lung injuries lead to heterogeneous ventilation behavior in the lung parenchyma, and conventional methods used to assess lung health, such as spirometry, fall short of providing regional information about lung function. Dynamic medical imaging and image registration offer a powerful tool for estimating the kinematic behavior of lung parenchyma in vivo. However, the difficulty of validating lung deformation estimated by image registration has curbed widespread adoption in the clinic. In-silico images, reconstructed from finite element (FE) simulations, provide a method to verify the results estimated through image registration (IR). Our objective in this study was to use in-silico computed tomography (CT) images, reconstructed from FE simulations, to assess the accuracy of an image registration method. In this study, we used dynamic CT (4DCT) images from human patients to reconstruct the lungs and generate an FE mesh. In-silico simulations were performed using the lung FE mesh, and the results were used to generate in-silico dynamic CT images matching the resolution of the actual 4DCT images. Image registration was performed on the actual and in-silico images, and the results were compared to those from the FE simulation. Results indicated good agreement in displacement estimated by the FE simulations and the image registration of the actual and in-silico CT images. The difference in predicted displacement image registration of the actual CT images and the FE simulations was greatest at the main bronchi, with a value of 2.7 mm. This result highlighted the effectiveness of the FE simulation-based method to generate in-silico CT images. The volumetric strain comparisons between actual 4DCT and the in-silico images were used to assess the method's accuracy. A new set of in-silico images was generated at a higher spatial resolution, resulting in improved agreement for the volumetric strain contours. We expect the method reported in this study to be applied to optimize medical imaging methods and investigate the behavior of various lung diseases under medical imaging.

1 Introduction

Lung injuries such as radiation-induced lung injury [1,2] and ventilator-induced lung injuries [3–5] can lead to spatial heterogeneities in ventilation. Conventional methods used to assess lung function and health, such as spirometry, provide primarily gross (overall) information on lung behavior at the organ level. There has been a growing interest in quantifying regional biomarkers to assess the health of the lung parenchyma [6]. Kinematic biomarkers, such as regional strain, can provide information on the probability of the onset and spread of lung injuries [7]. Recent ex-vivo studies on the surface of porcine lungs, conducted using digital image correlation, provided new insights into the effect of ventilation on regional lung strains [8,9]. Dynamic imaging and subsequent image registration can be used to assess the kinematic behavior of the lung parenchyma during in-vivo respiration [10–14]. Various algorithms of nonrigid image registration have been reported in the literature, such as block matching algorithms (e.g., Nifty-reg) [15,16] and Diffeomorphic Demons [17]. However, variations in results estimated by different image registration algorithms highlight the need for a benchmark that can serve as the ground truth to assess the accuracy of these algorithms.

The difficulty in validating regional strain biomarkers estimated through in-vivo image registration has been a major challenge against the widespread adoption of such biomarkers in the clinic. One common method to assess the performance of image registration methods is the estimation of displacements at major landmarks. These displacements are then compared against the displacement of these landmarks obtained through manual measurements. This method is limited to a few hundred landmarks at most, due to the time required to manually track landmarks through images. In addition, this method is most effective when the manual tracking is performed on airways.

In-vitro gel phantoms [18–20] have been used to assess image registration, where the phantoms are imaged while undergoing physical deformation through actuators. Image registration is performed on these images, and the results are compared against the prescribed deformation, which is measured through strain gauges. However, two important limitations of gel phantoms are (i) the difficulty associated with measuring regional strains in the bulk of such phantoms and (ii) the complex geometry and motion of the lungs, which can be difficult to replicate accurately. Furthermore, the complex microstructure of the lung parenchyma, particularly its high degree of porosity, can be challenging to replicate using gel phantoms.

In our previous study [7], we reported a method for generating in-silico computed tomography (CT) images from a finite element (FE) mesh to overcome the limitations of conventional gel phantoms. The primary advantage of an FE-mesh-based approach is that it can be used to generate realistic lung deformation through FE simulations, and FE simulations can be used to estimate regional strain throughout the bulk of the tissue. FE simulations provide the displacement and deformation behavior of the entire mesh and may serve as a reliable benchmark, since the results of FE simulations have been extensively studied and validated [6,21]. The well-established nature of FE modeling makes it a suitable benchmark to compare the results of image registration applied to dynamic CT (4DCT) images. Thus, results from FE simulations can serve as the ground truth for comparison with the results estimated through various image registration algorithms.

Dynamic computed tomography imaging of human patients is cautioned due to the excessive radiation associated with the method, thus limiting the available repository of 4DCT thoracic images. FE meshes can be used to generate dynamic CT images of both healthy lungs and lungs with various injuries and diseases, providing a robust training platform for the various deep learning tools being developed for lung image registration. The existing 4DCT images of patients can serve as the test set for such tools, thus significantly enhancing the training accuracy.

In this study, we reconstructed the lung geometry from one 4DCT image to create an FE mesh. This FE mesh was used to generate a set of in-silico images through FE simulations, which effectively served as an in-silico dynamic phantom. We performed image registration on the in-silico images and compared the results with those from the FE simulation and the image registration of the actual 4DCT images. The method presented in this study provides a pipeline for investigating the accuracy of image registration methods. In addition, in-silico phantoms can be used to simulate different imaging methods in order to assess their relative ability to evaluate various lung injuries. Similarly, phantom simulations can be used to better understand the manifestations of various lung injuries in medical images.

2 Methods

2.1 Segmentation and Reconstruction of Finite Element Mesh From Computed Tomography Scan Images.

Dynamic CT images were acquired from a subject undergoing a radiation therapy planning process (hereinafter referred to as actual 4DCT) to treat thoracic malignancies at the MD Anderson Cancer Center of the University of Texas, Houston, TX [22]. The resolution of the actual 4DCT images was 1.15 $\times$ 1.15 $\times$ 2.5 mm. Each 4DCT image set contained ten images acquired throughout one respiratory cycle, with five images capturing the inspiratory phase. The lungs and airways were segmented from the CT scans of the human lungs (n = 1) at end-expiration using materialise mimics to generate the lung geometry. This geometry was meshed using materialise 3-matic, with the nominal element size set to 1.5 mm. The edge length of the tetrahedral elements was determined based on two primary considerations. First, the edge length was required to be consistent throughout the entire lung geometry to ensure mesh uniformity. Second, it needed to represent the maximum allowable element size that would still maintain the morphological detail of the lung structure within the finite element mesh. The result is our initial lung phantom.

2.2 Physics-Informed Motion Registration to Replicate Lung Deformation.

The method to simulate respiration using the geometry at end-expiration was based on our previous work [23]. Physiological loads were applied to the diaphragm and outer surface of the end-expiratory mesh of the lungs to minimize the difference from the end-inspiratory mesh. Next, a displacement boundary condition was applied to the surface of the lungs to ensure that the surface matched the end-inspiratory mesh. A linear elastic model of the lungs was chosen for this work, as the focus was on the kinematic behavior of the lung parenchyma. Since the primary objective of the FE simulation was to match the end-inspiratory mesh, nominal stiffness values were selected, and the loads were modulated accordingly. The Poisson's ratio was set to 0.3 to enable compressible behavior. The modulation of loads was performed manually, with the primary objective being to minimize the difference between the end-expiratory and end-inspiratory mesh (Fig. S1 available in the Supplemental Materials on the ASME Digital Collection). Estimating displacement boundary conditions was automated based on the distance of each node on the surface of the deforming mesh to the surface of the target mesh. The accuracy of the simulation was estimated by calculating the Hausdorff distance between the geometry generated through the deformation of the end-expiratory mesh and the mesh reconstructed from the end-inspiratory phase of the 4DCT image.

2.3 Generation of Computed Tomography Images From the Finite Element Mesh.

Following the mesh generation, the FE lung mesh nodes were used to generate an in-silico dynamic image series. The method to generate in-silico images of our FE mesh was based on our previous study [24] but substantially modified to improve the replication of the CT imaging process. The voxel size of the in-silico reconstructed images was specified to match the voxel size of the actual 4DCT images (1.15 $\times$ 1.15 $\times$ 2.5 mm). The nodes of the FE mesh were binned into regions based on their height coordinate. All the nodes in each bin were flattened to the same height coordinate, and the size of the bin corresponded to the height of individual voxels. A simulated line emitter–detector pair was rotated around the lungs. The emitter was assumed to be a combination of an X-ray source and filters that emitted a parallel beam. The emitter projected a parallel beam toward the detector, and the nodes in the path of the beam were projected onto the line detector (Fig. 1). The projected points were used to generate a histogram whose bin sizes were set based on the required voxel size to create a sinogram image. The detector was rotated through 180 deg in steps of 1 deg at each height and then moved vertically. This process was repeated, and the sinogram images were stacked to generate a three-dimensional (3D) sinogram (Fig. 1). In addition, the emitter–detector pair was assumed to move in a circle, followed by a translation along the height axis, as opposed to the commonly used clinical method of helical motion. The 3D sinogram image was then used to reconstruct the CT image using the filtered back projection method with the Hann filter [25]. The Hann filter was chosen over the Ram-Lak filter to remove noise and graininess from the reconstructed image. This process was performed for each step in the FE simulation to generate a set of static CT images, which collectively served as a 4DCT image. Five images were generated throughout the inspiratory cycle to match the temporal resolution of the actual 4DCT images.

Fig. 1.

(a) Schematic illustrating the generation of the in-silico image derived from the finite element mesh. The number of nodes projected onto each pixel of the detector determines the intensity of the image. (b) The resulting 3D sinogram from the image generation.

2.4 Evaluation and Refinement of the In-Silico Images.

We used an open-source image registration algorithm (Nifty-reg) [15,16] to perform nonlinear nonrigid image registration to obtain voxel-based displacement. The lungs were segmented from the end-expiratory phase of the actual 4DCT image using slicer, and the segmentation was propagated through image registration to track the region of interest throughout the respiratory cycle.

The total displacement was calculated by summing the voxel displacements between consecutive timepoints, starting from the reference timepoint (at the beginning of inspiration) and ending at the current timepoint. Let $\mathbf{u}=[u_x\hspace{0.17em}{u}_y\hspace{0.17em}{u}_z]$ be the total displacement from the end-expiratory timepoint to the end-inspiratory timepoint in the undeformed configuration. The end-expiratory timepoint (timepoint with the lowest lung volume) served as the reference state when estimating displacement. The deformation gradient tensor $\mathbf{F}$ was calculated through the following equation:

$$\begin{array}{cc}\mathbf{F}& =\mathbf{I}+\left[\begin{array}{ccc}\frac{\partial u_x}{\partial X}& \frac{\partial u_x}{\partial Y}& \frac{\partial u_x}{\partial Z}\\ \frac{\partial u_y}{\partial X}& \frac{\partial u_y}{\partial X}& \frac{\partial u_y}{\partial Z}\\ \frac{\partial u_z}{\partial X}& \frac{\partial u_z}{\partial Y}& \frac{\partial u_z}{\partial Z}\end{array}\right]\end{array}$$ (1)

where $\mathbf{I}$ is the identity matrix, and [X Y Z] denote the coordinates in the undeformed configuration. $\mathbf{F}$ was used to estimate the right Cauchy–Green tensor $\mathbf{C}$ given by

$$\mathbf{C}={\mathbf{F}}^{\mathrm{T}}\mathbf{F}$$ (2)

The volumetric deformation J was estimated through the third invariant (determinant) of $\mathbf{C}$

$$J^2=\text{det}(\mathbf{C})$$ (3)

$J>1$ indicates volumetric dilation of the voxels, while $0<J<1$ indicates volumetric compression. This method of image registration and the calculation of volumetric strain was performed on the actual 4DCT image and the in-silico dynamic image generated from the FE simulations.

After evaluating the performance of the in-silico reconstructed images in the estimation of lung kinematic behavior, an additional in-silico dynamic image was generated at a smaller voxel size (higher resolution) (1 $\times$ 1 $\times$ 1 mm) to investigate the effect of voxel size on image registration performance.

3 Results

3.1 Error Assessment of Motion Registration Method.

The FE mesh was reconstructed from the geometry at end-expiration and end-inspiration (Figs. 2(a) and 2(b)). The end-expiratory FE mesh was used for FE simulations, and forces were applied to the diaphragm and outer surface of the lungs, such that it matched the end-inspiratory mesh. The results of the FE simulation were used to generate the in-silico dynamic CT images (Fig. 2(c)). The Hausdorff distance error on the surface was estimated to be 1.87 $\pm$0.83 mm with a median and maximum of 1.26 mm and 17 mm, respectively (Fig. 3(a)). In addition, the fractions of the lung surface where the Hausdorff distance exceeded 4 mm and 10 mm were 7.95% and 0.38%, respectively. The contour of the Hausdorff distance was limited to 4 mm to improve the visualization of the variation in Hausdorff distance error across the surface (Fig. 3(a)). Normalizing the error against the height of the lungs (250.38 mm) resulted in a mean percentage error of 0.7% and a maximum percentage error of 6.7%.

Fig. 2.

(a) Actual 4DCT image, (b) reconstructed lung geometry, and (c) in-silico dynamic image of the FE mesh at the end-expiratory and end-inspiratory phases of respiration. Scale bar represents 10 cm.

Fig. 3.

(a) Hausdorff distance error between the reconstructed end-inspiratory mesh (target) and end-expiratory mesh after FE simulation. (b) Maximum principal strain estimated by the FE simulation at the end of inspiration. The slice location for (b) is represented in the inset image.

3.2 Comparison of Displacement Estimated by Finite Element Simulation and Image Registration of the Actual Dynamic Computed Tomography, and In-Silico Images.

Estimating the strain in the FE simulation indicated that the maximum strain occurred in the region between the main bronchi and the diaphragm (Fig. 3(b)). The maximum (largest) principal strain in the lungs was found to be 0.204 $\pm$0.51. The maximum shear strain, computed as half the largest difference between principal strains, was found to be 0.187 $\pm$0.43. The comparison of displacement values indicated a good agreement between the displacement estimated through in-silico simulation and the image registration of the actual image and the in-silico image (Figs. 4(a)–4(c)). The maximum vertical displacement was found to be 11 mm, occurring in the inner half of the diaphragm. The error between the displacement of the actual 4DCT image and the FE simulation was greatest in the region below the left and right bronchi, when excluding the errors at surface edges caused by the motion registration (Fig. 4(d)). The error peaked in the region below the bronchi at 2.7 mm. The error between the displacement of the actual 4DCT image and the in-silico image was also highest below the left and right bronchi, but the values reached as high as 5.8 mm (Fig. 4(e)). The error was around 3 mm in the inner half of both lungs. In addition, the displacement halfway between the end-expiratory and end-inspiratory phases was also similar between the image registration of the actual image and the in-silico image (Fig. S2 available in the Supplemental Materials on the ASME Digital Collection).

Fig. 4.

Vertical displacement ( $u_z$) estimated through (a) image registration of the actual 4DCT image, (b) FE simulation of the reconstructed lung geometry, and (c) image registration of the in-silico dynamic image. Difference in estimated displacement between (d) actual 4DCT image and FE simulation, and (e) actual 4DCT image and in-silico dynamic CT image. The slice location is represented in the inset image.

3.3 Evaluation of Volumetric Strain and Refinement of In-Silico Images.

Comparison of volumetric strains revealed differences between the values estimated by image registration of the actual 4DCT images and those obtained by image registration of the in-silico images (Figs. 5(a) and 5(b)). Although the volumetric strain contours of the actual 4DCT images were qualitatively similar to their corresponding displacement contours, there were differences between those estimated from the in-silico image. The increased mesh resolution resulted in a moderate improvement and better qualitative agreement with the actual 4DCT images (Fig. 5(c)).

Fig. 5.

Volumetric strain (J) estimated through image registration of (a) the actual 4DCT image, (b) the in-silico dynamic image at the same resolution as the actual 4DCT image, and (c) high-resolution in-silico image.

4 Discussion

4.1 In-Silico Simulation Replicated In-Vivo Lung Deformation.

The method presented in this study was used to deform the end-expiratory lung geometry using physiological loads to match the end-inspiratory geometry. High Hausdorff distance error values between the end-expiratory geometry after the FE simulation and the end-inspiratory geometry were mainly observed at edges where the concavity of the surface changed. Additionally, the accuracy of the in-silico simulation was confirmed by the displacement contours, which were in good agreement with the displacement estimated by image registration. These results, along with our previous study [23], suggested that the in-silico simulation method can be used to estimate parenchymal deformation in vivo through the use of static CT imaging. Estimating displacement and strains without the high levels of radiation associated with dynamic CT imaging can facilitate the use of kinematic biomarkers to assess regional lung health and function.

4.2 In-Silico Image Performed Well in Comparison to the Actual Dynamic Computed Tomography Images.

The image registration results indicated good agreement between the actual 4DCT and the in-silico dynamic images for the displacement contour. The spatiotemporal resolution of the in-silico images was set to match that of the actual 4DCT images. The resolution was fixed to contrast the performance of the in-silico images with that of the 4DCT images when used to estimate regional kinematic biomarkers. Increasing the resolution to 1 $\times$ 1 $\times$ 1 mm demonstrated improvement in the volumetric strain contour and a better qualitative match with the actual 4DCT image. Increasing the accuracy of the in-silico images for a stronger agreement with the actual CT images will improve the capacity to generate physiologically realistic images of the lungs. However, further increases in the resolution will provide diminishing increases in accuracy as the spatial resolution of the in-silico images depends on the node density of the FE mesh. While the node density of the FE mesh can be increased for this purpose, such an increase also reaches a point of diminishing returns since the details captured by the FE mesh are dependent on the resolution of the actual CT image. Thus, using an actual high-resolution CT image would lead to the best image quality for the in-silico dynamic CT images.

4.3 Clinical Applications and Future Directions.

Various pathophysiological conditions can be simulated in silico using the phantom serving to simulate changes observable through CT imaging. Since the spatial resolution of the in-silico images depends on the actual CT image for reconstruction, a single static high-resolution CT image, followed by low-resolution dynamic CT images, can be used to generate synthetic high-resolution dynamic CT images. The resulting in-silico images can be used for image registration, enabling an improved understanding of regional parenchymal kinematics without subjecting the patient to excessive radiation. In addition, the generation of in-silico dynamic images can be used as training data for machine learning models to improve the training accuracy while minimizing the need for large repositories of thoracic CT images. The in-silico image generation method will be extended in future studies to account for the path length of the X-ray beam within the FE mesh when determining the final signal intensity. This extension is expected to enable the reduction of noise associated with the current method without the use of low-pass filters.

4.4 Limitations.

Only one 4DCT image was used for the investigation. The single case is insufficient to generalize the applicability of the reported model. Future studies will use multiple sets of 4DCT images to verify and validate the results presented in this study. Another limitation of this study is that the reconstruction of the final in-silico CT image was performed using the parallel-beam axial CT process and filtered back projection with a Hann filter. In contrast, conventional CT imaging employs conical beam helical imaging and complex signal processing and filtering methods to generate the sinogram and reconstruct the final CT image [25]. In addition, the accuracy of the in-silico images was assessed through visual inspection, with the current in-silico images being bright at the edges and noisier than the dynamic CT images. Future studies will aim to improve the processing method used to generate in-silico CT images and to improve the filtered back projection method. We expect these improvements to substantially reduce the difference between the in-silico images and dynamic CT images.

Image registration was performed using a single image registration algorithm to estimate kinematic biomarkers. Future studies will assess the performance of the in-silico image generation using multiple different image registration algorithms. Regarding the constitutive modeling, parenchymal tissue is commonly modeled as a hyperelastic material model [26]; however, since the primary objective of the FE simulation was to deform the end-expiratory mesh so that its surface matched that of the end-inspiratory mesh, a linear elastic model was sufficient to meet this objective. In addition, the stiffness of the material model can be assigned nominal values, since the loads can be adjusted accordingly to minimize differences between the meshes. However, the choice of material model becomes crucial when investigating stresses in lung tissue and their alterations in lung diseases and injuries. The choice of material would also become important in the presence of heterogeneities such as stiffer regions or nodules/tumors in the lungs. Therefore, an appropriate hyperelastic material model should be employed when investigating such cases.

5 Conclusion

In this study, we generated dynamic CT images based on an in-silico simulation of an FE mesh of human lungs. The deformation estimated through the image registration of the dynamic CT images was in agreement with that estimated through the image registration of the actual 4DCT images and those estimated by the in-silico simulation. The method presented in this study can be applied to assess the performance of image registration algorithms and assist with developing a pipeline for developing virtual medical imaging.

Supplementary Material

Supplementary Material

Supplementary Figure

Funding Data

• National Institutes of Health (R56HL172052; Funder ID: 10.13039/100000050).

• National Science Foundation (2244995; Funder ID: 10.13039/100000001).

Conflict of Interest

The authors declare no conflict of interest.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author, R.A., upon request.