Generation of synthetic megavoltage CT for MRI-only radiotherapy treatment planning using a 3D deep convolutional neural network

Abstract

Background:

Megavoltage computed tomography (MVCT) has been implemented on many radiotherapy treatment machines for on-board anatomical visualization, localization, and adaptive dose calculation. Implementing an MR-only workflow by synthesizing MVCT from MRI would offer numerous advantages for treatment planning and online adaptation.

Purpose:

In this work, we sought to synthesize MVCT (sMVCT) datasets from MRI using deep learning to demonstrate the feasibility of MRI-MVCT only treatment planning.

Methods:

MVCTs and T1-weighted MRIs for 120 patients treated for head-and-neck cancer were retrospectively acquired and co-registered. A deep neural network based on a fully-convolutional 3D U-Net architecture was implemented to map MRI intensity to MVCT HU. Input to the model were volumetric patches generated from paired MRI and MVCT datasets. The U-Net was initialized with random parameters and trained on a mean absolute error (MAE) objective function. Model accuracy was evaluated on 18 withheld test exams. sMVCTs were compared to respective MVCTs. Intensity-modulated volumetric radiotherapy (IMRT) plans were generated on MVCTs of four different disease sites and compared to plans calculated onto corresponding sMVCTs using the gamma metric and dose-volume-histograms.

Results:

MAE values between sMVCT and MVCT datasets were 93.3 ± 27.5, 78.2 ± 27.5, and 138.0 ± 43.4 HU for whole body, soft tissue, and bone volumes, respectively. Overall, there was good agreement between sMVCT and MVCT, with bone and air posing the greatest challenges. The retrospective dataset introduced additional deviations due to sinus filling or tumor growth/shrinkage between scans, differences in external contours due to variability in patient positioning, or when immobilization devices were absent from diagnostic MRIs. Dose distributions of IMRT plans evaluated for four test cases showed close agreement between sMVCT and MVCT images when evaluated using DVHs and gamma dose metrics, which averaged to 98.9 ± 1.0% and 96.8 ± 2.6% analyzed at 3%/3mm and 2%/2mm, respectively.

Conclusions:

MVCT datasets can be generated from T1-weighted MRI using a 3D deep convolutional neural network with dose calculation on a sample sMVCT in close agreement with the MVCT. These results demonstrate the feasibility of using MRI-derived sMVCT in an MR-only treatment planning workflow.

1. INTRODUCTION

Over the last decade there has been a substantial increase in the use of magnetic resonance imaging (MRI) in radiotherapy due to its superior soft tissue contrast relative to computed tomography (CT)¹. A critical step in radiotherapy dose calculation relies on obtaining patient-specific maps of electron density, typically derived from the Hounsfield Units (HUs) of CT scans acquired for treatment planning. Therefore, it is common practice in radiation oncology to acquire both an MRI (for anatomical visualization) and a CT (for electron density mapping). However, eliminating the CT scan and implementing an MR-only workflow would offer numerous advantages including streamlined clinical efficiency, removal of CT-to-MRI registration errors, and a reduction in the number of imaging procedures required for each patient². Early in the roadmap to developing an MR-only workflow, many groups investigated deriving synthetic CT images from MRIs for mapping electron density using voxel-based, atlas-based, or hybrid techniques^3–5. In recent years, machine learning algorithms such as deep convolutional neural networks (DCNNs) have demonstrated excellent results for medical image synthesis when translating between MRI and CT^6–8.

Of the techniques used to generate synthetic CT from MRI, all have focused on the synthesis of CT images acquired in the kilovoltage (70–140kVp) energy range, which remain the standard imaging modality in radiotherapy^9,10. kVCT scans exhibit high signal-to-noise (SNR) and contrast-to-noise (CNR) while providing sufficient electron density for photon radiotherapy. CT HU-to-electron density calibration curves are input into treatment planning systems for purposes of dose caculation^11,12 and provide dose calculation accuracy to within 2% even with large inaccuracies in calibration curves¹³. In addition to the conventional CT scan used for initial radiotherapy treatment planning, volumetric on-board imaging has been integrated into most linear accelerators (linacs) and proton therapy systems to provide image-guided radiotherapy (IGRT), allowing for pre-treatment localization, monitoring of anatomical changes, and online plan adaptation. On-board cone-beam CT (CBCT), for example, is routinely used for these purposes and has demonstrated utility for daily dose reconstruction^14–16. While kV imaging systems are widely used in many systems, they can introduce several disadvantages for treatment planning and dose calculation. Onboard kV CBCT imaging requires that additional hardware be integrated into the accelerator, resulting in different imaging and treatment isocenters. Additionally, kV imaging can be susceptible to artifacts caused by metallic material such as dental fillings or prosthetics^17,18. Megavoltage (MV) images mitigate many of the disadvantages exhibited by kV images and have been used to for both image guidance¹⁹ and for generating electron density maps for dose calculation^20–23 and adaptive radiotherapy planning^24–27. By utilizing the same high-energy x-ray beam for both imaging and treatment, the MV imaging isocenter coincides perfectly with the treatment isocenter while also evading problems related to streaking artifacts and beam hardening due to its higher photon energy spectrum. Several commercially available options for on-board volumetric MV imaging include the MVCT platform integrated into the TomoTherapy system (Accuray, Sunnyvale, CA)^28,29 and the MV-CBCT imaging platform integrated into the Halcyon system (Varian Medical Systems, Palo Alto, CA)³⁰.

In addition to MV and kV imaging, MRI is also being used for IGRT, offering real-time, continuous soft tissue visualization immediately prior to and during treatment delivery^31,32. Combined 0.35–1.5 T MRI-linacs have gained substantial popularity over the last several years³³ with a number of commercial options currently available. In a typical MR-linac treatment planning workflow, a CT alone or both CT and MRI are acquired for target definition and electron density mapping for dose calculation 1–2 weeks prior to the beginning of treatment³⁴. During online treatment adaptation, updated structure contours and HU values are acquired through deformable image registration of the pre-treatment MRI to the original treatment planning CT. Of note, some MR-linacs, such as the Elekta Unity system, come equipped with MV panels that can be used for portal imaging³⁵ and it is conceivable that these panels may also be used for online volumetric MV image reconstruction in the future.

In light of the advantages provided by MV imaging for dose calculation and availability of MV hardware-equipped MRI-linacs, we propose a clinical pipeline in which MRI-derived sMVCT be used for initial treatment planning and the already present high-energy MV photon beam for online dose adaptation on the MRI-linac. This paradigm shift would offer a solution that eliminates the need for kV imaging systems for both initial treatment planning and online plan adaptation. Furthermore, undesirable characteristics of MV imaging, such as low SNR and CNR, are less important if the CT scan is being used purely for photon dose calculation rather than providing anatomical detail, as proposed in this workflow. This study is the first to demonstrate that synthetic MVCT images can be generated from MRIs using a 3D U-Net deep learning model and used for treatment planning of head and neck cancer, illustrating the feasibility of our proposed workflow whereby the MRI is utilized for anatomical delineation while the MRI-derived sMVCT is used for radiotherapy dose calculation.

2. MATERIALS AND METHODS

2.A. Image data and preprocessing

T1-weighted (non-contrast) MR and MVCT images for 120 head and neck (H&N) cancer patients treated on an Accuray TomoHD system were retrospectively analyzed in this IRB-approved study. Diagnostic MRI scans were acquired in the head-first supine position approximately 0–6 weeks prior to treatment on 1.5 T scanners using a T1-weighted 3D sequence without contrast (repetition time 500–800ms, echo time 7–15ms, flip angle 70–110°, voxel size 1×1×3mm³). MVCT images were acquired in the head-first supine treatment position using a 5-point head and shoulder thermoplastic mask and tongue stent immediately prior to treatment delivery for purposes of image alignment on an Accuray TomoHD system (3.5MV photon spectra) with in-plane resolution 1×1mm² and slice thickness 3mm. For each patient, MVCT scans were rigidly co-registered to the space of their respective MRI scan in MIM (MIM Software Inc, v 6.8.3 Cleveland, OH) according to our institution’s clinical practice using automatic maximization of mutual information, followed by manual inspection and adjustment by a clinician with >8 years of experience in image fusion. MVCT images were cropped to match the MRI field-of-view and resampled to a uniform 512×512 matrix using nearest neighbor interpolation. All MR images were histogram-matched³⁶ and normalized to standardize intensity values of images acquired across different scanners. The final dataset of 120 paired MRI-MVCT images was divided into an 80%−5%−15% train-validation-test split for the deep learning model with 18 test datasets.

2.B. Deep learning architecture and model

A deep neural network based on a fully convolutional 3D residual U-Net architecture was implemented to map MR images to sMVCT images as shown in Figure 1. Input to the model were approximately 200,000 volumetric patches (dimensions ~64×64×32) generated from 100 paired 3D T1-weighted MRI and MVCT datasets. The encoding path consisted of the repeated application of two 3×3×3 convolution kernels, rectified linear unit (ReLU), group normalization, and 2×2×2 max pooling. The decoding path consisted of the application of two upsampling 3×3×3 convolution kernels, ReLU, and group normalization. Skip connections were used to feed outputs of encoding layers to corresponding decoding layers to recover spatial information lost during downsampling. The single-channel model output was a 3D sMVCT.

Figure 1.

U-Net architecture used in this study. Input to the model were 3D T₁w MRI patches and output were 3D sMVCT for the same patches.

Network and hyperparameter validation were performed on the validation dataset with the final U-Net initialized using random parameters and trained for 100 epochs using the Adam optimizer (initial learning rate 10⁻⁴) on an objective that minimizes the mean absolute error (MAE) between the measured 3D MVCT HU map and MRI-derived sMVCT HU map:

$$MAE=\frac{1}{N}{\sum }_{i=1}^N\left|MVCT\left(i\right)-sMVCT\left(i\right)\right|$$ (1)

In Equation 1, N is the total number of image voxels and MVCT(i) and sMVCT(i) are the ith voxels in the real and synthetic MVCTs, respectively. During validation, model optimization was run long enough to qualitatively produce realistic sMVCT images and minimize the MAE between real and synthetized MVCT images in the validation dataset. Model accuracy was evaluated on images in the test dataset that were withheld during model training. The U-Net was implemented in PyTorch and trained on a single NVIDIA Titan RTX 24GB GPU using a batch size of 60. Training the U-Net model took approximately 120 hours and reconstructing each 3D image dataset using the fully trained model took less than 1 minute. Inference was performed by dividing 3D H&N volumes into smaller overlapping volumetric patches (dimension ~[64,64,32]) with 50% overlap prior to applying the 3D U-Net. Volumes were reassembled by taking the sample mean of the resulting outputs at each 3D grid position to ensure the sMVCT was processed at native resolution.

2.C. Qualitative and quantitative evaluation of sMVCT images

Qualitative differences were assessed through visual side-by-side inspection of sMVCT versus MVCT images. The qualitative factors evaluated included overall shape of the anatomy, similarity in tissue contrasts, and the ability to distinguish air from bone, which remains an ongoing challenge in MR-to-CT synthesis given that both manifest as signal voids in most MR images³⁷. Quantitative differences were assessed in different tissue types by segmenting MVCT images into whole body, soft tissue, bone, and air-filled volumes. All images were masked to remove material outside of the patient external contour, including air and setup devices. The whole body was defined as all material within the mask, which covered tissues inside the head-and-neck field-of-view for each patient. MVCT HU threshold values for soft tissue and bone were determined by mapping typical kVCT HU to MVCT HU values through an electron density calibration curve. As such, within the whole body, the soft tissue volume was defined as voxels in the −120 to 100 HU range and the bone volume was defined as voxels with HU value greater than 100 HU. For each tissue type, MAE (mean and standard deviation) was calculated between sMVCT and MVCT images (Equation 1) and displayed as violin plots to visualize the error distribution for different tissue types in test patient datasets. Paired histograms for MVCT and sMVCT test datasets were generated to compare HU values at the voxel level. To quantify model performance in distinguishing bone versus air, dice coefficients were calculated between MVCTs and sMVCTs within the body, bone, and air-filled structures.

2.D. Radiotherapy treatment planning

Of the 18 test datasets, four representing a diversity of H&N sites including base-of-tongue, thyroid, neck, and nasal cavity were used to generate radiotherapy treatment plans to assess dosimetric impacts of using sMVCT images. Treatment planning was performed in RayStation (v.7.0 RayStation Laboratories, Stockholm, Sweden) using 6MV photon coplanar volumetric modulated arc therapy and an MVCT HU-to-electron density CT calibration curve for dose calculation. CT calibration curves were generated using CT data of a commercial tissue surrogate phantom acquired within 6 months of image acquisition as previously described by our group³⁸ to avoid instabilities in the calibration curve caused by target degradation³⁹. A collapsed cone convolution superposition algorithm was used for dose calculation. Contours were copied from the original CT planning dataset to the co-registered MVCT and sMVCT datasets. In cases where the MRI field-of-view was limited, contours were edited to be fully covered within the external contour. Planned dose, organ-at-risk (OAR) constraints, and planning target volume (PTV) coverage were reproduced based on the treatment plans used clinically for each patient. Dose was normalized to 95% of the PTV receiving 100% of the prescription. Plans generated on the MVCT datasets were recalculated onto sMVCT datasets using fixed plan parameters. Dose distributions for MVCT and sMVCT were compared by assessing several different clinically relevant dose-volume-histogram (DVH) endpoints and the gamma index⁴⁰ calculated at 3%/3mm and 2%/2mm locally at a 10% dose threshold.

3. RESULTS

3.A. Qualitative and quantitative evaluation of sMVCT images

Figure 2 shows T1-weighted MRI, MR-derived synthetic MVCT, real MVCT, and HU difference maps for four example test datasets. Qualitatively, synthetic MVCT images have similar contrast to real MVCT images with structural details including interfaces between air, bone, and tissue appearing well preserved in the synthetic images. For example, bony regions between synthetic and real images were generally well correlated in the mandible (Figure 2a), skull (Figure 2b,c), and vertebral bodies (Figure 2a,d). Structures containing air, such as the trachea (Figure 2a,d), also appeared well preserved.

Figure 2.

Axial slices of four representative T1-weighted MRI, sMVCT, MVCT, and HU difference map. All CT images are displayed using a range of 0 to 2000 HU for visualization. Dice coefficients for whole body structures were 0.97, 0.98, 0.98, and 0.96 for patients a, b, c, and d, respectively. Dice coefficients for bone structures were 0.65, 0.56, 0.73, and 0.53 for patients a, b, c, and d, respectively. Dice coefficients for air-filled structures were 0.64, 0.55, 0.67, and 0.57 for patients a, b, c, and d, respectively.

Across the 18 test patients, the MAE (mean ± standard deviation) between sMVCT and MVCT averaged over all voxels in the whole body, soft tissue, and bone volumes was 93.3 ± 27.5, 78.2 ± 27.5, and 138.0 ± 43.4 HU, respectively. Dice coefficients (mean ± standard deviation) between sMVCT and MVCT whole body, bone, and air-filled structures were 0.97 ± 0.01, 0.66 ± 0.09, and 0.60 ± 0.11, respectively. Mean error between sMVCT and MVCT images are displayed in Figure 3 for the four test patients used for radiotherapy treatment planning analysis (four left-most columns), as well as the average error over all 18 test patients (right-most column). As expected, error in soft tissue volumes was the lowest while error in bone was highest for all test datasets. Patient 4 appears to be an example in which bone and soft tissue were both underestimated in the sMVCT versus MVCT, which we suspect was caused by missing tumor volume and underestimation of bone near the auditory canal in the MR-generated sMVCT, which can be seen in Figure 6. At the time the MRI was acquired (about three weeks prior to the MVCT) the tumor was smaller, therefore, the MRI-predicted sMVCT exhibited less tissue within the sinus versus the MVCT.

Figure 3.

Violin plots depicting the error in whole body, soft tissue, and bone voxels between sMVCT and MVCT in test patients 1–4 (four leftmost columns), in addition to all error average over all 18 test patients (rightmost column). Patient 4 appears to be an example case in which bone and soft tissue were both underestimated in the sMVCT, which we suspect was caused by missing tumor volume and underestimation of bone near the auditory canal.

Figure 6.

Axial dose distributions for base-of-tongue, thyroid, neck, and nasal cavity treatment plans computed on real MVCTs and synthetic MVCTs with corresponding DVHs.

Differences between sMVCT and MVCT images could occasionally be seen when the tissue positioning or patient anatomy were different between the MRI and MVCT in our dataset, as shown in Figure 4. For example, the top panel of Figure 4 shows changes in the nasal cavity filling while the bottom example highlights differences caused by immobilization devices such as oral stents and thermoplastic head-and-neck masks which are often used in radiotherapy but not in diagnostic MRI.

Figure 4.

Axial slices of two representative T1-weighted MRI, sMVCT, MVCT, and HU difference map for cases demonstrating limitations in the dataset including anatomical differences and presence or absence of patient immobilization devices as shown by the yellow arrows.

Joint histogram plots of sMVCT HU and MVCT HU for all 18 test patients are shown in Figure 5. Some sMVCT datasets demonstrated lower intensity HU distributions versus the corresponding MVCT (for example, patients 4 and 8). This was typically seen in patients who had less soft tissue in the MRI (and thus, sMVCT) versus the MVCT. In most of these cases, the tumor increased in size between MRI acquisition and treatment commencement (when the MVCT was acquired).

Figure 5.

Heatmap plots for MVCT HU (x-axis) and sMVCT HU (y-axis) across the entire FOV for the 18 test patients. Points coinciding with the 45° reference line represent agreement between real and synthetic images. Treatment plans were generated for patients 1–4 and are depicted in Figure 6. HU values between −500 and 500 are shown for figure clarity as most values fell within this range.

3.B. Dosimetric evaluation of treatment plans

Figure 6 shows example dose distributions calculated on real and synthetic MVCT test image sets across four different H&N sites. Dose volume histograms for the PTV and relevant OARs are shown. Dose distributions are visually very similar between real and synthetic CT images. The majority of DVHs directly overlay each other, indicating close agreement in dose distributions between structures. Notable exceptions were the PTV and eye structures of test patient #4, who was treated to a tumor volume within the nasal cavity. We believe dose distribution disagreements in the PTV were driven by tumor growth within the sinus cavity that occurred between MRI and MVCT image acquisition. Small dose distribution disagreements were also seen in the eye structures for this patient because the eyes were relatively small structures directly abutting the PTV, therefore their DVHs were particularly sensitive to small deviations in dose distributions between the MVCT and sMVCT.

Table 1 shows dosimetric differences between the four treatment plans for PTVs and relevant OARs. Doses to 1% volume (D1), 95% volume (D95), 99% volume (D99), and mean dose are provided. Dose differences to all structures were within 1Gy (relative to 66–70Gy total prescription dose) except for the PTV, brain (D1), and cochlea for patient #4, which demonstrated maximum dose differences within 2Gy. In this case, sMVCT DVHs were systematically lower than MVCT DVHs due to significant changes in sinus filling between the datasets.

Table 1.

Dosimetric differences

		sMVCT-MVCT (cGy)
		D1	D95	D99	Mean
#1 Base-of-tongue	PTV	37	31	33	33
	Spinal cord	20	0	0	8
	Oral cavity	48	0	0	23
	Submand Glnd (L)	24	22	−6	15
	Submand Glnd (R)	31	−74	−82	−34
#2 Thyroid	PTV	48	5	−67	28
	Spinal cord	39	0	0	18
	Larynx	96	37	46	59
	Esophagus	27	−2	0	13
	Constrictor muscle	56	0	0	26
#3 Neck	PTV	17	13	35	24
	Parotid (L)	51	20	23	26
	Parotid (R)	59	49	23	41
	Submand Glnd (L)	0	31	4	26
	Submand Glnd (R)	0	11	12	−1
	Mandible	38	−3	−2	7
#4 Nasal cavity	PTV	−186	−134	−117	−173
	Brain	−149	−8	−1	−47
	Cochlea (L)	−118	−91	−84	−99
	Cochlea (R)	−197	−139	−133	−156
	Eye (L)	−54	−11	−11	−26
	Eye (R)	−67	−23	−19	−39

Dosimetric differences (cGy) between sMVCT and MVCT DVHs for structures of interest in each treatment plan.

Table 2 shows results of gamma values calculated to compare sMVCT and MVCT dose distributions using metrics of 3%/3mm and 2%/2mm at 10% local dose threshold criteria. Patients #1, #3, and #4 were within the 95% (2%/2mm) threshold typically used clinically for comparing two calculated dose distributions. The lowest gamma values were seen in patients #2 and #4. In the case of patient #2, minor dose disagreement was attributed to small differences in the tissue surface due to the immobilization used for the MVCT but not MRI, which impacted this plan more than others due to the location of the target volume being at the patient surface. Discrepancies in patient #4 were due to changes in sinus filling as previously described.

Table 2.

Gamma passing rates

	3%/3mm	2%/2mm
#1 Base-of-tongue	99.3	98.3
#2 Thyroid	97.5	93.8
#3 Neck	99.9	99.6
#4 Nasal cavity	98.9	95.4

Gamma passing rates between treatment plans computed with fixed plan parameters onto sMVCT and MVCT images and calculated using 10% local dose threshold at 3%3mm and 2%/2mm.

4. DISCUSSION

Our results suggest that synthetic MVCT images can be generated from T1-weighted MRIs using a 3D U-Net for H&N photon radiotherapy treatment planning. Synthetic MVCT images predicted by the model exhibited similar qualitative features to real MVCT images. MAE values measured between sMVCTs and MVCTs across whole body, soft tissue, and bone volumes were 93.3 ± 27.5, 78.2 ± 27.5, and 138.0 ± 43.4 HU, respectively, which is similar to deep learning approaches previously reported for synthetic kVCT. For example, Gupta et al reported MAE values of 81.0 ± 14.6, 17.6 ± 3.4, and 193.1 ± 38.3 HU for all whole body, tissue, and bone regions, respectively, when learning the relationship between T1 MR and kVCT⁴¹. For H&N-based MR-derived sCT, specifically, Wang et al reported MAE values of 97 ± 13, 131 ± 24, and 357 ± 44 HU for soft tissue, whole body, and bone, respectively, for T2 MR to kVCT learning⁴². In a systematic literature review of synthetic CT generation for MRI-only radiotherapy planning, voxel-based techniques using standard MRI sequences yielded MAE results between 11–135 HU and 91–99.9% gamma passing rate depending on MR technique, body site, and gamma criteria⁴. Of note, all previous studies in this realm investigated kV imaging, whereas our study investigated MV imaging. CT scans acquired at kV and MV energies yield different HU values for higher density materials such as bone, where the bilinear kV and linear MV CT calibration curves deviate from one another⁴³. For example, typical bony tissue with relative electron density of 1.2 will have HU values of ~400 in a kVCT and ~150 in an MVCT, which contributed to why our MV approach yielded comparable or lower MAE values in bone relative to prior kV approaches. In our work, dose distributions computed on synthetic images were very similar to those computed on real MVCT images, with gamma metrics >97% and >93% when computed at thresholds of 3%/3mm and 2%/2mm, respectively. Our dosimetric results are similar to those previously described in real versus synthetic CT comparisons, with Gupta et al⁴¹ and Wang et al⁴² also demonstrating overlapping DVH curves for treatment plans calculated on real versus synthetic images. While the four cases evaluated in this study help provide proof-of-concept of the MRI-to-MVCT model, clinical implementation of this technology would require evaluation and quality assurance across a larger number of datasets to quantify and mitigate any systematic under- or overestimations caused by the model used to generate synthetic images. In this work, we utilized the same rigid registration protocol that is implemented in our institution’s routine clinical practice. This leads to some observed ambiguity in distinguishing whether differences in real versus synthetic images were caused by anatomical misalignment or model performance. We expect that misalignment would be the most significant around the neck and jaw, which could be positioned differently between the MRI and MVCT images. This could potentially be alleviated through deformable image registration (DIR), but we chose not to use DIR in this study as its widespread clinical implementation in radiotherapy has been limited due to the large number of different algorithms and lack of quantitative quality control metrics for defining adequate registration⁴⁴. In examining images in the neck and jaw regions, we observed relatively good alignment with the rigid registration used for this study. We predict that improved anatomical alignment through DIR, MRI simulation using identical patient set-up, and/or a shorter time duration between MRI and MVCT acquisition will only increase model performance.

Despite the increased utilization of MRI in radiation oncology over the past decade, limitations remain for MR-only planning workflows. Geometric distortions, for instance, can compromise the spatial integrity of MRI images when used for radiation therapy planning⁴⁵. Comprehensive reviews of MRI distortions are available in the literature, but broadly speaking can be attributed to either the MRI system or to the patient anatomy. MRI system-level distortions are produced by phenomenon such as gradient nonlinearities, non-uniform main magnetic field, and eddy currents produced during sequences requiring rapidly pulsed gradients, such as diffusion weighted or echo planar imaging. These distortions become more pronounced further away from the magnet isocenter but can be largely mitigated through modern 3D distortion correction algorithms, increased gradient amplitude, increased bandwidth, and adequate shielding. For example, mean geometric distortions over a 40cm diameter volume-of-interest have been reported at <1 mm and <1.2mm for 1.5T and 3T diagnostic MRI scanners, respectively^46,47 while maximum geometric displacements of <1–2mm have been reported for both the Elekta Unity^48,49 and ViewRay MRIdian⁵⁰ systems. Patient-level distortions are typically caused by phenomena such as material magnetic susceptibility differences and chemical shifts. Susceptibility artifacts are most prevalent at air-tissue interfaces (for example, in the sinuses) and can be partially mitigated through patient-specific field map corrections. Without correction, distortions in the head can be as high as 0.3–0.6mm at 0.5T, 2–2.5mm at 1.5T, and 2–3.5mm at 3Tesla^51,52. Applying these corrections in the brain can decrease susceptibility-induced displacements from 4mm to 0.2mm⁵³. Despite these technical limitations, advantages of MRI-based planning can include more reliable anatomical definition and reduction in CT-MRI co-registration errors. CT-based contouring has been demonstrated as the most significant contributor to inaccuracies in radiotherapy with errors as high as 1–2cm reported in the literature^54–57. On the other hand, significant reductions in inter- and intra-observer variability have been reported with MRI-based treatment planning in disease sites such as the brain, prostate, and female pelvis^58–61. Furthermore, an MRI-only workflow would remove CT-to-MRI co-registration errors, which have been reported at 1–2mm depending on anatomical site^62,63. Overall, the benefits and limitations of CT-only, MRI-only, or combined CT-MRI workflows must be well understood by the clinical team and appropriate choice of imaging modality should be carefully considered in the context of a patients’ disease, anatomy, and other circumstances.

Many studies have been performed to investigate the translation between MRIs and traditional kVCT images for purposes of MR-only radiotherapy planning, but this work is the first to synthesize MVCT images from MRI. While advantages of MVCT include HU linearity with respect to material electron density and resistance to beam hardening effects^39,64–67, some disadvantages remain including lower image contrast and greater noise due to increased Compton scatter relative to the photoelectric effect²⁶. Image performance characterization of MVCT images has been reported in the literature. For example, Held et al²⁶ reported that MVCTs produced spatially stable image values regardless of image protocol and object size, despite being ~6x noisier than fan beam kVCT. MVCT image quality with respect to noise, uniformity, contrast, and spatial resolution was characterized by Meeks et al⁶⁴ who reported comparable image uniformity and spatial resolution to diagnostic fan beam kVCT. While MVCT images demonstrated reduced contrast resolution and signal-to-noise ratio, they were still able to provide sufficient contrast to delineate bone and soft tissue structures. Techniques have been implemented to minimize noise in MV imaging through statistical methods^68,69 and deep learning approaches^70,71, however, MVCT has been deemed adequate for photon dose calculation given linearity between contrast and electron density^20,72. Important to this work is that reduced SNR and CNR inherent to MV imaging becomes less important if the CT scan is being used purely for photon dose calculation, as proposed in this workflow, rather than providing anatomical detail.

Our results indicate that high quality synthetic CT images can be achieved even when derived from MR images acquired with some variability between scanners and acquisition parameters. A limitation of this study was due to the diagnostic nature of the MR image datasets, which resulted in different patient set-up and, oftentimes, a time delay between MR and MVCT acquisitions. Patients being treated for H&N cancer are particularly susceptible to experiencing changes in weight and/or tumor volume over the course of their treatment^41,73,74, suggesting that treatment planning images should be acquired as close to the beginning of treatment as possible to avoid differences between scans caused by anatomical changes in tumor volume, sinus filling, and/or weight. We anticipate these limitations will be largely mitigated through the utilization of radiation oncology-dedicated MR scanners for initial treatment planning, which will consist of standardized acquisition protocols, identical immobilization devices and patient setup as used for treatment, and a shorter time duration between simulation and treatment (ideally no more than 1–2 weeks). Furthermore, using onboard MV imaging for online dose adaptation, as in our proposed future workflow, will mitigate issues caused by anatomical changes occurring between original treatment planning images and onboard pre-treatment MRIs, eliminating the need for co-registration of these datasets. A remaining challenge when estimating photon attenuation maps from MRI includes accurately distinguishing between bone and air, as both produce low signal when imaged using conventional sequences. In this work, dice coefficients for whole body segmentation were 0.97 ± 0.01, while those for bone and air were lower at 0.66 ± 0.09 and 0.60 ± 0.11, respectively. To better distinguish between bone and air, methods such as ultrashort echo time (UTE) or zero echo time (ZTE) MRI utilize very short echo times capable of acquiring signal in bone (which has a very short transverse relaxation time)^75,76. While these sequences are not yet routinely acquired in most clinical radiation oncology practice, they show great promise for mapping photon attenuation from MRI⁷⁷ and will likely be implemented more routinely in future for this purpose. An additional strategy that could be used to increase the accuracy of mapping air is to modify the objective function utilized by the deep learning model. For example, an objective that uses different weights for different HU values to optimize towards specific HU values (for example, air) rather than a global MAE objective across all HU values within the body may result in superior mapping of air-filled structures.

While adaptive radiotherapy planning using combined MRI and MVCT for photon radiotherapy was the focus of this work, MVCT can also provide additional advantages in proton radiotherapy. Range uncertainty remains an ongoing challenge in proton radiotherapy, with uncertainties related to kV HU-to-stopping power ratio (SPR) calibration curves necessitating additional treatment margins on the order of 1–2% the proton range^78,79. In the energy range utilized for kV imaging acquisition, photons interact primarily though the photoelectric effect which is highly sensitive to the material molecular composition. For kVCT scans, the relationship between HU and electron density (to which SPR is linearly related) is not one-to-one and can exhibit uncertainties as large as 7%^43,80,81. In contrast, the primary mode of photon interaction for high energy x-rays is Compton scattering, which is predominantly dependent on electron density, resulting in a one-to-one relationship between HU and electron density at higher energies^82,83. MVCT has been demonstrated to provide higher accuracy for mapping both electron density and SPR^43,80 and deriving stoichiometric HU-to-SPR calibration curves⁸⁴. When it comes to both CT image synthesis for treatment planning and adaptive planning in proton radiotherapy, synthetic MVCTs may provide a more accurate alternative to synthetic kVCTs. Because higher energy accelerator requirements of MV fan-beam CT scanners limit their availability in the clinic, synthesizing MVCT images using deep learning offers a promising approach for utilizing these images in practice.

The U-Net is one of the most popular architectures for image-to-image domain translation and was used in this study to map MRI to synthetic CT using supervised learning^85,86. Other architectures incorporating weakly-supervised and unsupervised learning, such as generalized adversarial networks (GANs) and its variants (e.g. CycleGAN) have also demonstrated promising results for sCT generation^87,88, though initial evaluations of deep learning methods for medical image synthesis suggest that the supervised U-Net can achieve lower MAE, higher structural similarity, and higher peak signal-to-noise than synthetic CT images provided by traditional image transformation networks⁸⁹. Nevertheless, the method presented here is equally compatible with a GAN approach, which could be implemented by adding an additional adversarial loss to the objective in Equation 1. Overall, it is expected that the incorporation of additional loss components, modifications to the network backbone (3D U-Net), and future developments within the rapidly evolving field of image-to-image synthesis will further improve the results presented here.

5. CONCLUSIONS

In summary, this study is the first to synthesize MVCT images from T1w MR images using a deep learning U-Net model in a dataset of 120 patients treated for H&N cancer. MAE values between sMVCT and MVCT datasets were 93.3 ± 27.5, 78.2 ± 27.5, and 138.0 ± 43.4 HU for whole body, soft tissue, and bone volumes, respectively. Dose distributions evaluated for four test cases showed very close agreement between sMVCT and MVCT images when evaluated using DVHs and gamma dose metrics, which averaged to 98.9 ± 1.0% and 96.8 ± 2.6% analyzed at 3%/3mm and 2%/2mm, respectively. These results demonstrate the feasibility of using MRI-derived sMVCT in an MR-only treatment planning workflow.