Protocol selection formalism for minimizing detectable differences in morphological radiomics features of lung lesions in repeated CT acquisitions

Abstract.

Background

The accuracy of morphological radiomic features (MRFs) can be affected by various acquisition settings and imaging conditions. To ensure that clinically irrelevant changes do not reduce sensitivity to capture the radiomics changes between successive acquisitions, it is essential to determine the optimal imaging systems and protocols to use.

Purpose

The main goal of our study was to optimize CT protocols and minimize the minimum detectable difference (MDD) in successive acquisitions of MRFs.

Method

MDDs were derived based on the previous research involving 15 realizations of nodule models at two different sizes. Our study involved simulations of two consecutive acquisitions using 297 different imaging conditions, representing variations in scanners’ reconstruction kernels, dose levels, and slice thicknesses. Parametric polynomial models were developed to establish correlations between imaging system characteristics, lesion size, and MDDs. Additionally, polynomial models were used to model the correlation of the imaging system parameters. Optimization problems were formulated for each MRF to minimize the approximated function. Feature importance was determined for each MRF through permutation feature analysis. The proposed method was compared to the recommended guidelines by the quantitative imaging biomarkers alliance (QIBA).

Results

The feature importance analysis showed that lesion size is the most influential parameter to estimate the MDDs in most of the MRFs. Our study revealed that thinner slices and higher doses had a measurable impact on reducing the MDDs. Higher spatial resolution and lower noise magnitude were identified as the most suitable or noninferior acquisition settings. Compared to QIBA, the proposed protocol selection guideline demonstrated a reduced coefficient of variation, with values decreasing from 1.49 to 1.11 for large lesions and from 1.68 to 1.12 for small lesions.

Conclusion

The protocol optimization framework provides means to assess and optimize protocols to minimize the MDD to increase the sensitivity of the measurements in lung cancer screening.

1. Introduction

This study explores the application of quantitative imaging, a methodology seeking to extract measurable and objective data from radiological patterns. This extracted information holds the potential to enhance clinical decision-making and ultimately improve patient outcomes through more precise diagnosis and prognosis. Delta-radiomics features are well-known for their utility in multiple clinical applications, such as malignant tumor grading and noninvasive tumor analysis.¹ Delta radiomics in morphological radiomic features (MRFs) for lung lesions refers to the changes or differences observed in the quantitative analysis of morphological features extracted from radiological images of lung lesions over time or between different conditions. MRFs specifically pertain to the shape and size characteristics of lesions identified in medical images. In particular, MRFs have been recognized as factors relevant to the cause and progression of a disease.² However, their precise measurements have remained challenging as their measurements can be sensitive to variations in the scanning acquisition and reconstruction settings.^3–5 This is especially problematic in cases when disease progression cannot be reliably measured due to the impacts of the imaging protocols. The accurate measurement of disease progression is more challenging where there is a small biological change between a reference and the follow up scans. In such situations, it is unclear whether the changes indicate disease progression, response to treatment, or just variations in the applied imaging protocol.

To address this challenge, the quantitative imaging biomarkers alliance (QIBA) has released general guidelines for imaging protocols to minimize the variability across CT scans in the context of nodule volumetry.⁶ Many studies have also aimed to determine the optimal protocols for specific tasks, such as diagnosing or tracking the treatment response of a lung nodule.^7–10 The standard method for ensuring reproducibility of radiomics features relies on test–retest analysis in phantoms or patients.¹¹ In the majority of clinical trials, accessing the same patients who were imaged with multiple scanners is challenging.¹² Furthermore, relying solely on data-driven evaluation, it becomes challenging to simultaneously analyze multiple influencing parameters. In addition, identifying the optimal parameters based on the intercorrelation of scanner parameters, as dictated by the governing principles of CT, has not been addressed.^13–15

To establish the impact of imaging systems and protocols on delta-MRF, we developed a data-driven technique to assess quantification performance in terms of minimum detectable difference (MDD).¹⁶ The previous study¹⁶ established thresholds for MDDs that can be reliably measured from two imaging conditions. Using the same virtual imaging trial platform, in this study, we aimed to optimize the CT protocols for targeted MRFs; minimizing the MDD for each MRFs by selecting pair of imaging protocols that yield the lowest threshold for baseline and follow-up acquisitions assuming identical protocols for the two scans. We approximated parameterized models as surrogates of the MDDs for each MRFs to explain the relationship between the system parameters/lesion attributes and the corresponding thresholds. We then formulated optimization goals of minimizing the thresholds for each approximated model. To address the problems of protocol correlation, we identified correlation models to reflect the valid domain in which the imaging system’s physics remains feasible. Therefore, by solving the formulated optimization problem constrained with the correlation models, we obtained the optimal protocols for the targeted task, yielding the lowest threshold between two pairs of imaging scans. To investigate the variability and bias of the optimized protocols, we compared the MDDs derived by the proposed optimized protocols with those obtained by QIBA’s suggested protocols. The proposed framework offers a method of rigorously selecting optimal protocols. By employing a continuous surrogate function for the exploration of the feasible domain, the computational cost associated with intensive grid searches is minimized. Furthermore, the inclusion of physics-based correlation constraints ensures the practical feasibility of the solution. This approach facilitates the resolution of a multifactorial optimization problem, accounting for interactions among decision variables.

2. Methods

This section is structured as follows. We first explain the employed model selection to compute the MDD functions for each MRF. These MDD functions were parameterized over scanning parameters and lesion size. This follows with a description of the estimation of correlation models that define the relationships between protocols by utilizing the collected data in a phantom experiment. These correlation models delineate the permissible domain wherein the governing physics of the imaging system holds true. Subsequently, we formulate associated constrained optimization problems to minimize the MDDs over the feasible domain defined by the correlation models and boundary constraints. We target optimal protocols to minimize the MDD for each MRF, along with their connection to the lesion size. Finally, we describe a comprehensive examination of the variability and bias of MRF based on optimized protocols compared to those of the guidelines from QIBA.

2.1. Minimum Detectable Difference Calculation

MDD is a numerical measure that quantifies the least detectable change in a targeted measured feature between a pair of images.^16,17 Figure 1 shows a schematic of how MDD can be calculated, as reported in Ref. 16. A similar approach was utilized in this study to calculate MDDs for seven morphological radiomics features (MRFs) across various imaging conditions and lesions attributes. The mathematical expressions of the MRFs are provided in Table 1.

Fig. 1.

Schematic of the employed procedure to calculate the MDD values. We conducted a study on lesions with multiple instances of spiculations, where we employed active contour segmentation and extracted the ground truth MRF values. Next, using the same lesion models, we applied the MTF and NPS at different dose levels for the baseline acquisition. To do so, for each lesion size at a given spiculation level, five different noise realizations were generated. Subsequently, we segmented the noisy images with the same segmentation technique and obtained the MRFs. This procedure was repeated for follow-up acquisition settings and across various spiculation instances of the lesion at two different size categories. To establish a relationship between the measured difference features in noisy images and their corresponding ground truth difference at each binned spiculation, we utilized a linear model. The linear model maps the expected value of each measured MRF to the ground truth value. We then used the linear model to map the distribution of noisy image differences to the ground truth difference distribution. The 95% confidence of the cumulative distribution function was considered as the MDD for each MRFs.

Table 1.

List of the studied MRFs.

Feature	Description
Radius	$R=\frac{1}{n_b}\sum _I^{n_b}\Vert X_I-X_c\Vert$, where $n_b$ is the number of boundary voxels, $X_I$ is the boundary voxels, and $X_C$ is the center of mass coordinate of the lesion
Volume	$V$ is the volume of the binarized lesion’s segmentation mask, fitted to a mesh surface
Flatness	$F=\frac{{\lambda }_{\text{least}}}{{\lambda }_{\text{major}}}$, where ${\lambda }_{\text{least}}$ and ${\lambda }_{\text{major}}$ are the smallest and largest eigenvalue of the region-based 3D ellipsoid fitting, respectively
Major length	$L={\lambda }_{\text{major}}$
Surface area	$A$ is the surface of the binarized lesion’s segmentation mask, fitted to a mesh surface
Ellipsoid volume	$V_E=\frac{32}{3}pi\sqrt{{\lambda }_{\text{major}}{\lambda }_{\text{minor}}{\lambda }_{\text{least}}}$, where ${\lambda }_{\text{minor}}$ is the second largest eigenvalue of the region-based 3D ellipsoid fitting to the lesion mask
Ellipsoid surface area	$A_E=4\pi (\frac{a^p{b}^p+b^p{c}^p+a^p{c}^p}{3}{)}^p$, where $p=1.6075$, $a=2\sqrt{{\lambda }_{\text{major}}}$, $b=2\sqrt{{\lambda }_{\text{minor}}}$, and $c=2\sqrt{{\lambda }_{\text{least}}}$

As illustrated in Fig. 1, MDD calculations require repeated CT images of lesions at differing imaging protocols. For this study, the images were simulated using a typical voxel size (0.25 mm) and a defined contrast of 400 Hounsfield units (HU). The lesion sizes were set to be 6 and 17 mm, representing small and large nodules in accordance with lung-RADS categories.¹⁸ The modeled nodules had three spiculation levels ranging from low to high to represent nodules with diverse morphology. Each level contained five individual samples. This resulted in a total of 30 distinct lesions being utilized for the MDD calculation of each of the seven investigated MRFs.

The CT simulations were done using the technique described in Ref. 16. Five different scanner types were modeled (Table 2). Each nodule was rendered with modulated transfer function (MTF) and noise power spectrum (NPS) corresponding to a given dose, slice thickness, and imaging condition. For each scanner type, simulations were done at three different dose levels of 100%, 50%, and 25% of clinical doses,¹⁹ three slice thicknesses (0.625, 1.25, and 2.5 mm), and various reconstruction settings (Table 2), representing a diverse set of imaging protocols. The deployed protocols were based on clinically relevant practice for a standard chest scan at our institute. To minimize the impact of biological changes on the MDDs, the baseline and follow up acquisition protocols were assumed to be the same. Figure 2 illustrates the lesion models within a specific size category, along with simulations of one of the lesions across different protocols. It is worth mentioning that the variations in ground truth were attributed to differences in different instances of the lesions and the level of spiculation for each lesion size category.

Table 2.

List of the utilized scanner specifications and reconstruction kernels for MDD calculations in Ref. 16.

Vendor	Scanner	Reconstruction	Kernel
GE	Discovery 750 HD	ASiR-50%	Soft, standard, lung, and bone
	LightSpeed VCT	ASiR-50%	Soft, standard, and bone
	Revolution	ASiR-V-50%	Soft, standard, and bone
Siemens	Somatom definition flash	SAFIRE-1	I26f, I31f, and I50f
	Somatom force	ADMIRE-1	Br36d, Br40d, Br59d, and Br64d

Fig. 2.

(a) Examples of lesion models of a particular size exhibiting different degrees of spiculation are depicted. (b) Diverse acquisition simulations for one of the lesions are illustrated across various dose levels and reconstruction kernels.¹⁶

We recorded all variable factors during the MDD calculation for the next steps. These factors included a biological parameter, lesion size ($s$), and the system parameters ($D$, $T$, $f_{\mathrm{av}}$, $\mathrm{MTF}{f}_{50}$, and $\mathrm{MTF}{f}_{100}$) for each pair of images. The specific ranges of these influencing variables are listed in Table 3. The obtained MTF/NPS properties were measured from an ACR phantom study.

Table 3.

Modeled parameter space for the MDD calculation.

Variable name	Unit	Symbol	Ranges ({.} = discrete [.] = interval)
MTF’s frequency at 50%	${\mathrm{mm}}^{-1}$	$\mathrm{MTF}{f}_{50}$	[0.3 to 0.8]
MTF’s frequency at 100%	${\mathrm{mm}}^{-1}$	$\mathrm{MTF}{f}_{100}$	[0.0 to 0.5]
NPS’s frequency at average	${\mathrm{mm}}^{-1}$	$f_{\mathrm{av}}$	[0.19 to 0.66]
Slice thickness	mm	$T$	{0.625,1.250,2.500}
Lesion size	mm	$s$	{6,17}
Dose	mGy	$D$	[1.9 to 7.5]
Noise magnitude	HU	$\sigma$	[15 to 80]

MDD calculations require segmentation of nodules in the simulated image. To segment the nodules, the active contour method²⁰ was used.

2.2. Surrogate Minimum Detectable Difference Function

From the MDD calculation, we recorded noise magnitude ($\sigma$), average frequency of NPS ($f_{\mathrm{av}}$), and the frequencies associated with 50% and 100% of MTF ($\mathrm{MTF}{f}_{50}$ and $\mathrm{MTF}{f}_{100}$), as scalar surrogates of the NPS and MTF curves. $\mathrm{MTF}{f}_{100}$ indicates the edge enhancement feature of the utilized kernel, in which the MTF value at $\mathrm{MTF}{f}_{100}$ is greater than unity. To identify optimum conditions that minimize MDDs, the parameterized MDD functions were approximated over the recorded imaging protocols and lesion conditions. As such, the calculated MDDs were used to approximate MDD functions as polynomial functions where the imaging protocols and lesion conditions serve as influencing variables. Hence, we conducted a model selection procedure, through a fivefold cross validation, to identify the processing techniques, regression model, and polynomial degree that maximize a score index.

In this study, slice thickness and lesion size values were sparse and thus, they were considered as categorical variables, making the resultant feature space a combination of the categorical (slice thickness and lesion size) and numerical variables ($\mathrm{MTF}{f}_{50}$, $\mathrm{MTF}{f}_{100}$, $f_{\mathrm{av}}$, and $D$). The categorical features underwent preprocessing to transform them into one-hot encoded variables, whereas the selection of the numerical features’ processing algorithm involved choosing among standard scalar, min–max scaler, robust scaler, and max-absolute scaler methods. Similarly, for the corresponding MDD values, the preprocessing choice was made from the aforementioned techniques, along with the Cox–Box power transformer,²¹ followed by a min–max scaler. As a result, a total of 1188 sample points were employed for training and testing each model. It is worth mentioning that for each of these points, 150 individual samples were generated to derive the corresponding MDD values.

For the regression models, a set of nine linear regression models with polynomial mapping, encompassing a range of degrees within (1, 2) interval, were considered. The choices of the regression models were logistic, ridge, Lasso, elastic-net,²² Bayesian,²³ automatic relevance determination (ARD),²⁴ Poisson,²⁵ Huber, and least-angle (LARS) regressions.²⁶ The model selection method approximates the coefficient of polynomial models that maximize the coefficient of determination ($R^2$) score of the models’ predication with respect to the measured MDD for each distinct MRF. $R^2$ is a statistical measure that represents the proportion of the variance in the dependent variable (the variable being predicted) that can be explained by the independent variables (the variables used for prediction). Therefore, a high $R^2$ score indicates that the MDD functions prediction closely follows the trend of the actual MDD value. The MDD function can be mathematically expressed as

$$\begin{gathered} \gamma ={}^d\phi _{\mathrm{rf}}({}^b\mathrm{M}\mathrm{TF}{f}_{50},{}^b\mathrm{M}\mathrm{TF}{f}_{100},{}^{b}f_{\mathrm{av}},{}^{b}D,{}^{b}T,{}^f\mathrm{M}\mathrm{TF}{f}_{50},{}^f\mathrm{M}\mathrm{TF}{f}_{100},{}^{f}f_{\mathrm{av}},{}^{f}D,{}^{f}T,s), \\ \gamma ={}^d\phi _{\mathrm{rf}}({}^b\theta ,{}^f\theta ,s), \end{gathered}$$ (1)

where $\gamma$ and ${}^d\phi _{\mathrm{rf}}$ stand for the MDD value, and polynomial model with degree of $d$ for radiomics feature “rf”, respectively. Superscripts $b$ and $f$ stand for the baseline and follow-up acquisitions, respectively. $\theta$ represents the vector of variables whose size in this study is $N=5$ ($\theta =[\mathrm{MTF}{f}_{50},\mathrm{MTF}{f}_{100},f_{\mathrm{av}},D,T]$). Since, in this study, baseline and follow-up acquisition protocols are identical, one can reformulate Eq. (1) as

$$\begin{gathered} \gamma ={}^d\phi _{\mathrm{rf}}(\mathrm{MTF}{f}_{50},\mathrm{MTF}{f}_{100},f_{\mathrm{av}},D,T,s), \\ \gamma ={}^d\phi _{\mathrm{rf}}(\theta ,s). \end{gathered}$$ (2)

For each MRF, we split the data to a 30% to 70% portion for testing–training. The model selection procedure performed a grid search on the nine regression models, five preprocessing techniques, and polynomial degrees of 1 or 2. To avoid overfitting and gain more generalizability, the $L_1$ and $L_2$ regularization terms of regression models were set to be always $>0.1$. To select the proper regularization terms and other relevant hyper-parameters with models and preprocessings, fivefold cross validations were done over the training dataset. We then selected the model with the highest $R^2$ score on the pertained validation sets for each MFRs.

After the model selection procedure, to assess the sensitivity of the approximated MDD functions and gain insight into the predominant factors affecting MDDs, we conducted a measurement of permutation feature importance (PFI). The PFI quantifies the reduction in the model score resulting from the random shuffling of a single feature value. Consequently, the PFI serves as an indicator of the degree to which the decision variables impact the estimated model.

2.3. Correlation Models

The correlation models are mathematical expressions that represent the physical relationship between the system variables. In this study, all simulations were conducted consistent with the physics of the acquisition process. However, the employed models for each scanner were mutually independent. Consequently, a scanner-specific mathematical model was utilized for each individual scanner. Nonetheless, for the optimization stage, a scanner-agnostic model is required to encapsulate the acquisition physics across all utilized scanners. Based on Refs. ²⁷ and ²⁸, the following expression defines the relation of $\mathrm{MTF}{f}_{50}$ with noise magnitude ($\sigma$), in-plane pixel size ($\delta$), slice thickens ($T$), and dose ($D$):

$$D\times {\sigma }^2\times {\delta }^3\times T\propto \mathrm{MTF}{f}_{50}^{[4-5]}.$$ (3)

This relation contains uncertain fractional order polynomial models of $\mathrm{MTF}{f}_{50}$. To take advantage of the equation in a polynomial optimization problem, we approximated nonpolynomial terms with polynomial models. Therefore, substituting the right-hand side of Eq. (3) with a polynomial approximation of the fractional order, we reformulated Eq. (3) as

$$\sqrt{D\times {\sigma }^2\times {\delta }^3\times T}\propto {}^{2}g(\mathrm{MTF}{f}_{50}),$$ (4)

where ${}^{2}g(.)$ is a quadratic function. As we used a fixed pixel size ($\delta =0.25\mathrm{mm}$), we did not consider the $\delta$ as an input variable for the correlation model. We also know that $\sigma \propto f_{\mathrm{av}}$ and $\mathrm{MTF}{f}_{50}\propto f_{\mathrm{av}}$. As a result, including $f_{\mathrm{av}}$ in the formula leads us to estimate a correlation model with a tighter error margin as

$$\sqrt{D\times {\sigma }^2\times {\delta }^3\times T}\propto {}^2{g}_{\overline{\delta }}(\mathrm{MTF}{f}_{50},f_{\mathrm{av}}).$$ (5)

To approximate the coefficients and orders of ${}^2{g}_{\overline{\delta }}(.)$ functions, we utilized an experimental ACR phantom study. We measured $\sigma$ and $\mathrm{MTF}{f}_{50}$ (from a bone insert) at three doses over 561 imaging systems and kernels. All measured data were acquired with a fixed slice thickness ($T_{\mathrm{ACR}}=5\mathrm{mm}$) and fixed pixel size (${\delta }_{\mathrm{ACR}}=0.48\mathrm{mm}$). Consequently, we standardized $T$ using $T_{\mathrm{ACR}}$ and since $\delta$ were fixed in both ACR study and simulation, we regarded it as a fixed variable. We split the ACR data into a 70% to 30% ratio for training and testing, respectively. Then we utilized the developed ML pipeline to approximate ${}^2{g}_{\overline{\delta }}(.)$ such that the mean squared error of the approximated function with respect $\sqrt{D}\sigma$ is minimized. Therefore, we can express the correlation model as

$$D\times {\sigma }^2\times \frac{T}{T_{\mathrm{ACR}}}-{}^2{g}_{\overline{\delta }}{(\mathrm{MTF}{f}_{50},f_{\mathrm{av}})}^2=0.$$ (6)

In addition to the correlation model expressed by Eq. (6), the correlation between $f_{\mathrm{av}}$, $\mathrm{MTF}{f}_{50}$ was approximated as

$${}^{1}g(f_{\mathrm{av}},\mathrm{MTF}{f}_{50})=0.$$ (7)

2.4. Optimization Problem Formulation

The optimization problem was formulated as

$$\underset{\theta }{\min }{{}^d\phi }_{\mathrm{rf}}(\theta ,\overline{s})\mathrm{s.t.}g(\theta ,\sigma )\le 0,$$ (8)

where $\theta$ stands for the decision variables, ${{}^d\phi }_{\mathrm{rf}}(.)$ is the MDD model defined by Eq. (2), $\overline{s}$ is the lesion size, and $g(\theta ,\sigma )\le 0$ represents the constraints. Constraints were defined by the desired upper and lower bounds of the system’s parameters presented in Table 3 ($\mathrm{lb}\le g_b(\theta ,\sigma )\le \mathrm{ub}$), and the correlation constraints expressed by Eqs. (6) and (7).

Incorporating the constraints (6) and (7), we reformulated the optimization problem (8) as

$$\begin{gathered} \underset{\theta ,\sigma }{\min }{}^d\widehat{\phi }_{\mathrm{rf}}(\theta ,\overline{s}) \\ \mathrm{s.t.}\mathrm{lb}\le g_b(\theta ,\sigma )\le \mathrm{ub}, \\ {\theta }_4\times {\sigma }^2\times \frac{{\theta }_5}{T_{\mathrm{ACR}}}-{}^2{g}_{\overline{\delta }}{({\theta }_1,{\theta }_3)}^2=0, \\ {}^{1}g({\theta }_3,{\theta }_1)=0, \\ {\theta }_2+ϵ<{\theta }_1, \end{gathered}$$ (9)

where $\theta =[{\theta }_1,{\theta }_2,{\theta }_3,{\theta }_4,{\theta }_5]=[\mathrm{MTF}{f}_{50},\mathrm{MTF}{f}_{100},f_{\mathrm{av}},D,T]$ are decision variables. The last constraints indicate that $\mathrm{MTF}{f}_{50}$ (also known as ${\theta }_1$) is always greater than $\mathrm{MTF}{f}_{100}$ (also known as ${\theta }_2$). We empirically set $ϵ=0.3$. For a fixed lesion size $\overline{s}$, for the objective function, the formulated optimization problem was solved with Yalmip (MATLAB toolbox)²⁹ integrated with the Gurobi solver.³⁰ This package uses the linear programming relaxation based on the McCormick relaxation and convex envelope approximation to solve nonconvex problems. (In this study, all the MDD functions and correlation constraints were polynomial functions with an order of $<2$.)

2.5. Guideline Analysis

We conducted a controlled protocol analysis to evaluate the performance of the optimized protocol selection guideline. The data generation process follows the sampling-based MDD calculation framework described in Ref. 16. The control group had three categories of protocols. The first group had no restrictions on the selection of the parameters, and the resultant images were acquired by random selection of the imaging parameters from Table 3. The second group followed the QIBA recommended protocol⁶ with $\sigma \le 60\mathrm{HU}$, $0.3\le \mathrm{MTF}{f}_{50}\le 0.5{\mathrm{mm}}^{-1}$, and $0.65\le T\le 1.25\mathrm{mm}$. In the third group, the protocols were restricted to the optimized protocol for each lesion size where ${\sigma }_{\mathrm{opt}}-20\le \sigma \le {\sigma }_{\mathrm{opt}}+20$, $\mathrm{MTF}{f}_{50\mathrm{opt}}-0.1\le \mathrm{MTF}{f}_{50}\le \mathrm{MTF}{f}_{50\mathrm{opt}}+0.1$, $D\ge \frac{D_{\mathrm{opt}}}{2}$, and $T=T_{\mathrm{opt}}$. We then measured the resultant MDD of MRFs for each group. Table 4 summarizes the three groups of controlled protocols.

Table 4.

Features of the three protocol groups for controlled protocol analysis.

Protocol	$\mathrm{MTF}{f}_{50}$ (${\mathrm{mm}}^{-1}$)	Noise (HU)	Slice thickness (mm)	Dose (mGy)
Protocol A	0.3 to 0.8	15 to 80	[0.625, 1.25, 2.5]	[1.9, 3.75, 7.5]
QIBA	0.3 to 0.5	15 to 60	[0.625, 1.25]	[1.9, 7.5]
Optimized	$\mathrm{MTF}{f}_{50\mathrm{opt}}\pm 0.1$	${\sigma }_{\mathrm{opt}}\pm 20$	0.625	[3.75, 7.5]

3. Results

3.1. Surrogate Minimum Detectable Difference Function

In analyzing the results obtained from the grid search algorithm with fivefold cross-validation, second-order polynomial models had higher $R^2$ score across all MRFs. On average, the approximated models had a $R^2$ score of $0.95\pm 0.03$. Table 5 provides an overview of the preprocessing used for the features and MDD values, as well as the adopted models to approximate the MDD functions. Figure 3 demonstrates the PFI results for each MRF model. The obtained results indicated that for all MRFs except radius and major length, lesions size had a considerable impact on the predicted MDD values. In addition to the lesion size, the NPS average frequency $f_{\mathrm{av}}$ and dose ($D$) had the highest influence on the accuracy of the approximated functions. However, $\mathrm{MTF}{f}_{50}$ and $T$ had the higher impact on the accuracy of the radius and major length functions.

Table 5.

Employed preprocessing and models for each MRF.

MRF	Feature preprocessing	MDD preprocessing	Model
$R$	Max absolute scaler	Power transformer	ARD-regression
$V$	Robust scaler	Power transformers	Elastic net
$F$	Robust scaler	Power transformers	Bayesian ridge
$L$	Min–max scaler	Max absolute scaler	Poisson regression
$A$	Standard scaler	Max absolute scaler	Poisson regression
$V_E$	Robust scaler	Max absolute scaler	Poisson regression
$A_E$	Standard scaler	Max absolute scaler	Poisson regression

Fig. 3.

PFI analysis for MDD functions for all the MRFs: (a) radius, (b) major length, (c) volume, (d) flatness, (e) surface area, (f) ellipsoid volume area, and (g) ellipsoid surface area.

3.2. Optimization Results

Figure 4 depicts the optimal protocols for both small and large lesion sizes across all MRFs. The color bar indicates the normalized optimal values within the corresponding intervals of each protocol. The considered intervals for each protocol are reported in Table 3. At each row, the optimal system parameters for each lesion size are reported. For the smaller lesions, the higher dose (here 7.5 mGy), lower noise (20 to 25 HU), and thinner slices (here 0.625 mm) yielded lowest MDD. The results indicated that the optimal protocols for major length, surface area, radius, and ellipsoid volume were independent of the lesion size and remained unchanged regardless of the lesion size. In addition, the optimal slice thickness and dose were consistent across different lesion sizes and MRFs. The proposed framework provided a lesion-based and MRF-based guideline for protocol selection. In small lesions, except for surface area, the optimal protocols were associated with a 15 HU noise magnitude and an $\mathrm{MTF}{f}_{50}$ of $0.41{\mathrm{mm}}^{-1}$. For the surface area, the optimal noise magnitude and $\mathrm{MTF}{f}_{50}$ were 48 HU and $0.8{\mathrm{mm}}^{-1}$, respectively. For small lesions, the optimal edge enhancement varies with the MRF: major length, ellipsoid surface area, ellipsoid volume, and radius did not require edge enhancement ($\mathrm{MTF}{f}_{100}\le 0.05{\mathrm{mm}}^{-1}$), whereas other MRFs required different levels of edge enhancements. For large lesions, across all MRFs, except surface area and volume, the optimum protocols were associated with a noise magnitude of $<16\mathrm{HU}$, no edge enhancement ($\mathrm{MTF}{f}_{100}\le 0.05{\mathrm{mm}}^{-1}$), and an $\mathrm{MTF}{f}_{50}$ of $0.49{\mathrm{mm}}^{-1}$. The optimal protocols for surface area were the same for both large and small lesions. For volume, a noise magnitude of 48 HU, edge enhancement with a frequency of $0.25{\mathrm{mm}}^{-1}$, and an $\mathrm{MTF}{f}_{50}$ of $0.8{\mathrm{mm}}^{-1}$ found optimum.

Fig. 4.

From top to bottom, the MRFs were sorted based on their sensitivity to the lesion size. In general, the noise magnitude is the more important parameter for smaller lesions and should be adjusted to a lower magnitude than larger lesions, whereas higher sharpness is preferable for larger lesions.

In general, for optimal protocols across all MRFs (as indicated in the last row of Fig. 3), the noise magnitude and sharpness should increase by increasing the size of the lesions while minimum slice thickness and highest dose level are recommended for both lesions sizes.

3.3. Guideline Analysis

An analysis of the optimization results depicted in Fig. 4 suggested that effective kernels possess edge enhancement properties that fluctuate contingent upon lesion size and targeted MRF. Although $\mathrm{MTF}{f}_{100}$ displayed this unique characteristic, real-world clinical settings often lack prior knowledge of lesion size, precluding its inclusion in prospective protocol optimization. Additionally, the edge enhancement feature often arises inherently within the employed kernel. Consequently, achieving the desired edge-enhancement necessitates kernel alterations, a potentially limiting factor if scanner diversity is restricted. Furthermore, the proposed protocol selection guideline remained compliant with established protocols by solely utilizing parameters previously proposed by QIBA. The controlled protocol analysis (Fig. 5) shows reduction of MDD measurement variability across all MRF, when optimized protocols were used. The quantitative values of the obtained results are reported in Table 6. The results show that the MDDs for smaller lesions vary more in radius, flatness, and major length. In the other MRFs, larger lesions cause larger deviations. This finding illustrates the importance of lesion-based protocol optimization, which can only be achieved with the proposed method. Compared with the measured MDDs by the QIBA recommendation guideline, our optimum protocols reduced the coefficients of variations from 1.68 to 1.12 and 1.49 to 1.11, for the small and large lesions, respectively. Together, these reductions represent an average of 46% (13% to 57%) reduction in the variation of MDDs across all MRFs and lesions.

Fig. 5.

Controlled protocol analysis for different MRFs. Lesion size restrictions have reduced variations in optimal protocols. Moreover, the MDD for optimal protocol is lower across all MRFs: (a) radius, (b) major length, (c) volume, (d) flatness, (e) surface area, (f) ellipsoid volume area, and (g) ellipsoid surface area.

Table 6.

Mean value and standard deviation of the MDD for each MRF. The optimized protocols rendered images with less variability and lower MDD values in all cases.

	Small			Large
MRF	$A$	QIBA	Optimized	$A$	QIBA	Optimized
$R$	$0.25\pm 0.11$	$0.15\pm 0.04$	$0.12\pm 0.02$	$0.28\pm 0.08$	$0.21\pm 0.03$	$0.18\pm 0.02$
$V$	$8.72\pm 2.69$	$6.42\pm 1.56$	$5.32\pm 1.01$	$44.21\pm 14.21$	$33.46\pm 7.48$	$26.28\pm 5.02$
$F$	$0.031\pm 0.011$	$0.027\pm 0.006$	$0.026\pm 0.005$	$0.007\pm 0.002$	$0.005\pm 0.001$	$0.004\pm 0.001$
$L$	$0.40\pm 0.18$	$0.23\pm 0.05$	$0.19\pm 0.03$	$0.23\pm 0.07$	$0.17\pm 0.03$	$0.14\pm 0.02$
$A$	$59.50\pm 21.06$	$42.78\pm 9.12$	$36.69\pm 7.08$	$416.68\pm 254.67$	$258.61\pm 89.14$	$147.61\pm 35.00$
$V_E$	$18.39\pm 7.13$	$12.30\pm 3.26$	$10.21\pm 1.97$	$138.15\pm 43.25$	$100.41\pm 18.17$	$87.43\pm 12.90$
$A_E$	$7.97\pm 2.99$	$5.37\pm 1.37$	$4.50\pm 0.85$	$17.65\pm 5.48$	$12.60\pm 2.19$	$10.92\pm 1.43$

4. Discussion and Conclusions

This study offers a strategy for optimal CT acquisition for consecutive acquisitions to achieve the highest quantification accuracy and the smallest MDD in morphological radiomic features (MRFs). Results suggested that MDD values can be accurately predicted using second-order polynomial models (average score of $0.95\pm 0.03$) while using imaging parameters and lesion attributes as input parameters. The PFI analyzes the major contribution of lesion for accurate prediction of MDD values across all MRF models, except for radius and major length. Hence, accurate estimation of lesion size is vital for identifying protocols that minimize MDD. The NPS average frequency ($f_{\mathrm{av}}$) and dose were also identified as other major influential parameters in accurate prediction of MDD functions.

Based on the optimal protocol analysis, we determined the required imaging protocol for lesion sizes ranging from small to large across each MRF. MDD can be minimized using a higher dose and lower noise for smaller lesions across all studied cases. Furthermore, it was found that thinner slices and smooth kernels reduced MDD. These findings agree with the reported results in Ref. 16, in which each imaging parameter impact was evaluated independently. Compared to Ref. 16, the proposed method has the additional advantage of providing lesion size-specific recommendations for individual MRFs in consideration of the physical constraints of an imaging system. This provides a more tailored and accurate approach for protocol optimization. Specifically, for MRFs, such as major length, flatness, and radius, MDDs for smaller lesions exhibited higher variability. In contrast, in other MRFs, larger lesions showed higher deviations. This highlights the importance of lesion-based protocol optimization, which can be achieved using the proposed method.

This study addressed the challenge of quantifying MDDs in radiomic features of lung nodules across diverse imaging protocols. By employing approximated surrogate MRF functions and correlation models, an optimization problem was solved to identify the optimal imaging protocol that minimized the MDD while adhering to the physical constraints of the imaging systems. Findings revealed that the MDD varied depending on the lesion size for each morphological feature. The proposed optimized framework reduced bias and variability in MDD, leading to more accurate detection of radiomics features and consistent outcomes. When compared to the standard recommended protocols outlined by QIBA, the protocol selection guideline consistently achieved a significant reduction in the normalized MDD, ranging from 13% to 47%.

It is essential to recognize and address specific limitations within this study. Notably, the generalizability of the polynomial models for MDD prediction to diverse datasets and imaging systems necessitates more investigation. Additionally, exceptions observed in the influence of parameters, such as radius and major length, underscore the intricate nature of MDD calculation and the potential impact of unaccounted factors. Furthermore, the applicability of our lesion-based protocol optimization findings to varied lesion types and anatomical regions requires more complex simulation to accurately calculate the MDDs for such lesions.

In summary, the findings showed that to minimize the MDD, small slice thickness, and high dose acquisitions are optimal settings across different MRFs. In the case of small lesions, the ideal protocols included a noise magnitude of 15 HU and an $\mathrm{MTF}{f}_{50}$ of $0.41{\mathrm{mm}}^{-1}$, with the exception of surface area. For surface area, the optimal noise magnitude and $\mathrm{MTF}{f}_{50}$ were 48 HU and $0.8{\mathrm{mm}}^{-1}$, respectively. The optimal level of edge enhancement varied for small lesions depending on the specific MRF being considered. MRFs, such as major length, ellipsoid surface area, ellipsoid volume, and radius did not require any edge enhancement ($\mathrm{MTF}{f}_{100}\le 0.05{\mathrm{mm}}^{-1}$), whereas other MRFs required varying degrees of edge enhancement.

Regarding large lesions, the suggested optimal protocols for all MRFs, except surface area and volume, consisted of a noise magnitude below, no edge enhancement, no edge enhancement ($\mathrm{MTF}{f}_{100}\le 0.05{\mathrm{mm}}^{-1}$), and $\mathrm{MTF}{f}_{50}$ of $0.49\hspace{0.17em}{\mathrm{mm}}^{-1}$. The optimal protocols for surface area in large lesions were the same as those for small lesions. For volume, the optimum protocol was found to be associated with a noise magnitude of 48 HU, edge enhancement with a frequency of $0.25{\mathrm{mm}}^{-1}$, and $\mathrm{MTF}{f}_{50}$ of $0.8{\mathrm{mm}}^{-1}$.

The study presented a methodology that replaces the traditional computational approach to MDD calculation within each MRFs. Instead, a rapid optimization technique is employed to identify the optimal protocol configuration for a given lesion size. Leveraging the considerable difference in lesion sizes categories studied in this study, prior knowledge regarding lesion size might be obtained from scout images to inform the selection of the optimal protocol through the proposed approach. Notably, the results demonstrated a correlation between lesion size and optimal edge enhancement selection. Furthermore, for the majority of MRFs, the proposed method effectively reduces variability in MDD compared to existing suggestions. Future work will focus on incorporating additional measurable patient-specific factors, such as patient size, to refine the accuracy of protocol selection.

Biographies

Mojtaba Zarei is a PhD candidate and a researcher at the Center for Virtual Imaging Trials. He specializes in mathematics, statistics, and advanced learning algorithms. Through the implementation of physics-informed neural networks, machine learning, and optimization, he improves the reliability of safety-critical systems, such as medical imaging systems. In his master’s studies at the University of Tehran, he pioneered the development of secure control systems for complex robotic and autonomous systems.

Ehsan Abadi is an imaging scientist and an assistant professor at Duke University, specializing in radiology and electrical and computer engineering. He holds faculty positions in Medical Physics, Carl E. Ravin Advanced Imaging Laboratories, and the Center for Virtual Imaging Trials. He earned his PhD in electrical engineering from Duke in 2018. His research centers on quantitative imaging, CT imaging, lung diseases, computational human modeling, and medical imaging simulation.

Liesbeth Vancoillie holds her BSc, MSc, and PhD degrees in biomedical engineering from KU Leuven and completed her master’s degree along with postgraduate studies in medical radiation physics. Her PhD studies focused on the technical and task-based performance evaluation of synthetic mammograms using VITs. Her journey led her to become a medical physics expert at the University Hospitals Leuven and Jan Yperman Ziekenhuis in Belgium. Currently, he serves as a research manager at Duke University’s Radiology Department in the CVIT Lab.

Ehsan Samei serves as the Reed and Martha Rice Distinguished Professor and Chief Imaging Physicist, and a director of the Center for Virtual Imaging Trials at Duke University. With more than 360 referred papers, he strives to connect scientific research with clinical application, emphasizing patient-centered imaging and translational research. His work centers on smart use of AI, in vitro, in vivo, and in silico metrologies to enable quantitative, truth-based imaging.

Disclosures

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Code and Data Availability

The utilized data and code are proprietary materials for the Center for Virtual Imaging Trial.