Abstract.
Purpose
Radiomics have become invaluable for non-invasive cancer patient risk prediction, and the community now turns to exogenous assessment, e.g., from genomics, for interpretability of these agnostic analyses. Yet, some opportunities for clinically interpretable modeling of positron emission tomography (PET) imaging data remain unexplored, that could facilitate insightful characterization at voxel level.
Approach
Here, we present a novel deformable tubular representation of the distribution of tracer uptake within a volume of interest, and derive interpretable prognostic summaries from it. This data-adaptive strategy yields a 3D-coherent and smooth model fit, and a profile curve describing tracer uptake as a function of voxel location within the volume. Local trends in uptake rates are assessed at each voxel via the calculation of gradients derived from this curve. Intratumoral heterogeneity can also be assessed directly from it.
Results
We illustrate the added value of this approach over previous strategies, in terms of volume rendering and coherence of the structural representation of the data. We further demonstrate consistency of the implementation via simulations, and prognostic potential of heterogeneity and statistical summaries of the uptake gradients derived from the model on a clinical cohort of 158 sarcoma patients imaged with -fluorodeoxyglucose–PET, in multivariate prognostic models of patient survival.
Conclusions
The proposed approach captures uptake characteristics consistently at any location, and yields a description of variations in uptake that holds prognostic value complementarily to structural heterogeneity. This creates opportunities for monitoring of local areas of greater interest within a tumor, e.g., to assess therapeutic response in avid locations.
Keywords: heterogeneity, uptake gradient, positron emission tomography, sarcoma, spatial statistics, tumor characterization
1. Introduction
The standard of cancer care has become increasingly reliant on positron emission tomography (PET) imaging with -fluorodeoxyglucose (FDG). This has stimulated development of image-derived patient-adaptive quantitation of prognostic factors into therapy planning and management. Prognostic utility of semi-quantitative summaries of FDG-PET information and in particular variables derived from standardized uptake value (SUV) information in current clinical routine was demonstrated for a range of diseases.1–5 In addition to these, measures of intratumoral uptake heterogeneity in FDG-PET that exploit the whole volumetric uptake information have been associated with patient treatment outcomes in several cancer types.4,6–10
The development of radiomics now offers a more elaborate framework for such quantitative characterization, for many diseases.10–19 These multivariate predictive models, comprising of large, heterogeneous sets of features, are trained and validated via machine learning frameworks to achieve promising prognostic performance. Although some radiomic features [such as volume of interest (VOI) morphology or histogram intensity] have a clear practical meaning, these high-throughput methodologies tend to have inherently limited clinical interpretability. A growing number of studies now explore associations of radiomics models with exogenous assays, e.g., from genomics, with the aim to facilitate model interpretation in terms of the underlying tumor biology.20–23
Alternative quantitative methodologies can also be considered to facilitate clinical interpretation directly from modeling. Earlier works from our group presented a strategy based on a conceptual definition of intratumoral heterogeneity as a measured degree of conformity of the spatial PET tracer uptake distribution to an idealized ellipsoidal pattern. The approach thus consisted in comparing the volumetric uptake distribution against a rigid ellipsoidal reference,24 a lower degree of conformity indicating more scattered uptake at the core, as could be observed, e.g., in necrotic or more developed tumor masses. This strategy may be seen as a form of point pattern analysis; we are aware of only one other example of a spatial point process approach in the literature for cancer characterization25—although applied to analyze the distribution of cell nuclei locations in colon cancer from immunofluorescence imaging, a much different application. Our group later demonstrated that tumor characterization based on the ellipsoidal conformity approach offered additional prognostic opportunities complementary to commonly used radiomic features in sarcoma and lung cancer.26,27 However, tumor characterization from this approach remained restricted to an evaluation of lack of conformity to a rigid pattern, and could only produce a crude representation of local uptake characteristics. An alternative, data-adaptive strategy could also more appropriately capture complex tumor topologies, e.g., where a tumor grows along a bone.
A second model developed by the group6 was designed with the aim to adapt to spatial variations of the volumetric uptake data, based on a tubular representation of the data. An application of this strategy to a set of FDG-PET sarcoma studies showed that structural heterogeneity and other interpretable variables derived from this model at voxel level, such as median core phase of development, could bear prognostic value. This ad hoc model, however, had technical limitations. It required that smoothing of individual parametric components of the model be carried out separately, and it is not clear how this could impact volume rendering. Depending on data sampling and preprocessing choices, this approach may also not guarantee smoothness of the output volume in both axial and angular directions. This difficulty may be reinforced by the fact that variations along the sampling directions were not represented using continuous functionals of the volumetric coordinates, but were rather captured parametrically only. Another drawback of the scheme was that the fitted volumes may have inadequately large tail-ends. Although such large extreme slices did not appear to hinder significance of the overall metabolic assessment, they suggested that a better representation of the VOI could be achieved, and with it a more appropriate assessment of local activity.
These two former models of the spatial structure of PET tracer uptake in solid masses encouraged elaboration of refined techniques for assessment at voxel level. In this paper, we propose an alternative representation that provides a smooth, continuous evaluation of the volumetric uptake distribution and a 3D-coherent measure of uptake trends at any location. The approach involves a tubular representation of the tumor mass with a radial analysis of uptake, transverse to the tubular axis. A regularization formulation is used in this implementation to facilitate computation and provide simplified control over the implicit smoothness of multiple functions involved in the 3D representation of tumor uptake. A range of prognostic markers can be derived from the model fit, and we demonstrate their clinical potential via retrospective analysis of a clinical cohort. In particular, we derive uptake gradients to quantify the variation in uptake at voxel level. This variable enables mapping areas of fast and slow changes in tracer uptake, including information on the direction of change. Integration of this principle into a radiomics framework will be the scope of future work.
The proposed model and prognostic summaries derived from it are described in Sec. 2. Details on computational aspects are provided in Sec. 3, and complementary numerical analyses presented in Appendices. Section 4 explains an application of the methodology to clinical studies, including a comparison of quantitative analyses with previous models, and a demonstration of its prognostic potential. We discuss a number of technical and conceptual limitations and further directions in Sec. 5.
2. Modeling and Characterization of Intratumoral Tracer Uptake
This section presents the proposed modeling strategy. The key feature of this approach is the representation of the tracer uptake profile as a function of voxel (tubular) radius. We describe how to derive tracer uptake gradients for each voxel from this profile curve for local assessment of uptake trends, and summarize said curve into measures of intratumoral structural heterogeneity.
2.1. Tubular Modeling of Volumetric Tracer Uptake
The original imaging data , which consist of a collection of voxels defined by their uptake and 3D coordinates in the voxel coordinate domain , are first projected onto in the principal components basis, so as to align the tumor volume according to its direction of maximum spread. The 3D covariance matrix for principal components analysis is obtained from the distribution of ’s weighted by the uptake values, i.e.,
where is the uptake-weighted mean of the ’s. The projected data are resampled along the principle axis (i.e., along ) so as to obtain transverse slices of the reoriented volume. Transverse tumor spine coordinates are then obtained for each transverse slice using a form of uptake-weighted average smoothed along the principle axis.6 There is flexibility in the choice of sampling strategy here; we advise choosing a number of bins (or slices) so that it is proportional to the ratio between axial range of height values and within-plane resolution of the transverse image.6
Tubular coordinates are then obtained where , and . These coordinates become useful in defining the following nonlinear radial uptake profile:
(1) |
with infinite-dimensional parametrization corresponding respectively to voxel amplitude, core phase and radial deformation of the initial voxel tubular radius in the axial and angular directions. We call voxel phase of development the argument of this overall uptake profile function (i.e., the spatially uptake-adaptive radii), which provides an indication of local uptake trends at that voxel, in the context of the profile function .
A non-symmetric choice for may be appropriate given the asymmetric, heavy-tailed nature of radial uptake distributions observed from FDG-PET imaging data. A gamma distribution with shape parameter , and unit rate was used in this work. This allows for a reasonable, right-skewed output profile curve evaluated on a known scale.
The functional components of parameter are evaluated in terms of B-splines in the -coordinate, periodic B-splines in the -coordinate, and their tensor products,28 such that at any voxel ,
where , and denote finite projections of functionals , and on these B-spline bases. This spline-based evaluation confers continuous support to the proposed parametric uptake model Eq. (1). Parametrization becomes a -dimensional term in this implementation, with . Further details on the computational techniques used for model fitting and evaluation are provided in Sec. 3.
2.2. Uptake Profiling
Fitting model Eq. (1) yields a one-dimensional (1D) uptake profile curve as a function of voxel location (in the model referential), which is scaled by a voxel-varying amplitude term at each voxel. This allows for an assessment of phase of development at the voxel level, via its argument
(2) |
In this expression, quantifies uptake development near the tumor core, at transverse slice . Its interpretation depends on the choice of profile function , which influences the dynamic range of phase values. Typically, larger phase values correspond to early tumor areas, whereas small values indicate an ageing (e.g., necrotic or metabolically slower) core, i.e., to a locally more developed tumor subvolume.
Figure 1 shows this concept, using a scaled gamma(3,1) profile function . Since the phase variable defined by Eq. (2) corresponds to an adjusted radius, in this plot, the LHS of the axis corresponds to phases values at the tumor volume core (the smallest value in the range being ), and the RHS to values found at the boundary of the volume. Phase values lying around the profile mode [which in the gamma(3,1) case is ] correspond to highly active tumor voxels. Phase values correspond to a region in a more advanced phase of development, with a decreasing uptake trend. Larger values of correspond to marginal tissue or early tumor areas. As the fitted intratumoral uptake profile is a function of phase , typical tumor characteristics yield different uptake profile and uptake gradient signatures. A distribution of uptake gradients containing a large proportion of negative values is indicative of a tumor with a decelerating core, whereas the value of lower quantiles of that distribution would typically be greater for young tumors with an active core.
Fig. 1.
Uptake model fits for two typical homogeneous (top) and heterogeneous (bottom) FDG uptake patterns in sarcoma, with mid-volume transverse slices shown in left-most images, profiles derived from the regularized tubular model of volumetric uptake (second from left), and corresponding uptake gradient curves derived from the uptake profile curve (third from left). The histograms provide a comparison of the corresponding distributions of gradients, negative values announcing decreasing uptake or even necrosis at that location.
2.3. Extracting Uptake Gradient Information from the Model
The voxel phase quantitation of Sec. 2.2 provides an assessment of uptake trends current to the time of imaging, much like the actual tracer uptake information. But inspired by the concept of the diffusion process of an evolving cancer cell, we can further utilize model Eq. (1) to derive a localized assessment of these trends. Works such as that of Roose et al.29 proposed mathematical models of spherical cancer cell diffusion processes in a free-growth environment. However, here, we do not evaluate differential equations to capture such dynamics. Given a fitted profile (Fig. 1), the profile derivative can be seen to quantify the instantaneous rate of change in uptake with respect to a standard change in phase at any voxel. For interpretation it seems easier to use the negative derivative
so as to obtain a quantity that is positive when the measured uptake trend corresponds to an increase in activity, i.e., on the RHS of the profile mode, and negative when this rate is found to decrease, i.e., on the LHS of the profile mode.
Raw model gradient quantitation can be enhanced by weighting the gradient variable by the fitted voxel amplitude, with the fitted voxel phases , to obtain a quantitative measure of rate of uptake at the voxel level
(3) |
With this, the gradient information is adjusted for localized uptake patterns, which puts the emphasis on areas that combine higher activity and sharp changes in tracer uptake. This is useful to further separate active tumor areas from less active tissue. The right-hand-side half of Fig. 1 shows uptake gradient profiles and distributions for two typical sarcoma studies. Gradient values below the zero-line correspond to local areas of deceleration in tracer uptake. This evaluation helps to capture the typical radial positions where the highest rates of uptake change, as understood from model Eq. (1), can be found within the overall tumor volume.
Another practical advantage of Eq. (3) is that it produces an output variable that is expressed in the same units as the original observation, in our case SUV units, since the scale of profile is fixed by our choice of a distribution [e.g., a gamma(3,1)]; the term serves to adapt this profile to the observation scale. This feature makes the proposed variable comparable across patients with the same disease, i.e., it is interpretable in a clinical setting for a given type of cancer. This measure assesses local intratumoral development, rather than of global propagation of the disease.
Voxel-specific quantitation Eq. (3) can be mapped to the 3D voxel coordinates to produce an image of uptake gradients. The left panel of Fig. 2 shows a transverse view of uptake gradients Eq. (3) over a liposarcoma tumor imaged with FDG-PET, color-coded according to classes of percentiles of the distribution of gradients. This iso-quantile map highlights a cluster of voxels located toward the metabolic core of the tumor with decreasing metabolic activity. Voxels of high uptake gradient are also picked up on this map, with a distinctive pattern of higher gradients outward from the metabolic core, and toward the bottom of the transverse uptake slice. The corresponding smooth 3D output metabolic volume delineation is also shown on Fig. 2 (bottom). It demonstrates how end-slices close in at the extremities, a feature provided by the regularization scheme of Eq. (5). Figure 2 shows a similar analysis for another FDG-PET study of a nerve sheath (soft tissue) sarcoma in the pelvic area.
Fig. 2.
Two FDG-PET case studies (left and right) correspond respectively to an inner thigh sarcoma) and a soft tissue sarcoma in the pelvis. Top row: transverse slice (mid-section) of the imaging data (left) and corresponding color mapping of quantile levels of uptake gradients (right), showing successive masks for quantile levels in , as per the pie chart. This visualization emphasizes areas of higher uptake gradient in the VOI. Bottom row: 3D rendering of the overall tumor volume.
The sample of voxel uptake gradients may also be summarized into a selected percentile so as to produce a single-value quantitation of overall uptake trends. This approach provides the opportunity to define novel PET-derived biomarkers for tumor risk characterization. We recently introduced and demonstrated the prognostic utility of similar uptake gradients derived from the rigid ellipsoidal model for sarcoma26 and non-small cell lung cancer.27 Their derivation from the present smooth tubular model of the volumetric tracer uptake distribution allows for finer local definition of these uptake gradients. The potential of such statistical summaries for improved characterization of patient survival and disease progression is demonstrated in Sec. 4 on a set of FDG-PET sarcoma studies.
2.4. Intratumoral Heterogeneity
A measure of intratumoral heterogeneity can also be derived from the fitted values as a measure of goodness-of-fit of model Eq. (1) to the overall 3D uptake data. A slice-by-slice approach was required to compute the overall intratumoral heterogeneity with the initial modeling approach,6 as the fitting procedure was not 3D-coherent. Transverse heterogeneities were computed as a model lack-of-fit for each slice separately, yielding slice-specific heterogeneity scores for each slice , where denotes the -statistic for slice . An overall heterogeneity score is obtained by averaging these spinal assessments. Figure 3 shows this approach. Using the regularized approach Eq. (5), intratumoral heterogeneity is also defined as a lack of fit based on the statistic; however, it is computed at once across the 3D volume as
(4) |
Fig. 3.
(a) Views of a thigh liposarcoma VOI, with input user-specified ellipsoidal VOI (yellow lines). (b) Transverse heterogeneities can be calculated on a slice-per-slice analysis fo the VOI, along the tumor spine ( axis). In this plot the black curve (dots) depicts the transverse heterogeneity summaries obtained from our initial, non-regularized model,6 and the red curve (rectangles) those obtained from the regularized approach, which are more coherent overall. (c) The effect of regularization Eq. (5) is visible when comparing renderings of the output delineated 3D volume using our previous non-regularized approach6 (left) to the tumor volume output from the regularized scheme (right).
The regularized fit confers more robustness to this assessment. In particular, the transverse slices located at the volume extremities, which typically contain less active points, are subject to lower signal-to-noise ratios. This hinders the former procedure;6 the regularized fit is more coherent at the extremities by integrating the whole volume in the optimization process.
3. Model Fitting and Numerical Evaluation
This section discusses computational aspects of the proposed model and its implementation, including numerical evaluation of the regularized Levenberg–Marquardt process used for its optimization, and an initialization scheme. A numerical illustration of the modeling approach, and the effect of regularization using simulated data are provided in Appendix A.
3.1. Model Optimization
We propose to fit this radial uptake model using a discretized Laplacian-penalized objective function
(5) |
where is the 3D discretized Laplacian of . This penalty is introduced so as to reduce second-order variations of the spatial structure, to achieve a smoother volume. The discretization is based on the central difference formula. In the above formulation is a regularization parameter that controls overall smoothness of the combination of functions. Thus, for , the resulting estimate would tend toward the rougher weighted least squares solution, whereas a large value of would entail stronger penalty and, in turn, oversmoothing of the initial model.
Due to the non-linearity of the tubular model, both the data-fit and penalty components of regularization model Eq. (5) are non-quadratic. A Gauss−Newton approach is used for optimization.28,30 The requisite gradients for sensitivity matrices are based on divided differences of and of . Linearization gives
for . Let and be the divided difference gradients with respect to -components of and . The overall linearized uptake model at for yields objective function
(6) |
with Jacobian
and its Laplacian
Letting , and given the vectorized form of linearized objective function
(7) |
The optimal solution is
(8) |
which can be re-expressed as
3.2. Control on Regularization
We expect that the choice of a selection scheme for should be data-driven, and would have only a minor impact on overall performance. The regularization parameter in Eq. (8) may be set by generalized cross-validation,28,31 whereby this parameter would be defined as
using Hat matrix . Other strategies are possible, such as controlling the level of sensitivity of the fitted values with respect to the observations. In this line we chose to set such that the regression effective degrees of freedom (the trace of ) should approximate ,28,32 yielding satisfactory results for our data. This choice is arbitrary and may need adapting from one dataset to another.
3.3. Numerical Stability
A regularized Levenberg–Marquardt optimization with a penalty on second-order spatial variations was first considered33 to minimize Eq. (5), where smoothing parameter was controlled via generalized cross-validation. However, this approach proved sensitive to potential sampling disparities that may arise when dealing with spatially irregular volumes of interest, which could lead to the model Hessian being ill-conditioned. For this reason, we introduced the use of thin-plate splines28 for the evaluation of Eq. (5) so as to obtain increased numerical stability in ill-conditioned cases.
3.4. Positivity Constraints
To enforce positivity constraints on , , and , linearized solution Eq. (8) of the optimization scheme may be further updated to satisfy positivity constraints via an order-restricted linear model (a quadratic programming approach),34 i.e., Eq. (7) is minimized with respect to subject to positivity.
3.5. Initialization
The implementation of solution Eq. (8) requires initialization for the spline bases Eq. (7), and corresponding initial fit
This initialization can be set using from the output phases
and corresponding output fit from our former, simpler tubular representation6 of uptake
(9) |
This simpler scheme uses a Gaussian profile function , which can be converted into an approximate gamma profile function for use in the regularized construction Eq. (1).
Initial vectors for Eq. (7) are set based on a linear transformation of the preliminary set of values, so as to preserve the overall arrangement of voxel phases from the initial, unregularized tubular model fit. This ensures the overall uptake profile signatures derived from each model are coherent voxel-wise. Standardizing along the spinal direction yields initial phases which are used when deriving the initial (Gaussian) uptake profile Eq. (9).6 Final initial phase values for the regularized approach can then be for some with . Since each choice of profile distribution has a different range, the change in amplitude scale is also controlled by overall adjustment , for which an initial guess may be . The rigid transformation from original fit to regularization input fit using is adjusted by minimizing the nonlinear least squares difference between the two fits, yielding . Finally, this implies
where the final starting amplitude values are set equal along the -direction at each (projected) transverse slice , since the previous model did not include angular variation of the voxel amplitude parametrization. In the above expressions, we assume the same (user-specified) tubular sampling dimensions are used in both models, but if these were different, the original parametric vectors may be interpolated to the new parametric sampling space used for regularization.
3.6. Software Implementation
An open-source implementation of the original modelization6 in R35 is available online.36 Our implementation of Eq. (8) was mainly developed in R, with a number of computation modules written in Fortran, C and C++. This implementation is also available on GitHub.37
4. Results on Patient Data
This section presents a numerical comparison of tumor characterization derived from the proposed model over previous strategies on a sarcoma cohort, and results of analyses of overall survival (OS) and disease-free survival (DFS) assessing the potential prognostic benefit of statistical summaries of the distribution of uptake gradients , defined in Eq. (3).
4.1. Sarcoma Dataset
Analyses were carried out on a cohort of 158 sarcoma patients (73 deaths, 71 disease progressions) with over 10 years follow-up information, whose FDG-PET studies were acquired at the University of Washington School of Medicine using a GE Advance scanner.6,24 The dataset included routine information on patient age (mean 45 years, sd 16 years) and sex (66 female, 92 male) at baseline, metabolic tumor volume, grade (77 high, 56 intermediate, and 25 low grade), sarcoma type (41 bone, 12 cartilage, and 105 soft tissue), and quantitation of (mean 8.2, sd 5.6), 1cc- (mean 6.2, sd 4.6), (mean 2, sd 1), and total lesion glycolysis (TLG) at baseline. Two heterogeneity measures and derived from each of the ellipsoidal and regularized tubular models, along with first, second and third gradient quartiles derived from each of these two models also, were further included for analysis. In addition to these 19 routine and structural variables, an additional 64 radiomic features comprising of 16 shape features, 23 histogram-based features and 25 gray-level co-occurrence matrix (GLCM) features were included in the dataset, given their predominance in the recent literature. These were calculated in line with the Image Biomarker Standardisation Initiative guidelines,16 using an open-source implementation calibrated to these.36 Where required the input VOIs were linearly interpolated to cubic voxels of dimension and discretized to 64 gray levels using a fixed bin number for extraction of these radiomics features. When fitting the volumetric uptake models, most volumes of interest were resampled onto a grid of 26 transverse slices () and 25 angular sectors () in the projected space. A number of smaller input volumes required fewer bins in either direction, down to an -grid. A total of 83 variables were considered for analysis of OS and DFS via Cox proportional hazard models.
4.2. Comparison of Uptake Summaries
The proposed smooth tubular representation of volumetric tracer uptake yields some differences in uptake summaries, relative to previous modeling strategies proposed by this group, although preserving overall assessment of structural heterogeneity of the volumetric tracer uptake distribution. We illustrate this by comparing the distributions of corresponding metabolic summaries obtained for the set of 158 clinical sarcoma FDG-PET studies described in Sec. 4.1.
Figure 4 shows how the smooth tubular representation provides a reasonable alternative evaluation of volumetric heterogeneity, . Figure 4(a) shows a comparison of model fits achieved by the two models for two typical sarcoma uptake patterns, with a tighter description of voxel radial location, and therefore higher spatial resolution, achieved by the tubular approach. Heterogeneity quantitation obtained from the latter aligns strongly with that derived from the rigid-shape ellipsoidal representation [Fig. 4(b)], whose prognostic potential has been demonstrated for various cancers.26 Figure 4(c) further shows that the proposed tubular model offers finer differentiation of heterogeneity levels, compared to the earlier ad-hoc tubular modeling approach which does not span the [0, 100] range as effectively. Figure 5 shows difference in summaries of uptake gradients , over those derived from the rigid-shape ellipsoidal representation.26
Fig. 4.
(a) Fits obtained from the ellipsoidal (left) and regularized tubular (right) models, for the two studies of Fig. 1 (resp., top and bottom). (b) Comparisons of heterogeneity quantitations of 158 primary sarcoma tumors, derived from the ellipsoidal and regularized tubular models of volumetric uptake. Dot size is proportional to tumor volume. Spline regressions (thick lines) characterize the difference in distributions, with bootstrapped 95% confidence bands indicated in a yellow shade. (c) Same comparison of heterogeneities values, but obtained from the ad hoc and regularized tubular models.
Fig. 5.
Comparisons of gradient summaries from the clinical cohort of Fig. 4, derived from the ellipsoidal and regularized tubular models of volumetric uptake. The plots provide scaled comparisons of the distributions of the first, second, and third quartiles of the collections of gradients obtained from each study, respectively from left to right. Dot size is proportional to tumor volume. Spline regressions (thick lines) characterize the difference in distributions, with bootstrapped 95% confidence bands indicated in a yellow shade.
4.3. Univariate Relationships and Significance
Associations were considered where any two of the 83 variables had Pearson correlation above 0.70 in absolute value, or a significant association in the sense of a two-sided Mann–Whitney or chi-square test, or an analysis of variance, whichever was appropriate with respect to the nature of these variables. All test -values were corrected for false discovery rate (FDR).35,38 Variables , , and were found to be significantly associated with tumor grade (). It was also the case for the third quartile of rigid-shape ellipsoidal gradients (), and for 21 of the radiomic features. Variables age () and eight GLCM features () were found to be significantly associated with tumor type. No significant associations were found with patient sex. High correlation was found in 259 unique pairs of variables. In particular, this analysis indicated substantial alignment between and (), and (0.81), and and (0.98), as well as between and (0.71), , and (0.79), and (0.76), and and (0.94). Variables , , , and also all aligned strongly with (histogram coefficient of variation; 0.74, 0.73, 0.84, and 0.83 resp.), and with (0.71). All other strong correlations were amongst radiomic features. Figure 6 shows the correlation matrix for the subset of numerical features with respect to the 0.70 correlation threshold. This analysis shows rather limited associations between variables derived from the proposed 3D spatial modeling and classic radiomic features. As expected, strong correlations can be observed in particular among the subset of radiomic features.
Fig. 6.
Correlation matrix of the numerical features in the sarcoma dataset. Correlations within have been blanked out to highlight strong associations among the feature set, using a 4-color palette (shown on right-hand side).
Two variables were found to be significant univariate predictors of OS (structural heterogeneity and maximum histogram gradient (ih.max.grad, a radiomic feature), both with ). Variable was also found to be a significant univariate predictor of DFS after Bonferroni correction () but not after FDR correction. No other univariate associations with either OS or DFS were found.
4.4. Prognostic Significance
4.4.1. Variable selection
Unsupervised pre-filtering of the dataset of 83 features was performed to remove redundant variables on the basis of an absolute correlation .39 Tumor type was also removed to reduce confounding, on the basis of the analyses reported in Sec. 4.3. Following this process, a total of 41 features were removed, including , TLG, and , the rest being radiomic features. The resulting filtered subset of features was considered for prognostic analysis. A final set of features was selected after bootstrap analysis of regularized Cox proportional hazards models40 for both OS and DFS, to evaluate selection stability.41 A total of 100 bootstrapped fits were obtained with a lasso-like penalty, and another 100 bootstrapped fits with an elastic net penalty. The hyper-parameter for these models were tuned using 10-fold cross-validation nested within the bootstrapping framework. Five patients with incomplete data were removed from the multivariate survival analyses. Variables tumor grade, patient sex, , , the third quartile of uptake gradients derived from the regularized tubular approach, were selected for at least two thirds of the bootstrapped samples, for both elastic net and lasso-like regularizations. These features were also all selected frequently (i.e., at least 60% of the time) by a similar framework for DFS, as well as in similar bootstrap analyses of both forward and backward stepwise selections of Cox models for both OS and DFS. Following this combined assessment, this set of five variables was used in a final Cox model for further evaluation of prognostic performance.
4.4.2. Prognostic performance
On the basis of the feature selection described above, predictive performance of the multivariate prognostic model comprising of these five predictors was subsequently evaluated, also via bootstrapping. The multivariate Cox model was fitted to 100 bootstrap resamples and the median hazard ratios and associated -values were calculated from these fits for each covariate. linear (Cox) score predictions and corresponding concordance indices were derived for the out-of-bag (OOB) sample points from each fit. The median and nonparametric 95% confidence interval (CI) of OOB concordance indices were calculated to evaluate prediction performance. The same framework was used for OS and DFS. Results from these analyses are given in Table 1. For OS, all covariates were statistically significant at the 5% level in this model, with bootstrapped concordance index (CI: 0.61–0.78). For DFS, tumor grade, and were also statistically significant, yielding a model with bootstrapped concordance index (CI: 0.54–0.71).
Table 1.
Output from bootstrapped multivariate Cox analyses for OS and DFS for the sarcoma cohort ( for OS; for DFS). denotes model concordance, which measures the proportion of accurate outcome predictions in the cohort based on the model. -values below the 5% significance threshold are highlighted in bold.
Variable | OS (, CI: 0.61–0.78) | DFS (, CI: 0.54–0.71) | ||||
---|---|---|---|---|---|---|
Hazard ratio | 95% C.I. | -value | Hazard Ratio | 95% C.I. | -value | |
Grade (intermediate) | 0.60 | [0.34; 1.05] | 0.0746 | 0.78 | [0.45; 1.35] | 0.3739 |
Grade (low) | 0.23 | [0.07; 0.77] | 0.0157 | 0.32 | [0.13; 0.80] | 0.0170 |
Sex | 1.78 | [1.06; 2.97] | 0.0298 | 1.47 | [0.89; 2.40] | 0.1359 |
1.72 | [1.25; 2.31] | 0.0010 | 1.30 | [0.94; 1.78] | 0.1201 | |
1.56 | [1.19; 2.04] | 0.0011 | 1.41 | [1.09; 1.82] | 0.0107 | |
Gradient | 1.63 | [1.10; 2.41] | 0.0144 | 1.58 | [1.03; 2.43] | 0.0347 |
A Kaplan–Meier analysis was carried out on the medians of OOB linear score predictors for each sample point, to obtain a set of bootstrapped survival curves for low- and high-risk groups. The log-rank test -value from these Kaplan–Meier curve estimates indicated a statistically significant separation between the two risk groups ( for both OS and DFS).
We note that the resampling framework described in this section may have induced some selection bias, since the whole dataset was used for both feature selection (yielding the final five variable predictive model) and OOB prediction performance assessment. It nonetheless provides a reasonable compromise given the sample size available, and (unlike other schemes) yields a unique model to more conveniently illustrate the prognostic potential of the proposed methodology for tumor characterization, allowing for output such as those of Table 1 and Fig. 7. This bias was evaluated by comparison with an alternative resampling framework, further described in Appendix D. This analysis suggests that the reported results are not substantially compromised by potential feature selection bias.
Fig. 7.
Nonparametric Kaplan–Meier estimates of the survival functions following bootstrapping of the multivariate model of Table 1 (solid lines). Dashed lines indicate 95% confidence bands around the low-risk (blue) and high-risk (red) groups obtained from this analysis, showing a statistically significant risk group separation (log-rank test: for both OS and DFS).
5. Discussion
We proposed a novel modeling approach to analyze the spatial distribution of PET tracer uptake within the volume of interest of a solid tumor, and derived novel prognostic summaries from this representation. The modeling strategy consists in fitting a deformable tubular structure to the volumetric uptake distribution via 3D-coherent optimization and spatial regularization, penalizing high second-order variations of the model fit. This technique yields a profile curve that describes tracer uptake intensity as a function of voxel radii in the model’s own coordinate reference system. This approach offers a novel opportunity to assess local trends in the uptake pattern via the derivation of uptake gradients directly from the profile curve. A unique advantage of this flexible model is that this gradient information is locally coherent with the spatial uptake characteristics within the tumor volume. Intratumoral heterogeneity can also be assessed directly as a measure of lack-of-fit of this profile curve.
This smooth tubular representation of volumetric tracer uptake has some similarities with previous modeling strategies proposed by our group, namely an ad-hoc tubular model,6 and a former rigid-shape ellipsoidal description24 of the 3D uptake data. The gain offered by regularization and a more volumetrically coherent optimization of the novel tubular model over the previous ad-hoc approach was illustrated on patient data, with a more appropriate volume rendering and more consistent heterogeneity evaluation. This paper also demonstrated that heterogeneity obtained from the proposed model aligned strongly with that derived from the former rigid ellipsoidal representation (whose prognostic value was demonstrated for various cancers), but that it could achieve a tighter fit, thus yielding a locally more accurate representation of uptake data at the voxel level.
Analyses of simulated data demonstrated numerical stability and consistency of the proposed approach at varying noise levels. The model was also assessed on a clinical cohort of 158 sarcoma patients imaged with FDG-PET. The heterogeneity and uptake gradient summaries derived from the model were found to be statistically significant prognostic variables in both univariate and multivariate Cox proportional hazards analyses of patient survival. In particular, the third quartile of uptake gradients was retained as a significant prognostic indicator in multivariate prognostic models of both overall and DFS following a feature elimination process, and controling for tumor heterogeneity.
The scope of this paper is the conceptual presentation of the regularized tubular model, and the demonstration of the prognostic potential of structural heterogeneity and uptake gradient summaries derived from it, although this assessment included radiomics features. Follow-on work will be pursued to better understand the place for statistical modeling within the context of radiomics analyses. Our group previously demonstrated that tumor characterization based on the rigid ellipsoidal model offered additional prognostic opportunities complementary to common morphology-, intensity-, and texture-based radiomic features,26 which encourages investigations in this direction for the tubular representation.
The present contribution has a number of limitations and may be further extended in various ways. In the case of very large tumors, the optimization process can become taxing, with hundreds of model parameters to estimate via an iterative process that requires evaluation of second-order quantities; this was indeed a limitation in our study. (This issue is compounded by the fact that many of the core modules of the software are currently implemented in R.)35 Subsampling schemes and stochastic optimization strategies could be considered in future updates to reduce this cost and render this numerical process more feasible on mainstream computational platforms. Generalization of the approach could also be facilitated by a different choice for the uptake profile function , and a fully non-parametric alternative to the gamma distribution will be explored in future work. Whatever the choice of profile, however, the output prognostic variables extracted from the unimodal uptake profile fit will provide a locally detailed description of uptake activity.
In terms of clinical interpretation, it may be thought that given they assess local uptake trends, the collection of uptake gradients derived from the proposed model could offer a mechanism for projection of future uptake characteristics based on a single scan. Localized assessment of the intratumoral tracer uptake structure and the identification of subvolumes of higher rate of change in the uptake distribution derived from the model could create an opportunity both for improved patient prognosis and for advanced image-guided treatment, e.g., by guiding biopsy or radiotherapy. This will be explored in future work with follow-up imaging data at various treatment timepoints.
6. Conclusion
This paper described a smooth, data-adaptive spatial model of 3D PET uptake that represents the tumor activity in terms of a deformable tubular structure. The approach provides novel opportunities for detailed visualization and description of volumetric tracer uptake, thanks to a more coherent and accurate representation of its distribution at the voxel level over previous 3D modeling strategies. Consistency of this approach was illustrated via analyses on simulated volumetric data. A 1D uptake profile curve is obtained from this model fit as a function of radial voxel location (in the model referential), that can be summarized into prognostic descriptors of structural features of the PET data. Analysis of a clinical cohort of sarcoma patients showed strong alignment of intratumoral heterogeneity quantitation derived from this model, with that obtained from a well-established rigid-shape ellipsoidal modeling approach. This further demonstrates relevance of the proposed model for prognostic evaluation. Uptake gradients can also be obtained from the model fit at each voxel, to measure local uptake trends. These gradients can be summarized into additional pseudo-biomarkers, whose prognostic potential was demonstrated on the sarcoma cohort using multivariate prognostic models controled for intratumoral heterogeneity.
7. Appendix A: Simulation Framework
We describe numerical simulation output that illustrates the behavior of proposed model Eq. (1) and the considered regularization scheme. The simulated dataset consists of a smooth regular volumetric structure consisting of voxels arranged in 12 transverse slices, each slice containing voxels. This volume is consistent with a small-to-medium sarcoma volume of interest. In the example the simulated uptake distribution is tubular with a less active core. The spline bases are defined by a grid.
True simulated uptake values from model Eq. (1) were generated for fixed true values , setting and to be constant with respect to voxel angular position at each transverse slice, varying with successive voxel elevations respectively as follows:
and
also varied only with as . Figure 8 shows the noiseless simulated structure and its histogram is given in Fig. 9. The simulated uptake volume is also shown in Fig. 10. Initial values were set using , (which yields a mirrored -structure), and , .
Fig. 8.
From left to right: mid-volume transverse, sagittal and coronal slices of the simulated dataset, showing the smooth ellipsoidal structure and its decreasing core activity, using a common grayscale. Both coronal and sagittal views are presented with the transverse axis going along the vertical direction.
Fig. 9.
(a) Distribution of the noiseless simulated uptake structure (gray), and overlaid noisy uptake (); values below 2 correspond either to background voxels or cold volume core. (b) Mid-volume transverse slice of the synthetic image with , using the colorscale of Fig. 8.
Fig. 10.
Illustration of convergence and goodness-of-fit for one experiment with and . Left: 3D wireframe views of the true, initial, and final volumes, respectively. The volume boundaries correspond to isocontours defined by the first quartile () of the model evaluated at the corresponding values of parameters , i.e., respectively, , and . Right: distributions of model fit residuals for initial and final parameter estimates. Inset: RSS as a function of iteration step, demonstrating convergence of the nonlinear optimization of regularized model Eq. (1).
In “noisy” scenarios, randomness was incorporated in the form of an additive perturbation term with a normal distribution, i.e., for independent and identically distributed realizations , where controls the volumetric coefficient of variation. In these numerical experiments we used noise levels . Figure 9 shows the effect of noise level.
8. Appendix B: Illustration on a Single Experiment
Figure 10 shows the behavior of the proposed fitting procedure for the same typical experiment where and , demonstrating algorithmic convergence of the iterative nonlinear optimization scheme for regularized model Eq. (1), and a tight distribution of model fit residuals. This figure also underlines the large initial errors in model fit corresponding to the chosen initialization scheme.
9. Appendix C: Monte Carlo Analysis
Monte Carlo experiments were performed for each of four noise levels
and compared to the noiseless case. The value of regularization parameter was fixed for any simulated experiment, with reasonable values within . The Monte Carlo distributions of mean square errors (MSE) in model fit shown in Fig. 11 were obtained respectively without () and with ( and 0.2) regularization. The figure indicates a decrease in MSE with a decreasing image coefficient of variation. A comparison between these distributions highlights how regularization increases residual sum of squares (RSS), compared to the case, by increasing the bias in model fit to achieve smoother output volumes.
Fig. 11.
Monte Carlo distributions of the MSE in model fit for decreasing image coefficient of variation, with varying from 1 to 0, for various regularization magnitudes ( for bottom gray, middle red, and top blue boxplots in each grouping, respectively). Inset: same MSE distributions for the four noisy scenarios, but where both axes are presented on the log-scale to illustrate the rate of convergence of the MSE to the noise level. The slope of the fitted line (for ) is , indicating the expected parametric rate of MSE convergence (solid black line).
10. Appendix D: Alternative Analysis of the Clinical Cohort
An alternative framework was considered to assess selection bias of the analysis of Sec. 4.4 using repeated 70% to 30% splits of the entire dataset. For each such split, bootstrapping was performed on the 70% sub-sample to train regularized Cox models and select features on the basis of stability, using a 60% selection rate cutoff, similarly to the framework described in Sec. 4.4. In this setting, feature selection was thus nested within model training, and the corresponding final model was evaluated on hold-out data. This process was repeated on 100 different splits of the dataset to measure average hold-out prediction performance, each time using both elastic net and lasso regularizations. It is noteworthy that this alternative framework allowed the feature subset constituting the model to differ with each split. The mean hold-out accuracy under this framework was 0.67 (CI: 0.54–0.80). Increased variability was due to the variation in models used on the hold-out sets, and the framework itself. [The five-variable model of Sec. 4.4 yielded a mean accuracy of 0.70 (CI: 0.60–0.83) in this new framework.] From this analysis we suggest that selection bias inherent to the results reported on the five-variable model remains reasonable. Features selected over 50% of the time in this second framework included tumor grade, patient sex, , , the third quartile of uptake gradients , and two radiomic features, namely PCA elongation and .
Acknowledgments
This publication has emanated from research conducted with the financial support of Science Foundation Ireland under Grant Nos. PI-11/1027 and 12/RC/2289-P2, the latter being co-funded under the European Regional Development Fund. For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission. This work was also partly funded under National Institutes of Health / National Cancer Institute Grant No. ROI-CA-65537. This work complies with ethical standards.
Biography
Biographies of the authors are not available.
Disclosures
There are no conflicts of interest to report, financial, or otherwise.
Contributor Information
Eric Wolsztynski, Email: eric.w@ucc.ie.
Finbarr O’Sullivan, Email: f.osullivan@ucc.ie.
Janet F. Eary, Email: janet.eary@nih.gov.
References
- 1.Wahl R. L., et al. , “From RECIST to PERCIST: evolving considerations for PET response criteria in solid tumors,” J. Nucl. Med. 50(Suppl 1), 122S–150S (2009). 10.2967/jnumed.108.057307 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Jadvar H., Alavi A. A., Gambhir S., “-FDG uptake in lung, breast, and colon cancers: molecular biology correlates and disease characterization,” J. Nucl. Med. 50(11), 1820–1827 (2009). 10.2967/jnumed.108.054098 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Locke L., et al. , “FDG-PET quantification of lung inflammation with image-derived blood input function in mice,” Int. J. Mol. Imaging 2011, 356730 (2011). 10.1155/2011/356730 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Eary J., et al. , “Sarcoma mid-therapy F-18fluorodeoxyglucose positron emission tomography (FDG PET) and patient outcome,” J. Bone Joint Surg. 96(2), 152–158 (2014). 10.2106/JBJS.M.00062 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Luber B. S., et al. , “Response to early treatment evaluated with -FDG PET and PERCIST 1.0 predicts survival in patients with Ewing sarcoma family of tumors treated with a monoclonal antibody to the insulinlike growth factor 1 receptor,” J. Nucl. Med. 57(5), 735–740 (2016). 10.2967/jnumed.115.162412 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.O’Sullivan F., et al. , “A statistical modeling approach to the analysis of spatial patterns of FDG-PET uptake in human sarcoma,” IEEE Trans. Med. Imaging 30(12), 2059–2071 (2011). 10.1109/TMI.2011.2160984 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Davnall F., et al. , “Assessment of tumor heterogeneity: an emerging imaging tool for clinical practice?” Insights Imaging 3(6), 573–589 (2012). 10.1007/s13244-012-0196-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.van Gomez Lopez O., et al. , “Heterogeneity in fluorodeoxyglucose positron emission tomography/computed tomography of non-small cell lung carcinoma and its relationship to metabolic parameters and pathologic staging,” Mol. Imaging 13(9), 7290–2014 (2014). 10.2310/7290.2014.00032 [DOI] [PubMed] [Google Scholar]
- 9.Soussan M., et al. , “Relationship between tumor heterogeneity measured on FDG-PET/CT and pathological prognostic factors in invasive breast cancer,” PLoS One 9(4), e94017 (2014). 10.1371/journal.pone.0094017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Hatt M., et al. , “-FDG PET uptake characterization through texture analysis: investigating the complementary nature of heterogeneity and functional tumor volume in a multi-cancer site patient cohort,” J. Nucl. Med. 56(1), 38–44 (2015). 10.2967/jnumed.114.144055 [DOI] [PubMed] [Google Scholar]
- 11.Rahmim A., et al. , “Novel parametric PET image quantification using texture and shape analysis,” in Nucl.Sci. Symp. and Med. Imaging Conf. (NSS/MIC), IEEE, pp. 2227–2230 (2012). 10.1109/NSSMIC.2012.6551507 [DOI] [Google Scholar]
- 12.Orlhac F., et al. , “Multi-scale texture analysis: from -FDG PET images to pathological slides,” J. Nucl. Med. 57(6), 1823–1828 (2016). 10.2967/jnumed.116.173708 [DOI] [PubMed] [Google Scholar]
- 13.Lambin P., et al. , “Radiomics: the bridge between medical imaging and personalized medicine,” Nat. Rev. Clin. Oncol. 14, 749–762 (2017). 10.1038/nrclinonc.2017.141 [DOI] [PubMed] [Google Scholar]
- 14.Vallières M., et al. , “Responsible radiomics research for faster clinical translation,” J. Nucl. Med. 59(2), 189–193 (2019). 10.2967/jnumed.117.200501 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Hatt M., et al. , “Machine (deep) learning methods for image processing and radiomics,” IEEE Trans. Radiat. Plasma Med. Sci. 3(2), 104–108 (2019). 10.1109/TRPMS.2019.2899538 [DOI] [Google Scholar]
- 16.Zwanenburg A., et al. , “The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping,” Radiology 295(2), 328–338 (2020). 10.1148/radiol.2020191145 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Da-Ano R., Visvikis D., Hatt M., “Harmonization strategies for multicenter radiomics investigations,” Phys. Med. Biol. 65(24), 24TR02 (2020). 10.1088/1361-6560/aba798 [DOI] [PubMed] [Google Scholar]
- 18.Ferreira M., et al. , “[] FDG PET radiomics to predict disease-free survival in cervical cancer: a multi-scanner/center study with external validation,” Eur. J. Nucl. Med. Mol. Imaging 48, 1–12 (2021). 10.1007/s00259-021-05303-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Orlhac F., et al. , “How can we combat multicenter variability in MR radiomics? Validation of a correction procedure,” Eur.Radiol. 31(4), 2272–2280 (2021). 10.1007/s00330-020-07284-9 [DOI] [PubMed] [Google Scholar]
- 20.Pinker K., et al. , “Background, current role, and potential applications of radiogenomics,” J. Magn. Reson. Imaging 47(3), 604–620 (2018). 10.1002/jmri.25870 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Bodalal Z., et al. , “Radiogenomics: bridging imaging and genomics,” Abdom. Radiol. 44(6), 1960–1984 (2019). 10.1007/s00261-019-02028-w [DOI] [PubMed] [Google Scholar]
- 22.Gullo R. L., et al. , “Combining molecular and imaging metrics in cancer: radiogenomics,” Insights Imaging 11(1), 1–17 (2020). 10.1186/s13244-019-0795-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Lasocki A., et al. , “Neuro-oncology and radiogenomics: time to integrate?” Am. J. Neuroradiol. 41(11), 1982–1988 (2020). 10.3174/ajnr.A6769 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Eary J., et al. , “Spatial heterogeneity in sarcoma -FDG uptake as a predictor of patient outcome,” J. Nucl. Med. 49(12), 1973–1979 (2008). 10.2967/jnumed.108.053397 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Jones-Todd C. M., et al. , “Identifying prognostic structural features in tissue sections of colon cancer patients using point pattern analysis,” Stat. Med. 38(8), 1421–1441 (2019). 10.1002/sim.8046 [DOI] [PubMed] [Google Scholar]
- 26.Wolsztynski E., et al. , “Positron emission tomography-based assessment of metabolic gradient and other prognostic features in sarcoma,” J. Med. Imaging 5(2), 024502 (2018). 10.1117/1.JMI.5.2.024502 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Wolsztynski E., et al. , “Combining structural and textural assessments of volumetric FDG-PET uptake in NSCLC,” IEEE Trans. Radiat. Plasma Med. Sci. 3(4), 421–433 (2019). 10.1109/TRPMS.2019.2912433 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Wahba G., Spline Models for Observational Data, Vol. 59, pp. xii + 169, SIAM; (1990). [Google Scholar]
- 29.Roose T., Chapman S., Maini P., “Mathematical models of avascular cancer,” SIAM Rev. 49(2), 179–208 (2007). 10.1137/S0036144504446291 [DOI] [Google Scholar]
- 30.Gill P., Murray W., “Algorithms for the solution of the nonlinear least-squares problem,” SIAM J. Numer. Anal. 15(5), 977–992 (1978). 10.1137/0715063 [DOI] [Google Scholar]
- 31.Golub G., Heath M., Wahba G., “Generalized cross-validation as a method for choosing a good ridge parameter,” Technometrics 21(2), 215–223 (1979). 10.1080/00401706.1979.10489751 [DOI] [Google Scholar]
- 32.Ye J., “On measuring and correcting the effects of data mining and model selection,” J. Am. Stat. Assoc. 93(441), 120–131 (1998). 10.1080/01621459.1998.10474094 [DOI] [Google Scholar]
- 33.Wolsztynski E., et al. , “Analysis of spatial patterns in tumor imaging data: a method for assessment of tubularity,” in Proc. IEEE Nucl. Sci. Symp. and Med. Imaging Conf., Seattle, Washington, Vol. 56, pp. 1–5 (2014). 10.1109/NSSMIC.2014.7431031 [DOI] [Google Scholar]
- 34.Gromping U., “Inference with linear equality and inequality constraints using R: the package ic.infer,” J. Stat. Software 33(10), 1–31 (2010). 10.18637/jss.v033.i10 [DOI] [Google Scholar]
- 35.R Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna: (2015). [Google Scholar]
- 36.Wolsztynski E., “mia,” https://github.com/ericwol/mia/releases/latest (2017).
- 37.Wolsztynski E., “volumetrics,” https://github.com/ericwol/volumetrics/releases/latest (2022).
- 38.Benjamini Y., Yekutieli D., “The control of the false discovery rate in multiple testing under dependency,” Ann. Stat. 29, 1165–1188 (2001). 10.1214/aos/1013699998 [DOI] [Google Scholar]
- 39.Kuhn M., “Caret: classification and regression training,” R package version 6.0-90 (2021).
- 40.Simon N., et al. , “Regularization paths for Cox’s proportional hazards model via coordinate descent,” J. Stat. Software 39(5), 1–13 (2011). 10.18637/jss.v039.i05 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Meinshausen N., Bühlmann P., “Stability selection,” J. R. Stat. Soc.: Ser. B (Stat. Methodol.) 72(4), 417–473 (2010). 10.1111/j.1467-9868.2010.00740.x [DOI] [Google Scholar]