MRI-based radiomics model can improve the predictive performance of postlaminar optic nerve invasion in retinoblastoma

Abstract

Objectives:

To develop an MRI-based radiomics model to predict postlaminar optic nerve invasion (PLONI) in retinoblastoma (RB) and compare its predictive performance with subjective radiologists’ assessment.

Methods:

We retrospectively enrolled 124 patients with pathologically proven RB (90 in training set and 34 in validation set) who had MRI scans before surgery. A radiomics model for predicting PLONI was developed by extracting quantitative imaging features from axial T2W images and contrast-enhanced T1W images in the training set. The Kruskal–Wallis test, least absolute shrinkage and selection operator regression, and recursive feature elimination were used for feature selection, where upon a radiomics model was built with a logistic regression (LR) classifier. The area under the curve (AUC) of the receiver operating characteristic (ROC) curve and the accuracy were assessed to evaluate the predictive performance in the training and validation set. The performance of the radiomics model was compared to radiologists’ assessment by DeLong test.

Results:

The AUC of the radiomics model for the prediction of PLONI was 0.928 in the training set and 0.841 in the validation set. Radiomics model produced better sensitivity than radiologists’ assessment (81.1% vs  43.2% in training set, 82.4vs 52.9% in validation set). In all 124 patients, the AUC of the radiomics model was 0.897, while that of radiologists’ assessment was 0.674 (p < 0.001, DeLong test).

Conclusion:

MRI-based radiomics model to predict PLONI in RB patients was shown to be superior to visual assessment with improved sensitivity and AUC, and may serve as a potential tool to guide personalized treatment.

Introduction

Retinoblastoma (RB) is the most common primary intraocular malignancy in children, with an incidence of approximately 1 in 15,000 to 20,000 live births worldwide.¹ Because of the dramatic evolution of RB management and treatment options, the survival rate exceeds 95% in developed countries, while it remains at 40–80% in Asia and Africa.^2–4 Distant metastasis is the principal cause of disease-related death among RB patients. The presence of postlaminar optic nerve invasion (PLONI) is a poor prognostic indicator and is associated with a higher likelihood of local recurrence or systemic metastasis.^5–8 In recent years, substantial changes have taken place in terms of treatment regimens, with the increasing use of eye-sparing treatment strategies.^9,10 Although eye-salvage treatment has major advantages with regard to patient care, the lack of histopathologic examination for the detection of risk factors creates a challenge when making the decision whether to select enucleation or eye-sparing treatment. Therefore, accurate preoperative information about PLONI is crucial for identifying surgical candidates among those considered eye-sparing treatment strategies.

Magnetic resonance imaging (MRI) is an important technique to detect the presence of PLONI in patients who are considering eye-sparing treatment strategies.^11–14 Currently, contrast-enhanced MRI is the imaging modality of choice for preoperative RB staging and a standard procedure in clinical practice.¹⁴ Many authors have reported the role of MRI in retinoblastoma and have shown an unsatisfactory efficacy at identifying PLONI, with a relatively limited accuracy of 52–79% and a relatively low sensitivity with a wide range (37%–78%).^{11–13,15–17} This low efficacy has led to a considerable proportion of patients being understated or overstated. Thus, the performance of noninvasive assessment by MRI has room for improvement. Radiomics uses the high-throughput extraction of advanced quantitative features to objectively and quantitatively describe tumor phenotypes, and it is gaining importance in cancer research.^18,19 These quantitative features, which may fail to be appreciated by the naked eye, can potentially provide valuable diagnostic, prognostic or predictive information in oncology.²⁰ Recent studies^21–23 have shown that many radiomics features can significantly differentiate between early- and advanced-stage disease. To the best of our knowledge, there has been no MRI-based study on radiomics analysis for the preoperative prediction of PLONI in RB patients to date.

The purpose of the current study was to develop an MRI-based radiomics model with a logistic regression (LR) classifier to predict PLONI in RB patients and to compare its predictive performance with that of subjective radiologist assessment.

Methods and materials

Patients

This retrospective study adhered to the tenets of the Declaration of Helsinki and was approved by the local Ethics Committee of our hospital. The requirement for written informed consent was waived. Between January 2008 and January 2018, 124 consecutive patients with pathologically proven RB after enucleation were included in this study. The patient enrolment process for this study is shown in Figure 1. The inclusion criteria were as follows: (a) all patients had an MRI scan of the orbit with pre- and postcontrast images, and (b) all MRI scans were performed within 4 months of enucleation. The exclusion criteria included (a) poor quality of MR images due to motion artifacts and (b) the RB patients who were negative in PLONI and had received short-term treatment before enucleation. Among the 124 patients, nine patients had bilateral RB tumours, but only one tumour per patient was used in the data analysis (that with the shorter interval time between the MRI scan and enucleation). The pathological assessment of PLONI in the 124 RB patients showed that PLONI was confirmed in 54 globes and absent in 70 globes. Among the 54 RB patients with PLONI, 20 patients had received 1–3 cycles of CEV (carboplatin, etoposide, vindesine) intravenous chemotherapy (IVC). The median time between MR imaging and enucleation for the 20 patients was 54 days (range, 7–119 days). The median time between MR imaging and enucleation for the other 104 patients without any preoperative treatment was 16 days (range, 1–52 days).

Figure 1.

Flowchart of the patient enrolment process. PLONI: postlaminar optic nerve invasion. RB: retinoblastoma

The consecutive study population was divided into two groups according to the time point. The training set (recruited from January 2008 to June 2015) consisted of 90 patients (37 with PLONI and 53 without PLONI). The validation set (recruited from July 2015 to January 2018) consisted of 34 consecutive patients (17 with PLONI and 17 without PLONI).

MR image acquisition

All MR images were obtained with a 1.5-Tesla (Signa Highspeed, GE Healthcare, Milwaukee, USA, n = 49) or a 3-Tesla (GE HDxt, GE Healthcare, Milwaukee, USA, n = 46 or Discovery MR750; GE Healthcare, Milwaukee, WI, USA, n = 29) scanner. Precontrast axial T1W images (T1-WI), T2W images (T2-WI) and postcontrast enhanced T1-WI (CET1-WI) in the axial and coronal planes were acquired for all 124 patients. The imaging parameters are shown in Table 1. CET1-WI was obtained after an i.v. bolus injection of 0.1 mmol/kg gadopentetate dimeglumine. Fat suppression (FS) was used in the axial CET1-WI.

Table 1.

MR scanning parameters

Sequence	TR (ms)	TE (ms)	Field of view (mm)	Number of slices	Slice thickness (mm)	Slice gap (mm)	NEX	Matrix
T1-WI	400–500	8–11	160 × 160	16	3.0	0.3	2	256*384 (3.0T) 244*288 (1.5T)
T2-WI	2860–4440	115–120	160 × 160	16	3.0	0.3	1	256*384 (3.0T) 244*288 (1.5T)
CET1-WI	400–500	8–11	160 × 160	16	3.0	0.3	2	256*384 (3.0T) 244*288 (1.5T)

NEX, number of excitations; TE, echo time; TR, repetition time.

Region-of-interest segmentation and radiomics feature extraction

Manual segmentation for each of the 124 RB tumours was performed by two radiologists (radiologist one and radiologist 2, with 3 years and 15 years of experience, respectively, in reading head and neck images). The segmented region of interest (ROI) covered the whole intraocular tumour not including optic nerve and was delineated by the radiologists on both the axial T2-WI and CET1-WI on each slice. For each MR sequence, 1029 radiomics features were extracted on the Radcloud platform, which is a useful tool for extracting radiomics features with a large panel of engineered hard-coded feature algorithms (Huiying Medical Technology Co., Ltd. http://mics.radcloud.cn/).^24,25 The 1029 obtained features can be divided into four main categories: first order, shape feature, texture feature and higher-order statistical features. First-order statistics such as the mean, standard deviation, variance, maximum, median and range describe the intensity information in the MRI region of interest. Shape features such as volume, surface area, compactness and maximum diameter reflect the shape and size of the region. Texture features can quantify the regional differences in heterogeneity. The following higher order statistical features included the first-order statistics and texture features derived from wavelet transformation of the original images: exponential, square, square root, logarithm and wavelet (wavelet-LHL, wavelet-LHH, wavelet-HLL, wavelet-LLH, wavelet-HLH, wavelet-HHH, wavelet-HHL and wavelet-LLL). These features complied with the definitions produced by the Imaging Biomarker Standardization Initiative. We used the Radcloud platform to manage the imaging data and perform subsequent radiomics statistical analyses.

Different medical imaging factors cause inconsistencies in the image intensity information of tissues of the same nature. We used the formula for intensity normalization (Supplementary Material 1).

Inter- and intra-observer reproducibility evaluation

Interobserver and intra-observer reproducibility of ROI detection and radiomics feature extraction was initially determined by two radiologists using the T2-WI and CET1-WI data of 20 patients chosen by computer-generated random numbers. To assess intra-observer reproducibility, radiologist one repeated the generation of radiomics features twice within a 1-month period following the same procedure.

Feature selection and model construction

Feature selection and model construction were only performed on the training set, and the validation set was only used to evaluate the model performance. We applied the Kruskal–Wallis (K-W) test to compare the distributions of feature values across three different MR scanners, and features with p < 0.05 were removed to eliminate invalid data caused by different MR scanners. Then, the least absolute shrinkage and selection operator (LASSO) logistic regression algorithm was used to reduce the dimensionality of extracted features on the training set. The initial α was set to 0.1, and the tuning parameter λ was set to zero as the default. Finally, the recursive feature elimination (RFE) with the SVM (kernel: linear) estimator was performed as a multivariate analysis method to select the PLONI-related features. RFE constantly eliminates unimportant features to obtain the optimal feature set by calculating the importance of features. Logistic regression (LR) classifier analysis was conducted to develop a model for the prediction of PLONI in the training set. The performance of the radiomics model was then internally tested in an independent validation set with the formula derived from the training set. The radiomics workflow is shown in Figure 2.

Figure 2.

The radiomics workflow.

Radiologist assessment

In this study, PLONI was defined as abnormal enhancement of the optic nerve in continuity with the tumor on postcontrast MR images, regardless of whether the optic nerve thickened or not. The same two radiologists who were blinded to the pathologic findings classified the 124 patients into PLONI-positive or PLONI-negative groups based on the imaging interpretations determined by consensus. The performance of radiologists’ assessment was compared with that of the radiomics model with regard to the prediction of PLONI in RB patients.

Statistical analysis

Univariate and bivariate analyses were performed with SPSS v. 20 (SPSS Inc.). Statistical analyses were performed for both the training and validation sets. Student’s t-test and the chi-square test were used to assess differences in age and sex distributions between the training and validation sets. Dice’s coefficients were used to evaluate the intra- and interobserver consistency for the ROI segmentation and radiomics feature extraction with 20 randomly selected samples. We interpreted a coefficient of 0.81 to 1.00 as almost perfect agreement, 0.61 to 0.80 as substantial agreement, 0.41 to 0.60 as moderate agreement, 0.21 to 0.40 as fair agreement, and 0 to 0.20 as poor or no agreement. Receiver operating characteristic (ROC) analysis and the area under the curve (AUC) were used to assess the predictive performance of the radiomics model and visual assessment for PLONI. The cutoff values of the ROC curves were determined by the principle of maximizing the Youden index. The AUC of the radiomics model was compared with that of visual assessment by the DeLong test. p < 0.05 was considered to indicate a statistically significant difference.

Results

Clinical information in data analysis

The training set included 90 patients who were recruited between January 2008 and June 2015, and the validation set included 34 patients who were recruited between July 2015 and January 2018. There were no significant differences between the two groups in gender and age in either the training set or validation set (p > 0.05) (Table 2).

Table 2.

Gender and age distribution of training and validation set

	Training set			Validation set
	With PLONI (n = 37)	Without PLONI (n = 53)	P value	With PLONI (n = 17)	Without PLONI (n = 17)	P value
Gender
Male	21	36	0.279	11	10	0.724
Female	16	17		6	7
Age (mean ± SD, months)	30.56 ± 8.07	37.38 ± 22.69	0.376	32.81 ± 16.89	34.75 ± 19.10	0.620

Imaging feature selection and radiomics model development

Satisfactory inter-observer and intra-observer reproducibility was achieved for radiomics feature extraction. The mean Dice’s coefficients were greater than 0.81.

A total of 2058 features were extracted from MR images (1029 features from T2-WI and 1029 features from CET1-WI). First, the K-W test was performed to separate the relevant and redundant features, and 54 features from T2-WI and 136 features from CET1-WI remained. After applying the LASSO and RFE methods, we selected the three features from T2-WI and the six features from CET1-WI that were most strongly associated with PLONI in the training set (Table 3). To build the radiomics signature, the above nine features were selected for inclusion in the Rad-score prognostic model. Rad-scores for each patient in the training and validation sets are shown in Figure 3.

Table 3.

Nine selected MRI features for inclusion in the Rad-score prognostic model

Imaging technique	T2-w	CET1-w	T2-w + CET1-w
Imaging feature		wavelet-HLH_glrlm_GLNU	CE_wavelet-HLH_glrlm_GLNU
		original_shape_Max3D	CE_original_shape_ Max3D
		exponential_glszm_SAE	CE_exponential_glszm_SAE
		wavelet-HHH_ first order _Skewness	CE_wavelet-HHH_ first order _Skewness
		wavelet-LLH_ first order _Skewness	CE_wavelet-LLH_ first order _Skewness
		wavelet-LHL_glcm_IV	CE_wavelet-LHL_glcm_IV
	wavelet-HHL_ first order _Median		T2-w_wavelet-HHL_ first order _Median
	original_shape_MinAxis		T2-w_original_shape_MinAxis
	original_shape_SA		T2-w_original_shape_SA
intercept (non-feature)	−0.534	−0.498	−0.621

CET1-w, contrast-enhanced T1-weighted; GLNU, Gray Level Non-Uniformity; IV, Inverse Variance; Max3D, Maximum3DDiameter; MinAxis, Minor Axis; SA, Surface Area; SAE, Small Area Emphasis; T2-w, T2-weighted; coef, coefficients.

Figure 3.

Rad-score for each patient in the training (a) and validation cohort (b). The red bar with a Rad-score >0 indicated that the signature successfully classifies the RB patients with PLONI; the red bar with a Rad-score <0 indicated that the signature fails to the RB patients with PLONI. For the green bar, the contrary applied.

In the training set, the radiomics signature derived from CET1-WI yielded an AUC of 0.844 [95% confidence interval (CI): 0.611–0.926]. The radiomics signature from T2-WI yielded an AUC of 0.860 (95% CI: 0.781–0.940). In the validation set, the radiomics signature from CET1-WI yielded an AUC of 0.768 (95% CI: 0.544–0.904). The radiomics signature from T2-WI yielded an AUC of 0.830 (95% CI: 0.687–0.974). The radiomics signature derived from both CET1-WI and T2-WI yielded the highest AUC both in the training set and validation set: 0.928 (95% CI: 0.873–0.982) and 0.841 (95% CI: 0.707–0.974), respectively. A final radiomics model was developed based on the combination of CET1-WI and T2-WI features.

Performance comparison between the radiomics model and radiologist assessment

The overall performance of the radiomics model compared with that of the radiologist assessment is shown in Table 4. Radiomics model produced better sensitivity than radiologist assessment, achieving 87.7% [(81.1–43.2)/43.2] improved in the training set, 55.8% [(82.4–52.9)/52.9] improved in the validation set. In all patients, the radiomics signature from the combined CET1-WI and T2-WI features yielded an AUC of 0.897 (95% CI: 0.839–0.955). The AUC value for the radiologist assessment was 0.674 (95% CI: 0.576–0.773). In the ROC analysis, radiomics models both in the training set and validation reached satisfactory level of AUC (>0.80). In addition, the AUC values of the radiomics model were significantly higher than those of the radiologist assessment in the training set and in all 124 patients (both p < 0.001, DeLong test) (Figure 4), whereas radiomics model was slightly better than radiologist assessment in the validation set, but not statistically significant (AUC 0.841 vs 0.706, p = 0.184, DeLong test).

Table 4.

Overall performance of radiomics model compared with radiologist assessment in predicting PLONI

	Radiomics model			Radiologist assessment
	Primary set	Validation set	All cases	Primary set	Validation set	All cases
Sensitivity (%) (n)	81.1 (30/37)	82.4 (14/17)	81.5 (44/54)	43.2 (16/37)	52.9 (9/17)	46.3 (25/54)
Specificity (%) (n)	94.3 (50/53)	76.5 (13/17)	90.0 (63/70)	88.7 (47/53)	88.2 (15/17)	88.6 (62/70)
Accuracy (%) (n)	88.9 (80/90)	79.4 (27/34)	86.3 (107/124)	70.0 (63/90)	70.6 (24/34)	70.2 (87/124)
PPV (%) (n)	90.9 (30/33)	77.8 (14/18)	86.3 (44/51)	72.7 (16/22)	82.8 (9/11)	75.8 (25/33)
NPV (%) (n)	87.7 (50/57)	81.3 (13/16)	86.3 (63/73)	69.1 (47/68)	65.2 (15/23)	68.1 (62/91)
AUC	0.928	0.841	0.897	0.660	0.706	0.675

NPV, negative predictive value; PPV, positive predictive value.

Figure 4.

Receiver operating characteristic (ROC) curves for the prediction of PLONI in RB by the radiomics model and radiologist assessment in the training set (a), validation set (b), and all patients (c). The areas under the ROC curve (AUCs) for the radiomics model were 0.928 (a), 0.841 (b) and 0.897 (c), respectively. The AUCs for radiologist assessment were 0.660 (a), 0.706 (b) and 0.675 (c), respectively. The AUCs for the radiomics model were significantly higher than those for radiologist assessment in the training set and in all 124 patients (p < 0.001).

Moreover, all eight cases incorrectly classified as PLONI by radiologists were correctly predicted by the radiomics model. Among 28 false-negative cases classified by radiologists, 20 cases were correctly classified by the radiomics model. Two image examples are shown in Figures 5 and 6. A total of 10 false negative cases (7 cases in training set, 3 cases in validation set) were detected in radiomics model. Of these, there were 3 cases (2 cases in training set, 1 case in validation set) misclassified as negative PLONI by radiomics model but correctly identified by radiologist.

Figure 5.

Retinoblastoma in a 36-month-old male child without histopathological PLONI (1 day between MR imaging and enucleation). CET1-WI with fat suppression showed an irregular tumour in the left globe with focal enhancement at the nerve-globe junction (arrow). It was misclassified as a positive case by radiologist assessment and was correctly classified by the radiomics model.

Figure 6.

Retinoblastoma in a 45-month-old male child with pathologically confirmed PLONI (4 days between MR imaging and enucleation). Axial T2-WI (a) and axial CET1-WI with fat suppression (b) showed a large irregular tumour in the right globe. The lesion showed slight heterogeneous enhancement without obvious enhancement in the postlaminar optic nerve on CET1-WI. It was misclassified as a negative case by radiologist assessment and was correctly classified by the radiomics model.

Discussion

PLONI is considered to be one of the most important risk factors for metastasis and recurrence, which may alter treatment plans. Although histopathologic examination is the gold standard for identifying PLONI, MRI is the only method of detecting the presence of PLONI in patients undergoing eye-saving treatments.²⁶ Previous studies also investigated that some MRI features, such as the presence of optic nerve enhancement and tumor size, could be used to predict PLONI.^12,13 MR imaging was demonstrated to have acceptable diagnostic performance in detecting PLONI in patients with RB, but the sensitivity ranges from low to moderate with a wide range.^15,17,27,28 A recent meta-analysis by De Cho et al²⁹ reported the moderate sensitivity [61%; (95% CI, 46–74%)] for detecting PLONI by MRI, and demonstrated a moderate heterogeneity in both sensitivity (I² = 72.23%) and specificity (I² = 78.11%) among the studies. There was a moderate heterogeneity in the field of diagnostic test accuracy. In our study, the predictive performance of the radiologist assessment, which was based on the nerve enhancement parameter on postcontrast MRI, showed a relatively low sensitivity of 46%, which still was within the 95% confidence interval of the meta-analysis. The diagnostic performance of MR is acceptable for detecting PLONI, but there is still room for improvement in the detection sensitivity of PLONI. In this study, we developed an MRI-based radiomics model for the prediction of PLONI, which was proven to have much better predictive performance than the radiologist assessment (AUC values of 0.90 vs 0.68, sensitivity of 81.5 vs 46.3, respectively, for all 124 patients).

In this study, we found that all eight false-positive cases classified by radiologists were correctly classified by the radiomics model, and 20 (71%) of the 28 false negatives classified by radiologists were correctly classified by the radiomics model. The sensitivity and negative-predictive value (NPV) of the radiomics model were much better than those of the radiologist assessment for the prediction of PLONI, especially for the sensitivity (81.1% vs  43.2% in the training set, 82.4vs 52.9% in the validation set). The radiomics model improved the sensitivity of detecting PLONI (by 76 percentage points) while maintaining a high specificity (90.0% in radiomics model vs 88.6% in radiologist assessment). At the first step of feature selection, K-W test was performed to remove redundant features. Only 54 features from T2-WI and 136 features from CET1-WI were remained to ensure the universality of the radiomics model. Removing many features could lead to useful information removed for detection PLONI as well. This may lead to limited improvement in the specificity. But the specificity for both radiomics model and radiologists’ assessment was already enough good. The radiomics model accurately excludes PLONI, as reflected in the high NPV. In the current era of increasingly conservative treatment strategies for diseases, including advanced-stage retinoblastoma, ophthalmologists’ confidence in the use of conservative treatment strategies would be improved by this higher NPV. The higher sensitivity can help inform the choices made by patients’ families, and the option of primary enucleation can be offered to avoid delaying cancer treatment.⁹ A choice of many treatments is available to ophthalmologists, and several factors play a role in determining the initial treatment strategy for this cancer. We believe that this MRI-based radiomics model has the potential to be used by radiologists to improve their diagnostic performance with regard to predicting PLONI and patient management.

The radiomics signature derived from CET1-WI and T2-WI showed promising predictive performance for the detection of PLONI in both the training set (AUC of 0.93) and the validation set (AUC of 0.84). Nine descriptive radiomics features were enrolled in the model, some of which were also found by other authors to be useful in the evaluation of head and neck lesions, such as skewness and Max3D.^30–32 Most features selected for inclusion in the radiomics model were higher order statistics features and filtered wavelet features. These features extracted from intraocular tumour itself, not including optic nerve. One previous radiomics study³³ on ovarian cancer found that the skewness was significantly greater in the malignant group, signifying that more heterogeneity and clustering were associated with the development of cancer. Positive skewness was also reported to be significantly associated with more distant metastases in lung cancer.^34,35 According to the theory of radiomics, the features we extracted can reflect the spatial heterogeneity, tumour and gene expression of tumours.¹⁸ Intra-tumoural heterogeneity is associated with tumour aggressiveness, treatment response, and prognosis.^36,37 Winter et al³⁸ found intratumour heterogeneity in patients with RB, including gains in MYCN, may be determinants for the evolution of more aggressive tumours as shown in their studied. Studies by Ramirez-Ortiz³⁹ have also suggested that the incidence of metastatic disease for RB may be associated with biological heterogeneity. Thus, we speculated radiomics features extracted from intraocular tumour reflected the heterogeneity of RBs and associated with the invasion to the optic nerve.

Our study had several limitations. First, the number of patients was small relative to the high dimensionality of the radiomics features. This is because in the current era of the increasing use of conservative eye-sparing treatment strategies for RB, increasingly fewer primary enucleations of the affected eye are being performed in RB patients. Second, high-resolution MR imaging was not used due to the retrospective nature of the study. However, the sensitivity, specificity, and accuracy of our study involving visual assessment were similar to those of a study using high-resolution MRI¹² (sensitivity, specificity, accuracy: 0.46, 0.89, 0.70 vs 0.42, 0.85, 0.60). Third, an external validation of the model should be applied in the further study to improve the reliability and robustness of result, due to all the patients from a single center in this study. More clinical information, such as ICRB and a prolonged duration of symptoms, was not used in the data analysis. Finally, this was a retrospective study, and the utility of the model needs to be investigated in a prospective study.

In conclusion, this MRI-based radiomics model is superior to radiologist assessment for the prediction of PLONI in RB patients and can potentially help oncologists improve patient management.