The quality and clinical translation of radiomics studies based on MRI for predicting Ki-67 levels in patients with breast cancer

Abstract

Objective:

To evaluate the methodological quality of radiomics literature predicting Ki-67 levels based on MRI in patients with breast cancer (BC) and to propose suggestions for clinical translation.

Methods:

In this review, we searched PubMed, Embase, and Web of Science for studies published on radiomics in patients with BC. We evaluated the methodological quality of the studies using the Radiomics Quality Score (RQS). The Cochrane Collaboration’s software (RevMan 5.4), Meta-DiSc (v. 1.4) and IBM SPSS (v. 26.0) were used for all statistical analyses.

Results:

Eighteen studies met our inclusion criteria, and the average RQS was 10.17 (standard deviation [SD]: 3.54). None of these studies incorporated any of the following items: a phantom study on all scanners, cut-off analyses, prospective study, cost-effectiveness analysis, or open science and data. In the meta-analysis, it showed apparent diffusion coefficient (ADC) played a better role to predict Ki-67 level than dynamic contrast-enhanced (DCE) MRI in the radiomics, with the pooled area under the curve (AUC) of 0.969.

Conclusion:

Ki-67 index is a common tumor biomarker with high clinical value. Radiomics is an ever-growing quantitative data-mining method helping predict tumor biomarkers from medical images. However, the quality of the reviewed studies evaluated by the RQS was not so satisfactory and there are ample opportunities for improvement. Open science and data, external validation, phantom study, publicly open radiomics database and standardization in the radiomics practice are what researchers should pay more attention to in the future.

Advances in knowledge:

The RQS tool considered the radiomics used to predict the Ki-67 level was of poor quality. ADC performed better than DCE in radiomic prediction. We propose some measures to facilitate the clinical translation of radiomics.

Introduction

Surpassing lung cancer, breast cancer (BC) in females has become the most common cancer, according to global cancer statistics in 2020. ¹ Ki-67, the cell proliferation index, plays an important role in the molecular subtype classification and subsequent therapy selection in BC. Luminal A and B are two main types of BC and are basically differentiated by the level of Ki-67 and hormone receptor expression. Luminal B lesions tend to have a higher Ki-67 level, while Luminal A lesions are with a lower level. ² The cut-off value for the Ki-67 level is contentiously debated. As proposed by the St Gallen Consensus Conference, the cut-off value has changed three times: most recently in 2015, ³ it was changed to 20–29% from 20% in 2013 ⁴ and before that, it was 14% in 2011. ⁵ BC with a high Ki-67 index is more likely to be heterogeneous and aggressive ^6,7 and have a higher risk of recurrence, ⁸ and its therapy differs from the one with a lower Ki-67 index. A higher Ki-67 index was a significant indicator of better efficacy of chemotherapy for BC, ⁹ and a predictor for pathological complete response (pCR) before neoadjuvant chemotherapy. ¹⁰ In contrast, patients classified as Luminal A are more sensitive to endocrine therapy. Therefore, it is crucial to measure the Ki-67 index before selecting a specific therapy for patients with BC.

However, the Ki-67 index must be determined by biopsy and histopathological staining which requires both a long wait time and high cost. More importantly, the index is measured in a small sample taken from the tumor which may not represent the status of the entire tumor, as the proliferative activity may vary in different regions of the same tumor. ¹¹ Radiology is considered a promising solution because it has the advantage of reflecting the whole tumor. ¹² As an adjunct to mammography, MRI can yield a higher detection rate in females at high risk for BC. ¹³ Different sequence in MRI contains its own special imaging mechanism and predictive power, which is beneficial for radiomics data mining. Radiomics rarely depends on the experience level of the radiologist and it can objectively extract more imaging information to reveal tumor biomarkers, help direct a particular treatment, track response to treatment, discriminate the occurrence of relapse and predict prognosis. ^14–17 In addition, radiomics is expected to show the best sites for biopsy, thus reducing the error rate in pathology. ¹⁴

While the field of radiomics is growing rapidly and numerous studies are being continually published, there are obstacles to clinical translation because the current quality of the medical literature is low and some study results have been difficult to reproduce. ^18,19 Therefore, the radiomics quality score (RQS) was proposed as a tool for evaluating the radiomics literature, and was aimed to make radiomics more reliable in data selection, medical imaging, feature extraction, exploratory analysis, and modeling set. ²⁰

To the best of our knowledge, no systematic review has been conducted to assess the quality of radiomics literature based on MRI for predicting the Ki-67 level in patients with BC. Ki-67 is a promising index for clinical applications and radiomic modeling because it is expressed in almost all tumors and is strongly correlated with tumor size, ^21–24 which means that radiomic models may be applicable to many other tumors, not only BC. Therefore, in our review article, we aimed to use RQS to systematically evaluate the methodological quality of the literature, and point out implications and limitations as well as offer suggestions for future research.

Methods and material

We conducted and reported this systematic review based on the prespecified criteria outlined by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Supplementary Table 1.). The study protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO, registry number CRD42022333994). No formal ethical approval was required for this study because this is a systematic review article.

Search strategy

Two investigators independently searched PubMed, Embase, and Web of Science. Only studies published in English were included. The coverage dates for this review began from each database’s inception and ended on May 8, 2022. Keywords “breast cancer”, “radiomics”, “Ki-67”, and “MRI” were used in the database searches. After removing duplicates, the titles and abstracts of studies were screened for eligibility by two investigators. Any disagreements were adjudicated by a third investigator to reach a consensus. All studies deemed eligible by title and abstract screening underwent a full-text review by two independent investigators using the same criteria. Studies were eligible for inclusion if: (1) patients who were diagnosed with BC had undergone a biopsy for Ki-67 level and MRI before treatment and (2) radiomics was used to estimate the Ki-67 level in patients with BC. Studies comprising animal subjects, articles such as letters to the editor, reviews, and studies lacking data were excluded. We also manually screened the reference lists of all included studies and key journals in the field to include all potentially eligible studies.

Data extraction

We extracted items for the characteristics of the included studies such as first author, year of publication, study design, MRI magnet strength, MRI manufacturers, MRI sequence, region of interest (ROI) segmentation methods, feature process software, sample size, sensitivity, specificity and area under the curve (AUC) of models. The summary tables were thoroughly and independently reviewed by two investigators. Study authors were contacted for additional information or missing data if necessary.

Methodological quality and risk of bias assessment

Two investigators assessed the methodological quality of included studies by using the RQS, including 16 items with a total score of 36, and more scoring details can be seen in Table 1. And, we used the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) to evaluate the bias of diagnostic studies from four key aspects: the selection of cases, the implementation or interpretation of the test, the implementation and interpretation of the gold-standard, and the flow and timing of cases. The risk of bias was assigned to one of three categories: “low risk”, “high risk” or “unclear risk”, and if the answer to all informational questions was “yes”, then the risk of bias was assessed as low; if one answer to all questions was “no”, then there was a possibility of bias. The Cochrane Collaboration’s software (RevMan 5.4) was used to complete this evaluation process and generate corresponding figures. Two investigators performed the assessments independently, and conflicts regarding scoring were resolved through discussion with an expert.

Table 1.

The RQS ^a

	Criteria	Points
1	Image protocol quality—well-documented image protocols (e.g. contrast, slice thickness, energy, etc.) and/or usage of public image protocols allow reproducibility/replicability	+1 (if protocols are well-documented) +1 (if public protocol is used)
2	Multiple segmentations—possible actions are: segmentation by different physicians/algorithms/software, perturbing segmentations by (random) noise, segmentation at different breathing cycles. Analyse feature robustness to segmentation variabilities	+1
3	Phantom study on all scanners—detect interscanner differences and vendor-dependent features. Analyze feature robustness to these sources of variability	+1
4	Imaging at multiple time points—collect images of individuals at additional time points. Analyze feature robustness to temporal variabilities (e.g. organ movement, organ expansion/ shrinkage)	+1
5	Feature reduction or adjustment for multiple testing—decreases the risk of overfitting. Overfitting is inevitable if the number of features exceeds the number of samples. Consider feature robustness when selecting features	−3 (if neither measure is implemented) +3 (if either measure is implemented)
6	Multivariable analysis with non-radiomics features (e.g. EGFR mutation)—is expected to provide a more holistic model. Permits correlating/inferencing between radiomics and non-radiomics features	+1
7	Detect and discuss biological correlates—demonstration of phenotypic differences (possibly associated with underlying gene–protein expression patterns) deepens understanding of radiomics and biology	+1
8	Cut-off analyses—determine risk groups by either the median, a previously published cut-off or report a continuous risk variable. Reduces the risk of reporting overly optimistic results Discrimination	+1
9	Discrimination statistics—report discrimination statistics (e.g. C-statistic, ROC curve, AUC) and their statistical significance (e.g. p-values, confidence intervals). One can also apply resampling method (e.g. bootstrapping, cross-validation)	+1 (if a discrimination statistic and its statistical significance are reported) +1 (if a resampling method technique is also applied)
10	Calibration statistics—report calibration statistics (for example, Calibration-in-the-large/slope, calibration plots) and their statistical significance (e.g. p-values, confidence intervals). One can also apply resampling method (e.g. bootstrapping, cross-validation)	+1 (if a calibration statistic and its statistical significance are reported) +1 (if a resampling method technique is also applied)
11	Prospective study registered in a trial database—provides the highest level of evidence supporting the clinical validity and usefulness of the radiomics biomarker	+7 (for prospective validation of a radiomics signature in an appropriate trial)
12	Validation—the validation is performed without retraining and without adaptation of the cut-off value, provides crucial information with regard to credible clinical performance	−5 (if validation is missing) +2 (if validation is based on a data set from the same institute) +3 (if validation is based on a data set from another institute) +4 (if validation is based on two data sets from two distinct institutes) +4 (if the study validates a previously published signature) +5 (if validation is based on three or more data sets from distinct institutes) *Data sets should be of comparable size and should have at least 10 events per model feature
13	Comparison to ‘gold-standard’—assess the extent to which the model agrees with/is superior to the current ‘gold-standard’ method (e.g. TNM-staging for survival prediction). This comparison shows the added value of radiomics	+2
14	Potential clinical utility—report on the current and potential application of the model in a clinical setting (e.g. decision curve analysis).	+2
15	Cost-effectiveness analysis—report on the cost-effectiveness of the clinical application (e.g. QALYs generated)	+1
16	Open science and data—make code and data publicly available. Open science facilitates knowledge transfer and reproducibility of the study	+1 (if scans are open source) +1 (if region of interest segmentations are open source) +1 (if code is open source) +1 (if radiomics features are calculated on a set of representative ROIs and the calculated features and representative ROIs are open source
	Total points (36 = 100%)

AUC, area under the curve; EGFR, epidermal growth factor receptor;ROI, region of interest; RQS, radiomics quality score.

^aThe RQS was described in earlier work. ²⁰

Statistical analyses

Mean and variance was used for statistical description of RQS. We collected the sample size, sensitivity and specificity of each model based on ADC and DCE, and studies without sufficient data were excluded from data synthesis. Specific numbers of true-positive (TP), false-positive (FP), true-negative (TN) and false-negative (FN) were used to derive the pooled AUC value. The Cochrane Collaboration’s software (RevMan 5.4), Meta-DiSc (v. 1.4) and IBM SPSS (v. 26.0) were used for all statistical analyses.

Results

Search results

After removing duplicates, 56 unique articles were identified from electronic databases and then they were screened by reading abstracts. Finally, 18 studies fulfilled our inclusion criteria (Figure 1).

Figure 1.

The flow diagram of the study selection process.

Study characteristics

The characteristics and main results of the studies included are summarized in Table 2. These studies were published between 2017 and 2022, and their sample sizes varied from 27 to 922. In total, 4019 participants were included.

Table 2.

Characteristics of the 18 studies included

Author	Year	Study design	Number of patients	AUC		AUC if no grouping
				Training set	Validation set
Fan	2019	Retrospective	144	NA		S50+ADC and Multitask: 0.821 ± 0.074
Fan	2020	Retrospective	322	NA		original: 0.755 bicubic: 0.770 SRGAN: 0.801 EDSR: 0.818
Kayadibi	2021	Retrospective	154	14%: ADC: 0.785 DCEI: 0.696 BOTH: 0.755 20%: ADC: 0.744 DCE: 0.629 BOTH: 0.761	14%: ADC: 0.849 DCEI: 0.695 BOTH: 0.635 20%: ADC: 0.617 DCE: 0.741 BOTH: 0.618	NA
Liang	2018	Retrospective	318	T 2WI: 0.762	T 2WI: 0.74	NA
Liu	2020	Retrospective	328	T 2WI: 0.727 T1+C: 0.873 DWI: 0.674 multiparametric MRI: 0.888	T 2WI: 0.706 T1+C: 0.829 DWI: 0.643 multiparametric MRI: 0.875	NA
Ma	2018	Retrospective	377	NA		Naive Bayes: 0.773 SVM: 0.665 kNN: 0.489
Zhang	2019	Retrospective	128	0.75 ± 0.08	0.72	NA
Castaldo	2022	Retrospective	32	NA		LDA: 0.81 Logit Boost: 0.70 RF: 0.76
He	2020	Retrospective	88	NA		The first post-contrast frame: the AUC values among various regions of tumor and peritumoral stroma were all higher than 0.6
Holli-Helenius	2017	Retrospective	27	NA		Pre-contrast images: Sum entropy: 0.827 ± 0.065 Sum variance: 0.832 ± 0.076 Sum entropy+Sum variance: 0.876 ± 0.077
Jiang	2021	Retrospective	209	DCE: 0.825 DWI: 0.832 Both: 0.863	DCE: 0.816 DWI: 0.795 Both: 0.838	NA
Li	2021	Retrospective	351	Intra RS: 0.845 Peri RS: 0.835 Com RS: 0.875	Intra RS: 0.714 Peri RS: 0.692 Com RS: 0.749	NA
Monti	2018	Retrospective	49	NA		0.811 ± 0.005
Ni	2020	Retrospective	112	NA		Histogram: 0.926 (0.870–0.981) Texture: 0.718 (0.615–0.820) GLCM: 0.975 (0.949–1.000) RLM: 0.946 (0.905–0.988)
Saha	2018	Retrospective	922	NA	0.624	NA
Zhou	2020	Retrospective	126	Ktrans: 0.67 Kep: 0.69 Ve: 0.72 Vp: 0.74	NA	NA
Umutlu	2021	Retrospective	124	NA		All MRI and PET: 0.997
Zhang	2021	Retrospective	208	NA		The feature of Dependence Non-uniformity Normalized obtained the highest AUC of 0.8923 for Ki-67; The 3D tumor subregion related to FFK yielded the highest AUCs of 0.8080 for Ki-67

ADC, apparent diffusion coefficient; AUC, area under the curve; Com RS, combined radiomics signatures; DCE-MRI, dynamic contrast-enhanced magnetic resonance imaging; DWI, diffusion-weighted imaging; EDSR, enhanced deep super resolution; FFK, fast-flow kinetics; GLCM, gray level co-occurrence matrix; Intra RS, intratumoral radiomics signatures; Kep, flux rate constant; Ki-67, cell proliferation index; Ktrans, volume transfer constant; LDA, linear discriminant analysis; PET, positron emission tomography; Peri RS, peritumoral radiomics signatures; Pk-DCE MRI, pharmacokinetic dynamic contrast-enhanced magnetic resonance imaging; RF, random forest decision trees; RLM, run length matrix; ROI, region of interest; S50, the last time period of the DCE-MRI series; SRGAN, super-resolution generative adversarial network; SVM, support vector machine; T1+C, contrast-enhanced T1-weighted imaging; T ₂WI, T ₂ weighted imaging; Ve, extravascular extracellular volume fraction; Vp, plasma volume fraction; kNN, k-nearest neighbor.

Risk of bias assessment

QUADAS-2 was used in our risk of bias assessment. The results showed that most studies had a moderate risk of bias (Figure 2). The unclear risk was mainly in the index test, including the bias and applicability (Figure 3).

Figure 2.

The risk of bias summary for each study.

Figure 3.

The risk of bias graph for the 18 included studies.

RQS distribution of the 18 studies

The RQS has 16 items. Its maximum possible total score is 36 (100% on a percentage scale). The mean value of RQS for all 18 included studies was 10.17 (28.25%, standard deviation [SD]: 3.54). As shown in Figure 4, no study scored more than half of the maximum possible for the RQS. The RQS range was 0–13, with two studies receiving the highest score. Three studies had RQS of less than 10, and all of them lacked feature reduction and then three points were deducted from each RQS. The study by He et al received a zero in our evaluation; it had the lowest RQS.

Figure 4.

Calculation of the RQS for the 18 included studies. Maximum possible RQS is 36. Shading in the figure is as follows: grey indicates where points were deducted, light green indicates zero points, dark green indicates not fully scored items and orange indicates fully scored. RQS, radiomics quality score.

In our review of the 18 studies, no study received any score for each of these five items of the RQS: a phantom study on all scanners, cut-off analyses, prospective design, cost-effectiveness analysis, and open science and data. The sum of possible scores for these four items is 14 points (14 of the maximum possible RQS of 36, 38.89%). In contrast, all studies received a score on three items: detect and discuss biological correlates, discrimination statistics, and comparison to the gold standard. 4 of 18 (22.22%) studies lost three points on feature reduction. 1 of 18 (5.56%) studies lost five points on validation. The other 17 studies all used internal data set validation and 8 of them adopted cross-validation other than independent validation. The total possible points for validation are five when three or more external data sets are compared. But none of the studies performed external validation. Of note, there was only one study (Jiang et al) that received points for the potential clinical utility item, for its usage of decision curve analysis (DCA) and nomogram to advance the development of clinical applications. Moreover, it was also the only study using calibration curves of the nomogram to show the model-predicted and actual probabilities, so this study also received points for the calibration statistics item.

Predictive power of different MRI sequence

In the 18 studies, apparent diffusion coefficient (ADC), diffusion-weighted imaging (DWI), T ₂ weighted imaging (T ₂WI), dynamic contrast-enhanced MRI (DCE-MRI), and pharmacokinetic DCE (Pk-DCE) were sequences used for ROI segmentation, feature selection, and then for quantitative analysis. Our study compared the predictive power of ADC with DCE in radiomics prediction for the Ki-67 level (cut-off value: 14%). Six studies were included in the meta-analysis and eight data sets were pooled. As shown in Table 3, ADC showed a higher predictive power with a pooled AUC of 0.969 compared to 0.944 for DCE. However, the analysis was highly heterogeneous with the Spearman’s rank correlation rho estimates of 0.8, therefore, SROC (summary ROC) curve was used. Figure 5 was the SROC curve and forest plot.

Table 3.

ADC vs DCE in radiomics predictive power for the Ki-67 level

MRI sequence	Sensitivity	Specificity	AUC of sROC
ADC	0.801	0.844	0.969
DCE	0.802	0.819	0.944

ADC, apparent diffusion coefficient; AUC, area under the curve; DCE, dynamic contrast-enhanced; sROC, summary ROC.

Figure 5.

The SROC curve and forest plot. ADC, apparent diffusion coefficient; DCE, dynamic contrast-enhanced; SROC, summary ROC.

Discussion

Using the RQS as an evaluation tool, this review provides a systematic analysis of the methodological quality of 18 studies that used radiomics based on MRI to predict the Ki-67 level in a total of more than 4000 BC patients. While the results of this review support the efficacy of the booming field of radiomics in predicting the Ki-67 level, the studies we included were limited by the poor methodological quality.

MRI has the advantage of multisequence imaging, and plays an important role in breast imaging diagnosis. Some findings have shown that ADC performed well in predicting BC biomarkers, especially the Ki-67 index. ^25,26 In radiomics, researchers held disagreements about which sequence is of higher AUC. ^27–31 Our review compared the summary AUC of models based on DCE and ADC. While DCE was used most commonly, ADC showed a better AUC. Of note, Kayadibi et al compared these two Ki-67 cut-off values (14% vs  20%) and observed a significant result showing that the radiomics model based on 14% produced a higher AUC (0.849 vs 0.614 in the testing set). Our evaluation of the 18 included studies showed that most studies (12/18) chose 14% as the cut-off value. However, as mentioned above, the cut-off value is contentiously debated and has been changed multiple times. A long-term follow-up study showed that 20% was optimal for predicting the prognosis in luminal BC; this cut-off value can better distinguish the risk of recurrence and death than 14%. It is still an open question about which cut-off is more suitable for radiomics in BC. ³²

Improvement for radiomics quality is urgently needed

RQS, a tool promoted to quantify the quality of radiomics studies, has its unique benefits such as being a pioneering standard in the radiomics field, wide adaptability, and ease of management. Using 16 items, RQS can comprehensively evaluate whether a radiomics study is a good quality study or whether it has a deficiency. Lambin et al ²⁰ put more emphasis on these four items: a prospective study, feature selection, validation, and open science and data, suggesting that these four items might be more important for reflecting whether a study is of good quality or not. The prospective study design is recommended for the highest level of evidence, as well as for helping construct robust radiomics models and guaranteeing clinical validity. Disappointedly, none of the studies we evaluated were designed prospectively.

Four studies lost points on feature reduction. An included study by Saha et al ³³ serves as an example that extracted 529 DCE-MRI features to help compare the different AUCs obtained by previous radiomics studies which used small, limited, and various features. However, these investigators had to confront the overfitting consequence of compiling such a comprehensive set of features, and this could jeopardize the performance of the model. ²⁰ Finally, they got the AUCs of the models all below 0.7 which was an inferior result compared to other studies on this similar topic.

In our review, none of the 18 studies received any points for four items (phantom study on all scanners, cut-off analyses, cost-effectiveness analysis, and open science and data), indicating the insufficient attention and operation in the real-world practice of radiomics. A phantom study is designed to elevate the reproducibility and robustness of radiomic features because the imaging protocols for medical imaging scanners are not completely identical. However, previous reviews ^34–37 and our review found that no study published in the radiomics literature had carried out this test. A recent phantom study used the interscanner coefficient of variation (COV) and interscanner intraclass correlation coefficient (ICC) to assess the features extracted from two phantoms on eight scanners. ³⁸ Ray et al showed that first-order statistics were the most robust (having ICCs > 0.8) followed by kurtosis and skewness. Shape was the least robust metric, indicating that intervendor and interscanner sources of variability were variances to be addressed. And, we note that the RQS does not include an item about sample size, which is crucial for the validity of radiomics results. ³⁹ A sufficient number of patients facilitates the validation and, as we all know results obtained from a small number of patients are not so compelling, especially in radiomics which is based on mass data. Three of our included studies ^40–42 had sample sizes of less than 50 patients, which undoubtedly limited the interpretation of the radiomics results.

The information presented in Supplementary Table 3 shows great variability in the software used by different investigators for image segmentation and feature process (extraction and selection). It is also worthy of note to mention the intra- and inter-research heterogeneity in data sets derived from scan protocols, naming conventions of features, and modeling steps. ^14,43,44 In addition, it has been reported that different teams obtained different feature values even when using the same feature. ⁴³ The combination of these factors made it difficult when we tried to compare these studies and leave no opportunities for subsequent research studies to reproduce the original results. Taken together, this situation presents another problem underlying the current status of radiomics—standardization. Proposed by Zwanenburg et al. in 2020, ⁴³ using the Image Biomarker Standardization Initiative (IBSI) reference values can result in excellent reproducibility for features. However, in our review, we found that only three studies ^31,45,46 claimed to follow the IBSI reference when defining and calculating the features. Regarding feature extraction, semi- and fully automatic approaches are recommended because manual feature extraction can take a lot of time and add inter- and intra-observer variance. ⁴⁷ To improve the future of clinical translation, researchers should pay more attention to the use of uniform standards for software and algorithms when processing features.

No data in our reviewed studies were made publicly available. This situation was discouraging because openly sharing in-house data can greatly improve reproductivity and then the clinical translation can be achieved earlier. ⁴⁸ It was also reported that open science in radiomics can facilitate external validation, which is another key point in RQS. ⁴⁹ We believe that a large, publicly open and standardized radiomics database can provide tremendous power for clinical translation, where researchers can download data with little discrepancy in image acquisition and protocols. Furthermore, such a database can also be built by many institutions to meet the requirements of external validation more easily. ^49,50 The Cancer Imaging Archive (TCIA) (http://cancerimagingarchive.net/), an ever-growing archive of medical images consisting of a large quantity of clinical information, sets a good example for data sharing. ⁵¹ A study based on MRI has used the TCIA data set to predict molecular classifications of BC subtypes, ⁵² taking advantage of the multicenter data set validation. Moreover, future data sets are supposed to be based on the phantom study to lessen the impacts of the variance of intervendor and interscanner sources.

We still recommend that each radiomics study pre-score their study before submitting their work for publication and explain any incomplete items in the discussion. These measures will not only help improve their work but will also help editors and reviewers to objectively estimate the quality of the study.

There are several potential limitations of this review article. Primarily, we were limited by the small number of published studies in the literature. In addition, because of the large heterogeneity among them, we performed a meta-analysis of poor quality. Lastly, RQS itself has limitations. It is not a well-rounded tool, and it should be developed to keep pace with the rapid speed with which radiomics is improving.

Conclusion

Ki-67 is a common index to reflect the cell proliferation rate, whose radiomics model might be utilized in many other tumors. Many researchers have worked to predict it by radiomics, and our review indicated a promising result for the Ki-67 level prediction in patients with BC. However, according to the RQS, many aspects of these studies need improvement: the incorporation of a prospective study design, external validation, open science and data, and phantom analysis. The exploration of the most suitable MRI sequence for radiomics is warranted. Standardization of feature processing and establishment of a standard and open radiomics database might foster clinical translation.