Artificial intelligence-based models to assess the risk of malignancy on radiological imaging in patients with intraductal papillary mucinous neoplasm of the pancreas: scoping review

Introduction

Increased detection of pancreatic cystic neoplasms has drawn the attention of the medical community^1,2. Among these, intraductal papillary mucinous neoplasms (IPMNs) represent a serious challenge for clinicians because of their (low) premalignant potential. Despite extensive efforts, the treatment of IPMN remains controversial, which is reflected by differences in the current three major guidelines^1,3,4.

Most patients diagnosed with IPMN will be kept under surveillance, aimed at monitoring progression of the cyst, which may require surgical resection in highly selected patients. Still, the risk of clinicians missing IPMN progression to malignancy is concerning⁵, with burdensome consequences for the patient. This concern must be balanced against the risk of complications after major pancreatic surgery. Therefore, patient selection is crucial both to avoid unnecessary surgery for benign lesions, and to continue surveillance safely. Typically, diagnostic imaging plays a central role in guiding patient selection for, and the timing of, surgery. However, current imaging approaches fall short for optimal decision-making.

Machine learning assessment of radiological imaging may improve the assessment of IPMNs and add to the decision-making for surgery. This scoping review provides an overview of the available evidence on this topic.

Methods

Literature search

The Joanna Briggs Institute and PRISMA Extension for Scoping Reviews criteria were used for this scoping review^6,7. The review was performed in PubMed and Embase up to 16 December 2021. The search strategy is provided in Table S1. Two assessors worked independently on literature screening, evaluation of eligibility, and inclusion, with conflicts handled through discussion. The remaining literature was subjected to full-text analysis. Original studies on imaging-based machine learning models (Table S2) in IPMN, which reported on model performance in terms of malignancy assessment, were included.

Data extraction and analysis

Two reviewers extracted data independently. If consensus could not be established, disagreements were resolved by discussion. When these two reviewers could not reach agreement, a third independent assessor was involved. Data from the included studies were analysed descriptively. The primary outcome was the discriminatory ability of the models measured by the area under the curve (AUC), accuracy, C-index, and P value. Model performance of 0.75 or more was considered sufficient for research reporting on AUC, C-index, and accuracy.

Specific attention was paid to determining whether the discriminative models were compared with the reference standard of clinical care based on guidelines.

Methodological assessment

The methodological quality of the included studies was assessed using the modified Radiomics Quality Score (mRQS)⁸. Within the mRQS (Table S3), radiomics approaches can reach a total of 36 points on 16 aspects, and deep learning approaches can reach a total of 32 points on 14 aspects. Two independent assessors performed the methodological assessment. Discrepancies were resolved by discussion.

Results

Literature search

The literature search yielded 49 studies, of which 33 were excluded based on title and abstract screening, and four more studies after full-text screening. Eventually, 12 studies^9–20 fulfilled the eligibility criteria and were included in this scoping review (Fig. S1). Table 1 details the included studies and the data extracted.

Table 1.

Studies on imaging-based machine learning models assessing the malignant potential of intraductal papillary mucinous neoplasm

Reference	Country	Imaging	IPMN type	Target variable	Total n	Machine learning component	Data type	Outcome(s)
Hanania et al.9	USA	CT	BD and MD	Low-grade IPMN versus high-grade IPMN or PDAC	Total 52 (10-fold cross-validation)	PCA and LR	Radiomics	AUC 0.96
Permuth et al.10	USA	CT	BD, MD, and Mixed	Low-grade IPMN versus high-grade IPMN or PDAC	Total 38 (10-fold cross-validation)	LR and PCA	Radiomics	AUC 0.93
Attiyeh et al.11	USA	CT	BD	Low-grade IPMN versus high-grade IPMN or PDAC	Total 103 (10-fold cross-validation)	RF	Radiomics	AUC 0.76
Chakraborty et al.12	USA	CT	BD	Low-grade IPMN versus high-grade IPMN or PDAC	Total 103 (10-fold cross-validation)	RF and SVM	Radiomics	AUC 0.77
Kuwahara et al.19	Japan	EUS	Undefined	Low-grade IPMN versus high-grade IPMN or PDAC	Total 50 (10-fold cross-validation)	NN	Image	AUC 0.98
Corral et al.20	USA	MRI	BD and MD	Low-grade IPMN versus high-grade IPMN or PDAC	Total 139 (10-fold cross-validation)	NN	Image	AUC 0.78
Jeon et al.18	South Korea	MRI	BD, MD, and mixed	Low-grade IPMN versus high-grade IPMN or PDAC	248 (no validation)	LR	Radiomics	Entropy: OR 1.49–1.52 Compactness 2: OR 0.977–0.981
Harrington et al.13	USA	CT and EUS	BD and MD	Low-grade IPMN versus high-grade IPMN or PDAC	Training 103 Testing 33	RF	Radiomics	AUC 0.83
Polk et al.14	USA	CT	Undefined	Low-grade IPMN versus high-grade IPMN or PDAC	Total 51 (5-fold cross-validation)	LR	Radiomics	AUC 0.90
Tobaly et al.17	France	CT	BD, MD, and Mixed	Low grade IPMN versus high grade IPMN or PDAC	Training 296 Testing 112	LASSO and LR	Radiomics	AUC 0.84
Cui et al.16	China	MRI	BD	Low-grade IPMN versus high-grade IPMN or PDAC	Training 107 Testing 99	LASSO and LR	Radiomics	AUC 0.81–0.82
Cheng et al.15	China	CT and MRI	Undefined	Low-grade IPMN versus high-grade IPMN or PDAC	Total 60 (10-fold cross-validation)	LR and SVM	Radiomics	MRI: AUC 0.879–0.940 CT: AUC 0.811–0.864

IPMN, intraductal papillary mucinous neoplasm; BD, branch duct; MD, main duct; PDAC, pancreatic ductal adenocarcinoma; PCA, principle component analysis; LR, logistic regression; AUC, area under the curve; EUS, endoscopic ultrasound imaging; RF, random forest; SVM, support vector machine; NN, neural network; LASSO, least absolute shrinkage and selection operator.

Radiomics models

Of the 10 IPMN radiomics models, 9 were based on CT and 1 on MRI. All models aimed to distinguish low- or intermediate-grade dysplasia from high-grade dysplasia or IPMN with concomitant pancreatic ductal adenocarcinoma. The studies had AUCs ranging between 0.76 and 0.96^9–17; one¹⁸ showed that two features were independent variables for malignant IPMN with ORs ranging from 1.49 to 1.52 and 0.977 to 0.981.

Deep learning models

Of the two deep learning models, one was based on MRI and the other on endoscopic ultrasound imaging. The models aimed to distinguish low- or intermediate-grade dysplasia from high-grade dysplasia or IPMN with concomitant pancreatic ductal adenocarcinoma. These models reached AUCs of 0.78 and 0.98^19,20.

Comparison with reference standard and added value

Two of the included studies compared the developed model with available guidelines. Corral et al.²⁰ reported a similar diagnostic performance for MRI-based deep learning (AUC 0.78), and American Gastroenterology Association⁴ and Fukuoka (AUC 0.77)¹ guidelines. Conversely, the CT and MRI-based radiomics model of Cheng et al.¹⁵ had superior discriminative performance (MRI: AUC 0.94; CT: AUC 0.86) to that of the clinical and imaging model based on the Fukuoka guidelines (AUC 0.77)¹.

Methodological assessment

The median mRQS score for the radiomics studies was 11.5 of 36, ranging from 4 to 17. The deep learning studies had a median mRQS score of 9 of 32, ranging from 8 to 10. The included studies consistently scored poorly (25 per cent or fewer of the studies received all points) on 10 of 16 items in the mRQS: 2, 3, 4, 8, 10, 11, 12, 13, 15, and 16. Four items scored consistently positive (at least 75 per cent of the studies received all points): 5, 7, 9, and 14. Finally, two items scored moderately consistently (received all points in more than 25 per cent and less than 75 per cent of the studies): 1 and 6. Figure 1 summarizes the mRQS scoring.

Fig. 1.

Methodological assessment of included studies using modified Radiomics Quality Score

Green, all points given; orange, not all points given; red, no points or negative points given; –, not applicable to this study owing to data type.

Discussion

This scoping review identified 12 artificial intelligence-based models to assess the risk of malignancy in patients with IPMN on radiological imaging. Although model performance was generally promising, the methodological quality of the studies was relatively poor. Furthermore, none of the models were applied in a prospective clinical setting or determined the added value compared with current guidelines or a clinical expert panel. If methodologically robust models are developed, and evaluated in a prospective setting, they may have the potential to enhance decision-making in finding the best time for surgery in patients diagnosed with IPMN.

This review has several limitations. First, owing to publication bias, models that were not discriminative may not have been submitted or accepted for publication. The model performance presented in this review may therefore be optimistic. Second, all studies are based on retrospective surgical series. However, most IPMNs are addressed to surgery after surveillance if cyst progression is observed. The selection bias originating from including only patients with surgically resected tumours makes the value of these models unclear in the unresected population. Third, the methods of the included studies varied extensively. Therefore, extracting generalizable results from this overview is difficult.

Future research should concentrate on developing methodologically sound, generalizable, and clinically validated models. Multiple methodological elements are frequently missed or ignored, as is evident from the mRQS scores of the research included. Once robust and generalizable models have been constructed, their performance and value should be validated in clinical settings. Currently available studies have focused on assessing the discriminative performance of machine learning models for malignant IPMNs. However, ideally, models would exclude the presence of malignancy with a high negative predictive value and ‘safely’ advise surveillance in patients who would have been selected for surgical treatment according to current criteria. This would represent a true added value to current clinical practice.

This scoping review has provided evidence that 12 artificial intelligence-based machine learning models have sufficient capacity to evaluate the risk of malignancy in IPMN. However, the methodological quality of the included studies is inadequate, and the clinical value of the proposed models has not been proven. As a result, caution should be advised when interpreting these results, and the findings must be corroborated by additional high-quality studies. Future research should focus on developing rigorous models and investigating their usefulness in clinical practice to ensure that they are dependable tools for assessing the risk of malignancy in IPMN.

Funding

The authors have no funding to declare.

Author contributions

Alberto Balduzzi (Conceptualization, Methodology, Formal analysis, Investigation, Data Curation, Writing - Original Draft, Writing - Review & Editing, Visualization, Project administration), Boris Janssen (Conceptualization, Methodology, Formal analysis, Investigation, Data Curation, Writing - Original Draft, Writing - Review & Editing, Visualization, Project administration), Matteo De Pastena (Data Curation, Writing - Review & Editing), Tommaso Pollini (Data Curation, Writing - Review & Editing), Giovanni Marchegiani (Conceptualization, Writing - Review & Editing, Methodology, Supervision), Henk Marquering (Conceptualization, Writing - Review & Editing, Methodology, Supervision), Jaap Stoker (Conceptualization, Writing - Review & Editing, Methodology, Supervision), Inez Verpalen (Conceptualization, Writing - Review & Editing, Methodology, Supervision), Claudio Bassi (Conceptualization, Writing - Review & Editing, Methodology, Supervision), Marc Besselink (Conceptualization, Writing - Review & Editing, Methodology, Supervision), and Roberto Salvia (Conceptualization, Writing - Review & Editing, Methodology, Supervision).

Disclosure

The authors declare no conflict of interest.

Supplementary material

Supplementary material is available at BJS online.

Data availability

The raw data required to reproduce the scoping review findings were extracted from published articles.