Artificial Intelligence-assisted Diagnosis of Carotid Artery Calcifications on Panoramic Radiographs: A Meta-analysis

Abstract

Carotid artery calcifications (CACs), a known risk factor for stroke, can be detected on panoramic radiographs (PRs). However, this clinically significant pathophysiological condition has long been underdiagnosed due to insufficient training and expertise among dentists. Artificial intelligence (AI) may serve as a valuable tool to aid dentists in detecting CACs on PRs. This meta-analysis was conducted to assess the diagnostic accuracy of AI for CACs detection on PRs. A literature search was conducted on PubMed, Embase, Web of Science, Scopus and Cochrane Library up to 4 September 2025 without language limitation. The quality of studies was assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool. The performance of AI was assessed via the area under curve (AUC), sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR). Meta-analysis was conducted on Stata 14.0. This systematic review identified 8 relevant articles, 7 of which were eligible for meta-analysis. The analysis was conducted from two perspectives: per-side (evaluating the left and right sides of participants separately) and per-person. In the per-side analysis, the summary estimates indicated high diagnostic accuracy: sensitivity was 0.88 (95% CI: 0.84-0.92), specificity was 0.94 (95% CI: 0.91-0.97), the PLR was 15.8 (95% CI: 9.0-27.6), the NLR was 0.12 (95% CI: 0.08-0.18), the DOR was 129 (95% CI: 55-305), and AUC was 0.96. The per-person analysis yielded a pooled sensitivity of 0.90 (95% CI: 0.74-0.97) and specificity of 0.81 (95% CI: 0.73-0.87). The corresponding PLR was 4.6 (95% CI: 3.1-7.0), NLR was 0.12 (95% CI: 0.04-0.36), DOR was 37 (95% CI: 10-144), and the AUC was 0.86. These results indicate that AI may serve as a valuable tool to assist dentists in detecting CACs on PRs. However, large-scale, evidence-based studies are still needed to validate these findings.

Introduction

According to the World Health Organization (WHO), stroke is the world’s second most frequent cause of death, responsible for 11% of total deaths.¹ Stroke often leads to long-term disability and imposes a significant economic burden on patients, making it a critical concern for public health.² Atherosclerotic carotid plaques (ACPs) are recognised as a major etiological factor in stroke development.³ Their thickening and subsequent calcification can restrict blood flow, compromising oxygen delivery to the organs they supply.⁴ Consequently, early detection of carotid artery calcifications (CACs) plays a crucial role in stroke prevention.

Panoramic radiographs (PRs) are widely employed in initial examination in dental clinics, as they provide broad anatomical coverage, with minimal radiation exposure and low cost.^5,6 Beyond their primary diagnostic role, PR scans also incidentally detect CACs in approximately 3% to 15% of patients.⁷ Multiple studies have demonstrated the utility of PRs as a screening tool for detecting CACs.8, 9, 10, 11 However, identifying CACs on PRs from other calcified anatomical structures or pathological lesions remains challenging and highly subjective.¹² Even trained dental professionals often struggle to accurately identify CACs on PRs due to the need for substantial experience and expertise.¹³ As a result, conventional visual assessment of CACs on PRs has limited diagnostic accuracy.

With the rapid development of artificial intelligence (AI) technology, its application in medicine has attracted widespread attention. For instance, preliminary studies show that AI can match or even surpass the diagnostic accuracy of trained specialists in detecting lymph node metastases in tissue sections,¹⁴ melanomas in clinical photographs,¹⁵ and 14 common thoracic diseases in chest radiographs.¹⁶ AI is now widely integrated into dentistry,^17,18 including detecting dental caries,¹⁹ precisely segmenting anatomical structures like the mandibular nerve canal,²⁰ predicting an individual's sex from PRs,²¹ and identifying past dental treatments.²² Furthermore, AI has demonstrated considerable potential in identifying systemic health markers incidentally captured on dental radiographs, such as CACs on panoramic images. Sawagashira et al²³ introduced a machine learning (ML) classifier aimed at differentiating CACs or normal PRs, achieving a sensitivity of 0.936. In the study of Yoo et al,²⁴ a deep learning (DL) framework (CACSNet with EfficientNet-B4) achieved an exceptional accuracy of 0.985 in distinguishing PRs with CAC or normal. Similarly, Amitay et al²⁵ reported comparable high performance, attaining an accuracy of 0.940 using the InceptionResNetV2 architecture for the same classification task.

Despite the integration of AI into radiological imaging showing great potential in the diagnosis of CACs using PRs, there is still limited consensus on their reliability in real clinical use. Thus, the current study aims to systematically synthesise the current research evaluating the performance of AI models on CACs detection using PRs, covering all pertinent studies.

Methods

Eligibility criteria

The review question was structured using the PICO framework adapted for AI diagnostic studies:

• •Population (P): PRs.
• •AI Intervention (I): Any AI model, including ML or DL algorithms, designed for the automatic detection or classification of CACs on PRs. This includes models performing image-level classification, object detection, or semantic segmentation.
• •Comparator (C): A reference standard for CACs established by human clinicians.
• •Outcome (O): Diagnostic performance metrics, including but not limited to sensitivity, specificity, AUC, and related measures (PPV, NPV, DOR).

Search strategy

We conducted a search in five electronic databases (PubMed, Embase, Web of Science, Scopus and Cochrane Library) without language restriction, from inception to 4 September 2025. The search was performed in combination with the following terms: ‘artificial intelligence’ or ‘deep learning’ or ‘machine learning’ or ‘computer intelligence’ or ‘image processing’ or ‘neural network’ and ‘panoramic radiography’ or ‘radiographic image’ or ‘radiograph’ and ‘carotid artery’ or ‘carotid’ and ‘calcified’ or ‘calcium’ or ‘calcification’ or ‘plaque’ or ‘atherosclerosis’ or ‘atheroma’.

Study selection

Entries were managed using EndNote (version 9; Clarivate Analytics). After removing the duplicates, two independent reviewers (FFY and QZ) browsed the titles, abstracts, and full texts of the records. Any disagreement was resolved by consulting a third researcher (SYZ). In addition, the reference lists of the included studies were reviewed to identify any additional relevant articles.

Data collection

Two investigators (FFY and QZ) collected the data of each study independently. The information is as follows: authors, publication year, country, dataset size, AI platform, AI task type, reference standard for CACs, and metric results, such as accuracy, precision, sensitivity/recall, specificity, and AUC. Any discrimination would be discussed with a third investigator (SYZ).

Risk of bias assessment

Two investigators (FFY and QZ) assessed the risk of bias of the eligible studies using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool,²⁶ and disagreements were resolved by discussing with a third investigator (SYZ). The tool includes four domains focused on patient selection, index test, reference standard, and flow and timing, along with applicability concerns related to patient selection, index test, and reference standard. Risk of bias was recognised as ‘low’, ‘unclear’, or ‘high’ in each domain.

Data synthesis

Studies that provide sufficient raw data to calculate true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) were included for meta-analysis. All meta-analyses were conducted within the bivariate mixed-effects modelling framework, which is automatically implemented as the default analytical method in our statistical software (Stata's midas command)(Stata 14.0). Forest plots were used to display the pooled estimates and 95% confidence intervals (CI) for the sensitivity and specificity of AI in CAC detection. Heterogeneity across the studies was assessed using the Cochran-Q test and I² statistics, and I² > 50% and/or P < .05 was considered high heterogeneity. The Deeks test was conducted to assess the publication bias. Sensitivity analysis was done to evaluate the robustness of the results by removing eligible studies by sequence and recalculating the pooled results.

AI task type clarification

In this review, the included AI models primarily performed two types of tasks: classification and detection. Classification aims to determine whether the entire panoramic image contains CACs. Detection further localises calcified regions, typically outputting bounding boxes. Although the task types vary, all included studies share the core objective of identifying CACs, thus allowing their combined inclusion in the meta-analysis of diagnostic performance.

Selection of AI model performance data

If a study reported independent performance metrics for multiple models, each model was treated as a separate ‘study’ in the pooled analysis to reflect the full range of available techniques and their performance. For example, Amitay et al ²⁵ evaluated 4 distinct models, and all 4 were included as independent data points. When the best achievable performance of current methods was evaluated, we extracted the data corresponding to the model that achieved the highest reported accuracy or the highest AUC within that study.

Results

Literature selection

After the initial database search, a total of 170 records were identified. After removing 37 duplicates, 133 records were identified, of which 123 publications were excluded after reading titles and abstracts. Following full text screening, 823, 24, 25^,27, 28, 29, 30, 31 studies were included in the review, and 7^24,25,27, 28, 29, 30, 31 studies were included in meta-analysis. The flowchart of the selection process is shown in Figure 1.

Fig. 1.

Summary of the search and process of the study selection.

Study description

The characteristics of the 8 articles published between 2013 and 2025 are described in Table 1. These studies were conducted in four different countries across Asia and Europe. With the exception of the study by Lee et al,³⁰ all investigations were single-centre in design. Regarding the devices used to obtain PRs, 2 studies utilised a single device^{23, 25}; 4 used 2 different devices,^24,27, 28, 29 and 2 did not specify the device used.^30,31 The datasets varied in the number of PRs images used, ranging from a minimum of 65 to a maximum of 1400. The marking of CAC images on PRs was performed by clinical specialists. Specifically, 1 specialist in 3 studies,^23,24,30 2 specialists in 2 studies,^28,29 and 3 or more specialists in 3 studies.^25,27,31 A total of 13 AI models were used, with deep learning (DL) being the most commonly employed approach, while only 1 study utilised machine learning (ML).²³ The number of models per study also varied: 5^{23,27,28,30,31} studies employed a single model, while others used 3,²⁹ 4,²⁵ and 5²⁴ models, respectively. These studies reported various outcomes, including accuracy, sensitivity/recall, specificity, precision, F1-score, as well as AUC. All 8 studies were retrospective.

Table 1.

Study characteristics of included primary studies.

Study	Country	Dataset size	Reference Standard	Number of devices used	Patient source	Study design	AI model	AI task type	Results (%)
Sawagashira et al23	Japan	100	One dental radiologist	1 device	Single centre	Retrospective	SVM	Detection	Sensitivity 90.0
Kats et al28	Israel	65	Two experts in oral medicine and maxillofacial radiology	2 devices	Single centre	Retrospective	Faster R-CNN	Detection	Sensitivity 75.0 Specificity 80.0 Accuracy 83.0
Lee et al30	Korea	1400	One dental radiography expert	NA	Multiple centre	Retrospective	Faster R-CNN	Detection	Sensitivity 95.8 Precision 42.3 Specificity 95.9
Song et al31	Korea	60	Two oral and maxillofacial radiologists and two general dentists	NA	Single centre	Retrospective	FAST-RCNN	Detection	Sensitivity 77.4 Specificity 71.7 Accuracy 72.7
Amitay et al25	Israel	500	Two certified oral and maxillofacial physicians who were trained by one oral radiology specialist	1 device	Single centre	Retrospective	InceptionResNetV2	Classification	F1-score 82.0 Sensitivity 82.0 Precision 84.0 Specificity 97.0 Accuracy 94.0
							InceptionResNetV2 + XGBoost	Classification	F1-score 80.0 Sensitivity 68.0 Precision 73.0 Specificity 91.0 Accuracy 91.0
							DenseNet169	Classification	F1-score 79.0 Sensitivity 80.0 Precision 81.0 Specificity 96.0 Accuracy 93.0
							EfficientNetV2M	Classification	F1-score 79.0 Sensitivity 78.0 Precision 80.0 Specificity 96.0 Accuracy 93.0
Vinayahalingam et al27	Germany	370	One dental radiologist, two oral and maxillofacial surgeons, and an oral Surgeon	2 devices	Single centre	Retrospective	Faster R-CNN+ Swin-T	Detection	Precision 89.5 Sensitivity 88.1 Specificity 89.7 F1-score 88.8 AUC 95.0 Average precision 94.2
Yoo et al24	Korea	400	One dental radiologist	2 devices	Single centre	Retrospective	CACSNet+VGG16	Classification	Sensitivity 88.0 Specificity 95.0 Accuracy 92.3 AUC 97.2
							CACSNet+MobileNet V2	Classification	Sensitivity 92.0 Specificity 98.8 Accuracy 96.2 AUC 99.6
							CACSNet+ResNet101	Classification	Sensitivity 94.0 Specificity 96.3 Accuracy 95.4 AUC 99.2
							CACSNet+DenseNet121	Classification	Sensitivity 94.0 Specificity 98.8 Accuracy 96.9 AUC 99.5
							CACSNet+EfcientNet-B4	Classification	Sensitivity 98.0 Specificity 98.8 Accuracy 98.5 AUC 99.6
Kuwada et al29	Japan	580	Two oral and maxillofacial radiologists	2 devices	Single centre	Retrospective	GoogLeNet YOLOv7+GoogLeNet YOLOv7	Classification Classification+Detection Detection	Per side: YOLOv7+GoogLeNet Sensitivity 0.88 Specificity 0.89 PPV 0.83 NPV 0.93 Accuracy 0.89 AUC 0.89 YOLOv7 Sensitivity 0.87 Specificity 0.82 PPV 0.74 NPV 0.91 Accuracy 0.84 AUC 0.84 Per person: GoogLeNet Sensitivity 0.68 Specificity 0.75 PPV 0.73 NPV 0.70 Accuracy 0.71 AUC 0.71 YOLOv7+GoogLeNet Sensitivity 0.93 Specificity 0.80 PPV 0.82 NPV 0.91 Accuracy 0.86 AUC 0.86 YOLOv7 Sensitivity 0.98 Specificity 0.83 PPV 0.85 NPV 0.97 Accuracy 0.90 AUC 0.90

AI, artificial intelligence; AUC, area under curve; DL, deep learning; ML, machine learning; NPV, negative predictive value; PPV, positive predictive value.

Risk of bias and applicability

The risk of bias and applicability assessment are shown in Figure 2. Detailed assessment results are shown in Supplementary Figure 1. Across the 8 studies, the risk of bias was variable: patient selection was ‘high’ in 4, ‘low’ in 2, and ‘unclear’ in 2. The index test and reference standard were predominantly ‘low’ or ‘unclear’ in risk. However, the domain of flow and timing was notably ‘high’ risk in six studies. No studies raised concerns regarding applicability.

Fig. 2.

Risk of bias assessment using quality assessment tool for diagnostic accuracy studies (QUADAS-2).

Meta-analysis

Pooled performance of AI models in the side-based evaluation

A total of six studies reported the performance of AI models in the side-based (left and right sides of participants) evaluation.^24,25,28, 29, 30, 31 Amitay et al²⁵ employed 4 different AI models, Yoo et al²⁴ utilised 5 different AI models, and Kuwada et al²⁹ used 2 different AI models; therefore, a total of 14 models were included. The pooled sensitivity was 0.88 (95% CI: 0.84, 0.92), pooled specificity was 0.94 (95% CI: 0.91-0.97), PLR was 15.8 (95% CI: 9.0-27.6), NLR was 0.12 (95% CI: 0.08-0.18), and DOR was 129 (95% CI: 55-305) (Figure 3, Supplementary Figures 2 and 3). The summary receiver operating characteristic (SROC) curve for DL-based diagnosis of CACs is presented in Figure 4, with an area under the curve (AUC) of 0.96, indicating excellent overall diagnostic accuracy. To investigate potential sources of the substantial statistical heterogeneity observed in the main analyses, we performed subgroup analyses based on three domains: AI task type (classification vs detection), dataset size (<500 images vs ≥500 images), and the number of specialists establishing the reference standard (1 vs ≥2). The detailed results are presented in Table 2. Studies employing classification models demonstrated notably higher pooled specificity (0.97, 95% CI: 0.95-0.98) and diagnostic odds ratio (DOR: 232.2) compared to detection models (specificity: 0.85, DOR: 38.46). Studies with smaller datasets (<500 images) showed higher pooled sensitivity (0.91 vs 0.85) but also higher heterogeneity in specificity (I² = 94.1%). Notably, studies that used a single specialist yielded significantly higher pooled sensitivity, specificity, and DOR than those with multiple specialists (≥2). The sensitivity analysis showed that the research findings are stable and reliable (Supplementary Figure 4). The Deeks’ funnel plot revealed no evidence of publication bias (P = 0.15 > 0.05), as shown in Figure 5. We extracted the highest accuracy data from the three studies for inclusion in the meta-analysis. When the highest performance was selected from the three studies, yielded a pooled sensitivity of 0.90 (95% CI: 0.81–0.95), pooled specificity of 0.92 (95% CI: 0.82-0.97), PLR of 11.7 (95% CI: 4.6-29.7), NLR of 0.11 (95% CI: 0.05-0.23), pooled AUC of 0.96 (95% CI: 0.94-0.98), and DOR of 109 (95% CI: 22-547) (Supplementary Figures 5-8).

Fig. 3.

Forest plot of sensitivity and specificity of the AI models for CACs diagnosis (per-side).

Fig. 4.

SROC for CACs diagnosis (per-side).

Table 2.

The results of subgroup analyses.

Subgroup	Number of studies	Sensitivity (95%CI)	I2	Specificity (95%CI)	I2	DOR (95%CI)
AI task type
Detection	4	0.87 (0.74, 0.94)	68.63	0.85 (0.70, 0.94)	97.73	38.46 (7.16, 206.58)
Classification	9	0.89 (0.82, 0.93)	67.1	0.97 (0.95, 0.98)	58.02	232.2 (88.59, 608.62)
Dataset size
<500 images	7	0.91 (0.83, 0.95)	66.6	0.96 (0.88, 0.99)	94.1	210.12 (40.31, 1095.30)
≥500images	7	0.85 (0.78, 0.91)	63.33	0.93 (0.90, 0.96)	86.57	83.72 (39.21, 178.72)
Number of reference specialists
1	6	0.94 (0.91, 0.96)	3.8	0.96 (0.95, 0.97)	0	392.6 (234.15, 658.28)
≥2	8	0.82 (0.76, 0.86)	15.62	0.91 (0.84, 0.95)	87.15	42.64 (21.71, 83.72)

Fig. 5.

Deek’s funnel plot of included studies.

Pooled performance of AI models in the person-based evaluation

The performance of AI models based on the number of PRs was evaluated in only two studies.^27,29 As the study by Kuwada et al²⁹ employed 3 distinct models, a total of 4 analyses were included in this assessment. The pooled sensitivity was 0.90 (95% CI: 0.74-0.97), specificity was 0.81 (95% CI: 0.73-0.87), PLR was 4.6 (95% CI: 3.1-7.0), NLR was 0.12 (95% CI: 0.04-0.36), DOR was 37 (95% CI: 10-144) and AUC = 0.86 (Figure 6, Supplementary Figures 9-11).

Fig. 6.

Forest plot of sensitivity and specificity of the AI models for CACs diagnosis (per-person).

Discussion

This systematic review and meta-analysis comprehensively evaluated the diagnostic accuracy of AI models for detecting CACs on panoramic radiographs (PRs). All seven studies included in the primary analysis demonstrated promising diagnostic performance, supporting the potential of DL-based AI technology as a viable tool for clinical diagnostics. For studies evaluating AI models based on per-side, the pooled sensitivity of 0.88 (95% CI: 0.84-0.92) and specificity of 0.94 (95% CI: 0.91-0.97) indicate that AI models can identify both true positives and true negatives with high reliability (Figure 3). This is further corroborated by an outstanding summary AUC of 0.96 and a DOR of 129, which collectively signify a robust overall diagnostic performance. These results affirm the potential of AI to mitigate the long-standing challenge of underdiagnosis of CACs in dental practice.

To our knowledge, only one previous systematic review and meta-analysis has been published on this topic, demonstrating that AI methods for identifying CACs in PRs achieve high sensitivity and specificity.³² While this finding aligns with our results and underscores the promise of AI-based screening, our study differs in several key aspects. Most notably, we calculated TP, TN, FP and FN values from the primary data when they were not directly provided. This methodology allowed for the inclusion of seven studies in our meta-analysis, compared to only three in the previous review, thereby yielding a more comprehensive and robust evidence base. Second, we assessed the diagnostic accuracy of AI for identifying CACs on PRs from side-based and person-based aspects. And our analysis revealed a notable performance difference between the two, as summarised in Table 1 and detailed in the per-person meta-analysis, the pooled specificity dropped to 0.81 (95% CI: 0.73-0.87) (Figure 6) and the AUC to 0.86 (Supplementary Figure 11). This discrepancy is clinically vital. It indicates that AI models may perform better on smaller, focused areas. Evidence from one study showed that 70 × 70 pixel image patches provide better diagnostic performance for assessing the mandibular canal and impacted third molar relationship than 140 × 140 pixel patches.³³ Furthermore, employing an area-detection technique before or simultaneously with classification improved the deep learning model's person-based performance in identifying CACs on PRs. The results of our subgroup analyses offer important, albeit preliminary, insights into the performance heterogeneity of AI models for CAC detection. The trend showing superior specificity for classification models over detection models may reflect the inherent challenge for detection algorithms to precisely localise calcifications without generating false positives on similar-looking anatomical structures. Conversely, the tendency for smaller datasets to be associated with higher sensitivity and specificity underscores the risk of overfitting and the critical need for larger, more diverse training sets to ensure generalizable performance. Perhaps most intriguing is the finding that studies using a single annotator as the reference standard reported significantly better performance metrics than those with multiple annotators. This counterintuitive result likely does not indicate that single-annotator standards are superior; rather, it may highlight the lack of a robust, consensus-driven ‘gold standard’ for CACs on PRs, and that models trained against a single subjective interpretation may not generalise well to broader clinical practice.

The clinical significance of these findings is profound. CACs visible on PRs are indicative of subclinical atherosclerotic disease, a major risk factor for ischemic stroke. Traditional methods for detecting these calcifications, such as ultrasound, CT angiography, and magnetic resonance imaging (MRI), can be invasive and costly.34, 35, 36, 37 These drawbacks make it unsuitable for widespread screening. PRs, routinely used in dental therapy, are low-cost and low-radiation, and can yield important incidental findings of CACs.^38,39 However, conventional visual assessment of CACs is notoriously challenging and subject to high inter-observer variability, even among trained specialists. Integrating AI offers a transformative solution: a consistent and accurate automated screening tool.⁴⁰ This can standardise detection in dental clinics, identifying at-risk patients who would otherwise go undiagnosed until a stroke.

While our study provides a strong evidence-based assessment of AI for detecting CACs on PRs, several important limitations warrant discussion. First, manual annotation of CACs (by a physician) is challenging, as they can be difficult to distinguish from other calcified anatomical structures in the same region, such as calcified triticeous cartilage. Definitive diagnosis of CACs requires Doppler ultrasonography,²⁸ however, due to the retrospective nature of included studies, Doppler ultrasonographic screening could not be conducted. This limitation may have introduced potential inaccuracies in reference annotations. Second, although we conducted a comprehensive systematic search, the final number of eligible studies was limited. Consequently, more studies are needed to validate these findings. Third, all studies were single-centre investigations. To improve clinical applicability and generalizability, many AI researchers recommend adopting multicentre study designs or external validation approaches.⁴¹ Furthermore, the generalizability of the model is limited by its training on PRs from only one or two imaging devices. Performance may therefore decline in real-world clinical environments where different scanner models are used. Last, the exclusion of unclear diagnoses and low-quality, blurry images resulted in artificially elevated model performance in most studies.

Conclusion

This review indicates that deep learning algorithms show promise in assisting dental professionals with detecting CACs on PRs. However, their current effectiveness is constrained by the small size and limited diversity of existing datasets. These findings are therefore preliminary and warrant cautious interpretation. Future research utilising larger, more diverse datasets is essential to validate the accuracy of these algorithms and establish their reliability for clinical practice.

Ethics statement

This review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and registered in PROSPERO (no: CRD420251140383).

Author contributions

Contributed to conception, design, data acquisition and analysis, and drafted the manuscript: Ye.; Contributed to data acquisition and analysis, and revised the manuscript: Zhou.; Contributed to data acquisition and analysis: Zhang.

Conflict of interest

None disclosed.