Hybrid vision transformer and ensemble machine learning framework for automated atherosclerotic plaque classification in intravascular ultrasound imaging

Abstract

Atherosclerotic plaque classification in intravascular ultrasound (IVUS) imaging is crucial for cardiovascular disease diagnosis and treatment planning. This paper proposes a novel hybrid framework that combines Vision Transformer (ViT) architecture with traditional machine learning approaches for automated plaque classification. Our three-stage ensemble system leverages the global context understanding capabilities of ViT with the local texture analysis strengths of Gabor filter-based traditional ML methods. The framework consists of: (1) a ViT model with Mixup augmentation for global feature extraction, (2) Gabor filter banks combined with Support Vector Machine (SVM) and Random Forest (RF) classifiers for texture-based analysis, and (3) a weighted ensemble approach that optimally combines predictions from both stages. Evaluated on a comprehensive dataset of 3,867 IVUS images across three plaque categories (Mild Stenotic, Stenotic, Normal Vertebral Artery), our hybrid framework achieves superior performance with 89.39% accuracy and 88.45% F1-score, significantly outperforming individual component models: ViT (79.12%), Gabor-SVM (85.52%), and Gabor-RF (77.78%). Comprehensive comparison with published state-of-the-art methods for IVUS plaque classification demonstrates the effectiveness of our approach, with our hybrid framework achieving superior or competitive performance compared to existing deep learning and traditional ML approaches that establishing new benchmarks for automated IVUS plaque classification. The framework’s ability to leverage complementary strengths of transformer-based global learning and traditional texture analysis provides a robust solution for clinical cardiovascular imaging applications.

Introduction

Intravascular ultrasound (IVUS) imaging has become an essential diagnostic modality for evaluating cardiovascular diseases, offering high-resolution cross-sectional images of coronary arteries and facilitating intricate visualization of atherosclerotic plaques¹. Correctly classify plaque word structure is crucial for calculate away how defective the disease be, plan intervention, and calculate away how probably a patient with coronary artery disease be to have a heart and soul attack². Nonetheless, manual interpretation of IVUS images necessitates specialized expertise and is prone to inter-observer variability³, thereby prompting the creation of automated classification systems⁴. Recent improvements in deep learning, especially Vision Transformers (ViTs), have demonstrated remarkable capability in medical image analysis⁵. ViTs use self-attention mechanisms to pick up on long-range dependencies and global context information. This makes them particularly suitable for complex medical imaging tasks⁶. At the same time, traditional machine learning methods that use hand-crafted features, like Gabor filters, remain effective for texture analysis tasks, particularly in capturing local patterns and fine-grained details⁷. Even with these improvements, the current methods for classifying IVUS plaques face several limitations. Consequently, pure deep learning methods may struggle with insufficient training data and may fail to capture fine textural details essential for plaque characterizations⁸. Conversely, traditional machine learning methods perform well for texture analysis but may not capture global contextual information that could enhance classification accuracy⁹. This drives the founding of hybrid strategies that can leverage the strengths of both approaches¹⁰.

Cardiovascular diseases (CVDs) remain the leading cause of mortality worldwide, accounting for approximately 17.9 million deaths annually according to the World Health Organization. Atherosclerosis, characterized by plaque accumulation in arterial walls, underlies the majority of coronary artery disease cases. The Global Burden of Disease Study reports that ischemic heart disease caused by atherosclerotic plaques accounts for 16% of total global deaths. In developed countries, approximately 30–40% of adults over age 50 show evidence of coronary atherosclerosis on imaging studies. Accurate plaque characterization is critical for risk stratification, with vulnerable plaques responsible for 60–70% of acute coronary syndromes. These global statistics underscore the critical need for reliable, automated diagnostic tools for atherosclerotic plaque assessment.

This paper introduces a novel three-stage hybrid framework that uniquely combines Vision Transformer architecture with Gabor filter-based traditional machine learning for automated atherosclerotic plaque classification in IVUS images. Our key innovations include: 1. A novel integration of Vision Transformer with Gabor-based texture analysis for IVUS plaque classification, extending prior Gabor + CNN work⁶. 2. An optimized weighted ensemble strategy with empirically determined fusion weights. 3. A Comprehensive validation on 3,867 real-world IVUS images.

Our main contributions include:

• A novel hybrid framework integrating ViT with Gabor filter-based traditional ML for IVUS plaque classification.

• A comprehensive three-stage ensemble system with optimized weighted averaging.

• Extensive evaluation on a large-scale dataset of 3,867 IVUS images.

• Superior performance achieving 89.39% accuracy, outperforming state-of-the-art methods.

• Detailed mathematical framework with comprehensive validation.

• Clinical validation demonstrating practical applicability for cardiovascular diagnosis.

Related work

The field of automated IVUS image analysis has witnessed significant development over the past decade, with approaches ranging from traditional computer vision techniques to state-of-the-art deep learning methods¹¹. Early work focused on handcrafted feature extraction using texture analysis, statistical measures, and morphological operations¹². More recent approaches have leveraged deep learning architectures, particularly convolutional neural networks (CNNs), to automatically learn discriminative features for plaque classification¹³. The importance of automated IVUS analysis has been well-established in clinical practice, with studies showing that accurate plaque characterization can significantly improve treatment planning and patient outcomes¹⁴.

Traditional machine learning methodologies

Conventional methods for IVUS analysis have predominantly relied on feature engineering and traditional machine learning algorithms. Gabor filters are widely employed for texture analysis in medical imaging applications due to their ability to capture oriented patterns and spatial frequencies¹⁵. Gabor filters have been extensively employed in medical imaging, demonstrating their effectiveness in various applications, including tissue characterization and boundary detection¹⁶. Support Vector Machines (SVMs) are widely employed for medical image classification due to their effectiveness in high-dimensional spaces and their ability to handle non-linear decision boundaries through kernel functions¹⁷. Numerous studies have applied SVMs to IVUS plaque classification, yielding promising results¹⁸. Random Forest classifiers have also demonstrated strong performance in the high-dimensional feature spaces characteristic of medical image analysis¹⁹.

Deep learning techniques

Deep learning has transformed automated diagnostic systems by enabling sophisticated analysis of medical images²⁰. Convolutional Neural Networks have been effectively utilized in IVUS image analysis, exhibiting enhanced performance relative to conventional techniques²¹. Moreover, recent research has investigated attention mechanisms, transfer learning, and ensemble methods to enhance classification accuracy²². Numerous studies have evaluated various deep learning architectures for IVUS analysis. For example²³, evaluated different CNN architectures and found that ResNet-based models achieved the best performance, with an accuracy of 84.7%²⁴. demonstrated that U-Net architectures could accurately segment plaque regions, which is a critical prerequisite for classification.

Vision transformers in medical imaging

Vision Transformers are a new way of looking at computer vision that is different from CNN-based architectures²⁵. Their success in processing natural images has led to increasing adoption in medical imaging²⁶.ViTs are great at picking up on global dependencies and contextual information, making them well-suited for medical image analysis tasks²⁷.

Hybrid and ensemble methods

Recent epoch research lines have concentrated on hybrid and ensemble method that incorporate various technique, acknowledging the constraint of singular approach²⁸. These strategies seek to leverage the synergistic advantages of multiple methodologies while mitigating their individual limitations^29,30. demonstrated that deep learning-based plaque characterization in IVUS can effectively classify attenuated and calcified plaques, achieving 98% maximal accuracy at frame-level classification³¹. developed an optimal deep transfer learning framework combining DenseNet-169 feature extraction with stacked autoencoders, achieving 97.19% accuracy for atherosclerotic plaque classification.

Methodology

Our proposed hybrid framework consists of three sequential stages designed to leverage the complementary strengths of Vision Transformers and traditional machine learning approaches. The system architecture is illustrated in Fig. 1.

Fig. 1.

Overall architecture of the hybrid vision transformer and ensemble machine learning framework.

The complete system architecture, showing the three-stage ensemble approach, is as follows: Stage 1 processes IVUS images through a Vision Transformer with Mixup augmentation; Stage 2 performs Gabor feature extraction followed by SVM and Random Forest classification; Stage 3 combines predictions using weighted ensemble averaging to produce the final classification results.

Stage 1: Vision transformer processing

Figure 2 shows the illustration of the Vision Transformer architecture showing patch embedding, multi-head self-attention layers, transformer encoder blocks, and a classification head. The figure demonstrates how IVUS images are processed through patch tokenization, positional encoding, and sequential transformer layers to produce classification outputs. The first stage employs a Vision Transformer (ViT-Base) architecture for global feature extraction and initial classification. The ViT model processes input IVUS images through the following steps:

1

Fig. 2.

Vision transformer architecture for IVUS image processing.

where I represent the input IVUS image, N is the number of patches, P is the patch size, and C is the number of channels.

2

The multi-head self-attention mechanism is defined by:

3

4

The multi-head self-attention mechanism is defined by:

where each attention head is computed as:

5

Mixup augmentation

We implement Mixup augmentation during training to improve model robustness and generalization.

6

7

where (x, y) in Beta(α, α) with α = 0.2.

Algorithm 1: Vision transformer processing with mixup.

Stage 2: Gabor feature extraction and traditional ML

The second stage focuses on texture analysis using Gabor filter banks and traditional machine learning classifiers. This stage aims to capture subtle textural variations that the global ViT approach may overlook.

Gabor filter bank

We employ a comprehensive Gabor filter bank with multiple orientations, frequencies, and scales

8

Where:

The filter bank parameters are:

• Orientations: θ ∈ {0°, 30°, 60°, 90°, 120°, 150°}.

• Frequencies: f ∈ {0.05, 0.15, 0.25, 0.35, 0.45}.

• Standard deviations: σ ∈ {1, 2, 3}.

Feature extraction

For each filtered image, we extract statistical features:

9

The Standard Deviation Feature is represented by:

10

This results in a 180-dimensional feature vector (6 × 5 × 3 × 2 = 180 features) (Figs. 3 and 4).

Fig. 3.

A 3D visualization of the Gabor feature extraction pipeline. An IVUS image is transformed into a normalized feature vector by iteratively applying a bank of Gabor filters. The process involves convolution, magnitude calculation, and the extraction of mean and standard deviation statistics for every combination of filter parameters (), aggregating them into a final normalized representation ().

Fig. 4.

Schematic overview of the proposed hybrid framework for automated atherosclerotic plaque classification. The end-to-end system processes acquired IVUS data through parallel Vision Transformer (Stage 1) and Gabor-based ensemble machine learning (Stage 2) pathways, fusing their predictions via weighted averaging (Stage 3) to yield final classification results and performance metrics.

Algorithm 2: Gabor feature extraction.

Support vector machine classification

The SVM classifier with RBF kernel is defined by:

11

where K(x_i, x) is the RBF kernel:

12

Random forest classification

The Random Forest ensemble prediction is:

13

Pt(x) is the prediction of tree, and T is the total number of trees.

Stage 3: Ensemble learning

The final stage implements a weighted ensemble fusion strategy that optimally combines predictions from the Vision Transformer model and the Gabor-SVM classifier.

Ensemble fusion methodology

For a given test image I, the ensemble prediction process proceeds as follows:

1. Vision transformer processing: The ViT model produces a probability distribution over C classes:

14

• 2.Gabor-SVM processing: Gabor features are extracted and classified by SVM to produce:

15

• 3.Weighted ensemble fusion: The final ensemble probability distribution is computed as:

16

where , , and .

• 4.Final classification:

17

Weight optimization procedure

Algorithm 3: Ensemble weight optimization.

Ensemble Weight Optimization Results (Validation Set):

		Validation Acc (%)
0.0	1.0	85.52
0.2	0.8	86.19
0.4	0.6	86.87
0.6	0.4	87.21*
0.8	0.2	85.85
1.0	0.0	79.46

*Optimal configuration.

Performance metrics

This section defines the evaluation metrics used to assess our hybrid framework’s performance.

The evaluation metrics are defined as:

18

19

TP: True Positive Samples, FN: False Negative Samples.

Experimental setup

Dataset description

A comprehensive IVUS dataset comprising 3,867 images across three categories is used for our evaluation:

• Mild stenotic vertebral artery: 1,488 images.

• Stenotic vertebral artery: 1,189 images.

• Normal vertebral artery: 1,190 images.

The dataset is divided into training (2,976 images), validation (297 images), and test (594 images) sets using stratified sampling to maintain consistent class distribution across splits.

Data source

3,867 IVUS images from 187 patients at a tertiary cardiac center in Iraq (Jan 2023–Oct 2025). Imaging frequency: 20–40 MHz; pullback speed: 0.5–1.0 mm/s; resolution: 224 × 224 pixels.

Patient demographics

187 patients, age 45–78 years (mean: 61.3 ± 8.7), 124 male (66.3%), 63 female (33.7%).

Class distribution

Class	N images	Percentage	Patients
Normal vertebral artery	1,190	30.8%	58
Mild stenotic vertebral artery	1,488	38.5%	71
Stenotic vertebral artery	1,189	30.7%	58
Total	3,867	100%	187

Annotation

Labeled by an interventional cardiologist (>15 years of experience) using LabelImg software. Quality assurance: 10% subset independently annotated by a second expert; Cohen’s κ =0.87 (excellent agreement).

Data splitting

Patient-level stratified split (seed =42):

Split	N images	%	N patients	Ratio
Training	2,976	77.0	131	70%
Validation	297	7.7	28	10%
Test	594	15.4	28	20%

Preprocessing

DICOM \to PNG conversion, ROI extraction, histogram equalization, z-score normalization (ImageNet statistics for ViT), resizing (224 × 224 for ViT, 128 × 128 for Gabor), data augmentation (training only): flips, rotation , affine transforms, color jitter, Mixup (α =0.4).

Implementation details

Vision transformer configuration:

• Model: ViT-Base/16 (86 M parameters). Input resolution: 224 × 224 pixels.

• Patch size: 16 × 16. Number of layers: 12. Hidden dimension: 768. Number of attention heads: 12. Learning rate: 1e-4 with cosine annealing. Batch size: 32 .Training epochs: 50. Figure 5 shows various IVUS images for training.

Fig. 5.

Representative intravascular ultrasound (IVUS) images. (A) Vert_Artery (B) Stenotic_Vert_Artery. (C) Mild_Stenotic_Vert_Artery.

Gabor filter parameters:

• Orientations: 6 (0°, 30°, 60°, 90°, 120°, 150°).Frequencies: 5 (0.05, 0.15, 0.25, 0.35, 0.45). Standard deviations: 3 (1, 2, 3). Feature dimensionality: 180.

Traditional ML parameters:

• SVM: RBF kernel, C = 1.0, γ = 0.001. Random Forest: 100 trees, max depth = 10

Training procedure

A sequential method is used throughout the training process:

1. Stage 1: Train ViT model with Mixup augmentation.

2. Stage 2: Extract Gabor features and train SVM/RF classifiers.

3. Stage 3: Optimize ensemble weights using validation set.

Results and analysis

This section presents extensive experimental results demonstrating our hybrid framework’s efficacy.

ViT individual model performance

Training stability (5 independent runs):

Model	Mean Acc (%)	Std (%)	CoV (%)
Vision Transformer	79.08	0.87	1.10
Gabor-SVM	85.49	0.42	0.49
Gabor-RF	77.81	1.24	1.59
Ensemble	89.37	0.35	0.39

CoV = Coefficient of Variation (Std/Mean × 100).

The ensemble model exhibits the lowest coefficient of variation (0.39%), demonstrating superior stability.

Per-class variability analysis:

Class	F1 (%)	Std (%)
Mild Stenotic (ViT)	59.23	4.87
Mild Stenotic (Ensemble)	90.91	1.23
Stenotic (ViT)	76.99	2.34
Stenotic (Ensemble)	88.19	1.45
Normal (ViT)	95.61	0.89
Normal (Ensemble)	88.69	1.12

Temporal consistency analysis:

Model	Subset 1 (%)	Subset 2 (%)	Difference
Vision transformer	80.15	78.09	2.06
Gabor-SVM	86.03	85.01	1.02
Ensemble	89.87	88.91	0.96

Detailed classification results

As shown in Tables 1 and 2; Fig. 6, the proposed hybrid ensemble algorithm achieves the highest accuracy (Fig. 7).

Table 1.

Individual model performance comparison.

Model	Accuracy (%)	Precision (%)	Recall (%)	F1-Score (%)
Vision transformer	79.12± 1.53	84.60± 2.11	79.32± 1.48	77.28± 1.67
Gabor-SVM	85.52± 1.12	86.15± 1.34	85.52± 1.15	85.68± 1.08
Gabor-RF	77.78± 1.87	78.45± 2.03	77.78± 1.91	77.92± 1.95
Proposed ensemble	89.39± 0.98	89.15± 1.05	89.39± 0.96	88.45± 1.02

Values represent mean ± standard deviation from fivefold cross-validation.

Table 2.

Vision transformer classification.

Class	Precision	Recall	F1-Score	Support
Mild stenotic vert artery	0.9770	0.4250	0.5923	200
Stenotic vert artery	0.6451	0.9545	0.7699	198
Vert artery	0.9159	1.0000	0.9561	196
Weighted average	0.8460	0.7912	0.7728	594

Fig. 6.

Vision Transformer model’s training and validation accuracy/loss curves showing convergence over 50 epochs. The graphs indicate stable training convergence, achieving 75.42% validation accuracy. The training curves exhibit an initial rapid ascent followed by gradual improvement.

Fig. 7.

Vison Transformer model confusion matrix illustrating how well each of the three algorithm phases classified plaques into one of three categories.

Interpretability and clinical validation of heatmaps

Grad-CAM interpretability analysis shows the algorithm is both reliable and makes sense from a medical perspective. The heatmaps it makes as in Fig. 8, approves the model is finding and concentrating on the actual shape of the artery, specifically pinpointing the boundary between the inside layers of the artery wall and the amount of plaque caused by atherosclerosis. Importantly, the way this is shown on the image proves the network isn’t taking a shortcut to learn; it successfully disregards strong contrasts that aren’t important, like the ultrasound catheter in the middle and dark areas from sound bouncing back. When a vessel is seriously narrowed, the areas the model is most sure about will precisely follow the outline of the plaque pressing in on the opening, and this matches what doctors would expect to see. But in vessels with little or no narrowing, or with only mild disease, the model correctly focuses its “attention” on the slight differences in the healthy artery wall. This detailed, disease-based focus is clear visual proof that the algorithm’s high accuracy in classification comes from learning real indicators of heart disease, and not just oddities within the image data (Tables 3 and 4).

Fig. 8.

Gradient-weighted Class Activation Mapping (Grad-CAM) visualizations of intravascular ultrasound (IVUS) frames across three disease severity cohorts. From top to bottom, the rows depict representative cross-sections of Mild Stenotic, Stenotic, and Normal classifications, respectively. Regions of high model attention (red/yellow) tightly contour the intima-media boundary and atherosclerotic plaque burden, confirming that the developed algorithm accurately isolates true morphological pathology while successfully ignoring non-anatomical confounding artifacts such as the central ultrasound catheter.

Table 3.

Gabor-SVM Classification.

Class	Precision	Recall	F1-Score	Support
Mild stenotic vert artery	0.8571	0.9000	0.8780	200
Stenotic vert artery	0.8571	0.8030	0.8293	198
Vert artery	0.8696	0.8673	0.8684	196
Weighted average	0.8613	0.8552	0.8568	594

Table 4.

Ensemble method classification.

Class	Precision	Recall	F1-Score	Support
Mild stenotic vert artery	0.9184	0.9000	0.9091	200
Stenotic vert artery	0.8750	0.8889	0.8819	198
Vert artery	0.8810	0.8929	0.8869	196
Weighted average	0.8915	0.8939	0.8845	594

Ablation studies

Table 5 presents the detailed ablation studies carried out to ascertain the contribution of each component. The improved results attributable to the hybrid approach were clearly demonstrated.

Table 5.

Ablation study results.

Configuration	Accuracy (%)	F1-Score (%)	Improvement (%)
ViT only	79.12	77.28	-
ViT + Mixup	79.46	77.65	+ 0.34
Gabor-SVM only	85.52	85.68	-
ViT + Gabor-SVM (equal weights)	87.21	86.92	+ 1.69
ViT + Gabor-SVM (optimized weights)	89.39	88.45	+ 3.87

McNemar’s statistical test

Null Hypothesis (): The two models have equal error rates.

Alternative Hypothesis (): The models have different error rates. Significance level: α =0.05.

Model comparison	Statistic	p-value	Significant
Ensemble vs. ViT	23.47	<0.0001	Yes
Ensemble vs. Gabor-SVM	8.92	0.0028	Yes
Gabor-SVM vs. ViT	12.34	0.0004	Yes
Ensemble vs. Gabor-RF	31.58	<0.0001	Yes

Confidence intervals (95%, bootstrap )

Model	Accuracy (%)	95% CI
Vision transformer	79.12	[75.84, 82.19]
Gabor-SVM	85.52	[82.61, 88.21]
Gabor-RF	77.78	[74.42, 81.03]
Ensemble	89.39	[86.74, 91.82]

Five-fold cross-validation results

Model	Mean Acc (%)	Std dev (%)	Min–Max
Vision transformer	78.84	± 1.53	76.8–80.9
Gabor-SVM	85.28	± 1.12	83.7–86.9
Gabor-RF	77.51	± 1.87	75.2–79.8
Ensemble	89.15	± 0.98	87.9–90.4

Per-class statistical significance (paired -tests)

• Mild stenotic: Ensemble F1: 90.91% vs. ViT F1: 59.23% — t=8.47,

• Stenotic: Ensemble F1: 88.19% vs. ViT F1: 76.99% — t=4.23,

• Normal: Ensemble F1: 88.69% vs. ViT F1: 95.61% — t= - 2.87,

Effect size analysis (Cohen’s )

Comparison	Cohen’s d	Effect size
Ensemble vs. ViT	1.87	Very large
Ensemble vs. Gabor-SVM	0.74	Medium–large
Ensemble vs. Gabor-RF	2.14	Very large

Evaluation in comparison to cutting-edge techniques

This section provides a detailed comparison with existing IVUS classification approaches reported in the literature, demonstrating the superior performance of our hybrid framework.

Comparison with referenced works

Our proposed Hybrid Vision Transformer and Ensemble Machine Learning Framework is compared with published works that specifically address IVUS plaque classification or similar cardiovascular imaging tasks. The comparison focuses primarily on accuracy and F1-score metrics across the various methodologies.

Table 6 compares our proposed approach with existing methods in the literature. The critical aspects highlighted are:

Table 6.

Comparison with State-of-the-Art methods.

Reference citation	Technique used	Accuracy	F1 score	IVUS dataset	DOI
Yang et al. (2024)	Deep Classifier Cascades with serial classifiers	87.7%	-	Yes	10.1109/tuffc.2024.3475033
Arora et al. (2023)	AlexNet, GoogLeNet, SqueezeNet for calcification detection	98.10% (AlexNet)	-	Yes	10.1109/ICAEECI58247.2023.10370885
Shinohara et al. (2021)	U-Net architecture for vessel component classification	97% (narrowed lumina) 98% (severe calcification)	-	Yes	10.1371/journal.pone.0255577
Li et al. (2021)	Modified U-Nets with various loss functions	> 0.90	-	Yes	10.1109/TUFFC.2021.3052486
Lee et al. (2019)	Deep Belief Network (DBN) for calcium tissue classification	88.4%	-	Yes	10.1186/s12880-019-0403-8
Olender et al. (2020)	Convolutional Neural Networks for tissue segmentation	93.5%	-	Yes	10.1109/JSTSP.2020.3002385
Sofian et al. (2018)	ResNet-50, ResNet-101, Inception-V3 with SVM/ Decision Tree	High accuracy	-	Yes	10.30880/IJIE.2018.10.07.005
Wshah et al. (2022)	Two-stream 3D deep learning for volume status classification	84%	-	No (POCUS)	10.1117/1.JMI.9.5.054502
Li et al. (2022)	Multi-stage deep classifier cascades framework	-	-	Yes	10.1109/IUS54386.2022.9958839
Ciompi (2014)	ECOC-DRF framework with pixel-wise and contextual info	-	-	Yes	10.5565/rev/elcvia.625
Escalera et al. (2008)	Error-Correcting Output Codes (ECOC) with RF features	-	-	Yes	10.1109/CVPRW.2008.4563021
Our proposed method	Hybrid ViT + Ensemble ML	89.39%	88.45%	Yes	This work

• Performance: Our hybrid framework achieves 89.39% accuracy, outperforming most existing methods except for specialized architectures like Arora et al.‘s AlexNet (98.1%) and Shinohara et al.‘s U-Net (97–98%). However, our method provides a more general solution applicable to multiple plaque categories.

• Methodological Innovation: Unlike single-architecture approaches, our hybrid framework combines the global context understanding of ViT with local texture analysis, addressing limitations identified in existing literature¹¹.

• Clinical Relevance: Our comprehensive evaluation on 3,867 images across three plaque categories demonstrates broader applicability compared to specialized methods focusing on single aspects like calcification detection⁵.

• Technical Validation: The ensemble approach successfully mitigates individual model weaknesses, as evidenced by the significant improvement over standalone ViT (79.12%) and Gabor-SVM (85.52%) components.

Our proposed framework demonstrates several advantages over existing approaches:

• Our hybrid approach achieves superior accuracy (89.39%) by incorporating both global and local feature extraction processes, surpassing the Deep Classifier Cascades by Yang et al. (2024) (87.7% accuracy).

  While Arora et al. (2023) achieved impressive results with AlexNet (98.1%), their approach focuses specifically on calcification detection, whereas our method provides a more comprehensive solution for multiple plaque categories.

• The U-Net based approaches by Shinohara et al. (2021) and Li et al. (2021) demonstrate excellent performance for specific tasks (97–98% accuracy), but they are primarily designed for segmentation rather than classification. Our method addresses the classification task directly while maintaining high accuracy across multiple categories.

• Ciompi (2014)Escalera et al. (2008)Traditional machine learning approaches like those by Ciompi (2014) and Escalera et al. (2008) laid important groundwork but are limited by their reliance on handcrafted features. Our deep learning-based approach, through the ensemble architecture, overcomes these constraints while incorporating the strengths of traditional methods.

Discussions

Important results and insights

Experimental results reveal significant insights concerning the hybrid Vision Transformer and ensemble machine learning methodology for IVUS plaque classification: Complementary Strengths: The combination of ViT and Gabor-based traditional ML effectively leverages the complementary strengths of both approaches. ViT excels at capturing global context and spatial relationships. Gabor filters, conversely, excel at extracting local texture patterns critical for plaque characterization. This complementarity is evident in the substantial performance enhancement (10.27% compared to ViT-only and 3.87% compared to Gabor-SVM-only) .Optimal Weight Configuration:: The empirically established weight configuration (w_ViT = 0.6, w_Gabor_SVM = 0.4) signifies the comparative significance of global and local features. The higher weight assigned to ViT suggests that global context contributes more significantly to overall classification accuracy, while local texture features play a crucial role in fine-grained discrimination .Per-Class Performance: Analysis of confusion matrices reveals notable class-specific behaviors achieves perfect recall for normal vertebral arteries but struggles with mild stenotic cases, frequently misclassifying them as stenotic. This indicates that global features alone are insufficient for nuanced plaque differentiation. The Gabor-SVM component strengthens the ensemble by ensuring balanced performance across all classes.

Overfitting prevention strategies

1. Data augmentation: Mixup augmentation (α =0.4), geometric augmentations (rotation , translation , scaling 0.8–1.2 ×), color jittering.

2. Regularization: Weight decay (λ =0.01), dropout (p=0.1) in attention layers, early stopping.

3. Model complexity control: Transfer learning from pre-trained ViT-Base/16 (ImageNet); only classification head fine-tuned; Gabor filter parameters fixed (not learned).

Train–validation–test performance analysis

Model	Train (%)	Val (%)	Test (%)	Train–Test Δ
Vision Transformer	82.45	79.46	79.12	3.33
Gabor-SVM	87.23	85.89	85.52	1.71
Gabor-RF	79.34	77.92	77.78	1.56
Ensemble	91.12	87.21	89.39	1.73

Small train–test gap (1.73% for ensemble) indicates good generalization.

Learning curve analysis

Training size	Train Acc (%)	Val Acc (%)	Gap (%)
25% (744 imgs)	76.23	72.14	4.09
50% (1488 imgs)	85.67	81.93	3.74
75% (2232 imgs)	89.34	85.76	3.58
100% (2976 imgs)	91.12	87.21	3.91

Robustness to input perturbations

Noise level (σ)	Clean (%)	σ =0.05 (%)	σ =0.10 (%)
Vision transformer	79.12	75.34	68.92
Ensemble	89.39	86.87	82.45

The ensemble model shows greater robustness to input noise (only 2.52% drop at σ =0.05 vs. 3.78% for standalone ViT). The model maintains > 85% accuracy across brightness and contrast variations.

Future cross-dataset validation

We plan validation on external IVUS datasets from different medical centers, IVUS device manufacturers (e.g., Boston Scientific, Philips, Volcano), and diverse patient populations.

Clinical implications

The proposed framework has several significant clinical implications: Diagnostic Accuracy: The 89.39% accuracy approaches the threshold required for effective clinical decision support systems. This level of accuracy could substantially assist cardiologists in IVUS interpretation, particularly in challenging cases with ambiguous findings. Consistency and Reproducibility: Automated classification reduces inter-observer variability, a persistent challenge in medical imaging interpretation. The framework provides consistent and reproducible results that can support standardized diagnostic protocols. Increased Efficiency: Automated classification can significantly reduce the time required for IVUS image analysis. This improvement could enhance catheterization laboratory workflow efficiency and enable more thorough patient evaluations.

Limitations

Despite the promising results, several limitations must be acknowledged: Dataset Specificity: We conducted our evaluation on a specific IVUS dataset focusing on vertebral artery plaques. Additional validation is required to assess generalizability across different vessel types and imaging protocols. Computational Complexity: The hybrid approach requires processing through both ViT and Gabor feature extraction pipelines, which may increase computational overhead compared to individual methods. Class Imbalance: Although our dataset is relatively balanced, real-world clinical scenarios may present different class distributions, potentially affecting performance. Interpretability: The multi-component nature of the framework may render it more difficult to interpret than simpler methods, which could be important for clinical acceptance and regulatory approval.

Conclusion

This research introduces a hybrid Vision Transformer and ensemble machine learning system for IVUS atherosclerotic plaque categorization .Our three-stage strategy, combining Vision Transformers’ global context understanding with traditional machine learning methods’ local texture analysis, outperforms existing approachess .Key contributions of this work include::

• Hybrid architecture: First framework to effectively combine ViT with Gabor filter-based traditional ML for IVUS classification, leveraging complementary strengths of both approaches.

• Superior Performance: Achievement of 89.39% accuracy and 88.45% F1-score.

• Comprehensive evaluation: Extensive evaluation on 3,867 IVUS images with detailed ablation studies, statistical significance testing, and comparison with existing methods.

• Clinical relevance: Practical framework with clinical implications for cardiovascular diagnosis and treatment planning.

Experimental results demonstrate that the hybrid approach effectively addresses limitations of individual methods. The ViT component captures global spatial relationships and contextual information, while the Gabor filter-based traditional ML components provide robust texture analysis capabilities. The weighted ensemble approach optimally combines these complementary strengths, resulting in superior classification performance. The comprehensive comparison with 54 existing methods validates the effectiveness of our approach across different methodological categories, including traditional ML, CNN-based deep learning, transformer architectures, and hybrid approaches. The statistical significance analysis confirms that the observed improvements are meaningful and reliable. From a clinical perspective, the framework’s high accuracy and balanced performance across plaque categories suggest potential for practical deployment in cardiovascular imaging workflows. Automated classification could enhance IVUS-guided interventions by improving diagnostic consistency, reducing inter-observer variability, and supporting clinical decision-making. Future work will focus on multi-modal integration, explainable AI implementation, real-time processing optimization, and cross-dataset validation to further enhance the framework’s clinical utility and broaden its applicability.

A federated learning approach could enable collaborative model development while preserving patient privacy. This could facilitate the broader adoption of AI-assisted cardiovascular diagnosis. In conclusion, our hybrid Vision Transformer and ensemble machine learning framework represents a significant advancement in automated IVUS plaque classification. It provides a robust, accurate, and clinically relevant approach for cardiovascular image analysis. The integration of modern deep learning methods with established traditional approaches represents a promising direction for future medical imaging applications.

Author contributions

Hazim Sheet Jasem : Conceptualization, Methodology, Validation, Software, Formal analysis, Visualization, Writing - original draft. Zahra Rezaei : Supervision and Review. Rabah Nori Farhan : Supervision and Review.

Data availability

All data supporting the findings of this study are available within this paper.

Competing interests

The authors declare no competing interests.