A novel superpixel based Vision Transformer for improving interpretability in glaucoma screening

Abstract

Interpretability remains one of the major challenges in the clinical adoption of deep learning models for medical image analysis. In ophthalmology, particularly for glaucoma screening, explainable artificial intelligence (XAI) methods are essential for ensuring trust and diagnostic transparency. This study introduces the Superpixel-based Vision Transformer (SpxViT), a model designed to enhance interpretability while maintaining competitive accuracy. SpxViT replaces the traditional fixed grid tokenization of Vision Transformers. (ViTs) with a superpixel-based approach that preserves semantic boundaries within the retinal image. Two variants, SpxViT_fix and SpxViT_var, were evaluated on public and private glaucoma datasets. Results demonstrate that SpxViT achieves comparable accuracy to ViT-B/16 (91.9% vs. 92.5%) while producing more clinically consistent attention maps focused on the optic disc and cup.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-026-39730-x.

Introduction

Glaucoma is a progressive optic neuropathy and one of the leading causes of irreversible blindness worldwide¹. Early detection and monitoring are essential to prevent vision loss, yet the diagnostic process heavily depends on expert evaluation of structural changes in the optic nerve head. While a definitive glaucoma diagnosis relies on a combination of clinical tests including intraocular pressure and visual field analysis, automated screening based on fundus images serves as a crucial first step in identifying patients who require further specialized evaluation. Artificial Intelligence (AI) and deep learning approaches have demonstrated strong potential to assist clinicians in detecting glaucoma from retinal fundus images^2,3 However, despite their promising performance, these models remain limited by a major barrier, the lack of interpretability⁴. In clinical settings, physicians must not only know what the model predicts but also why it does so.

Traditional Convolutional Neural Networks (CNNs)^5,6 and Vision Transformers (ViTs)^7,8 achieve high accuracy in image classification, but their “black-box” nature makes it difficult to understand which image regions influence predictions. This limitation restricts their integration into diagnostic workflows. To address this, explainable AI (XAI) methods, such as saliency maps⁹, Grad-CAM¹⁰, and attention-based models^11,12, have been explored. Yet, many of these techniques provide only post hoc explanations that may not accurately represent the model’s reasoning process^11,13.

Recent advances in deep learning have highlighted the importance of learning meaningful visual representations that go beyond regular grid-based sampling. Transformer-based architectures have demonstrated strong performance by leveraging attention mechanisms to focus on task-relevant image regions, both in natural and medical imaging applications. Several works have explored adaptive or structurally informed representations to better capture discriminative visual cues, including convolution–Transformer hybrid architectures that combine local feature extraction with global context modelling¹⁴, as well as Transformer-based models tailored for medical image analysis¹⁵. In parallel, increasing attention has been paid to the explainability of visual classifiers, with instance-wise and attention-based analyses being widely used to interpret model decisions¹⁶. These trends motivate the exploration of alternative tokenization strategies that can preserve semantic coherence and improve interpretability, particularly in clinical scenarios where model transparency is critical.

The Superpixel-based Vision Transformer (SpxViT) proposed in this work introduces interpretability directly at the tokenization level. Instead of using fixed-size square patches as in standard ViTs, SpxViT employs superpixelbased tokenization to create input tokens that correspond more closely to meaningful visual structures (e.g., optic cup, disc, and surrounding tissues). This design ensures that each token captures semantically coherent regions, making the resulting attention maps inherently more interpretable.

Recent research has highlighted the growing interest in superpixel-based tokenization for Vision Transformers^17–20. Traditional Vision Transformers (ViTs)²¹ rely on a sequence of fixed-size, non-adaptive square patches for image classification. Critics argue that this uniform slicing method is an oversimplification that fails to consider the correlation between patches, which can negatively impact the model’s adaptability and generalization¹⁸. Moreover, a fixed patch embedding method results in arbitrary mixing of visual concepts within a single token, disregarding semantic boundaries. Superpixel-based approaches emerge as a powerful alternative because they adapt to image content, dividing the image into irregular regions that group adjacent pixels sharing similar characteristics like color, texture, or brightness into meaningful, semantically coherent blocks¹⁸.

Several novel architectures leverage superpixels to overcome the limitations of fixed patches and conventional positional encoding. The Vision Transformer with Superpixel Slicing (SSVT) method proposes a superpixel slicing scheme that dynamically generates a sequence of patch embeddings based on the image’s features¹⁸. This dynamic approach allows for the creation of a progressive positional encoding designed to bring slices of the same object closer together in the sequence, thereby capturing the complete positional information of the target object. Another architecture, the Superpixel-Tokenized Vision Transformer (SuiT)²⁰, addresses the challenges presented by superpixels’ irregular shapes, sizes, and variable locations through a specific two-stage pipeline. This pipeline first creates pixel-level embeddings (combining local and parameter-efficient positional features) and then uses Superpixel-aware Aggregation (employing average and max pooling) to generate fixed-dimensional tokens.

Other contributions focus on modularity and interpretability, which are inherent strengths of superpixel representation. The Superpixel Transformer (SPFormer)¹⁹ enhances ViT by utilizing Superpixel Cross Attention (SCA) to iteratively refine superpixel and pixel features. SPFormer highlights the inherent explainability afforded by its superpixel structure, which aligns seamlessly with semantic boundaries even when trained only with category labels. Similarly, Modular Superpixel Tokenization (SPiT)¹⁷ proposes a generalized framework that treats tokenization as a modular component, decoupling it from the transformer backbone. SPiT uses an efficient on-line superpixel tokenization approach combined with scale- and shape-invariant positional embeddings to handle irregular partitions.

Building upon these advances, SpxViT adapts the concept of semantically meaningful tokens to the field of ophthalmology, emphasizing not only performance but also clinical interpretability. Our method is based on the principle of decoupling the tokenisation process from the transformer backbone architecture, as proposed by the SPiT model¹⁷. The key difference in our approach, as will be demonstrated below, lies in our proposal to generate superpixels using efficient, well-established techniques and to construct embeddings that are simpler and more aligned with conventional ViTs. This implies a different approach to handling patch irregularity. While the SPiT model explores complex feature extraction (colour, texture, shape, scale, and position) to maximise information content in irregular superpixels, our work shows that explainability benefits can be obtained by processing superpixels in a manner similar to that used by a classic ViT for regular patches. This study aims to:

1. Propose a Vision Transformer variant (SpxViT) that integrates superpixelbased tokenization while preserving the original ViT architecture.

2. Evaluate whether superpixel tokens improve interpretability without compromising diagnostic accuracy.

3. Analyze attention behavior across glaucoma and healthy cases to assess clinical consistency.

4. Compare SpxViT against baseline models (ViT-B/16, SPiT).

Unlike prior studies that modify the backbone of ViT to integrate superpixel attention, SpxViT focuses solely on the tokenization stage, ensuring direct comparability with the standard ViT-B/16. The novelty of this work lies not in introducing a new transformer architecture, but in demonstrating how an interpretable tokenization mechanism can make attention maps more clinically meaningful while maintaining high accuracy.

By aligning deep learning representations with anatomically and clinically relevant regions, SpxViT bridges the gap between computational interpretability and medical explainability. The results show that SpxViT delivers interpretable attention focusing on the optic disc and cup, which are essential regions for glaucoma assessment. This approach promotes transparency and reliability, contributing to safer clinical adoption of AI-based diagnostic tools.

Materials and methods

This section describes the datasets, preprocessing pipeline, model architecture, and explainable technique used in this study.

Datasets

Two datasets were used for this study:

1. Rim-ONE DL (public dataset)²²: a well-known glaucoma database containing optic disc-centered fundus images, used for benchmarking deep learning approaches.

2. Private dataset (University Hospital of the Canary Islands): consisting of anonymized color fundus photographs collected under ethical approval (protocol CHUC_2023_41).

Together, the datasets comprise 739 images (glaucomatous and non-glaucomatous), divided using five-fold cross-validation to ensure robust evaluation and reduce bias. All experiments comply with the Declaration of Helsinki, and patient identifiers were removed prior to processing. CHUC_2023_41, 27 April 2023). The confidentiality of personal data was guaranteed.

Glaucomatous grading was established by ophthalmology specialists using standard clinical criteria. An anatomical criterion was primarily applied, considering a cup-to-disc ratio greater than 0.7 as indicative of glaucoma. This assessment was complemented by additional clinical findings, including elevated intraocular pressure (> 22 mmHg), reproducible focal defects in standard white-on-white perimetry, and structural abnormalities of the optic nerve head such as localized neuroretinal rim thinning or notching. Diagnosis was therefore based on a combination of anatomical and functional indicators.

Additionally, medical experts manually segmented the disc and cup in all these images. These two regions are crucial for diagnosing glaucoma. For the public RIM-ONE DL dataset, optic disc and cup segmentations were independently performed by five ophthalmology specialists, each with more than at least five years of experience in glaucoma diagnosis. For the private dataset from the University Hospital of the Canary Islands (HUC), segmentations were independently performed by two glaucoma specialists, with more than ten years of clinical experience. In both datasets, the final segmentation used in this study corresponds to the average segmentation computed from all available expert annotations. As a result, discrepancies between individual experts were mitigated. Figure 1 shows two images extracted from Rim-ONE DL, together with their manual segmentations.

Fig. 1.

Images of an eye with glaucoma (left) and a healthy eye (right) extracted from²². The bottom row shows the cup and disc areas manually segmented by medical experts.

Data preprocessing and superpixel generation

Images were resized to 224 × 224 pixels. Superpixels were generated using the SLIC (Simple Linear Iterative Clustering) algorithm²³. SLIC is an adaptation of the k-means clustering algorithm that groups pixels into perceptually meaningful regions that correspond to clusters in the CIELAB colour image space.

Rather than dividing the image into 196 regular 16 × 16 pixel patches, SLIC is used for generating 196 irregular patches based on superpixels extracted from the image. Although other methods were also tested, such as SEEDS (Superpixels Extracted via Energy-Driven Sampling)²⁴, ERS (Entropy Rate Superpixel Segmentation)²⁵ and LSC (Linear Spectral Clustering Superpixel)²⁶, SLIC was ultimately chosen as it offered the best-suited superpixels to the image structures in the simplest and fastest way. Another criterion for selecting the algorithm was the need to always obtain 196 superpixels per image, which is essential for emulating the structure of ViTB/16. This can be achieved by adjusting the SLIC compactness parameter, which allows users to customise the size and shape of the generated superpixels.

Two tests were performed to adjust the compactness value with the aim of generating 196 consistent superpixels. The first test examined how different fixed compactness values affected the generation of superpixels in the analysed retinography images. In our images, a compactness value of 8 consistently produced 196 or fewer superpixels. The value of 8 was determined empirically through a series of experimental runs testing different compactness parameters. It was identified as the threshold value that consistently ensured the generation of a maximum of 196 superpixels across the entire dataset, a necessary constraint to maintain compatibility with the fixed input size of the standard ViT-B/16 architecture without exceeding its token limit. Figure 2 shows the superpixels generated by SLIC on a retinography image with a compactness value of 8, and as can be seen, 194 superpixels were generated.

Fig. 2.

Superpixels obtained with SLIC with the compactness parameter set to 8.

In the second test, the SLIC compactness value was varied iteratively for each image until the exact number of 196 superpixels was achieved. Figure 3 shows the superpixels generated by SLIC on a retinography image with the compactness value varied. This approach forces the generation of 196 tokens per image iteratively. However, as can be seen by comparing Figs. 2 and 3, the resulting superpixels are less suited to the image structures and are much squarer and more regular in shape. They bear a closer resemblance to the regular tokens employed in traditional ViTs.

Fig. 3.

Superpixels obtained with SLIC with variable compactness parameter.

Two versions of the model were evaluated:

1. SpxViT_fix: using a fixed compactness parameter of 8.

2. SpxViT_var: using a variable compactness parameter automatically adjusted per image. In SpxViT_var, the SLIC compactness parameter is adjusted per image to achieve exactly 196 superpixels; therefore, the number of tokens remains constant across images and batches.

In this work, the number of superpixels K is fixed to 196 to match the ViT-B/16 token sequence length; therefore, K is not treated as a free hyperparameter. Superpixel generation is performed independently for each image as a preprocessing step and introduces additional computational cost compared to standard ViT patch tokenization. In our experiments, the average preprocessing time per image was 0.14 ms for ViT, 50.96 ms for SpxViT_fix, and 129.32 ms for SpxViT_var. Although superpixel computation introduces a non-negligible preprocessing overhead, this cost is incurred only once per image and does not affect model inference time. In clinical or offline analysis scenarios, this overhead is acceptable given the substantial gains in interpretability achieved by superpixel-based tokenization.

The superpixels generated for the two SpxViT model variations are sorted from left to right and from top to bottom, with the value of the top-left corner of each superpixel taken into account. This sorting method is similar to that used for the ViT-B/16 model.

SpxViT architecture

SpxViT preserves the original ViT-B/16 backbone²¹, ensuring that differences in performance stem only from the tokenization strategy. Apart from the novel superpixel-based tokenization, the architecture remains unchanged: it employs a 12-layer Transformer encoder with an embedding dimension of 768 and 12 attention heads. Compared to the classic ViT architecture, SpxViT modifies the tokenisation and embedding construction processes to enable irregular tokens to be used in the model.

Like the ViT-B/16 architecture, SpxViT accepts images with dimensions of 224 × 224. Each image is tokenized into a sequence of superpixel embeddings, replacing the standard 16 × 16 patch tokens.

Once the patches have been extracted, each 16 × 16 patch in ViT-B/16 is converted into a 768-value vector by concatenating the RGB values of the 256 pixels. However, in the case of SpxViT, each superpixel is composed of a different number of pixels. This means that flattening its RGB values results in vectors of different sizes. For this reason, vectors are first constructed by concatenating the RGB values of the superpixel pixels. Then, linear interpolation is applied to resize the R, G, and B values to 256 each so that the final vector has 768 values. Unlike traditional methods that use mean pooling, which can blur local features, our interpolation-based flattening preserves the internal signal of the superpixel. In the case of SpxViT_fix, the version of the model that uses a fixed compactness value of 8, if fewer than the desired 196 superpixels are generated, empty tokens are added until that value is reached.

Next, in ViT-B/16, each of the 196 flattened vectors is mapped to the Transformer dimension (768 in ViT-B/16) by a trainable single-layer MLP. In SpxViT, this mapping is performed using a two-layer trainable MLP because we have empirically verified that this improves the model’s accuracy.

To bridge the gap between irregular superpixel shapes and the fixed-size requirements of the Transformer, each token is constructed by flattening the RGB values and applying linear interpolation to reach a standardized length of 768 (256 per channel), as previously mentioned. This ensures that even though superpixels vary in area, their representative embeddings remain consistent in dimension.

From this point onwards, the SpxViT model is identical to the ViT-B/16 model. Standard learnable 1D positional embeddings are then added to the token sequence to retain spatial information, following the original ViT formulation. Even though the boundaries are irregular, the consistency of the superpixel generation across the dataset allows the model to learn the spatial arrangement of the retina during training, effectively mapping each token to its corresponding anatomical region. The final embeddings are then fed into the Transformer’s encoding layers, which consist of alternating layers of Multi-head Self-Attention (MSA) and MLP blocks, as defined in²¹. However, the output layer has been adapted specifically for binary image classification.

To ensure compatibility with the standard ViT architecture and allow for efficient batch processing, our model manages tokens without the need for masking or dynamic batching. First, each superpixel is converted into a fixed-size vector of 768 elements through linear interpolation of its flattened RGB values. Second, a constant sequence length of 196 tokens is maintained across all images: in the SpxViT_var variant, the SLIC parameters are iteratively adjusted to produce exactly 196 superpixels, while in the SpxViT_fix variant, ‘empty tokens’ are used as padding if the algorithm generates fewer than the target number of superpixels. This standardization allows for traditional batch training with a fixed batch size.

The resulting 196 superpixel tokens are processed as a sequence, bypassing the need for a complex graph-like structure. The spatial dependency between adjacent superpixels is implicitly captured through the global self-attention mechanism and the inclusion of learnable positional embeddings, maintaining the model’s efficiency and compatibility with the original ViT-B/16 architecture.

Figure 4 shows the SpxViT architecture. In Fig. 4 the superpixels are much larger than those used in practice (see Figs. 2 and 3) but have been drawn this way for clarity.

Fig. 4.

Overview of the SpxViT model. The modifications it introduces to the architecture of the classic ViT model are highlighted in the box.

Explainable technique used: Attention Rollout

Attention-based methods are a type of explainability technique that uses the self-attention mechanism inherent in Transformer architecture to understand how models make decisions during image classification. In Transformer based architectures, the attention matrix generated in each layer of an attention block quantifies the relative importance of each pair of input tokens. In models with Multi-Head Attention (MHA), this results in several attention matrices per layer, which are concatenated and projected through a linear layer. Although direct visualisation of attention weights (’raw attention’) is the most popular and straightforward way of interpreting Transformers, it has been shown to be an unreliable method in isolation due to information being mixed across layers and a lack of ’token identifiability’ in the upper layers²⁷. Raw attention weights tend to become uniform in deeper layers, making it difficult to extract interpretable patterns. Furthermore, they do not always correlate with the actual importance of input tokens.

To overcome the limitations of raw attention, Abnar and Zuidema²⁷ proposed the Attention Rollout method. This method calculates the cumulative matrix product of normalised attention matrices (often with added identity) from the input to a specific layer. This models the flow of information from the input tokens to the CLS class token or another target token. This approach allows the indirect contribution of each token to the model output to be estimated, offering a more comprehensive visualisation of the hierarchical reasoning learnt by the network. Specifically, the Attention Rollout algorithm operates as follows:

1. It begins with the attention matrix of the first layer.

2. For each subsequent layer, it multiplies the attention matrix of the current layer by the Attention Rollout matrix of the previous layer.

3. The rows of the final Attention Rollout matrix are normalised to ensure that the total Attention Flow sums to 1.

Once the final attention rollout matrix has been obtained, the first column is extracted to estimate the self-attention weights of the CLS class token for each remaining token in the sequence (i.e. the other patches comprising the input image). These weights indicate which input patches the model focused on the most when predicting the class represented by the CLS token. Finally, the image is remodelled into its original 2D form, with the weights acting as a saliency map that adjusts the brightness of the image. Since each token in SpxViT represents a specific superpixel, all pixels belonging to a given superpixel are assigned its corresponding attention score. This results in the anatomically-aligned attention maps, which eliminate the typical grid-like artifacts of traditional ViTs. This explainability method was employed in this study to generate the analysed saliency maps.

Experimental design

Objectives of the experiments

This research primarily aims to evaluate the efficiency of SpxViT and examine the effect of its capacity to accept irregular tokens from the input image on this efficiency.

To this end, the classification results obtained using the two SpxViT versions (SpxViT_fix and SpxViT_var) will be compared with those obtained using the classic Vision Transformer version (ViT-B/16)²¹ on which the SpxViT modifications were implemented, as well as with the SPiT method¹⁷.

The SPiT method has been selected for comparison because it is the superpixel-based method most similar to SpxViT. The authors have also made their code available on GitHub²⁸, enabling reproducibility. Although SPiT and SpxViT are both based on the principle of decoupling the tokenisation process from the transformer backbone architecture, SPiT performs tokenisation significantly differently. Firstly, SPiT accepts input images measuring 384 × 384 pixels. Secondly, SPiT constructs vectors by extracting colour, shape, and texture features from superpixels, whereas SpxViT collects the RGB values of superpixels as is done in classic ViTs. Thirdly, SPiT processes a different number of superpixels for each image, whereas SpxViT processes a fixed number of 196 superpixels per image (the same as ViT-B/16).

As the purpose of incorporating superpixels into the ViT structure is to enhance the subsequent interpretability of the model, a second phase of experimentation will be carried out to confirm this. During this phase, attention maps will be constructed using the Attention Rollout technique. These attention maps will be analysed to determine the most effective model for each tested architecture: ViT-B/16, SPiT, SpxViT_fix and SpxViT_var. The aim is to establish which architectural approach produces the most coherent and interpretable maps, and to confirm whether incorporating superpixels into the model assists in identifying regions of the image that are semantically relevant for classification purposes.

Training configuration

A total of 739 retinographs were used in the experiments. 20% of the data was set aside for the test set and the remaining 80% was divided into five equalsized blocks for five-fold cross-validation. Five-fold cross-validation procedures were carried out for each architecture to adjust the hyperparameters, with the aim of optimising performance in terms of accuracy and the loss function. The objective of these experiments was specifically to determine the optimal values for the batch size, learning rate, and number of epochs. Table 1 shows the final hyperparameter values chosen for each architecture.

Table 1.

Training hyperparameters used for each architecture.

Architecture	ViT	SPiT	SpxViT_fix	SpxViT_var
Batch size	16	32	32	32
Learning Rate	1 × 10−6	1 × 10−5	1 × 10−5	1 × 10−5
Optimizer	Adam	Adam	Adam	Adam
Input size	224	384	224	224
Loss function	CrossEntropyLoss	CrossEntropyLoss	CrossEntropyLoss	CrossEntropyLoss
Epochs	500	600	500	500

With the hyperparameter values already adjusted, the five- fold crossvalidation procedure was repeated once more for each architecture, in order to select the best model from among the five resulting models. The aim is to compare the accuracy of the best model from each architecture.

All training and evaluation scripts, including data preprocessing pipelines, are publicly available at²⁹.

Metrics used

In this research, precision, sensitivity, specificity, balanced accuracy, and F1-score were used to evaluate models’ performance³⁰.

Modifications introduced in the models for calculating Attention Rollout

The second phase of experimentation uses the Attention Rollout technique to study the explainability of the models. In order to construct these attention maps, access to the attention matrices of all ViT layers is required.

This study used the PyTorch library³¹. The multi-head attention layers in the predefined ViT-B/16 model cannot be configured to return the attention matrices. Therefore, the ViT-B/16 model was programmed from scratch to enable normal operation and access to the these matrices. SpxViT, based on ViT-B/16, was also developed from scratch for this purpose. As the predefined ViT-B/16 version in PyTorch could not be used, its pre-trained weights could not be used either.

We acknowledge that training models from scratch rather than initialising them with ImageNet pre-trained weights may limit generalisation, but this is the only way to access attention matrices. One advantage is that this approach allows for a more controlled evaluation of interpretability mechanisms, as it enables us to assess whether meaningful attention patterns and interpretable structures could emerge without transfer learning. This is a relevant consideration for small medical datasets.

Results and discussion

Performance comparison

Table 2 presents and compares the experimental results obtained with the proposed SpxViT models and the baseline architectures.

Table 2.

Classification results of the four architectures on the test set. Metrics: accuracy (ACC), sensitivity (Sen), specificity (Spe), balanced accuracy (BAL ACC), and F1-Score.

Model	Fold	ACC	Sen	Spe	BAL ACC	F1 Score
ViT-B/16	1	0.8919	0.8904	0.8933	0.8919	0.8904
	2	0.9189	0.8904	0.9467	0.9185	0.9155
	3	0.9257	0.8767	0.9733	0.9250	0.9209
	4	0.9054	0.8767	0.9333	0.9050	0.9014
	5	0.9189	0.9315	0.9067	0.9191	0.9189
	Mean	0.9122	0.8931	0.9307	0.9119	0.9094
	Std	0.0135	0.0225	0.0318	0.0134	0.0131
SPiT	1	0.8783	0.9452	0.8133	0.8792	0.8846
	2	0.8783	0.8904	0.8533	0.8718	0.8724
	3	0.8918	0.9041	0.8800	0.8920	0.8918
	4	0.8513	0.8904	0.8133	0.8518	0.8552
	5	0.8918	0.9726	0.8133	0.8929	0.8987
	Mean	0.8783	0.9205	0.8346	0.8775	0.8805
	Std	0.0165	0.0368	0.0307	0.0169	0.0172
SpxViT_fix	1	0.8243	0.9178	0.7333	0.8256	0.8375
	2	0.8243	0.8630	0.7867	0.8248	0.8289
	3	0.8919	0.8630	0.9200	0.8915	0.8873
	4	0.8176	0.9589	0.6800	0.8195	0.8383
	5	0.8649	0.8630	0.8667	0.8648	0.8630
	Mean	0.8446	0.8931	0.7973	0.8452	0.8510
	Std	0.0324	0.0438	0.0973	0.0316	0.0239
SpxViT_var	1	0.8446	0.9315	0.7600	0.8458	0.8553
	2	0.8986	0.9178	0.8800	0.8989	0.8993
	3	0.8986	0.8904	0.9067	0.8985	0.8966
	4	0.9189	0.9178	0.9200	0.9189	0.9178
	5	0.8851	0.9726	0.8000	0.8863	0.8931
	Mean	0.8892	0.9260	0.8533	0.8897	0.8924
	Std	0.0277	0.0300	0.0699	0.0272	0.0228

The ViT-B/16 architecture demonstrated the highest overall mean performance achieving the best mean results for accuracy, specificity, balanced accuracy and F1-Score. SpxViT_fix generally recorded the lowest mean scores across all primary metrics. Analyzing each model individually reveals the following observations:

• VIT-B/16 is the model that consistently performs best overall and is the most balanced in terms of sensitivity and specificity. It demonstrates high overall accuracy, ranging from 89.19 to 92.57%. The model shows an impressive ability to identify positives, with sensitivity values spanning 87.67% to 93.15%. Notably, the fold 3 model exhibited exceptional specificity, reaching 97.33%, indicating an outstanding capacity to identify negatives. Its F1 scores, ranging from 89.04 to 92.09%, also demonstrate solid performance.

• SPiT offers slightly lower levels of accuracy than VIT-B/16, with values ranging from 85.13% (fold 4) to 89.18% (folds 3 and 5). It demonstrates an extremely high ability to detect positives, with sensitivity values ranging from 89.04 to 97.26%. However, its specificity is significantly lower than its sensitivity, ranging from 81.33 to 88.00%. Its F1 scores (85.52%–89.87%) are competitive but not as high as VIT-B/16’s.

• Among the four models, SpxViT_fix shows the lowest overall accuracy values, ranging from 81.76% (fold 4) to 89.19% (fold 3). Like SPiT, it has high sensitivity, with values ranging from 86.30 to 95.89%. However, its specificity is the lowest among the models, varying from 68.00 to 92.00%. The value of 68.00% in the fold 4 model is particularly low, indicating considerable weakness in identifying negatives. Its F1 scores (82.89%–88.73%) are generally the lowest, reflecting the imbalance between precision and recall due to low specificity. Overall, SpxViT_fix is clearly the model with the weakest performance, particularly regarding specificity.

• SpxViT_var demonstrates high accuracy, with values ranging from 84.46% (fold 1) to 91.89% (fold 4). It has a high sensitivity ranging from 89.04 to 97.26%, the highest value recorded in the table alongside SPiT. Its specificity ranges from 76.00 to 92.00%. This is better than that of SPiT and SpxViT_fix but is generally slightly lower than that of VIT-B/16. It achieves F1-score values ranging from 85.53 to 91.78%. The value of 91.78% (fold 4) is one of the best F1 scores in the table and almost matches VIT-B/16’s maximum score. SpxViT_var therefore offers a solid performance, providing a promising balance between sensitivity and specificity that is superior to that of SPiT and SpxViT_fix and often approaching the performance of VIT-B/16, particularly with regard to the F1-score.

Finally, a more detailed analysis of the comparison between architectures is conducted by studying the results of the Balanced Accuracy (BAL ACC) metric, which considers sensitivity and specificity equally. Table 2 shows that ViT-B/16 achieves a balanced accuracy of 92.5%, SPiT achieves 89.29%, SpxViT_fix achieves 89.15%, and SpxViT_var achieves 91.89%. The SpxViT_var model has balanced accuracy values that are most similar to those of ViT-B/16. This is significant because, in this model, the superpixels are much more square and regular in shape, making them more similar to the regular tokens used in ViT-B/16 (see Fig. 3). Figure 5 provides a more comprehensive comparison of the models’ balanced accuracy values.

Fig. 5.

Boxplot of the balanced accuracy values of the four models when classifying the test set: minimum value, maximum value, median, and quartiles 1 and 3.

The ViT-B/16 architecture’s best model outperforms the best models of the other architectures in balanced accuracy, albeit slightly. The introduction of superpixels in the ViT architecture appears to slightly lower its accuracy, although this decrease is not too significant and may be offset if this change is subsequently shown to substantially improve the model’s explainability. In scenarios where the model must be integrated into medical practice, transparent decision-making processes can be as important as diagnostic accuracy, particularly to build specialist confidence. In this sense, a slight loss of performance would only be justifiable if it were accompanied by a substantial improvement in visualisation and interpretation capabilities, an aspect that will be evaluated in the following section.

Explainability study

The second experimental phase analyses the explainability of the best models identified for each architecture using their attention rollout maps. Figure 6 illustrates these maps. White and yellow indicate regions that the model considers most relevant for classification, i.e. regions with higher attention values.

Fig. 6.

Attention maps generated using the Attention Rollout technique for the best models of the four architectures studied: ViT-B/16, SPiT, SpxViT_fix, and SpxViT_var. First row: healthy eye; second row: eye with glaucoma.

For glaucoma diagnosis, the clinically significant regions are the optic disc and the excavation (cup). Therefore, a model trained to detect glaucoma is expected to focus on these areas in both normal and glaucomatous eyes. As can be seen from analysing the maps in Fig. 6, most show high attention values in the optic disc area. Some even emphasise the excavation area more. However, in some cases, the area of attention is located at the bottom of the image, outside the optic disc area. To determine whether the model classifies according to consistent criteria, it is necessary to precisely quantify where attention falls.

Another important consideration is how easily key image elements can be identified in the attention maps produced. In the ViT-B/16 maps, for instance, the regular grid structure into which the image is divided for processing is clearly visible. While these ’gridded’ maps offer an approximate view of the regions that attract the model’s attention, they sacrifice a lot of resolution. In contrast, architectures that use superpixels produce maps with much more recognisable semantic shapes. This enables the analyst to identify specific structures in the image, thereby improving their interpretation and understanding of the model’s response. It is necessary to conduct a qualitative analysis of how medical experts perceive this aspect. Therefore, this study will conduct a comparative analysis of the attention maps from quantitative and qualitative points of view.

Quantitative study

To quantitatively study which areas of the image attract the attention of the four models, segmented images were used. These were provided by medical experts at the University Hospital of the Canary Islands (see Fig. 1). Therefore, the distribution of heat maps in these three regions can be studied quantitatively.

Firstly, as we are interested in analysing locations with the highest attention values, a threshold of 70% is applied to all attention maps, with values below this threshold being discarded, as shown in Fig. 7. This threshold value identifies the most significant attention.

Fig. 7.

Original image and attention maps generated using the Attention Rollout technique for the best models of the four architectures studied: ViT-B/16, SPiT, SpxViT_fix, and SpxViT_var (first row) and thresholding of each map at 0.7 (second row).

Next, the intersection of these thresholded maps with the segmented regions of the original image (cup, disc, and background) is calculated, as shown in Fig. 8. The distribution of attention of each map can be calculated by counting the number of pixels of these maps located in each region studied (cup, disc and background). In the example shown in Fig. 8, this calculation results in the following:

• ViT-B/16: 2.44% attention on the cup, 96.30% on the disc and 1.26% on the background.

• SPiT: 13.07% attention on the cup, 28.51% on the disc and 58.42% on the background.

• SpxViT_fix: 28.90% attention on the cup, 68.71% on the disc and 2.39% on the background.

• SpxViT_var: 2.49% attention on the cup, 93.57% on the disc and 3.94% on the background.

Fig. 8.

First row: segmented image and thresholded attention maps. Second row: intersection of the thresholded maps with the segmented image.

This methodology was used to calculate the distribution of attention of the attention maps generated by the four model architectures for the 148 images of the test set. Table 3 shows the mean and standard deviation of the attention distributions of all maps in the three regions considered (cup, disc and background) by architecture, to provide an overview of how attention is distributed.

Table 3.

Mean and standard deviation of the attention distribution in the three regions studied (cup, disc and background) of the attention maps of the four models on the test set (148 images).

Region	ViT-B/16		SPiT		SpxViT_var		SpxViT_var
	Mean	Std	Mean	Std	Mean	Std	Mean	Std
Cup	39.86%	29.87	39.16%	26.36	61.43%	33.36	40.90%	34.13
Disc	55.34%	29.87	34.81%	20.15	33.04%	30.06	52.00%	31.79
Background	4.80%	11.33	26.03%	23.68	5.53%	11.04	7.10%	13.05

Table 3 shows that SpxViT_fix has the highest average attention to the cup (61.43%). This suggests that this model focuses a significantly higher proportion of its attention on this region than the other models do. The other models (ViT-B/16, SPiT and SpxViT_var) distribute their attention more evenly across the cup, with values ranging from 39.16 to 40.90%. There is considerable variability in how attention is distributed in the cup from one image to another or between different attention maps, as indicated by the generally high standard deviations for attention in the cup (ranging from 26.36 to 34.13).

The ViT-B/16 model pays the most attention to the disk on average (55.34%). SpxViT_var also pays considerable attention to the disk (52.00%). By contrast, SPiT (34.81%) and SpxViT_fix (33.04%) show significantly less average attention to the disk, suggesting that these models prioritise this region less or distribute their attention more broadly elsewhere. The standard deviations on the disk are also high, indicating variability in the distribution of attention.

Most models tend to pay little attention to the background, which is desirable given that background information is generally less relevant to medical analysis. However, SPiT pays considerably more attention to the background than other models, 26.03%, with a standard deviation of 23.68. This suggests that it is less efficient than the other models at ignoring noise or irrelevant areas. In contrast, ViT-B/16 (4.80%), SpxViT_fix (5.53%) and SpxViT_var (7.10%) pay minimal attention to the background. This indicates that these models are more effective at focusing on regions of interest. The standard deviation for the background is lower for ViT-B/16 and SpxViT_fix (around 11.04–11.33) than for SPiT.

This quantitative study shows that there is considerable variation in how different models distribute their attention across the cup, disc and background regions. While SpxViT_fix appears to focus primarily on the cup, ViT-B/16 and SpxViT_var demonstrate a greater focus on the disc. SPiT devotes the highest proportion of attention to the background, which suggests that the model may not be making decisions based on reliable factors.

From a medical point of view, the main concern is to check that the model focuses on the anatomical structures that are most relevant for diagnosis: the cup and the optic disc. This is because the cup-to-disc ratio is a crucial indicator of diseases such as glaucoma. Any part of the image that does not correspond to the cup or disc is irrelevant information, or ’noise’, for the diagnosis. A reliable model should largely ignore this information. For this reason, SPiT is the least desirable of the four models following this explainability study.

On the other hand, SpxViT_fix exhibits the highest average attention to the cup (61.43%). While this is excellent for cup detection, it pays the least attention to the disc among the relevant models (33.04%). A medical expert needs to evaluate both structures (cup and disc) to gain a complete understanding. A disproportionate focus on the cup could cause the model to overlook critical details in the disc or the relationship between the two structures.

The ViT-B/16 and SpxViT_var models distribute their attention evenly between the cup and disc, which is very positive. In addition, the average attention given to the background is very low in both models. This suggests that these models are highly efficient and able to ignore noise while maintaining a strong focus on the two structures of interest. From a quantitative point of view, these two models are the most consistent.

Qualitative study

Another important consideration regarding these attention maps is whether they can be easily interpreted by medical experts. To address this issue, a practical evaluation was carried out, in which a specialist physician was provided with a sample of attention maps from 20 retinographs, split evenly between ten normal eyes and ten eyes with glaucoma. Four attention maps were generated for each retinography, similar to those shown in Fig. 6.

The expert ranked the attention maps generated with each method, from the point of view of interpretability, from best (score = 1) to worst (score = 4), taking into account the following criteria: 1) the concentration of attention in regions clinically important for the diagnosis of glaucoma or normality (mainly the neuroretinal ring and papillary excavation), 2) consistency of the map’s heat pattern with the morphology of the optic nerve, and 3) absence of artificial patterns or artifacts that do not correlate with actual characteristics of normality or glaucoma.

Table 4 shows the interpretability scores assigned by the medical expert to the attention maps generated by each of the four models evaluated. The images from this study and the score given to each attention map by the expert are provided in the Supplementay Information.

Table 4.

Interpretability score (best = 1, worst = 4) of the attention maps for each model, as assigned by the medical expert.

Model	Score = 1 (best)	Score = 2	Score = 3	Score = 4 (worst)
ViT	0	2	4	14
SPiT	2	11	5	2
SpxViT_fix	17	0	1	2
SpxViT_var	1	7	10	2

As shown in Table 4, the following points can be highlighted:

• ViT showed the lowest performance in terms of interpretability. 14 of its 20 maps were classified with the worst score (4), and none received the best score (1). This suggests that, for the most part, the medical expert found ViT’s attention maps difficult to interpret.

• SPiT showed a more balanced performance. 11 of its maps received a score of 2, and only two were classified as the worst (score 4). It received two scores of 1 (the best score) and five scores of 3, indicating moderate to good interpretability in most cases.

• SpxViT_fix notably stands out in terms of interpretability. 17 of its 20 maps received the best score (1), none received a score of 2, and only one received a score of 3. Two received a score of 4. These results suggest that the attention maps generated by SpxViT_fix were by far the easiest and clearest for the medical expert to interpret.

• SpxViT_var achieved an intermediate performance. Most of its maps were concentrated in scores of 2 and 3 (7 and 10, respectively), with only one receiving the highest score of 1 and two receiving the lowest score of 4. This indicates that its interpretability was acceptable, but not at the level of SpxViT_fix.

In summary, Table 4 highlights that SpxViT_fix was the model whose attention maps proved to be the most interpretable for the medical expert, obtaining the highest score in the “best” category. In contrast, ViT generated the least interpretable maps. These findings confirm that superpixel-based methods offer superior interpretability to regular token-based models.

Furthermore, the robustness of the superpixel generation process was evaluated across a wide range of image qualities. As shown in the Supplementary Material, the SpxViT model maintains its ability to delineate the optic disc and cup even in images with suboptimal illumination or noise. Specifically, in Table A.1 Healthy eyes, in the first and third rows, even with variations in lighting and contrast, the SpxViT_fix model correctly identifies the relevant clinical structures, receiving the highest interpretability score from the expert. Similarly, in Table A.2 Glaucomatous eyes, in the sixth row, the model effectively handles glaucomatous eyes where the optic disc boundaries are more challenging to define. This confirms that the SLIC-based tokenization reliably captures anatomical structures regardless of common artifacts in fundus photography.

Conclusions

This study presented SpxViT, a superpixel-based Vision Transformer designed to enhance the interpretability of ViT architectures for glaucoma screening from fundus images. By replacing traditional square patch tokenization with superpixel-based segmentation, the model generates semantically coherent tokens that better reflect the anatomical structures of the optic disc and cup. Rather than introducing a new architecture, this work highlights how modifying the tokenization strategy of existing transformers can substantially improve interpretability without sacrificing accuracy.

In the first experimental phase, a performance comparison between the four studied architectures (ViT-B/16, SPiT, SpxViT_fix and SpxViT_var) was carried out. Experimental results demonstrated that ViT-B/16 performed best overall, achieving 92.5% balanced accuracy. SpxViT_var performed similarly, achieving 91.89% balanced accuracy.

The second experimental phase focused on studying the explainability of the models using the Attention Rollout technique. This involved quantitative and qualitative analysis of the distribution of attention in clinically relevant regions: the cup, the disc and the background. From a quantitative point of view, ViT-B/16 and SpxViT_var were the most consistent models, presenting an excellent balance of attention between the cup (around 40%) and the disc (around 50–55%), as well as very low and desirable attention in the background (less than 7.1%). This suggests that these models are highly efficient at focusing on structures of interest while ignoring irrelevant information.

From a qualitative point of view, it was observed that the attention maps obtained with ViT-B/16 have a grid structure, which makes them difficult to interpret. In contrast, architectures that use superpixels generate maps with shapes that are much more easily recognised from a semantic point of view. This makes it easier for medical experts to recognise specific structures in the image, thereby improving the interpretation and understanding of the model’s response. In this regard, SpxViT_fix is notable for its high interpretability; its attention maps were the easiest for a medical expert to understand.

In short, the introduction of superpixels to the ViT architecture may slightly decrease classification accuracy compared to ViT-B/16, as observed in balanced accuracy. However, this is offset by a substantial improvement in model explainability. The SpxViT_var model is a highly promising option, offering competitive classification performance and, importantly, a distribution of attention that is quantitatively consistent with medical criteria (strong focus on the cup and disc, and minimal attention to the background). It is also more interpretable for specialists due to the semantic nature of superpixels. This balance between accuracy and transparency is essential for fostering trust and effectively integrating AI into the clinical diagnosis of glaucoma.

Although image quality can affect any segmentation-based method, the use of a sufficient number of superpixels (196) ensures that the optic disc is reliably captured. The inherent contrast of the optic nerve head allows the SLIC algorithm to maintain semantic boundaries even in suboptimal imaging conditions. This reliability is reflected in the high interpretability scores assigned by the ophthalmologist, who found the SpxViT attention maps to be more aligned with diagnostic features than those of grid-based models.

Despite its overall superior performance in clinical interpretability, SpxViT is not exempt from failure cases. As shown in the Supplementary Information, Table A.1 Healthy eyes, in the sixth and eighth rows, the attention maps generated with SpxVit_fix are the lowest rated by the specialist. In these images, the superpixels do not clearly separate the excavation of the neuroretinal ring in healthy optic discs, and this can lead to confusion as it resembles the attention map of a glaucomatous optic disc.

Overall, SpxViT contributes to bridging the gap between interpretability and performance in deep learning-based ophthalmology. By grounding model reasoning in clinically meaningful regions, it fosters transparency and trust in AI-assisted glaucoma diagnosis.

Despite its promising results, this study has several limitations:

1. Dataset size: the total number of images (739) is limited, potentially restricting generalization.

2. Training from scratch: models were not pretrained on large-scale datasets (e.g., ImageNet), which could limit their generalization capacity.

3. Fixed number of superpixels: in this study, the number of superpixels is fixed to 196 in order to preserve full compatibility with the ViT-B/16 architecture and enable a controlled comparison. As a result, the impact of varying on classification performance and attention quality is not analyzed and remains an open direction for future work.

4. Computation cost: the SLIC preprocessing step increases the total computation time compared to classical patch extraction. Specifically, average preprocessing time per image increases from 0.14 ms (ViT) to 50.96 ms for SpxViT_fix and 129.32 ms for SpxViT_var. Although this represents a non-negligible overhead, it is incurred only once per image and does not affect the model’s core inference time. In a clinical context, this additional fraction of a second is an acceptable trade-off for the substantial improvements in model explainability and diagnostic trust.

5. Variations in image exposures: no dedicated illumination normalization was applied, which may affect attention localization in images with strong exposure variations. While this reflects real-world acquisition conditions, such variability may influence superpixel segmentation and attention distribution, particularly in underexposed or overexposed images.

6. Subjective interpretability scoring: although expert ratings were consistent, a larger pool of clinicians could provide more robust qualitative validation.

7. Generalization: the method was validated on fundus images only.

Nevertheless, the design principles behind SpxViT are broadly applicable. Superpixel-based tokenization could enhance interpretability in other medical imaging domains such as diabetic retinopathy, macular degeneration, or cardiovascular imaging, where understanding regional focus is essential. Future work will explore three directions:

1. Investigating how to minimise the small performance gap between SpxViT_var and ViT-B/16 without compromising the interpretability improvements offered by superpixels.

2. Conducting multi-expert interpretability studies to evaluate the consistency of attention explanations across different clinicians.

3. The model’s performance on completely unseen datasets from different clinical environments or captured with different fundus cameras remains to be tested. Future work should focus on external validation using large-scale, multicenter datasets to ensure the robustness and clinical reliability of the SpxViT architecture in diverse real-world settings.

4. Investigating how the pre-trained weights of the ViT-B/16 model could be adapted for the SpxViT architecture could significantly improve its performance and generalisation.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

Conceptualization: Jorge Hernández, Silvia Alayón, José F. Sigut. Data curation: Tinguaro Díaz-Alemán. Formal analysis: Jorge Hernández, Silvia Alayón, José F. Sigut. Methodology: Jorge Hernández, Silvia Alayón. Software: Jorge Hernández. Supervision: Silvia Alayón. Validation: Tinguaro Díaz-Alemán. Writing—original draft: Jorge Hernández, Silvia Alayón.

Funding

Work co-financed by the Canary Islands Agency for Research, Innovation and Information Society of the Regional Ministry of Economy, Knowledge and Employment, and by the European Social Fund (ESF) Integrated Operational Programme of the Canary Islands 2014–2020, Axis 3 Priority Theme 74 (85%). Grant number TESIS2022010056. Work co-financed by Project (PROID2024010027) of the Canary Islands Agency for Research, Innovation and the Information Society (ACIISI) and by the European Regional Development Fund within the framework of the FEDER programme Canarias 2021–2027.

Data availability

One of the datasets used in this study is the Rim-ONE DL dataset, which is publicly available at https://github.com/miag-ull/rim-one-dl. The other dataset, which was compiled by medical experts at the University Hospital of the Canary Islands, is not publicly available due to ethical and privacy restrictions. However, it can be provided upon reasonable request to the corresponding author and with the permission of the hospital’s Ethics Committee.

Declarations

Competing interests

The authors declare no competing interests.

Ethics approval and consent to participate

This study was conducted in accordance with the Declaration of Helsinki and approved by the Research Ethics Committee of the Canary Islands University Hospital (CHUC_2023_41, 27 April 2023). Confidentiality of personal data was guaranteed. Written informed consent was obtained from all participants involved in the study.

Patient consent for publication

Informed consent was obtained from all subjects involved in the study.