A fast fully automated approach for evaluating calcified lesions in intravascular ultrasound

Abstract

Aims

Catheter-based coronary intervention is an effective treatment for acute coronary syndrome. However, calcified plaques pose significant challenges within these procedures, as they complicate stent deployment and passage of devices. This study presents novel research for calcium detection in intravascular ultrasound data dealing with weak labelling.

Methods and results

Encoding–decoding network architecture is adapted to predict the angle-wise existence of calcification in intravascular ultrasound (IVUS) data, enabling the interpretation of predictive behaviour by observing the attention map at the penultimate layer. We explore several practical factors, including the training of five candidate models and their average ensembling, enhancing dataset diversity, and optimizing filter size for post-processing. The algorithm is developed using stored DICOM data from 42 patients. Employing a five-fold cross-validation training strategy, we achieve an average accuracy of 0.90 and a Dice score ranging of 0.88 in testing performance. The attention map shows that the trained model learned to use image features to support its decisions, resembling the manner of trained IVUS experts. Comparing the computed calcium scores to the ground truth confirms that using average resembling followed by post-processing with a filter size of 61 A-lines and 13 frames yields the best performance. The window is surprisingly larger than what is typically used in the literature, likely due to its high noise ratio in IVUS images.

Conclusion

The detection and quantification pipeline holds promise for accelerating clinical research, particularly in elucidating the impact of calcified plaques on intervention procedures and prognosis. It may also serve as a powerful tool to assist in decision-making during interventions.

Introduction

Minimally invasive catheter-based revascularization is the most commonly employed treatment for coronary artery diseases (CAD). In percutaneous coronary intervention (PCI), a stent is implanted into the coronary artery segment, where impaired blood flow causes symptoms. The stent is expanded towards the artery wall by inflating a high-pressure balloon, pushing aside any obstructing material, such as atherosclerotic plaque or thrombus. Percutaneous coronary intervention aims to achieve a patent open lumen and a fully expanded stent with sufficient radial strength to support the vessel wall. Calcified plaques are non-compliant and can limit balloon expansion, leading to a smaller lumen area and stent under-expansion.^1,2,3 Coronary calcification can also complicate the use of interventional devices.^4,5

Intravascular ultrasound (IVUS) is a catheter-based imaging technique that visualizes the vessel wall using high-frequency sound waves. Most IVUS systems use a rotating single-element transducer. Because hard calcium strongly reflects acoustic waves, calcified plaques appear in IVUS as bright linear features, resulting from the reflection of the ultrasound pulse off the surface of the calcium, and an adjacent shadow region where the ultrasound pulse is attenuated and no echoes are generated (see Figure 1A). This distinct pattern makes IVUS data highly suitable for the automatic detection and quantification of calcified coronary plaques.

Figure 1.

The acquired images were labelled every 1 mm and transformed into polar coordinates, which were fed to train the Detection Module. The module generated outputs of 0s (non-calcified) and 1s (calcified). The two cyan-coloured lines in the Acquired IVUS Image span the drawn angle of a calcified lesion, and the red line marks the cutting plane with which the longitudinal view was generated to demonstrate frame selection in the lower panel. The angle drawing always centres on the image centre, which is also the centre of the imaging catheter. The Detection Module contains a U-Net and a sum unit.

Previous efforts to identify coronary calcium in IVUS data have explored a range of methods. For instance, automatic thresholding was employed to quantify the angle of calcification,⁶ yielding a sensitivity of 84% and specificity of 72%. This algorithm was developed using images of 14 human left anterior descending coronary arteries acquired in vivo during PCI. In another study,⁷ leveraging prior knowledge that calcified plaque of interest is typically deposited only in the intimal layer, a deformable geometric model was applied to detect the front and back borders of the intima. Subsequently, a Bayesian classifier was utilized to identify calcium in the region in between, achieving a sensitivity for coronary calcium detection ranging from 85% to 90% in data obtained from seven patients. We previously⁸ adopted a step-wise approach, extracting statistical features and combining them using a radial basis function support-vector classification model. This methodology was validated using data from three vendors, with 35 pullbacks from each. Sensitivity rates were reported to be 77% in Infaredx data, 90% in Volcano data, and 85% in Boston Scientific data. Moreover, a deep-learning plaque characterization framework was trained using data from 598 patients, achieving a sensitivity of 86% and specificity of 97% specificity.⁹ In a comparative study,¹⁰ U-Net and DeepLabV3 were evaluated for calcium segmentation, revealing that the multi-task learning approach with auxiliary data of lumen and vessel contours outperformed the single-task learning approach and transfer learning.

The aforementioned studies focused on detecting calcium at either the pixel level or the region level. Notably, some studies concentrated on detecting calcified plaque by identifying it at the image level. Sofian et al.  ¹¹ employed the Directed Acyclic Graph network to extract image features and perform classification for detection. They evaluated three types of classifiers, demonstrating exceptional performance, particularly in the analysis of 20MHz IVUS images. Furthermore, they extended their investigation by comparing InceptionResNet-V2 and DenseNet-201 for feature extraction and evaluating the performance of seven classifiers for detection in another study.¹² In a separate study by Lee et al.,¹³ images were partitioned into small patches labelled with or without dense calcium. Image features were extracted from each patch, and the dimensions were further reduced using principal component analysis. Subsequently, a deep learning network was trained to detect tissues with dense calcium, achieving a sensitivity of 92.8% and specificity of 85.1% in images obtained from 26 patients.

Previous efforts have explored the capabilities of deep learning networks for detecting calcified lesions, employing datasets ranging from tens to thousands of patients and utilizing up-to-date deep learning models. In fact, calcified lesions are often labelled based on angle in clinical research; for each direction from the catheter centre, the image vector is marked as calcified or not (see Figure 1A–C). There is a need for a method to handle this weak-labelling task.

In this study, we address the weak-labelling challenge by proposing a novel solution that employs a deep learning network. Our approach integrates an encoding-decoding network structure with a summation layer. By training this network, the penultimate layer generates an attention map, which is then used to predict angle-wise labels. This also allows us to interpret the network’s learning behaviour through the attention map. The proposed method ensures high accuracy while maintaining low computational time. We further demonstrate the practical application of our detection results in extracting quantification metrics used in clinical analysis.

Methodology

We applied minimal pre- and post-processing steps and focused on interpreting the training process so that the algorithm could learn the most patterns from the currently available data for this study. The main processing pipeline includes image transformation, deep-learning detection, morphological operations, and quantification of imaging biomarkers. An overview of the proposed workflow, from raw IVUS data to quantitative biomarker extraction, is illustrated in the Graphical abstract.

Data acquisition

Following the IVUS convention, the 1-D echo signal collected from each direction is referred to as the A-line. By rotating the catheter, the A-lines are collected circumferentially to form a signal array, which is referred to as a B-scan. A stack of multiple B-scans was acquired by pulling the imaging tip backward, from distal to proximal, inside the vessel during image acquisition, at a constant speed. The resulting volumetric dataset is called a pullback. For visualization, B-scans are usually transformed into Cartesian coordinates (‘scan conversion’; see Figure 1A)¹⁴ and saved as a stack of images, one for each frame in the pullback, in DICOM format. These pullbacks were used in this study. All IVUS pullbacks were acquired in the native coronary arteries of patients undergoing PCI at the Erasmus MC. Data were retrospectively collected from the clinical database of the Department of Cardiology, Erasmus MC. We selected the data only to include the pullbacks with moderately or heavily calcified lesions acquired with the Boston Scientific imaging station (40 MHz, Atlantis SR Pro, and OptiCross Coronary Imaging Catheter, Boston Scientific Corp., Natick, MA, USA). In all datasets, the frame rate was 30 frames/s and the pullback speed was 0.5 mm/s. We first selected pullbacks from 35 patients for primary development. We further included seven more pullbacks with heavy calcium as an enhancing dataset.

The initial developing data set is denoted as ${\mathcal{D}}_1$, and enhancing data is denoted as ${\mathcal{D}}_2$ Dataset ${\mathcal{D}}_1$ contains 1654 labelled images ($47.26\pm 22.28$ over pullbacks) out of 35 pullbacks. Dataset ${\mathcal{D}}_2$ contains 231 labelled images ($33.00\pm 2.98$) generated from seven additional pullbacks with heavily calcified lesions.

The Institutional Review Board of the Erasmus Medical University Center waived ethical approval for the present study due to the retrospective nature of the data used (MEC-2019-0731). Each pullback was anonymized such that our process was compliant with the European General Data Protection Regulation, and no identifying information was accessible to the algorithm. An example of the acquired IVUS image is shown in Figure 1A. The calcified plaque was labelled in the cross-sectional images by an expert drawing angular wedges in the QCU-CMS (LKEB, Division of Image Processing, Leiden, The Netherlands). Each pullback was labelled using a constant frame selection, i.e. selecting 1 image to label per 1 mm. All images were labelled by two experienced observers (TN and AZP) with extensive expertise in IVUS interpretation. The observers strictly followed a standardized annotation protocol and jointly reviewed challenging cases. In instances of disagreement or uncertainty, a third senior observer (JD) adjudicated the final label. The resulting consensus served as the reference standard for model training and evaluation. This consensus-based scheme ensures that the algorithm learns to recognize the patterns defined in the consensus protocol rather than the specific labelling behaviour of a single observer.

Image transformation

A calcified lesion in an IVUS A-line features a cluster of bright rich image intensities along the lumen structures, followed by a low signal, where the calcium reflection casts an acoustic shadow on deeper-lying tissues. Because the IVUS image is initially acquired in polar coordinates and later transformed into Cartesian coordinates using interpolation for convenient interpretation, this trait can be better represented in polar coordinates. However, the raw data in the polar domain are not available; therefore, we transform the stored Cartesian images back into the polar domain, where each one-dimensional signal (A-line) receives one label, that is, 1 (calcified) or 0 (non-calcified) (see Figure 1E).

Detection using deep learning

In the following, we denote the dataset of polar images normalized into a scale of $[0,1]$ as a bitmap of dimension $N_f\times N_a\times N_d$ (number of frames × number of A-lines × number of pixels in depth) and resolution $r_f\times r_a\times r_d$ in millimetres (mm). $N_a$ and $N_d$ were chosen to be fixed values for all images. The upper and lower parts of the polar image were located at the seamline in the Cartesian domain. To ensure the detection of naturally distributed calcium at this seamline, we applied padding with $N_k$ circular repeating A-lines to both image and ground-truth labels. We applied the U-Net segmentation backbone deep learning network to fulfil the purpose of this detection task (see Figure 1D). Taking padded polar images with a size of $(N_a+2N_k)\times N_d$ as input, U-Net backbone outputs an image as the attention map of the same size ($(N_a+2N_k)\times N_d$).

To output the label per A-line rather than per pixel, we aggregate the values from all pixels for each A-line by summing them, and then applying a sigmoid activation function (Equation (1)). The model generated outputs for all A-lines in each image in a dimension of $(N_a+2N_k)\times 1$.

$$\mathcal{S}(x)=\frac{1}{1+e^{-x}}.$$ (1)

Concretely, if we denote the output of the U-net as ${\mathcal{Q}}^{\theta }=\{{\mathcal{Q}}_{\alpha ,\delta }^{\theta }{\}}_{\alpha \in {\mathcal{A}}_0,\delta \in \mathcal{D}}$, where $\mathcal{Q}$ is the U-net backbone with trained parameters θ taking the input image with a dimension of $(N_a+2N_k)\times N_d$, α is the A-line index from the set ${\mathcal{A}}_0=\{\alpha \le N_a+2N_k|\alpha \in {\mathbb{N}}^{+}\}$, and δ is the depth index from the set $\mathcal{D}=\{\delta \le N_d|\delta \in {\mathbb{N}}^{+}\}$. The output of the overall model is:

$${\mathcal{U}}^{\theta }=\{{\mathcal{U}}_{\alpha }^{\theta }{\}}_{\alpha \in \mathcal{A}}$$ (2)

$${\mathcal{U}}_{\alpha }^{\theta }=\mathcal{S}\left({\displaystyle \sum _{\delta \in \mathcal{D}}}{\mathcal{Q}}_{\alpha ,\delta }^{\theta }\right).$$ (3)

The output of the padded image will be recovered to fit the size of the original image by taking a subset of ${\mathcal{A}}_0$, i.e. $\mathcal{A}=\{N_k<\alpha \le N_a+N_k|\alpha \in {\mathcal{A}}_0\}$.

It is worth noting that the computational step listed in Equation (3) does not involve any trainable parameters. Therefore, the model is primarily trained to adjust the scalar ${\mathcal{Q}}_{\alpha ,\delta }^{\theta }$ at each pixel in the attention map to contribute to the final determination of calcified lesions. The magnitude of the scalar indicates the model’s confidence in categorizing the A-line, while the sign of the scalar suggests whether it is positively or negatively associated with a calcified plaque.

Five-fold cross validation

A five-fold cross-validation was applied at the pullback level. This avoids the reporting bias caused by involving similar images from the same pullback in both training and testing. As shown in Figure 2A, pullbacks were randomly split into five folds, each containing seven pullbacks. In turn, three folds were used for training, one fold was used for validation, and one fold was used for reporting testing performance. The training for each training-validation-testing data split was performed multiple times (5×) using random initialization, and the trained model with the top performance in the validation data was chosen to report the performance in the testing data.

Figure 2.

Data splitting and model training for (A) reporting results in the testing data; (B) choosing the final prediction model.

Final model selection

The final model was trained using four folds and one fold for validation (Figure 2B). For each training-validation data combination, training was repeated multiple times (5×) and the model with the best validation performance was chosen as the candidate model. Repeating the training using five different combinations of training and validation data resulted in five candidate models, which were merged by averaging the attention layers of the trained model followed by sigmoid activation to obtain the final prediction.

Post-processing

Training the model using ground-truth data will enable the model output for each A-line to gradually become very close to 0 or 1. To obtain a binary detection image, we introduce a threshold probability $p_0$, such that:

$${\mathcal{U}}_{\varphi ,\alpha }^{\theta ,p_0}=\{\begin{array}{cc}1& \text{if}{\mathcal{U}}_{\varphi ,\alpha }^{\theta }\ge p_0\\ 0& \text{otherwise}.\end{array}$$

Here, we introduce the variable ϕ to represent the frame index. The overall result for one pullback is a 2-D matrix with angular location and frame location, where $\varphi \in {\mathcal{F}}^{\prime }=\{\varphi \le N_f^{\prime }|\varphi \in {\mathbb{N}}^{+}\}$. The number of frames $N_f^{\prime }$ may differ for each pullback. As the network was trained to optimize the Dice loss, the predicted probabilities tended to cluster near 0 or 1. Therefore, a threshold of 0.5 was empirically used to obtain binary predictions, consistent with standard practice in binary segmentation models. We further refined the result ${\mathcal{U}}^{\theta ,p_0}=\{{\mathcal{U}}_{\varphi ,\alpha }^{\theta ,p_0}\}$ using morphological closing and opening. The former removes small holes whereas the latter removes unreasonably small objects. Both operations apply a predefined rectangular structuring element, which covers $n_a$ A-lines and $n_f$ frames.

Evaluating matrices

The performance in the testing data was reported in Dice indices (Dice), accuracy (Accu), precision (Prec also called positive predictive value), recall (Reca also called true positive rate), and ${\text{F}}_1$-score ($F_1$). The Dice index was calculated per frame and reported as the mean ± standard deviation. The total accuracy, precision, recall, and ${\text{F}}_1$-score were calculated using A-lines of all frames. For the computation of accuracy, precision, recall, ${\text{F}}_1$-score and Dice score, we used the Boolean values. We denote the ground-truth of frame ϕ as $g_{\varphi }=\{g_{\varphi ,\alpha }{\}}_{\alpha \in \mathcal{A}}$, where $\varphi \in \mathcal{F}=\{\varphi \le N_f|\varphi \in {\mathbb{N}}^{+}\}$, $N_f$ denotes the total number of frames that have been labelled. All the matrices are calculated as follows:

$$\begin{array}{rl}Dic{e}_{\varphi }& =\{\begin{array}{cc}\frac{2{\displaystyle \sum _{\alpha \in \mathcal{A}}}{\mathcal{U}}_{\varphi ,\alpha }^{\theta ,p_0}{g}_{\varphi ,\alpha }}{{\displaystyle \sum _{\alpha \in \mathcal{A}}}({\mathcal{U}}_{\varphi ,\alpha }^{\theta ,p_0}+g_{\varphi ,\alpha })}& ,\mathrm{if}{\displaystyle \sum _{\alpha \in \mathcal{A}}}({\mathcal{U}}_{\varphi ,\alpha }^{\theta ,p_0}+g_{\varphi ,\alpha })\ne 0,\\ 1& ,\mathrm{otherwise}.\end{array}\\ Accu& =\frac{{\displaystyle \sum _{\varphi \in \mathcal{F},\alpha \in \mathcal{A}}}(t{p}_{\varphi ,\alpha }+t{n}_{\varphi ,\alpha })}{N_f{N}_a},\\ Prec& =\frac{{\displaystyle \sum _{\varphi \in \mathcal{F},\alpha \in \mathcal{A}}}t{p}_{\varphi ,\alpha }}{{\displaystyle \sum _{\varphi \in \mathcal{F},\alpha \in \mathcal{A}}}(t{p}_{\varphi ,\alpha }+f{p}_{\varphi ,\alpha })},\\ Reca& =\frac{{\displaystyle \sum _{\varphi \in \mathcal{F},\alpha \in \mathcal{A}}}t{p}_{\varphi ,\alpha }}{{\displaystyle \sum _{\varphi \in \mathcal{F},\alpha \in \mathcal{A}}}(t{p}_{\varphi ,\alpha }+f{n}_{\varphi ,\alpha })},\\ F_1& =\frac{2Prec\times Reca}{Prec+Reca},\end{array}$$

Bio-marker extraction using detected calcification

After calcified plaque detection, the result can be used to extract quantification numbers, including the total IVUS-calcium score (ICS), local IVUS-calcium score within x mm in the neighbouring (${\text{ICS}}_{x-\mathrm{mm}}$), the length of the longest calcified lesion (${\text{LC}}_{max}$), and the existence of calcified lesions under specific conditions, for example, whether there is a calcified lesion distributed more than ${180}^{\circ }$ over several frames covering 2 mm, ${\mathbf{1}}_{2-\mathrm{mm}}^{{180}^{\circ }}$.

• ICS: Following the definition in Ref.⁸, the total ICS is calculated as:

  $$\mathrm{ICS}=\frac{1000}{N_a\cdot N_f}{\displaystyle \sum _{\alpha \in \mathcal{A},\varphi \in {\mathcal{F}}^{\prime }}}{\mathcal{U}}_{\varphi ,\alpha }^{\theta ,p_0}$$ (4)
• ${\text{ICS}}_{x-\mathrm{mm}}$ : The ${\text{ICS}}_{x-\mathrm{mm}}$ is a function of the longitudinal (frame) location that describes the local distribution of the calcified lesion. We used a convolution operation to achieve fast computation.

  $${\mathrm{ICS}}_{x-\mathrm{mm}}(\varphi )=\frac{1000}{N_a\cdot l({\mathcal{W}}_{\varphi ,x})}\times {\displaystyle \sum _{\alpha \in A}}{\displaystyle \sum _{\varphi \in {\mathcal{F}}^{\prime }\cap {\mathcal{W}}_{\varphi ,x}}}{\mathcal{U}}_{\varphi ,\alpha }^{\theta ,p_0},$$

  where ${\mathcal{W}}_{\varphi ,x}=[\varphi -\frac{x}{2r_f},\varphi +\frac{x}{2r_f}]$ is the frame window scanning the detection result matrix with a length of x mm, which corresponds to $l({\mathcal{W}}_{f,x})$ frames.

• ${\text{LC}}_{max}$ : The length of calcified lesions was calculated for each isolated plaque. As the detection results are represented in polar coordinates, obtaining circular connectivity is the main challenge. We introduced a mapping from the polar representation to a 2-D disk representation, such that the 3-D connectivity has a very low computational cost. The mapping is illustrated in Figure 3. The specific steps are as follows.

  1. In Figure 3 Step.1, we show an artificial output array from the deep learning detection network with $N_f$ rows and $N_a$ columns. It can be considered as a tube mesh unfolded along the long edge to form a rectangular mesh. $N_f$ is the number of frames of pullback that varies for each case, and $N_a$ is the number of A-lines for which we used a constant number. The mapping may be illustrated by considering three regions resembling calcified plaques in a simple rectangular shape. As shown, A and C are two independent regions, whereas B.1 and B.2 are supposed to belong to one intact region.

  2. To reconnect B.1 and B.2, a natural method is to fold the rectangular mesh back to the original 3-D tube mesh. However, doing this is very time- and computation-intensive. We propose the use of conformal mapping to recover connectivity and use fewer computational resources. As shown in Figure 3 Step 2, we map it into a disk mesh, where B.1 and B.2 are connected to form an intact region. Each calcified region can then be labelled using a fast-connected component labelling algorithm.¹⁵ To avoid connecting Region A to Region B in Step 2, we first 0-padded the rectangular mesh before mapping (the underneath green-bounded rectangle in Step 1, which becomes a green circle in the centre in Step 2).

  3. The labelled image can then be transformed back into the original space (Step 3). The length of each calcified region was simply the number of frames covered.

  An example using real data is demonstrated in Figure 4, the region coloured in green could be labelled as one calcification in the disk mesh in A.

• ${\mathbf{1}}_{x-\mathrm{mm}}^{{180}^{\circ }}$ : the existence of 180°-calcium within x mm. In clinical studies, this indicator is computed over 2, 4, and 5 mm to analyse the impact of calcified lesions on prognostic outcomes, such as stent failure, and long-term PCI recoveries.⁵ The following equation was applied for this calculation:

  $${\mathbf{1}}_{x-\mathrm{mm}}^{{180}^{\circ }}={\displaystyle \sum _{\varphi \in \mathcal{F}\cap {\mathcal{W}}_{\varphi ,x}}}\left[{\displaystyle \sum _{\alpha \in \mathcal{A}}}{\mathcal{U}}_{\varphi ,\alpha }^{\theta ,p_0}\le \frac{180\cdot N_a}{360}\right]\ge l({\mathcal{W}}_{\varphi ,x})$$

Figure 3.

Domain mapping for the calculation of the length of calcium.

Figure 4.

(A) Detected labels converted in disk mesh and labelled based on the connectivity. (B) Converting the labelled calcified overview into its original rectangular mesh.

Experiment and results

Implementation details

The main structure of the segmentation network was based on the U-Net architecture, following its classical design. Specifically, we used iterative $3\times 3$ convolutional filters in both the encoder and decoder paths, and $2\times 2$ max-pooling layers with a stride of 2 for down-sampling blocks. Zero-padding was applied to all convolutional layers to preserve spatial dimensions. In addition, the input images were preprocessed with circular padding to maintain continuity along the upper and lower boundaries of the polar representation, reflecting the seamless nature of IVUS cross-sectional images. Concretely, the original image with $N_a=512$ A-lines was circularly padded with $N_k=100$ A-lines at both the top and bottom, resulting in an input size of 712 A-lines while keeping the depth dimension unchanged at $N_d=256$. The predicted output had a dimension of $712\times 1$ corresponding to all input A-lines. However, only the outputs associated with the non-padded A-lines were included in the loss computation, resulting in an effective output size of $512\times 1$, and for an entire pullback, the output has a size of $512\times N_f$, which varies for different pullbacks. The model was trained with the Adam optimizer using default settings¹⁶ to optimize the Dice loss. The training was performed for a maximum of 250 epochs. Both training and prediction were performed on an HPZ4 workstation equipped with an Intel XeonW-2235 3.8 GHz 6-core processor, 128 GB ($4\times 32$ GB) RAM, and an NVIDIA GeForce RTX 3090 24 GB GPU.

Experiments

The following validation experiments were performed. Using five-fold cross-validation, we optimized and tested the model using independent datasets. This hints at how well the final model performs approximately on new data in the real world. Subsequently, we achieved the final models using the method described in Final model selection section. Using the final model, we computed the ICS and presented a comparison of the detected result and the ground truth in Bland–Altman plots and Pearson correlation coefficients. We further rationalize the use of the average ensemble by demonstrating its improved performance compared to the majority voting strategy. Furthermore, we attempted to search for an optimal operational window for post-processing. The results are listed below, and the prediction time is reported.

Detection performance of five-fold cross-validation

The results of the testing dataset of five-fold cross-validation are shown in Table 1. Results show that our method achieved an average accuracy of 0.96, Dice scores of 0.88. The precision, recall, and F1-score are 0.90, 0.77, and 0.83, respectively.

Table 1.

Detection performance of five-fold cross-validation in  ${\mathcal{D}}_{\mathbf{1}}$

CV	Prec	Reca	Accu	$F_1$	Dice
0	0.94	0.74	0.96	0.83	0.86 ± 0.26
1	0.84	0.79	0.94	0.81	0.86 ± 0.23
2	0.93	0.83	0.96	0.88	0.90 ± 0.21
3	0.91	0.87	0.98	0.89	0.92 ± 0.19
4	0.88	0.64	0.93	0.74	0.90 ± 0.20
Average	0.90	0.77	0.96	0.83	0.88

The attention map

The attention map is the output of the U-Net backbone, i.e., the penultimate layer of the proposed network. After training, each pixel of this image indicates its contribution to the final decision of the A-line it belongs. This provides insight into whether the algorithm was successfully trained to learn relevant features. Two examples from the testing dataset are shown in Figure 5. By observing the sign and magnitude of each pixel, we confirm that pixels with positive contribution and large magnitude are located in regions with bright sharp edges and dark shadows, which are two key factors defining a calcified lesion. Meanwhile, non-calcium A-lines are distinguished where pixels with negative contribution and large magnitude predominate in regions with rich texture in the deeper regions.

Figure 5.

Visualizing examples of detection in the testing dataset.

Quantifying the calcium score

We computed the total IVUS-calcium score (ICS) using equation (4) using the detection results. By comparing this score to that computed using the ground truth (GT) in a Bland–Altman plot, we experimented with several factors that may improve the performance further, including applying majority voting (MV) or average ensembling (ES), applying post-processing (PP), and enlarge the database ${\mathcal{D}}_1$ + ${\mathcal{D}}_2$. In Table 2, we report calcium scores in different experimental settings, including ground-truth (GT), majority voting (MV), majority voting combined with post-processing (MV+PP), an ensemble combined with post-processing (ES), and an ensemble combined with post-processing (ES + PP) trained using all the available datasets (${\mathcal{D}}_1$ + ${\mathcal{D}}_2$). In ES + PP, the model was trained using the combined dataset, whereas the quantification number was reported only in ${\mathcal{D}}_1$ for a fair comparison. Furthermore, the numbers were compared with the ground truth in the Bland–Altman plot, and the Pearson correlation coefficients are shown in Figure 6A–D.

Table 2.

IVUS-calcium score (ICS)

	GT (${\mathcal{D}}_1$)	MV (${\mathcal{D}}_1$)	MV$+$PP (${\mathcal{D}}_1$)	ES+PP (${\mathcal{D}}_1$)	ES+PP (${\mathcal{D}}_1$ + ${\mathcal{D}}_2$)
mean	163.46	136.76	138.78	155.17	164.15
std	130.39	89.37	92.49	102.04	131.01
min	6.29	3.88	1.79	6.00	0.53
25%	75.80	67.96	67.74	79.42	72.13
50%	144.43	127.31	126.71	141.58	138.80
75%	212.94	183.01	186.49	208.90	223.81
max	718.95	366.29	372.38	437.85	676.84

GT, ground-truth; MV, majority voting; PP, post-processing; ES, ensemble.

Figure 6.

(A) Bland–Altman of ICS comparing MV (${\mathcal{D}}_1$) to GT. (B) Bland–Altman of ICS comparing MV+PP (${\mathcal{D}}_1$) to GT. (C) Bland–Altman of ICS comparing ES+PP (${\mathcal{D}}_1$) to GT. (D) Bland–Altman of ICS comparing ES+PP (${\mathcal{D}}_1$ + ${\mathcal{D}}_2$) to GT.

Post-processing, majority voting, and ensemble

By comparing the quantitative numbers, we observed that applying MV+PP results in a mean slightly closer to the ground truth and lower standard deviation values than applying only majority voting (MV). Using an ensemble (ES) instead of majority-voting (MV) yields mean and standard deviation values much closer to the GT. Eventually, developing an algorithm using expanded data (${\mathcal{D}}_1$ + ${\mathcal{D}}_2$) would result in an even smaller difference. From the Bland–Altman plots, we observed that post-processing improved the correlation coefficient by 0.02 (Figure 6A vs. Figure 6B). The ensemble model provided an ICS that was closer to the ground-truth calcium score and had a higher correlation ($r=0.93$) (Figure 6B vs. Figure 6C).

Optimizing the post-processing

To search for the optimal operational window for post-processing, we applied different sizes of filters using the number of A-lines chosen from $\{21,31,41,\dots ,151\}$, and the number of frames chosen from $\{5,7,9,\dots ,33\}$. The total squared error compared to the ground-truth calcium scores is shown in Figure 7. The optimal filter for post-processing covered 13 frames and 61 A-lines. The final prediction ensemble model was trained using all available data (${\mathcal{D}}_1$ and ${\mathcal{D}}_2$), and the optimal window for post-processing was applied. It provides the highest correlation ($r=0.99$), and lowest bias in the Bland–Altman plot when compared to the ground-truth calcium scores (see Figure 6D).

Figure 7.

Effect of changes in the post-processing structuring element on the logarithm of sum of squared errors of calcium scores.

Computation time

Finally, the prediction times are reported in Table 3. The algorithm performs automated calcium detection in pullbacks containing 709 to 5920 frames in 0.36 to 2.91 min, 0.03 s/frame on average. And a visual segmentation example in stented data is demonstrated in Figure 8.

Table 3.

Predicting time

	Predicting time (min)	Number of frames	Time/Frame (ms)
mean	1.51	3077	29.77
std	0.63	1313	1.71
min	0.36	709	29.04
25%	1.01	2059	29.23
50%	1.50	3064	29.33
75%	1.88	3855	29.62
max	2.91	5920	39.34

Figure 8.

Visualizing examples of detection in stented IVUS, which was not included in the training set. Red pie boundaries mark the detected calcification.

Discussion

Although previous deep learning frameworks achieved excellent detection performance, they primarily focused on pixel- or image-level classification, which is not sufficient for clinical interpretation. In practice, interventional guidance requires quantification of calcium distribution (coverage degree, plaque burden, and length). Moreover, pixel-wise delineation is limited by the acoustic shadowing caused by calcium, which prevents accurate boundary detection. In this study, we developed an automated calcium quantification pipeline in IVUS images dealing with weak labelling. The pipeline comprises a detection module and a computational module. For detection, we used a learning approach combining U-Net backbone with a sum operation to output the detection per A-line. The convolution filters applied in U-Net backbone consider neighbouring information during prediction. Because it is connected to a sum operation, the output of the trained U-Net backbone as an attention map shows intuitively the contribution of each pixel to the final decision.

Compared to the previous SVM method,⁸ our method considered neighbouring information and achieved significantly improved accuracy with a higher predicted true-negative value. We report higher Dice scores compared with a deep learning network trained using data from 598 patients,⁹ where a Dice score of 0.79 was reported. Additionally, our prediction speed was also improved compared to that reported in Ref.⁹ (0.05 s/frame) using a comparable GPU (V100-32G).

The images of features demonstrated in Figure 5 show that the detection model has learned relevant structures for recognizing a calcified lesion: positive areas include both the sharp reflection and the trailing shadow region. In contrast, negative areas are primarily located in the textured deeper layers, which are invisible in the presence of a calcification. Therefore, we are convinced that the detection model was properly trained. Moreover, including seven pullbacks with more circumferential calcium images further enhances the model’s performance. This can be observed by comparing Figure 6A–C with Figure 6D, where one previously underestimated pullback was addressed. This suggests that greater data diversity can improve performance.

The calcification quantification pipeline was designed to involve minimum processing steps. We only applied a coordinate transformation for pre-processing to enable detection in polar images. For post-processing, we applied morphological closing and opening for their simplicity and efficiency to further improve detection when comparing total calcium scores (Figure 6B vs. Figure 6A). We experimented with the window size of the kernel to be applied.

The algorithm was developed to detect calcified lesions within the arterial tunica intima. In our current dataset and evaluations, we did not observe any noticeable differences in detection performance between superficial and deep calcium (see A and B in Figure 8), suggesting that the model captures feature common to both locations. However, for calcifications located outside the internal elastic membrane, the dataset does not contain sufficient examples to assess detection behaviour, and further study would be required to evaluate performance in these cases.

Although guidewire and stent struts may appear similar to calcification in IVUS images, the model demonstrated robust discrimination capability. The proposed model likely captured fine structural cues and contextual information that differentiate these components, consistent with the distinctions recognized by expert observers (see Figures 5 and 8C–F).

The proposed network architecture was intentionally designed based on the standard U-Net backbone, without introducing additional trainable parameters or novel architectural components. The final summation layer serves only to aggregate angular information and reduce the output dimensionality, rather than representing a new learnable module. As such, a separate ablation study was not conducted in this proof-of-concept work. Nevertheless, further ablation analyses may be valuable in future studies to assess the contribution of architectural and post-processing choices to the overall performance.

Calcification-associated narrowing affects over 30% of PCI target lesions, posing a significant challenge to procedural success.⁵ An understanding of the interaction between quantitative measures of coronary calcium and procedural outcomes can help to make PCI more predictable and safer, and extensive research has been dedicated to exploring this relation. For instance, severe calcification is characterized by a maximum covering angle exceeding ${270}^{\circ }$^17,18 while a length of 5 mm serves as a cutoff for defining calcium scores.³ These numbers have been determined based on painstaking manual analysis. Due to the abundance of frames in IVUS pullbacks, only sub-sampled data were analysed for research with manual labelling. In this study, we introduce algorithms capable of automatically computing these quantification metrics across entire pullbacks, with unparalleled accuracy, precision, and F1 scores. This advancement holds promise for reliable automation of studies underpinning decision support in intravascular-imaging guided PCI. Moreover, our work facilitates the comparison of plaque treatment techniques and hopefully can aid in the stratification of device selection.

Conclusion

In this work, we proposed a calcium detection pipeline where the U-Net backbone was applied to handle weak labelling in IVUS data. The predicting behaviour can be interpreted by observing the attention layer. The results demonstrate that the pipeline can perform fast, robust, and fully automated detection. We experimented with several practical factors, including training five candidate models and merging their trained parameter sets, completing the training database to be more diverse, and seeking the optimal size of the filter for post-processing. After addressing these practical issues, the detection pipeline achieves sufficient accuracy of 0.96 and Dice scores of 0.88, ready for further clinical validation. Furthermore, we provide an automated and fast calculation of the quantitative indices that are currently commonly used in clinical research. The detection and quantification pipeline can accelerate clinical research on how calcified plaques affect PCI procedures and patient outcomes, and provide data supporting decisions on lesion preparation strategy in PCI.

Funding

This work was supported by Boston Scientific Corporation. S.L. received research support from the Open eScience Call 2022 project NLeSC.C.22.0215. R.C. and M.d.B. received financial support from the Dutch Research Council (NWO) project VI.C.182.042. J.D. received institutional grant/research support from Abbott Vascular, Boston Scientific, ACIST Medical, Medtronic, Microport, Pie Medical, and ReCor medical, and consultancy and speaker fees from Abbott Vascular, Abiomed, ACIST Medical, Boston Scientific, Cardialysis BV, CardiacBooster, Kaminari Medical, ReCor Medical, PulseCath, Pie Medical, Sanofi, Siemens Health Care, and Medtronic. G.v.S. is a cofounder of, and has equity in, Kaminari Medical BV. In the past 3 years, he was the PI on research projects, administered by Erasmus MC, that received research support from FUJIFILM VisualSonics, Shenzhen Vivolight, Boston Scientific, Waters and Mindray.

Data availability

The data underlying this article will be shared on reasonable request to the corresponding author.