Quantitative ultrasound image analysis of axillary lymph nodes to differentiate malignancy from reactive benign changes due to COVID-19 vaccination

Abstract

Purpose

The aim of this study is to assess the potential of quantitative image analysis and machine learning techniques to differentiate between malignant lymph nodes and benign lymph nodes affected by reactive changes due to COVID-19 vaccination.

Method

In this institutional review board–approved retrospective study, we improved our previously published artificial intelligence model, by retraining it with newly collected images and testing its performance on images containing benign lymph nodes affected by COVID-19 vaccination. All the images were acquired and selected by specialized breast-imaging radiologists and the nature of each node (benign or malignant) was assessed through a strict clinical protocol using ultrasound-guided biopsies.

Results

A total of 180 new images from 154 different patients were recruited: 71 images (10 cases and 61 controls) were used to retrain the old model and 109 images (36 cases and 73 controls) were used to evaluate its performance. The achieved accuracy of the proposed method was 92.7% with 77.8% sensitivity and 100% specificity at the optimal cut-off point. In comparison, the visual node inspection made by radiologists from ultrasound images reached 69.7% accuracy with 41.7% sensitivity and 83.6% specificity.

Conclusions

The results obtained in this study show the potential of the proposed techniques to differentiate between malignant lymph nodes and benign nodes affected by reactive changes due to COVID-19 vaccination. These techniques could be useful to non-invasively diagnose lymph node status in patients with suspicious reactive nodes, although larger multicenter studies are needed to confirm and validate the results.

1. Introduction

On 11th March 2020, the World Health Organization (WHO) declared the ongoing COVID-19 outbreak a global pandemic. As of early June 2022, more than 530 million people worldwide have been infected by SARS-CoV-2 (the strain of coronavirus that causes COVID-19), and more than 6.3 million have died according to WHO’s statistics [1]. COVID-19 has changed the landscape of global society and has converted in a public emergency.

Obtaining a vaccine against this coronavirus has been a global priority in the fight against the pandemic. In December 2020 the most developed countries worldwide began a massive vaccination program. mRNA-based vaccines Comirnaty (Pfizer/BioNTech) and Spikevax (Moderna), vector-based vaccines Vaxzevria (AstraZeneca) and Janssen COVID-19 vaccine (Johnson&Johnson/Janssen) and others, were distributed over the world’s main countries. As of 1st June 2022, 66.0% of the world population has received at least one dose and 60.2% is fully vaccinated (excluding booster shots), according to OurWorldInData [2]. During this time, like any drug, a significant number of adverse reactions to the new COVID-19 vaccines have been observed [3]. Most common side effects are mild-to-moderate, and similar to those described for other vaccines: pain in the injection site, fatigue, headaches, fever, chills, and muscle and joint pains [3], [4].

Other less common side effects such as axillary and supraclavicular lymphadenopathy have been reported [3], [4]. In the Moderna trial, axillary swelling and tenderness on patient survey was reported in 10.2% people in the vaccine group after first dose (4.8% in placebo) and 14.0% after second dose (3.9% in placebo) [5]. While not reported as an adverse event in Pfizer-BioNTech trials, more cases are reported in the COVID-19 vaccine group (64) than in the placebo group (6) [6]. AstraZeneca does not detail these reactions in its trials, but simply labels them as “uncommon” in the vaccine characteristics [7]. No data or information related to this adverse reaction was reported in Janssen trials [8]. Globally, recent studies performed in general population have reported that the incidence of post-vaccination lymphadenopathy ranged from 14.5% (after a single dose) to 53% [9].

Regardless of the frequency of this type of adenopathy related to vaccination, radiologists and oncologists should be aware of it to obviate unnecessary changes in patient management, particularly for cancer patients [10]. Recognizing this as a potential differential diagnosis is crucial to be able to provide appropriate follow-up recommendations [11].

In the oncology field, lymph node status has always been a key factor for the staging, prognosis and treatment of cancer. Ultrasound examination of axillary lymph nodes has become common practice in the pre-surgical assessment of breast cancer patients [12], [13]. Once suspicious nodes have been detected, invasive techniques are performed to confirm malignancy such as ultrasound-guided fine needle aspiration (FNA) or core needle biopsy (CNB) [13], [14].

To reduce or avoid the use of these invasive techniques, several studies have used artificial intelligence (AI) techniques to non-invasively predict axillary lymph node involvement in breast cancer from medical images. Our previous study published in 2019 [15] demonstrated the potential of image analysis techniques to detect the microstructural and compositional changes that occur in lymph nodes to noninvasively diagnose metastatic involvement. Subsequently, Sun et al. [16], Lee et al. [17], Zhou et al. [18] and Sun et al. [19] confirmed that deep learning techniques and more specifically convolution neural networks can effectively predict axillary lymph node metastasis.

Building upon our previous study researching AI methods to diagnose metastatic involvement in axillary lymph nodes from ultrasound images [15], the objective of this study is to assess the potential of similar image analysis and machine learning techniques to differentiate between malignant lymph nodes affected by metastatic breast cancer and benign lymph nodes affected by reactive changes due to COVID-19 vaccination.

2. Methods

2.1. Study design and patients

This was an institutional review-board approved retrospective study authorized by the ethics committee of the Hospital Clínic de Barcelona (HCB/2020/0863) with a waiver of the need to obtain patient informed consent.

In this study, we improved our previous AI technique [15], retraining the old model with newly collected images. Subsequently, we tested the resulting model using new images which contain benign lymph nodes affected by COVID-19 vaccination.

To improve our last model, specialized breast-imaging radiologists of Hospital Clínic de Barcelona retrospectively selected nodes from patients attending the clinic for routinary breast and axillary lymph node radiological assessment between November 2018 and February 2020. These patients were either breast cancer patients, or non-breast cancer patients who underwent an examination for other reasons (not related with the COVID-19 vaccine, still non-existent at that time). None of these patients were overlapped in time with patients of our previous study.

To test the model and obtain the final results, the radiologists retrospectively selected all viable nodes from the first months of COVID vaccination campaign (from March 2021 to June 2021). This was the period where more benign suspicious nodes were analyzed, as there was no certainty that COVID vaccination could affect the axillary nodes. These nodes were from breast cancer patients or non-breast cancer patients who underwent an examination for possible lymphadenopathy caused by COVID-19 vaccination (with Moderna, Pfizer-BioNTech or AstraZeneca vaccines).

The nature of each node was assessed through a strict clinical protocol outlined below. Only images from nodes which met the inclusion criteria detailed in this clinical protocol were collected. Images with artefacts, pointers or callipers, were also discarded. If more than one image per node was available, radiologists selected only the best image (the one with best resolution and saturation-darkness ratio) to use in this study.

2.2. Image acquisition

All the collected images were ultrasound images in B-mode of an axillary lymph node acquired by specialized breast-imaging radiologists during each patient examination with no prior guidelines. The images were stored in DICOM (Digital Imaging and Communication in Medicine) format and later anonymised.

The ultrasound devices used to acquire the images were an Acuson Antares (Siemens AG, Munich, Germany) with a 10–13 MHz linear transducer and an Aplio i800 (Canon Inc., Ōta, Tokyo, Japan) with a 10 MHz linear transducer.

2.3. Clinical management

The protocol used in the hospital to determine the nature (benign or malignant) of axillary lymph nodes was the same as in our previous study [15]. This protocol is based on Bedi’s criteria [20], which defines the degree of suspicion of metastatic involvement for each axillary lymph node (Fig. 1 ). When a node is considered suspicious for metastasis (Bedi score of 3 or higher), an ultrasound-guided FNA or CNB was performed by the radiologist to confirm malignancy, even in the case of recent vaccination.

Fig. 1.

Drawings and definitions of the different types of nodes proposed by Bedi[20]accordingtotheir degrees of suspicion. Courtesy of Coronado-Gutierrez et al.[15].

For breast cancer patients (with or without history of recent vaccination), only confirmed malignant nodes with FNA or CNB were selected to use as “cases”. Nodes with negative result or without a FNA or CNB result were discarded, since we could not be sure of their real nature.

In case of non-breast cancer patients used to retrain the model, only nodes not suspicious of metastasis (Bedi score lower than 3) or nodes with benign confirmation via ultrasound-guided FNA or CNB were retained. All these nodes were assumed negative for metastasis and used as “controls”.

For non-breast cancer patients used to test the model (patients with history of recent vaccination), only suspicious nodes (Bedi score of 3 or higher) with negative breast findings on ultrasound were used. All these nodes were assumed negative for metastasis and used as “controls” since as far as we know, they continued to have all negative follow-ups at the time of this study.

2.4. Image analysis

The proposed technique had three different steps: (1) manual delineation of the region of interest (ROI); (2) quantitative image analysis from ROIs; and (3) classification with machine learning methods. These three steps were performed for both training and testing subset images.

(1) From each image, the selected lymph node was segmented out manually by an expert radiologist using a graphical user interface, obtaining the node’s ROI.

(2) Subsequently, each outlined ROI was quantitatively analysed to obtain textural features useful to train the machine learning methods. To do this task, image processing techniques were used to encode the ROI pixels values into a meaningful representation of the textural patterns it contained. In this case, a simpler, yet efficient, method than that used in the previous study was selected to perform this task: a multiresolution approach based on local binary pattern (LBP) method [21]. This method is based on recognizing that certain LBPs, termed “uniform”, are fundamental properties of a local image texture. Indeed, their histogram of occurrence has proven to be a very powerful texture feature. This technique is very robust in terms of grey-scale and rotation variations.

(3) Finally, the computed features and the histopathologic outcome of each image were used to train and test supervised machine learning methods to distinguish between nodes that were positive for metastasis and those that were negative even if they had reactive changes due to COVID-19 vaccination. As in our previous study, we again used sparse partial least squares (SPLS) method [22] to perform this task. This method is a variant of the widely used PLS method, which is a technique that reduces the predictors to a smaller set of uncorrelated components and performs least squares regression on these components, instead of on the original data. SPLS produces sparse linear combinations of the original PLS predictors to achieve good predictive performance and variable.

2.5. Statistical analysis

The performance of the final method was evaluated using the test set images. This set was a compound of affected metastatic nodes and benign nodes affected by COVID-19 vaccination.

Receiver operating characteristic (ROC) curve was computed to report overall classifier performance and then typical binary performance metrics were reported at the optimal cut-off point of the ROC curve (chosen as that maximizing accuracy). The same ROC curve was computed with Bedi scores annotated by expert radiologists to compare the differences in performance with the proposed method.

All the methods used in the image analysis and in the statistical analysis processes, were developed using Matlab R2017a (MathWorks, Inc., Natick, MA, USA).

3. Results

In our previous study [15] the proposed classification method was trained with 118 images from 105 distinct patients: 53 affected metastatic nodes (cases) and 65 unaffected nodes (controls).

In this study, a total of 180 new images from 154 different patients were recruited: 71 images (10 cases and 61 controls) were used to retrain the old model and 109 images (36 cases and 73 controls) were used to evaluate the performance of the proposed method. The prevalence of metastatic nodes in the training set (previous train subset plus new retrain subset) and testing set was 33% in both cases.

Table 1 shows the demographic characteristics of the 71 new images from 60 different patients recruited to retrain the old model and Table 2 shows the demographic characteristics of the 109 new images from 94 different patients recruited to test the new model.

Table 1.

Demographic characteristics of patients used to retrain the old model.

	Patients	Nodes
Diagnosed Breast Cancer	10	10
Invasive Ductal Carcinoma (IDC)	8	8
Invasive Lobular Carcinoma (ILC)	1	1
Other carcinomas	1	1
Non-diagnosed Breast Cancer	50	61
TOTAL	60	71

Table 2.

Demographic characteristics of patients used to test the new model.

	Patients	Nodes
Diagnosed Breast Cancer	36	36
Invasive Ductal Carcinoma (IDC)	33	33
Invasive Lobular Carcinoma (ILC)	1	1
Other carcinomas	2	2
Non-diagnosed Breast Cancer (reactive changes by COVID-19 vaccination)	58	73
TOTAL	94	109

To point out visual similarities between malign nodes and those affected by COVID-19 vaccination, Fig. 2 shows an example of two nodes with the same Bedi score (5) but different nature: one is a benign node affected by COVID-19 vaccination while the other is a malignant node. Both are morphologically similar (similar contours of the nodes and fatty hilum).

Fig. 2.

Ultrasound images of 2 different nodes: (a) benign node with reactive changes due to COVID-19 vaccination; (b) metastatic node of invasive ductal carcinoma. Arrows point out the node’s cortex and arrow heads their fatty hilum.

Fig. 3 shows the ROC curve obtained by the proposed classification learning method on the test set after being retrained using the new training set. The overall area under the ROC curve (AUC) was 88.4%, and the optimal accuracy achieved was 92.7%, with 77.8% and 100% sensitivity and specificity, respectively (Table 3 ).

Fig. 3.

ROC curve obtained by the proposed method (blue line) versus visual inspection by expert radiologists using Bedi’s criteria (red line) on the same test images.

Table 3.

Performance metrics of the proposed method and the visual scoring at optimal cutoff point (best accuracy) of the ROC curves. The numbers in brackets represent the 95% confidence interval and the numbers in parentheses the number of correct answers among the total. (AUC = Area Under the ROC Curve; ACC = Accuracy; SENS = Sensitivity; SPEC = Specificity; PPV = Positive Predictive Value; NPV = Negative Predictive Value).

	AUC	ACC	SENS	SPEC	PPV	NPV
Proposed method	88.4%	92.7% [86.0 96.8] (101/109)	77.8% [60.8 89.9] (28/36)	100.0% [95.1 100.0] (73/73)	100.0% [87.7 100.0] (28/28)	90.1% [81.5 95.6] (73/81)
Visual scoring	62.6%	69.7% [60.2 78.2] (76/109)	41.7% [25.5 59.2] (15/36)	83.6% [73.0 91.2] (61/73)	55.6% [35.5 74.5] (15/27)	74.4% [63.6 83.4] (61/82)

Fig. 3 also shows the ROC curve obtained with Bedi scores annotated by expert radiologists on the test images, to visualize the differences with the proposed method. In this case, the AUC was 62.6% and the optimal accuracy was 69.7%, with 41.7% sensitivity and 83.6% specificity (Table 3).

4. Discussion

In this study, AI techniques presented in our previous publication [15] to non-invasively diagnose axillary lymph node involvement in breast cancer were applied to observe its potential to differentiate between malignant nodes and benign nodes affected by COVID-19 vaccination. The proposed method was evaluated using axillary lymph node images from ultrasound explorations of breast cancer and non-breast cancer patients recorded in Hospital Clínic de Barcelona. The training (and retraining) set images were all malignant and normal benign nodes, and the testing set images were malignant and benign nodes affected by COVID-19 vaccination, which are morphologically similar.

Results showed that the proposed image analysis and machine learning methods can effectively differentiate malignant nodes from benign ones even if they had reactive changes due to COVID-19 vaccination. The proposed technique reached 92.7% accuracy, with 77.8% sensitivity and 100% specificity at the optimal cut-off point. These results demonstrate that these techniques have potential to detect unaffected lymph nodes even if they look visually similar to a metastatic one to the naked eye.

Secondly, the visual Bedi score made by radiologists showed lower results compared with the proposed method. The ROC curve obtained with this visual scoring reached an AUC of 62.6% while the proposed method got 88.4%. This direct comparison on the same images clearly suggests that the proposed methods can improve radiologists’ performance and also indicate that visual techniques are not enough to correctly classify nodes in patients after COVID-19 vaccination. Unaffected reactive nodes could be easily confused with reactive ones, especially in patients with breast cancer. For this reason, current clinical protocols could greatly benefit from the techniques proposed in this study.

These results are in line with Sun et al. [16], Lee et al. [17] and Zhou et al. [18], who obtained an AUC between 75.9% and 95.7%. Still, these studies used ultrasound images of primary breast cancer tumours and not of axillary lymph nodes. Our results show higher performance compared to the recent study of Sun et al. [19] which used axillary lymph node images. With a similar test sample size, they obtain an AUC of 72% and an accuracy, sensitivity and specificity of 72.6%, 65.5% and 78.9% respectively. Moreover, more complex methods based on convolutional neural networks were used in these studies compared to our proposed algorithm and none of them related lymph node involvement with lymphadenopathy due to COVID-19 vaccination.

The proposed techniques could be implemented as an online software or even integrated in the ultrasound machine, to facilitate its use in real clinical practice. This software could be used by radiologists right after an axillary lymph node exploration and the results would be obtained in less than thirty seconds. Processing time in our computers using Matlab R2017a was less than one second. To this, one must add the time radiologists’ need to manually delineate the lymph node, which in our case was less than 20 seconds by using our platform. In the future, if larger databases were collected, an automatic node’s delineation could probably be implemented using deep learning algorithms given their recent success stories in similar applications [23].

Focusing on clinical practice, a software using these techniques could greatly benefit clinicians, radiologists and patients. This software could be used to support decision making in both cancer and non-cancer patients with suspicious lymph nodes (Bedi score of 3 or higher) during ultrasound examination. In this way, unnecessary biopsies would be avoided, thus improving patient management and treatment. Another potential application of these techniques could be after neoadjuvant chemotherapy (NCT), by providing a non-invasive way of evaluating axillary lymph node status which often becomes negative after NCT, and avoiding therefore unnecessary lymph node excisions.

This study has several strengths. First of all, we tested the proposed method with images of benign nodes affected by COVID-19 vaccination, considerably different from the benign nodes used in the training set (not affected by COVID-19 vaccination). Secondly, unlike our previous study where the images were collected avoiding any type of ultrasound post-processing, the new collected images were obtained with no prior guidelines and images with post-process were accepted. As the main limitation of the study, the small size of the data test set warrants the need to confirm these results in larger multicenter studies before they can be applied in clinical practice.

5. Conclusions

The results show the potential of image analysis and machine learning methods to differentiate between malignant lymph nodes affected by metastatic breast cancer and benign lymph nodes affected by reactive changes due to COVID-19 vaccination. These techniques could be useful to non-invasively diagnose lymph node status in patients with and without diagnosed breast cancer with suspicious reactive nodes. Larger, multicenter studies are needed to confirm and validate the results described in this study.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.