Recent Updates on Applications of Artificial Intelligence for Nuclear Medicine Professionals: Bone Scintigraphy

Introduction

We face a new world of artificial intelligence (AI) when we wake up. It is like a labyrinth that changes in real time and can be really difficult to navigate. Small marks of researchers who went before often remain in various forms and lead the followers even in such a labyrinth. Introducing the latest research that sometimes leads to the right path and sometimes tells us that it is a dead end. This editorial aims to introduce those markers. We introduce recent applications of AI to nuclear medicine professionals starting in the field of bone scintigraphy.

Bone scintigraphy is one of the most basic and important tests in the field of nuclear medicine. Over the past few decades, it has played a key role in the diagnosis and treatment of various bone diseases. Nevertheless, there are still many ways to further improve the precision and efficiency of bone scintigraphy, especially with the recent advances in AI and the benefits of applying it to the field of bone scintigraphy. AI has shown excellent performance in areas such as image analysis, pattern recognition, and data processing, and it is expected that it can be applied to nuclear medicine tests such as bone scintigraphy to provide more accurate and faster diagnosis. Therefore, we aim to shed light on how AI can be effectively applied to bone scintigraphy and the potential benefits it can bring.

Adapting a Low-count Acquisition of the Bone Scintigraphy using Deep Denoising Super-resolution Convolutional Neural Network

Toshimune Ito, Takafumi Maeno, Hirotatsu Tsuchikame, Masaaki Shishido, Kana Nishi, Shinya Kojima, Tatsuya Hayashi, Kentaro Suzuki

Tokyo, Japan

Phys Med. 2022;100:18–25.  10.1016/j.ejmp.2022.06.006

Background

Bone scintigraphy is a diagnostic tool that uses gamma ray emissions from radionuclides introduced into the body to detect various bone diseases, including metastatic bone tumors. While radiation exposure is minimized, the randomness of these emissions leads to statistical fluctuations, which increases statistical noise and reduces the signal-to-noise ratio (SNR) at lower doses. This issue constrains the clinical application of low count bone scintigraphy images.

Recent advances in deep neural networks have shown significant potential in image processing tasks, including denoising and super-resolution imaging. Convolutional neural networks (CNNs) and super-resolution convolutional neural networks (SRCNNs) have been used to denoise raw images and enhance the resolution of low-resolution images. However, studies on noise reduction processing using super-resolution processing in the context of deep learning for nuclear medicine imaging are scarce.

This study investigated the applicability of deep denoising super-resolution convolutional neural networks (DDSRCNN) in improving low count imaging in bone scintigraphy. DDSRCNN was compared to de-noising convolutional natural networks, Gaussian processing, and nonlinear diffusion processing.

Methods

This study used bone scintigraphy images from 156 patients who underwent this procedure within the period of June to September 2020. The bone scintigraphy was performed by chest planar acquisition three hours after the intravenous administration of 740 MBq ^99mTc-hydroxymethylene diphosphonate. The images were acquired with a 256 × 256 matrix of 2.46-mm pixels at 0.898 × magnification, a 15% energy window at the center of the Photopeak energy (140 keV), and a total count of 1000 kilocounts (kct). The Lister tool function was used to generate low count images with total counts of 500 kct, 400 kct, 300 kct, 200 kct, and 100 kct, assuming short collection time condition.

The DDSRCNN architecture is based on a series of convolutional and deconvolutional layers. The encoder module is responsible for extracting meaningful features from input data. The decoder module scales the image and recovers details from the given features. Skip connections are introduced between the corresponding convolutional and deconvolutional layers to increase the efficiency of deep network training. As the convolutional layer becomes deeper, image details are inevitably lost, making recovery by deconvolution difficult. By propagating the feature map through skip connections, a greater amount of image detail is preserved, improving the accuracy of image restoration.

The effectiveness of the DDSRCNN model was evaluated by comparing it to traditional denoising image processing methods: Gaussian processing, nonlinear diffusion processing, and denoising convolutional natural networks. The metrics used for this comparison were peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM).

Results

The DDSRCNN outperformed other methods in terms of PSNR and SSIM for all total counts, which holds promise for imaging patients using less radiopharmaceuticals. The researchers also found that in this study, the DDSRCNN model trained on a dataset containing a mixture of 100 kct and 500 kct images performed better than the model trained on 100 kct images alone. This result suggests that there is a gradual noise component, and therefore careful consideration of the noise characteristics in nuclear medicine images is important in the development of denoising models.

Reviewer’s Comments

The results of this study have important implications for the practical application of bone scintigraphy imaging in a clinical environment. Bone scintigraphy, which is mainly used in the field of nuclear medicine imaging, is simple and widely used, but it is inconvenient for patients to wait for 2 to 4 h and to lie in a rigid position while the whole body scintigraphy is completed. Due to the nature of scintigraphy, the image using a small amount of radionuclide was difficult to read clinically because of the noise. This AI will benefit clinically by removing noise, especially in bone imaging of patients with conditions such as chronic kidney disease, which is associated with poor image quality.

Accumulation of techniques such as this study will enable faster scans by removing noise from images using lower radionuclide doses. This not only enhances image quality but also significantly reduces radiation exposure to patients, emphasizing a safer clinical approach. By optimizing the use of medical resources and minimizing equipment occupancy, facilities can accommodate higher acquisition volumes, providing more options for equipment operation and further ensuring patient safety.

Artificial Intelligence-based Analysis of Whole-body Bone Scintigraphy: The Quest for the Optimal Deep Learning Algorithm and Comparison with Human Observer Performance

Ghasem Hajianfar, Maziar Sabouri, Yazdan Salimi, Mehdi Amini, Soroush Bagheri, Elnaz Jenabi, Sepideh Hekmat, Mehdi Maghsudi, Zahra Mansouri, Maziar Khateri, Mohammad Hosein Jamshidi, Esmail Jafari, Ahmad Bitarafan Rajabi, Majid Assadi, Mehrdad Oveisi, Isaac Shiri, Habib Zaidi

Switzerland, Iran, Canada, the Netherlands, Denmark

Z Med Phys. 2023:S0939–3889(23)00008–9. 10.1016/j.zemedi.2023.01.008

Background

Whole-body scintigraphy (WBS) has become an important diagnostic tool due to its high sensitivity, cost-effectiveness, and ability to provide comprehensive body imaging. However, it has the disadvantage of being difficult to differentiate between benign and malignant lesions in the early stages of disease and is susceptible to subjective interpretation. Furthermore, while its sensitivity is as high as 95%, it lacks specificity and requires sufficient skill to differentiate between benign and malignant disease with increased bone turnover.

However, with the advent of AI, specifically deep learning algorithms, solutions are emerging to address these issues. Deep learning, specifically convolutional neural networks (CNNs), has been shown to have significant potential for diagnosing malignant bone disease from WBS images. In particular, because WBS inherently produces two planar scans, including anterior and posterior views, the recently emerging multi-view strategy integration method can be applied.

The authors created several models based on the above and evaluated their classification performance on the following tasks: classifying patients into normal and abnormal subjects, and distinguishing patients with malignant bone disease from patients diagnosed with other abnormalities.

Methods

This retrospective, multicenter study initially included WBS from 7188 patients obtained between October 2015 and September 2019. However, several exclusions were made due to factors such as incomplete or inaccessible records, low-quality images, subcutaneous injections, and skin surface contamination, resulting in 3772 patients being included in the primary analysis. Of these, 2313 cases were identified as abnormal, of which 65 were discarded due to lack of definitive reports, leaving 2248 abnormal cases for secondary analysis. Ground truth was determined by two nuclear medicine physicians (NMPs) who considered each patient's medical history, laboratory tests, pathology, and current WBS along with additional spot images.

WBS were performed 2–4 h after intravenous injection of 555 to 925 MBq of ^99mTc-methylene diphosphonate using a dual-head gamma camera (Siemens Symbia Encore, Siemens ECAM IP1 and Mediso AnyScan S) equipped with low-energy, high-resolution parallel hole collimators in the supine arm-down position. The energy acquisition window was centered at 140 keV with a 20% window, a scan speed of 12–15 cm/min in continuous mode, and a matrix size of 1024 × 256.

A single input strategy was implemented using ten different CNNs: VGG19, MobileNetV2, ResNet50V2, ResNet101V2, ResNet152V2, InceptionV3, InceptionResNetV2, DenseNet121, DenseNet169, and DenseNet201 with squeeze and excitation (SE). The dual-input strategy involved combining these ten CNNs with three unique aggregation methods: SE, Spatial Pyramid Pooling (SPP), and Attention-Augmented (AA). A total of 50 separate models were developed using this methodology. These included 20 models from the single-input strategy (with ten models representing anterior views and the remaining ten models representing posterior views) and 30 models from the dual-input strategy, obtained by cross-combining the ten CNN models and the three aggregation methods.

Three NMPs with varying levels of experience were involved in the analysis, and their performances to classify normal/abnormal and malignancy/benign served as a benchmark for the effectiveness of the AI-based diagnostic approach.

Results

In the first normal/abnormal classification analysis, the models with the highest area under curve (AUC) were InceptionV3_Ant (0.69), ResNet50V2_Post (0.67), DenseNet121_AA (0.72), InceptionResNetV2_SE (0.68), and InceptionV3_SPP (0.70). In the second malignancy/benign differentiation analysis, the top five models in terms of AUC were DenseNet201_Ant (0.67), DenseNet121_Post (0.64), InceptionResNetV2_AA (0.70), DenseNet121_SE (0.64), and InceptionResNetV2_SPP (0.72).

The performances of these deep learning models were compared with three NMPs. In the first analysis, the NMPs achieved AUCs of 0.65, 0.64, and 0.60, respectively, compared to the DenseNet121_AA model, which achieved an AUC of 0.72. In the second analysis, the NMPs achieved AUCs of 0.74, 0.70, and 0.77, respectively, while the InceptionResNetV2_SPP model achieved an AUC of 0.72. According to the DeLong test, DenseNet121_AA significantly outperformed all NMPs in the first analysis (p < 0.05). In the second analysis, there was no significant difference in performance between the models and the NMPs.

Reviewer’s Comments

Since the development of AI, several models have been grafted to classify nuclear medicine images. In this study, feature extraction was performed using widely accepted network models, and multiple input models were combined to accommodate the feature that the anterior and posterior views of bone scintigraphy were obtained simultaneously. These methods raise considerations for researchers who want to develop a new CNN model for bone scintigraphy.

The results of this study showed that the model’s ability to discriminate between normal and abnormal scans and discriminate between malignant and benign cases was as good as or comparable to that of NMPs. If these models lead to clinical implementation, it will lead to improved diagnostic precision in the clinical environment, thereby mitigating the occurrence of misdiagnosis and strengthening overall trust in patient care.

However, for clinical application, the performance of the model should also be evaluated for factors frequently encountered in clinical practice, such as poor quality images, subcutaneous injections, full bladder, and urinary catheter, which were excluded in this study. Good quality data is needed for model development and research, but it seems necessary to consider how to handle low-quality data in order to enter clinical practice.

Using the STEGO Neural Network for Scintigraphic Image Analysis

Ivan Ulitin, Marina Barulina, Marina Velikanova

Saratov, Russia

Engineering Proceedings. 2023;33(1):5. 10.3390/engproc2023033005

Background

With the advent of the transformer architecture, various models have emerged that use the vision transformer as their backbone to extract features from images. Among these, the Self-Supervised Transformer with Energy-Based Graph Optimization (STEGO) stands out as a state-of-the-art model in 2022. This model is a collaborative effort between renowned institutions: MIT, Microsoft, Cornell University, and Google.

While STEGO is not the only model that performs unsupervised learning segmentation, its development by a collaboration of prestigious AI teams holds significant weight. If the segmentation task is used as a preprocessor for connecting to other tasks after segmentation, using a widely recognized model like STEGO to process the raw data will allow subsequent pipelines to connect seamlessly, speeding up technology adoption. Given its potential, this research is in the context of exploring and extending the application of unsupervised learning models, a branch of AI models, to the field of scintigraphy, a radionuclide diagnostic procedure that visualizes physiological and functional changes.

Previous research has shown that machine learning methods in radionuclide diagnostics, such as the diagnosis of cancer bone metastases, can yield promising results. However, these methods often require large amounts of training data to identify features and locations of foci in images. STEGO, being an unsupervised learning method, overcomes this challenge as it doesn't necessitate a segmented ground truth for the segmentation task. It employs correspondence distillation loss, combining feature correspondences and segmentation correspondences, to execute the segmentation task without the need for ground truth. Recognizing its potential and the speed it offers in technical applications, we decided to introduce a paper that applies STEGO to bone scintigraphy.

Methods

The dataset used in this study was a DICOM file containing anterior and posterior view information for 57 patients, 54 men and 3 women, with renal or prostate cancer with bone metastases. While scintigraphy images can be displayed in a variety of color palettes, color affects the result of the STEGO method, so the images of the same case were prepared in three different color palettes (blue/yellow, blue/green/red/yellow, and warm metal).

The authors extracted the features using the frozen visual backbone to the extracted features and calculated the feature correspondences and also applied the segmentation head to compute the segmentation correspondences. The model was trained by calculating the correspondence distillation loss using these two correspondences. To perform semantic segmentation with this model, the linear probe prediction method and the clustering prediction method were applied to perform segmentation on each image, which was transformed into a good low-resolution image of 246 × 246 pixels. The resulting images were examined for their ability to discriminate the foci in the original images.

Results

Using linear probes to separate the classes resulted in a relatively good separation between metastatic and non-metastatic regions. In particular, metastatic regions in the ribs and spine are clearly identified as foci with clear boundaries in the STEGO output image using linear probe prediction. On the other hand, cluster prediction did not segment well in general, which the authors attributed to the fact that the colors appear almost uniformly in the image.

The authors found that STEGO’s results are affected by the color palette that makes up the scintigraphy image: an image with a blue/yellow color palette resulted in a false positive segmentation in the case of a patient with metastases in the spine, creating a large area of metastases between the patient’s arm and thigh, and also observed that STEGO’s segmentation performance degrades when the color palette changes to rainbow colors. Therefore, not all possible color palettes allow for high quality image segmentation.

Reviewer’s Comments

As AI advances, there is a growing need to leverage data. However, building ground truth remains a significant challenge due to its complex and labor-intensive nature. As a result, there is a growing emphasis on unsupervised learning methods, particularly self-supervised learning, which can avoid the need for labeled data.

Innovative techniques such as STEGO, which can skillfully extract features from images, provide a highly efficient way to build models without the need for ground truth data. These methods can also be used to automatically extract and use salient features from images to build higher-level models for classification and prediction tasks. This efficiency not only helps to speed up model development but also makes it easier to use the model in a wider range of applications.

This approach is particularly promising for medical image analysis. Similar to the step-by-step process performed in actual clinical reading, these techniques facilitate a step-by-step approach based on the visual features of an image. By providing information that predicts the level and area of a lesion along with interpretation information, these methods contribute significantly to the advancement of explainable AI.

This study has several limitations, such as experimenting with a limited dataset, only looking at bone scintigraphy from patients with renal or prostate cancer, and not providing specific intersection values for each case. Nevertheless, we praise the pioneering courage in trying to solve the segmentation problem by applying the STEGO method, which has emerged hotly in the semantic segmentation, to bone scintigraphy.

Conclusion

Just as navigating a labyrinth requires keen perception and innovative strategies, applying AI to bone scintigraphy presents similar challenges. The application of deep learning algorithms such as CNNs and unsupervised learning methods such as STEGO hold great promise for improving the analysis of scintigraphy image analysis. However, this complex journey also reveals numerous considerations, such as the impact of color palette on AI performance, reminding us that it must be approached carefully to avoid missteps. Finding the end of this labyrinth would not be easy, but the deeper we dig, the more we will realize AI’s potential to revolutionize medical imaging.

Author Contribution

Conceptualization: H-SHB and K-SP. Writing—original draft: K-SP. Writing—review and editing: H-SHB.

Data Availability

Not applicable.

Compliance with Ethical Standards

Conflict of Interest

The authors declare no competing interests.

Ethical Statement

All content in this editorial was in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

As an editorial article, obtaining informed consent was waived.

Consent for Publication

None.