Detection of Brain Metastases with Deep Learning Single-Shot Detector Algorithms

Beginning in 2012, deep learning (DL), a variant of machine learning, has dominated competitions in image classification (ie, determining whether an image contains a cat, dog, or airplane). This class of algorithms was enabled by so-called convolutional neural networks (CNNs), a class of artificial neural networks (ANNs). ANNs are computing systems vaguely inspired by the biologic neural networks, and CNNs bear some similarity to biologic visual systems. In 2015, a DL method was said to achieve “superhuman” performance in large-scale image classification (1). CNN models have also performed well on semantic segmentation, or assigning labels to pixels or voxels in images. In 2016, a CNN system was shown to be statistically indistinguishable from human readers in the detection of lesions at mammography (2). Basic ANN technology was introduced in the 1940s (3), but it did not perform well on real problems. The current version of this technology has gained performance through the increase in the number of layers in the networks. Adding layers in the CNN has resulted in substantial growth of the needed computational power; this gain has been made practical by the relatively recent use of so-called graphic processing units for computations of this kind.

The study by Zhou et al in this issue of Radiology (4) aims to provide a fully automated method for detecting metastases with contrast material–enhanced T1-weighted MRI of the brain. The work is motivated by stereotactic radiosurgery planning for treatment of the metastases, where currently, medical experts perform detection and segmentation. The authors applied a recent method from computer vision research, single-shot detection (SSD), to achieve good results (eg, detecting almost all metastases larger than 6 mm with a low false-positive rate).

Dr Zhou et al refer to several previous efforts that use CNNs for segmenting brain metastases. However, the quality of prior results reported in the literature was not deemed acceptable, that is, the false-positive rate was too high. To address this situation, Zhou et al identified an intriguing method from computer vision research, SSD. SSD uses a deep network to (essentially) directly predict object detections in the form of bounding boxes that surround objects in images. This is in contrast to prior methods that used several stages of processing to arrive at detections. The SSD approach enabled fast detection of objects in images (5).

Zhou et al used the SSD approach for the detection of brain metastases in postcontrast T1-weighted MRI to good effect (see Key Results in the article). In addition, the system is fast enough for clinical workflow (1 second per patient). Going forward, many medical imaging applications may benefit from such fast detection.

A full solution for stereotactic radiosurgery planning will need, in addition to detection, metastases segmentation. Zhou et al point out that the SSD detections (bounding box predictions) could be used as masks to improve the false-positive rates of semantic segmentation. This sounds like a good approach. The authors also point out that room for improvement exists in the performance of the algorithm for detecting small metastases and that this might be alleviated by larger training data and by network design changes (ie, extending some fully connected layers from two dimensions to three dimensions).

Looking forward, it may be that progress in generalpurpose CNN segmentation methods may lead to acceptable performance, without the need for a separate detection mechanism. However, this remains to be seen.

Additional development, and substantial testing, will be needed to achieve a system that could be used routinely in stereotactic radiosurgery planning. Because DL methods can be sensitive to the details of image acquisition protocols, special care will be needed to ensure generality across MRI sites and vendors; here domain adaptation methods may be useful (6).

In summary, the study by Zhou et al represents an important step toward the use of CNN for metastasis detection. We applaud the authors for this interesting contribution.

Biography

Dr Kikinis has been a professor of radiology at Harvard Medical School since 2004 and the founding director of the Surgical Planning Laboratory since 1990. From 2014 through 2020, he was also Institute Director Fraunhofer Mevis and professor of computer science at the University of Bremen. Dr Kikinis’ main scientific interest is in medical image computing and image-guided therapy.

Dr Wells is a professor of radiology at Harvard Medical School and a research scientist at the MIT Computer Science and Artificial Intelligence Laboratory. He received the IEEE Computer Society Helmholtz Prize for fundamental contributions in computer vision and the Enduring Impact Award of the MICCAI Society. His research focuses on segmentation and registration for image-guided surgery, with recent emphasis on deep learning.