Discrimination of smoking status by MRI based on deep learning method

Abstract

Background

This study aimed to assess the feasibility of deep learning-based magnetic resonance imaging (MRI) in the prediction of smoking status.

Methods

The head MRI 3D-T1WI images of 127 subjects (61 smokers and 66 non-smokers) were collected, and 176 image slices obtained for each subject. These subjects were 23–45 years old, and the smokers had at least 5 years of smoking experience. Approximate 25% of the subjects were randomly selected as the test set (15 smokers and 16 non-smokers), and the remaining subjects as the training set. Two deep learning models were developed: deep 3D convolutional neural network (Conv3D) and convolution neural network plus a recurrent neural network (RNN) with long short-term memory architecture (ConvLSTM).

Results

In the prediction of smoking status, Conv3D model achieved an accuracy of 80.6% (25/31), a sensitivity of 80.0% and a specificity of 81.3%, and ConvLSTM model achieved an accuracy of 93.5% (29/31), a sensitivity of 93.33% and a specificity of 93.75%. The accuracy obtained by these methods was significantly higher than that (<70%) obtained with support vector machine (SVM) methods.

Conclusions

The deep learning-based MRI can accurately predict smoking status. Studies with large sample size are needed to improve the accuracy and to predict the level of nicotine dependence.

Introduction

Smoking is a major public health problem and the leading cause of preventable deaths worldwide (1,2). More than six million people die from smoking related diseases all over the world each year. The World Health Organization (WHO) has classified nicotine dependence as a disease. Nicotine dependence is a dependence on psychoactive substance, its initiation and development are closely related to the changes in the brain structure and function, and it can cause cerebral vascular disease and Alzheimer’s disease (3,4).

Studies have shown that nicotine dependence leads to the changes in brain structure (5-8), but these changes are typically subtle and hard to macroscopically identify. Currently, it is difficult for clinicians to identify and interpret the changes in the brain by magnetic resonance imaging (MRI), and thus the assessment and prediction of smoke cessation are still a challenge in clinical practice. Despite the rapid development of artificial intelligence in recent years, little is known about the application of deep learning methods in the diagnosis and treatment of clinical diseases, especially in the studies about the discrimination of human brain structure. In the present study, advanced deep learning methods were employed to discriminate the brain structure of smokers, aiming to find a new way to investigate the brain characteristics of smokers.

Methods

Subjects

Both smoking and non-smoking subjects were recruited and included as smoking group (SG) and control group (CG). The SG and CG subjects were enrolled based on the following inclusion criteria: subjects were aged 23–45 years; subjects were healthy; the education level was bachelor degree or higher; they had normal hearing and vision; they had no color blindness and were right-handed, and there were no contraindications to MR examination; smokers started to smoke when they were 18 years old or older and had a history of smoking for more than 5 years. The exclusion criteria were as follows: subjects had central nervous system or mental illness, cardiovascular disease (CVD), respiratory system diseases, the endocrine system diseases, diseases of the digestive system or other important diseases; subjects were susceptible to depression or anxiety.

A total of 127 subjects were recruited into this study. In SG group, 61 smokers were enrolled from the Smoking Cessation Clinic of Beijing Chaoyang Hospital between August 2012 and October 2014; in CG group, 66 non-smokers were enrolled from the Clinic Health Center of Beijing Chaoyang Hospital between September 2013 and October 2014. Both groups had comparable demographics and personality characteristics, including age, gender, marital status, dietary habits, education level, eyesight, depression and anxiety [Beck Depression Inventory (BDI); Hamilton Anxiety Scale (HAMA)] (Table 1).

Table 1. Clinical characteristics of smokers and non-smokers in this study.

Characteristic	Smokers (n=61)	Non-smokers (n=66)	P
Age (year)	31.75±5.91	33.01±6.49	0.073
Gender (M/F)	61/0	66/0
Marital status (married/not married)	47/14	52/14
Level of education (year)	16.57±1.54	17.01±1.25	0.664
Handedness (R/L)	61/0	66/0
Eyesight	4.79±0.21	4.80±0.20	0.691
BDI	2.95±1.16	3.09±1.08	0.768
HAMA	0.10±0.30	0.08±0.27	0.370

M, male; F, female; R, right; L, left; BDI, Beck Depression Inventory; HAMA, Hamilton Anxiety Scale.

MRI

MRI data were acquired in the Imaging Center of Beijing Chaoyang Hospital Affiliated to the Capital University of Medical Sciences using a Siemens Magnetom Trio 3T MRI system (gradient field strength: 45 mT/m; gradient field switching rate: 200 mT/m/ms).

MRI images were acquired using 3D T1-weighted MPRAGE T1 sequence with a sagittal view. The dataset contains 176 slices with a slice thickness of 1 mm. The Field of view is 256 mm × 256 mm with 256×256 pixels. The sequence was repeated twice. Figure 1 shows an example volume of 176 slices.

Figure 1.

The head sagittal MR T1WI images (176 slices).

Deep learning decision

The convolutional neural network (CNN) was proposed in LeNet-5 in 1998 (9). The conventional CNN can only deal with 2D input images. Each image has a corresponding tag, which describes the outcome of the image and is entered into the network for training. The experimental data in this study were from the brain MR images. For each sample, there were 176 images corresponding to 176 slices of the MRI volume, and each sample had a corresponding tag. In order to process the volumetric datasets in this study, a deep 3D CNN model (Conv3D) was used.

Approximate 25% of objectives from each group were enrolled randomly as the testing samples, including 15 smokers and 16 non-smokers. The remaining subjects served as training samples. The MRI volumes with their tags as training samples were inputted into the Conv3D network to train the model. In the training phase, the parameters in the network are updated after optimization of the loss function. Then, the trained model was used to test the MRI volume in test dataset. The test results were got finally. This model was trained with the training data and tested on the test datasets. The accuracy, sensitivity and specificity on the smoking status were evaluated.

C3D (10) is a neural network which supports 3D convolution operations. The parameters were first pre-trained on the Sports-1M data set, and then this model was fine-tuned with the training data to predict smoking status from MR images. Figure 2 shows the flow chart of the training and testing procedures of the Conv3D model. Each training dataset on the left contains multiple images, and each test dataset on the right contains the same number of images.

Figure 2.

The train and test flow chart of Conv3D convolutional neural networks.

Because the Conv3D method is based on C3D model, the input training and test data has to follow the format that is recognized by the C3D model [1, 3, 16, 112, 112]. Each sample should have 3 groups of images, and each group contains 16 images with the resolution of 112×112 pixels. In order to maximize the information from each brain MRI volume, processing was initiated from the middle slice out of the 176 slices, every other image going in both directions was extracted. Twenty-four images were extracted from each direction, and 48 images were inputted into the model.

The CNN models are commonly used in the processing of image data, and another neural network structure (11), the recurrent neural network (RNN), is widely used in the processing of sequential datasets (12). RNN permits data persistence. Figure 3 shows the block diagram of a typical RNN module A, which receives an input xi and outputs hi. The recurrent nature of RNN allows the information to propagate to the next step. RNN can be regarded as implementing the same neural network block multiple times, and each block passes the information to the next. The chain structure in Figure 3 indicates that RNN is a sequence of operations, and is an appropriate method to process sequential data. In the past few years, RNN has achieved success in the field of voice recognition, language modeling and translation, as well as photo description.

Figure 3.

Block diagram of a typical RNN module. RNN is a sequence of operations, and is the appropriate method to process sequential data. RNN, recurrent neural network.

The long short-term memory (LSTM) is a special RNN architecture, which enables learning from experience. LSTM was proposed by Hochreiter & Schmidhuber in 1997 (13,14), and has been improved over year (15). Figure 4 shows a block diagram of the ConvLSTM model, which uses the LSTM architecture. The input data is a series of images, which correspond to the multiple slices of an MRI volume. Image features were extracted from each image using CNN models, and the features were inputted into the LSTM model. The entire volume data were trained with its corresponding tag (smoking or non-smoking). In this study, a 3-layer LSTM model was used, and the central 100 slices were applied as the input data. A 1,024-dimensional feature set was extracted from each slice using GoogleNet. The last layer of the ConvLSTM model was a full-connected layer. We also compared with support vector machine (SVM) model using the features extracted by GoogleNet. First, the features for slices are concatenated. Then, PCA was used for dimension reduction. Finally, the linear SVM was trained by the features.

Figure 4.

The flow chart of ConvLSTM model. The input data are a series of images, which correspond to the multiple slices of an MRI volume.

The deep learning model was implemented by using the Lasagne (Lightweight library to build and train neural networks in Theano. http://lasagne.readthedocs.org/) framework based on Theano (Theano Development Team. Theano: A Python framework for fast computation of mathematical expressions. http://arxiv.org/abs/1605.02688), on a workstation with NVIDIA Titan X GPU that has 12GB memory. Based on implementation and hardware configuration, the model took 0.021 s to process every image during testing.

Results

Four-fold cross validation was used in this study, and all the data mentioned were the averages from four examinations.

Training and testing results based on the Conv3D deep learning model

The accuracy curve based on the Conv3D deep learning model is shown in Figure 5. The accuracy generally increased with the increase in the number of epochs. After 55 epochs, the model became stable, and achieved high accuracy on both training and test data. Eventually, the accuracy of the Conv3D model was 93.7% on the training set and 80.6% on the test set.

Figure 5.

The accuracy of Conv3D model with respect to the number of epochs. The accuracy generally increased with the increase in the number of epochs.

Figure 6 shows the confusion matrix of the Conv3D results on the test data. The sensitivity, specificity and overall accuracy of Conv3D deep learning model was 80%, 81.3% and 80.6%.

Figure 6.

The confusion matrix of Conv3D model.

Training and test results Based on the ConvLSTM model

The accuracy curve based on ConvLSTM deep learning model is shown in Figure 7. The accuracy on the training data reached 100% beyond 70 epochs. However, the accuracy on the test data continued to increase gradually with the increase in the number of epochs. Eventually, the accuracy of ConvLSTM model was 100% on the training set and 93.5% on the test set.

Figure 7.

The accuracy of ConvLSTM model with respect to the number of epochs. The accuracy on the training data reached 100% beyond 70 epochs.

Figure 8 shows the loss value in the process of training of the ConvLSTM model. The loss magnitude decreased with the increase in the number of epochs, indicating the convergence of the model.

Figure 8.

The loss value with respect to the number of epochs. The loss magnitude decreased with the increase in the number of epochs, indicating the convergence of this model.

Figure 9 shows the confusion matrix of the ConvLSTM results on the test data. The table above calculated to the deep learning model ConvLSTM the sensitivity, specificity and overall accuracy was 93.33%, 93.75% and 93.5%.

Figure 9.

The confusion matrix of ConvLSTM model.

Comparison between Conv3D and SVM and ConvLSTM models

Figure 10 shows the comparison of ROC curves of Conv3D and ConvLSTM models. Table 2 shows the comparison of performances of Conv3D and SVM and ConvLSTM models. As shown in the table above, the image recognition accuracy based on ConvLSTM (93.5%) was better than that on SVM (83.8%) and Conv3D (80.6%).

Figure 10.

The ROC curves of Conv3D model and ConvLSTM model. ROC, receiver operating characteristic.

Table 2. The experimental data contrast.

Methods	Conv3D (%)	SVM (%)	ConvLSTM (%)
Sensitivity	80.0	84.0	93.33
Specificity	81.3	83.75	93.75
Overall accuracy	80.6	83.8	93.5

SVM, support vector machine.

Discussion

MRI can detect brain anatomical structure characteristics in vivo and reveal the relationship between human behavior and corresponding brain structure. MRI has already been used in the study on brain morphology (such as thickness, volume, brain region, etc.), gray matter development, aging and disease pathology (16,17). High-resolution thin volumetric MRI provides an important method to build the structural characteristics of living human brain, and the modern deep learning models and other artificial intelligence methods can provide us tools to analyze the images. Although RNN are originally designed for temporal features, our results suggest that their application can be expanded for classification of datasets acquired at one snapshot.

Mechelli et al. (18,19) analyzed the structure images of brain, found the correlation of brain gray matter volume among multiple regions, and concluded that the brain morphology and coordination were associated with genetic and acquired plasticity. Zhang et al. (20,21) found that the prefrontal gray thickness of smokers decreased, and the corresponding area was less stimulated by smoking, which was associated with the change in cognitive function of smokers. They analyzed gray matter thickness in 48 smokers and non-smokers with respect to the degree of smoking addiction, smoking amount and duration, and found that the left prefrontal cortex became thinner after long-term smoking, which indicated that long-term heavy smoking could lead to the destruction of gray matter. Many studies have confirmed that long-term smoking may lead to specific brain atrophy, which in turn causes the change in cognitive function (22,23). It has been reported that corpus callosum structural integrity is impaired in smokers, but it can slowly recover in a long time after smoking cessation (4,24,25). Zanchi et al. (26) investigated 18 non-smokers and 14 smokers in 2015, and used MRI diffusion analysis to evaluate white matter and combined VBM method to evaluate the cortex. They found that smoking caused the damage to the connection between right side of the former insula and the anterior cingulate.

In our previous study, DARTEL based VBM method was used to evaluate the gray matter composition in 53 controls and 53 chronic smokers in 2015. Results showed that the smokers had significantly lower brain gray matter volume after the superior temporal gyrus, insula, cingulate, precuneus as compared to controls, and the difference correlated with the amount of cigarettes per year and the age of smokers (17). However, VBM analysis method relies on group analysis, and it is difficult to examine individual brain structure change. Given the blindness of smoking cessation, high relapse rate, and other clinical difficulties, it is necessary to further understand the changes in the brain structure and function of smokers which may provide theoretical basis for individualized cessation treatment and improve the success rate of smoking cessation.

With the rapid development of artificial intelligence in recent years, machine-learning methods have been widely used for image recognition, speech recognition and other procedures. In the present study, we for the first time used Conv3D and ConvLSTM models to predict nicotine dependence based on brain MR images. The ConvLSTM model achieved an overall accuracy of 93.5%, which was better than that of Conv3D model (80.6%). The present study showed the accuracy was significantly higher than that reported in a previous study (64.04%) on SVM (27). In addition, our methods do not require manual extraction of the features from the images. The models rely on CNN to extract features from the volumetric datasets, which may guide radiologists to study the brain composition of smokers and predict the effectiveness of cessation plans in the future.

In conclusion, state-of-the-art deep learning models are used to predict smoking status based on MRI images of individual human subject and provide a tool for the intelligent medical image interpretation. Though there is no direct clinical issues, we can see that the potential of deep learning methods to assist the doctor in the aspect of image feature recognition. It provides a tool to analyze nicotine dependence and develop cessation plans. However, more studies with larger sample size are needed to refine our models to differentiate nicotine dependence of different degrees and to confirm our findings.

Ethnical Statement: The study was conducted in accordance with the Committee for Human Research in our institution and followed all regulations (No. 12-KE-21). Informed consent was obtained before scans.