Classification of Activated Microglia by Convolutional Neural Networks

Abstract

Microglia are the resident macrophages in the central nervous system. Brain injuries, such as traumatic brain injury, hypoxia, and stroke, can induce inflammatory responses accompanying microglial activation. The morphology of microglia is notably diverse and is one of the prominent manifestations during activation. In this study, we proposed to detect the activated microglia in immunohistochemistry images by convolutional neural networks (CNN). 2D Iba1 images (40μm) were acquired from a control and a cardiac arrest treated Sprague-Dawley rat brain by a scanning microscope using a 20X objective. The training data were a collection of 54,333 single-cell images obtained from the cortex and midbrain areas, and curated by experienced neuroscientists. Results were compared between CNNs with different architectures, including Resnet18, Resnet50, Resnet101, and support vector machine (SVM) classifiers. The highest model performance was found by Resnet18, trained after 120 epochs with a classification accuracy of 95.5%. The findings indicate a potential application for using CNN in quantitative analysis of microglial morphology over regional difference in a large brain section.

I. Introduction

As resident macrophages in the central nervous system, microglia are activated in response to neuroinflammation in many brain diseases and injuries, such as traumatic brain injury, stroke, and hypoxic-ischemic brain injury after cardiac arrest (CA) [1]–[4]. Microglial morphology is closely related to its activation status, hence the morphological analysis of microglia has been widely used to provide quantitative indices of neuroinflammation [5]–[7]. Several machine learning and deep learning algorithms, with supervised or unsupervised data, have been used to characterize the state of microglia in various animal disease models. Unsupervised algorithms (e.g., K-means or hierarchical clustering methods) require cell morphology parameters calculated from segmented cell body images to identify the activation states of microglia [8]–[12]. On the other hand, supervised learning algorithms, such as Support Vector Machines (SVM) or Convolutional Neural Networks (CNN), require extensive manual labeling of segmented cell images [13], [14]. Silburt and Aubert [15] trained an SVM to identify activated microglia and astrocytes, using cellular features extracted from sliding-window images of mouse brain slices with focused ultrasound treatment. However, existing models are mostly limited to detecting microglia in a relatively small and uniform region with simple brain structure. A few models attempt to detect region-specific microglial activation in a large field of view, but lose the resolution necessary to detail individual cells.

In this study, we propose to utilize CNNs for the detection of activated microglia in two pathological conditions from different brain regions. The training data were prepared for individual microglia images curated from the cortex and midbrain of control and cardiac arrested rat brains. Results of residual neural networks were compared between Resnet18, Resnet50, Resnet101, and a three-convolution layer CNN (CNN3CL). We also compared the CNNs with SVM classifiers trained by raw cell images or by the features extracted from Resent18. Results show superior performance with Resnet18 trained after 120 epochs for microglia classification.

II. Materials and Methods

A. Animal model and immunohistochemistry stain

Ischemic brain injury was induced in one-week-old female Sprague-Dawley rats by 12-minute CA surgery and compared with the non-surgical control brain [4]. All rat brains were perfused with 4% paraformaldehyde, extracted, then sectioned at 40 μm thickness. Immunohistochemistry (IHC) staining was performed with the primary ionized calcium-binding adapter (Iba1) antibody, followed by brightfield microscope images taken by 20X objective in 0.464 μm/pixel resolution. Slices located at ~5 mm posterior to the bregma were studied.

B. Image data sets

The locations of microglia were obtained by applying multiple threshold steps on the grayscale image followed by manual corrections to fine-tune the bounding box of each cell. Each single-cell image was then resized to 120×120 pixels by zero-padding for CNNs. Classification of single-cell image data was evaluated by (1) four classes according to the pathological-regional conditions: the control cortex (Ctrl-COR), control midbrain (Ctrl-MB), CA cortex (CA-COR), and CA midbrain (CA-MB), or (2) two classes by pathological conditions: Ctrl and CA. In total, 18524, 7928, 18078, and 9803 images were taken from Ctrl-COR, Ctrl-MB, CA-COR, and CA-MB, respectively. 6000 and 12000 images were randomly selected in each of the four-class and two-class data sets, respectively, for training and the remainders for validation.

C. CNN and SVM classifiers

The pre-trained Resnet18, Resnet50, and Resnet101 for 1000 object categories were modified with the input image size to 120×120 pixels and number of the classes in the output layer for transfer learning [16], [17]. For comparison, a three convolutional layer CNN, CNN3CL, consisted of three repeated convolution - Rectified Linear Unit (ReLu) - max-pooling layers, followed by fully connected - ReLu - fully connected layers, and a softmax layer to the output, which had been trained from random biases and weights [18].

All CNNs were trained by homemade Matlab programs using stochastic gradient descent with momentum (SGDM) algorithm, with the momentum = 0.9, piecewise learning rate schedule, shuffle at every epoch, and the initial learning rate set to 0.001 for CNN3CL and 0.1 for the Residual Networks. Training times performed on an NVIDIA GeForce GTX 2070 GPU with 8 GB memory for 60 epochs (eps) were 24, 43, 143, and 292 minutes for CNN3CL, Resnet18, Resnet50, and Resnet101, respectively. Moreover, the Resnet18-120eps was trained with an additional 60 epochs, totaling 120 eps, in 48 minutes from the previous weights.

Multi-class SVM classifiers with a linear kernel were trained by a fast stochastic gradient descent solver [19] using the labeled grayscale images (SVM@images) or the features extracted from layers in Resnet18-60eps or Resnet18-120eps.

III. Results

A. Four-class classification

Activated microglia were present in the hypertrophic or amoeboid form in the inflamed brain after CA, whereas resting microglia in the control tissue were ramified morphology [4], [20], [21]. Fig. 1 shows the single microglia images in bounding boxes from the cortex and midbrain of the control and CA brains. Instead of labeling individual microglia through their morphology manually, we created the label based on the pathological condition, CA or control, and their location in the brain, cortex or midbrain. Microglia showed large diversity and morphological varieties in all four classes: Ctrl-COR, Ctrl-MB, CA-COR, and CA-MB.

Fig. 1.

Single-cell images prepared from the cortex and midbrain of control and cardiac arrest (CA) surgery rat brains, (A) Ctrl-Cortex, (B) CA-Cortex, (C) Ctrl-Midbrain (D) CA-Midbrain.

Microglia in the CA brains were less branched than those in the control brains. Although the morphological differences of cells between the four classes appeared to be smaller than the differences within each class, the CNNs were able to distinguish the subtle features among the four classes. As shown in Table 1, in the tests of different CNNs, the best training accuracy and F1 were 95.5% and 0.96 from Resnet18-120eps, respectively, while the best validation accuracy and F1 were 67.7% and 0.64 from Resnet18-60eps.

TABLE I.

Accuracy and F1 score of CNNs and SVM results

Networks:╲Datasets:	Four Classes (Ctrl - COR/MB, CA - COR/MB)				Two classes (Ctrl and CA)
	Training		Validation		Training		Validation
	Accuracy	F1	Accuracy	F1	Accuracy	F1	Accuracy	F1
CNN3CL (60eps)	68.5%	0.69	56.7%	0.55	84.1%	0.84	77.3%	0.77
Resnet18-60eps	73.6%	0.74	67.7%	0.64	92.3%	0.92	79.7%	0.80
Resnet18-120eps	95.5%	0.96	63.5%	0.59	98.8%	0.99	78.4%	0.78
Resnet50-60eps	93.5%	0.94	65.1%	0.62	97.7%	0.98	78.0%	0.78
Resnet101-60eps	88.5%	0.89	59.4%	0.53	93.3%	0.94	79.3%	0.79
SVM@images	40.1%	0.40	26.8%	0.29	61.5%	0.62	58.1%	0.58
SVM@Resnet18-120eps-L12	77.7%	0.78	63.7%	0.58	80.7%	0.81	69.7%	0.70

The training accuracy of Resnet18-120eps (95.5%) was higher than that of Resnet18-60eps (73.6%) yet their validation accuracies were opposite, at 63.5% vs. 67.7%. The relatively high training accuracies suggest the overfitting of CNNs. Fig. 2 illustrates the confusion matrices of the four classes from Resnet18-60eps and Resnet18-120eps on training and validation sets. The true positive rates (TPR) and positive predictive values (PPV) of Resnet18-120eps were all higher than 90% in the training dataset. In contrast, Resnet18-60eps misidentified more microglia in Ctrl-COR as Ctrl-MB, whereas microglia in CA-MB were mislabeled as CA-COR and vice versa. The difference implied that some microglia may have similar morphology in the cortex and midbrain.

Fig. 2.

Confusion matrix of four testing classes as shown in Figure 1. (A) Resnet18 - 60 epochs of training, (B) Resnet18 - 60 epochs of validation, (C) Resnet18 -120 epochs of training, (D) Resnet18 - 120 epochs of validation datasets. TPR: true positive rate; FNR: false negative rate; PPV: positive predictive value; FDR: false discovery rate.

B. Two-class classification

Fig. 3 and the second part of Table 1 demonstrate the results of CNNs for the detection of microglia in two classes (Ctrl vs CA). Compared with the four-class results, both the accuracy and F1 score of the two-class CNNs improved, especially the validation accuracy increased from 56–67% to 77–79%. As shown in Fig. 3, Resnet18-60eps and Resnet18-120eps achieved 90% and 98% of TPR and PPV, respectively, in the training data, while they were over 75% in the validation dataset. Results suggest that CNN can recognize different microglia morphology in the CA brain with high accuracy.

Fig. 3.

Confusion matrix of two testing classes in control and cardiac arrested (CA) brain tissues. (A) Resnet18 - 60 epochs of training, (B) Resnet18 - 60 epochs of validation, (C) Resnet18 -120 epochs of training, (D) Resnet18 - 120 epochs of validation datasets. TPR: true positive rate; FNR: false negative rate; PPV: positive predictive value; FDR: false discovery rate.

C. Prediction Images of CNNs

As demonstrated in Fig. 4, images from the cortex and midbrain of the control and CA brain were fused with the colored bounding boxes of the four-class ground truth labels and the trained networks: green for Ctrl-COR, blue for Ctrl-MB, red for CA-COR, and yellow for CA-MB. Labeled images showed a high correlation to the prediction of each CNN model. Resnet50 and Resnet101 had lower accuracy compared to Resnet18, which indicates increasing the complexity of the model has no benefit to the current dataset. Meanwhile, the underfitting of CNN3CL implied that the features between the cortex and midbrain of either control or CA brain (Fig. 4B) cannot be captured accurately by the only three convolutional layers.

Fig. 4.

Predictions of CNNs and SVMs in the cortex and midbrain. (A) Ground truth, (B) CNN3CL (CNN with three convolutional layer), (C) Resnet18 - 60 epochs (D) Resnet18 - 120 epochs, (E) Resnet50 - 60 epochs, (F) Resnet101 - 60 epochs, (G) SVM classifier trained from labeled images, (H) SVM classifier from features of layer 12 of Resnet18 - 120 epochs.

D. SVM from layer features of Resnet18

The SVM classifiers were trained by the same four-class training images (Fig. 4G) or by the extracted features from Resnet18. As shown in Fig. 5A, the accuracy of the SVM classifiers increased with the depth of the feature extraction layers. Generally, the SVM classifiers from Resnet18-120eps had higher training accuracy than those from Resnet18-60eps after the features were extracted from about layer 20, but their accuracies barely improved. Fig. 4G and 4H show the predicted images of the SVM classifier (SVM@images) trained by the raw images or by the features extracted from layer 12 of Resnet18-120eps (SVM@Resnet18-120eps-L12). The training accuracy of SVM@images was only 40.13%, which means it was difficult to classify images without extracting image features. The results from SVM@Resnet18-120eps-L12 were similar to CNN3CL because the architecture of the first 12 layers in Resnet18 also contains three convolution layers. Fig. 5B and 5C show examples of the four-class cell images and their feature maps obtained from layer 12 of Resnet18-120ep. The high-intensity values of the soma and cell body, as well as the large area of surrounding background, were the major features contributing to the judgments of the CNNs and SVM classifiers.

Fig. 5.

SVM classifier using CNN features. (A) Training and validation accuracies of SVM classifiers by features from the layer of Resnet18 - 120 epochs, the layer 0 is from the original images, (B) and (C) microglia images of four classes and their 64 feature maps (size of 30×30) from layer 12 of Resnet18 - 120 epochs (a.u.: arbitrary units).

IV. Discussion

CNN and SVMs are increasingly used for biomedical image classification [1], [13]–[15], [22]–[24]. For the analysis of microglia images, most studies utilize a classification system based on the morphological phenotypes of microglia, such as ramified, hyper-ramified, activated, rod, and amoeboid morphotypes [13], [14]. In this study, CNNs were tested on identifying microglia based on a classification definition given simply by anatomy and pathological conditions. Results showed that CNNs can accurately identify resting microglia in the control brain as well as mostly activated microglia in the CA brains without the inputs of microglial morphotypes.

The trained Resnet had lower accuracy in validation, indicating overfitting of the CNN, possibly due to the complexity and diversity of microglia and the imbalance in the number of cells between each class in the validation set (24602 cells from the cortex and 5731 cells from the midbrain). It is possible to prevent overfitting by adding more training data or training with data augmentation or cross-validation [25], [26]. SVM was able to classify the microglia based on the features extracted from segmentation results, such as cell area, cell perimeter, and cell circularity [13]. SVM performed inferiorly when trained with the cell images directly, because each image pixel was treated as an independent variable. This may result in the loss of correlation between pixels (Fig. 4G). When using the features extracted from Resnet18, SVM can classify microglia adequately, which demonstrates the value of feature extraction by CNNs. Accordingly, SVM may be useful to understand how a CNN works by evaluating the features extracted from different CNN, therefore helping the design of a better architecture for specific datasets.

It is worthy of note that the four- or two-class ground truth systems that were tested in this study did not consider the fact that microglia may have the same morphotype in different regions, or even in different pathological conditions. The classification system in the tested ground truths did not reflect the actual spatial distribution of microglia by morphology in the brain tissues. Interestingly, even with the ground truths that defined microglia inaccurately, the CNNs still recognized the morphological features from the majority of microglia that were correctly defined, which were the resting microglia in the control brain, and the activated microglia in the CA brain. CNN3CL and Resnet18-60eps both showed that some microglia in the center cortex appeared similar to those in the midbrain (Fig. 4B and 4C). The “false” classification of microglia in the cortex and midbrain by CNNs revealed that the microglia features were not properly classified by the ground truth. As suggested by Fernández-Arjona’s study [11], microglia should be classified by a morphometric parameter grading system in a region-specific manner. These results suggested that it may be possible to utilize CNNs and a properly defined classification system to create a morphological atlas by detecting the distributions of specific cell features in the brain. Furthermore, microglia morphology can vary not only by regional and pathological conditions, but also by different staining methods, slice thickness, microscope focus, camera exposure and color tone, etc. More training data is required to enable CNNs to be widely applicable to analyze stained slices of varying image quality.

V. Conclusion

In this work, CNNs successfully detected microglial activation in the CA treated brain compared to that of the control brain. The CNNs also distinguished microglia between cortex and midbrain, which suggests a potential for CNN to perform precise regional quantification of microglia in a large brain section.