Secondary Pulmonary Tuberculosis Recognition by 4-Direction Varying-Distance GLCM and Fuzzy SVM

Abstract

Tuberculosis (TB) is an infectious disease caused by Mycobacterium tuberculosis. Our study investigates the recognition of secondary pulmonary (SPTB). A novel F3 model is proposed. The first F means using a four-direction varying-distance gray-level co-occurrence matrix (FDVDGLCM) to analyze the chest CT images; the second F means a five-property feature set (FPFS) from the FDVDGLCM results; the third F means fuzzy support vector machine (FSVM). Besides, a slight adaption of multiple-way data augmentation is used to boost the training set. The 10 runs of 10-fold cross-validation demonstrate that this F3 model achieves a sensitivity of 93.68% ± 1.75%, a specificity of 94.17% ± 1.68%, a precision of 94.17% ± 1.55%, an accuracy of 93.92% ± 1.05%, an F1 score of 93.91% ± 1.07%, an MCC of 87.88% ± 2.09%, and an FMI of 93.92% ± 1.06%. The AUC is 0.9624. The FSVM can give better performance than ordinary SVM. The proposed F3 model is superior to six state-of-the-art SPTB recognition models.

Introduction

Tuberculosis (TB) is an infectious disease caused by Mycobacterium tuberculosis [1]. TB outside of the lung is called extrapulmonary TB, such as bones or other organs. TB infects the lungs typically; thus is named pulmonary TB (PTB) [2].

Typical symptoms of PTB are high temperature [3], night sweats, extreme fatigue, weight loss, poor appetite, etc. Respiratory signs comprise hemoptysis, cough, expectoration, chest tightness, chest pain, or dyspnea in different grades. In the world, PTB is one of the top ten fatality causes and the foremost cause from a single contagious agent.

PTB can be classified into primary PTB and secondary PTB (SPTB) [4]. we fix on SPTB. The most common locations of SPTB lesions: apical and posterior segments of the upper lobe and dorsal segments of the lower lobe. CT manifestations: Exudative patchy shadows of varying density, miliary shadows, tuberculosis bulbs, caseous pneumonia, proliferative nodular shadows, satellite foci, cavities, tree bud signs, bronchial dissemination, cord strips and calcifications, and numerous other forms.

In current years, experts are prone to harness artificial intelligence (AI) methods to create smart SPTB diagnosis models. For instance, Bagci, Kubler, Luna, Jain, Bishai and Mollura (2013) [5] created a computer-aided detection and quantification model for cavitary TB. Based on a support vector machine (SVM), the authors proposed a shape-based SVM (SSVM) to discover airways and cavities. Li, Huang and Jin (2018) [6] presented an AE-CNN classifier, which stands for autoencoder and convolutional neural network. James-Reynolds, Currie and Gao (2020) [7] proposed a depth-ResNet for TB severity level analysis. Xie, Wu, Han, Wang, Wu, Cui, Feng, Zhu and Chen (2020) [8] offered a computer-aided system (CAS) for the detection of TB in radiographs. Based on convolutional neural network (CNN), Zhang, Nayak, Zhang and Wang (2020) [9] presented an improved CNN (ICNN) for TB recognition. Rajpurkar, O'Connell, Schechter, Asnani, Li, Kiani, Ball, Mendelson, Maartens, van Hoving, Griesel, Ng, Boyles and Lungren (2020) [10] presented a CheXaid model to assist physicians in making the diagnosis of TB.

Nevertheless, the performances of the above methods still have rooms to improve. In order to recognize SPTB more accurately, we propose a new SPTB recognition system in this study. The motivation is that gray-level co-occurrence matrix (GLCM)—an image texture descriptor—may produce a measure of chest CT images to distinguish SPTB from healthy people’s scanning. The contributions of our method are five points:

• (i)A 4-direction varying-distance GLCM (FDVDGLCM) is introduced as the feature extractor.
• (ii)A 5-property feature set is extracted from FDVDGLCM matrix.
• (iii)The fuzzy support vector machine (FSVM) is harnessed as the classifier.
• (iv)A slight adaption of multiple-way data augmentation is used to boost the training set.
• (v)Multiple runs of cross-validation results show that our model is superior to six state-of-the-art models.

Dataset and preprocessing

The raw dataset V₁ in this study is from Ref. [9], in which 66 SPTB and 66 healthy control (HC) are enrolled, and 144 SPTB images and 144 HC images are generated from the 3D chest CT scanning. Table 1 shows the demographics of our dataset.

Table 1.

Demographics of our dataset

Class	No. of subjects	No. of images
SPTB	68	144
HC	66	144

A four-step preprocessing is carried out as described in Fig. 1. First, the raw image set is gray-scaled to V₂, then, histogram-stretching (HS) is run on V₂ to get V₃. Afterwards, cropping is performed to remove the texts and rulers and check-up beds in the images, and we obtain V₄. Finally, all the images are downscaled to the size of [W, W]. The final image set is symbolized V₅. Algorithm 1 itemizes the pseudocode of the four-step preprocessing.

Fig. 1.

Diagram of preprocessing

Figure 2a, c show one sample of SPTB and one sample of HC. Figure 2b delineates the lesions within five red ellipses. Multiple nodule shadows of different sizes are seen in the posterior segment of the upper lobe tip of both lungs.

Fig. 2.

Two samples of our preprocessed dataset

Methodology

Gray-level co-occurrence matrix

Table 2 presents the acronym list for ease of reading. Suppose we have a gray-level image I with the size of m × n and p different pixel values, the gray-level co-occurrence matrix (GLCM) computes how often pairs of pixels with a specific value and offset occur. GLCM is a powerful approach to texture analysis with loads of medical applications such as lung cancer classification [11], similar to our task.

Table 2.

Abbreviation list

Abbreviation	Full title
AI	artificial intelligence
AUC	the area under the curve
CNN	convolutional neural network
DA	data augmentation
DF	dual form
FA	fuzzy altitude
FD-GLCM	four-dimension GLCM
FDVDGLCM	four-direction varying-distance GLCM
FMF	fuzzy membership function
FMI	Fowlkes–Mallows index
FPFS	five-property feature set
FSVM	fuzzy SVM
GLCM	gray-level co-occurrence matrix
HC	healthy control
HO	hyperparameter optimization
LR	logistic regression
MCC	Matthews correlation coefficient
MDA	multiple-way data augmentation
MSD	mean and standard deviation
PTB	pulmonary TB
ROC	receiver operating characteristic
SPTB	second pulmonary tuberculosis
SVM	support vector machine
TB	tuberculosis

The GLCM matrix G is defined as

$$G_{\mathrm{\Delta }x,\mathrm{\Delta }y}\left(i,,,j\right)={\sum }_{x=1}^n{\sum }_{y=1}^m\left\{\begin{array}{cc}1& \mathrm{if}I\left(x,,,y\right)=i\wedge I\left(x,+,\mathrm{\Delta },x,,,y,+,\mathrm{\Delta },y\right)=j\\ 0& \mathrm{otherwise}\end{array}\right)$$ 1

where ∧ means and operation; i and j are pixel values; x and y are spatial positions in the image I; (Δx, Δy) are the offsets defining the spatial relation. Note the GLCM matrix is a p × p matrix, i.e., G_{Δx, Δy} ∈ ℝ^p × p. Some advanced GLCM techniques, such as local-GLCM [12], may be tested in our future study.

Four-direction mechanism

In practice, the (x, y) are often replaced by row and column coordinates (r, c). Subsequently, the offset is expressed in the form of (Δr, Δc). As can be seen here, the offset [0, 1] corresponds to 0°, similarly, [−1, 1], [−1, 0], and [−1, −1] correspond to 45°, 90°, and 135°, respectively. The directional angles of GLCM in our study are set to the following four directional angles.

$$\theta =\left[0,{}^{\circ },,,45,{}^{\circ },,,90,{}^{\circ },,,135,{}^{\circ }\right]$$ 2

Other angles [180°, 225°, 270°, 315°] can be chosen, but they will give the same results as [0°, 45°, 90°, 135°], respectively. That is the reason why we only choose this four-direction mechanism (Fig. 3).

Fig. 3.

Diagram of four-direction (r, c)-based offset

The following Fig. 4 shows an example of four-direction GLCM (4D-GLCM). For example, Fig. 4a shows the element in G_{0, 1}(3, 1) = 1, because there is one instance in the left-hand image where one horizontally adjacent pixels have the values of 3 and 1. The element G_{0, 1}(7, 4) = 2 is since there exist two occurrences in the raw image where two horizontally adjacent pixels have the values of 7 and 4.

Fig. 4.

A toy example of 4-direction GLCM

The varying-distance mechanism

All the above four offsets have a distance d of 1, as clearly shown in Fig. 3. Srivastava, Rajitha, Agarwal and Singh (2020) [13] introduced the varying-distance mechanism. In their paper, d ranges from 1 to the width of the image. Nevertheless, we find by the trial-and-error method that d assigned with a value larger than three does not improve the performance.

Figure 5 shows the varying distance where the maximum distance if D = 3. Hence, we can write the pseudocode of our four-direction varying-distance GLCM (FDVDGLCM) in Algorithm 2. Note, the offset (Δr, Δc) can be expressed at the form of (d, θ) and each GLCM matrix G_{d, a} ∈ ℝ^p × p. The final overall GLCM matrix $\mathbb{G}\in {\mathbb{R}}^{p\times p\times 4\times D}$ can be expressed as

$$\mathbb{G}=C_{d=1}^D{C}_{\theta =\left(0,45,90,135\right)}{G}_{d,\theta },$$ 3

where C means the concatenation function.

Fig. 5.

The diagram of varying-distance mechanism 1 ≤ d ≤ D = 3

Figure 6 displays a toy example of varying distance in which d = 1, 2, 3. As we can observe, the larger the distance d is, the GLCM will analyze pairs of voxels further away from each other.

Fig. 6.

A toy example of varying-distance d = 1, 2, 3

Five-property feature set of FDVDGLCM

Based on FDVDGLCM, we calculate a five-property feature set (FPFS). This study sets the number of properties as N_F = 5. First is the contrast, indicating the linear dependency of gray levels of two neighboring pixels. For constant image, the contrast is 0.

$${\mathrm{Contrast}}_{d,\theta }={\sum }_{i=1}^p{\sum }_{j=1}^p{\left(i,-,j\right)}^2{G}_{d,\theta }\left(i,,,j\right)$$ 4

Second is the correlation, measuring how correlated a pixel is to its neighbor over the whole GLCM image.

$${\mathrm{Correlation}}_{d,\theta }={\sum }_{i=1}^p{\sum }_{j=1}^p\frac{\left(i,-,{\mu }_i\right)\left(j,-,{\mu }_j\right)G_{d,\theta }\left(i,,,j\right)}{{\sigma }_i{\sigma }_j}$$ 5

where μ_i and μ_j are the GLCM mean defined as

$${\mu }_i={\sum }_{i=1}^p{\sum }_{j=1}^{p}i\times G_{d,\theta }\left(i,,,j\right)$$ 6

$${\mu }_j={\sum }_{i=1}^p{\sum }_{j=1}^{p}j\times G_{d,\theta }\left(i,,,j\right)$$ 7

The GLCM variances σ_i and σ_j are defined as:

$${\sigma }_i=\sqrt{{\sum }_{i=1}^p{\sum }_{j=1}^p{\left(i,-,{\mu }_i\right)}^2{G}_{d,\theta }\left(i,,,j\right)}$$ 8

$${\sigma }_i=\sqrt{{\sum }_{i=1}^p{\sum }_{j=1}^p{\left(j,-,{\mu }_j\right)}^2{G}_{d,\theta }\left(i,,,j\right)}$$ 9

Note, correlation is within the range of [−1, 1]. The −1 stands for a perfectly negative correlation, while +1 is a perfectly positive correlation.

The third is the energy property, calculating the sum of squared GLCM elements. It measures the textural uniformity of an image. Its value ranges from 0 to 1. The definition is

$${\mathrm{Energy}}_{d,\theta }={\sum }_{i=1}^p{\sum }_{j=1}^p{G}_{d,\theta }^2\left(i,,,j\right)$$ 10

The fourth is homogeneity, a value that measures the closeness of the distribution of elements in the GLCM to the GLCM diagonal. Again, its range is from 0 to 1. The definition is:

$${\mathrm{Homogeneity}}_{d,\theta }={\sum }_{i=1}^p{\sum }_{j=1}^p\frac{G_{d,\theta }\left(i,,,j\right)}{1+\left|i,-,j\right|}$$ 11

The above four properties are commonly used to extract features in GLCM. However, Athertya and Kumar (2021) [14] stated that entropy-based measure obtained from GLCM alone is sufficient to detect focal changes, indicating entropy's importance. Hence, we introduce the final property

$${\mathrm{Entropy}}_{d,\theta }=-{\sum }_{i=1}^p{\sum }_{j=1}^p{G}_{d,\theta }\left(i,,,j\right)\times ln{G}_{d,\theta }\left(i,,,j\right)$$ 12

The five properties run on all the GLCM matrix G_{d, θ} are concatenated and form the FPFS $\mathbb{F}$.

$$\mathbb{F}=C_{d=1}^D{C}_{\theta =\left(0,45,90,135\right)}\left\{\begin{array}{c}\mathrm{Contrast}\left(G_{d,\theta }\right)\hfill \\ \mathrm{Correlation}\left(G_{d,\theta }\right)\hfill \\ \mathrm{Energy}\left(G_{d,\theta }\right)\hfill \\ \mathrm{Homogeneity}\left(G_{d,\theta }\right)\hfill \\ \mathrm{Entropy}\left(G_{d,\theta }\right)\hfill \end{array}\right\},$$ 13

Support vector machine

The FPFS is sent to the classifier. Note, we do not choose deep learning [15, 16] due to the inherent characteristics and dimensions of FPFS $\mathbb{F}$. Instead, we find that traditional classifier—the support vector machine (SVM) [17]—exhibits excellent performances in handling the FPFS describe in Section 3.4. Additionally, a variant of SVM—fuzzy SVM (FSVM)—is introduced to our method.

The SVM is a standard robust classifier [18]. Suppose there are two classes of positive and negative, and the dataset is

$$\left\{\left(g_n,,,t_n\right),|,p_n,\in ,{\mathbb{R}}^d\right\},n=1,\cdots ,N,$$ 14

where N stands for the number of samples, g_n the feature vector of n-th sample data, d the dimension of input features, and t_n the corresponding target label.

$$t_n=\left\{\begin{array}{cc}+1& \mathrm{class}=\mathrm{positive}\\ -1& \mathrm{class}=\mathrm{negative}\end{array}\right).$$ 15

The goal of SVM [19] is to build a hyperplane with the dimensions of (d − 1) that can part the positive class from the negative class [20]. The SVM model is alike to logistic regression (LR), because both are driven by linear functions. The hyperplane of SVM can be written as:

$${\mathit{w}}^{T}g-b=0,$$ 16

where w and b have analogous mathematical meanings as those in LR. w stands for the weights and b the bias. The solutions of w and b are:

$$\left\{\begin{array}{c}\underset{\mathit{w},b}{min}\frac{1}{2}{∥\mathit{w}∥}^2\hfill \\ \mathrm{s}.\mathrm{t}.t_n\left({\mathit{w}}^T,g_n,-,b\right)\ge 1,n=1,\cdots ,N\hfill \end{array}\right).$$ 17

In cases that input features g_n encloses noises, we utilize the “soft margin” method, which adds the positive slack vector ρ

$$\mathbf{\rho }=\left({\rho }_1,,,{\rho }_2,,,\cdots ,,,{\rho }_n,,,\cdots ,,,{\rho }_N\right).$$ 18

Equation (17) is then rephrased as:

$$\left\{\begin{array}{c}\underset{\mathit{w},\mathbf{\rho },b}{min}\frac{1}{2}{∥\mathit{w}∥}^2+L{\mathit{e}}^T\mathit{\rho }\hfill \\ \mathrm{s}.\mathrm{t}.\left\{\begin{array}{c}{t}_n\left({\mathit{w}}^T,g_n,-,b\right)\ge 1-{\rho }_n\hfill \\ {\rho }_n\ge 0\hfill \end{array}\right),,n=1,\cdots ,N\hfill \end{array}\right),$$ 19

where e stands for a vector of ones of N-dimension and L the error penalty [21].

The constraint optimization problem in Eq. (19) is solved by the Lagrange multiplier [22] method as:

$$\underset{\mathbf{w},\mathbf{\rho },b}{min}\underset{\phi ,\gamma }{max}\left\{\frac{1}{2},{∥\mathbf{w}∥}^2,+,L,{\mathit{e}}^T,\mathbf{\rho },-,{\sum }_{n=1}^N,{\phi }_n,\left[t_n,\left({\mathbf{w}}^T,g_n,-,b\right),+,{\rho }_n,-,1\right],-,{\sum }_{n=1}^N,{\rho }_n,{\gamma }_n\right\},$$ 20

The dual form (DF) [23] is utilized to elucidate Eq. (20) because the min-max optimization is challenging. We get:

$$\left\{\begin{array}{c}\underset{\phi }{max}{\sum }_{n=1}^N{\phi }_n-\frac{1}{2}{\sum }_{n=1}^N{\sum }_{m=1}^N{\phi }_m{\phi }_n{t}_m{t}_n{g}_m^T{g}_n\hfill \\ \mathrm{s}.\mathrm{t}.\left\{\begin{array}{c}L\ge {\phi }_n\ge 0\hfill \\ {\sum }_{n=1}^N{t}_n{\phi }_n=0\hfill \end{array}\right),n=1,\cdots ,N\hfill \end{array}\right).$$ 21

Fuzzy membership function

The fuzzy membership function (FMF) [24] is introduced to each training sample (g_n, t_n). Note, the raw training sample includes only the input and target [25], while the fuzzy sample data comprises the input, the target, and the fuzzy altitude (FA) [26]. Mathematically, Eq. (14) is generalized to

$$\left\{\begin{array}{c}\left\{\left(p_n,,,t_n,,,a_n\right),|,p_n,\in ,{\mathbb{R}}^d\right\},n=1,\cdots ,N\hfill \\ \mathrm{s}.\mathrm{t}.,0<a_n\le 1\hfill \end{array}\right)$$ 22

where a_n is the FA of the sample g_n towards class t_n.

Let the mean of class +1 be m₊ and the mean of class −1 be m₋. We have [27]:

$$m_{+}=\underset{n}{\mathrm{mean}}\left(g_n,|,t_n,=,+,1\right),$$ 23

$$m_{-}=\underset{n}{\mathrm{mean}}\left(g_n,|,t_n,=,-,1\right).$$ 24

We can deduce the radii of class +1 and class −1 as:

$$r_{+}=\underset{\left\{g_n,:,t_n,=,+,1\right\}}{max}\left|m_{+},-,g_n\right|,$$ 25

$$r_{-}=\underset{\left\{g_n,:,t_n,=,-,1\right\}}{max}\left|m_{-},-,g_n\right|.$$ 26

The FMF is described on the means and radii of two classes t_n =  + 1 and t_n =  − 1:

$$a_n=\left\{\begin{array}{cc}1-\frac{\left|m_{+},-,g_n\right|}{r_{+}+\upsilon }& t_n=+1\\ 1-\frac{\left|m_{-},-,g_n\right|}{r_{-}+\upsilon }& t_n=-1\end{array}\right),n=1,2,\cdots ,N$$ 27

Here υ is a positive parameter, to make sure the FA a_n is larger than zero, viz., a_n > 0.

Fuzzy SVM

Assume vector a is the fuzzy membership vector [28] of all samples:

$$\mathit{a}=\left(a_1,,,a_2,,,\cdots ,,,a_n,,,\cdots ,,,a_N\right)$$ 28

We can deduce the hyperplane [29] of FSVM as

$$\left\{\begin{array}{c}\underset{\mathit{w},\mathbf{\rho },b}{min}\frac{1}{2}{∥\mathit{w}∥}^2+L{\mathit{a}}^T\mathit{\rho }\hfill \\ \mathrm{s}.\mathrm{t}.\left\{\begin{array}{c}{t}_n\left({\mathit{w}}^T,g_n,-,b\right)\ge 1-{\rho }_n\hfill \\ {\rho }_n\ge 0\hfill \end{array}\right),n=1,\cdots ,N\hfill \end{array}\right)$$ 29

Note, a smaller a_n decreases the influence of the slack vector ρ_n so that the corresponding sample g_n is deemed less substantial. The Lagrangian [30] is attained similarly as

$$\underset{\mathbf{w},\mathbf{\rho },b}{min}\underset{\phi ,\gamma }{max}\left\{\frac{1}{2},{∥\mathbf{w}∥}^2,+,L,{\mathbf{a}}^T,\mathbf{\rho },-,{\sum }_{n=1}^N,{\phi }_n,\left[t_n,\left({\mathbf{w}}^T,g_n,-,b\right),+,{\rho }_n,-,1\right],-,{\sum }_{n=1}^N,{\rho }_n,{\gamma }_n\right\}$$ 30

Its DF is attained as

$$\left\{\begin{array}{c}\underset{\phi }{max}{\sum }_{n=1}^N{\phi }_n-\frac{1}{2}{\sum }_{n=1}^N{\sum }_{m=1}^N{\phi }_m{\phi }_n{t}_m{t}_n{g}_m^T{g}_n\hfill \\ \mathrm{s}.\mathrm{t}.\left\{\begin{array}{c}0\le {\phi }_n\le a_{n}L\\ {\sum }_{n=1}^N{t}_n{\phi }_n=0\end{array},,,n,=,1,,,\cdots ,,,N\right)\hfill \end{array}\right)$$ 31

Implementation of our F3 model

If we summarize all the techniques, the pseudocode of our F3 model can be recapped in Algorithm 3. We dub our model F3. The first F stands for the FDVDGLCM, the second F for the FPFS, and the last for FSVM. From Algorithm 3, we observe that our F3 model can work in two phases. Within Phase I, the input is a dataset D, and the output is the trained classifier $\mathbb{M}$. Within Phase II, the trained classifier $\mathbb{M}$ is harnessed to classify the category of the input image I. The output is the label O.

Cross-validation and multiple-way data augmentation

V-fold cross-validation is employed to run our F3 model due to the relatively small size of our dataset. Hold-out validation is not chosen since it works well for a larger dataset. In r-th run (1 ≤ r ≤ R) of the V-fold cross-validation, the whole dataset D (144 SPTB image and 144 HC images) is divided into V folds.

$$D\mapsto \left\{D_r,(1),,,D_r,(2),,,\cdots ,,,D_r,(v),,,\cdots ,,,D_r,(V)\right\},r=1,2,\cdots ,\mathrm{R}$$ 32

where D_r(v) stands for the v-th fold of the whole dataset at r-th run.

At v-th (1 ≤ v ≤ V) trial, the v-th fold is pinched out as the test set [31], and the rest V − 1 folds are selected as training set:

$$\left\{\begin{array}{cc}\mathrm{Training}\mathrm{Set}\hfill & \left\{D_r,(1),,,\cdots ,,,D_r,\left(v,-,1\right),,,D_r,\left(v,+,1\right),,,\cdots ,,,D_r,(V)\right\}\hfill \\ \mathrm{Test}\mathrm{Set}\hfill & \left\{D_r,(v)\right\}\hfill \end{array}\right),v=1,2,\cdots ,V,r=1,2,\cdots ,R$$ 33

Note here; the training set is augmented via the multiple-way data augmentation (MDA) method described in Ref. [9]. We perform a slight adaption to the MDA. In the original MDA [9], N_D = 7 different data augmentation (DA) [32] techniques are employed: rotation, Gamma correction, scaling, random translation, noise injection, horizontal and vertical shear. Nevertheless, each data augmentation in their paper generates a different number of new images. This study makes all the N_D = 7 data augmentation generates the same number of new N_N = 30 images. Suppose a given image is d, the MDA brings an augmented set as

$$d\mapsto \left\{\begin{array}{cc}d& h(d)\\ \underset{N_N}{\underbrace{f_{\mathit{DA}}^1(d)}}& \underset{N_N}{\underbrace{f_{\mathit{DA}}^1\left[h,(,d,)\right]}}\\ \cdots & \cdots \\ \underset{N_N}{\underbrace{f_{\mathit{DA}}^D(d)}}& \underset{N_N}{\underbrace{f_{\mathit{DA}}^D\left[h,(,d,)\right]}}\end{array}\right\},$$ 34

where f_DA denotes a type of DA method and h(d) is the horizontally mirrored image of d.

The F3 model is trained on the augmented training set. The trained model is dubbed M(r, v), and the corresponding confusion matrix is dubbed C(r, v). After all the V-fold trials, the confusion matrix of r-th run is summarized as

$$C(r)={\sum }_{v=1}^{V}C\left(r,,,v\right)$$ 35

where K indicators I(r, k), k = 1, 2, …, K are deduced from the confusion matrix, which will be explained in the next section. Based on R runs, the mean and standard deviation (MSD) [33] of all K indicators are calculated as the form of a(k) ± b(k), which is defined as:

$$\left\{\begin{array}{c}a(k)=\frac{1}{R}\times {\sum }_{r=1}^{R}I\left(r,,,k\right)\hfill \\ b(k)=\sqrt{\frac{1}{R-1}\times {\sum }_{r=1}^R{\left[I,\left(r,,,k\right),-,a,(k)\right]}^2}\hfill \\ k=1,\cdots ,K\hfill \end{array}\right)$$ 36

Figure 7 shows the schematic of V-fold cross validation. This study let V = 10, viz., a ten-fold cross-validation is run. Besides, this V-fold cross-validation runs R times. At each run, the data division is reset randomly. Algorithm 4 recaps the pseudocode of R-run of V-fold cross-validation procedure.

Fig. 7.

Schematic of V-fold cross-validation

Measures and indicators

K = 7 indicators are defined. The SPTB is a positive class, while the HC is a negative class. The confusion matrix is defined as

$$C=\left[\begin{array}{cc}\mathrm{TP}& \mathrm{FN}\\ \mathrm{FP}& \mathrm{TN}\end{array}\right]\stackrel{\mathrm{def}}{=}\left[\begin{array}{cc}{c}_{11}& c_{12}\\ c_{21}& c_{22}\end{array}\right]$$ 37

The definitions of TP, FN, FP, and TN are listed in Table 3. Note, P stands for the actual positive class, so P = TP + FN. Similarly, N stands for the actual negative class; hence N = FP + TN.

Table 3.

Definitions in the confusion matrix

Abbreviation	Full Name	Symbol	Meaning
P	Positive Class	c11 + c12	SPTB
N	Negative Class	c21 + c22	HC
TP	True Positive	c11	SPTB is correctly classified into SPTB.
FN	False Negative	c12	SPTB is wrongly classified into HC.
FP	False Positive	c21	HC is wrongly classified into SPTB.
TN	True Negative	c22	HC is correctly classified into HC.

Three common indicators—Sensitivity, Specificity, and Precision—are defined below

$$\left\{\begin{array}{c}\mathrm{Sen}=\frac{c_{11}}{c_{11}+c_{12}}\hfill \\ \mathrm{Spc}=\frac{c_{22}}{c_{21}+c_{22}}\hfill \\ \mathrm{Prc}=\frac{c_{11}}{c_{11}+c_{21}}\hfill \end{array}\right).$$ 38

Accuracy [34] is defined as:

$$\mathrm{Acc}=\frac{c_{11}+c_{22}}{c_{11}+c_{12}+c_{21}+c_{22}}.$$ 39

F1 score considers both the precision and the sensitivity [35]. It is the harmonic mean of the previous two measures: precision and sensitivity. F1 score [36] is defined as

$$\mathrm{F}1={\left(\frac{{\mathrm{Sen}}^{-1}+{\mathrm{Prc}}^{-1}}{2}\right)}^{-1}=\frac{2\times c_{11}}{2\times c_{11}+c_{12}+c_{21}}.$$ 40

Two other indicators—Matthews correlation coefficient (MCC) [37] and Fowlkes–Mallows index (FMI)—are defined below:

$$\mathrm{MCC}=\frac{c_{11}\times c_{22}-c_{21}\times c_{12}}{\sqrt{\left[c_{11},+,c_{21}\right]\times \left[c_{11},+,c_{12}\right]\times \left[c_{22},+,c_{21}\right]\times \left[c_{22},+,c_{12}\right]}},$$ 41

$$\mathrm{FMI}=\sqrt{\frac{c_{11}}{c_{11}+c_{21}}\times \frac{c_{11}}{c_{11}+c_{12}}}.$$ 42

The receiver operating characteristic (ROC) curve [38] and the area under the curve (AUC) [39] are introduced to provide a graphical plot and a quantitative value of measuring the proposed F3 model, respectively. ROC and AUC are calculated via the following: (i) ROC plot is produced by charting the TP rate against the FP rate at different threshold degrees. (ii) AUC [40] is calculated by measuring the entire 2D area underneath the ROC curve from (0, 0) to (1, 1).

Experiment design

The experiments run on the programming platform of MATLAB 2021a. The parameters are listed in Table 4. The directional angles in GLCM are θ = [0°, 45°, 90°, 135°]. The varying-distance ranges from 1 to D = 3. The number of properties extracted from FDVDGLCM is N_F = 5. Besides, K = 7 indicators are deduced from the confusion matrix. The experiment runs a 10-fold cross-validation ten runs. N_D = 7 different data augmentation methods are utilized. Each DA generates N_N = 30 new images.

Table 4.

Parameters and their values

Parameter	Value
θ	[0°, 45°, 90°, 135°]
D	3
NF	5
K	7
V	10
R	10
ND	7
NN	30

Results and discussions

Result of MDA

Grab Fig. 2a as the raw training image d, Fig. 8a-g shows the results of seven different DA methods. The horizontally mirrored image h(d) and its corresponding DA results are not displayed because of the page limit. As is observed from Fig. 8, MDA provides loads of diversity to the training set, thus improving the performance of the whole recognition system.

Fig. 8.

Results of 14-way MDA

Statistical performance of our F3 model

The performances of 10 runs of 10-fold cross-validation are reported in Table 5. The MSD values of the seven indicators are listed as: sensitivity (93.68% ± 1.75%), specificity (94.17% ± 1.68%), precision (94.17% ± 1.55%), accuracy (93.92% ± 1.05%), F1 score (93.91% ± 1.07%), MCC (87.88% ± 2.09%), and FMI (93.92% ± 1.06%).

Table 5.

Ten runs of our F3 model

Run	Sen	Spc	Prc	Acc	F1	MCC	FMI
1	93.06	94.44	94.37	93.75	93.71	87.51	93.71
2	90.28	93.75	93.53	92.01	91.87	84.08	91.89
3	92.36	96.53	96.38	94.44	94.33	88.97	94.35
4	95.14	95.83	95.80	95.49	95.47	90.97	95.47
5	93.75	95.83	95.74	94.79	94.74	89.60	94.74
6	93.75	92.36	92.47	93.06	93.10	86.12	93.11
7	95.83	90.97	91.39	93.40	93.56	86.91	93.59
8	95.83	93.75	93.88	94.79	94.85	89.60	94.85
9	94.44	94.44	94.44	94.44	94.44	88.89	94.44
10	92.36	93.75	93.66	93.06	93.01	86.12	93.01
MSD	93.68 ±1.75	94.17 ±1.68	94.17 ±1.55	93.92 ±1.05	93.91 ±1.07	87.88 ±2.09	93.92 ±1.06

If we sum up the confusion matrixes of ten runs, the resultant confusion matrix is shown in Fig. 9. We can observe that 91 SPTBs are wrongly classified to HC while 84 HCs are wrongly classified to SPTB at ten runs.

Fig. 9.

Sum of confusion matrixes of 10 runs

SVM versus FSVM

The effectiveness of FSVM is validated in this ablation experiment. If we relace the FSVM classifier with the ordinary SVM classifier, the ten-run results are itemized in Table 6. The comparison is plotted in Fig. 10, where it clearly indicates our FSVM is superior to ordinary SVM. The ROC curves of both classifiers are drawn in Fig. 11, where we can observe the FSVM has an AUC of 0.9624, greater than that of SVM of 0.9483.

Table 6.

Ten runs using ordinary SVM model

Run	Sen	Spc	Prc	Acc	F1	MCC	FMI
1	91.67	90.28	90.41	90.97	91.03	81.95	91.04
2	93.06	93.75	93.71	93.40	93.38	86.81	93.38
3	91.67	90.97	91.03	91.32	91.35	82.64	91.35
4	92.36	91.67	91.72	92.01	92.04	84.03	92.04
5	90.97	92.36	92.25	91.67	91.61	83.34	91.61
6	90.28	91.67	91.55	90.97	90.91	81.95	90.91
7	92.36	91.67	91.72	92.01	92.04	84.03	92.04
8	90.97	90.28	90.34	90.62	90.66	81.25	90.66
9	93.06	93.75	93.71	93.40	93.38	86.81	93.38
10	88.89	95.14	94.81	92.01	91.76	84.19	91.80
MSD	91.53 ±1.30	92.15 ±1.61	92.13 ±1.50	91.84 ±0.96	91.82 ±0.94	83.70 ±1.92	91.82 ±0.94

Fig. 10.

Error bar comparison between SVM and our FSVM

Fig. 11.

ROC curve comparison between SVM and our FSVM

Comparison to state-of-the-art models

The experiment compares the proposed F3 model with six state-of-the-art SPTB recognition models: SSVM [5], AE-CNN [6], depth-ResNet [7], CAS [8], ICNN [9], and CheXaid [10]. The results are itemized in Table 7. We can observe that our F3 model achieves the best at six indicators out of all the seven indicators except sensitivity, of which the F3 model is merely 0.07% lower than the best model as ICNN [9]. The error bar comparison is depicted in Fig. 12.

Table 7.

Comparison to State-of-the-art SPTB recognition models based on ten runs of ten-fold CV

Model	Sen	Spc	Prc	Acc	F1	MCC	FMI
SSVM [5]	83.82 ± 2.07	86.53 ± 1.80	86.18 ± 1.51	85.17 ± 1.05	84.96 ± 1.13	70.41 ± 2.09	84.98 ± 1.13
AE-CNN [6]	87.57 ± 2.06	88.33 ± 2.47	88.30 ± 2.09	87.95 ± 1.37	87.91 ± 1.35	75.95 ± 2.72	87.92 ± 1.34
depth-ResNet [7]	88.75 ± 1.42	91.60 ± 1.48	91.38 ± 1.32	90.17 ± 0.61	90.03 ± 0.63	80.40 ± 1.23	90.05 ± 0.63
CAS [8]	88.47 ± 1.40	91.32 ± 1.19	91.08 ± 1.07	89.90 ± 0.68	89.75 ± 0.72	79.84 ± 1.36	89.76 ± 0.71
ICNN [9]	93.75 ± 0.98	93.19 ± 2.33	93.29 ± 2.07	93.47 ± 0.95	93.50 ± 0.87	86.98 ± 1.86	93.51 ± 0.86
CheXaid [10]	89.93 ± 2.32	90.83 ± 0.72	90.76 ± 0.58	90.38 ± 1.04	90.33 ± 1.17	80.79 ± 2.03	90.33 ± 1.16
F3 (Ours)	93.68 ± 1.75	94.17 ± 1.68	94.17 ± 1.55	93.92 ± 1.05	93.91 ± 1.07	87.88 ± 2.09	93.92 ± 1.06

Fig. 12.

Error bar comparison of state-of-the-art SPTB recognition models

The reasons why the proposed F3 model is the best are three points: (a) The 5-property from FDVDGLCM is used as the feature descriptor. (b) The FSVM is chosen, which has been proved to give better performance than ordinary SVM. (c) A slight adaption of multiple-way data augmentation is used to boost the training set.

Note ICNN [9] has better sensitivity than our F3 model. The reason is the ICNN used two advanced technologies: stochastic pooling and hyperparameter optimization (HO). The HO strategy may help our F3 model; hence, we shall try to integrate HO into our model in future studies.

(Bold means the best.)

Conclusion

This study proposes a novel F3 model for SPTB recognition. The F3 model means (i) using FDVDGLCM to analyze the chest CT images, (ii) using FPFS to extract five properties, and (iii) using FSVM as the classifier. Besides, a slight adaption of multiple-way data augmentation is used to boost the training set. The ten runs of 10-fold cross-validation demonstrate that this F3 model is superior to six state-of-the-art models.

After reviewing our model, we observe four shortcomings: (i) The dataset is moderately small. (ii) The classification task is only a binary classification problem. (iii) Some advanced classification tools are yet to be verified. (iv) the hyperparameters can be optimized to improve the performances.

In future studies, we intend to endeavor to gather more SPTB subjects to expand the experimental dataset. Chest CT images of other infectious diseases will be added, such as community-acquired pneumonia and COVID-19. We shall test the recently proposed graph convolutional network, attention network, etc. Finally, the HO strategy will be used to integrate into our model.