Directional mutation and crossover boosted ant colony optimization with application to COVID-19 X-ray image segmentation

Abstract

This paper focuses on the study of Coronavirus Disease 2019 (COVID-19) X-ray image segmentation technology. We present a new multilevel image segmentation method based on the swarm intelligence algorithm (SIA) to enhance the image segmentation of COVID-19 X-rays. This paper first introduces an improved ant colony optimization algorithm, and later details the directional crossover (DX) and directional mutation (DM) strategy, XMACO. The DX strategy improves the quality of the population search, which enhances the convergence speed of the algorithm. The DM strategy increases the diversity of the population to jump out of the local optima (LO). Furthermore, we design the image segmentation model (MIS-XMACO) by incorporating two-dimensional (2D) histograms, 2D Kapur's entropy, and a nonlocal mean strategy, and we apply this model to COVID-19 X-ray image segmentation. Benchmark function experiments based on the IEEE CEC2014 and IEEE CEC2017 function sets demonstrate that XMACO has a faster convergence speed and higher convergence accuracy than competing models, and it can avoid falling into LO. Other SIAs and image segmentation models were used to ensure the validity of the experiments. The proposed MIS-XMACO model shows more stable and superior segmentation results than other models at different threshold levels by analyzing the experimental results.

Nomenclature

ACOR

Ant colony optimization

XMACO

Ant colony optimization with directional crossover strategy and the directional mutation strategy

MIS-XMACO

A new multilevel image segmentation model

DX

Directional crossover

DM

Directional mutation

FT

Friedman test

WSRT

The Wilcoxon signed-rank test

SSIM

The structural similarity index

SIA

Swarm intelligence algorithm

LO

Local optima

COVID-19

Corona virus disease 2019

$FEs$

The current number of evaluations

MLIS

multilevel image segmentation

$rand$

A random number between $\left[\mathrm{0,1}\right]$

PSNR

The peak signal-to-noise ratio

FSIM

The feature similarity index

1. Introduction

In recent years, a novel coronavirus disease known as the COVID-19 has emerged. Pneumonia caused by the COVID-19 disease has wreaked havoc on humanity. In March 2020, the World Health Organization declared COVID-19 to be a global pandemic. The continual evolution and spread of the coronavirus disease have put tremendous pressure on social and medical systems of various countries. Currently, many countries have developed COVID-19 nucleic acid detection reagents, but the detection reagents are not perfect. The false-negative rate for nucleic acid testing is approximately 20%. Many researchers have studied lung X-ray images of a large number of COVID-19 patients in recent years [[1], [2], [3]]. Numerous studies have shown that diagnostic lung X-ray images of suspected patients are the most feasible method to detect COVID-19 pneumonia, a serious potential complication of the coronavirus disease. Therefore, the accurate and efficient segmentation of COVID-19 X-ray images is an essential component of successfully diagnosing and treating dangerous conditions. Multilevel image segmentation (MLIS) [4], a widely used image segmentation technique in recent years, enables the efficient processing of complex data. Some current medical diagnostic systems use MLIS as one of the important basic technical tasks in the medical system.

Optimization methods include a comprehensive class of deterministic and stochastic solvers that can be adapted according to the nature of problems and characteristics of the target surfaces [5,6]. Swarm intelligence algorithms (SIAs), also known as metaheuristic algorithms, are inspired by the behavior of plants and animals in nature that hunt cooperatively in groups. SIA has evolved over the years. Researchers have continued to develop many algorithms, such as the sine cosine algorithm (SCA) [7], different evolution (DE) [8], the whale optimizer (WOA) [9], particle swarm optimization (PSO) [10], moth-flame optimization (MFO) [11], the bat-inspired algorithm (BA) [12], Harris hawks optimization (HHO)1 [13], the hunger games search (HGS)2 [14], the colony predation algorithm (CPA) [15], weighted mean of vectors (INFO)3 [16], the slime mold algorithm (SMA)4 [17], and the Runge Kutta optimizer (RUN)5 [18]. Marco Dorigo first proposed ant colony optimization (ACO) to solve discrete optimization [19]. Then, researchers in works [20,21] proposed ACO for continuous domains to solve continuous-type problems, and they named this approach “ACOR”. ACOR is a classical and efficient algorithm based on ant colony foraging behavior, and researchers have conducted many studies using this algorithm.

In MLIS, the threshold set can improve the intensity of pneumonia shadows after the segmentation of COVID-19 X-ray images. Therefore, researchers have combined various optimization algorithms, such as swarm intelligence, into image segmentation models to improve the efficiency of obtaining threshold sets. For example, Zou et al. [22] presented an enhanced thresholding model with Shannon entropy difference and dynamic synergic entropy, and they applied this to MLIS. Zhao et al. [23] proposed an adaptive thresholding segmentation model based on a multiobjective artificial bee colony optimizer that combines the between-class variance function and the interval-valued fuzzy entropy function. Zhang et al. [24] presented a novel image segmentation model that combines the method of gray image enhancement and an improved edge detection strategy for the image processing technology of cesarean section of placenta previa. Wang et al. [25] presented a novel multiverse optimizer with minimum cross entropy thresholding to solve multilevel segmentation problems. In conclusion, multilevel image segmentation methods combined with SIAs have become a popular research area in the field of image segmentation. This is because the SIAs are perfectly suited to MLIS models and because the SIAs have strong power and potential for improvement.

This paper presents a novel XMACO algorithm that introduces the directional crossover strategy and the directional mutation strategy into ACOR. The DX improves the searchability of XMACO and thus, the convergence speed of the algorithm. The DM improves the population diversity of XMACO through population mutation and thus, the ability of the algorithm to jump out of the local optima (LO). The XMACO can quickly converge to an effective local search region in the early stage of optimization. At the same time, the guarantee of population diversity allows the algorithm to reel in and out of the local search. Furthermore, benchmark functions are designed to experimentally demonstrate the performance of XMACO. The function set is composed of IEEE CEC2014 and IEEE CEC2017. The benchmark function experiments of XMACO include a variety of performance experiments, including an assessment of the effect of the two strategies, a comparison with other original algorithms, and a comparison with other improved algorithms. The benchmark function experimental results were analyzed using the variance, the mean, the Wilcoxon signed-rank test (WSRT) [26], the Friedman test (FT) [27], and other data analysis methods.

ACOR excels in many areas of application. We propose a novel MIS-XMACO model applied to multilevel COVID-19 X-ray image segmentation. The proposed model uses the 2D histogram strategy to represent the image information; 2D Kapur's entropy represents the threshold set level from the model. Moreover, 2D Kapur's entropy is the objective function (fitness) of the XMACO. We consider that MIS-XMACO needs to maintain stability for image segmentation at different threshold levels. Three levels of the low threshold set (2, 4, 6), and three levels of the high threshold set (12, 16, 20) are designed for the segmentation model. Moreover, the image segmentation results are effectively and fairly analyzed by the structural similarity index (SSIM) [28], peak signal-to-noise ratio (PSNR) [29], feature similarity index (FSIM) [30], etc. Finally, the experimental results are evaluated using the WSRT, mean, and variance methods.

This paper's contributions and innovations present an improved version of ACOR, named XMACO, that combines directional crossover strategy with directional mutation strategy. We designed a series of comparative algorithm experiments on XMACO to verify that XMACO has better convergence speed, accuracy, and ability to jump out of LO on the benchmark function set. Second, this paper presents the first XMACO-based multilevel COVID-19 X-ray image segmentation method, named MIS-XMACO. The proposed method combines the nonlocal means strategy, 2D histogram, and 2D Kapur's entropy. The experimental data analysis proves that XMACO improves the convergence speed, accuracy, and ability to jump out of the LO compared to other approaches, and the evaluation results demonstrate that MIS-XMACO has better segmentation results and more stability than other models for multilevel COVID-19 X-ray image segmentation.

The remaining sections of the paper are described as follows: Section 2 focuses on and explores more details of the current study. Section 3 describes the structure of the XMACO algorithm and proposes the MIS-XMACO image segmentation model. Section 4 designs benchmark function experiments to verify the XMACO algorithm and designs image segmentation experiments based on COVID-19 X-ray images to verify the MIS-XMACO model. The conclusions of the paper and potential future works are presented in Section 5.

2. Related works

With the development of image processing technology [[31], [32], [33], [34]], image segmentation, as a fundamental technical tool for image processing, is receiving increasing attention from researchers. For example, Zhao et al. [35] proposed an enhanced 2D Otsu image segmentation model combined with PSO. However, the original PSO-based model designed by the researcher has drawbacks. Because the optimization capability of PSO is not sufficiently high, the convergence speed is slow, and it easily falls into LO in the face of multimodal functions. Anter et al. [36] presented a novel computed tomography (CT) image segmentation model based on a bioinspired ant lion optimizer, which combined the improved fuzzy c-means with chaos theories. Abdel-Basset et al. [37] introduced an image segmentation model based on 2D Kapur's entropy combined with SMA and WOA for COVID-19 X-rays. Verma et al. [38] proposed an image segmentation model based on hybrid fuzzy c-means and PSO. Additionally, Shih et al. [39] proposed a new adaptive mathematical morphology model that combined one-dimensional (1D) histogram-based modeling for image segmentation. Among them, the multilevel image segmentation method is robust and efficient. Recently, researchers have proposed many MLIS methods. These include the MLIS model based on contour detection and hierarchical image segmentation and principal component analysis based on image segmentation, pixel clustering and color image segmentation. Among them, 2D histogram-based MLIS has strong image denoising abilities.

Image segmentation has been applied to solve many practical problems, such as leaf spot disease image segmentation [40], breast cancer image segmentation [41,42], and lupus nephritis image segmentation. Today, many COVID-19 X-ray image segmentation methods have been proposed by researchers. Gopatoti et al. [43] presented a convolutional neural network chest X-ray radiography image segmentation model that combined SegNet, U-Net, and the gray wolf optimization algorithm. Tahir et al. [44] proposed a novel method with segmentation networks, U-Net++, and feature pyramid networks for COVID-19 lung image segmentation. Degerli et al. [45] proposed a new approach for the joint localization, severity grading, and detection of COVID-19 from X-ray images by generating so-called infection maps. Zhang et al. [46] presented a novel multiscale feature capture block model with multilevel feature aggregation to effectively capture multiscale features for COVID-19 X-ray image segmentation. Joshi et al. [47] proposed a new saliency-based region detection and image segmentation model for COVID-19 X-ray image segmentation. Su et al. [48] proposed a multilevel thresholding image segmentation model that combines an improved artificial bee colony algorithm for COVID-19 X-ray images. Zhao et al. [49] presented a novel ant colony algorithm (ACO), which combined horizontal crossover search and vertical crossover search to optimize multithreshold image segmentation. Yan et al. [50] proposed adaptive local threshold segmentation, which combines Fourier spatial filtering for digital speckle pattern interferometry. Xing et al. [51] proposed an improved emperor penguin optimization for multilevel threshold values for color image segmentation. Xiao et al. [52] presented an enhanced Otsu's model with a three-dimensional histogram for image segmentation. Wunnava et al. [53] proposed an improved Harris hawk optimization combined with an enhanced two-dimensional gray gradient strategy for multilevel thresholding. Yang et al. [54] proposed a thresholding segmentation method that combines a novel 2D histogram that consists of gray level and the local relative entropy.

By summarizing the above methods, we see that many of them are based on optimization algorithms to better select the optimal threshold set. This improves the time, accuracy, and efficiency of image segmentation. However, many image segmentation models still have shortcomings through the analysis of the above models, including image denoising, intensity inhomogeneity, and optimization algorithm performance. The MLIS model based on the 2D histogram strategy is proposed in this paper. Compared to 1D histograms, 2D histograms can describe the information of an image more completely using grayscale values and nonlocal means. Additionally, the 2D histogram is not as sensitive to pattern noise as the 1D histogram. Moreover, the proposed model incorporates an improved SIA with 2D Kapur's entropy as the objective function of the proposed algorithm to provide the optimal threshold set for the image segmentation model.

3. The proposed method

3.1. Proposed XMACO algorithm

This section presents the structure of the proposed algorithm, XMACO. The performance of XMACO is improved by introducing the DX and DM mechanisms into ACOR. It should be noted that the variable settings in each strategy in this section are independent.

3.1.1. Directional crossover

The study [55] found that the DX strategy can obtain a strong search ability according to the information of the best individual. Inspired by this paper, introducing the DX strategy into the original ACOR can improve its convergence speed and accuracy. The principle of the DX strategy is as follows:

The key parameters in the DX strategy are crossover probability ($p_{cv}$), directional probability ($p_d$), and multiplying factor ($\mathrm{\alpha }$). $p_1$ and $p_2$ represent two random individuals. $p_{mean}^j$ and $p_{best}^j$ are the mean and the best of two random individuals in the jth dimension. The formula for the DX is Eqs. (1), (2). Pseudocode Algorithm 1 shows the formula and the flow of DX.

$$val=\{\begin{array}{l}1-{0.5}^{e^{\left[\frac{\left|p_1^j-p_2^j\right|}{\left(y_u^j-y_l^j\right)}\right]}},if{p}_1^j\ne p_2^j\hfill \\ 1-{0.5}^{e^{\left[\frac{\left|p_{best}^j-p_{mean}^j\right|}{\left(y_u^j-y_l^j\right)}\right]}},if{p}_1^j=p_2^{j}and{p}_{best}^j\ne p_{mean}^j\hfill \end{array}$$ (1)

$$\beta =\frac{r}{{\alpha }^2}$$ (2)

When $p_1^j\ne p_2^j$, Eq. (3) and Eq. (4) are determined from the directional probability ($p_d$).

$$c_1^j=val\times \left(p_1^j\pm p_2^j\right)\pm {\alpha }^{r_1}{e}^{\left(1-\beta \right)}\left(1-val\right)\times \left|p_1^j-p_2^j\right|$$ (3)

$$c_2^j=\left(1-val\right)\times \left(p_1^j\pm p_2^j\right)\pm {\alpha }^{(1-r_1)}{e}^{\left(-\beta \right)}val\times \left|p_1^j-p_2^j\right|$$ (4)

When $p_1^j=p_2^j$ and $p_{best}^j\ne p_{mean}^j$, Eq. (5) and Eq. (6) are determined from the directional probability ($p_d$).

$$c_1^j=val\times \left(p_{best}^j+p_{mean}^j\right)\pm {\alpha }^{r_1}{e}^{\left(1-\beta \right)}\left(1-val\right)\times (p_{best}^j-p_{mean}^j)$$ (5)

$$c_2^j=\left(1-val\right)\times \left(p_{best}^j+p_{mean}^j\right)\pm {\alpha }^{(1-r_1)}{e}^{\left(-\beta \right)}val\times (p_{best}^j-p_{mean}^j)$$ (6)

where $val$ and $\beta$ represent the two variables that change with iterations. $r$ is a random variable, and $r\in \left(\mathrm{0,1}\right)$. $y_u^j$ and $y_l^j$ represent the upper and lower bounds of the objective function in the jth dimension, respectively. Eqs. (3), (4), (5), (6) will determine the $\pm$ in formulas based on $p_d$. $c_1$ and $c_2$ are new individuals generated by the DX strategy, $\alpha =0.95$, $r_1\in \left(\mathrm{0,1}\right)$. We suppose the individual dimension is $d$.

Algorithm 1

The Pseudocode of DX strategy

3.1.2. Directional mutation

The tendency of the original ACOR to fall into an LO is another reason why the algorithm cannot obtain an optimal solution. The DM strategy in Ref. [55] exhibits an extremely strong balance of population diversity. An increased population diversity can help the original ACOR gain the ability to escape the LO. The principle of the DM strategy is as follows.

The DM strategy randomly mutates the individual position by a distance in any direction in the search space. We suppose that the individual dimension is $d$. $y$ represents an individual. $y_{best}$ represents the best individual. $y_{best}$ guides the position update direction of $y$. The formula for the DM strategy is shown in Eqs. (7), (8), (9), (10).

$${\beta }_1=e^{\left(2r-\frac{2}{r}\right)}$$ (7)

$${\beta }_2=e^{\left(r-\frac{2}{r}\right)}$$ (8)

If $y_{best}^j\ge y^j$, then the formula is as follows:

$$c^j=\{\begin{array}{l}{y}^j+{\beta }_1\left(y_u^j-y^j\right),if{r}_2\le p_d\hfill \\ y^j-{\beta }_2\left(y^j-y_l^j\right),otherwise\hfill \end{array}$$ (9)

If $y_{best}^j<y_i^j$, then the formula is as follows:

$$c^j=\{\begin{array}{l}{y}^j-{\beta }_1\left(y^j-y_l^j\right),if{r}_2\le p_d\hfill \\ y^j+{\beta }_2\left(y_u^j-y^j\right),otherwise\hfill \end{array}$$ (10)

where ${\beta }_1$ and ${\beta }_2$ are the variables. $y^j$ represents the value of the individual in the jth dimension. $r=0.8$, and $r_2\in \left(\mathrm{0,1}\right)$. $p_d$ represents the directional probability, and $p_d=0.4$. $p_m$ represents the mutation probability, and $p_m=0.1$. $y_u^j$ and $y_l^j$ represent the upper and lower bounds of the objective function in the jth dimension, respectively. $c$ is a new individual generated by the DM strategy.

3.1.3. The proposed XMACO's structure

Initialization of the population strategy and archive mechanism: ACOR initializes two populations in the initial phase, $Archive$, the archive population, and $POP$, the initial population of the algorithm. Both $Archive$ and $POP$ are initialized with an initial randomized strategy based on the Gaussian function. Eq. (11) and Eq. (12) are the initialization formulas.

$$POP=ub\times N\left(\mathrm{0,1}\right)+lb\times N\left(\mathrm{0,1}\right)$$ (11)

$$Archive=ub\times N\left(\mathrm{0,1}\right)+lb\times N\left(\mathrm{0,1}\right)$$ (12)

where $ub$ is the upper bound of the objective function, and the size of $ub$ is $1\times dim$. $lb$ is the lower bound of the objective function, and the size of $lb$ is $1\times dim$. $N\left(\mathrm{0,1}\right)$ is a Gaussian function.

The original ACOR was designed to mimic the pheromone mechanism of ant colony foraging. The algorithm selects the guided individual based on the individual weights in the archive. The size of the archive is $k$. The archive holds the best $k$ individuals of the initial population, $POP$. $s_i$ represents the ith individual in the archive. $f\left(s_i\right)$ is the objective function. The formula for ith individual weight, $w_i$, and normalized probability, $p_i$, is Eq. (13), (14).

$$w_i=\frac{1}{qk\sqrt{2\pi }}{e}^{-\frac{{\left(i-1\right)}^2}{2q^2{k}^2}}$$ (13)

$$p_i=\frac{{\omega }_i}{{\sum }_{r=1}^k{\omega }_r}$$ (14)

where $w_i$ is a Gaussian function conforming to $N\left(l,qk\right)$. $q$ is a constant that controls the probability of individual selection in the archive. $q$ is the algorithm that controls the local search and global search parameters. $p_l$ is the probability of selecting the guided individual. Fig. 1 a shows the foraging process of ants along with pheromones. $An{t}_1$ leaves pheromones in the paths where food is available, and other fellow ants will find food faster using the pheromones. Fig. 1b shows the structure of the archive.

Fig. 1.

(a) Ant colony foraging (b) ACOR's archive structure.

First, for the location update strategy, ACOR updates the search step based on the continuous probability density (Gaussian) strategy. The algorithm establishes the concept of the Gaussian kernel [20] to describe the multidimensional objective function problem, $G\left(x\right)=N\left(\mu ,\sigma \right)$. The algorithm selects the guide individual $s_l$ from the archive by sampling based on Eq. (13). Then, the Gaussian kernel update step is calculated based on the guide individual, $s_l$. The formulas of probability density strategy are Eq. (15), (16), (17).

$${\mu }^j=\left\{{\mu }_1^j,{\mu }_2^j,\dots {\mu }_i^j\right\}=\left\{s_1^j,s_2^j,\dots s_i^j\right\}$$ (15)

$${\sigma }_i^j=\xi \times \sum _{e=1}^k\frac{|s_e^j-s_l^j|}{k-1}$$ (16)

$$PO{P}_i^j=N\left({\mu }_i^j,{\sigma }_i^j\right)=s_l^j+{\sigma }_i^j\times rand\left(\mathrm{0,1}\right)$$ (17)

The Gaussian kernel $G\left(x\right)$ is a Gaussian distribution function that conforms to a mean of ${\mu }_i^j$ with a variance of ${\sigma }_i^j$. $s_l^j$ is the value of the guide individual in the jth dimension. $\xi$ is the pheromone evaporation rate, and $\xi =1$. After the population executes the continuous probability density function strategy, the XMACO then performs the DX strategy to enhance the exploration capability of the algorithm and enhance the diversity of the population through the DM strategy. The position update formula is in Eq. (18-22).

$$n_1=rand,n_2=rand$$ (18)

$$\left[newPO{P}_1,newPO{P}_2\right]=DX(Archive\left(1\right),POP\left(n_1\right),POP\left(n_1\right),lb,ub)$$ (19)

$$newPO{P}_1=DM(Archive\left(1\right),newPO{P}_1,lb,ub)$$ (20)

$$newPO{P}_2=DM(Archive\left(1\right),newPO{P}_2,lb,ub)$$ (21)

$$PO{P}_i=\{\begin{array}{l}newPO{P}_{1}iffobj\left(newPO{P}_1\right)<fobj\left(newPO{P}_2\right)<fitnes{s}_i\hfill \\ newPO{P}_{2}iffobj\left(newPO{P}_2\right)<fobj\left(newPO{P}_1\right)<fitnes{s}_i\hfill \end{array},$$ (22)

where $n_1$ and $n_2$ are two random numbers, and $n_1\in [1,N]$ and $n_2\in [1,N]$. $Archive\left(1\right)$ is the best individual. $lb$ is the lower bound of the objective function. $ub$ is the upper bound of the objective function. $fitnes{s}_i$ is the fitness value of the ith individual. When a new population, $POP$, is formed, XMACO ranks the populations ($POP$ +$Archive$) of size ($N+k$) and finally takes the top $k$ ranked individuals into the archive. Algorithm 2 is the pseudocode of the proposed algorithm.

Algorithm 2

Pseudocode of XMACO

This section analyzes the time complexity composition of the proposed XMACO algorithm in detail, including the initialization strategy, probability density strategy, directional crossover strategy, and directional mutation strategy. The pheromone archive size is $k$. $E$ represents the total number of $MaxFEs$ evaluated by the algorithm. $N$ represents the size of the algorithm population. And, $d$ represents the dimensionality of the algorithm objective function. The time complexity of the original ACOR is $O\left(ACOR\right)=O\left(k\right)+O(E\times (k+N\times (k+d\times k)))$. The time complexity of XMACO's population initialization strategy is $O(k+N)$. The time complexity of the probability density function strategy is $O(E\times (N\times (d\times k)))$. The time complexity of the DX strategy is $O(E\times (N\times d))$. The time complexity of the DM strategy is $O(E\times (N\times d))$. The time complexity of the XMACO algorithm is $O\left(XMACO\right)=O\left(k+N\right)+O(E\times (k+N\times (k+d\times k+d+d)))$. In this paper, the population size of the archive for the algorithm is set to $k=10$. The algorithm population size is $N=30$. According to the maximum time complexity criterion, $O\left(XMACO\right)\approx O\left(E\times N\times d\times k\right)$. In addition, $O\left(ACOR\right)\approx O\left(E\times N\times d\times k\right)$. The time complexities of the algorithms XMACO and original ACOR are analyzed by calculating them separately. The time complexity of XMACO is within a manageable performance improvement.

3.2. Proposed MLIS method

This section proposes the MIS-XMACO model and introduces the image segmentation process. It should be noted that the variable settings in each strategy in this section are independent.

3.2.1. Nonlocal means and 2D histogram strategy

The information in the 2D histogram is generated by the nonlocal means and the grayscale values of the pixels. In 2005, Buades et al. [56] proposed the nonlocal means method for removing redundant information from images. In this paper, we refer to nonlocal means as a method to remove “noise” from an image and preserve as much valid information as possible. The nonlocal means represent the similarity of all pixel points in the image. The similarity is calculated using the Euclidean metric. We assume that $v\left(y\right)$ represents the grayscale value of pixel $y$ in the image, and that $u\left(x\right)$ represents the corresponding filter value of pixel $x$ in the image. $u\left(x\right)$ will be calculated by Eq. (23), (24), (25), (26).

$$u\left(x\right)=\sum _{y\in I}w\left(x,y\right)\times v\left(y\right)$$ (23)

$$w\left(x,y\right)=\frac{1}{Z\left(x\right)}\exp \left(-\frac{{\left|\left|V\left(x\right)-V\left(y\right)\right|\right|}^2}{h^2}\right)$$ (24)

$$V\left(x\right)=\frac{1}{m\times m}\sum _{v\in Q\left(x\right)}v\left(x\right)$$ (25)

$$V\left(y\right)=\frac{1}{m\times m}\sum _{v\in Q\left(y\right)}v\left(y\right)$$ (26)

where $I$ is the total number of pixels in the image. $w\left(x,y\right)$ represents the similarity between pixel $x$ and pixel $y$. $Z\left(x\right)$ is the normalization factor. ${\left|\left|V\left(x\right)-V\left(y\right)\right|\right|}^2$ is the Euclidean distance between $x$ and $y$. $h$ is called the smoothing parameter, and $h$ controls the noise level of the image. The larger that $h$ is, the more noise thqt is removed from the image and the higher the image level. $V\left(x\right)$ and $V\left(y\right)$ represent a rectangular neighborhood window of size $m\times m$ centered at $x$ and $y$, respectively. We suppose that the image is of size $M\times U$, $p\in \left[0,M\right]$, and $q\in \left[0,U\right]$. The grayscale value of the pixel is $v\left(q,p\right)$. The nonlocal means of the pixel are $u\left(q,p\right)$. $v\left(q,p\right)\in \left[0,254\right]$ and $u\left(q,p\right)\in \left[0,254\right]$. The 2D histogram is calculated by Eq. (27).

$$P_{ij}=\frac{h_{ij}}{M\times U}$$ (27)

where $i$ represents the value of pixel grayscale value $v\left(q,p\right)$. $j$ represents the value of the nonlocal mean $u\left(q,p\right)$. $h_{ij}$ represents the level of pixel number at pixel point with grayscale value $i$ and nonlocal mean $j$. $P_{ij}$ represents the normalized representation of the pixel density in the 2D histogram. We can produce the final 2D histogram shown in Fig. 2 and the corresponding plane view.

Fig. 2.

The 2D histogram and the 2D plan view.

3.2.2. 2D Kapur's entropy

2D Kapur's entropy measures the degree of homogeneity and redundancy among substances. In recent years, researchers have applied the concept of 2D Kapur's entropy [57] to measure the threshold level set in image segmentation. In Fig. 2, the horizontal axis $S=\left\{s_1,s_2,\dots s_r,\mathrm{..}{s}_{255}\right\}$ represents the grayscale value of the pixel; the vertical axis $T=\left\{t_1,t_2,\dots t_r,\mathrm{..}{t}_{255}\right\}$ represents the nonlocal mean value of the pixel. We suppose that $L$ represents the number of thresholds to be segmented by the image segmentation model. $t{h}_r$ is a threshold of the image. The threshold set for image segmentation is $\left\{t{h}_1,t{h}_2,\dots t{h}_r,\dots t{h}_M\right\}$, and $1\le M\le 255$. The 2D Kapur's entropy $\phi \left(S,T\right)$ is used as the objective function value of XMACO. $\phi \left(S,T\right)$ is calculated by threshold set $\left\{t{h}_1,t{h}_2,\dots t{h}_r,\dots t{h}_M\right\}$. $\phi \left(S,T\right)$ is calculated by Eq. (28), (29), (30).

$$\phi \left(S,T\right)=F\left(t{h}_1,t{h}_2,\dots t{h}_r,\dots t{h}_M\right)=E_0+E_1+\dots E_r+\dots E_M$$ (28)

$$E_r=-\sum _{i=s_{r-1}}^{s_r-1}\sum _{j=t_{r-1}}^{t_r-1}\frac{P_{i,j}}{w_r}\ln \frac{P_{i,j}}{w_r}$$ (29)

$$w_r=\sum _{i=t{h}_{r-1}}^{t{h}_r-1}{P}_{i,j}=\sum _{i=s_{r-1}}^{s_r-1}\sum _{j=t_{r-1}}^{t_r-1}{P}_{i,j}$$ (30)

3.2.3. The proposed MIS-XMACO model

We introduce the XMACO algorithm in the multilevel image segmentation model to optimize 2D Kapur's entropy value of the threshold set. The MIS-XMACO model provides a precise threshold set for image segmentation. The COVID-19 X-rays are segmented to separate the medical diagnostic target from the background, completing the fundamental work for the entire image diagnosis. In 1981, Pun et al. [58] proposed an automatic threshold selection method based on the histogram of pixel probability distribution to solve the one-threshold segmentation problem. In 1985, Kapur et al. [59] introduced the concept of 2D Kapur's maximum entropy, which is more efficient to compute, to the single threshold partitioning problem based on previous work. When researchers face multilevel segmentation problems, it is inefficient and unreasonable to use one-threshold segmentation methods to solve multilevel problems. Additionally, the 1D histogram-based one-threshold segmentation technique is incomplete for representing image information, while the 1D histogram is also too sensitive to pattern noise. We propose a multilevel segmentation model based on the nonlocal mean method, the 2D histogram strategy, and 2D Kapur's entropy. The model also avoids the limitation of using the local mean for image information conveyance.

The segmentation process of the proposed MIS-XMACO model is: (1) conversion of COVID-19 X-ray images into grayscale images; (2) generation of nonlocal mean images using grayscale values of COVID-19 X-ray image pixels; (3) generating 2D histograms based on two types of image information: grayscale values and nonlocal mean values; and (4) the threshold set as the population of the XMACO, while 2D Kapur's entropy is the objective function of the XMACO. XMACO uses the pixel information in the 2D histogram to obtain the maximum 2D Kapur's entropy. Finally, the proposed MIS-XMACO model generates segmented grayscale and color images. Fig. 3 dynamically demonstrates the image segmentation process of the proposed model.

Fig. 3.

Segmentation process of the MIS-XMACO model.

4. Experiments and results

In this section, a series of benchmark function comparison experiments based on the XMACO algorithm is designed. The benchmark function sets for the experiments are selected from the function sets of IEEE CEC2014 and IEEE CEC2017. Meanwhile, the proposed multilevel image segmentation model (MIS-XMACO) is applied to COVID-19 X-ray image segmentation. This experiment was conducted at low threshold levels 2, 4, and 6 and high threshold levels 12, 16, and 20 to ensure stability and fairness. In addition, to ensure the same environment, all experiments were conducted on a windows server windows 10 pro with an AMD ryzen 9 3900 × 12-core processor (3.79 GHz) and 20 GB of RAM. The coding was performed using MATLAB 2020.

4.1. Comprehensive experiments

4.1.1. Benchmark function validation

To effectively demonstrate the improvement of XMACO in terms of convergence speed and the ability to jump out of LO, we demonstrate the performance of XMACO in two directions. To verify the effect of the two strategies (DX and DM) on the proposed algorithm, we designed a comparison experiment among different XMACOs. To verify the outstanding performance of XMACO, we also designed comparison experiments between XMACO and other algorithms. The results of the experiments were analyzed by the standard deviation, mean, Wilcoxon signed-rank test (WSRT) [26] and Friedman test (FT) [27].

In function comparison experiments, to guarantee the fairness of the experiments, all algorithms were put into the test under the same conditions. In all experiments, the participating algorithms used the evaluation principles (FEs) to limit the number of cycles of the algorithm. The number of evaluations is increased each time the algorithm objective function value (fitness value) was calculated. The evaluation principle ensures the validity and certainty of experiments. The total number of evaluations (MaxFEs) was set to 300,000. All algorithms had a population size of 30. To reduce the randomness generated by experiments, all algorithms were tested 30 times.

4.1.2. Experimental setup of the benchmark function set

The benchmark function set is selected from two function sets, IEEE CEC2014 and IEEE CEC2017. The function set includes several types of functions, including unimodal, simple multimodal, hybrid, and composition functions. The details of the benchmark function set are shown in Table A1 in the supplementary material.

Table 1.

Versions of various XMACOs.

Algorithm	DX	DM
ACOR	0	0
DACO	1	0
MACO	0	1
XMACO	1	1

4.1.3. The impact of directional crossover and directional mutation

First, this paper shows the comparison experiment among different XMACO variants, including ACOR, DACO, MACO, and XMACO. Table 1 shows the details of XMACOs. “0” means that the strategy is not introduced into the algorithm. “1” means the strategy is introduced into the algorithm. Table A2 in the supplementary material represents the mean (AVG) and the standard deviation (STD) obtained by this experiment (the smaller value of “AVG” represents the better result for solving the function; the smaller value of “STD” proves the more stable result for solving the function). The table shows that XMACO has the best “AVG” and “STD” values on almost all functions (marked in bold). Combining the results of both analyses, XMACO demonstrates a more stable, accurate optimization capability than other comparative algorithms.

Table 2.

The analysis results of WSRT and FT.

Item	XMACO	ACOR	DACO	MACO
+/−/ =	∼	24/0/6	17/1/12	29/0/1
Mean	1.673	2.631	2.549	3.147
Rank	1	3	2	4

In Table 2, “+/−/ = ” is the WSRT analysis result of this experiment. “Mean” is the FT analysis result. XMACO's mean is 1.673, which is the best performance compared to DACO, MACO, and ACOR. Meanwhile, the WSRT rank shows that XMACO is best. In addition, XMACO outperformed (”+“) DACO on at least seventeen functions, was inferior (”-“) to DACO on one, and had the same (” = “) performance as DACO on twelve. Table A3 in the supplementary material shows the experimental WSRT's p value. In the table, XMACO shows smaller p values (<0.05) than DACO, MACO, and ACOR in almost all functions. It is proven that XMACO has significant reliability for the comparison results.

Table 3.

The analysis results of WSRT and FT.

	XMACO	ACOR	DE	HHO	MFO	PSO	SCA	WOA	BA
+/−/ =	∼	21/1/8	17/6/7	24/5/1	30/0/0	25/1/4	29/1/0	29/1/0	22/2/6
Mean	2.1822	3.2328	3.1050	4.9606	6.4700	5.2311	7.6478	6.5739	5.5967
Rank	1	3	2	4	7	5	9	8	6

Fig. 4 shows the convergence curves of the experiment. By observing the convergence curves, we see that XMACO has a better convergence speed and accuracy than the other comparison algorithms and can jump out of the LO. In addition, the remaining convergence curves of the benchmark function set are shown in the supplementary material, including Figure B1, Figure B2 and Figure B3.

Fig. 4.

The convergence curve of this benchmark function experiment.

In addition, to better theoretically analyze XMACO, the experiment is designed to test the algorithm's exploration and exploitation capabilities. In F9 of Fig. 5 , the average exploration value of MACO increases from 3.94% to 5.58%; the average exploration value of DACO increases from 3.94% to 12.10%. Therefore, the MACO and DACO are improved explorations to balance the exploitation of algorithms. However, the average exploration value of XMACO reached 24.69%. This suggests that XMACO can better balance the algorithm's exploration and exploitation capability, thus improving the convergence speed and accuracy of the algorithm.

Fig. 5.

The experimental results of the exploration and exploitation capabilities test.

We also design a deeper analysis experiment to analyze the ability of the XMACO algorithm to jump out of LO. Fig. 6 shows the experimental results from two aspects: (a) changes in the population diversity of variants (the average distance between individuals in the population reflects the population diversity); and (b) changes in the population search trajectories of XMACO and ACOR (the rich diversity of populations can help the algorithm to change the search trajectory). In Fig. 6a, on F6, F22, and F24, MACO possessed a higher population diversity than ACOR in the late search phase. Moreover, XMACO exhibited better population diversity. It is demonstrated that the DX and DM strategies can increase the population diversity of the algorithm. Therefore, the change in population diversity of XMACO in Fig. 6a corresponds to the change in population trajectory in Fig. 6b, allowing the algorithm to jump out of LO.

Fig. 6.

(a) Changes in diversity during optimization of different variants, (b) Changes in population search trajectories during optimization by XMACO and ACOR.

4.1.4. Comparison with other algorithms

Second, we designed a comparison experiment between XMACO and the eight original algorithms (ACOR, DE, HHO, MFO, PSO, SCA, WOA, and BA), as well as a comparison between XMACO and seven improved algorithms (RCACO, ASCA_PSO, CDLOBA, HGWO, DSMFO, CGSCA, and AMFOA). In addition, we followed fair evaluation instructions. A fair comparison is an accepted logical process in the artificial intelligence, optimization, and neural network communities to ensure that there is no bias in the results due to the various testing conditions. Table A4 and Table A6 in the supplementary material show the AVG and STD results obtained by this experiment. The tables show that the proposed method has the best AVG and STD values on almost all functions. It is evident that XMACO achieves optimal solution quality and is more stable on most of the functions than the other comparison algorithms.

Table 4.

The analysis results of WSRT and FT.

	XMACO	RCACO	ASCA_PSO	CDLOBA	HGWO	DSMFO	CGSCA	AMFOA
+/−/ =	∼	8/7/15	30/0/0	25/2/3	26/4/0	25/5/0	26/4/0	25/5/0
Mean	2.2961	2.3489	4.3589	4.4556	4.4461	5.9367	4.8567	7.3011
Rank	1	2	3	5	4	7	6	8

Table 3 shows the results of the analysis for the comparison of the original algorithms. XMACO is first in both the WSRT and FT rankings in the table. In addition, in terms of the results of “+/−/ = ”, the algorithm has better performance than DE on 17 functions. Table 4 shows the results of the analysis for improved algorithm comparison. XMACO is also first in both the WSRT and FT rankings, indicating that XMACO can perform better or equal to other algorithms in most functions. Table A5 and Table A7 in the supplementary material show the experimental p values. XMACO's WSRT p value is less than 0.05 for most of the functions compared with the original algorithm and the improved algorithm, which proves that the algorithm has significant reliability of the comparison results.

The convergence curves of XMACO and the other algorithms are shown in Fig. 7 . The five convergence curves of F7, F20, F21, F23, and F25 show that XMACO has better convergence speed and accuracy. We find that XMACO can jump out of the LO from the convergence curves of F19 and F29. The remaining convergence curves of the benchmark function set are shown in the supplementary material, including Figure B4, Figure B5 and Figure B6.

Fig. 7.

The convergence curve of this benchmark function experiment.

In addition, the convergence curves of XMACO and the other improved algorithms are shown in Fig. 8 . XMACO in F7, F20, F21, and F23 exhibits faster convergence than the other improved algorithms. The F3 and F30 convergence curve algorithms show the ability to jump out of the LO. The remaining convergence curves of the benchmark function set are shown in the supplementary material, including Figure B7, Figure B8 and Figure B9 . The convergence curves of XMACO with other popular algorithms demonstrate the ability to enhance jumping out of LO and fast convergence.

Fig. 8.

The convergence curve of this benchmark function experiment.

Fig. 9.

The original COVID-19 X-ray images and the corresponding 2D histograms.

4.2. Experiment on multilevel COVID-19 X-ray image segmentation

This section effectively demonstrates that the MIS-XMACO model has favorable segmentation effects on COVID-19 X-ray image segmentation. The proposed model has experimented with ten models for multilevel segmentation on eight COVID-19 X-ray images. The image segmentation experiments were conducted at six levels of low thresholds (2, 4, and 6) and high thresholds (12, 16, and 20) to demonstrate the superiority and stability of the proposed model.

4.2.1. Setup of the experiment

The source data for this image segmentation experiment were eight real COVID-19 X-ray images from a public database collected by Cohen et al. [60]. Fig. 9 shows images and their 2D histograms for A, B, C, D, E, F, G, and H. The 2D histogram of the COVID-19 X-ray image shows the pixel information. The image segmentation experiments all involve the MIS-XMACO model and MIS-ACOR, MIS-HHO, MIS-m_SCA, MIS-IGWO, MIS-IWOA, MIS-SCADE, MIS-MVO, MIS-CLPSO, and MIS-BLPSO. Table A8 in the supplementary material shows the parameter settings of all segmentation models involved in this experiment.

To ensure the fairness and reliability of the experimental results, the number of iterations for all models was 100, the size of the COVID-19 X-ray segmentation image was 512 × 400, and the population size of the segmentation model was set to 20. To eliminate the randomness of the experiments, all experiments were run 30 times independently. We evaluated the results of the COVID-19 X-ray segmentation experiments using PSNR, SSIM, and FSIM and performed comparative analyses using the mean, standard deviation, and WSRT.

4.2.2. Details of the data analysis methods

The results of the image segmentation experiments were analyzed by the feature similarity index, structural similarity index, and peak signal-to-noise ratio. The better the data processing results obtained by the PSNR, FSIM, and SSIM methods, the better the designed image segmentation model. Eq. (31), (32), (33), (34) show the specific formula details of PSNR, FSIM, and SSIM.

$$PSNR=20\times {\log }_{10}\left(\frac{255}{RMSE}\right)$$ (31)

$$RMSE=\sqrt{\frac{{\sum }_{i=0}^{M-1}{\sum }_{j=0}^{N-1}{\left(I_{ij}-Se{g}_{ij}\right)}^2}{M\times N}}$$ (32)

$$SSIM=\frac{\left(2{\mu }_I{\mu }_{Seg}+c_1\right)\left(2{\sigma }_{I,Seg}+c_2\right)}{\left({{\mu }_I}^2+{{\mu }_{Seg}}^2+c_1\right)\left({{\sigma }_I}^2+{{\sigma }_{Seg}}^2+c_2\right)}$$ (33)

$$FSIM=\frac{{\sum }_{I\in \Omega }{S}_L\left(X\right)P{C}_m\left(X\right)}{{\sum }_{I\in \Omega }P{C}_m\left(X\right)}$$ (34)

The PSNR result shows the difference between the split image and the original image. The larger the PSNR value of the segmentation result, the better the image segmentation model performs. The SSIM results show the similarity between the original image and the segmented image. The smaller the SSIM value of the segmentation result, the better the image segmentation model performs. FSIM shows the quality score of the significance of a local structure. The larger the FSIM value of the segmentation result, the better the image segmentation model performs.

4.2.3. Experiment result analyses

In the field of image segmentation, the validity and accuracy of the model need to be rigorously analyzed by effective methods. Table A9-Table A10 in the supplementary material depict the STD and AVG obtained for all models after FSIM. The MIS-XMACO model obtained a total of 26 optimal FSIM evaluation AVGs; that is, it obtained the maximum FSIM evaluation results and 30 minimum STDs among all segmentation threshold levels. The MIS-ACOR model obtained a total of 11 optimal FSIM evaluation AVGs and 8 minimum STDs. Table A11-Table A14 in the supplementary material depicts the STDs and AVGs obtained for all models after PSNR and SSIM. The data in bold in the tables represent the optimal values. The AVG and the STD obtained by MIS-XMACO are better than those of comparison models at all threshold levels. These results show that the proposed model can obtain better and more stable segmentation results in most images.

To further demonstrate the segmentation capability of the model, Table A15-Table A17 in the supplementary material show the rank results of WRST of the evaluated results of PSNR, SSIM, and FSIM in all models at all thresholds. The proposed model is first in both WSRT rankings. In addition, in terms of the results of “+/−/ = ”, the proposed model has better or equal performance than other comparative models. Based on the evaluation results of PSNR, FSIM, and SSIM, Fig. 10 -Fig. 12 show the average values obtained by the MIS-XMACO model for all threshold levels on the dataset. The model obtained the maximum value on all three evaluation methods and showed better segmentation results. The experimental results show that the proposed model performs and ranks better on all COVID-19 X-ray images for both the FSIM, SSIM, and PSNR analysis results. Therefore, MIS-XMACO obtained superior results in COVID-19 X-ray image segmentation.

Fig. 10.

Average results of FSIM assessment for all threshold levels.

Fig. 12.

Average results of SSIM assessment for all threshold levels.

To more clearly show the segmentation results of all models on the eight COVID-19 X-ray images, Table A18-Table A19 in the supplementary material show the 2D Kapur's entropy values obtained by the models on all threshold sets. The larger value obtained for 2D Kapur's entropy as the model's objective function proves that the image segmentation is more effective. The MIS-XMACO model in the table obtains the maximum 2D Kapur's entropy for most of the images. Therefore, the data demonstrate that the model can find the optimal threshold sets reliably. Meanwhile, Figure B10 in the supplementary material and Fig. 13 show the images of the maximum 2D Kapur's entropy convergence curves obtained for MIS-XMACO and other segmentation models at 6 and 12 thresholds, respectively. The curves show more intuitively that the proposed model has better convergence speed and convergence accuracy. Moreover, the proposed model shows good convergence at different threshold levels, which proves that the model has good adaptability and stability.

Fig. 13.

All convergence curves obtained by all models at threshold level 12.

Figure B11-Figure B18 in the supplementary material show the results of threshold set (when the threshold value is 6) segmentation for all models on the gray value distribution maps of all COVID-19 X-ray images. Based on the observation and analysis of the segmentation results, it can better demonstrate the segmentation effect of the MIS-XMACO model. Fig. 14 shows all segmentation models' multilevel segmentation effect images of COVID-19 X-ray D. Among them, MIS-XMACO can segment the image into different regions, and the color mapping results also show that the different regions have relatively clear boundaries.

Fig. 11.

Average PSNR assessment results for all threshold levels.

Fig. 14.

COVID-19 X-ray D segmentation results of all models at threshold level 20.

After analyzing all experimental results, we found that the proposed multilevel image segmentation model can obtain higher quality segmentation results when performing COVID-19 X-ray image segmentation. However, the COVID-19 case is not enough to verify the effectiveness of the proposed XMACO-based optimizer. In future work, the method can also be used to tackle more problems, such as human motion capture, active surveillance, service ecosystem, location-based services, information retrieval services, fault diagnosis, pharmacoinformatic data mining, urban road planning, and Kayak cycle phase segmentation. Moreover, XMACO can also be extended to multiobjective or many optimization versions to solve more complex tasks.

5. Conclusions and future works

To improve the efficiency of image segmentation of novel coronavirus pneumonia X-rays, this paper innovatively proposes an MLIS model that can be applied to COVID-19 X-ray image segmentation, MIS-XMACO. The proposed image segmentation model combines a 2D histogram strategy, a nonlocal mean method, 2D Kapur's entropy, and the swarm intelligence algorithm, XMACO. The MIS-XMACO model is applied to eight real COVID-19 X-rays to complete the image segmentation experiments. The XMACO algorithm based on 2D Kapur's entropy and 2D histogram selects the optimal set of thresholds for the X-ray images. Image segmentation experiments are performed on six thresholds at different high and low levels to ensure the experiments' accuracy and the model's stability. We compared MIS-XMACO with ten similar models and evaluated the performance using the FSIM, SSIM, and PSNR methods. The analysis results effectively illustrate the experimental data of the variance, the mean, and WSRT of the COVID-19 X-ray image segmentation results. The segmentation data, segmentation images of COVID-19 X-ray, and 2D Kapur's entropy convergence curve all demonstrate that MIS-XMACO has a better segmentation effect than similar models and maintains stability at different threshold levels.

As the core of the image segmentation model proposed, the ACOR is combined with the directional crossover strategy and the directional mutation strategy, named XMACO, for the first time in this paper. Among them, the directional crossover strategy is effectively applied in the end stage of ant foraging, which can improve the population's convergence speed and accuracy. The directional mutation strategy is applied at the end stage of the directional crossover strategy to enhance the diversity of the population, allowing the algorithm to improve its ability to jump out of the LO. We designed XMACO to evaluate the algorithm's performance on a benchmark function set consisting of IEEE CEC2014 and IEEE CEC2017. The performance of XMACO was evaluated by WSRT, FT, and other methods and confirmed that XMACO has better convergence and searchability than other similar algorithms and the original ACOR. However, the time complexity of the proposed XMACO increases due to the change in the algorithm structure. Therefore, all experiments in this paper were computed using parallel CPUs to improve the computational efficiency of the algorithm.

The work in this paper improves the effectiveness and efficiency of image segmentation in COVID-19 X-rays. Nevertheless, image segmentation is a fundamental task in the medical diagnosis of pneumonia caused by the novel coronavirus. In the future, we will continue our constructive work on feature extraction, feature selection, and classification of COVID-19 X-rays to improve the efficiency of medical diagnosis of patients with novel coronavirus pneumonia. Moreover, MIS-XMACO, as an efficient image segmentation model, will be applied to more kinds of medical image segmentation, aiming to make a more significant contribution to the medical field. Meanwhile, XMACO, as a more efficient optimization algorithm, will be applied to more fields, including engineering design problems, economic emission scheduling problems, feature selection, etc.

Declaration of competing interest

The authors declare that there is no conflict of interests regarding the publication of article.

Appendix A. Supplementary data

The following is/are the supplementary data to this article: