Skip to main content
Scientific Reports logoLink to Scientific Reports
. 2025 Jul 16;15:25707. doi: 10.1038/s41598-025-09211-8

Energy-based segmentation methods for images with non-Gaussian noise

Jiatao Zhong 1, Shiyin Du 1, Canruo Shen 1, Yiting Chen 1, Min Gao 1,3, Medha Naidu 1, Xiaoping Shi 1,, Yuejiao Fu 2
PMCID: PMC12267753  PMID: 40670447

Abstract

This paper proposes an energy-based segmentation method facilitated by the change point detection. We apply the Kullback–Leibler (KL) divergence to demonstrate the feasibility of our method for non-Gaussian noisy images. Notably, the algorithm automatically determines whether the model is solvable using a Gaussian approach and, if not, effortlessly switches to a non-Gaussian alternative. It can also automatically determine the optimal number of classifications. Furthermore, its iterative nature enables the detection and segmentation of small regions that other methods often fail to capture. Compared to the traditional maximum between-class variance technique and recent statistical approaches, this method provides improved thresholding accuracy for bimodal grayscale images. Moreover, in the context of multiple threshold identification, the proposed method outperforms Subtractive Clustering K-Means with Filtering, Sparse Graph Spectral Clustering, Gaussian mixture on Markov random field, and Adaptive Thresholding in segmenting multimodal grayscale images.

Keywords: Energy, Kullback–Leibler divergence, Multimodal grayscale images

Subject terms: Statistics, Computational science

Introduction

Image segmentation is a fundamental technique with important applications across various disciplines. For instance, numerous CT scans and X-rays rely on segmentation to detect tumors or locate lesions within organs. In the case of breast cancer, it is estimated that approximately 15 Canadian women would succumb to the disease daily in 20241. Detecting Lymph node plays a crucial role in both staging and treatment of breast cancer2. Beyond medical image detection, rapid and accurate wildfire detection from satellite imagery has become increasingly urgent. In 2023, Canadian wildfires burned nearly 7.8 million hectares of forest, representing over a quarter of global tree cover loss3. Image segmentation is also crucial in the field of biology. For instance, Gooh et al.4 used optical manipulation to collect 400 images of Arabidopsis early embryogenesis at regular intervals to observe cell division. Accurate cell counting was a prerequisite for this analysis, necessitating effective image segmentation methods.

It is essential to identify or develop robust image segmentation methods to address these issues. A widely known algorithm used in image segmentation is Otsu’s method5. It has been cited as an effective technique6,7, but the Otsu threshold significantly deviates from the intersection point, skewing towards the center of the class with a larger variance. This causes it to fail when handling peaks with differing variances8. Besides Otsu’s method, there are several other methods using thresholds to segment the pictures, for example, Kapur et al.9 introduced an entropy-based technique for image segmentation, where entropy quantifies the amount of image information. The maximum entropy algorithm aims to determine the optimal threshold by maximizing the combined entropy of the background and foreground segments. Kittler and Illingworth10 developed a minimum error thresholding technique. More recent methodologies have explored the use of semi-atomic algorithms for histogram thresholding11. Zhang and Hu12 developed image segmentation based on the 2D Otsu method with histogram analysis and demonstrated that the two-dimensional Otsu method behaves better in segmenting images of low signal-to-noise ratio than the one-dimensional method. However, these methods lack rigorous mathematical or statistical validation of their algorithms. Our approach seeks to apply a statistical method for change point detection based on the Energy-Based Model (EBM), extending single-threshold segmentation to multiple-threshold segmentation. Building on the traditional approach, we developed the Energy-Based Model segmentation method as an enhancement. The EBM can represent any statistical distribution and effectively simulate the gray-level histograms of images. This capability suggests that EBM could be applied to image segmentation tasks based on histogram data.

The objective of EBM is to derive an energy function that accurately represents the probability distribution of data. Originating from physics, EBM often describes the potential energy of a system. The model is inspired by the Boltzmann distribution, in which the state of a system at a given temperature follows a probability distribution inversely proportional to the energy13. Over time, EBM has been extensively applied within the field of computing, the EBM approach offers a unified theoretical framework applicable to a wide range of learning models, encompassing traditional discriminative and generative approaches, as well as graph-transformer networks, conditional random fields, maximum margin Markov networks, and various manifold learning techniques14. Furthermore, the exponential tilting density has notable connections with EBM and can represent any statistical distribution. Compared to probabilistic methods, EBM could be interpreted as a non-probabilistic factor graph, offering significantly greater flexibility in designing architectures and establishing training criteria. Thus, EBM has considerable potential for wide application in statistics.

Our primary objective is to detect appropriate thresholds in the gray-level histograms of images such as CT scans, X-rays, cell images, and satellite wildfire images. The localization of the threshold is crucial for this task, which differs from standard imaging because even small errors can seriously affect the results of professional image segmentation. As for Fig. 1, we converted the colour image to a grayscale image and plotted its grayscale histogram; based on this histogram, we could then identify the change point (a specific grayscale value) in a bimodal histogram. This grayscale value was then used as the threshold value, converting pixels with values above it to black and those below it to white. However, for multiple change points, binary segmentation is insufficient. As shown in Fig. 2, since there are five change points, we need to divide the image into six segments, corresponding to six distinct colours.

Fig. 1.

Fig. 1

The process of a change point segmentation.

Fig. 2.

Fig. 2

The process of multiple change point segmentation.

Determining thresholds is essential for effective image segmentation. To achieve this, we applied EBM to estimate the image histogram and used a change point detection method to facilitate segmentation.

Modelling

EBM with non-Gaussian noise

Based on the Central Limit Theorem, the normalized grayscale histogram of many images can be viewed as one-dimensional Gaussian distributions or Gaussian Mixtures15. There exists a baseline density such as a one-dimensional Gaussian density defined as

graphic file with name d33e325.gif 1

where Inline graphic represents the mean, and Inline graphic represents the standard deviation. Since Gaussian distribution belongs to the exponential family, (1) can be also written as

graphic file with name d33e348.gif 2

where Inline graphic, Inline graphic, Inline graphic. It is worth noting that some normalized grayscale image histograms can not be modeled as Gaussian distributions or as Gaussian Mixtures16. To tackle this problem, we developed an extended probabilistic model including both Gaussian and Non-Gaussian distributions. Given a baseline density Inline graphic, we can get the optimal extension according to Kullback–Leibler (KL) divergence which was defined as

graphic file with name d33e384.gif 3

where a smaller KL divergence value indicates that the distribution Inline graphic is closer to Inline graphic. Therefore, minimizing the KL divergence can be interpreted as a way of finding the Inline graphic that best approximates Inline graphic.

Next, we aim for Inline graphic to closely approximate a reference density Inline graphic in an information-theoretic sense. Since we aim to encompass a broader range of potential expressions, according to Shi et al.17, we let Inline graphic be a positive measure on Inline graphic and define Inline graphic as the collection of positive probability densities Inline graphic on Inline graphic such that

graphic file with name d33e464.gif 4

where Inline graphic, Inline graphic, Inline graphic and Inline graphic are real numbers. In particular, Inline graphic represents a special case of Inline graphic with Inline graphic. If Inline graphic and Inline graphic, then Inline graphic. Let Inline graphic be some real constants such that

graphic file with name d33e539.gif 5

In light of Shi et al.17, the optimal choice in terms of image information should be the one that minimizes the KL divergence;

graphic file with name d33e550.gif 6

which can also be represented by

graphic file with name d33e558.gif 7

This represents a specific form of EBM, where Inline graphic, Inline graphic is a vector of predictors with Inline graphic dimensions, and Inline graphic. Note that this model remains feasible even for large p, indicating its ability to handle images with non-Gaussian noise.

EBM with single change point

By the definition of the energy-based model in (7), and in conjunction with the process of research on change points by Jin et al.18, we develop a change point detection method based on the energy model. To facilitate image segmentation, we need to extend EBM Inline graphic to EBM with change point G(x), as defined below:

graphic file with name d33e617.gif 8

where Inline graphic is a sequence of Inline graphic dimensional vector of predictors, Inline graphic and Inline graphic are unknown Inline graphic dimensional vector of regression coefficients, Inline graphic is an unknown change point location.

Since the normalized grayscale histogram of an image is bounded and discrete in practical applications, we define Inline graphic as the vertical coordinates of the histogram and Inline graphic as the horizontal coordinates, introducing the error term Inline graphic for normalization:

graphic file with name d33e681.gif 9

where Inline graphic, Inline graphic is a sequence of Inline graphic dimensional vector of predictors, Inline graphic and Inline graphic are unknown regression coefficients, and ab represent the endpoints of the pixel value range, Inline graphic, Inline graphic is the total number of combinations of pixel values (e.g., Inline graphic for an 8-bit image).

Change point detection in grayscale histogram

In light of Csörgő & Horvath19 and Jin et al.18, whose methods for detecting change points in linear regression are useful and have been rigorously validated through mathematical proofs, we used the least squares regression to estimate unknown change point Inline graphic because polynomial regression is a kind of linear regression. We divide the data sequence into two segments using Inline graphic as the dividing point and employ least squares polynomial regression within each segment, resulting in the following least squares estimation of coefficients:

graphic file with name d33e768.gif 10

where Inline graphic, Inline graphic, Inline graphic, and Inline graphic. Then, we apply these coefficients to estimate Inline graphic,

graphic file with name d33e806.gif 11

Following Csörgő & Horvath19, we specify the following least squares criterion

graphic file with name d33e818.gif 12

The least squares estimator of Inline graphic is

graphic file with name d33e832.gif 13

Subsequently, we employed the likelihood ratio test to validate the existence of a change point at Inline graphic. Following Csörgő & Horvath19 , under a set of assumptions C.1–C.5 (See Supplementary Equations S1 online), the test statistic is given by

graphic file with name d33e853.gif 14

where Inline graphic We claim that there exists a change point at Inline graphic, if

graphic file with name d33e873.gif 15

where Inline graphic, Inline graphic, α is the significance level, and Inline graphic denotes Gamma function.

Next, the asymptotic distribution of the statistic is presented. Let

graphic file with name d33e900.gif 16
graphic file with name d33e906.gif 17
graphic file with name d33e912.gif 18

From (16)–(18), there is

graphic file with name d33e926.gif 19

To verify this asymptotic result, we conducted a Monte Carlo simulation. We selected different sample sizes, Inline graphic, and generated random variable sequences Inline graphic that satisfy the required conditions for each sample size, thereby obtaining the sample distribution of the statistic Inline graphic. Next, we estimated the critical value Inline graphic at the significance level Inline graphic, which satisfies

graphic file with name d33e965.gif

By solving for Inline graphic with Inline graphic, we obtain Inline graphic. We performed 10,000 simulations and obtained the 95th percentile of Inline graphic as 3.264, 3.302, and 3.323, corresponding to n = 256, 512, and 1024. We observed that the 95th percentile converges to an asymptotic value of 3.663, which confirms the validity of the asymptotic distribution in (19).

To further illustrate and evaluate the performance of the proposed procedures, we give an example to compare this method with the traditional Otsu’s method through a Monte Carlo experiment.

Example 1

Let Inline graphic, Inline graphic, and Inline graphic in (9). We defined

graphic file with name d33e1029.gif

where Inline graphic. The density histogram of Inline graphic is shown as Fig. 3.

Fig. 3.

Fig. 3

The density histogram of Inline graphic.

We divided the data sequence into two segments using Inline graphic as the dividing point. According to (12), we can get

graphic file with name d33e1080.gif

Next, we employed the Energy Based Model Segmentation (EBS) change point detection method in (13) to identify the change point. Using the EBS method yields the value of the statistic as (a) in Fig.4

graphic file with name d33e1091.gif
Fig. 4.

Fig. 4

Comparison of EBM and Otsu’s method.

We then applied (15) to evaluate the significance of the change point. With Inline graphic, the observed test statistic is

graphic file with name d33e1107.gif

Therefore, Inline graphic was a significant change point estimation.

For comparison purposes, Otsu’s detection method is introduced below. We found that this well-known algorithm is, in fact, equivalent to detecting change point within the grayscale image histogram. Therefore, we provide a rigorous mathematical definition for it and reformulate it within the context of change point detection.

Let the pixels of a given picture be represented in Inline graphic gray levels Inline graphic. The number of pixels at level i is denoted by Inline graphic and the total number of pixels by Inline graphic. In order to simplify the discussion, the gray-level histogram is normalized and regarded as a probability distribution:

graphic file with name d33e1152.gif

Suppose that we divide the pixels into two classes (background and foreground) using Inline graphic as the dividing point, Inline graphic. Pixels with levels Inline graphic belong to one class, while pixels with levels Inline graphic belong to the other class. In the context of change point detection, Otsu’s method is equivalent to finding the optimal change point by maximizing the between-class variance:

graphic file with name d33e1183.gif 20

and

graphic file with name d33e1191.gif 21

The details of (20) can be found in Supplementary Methods S2 online.

As illustrated in Fig.4, EBM method identified the change point efficiently, where Inline graphic, whereas the Otsu method failed to do so. This can be explained by the fact that the Otsu threshold tends to deviate from the intersection point of grayscale histogram, skewing towards the center of the class with larger variance8.

Example 2

A dataset of Inline graphic grayscale images was synthetically generated to evaluate the performance of various image segmentation algorithms under challenging conditions. Each image has dimensions of SZ Inline graphic SZ pixels, where SZ denotes the image size and is set to 100. Initially, we tested the algorithms on a simpler set of images, where all methods performed well. To better assess their robustness in complex scenarios, we increased the difficulty of the test set. The generation process for each image, denoted as I, is described in the Supplementary Method S3 online. The resulting 1000 images are highly noisy and challenging to segment. Representative examples are shown in Fig. 5.

Fig. 5.

Fig. 5

Representative samples from the synthetic dataset, illustrating diversity in geometric shapes, grayscale values, sizes, and positions.

The performance of the segmentation methods is evaluated using the F score, specifically the Inline graphic score20, which emphasizes recall over precision. Let GT be the ground truth binary mask (where Inline graphic for the foreground, Inline graphic for the background), derived directly from the generated mask M. Let BS be the binary segmentation mask produced by a given algorithm for the image I. We define the following quantities based on pixel-wise comparison between BS and GT:

graphic file with name d33e1309.gif

where TP is the number of pixels correctly identified as foreground, FP is the number of pixels incorrectly identified as foreground (background pixels classified as foreground), FN is the number of pixels incorrectly identified as background (foreground pixels classified as background), Inline graphic is the indicator function. From these quantities, we calculate Precision (P) and Recall (R) as follows:

graphic file with name d33e1337.gif

where Precision (P) represents the proportion of pixels identified as foreground that are actually foreground, and Recall (R) denotes the proportion of actual foreground pixels correctly identified. The general F score (Inline graphic) is the harmonic mean of Precision and Recall, weighted by a factor Inline graphic:

graphic file with name d33e1365.gif

In biomedical image analysis, the Inline graphic score is a commonly used evaluation metric. This choice gives R twice the importance of Precision:

graphic file with name d33e1382.gif

We have used different methods for image segmentation, including the Otsu method and the recent change point detection methods: Inspect21, Wild Binary Segmentation (WBS)22, Standard Binary Segmentation (SBS)22, Graph-based change-point (GCP)23 as well as the non-threshold segmentation method Bayesian Gaussian Mixture24 (BGM). Table 1 shows that EBS achieves the highest average Inline graphic score, demonstrating strong overall performance.

Table 1.

Average Inline graphic scores for different segmentation methods on the synthetic dataset.

Method Otsu EBS Inspect WBS SBS GCP BGM
Average Inline graphic score 0.1791 0.2298 0.1352 0.1865 0.1585 0.0659 0.2113

Algorithm

This section outlines the steps in EBS and Adaptive Multiple change point Energy-based Model Segmentation (MEBS) algorithms.

EBS

Based on the theorem of the EBM method, the procedural steps of energy-based image segmentation are delineated as follows:

Algorithm 1.

Algorithm 1

Algorithm for image processing

In Algorithm 1, the CalculateTauHat function uses parameters a and b to define the grayscale value interval for histogram analysis, defaulting to the full range (e.g., 0–255). The trim parameter (default False) controls whether this interval is automatically adjusted to exclude histogram tail-end data, typically based on cumulative distribution thresholds (e.g., removing the lowest and highest Inline graphic). This focuses on the analysis on of the main grayscale distribution. The parameter p (default 2) sets the polynomial degree used to fit the histogram within the (possibly trimmed) interval. Together, these parameters guide the calculation of the optimal segmentation threshold Inline graphic. The algorithm processes an image with N pixels and a gray range M. Initially, the algorithm computes the histogram H by examining each of the N pixels, an operation with a time complexity of O(N). Following this, it processes the CalculateTauHat function, which involves performing change point detection using polynomial fits on the histogram H. Given that the histogram consists of M bins corresponding to the gray range, this subsequent step has a computational complexity of Inline graphic. Therefore, the total computational complexity for this algorithm is Inline graphic.

It is worth noting that the EBS could only handle images with bimodal grayscale histograms, making it challenging to process images with histograms containing more than two peaks. The MEBS method extends the original EBS by detecting multiple change points through a bi-section approach, enabling it to process complex, multi-modal images beyond the bimodal limitation of EBS. Specifically, EBS serves as the basis for MEBS, and in the special case of Inline graphic, EBS can be considered the initial step within the MEBS framework.

MEBS

The limit theorem in change point analysis typically requires a large sample, conventionally defined as 30 or more observations25. Since we need two regressions within a region to find the change point, the minimum number of regions is 60. Therefore, According to (12), we get

graphic file with name d33e1594.gif 22

Therefore, we set Inline graphic. Based on Algorithm 1 of EBS, The algorithmic flow chart of MEBS is shown in Fig.6:

Fig. 6.

Fig. 6

MEBS flowchart.

The definitions of Inline graphic could be found in (9), (13), (14), and (15). List.append(x) means adding x to the List. The computational complexity of the MEBS algorithm is Inline graphic, where L is the number of significant thresholds found. Fig.6 demonstrates that if the CalculateTauHat function, applied to a given interval with polynomial degree p, fails to detect a significant threshold, the algorithm does not immediately discard the interval. Instead, the algorithm increases the polynomial degree from p to Inline graphic and refits the histogram data for segmentation. Automatically increasing p enhances the robustness and capability of the algorithm in detecting thresholds in histograms with complex or subtle patterns. The recursion terminates only when either a significant threshold cannot be identified after incrementing the polynomial degree p, or the interval length falls below a predefined minimum threshold.

Empirical applications

Based on the algorithm presented, we now illustrate their applications through several empirical examples. All applications were implemented on a personal computer equipped with a 3.8 GHz CPU, utilizing Python 3.11 as the programming language.

EBS applications

Lung fluids segmentation

We acknowledge the use of this Inline graphic original image from Nouri, S. et al.26. The image is licensed under CC BY-NC 4.0 (https://creativecommons.org/licenses/by-nc/4.0/). It indicates bilateral pleural effusion (indicated by blue arrows) in a 67-year-old female patient with acute myeloid leukemia (AML), suggesting pulmonary edema. Fluids within the lungs are particularly insidious, rendering their isolation challenging. As illustrated in Fig. 7, the EBS method appears to segment bilateral pleural effusions with clarity effectively. In contrast due to its lack of sensitivity to finer details, Otsu’s method failed to discern the bilateral pleural effusions. Although Modified Otsu27 aligns more closely with EBS than Otsu, it still inherits Otsu’s limitations in single-threshold segmentation. Hommo-Ostu28 integrates homomorphic transform with Otsu segmentation to enhance contrast between detected regions and normal tissue, but its effectiveness in pleural effusion segmentation remains limited.

Fig. 7.

Fig. 7

Comparison of EBS, Otsu, Modified Otsu, and Hommo-Otsu’s methods for lung fluids segmentation.

Brain tumor segmentation

Medical image analysis techniques using brain X-ray are used for early diagnosis and treatment planning, assisting radiologists in identifying and evaluating brain tumors. We acknowledge using this Inline graphic original image from the Brain Tumor Dataset29. The contents of this dataset are licensed under DbCL v1.0 (https://opendatacommons.org/licenses/dbcl/1-0/). It shows the brain X-ray of a tumor patient, with the tumor’s location indicated by the red arrow. As shown in Fig. 8, the EBS method effectively highlights the tumor contour, though some white noise is still present in surrounding areas. In contrast, other methods struggle to segment the tumor, focusing mainly on the overall brain contour.

Fig. 8.

Fig. 8

Comparison of EBS, Otsu, Kapur and efficient brain tumor’s methods for brain tumor segmentation.

Applications of MEBS

Detailed descriptions of the MEBS operations on individual images can be found in Supplementary Discussion S4 online.

Breast tumor detection in mammograms

Comparison of MEBS and Other Segmentation Methods for Breast Tumor Detection in Mammograms

The Inline graphic images in Fig. 9 depict breast tumors in the mammogram from Alsolami et al.30. We acknowledge their work, which is licensed under CC0 1.0 (https://creativecommons.org/publicdomain/zero/1.0/). These X-ray images were annotated and validated by three different radiologists, Dr. Sawsan Ashoor, Dr. Samia Alamoud, and Dr. Gawaher Al Ahadi, and the breast cancer images were segmented through hand-drawing on the suspicious areas. The structure of this particular tumor closely resembled the complex architecture of a healthy breast. Therefore, it is challenging to identify any distinctive or differentiating features from normal breast tissue. Due to the multiple peaks in the grayscale histogram of these images, simple threshold detection is insufficient. Therefore, we apply the Adaptive Multiple change point Energy-based Model Segmentation (MEBS). We compare its performance to several methods, including Subtractive Clustering K-Means with Filtering (SCKF)31, Sparse Graph Spectral Clustering (SGSC)32, Gaussian mixture model based on Markov random field (MRF-GMM)33, and Adaptive Thresholding34. MEBS identified the optimal number of clusters as Inline graphic. Since the other methods require manual selection of the cluster number, we fixed this value for all methods to facilitate a direct comparison. The results demonstrate that tumor regions seem more accurately detected using MEBS, particularly in Fig.9 (Inline graphic) and Fig.9 (Inline graphic). MEBS appears to segment nearly the entire tumor, whereas the other methods fail to achieve this level of precision. Tumor segmentation precision was quantitatively evaluated using the Inline graphic score, as presented in Table 2. Details of the evaluation procedure are provided in Supplementary Method S5 online. The results indicate that MEBS consistently achieves the highest or near-highest Inline graphic scores, significantly outperforming SCKF and MRF-GMM. Compared to the SGSC method, MEBS performs comparably overall.

Fig. 9.

Fig. 9

Comparison of MEBS and other methods for breast tumor detection (Inline graphic).

Table 2.

Inline graphic scores in breast tumor detection.

Method A B C D
MEBS 0.503 0.679 0.467 0.842
SGSC 0.494 0.711 0.483 0.841
SCKF 0.452 0.267 0.301 0.137
MRF-GMM 0.466 0.359 0.280 0.154

Cells counting analysis

The Inline graphic pixel cell image is from Gooh et al.4 and used with the authors’ permission. They operated optical manipulation of Arabidopsis’s early embryogenesis and took a picture at every other unit of time. Counting these cells within each unit of time can provide insights into the growth trends of plants. In addition to cell count, other characteristics such as morphological changes, size, density, and position of cells can also be considered. By analyzing variations in these features, it is possible to gain a more comprehensive understanding of the developmental stages of plant embryos, which may be important for future agricultural applications. Image segmentation is crucial before cell counting can be conducted. Due to interference from the cellular matrix illustrated in Fig. 10, it is challenging for image segmentation techniques to produce precise outcomes. We selected the 1st, 101st, 201st, and 301st images to apply different segmentation methods and evaluate their effectiveness. According to Fig. 11, MEBS, Adaptive Mean Threshold, and Adaptive Gaussian Threshold appear to perform better than the other methods. In Fig. 11A, MEBS demonstrated notable effectiveness in segmenting individual cells. Other clustering methods were ineffective because they could not eliminate the noise produced by the original epidermal layer and matrix of the embryo, resulting in suboptimal cell segmentation. However, although the Adaptive Mean Threshold and Adaptive Gaussian Threshold performed segmentation as effectively as MEBS, they could only binarize the images to black and white, resulting in a substantial loss of information. On the other hand, MEBS segments the cells but also classifies them into different colors. The results could have potential applications in identifying differences in the cell division process. Table 3 further highlights the overall superior performance of MEBS compared to other methods, based on the Inline graphic score.

Fig. 10.

Fig. 10

Embryonic epidermal layer and matrix.

Fig. 11.

Fig. 11

Comparison of MEBS and other methods for cells counting analysis.

Table 3.

Inline graphic scores in cells counting analysis.

Method A B C D
MEBS 0.826 0.850 0.911 0.860
SCKF 0.528 0.860 0.957 0.347
SGSC 0.802 0.060 0.942 0.870
MRF-GMM 0.827 0.462 0.952 0.853

Wildfire spark detection

These 668 Inline graphic 478 pixel images, originally acquired by the MODIS instrument aboard the Terra Earth-orbiting satellite, were obtained from NASA’s LAADS DAAC35. Hyperspectral images taken from satellites are well suited to analyze wildfires. Hyperspectral images are characterized by having many discrete layers or bands over a range of wavelengths. Specific bands will be sensitive to different features. For example, some bands will capture clouds, snow, or for our analysis, fire well36. Consequently, the image is plagued with substantial noise, rendering conventional image segmentation methods ineffective in this scenario. For this kind of image, we do not quantitatively compare them because the wildfire sparks are just a few pixels, and it is straightforward to see whether they are segmented or not. As illustrated in Fig.1236, the tendency of the other methods toward excessive generalization impairs their ability to detect spark points, significantly compromising their segmentation results accurately. In contrast, MEBS and SGSC have a strong ability to recognize images and detect sparks effectively. It is interesting to note that both MEBS and SGSC successfully identified two red sparks, as indicated by the arrows in Fig. 12C. This robustness is particularly important when dealing with satellite remote sensing data, which are often subject to noise and uncertainty. This demonstrates the potential of MEBS for future wildfire spark detection applications, providing a promising tool for effective wildfire spark prevention and control strategies.

Fig. 12.

Fig. 12

Comparison of MEBS and other methods for wildfire spark detection.

Conclusions

This paper introduces an energy-based image segmentation method inspired by energy functions in physics and incorporates change point detection within a stochastic process framework. The advantage of this approach is demonstrated through the use of KL divergence. Building on this foundation, we propose an innovative and effective method for applying change point detection to image segmentation. The EBS method exhibits refined segmentation capabilities, particularly in capturing complex details that are often imperceptible to the human eye. In empirical studies, we constructed a highly challenging test dataset, and comparative analyses show that EBS achieves greater advantages over competing methods. In practical applications, EBS demonstrates high accuracy in segmenting medical images. Furthermore, we extend EBS to the MEBS framework, which can automatically determine the optimal number of categories and performs well when processing multi-modal images. MEBS achieves high precision in detecting sparks in wildfire imagery and, based on the Inline graphic score evaluation, outperforms other methods in segmenting diverse types of images.

One limitation of the current study is its focus on grayscale images. For color images, the different-channel intensities can be converted into a single grayscale intensity using weighted-sum methods37,38. In light of the findings reported by Shi et al.39, our method can be readily extended to handle color image segmentation.

Supplementary Information

Acknowledgements

We would like to thank the three reviewers and the editor for their valuable suggestions, which have significantly improved the presentation of this paper. We thank Dr. Daisuke Kurihara for providing the cell image data. We also thank Dr. Fateme Sadat Hosseini, Dr. Mahmoud Shaheen, Dr. Asmaa Saad, and NASA LAADS DAAC for making the datasets used in this study available.

Author contributions

J.Z. and X.S. designed this research. J.Z. performed this research and wrote the main manuscript. S.D. assisted in modelling and empirical applications. C.S. assisted in evaluating the performance of the MEBS method in image histogram analysis. Y.C. performed change point detection in empirical experiments and evaluated the performance of methods. M.G. assisted in model development and performed sensitivity analysis. M.N., X.S., and Y.F. reviewed and revised the manuscript. All authors reviewed the manuscript.

Funding

Dr. Shi discloses support for the research of this work from the Natural Sciences and Engineering Research Council of Canada [grant number RGPIN-2022-03264], the NSERC Alliance International Catalyst [grant number ALLRP 590341-23] and the University of British Columbia Okanagan (UBC-O) Vice Principal Research in collaboration with UBC-O Irving K. Dr. Fu discloses support for the research of this work from the Natural Sciences and Engineering Research Council of Canada [grant RGPIN 2018-05846].

Data Availability

The data and code will be made available on reasonable request to the corresponding author of the paper. The content list is available in Supplementary Data S6 online.

Declarations

Competing interests

The authors declare no competing interests.

Human participants or animals

All human data used in this study were accessed in an anonymized or de-identified format. This research involved only the secondary analysis of these existing data. There was no direct interaction with human participants or animals by the researchers involved in this study, nor was there direct handling of the original plant specimens. Consequently, as the study relied on existing, publicly available or permitted, and anonymized/de-identified data, formal ethical approval from an Institutional Review Board (IRB) or Ethics Committee was not required for this specific secondary analysis. The use of all datasets adhered strictly to their respective licensing terms and the permissions granted.

Footnotes

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-025-09211-8.

References

  • 1.Canadian Cancer Society. Breast Cancer Statistics. https://cancer.ca/en/cancer-information/cancer-types/breast/statistics (2024). Accessed 12 Aug 2024.
  • 2.Cascetta, K. What does it mean if breast cancer spreads to your lymph nodes? https://www.healthline.com/health/breast-cancer/breast-cancer-lymph-nodes (2023). Accessed 12 Aug 2024.
  • 3.MacCarthy, J., Tyukavina, A., Weisse, M. J., Harris, N. & Glen, E. Extreme wildfires in Canada and their contribution to global loss in tree cover and carbon emissions in 2023. Glob. Change Biol.30, e17392 (2024). [DOI] [PubMed] [Google Scholar]
  • 4.Gooh, K. et al. Live-cell imaging and optical manipulation of arabidopsis early embryogenesis. Dev. Cell34, 242–251 (2015). [DOI] [PubMed] [Google Scholar]
  • 5.Otsu, N. et al. A threshold selection method from gray-level histograms. Automatica11, 23–27 (1975). [Google Scholar]
  • 6.Yu, B., Jain, A. K. & Mohiuddin, M. Address block location on complex mail pieces. In Proceedings of the Fourth International Conference on Document Analysis and Recognition, vol. 2, 897–901 (IEEE, 1997).
  • 7.Kurita, T., Otsu, N. & Abdelmalek, N. Maximum likelihood thresholding based on population mixture models. Pattern Recogn.25, 1231–1240 (1992). [Google Scholar]
  • 8.Xu, X., Xu, S., Jin, L. & Song, E. Characteristic analysis of otsu threshold and its applications. Pattern Recogn. Lett.32, 956–961 (2011). [Google Scholar]
  • 9.Kapur, J. N., Sahoo, P. K. & Wong, A. K. A new method for gray-level picture thresholding using the entropy of the histogram. Comput. Vis. Graph. Image Process.29, 273–285 (1985). [Google Scholar]
  • 10.Kittler, J. & Illingworth, J. Minimum error thresholding. Pattern Recogn.19, 41–47 (1986). [Google Scholar]
  • 11.Raju, P. D. R. & Neelima, G. Image segmentation by using histogram thresholding. Int. J. Comput. Sci. Eng. Technol.2, 776–779 (2012). [Google Scholar]
  • 12.Zhang, J. & Hu, J. Image segmentation based on 2d otsu method with histogram analysis. In 2008 International Conference on Computer Science and Software Engineering, vol. 6, 105–108 (IEEE, 2008).
  • 13.Lindley, D. Boltzmann’s atom: the great debate that launched a revolution in physics (Simon and Schuster, 2001).
  • 14.LeCun, Y. et al. A tutorial on energy-based learning. Predicting structured data1 (2006).
  • 15.Bouman, C. A., Shapiro, M., Cook, G., Atkins, C. B. & Cheng, H. Cluster: An unsupervised algorithm for modeling Gaussian mixtures (Springer, 1997). [Google Scholar]
  • 16.Ruderman, D. L. & Bialek, W. Statistics of natural images: Scaling in the woods. Phys. Rev. Lett.73, 814 (1994). [DOI] [PubMed] [Google Scholar]
  • 17.Shi, X., Wang, X.-S., Wong, A. & Lin, W. Approximate inference with exponential tilting densities: theory andapplications. Statistical Science accepted on Dec 28, 2024 (2024).
  • 18.Jin, B., Wu, Y. & Shi, X. Consistent two-stage multiple change-point detection in linear models. Can. J. Stat.44, 161–179 (2016). [Google Scholar]
  • 19.Csorgo, M. & Horváth, L. Limit theorems in change-point analysis (1997).
  • 20.Van Rijsbergen, C. J. Information Retrieval, 2nd edn. (1979).
  • 21.Wang, T. & Samworth, R. J. High dimensional change point estimation via sparse projection. J. R. Stat. Soc. Ser. B Stat Methodol.80, 57–83 (2018). [Google Scholar]
  • 22.Fryzlewicz, P. Wild binary segmentation for multiple change-point detection. Ann. Stat. (2014).
  • 23.Shi, X., Wu, Y. & Rao, C. R. Consistent and powerful graph-based change-point test for high-dimensional data. Proc. Natl. Acad. Sci.114, 3873–3878 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Pedregosa, F. et al. Scikit-learn: Machine learning in python. J. Mach. Learn. Res.12, 2825–2830 (2011). [Google Scholar]
  • 25.Cohen, J. Approximate power and sample size determination for common one-sample and two-sample hypothesis tests. Educ. Psychol. Measur.30, 811–831 (1970). [Google Scholar]
  • 26.Nouri, S., Mirhosseini, N., Naghibi, N. & Hasanian, M. Thoracic ct scan findings in patients with confirmed hematologic malignancies admitted to the hospital with acute pulmonary symptoms. Int. J. Cancer Manag.16 (2023).
  • 27.Faragallah, O. S., El-Hoseny, H. M. & El-sayed, H. S. Efficient brain tumor segmentation using otsu and k-means clustering in homomorphic transform. Biomed. Signal Process. Control84, 104712 (2023). [Google Scholar]
  • 28.Singh, S., Mittal, N., Singh, H. & Oliva, D. Improving the segmentation of digital images by using a modified otsu’s between-class variance. Multim. Tools Appl.82, 40701–40743 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Shaheen, M. Brain Tumor Dataset. https://www.kaggle.com/datasets/mahmoudshaheen1134/brain-tumor-dataset (2022). Accessed 29 April 2025.
  • 30.Alsolami, A. S. et al. King abdulaziz university breast cancer mammogram dataset (kau-bcmd). Data6, 111 (2021). [Google Scholar]
  • 31.Deeparani, K. & Sudhakar, P. Efficient image segmentation and implementation of k-means clustering. Mater. Today Proc.45, 8076–8079 (2021). [Google Scholar]
  • 32.Palnitkar, R. & Neto, J. F. S. R. A sparse graph formulation for efficient spectral image segmentation. In 2024 IEEE International Conference on Image Processing (ICIP), 1410–1416 (IEEE, 2024).
  • 33.Zhang, Y., Brady, M. & Smith, S. Segmentation of brain mr images through a hidden markov random field model and the expectation-maximization algorithm. IEEE Trans. Med. Imaging20, 45–57 (2001). [DOI] [PubMed] [Google Scholar]
  • 34.Library, O. Opencv: Open Source Computer Vision Library (2024). Accessed 02 Aug 2024.
  • 35.Level 1 and Atmospheres Archive and Distribution System (LAADS) Distributed Active Archive Center (DAAC). Terra/MODIS MOD021KM, Level 1B Calibrated Radiances - 1km (March 2019) (2019).
  • 36.Giles, P. A summary of research on fire detection problem and change-point analysis using modis data. Tech. Rep., Research report with Dr. Xiaoping Shi, unpublished (2019).
  • 37.Jack, K. Video Demystified: A Handbook for the Digital Engineer (Elsevier, 2011). [Google Scholar]
  • 38.Pratt, W. K. Digital Image Processing: PIKS Scientific Inside Vol. 4 (Wiley, 2007). [Google Scholar]
  • 39.Shi, X., Wu, Y. & Rao, C. R. Consistent and powerful non-euclidean graph-based change-point test with applications to segmenting random interfered video data. Proc. Natl. Acad. Sci.115, 5914–5919 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Data Availability Statement

The data and code will be made available on reasonable request to the corresponding author of the paper. The content list is available in Supplementary Data S6 online.


Articles from Scientific Reports are provided here courtesy of Nature Publishing Group

RESOURCES