Development of an image processing software for quantification of histological calcification staining images

Abstract

Quantification of the histological staining images gives important insights in biomedical research. In wet lab, it is common to have some stains off the target to become unwanted noisy stains during the generation of histological staining images. The current tools designed for quantification of histological staining images do not consider such situations; instead, the stained region is identified based on assumptions that the background is pure and clean. The goal of this study is to develop a light software named Staining Quantification (SQ) tool which could handle the image quantification job with features for removing a large amount of unwanted stains blended or overlaid with Region of Interest (ROI) in complex scenarios. The core algorithm was based on the method of higher order statistics transformation, and local density filtering. Compared with two state-of-art thresholding methods (i.e. Otsu’s method and Triclass thresholding method), the SQ tool outperformed in situations such as (1) images with weak positive signals and experimental caused dirty stains; (2) images with experimental counterstaining by multiple colors; (3) complicated histological structure of target tissues. The algorithm was developed in R4.0.2 with over a thousand in-house histological images containing Alizarin Red (AR) and Von Kossa (VK) staining, and was validated using external images. For the measurements of area and intensity in total and stained region, the average mean of difference in percentage between SQ and ImageJ were all less than 0.05. Using this as a criterion of successful image recognition, the success rate for all measurements in AR, VK and external validation batch were above 0.8. The test of Pearson’s coefficient, difference between SQ and ImageJ, and difference of proportions between SQ and ImageJ were all significant at level of 0.05. Our results indicated that the SQ tool is well established for automatic histological staining image quantification.

Introduction

Histological staining is a method used for labeling and identifying tissues and cells in biomedical research and diagnosis. Histological staining of biological calcification and mineralization relates to skeletal tissue development [1, 2] and tooth formation [3], meanwhile histological staining of pathological calcification in heart valves and arteries could be an early sign of heart diseases [4, 5]. Alizarin Red (AR) and Von Kossa (VK) are two commonly used histological staining methods for the use of identifying calcium components deposited in tissues and cells. Both AR and VK staining methods are widely used in testing osteoblast differentiation, osteocyte and its histopathology-physiological calcification changes in the field of orthopedics and dentistry studies. They are also used for identifying pathological calcification in cardiovascular diseases and other related fields. Therefore, an accurate quantification of the calcified region in AR and VK staining assays at cell and tissue levels is valuable for biological experimental research, disease diagnosis and drug development.

Image recognition is widely applied in quantification of staining areas in the histological staining images [6–10], in which most of these algorithms were developed for recognizing the calcified region based on the object pattern rather than the degree of its color intensity. To the best of our knowledge, there is no methods or tools specifically designed for solving the staining quantification work with massive noisy stains, which is however a common issue in research labs. Images are usually preprocessed by cleaning up its major dirty background manually prior to the automatic quantification phase. An automated tool to quantify the staining results from images is lacked. Therefore, the goal of this study is to develop a tool that could complete the quantification work without manually preprocessing, meanwhile, report results in a batch processing manner.

As of today, the most well-used and widely accepted tool for analyzing histological staining images is ImageJ [10–12], developed by the National Institutes of Health (NIH, USA). This light software of ImageJ is a quick starter for researchers to have the Region of Interest (ROI) outlined by hand-drawing or thresholding, and then report the quantitative outputs accordingly. The use of ImageJ is a general good practice in most laboratories; however, some limitations would hinder its use to reflect the subtle differences at cellular and tissue levels. Firstly, to achieve the best results, most images request to be preprocessed and adjusted manually image-by-image prior to the ROI recognition step, and hence it prevents the use of the software to execute image quantification in a batch processing manner. Secondly, it lacks stability to standardize the outlining and thresholding of the ROI so that the reported results could be biased upon human’s observation. Moreover, since the color and texture can be conterminous between the background noise and the ROI, it is difficult for an automatic recognition to distinguish the background noise from ROI. Current solution for maintaining measurement accuracy for images with massive noises is to define the ROI manually, which is adopted by many software [13] including a commercially available staining image analysis software TissueQuest (TissueGnostics GmbH, Austria).

Behind those wrapped up software, Otsu’s method and Triclass thresholding method are two state-of-art thresholding algorithms for automatic image segmentation. Otsu’s method searches one threshold that segments the image into two classes by minimizing the intra-class variances. Triclass thresholding method separates the image into three classes by the same idea (i.e., minimizing the intra-class variances) and iteratively execute the segmentation until the class in the middle of foreground and background is completely segmented into the upper and lower classes [14, 15]. Triclass thresholding can be a better choice over Otsu’s method in the case that one or more strong signals exhibited in the image [15]. However, since both Otsu’s method and Triclass thresholding method are solely based on pixel values, they could be unable to distinguish the foreground if its background noise has equal or greater pixel values, or the background noise is overlaid with ROI.

Here, we report an unsupervised algorithm for automatic recognition and quantification of histological and cytological staining, especially for calcium staining and quantification (Fig 1).

Fig 1. Schematic diagram of the research question and design.

Schematic diagram summarized the entire schematic diagram of this study from data collection to image quantification via a set of experimental efforts and computational process. The cell sample was originally collected from human derived primary stem cells, and was transferred to 3D pellets for chondrogenesis and calcification. Next, the calcification assay was assigned by applying two types of staining methods and the staining images were photographed for automatic quantification afterwards. Finally, the quantitative readouts along with visualized results were displayed by using the designated SQ tool for staining quantification. This diagram is created with BioRender.com.

Materials and methods

Histological mineralization image acquisition

We obtained more than a thousand histological images of AR and VK staining from our well-established stem cell pellet culture and osteogenic induction model [16–18] (Fig 1, top panel). In brief, 2 × 10⁵ human cartilage tissue specific stem/progenitor cells were isolated and cultured into pellets follow our previously protocol [16–18], and using human derived cells as study material was approved by The Chinese University of Hong Kong, the Joint Chinese University of Hong Kong–New Territories East Cluster Clinical Research Ethics Committee (The Joint CUHK-NTEC CREC, Ref. No. 2019.078) with written informed consent obtained. Cells were spun down in V-shaped bottom 96-well plates and cultured for 7 days in high glucose DMEM (Gibco, Waltham, Massachusetts, USA) containing 1× ITS (Gibco), 1× antibiotic-antimycotic (Gibco), 10 ng/ml TGF-β3 (PeproTech, Rocky Hill, NJ, USA), 50 μg/ml ascorbic acid (Sigma, St Louis, MO, USA), 40 μg/ml proline (Sigma), and 0.1 μM dexamethasone (Sigma). The medium was changed every 3–4 days. For induction of mineralization, the cultured pellets were treated with osteogenic medium consisting of high-glucose DMEM (Gibco), 10% FBS (Gibco), 1 × antibiotic-antimycotic (Gibco), 10 mM β-glycerolphosphate (Sigma), 0.1 μM dexamethasone (Sigma), and 50 μg/ml sodium ascorbate (Sigma) for 14 days with medium changed every 3–4 days. These 3D-cultured pellets were then harvested and fixed in 4% paraformaldehyde overnight, followed by tissue processing, paraffin embedding, and sectioning. Five μm-thick sections of the pellets were stained with 2% AR solution (Beyotime, Shanghai, China) at pH 4.2 for 1 minute for AR staining, or with 1% silver nitrate solution (Sigma) for 1 hour under light and counterstained with nuclear fast red for VK staining. The stained sections were dehydrated, mounted, and photographed with Nikon Ni-U Eclipse Upright Microscope (Nikon, Tokyo, Japan).

For external validation (EV) of the algorithm, a set of 50 histological images from previously published papers were selected for representative scenarios of calcification assays [1, 2, 4, 5, 19–36], which included but not limited to developmental of skeletal tissues in developmental biology study [1, 2], calcification in 3D cultured of immortalized mouse myoblast C2C12 cell pellets [19], and calcified blood vessels [4, 5].

Development of staining quantification (SQ) tool for image quantification

HOS transformation

The Higher Order Statistics (HOS) method conceptualized by Park, Jungwoo et al. [37], was primarily aimed at conducting image segmentation for image sequences with low depth-of-field (DOF). In statistics, the first two moments can derive the mean and variance which depict the average value of a distribution and how spread out the distribution is. The third and fourth moments indicate the asymmetry of a distribution and how heavy its tails or extreme values are [38]. The HOS method calculates the sample mean of the fourth-order central moment at pixel point (x, y) with a set of its neighboring pixels in a grayscale image. Using the concept of fourth moments, it is possible to transform the original pixel values to their higher order statistics so that a focused object could be segmented from defocused or noisy background. The HOS transformation could be written as:

$${\widehat{m}}^{\left(4\right)}\left(x,y\right)=\frac{1}{N_{\eta }}\sum _{(s,t)\in \eta (x,y)}{\left(I\right(s,t)-\widehat{m}(x,y\left)\right)}^4$$

where η(x, y) is a set of neighboring pixels centering at (x, y), $\widehat{m}(x,y)$ is the sample mean of I(s, t).

Since the range of HOS values could be extremely large, the value is down scaled to limit its range to [0, 255]. Therefore, the HOS value can be further written as:

$$HOS\left(x,y\right)=\mathrm{m}\mathrm{i}\mathrm{n}(255,{\widehat{m}}_R^{\left(4\right)}(x,y)/300)$$

where 300 is an appropriate down scaling factor [37].

Unlike the traditional unsupervised image segmentation algorithms, which usually find optimal thresholding value solely based on original pixel values and/or its variance, the HOS method exploits pixels with each pixel surrounded and calculates its kurtosis to identify focused objects by how heavy the kurtosis of a point with its surrounding points could be rather than the pixel value itself.

Local density filtering

After the HOS transformation, the original pixel values would be transformed to binary indicators in which focused area tend to have densely distributed points whereas in the defocused or noisy area, the points would be sparsely distributed. This gave a chance for density peaks clustering (DPC) to be able to filter most of the sparse points (i.e. noisy background) [39–41]. The main idea of DPC is to compute local density for each point and its distance to points of higher density. For each point p_i, i = 1,…,N, there are k Nearest Neighbor (KNN) points of p_i, which we denote it as q_j, j = 1,…k. Then the average distance of p_i to its nearest neighbors can be defined as:

$$\overline{d_i}=1/k\bullet {\sum }_{j=1}^k{d}_{ij}$$

where d_ij is the Euclidean distance between p_i and q_j. Thus, the local density (LD) can be defined as:

$$LD\left(p_i\right)=1/k\bullet \sum _{q_j\in KNN\left(p_i\right)}exp\left(\frac{-d_{ij}}{\stackrel{-}{d_i}}\right)$$

LD(p_i) indicates the magnitude of local density: the lower value of LD(p_i), the higher possibility that it should be filtered [42].

Denoising

With the application of local density filter, ROI would be most likely to be outlined in a rough manner that some very little noise might still leave there. To eliminate those minor objects (noise), we ordered connected objects in sequence according to their size of area, and then eliminate objects with least size of area. The proportion of objects to be deleted was based on a threshold set up by users (normally it is set to be 95%, alike to the confidence interval for a level of significant statistical testing).

Positive staining identification

Positive staining region was identified by setting up an appropriate threshold within ROI. Specifically, the positive staining thresholding value for AR was set at 100 (in the range of 0–255) and the positive staining thresholding value for VK was set at 140 (in the range of 0–255).

Staining quantification with ImageJ

The histological stained images were also processed and quantified manually with ImageJ (NIH, USA) by two observers independently (XL and YTC). After importing a microscopic photo into ImageJ, the total area of the section was selected either by the “Wand Tool” or by hand drawing if the Wand Tool failed to correctly identify the section. The total area of ROI was then quantified by the “Measure” function of ImageJ and the output was the pixel values. After converting the microscopic photo to 8-bit grayscale image and inverting the image intensity, the positively stained area and intensity were quantified using the “Threshold” function. The positive staining threshold values for Alizarin Red staining was set as 100 (out of 0–255) and the positive staining threshold values for von Kossa staining was set as 140 (out of 0–255). The positive staining intensity percentage was then calculated with the same formula as SQ tool [43]:

$$\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{y}\mathrm{\%}=\frac{\sum \mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{y}\mathrm{o}\mathrm{f}\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{l}\mathrm{y}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{d}\mathrm{r}\mathrm{e}\mathrm{g}\mathrm{i}\mathrm{o}\mathrm{n}}{\sum \mathrm{m}\mathrm{a}\mathrm{x}\mathrm{i}\mathrm{m}\mathrm{u}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{y}\mathrm{o}\mathrm{f}\mathrm{R}\mathrm{O}\mathrm{I}}\times 100\%$$

$$=\frac{\sum \mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{y}\mathrm{o}\mathrm{f}\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{l}\mathrm{y}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{d}\mathrm{r}\mathrm{e}\mathrm{g}\mathrm{i}\mathrm{o}\mathrm{n}}{\mathrm{a}\mathrm{r}\mathrm{e}\mathrm{a}\mathrm{o}\mathrm{f}\mathrm{R}\mathrm{O}\mathrm{I}\times 255}\times 100\%$$

Metrics evaluation and statistical analysis

The main quantitative results from the software included (1) total area, (2) total intensity (in ROI), (3) stained area, and (4) stained intensity (the staining signal within ROI). The comparison was conducted between SQ tool and ImageJ. The accuracy of SQ tool was evaluated by comparing the above four quantitative measurements of SQ tool with ImageJ. AR, VK, and EV images were evaluated separately with a batch of 50 samples each.

Setting ImageJ as a benchmark, the metrics used for evaluating the accuracy were:

• Pearson’s correlation coefficient:

$$r=\frac{N\sum xy-\sum x\sum y}{\sqrt{\left[N\sum x^2-{\left(\sum x\right)}^2\right]\left[N\sum y^2-{\left(\sum y\right)}^2\right]}}$$

where N = 50;

$x=asetofquantitativeresultsmeasuredbySQtool;$

$y=asetofquantitativeresultsmeasuredbyImageJ$.

• Difference in percentage:

$$dif{f}_{pct}=\frac{x-y}{y}\times 100\%$$

• Difference of proportion in percentage:

$$prop.dif{f}_{pct}=\frac{x_{prop}-y_{prop}}{y_{prop}}\times 100\%$$

where $x_{prop}=\frac{x_{stained}}{x_{total}};y_{prop}=\frac{y_{stained}}{y_{total}}$

The 95% confidence interval of the average value for difference in percentage and difference of proportion in percentage was calculated for each batch, and t test was conducted at a significance level of 0.05. The null hypothesis was: the absolute value of true mean is greater than 0.05.

• Success rate:

A difference in percentage and a difference of proportion in percentage less than 0.05 will be considered as a success image recognition, and the success rate will be calculated for each batch accordingly.

$$successrate=\frac{N\left(dif{f}_{pct}\right(orprop.dif{f}_{pct})<0.05)}{N}$$

Results

Overall workflow of SQ tool

The methodology of SQ tool includes three main phases: Higher Order Statistics (HOS) transformation, local density filtering, and denoising. Fig 2 shows the complete workflow of image processing and quantification, including six steps: 1) grayscale; 2) HOS transformation; 3) local density filtering 4) dilate 5) fill in solid 6) denoising. The details of the main methodologies are as below.

Fig 2. Workflow of total tissue and stained area segmentation.

Input image is a histological staining image of Alizarin Red Stating of stem cell pellet underwent osteogenic differentiation. The study subject in this image is stained in light yellow; the positive staining area is in orange red, and the cloudy light pink zones are noise signals (bar = 200 μm). The automatic staining quantification (SQ) algorithm includes six steps successively, in which three major steps (HOS transform, local density filter, denoising) were conducted to check with different types of noises.

Comparison of HOS transformation with conventional methods

Fig 3 shows the results from three scenarios recognized by HOS method compared with two traditional methods (Otsu’s method and Triclass thresholding method) [14, 15]. The binary indicators in Fig 3 indicates the primary interested area with white (labeled as 1) and not-interested area with black (labeled as 0). HOS method has superior performance than the above-mentioned methods in identifying the tissue area in all three scenarios. In Fig 3 scenario 1, in the case that positive signal is weak, the blurry dirt is all over the background and ROI and its intensity is stronger than ROI, Otsu’s method fails to exclude the blurry dirt (pixel value is 572,790) and Triclass thresholding fails to include most part of ROI (pixel value is 61,719). On the other hand, HOS method identifies the ROI without blurry dirt (pixel value is 627,958). In Fig 3 scenario 2, in the case that there is no positive signal and the blurry dirt is connected or overlaid with ROI, Otsu’s method is unable to distinguish between the connected blurry dirt and ROI (pixel value is 1,603,327) and Triclass thresholding fails to include most part of ROI (pixel value is 270,623). On the other hand, HOS method separates the connected blurry dirt from ROI and have the ROI identified correctly (pixel value is 1,035,680). In Fig 3 scenario 3, in the case that the positive signal is strong and massive compared to the rest part of ROI, although Triclass thresholding was reported to be superior if one or more strong signals are exhibited in the image [15], in current practice Otsu’s method and Triclass thresholding fails to identify most part of the ROI (pixel value is 847,351 for Otsu’s method, 939,787 for Triclass method). In contrast, HOS method correctly identifies ROI with the weak intensified area included (pixel value is 1,663,425). More details of ROI are reserved by HOS method because it transformed pixel values to the sample mean of the fourth-order central moment at pixel point (x, y) with a set of its neighboring pixels.

Fig 3. Comparison of HOS transformation with Otsu’s method and Triclass thresholding.

A comparison of HOS transformation with two traditional segmentation methods. HOS transformation outperformed the other two methods in three different scenarios which stand for most representative problems in histological images of Alizarin Red Stating from stem cell pellet underwent osteogenic differentiation, bar = 200 μm. Pixel values of the identified region were displayed in each graph. Scenario 1 shows that when the image contains dirty background and tiny positive stained signal, in the meantime, the dirty background might have stronger intensity than the ROI, the former two methods were unable to clear dirty background or outline the ROI exclusively. Scenario 2 shows that when there is no positive stained signal and the dirty background was completely blended with the ROI, the overlapping area cannot be distinguished or the method failed to outline the ROI. SQ algorithm successfully segmented the blending dirt and gave clear ROI outline. Scenario 3 shows that when the image contains massive positive stained signal, the former two methods would usually miss some part of ROI (e.g., some weak signals) since the color contrast is too strong.

Data readouts from SQ tool in recognizing of Alizarin Red staining and von Kossa staining

With above method established, the main outputs of SQ tool at subject recognition step included: two black and white pictures indicating tissue area (ROI) and staining area recognized (as the representative images in Fig 4), and the descriptive statistics such as intensity, area, and percentage of the staining area.

Fig 4. Reprehensive images ROI and positive stain recognition of SQ tool of histological calcification images.

(A) Reprehensive images of stem cell pellets underwent osteogenic differentiation with Alizarin Red, and (B) Reprehensive images of stem cell pellets Von Kossa staining (bar = 200 μm). Original images can be found in S2A Fig (n = 50 for AR) and S2B Fig (n = 50 for VK). (C-E) External Validation calcification-stained images: to validate the ROI auto-recognition function of SQ algorithm, we collected histological images from published papers about mineralization of chondrogenic pellets (C), brachiocephalic artery study, bone biology study, bone developmental biology study (D), and vascular calcification study (E). These images are all stained with von kossa to identify tissue calcification, and counterstained with hematoxylin erosion (H&E), or von Kossa/McNeal’s tetrachrome staining. Original images of EV images (n = 50) are listed in S2C Fig, with permissions of reprint.

To validate the SQ tool, we examined 50 images of AR staining (Fig 4A, S1A Fig), and another 50 images of VK staining (Fig 4B, S1B Fig) of the stem cell pellet cultures generated in our lab. Besides in-house data, the accuracy of readouts of SQ tool was further verified with external published images represent the most common calcification scenarios in biomedical research. A total of 50 random images including Alizarin Red and von Kossa staining were collected from published papers [1, 2, 4, 5, 19–36] (Fig 4C–4E, and S1C Fig).

Comparison of the readouts from SQ tool with ImageJ

We then compared the results obtained from SQ tool and ImageJ, and discussed for the metrics of Pearson’s correlation coefficient, difference in percentage between SQ and ImageJ, difference of proportion in percentage between SQ and ImageJ for the batch of AR, VK, and EV set, respectively (Figs 5 and 6). Table 1 shows the complete comparison of SQ and ImageJ in multi-levels. The difference in percentage and difference of proportion in percentage was measured by its sample mean, 95% confidence interval, p value of t test with null hypothesis of true mean being greater than 0.05, and success rate.

Fig 5. Pearson’s coefficient between SQ and ImageJ.

Pearson’s coefficient between SQ (x axis) and ImageJ (y axis) measured for the metrics of total area, total intensity, stained area, and stained intensity in the three batches of 50 samples for Alizarin Red, Von Kossa, and external validation, respectively. The blue line is the diagonal line with slope = 1. Pearson’s coefficient is shown in the lower right of each graph.

Fig 6. Difference of proportion between SQ and ImageJ.

Difference of proportions between SQ and ImageJ measured for the metrics of stained area out of total area, and stained intensity out of total intensity in the three batches of 50 samples for Alizarin Red, Von Kossa, and external validation, respectively were shown in each graph. The red line is the average of difference of proportion in percentage and their values were shown correspondingly.

Table 1. Evaluation metrics of SQ tool compared with ImageJ in multi-levels.

Metrics—level1	Metrics—level2	batch	value	95% CI (%)	p value
Pearson’s coefficient	total area	Alizarin Red	0.9993	-	<0.0001
		Von Kossa	0.9997	-	<0.0001
		External Validation	0.9990	-	<0.0001
	total intensity	Alizarin Red	1.0000	-	<0.0001
		Von Kossa	0.9999	-	<0.0001
		External Validation	0.9992	-	<0.0001
	stained area	Alizarin Red	1.0000	-	<0.0001
		Von Kossa	0.9999	-	<0.0001
		External Validation	0.9973	-	<0.0001
	stained intensity	Alizarin Red	1.0000	-	<0.0001
		Von Kossa	0.9999	-	<0.0001
		External Validation	0.9958	-	<0.0001
Difference in percentage (%)	total area	Alizarin Red	2.48	(1.60, 3.36)	<0.0001
		Von Kossa	1.47	(1.02, 1.92)	<0.0001
		External Validation	2.67	(1.16, 4.18)	0.0020
	total intensity	Alizarin Red	0.38	(0.03, 0.72)	<0.0001
		Von Kossa	0.53	(0.36, 0.69)	<0.0001
		External Validation	-0.04	(-1.46, 1.37)	<0.0001
	stained area	Alizarin Red	0.21	(-0.62, 1.05)	<0.0001
		Von Kossa	0.01	(-0.63, 0.65)	<0.0001
		External Validation	-1.68	(-3.23, -0.14)	<0.0001
	stained intensity	Alizarin Red	0.37	(-0.52, 1.26)	<0.0001
		Von Kossa	0.13	(-0.48, 0.74)	<0.0001
		External Validation	-1.84	(-4.16, 0.48)	0.0052
Proportional difference in percentage (%)	stained area out of total area	Alizarin Red	-1.53	(-2.71, -0.34)	<0.0001
		Von Kossa	-1.13	(-1.76, -0.49)	<0.0001
		External Validation	-3.94	(-5.74, -2.14)	<0.0001
	stained intensity out of total intensity	Alizarin Red	-0.04	(-1.07, 0.99)	<0.0001
		Von Kossa	-0.46	(-1.09, 0.18)	<0.0001
		External Validation	-1.81	(-3.96, 0.34)	<0.0001

Pearson’s correlation coefficient

As shown in Table 1 and Fig 5, the Pearson’s coefficient between data obtained from SQ and ImageJ for total area, total intensity, stained area, and stained intensity in the three batches (AR, VK, and EV) were all high. Fig 5 shows that most points were lying on the diagonal line or around it and Table 1 shows that the 12 measurements were all near to 1. These results indicate that the quantitative results of SQ and ImageJ are highly correlated with almost a perfect positive relationship. The Pearson’s coefficients provided a preliminary analysis and comparison of the four measurements between SQ and ImageJ and showed that the results obtained by SQ tool were consistent with results obtained by ImageJ.

Difference in percentage between SQ and ImageJ

Table 1 shows minute difference in percentage between SQ and ImageJ for total area, total intensity, stained area, and stained intensity in the three batches (AR, VK, and EV). The absolute value of average sample mean of the 12 quantitative measurements were all less than 0.05 (ave. Mean <0.05) and their corresponding p value were all significant (p<0.05). This illustrates that the null hypothesis of true mean being greater than 0.05 can be rejected. Among the 12 measurements, total intensity in the EV batch, stained area in the AR and VK batch, stained intensity in the AR, VK and EV batch have 95% CI for average sample mean lying through zero, which means that those measurements have 95% confidence to conclude that SQ had no difference from ImageJ in terms of above 6 measurements. S2 Fig shows the scatterplots of the four quantitative measurements in the three batches. Most points were randomly distributed around zero, with a few outliers.

Setting the difference less than 0.05 as a standard for success, the success rate was also shown in Table 1. SQ tool could automatically recognize and evaluate more than 90% of the AR and VK images and more than 80% of the EV images and the results were as good as obtained manually from Image J.

Difference of proportion in percentage between SQ and ImageJ

Since the proportion of stained area (stained area/total area) and the proportion of stained intensity (stained intensity/total intensity) are two mostly useful readouts in data quantification, the difference of proportion was calculated in addition to the directly readouts difference. Table 1 shows the difference of proportion in percentage. Similarly, the absolute value of sample mean of the 6 quantitative measurements were all less than 0.05 (ave. Mean <0.05) and their corresponding p value was <0.0001. We can conclude that there is more than 99.9% confidence that the true mean of difference of proportion in percentage between SQ and ImageJ were less than 0.05 (ave. Mean). The 95% CIs included zero for difference of stained intensity in all three batches (AR, VK, and EV).

Fig 6 also shows that most points of the difference of proportion were randomly distributed around zero. In terms of the success rate of data processing, SQ tool could automatically evaluate the stained area and staining intensity in more than 80% (Table 1) of the images (S1A–S1C Fig), and the results were as good as obtained manually from Image J.

The histogram in Fig 7 shows the difference of proportion for the measurements of area (Fig 7A1–7C1) and intensity (Fig 7A2–7C2). The highest frequency values were around zero, with a few observations being very low or high, which suggests manual check is needed for these outliers.

Fig 7. Histogram of proportional difference in percentage.

The batch of 50 samples for Alizarin Red(A), Von Kossa(B), and external validation set(C) were arranged vertically to show the histogram of proportional difference in percentage between SQ and ImageJ for the stained area out of total area(1), and the stained intensity out of total intensity(2).

Discussion

We developed an unsupervised image recognition algorithm incorporated in the SQ Tool that could process images stained in different batches with diverse noisy background, and could standardize the staining intensity and provide reliable quantification results of the common calcification staining images. The SQ tool is highly automatic with the ability of high-throughput batch processing and parameter-tuning. We firstly collected histological images of AR and VK, which were generated from our laboratory to evaluate the degrees of stem cell osteogenesis with a three-dimensional in vitro tissue culture model [18, 44], and developed the algorithm by R 4.0.2 (Fig 1). The image was taken the step of grayscale at first, and then the background noise was optimized by the steps of HOS transformation, local density filtering, dilate, fill and denoising. We then validated the application of this algorithm in other scenarios with histological images of skeletal tissue development, dental slides, and calcified arteries [1, 2, 4, 5, 19].

To date, the widely used quantification tool for histological staining images was ImageJ. Some of recent research have developed analogous image analysis tools for quantification, but they either acquire basic quantitative measurements from ImageJ to achieve derivative functions [10], or the tools were designed to quantify the input batch images without noisy stains [9]. We then compared the readouts from SQ tool to the readouts obtained from ImageJ in several metrics including results of area and intensity in both stained region and total ROI region. The Pearson’s coefficients were all above 0.99, and the hypothesis tests of difference in percentage (<0.05) were all showed significance. The quantitative evaluations demonstrate that SQ tool can produce results as good as ImageJ. Thus, we concluded that SQ tool is smarter and quicker in quantification of calcification staining signals compared to ImageJ. Given the accuracy as we tested, SQ tool can get similar results as ImageJ and solve the specific quantification problem of histological staining images on tissue or cell with complicated backgrounds.

Compared to ImageJ, the SQ tool is highly automatic and high-throughput, thus saves time and labor cost, and has advantages in the following aspects: (i) image preprocessing is time-consuming and could add uncontrolled artificial factors. SQ tool does not require images to be preprocessed to remain only interested regions. Images can be diagnosed directly from experimental outputs. (ii) A large amount of images can be diagnosed simultaneously, so that they would be ruled under same thresholding criteria. This will enhance the reliability to the quantification and avoid the observer bias during manual thresholding. (iii) The software is semi-parametric, which means that parameters such as binary cut-off value for HOS transformation, degree of dilate, percent cut-off value for denoising are enabled to make adjustment. (iv) The most important feature of SQ tool is to automatically recognize noisy stains blended with ROI. It is common to have debris and other noisy signals in the histological staining images obtained from wet lab, and occasionally some stains would be blended with the background and get out of the interested region (e.g., Fig 3, Scenario 1; Fig 4A top left). Sometimes the unwanted stains (e.g., red and pink stains generated by Eosin) have the same or stronger color and intensity as those calcification staining signals, and they are difficult to distinguish by human’s eyes. The essential difference between noisy stains and the interested stains in ROI is that the noisy stains are defocused during the experimental process of dyeing. Hence, in this study, the concept of HOS transformation is to treat those noisy stains as defocused regions and such that disturbing stains would be transformed to very dispersive points compared to the focused stains in ROI (Fig 3). With that being processed, computers can outline ROI in a manner of separation between dispersive points and centralized points.

SQ tool is a lightweight and functional tool with easy application in biomedical research, compared to the supervised machine learning methods such as neural network (NN)-based algorithms, whose labeling and model training can take an enormous amount of time and computation resources. NN-based algorithms would be superior in recognizing certain objects (e.g. cars, animals, human beings) given the pattern of those objects and it requires a vast amount of images with similar kind of pattern to train the model. On the other hand, the SQ tool created a label-free segmentation that requires light computational resource with fast processing speed.

Despite that the tool has achieved promising results with very high level of accuracy, there are several critical challenges and flaws need to be considered for further development. For instance, the default parameters can be adjusted for a batch of images, but prior check is required for each batch and the parameters would still need to be tuned by observer. Future revision could be to add machine learning algorithm to fine-tune the parameters instead of offering a list of default parameters static for all images.

In conclusion, we developed a lightweight, fast diagnostic image recognition tool specifically aimed at quantification of staining images. The use of it could be extended to other staining image recognitions not limited to AR and VK staining.

Supporting information

S1 Fig. Original images of histological calcification assays.

Three batches of samples for (A) Alizarin Red (from 50 random samples), (B) Von Kossa staining (from 50 random samples) of stem cell cultured pellets. Samples of stem cell pellet underwent osteogenic differentiation with Alizarin Red and Von Kossa staining positive signals exhibited in an ascending order according to the percentage of positive staining area (bar = 200 μm). (C) External Validation calcification-stained images (from 22 published papers. Images with permission of reuse were listed, images with no permission of reused regranted were included in the reference [20, 21, 27, 32–34]). To validate the ROI auto-recognition function of SQ algorithm, we collected histological results images and from published papers about mineralization of chondrogenic pellets, brachiocephalic artery study, bone biology study, bone developmental biology study, and vascular calcification study with permissions of reprint. These images are all stained with von kossa to identify the tissue calcification, and counterstained with hematoxylin erosion (H&E), or von Kossa/McNeal’s tetrachrome staining.

(PDF)

S2 Fig. Difference in percentage between SQ and ImageJ.

Difference in percentage between SQ and ImageJ measured for the metrics of total area, total intensity, stained area, and stained intensity in the batch of 50 samples for Alizarin Red, Von Kossa, and external validation, respectively. The red line is the average of difference in percentage and the value is shown in each graph.

(PDF)

A list of abbreviations

AR

Alizarin Red

VK

Von Kossa

EV

External Validation

SQ

Staining Quantification

ROI

Region of Interest

HOS

Higher Order Statistics

NN

Neural Network

DPC

Density Peaks Clustering

Data Availability

All relevant image data are within the paper and its Supporting Information files. The installation package of the software and its source code are available at the GitHub repository: https://github.com/heaena/Staining-Quantification.

Funding Statement

This work was supported by: (i) the National Key R&D Program (Project Number 2019YFA0111900 to YJ), administered by the Ministry of Science and Technology of the People’s Republic of China (MOST, China), (ii) this work described in this paper was partially supported by a grant from the NSFC/RGC Joint Research Scheme sponsored by the Research Grants Council of the Hong Kong Special Administrative Region, China and the National Natural Science Foundation of China (Project No. N_CUHK483/22), (iii) Health@InnoHK, Innovation and Technology Commission, the Government of the Hong Kong Special Administrative Region of the People’s Republic of China (CNRM, to YJ). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.