Algorithms for differentiating between images of heterogeneous tissue across fluorescence microscopes

Abstract

Fluorescence microscopy can be used to acquire real-time images of tissue morphology and with appropriate algorithms can rapidly quantify features associated with disease. The objective of this study was to assess the ability of various segmentation algorithms to isolate fluorescent positive features (FPFs) in heterogeneous images and identify an approach that can be used across multiple fluorescence microscopes with minimal tuning between systems. Specifically, we show a variety of image segmentation algorithms applied to images of stained tumor and muscle tissue acquired with 3 different fluorescence microscopes. Results indicate that a technique called maximally stable extremal regions followed by thresholding (MSER + Binary) yielded the greatest contrast in FPF density between tumor and muscle images across multiple microscopy systems.

1. Introduction

Histopathology is the clinical standard for cancer diagnosis; however, it requires tissue processing, laboratory personnel and infrastructure, and a highly trained pathologist to diagnose the tissue [1, 2]. Thus, it is resource-intensive to diagnose tissue at the point of care using histopathology. Microscopy is a technique that can enable the intraoperative visualization of tissue without the need for tissue processing. In particular, many groups have combined fluorescence microscopy with vital fluorescent stains to visualize tissue morphology. For example, several groups have combined fluorescence microscopy with fluorescent stains such as 4',6-diamidino-2-phenylindole (DAPI) in order to visualize microstructures in breast [3], cervical [4], and prostate carcinomas [4]. Other groups have used acriflavine [5, 6] and acridine orange [7–9] in order to visualize nuclear features in oral and oropharyngeal [5], skin [7], ovarian [9], and esophageal cancers [6].

While these different approaches can provide real time visualization of tissue morphology, automated interpretation of images is needed to enable quantitative diagnoses at the point of care. In order to enable automated interpretation of tissue images, various segmentation methods have been used to isolate and quantify nuclear features in fluorescence microscopy images of breast [3], cervical [4], prostate [4], and oral [5] tissues. Specifically, several groups have leveraged global thresholding to delineate cell nuclei from surrounding tissue [3, 5, 6]. While global thresholding is a relatively simple algorithm and therefore is easy to use, it requires uniform background intensity, which is often not present in images of heterogeneous tissue. Other groups have employed more complex techniques, such as adaptive window thresholding or local maxima detection [10–12], high pass filtering [13], Otsu’s method [14], or the circle transform [15] to segment nuclei in various tissue samples. The advantages and disadvantages of these various approaches have been laid out in previous work published by our group [16]. Briefly, while adaptive threshold methods are simple, they require varying the window size across an image and adjusting the threshold within each window, which can quickly become unwieldy [10–12]. High pass filtering is simple and easy to tune; however it is sensitive to noise within an image [13]. Otsu’s method is also relatively simple to tune, but often unable to distinguish overlapping or touching features [14]. Lastly, the circle transform is useful for delineating overlapping features or nuclei within an image, but is sensitive to small variations in background intensity [15].

Previously our group has combined topical fluorescent stains called acriflavine and acridine orange (AO) with fluorescence microscopy to enable real time visualization of tissue morphology in a preclinical genetically engineered mouse sarcoma model [16, 17]. Specifically, our group used a high resolution fluorescence microendoscope (HRME) in combination with acriflavine, which reversibly binds to nucleic acids, such as DNA and RNA, to acquire real-time images of tissue morphology in resected sarcoma tumor margins [16]. In a separate study, our group used a structured illumination microscopy (SIM) system in combination with acridine orange, which also reversibly binds to nucleic acids, to acquire images of tissue morphology in the same sarcoma model over a larger field of view but with comparable resolution to the HRME [17]. We have previously investigated two different segmentation methods – sparse component analysis followed by the circle transform (SCA + CT) [16] and maximally stable extremal regions (MSER) [18]. Both methods require little human intervention or supervision and were able to segment fluorescent positive features (FPFs) in heterogeneous images of sarcoma tumor margins. While fluorescent positive features (FPFs) may correspond to nuclei, in some cases RNA and DNA are concentrated within the nucleoli of neoplastic cells and single-stranded RNA is present in the cytoplasm; therefore, we refer to these fluorescent positive features as FPFs throughout this work.

The objective of this study was to assess the ability of various segmentation algorithms to isolate FPFs in heterogeneous images and identify an approach that can be used across multiple fluorescence microscopes with minimal tuning between systems. Specifically, we compared high pass filtering (HPF), global thresholding (GT), Otsu’s method (Otsu), SCA followed by thresholding (SCA + Binary), SCA + CT, MSER followed by thresholding (MSER + Binary), and MSER followed by the circle transform (MSER + CT). Algorithms were compared by first applying each approach to simulated images with no background and with a heterogeneous background. In order to assess the diagnostic potential of each algorithm, we also applied each algorithm to fluorescent images of acridine orange stained sarcoma and muscle tissue acquired with three microscopes, including a confocal microscope, the HRME, and SIM systems.

2. Methods and materials

2.1 Image segmentation algorithms

The seven different segmentation algorithms chosen for this study were high pass filtering (HPF), global thresholding, Otsu’s method (Otsu), sparse component analysis followed by thresholding (SCA + Binary), sparse component analysis followed by the circle transform (SCA + CT), maximally stable extremal region followed by thresholding (MSER + Binary), and maximally stable extremal region followed by the circle transform (MSER + CT). A summary of the different algorithms is presented in Table 1. All image processing and analysis was carried out using MATLAB (2013b, Mathworks INC., Natick, MA). HPF, global thresholding and Otsu were chosen because they are simple image segmentation techniques previously implemented by various groups to isolate FPFs [3, 5, 6, 13, 14]. HPF was implemented through convolution of a Gaussian filter with a standard deviation of 20 pixels with each normalized image. Then the resulting low pass information was subtracted from the image yielding the high pass filtered image. The standard deviation was empirically chosen such that a majority of the FPFs were isolated with HPF. Global thresholding and Otsu’s method were both implemented using the MATLAB command im2bw. For global thresholding, the level or threshold was empirically chosen to be 0.2 for a normalized image that varied from 0 to 1. In particular 0.2 was chosen because it was very close to the automatically select threshold level selected by Otsu’s method. For Otsu’s method, the level of threshold was chosen using the MATLAB command graythresh, which chooses the threshold that minimizes the intraclass variance of the white and black pixels.

Table 1. Summary of image segmentation algorithms.

Segmentation algorithm	Description
Simple algorithms
High pass filtering (HPF)	Isolates high frequency content within an image; can be sensitive to noise
Global thresholding	Isolates pixels within an image that have higher intensity than a manually selected threshold
Otsu’s method (Otsu)	Isolates pixels within an image that have higher intensity than an automatically selected threshold, which is chosen by minimizing the intraclass variance of the white and black pixels
Complex algorithms
Sparse component analysis followed by thresholding (SCA + Binary)	Separates different morphological structures into mathematically discrete components, then a threshold is applied to create a binary image from which features can be quantified
Sparse component analysis followed by the circle transform (SCA + CT)	Separates different morphological structures into mathematically discrete components, then the circle transform is applied to quantify features
Maximally stable extremal regions followed by thresholding (MSER + Binary)	Automatically selects local thresholds for each component within a given size range, then a threshold is applied to create a binary image from which features can be quantified
Maximally stable extremal regions followed by the circle transform (MSER + CT)	Automatically selects local thresholds for each component within a given size range, then the circle transform is applied to quantify features

The SCA methodology has been fully described elsewhere [16, 19]. Briefly, SCA is a computational technique that separates different morphological structures into mathematically discrete components. In particular, SCA was used here to isolate FPFs from the background of each image. In order to apply SCA to images, three regularization parameters τ_nuclei τ_muscle, τ_adipose must be selected by the user (see [16] for detailed notation). The process to select the regularization parameters has previously been described in detail [16]. The parameters τ_nuclei = 0.10, τ_muscle = 1.0 and τ_adipose = 1.0 gave strong performance across images and microscopy systems; this set was applied to all images in this study. The output that contains the FPFs from SCA can then be thresholded, which results in SCA + Binary, or the circle transform can be applied, resulting in SCA + CT. For the circle transform, the minimum and maximum size of the circular objects was set to a pixel value that corresponded to a 2 - 15 µm diameter, which covers the range of FPF sizes present in the simulated and tissue images.

Lastly, maximally stable extremal regions (MSER) was also used to segment FPFs from images. MSER has previously been used within the image processing community to reconstruct 3D scenes [20] and has been fully described elsewhere [21]. Specifically, MSER uses intensity thresholding in order to isolate components in an image. However, rather than using a global threshold, MSER automatically selects local thresholds for each component within a given size range. In this study, MSER has been adapted to isolate FPFs from each image. In order to apply MSER to images, five parameters had to be selected. The process for selecting these parameters has been described in detail [18]. Briefly, the five parameters include minimum area (MinArea), maximum area (MaxArea), maximum variation (MaxVariation), minimum diversity (MinDiversity), and Delta. The two simplest parameters to select are MinArea and MaxArea, which are related to the expected size of features. Specifically, in order to cover the range of FPF sizes present in the simulated and tissue images, MinArea was set to a pixel area that corresponds to a 2 µm diameter and MaxArea was set to a pixel area that corresponds to a 15 µm diameter. The three other parameters, MaxVariation, MinDiversity, and Delta are related to the intensity thresholds and were systematically tuned by applying a range of values to a set of 30 representative images [18]. MSER segmentation results were found to be insensitive to changes in MaxVariation and MinDiversity. However, segmentation results were sensitive to changes in Delta, and if Delta was set too low it could result in over-segmentation or if Delta was set too high it could result in under-segmentation (missing FPFs) [18]. Ulimately, the parameters MaxVariation = 10, MinDiversity = 0.5, and Delta = 10 resulted in accurate segmentation of FPFs across images and microscopy systems indicating that these parameters appear generalizable to all fluorescence images included in this study. The output that contains the FPFs from MSER can then be thresholded, which results in MSER + Binary, or the circle transform can be applied, resulting in MSER + CT.

2.2 Imaging systems

Three different microscopy systems were used in this study, including a confocal microscope, a high resolution microendoscope (HRME), and a structured illumination microscope (SIM). Table 2 contains a summary of the different characteristics of each of these systems. Briefly, an upright confocal microscope (Zeiss Axio Examiner with fixed stage microscope) was used as the gold standard system. We used a 405nm Diode laser and a 10x objective (Dry Zeiss Plan- Apochromat 1063-139 WD 2.0mm, NA = 0.45) to collect images. A pinhole of 2.05 airy unit was used to collect all images, which resulted in a calculated optical section thickness of 10.6 µm. A high resolution microendoscope (HRME), which has been previously described in [16], was also used to collect fluorescent images in this study. The system consisted of a 455 nm light emitting diode (Luxeon V Star, LXHL-LR5C), excitation filter (Semrock, FF01-452/45-25), dichroic mirror (Chroma 485 DCLP), emission filter (Semrock FF01-550/88-25), CCD Camera (Point Grey Research, GRAS-14S5), and a coherent fiber bundle (Sumitomo, IGN-08/30). The field of view (FOV) of the system was approximately 750 µm in diameter; however, images were cropped to remove the vignetting caused by the rim of the coherent fiber bundle, which resulted in a FOV of 445 x 526 µm. The resolution of the system was 4.4 µm. A low pass Gaussian filter was applied to each image to remove the high-frequency fiber bundle pattern before further analysis. The widefield structured illumination microscopy (SIM) system [17] consists of a broadband super continuum laser (Fianium SC400), an excitation filter centered at 480 nm, a 6x beam expander, a polarizing beam splitter, an LCoS SLM display chip (Holoeye LCR-720), a microscope objective (Nikon 4xE Plan Fluor, NA = 0.1), a dichroic beam splitter (505nm), an emission filter (peak 520nm), a 200 mm focal length tube lens (Nikon MXA20696), and a CCD camera (LaVision Imager 3 QE). Structured illumination theory has been previously described in detail elsewhere [17, 22]. Briefly, the sample is illuminated with a sinusoidal pattern of a specified frequency, and a series of three images are acquired in which the sinusoidal pattern is slightly shifted, such that the three images differ only in phase shift. A demodulation method is then applied in order to extract only the in-focus fluorescence information of each image. This procedure removes any fluorescence emitted from outside the focal plane, and the resulting sectioned image only contains information from the plane of focus. In previous work, our group illustrated that an absolute illumination pattern spatial frequency of 31.7 mm⁻¹ which corresponded to an optical section thickness of 120 µm was a good compromise between optical section thickness and SNR reduction [17]. Thus, a spatial frequency of 31.7 mm⁻¹ was used to collect all structured illumination images included in this work.

Table 2. Microscopy system characteristics.

	Confocal	HRME	SIM
Objective	10x	10x	4x
Resolution	1.0µm	4.4µm	3.2µm
Number of pixels	512 x 512	606 x 716	1040 x 1376
Field of view	848.5 x 848.5µm (0.72 mm2)	445 x 526µm (0.23 mm2)	1560 x 2064µm (3.21mm2)
µm/pixel	1.66	0.735	1.50
Optical section thickness estimate (µm)	10 µm	>200 µm	120 µm

2.3 Simulations

Simulated images were used to quantify the errors associated with each segmentation approach. The generation of simulated images has been described in detail elsewhere [16]. Briefly, all simulated FPFs were drawn the same way using MATLAB. Specifically, random locations were selected within a 1000 x 1000 pixel image. Then the fspecial command was used to create a disk with a specified diameter at each location. FPFs were simulated with varying diameters ranging from 4 up to 18 pixels (3 to 13 µm). The densities of FPFs were simulated from 128 to 1920 nuclei/mm². A different combination of FPF diameter and density was selected for each region of the simulated images. These input values represented the ground truth in our simulations. These ranges of diameters and densities were selected to fully cover the biologically observed range of FPFs observed in tumor and normal images [16]. Each FPF was then blurred with a Gaussian filter that had a standard deviation of 1.1 pixels. While FPFs appear mostly bright against the surrounding tissue, there is a small falloff in intensity around the edges that we observed in tissue images; thus, the blurring step simulated the small falloff in intensity observed experimentally. A heterogeneous background of muscle was simulated as longitudinal muscle fibers and the FPFs were added to the longitudinal muscle fibers through a weighted addition. Specifically, the intensity ratio between the FPFs and the underlying muscle was 1.4, which was based on observations from experimental images. In order to determine the accuracy of each algorithm, we calculated the percent error associated with density. Density rather than diameter was chosen to determine percent error because density has been shown to be the more important diagnostic parameter in a series of both preclinical and clinical studies completed by our group [16, 21, 23]. Specifically, we used the equation:

$$\begin{array}{l}{\%}_{error}=\frac{(True density- estimated density)}{True density}\\ \end{array}$$ (1)

where true density represents the values that were inputs in our simulations (i.e. the ground truth) and estimated density are the density values obtained after applying each image segmentation algorithm.

2.4 Tissue imaging and analysis

The tumors used in this study were generated from conditional p53 and either Kras^G12D or Braf^V600E using methods previously described [24, 25]. Tumors were excised as described by Mito et al [26]. Mice were euthanized immediately prior to surgical tumor resection. Within ten minutes of euthanasia, the tumor was excised from the leg. The excised tissue was flash frozen using liquid nitrogen. The tissue was then imbedded in optical cutting temperature compound (Tissue-Tek) and serially sectioned with a Leica cryostat, with alternating 50 µm and 5 µm sections being cut. The 5 µm sections were submitted for hematoxylin and eosin (H&E) staining and imaged with a standard transmission microscope. The 50 µm sections were mounted onto a glass slide and stained by topically applying 3-5 drops of acridine orange (AO). AO was selected as a contrast agent to stain morphological tissue features. AO is an intravital dye that stains RNA and DNA, skeletal muscle, and collagenous stroma [5, 27–29]. AO (0.01% w/v) was dissolved in phosphate buffer solution and topically applied to tissue prior to imaging. Each slide was then washed with 3-5 drops of PBS. Excess liquid was drained off of the slide. A total of 6 tumor and 6 muscle images were acquired with each microscope.

After segmenting FPFs from all tissue images using each algorithm, we determined the density of FPFs within each tumor and muscle image by calculating the number of FPFs per image and normalizing by the field of view, such that all density measurements are reported in FPFs/mm². For HPF, global thresholding, Otsu’s, SCA + Binary, and MSER + Binary, a connected components algorithm, in which connected pixels are assumed to belong to the same FPF, was applied to extract the number of FPFs from the segmented images. For SCA + CT and MSER + CT, the circle transform yields the estimated radii of the circles detected from which the number of circles (or FPFs) in each segmented image is determined. As a point of comparison, the number of nuclei in images of tumor and muscle H&E sections were counted through using the cell counter plugin in ImageJ (1.47v, National Institutes of Health, USA). Then values were normalized by the field of view, such that all measurements are reported as nuclei/mm². Lastly, Wilcoxon rank sums (non-parametric, two-tailed, alpha = 0.95) were used to determine whether density values were significantly different between tumor and muscle images for each microscope. A significance level of p<0.05 was considered to reject the null hypothesis for all analyses.

3. Results

3.1 Application of various image segmentation algorithms to simulated images

A simulated image with FPFs varying in size and density is shown in Fig. 1. Each segmentation algorithm was applied to this image and an overlay of the FPFs segmented by each algorithm can be seen in Fig. 1(A). A contour plot showing the percent error associated with calculating the FPF density with each algorithm can be seen in Fig. 1(B). As seen, in the absence of a background, all algorithms have relatively low error in segmenting FPFs, regardless of size or density. In a few instances (such as global thresholding, SCA + Binary, and MSER + Binary) errors are observed on the right hand side of the image when the larger FPFs are close together. In particular, for both SCA + Binary and MSER + Binary, large error is reported in the bottom right corner of the simulated image. This error is due to the fact that two FPFs in that region are close together and are incorrectly being counted as a single FPF. This results in a high error because the true density value (and therefore the denominator in our percent error equation) is small. For both SCA + CT and MSER + CT a few small errors are observed on the left hand side of the image when FPFs are very small. These errors are likely due to the CT being unable to detect features that are less than 5 pixels in diameter.

Fig. 1.

All algorithms achieve low errors when applied to a homogeneous simulation. An image simulation with various sizes and densities of FPFs was segmented with the following algorithms: HPF, Global Thresholding, Otsu, SCA + Binary, SCA + CT, MSER + Binary, and MSER + CT. The areas that were segmented with each method were false colored green and overlaid onto the original image. Overlays for each method can be seen in (A). The percent error associated with calculating the density of FPFs was determined for each method and is shown as a contour plot in (B). The colorbar indicates the percent error (%).

In order to simulate heterogeneous tissue, a striated background representing longitudinal muscle fibers was added to the simulated FPF image seen in Fig. 2. The output from each algorithm is shown as an overlay in Fig. 2(A). A contour plot showing the percent error associated with calculating the true density with each algorithm can be seen in Fig. 2(B). When applied to a heterogeneous image, the basic algorithms such as HPF, Global Thresholding, and Otsu’s method results in high percent errors. SCA + CT and MSER + CT segment larger FPFs with low percent error, but are unable to accurately segment smaller FPFs due to the limitation of the CT. Both SCA + Binary and MSER + Binary successfully segment all FPFs and achieve the lowest percent errors when calculating density in the image.

Fig. 2.

Complex algorithms achieve low errors when applied to a heterogeneous simulation. An image simulation with various sizes and densities of FPFs was added to a muscle simulation to create a heterogeneous image. The simulation was segmented with the following algorithms: HPF, Global Thresholding, Otsu, SCA + Binary, SCA + CT, MSER + Binary, and MSER + CT. The areas that were segmented with each method were false colored green and overlaid onto the original image. Overlays for each method can be seen in (A). The percent error associated with calculating the density of FPFs was determined for each method and is shown as a contour plot in (B). The colorbar indicates the percent error (%).

3.2 Evaluating the potential diagnostic performance of SCA + Binary, SCA + CT, MSER + Binary, and MSER + CT across multiple microscopy systems

In order to assess the diagnostic potential of SCA + Binary, SCA + CT, MSER + Binary, and MSER + CT across multiple microscopy systems, these four algorithms were applied to acridine orange stained tumor and muscle tissue images acquired with the confocal, HRME and SIM systems. Representative confocal, HRME, and SIM images of tumor and muscle tissue are shown in Fig. 3. The corresponding H&E sections are shown in Fig. 3 column 1. The output from the SCA + Binary, SCA + CT, MSER + Binary, and MSER + CT algorithms are shown as overlays in Fig. 3 columns 3-6 respectively. As seen, MSER + Binary leads to the largest observed difference between the density of FPFs in tumor images compared to muscle images for all microscopy systems. Both SCA + CT and MSER + CT underestimate the density of FPFs, particularly in confocal and SIM images. SCA + Binary incorrectly isolates large portions of the muscle confocal image, but does reasonably well at isolating features in the HRME and SIM images.

Fig. 3.

MSER + Binary leads to the largest observed difference between the density of FPFs in tumor images compared to muscle images Representative images of acridine orange stained tumor and muscle acquired with the confocal, HRME, and SIM systems are shown in A-C, respectively. Images of the corresponding H&E sections are shown in column 1. The original images of tumor and muscle acquired with the different fluorescent microscopes are shown in column 2. The outputs from SCA + Binary, SCA + CT, MSER + Binary, and MSER + CT were false colored green and overlaid onto the original images. Scale bar 200µm.

Lastly, we determined the average density of FPFs observed in tumor images and the average density of FPFs observed in muscle images acquired with the confocal, HRME, and SIM systems. As a point of reference, we also determined the average density of nuclei in H&E sections through manually counting nuclei in representative H&E images of tumor and muscle. The average tumor and muscle FPF density for SCA + Binary, SCA + CT, MSER + Binary, and MSER + CT are shown in Fig. 4. The p values associated with each algorithm and microscopy system are shown in Fig. 4(E). As seen, MSER + Binary is the only algorithm that yielded significant differences in FPF densities between tumor and muscle tissue for all microscopy systems.

Fig. 4.

Average FPF density for tumor and muscle images analyzed using SCA + Binary, SCA + CT, MSER + Binary, and MSER + CT. Density values obtained from manually counting nuclei in H&E images of tumor and muscle is shown in A. Densities seen in the confocal, HRME, and SIM systems are shown in B-D respectively. Asterisks indicate significance (*p<0.05). The p values associated with each algorithm and microscopy system are shown in E.

4. Discussion

In this study, we compared seven segmentation algorithms: HPF, global thresholding, Otsu, SCA + Binary, SCA + CT, MSER + Binary, and MSER + CT with the goal of determining whether one algorithm can segment a large fraction of FPFs from heterogeneous images and can operate effectively in high density FPF regions commonly seen in tumor images. Applying each algorithm to simulated images revealed that all algorithms performed comparably when segmenting FPFs from a homogeneous image that had no background (Fig. 1). However, the more basic algorithms, such as HPF, global thresholding, and Otsu had high errors associated with estimating the true density of FPFs in the heterogeneous image (Fig. 2). This highlights the need for more complex approaches for segmenting FPFs from heterogeneous images. We evaluated the potential diagnostic performance of the more complex algorithms, which included SCA + Binary, SCA + CT, MSER + Binary, and MSER + CT on tumor and muscle images from a genetically engineered mouse sarcoma model. While several algorithms led to significant differences in FPF density between tumor and muscle images, MSER + Binary is the only algorithm that yielded significant differences between tumor and muscle images with all three microscopy systems. Additionally, the density values achieved with MSER + Binary most closely match the density values observed in H&E images of tumor and muscle (Fig. 4). In particularly, the values achieved with MSER + Binary in confocal images (Fig. 4(B)) appear very similar to the H&E density values (Fig. 4(A)). The values achieved with MSER + Binary in the HRME (Fig. 4(C)) and SIM (Fig. 4(D)) appear to be a little higher than the H&E density values; however, this is expected because the three microscopes used in this work had different optical section thicknesses. Specifically, the confocal microscope had the thinnest optical section thickness (10 µm), which most closely matched the 5 µm thickness of the H&E sections.

A significant strength of MSER + Binary is that little to no tuning was required between microscopy systems. Specifically, the only parameters that changed between the microscopy systems used in this study were MinArea and MaxArea because each system had a slightly different pixel resolution. In the future, comparisons could be extended to include additional samples and include additional types of microscopy systems. In addition, the variation of FPF intensity could be varied within each simulation to further explore the diagnostic potential of each segmentation algorithm. Compared to human observers, the computer algorithm can better accomplish automated and quantitative interpretation of images, while different human observers might give different results. For example, in a previous study, we found that pathologists disagreed about the diagnosis of H&E sections of mouse sarcoma margins approximately 15% of the time [23]. The computer also has the ability to perform high throughput analysis. Additional studies will be needed in borderline cases where human pathologists may disagree on the diagnosis in order to compare the performance of algorithms with multiple human observers; however, these studies are outside the scope of the current manuscript.

This study lays the groundwork for how fluorescence microscopy and topical fluorescent stains (such as acridine orange) could be combined with segmentation algorithms (such as MSER + Binary) to enable rapid automated quantification of the size and density of image features. This combination of technologies could enable rapid visualization of tissue morphology and quantification of tissue features for automated diagnosis at the point of care. One possible application of this work is to use low cost fluorescence microscopes and segmentation algorithms to evaluate fresh tissues in environments with limited laboratory infrastructure and pathology, which could help guide treatment decisions [30]. Another potential application of this work is to leverage fluorescence microscopy and segmentation algorithms to enable the detection of residual disease in tumor margins during surgical resection [31, 32]. Based on the intended application, specific fluorescence microscopes may be selected. For example, the SIM system has a FOV that is over 13X larger than the FOV of the HRME, but both systems have comparable lateral resolution. Thus, if an application requires surveying a larger area of tissue, the SIM system may be the better microscope to employ. Additionally, the three microscopes used in this work had different optical section thicknesses and therefore yielded different contrast between FPF and the background. Thus, if an application requires a thinner optical section thickness due to the need for high achievable contrast, then the confocal or SIM system might be a better choice than the HRME. However, this work more broadly represents a process used to identify the best algorithm for various fluorescence microscopes. In particular, we explore a variety of image characteristics that can vary between microscopy systems, such as FOV, resolution, and optical section thickness.

In conclusion, MSER + Binary segmented FPFs from simulations with high accuracy and consistently yielded the greatest contrast in the density of FPFs calculated in tumor images compared to muscle images across multiple microscopy systems Thus, MSER + Binary is a robust approach for segmenting FPFs from heterogeneous images regardless of the microscopy system used and is an appropriate approach to enable automated interpretation of fluorescence microscopy images. This information can be leveraged to select the optimal combination of tools to enable automated segmentation and quantification of high resolution thick tissue images for a variety of applications.

Funding

Department of Defense (W81XWH-09-1-0410); National Institutes of Health (NIH) (1R01EB011574).