Approximating Cellular Densities from High-Resolution Neuroanatomical Imaging Data

Abstract

Characterizing the cellular architecture (cytoarchitecture) of tissues in the nervous system is critical for modeling disease progression, defining boundaries between brain regions, and informing models of neural information processing. Extracting this information from anatomical data requires the expertise of trained neuroanatomists, and is a challenging task for inexperienced analysts. To address this need, we present an unbiased, automated method to estimate cellular density of retinal and neocortical datasets. Our approach leverages the fact that within retinal and neurocortical datasets, cells are organized into “layers” of constant density to approximate cytoarchitecture with a small number of known basis elements. We introduce methods for patch extraction, cell detection, and sparse approximation of inhomogeneous Poisson processes to differentiate changes in cellular densities and detect layers. Our results demonstrate the feasibility of using automation to reveal the cytoarchitecture of large-scale biological samples.

I. INTRODUCTION

Mapping the cellular architecture, or cytoarchitecture, within the nervous system is the first step in identifying the origin of tissue samples, performing comparative neuroanatomy, and modeling neurological disease states. A key step in delineating neural cytoarchitecture and mesoscale brain mapping involves the identification of “layers”, or regions within the sample where cellular density is approximately homogeneous. The cerebral cortex is a key brain region in which six layers have been identified and thoroughly studied [1]. Identifying layers is also integral to studying retinitis pigmentosa (RP), a family of retinal degeneration diseases in which the width and density of retinal layers is indicative of the stage of the disease [2].

Characterizing cytoarchitectural boundaries presently relies on trained neuroanatomists to manually annotate photomicrographs [3]. As neuroscience datasets scale upwards, having a human visually define structures becomes inefficient, impractical, and risks skewed data. It is therefore imperative that automated quantification be integrated into experimental and diagnostic methods to ensure large datasets are analyzed in an unbiased manner.

In the following paper, we introduce a method for Approximating Cellular Densities (ArCaDe), which estimates the spatially-varying density of cells within high-resolution neuroanatomical images. To provide robust estimates of cell densities that also reveal layer transitions, we developed an approach for sparse approximation from count data that estimates the density function with a total-variation (TV) approach [4]. The main idea behind our method is to estimate a density function that confirms the cellular count data while also minimizing the total variation of the estimated density. We additionally provide supporting methods to perform extraction of image patches and cell detection that allow efficient estimation of layering structure.

We applied ArCaDe to synthetic data, as well as image datasets from visual and somatosensory cortex [5] and from retinal samples from a rd10 mouse model [2]. In each case, we demonstrated that ArCaDe is able to achieve high accuracy of cell counts and estimate the underlying density effectively. Our approach for estimating cellular density exploits known structure in the density function to achieve higher stability and robustness to noise.

By combining computer vision methods for cell detection [6] with state-of-the-art methods density estimation [7], we were able to provide an end-to-end tool for estimating cytoarchitecture from neuroanatomical data. Furthermore, our method can be applied to a variety of imaging datasets with minimal re-training. We show that quantitative estimates of neural densities can be obtained with high accuracy, leading to new models of brain organization and tools for comparing brains across disease conditions.

II. Cytoarchitecture Approximation

Our approach for Approximating Cellular Densities (ArCaDe) consists of three main steps (Fig. 1). The first step of the method extracts patches from the image that run perpendicular to the surface of the sample to reveal the layering structure within the tissue. The second step detects cells within the image and produces cell counts for each extracted patch. The last step of the method approximates the spatially-varying density of cells using a TV-minimization approach. As a result of using ArCaDe, we can produce a spatially varying map of the density and width within each layer of the tissue sample. We provide MATLAB code and a demo at github.com/nerdslab/arcade.

Fig. 1. Overview of the Approximating Cellular Densities (ArCaDe) method.

(A) A Nissl image of somatosensory cortex from the Allen Institute’s Reference Atlas [5] with exemplar patches overlaid. (B) A demonstration of the fitted polynomial around the membrane of the cortex in the presence of noise. (C) From top to bottom: A patch from the image to visualize the cellular distribution from the top (pia) to the bottom of the cortex (Layer 6). The red lines indicate the transitions between layers as identified by a trained neuroanatomist. Below, detected cell bodies in red overlaid on the same image patch. Finally, the estimated density function obtained with our TV-minimization approach.

A. Step 1: Patch Extraction

The first step of ArCaDe extracts small patches from the image data that run perpendicular to the surface of the sample (Fig. 1A). The cellular layers of both cortex and retina run parallel to the surface of the tissue, so when cutting through the sample perpendicular to the surface, we observe sharp transitions in cell density.

To extract patches, we begin by identifying the surface of the sample by thresholding the image data and performing a connected components analysis to produce a binary image with the tissue sample labeled as ones and the rest of the image labeled as zeros. Next, we compute the 2D gradient of the binary image to estimate the membrane (surface) outlining the tissue sample. We fit a polynomial function of even order to account for noise around the surface (Fig. 1B). The directions of the tangent and the normal are computed for each smoothed boundary pixel. Using a designated patch width and a patch length, we can extract the patch producing a matrix of pixel values for the region of interest orthogonal to the surface of the structure.

B. Step 2: Cell Detection Method

The second step of ArCaDe converts image data into cell counts. We developed an approach for cell detection from 2D histology images by adapting a cell detection method developed for 3D neuroanatomy data [6].

1). Computing Class Probabilities:

To convert image data into cell counts, we start by producing probability maps of encoded pixels of either cells (ones) or background (zeros) in either a supervised or an unsupervised approach. For our supervised approach, we train a random forest classifier to distinguish cells from their background using an interactive segmentation tool called ilastik [8], a general-purpose framework that requires the user to input a small amount of training data to identify cells and background in the image. For our unsupervised approach, we use a Gaussian Mixture Model (GMM) to estimate the probability that a pixel in the image corresponds to the foreground (e.g. stained cells) or background [9]. The probability maps that result from either our supervised or unsupervised strategy are then input into our cell detection method to generate accurate cell counts.

2). Iterative Cell Detection:

After obtaining probability maps from the image data, we then apply a greedy sparse recovery approach to iteratively select circular objects (cell bodies) of a fixed size from the probability map. The algorithm begins by thresholding the probability map and then convolving the thresholded probabilities with a circular template to find the point in the map with maximum correlation. After finding the point maximally correlated with the template, a small window is removed around the matching pixel to zero out the probability map at this point in space. The method iteratively finds more cells until a stopping criterion (i.e., when the correlation is less than a threshold) is satisfied. The output of the cell detection method contains the center of all detected cells within the image (Fig. 2). This cell detection method provides an automated approach to convert a histological image (e.g., Nissl stained brain slice) into a collection of cell counts to be incorporated into our density estimation method.

Fig. 2. Cell detection results on cortical and retinal datasets.

(A) A Nissl-stained visual cortex image from the Allen Institute’s Reference Atlas [5], computed probability maps, and detected cells overlaid on the probability map (red circles) at 1.6x zoom (top row) and 12x zoom (bottom row). (B) Cell detection results overlaid on a retinal sample in a rd10 mouse (each detected cell is displayed as a different color).

3). Model Selection:

To apply our cell detection method to large images, we created a method which divides the image data into smaller blocks, runs the cell detection in parallel, and then merges the estimates. Additionally, we implemented a hyper-parameter optimization method that uses ground truth data to tune the hyper-parameters used in the cell detection step. These parameters include: the threshold applied to the probability map, size of template, size of circular window used when removing a cell from the probability map, and the stopping criterion.

C. Step 3: Poisson Density Estimation

After obtaining cell counts from each image patch, the next step is to obtain a density function that approximates the count data. We leverage the fact that the layering structure of neural tissue can be modeled by a piecewise-constant density function. Thus we can use sparse approximation techniques in conjunction with maximum likelihood estimation (MLE) to find efficient representations of cellular densities.

1). Density Estimation for Inhomogeneous Poisson Processes:

To model the cell density in a patch of tissue as a function of the depth, we start by binning the space into M disjoint depth intervals and model the number of cells in each bin as a Poisson random variable, where z_m ~ Poisson(R_m) denotes the number of cells observed in the m^th bin and R_m denotes the density (rate) of the m^th bin. We focus on a linear model where the density can be expressed as a linear combination of elements from a known basis A as $R_m^{\left(x\right)}={\left[Ax\right]}_m$, where A is a M × N matrix containing N basis elements each of M dimensions and $x\in {ℝ}^N$. Under this model, the negative log-likelihood of observing z is given by

$$\mathcal{L}(x\mid z)=1^{T}Ax-z^T\text{log}\left(Ax\right),$$

where z = [z₁, …, z_M] contains the number of cells in each bin and the logarithm is applied element-wise to Ax.

2). Sparse Maximum Likelihood Estimation:

Estimating the density function $R_m^{\left(x\right)}$ typically involves solving a MLE problem to find the parameters x that best match the counts z. However, when dealing with noisy biological data, applying additional regularization to our estimation problem helps to mitigate overfitting. In this case, rather than using a standard MLE approach, we leverage additional structure in our cell density estimation problem.

In particular, we can model the distribution of cells across cortical and retinal samples as a piecewise-constant function, where each layer is assumed to have nearly constant density [1]. In most cases, we have strong prior information about the specific number of layers and that the number of layers K is small relative to the number of bins M used to discretize the sample, i.e., K ≪ M.

Using this information, we can thus pose our density estimation problem as the following convex optimization problem:

$$\underset{x}{\text{arg min}}\mathcal{L}\left(x\mid z\right)+\lambda \Vert x{\Vert }_{TV},$$ (1)

where ‖x‖_TV = Σ_m |x_m+1 − x_m| is referred to as the “total variation” norm [4] and the regularization parameter λ is set to achieve a certain sparsity level based upon the number of layers that we expect in a sample. In our specific application, we will choose A to be a diagonal matrix; however, our approach can be used for other choices of bases without modification. In essence, the TV-norm penalizes the number of transitions in the density estimate and thus favors density functions that are piecewise-constant.

3). Algorithm:

An algorithm for solving (1) is presented in Algorithm 1. This algorithm uses a majorization-minimization strategy and employs the majorizer

$$\Vert x{\Vert }_{TV}\le \frac{1}{2}{\Vert {\Lambda }_{\left(Dx\prime \right)}^{1/2}Dx\Vert }_2^2+\frac{1}{2}\text{Tr}\left({\Lambda }_{\left(Dx\prime \right)}^{-1}\right)$$

with [Dx]_m = [x]_m+1 − [x]_m and diagonal matrix [Λ_(y)] with diagonal entries equal to [Λ_(y)]_mm = |y_m|⁻¹. Using this majorizer allows us to solve our optimization problem with smooth gradients, sidestepping the non-differentiability of the ℓ₁-norm. The parameter ϵ is chosen to be small (but non-zero) to condition the majorization, preventing singularities but otherwise having a negligible effect on the solution. The scale factor α is automatically kept in a moderate range to ensure that the iterations are feasible and the Hessian is nonsingular. Since A and D are banded matrices, the Hessian (H) is also banded and the linear system (H + αI)⁻¹f in Algorithm 1 can be solved efficiently.

4). Model Selection:

In order to find an appropriate value of the regularization parameter λ, we implemented an iterative method which solves (1) using Algorithm 1 for a fixed value of λ, computes the sparsity (number of non-zeros) of the solution vector x, increases λ, and uses the previous solution as a warm start for the next iteration. This repeats until an appropriate amount of regularization is applied to achieve a fixed sparsity level. This iterative approach allows us to achieve a sparsity level which can be set to identify a specific number of layers in the sample.

III. RESULTS

To demonstrate the efficacy of our method, we started by testing our cell detection and patch extraction methods on patches from two biological datasets. The first dataset was obtained from a Nissl-stained visual cortical section from the Allen Institute’s Reference Atlas (Fig. 2A) and the second sample was obtained from a retinal tissue section stained with toluidine blue (Fig. 2B); both staining methods highlight the layer-specific regions within each sample. This retinal dataset was obtained from the rd10 strain of mice, an accepted animal model of RP (retinal degeneration) [2]. Our evaluations were selected to test the utility of our method on diverse image datasets.

To test our cell detection methods, we obtained ground truth estimates of all cell positions within a segment of an image obtained from the Allen Institute’s Reference Atlas using ITK-Snap [10]. We used this information to perform a hyper-parameter optimization, resulting in a f1-score (harmonic mean of precision and recall) of 0.9031 for the supervised method (random forest classifier) and 0.9251 for the unsupervised method (GMM). To test our cell detection method on the retinal sample, we trained a classifier to recognize cells of two different sizes and obtained f1-scores of 0.9787 for cells in the first two layers (ganglion cells, inner nuclear layer) and 0.8861 for cells in the third layer (outer nuclear layer) (Fig. 2B). Our results suggest that ArCaDe can be used to reliably detect cells from both cortical and retinal datasets.

Next, we sought to demonstrate that our method can reliably estimate cellular densities and reveal layered structure within each dataset. To achieve this, we created two synthetic datasets that realistically modeled the biological layering of the width and densities from primary somatosensory cortex (S1) and primary visual cortex (V1) [5]. We applied our density estimation strategy to counts generated from these density functions (Fig. 3A). Results from the synthetic data suggest that TV-minimization provides a good approximation to the underlying density function and estimates of transitions between layers.

Fig. 3. Evaluation of cytoarchitecture estimation approach.

(A) Synthetic examples of density estimation in noisy data modeled from visual cortex (left) and somatosensory cortex (right). From top to bottom, average density estimate and sparse layer transitions as measured by the total-variation norm (50 trials). (B) Results of density estimation for a patch of cortical data (left) and retinal data (right). From top to bottom: an extracted image patch with annotated layer transitions in red, density estimates, and sparse layer transitions as measured by the TV-norm.

After confirming that our density estimation method can be successfully applied to synthetic data, we applied our method to image patches from S1, V1, and retinal samples (Fig. 3B). We compared these estimates against a more standard method for density estimation, a kernel density approach which smooths the count data. Upon comparing our results to that of annotations from a trained neuroanatomist, we observed that our method was able to accurately capture the layering structure from a noisier patch of cortical data and a patch from the retina. Our results demonstrate that ArCaDe can automatically distinguish layers from diverse neural datasets.