Detection of the optimal neuron traces in confocal microscopy images

Abstract

Quantitative methods of analysis of neural circuits rely on large datasets of neurons reconstructed accurately in three dimensions (3D). Due to the complexity of neuronal arbors, large datasets of reconstructed neurons must be generated with automated algorithms. Here, we attempted to automate the process of neuron tracing from sparsely labeled 3D stacks of confocal microscopy images. Our algorithm involves two steps. In the first step, the segmented image of neurites in the stack is voxel-coded. Centers of intensity of consecutively coded wave fronts are connected into a branched structure, which represents a coarse trace of the neurites. In the second step, this trace is optimized with the modified active contour method, which tends to maximize the intensity along the trace while keeping it under tension. To assess the performance of the algorithm we used manual reconstructions of neurons and converted them into artificial stacks of intensity images. These images were traced using the developed algorithm and quantitatively compared to the corresponding manual traces. The optimal traces were on average 6.0% shorter than the manual traces. This reduction in length resulted from the smoothness of the optimal traces, which, in comparison to the manual ones, were built out of shorter segments, and, as a result, were 3.3% less tortuous. The average distance between the optimal and the manual traces was 0.14 μm, and the average distance between their corresponding branch-points was 2.2 μm, illustrating good agreement between the traces.

Introduction

It is virtually impossible to understand how the brain functions without detailed knowledge of neural circuits. Axons and dendrites of neurons can be labeled with a multitude of cell labeling techniques and visualized in three-dimensions (3D) with confocal scanning microscopy (Conchello and Lichtman, 2005). Such images contain valuable information about the connectivity in the brain, despite the facts that only small subsets of neurons are labeled in typical experiments to allow for quantitative analysis, and that synaptic connectivity cannot be resolved reliably on the level of light microscopy. The first restriction can be partially alleviated by new circuit labeling techniques such as multicolor Brainbow (Livet et al., 2007; Lichtman et al., 2008), which allows for labeling of a much larger subsets of neurons through the introduction of different colors. Transsynaptic tracing (Wickersham et al., 2007), which identifies the pre-synaptic partners of a given cell, or array tomography (Micheva and Smith, 2007) can alleviate the second restriction on the visualization of neural circuits.

In spite of the above limitations, quantitative methods of analysis of axonal and dendritic arbors of sparsely labeled neurons have already resulted in numerous findings about synaptic connectivity [see e.g. (Elston and Rosa, 1997; Kozloski et al., 2001; Duan et al., 2002; Stettler et al., 2002; Lubke et al., 2003; Binzegger et al., 2004; Stepanyants et al., 2004; Li et al., 2005; Shepherd et al., 2005; Markram, 2006; Jefferis et al., 2007; Lee and Stevens, 2007; Stepanyants et al., 2008)]. These methods rely on accurate reconstruction of labeled neurons, which currently can only be achieved with manual reconstruction software tools, such as the ones included in Neurolucida (MicroBrightField Inc.), NeuronJ (Meijering et al., 2004), NeuronStudio (CNIC, Mount Sinai School of Medicine), and FilamentTracer (Bitplane Inc.). However, the manual reconstruction is extremely time consuming and depends on the diligence and the subjective judgment of the user. There is an obvious need to automate the neuron reconstruction process.

Automated reconstruction of neurites from confocal microscopy stacks of images typically contains several of the following steps (Russ, 2007): image restoration (denoising and deconvolution), registration of images within individual image stacks, image segmentation, tracing, feature extraction (cell bodies, spines, boutons, synapses, branch-points), and registration of stacks of images. The purpose of this study is to develop and evaluate the performance of an automated algorithm for neurite tracing. To focus on this aim, we assumed that all the remaining reconstruction steps have been already accomplished or will be performed after the tracing is completed. In particular, we assumed that neurites had been successfully segmented from the background, i.e. they contain no loops or broken segments.

There are generally several classes of algorithms, which can be used to obtain the traces of segmented liner branched structures in 3D. One class of algorithms is based on image thinning methods, which are used for extracting skeletons from binary images [see e.g. (Lee et al., 1994; Palagyi and Kuba, 1998)]. The basic idea behind these methods is an iterative removal of voxels from the surface of the segmented image in a way that preserves the topology of the contained structure. When applied to noisy images of neurites, such algorithms produce voxellated skeletons, which do not follow precisely the intensity in the image and often contain erroneous short branches.

Multi-scale 3D line enhancement filters (Sato et al., 1998; Streekstra and van Pelt, 2002), that are based on the eigenvalues of the Hessian matrix, can facilitate centerline visualization and detection. These filters can extract centerline voxels in non-branched linear structures. Once the centerline is detected, fitting models can be used to determine the most optimal trace of the linear structure. One of the most frequently used models is the active contour model introduced by Kass (Kass et al., 1988; Frangi et al., 1999; Schmitt et al., 2004) that fits the shape of a curve to the skeleton. In most of these models, the user sets the branch-points and they remain fixed during the fitting process.

Another approach is vectorial tracking (Can et al., 1999; Al-Kofahi et al., 2002; Srinivasan et al., 2007; Wang et al., 2007). This class of algorithms exploits local image properties to recursively trace linear structures. The algorithm starts from a seed point, which is typically provided by the user, and tracks the structure of interest by exploring the nearby voxels. The algorithm can be combined with a dynamic real-time updating of the trace direction by the user (Meijering et al., 2004). Vectorial tracking is suitable for semi-automated image analysis of high contrast images. However, this method is not appropriate for precise fully automated tracing of branched structures because the branch- and end-points of the structure have to be specified by the user.

Currently, there are no fully automated tools, which would reliably trace complex neuronal morphology, e.g. axonal arbors of cortical pyramidal neurons. Most methods require from the user to perform pattern recognition tasks, such as identification of branch- and end-points. In this study, we described an automated algorithm for determining the optimal traces of already segmented neurite images. This computerized method of neuron tracing requires no user involvement. It consists of two steps, voxel-coding and optimal tracing, which are described in the first part of this paper. In the second part, we compared the results obtained with the automated tracing procedure to the manual neuron reconstructions.

Materials and Methods

Manual neuron traces

In this study we used dendritic arbors of 20 pyramidal cells reconstructed from layer 3 of monkey superior temporal cortex and prefrontal area 46 (Duan et al., 2002; Duan et al., 2003). These neurons were retrogradely labeled with Lucifer Yellow and reconstructed in 3D with custom-designed morphometry software NeuroZoom (Nimchinsky et al., 1996; Nimchinsky et al., 2000). Neuron reconstructions were obtained from http://NeuroMorpho.Org (Ascoli, 2006) in the SWC format (Cannon et al., 1998).

Converting neuron traces into image stacks

For every manually reconstructed cell, we generated a 3D stack of intensity images by using the following procedure. We assumed that the underlying neurite fluorescence, F_N, is distributed uniformly throughout the arbor of the labeled neuron. Since in the present study we are only interested in finding the optimal traces of already segmented images, background level of fluorescence, F_B , was assumed to be zero. To simulate the light scattering in the tissue and in the microscope we linearly blurred the fluorescence of the labeled tissue by convolving it with a point spread function (PSF). For convenience, the PSF was chosen to be Gaussian with equal standard deviations in all three dimensions, σ_x,y,z = 1μm. As a result of this procedure, we obtained the expected fluxes of photons arriving at the detector from individual locations in the tissue. These fluxes, integrated at the detector over time, result in the expected counts of photons arriving from individual voxels in the image stack, μ(x,y,z). For simplicity, voxel size was chosen to be s = 1μm in all three dimensions. The actual photon counts, n(x,y,z), were randomly chosen from the Poisson distributions, P(n | x,y,z), with the corresponding expected values, μ(x,y,z). This procedure can be summarized as:

$$\begin{array}{cc}\hfill & PSF(x,y,z)=\frac{1}{{\left(2\pi \right)}^{3∕2}{\sigma }_x{\sigma }_y{\sigma }_z}\exp \left[-\frac{x^2}{2{\sigma }_x^2}-\frac{y^2}{2{\sigma }_y^2}-\frac{z^2}{2{\sigma }_z^2}\right]\hfill \\ \hfill & \mu (x,y,z)=s^3\int F(x^{\prime },y^{\prime },z^{\prime })PSF(x-x^{\prime },y-y^{\prime },z-z^{\prime })d{x}^{\prime }d{y}^{\prime }d{z}^{\prime }\hfill \\ \hfill & P(n\mid x,y,z)=\frac{\exp [-\mu (x,y,z)]\mu {(x,y,z)}^n}{n!}.\hfill \end{array}$$ (1)

In this expression, F(x,y,z) is the underlying fluorescence of the tissue. It assumes the values of F_N for points belonging to the interior of the neurites and F_B for the exterior points.

To create an artificial image stack from a manually traced neuron we first voxellated the trace by assigning values of F_N = 20 μm⁻³ to all the voxels overlapping with the trace and F_B = 0 μm⁻³ to the rest of the voxels. This procedure yielded the function F(x,y,z), which was next convolved (second expression in Eq. 1) with the PSF to produce the expected photon counts in the image stack, μ(x,y,z). Next, the actual photon counts, n(x,y,z), were randomly generated for each voxel from the corresponding Poisson distributions (last expression in Eq. 1). In order to avoid loops, voxels with zero actual photon counts in the interior of neurites were assigned the photon counts of 1. Finally, the intensity I(x,y,z) in the image stack was obtained by scaling down the actual photon counts to the [0 1] interval. The value of F_N = 20 μm⁻³ was chosen because it produced image stacks, where the intensity along the neurites was similar to or more variable than that in typical confocal microscopy images of neurites (Figure 1). Our method is not sensitive to particular values of this parameter.

Figure 1.

Artificially created image stacks. A. Maximum intensity projection of an image stack, which was artificially created from a manually traced pyramidal cell dendrite. B. yz and C. xy maximum intensity projections of the selected region from A. The manual trace is shown in green. Scale bars are 50 μm in A and 10 μm in B and C.

The optimal trace

We begin the description of the method for finding the optimal traces of linear branched structures in confocal microscopy images by first looking at a simplified example. Consider a 3D continuous intensity image, I (r⃗), of a non-branched line with known start- and end-points, r⃗_i and r⃗_f. One can employ active contour models (Kass et al., 1988) to obtain the optimal trace of this image. To this end, a fitness function has to be constructed which favors smooth traces connecting points r⃗_i and r⃗_f in a way that maximizes the intensity along the trace.

We used a parametric representation of the trace line, r⃗ (t), t ∊ [0,1], which originates at the point r⃗ (0) = r⃗_i and terminates at the point r⃗ (1) = r⃗_f. On the one hand, the optimal trace has to pass through the high intensity places in the image and on the other hand, the trace has to be kept under tension to prevent it from zigzagging from one high intensity voxel to another. The trade-off between these two requirements can be captured by the maximization of the fitness functions containing some of the following terms (Kass et al., 1988; Frangi et al., 1999; Schmitt et al., 2004),

$$F=a{\int }_0^{1}I\left(\overrightarrow{r}\left(t\right)\right)dt+b{\int }_0^{1}I\left(\overrightarrow{r}\left(t\right)\right)\mid \frac{d\overrightarrow{r}\left(t\right)}{dt}\mid dt-c{\int }_0^1\mid \frac{d\overrightarrow{r}\left(t\right)}{dt}\mid dt-d{\int }_0^1{\mid \frac{d\overrightarrow{r}\left(t\right)}{dt}\mid }^{2}dt.$$ (2)

Other terms in the fitness functions had also been considered in the past (Kass et al., 1988; Frangi et al., 1999).

The first two terms in this fitness function benefit from routing the trace through the high intensity regions. The first term gives the average intensity along the trace, while the second corresponds to the integrated intensity along the trace. The third and fourth terms were included to keep the trace under tension. The third term corresponds to the length of the trace, and the fourth term can be interpreted as the elastic energy due to stretching of the trace. Adjustable coefficients a,b,c,d > 0 capture the relative contributions of the four terms to the overall fitness, F. It was empirically determined that it is sufficient to include any two terms, where one is related to intensity and the other is related to tension. Here, in order to simplify the description of the methods, we considered the fitness function, which consists of the first and the last terms of Eq. 2. All the results presented in this study were obtained by maximizing this fitness function.

Since confocal microscopy stacks of images are voxellated and not continuous, it is more convenient to rewrite the fitness function in a discrete notation. To do this, we approximated the trace with a broken line consisting of N −1 straight segments connecting N vertices, and replaced the integration with summation,

$$F\left(\left\{{\overrightarrow{r}}_k\right\}\right)=\frac{1}{\lambda }\sum _{k=1}^{N}I\left({\overrightarrow{r}}_k\right)-\alpha \lambda \sum _{k=1}^{N-1}{\mid {\overrightarrow{r}}_{k+1}-{\overrightarrow{r}}_k\mid }^2.$$ (3)

Coefficients a and d of Eq. 2 were replaced with 1/λ and αλ , where λ denotes the average density of segments along the trace (number of segments per unit length of the trace), and the dimensionless parameter α > 0 controls the tension in the trace. The fitness function in Eq. 3 scales linearly with the overall length of the trace. The parameter λ in this equation was introduced to make the two terms of the fitness function independent on the segment density, which is an adjustable parameter.

To make use of the above expression, we need to be able to estimate the intensity in the image stack at the positions specified by vertices, r⃗_k (the first term in the Eq. 3). To this end, intensity in the image stack is interpolated onto every position r⃗_k by using a Gaussian interpolation procedure (Stepanyants et al., 2008). This results in,

$$F\left(\left\{{\overrightarrow{r}}_k\right\}\right)=\frac{1}{\lambda }\sum _{k=1}^N\sum _{i}I\left({\overrightarrow{R}}_i\right)s^3\frac{e^{-{({\overrightarrow{r}}_k-{\overrightarrow{R}}_i)}^2∕2{\sigma }^2}}{{\left(2\pi \right)}^{3∕2}{\sigma }^3}-\alpha \lambda \sum _{k=1}^{N-1}{\mid {\overrightarrow{r}}_{k+1}-{\overrightarrow{r}}_k\mid }^2.$$ (4)

Parameter s in this equation denotes the voxel size and vector R⃗_i specifies the position of the center of the voxel i. Summation over index i in the first term of Eq. 4 runs over all the voxels in the image stack. In practice, due to a fast decay of the Gaussian function (σ ≈ 1μm ), this sum can be restricted to a small number of voxels in the vicinity of the point r⃗_k.

If the layout of the initial trace is sufficiently close to that of the optimal solution, the optimization problem of Eq. 4 can be solved with an iterative procedure referred to as the gradient ascent algorithm (Boyd and Vandenberghe, 2004):

$$\begin{array}{c}\hfill {\overrightarrow{r}}_k^{n+1}={\overrightarrow{r}}_k^n+\beta {\phantom{\mid }\frac{\partial F\left(\left\{{\overrightarrow{r}}_k\right\}\right)}{\partial {\overrightarrow{r}}_k}\mid }_{\left\{{\overrightarrow{r}}_k\right\}=\left\{{\overrightarrow{r}}_k^n\right\}}=\hfill \\ \hfill ={\overrightarrow{r}}_k^n+\beta \left[-\frac{1}{\lambda }\sum _i({\overrightarrow{r}}_k^n-{\overrightarrow{R}}_i)I\left({\overrightarrow{R}}_i\right)s^3\frac{e^{-{({\overrightarrow{r}}_k^n-{\overrightarrow{R}}_i)}^2∕2{\sigma }^2}}{{\left(2\pi \right)}^{3∕2}{\sigma }^5}-2\alpha \lambda (2{\overrightarrow{r}}_k^n-{\overrightarrow{r}}_{k+1}^n-{\overrightarrow{r}}_{k-1}^n)\right]\hfill \end{array}$$ (5)

In this expression, ${\overrightarrow{r}}_k^n$ denotes the position of the vertex k of the trace at an iteration step n of the gradient ascent algorithm. As the start- and end-points of the trace are fixed, index k in Eq. 5 runs from 2 to N −1. Parameter β > 0 controls the size of the iteration step.

The gradient ascent procedure in Eq. 5 is easily generalized on the case of a branched structure,

$${\overrightarrow{r}}_k^{n+1}={\overrightarrow{r}}_k^n+\beta \left[-\frac{1}{\lambda }\sum _i({\overrightarrow{r}}_k^n-{\overrightarrow{R}}_i)I\left({\overrightarrow{R}}_i\right)s^3\frac{e^{-{({\overrightarrow{r}}_k^n-{\overrightarrow{R}}_i)}^2∕2{\sigma }^2}}{{\left(2\pi \right)}^{3∕2}{\sigma }^5}-2\alpha \lambda \sum _l({\overrightarrow{r}}_k^n-{\overrightarrow{r}}_{k_l}^n)\right].$$ (6)

Here, index k_l enumerates all the segments which are connected to the vertex k. If vertex k is an intermediate-point, the second sum in Eq. 6 contains only two terms, and Eq. 6 reduces to Eq. 5. For a bifurcation point, the second sum will contain three terms, for a trifurcation – four, etc. No calculations are needed for the start- and end-points of the branched structure, as these points must remain stationary.

For small values of parameter α in Eq. 6, the trace becomes very stretchy and the optimal solution can double on itself at high intensity regions in the image stack. This can happen if the fitness function, Eq. 4, benefits more from threading the trace through the same high intensity region several times than it is penalized for the increase in the length of the trace. To prevent such solutions of the optimization problem, the trace must be sufficiently stiff. Analysis of Eq. 4 resulted in the following lower bound for the parameter α,

$${\alpha }_{\min }=\underset{\overrightarrow{R}}{\max }\left[\sum _{i}I\left({\overrightarrow{R}}_i\right)s^3\frac{e^{-{(\overrightarrow{R}-{\overrightarrow{R}}_i)}^2∕2{\sigma }^2}}{{\left(2\pi \right)}^{3∕2}{\sigma }^3}\right].$$ (7)

The maximum in this expression is taken over all the positions R⃗ on the segmented image. The value of α_min depends on the distribution of intensity in the image stack, the voxel size s , and the parameter σ. In practice, for images of neurites with intensity in the [0 1] range, the value of α_min is roughly 0.1−0.3.

Very large values of the parameter α may result in traces which are excessively straight and do not faithfully follow the intensity in the image in places where neurites make sharp turns. For some value of this parameter, α_max, the algorithm will trace the most tortuous parts of the arbor along a straight line, rather than following the image intensity. Hence, the numerical value of α_max is dependent on the type of the imaged neurites. For the dataset of images used in this study, it was empirically established that α_max = 1.5.

Since parameter β controls the step of the gradient ascent algorithm, increasing the value of this parameter would lead to a faster convergence of the trace to the optimal solution. However, large values of this parameter may result in unstable or diverging traces. Stability analysis of the above iterative procedure places an upper bound on the possible values of the step size at β_max = 1/(4αλ).

To further improve the stability of the trace, long segments were subdivided and short segments were combined at every iteration step of the gradient ascent algorithm. The following values of the parameters were used in this study: s = 1μm, σ = 1μm , λ = 1.0 μm⁻¹, α = 0.3, β = β_max/2 = 0.42 μm . These numerical parameters led to stable convergence of the trace to the optimal solution. The fitness function, Eq. 4, increased monotonically throughout the iterations and the procedure was deemed convergent to the optimal solution after 40 iteration steps.

Results

To evaluate the performance of the algorithm, we converted the dendritic arbors of 20 pyramidal cells, which were manually reconstructed from layer 3 of monkey superior temporal cortex and prefrontal area 46 (Duan et al., 2002; Duan et al., 2003), into artificial stacks of images (Figure 1). This was done by blurring the manual traces and introducing Poisson noise to the images in order to imitate low intensity imaging (see Materials and Methods section for the details). Voxel size in the artificial image stacks was chosen to be 1 μm³. Creating the artificial stacks of images from manually traced neurons, rather than using real confocal image stacks, was done for two reasons. First, we wanted to be able to compare the results of our algorithm to both the intensity images and the manual traces. Second, we wanted to avoid the segmentation problem, which is beyond the scope of this study. For the same reason, the background intensity in the artificial image stacks was set to zero, and we only considered cells which did not lead to the formation of loops in the process of conversion.

Below, we describe an automated tracing procedure, which should be performed following the segmentation of neurites. Namely, we assumed that voxels of interest had been successfully separated from their background, as shown in Figure 1A. The segmented voxels need to be further analyzed to extract the optimal 3D trace of the neurites. This was done by using the following two steps: voxel-coding and optimal tracing.

The voxel-coding and optimal tracing

A simple way to obtain the topology of a 3D segmented structure and to examine its connectedness is the voxel-coding algorithm (Zhou et al., 1998; Zhou and Toga, 1999; Shikata et al., 2004; Yamasaki et al., 2006). In this recursive algorithm, a label (a natural number) is associated with each voxel of the segmented image (Figure 2A). For each connected region in the image stack, the voxel-coding algorithm was initiated by selecting a seed voxel, which in principle can be randomly chosen. The seed voxel was assigned the label of 1. At the next step, the voxel-coding algorithm finds all the voxels that are neighboring to the seed voxel. The neighborhood of a voxel is specified by a 3D structural element, which in this study was the 26 nearest neighbors. This neighborhood was referred to as the wave front of the seed voxel, and label 2 was assigned to all of its voxels. At each step, a new wave front was created and consecutively labeled. This encoding procedure was repeated until all of the segmented voxels in the image were labeled.

Figure 2.

Illustration of the voxel-coding method. A. Maximum intensity projection of a small part of a dendritic arbor. A 3D wave is initiated at an arbitrary voxel in the image (yellow circle). At subsequent steps, the new wave fronts are determined based on the choice of the structural element (3×3×3 cube in the figure). The branch-point (green asterisk) is defined as the center of intensity of the wave front that splits in two at the subsequent step. The end-points (blue asterisks) correspond to the centers of intensity of the terminal wave fronts. Intermediate-points are the centers of intensity of the remaining wave fronts. B. Subsequent end-, branch-, and intermediate-points are connected with straight segments to form the wave trace. This trace (yellow line) is shown superimposed on the maximum intensity projected image from Figure 1B. Scale bars is 10 μm.

As the wave propagated through the branch-points, voxel-coding produced wave fronts consisting of two or more disconnected regions. For every disconnected wave front region the center of intensity (center of mass) was obtained. These points formed the wave trace (Figure 2B). The end-points in the wave trace corresponded to the centers of intensity of the terminal wave front regions. The branch-points were defined as the centers of intensity of the front regions, which divided at the subsequent steps. The remaining wave trace points were termed intermediate. The coordinates of the end-, branch-, and intermediate-points, and their topological structure was conveniently recorded in the SWC file format (Cannon et al., 1998). In this study, there was only a single segmented region in each image stack (the entire dendritic arbor), and the voxel-coding algorithm was initialized at the soma of each cell. In general, the described procedure must be applied to all the disconnected neurites present in the segmented image stack.

Because the voxel-coding procedure was performed on a binary segmented image, the resulting wave trace did not follow precisely the intensity in the image stack. The wave trace was not smooth and the branch-points were placed imprecisely (Figure 2B). Next, the optimal tracing procedure was applied to correct for these artifacts (see Materials and Methods). This procedure was based on the modified active contour method (Kass et al., 1988; Frangi et al., 1999; Schmitt et al., 2004), and it used the wave trace as a starting layout. The optimal tracing procedure iteratively moved the intermediate- and branch-points of the trace and converged towards the optimal structure within few tens of iterative steps, while keeping the end-points stationary.

Comparison of the optimal and the manual traces

First, we obtained the optimal traces for the 20 artificially created image stacks and visually examined how well these traces fit the intensity images (Figure 3). Figures 3B and 3C illustrate that the optimal trace provides a reasonably good 3D fit to the intensity image. Second, we quantitatively compared the optimal traces to the corresponding manual reconstructions. For this comparison, we selected five features of neuron morphology: the total arbor length, the tortuosity of neuronal branches, the number of end-points, the proximity of the arbors, and the placement of branch-points.

Figure 3.

The optimal trace. A. The optimal trace (red line) is shown superimposed onto the maximum intensity projected image from Figure 1. B. yz and C. xy maximum intensity projections of the selected region from A. Scale bars are 50 μm in A and 10 μm in B and C.

Figure 4A shows the comparison of the total arbor lengths between the manual and the optimal traces. The 20 data points represent individual analyzed cells. The length of the optimal trace was highly correlated (adjusted R² = 0.9995) with the length of the manual trace, and was on average 6.0±0.9% (mean ± SD) shorter. This difference resulted from the fact that manual neuron traces are composed of straight segments of variable lengths, which are typically several micrometers long. The optimal trace was formed by shorter segments (1 μm on average), and it provided a smoother trace of the intensity image, resulting in a shorter length.

Figure 4.

Comparison of the optimal and the manual neuron traces. A. Comparison of lengths of the optimal and the manual traces. Individual points represent different cells. B. Comparison of tortuosities of the optimal and the manual traces. Tortuosity index, T, is calculated for non-branching dendritic segments of different lengths as the ratio of the geometric distance between the segment end-points, R, and the segment length, L (inset). C. Comparison of the numbers of end-points in the optimal and the manual traces.

This point is reflected in the comparison of the optimal and the manual trace tortuosities (Stepanyants et al., 2004). Tortuosity index, T, of a dendritic branch segment is defined as the ratio of the segment length, L, to the geometric distance between its end-points, R (see the inset of Figure 4B). We calculated the tortuosities for non-branching segments of different lengths, for all the 20 traced neurons. Segments containing branch-points were excluded from this analysis, as their tortuosity indices are influenced by branching. The results are shown in Figure 4B where the tortuosity indices of the manual and the optimal traces are plotted for segments of different lengths. As was expected, the tortuosity index is an increasing function of the segment length (Stepanyants et al., 2004). The optimal trace tortuosity index was roughly 3.3% smaller than that for the manual trace, indicating a smoother fit to the intensity image.

The voxel-coding procedure is sensitive to the smoothness of the segmented image. Because this procedure is applied to binary images of neurites, relatively large (few voxels in size) irregularities on the surface of the neurites may result in creation of short erroneous terminal branches. Similarly, voxel-coding can omit some of the existing terminal branches that are short. In this study, these branches were only few voxels (or micrometers) in length. In Figure 4C we compared the numbers of end-points in the manual and the optimal traces. It can be seen that the numbers of end-points in the manual and the optimal traces are highly correlated (adjusted R² = 0.99), and that for many of the analyzed cells theses numbers are the same. However, for some cells the difference in the numbers of terminal branches was as high as 2. The average absolute difference, 0.85 terminal branches per cell, was small in comparison with the total number of terminal branches per cell, 42±10. As the erroneous terminal branches and the branches that were missed by the algorithm were only few micrometers in length, they have little impact on the overall cell morphology.

To further quantify the accuracy of the automated tracing procedure, we defined the measure of proximity between two traces. To this end, we first re-segmented both traces into short segments (shorter than 0.25 μm) and determined the centers of these segments. Second, for every segment center on the first trace we found the corresponding closest segment center on the second trace and determined the Euclidian distance between the two. Third, a weighted average of these distances was calculated by using the products of the corresponding segment lengths as weights. Fourth, the traces were switched around and the last two steps were repeated producing an alternative weighted average. Finally, the two weighted distances were averaged resulting in the distance measure that is symmetric with respect to both traces. According to this measure, the average distance between the segments in the manual and the wave traces was 0.31 μm, and 95% of the corresponding segment centers were separated with distances in the 0.005 μm − 2.03 μm range (95% confidence interval). In comparison, the average distance between the segments in the manual and the optimal traces was much lower, 0.14 μm (0.004 μm − 0.63 μm 95% confidence interval), emphasizing the importance of the optimal tracing procedure.

As the final comparison between the manual and the optimal traces, we examined the accuracy of branch-point placement. The average distance between the corresponding branch-points in the manual and the optimal traces was 2.2 ± 1.3 μm. Since the size of a typical bifurcation region in our images was more than 10 μm in diameter, the branch point placement was deemed sufficiently accurate. As was expected (Figure 2B), the average distance between the corresponding branch-points in the manual and the wave traces was much larger, 4.0 ± 2.0 μm, again illustrating the benefits of the optimal tracing procedure.

Discussion

Algorithms for automated neuron reconstruction are essential for realistic modeling of neural circuits. In recent years, there has been a dramatic increase in the number of studies, which rely on large sets of high quality neuron reconstructions [see (Stepanyants and Chklovskii, 2005; Ascoli, 2007) for review], and this trend will continue to grow with the advances in experimental techniques and computational capabilities.

In this paper, we described an algorithm that could be used as part of an automated procedure for reconstruction of neural circuits imaged with confocal microscopy. The algorithm was aimed at automating the process of detection of the optimal traces from already segmented neurite images. Segmenting neurites from their background can be a very difficult problem when the image quality is low or the density of labeled neurites is high. Poor labeling or image quality can lead to low contrast and beaded appearance of neurites. Segmentation of such images often results in broken structures that need to be reconnected afterward. Densely labeled neurites may result in the appearance of loops in the segmented image. Such loops will have to be subsequently resolved, which presents a substantial challenge.

As a starting point of our method, we created segmented stacks of images and focused on the automated and accurate neuron tracing. To this end, we blurred 20 manually traced dendritic arbors of pyramidal cells and transformed them into segmented 3D stacks of intensity images. The automated tracing algorithm presented in this study consists of two parts. First, we voxel-coded the segmented intensity image (Zhou et al., 1998; Zhou and Toga, 1999; Shikata et al., 2004; Yamasaki et al., 2006) and extracted the topology of the 3D neuronal structure. Second, to find the optimal trace of the segmented neurites, we smoothed and optimized this structure by using a modified active contour method (Kass et al., 1988; Frangi et al., 1999; Schmitt et al., 2004).

To test this algorithm, we obtained the optimal traces of the 20 artificially created image stacks and compared them to the initial, manual traces. The comparison showed no major differences between the traces, with the average distance between the manual and the optimal traces being only 0.14 μm. The optimal traces were on average 6.0% shorter than the manual ones. This was a consequence of the fact that in comparison to typical manual neuron traces, where the segment lengths often exceed 5−10 μm, the optimal traces in this study were made of shorter segments. As a result, they provided a smoother representation of the intensity image. This point is reflected in the fact that optimal traces were 3.3% less tortuous than the manual ones.

As mentioned in the results section, the voxel-coding algorithm, due to its binary nature, did not faithfully reproduce the number of terminal branches. We quantified the extent of this artifact by comparing the numbers of end-points in the tree structures of the manual and the optimal traces. The discrepancy was on average 0.85 terminal branches per cell. New erroneous branches or branches that were missed by the algorithm were only few micrometers in length and have very little impact on the morphological characteristics of the cells and on synaptic connectivity. Once the optimal trace of a neuron is detected, it becomes feasible to eliminate the erroneous branches and to find the missed branches by searching in the volume surrounding the optimal trace.

While detecting the optimal neuronal traces from the stacks of intensity images we made no attempt to render the shapes of individual neuronal branches in 3D (Schmitt et al., 2004). The reason for this is the fact that the thickness of axons and dendrites of most neural classes in the cerebral cortex is below the resolution of confocal microscopy. For example, the average (by length) radius of the dendritic branches of cortical pyramidal cells in the mouse neocortex is 0.45 μm, while the average radius of axonal branches is roughly 3 times smaller (Braitenberg and Schüz, 1998). Hence, the details of neuronal morphology related to the thickness of branches cannot be resolved reliably with confocal microscopy. These types of analyses are better addressed with electron (Fiala, 2005; Briggman and Denk, 2006) or high-resolution light microscopy (Micheva and Smith, 2007; Cai et al., 2008).

The ultimate goal of this study is to facilitate a complete reconstruction of sparsely labeled subsets of neural circuits on a large scale, e.g. on the scale of the entire mouse brain. Such reconstruction will require fast and accurate processing of large numbers of confocal image stacks. All calculations in this study were performed on a 3.6 GHz computer and required approximately 2 minutes of processing time per cell, which was divided evenly between the voxel-coding and the optimal tracing parts of the algorithm. In addition, comparison of the manual and the optimal traces showed that the algorithm is sufficiently accurate, which makes it potentially useful for the large-scale neural circuit reconstruction projects.