EXTRACTION AND ANALYSIS OF ACTIN NETWORKS BASED ON OPEN ACTIVE CONTOUR MODELS

Abstract

Network structures formed by actin filaments are present in many kinds of fluorescence microscopy images. In order to quantify the conformations and dynamics of such actin filaments, we propose a fully automated method to extract actin networks from images and analyze network topology. The method handles well intersecting filaments and, to some extent, overlapping filaments. First we automatically initialize a large number of Stretching Open Active Contours (SOACs) from ridge points detected by searching for plus-to-minus sign changes in the gradient map of the image. These initial SOACs then elongate simultaneously along the bright center-lines of filaments by minimizing an energy function. During their evolution, they may merge or stop growing, thus forming a network that represents the topology of the filament ensemble. We further detect junction points in the network and break the SOACs at junctions to obtain “SOAC segments”. These segments are then re-grouped using a graph-cut spectral clustering method to represent the configuration of actin filaments. The proposed approach is generally applicable to extracting intersecting curvilinear structures in noisy images. We demonstrate its potential using two kinds of data: (1) actin filaments imaged by Total Internal Reflection Fluorescence Microscopy (TIRFM) in vitro; (2) actin cytoskeleton networks in fission yeast imaged by spinning disk confocal microscopy.

1. INTRODUCTION

All eukaryotic cells have actin proteins. Actin proteins can self-assemble into long polymers to build networks and bundles of filaments that provide cellular mechanical integrity, generate forces for cell movement, and act as tracks for intracellular transport by motor proteins. Fluorescence microscopy is widely used to study the kinetics of single actin filament growth in vitro [1, 2, 3], the assembly mechanism of the contractile ring during cytokinesis [4], and the 3-D structure of microtubules [5]. Systematic analysis of the topology and properties of cytoskeletal structures in such images can provide significant insights into their conformations and dynamics. Fig. 1 demonstrates several kinds of microscopic images in which actin networks or intersecting actin filaments are present. Fluorescently-labeled actin filaments imaged by Total Internal Reflection Fluorescence Microscopy (TIRFM) grow parallel to a glass slide, and some long filaments intersect with each other in the last several frames of the TIRFM time-lapse sequence (Fig. 1(a)). In the radial projection of 3D confocal microscopy images, actin filament bundles manifest themselves as a meshwork during mitosis (Fig. 1(b)). Actin cables inside a cell imaged by spinning-disk confocal microscopy also form a cytoskeleton network (Fig. 1(c)). To analyze these network structures, an important step is to accurately extract them from the microscopy images.

Fig. 1.

(a) Intersecting filaments in a TIRFM experiment. (b) Actin meshwork in a 2D radial projection of a 3D confocal microscopy volume of a dividing cdc25–22 fission yeast cell labeled by GFP-CHD. Vertical axis is arc length [4]. (c) One 2D confocal actin cables slice of a fission yeast cell labeled by GFP-CHD. The cell radius is 1.73 μm.

Parametric active contour models (snakes) [6] have been reported as an effective method to segment curvilinear structures such as actin filaments, actin cables in fission yeast, and microtubules. Li et al. [7, 8, 9] used Stretching Open Active Contours (SOACs) to segment and track individual filaments in a TIRFM image sequence. Smith et al. [10] further applied this method to quantify the conformations and dynamics of actin cables in 3D confocal microscopy images. Related methods have also been utilized to trace microtubules. Nurgaliev et al. [11] employed active contour models and Monte Carlo simulations to locate microtubules in 3D electron tomography images. Hadjidemetriou et al. [12] tracked microtubule tips automatically using consecutive level sets methods. Sargin et al. [13] traced microtubule bodies using second order derivatives of Gaussian filters and iterative calculation of geodesic paths. Saban et al. [14] automatically located microtubule tips in the first frame and tracked tips by searching for the closest match in subsequent frames.

These methods, however, do not explicitly model the topology of intersecting filaments and filament meshworks. Furthermore, although SOACs demonstrate successful application to single filament segmentation and tracking [7, 8, 9, 10], they are not directly applicable to extracting filament meshworks or segmenting multiple intersecting filaments simultaneously. The reason is that manual initialization is required for each filament and the behavior between different SOACs is not explicitly regulated.

While other methods often segment individual actin filaments one by one, our automated method simultaneously extracts and segments all the filaments in a full-field image. Then the topology of the network structure can be further analyzed and the relationship among filaments can also be retrieved. Previous studies have addressed the network morphology and distribution of intermediate filaments [18, 19, 20]. Here, we introduce a different approach in which a large number of SOACs [7] evolve simultaneously. Furthermore, we explicitly regulate their behavior when they meet, cross, and overlap with each other. When the evolution is converged, we dissect each SOAC at the junctions of the extracted network and re-group these dissected “SOAC segments” to obtain the configuration of actin filaments.

The main contributions of the paper are: (1) A fully automated method for initializing multiple SOACs along intensity ridges in the image; (2) A complete set of mechanisms for regulating SOACs’ behavior during their simultaneous evolution, so that a neat network can be extracted efficiently from noisy images; (3) Re-organization of SOAC segments using Normalized Cuts [17] so that the newly grouped SOACs correspond to physical actin filaments.

2. METHODOLOGY

2.1. Automatic Initialization of Multiple SOACs

Since noise can induce many false positives in detected ridge points, we first obtain the smoothed image I_G by Gaussian filtering with σ equal to the typical filament width (2–3 pixels). Though more sophisticated methods, such as Hessian-based vessel enhancement filtering [15], enhance the appearance of filaments, Gaussian filtering is adequate as suggested by our experiments.

Candidate points for multiple SOAC initialization are then detected. They are ridge points with locally maximum intensity along a certain direction. Since any direction in 2D can be decomposed into x and y components, a ridge point can be found by inspecting the plus-to-minus sign change in the gradient field G = [G_x, G_y] of I_G along x and y directions independently [16]. A pixel at location (i, j) is a vertical ridge point if G_x(i, j − 1) > 0 and G_x(i, j) < 0; likewise, it is a horizontal ridge point if G_y(i, j − 1) > 0 and G_y(i, j) < 0. In other words, the sign of G_x (or G_y) changes from positive to negative at a ridge point. If centered difference is used to estimate G, the ridge points detected may lie next to the pixels with locally maximum intensity. However, this does not incur any problem since SOACs can automatically move to the centerline of a filament during their evolution. Unlike [16], we do not detect other patterns of sign changes in G because +− sign changes already give enough candidate points (Fig. 2).

Fig. 2.

Ridge points detected along x or y directions are labeled green or blue, respectively; the ones detected in both directions are labeled red.

Finally we initialize each SOAC from locally connected vertical/horizontal ridge points using 8-connected-component analysis. Vertical and horizontal ridge points are used separately so that the order to form a SOAC can be determined conveniently. For example, from a connected component of horizontal ridge points, we can initialize a horizontal SOAC starting from the point with minimum x coordinate, up to the one with maximum x coordinate. This automatic initialization generates short vertical (in green) and horizontal (in blue) SOACs (Fig. 3).

Fig. 3.

Initial SOACs initialized from ridge points. Green and blue ones are initialized from vertical and horizontal ridge points, respectively.

2.2. Simultaneous Evolution of Multiple SOACs

2.2.1. Dynamic Deformation with Stretching Energy

In the continuous domain, a stretching open active contour model can be represented as a parametric curve, r(s) = (x(s), y(s)), s ∈ [0,1]. A SOAC evolves by minimizing the contour energy E, where E = E_int + E_ext. E_int is an internal energy term, which maintains the continuity and smoothness of the contour; E_ext is an external energy term, which pushes the contour towards salient image features, such as bright ridges. The internal energy E_int is defined as:

$$E_{\mathit{int}}={\int }_0^1(\alpha (s){\mid {\mathbf{r}}_s(s)\mid }^2+\beta (s){\mid {\mathbf{r}}_{ss}(s)\mid }^2)ds,$$ (1)

where α(s) and β(s) are the relative weights between the first-order and second-order terms. The external energy E_ext is also composed of two terms: an image term E_img = I_G and a stretching term E_str. The definition of the external energy is:

$$E_{\mathit{ext}}={\int }_0^{1}k·(E_{\mathit{img}}(\mathbf{r}(s))+E_{\mathit{str}}(\mathbf{r}(s)))ds,$$ (2)

where k is a constant balancing the internal and external energy.

To make a SOAC grow along the filament, we add stretching forces to its ends [7]. The force direction is along the tangential direction at the ends and its magnitude is proportional to the image intensity I_G (r (s)) at a SOAC point. The gradient of the stretching term is defined by:

$$\nabla E_{\mathit{str}}(\mathbf{r}(s))=\{\begin{array}{ll}-\frac{{\mathbf{r}}_s(s)}{\mid {\mathbf{r}}_s(s)\mid }·k_{\mathit{str}}·I(\mathbf{r}(s))\hfill & \text{if}s=0,\hfill \\ \frac{{\mathbf{r}}_s(s)}{\mid {\mathbf{r}}_s(s)\mid }·k_{\mathit{str}}·I(\mathbf{r}(s))\hfill & \text{if}s=1,\hfill \\ 0\hfill & \text{if}0<s<1,\hfill \end{array}$$ (3)

where k_str is the coefficient balancing image and stretching term. So the external force field is:

$$\nabla E_{\mathit{ext}}(\mathbf{r}(s))=\nabla I_G(\mathbf{r}(s))+\nabla E_{\mathit{str}}(\mathbf{r}(s))$$ (4)

Combining all terms, minimizing the contour energy E makes SOACs grow along the bright ridges while keeping their continuity and smoothness. Summing up all terms and deriving the EulerLagrange equation, a SOAC at iteration i are computed from the one at iteration i − 1 as follows:

$$x_i={(A+\gamma I)}^{-1}(\gamma x_{i-1}-\partial E_{\mathit{ext}}(x_{i-1},y_{i-1})/\partial x),$$ (5a)

$$y_i={(A+\gamma I)}^{-1}(\gamma y_{i-1}-\partial E_{\mathit{ext}}(x_{i-1},y_{i-1})/\partial y),$$ (5b)

where A is the pentadiagonal banded matrix containing the internal continuity and smoothness constraints defined by (1), I is the identity matrix and γ is the step size [6]. All SOACs need to be re-sampled after each iteration to keep the distance between adjacent SOAC points. The contour energy E is minimized at each iteration. When the internal forces and external forces are balanced, the system enters a state of equilibrium. If a certain SOAC’s length and position does not change after a predefined number of iterations, it is considered converged and it stops evolving. After all SOACs are converged, SOACs that are too short to be a filament are deleted. The length threshold is set to 10 pixels.

2.2.2. Merging, Deletion and Growth Termination of Multiple SOACs

Because several initial SOACs may lie on the same filament, they will partly or fully overlap as they elongate. Furthermore, a SOAC may elongate erroneously to another filament at intersections, leading to incorrect segmentation. To solve this problem arising from simultaneous evolution, SOACs are merged, deleted or stopped elongating, to obtain a clear topology of the filament ensemble. Merging and deletion also greatly reduces the number of SOACs in action, making the evolution more efficient.

Let us denote the set of SOACs by S, and its cardinality by |S|. At each iteration, we do pairwise calculation on a pair of SOACs r_i and r_j, i, j ∈ [1, |S|]. For each end point r_i(t), t = 0, 1 of r_i, we locate the last point r_i(p) close to r_j counting from r_i(t), and the closest point r_j(q) to r_i(p) on r_j. The L₂-norm distance threshold D_thresh for measuring closeness is set to 1 pixel.

Case I: SOAC merging

If q = 0,1 and the angle θ between the two tangential vectors at r_i(t) and r_j(q) is sufficiently close to π and greater than θ_thresh, we merge r_i with r_j by concatenating the non-overlapping part of r_i to r_j and then deleting r_i (Fig. 4(a)). We set θ_thresh = 2π/3.

Fig. 4.

Illustration of cases for SOAC merging, deletion, and growth termination. (a) r_i merges with r_j. (b) r_i is deleted. (c) & (d) r_i stops growing at r_i(t). We attach r_i(t) to r_j(q) when the overlapping region exceeds a predefined threshold O_thresh.

Case II: SOAC deletion

If t = 0, p = 1 or t = 1, p = 0, indicating that r_i is completely covered by r_j, we delete r_i (Fig. 4(b)).

Case III: SOAC growth termination

If 0 < q < 1 and the length of the overlapping part of r_i exceeds a predefined threshold O_thresh (Fig. 4(c)), we delete the overlapping part and attach the end of r_i(t) to r_j(q) (Fig. 4(d)). When an end is fixed, we also set the stretching coefficient k_str at that end to zero.

Other than the above three cases, we do not interrupt the SOACs’ evolution.

2.3. Dissecting SOACs and Re-Grouping SOAC Segments

2.3.1. Dissecting SOACs

Even though we regulate SOACs’ behavior in the evolution process, we still cannot guarantee that each converged SOAC corresponds to an actual filament. This is because a SOAC may evolve to represent different filaments, or one filament is represented by several different SOACs. This situation is illustrated in Fig. 5, where green and blue indicate different SOACs.

Fig. 5.

(top) Three intersecting filaments. (bottom) Converged SOACs do not correspond to the actual filaments. Green and blue ones are different SOACs.

To analyze the topology of the extracted filament network represented by the SOACs, we first dissect each SOAC at junction points into “SOAC segments”. Junction points are those SOAC points having one or more neighboring SOAC points that belong to another SOAC. Fig. 6(a) shows an example of detected junctions in the network.

Fig. 6.

(top) Detected junctions shown in red points. (bottom) SOAC segment grouping results; the segments covering the same filament are in the same color; the blue arrow indicates an overlapping common segment that is grouped to both the green and red groups.

2.3.2. Grouping the SOAC Segments

After obtaining the SOAC segments, we can examine which SOAC segments correspond to the same actin filament. Before grouping, we need to detect “common segments” in all the SOAC segments. A common segment is a segment where two or more filaments overlap. The observation for detecting a common segment is that its two ends are both junctions and its average intensity is the highest among all the segments sharing these two junctions. After that, we can form “grouping subsets” of SOAC segments: a grouping subset consists of SOAC segments sharing a junction or sharing a common segment. We then use Normalized Cuts [17] to group segments in each grouping subset. Normalized Cuts is a graph partitioning algorithm for data clustering. It minimizes the global criterion which measures both the total dissimilarity between different subgroups of a graph as well as the total similarity within the subgroups.

Under the graph partitioning framework, a grouping subset can be represented as a small complete weighted undirected graph G_k = (V_k, E_k), k = 1, …, K, where K is the number of grouping subsets. Each SOAC segment in the grouping subset G_k is a node in V_k and the edge weight w_ij specifies the junction continuity, which is defined by the local orientations at the end points of segments. Because the actin filaments are smooth linear structures, two segments sharing a junction are more likely to belong to the same filament if they are continuous and smooth across the junction. So we use this prior information to construct the affinity matrix W = {w_ij}, which is an N × N symmetric matrix and N is the number of segments in the grouping subset (i.e. number of nodes in the subset’s graph G_k). Usually N equals 3 or 4. To compute w_ij, we use the normalized angle between every two tangential vectors at the end points of the segments. According to the results of filament curvature distribution analysis [10], we use a sample distance of 20 pixels to estimate the direction of tangent vectors at end points.

We specify the number of groups as 2 for a grouping subset if N = 3, 4. Occasionally N can be 5 or 6, in which case we let the number of groups be 3.

At the final step, we successively merge the resulting groups if they share segments until all the groups are disjoint. Fig. 6(b) illustrates an example of grouping results. Note that the common segment belongs to both groups adjoining it.

3. APPLICATION TO EXPERIMENTAL DATA

3.1. Experimental Image Data

We use TIRFM image sequences from [3] for validation. In the experiment, polymerization of muscle Mg-ADP-actin is monitored in the presence of inorganic phosphate (Pi) and actin monomers. The pixel size is 0.17 μm. To evaluate our method, we use the last frames of the sequences where the filaments are more crowded and have more intersections than previous frames. A test image is shown in Fig. 1(a). Note that its non-uniform illumination makes the magnitude of stretching force inconsistent across the image. We correct this problem by subtracting the non-uniform background estimated by a gray-scale opening morphological operation.

We also tested our method using a 2D radial projection of a 3D confocal microscopy image of actin in a dividing fission yeast labeled by GFP-CHD [4] (Fig. 1(b)) as well as one 2D confocal slice showing actin cables (Fig. 1(c)). Radial projections are obtained from stacks of images of cells expressing GFP-CHD by radially projecting image data onto a 2D strip which corresponds to the membrane unfolded longitudinally from the middle of the cell. These images of cdc25–22 fission yeast cells reveal an intricate dynamic actin meshwork establishing connections among myosin nodes.

3.2. Segmentation Examples and Evaluation

We select 55 filaments from one of the last frames of TIRFM sequences to measure the segmentation error of our method. For each filament, a SOAC generated by [10] and subsequently modified by a human expert serves as the ground truth. The body distance d(r_c, r_t) between our computed SOAC r_c and the corresponding ground truth SOAC r_t is defined as:

$$d({\mathbf{r}}_c,{\mathbf{r}}_t)=\underset{i}{max}\{\underset{j}{min}d\{{\mathbf{r}}_c(i),{\mathbf{r}}_t(j)\}\}$$ (6)

where r_c(i) and r_t(j) is the ith and jth points of SOAC r_c and r_t respectively. Since the tip location is important, we also compute the L₂ distance between locations of the two end points of our computed SOAC and those of the ground truth SOAC. Fig. 7 shows the segmentation results of our method, compared with the ground truth. Table 1 shows the segmentation error statistics of our method. Fig. 8 illustrates a typical case in which the SOAC segments whose orientations are more consistent are grouped together into one filament during the Normalized Cuts grouping step. Application to confocal microscopy images are shown in Fig. 9.

Fig. 7.

(top) Segmentation results. (bottom) Ground truth.

Table 1.

Body and tip segmentation error statistics of 55 filaments. (Unit: pixel)

	Mean	Maximum	Standard Deviation
Body	2.7312	8.3775	2.0266
Tip1	2.3645	11.2704	2.2668
Tip2	2.1377	8.3775	1.7210

Fig. 8.

(left) Two converged SOACs. (right) Re-grouped SOAC segments at that junction. The top part of the left filament (left) is correctly re-grouped with the right filament (right) based on local orientation continuity using Normalized Cuts.

Fig. 9.

Filament network extraction in confocal microscopy images (see Fig. 1) (a) and (c) are converged SOACs; (b) and (d) are re-grouping results giving the filaments.

4. CONCLUSION

In this paper, we proposed an automated method to simultaneously segment intersecting filaments that form a network structure. We further analyze the topology of the filament network and re-organize the SOACs to represent the physical actin filaments. Experimental and validation results demonstrate the performance of this method. One advantage of our method is that it can overcome the disconnectivity problems induced by intensity gaps or faint filaments in the image when extracting actin networks by thinning. One limitation of our current method is that it cannot distinguish a filament from an overlapping filament if the two filaments overlapping part covers an end point of one of the filaments. In the future we plan to employ SOAC width and intensity information to better identify overlapping parts of filaments.