Convolutional nets for reconstructing neural circuits from brain images acquired by serial section electron microscopy

Abstract

Neural circuits can be reconstructed from brain images acquired by serial section electron microscopy. Image analysis has been performed by manual labor for half a century, and efforts at automation date back almost as far. Convolutional nets were first applied to neuronal boundary detection a dozen years ago, and have now achieved impressive accuracy on clean images. Robust handling of image defects is a major outstanding challenge. Convolutional nets are also being employed for other tasks in neural circuit reconstruction: finding synapses and identifying synaptic partners, extending or pruning neuronal reconstructions, and aligning serial section images to create a 3D image stack. Computational systems are being engineered to handle petavoxel images of cubic millimeter brain volumes.

Introduction

The reconstruction of the C. elegans nervous system by serial section electron microscopy (ssEM) required years of laborious manual image analysis [1]. Even recent ssEM reconstructions of neural circuits have required tens of thousands of hours of manual labor [2]. The dream of automating EM image analysis dates back to the dawn of computer vision in the 1960s and 70s (Marvin Minsky, personal communication) [3]. In the 2000s, connectomics was one of the first applications of convolutional nets to dense image prediction [4]. More recently, convolutional nets were finally accepted by mainstream computer vision, and enhanced by huge investments in hardware and software for deep learning. It now seems likely that the dream of connectomes with minimal human labor will eventually be realized with the aid of artificial intelligence (AI).

More specific impacts of AI on the technologies of connectomics are harder to predict. One example is the ongoing duel between 3D EM imaging approaches, serial section and block face [5]. Images acquired by ssEM may contain many defects, such as damaged sections and misalignments, and axial resolution is poor. Block face EM (bfEM) was originally introduced to deal with these problems [6]. For its fly connectome project, Janelia Research Campus has invested heavily in FIB-SEM [7], a bfEM technique that delivers images with isotropic 8 nm voxels and few defects [8]. FIB-SEM quality is expected to boost the accuracy of image analysis by AI, thereby reducing costly manual image analysis by humans. Janelia has decided that this is overall the cheapest route to the fly connectome, even if FIB-SEM imaging is expensive (Gerry Rubin, personal communication).

It is possible that the entire field of connectomics will eventually switch to bfEM, following the lead of Janelia. But it is also possible that powerful AI plus lower quality ssEM images might be sufficient for delivering the accuracy that most neuroscientists need. The future of ssEM depends on this possibility.

The question of whether to invest in better data or better algorithms often arises in AI research and development. For example, most self-driving cars currently employ an expensive LIDAR sensor with primitive navigation algorithms, but cheap video cameras with more advanced AI may turn out to be sufficient in the long run [9].

AI is now highly accurate at analyzing ssEM images under “good conditions,” and continues to improve. But AI can fail catastrophically at image defects. This is currently its major shortcoming relative to human intelligence, and the toughest barrier to creating practical AI systems for ssEM images. The challenge of robustly handling edge cases is universal to building real-world AI systems, and makes the difference between a self-driving car for research and one that will be commercially successful.

In this journal, our lab previously reviewed the application of convolutional nets to connectomics [10]. The ideas were first put into practice in a system that applied a single convolutional net to convert manually traced neuronal skeletons into 3D reconstructions [11], and a semiautomated system (Eyewire) in which humans reconstructed neurons by interacting with a similar convolutional net [12].¹ A modern connectomics platform (Fig. 1) now contains multiple convolutional nets performing a variety of image analysis tasks.

Figure 1.

Example of connectomic image processing system. Neuronal boundaries and synaptic clefts are detected by convolutional nets. The boundary map is oversegmented into supervoxels by a watershed-type algorithm. Supervoxels are agglomerated to produce segments using hand-designed heuristics or machine learning algorithms. Synaptic partners in the segmentation are assigned to synapses using convolutional nets or other machine learning algorithms. Error detection and correction is performed by convolutional nets or human proofreaders. Gray dashed lines indicate alternative paths taken by some methods.

This review will focus on ssEM image analysis. At present ssEM remains the most widely used technique for neural circuit reconstruction, judging from number of publications [5]. The question for the future is whether AI will be able to robustly handle the deficiencies of ssEM images. This will likely determine whether ssEM will remain popular, or be eclipsed by bfEM.

Serial section EM was originally done using transmission electron microscopy (TEM). More recently, serial sections have also been imaged by scanning electron microscopy (SEM) [15]. Serial section TEM and SEM are more similar to each other, and more different from bfEM techniques. Both ssEM methods produce images that can be reconstructed by very similar algorithms, and we will typically not distinguish between them in the following.

Alignment

The connectomic image processing system of Fig. 1 accepts as input a 3D image stack. This must be assembled from many raw ssEM image tiles. It is relatively easy to stitch or montage multiple image tiles to create a larger 2D image of a single section. Adjacent tiles contain the same content in the overlap region, and mismatch due to lens distortion can be corrected fairly easily [16]. More challenging is the alignment of 2D images from a series of sections to create the 3D image stack. Images from serial sections actually contain different content, and physical distortions of sections can cause severe mismatch.

The classic literature on image alignment has included two major approaches. One is to find corresponding points between image pairs, and then compute transformations of the images that bring corresponding points close together. In the iterative rerendering approach, one defines an objective function such as mean squared error or mutual information after alignment, and then iteratively searches for the parametrized image transformations that optimize the objective function. The latter approach is popular in human brain imaging [17]. It has not been popular for ssEM images, possibly because iterative rerendering is computationally costly.

The corresponding points approach has been taken by a number of ssEM software packages, including TrakEM2 [18], AlignTK [19], NCR Tools [20], FijiBento [21], and EM_aligner [22]. As an example of the state-of-the-art, it is helpful to examine a recent whole fly brain dataset [23], which is publicly available. The alignment appears outstanding at most locations. Large misalignments do occur rarely, however, and small misalignments can also be seen. This level of quality is sufficient for manual reconstruction, as humans are smart enough to compensate for misalignments. However, it poses challenges for automated reconstruction.

To improve alignment quality, several directions are being explored. One can add extensive human-in-the-loop capabilities to traditional computer vision algorithms, enabling a skilled operator to adjust parameters on a section-by-section basis, as well as detect and remove false correspondences [24]. Another approach is to reduce false correspondences from normalized cross correlation using weakly supervised learning of image encodings [25].

The iterative rerendering approach has been extended by Yoo et al. [26], who define an objective function based on the difference between image encodings rather than images. The encodings are generated by unsupervised learning of a convolutional autoencoder.

Another idea is to train a convolutional net to solve the alignment problem, with no optimization at all required at run-time. For example, one can train a convolutional net to take an image stack as input, and return an aligned image stack as output [27]. Alternatively, one can train a convolutional net to take an image stack as input, and return deformations that align it [28]. Similar approaches are already standard in optical flow [29,30].

Boundary detection

Once images are aligned into a 3D image stack, the next step is neuronal boundary detection by a convolutional net (Fig. 1). In principle, if the resulting boundary map were perfectly accurate, then it would be trivial to obtain a segmentation of the image into neurons [4,10].

Detecting neuronal boundaries is challenging for several reasons. First, many other boundaries are visible inside neurons, due to intracellular organelles such as mitochondria and endoplasmic reticulum. Second, neuronal boundaries may fade out in locations, due to imperfect staining. Third, neuronal morphologies are highly complex. The densely packed, intertwined branches of neurons make for one of the most challenging biomedical image segmentation problems.

A dozen years ago, convolutional nets were already shown to outperform traditional image segmentation algorithms at neuronal boundary detection [4]. Since then advances in deep learning have dramatically improved boundary detection accuracy [31–34], as evidenced by two publicly available challenges on ssEM images, SNEMI3D² and CREMI.³

How do state-of-the-art boundary detectors differ from a dozen years ago? Jain et al. [4] used a net with six convolutional layers, eight feature maps per hidden layer, and 34,041 trainable parameters. Funke et al. [34] use a convolutional net containing pathways as long as 18 layers, as many as 1,500 feature maps per layer, and 83,998,872 trainable parameters.

State-of-the-art boundary detectors use the U-Net architecture [35,36] or variants [31,33,34,37,38]. The multiscale architecture of the U-Net is well-suited for handling both small and large neuronal objects, i.e., detecting boundaries of thin axons and spine necks, as well as thick dendrites. An example of automated reconstruction with a U-Net style architecture is shown in Fig. 2.

Figure 2.

Automated reconstruction of pyramidal neuron in mouse visual cortex by a system like that of Fig. 1. The reconstruction is largely correct, though one can readily find split errors (red, 4) and merge errors (yellow, 6) in proofreading. 6 of these errors are related to image artifacts, and merge errors often join an axon to a dendritic spine. Cyan dots represent automated predictions of presynaptic sites. Predicted postsynaptic terminals were omitted for clarity. The 200 × 100 × 100 m³ ssEM dataset was acquired by the Allen Institute for Brain Science (A. Bodor, A. Bleckert, D. Bumbarger, N. M. da Costa, C. Reid) after calcium imaging in vivo by the Baylor College of Medicine (E. Froudarakis, J. Reimer, Andreas S. Tolias). Neurons were reconstructed by Princeton University (D. Ih, C. Jordan, N. Kemnitz, K. Lee, R. Lu, T. Macrina, S. Popovych, W. Silversmith, I. Tartavull, N. Turner, W. Wong, J. Wu, J. Zung, and H. S. Seung).

State-of-the-art boundary detectors make use of 3D convolutions, either exclusively or in combination with 2D convolutions. Already a dozen years ago, the first convolutional net applied to neuronal boundary detection used exclusively 3D convolutions [4]. However, this net was applied to bfEM images, which have roughly isotropic voxels and are usually well-aligned. It was popular to think that 3D convolutional nets were not appropriate for ssEM images, which had much poorer axial resolution and suffered from frequent misalignments. Furthermore, many deep learning frameworks did not support 3D convolutions very well or at all. Accordingly, much work on ssEM images has used 2D convolutional nets [39,40], relying on agglomeration techniques to link up superpixels in the axial direction. Today it has become commonly accepted that 3D convolutions are also useful for processing ssEM images.

The current SNEMI3D [33] and CREMI leaders [32,34] both generate nearest neighbor affinity graphs as representations of neuronal boundaries. The affinity representation was introduced by Turaga et al. [41] as an alternative to classification of voxels as boundary or non-boundary. It is especially helpful for representing boundaries in spite of the poor axial resolution of ssEM images [10]. Parag et al. [42] have published empirical evidence that the affinity representation is helpful.

Turaga et al. [43] pointed out that using the Rand error as a loss function for boundary detection would properly penalize topological errors, and proposed the MALIS method for doing this. Funke et al. [34] have improved upon the original method through constrained MALIS training. Lee et al. [33] used long-range affinity prediction as an auxiliary objective during training, where prediction error for the affinities of a subset of long-range edges can be viewed as a crude approximation to the Rand error.

Handling of image defects

The top four entries on the SNEMI3D leaderboard have surpassed human accuracy as previously estimated by disagreement between two humans. But the claim of “superhuman” performance comes with many caveats [33]. The chief one is that the SNEMI3D dataset is relatively free of image defects. Robust handling of image defects is crucial for real-world accuracy, and is where human experts outshine AI.

One can imagine several ways of handling image defects: (1) Heal the defect by some image restoration computation. (2) Make the boundary detector more robust to the defect. (3) Correct the output of the boundary detector by subsequent processing steps. Below, we detail different types of image defects, their effects on reconstruction, and efforts to account for them.

Missing sections are not infrequent in ssEM images. Entire sections can be lost during cutting, collection, or imaging. The rate of loss varies across datasets, e.g., 0.17% in Zheng et al. [23] and 7.56% in Tobin et al. [44]. It is also common to lose part of a section. For example, one might exclude from imaging the regions of sections that are damaged or contaminated. One might also throw out image tiles after imaging, if they are inferior in quality. Or the imaging system might accidentally fail on certain tiles.

In an ssEM image stack, a partially missing section typically appears as an image with a black region where data is missing. An entirely missing section might be represented by an image that is all black, or might be omitted from the stack.

Traceability of neurites by human or machine is typically high if only a single section is missing. Traceability drops precipitously with long stretches of consecutive missing sections.⁴

Partially and entirely missing sections are easy to simulate during training; one simply blacks out part or all of an image. Funke et al. [34] simulated entirely missing sections during training at a 5% rate. Lee et al. [33] simulated both partially and entirely missing sections at a higher rate, and found that convolutional nets can learn to “imagine” an accurate boundary map even with several consecutive missing sections.

Misalignments are not image defects, strictly speaking. They arise during image processing, after image acquisition. From the perspective of the boundary detector, however, misalignments appear as input defects. Progress in alignment software and algorithms is rapidly reducing the rate of misalignment errors. Nevertheless, misalignments can be the dominant cause of tracing errors, because boundary detection in the absence of image defects has become so accurate.

Lee et al. [33] introduced a novel type of data augmentation for simulating misalignments during training. Injecting simulated misalignments at a rate much higher than the real rate was shown to improve the robustness of boundary detection to misalignment errors.

Januszewski et al. [45] locally realigned image subvolumes prior to agglomeration, in an attempt to remove the image defect before further image processing.

Cracks and folds are common in ssEM sections. They may involve true loss of information, or cause misalignments in neighboring areas. We expect that improvements in software will be able to correct misalignments neighboring cracks or folds.

Knife marks are linear defects in ssEM images caused by imperfect cutting at one location on the knife blade. They can be seen in publicly available ssEM datasets [15].⁵ They are particularly harmful because they occur repeatedly at the same location in consecutive serial sections, and are difficult to simulate. Even human experts have difficulty tracing through knife marks.

Agglomeration

There are various approaches to generate a segmentation from the output of the boundary detector. The naïve approach of thresholding the boundary map and computing connected components can lead to many merge errors caused by “noisy” prediction of boundaries. Instead, it is common to first generate an oversegmentation into many small supervoxels. Watershed-type algorithms can be used for this step [46]. The number of watershed domains can be reduced by size-dependent clustering [46], seeded watershed combined with distance transformation [32,34], or machine learning [47].

Supervoxels are then agglomerated by various approaches. A classic computer vision approach is to use statistical properties of the boundary map [48], such as mean affinity [33] or percentiles of binned affinity [34]. A score function can be defined for every pair of contacting segments. At every step, the pair with the highest score is merged. This simple procedure can yield surprisingly accurate segmentations when starting from high quality boundary maps, and can be made computationally efficient for large scale segmentation.

Agglomeration can also utilize other information not contained in the boundary map, such as features extracted from the input images or the segments [49–52]. Machine learning methods can also be used directly without defining underlying features to serve as the scoring function to be used in the agglomeration iterations [53].

Supervoxels can also be grouped into segments by optimization of global objective functions [32]. Success of this approach depends on designing a good objective function and algorithms for approximate solution of the typically NP-hard optimization problem.

Error detection and correction

Convolutional nets are also being used to automatically detect errors in neuronal segmentations [54–57]. Dmitriev et al. [56] leverage skeletonization of candidate segments, applying convolutional nets selectively to skeleton joints and endpoints to detect merge and split errors, respectively. Rolnick et al. [55] train a convolutional net to detect artificially induced merge errors. Zung et al. [57] demonstrate detection and localization of both split and merge errors with supervised learning of multiscale convolutional nets.

Convolutional nets have also been used to correct morphological reconstruction errors. Zung et al. [57] propose an error-correcting module which prunes an “advice” object mask constructed by aggregating erroneous objects found by an error-detecting module. The “object mask pruning” task (Fig. 3b) is an interesting counterpoint to the “object mask extension” task implemented by other methods (Fig. 3a) [45].

Figure 3.

Illustration of (a) object mask extension [40,45] and (b) object mask pruning [57]. Both employ attentional mechanisms, focusing on one object at a time. Object mask extension takes as input a subset of the true object and adds missing parts, whereas object mask pruning takes as input a superset of the true object and subtracts excessive parts. Both tasks typically use the raw image (or some alternative representation) as an extra input to figure out the right answer, though object mask pruning may be less dependent on the raw image [57].

Synaptic relationships

To map a connectome, one must not only reconstruct neurons, but also determine the synaptic relationships between them. The annotation of synapses has traditionally been done manually, yet this is infeasible for larger volumes [58]. Most research on automation has focused on chemical synapses. This is because large volumes are typically imaged with a lateral resolution of 4 nm or worse, which is insufficient for visualizing electrical synapses. Higher resolution would increase both imaging time and dataset size.

A central operation in many approaches is the classification of each voxel as “synaptic” or “non-synaptic” [59–63]. Increasingly, convolutional nets are being used for this voxel classification task [58,64–69]. Synaptic clefts are then predicted by some hand-designed grouping of synaptic voxels.

A number of approaches are used to find the partner neurons at a synaptic cleft [58,64,66,67,70]. For example, Dorkenwald et al. [58] extract features relating the predicted cleft segments to candidate partners by overlap with these partners, as well as their contact site, and feed these features to a random forest classifier for a final “synaptic” or “non-synaptic” classification for inferring connectivity. Parag et al. [70] pass a candidate cleft, local image context, and a candidate pair of partner segments to a convolutional net to make a similar “synaptic” or “non-synaptic” judgment. Turner et al. use a similar model, yet they instead use the cleft as an attentional input to predict the voxels of relevant presynaptic and postsynaptic partners (N Turner et al., submitted).

Another group of recent approaches do not explicitly represent synaptic clefts [69,71]. Buhmann et al. [69] detect sites where presynaptic and postsynaptic terminals are separated by specific spatial intervals, using a single convolutional net to predict both location and directionality of synapses. Staffler et al. [71] extract contact sites for all adjacent pairs of segments within the region of interest, and use a decision tree trained on hand-designed features of the “subsegments” close to the contact site in order to infer synaptic contact sites and their directionality of connection.

Distributed chunk-wise processing of large images

In connectomics, it is often necessary to transform an input image into an output image (Fig. 1). The large input image is divided into overlapping chunks, each of which is small enough to fit into RAM of a single worker (CPU node or GPU). Each chunk is read from persistent storage by a worker, which processes the chunk and writes the result back to persistent storage. Additional operations may be required to insure consistency in the regions of overlap between the chunks. The processing of the chunks is distributed over many simultaneous workers for speedup.

To implement the above scheme, we need a “chunk workflow engine” for distributing chunk processing tasks over workers, and handling dependencies between tasks. We also need a “cutout service” that enables workers to “cut” chunks out of the large image. This includes both read and write operations, which may be requested by many workers in parallel. The engine and cutout service may function with a local cluster and networkattached storage, or with cloud computing and storage.

DVID is a cutout service used with both network-attached and Google Cloud Storage [72]. Requests are handled by a server running on a compute node in the local cluster or cloud. For ease of use, DICED is a Python wrapper that makes a DVID store behave as if it were a NumPy array [73]. bossDB uses microservices (AWS Lambda) to fulfill cutout requests from an image in Amazon S3 cloud storage [74]. ndstore shares common roots with bossDB [75], but uses Amazon EC2 servers rather than microservices. It is employed by neurodata.io, formerly known as the Open Connectome Project.

For cloud storage, the cutout service can also be implemented on the client side (Fig. 4). CloudVolume [76] and BigArrays.jl [77] convert cutout requests into native cloud storage requests. Both packages provide convenient slicing interfaces for Python NumPy arrays [76] and Julia arrays [77] respectively.

Figure 4.

Two approaches to chunk cutout services. (a) Server-side cutout. A server responds to a cutout request by loading the required image blocks (light blue), performing the cutout (blue), and sending the result back to a client. (b) Client-side cutout. A client requests the chunk blocks from cloud storage, and performs the cutout. A read operation is shown here; write operations are also supported by cutout services.

The chunk workflow engine handles task scheduling, the assigning of tasks to workers. In the simplest case, processing the large image can be decomposed into chunk processing tasks that do not depend on each other at all. Then scheduling can be handled by a task queue. Tasks are added to the queue in arbitrary order. When a worker is idle, it takes the first available task from the queue. The workers continue until the queue is empty. The preceding can be implemented in the cloud using AWS Simple Queue Service [78,79]. Task scheduling is more complex when there are dependencies between tasks. If workflow dependencies are represented by a directed acyclic graph (DAG), the workflow can be executed by, for example, Apache Airflow [80].

In large scale distributed processing, errors are almost inevitable. The most common error is that a worker will fail to complete a task. Without proper handling of this failure mode, chunks will end up missing from the output image. A workflow engine typically judges a task to have failed if it is not completed within some fixed waiting time. The task is then reassigned to another worker. This “retry after timeout” strategy can be implemented in a variety of ways. In the case of a task queue, when one worker starts a task, the task is made invisible to other workers for a fixed waiting time. If the worker completes the task in time, it deletes the task from the queue. If the worker fails to complete the task in time, the task becomes visible in the queue again, and another worker can take it.

One can reduce cloud computing costs by using unstable instances (called “preemptible” on Google and “spot” on Amazon). These are much cheaper than on-demand instances, but can be killed at any time in favor of customers using on-demand instances or bidding with a higher price. When an instance is killed, its tasks will fail to complete. Therefore, the error handling described above is not only important for insuring correct results, but also for lowering cost.

Beyond boundary detection?

The idea of segmenting a nearest neighbor affinity graph generated by a convolutional net that detects neuronal boundaries is now standard (Fig. 1). One intriguingly different approach is the flood-filling network (FFN) of Januszewski et al. [45]. The convolutional net receives the EM image as one input, and a binary mask representing part of one object as an “attentional” input. The net is trained to extend the binary mask to cover the entire object within some spatial window (Fig. 3a). Iteration of the “flood filling” operation can be used to trace an object that is much larger than the spatial window. “Flood filling,” also called “object mask extension” in the computer vision literature, directly reconstructs an object. This is a beautiful simplification, because it eliminates postprocessing by a segmentation algorithm (e.g. connected components or watershed) as required by boundary detection.

That being said, whether FFNs deliver accuracy superior to state-of-the-art boundary detectors is unclear. On the FIB-25 dataset [81], FFNs [45] outperformed the U-Net boundary detector of Funke et al. [34], but this U-Net may no longer be state-of-the-art according to the CREMI leaderboard. In the SNEMI3D challenge, the accuracies of the FFN and boundary detector approaches are merely comparable [45].

Currently FFNs have high computational cost relative to boundary detectors, because every location in the image must be covered so many times by the FFN [45]. Nevertheless, one can imagine that FFNs could become much less costly in the future, riding the tailwinds of huge industrial investment in efficient convolutional net inference. Boundary detectors will at the same time become less costly, but watershed-type postprocessing of boundary maps may not enjoy similar cost reduction.

Another intriguing direction is the application of deep metric learning to neuron reconstruction. As noted above, Lee et al. [33] have shown that prediction of nearest neighbor affinities can be made more accurate by also training the network to predict a small subset of long-range affinities. For predicting many or all long-range affinities, it is more efficient for a convolutional net to assign a feature vector to every voxel, and compute the affinity between two voxels as a function of their feature vectors. Voxels within the same object receive similar feature vectors and voxels from different objects receive dissimilar feature vectors. This idea was first explored in connectomics by Zung et al. [57] in an error detection and correction system.

Given the representation from deep metric learning, it is tempting to reconstruct an object by finding all feature vectors that are close in embedding space to a seed vector. This approach has been followed by the computer vision literature [82,83], effectively portraying deep metric learning as a “kill watershed” approach like FFNs. However, seed-based segmentation turns out to be outperformed by segmenting a nearest-neighbor affinity graph generated from the feature vectors [84]. Seed-based segmentation often fails on large objects, because the convolutional net cannot learn to generate sufficiently uniform feature vectors inside them.

Whether novel “object-centered” approaches like FFNs and deep metric learning can render boundary detectors obsolete remains to be seen. The showdown ensures suspense in the future of AI for connectomics.

Conclusion

The automation of ssEM image analysis is important because ssEM is still the dominant approach to reconstructing neural circuits [5]. It is also highly challenging for AI because ssEM image quality is poor relative to FIB-SEM. The field has made dramatic progress in increasing accuracy, but human effort is still required to correct the remaining errors of AI. Due to lack of space, we are unable to describe the considerable engineering effort currently going into semi-automated platforms that enable human proofreading of automated reconstructions. These platforms must adapt to leverage the increasing power of AI. As the error rate of automated reconstruction decreases, it may become more timeconsuming for the human expert to both detect and correct errors. To make efficient use of human effort, the user interface must be designed and implemented with care. Automated error detection may be useful for focusing human effort on particular locations [54], and automated error correction may be useful for suggesting interventions.

In spite of the gains from AI, the overall amount of human effort employed by connectomics may increase because the volumes being reconstructed are also increasing. Even a cubic millimeter has never been fully reconstructed yet, and a human brain is larger by six orders of magnitude. The demand for neural circuit reconstructions should also grow dramatically as they become less costly. Therefore AI may actually increase the total amount of human effort devoted to connectomics, at least in the short term, bucking the conventional wisdom that AI is a jobs destroyer.

Automation of image analysis is spurring interest in increasing the throughput of image acquisition. When image analysis was the bottleneck, increasing image quality was perhaps more important. Now that the bottleneck is being relieved, there will be increased demand for high throughput SEM and TEM image acquisition systems [23,85]. With continued advances in both image acquisition and analysis, it seems likely that ssEM reconstruction of cubic millimeter volumes will eventually become routine. It is unclear, however, whether ssEM image acquisition can scale up to cubic centimeter volumes (whole mouse brain) or larger. New bfEM approaches are being proposed for acquiring such exascale datasets [86]. The battle between ssEM and bfEM continues.