Leginon: New features and applications

Abstract

Leginon is a system for automated data acquisition from a transmission electron microscope. Here we provide an updated summary of the overall Leginon architecture and an update of the current state of the package. We also highlight a few recent developments to provide some concrete examples and use cases.

1. INTRODUCTION

Leginon is a modular system for automatically acquiring images from a transmission electron microscope (TEM). ¹ , ² , ³ , ⁴ This software has been used for many years (the primary papers have been cited ~800 times) to collect data for either single particle cryo electron microscopy (cryoEM) or cryo electron tomography (cryoET). Recently, functionality has also been added to provide for data collection for microcrystal electron diffraction (microED) ⁵ an approach of interest to organic chemists and structural biologists.

High‐resolution electron microscopy requires a series of targeting and image acquisition steps designed to obtain the highest quality high magnification images in the shortest amount of time. CryoEM single particle analysis and cryoET projects require imaging vitrified samples suspended as a thin layer over a fenestrated (holey) substrate, typically carbon or gold, supported by an electron microcopy grid consisting of a metal mesh (see Figure 1). The goal of imaging is to locate holes where the vitrified ice is of optimal thickness and the particles are well distributed in the ice, adjust certain microscope parameters, and then acquire a final high magnification image or movie tilt series. For microED, the targets of interest are potentially well‐ordered microcrystals with suitable dimensions for data collection (see Section 3.1). Note that the overall quality and distribution of sample across a typical EM grid does not allow for a brute force approach to targeting where, for example, high magnification images would be systematically acquired as a raster across the EM grid; this would lead to a very high number of unsuitable images and be unacceptably inefficient given the high cost of using these instruments. Leginon thus manages and automates the task of selecting targets, at a series of increasing magnifications, with the goal of selecting optimal targets to provide high‐quality high‐magnification movies; once set up it can run uninterrupted and unattended for many days at a time.

FIGURE 1.

Multi‐Scale Imaging and Leginon accessory functions: Images at higher magnification are acquired by defining targets on the parent images. Acquisition and Targeting node classes string these processes together. Additional targets can be added to perform other automated tasks. Focusing targets (blue) are used to adjust eucentric height and perform low‐dose focusing. Reference targets (violet) are used to perform periodic alignments such as the energy filter zero‐loss slit. The reference node may also pass data to an accessory node to do additional tasks such as ice thickness measurement

There are a number of other academic and commercial automated, or semi‐automated, software systems that perform similar data collection (reviewed in Reference 6), most notably SerialEM ⁷ and EPU (Thermo Fisher Scientific). Leginon is unique among these in its centralized approach to multiple microscopes and project management. Leginon is closely integrated with a web server that provides users with live feedback and the opportunity for remote collaborations during data collection. In addition, the Leginon database and webviewer also serve as the basis for live analysis of the data using Appion, ⁸ software that “wraps” a large number of other packages. The close links with a database, a webviewer and integration with Appion means that preliminary analysis allows data filtering and feedback to be concurrent with the data collection.

The current Leginon framework was published in 2005. ³ Updates to support new microscopes, cameras and peripheral services are included in version releases on an ongoing basis. ¹ Other functionalities added since the initial publication have been scattered in various technical publications and/or with papers focused on the biological results. These include support for tomography, ⁴ robotic grid screening, ⁹ , ¹⁰ live processing with Appion, ⁸ random conical tilt data collection, ¹¹ specialized schemes for tilted image collection, ¹² ice thickness measurement, ¹³ and various use cases. ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ The goal of this paper is to provide an updated summary of the overall Leginon architecture and an update of the current state of the package. We also highlight a few recent developments to provide some concrete examples and use cases.

2. DESIGN

Leginon architecture (Figure 2) has not fundamentally changed from that described in the original publication in 2005. At its base is a database for storing and accessing metadata, and a file system for storing acquired raw images and movies as well as processed data. Leginon accesses these two systems to perform two main functions:

FIGURE 2.

Leginon architecture: The microscope, cameras and other instruments (phase plates, energy filters, etc.) are controlled by facility staff or local users through the Leginon user interface (see Figure 7). Database and File Servers store the data and metadata which is viewed using a web browser; Appion also uses the web viewer as a user interface for live preprocessing (see Figure 3). A new web interface was recently added to provide remote manual targeting and on‐site support requests (see Figure 8)

• Instrument controls, written in python, using an object‐relational mapping (ORM) system developed specifically for the project. These controls also require scripting access to the instrument, usually provided by the instrument manufacturer, or sometimes by third‐party middleware, for example, in the case of the Gatan camera access is via SEMCCD.dll from SerialEM.

• Data display as a web service through the software bundle LAMP (Linux, Apache, MySQL, PHP). Originally, the operating system used for the web service software bundle was not limited to Linux. However, to reduce support overhead, we now only officially support the more stable version of CentOS.

At an institution like ours, ² multiple microscopes, multiple projects, and multiple user groups means that a high level of organization is critical. The centralized database underlying Leginon provides the organization required to support large scale management and also makes possible live‐processing through Appion. Appion is distributed with Leginon as an add‐on functionality. Appion uses the same database, file storage system, and the same ORM in its python wrapper scripts, and the web service acts as its user interface. If processing is desired as an add‐on functionality, Appion uses the same systems in its python wrapper scripts. With so many rapid developments of processing software, we have found it difficult to keep Appion current and we no longer recommend it as a full workflow package that proceeds all the way to 3D reconstruction and refinement. Instead, we focus on offering real time feedback during data collection, including full functionality and analysis for motion correction, ice thickness evaluation, CTF fitting, and particle picking (Figure 3). We also offer tools for importing and exporting data to and from Appion. For example, users can export a star file including CTF estimates from Appion or import 2D classification and 3D refinement jobs from cryoSPARC, ¹⁸ and can import the final results from a cryoSPARC run for archiving. For each data collection session, we have also added an option to display an overall summary of the experiment in a report form (Figure 4). This report can be sent as a PDF file that can be shared with collaborators or archived for future reference.

FIGURE 3.

Web viewers: (a) Integrated Leginon and Appion web tools provide real‐time display of acquired images and results from on‐the‐fly particle picking, CTF estimation, motion correction and ice thickness. (b) Similar tools are available for observing and monitoring microED data collection

FIGURE 4.

Summary report: Data summarized includes (a) image statistics; (b) ice thickness; (c) defocus and CTF estimates; (d) overall grid atlas; (e) experimental setup; (f) experimental methods; (g) image processing results uploaded by the operator

From an end user's point of view, the instrument control python programs are workflows made from functional modules called “nodes.” The position of the nodes in the workflow in relation to others are defined by bindings. The basic workflow of Leginon requires multi‐scale imaging (MSI) (Figure 1). At each scale, an acquisition node outputs child images taken at a given set of microscope and camera conditions based on input target positions defined on an existing parent image. A targeting node follows each imaging node that accepts an image as its input and outputs the targets chosen within the node either automatically, or optionally, manually as selected by the user. Images are acquired at progressively higher magnifications; at the highest magnification where the final desired data is recorded, a specific subclass of acquisition nodes is used to perform specialized tasks such as acquiring direct detector movies, tomography tilt series, or diffraction tilt series. In addition, each node class instance can have unique settings in the workflow to give it a more refined behavior. A number of accessory nodes handle management of the microscope and camera preset values, autofocusing, adjustment of the targets to account for specimen drift when returning to targets queued up previously, etc. A recently developed accessary node provides ice thickness measurements for every high magnification image. ¹³ Together, the nodes, bindings, and the corresponding settings can be saved as an application. While any user can create their own applications, most users use the many applications that are provided standard with Leginon (see Table 1 for some of the more common applications).

TABLE 1.

Standard MSI applications in Leginon

Name	Primary use
MSI‐T	Single particle data collection (typically from holey C or Au EM grids)
MSI‐Tomography	Cryo electron tomography data collection
MSI‐Raster‐Screen	Data collection from multiple grids (typically for negative stain screening)
MSI‐PP‐Stitch	Data collection using phase plates
MSI‐RCT	Data collection using random‐conical tilt geometry
MSI‐T Tilt	Data collection using a tilted stage
MSI‐Diffr	Data collection for micoED

Leginon is distributed as open‐source under the Apache2 license. The code and documentation are freely available ³ and a Docker container image is available ⁴ that provides simulated data collection so as to provide an opportunity to learn the overall operation, set up of automated target finding, and the general feel of running an automated data collection system.

3. MAJOR NEW FEATURES

Since 2005, approximately 17,000 revisions have been made to the Leginon and Appion code base. These correspond to numerous enhancements and advances; only a few of the most illustrative or important ones will be described here. A more comprehensive list is available at leginon.org.

3.1. Microcrystal electron diffraction workflow

The philosophy behind Leginon is both to provide a full workflow (from project selection, to data collection, and data display), and to provide good quality assurance to maximize efficiency. The recently developed microcrystal electron diffraction application, MSI‐Diffr, workflow provides an illustrative example (Figure 5).

FIGURE 5.

Leginon microED workflow: First, an atlas is collected to locate good grid squares with crystals. The user then selects grid squares and areas of interest within grid squares for higher magnification images. These images are fed into a queue of images for user targeting. The user can then target suitable crystals from these images and add them to the exposure collection queue. The user has the option to test the diffraction of any crystal with Diffraction Preview Mode. This immediately takes a single diffraction image from the targeted crystal so that the user can decide whether to add that crystal to the queue or omit it based on a visual inspection of the diffraction pattern displayed in the Web Viewer (Figure 3b). Once a sufficient number of crystals have been targeted, the user submits the queue. Leginon then automatically moves to each crystal target, and performs eucentric focusing. The stage is then set to the starting tilt angle and rotated at a constant speed to the desired end point while the camera records diffraction data in rolling shutter mode. The image files are saved, and converted to MRC format, for the web viewer, and SMV format for import into standard crystallographic software. This process repeats until all crystal targets have been processed. Note that the data collection parameters can be altered by the user at any time during the execution of the queue of crystal targets

Continuous rotation electron diffraction using a transmission electron microscope (TEM), also known as microcrystal electron diffraction (MicroED), continuously records rotation data in diffraction mode from three‐dimensional crystals without applying a beam tilt. Due to the much stronger interactions between electron and matter, significantly smaller crystals, typically 0.2–1 μm in length, can be used compared to the much larger crystals required for X‐ray crystallography (~5–10 μm for a synchrotron source and ~50 μm for a lab source). The method has been developed over several years for protein crystals ¹⁹ but recently gained attention when it was demonstrated that the method could also be applied to small organic crystals, to produce data at sub‐Angstrom resolution which can be phased using ab initio methods. ²⁰ , ²¹ This application to small molecules (limited to <1,000 Da) in particular makes this a valuable tool for medicinal chemistry, chemical biology, natural product research, organic semiconductor research and synthetic organic chemistry. We developed a Leginon implementation of the MicroED workflow for a TFS Glacios equipped with Ceta‐D camera, that does not requiring a frame saving image server. The application is as simple to use as the tomography application, only requiring the user to manually target potential crystals of interest. It includes accurate crystal position recall from a queued list of targets through iterative movement in image search mode, a eucentric height correction for each target, extra lens normalization required by the diffraction mode setup, automated aperture selection, coordination of imaging to diffraction mode switching with automatic beamstop insertion and retraction, and synchronized stage rotation with continuous data collection. It also uses the true rolling‐shutter mode of the camera which avoids gaps in data collection. The recorded diffraction series are uploaded into the Leginon database along with other metadata, and also automatically converted to crystallographic SMV format.

Two specialized nodes were added to Leginon in this implementation: (a) a preview of potential crystals and (b) acquisition of rolling‐shutter movie series in diffraction mode while the specimen stage is continuously tilted. Furthermore, we added workarounds to address the lack of official scripting control of the beamstop as well as the initiation and saving of true rolling shutter movie acquisition. This was achieved using AutoIt ⁵ scripting called within Leginon functions. AutoIt interacts with the graphical interface of the microscope control system running on the Windows PC to control required functions. AutoIt can access most Windows objects from their class instance rather than the primitive display position used by other programs; this has the advantage of higher reliability. Overall, these features deliver a high success rate and high quality for automated microED data collection.

The crystal preview node is an acquisition node subclass. It is a good example of the quality assurance tools built into Leginon. Users can choose to collect a single diffraction image, over a small continuous tilt range, using this node to help decide whether a potential crystal should be included in the queue of crystals destined for fully automated data collection. A single diffraction image can be used to estimate resolution, identify pathologies such as streaky spots (high mosaicity) and the presence of multiple lattices, determine if a crystal is too thick to penetrate, provide a rough estimation of unit cell dimensions (important for distinguishing contaminants from the target of interest), and can guide the user to select the most optimal data collection strategy (particularly camera length selection and dose per frame). Using this node ensures that the operator only queues the highest quality crystals for final data collection with an optimal collection strategy, improving the overall efficiency and quality of the data collected. A small continuous tilt is used because electron diffraction can be very sparse at a fixed angle. The default tilt range for a preview image is 5°, but any value can be specified by the operator.

The second specialized node performs the main tasks of continuous rotation electron diffraction: centering each crystal in X, Y, and Z, setting up diffraction imaging conditions, inserting the beamstop, and continuously tilting the crystal while acquiring diffraction images in rolling shutter mode. In the microED implementation in SerialEM ²² centering the crystal and beamstop position were performed manually for each target during a one‐time set up for microED collection; this was required because the microscopes manufacturer did not provide external control of the beamstop. Following this initial setup in SerialEM, each crystal was centered by recalling and setting a saved stage position, and a diffraction movie was acquired using a multithreaded global shutter mode. Since digital cameras do not allow for acquiring exposures during the readout phase of this shutter mode, gaps in the “continuous” movie occur and can be up to 30% of the cycle. This is not the case in rolling shutter mode. In the Leginon implementation, each crystal is iteratively centered to an accuracy of ~0.1 μm, monitored by acquiring images in imaging mode. This level of accuracy is especially important when operating with a 20 μm C2 aperture which on our Glacios corresponds to a beam diameter of ~0.6 μm, which is often about the same size as, or smaller than, the dimensions of the crystals of interest. A very small beam is advantageous because it reduces the background signal from the grid substrate and other crystals/debris that might be in the beam path and eliminates the need for a selected area aperture.

Once the crystal has been adequately centered, Leginon switches out of imaging mode and into diffraction mode. It is very important that the beam remains stably centered behind the beamstop after this switch to diffraction mode to protect the camera and ensure the quality of the data. A series of lens normalizations were added to achieve the required reproducibility and reduce beam movement during collection. Next, the stage is tilted to the selected start angle and then the stage is rotated at the selected tilt speed for the selected tilt range and diffraction data is acquired. To ensure that a constant speed is achieved, a 2‐s delay was added before the beam is un‐blanked and data collection is initiated. As with other acquisition nodes in Leginon, many targets can be selected and then a large batch can be submitted for automated data collection. Overnight data collection can collect hundreds of datasets from a single grid without the need for user intervention. Overall, these features reliably deliver high quality data in a highly automated fashion.

The Leginon full workflow approach does not end when the data is written onto the local disk. For example in the microED application, services are made available to automatically transfer the raw movies to a central storage system, and upload these, and their associated metadata, to the Leginon database, where they may be viewed and assessed by anyone with permission and access to a web browser (see Figure 3b). This viewer displays diffraction images paired with images of the targeted crystal and includes a ruler to measure the resolution of a given reflection. The movies are also converted to SMV crystallographic format with all of the necessary metadata required for data processing in standard crystallographic software packages. As SMV format does not allow for negative pixel values, an offset value is added to all pixel values. This value is reported in the image header as the LEGINON_OFFSET and corresponds to the most negative value recorded. Additionally, a suggested pedestal value (IMAGE_PEDESTAL) is determined by examining the minimum pixel values across the dataset after applying the offset and selecting the largest of these. For crystallographic software packages that allow for negative pixel values, such as DIALS, ²³ this pedestal can be applied to better approximate the true baseline. Additionally, a DIALS format class was created for data collected using a Ceta‐D camera operated in rolling shutter mode. ⁶ In the future we plan to incorporate DIALS into Appion to provide on‐the‐fly data processing to help the operator collect the best possible data as efficiently as possible.

3.2. Ice thickness measurement

A critical aspect of acquiring useful cryoEM images is selecting targets with optimal vitreous ice thickness. Ice that is too thin can exclude particles, whereas ice that is too thick results in both addition of unnecessary noise and increases the possibility that particles will be overlapping in the final projection image. Somewhat surprisingly, measurement of ice thickness during data collection has become routine only recently. ¹³ Leginon includes two tools to measure ice thickness of final high‐magnification images. For microscopes with an energy filter, it uses the ratio of intensity of images with and without the slit inserted. By calibrating the apparent mean free path of electrons through ice and providing this value in the Leginon settings, Leginon can automatically determine the ice thickness. For microscopes without an energy filter, a comparison of intensity over vacuum with intensity over ice allows determination of ice thickness using Beer's Law. Again, a scattering coefficient must be provided to the Leginon interface. Mean‐free paths and scattering coefficients for several microscope configurations are listed in Reference 13. We have also since added:

• Glacios or TFS Talos Arctica (200 keV), no energy filter; 100 μm objective; ALS = 1,055;

• TFS Talos Arctica (200 keV), no energy filter; 70 μm objective; ALS = 745.

While these are good starting values for Leginon users they should be verified using tomography for a specific instrument. Using the energy filter provides the most accurate method as it does not require a measured fixed intensity and so it is unaffected by changes in beam intensity. It does, however, add several seconds of time to each image, since two additional images need to be taken and the energy filter slit needs to be removed and inserted. In practice, the ice thickness can be measured using both methods during screening to establish that the ALS method is providing accurate results, and then the EF method need only be done once in a while (e.g., every 100 images), to ensure that the beam intensity is holding steady.

While it is advantageous to measure the ice thickness for all images, it is even more valuable to correlate this final ice thickness measurement to the nominal ice thickness estimated from the intensity in the holes using lower magnification images. This is useful as it is often difficult to estimate ice thickness by eye, particularly on gold grids. The Leginon Summary page provides a plot of the hole thickness estimation value versus the final, measured thickness in the high magnification images (Figure 6). The graph is linear, with some scattering, and correlation coefficients are generally above 0.9. Using this correlation, it is straightforward to have Leginon only acquire images of targets which are in the desired thickness range. Users at both New York Structural Biology Center (NYSBC) and New York University (NYU) find this feature exceptionally valuable, as it allows them to focus on taking images with optimal ice thickness using the limited microscope time they have available. We are currently exploring methods to predict ice thickness at an even earlier step in the process with the goal of indicating estimated thickness values in the atlas view.

FIGURE 6.

Ice thickness monitoring: Plot of predicted hole ice thickness versus measured high magnification target ice thickness indicates close correlation. To obtain ice thickness between 20 and 80 nm, the hole thickness values should be constrained to values from 0.05 to 0.14 or more precise values can be calculated from the fit

3.3. Phase plate characterization

An extensive array of Volta phase tools have been developed for Leginon over the last few years and continue to be available despite the current relatively low interest in this once highly anticipated hardware option. The features include a fully automated workflow for MSI imaging, charging of the phase plate spot, and moving to the next spot after a user‐defined number of images has been acquired. The carbon areas of the phase plate do not behave uniformly, and thus some areas are unsuitable for imaging. A Leginon node was created for characterizing all phase plate spots and unsuitable spots are automatically avoided during automated data collection. We also control the direction of the beam tilt used in the autofocusing routine to direct the focusing position to a spent phase plate spot rather than a fresh phase spot.

3.4. Leginon remote

The Leginon main user interface (Figure 7) is written in wxPython. It is feature rich and comes with several automated algorithms for selecting targets. However, we have found a significant number of users that require very specific targeting, for example, to identify sparse filaments or during initial screening to identify which hole, or region inside a hole, yields the best ice and particle density. Our facility, based at the Simons Electron Microscopy Center (SEMC), serves the New York cryoEM and cryoET community through nine institutional memberships to the NYSBC. We also serve the national community through the National Resource for Automated Molecular Microscopy ⁷ and the National Center for CryoEM Access and Training. ⁸ Many users would like at least limited remote control of grid targeting. This can be provided through VNC or some other remote‐desktop software, which is used routinely by our staff. However, many external users are not seasoned microscopists and would face a steep learning curve in using the powerful Leginon GUI, as well as potentially doing some harm to the microscope if provided with full control of all options. In addition, there are potential security concerns in providing VPN access to a very large number who are widely geographically dispersed.

FIGURE 7.

Leginon GUI interface: (a) Workflow list and node selection. (b) Toolbar for the selected node in (a), showing Exposure_Targeting in this case. (c) Continuous log of events including warning and errors. (d) Node information panel, in this case providing image statistics. (e) Auto‐targeting types and settings arranged by processing steps

To address the need for limited off site control, we have developed the “Leginon‐Remote” application, which provides users with the ability to remotely select targets, pause and continue data collection sessions, and chat with on‐site staff, all through a standard web browser (Figure 8). The application is written in Python and Javascript using the Django web development framework. It uses a Nginx web server, a Postgres database to store metadata, Slack for live user‐staff communication, and Docker for deployment, data, and service management. While a full Leginon installation is required to use “Leginon‐Remote,” the source code and deployment are isolated from Leginon and the two applications communicate via REST HTTP requests. “Leginon‐Remote” is open source distributed under the MIT license and instructions for installation, configuration, and operation are available via github. ⁹

FIGURE 8.

Leginon‐remote interface: (a) interactive target selection; (b) Leginon operation status showing nodes requiring remote intervention and nodes that are busy; (c) remote tools available include: grid atlas refresh, queue submission, pause/resume, and column valve closing; (d) session progress; (e) live chat with staff (need some targets on this image)

3.5. Aberration correction of large beam‐image shift

Large beam‐image shifts are used to target many holes for every stage move and defocus setting. ²⁴ This approach significantly speeds up data collection at the cost of introducing aberrations into the images. As an averaging technique, cryoEM does not appear to require every image to be perfect as long as the average converges to the correct answer. ¹⁷ However, significant aberration affects CTF estimation as well as FSC‐based half‐maps assessment of resolution. ²⁵ Fortunately, the strongest aberrations induced by beam‐image shift, mainly coma and astigmatism, can be corrected both during and after data collection (Relion 3.1, CryoSPARC 2). The calibrations necessary to achieve coma correction during data collection are semi‐automated; implementation of this feature in Leginon is similar to that in SerialEM (Xu and Mastronarde, personal communication). In general, these calibrations are sufficient to treat all images as being in the same beam‐tilt group when final beam tilt corrections are performed in software using Relion or CryoSPARC. However, if these calibrations are somewhat off, or if the coma‐correction was accidentally turned off in Leginon, then it is critical to correct for these effects to obtain the highest resolution. Since images can be collected with image shifts of up to 8 μm in various directions, they need to be divided into similar image shift groups prior to software correction. As all data collection parameters are stored in the Leginon database a simple program, tiltgroupwrangler.py, is provided to sort the tilt groups. It requires downloading the star file from Leginon that contains CTF values and beam tilts for all groups. After this data is read into tiltgroupwrangler.py, a plot is made of all x and y image shift values and a slider is used to determine the number of groups (Figure 9), with 50–100 being sufficient. Grouping is achieved using k‐means clustering, and the program outputs a Relion 3.0 file with images divided into groups. Beam tilt corrections can then be performed in software by treating the particles from images in these groups as separate sets. A module is also available to do the same grouping for particles processed with CryoSPARC, though this requires the availability of a CryoSPARC python library to write the required cs files.

FIGURE 9.

(a) Main interface of tiltgroupwrangler.py. Using the star file provided by Leginon as input, various tools allow examination and modification of the information; a slider is used to select the number of groups to determine. (b) Plot of the tilt groups found; raw tilt values are displayed as blue dots, centroids of each group as red dots, and grouping is visualized by the map colors

4. QUALITY ASSURANCE

Leginon focus on the quality of data collection by using an intelligent, rather than brute‐force, approach to selecting targets for imaging. Various hole‐finding algorithms already described in the original publication and in the on‐line manual are aimed at being selective in the targeting process. Overall our goal is to make the use of the microscope more efficient by monitoring its activity, and to improve image quality by analyzing the quality during the data collection.

4.1. Appion image rejector

Image rejector analyzes the movie drift speed, CTF estimation confidence scores, and presence of ice crystal and automatically hides movies that fail user specified criteria. The ice crystal detection checks the radial average of power spectra for the presence of a sharp peak at the expected resolution. The list can also be used to remove poor quality data to reduce storage usage.

4.2. Error and idle notifications

Despite best efforts by the microscope and camera manufacturers, as well as of the automation software developers, hardware and software problems may still arise during an automated data collection session. Microscope operators would like to monitor data collection even during off‐hours to ensure data quality. To reduce stress on the operators while also minimizing idle time, Leginon actively reports unusual events through text messages or emails. We currently use the Slack Application ¹⁰ to post these notifications to a specific channel that microscope operators subscribe to. Leginon also monitors active usage of the microscope and the camera and if the instrument is idle for a user specified set time, a Slack notification is posted. These idle notifications catch user errors as well as instrument errors that stall the workflow and have been very valuable in improving the overall efficiency of data collection.

4.3. Black stripe detector

In our experience using the Gatan K2 Summit detector, we have noticed that images will occasionally include large areas that are blank. The camera is divided into a series of 8 “stripes,” and data from each stripe is sent to the digitizer independently. Software issues, which are still not well characterized, occasionally cause one or more of these channels to lose connection, resulting in a “black stripe” of all zero data from this channel. As we operate our microscopes on a 24/7 schedule, this problem quite often occurs during hours when staff are not actively observing data collection. The “Black Stripe” node was created to monitor this problem. It actively scans all high magnification images for the presence of a completely blank channel. Upon detection, it sends an error message to Leginon, and if the Leginon error notification is turned on, all EM staff receive this message via Slack. Quite often someone is available to pause the imaging and address the issue. Fortunately, we have found that in most cases, it can be solved by using the Gatan camera software to turn the camera off and on again. Typically new gain references must then also be acquired. Both actions can be performed remotely and are generally completed within an hour. If someone does not attend to the matter the default option is to continue data collection, since occasionally the black stripes will disappear over time. Should we notice the same problem occurring on the Gatan K3 cameras, we will update this node to include these cameras.

4.4. Report generation

All information associated with data collection and analysis is stored in a database, which we query to generate and display summary tables and histograms of various parameters in the form of a “report” (Figure 4). In particular, we include imaging and defocus summary tables, histograms of ice thickness, an image of the grid atlas, and a table providing experimental details including TEM parameters (magnification, high tension, defocus range, spot size, intensity, Cs, and energy filter settings) and camera parameters (dimension, binning, exposure time, pixel size, dose rate, total dose, frame rate and total frames). We also provide Experimental Methods, Acknowledgment, and Reference sections. To include other information computed outside of Appion, we have implemented an option to import 2D classification and 3D refinement jobs from cryoSPARC into the report. Users can select which cryoSPARC server to connect to by indicating an IP address and entering the project and job ID of the 2D classification or 3D refinement jobs. The image of the 2D classes and the FSC curve computed by cryoSPARC is then also included in the report (Figure 4). These reports are a nice way to observe the overall progress of the experiment and share this information with remote users or collaborators.

4.5. Benchmarking workflows

To ensure the instrumentation and software meets the needs of a cryoEM or cryoET experiment, our workflows are subjected to regular benchmark tests. ¹⁷ While the information limit of the microscope is checked during daily routine alignments, we also run a full workflow benchmark on an annual basis and also after any major instrument upgrades or repairs. Test samples, such as apoferritin or aldolase, are used to collect data under standard conditions to identify any bottlenecks and track instrumentation performance. During these annual microscope benchmark tests, the full Leginon and Appion pipelines are used with on‐the‐fly feedback to verify that the IT infrastructure can handle continuous data collection. If issues are found, then preventative maintenance may be scheduled to maintain the capacity and efficiency of high‐end instrumentation. The results of one of these benchmark tests are shown (Figure 10).

FIGURE 10.

Annual microscope checkout results: (a) Isosurface representation of the apoferritin map using 325,636 particles from 564 movies taken on a TFS Titan Krios with Gatan GIF/K3 at 0.825 Å/pixel and 54e⁻/Ų total dose. (b) Representative 2D class averages; box size 27.4 × 27.4 nm. (c) An α‐helical segment from one β subunit (PDB: 6V21) is shown in heteroatom representation docked into the corresponding region of the reconstruction. (d) FSC Plot. (e) Guinier Plot

5. DISCUSSION

Advances in camera and microscope technology have allowed cryoEM data collection to become fairly routine. Since high‐end microscopes and cameras are both very expensive and in very high demand, we try to provide the most efficient data collection and ensure that it is of high quality, while reducing the burden on the microscopists managing the instruments. Highlighted above are some of the tools we have added in the last few years to provide new features, improve efficiency, and strengthen quality assurance. With these enhancements, combined with the underlying database that provides and organizes records of all previous data collections, we believe Leginon is a good platform to use to improve future data quality.

One of our goals for the future is to routinely collect ~10,000 movies during a 24 hr user session, including screening time. This translates to over 400 movies per hour, or <9 s per movie. There are several limiting factors that must be addressed to achieve this aim. Firstly, since grid and specimen quality are still the most variable parameters that determine the success of cryoEM projects, screening of suitable areas on the grid is still a necessary step and often the most time‐consuming period for the microscope operator. It is not unusual that 4–6 hr are spent on this process as the operator and researcher collaborate to find the best grid and the best imaging strategy. The currently typically quoted data collection speeds usually ignore the time spent on this step. It would be highly desirable to automate this process by implementing a pipeline to manage grid exchanges unattended by an operator assisted by machine‐learning algorithms to rapidly optimize targeting strategies based on previous experience on similar samples.

The core of the data acquisition speed limit is the time required to acquire and save consecutive movies. As a result, acquisition parameters can have a significant impact and for fast cameras such as the Gatan K3, can change the speed of data collection significantly. For example, the upper limit of the Gatan K3 camera data collection speed, using the SEMCCD.dll, changes depending on collection strategies, including super‐resolution mode and using a multithreading acquisition known as “early‐return” feature. A 50‐frame movie collection can cycle within 3 s using the LZW Tiff format in standard image counting mode with 39 ms per frame. In super‐resolution mode, twice the time, that is, 6 s, is required for the same operation. The gains provided by super‐resolution data quality have to be considered against the loss in number of images that can be acquire in a fixed period. Another popular strategy used on the K2 camera is the use of the SEMCCD.dll “early‐return” feature where the acquisition request thread is returned before file saving is completed. When targeting time is dominated by stage movements, this is a highly desirable method since the stage settling can occur while movies are being saved. However, we observed on the K3 camera, that even though the first acquisition of the super‐resolution movie takes only 3 s, later movies have to wait for the previous movie to be saved, paying the penalty of buffering the movie in memory, and thus the time spent on subsequent movie acquisitions becomes 8 s. This time difference and the fact that it is common now to take many beam‐image shifts targeting that requires negligible settling time before any stage movement, means that the early‐return multithreading option is now usually a disadvantage.

We also need to address some data acquisition overhead in Leginon. As a distributed and multi‐instrument system, Leginon requires more overhead in transferring information than similar packages like SerialEM, EPU, and Latitude. For example, to transfer a single Gatan K3 super‐resolution dark‐corrected image as an array (11,520 × 8,184 pixels; 180 MB) through a non‐dedicated 10G network, followed by loading and applying a gain correction, and then saving the resulting image can take up to 6 s. We work around this limitation by transferring a reduced size image. However, there are almost certainly more time savings to be discovered.

Another obvious limiting factor to data collection speed is the time required for lens normalization. Electro‐magnetic lenses and deflectors in electron microscopes exhibit strong hysteresis requiring elaborate lens normalizations to ensure reproducible results when changing settings such as magnification and spot size etc. When beam size, magnification, and defocus are all changed, as required in the MSI workflow, multiple lens normalization become necessary, and it can take 6–9 s to complete them all. These time‐consuming processes will need to be consolidated without losing the benefits to further improve data collection speed.

Currently, Leginon still has a 2–3‐s overhead, in the best case, from each beam‐image shifted movie as a result of calibration queries, calculating the required changes, changing the instrument parameters, and waiting for instrument stabilization. The rate of data collection, including overhead from autofocusing and nitrogen filling averages between 240–400 movies per hour. The rate is determined by the exposure time, the total number of frames and the number of beam‐image shifted images targeted per stage move.

Other factors which affect collection speed are out of control of software but could be addressed by future hardware improvements. Nitrogen refills take about 10 min every 8 hr on the TFS microscopes. A larger dewar inside the microscope would reduce this frequency. Energy filters in our experience need to have slit centering checked every 1–2 hr, which takes about 3 min. More stable filters or better room temperature control would help reduce this frequency. Dark references on the direct detectors need to be updated several times per day, and ideally this requirement could be reduced. Finally, grid quality is often not ideal. If there are many holes which are too thick or too thin, the number of good targets per stage movement is reduced, which effectively slows down the collection. We generally aim to collect a high proportion of good images rather than collecting the absolute maximum number possible and letting the user sort out the bad one's post‐collection.

CryoEM data collection automation has come a long way since the last paper describing Leginon was published in 2005. We will continue to update it to use new instruments and add features to improve the efficiency of high‐quality data collection and assessment.

AUTHOR CONTRIBUTIONS

Anchi Cheng: Methodology; software; supervision; visualization; writing‐original draft; writing‐review and editing. Carl Negro: Methodology; software; writing‐original draft. Jessica Bruhn: Data curation; validation; visualization; writing‐review and editing. William Rice: Data curation; methodology; software; validation; visualization; writing‐original draft; writing‐review and editing. Sargis Dallakyan: Software; visualization; writing‐original draft. Edward Eng: Data curation; validation; visualization. David Waterman: Software; writing‐review and editing. Clint Potter: Conceptualization; funding acquisition; project administration; resources; supervision; writing‐review and editing. Bridget Carragher: Conceptualization; funding acquisition; project administration; resources; supervision; writing‐original draft; writing‐review and editing.

Cheng A, Negro C, Bruhn JF, et al. Leginon: New features and applications. Protein Science. 2021;30:136–150. 10.1002/pro.3967