eDetect: A Fast Error Detection and Correction Tool for Live Cell Imaging Data Analysis

Summary

Live cell imaging has been widely used to generate data for quantitative understanding of cellular dynamics. Various applications have been developed to perform automated imaging data analysis, which often requires tedious manual correction. It remains a challenge to develop an efficient curation method that can analyze massive imaging datasets with high accuracy. Here, we present eDetect, a fast error detection and correction tool that provides a powerful and convenient solution for the curation of live cell imaging analysis results. In eDetect, we propose a gating strategy to distinguish correct and incorrect image analysis results by visualizing image features based on principal component analysis. We demonstrate that this approach can substantially accelerate the data correction process and improve the accuracy of imaging data analysis. eDetect is well documented and designed to be user friendly for non-expert users. It is freely available at https://sites.google.com/view/edetect/ and https://github.com/Zi-Lab/eDetect.

Graphical Abstract

Highlights

• •eDetect is a user-friendly software for live cell imaging data analysis
• •It provides an easy solution to detect and correct errors in imaging data analysis
• •It enables precise cell lineage reconstruction from long-term imaging datasets
• •It is compatible with CellProfiler and performs well in different imaging datasets

Automation in Bioinformatics; Bioinformatics; Cell Biology

Introduction

Live cell imaging has been widely used in monitoring cellular processes and dynamics at the single-cell level (Ni et al., 2018, Specht et al., 2017, White et al., 2018). During the course of these studies, large volumes of time-lapse images are often generated by automatic microscopes, which poses a challenge in quantifying cellular dynamics from these datasets with high precision (Skylaki et al., 2016). In the past, computational tools have been developed to enable automated analysis of large image datasets (Blanchoud et al., 2015, Carpenter et al., 2006, Cooper et al., 2017, de Chaumont et al., 2012, Held et al., 2010, Hilsenbeck et al., 2017, Liepe et al., 2016, Winter et al., 2016). However, the errors in the automated analyses impaired the accuracy in quantifying long-term single-cell dynamics. In most cases, fully automatic algorithms cannot guarantee 100% accuracy (Ulman et al., 2017). Even a small portion of errors can result in a much larger amount of false cell lineages (Skylaki et al., 2016). Therefore manual inspection and correction are needed (Hilsenbeck et al., 2016).

Several tools have provided functions to manually correct image data analysis results. For example, CellProfiler and qTfy allow the users to manually edit the contours of segmented objects one by one (Carpenter et al., 2006, Hilsenbeck et al., 2016). In addition, CellTracker, NucliTrack, LEVER, and tTt support manual track editing (Cooper et al., 2017, Hilsenbeck et al., 2016, Piccinini et al., 2016, Winter et al., 2016). These tools are useful, but an approach is still needed to improve the efficiency of data curation.

Here, we report an error detection and correction tool for live cell imaging data analysis (eDetect). In eDetect, we proposed a new gating approach to classify flawed segmentation and tracking results by visualizing image features in scatterplots based on principal-component analysis (PCA). As cell objects with similar appearance often aggregate in the scatterplot, we can easily detect a group of errors from voluminous time-lapse images. Similar errors can be efficiently corrected with batch operations in eDetect. In addition, eDetect uses multiple quality control steps to achieve high accuracy in the final live cell imaging analysis results. In summary, eDetect is a comprehensive software tool that integrates cell segmentation, feature extraction, cell tracking, and efficient data curation.

Results

eDetect Provides a Fast Way to Detect Segmentation Errors

We first used eDetect to perform cell segmentation on the live cell imaging data of mouse stem cells, which is a benchmark dataset (Fluo-N2DH-GOWT1) from the Cell Tracking Challenge (Maška et al., 2014, Ulman et al., 2017). The results of automatic segmentations are shown in the image display window (Figure 1A). We found that the application of automatic segmentation algorithms can result in different types of errors such as under-segmentations (multiple adjacent nuclei are labeled as one object), over-segmentations (one nucleus is divided into multiple objects), and irregular shapes. To efficiently detect these errors, eDetect provides a function of segmentation gating, which uses a scatterplot to visualize image features based on PCA. Each point in this plot represents a segmented object. As shown in Figure 1B, the correct and wrong segmentation objects are well separated and displayed as different groups. By clicking on the points in the scatterplot, the corresponding segmentation objects are highlighted in the image display window. In this way, the users can easily inspect and identify different groups of segmentation errors.

Figure 1.

Main Features of eDetect

(A) Main interface of eDetect.

(B) Segmentation gating helps the users to separate correct and incorrect segmentation objects into different groups. Representative correct and incorrect segmentation objects are shown for different groups.

(C) Batch correction of segmentation errors. Segmentation gating enables splitting a batch of under-segmentations (red points in panel B) or deleting irregular segmentations (green points in panel B) in all frames with one batch operation.

(D) Cell pair gating function in eDetect. The red polygon gate indicates the likely misassigned cell divisions in the cell tracking. An example of misassigned cell division is shown.

eDetect Can Correct a Group of Errors with Batch Operations

To further improve the efficiency of error correction, eDetect provides both manual and batch operations for splitting and deleting cell segmentations. When the same type of segmentation errors are grouped together in the PCA plot, eDetect can correct these types of errors with a batch operation after selecting them with a polygon. For example, the cluster of under-segmentations can be corrected by one click of splitting objects in eDetect (Figure 1C). Similarly, the irregular non-cell segmentations can be easily discarded with one “Delete objects” operation.

Multiple Quality Control Steps Improve the Accuracy of Live Cell Imaging Data Analysis

Although segmentation gating can assist the users to correct segmentation errors efficiently, some mistakes may still exist. Therefore eDetect provides additional steps to improve the accuracy of data analysis. To illustrate this feature, we applied eDetect to analyze a live cell imaging dataset (HaCaT-FUCCI) from human HaCaT cells that stably express the CFP-H2B nuclear marker and mCherry-Geminin FUCCI cell cycle indicator proteins (Sakaue-Sawano et al., 2008).

To improve the precision of cell tracking, eDetect uses cell pair gating to visualize identified cell divisions in another PCA-based scatterplot. eDetect automatically locates and marks the corresponding pair of cells in the original image when the user points at a cell division pair (Figure 1D). By inspecting the highlighted pairs of cells, the users can find and correct misassigned cell divisions.

As the last quality control module in eDetect, cell lineages display window assists the users to identify missed errors from previous steps. This is implemented by visualizing different measurements (e.g., nuclear area) of the objects in cell lineages with a heatmap. In general, nuclear area changes gradually under normal cell growth and only shows abrupt changes during cell divisions. The users can easily find abnormalities such as abnormal cell divisions, abrupt changes of nuclear sizes, and incomplete cell lineages (Figure 2). eDetect can assist us to check and correct these abnormalities in convenient ways. For example, eDetect has an automatic outlier detection function to mark abrupt changes within cell lineage branches. In addition, it can generate a synchrogram (a sequence of images of the selected cell lineage), which facilitates the check on the correctness of cell lineages (Sigal et al., 2006). After multiple error correction steps, we can accurately quantify the dynamics of the cell cycle sensor protein mCherry-Geminin in individual HaCaT cells (Figure S1).

Figure 2.

Cell Lineages Display in eDetect Improves the Accuracy of Live Cell Imaging Analysis Results

eDetect enables the detection of other overlooked errors with heatmap visualization. Two examples of errors in cell lineages results are zoomed in and shown on the right: (i) abnormal cell divisions (the cell division between frame #48 and #49 is misassigned), and (ii) outliers (the highlighted nucleus in frame #203 has an inaccurate segmentation).

Compatibility of eDetect with CellProfiler

eDetect can be used together with CellProfiler to maximize its interoperability. The users can first perform image preprocessing and segmentation with CellProfiler and then import the results into eDetect for error detection and correction. As CellProfiler provides advanced algorithms and pipelines for image analysis, the compatibility of eDetect with CellProfiler will improve the broad application of eDetect for live cell imaging analysis.

Evaluation of eDetect Performance to Detect and Correct Errors in Live Cell Imaging Data Analysis

To evaluate the effectiveness and time cost of eDetect in error detection and correction, we first performed automatic data analysis in CellProfiler (CellProfiler*) and eDetect (eDetect*) without any corrections. In parallel, we imported the automatic analysis results from CellProfiler and then performed additional error correction with eDetect (CellProfiler+). In addition, we implemented automatic image analysis and error correction with eDetect (eDetect). Performance was quantified based on segmentation accuracy (SEG), tracking accuracy (TRA), complete tracks (CT) and recall of complete lineages (RCL). The RCL score measures the fraction of cells (including their descendants) that are correctly tracked from the first frame to the last frame of the movie. A detailed description of the performance metrics is provided in the Transparent Methods. The performance scores were calculated for four datasets, consisting of an imaging dataset HaCaT-FUCCI and three training datasets from the Cell Tracking Challenge.

As shown in Figure 3, eDetect can improve the performance of data analyses on different datasets by implementing error correction (comparing eDetect* and eDetect). The average values of SEG and TRA for the tested four datasets are increased by 0.04 and 0.10, respectively. More importantly, the average values of CT and RCL are increased to a larger extent, by 0.55 and 0.38, respectively. Similar performance augmentation is observed by comparing CellProfiler* and CellProfiler+. The improvement of CT and RCL values is more apparent when a long-term live cell imaging dataset is analyzed. For the 2-day imaging dataset HaCaT-FUCCI, although automatic algorithms in CellProfiler* and eDetect* reached very good SEG and TRA scores, the corresponding CT and RCL values are very low. This result indicates that a small portion of segmentation and tracking errors occurring in the automatic analysis can lead to a lot of errors in CT, which need to be corrected to reach valid conclusions. By performing error detection and correction in eDetect, the RCL value was increased from 0.111 to 1 with about 9 min of error correction on HaCaT-FUCCI dataset. The RCL value is especially important to draw biological conclusions when a reconstruction of complete cell lineages is needed. Therefore, eDetect is very useful especially for long-term imaging data analysis.

Figure 3.

The Performance of Error Detection and Correction Function in eDetect

The performance values were obtained with or without additional error correction steps in eDetect. CellProfiler*: automatic pipeline analysis with CellProfiler. CellProfiler+: automatic pipeline analysis was performed with CellProfiler first and then the results were imported to eDetect and additional error detection and correction was performed. eDetect*: automatic analysis with eDetect. eDetect: automatic analysis plus error detection and correction in eDetect.

In general, eDetect is efficient and has a low time cost (Figure 3). The execution time (TIM) for the automatic analysis step is less than 1 min for six tested imaging videos running on windows with eight CPU cores (Intel Core i7-6,700 Processor, 3.4 GHz). The time used in the error detection and correction steps (TIM_EDC) depends on the performance of automatic segmentation algorithms and the size of the image dataset. The error correction steps took about 10–90 min for six of the analyzed videos with total number of frames ranging between 65 and 206. For the video #2 of Fluo-N2DH-SIM+ dataset, the performance measures are relative poor when automatic analyses were performed in both CellProfiler* and eDetect*. This might be caused by the artificial noise introduced in this simulated dataset. In this case, several hours of manual correction of segmentations are needed to substantially improve the accuracy of data analysis.

We next compared eDetect with the top-three performing methods from Cell Tracking Challenge using three benchmarking datasets (Figure 4). In terms of the SEG and TRA scores, eDetect has comparable performance to those of top-performing methods. It is important to note that, with the function of error detection and correction, eDetect could achieve the best performance in the measurements of complete tracks and recall of complete lineages. These results highlight the importance of error correction in long-term live cell imaging data analysis.

Figure 4.

Comparison of eDetect with Top Three Performing Methods from the Cell Tracking Challenge

For each dataset, the performance scores were computed by the average of scores from two training videos. The color code below indicates the top three performing methods submitted by the corresponding participants in Cell Tracking Challenge.

Discussion

The quantification of long-term single-cell dynamics is crucial for understanding cell signaling dynamics and cell fate in development. Although improving the accuracy of automatic algorithms is very important for live cell imaging data analysis (Maška et al., 2014, Ulman et al., 2017), an efficient curation tool is also urgently needed because a small fraction of errors, especially occurring earlier in the videos, could lead to a large fraction of errors in the complete cell lineages. In eDetect, we proposed a PCA-based approach to detect and correct errors, which substantially reduces the time for manually pinpointing errors from massive image datasets. In addition, eDetect can help us to correct or remove errors with one-click batch operations. Last, but not least, multiple quality control steps ensure a high precision in the final imaging analysis results.

Despite its effectiveness, segmentation gating in eDetect is inherently limited: it requires the users to customize PCA input variables. In practice, it is difficult to completely separate correct and incorrect segmentations at once. The users can achieve accurate segmentations by applying multiple gating steps with different PCA inputs even if each round of gating is not ideal. Moreover, overlooked errors can still be found in subsequent quality control steps. On the other hand, supervised learning methods have been previously used for phenotypic analysis of high-content imaging data (Piccinini et al., 2017, Ramo et al., 2009, Smith et al., 2018). In the future, these methods could be potentially used in eDetect to detect segmentation errors.

Here, we demonstrated eDetect's capability to detect and correct errors in live cell imaging data analysis. Making sense of large imaging datasets is a demanding challenge that requires powerful and efficient computational tools. The development of eDetect will be valuable for improving the precision of live cell imaging data analysis.

Limitations of This Study

Current release of eDetect does not provide multiple algorithms for segmentation and tracking. The performance of automatic analysis in eDetect may not be suitable for certain datasets. The compatibility of eDetect with CellProfiler offers the choice of various segmentation and tracking methods from CellProfiler, but it requires additional pipeline setup work. In the future, eDetect could be improved by integrating more established methods for segmentation and tracking. In addition, there is also a strong demand for the analysis of high-throughput time-lapse datasets. Although eDetect can in principle handle these datasets, it would require a large amount of manual labor. To address this challenge, it would be helpful to apply machine learning methods for error detection and correction.

Methods

All methods can be found in the accompanying Transparent Methods supplemental file.

Published: March 29, 2019