DeepCell Kiosk: scaling deep learning–enabled cellular image analysis with Kubernetes

Abstract

Deep learning is transforming the analysis of biological images, but applying these models to large datasets remains challenging. Here we describe the DeepCell Kiosk, cloud-native software that dynamically scales deep learning workflows to accommodate large imaging datasets. To demonstrate the scalability and affordability of this software, we identified cell nuclei in 10⁶ 1-megapixel images in ~5.5 h for ~US$250, with a cost below US$100 achievable depending on cluster configuration. The DeepCell Kiosk can be downloaded at https://github.com/vanvalenlab/kiosk-console; a persistent deployment is available at https://deepcell.org/.

While deep learning is an increasingly popular approach to extracting quantitative information from biological images, its limitations substantially hinder its widespread adoption. Chief among these limitations are the requirements for expansive sets of training data and computational resources. Here we sought to overcome the latter limitation. While deep learning methods have remarkable accuracy for a range of image-analysis tasks, including classification¹, segmentation^2–4 and object tracking^5,6, they have limited throughput even with GPU acceleration. For example, even when running segmentation models on a GPU, typical inference speeds on megapixel-scale images are in the range of 5–10 frames per second, limiting the scope of analyses that can be performed on images in a timely fashion. The necessary domain knowledge and associated costs of GPUs pose further barriers to entry, although recent software packages^7–11 have attempted to solve these two issues. While cloud computing has proven effective for other data types^12–15, scaling analyses to large imaging datasets in the cloud while constraining costs is a considerable challenge.

To meet this need, we have developed the DeepCell Kiosk (Fig. 1a). This software package takes in configuration details (user authentication, GPU type, etc.) and creates a cluster on Google cloud that runs predefined deep learning–enabled image-analysis pipelines. This cluster is managed by Kubernetes, an open-source framework for running software containers (software that is bundled with its dependencies so it can be run as an isolated process) across a group of servers. An alternative way to view Kubernetes is as an operating system for cloud computing. Data are submitted to the cluster through a web-based front-end, a command line tool, or an ImageJ plugin. Once submitted, they are placed in a database where the specified image analysis pipeline can pick up the dataset, perform the desired analysis and make the results available for download. Results can be visualized by a variety of visualization software tools^16,17.

Fig. 1 |. Architecture and performance of the DeepCell Kiosk.

a, Data flow through the DeepCell Kiosk. Images submitted to a running Kiosk cluster are entered into a queue. Once there, images are processed by objects called consumers, which execute all the computational operations in a given pipeline. Consumers perform most computations themselves but access deep learning by submitting data to TensorFlow Serving and capturing the result. This separation allows conventional operations and deep learning operations to occur on different types of nodes, which is essential for efficient resource allocation. After processing, the result is returned to the queue, ready to be downloaded by the user. b, Left, benchmarking of inference speed demonstrating scaling to large imaging datasets (see the Supplementary Information for details). Throughput is ultimately limited by data transfer speed. right, an analysis of cost demonstrating affordability. network costs are incurred by inter-zone network traffic in multi-zone clusters, which are more stable and scale faster than their single-zone counterparts. This cost can be avoided when users configure a single-zone cluster. Benchmarking was performed once for 1 million (M) image runs and in triplicate otherwise. Error bars, when present, represent the s.d. of replicate measurements. c, The DeepCell Kiosk enables construction and scaling of multi-model pipelines. For example, a live-cell imaging consumer can access deep learning models for both segmentation and tracking. The live-cell imaging consumer places the frames for a movie into the queue for a segmentation consumer, where they are processed in parallel. Once segmented, the images are processed by a tracking consumer to link cells together over time and to construct lineages. The results, which consist of label images of segmented and tracked cells as well as a JSOn file describing mother–daughter relationships, are uploaded to a cloud bucket, from which they can be downloaded by the user.

To ensure that image analysis pipelines can be run efficiently on this cluster, we made two software design choices. First, image analysis pipelines access trained deep learning models through a centralized model server in the cluster. This strategy enables the cluster to efficiently allocate resources, as the various computational steps of a pipeline (deep learning and conventional computer vision operations) are run on the most appropriate hardware. Second, the DeepCell Kiosk treats hardware as a resource that can be allocated dynamically, as opposed to a fixed resource. This conceptual shift means that cluster size scales to meet data analysis demand: small datasets lead to small clusters, while large datasets lead to large clusters. This feature, which has previously been demonstrated for other biological data types¹³, is made possible by our use of Kubernetes, which enables us to scale analyses to large datasets, reducing analysis time while constraining costs (Fig. 1b). A full description of the DeepCell Kiosk’s software architecture, scaling policies, and benchmarking is provided in the Supplementary Information.

Given the wide range of image analysis problems that deep learning can now solve, we designed the DeepCell Kiosk with the flexibility to work with arbitrary collections of deep learning models and image analysis pipelines. Pipelines can exploit multiple deep learning models—and even other pipelines—while still leveraging the scaling ability provided by Kubernetes. For example, we constructed a pipeline (Fig. 1c) that pairs deep learning models for cell segmentation and cell tracking⁵ to analyze live-cell imaging data. Additional examples are described in the Supplementary Information, and documentation on creating custom pipelines is available at https://deepcell-kiosk.readthedocs.io.

In conclusion, the DeepCell Kiosk enables cost-effective scaling of customizable deep learning–enabled image analysis pipelines to large imaging datasets. The dynamic nature of our scaling enables individual users to process large datasets. Further, it allows a single cluster to serve an entire community of researchers—much like BLAST¹⁸ has rendered sequence alignment accessible to the scientific community through a web portal. At the cost of network latency, software ecosystems like ImageJ can access complete deep learning–enabled pipelines hosted by such a cluster through a plugin. While the data analyzed here came from in vitro cell culture, our emphasis on deep learning makes it possible for our software to analyze tissue data, including spatial genomics data¹⁹. We believe that, by reducing costs and time to analysis, this work should accelerate the rate of biological discovery and change the relationship that biologists have with imaging data. Further, by focusing on deployment, this work highlights aspects of the performance of deep learning models beyond accuracy (such as inference speed) that substantially impact their utility. Finally, this work highlights the growing definition of the term ‘software’. Just as deep learning has made data a part of the software stack, Kubernetes can do the same for hardware. Jointly developing data, code, and computing should substantially improve the performance of scientific software crafted in the era of Software 2.0.

Methods

Data collection.

Cells were imaged with either a Nikon Ti-E or Nikon Ti2 fluorescence microscope controlled by either Micro-Manager or Nikon Elements. Data were processed using the scientific computing stack for Python.

Reporting Summary.

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Data availability

All data that were used to generate the figures in this paper are available at https://deepcell.org/data and at https://github.com/vanvalenlab/deepcell-tf under the deepcell.datasets module.