Data-driven microscopy allows for automated context-specific acquisition of high-fidelity image data

Oscar André; Johannes Kumra Ahnlide; Nils Norlin; Vinay Swaminathan; Pontus Nordenfelt

doi:10.1016/j.crmeth.2023.100419

. 2023 Mar 6;3(3):100419. doi: 10.1016/j.crmeth.2023.100419

Data-driven microscopy allows for automated context-specific acquisition of high-fidelity image data

Oscar André ¹, Johannes Kumra Ahnlide ¹, Nils Norlin ^2,³, Vinay Swaminathan ⁴, Pontus Nordenfelt ^1,^5,^∗

PMCID: PMC10088093 PMID: 37056378

Summary

Light microscopy is a powerful single-cell technique that allows for quantitative spatial information at subcellular resolution. However, unlike flow cytometry and single-cell sequencing techniques, microscopy has issues achieving high-quality population-wide sample characterization while maintaining high resolution. Here, we present a general framework, data-driven microscopy (DDM) that uses real-time population-wide object characterization to enable data-driven high-fidelity imaging of relevant phenotypes based on the population context. DDM combines data-independent and data-dependent steps to synergistically enhance data acquired using different imaging modalities. As a proof of concept, we develop and apply DDM with plugins for improved high-content screening and live adaptive microscopy for cell migration and infection studies that capture events of interest, rare or common, with high precision and resolution. We propose that DDM can reduce human bias, increase reproducibility, and place single-cell characteristics in the context of the sample population when interpreting microscopy data, leading to an increase in overall data fidelity.

Keywords: data-driven microscopy, event-driven microscopy, adaptive feedback microscopy, high-content screening, automation, cell migration, host-pathogen interactions, image analysis, data analysis

Graphical abstract

Highlights

•
We introduce a data-driven approach to microscopy
•
The method enables high-fidelity imaging of relevant context-specific phenotypes
•
It enables curation of population-wide estimates by high-fidelity verification
•
It can automatically target multi-labeled cells, cell migration, and infection events

Motivation

Even state-of-the-art implementations of high-content screening and event-driven feedback microscopy still fall short in placing high-resolution single-cell data in a population-wide context. This is in contrast to other single-cell techniques, such as flow cytometry, where it is natural to assess the whole sample population. Even when overview sample information is theoretically available, such as with high-content imaging, the information is not used to control the acquisition of images or resample cells of interest at higher resolution. We sought to provide a general solution that combines image analysis and acquisition, allowing a data-driven approach to microscopy where acquisition and content analysis is coupled to achieve higher fidelity.

André et al. present data-driven microscopy (DDM), an approach that uses real-time characterization of a microscopy sample to enable data-driven high-fidelity imaging. DDM can reduce human bias, increase reproducibility, and place single-cell characteristics in the context of the sample population when interpreting microscopy data.

Introduction

Today, researchers can collect thousands of images and accumulate millions of data points on cellular processes in just a few hours. However, microscope image acquisition is still primarily based either on manual decisions by the end user or brute-force approaches, such as slide imaging, where a whole sample is scanned. Coupled with automated image analysis pipelines, high-content screening (HCS) approaches enable researchers to extract rich and unbiased information from datasets that would otherwise overwhelm any human operator.¹^,²^,³^,⁴^,⁵^,⁶^,⁷^,⁸^,⁹ HCS has become a main method for assessing large quantities of single-cell data in a population-wide context.¹⁰^,¹¹^,¹²^,¹³^,¹⁴ However, in most high-resolution microscopy applications, images are typically only acquired from selected points, lacking population context and risk of bias, especially since data selection is often left to human operators. Even more demanding is ensuring that relevant data are acquired during live imaging, such as when recording high-resolution cell migration data or capturing host-pathogen interactions.

Integrating image analysis, including machine-learning classification algorithms, into the data selection has improved the efficiency and reduced the overall bias of image acquisition. These solutions, referred to as event-driven feedback microscopy or intelligent microscopy, allow for the high-throughput targeted imaging of cells of interest in high-spatiotemporal settings.¹⁵^,¹⁶ There are both proprietary (Nikon, Zeiss, Leica, Olympus) and open-source solutions available for controlling microscope hardware¹⁷^,¹⁸^,¹⁹ and doing feedback microscopy¹⁶^,²⁰^,²¹^,²² where the microscope can be programmed to respond to image content in real time. Such decisions are typically based on predetermined criteria of image characteristics through image segmentation and are not adaptive to the data distribution of a particular sample population. In parallel to hardware control development, image analysis software has continuously developed. The Open Microscopy Environment (OME²³) curates the BIO-Formats²⁴ software package, allowing images to be accessed in a standardized manner, and there are both increasing capabilities of proprietary software and languages (NIS Elements, Leica, MATLAB) and open source-based options (Carpenter et al.,²⁵ Schneider et al.,²⁶ Allan et al.,²⁷ Python, Julia). Accompanying image analysis is typically applied post-acquisition and not to enhance image acquisition. However, there are no generally accepted strategies for how feedback-based microscopy could efficiently integrate image analysis and hardware control in a way that allows the information from image analysis pipelines to inform and drive hardware control. Importantly, no solutions are available that focus on targeted image acquisition based on analyzing population characteristics to find relevant and representative objects or events.

Here, we identify and adopt an analogous solution to this microscopy problem from flow cytometry where targeted cells or objects based on overall feature distributions are placed in a population-wide context.²⁸^,²⁹^,³⁰ Imaging flow cytometers and image-based cell sorting show the strength of such an approach for image-based data³¹^,³²^,³³ while lacking the resolving power and flexibility of advanced light microscopy. We introduce a general microscopy framework, data-driven microscopy (DDM), which uses population-wide data to improve and control microscopy and enables cross-experiment image data validation. A data-independent acquisition phase performs high-throughput imaging and generates a population-wide phenotype assessment. These data include the relative coordinates of each data point for the system, which feeds into the automated data-dependent acquisition of selected phenotypes. Combining the two datasets yields population-wide data with high-fidelity object characterization data. DDM also inherently includes what constitutes an objective representation of the sample population. The general DDM framework applies to any motorized microscope, and all steps can be fully automated, including phenotype targeting. We have implemented a data link for Nikon microscopes, with the possibility of expanding to other microscopes using micromanager or vendor-specific connection solutions for microscope-server communication. As a proof of principle, we apply DDM to target and enhance HCS quantification of multi-labeled cells, carefully assess cancer cell migration phenotypes using live feedback microscopy, and monitor cell populations for bacterial infection events to collect high-resolution data, in all cases without human input with regards to data acquisition. Placing the combined data in a population context yields a more robust, reproducible, and efficient method for selecting and acquiring data in microscopy.

Results

DDM as an approach for automated targeted image acquisition of relevant data

DDM was built around two imaging strategies interconnected through a shared server database. First, a data-independent acquisition (DIA) step (Figure 1.1) aims at providing complete population characterization at a single-cell level. This is achieved by coupling the acquisition pipeline to real-time analysis of images acquired by the microscope. The resulting data are continuously stored in a database (Figure 1.2). From this database, the user can define cells of interest by exploring and filtering on features (gating) either post-acquisition or in real time. The characterization and gating of cells of interest feed into the second imaging strategy, data-dependent acquisition (DDA), which performs targeted high-fidelity imaging of the cells of interest (Figure 1.3; for the practical implementation, see Figure S1). The fidelity is increased by either increasing the spatiotemporal resolution or by imaging with a different modality. DDA can be performed subsequent to DIA, where gating is done in real time using stored stage coordinates or with gates from a separate DIA experiment. The data are returned and analyzed within the database, resulting in high-fidelity data in the context of the entire sample population (Figure 1.4). With access to the combined datasets, the user can further explore the data in their relative contexts and uncover new insights that motivate further experiments (Figure 1.5). Taken together, a synergistic relationship can be established between the two strategies, increasing the fidelity as well as the relevance of the data by placing phenotypes of interest in their population context.

Data-driven microscopy as an approach for automated targeted image acquisition of relevant data

Data-independent acquisition (DIA) aims at characterizing the full sample population in real time (1). Generated single-cell data are continuously stored in a database, allowing for filtering of data and single-cell targeting (2). Predefined or generated targeting criteria are fed back to the microscopy for data-dependent acquisition (DDA; 3). The high-fidelity data are stored in a database, which is interconnected to the DIA database (4). The interconnection of DIA and DDA databases allows for high-fidelity data to be placed in the sample-wide context, allowing for potential new insights that motivate future experiments (5).

DDM enables automated high-fidelity sampling of targeted multi-labeled subpopulations

In HCS applications, images are normally acquired at a fixed low magnification, with object classification only sometimes verified at higher magnifications and, in that case, typically by manual subsampling. To test whether DDM could enable automated high-fidelity HCS, we developed a data-driven enhanced variant whereby automated subsampled high-magnification data could curate low-magnification population-wide data (Figure 2A). As a test case, we set out to assess transfection efficiency in cells transfected with multiple fluorescent protein-based plasmids (actin [Lifeact-Crimson], tubulin [alpha-tubulin Scarlet], and integrin alpha-5 [GFP] and beta-1 subunits). Integrins will only fold if both subunits have been successfully expressed. The samples were fixed before we acquired the entire cell population of 8 wells in two separate experiments (Figure 2B) and characterized the signal of each plasmid at a single-cell level. The reproducibility between sample wells was evaluated using population feature distributions (Figure S2). Figure 2C shows an example field of view containing cells with different levels of plasmid expression. Each cell (N = 109,065) was categorized (non-, single, double, or triple transfected) according to a signal threshold per channel (Figure 2D). This categorization was based on predetermined thresholding criteria generated by curating randomized cells. Following the DIA procedure, an equal number of cells (N = 50) were sampled from each category per well and experiment (total N = 3,200) and automatically targeted for imaging in DDA (Figure 2E). Figure 2F shows an example of a targeted triple-transfected cell in high resolution (60× magnification); in this case, the same cell as shown in Figure 2C.

The resulting high-resolution data from DDA were used to test the accuracy of the targeting criteria for each group in DIA. The distribution of cells per transfection category assigned during the DIA was heavily skewed toward non-transfected (41.9% ± 1%) or single transfected (53.5% ± 1%). Only small fractions of cells were identifiable as expressing two (4.6% ± 0.1%) or more (<1%) (Figure 2G). A confusion matrix between curated cells (N = 278) from the DDA and the predicted category of the DIA revealed an overly conservative estimation of the transfection categories (Figure 2H). The prediction of single-transfected cells in DIA was inaccurate, as non-transfected cells were confirmed to be single transfected 57% of the time compared with the predicted 29%. The DIA analysis module accurately predicted those believed to be triple-transfected cells (96%) but also misclassified triple transfected as double transfected (51%) or single transfected (23%), leading to a large proportion of false negatives. With the confusion matrix, the inaccuracy in classification during the DIA could be corrected (Figure 2I).

The transfection data illustrate that low-magnification DIA (comparable to HCS) is useful for the initial acquisition and targeting of population data but with clear misclassification of image data. However, after the DDM-enhanced analysis with high-resolution DDA data, the original DIA data could be corrected, increasing data fidelity and hit ratio multiple folds, resulting in a large (N = 109,065), high-quality dataset. Thus, we conclude that DDM can accurately characterize the distribution of multi-labeled cells through DIA, like traditional HCS, and subsequently perform high-magnification targeted imaging of subpopulations through DDA, leading to higher-fidelity HCS through data-driven enhancement.

DDM allows for the identification and acquisition of bacteria-cell interactions

When doing live imaging of host-pathogen interactions, a major issue is to predict where in the sample such events might happen, especially as many of the interactions might be rare events. To validate the live cell and the capturing of rare events capabilities of DDM, we developed a data-driven approach to target live single-cell interactions between cells and bacteria. The goal was to be able to capture high-resolution time-lapse data of bacteria interacting with human cells. We used HeLa cells expressing mScarlet-Lifeact and the bacterial pathogen Yersinia pseudotuberculosis expressing GFP. This bacterium injects actin-modulating toxins that can disrupt the host cell actin cytoskeleton,³⁴ an effect that is possible to visualize and analyze using high-resolution live imaging. Interactions were categorized according to a defined minimum distance between each bacterium and its closest neighboring cell (Figure 3A), which was persistent across frames. The distance was empirically determined (5 pixels, ∼3.7 μm) when establishing this DDM pipeline and is about 2–3 times the size of a Yersinia bacterium (1–2 μm in length). Through DIA, the sample populations of HeLa cells (N = 16,988) and Y. pseudotuberculosis (N = 990) and their interactions (N = 120) were monitored (Figure 3B). The bacteria can be seen spreading across the sample over time (green area in Figure 3B) and then becoming associated with cells (red dots in Figure 3B). It is expected that some of the spread comes from dividing bacteria, with Yersinia typically dividing every hour under these conditions. When viewed at high magnification, all these identified interactions indeed turned out to be true persistent host-bacteria events that could be used for high-resolution time-lapse acquisition, representing a hit rate of 100%. Comparatively, a traditional approach for acquiring high-resolution cell-bacteria interactions based on manual monitoring would achieve an estimated hit rate of 1.4% (with DDM capable of capturing all events) in the same experiment based on the fraction of interacting bacteria and cells in the sample. In addition, population data on the sample would not be available with standard approaches, making assessing the overall association inaccessible. Figure 3C shows three examples of bacteria-cell interactions in DIA (10× magnification; images taken every 10 min). Upon meeting the predefined number of interactions in DIA, bacteria-cell interactions were automatically targeted for high-fidelity live-cell imaging in DDA (n experiments = 2, N total = 120). Figure 3D shows three examples from DDA where also the effects of Yersinia-induced actin disruption are visible (note, the same cells as in Figure 3C; see Video S1). Figure 3E shows a representative outcome of a bacteria-cell interaction over time. Thus, this module demonstrates the adaptability and capacity of DDM to acquire population-wide data and the subsequent targeting of temporally dependent and rare events for live-cell imaging over time.

DDM allows for the identification and acquisition of temporally dependent rare events

(A) A live-cell imaging system of HeLa mScarlet-LifeAct cells and *Yersinia pseudotuberculosis* was set up. Attachment between bacteria (N = 990) and cells (N = 16,988) was monitored and, upon identification, targeted for DDA (N = 120).

(B) Monitoring of bacteria and interactions over time revealed the interaction to be sparse across the sample.

(C) Example images of three attached bacteria to cells imaged in DIA (10× magnification; scale bars are 25 μm).

(D) Example images of the same three cells targeted and imaged in DDA (60× magnification; scale bars are 10 μm).

(E) *Y. pseudotuberculosis* disrupt host cell function by interfering with actin filament formation through the injection of multiple toxins through its type III secretion system. The interaction monitored over time with images taken every 2 min reveals a clear effect on the host cell actin morphology and dynamics by the bacteria.

Data are based on two independent experiments. Out-of-focus light was removed in post-processing using NIS Elements Clarify.ai. N represents number of cells.

Video S1. Examples of captured infection events, related to Figure 3

Montage of infection events captured through DDM (blue - DNA, orange - actin, green - bacteria). Out-of-focus light was removed in post-processing using Nis elements Clarified.ai.

Download video file^{(8.6MB, mp4)}

DDM allows for the characterization and high-spatiotemporal imaging of migratory subpopulations

To further test the capacity of DDM on live feedback microscopy, we decided to study cancer cell migration at high spatiotemporal resolution. An adaptive feedback microscopy module was developed and implemented with DDM (Figure 4A). Through DIA, H1299 mKate-paxillin cells (N = 24,940) were imaged every 10 min for 6 h, and single-cell migration was characterized. The parameter space was collapsed using uniform manifold approximation and projection (UMAP) analysis to categorize the migration modes. To investigate this space, single-cell migration tracks from different regions were overlayed onto the UMAP (Figures S3A and S3B). As expected, a wide range of phenotypes in terms of migratory behavior were present. To investigate further, we grouped the cells into their top and bottom percentiles (>90^th and <10^th, respectively) in terms of mean migration speed. This separation revealed distinct phenotypes (Figure S3C), with the bottom percentile containing different migratory behaviors compared with the longer and generally smooth tracks in the top percentile (Figure 4B). Grouping the cells based on their meandering index also showed a distinction between the respective groups. In the bottom percentile, cells were migrating around their original starting position, some of them in long smooth tracks. The top percentile migrates greater distances than the bottom percentile and in straighter paths (Figure 4C). So far, analysis of the low-resolution DIA data shows a wide heterogeneity in the migratory behavior of the cells in the population.

To understand cell migratory behavior in more detail, we decided to target cells for data-driven imaging at high magnification. Through DIA, cells were imaged every 10 min for 100 min total, and single-cell migration was characterized. Figure 4D shows cells successfully tracked for the duration of the DIA. Since mean speed as a metric previously displayed distinct migratory phenotypes in the fastest and slowest (>90^th and <10^th percentiles, respectively) migrating extremes (Figure 4B), we decided to sample (N = 100) the fastest cells for DDA. Figure 4E shows a targeted fast-migrating cell, in this case the same cell as one of the highlighted cells in Figure 4D (see Videos S2 and S3). In DDA, images were taken every 3 min for 3 h, and the speed was monitored. Because both DIA and DDA were performed on the same system, the continuous tracking of cells between modalities could easily be performed (Figure 4F). This could also be done on neighboring cells next to the targeted cells, which increased the overall dataset of fast- and slow-migrating cells to 391 (165 and 226, respectively; Figure 4G). The final dataset also included 958 unique observations of cells in the intermediate groups (10^th–90^th percentiles), which further increased the total sample size to 1,336. The increased spatial and temporal resolution of the DDA data resulted in a much more detailed tracking of cell migration, clearly seen in the increased variance of both groups in DDA (Figure 4H). In addition, although we see a decrease in the mean speed of the fast-migrating cells in DDA, they were continuously faster than the slow-migrating cells, indicating that most of the cells in each targeted group maintain their speed profile over the duration of the experiment. The DIA data confirm an overall regression to the population mean over time in terms of migration speed while maintaining the speed group allocation (Figure S3D). Taken together, DDM enables monitoring cells in a population-wide context in combination with an automated targeted high-spatiotemporal acquisition, resulting in increased overall fidelity and integrity of the data.

Video S2. Examples of fast-migratory cells, related to Figure 4

Montage of fast migratory cells captured through DDM (DIC + blue - DNA). Out-of-focus light was removed in post-processing using Nis elements Clarified.ai.

Download video file^{(2.8MB, mp4)}

Video S3. Examples of slow-migratory cells, related to Figure 4

Montage of slow migratory cells captured through DDM (DIC + blue - DNA). Out-of-focus light was removed in post-processing using Nis elements Clarified.ai.

Download video file^{(2.6MB, mp4)}

Discussion

We propose a data-centric view for image acquisition, where the properties of the data distributions of a given sample should control where and how image acquisition should be made. Through a data-driven approach, the data can influence the acquisition decision without prior knowledge about the sample and also be able to curate results between and across experiments. To achieve this, we developed a methodology for image data acquisition and selection, DDM, which uses a combination of automated multimodality imaging. Through DIA, we acquire population-wide data and profile single-cell phenotypes in the context of the population distribution. Using these data, we can acquire further data with other imaging modalities (DDA) that provide additional information about targeted phenotypes of interest. DDM is synergistic in that DDA complements DIA data, making both more relevant, leading to enhanced fidelity of the combined image data.

With DDM, we have implemented a data-centric approach to image acquisition. The initial scan to collect overview data also leads to one of the most obvious benefits of the approach in that it provides coordinates and basic features of the objects in the sample. This basic sample overview data allow for real-time analysis of population context based on objective data rather than the subjective experience of the microscopist. The most similar example is from how flow cytometry is typically used, with an initial sample overview based on forward- and side-scatter data.³⁵ Also similar is how that population data can then be filtered (gating) on additional channels to decide which data should be collected. Microscopy has the advantage over flow cytometry in that it can also offer spatial information on the cells being imaged, allowing for more characteristics to filter on. Imaging flow cytometry³¹ demonstrates many of these benefits but lacks the spatial resolution and ability to study live cells in an environmental context, such as relative localization to other cells, migrating cells, or multi-cellular environments. DDM also inherently provides information on what constitutes representative objects in the sample population. Cells can be considered representative when they are placed in the context of the population feature distribution; it also opens the possibility to do analysis and targeting of outlier cells, as we did with the cell migration analysis, where we studied the slowest- and fastest-migrating cells in more detail, demonstrating the versatility and power of DDM.

We have shown that DDM enhances existing advanced imaging pipelines and increases the throughput of biologically relevant data. The novelty of DDM compared with traditional HCS methods¹^,¹¹^,¹³^,³⁶ is that the image acquisition can easily be adapted according to the questions being asked and based on the population characteristics. The improvement is especially true for high-resolution applications and is exemplified through the transfection experiment we performed. If a traditional HCS approach had been used, it would have resulted in a large, low-quality dataset with misrepresented transfection distribution. If a traditional high-magnification approach had been used, it would have resulted in a small sampling of high-quality data with an uncertain transfection distribution. For live interaction studies, using DDM leads to a dramatic increase in hit rate for high-resolution data collection compared with traditional approaches and the valuable addition of population-wide context compared with other feedback microscopy solutions. DDM uses the best aspect of each modality and controls acquisition in an automated and efficient manner, resulting in large, context-aware datasets with high fidelity.

DDM allows for significant benefits post-acquisition compared with the current state of the art. To increase introspection and reproducibility, DDM inherently logs all operations performed, as well as the state of the running experiment, providing clear status updates to the user. Since DDM essentially provides a population fingerprint for each experiment, it makes it less prone to human error and bias and could provide a basis for automated curation of large datasets,³⁷ making it a powerful approach for machine-learning approaches.³⁸^,³⁹^,⁴⁰^,⁴¹ It should be possible to map the data collected through DDA backward onto the DIA data. In the future, this could potentially remove the need for DDA for specific applications if sufficient previous data have been acquired. A similar development has been shown within the mass spectrometry field⁴² where DIA data can be mined after acquisition to provide high-quality data for new questions.

In this work, as proof of principle, we have established the DDM framework on Nikon microscopes. The framework is compatible with any microscope as long as the controlling software can send images to the server or invoke external programs to achieve this. The ability to fully interoperate the popular programming language Python allows DDM to be easily integrated into other already existing workflows and methods. In summary, we believe that DDM offers a useful framework for a more robust and unbiased acquisition of high-fidelity microscopy data.

Limitations of the study

To promote the adaptation and integration of our method by other researchers, we have aimed to provide the necessary tools for custom analysis integration through a scaled-down application programming interface (API) and instructions for running the code locally with supplemented instructions. It still lacks some components for a more widespread adaptation, most notably by researchers without expertise in coding. For instance, the lack of a graphical user interface for data visualization and selection during DDA may prohibit the adaptation by researchers incapable of setting up a complementary environment for data visualization. During the development of our method, we continuously analyzed and visualized the incoming data using Jupyter (https://jupyter.org/), an online notebook interface for running code on a local server. Furthermore, our method was solely tested on Nikon hardware, with the supplemented JOBs software. JOBs is a simple yet powerful tool provided by Nikon that allows for the design of workflows for hardware control. We are dedicated in the development of μManager⁴³ interfaces that would allow for the design of equivalent workflows as presented in this study. Finally, the use of our method relies on an unaltered sample during imaging. This limits the implementation and use of our method to single-system multi-modal microscopes and minimizes the extent to which reagents can be introduced to the sample.

STAR★Methods

Key resources table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Chemical, peptides and recombinant proteins

Hoechst	Themo Scientific	Cat#62249
HEPES	Gibco	Cat#15630056
Phosphate buffered saline (PBS)	Gibco	Cat#20012068
Bovine Serum Albumin	VWR	Cat#422351S
Fibronectin	Sigma Aldrich	Cat# F0895
Paraformaldehyde (PFA)	Themo Scientific	Cat#28908
Ampicillin	Sigma Aldrich	Cat#A9518
Kanamycin	Sigma Aldrich	Cat#K1377
Dulbecco's Modified Eagle's Media (DMEM)	Gibco	Cat#61965-059
Roswell Park Memorial Institute (RPMI 1640)	Gibco	Cat#61870044
Fetal Bovine Serum (FBS)	Gibco	Cat#10270106
Luria-Bertani's (LB) Media	Sigma Aldrich	Cat#L3022

Critical commericial assays

QIAGEN Plasmid Mini Kit	QIAGEN	Cat#12143/12145
Transfection kit	Invitrogen	Cat#L3000008

Experimental models: Cell lines

HeLa WT	Sigma Aldrich	Cat#93021013-1VL; RRID: CVCL_0030
H1299 WT	Shafqat-Abbasi et al.⁴⁴	N/A
H1299 WT - mKate-Paxillin	Shafqat-Abbasi et al.⁴⁴	N/A

Recombinant DNA

Plasmid: pmScarlet_alphaTubulin_C1	Addgene	Cat#85045; RRID: Addgene_85045
Plasmid: pcDNA3.1 LifeAct E2-Crimson	This paper	N/A
Plasmid: pEF1 ITGB1 wt	Timothy Springer Lab	N/A
Plasmid: ITGA5-GFP	Addgene	Cat#15238; RRID: Addgene_15238
Plasmid: GFP plasmid	This paper	N/A

Bacterial strains

Yersinia Psuedotuberculosis WT	Maria Fällman Lab	N/A

Software and algorithms

Julia	https://julialang.org/	N/A
Nim programming language	https://nim-lang.org/	N/A
DDMFramework.jl	https://github.com/nordenfeltLab/DDMFramework.jl	https://doi.org/10.5281/zenodo.7590417
DDMServer.jl	https://github.com/nordenfeltLab/DDMServer.jl	https://doi.org/10.5281/zenodo.7670863
DDMTransfection.jl	https://github.com/nordenfeltLab/DDMTransfection.jl	https://doi.org/10.5281/zenodo.7590424
DDMInfection.jl	https://github.com/nordenfeltLab/DDMInfection.jl	https://doi.org/10.5281/zenodo.7671048
DDMMigration.jl	https://github.com/nordenfeltLab/DDMMigration.jl	https://doi.org/10.5281/zenodo.7590361
Microscope CLI	https://github.com/nordenfeltLab/ddm_microscope_cli	https://doi.org/10.5281/zenodo.7670892

Other

μ-Slide 8-Well Glass Bottom slides	Ibidi	Cat#80821
μ-Slide 4-Well Ph+ Glass Bottom slides	Ibidi	Cat#80447

Open in a new tab

Resource availability

Lead contact

Further information and requests for resources should be directed to the lead contact, Pontus Nordenfelt (pontus.nordenfelt@med.lu.se).

Materials availability

Reagents in this study are available by request to the lead contact, Pontus Nordenfelt (pontus.nordenfelt@med.lu.se).

Experimental model and subject details

Cell culture

The cell lines herein are not members of the ATCC list of commonly misidentified cell lines. All cells were maintained and used between passages 5–25. Human cervix epithelioid carcinoma cells (HeLa; Sigma Aldrich) were cultured in Dulbecco’s Modified Eagle’s (DMEM; Gibco, Thermo Fisher Scientific) supplemented with 10% heat-inactivated fetal bovine serum (FBS) (Gibco) at 37°C in a humidified 5% CO2 incubator. For imaging, μ-slide 8-well glass-bottom slides (Ibidi) were coated with 2 μg/mL fibronectin human plasma (FN) (Sigma Aldrich) and incubated for 30 min at 37°C. Cells were transfected in suspension (70,000 cells/mL) with 500 ng/plasmid using Lipofectamine 3000 (Invitrogen) according to the manufacturer’s protocol. All plasmids were transfected together (total 2 μg DNA per transfection) and are listed in the key resources table. Cells were plated at 20,000 cells cm² in each well and incubated for 48 h before fixing with 4% paraformaldehyde (PFA; Thermo Scientific) for 15 min at room temperature (RT). For live imaging of host-bacteria interactions, various cell densities were tested for seeding to achieve a suitable cell confluency level (we used 20,000 cells per well in the included experiments), where it was possible to find both interacting and non-interacting bacteria.

H1299 cells expressing mKate-Paxillin and H1299 WT⁴⁴ were kindly provided by Staffan Strömblad. H1299 cells (mKate-Paxillin and WT) were cultured in Roswell Park Memorial Institute (RPMI 1640; Gibco) supplemented with 10% heat-inactivated FBS at 37°C in a humidified 5 % CO2 incubator. For imaging, μ-Slide 4-well Ph+ glass bottom slides (Ibidi) were coated with 2 μg/mL FN and incubated for 30 min 37°C. Slides were blocked in 1 % BSA in 1X PBS for 30min at 37°C. Cells were plated at 6 000 cells cm2 and incubated overnight. Before imaging, cell-permeable Hoechst (0.05 nM) (Thermo Scientific) and HEPES (10 mM) (Gibco) were added to the media.

Bacterial preparation

GFP expressing WT Yersinia pseudotuberculosis was previously generated in lab. Overnight cultures of bacteria were diluted (50:2000) in Luria-Bertani’s media (100 μg/mL Ampicillin) and grown in a 26°C shaking environment for 2 hours. Activation of the bacteria was done by incubating the suspension cultures at 37°C for 1 hour prior to imaging. To increase the likelihood of single bacteria-cell interactions, 1 μL of the suspension cultures were added to the edge of the well before imaging and allowed to disperse across the sample over time. The original strain was kindly provided by Maria Fällman.

Method details

Live fluorescence microscopy

Images were acquired using an inverted Nikon Ti2-E wide-field fluorescence microscope with a Nikon Plan Apo _λ 10x 0.45 numerical aperture (NA) objective lens and Perfect Focus System (PFS) for maintenance of focus over time. Excitation and emission light were passed through DAPI (Exc. 379-405nm, Em. 414-480nm), FITC (Exc. 457-487nm, Em. 503-538nm), TRITC (Exc. 543-566nm, Em. 582-636nm) and Cy5 (Exc. 590-645nm, Em. 659-736nm) filter cubes, all from Semrock. Samples were kept in a humidified atmosphere at 37°C and 5 % CO2 using an environmental chamber (Okolab). Images were acquired on a Nikon DS-Qi2 CMOS camera. The imaging of the samples was automated by generating stage positions covering the sample area using JOBS (NIS-Elements extension; Nikon) and a Nikon TI-S-ER motorized stage with an encoder. For time-lapse imaging, images were collected every 10 min for 100 min. The same system was also used with a Nikon CFI SR Plan Apo IR 60XAC WI / 1.27NA with a software-driven TI2-N-WID Water Immersion Dispenser for imaging of fixed samples or time-lapse imaging of live cell migration every 3 min for 3 hours.

DDM framework

The framework was developed in Julia (v.1.6.0) and deployed on a local server to handle image analysis and data informatics. Image analysis packages with a shared schema were developed (see image analysis) and loaded into the framework. For each experiment, the corresponding analysis package was initiated with an experiment-specific configuration. Images were transferred to the server using an in-house developed command line interface (CLI; available on Github) for analysis. We plan to add plugins for other microscope vendors and μ-manager¹⁸ in the future. NIS Elements JOBS was used to query the server through the CLI for analysis output. See Methods S1.

Image analysis

All described image analyses were written in Julia (version 1.6.0) and are available on Github. Each analysis is self-contained in plugins loaded by the framework. The analysis parameters are specified during instantiating each plugin through a standardized JSON format (see Methods S2).

Transfection plugin

Low magnification (10X) images of multi-labeled fixed HeLa cells were acquired as described above. Image background segmentation was performed on the DAPI channel (cell nuclei). Using the foreground segments as seeds, a local thresholding (Otsu’s thresholding) window was used to generate segments of the cell fluorescence in each channel. Segments overlapping the seeds were selected and intensity features (i.e. mean, median, standard deviation) were measured and normalized to the mean sample background. Cells were sampled for data-dependent imaging based on the local relative intensity for every label.

Infection plugin

Low magnification time-lapse images of stable mScarlet-LifeAct expressing HeLa cells and WT Y. pseudotuberculosis were acquired as described above. Background segmentation was performed on each channel (DAPI, FITC, TRITC) to acquire segments of fluorescent actin, bacteria and nuclei. Bacteria-cell interactions were determined based on thresholding the Euclidean distance (> 5 pixels, ∼3.7 μm) between each bacterium and the closest segmented actin signal, as well as consecutive tracking of the bacteria for a minimum of 2 frames (20 minutes). Cells were sampled for data-dependent imaging based on a per experiment pre-defined number of interactions.

Migration plugin

Low magnification time-lapse images of stable mKate-paxillin expressing H1299 cells were acquired as described above. Background segmentation was performed on the DAPI channel (cell nuclei). Features describing cell migration (e.g. mean speed, meandering index) were obtained by tracking the coordinates of the nuclei and finding the optimal assignment of coordinates, in terms of least total Euclidean distance, between each pair of frames using the Munkres algorithm.⁴⁵ Background segmentation was performed on the TRITC channel (paxillin) to obtain intensity features (i.e. mean, median, standard deviation). Cells sampled from the top and bottom 10^th quantiles in terms of mean speed were selected for data-dependent imaging.

Automated data-dependent microscopy

Image features extracted from each experiment (e.g. cell morphometrics, fluorescence signal) were stored in a database and the recorded stage position of each data point was accessed through a query using the CLI. Each stage position was recorded to a local file on the microscope station and loaded into NIS-Elements for imaging using JOBS. The stage positions were imaged differently depending on the microscope: (a) for the live-fluorescence microscopy, the JOBS was programmed to switch to the Nikon CFI SR Plan Apo IR 60XAC WI / 1.27NA objective, apply water using the water dispenser, perform autofocus and subsequently image each point, (b) for the TIRF and SIM microscope, the JOBS was programmed to let the user switch to the respective Nikon CFI Apo TIRF 100X Oil / 1.49 NA objective, manually apply immersion oil and calibrate the sample to its original position, perform autofocus and image each point. See File S2 for the JOBs templates used in this paper.

Quantification and statistical analysis

We used Julia v1.7 for data analysis and visualization. All figure plots were rendered using Makie.jl.⁴⁶ Statistical details can be found in the figure legends.

Additional resources

To ease the setup of the DDMFramework, we are providing a demo containing all necessary code and instructions for setting up a server with the plugins used for this paper. We also include dataset of images of migrating H1299 cells used in this paper for simulating the acquisition of data in real time. See File S1. We also provide a simple API for the integration of custom image analysis pipelines. Further details can be found at: https://github.com/nordenfeltLab/DDMFramework.jl.

Acknowledgments

We thank the Lund University Bioimaging Center (LBIC) for use of fluorescence microscopes. We thank the Emil and Eva Cornell Foundation, the Swedish Research Council (Vetenskapsrådet, 2019-02355), the Swedish Cancer Foundation (Cancerfonden), the Strategic Research Foundation (SSF),VINNOVA (2020-04702), and Per-Eric and Ulla Schyberg’s Foundation for funding. The microscope and objective assets in the figures were created using https://biorender.com.

Author contributions

Conceptualization, O.A., J.K.A., and P.N.; experimentation and data analysis, O.A. and J.K.A.; writing – original draft, O.A. and P.N. All authors contributed to reading and editing the manuscript.

Declaration of interests

O.A., J.K.A., and P.N. have a patent pending related to the findings in this paper and are founders of Oden Vision AB.

Published: March 6, 2023

Footnotes

Supplemental information can be found online at https://doi.org/10.1016/j.crmeth.2023.100419.

Supplemental information

Document S1. Figures S1–S3 and Methods S1 and S2

mmc1.pdf^{(3.1MB, pdf)}

File S1. Demo of DDMFramework, related to STAR Methods section additional resources

A.zip file containing a demo for running the DDMFramework with the DDMMigration plugin on a local machine. Please follow the included readme.pdf file for further instructions for setting up your environment and running the server. The demo contains image files and instructions for how to send them for analysis on the server, including examples of querying the database.

mmc5.zip^{(127.8MB, zip)}

File S2. JOBs templates, related to STAR Methods section automated data-dependent microscopy

The acquisition of images on the microscope was automated using JOBs. Minor modifications were made for each plugin to accommodate for live cell imaging and the nature of the experiment. In general, settings (e.g. excitation wavelengths, autofocus settings, storage and run variables) for the pipeline are defined at the start of the program. The program performs the following for a well; the data-independent acquisition performs a timelapse and generates points covering approximately 45–85\% of the wells (note: for the fixed transfection experiments, no timelapse is performed). Autofocus is performed for the initial image of the generated points. Images are captured for each point and saved to the defined storage location. A macro initiates the in-house developed command-line interface that posts the image to the server running the DDMFramework with the corresponding loaded plugin. Depending on the experiments, ultimately the CLI performs a request action querying the database for cells of interest. If the response contains coordinates, the data-dependent acquisition begins with switching to the 60X water immersion objective, and water is applied to the objective through an automated water dispenser. Autofocus is performed for each coordinate and images are taken according to the JOBs setup.

mmc6.zip^{(2.1MB, zip)}

Document S2. Article plus supplemental information

mmc7.pdf^{(8.8MB, pdf)}

Data and code availability

•
Microscopy data reported in this paper will be shared by the lead contact upon request.
•
All original code has been deposited at https://github.com/nordenfeltLab/ and are publicly available as of the date of the publication. DOIs are listed in the key resources table.
•
Any additional information required to run and reanalyze the data reported in this paper is available from the lead contact upon request.

References

1.Battich N., Stoeger T., Pelkmans L. Image-based transcriptomics in thousands of single human cells at single-molecule resolution. Nat. Methods. 2013;10:1127–1133. doi: 10.1038/nmeth.2657. [DOI] [PubMed] [Google Scholar]
2.Snijder B., Sacher R., Rämö P., Damm E.-M., Liberali P., Pelkmans L. Population context determines cell-to-cell variability in endocytosis and virus infection. Nature. 2009;461:520–523. doi: 10.1038/nature08282. [DOI] [PubMed] [Google Scholar]
3.Neumann B., Held M., Liebel U., Erfle H., Rogers P., Pepperkok R., Ellenberg J. High-throughput RNAi screening by time-lapse imaging of live human cells. Nat. Methods. 2006;3:385–390. doi: 10.1038/nmeth876. [DOI] [PubMed] [Google Scholar]
4.Neumann B., Walter T., Hériché J.K., Bulkescher J., Erfle H., Conrad C., Rogers P., Poser I., Held M., Liebel U., et al. Phenotypic profiling of the human genome by time-lapse microscopy reveals cell division genes. Nature. 2010;464:721–727. doi: 10.1038/nature08869. [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Crainiciuc G., Palomino-Segura M., Molina-Moreno M., Sicilia J., Aragones D.G., Li J.L.Y., Madurga R., Adrover J.M., Aroca-Crevillén A., Martin-Salamanca S., et al. Behavioural immune landscapes of inflammation. Nature. 2022;601:415–421. doi: 10.1038/s41586-021-04263-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
6.Bray M.-A., Singh S., Han H., Davis C.T., Borgeson B., Hartland C., Kost-Alimova M., Gustafsdottir S.M., Gibson C.C., Carpenter A.E. Cell Painting, a high-content image-based assay for morphological profiling using multiplexed fluorescent dyes. Nat. Protoc. 2016;11:1757–1774. doi: 10.1038/nprot.2016.105. [DOI] [PMC free article] [PubMed] [Google Scholar]
7.Aguilar-Avelar C., Soto-García B., Aráiz-Hernández D., Yee-de León J.F., Esparza M., Chacón F., Delgado-Balderas J.R., Alvarez M.M., Trujillo-de Santiago G., Gómez-Guerra L.S., et al. High-throughput automated microscopy of circulating tumor cells. Sci. Rep. 2019;9:13766. doi: 10.1038/s41598-019-50241-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Mahecic D., Stepp W.L., Zhang C., Griffié J., Weigert M., Manley S. Event-driven acquisition for content-enriched microscopy. Nat. Methods. 2022;19:1262–1267. doi: 10.1038/s41592-022-01589-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Alvelid J., Damenti M., Sgattoni C., Testa I. Event-triggered STED imaging. Nat. Methods. 2022;19:1268–1275. doi: 10.1038/s41592-022-01588-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Zanella F., Lorens J.B., Link W. High content screening: seeing is believing. Trends Biotechnol. 2010;28:237–245. doi: 10.1016/j.tibtech.2010.02.005. [DOI] [PubMed] [Google Scholar]
11.Boutros M., Heigwer F., Laufer C. Microscopy-based high-content screening. Cell. 2015;163:1314–1325. doi: 10.1016/j.cell.2015.11.007. [DOI] [PubMed] [Google Scholar]
12.Mattiazzi Usaj M., Styles E.B., Verster A.J., Friesen H., Boone C., Andrews B.J. High-content screening for quantitative cell biology. Trends Cell Biol. 2016;26:598–611. doi: 10.1016/j.tcb.2016.03.008. [DOI] [PubMed] [Google Scholar]
13.Garvey C.M., Spiller E., Lindsay D., Chiang C.-T., Choi N.C., Agus D.B., Mallick P., Foo J., Mumenthaler S.M. A high-content image-based method for quantitatively studying context-dependent cell population dynamics. Sci. Rep. 2016;6:29752. doi: 10.1038/srep29752. [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Mattiazzi Usaj M., Sahin N., Friesen H., Pons C., Usaj M., Masinas M.P.D., Shuteriqi E., Shkurin A., Aloy P., Morris Q., et al. Systematic genetics and single-cell imaging reveal widespread morphological pleiotropy and cell-to-cell variability. Mol. Syst. Biol. 2020;16:e9243. doi: 10.15252/msb.20199243. [DOI] [PMC free article] [PubMed] [Google Scholar]
15.Tischer C., Hilsenstein V., Hanson K., Pepperkok R. Adaptive fluorescence microscopy by online feedback image analysis. Methods Cell Biol. 2014;123:489–503. doi: 10.1016/b978-0-12-420138-5.00026-4. [DOI] [PubMed] [Google Scholar]
16.Almada P., Pereira P.M., Culley S., Caillol G., Boroni-Rueda F., Dix C.L., Charras G., Baum B., Laine R.F., Leterrier C., Henriques R. Automating multimodal microscopy with NanoJ-Fluidics. Nat. Commun. 2019;10:1223. doi: 10.1038/s41467-019-09231-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Pinkard H., Stuurman N., Corbin K., Vale R., Krummel M.F. Micro-Magellan: open-source, sample-adaptive, acquisition software for optical microscopy. Nat. Methods. 2016;13:807–809. doi: 10.1038/nmeth.3991. [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Edelstein A., Amodaj N., Hoover K., Vale R., Stuurman N. Computer control of microscopes using μManager. Curr. Protoc. Mol. Biol. 2010;Chapter 14:Unit14.20. doi: 10.1002/0471142727.mb1420s92. [DOI] [PMC free article] [PubMed] [Google Scholar]
19.Pinkard H., Stuurman N., Ivanov I.E., Anthony N.M., Ouyang W., Li B., Yang B., Tsuchida M.A., Chhun B., Zhang G., et al. Pycro-Manager: open-source software for customized and reproducible microscope control. Nat. Methods. 2021;18:226–228. doi: 10.1038/s41592-021-01087-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
20.Tosi S., Lladó A., Bardia L., Rebollo E., Godo A., Stockinger P., Colombelli J. AutoScanJ: a suite of ImageJ scripts for intelligent microscopy. Front. Bioinform. 2021;1:627626. doi: 10.3389/fbinf.2021.627626. [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Fox Z.R., Fletcher S., Fraisse A., Aditya C., Sosa-Carrillo S., Petit J., Gilles S., Bertaux F., Ruess J., Batt G. Enabling reactive microscopy with MicroMator. Nat. Commun. 2022;13:2199. doi: 10.1038/s41467-022-29888-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
22.Conrad C., Wünsche A., Tan T.H., Bulkescher J., Sieckmann F., Verissimo F., Edelstein A., Walter T., Liebel U., Pepperkok R., Ellenberg J. Micropilot: automation of fluorescence microscopy–based imaging for systems biology. Nat. Methods. 2011;8:246–249. doi: 10.1038/nmeth.1558. [DOI] [PMC free article] [PubMed] [Google Scholar]
23.Goldberg I.G., Allan C., Burel J.-M., Creager D., Falconi A., Hochheiser H., Johnston J., Mellen J., Sorger P.K., Swedlow J.R. The Open Microscopy Environment (OME) Data Model and XML file: open tools for informatics and quantitative analysis in biological imaging. Genome Biol. 2005;6:R47. doi: 10.1186/gb-2005-6-5-r47. [DOI] [PMC free article] [PubMed] [Google Scholar]
24.Linkert M., Rueden C.T., Allan C., Burel J.M., Moore W., Patterson A., Loranger B., Moore J., Neves C., Macdonald D., et al. Metadata matters: access to image data in the real world. J. Cell Biol. 2010;189:777–782. doi: 10.1083/jcb.201004104. [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Carpenter A.E., Jones T.R., Lamprecht M.R., Clarke C., Kang I.H., Friman O., Guertin D.A., Chang J.H., Lindquist R.A., Moffat J., et al. CellProfiler: image analysis software for identifying and quantifying cell phenotypes. Genome Biol. 2006;7:R100. doi: 10.1186/gb-2006-7-10-r100. [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Schneider C.A., Rasband W.S., Eliceiri K.W. NIH Image to ImageJ: 25 years of image analysis. Nat. Methods. 2012;9:671–675. doi: 10.1038/nmeth.2089. [DOI] [PMC free article] [PubMed] [Google Scholar]
27.Allan C., Burel J.-M., Moore J., Blackburn C., Linkert M., Loynton S., MacDonald D., Moore W.J., Neves C., Patterson A., et al. OMERO: flexible, model-driven data management for experimental biology. Nat. Methods. 2012;9:245–253. doi: 10.1038/nmeth.1896. [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Steiner G.E., Ecker R.C., Kramer G., Stockenhuber F., Marberger M.J. Automated data acquisition by confocal laser scanning microscopy and image analysis of triple stained immunofluorescent leukocytes in tissue. J. Immunol. Methods. 2000;237:39–50. doi: 10.1016/s0022-1759(99)00240-9. [DOI] [PubMed] [Google Scholar]
29.Ecker R.C., Steiner G.E. Microscopy-based multicolor tissue cytometry at the single-cell level. Cytometry A. 2004;59:182–190. doi: 10.1002/cyto.a.20052. [DOI] [PubMed] [Google Scholar]
30.Händel N., Brockel A., Heindl M., Klein E., Uhlig H.H. Cell-cell-neighborhood relations in tissue sections—a quantitative model for tissue cytometry. Cytometry A. 2009;75:356–361. doi: 10.1002/cyto.a.20705. [DOI] [PubMed] [Google Scholar]
31.Barteneva N.S., Fasler-Kan E., Vorobjev I.A. Imaging flow cytometry. J. Histochem. Cytochem. 2012;60:723–733. doi: 10.1369/0022155412453052. [DOI] [PMC free article] [PubMed] [Google Scholar]
32.LaBelle C.A., Massaro A., Cortés-Llanos B., Sims C.E., Allbritton N.L. Image-based live cell sorting. Trends Biotechnol. 2021;39:613–623. doi: 10.1016/j.tibtech.2020.10.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
33.Schraivogel D., Kuhn T.M., Rauscher B., Rodríguez-Martínez M., Paulsen M., Owsley K., Middlebrook A., Tischer C., Ramasz B., Ordoñez-Rueda D., et al. High-speed fluorescence image–enabled cell sorting. Science. 2022;375:315–320. doi: 10.1126/science.abj3013. [DOI] [PMC free article] [PubMed] [Google Scholar]
34.Rosqvist R., Forsberg A., Wolf-Watz H. Intracellular targeting of the Yersinia YopE cytotoxin in mammalian cells induces actin microfilament disruption. Infect. Immun. 1991;59:4562–4569. doi: 10.1128/iai.59.12.4562-4569.1991. [DOI] [PMC free article] [PubMed] [Google Scholar]
35.McKinnon K.M. Flow cytometry: an overview. Curr. Protoc. Immunol. 2018;120:5.1.1–5.1.11. doi: 10.1002/cpim.40. [DOI] [PMC free article] [PubMed] [Google Scholar]
36.Sommer C., Hoefler R., Samwer M., Gerlich D.W. A deep learning and novelty detection framework for rapid phenotyping in high-content screening. Mol. Biol. Cell. 2017;28:3428–3436. doi: 10.1091/mbc.e17-05-0333. [DOI] [PMC free article] [PubMed] [Google Scholar]
37.Caicedo J.C., Cooper S., Heigwer F., Warchal S., Qiu P., Molnar C., Vasilevich A.S., Barry J.D., Bansal H.S., Kraus O., et al. Data-analysis strategies for image-based cell profiling. Nat. Methods. 2017;14:849–863. doi: 10.1038/nmeth.4397. [DOI] [PMC free article] [PubMed] [Google Scholar]
38.Hu C., He S., Lee Y.J., He Y., Kong E.M., Li H., Anastasio M.A., Popescu G. Live-dead assay on unlabeled cells using phase imaging with computational specificity. Nat. Commun. 2022;13:713. doi: 10.1038/s41467-022-28214-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
39.Godinez W.J., Hossain I., Lazic S.E., Davies J.W., Zhang X. A multi-scale convolutional neural network for phenotyping high-content cellular images. Bioinformatics. 2017;33:2010–2019. doi: 10.1093/bioinformatics/btx069. [DOI] [PubMed] [Google Scholar]
40.Buggenthin F., Buettner F., Hoppe P.S., Endele M., Kroiss M., Strasser M., Schwarzfischer M., Loeffler D., Kokkaliaris K.D., Hilsenbeck O., et al. Prospective identification of hematopoietic lineage choice by deep learning. Nat. Methods. 2017;14:403–406. doi: 10.1038/nmeth.4182. [DOI] [PMC free article] [PubMed] [Google Scholar]
41.Christiansen E.M., Yang S.J., Ando D.M., Javaherian A., Skibinski G., Lipnick S., Mount E., O’Neil A., Shah K., Lee A.K., et al. Silico labeling: predicting fluorescent labels in unlabeled images. Cell. 2018;173:792–803.e19. doi: 10.1016/j.cell.2018.03.040. [DOI] [PMC free article] [PubMed] [Google Scholar]
42.Röst H.L., Rosenberger G., Navarro P., Gillet L., Miladinović S.M., Schubert O.T., Wolski W., Collins B.C., Malmström J., Malmström L., Aebersold R. OpenSWATH enables automated, targeted analysis of data-independent acquisition MS data. Nat. Biotechnol. 2014;32:219–223. doi: 10.1038/nbt.2841. [DOI] [PubMed] [Google Scholar]
43.Edelstein A.D., Tsuchida M.A., Amodaj N., Pinkard H., Vale R.D., Stuurman N. Advanced methods of microscope control using μManager software. J. Biol. Methods. 2014;1:e10. doi: 10.14440/jbm.2014.36. [DOI] [PMC free article] [PubMed] [Google Scholar]
44.Shafqat-Abbasi H., Kowalewski J.M., Kiss A., Gong X., Hernandez-Varas P., Berge U., Jafari-Mamaghani M., Lock J.G., Strömblad S. An analysis toolbox to explore mesenchymal migration heterogeneity reveals adaptive switching between distinct modes. Elife. 2016;5:e11384. doi: 10.7554/elife.11384. [DOI] [PMC free article] [PubMed] [Google Scholar]
45.Munkres J. Algorithms for the assignment and transportation problems. J. Soc. Ind. Appl. Math. 1957;5:32–38. doi: 10.1137/0105003. [DOI] [Google Scholar]
46.Danisch S., Krumbiegel J. Makie.jl: flexible high-performance data visualization for Julia. J. Open Source Softw. 2021;6:3349. doi: 10.21105/joss.03349. [DOI] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Video S1. Examples of captured infection events, related to Figure 3

Montage of infection events captured through DDM (blue - DNA, orange - actin, green - bacteria). Out-of-focus light was removed in post-processing using Nis elements Clarified.ai.

Download video file^{(8.6MB, mp4)}

Video S2. Examples of fast-migratory cells, related to Figure 4

Montage of fast migratory cells captured through DDM (DIC + blue - DNA). Out-of-focus light was removed in post-processing using Nis elements Clarified.ai.

Download video file^{(2.8MB, mp4)}

Video S3. Examples of slow-migratory cells, related to Figure 4

Montage of slow migratory cells captured through DDM (DIC + blue - DNA). Out-of-focus light was removed in post-processing using Nis elements Clarified.ai.

Download video file^{(2.6MB, mp4)}

Document S1. Figures S1–S3 and Methods S1 and S2

mmc1.pdf^{(3.1MB, pdf)}

File S1. Demo of DDMFramework, related to STAR Methods section additional resources

mmc5.zip^{(127.8MB, zip)}

File S2. JOBs templates, related to STAR Methods section automated data-dependent microscopy

mmc6.zip^{(2.1MB, zip)}

Document S2. Article plus supplemental information

mmc7.pdf^{(8.8MB, pdf)}

Data Availability Statement

•
Microscopy data reported in this paper will be shared by the lead contact upon request.
•
All original code has been deposited at https://github.com/nordenfeltLab/ and are publicly available as of the date of the publication. DOIs are listed in the key resources table.
•
Any additional information required to run and reanalyze the data reported in this paper is available from the lead contact upon request.

[bib1] 1.Battich N., Stoeger T., Pelkmans L. Image-based transcriptomics in thousands of single human cells at single-molecule resolution. Nat. Methods. 2013;10:1127–1133. doi: 10.1038/nmeth.2657. [DOI] [PubMed] [Google Scholar]

[bib2] 2.Snijder B., Sacher R., Rämö P., Damm E.-M., Liberali P., Pelkmans L. Population context determines cell-to-cell variability in endocytosis and virus infection. Nature. 2009;461:520–523. doi: 10.1038/nature08282. [DOI] [PubMed] [Google Scholar]

[bib3] 3.Neumann B., Held M., Liebel U., Erfle H., Rogers P., Pepperkok R., Ellenberg J. High-throughput RNAi screening by time-lapse imaging of live human cells. Nat. Methods. 2006;3:385–390. doi: 10.1038/nmeth876. [DOI] [PubMed] [Google Scholar]

[bib4] 4.Neumann B., Walter T., Hériché J.K., Bulkescher J., Erfle H., Conrad C., Rogers P., Poser I., Held M., Liebel U., et al. Phenotypic profiling of the human genome by time-lapse microscopy reveals cell division genes. Nature. 2010;464:721–727. doi: 10.1038/nature08869. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib5] 5.Crainiciuc G., Palomino-Segura M., Molina-Moreno M., Sicilia J., Aragones D.G., Li J.L.Y., Madurga R., Adrover J.M., Aroca-Crevillén A., Martin-Salamanca S., et al. Behavioural immune landscapes of inflammation. Nature. 2022;601:415–421. doi: 10.1038/s41586-021-04263-y. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib6] 6.Bray M.-A., Singh S., Han H., Davis C.T., Borgeson B., Hartland C., Kost-Alimova M., Gustafsdottir S.M., Gibson C.C., Carpenter A.E. Cell Painting, a high-content image-based assay for morphological profiling using multiplexed fluorescent dyes. Nat. Protoc. 2016;11:1757–1774. doi: 10.1038/nprot.2016.105. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib7] 7.Aguilar-Avelar C., Soto-García B., Aráiz-Hernández D., Yee-de León J.F., Esparza M., Chacón F., Delgado-Balderas J.R., Alvarez M.M., Trujillo-de Santiago G., Gómez-Guerra L.S., et al. High-throughput automated microscopy of circulating tumor cells. Sci. Rep. 2019;9:13766. doi: 10.1038/s41598-019-50241-w. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib8] 8.Mahecic D., Stepp W.L., Zhang C., Griffié J., Weigert M., Manley S. Event-driven acquisition for content-enriched microscopy. Nat. Methods. 2022;19:1262–1267. doi: 10.1038/s41592-022-01589-x. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib9] 9.Alvelid J., Damenti M., Sgattoni C., Testa I. Event-triggered STED imaging. Nat. Methods. 2022;19:1268–1275. doi: 10.1038/s41592-022-01588-y. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib10] 10.Zanella F., Lorens J.B., Link W. High content screening: seeing is believing. Trends Biotechnol. 2010;28:237–245. doi: 10.1016/j.tibtech.2010.02.005. [DOI] [PubMed] [Google Scholar]

[bib11] 11.Boutros M., Heigwer F., Laufer C. Microscopy-based high-content screening. Cell. 2015;163:1314–1325. doi: 10.1016/j.cell.2015.11.007. [DOI] [PubMed] [Google Scholar]

[bib12] 12.Mattiazzi Usaj M., Styles E.B., Verster A.J., Friesen H., Boone C., Andrews B.J. High-content screening for quantitative cell biology. Trends Cell Biol. 2016;26:598–611. doi: 10.1016/j.tcb.2016.03.008. [DOI] [PubMed] [Google Scholar]

[bib13] 13.Garvey C.M., Spiller E., Lindsay D., Chiang C.-T., Choi N.C., Agus D.B., Mallick P., Foo J., Mumenthaler S.M. A high-content image-based method for quantitatively studying context-dependent cell population dynamics. Sci. Rep. 2016;6:29752. doi: 10.1038/srep29752. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib14] 14.Mattiazzi Usaj M., Sahin N., Friesen H., Pons C., Usaj M., Masinas M.P.D., Shuteriqi E., Shkurin A., Aloy P., Morris Q., et al. Systematic genetics and single-cell imaging reveal widespread morphological pleiotropy and cell-to-cell variability. Mol. Syst. Biol. 2020;16:e9243. doi: 10.15252/msb.20199243. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib15] 15.Tischer C., Hilsenstein V., Hanson K., Pepperkok R. Adaptive fluorescence microscopy by online feedback image analysis. Methods Cell Biol. 2014;123:489–503. doi: 10.1016/b978-0-12-420138-5.00026-4. [DOI] [PubMed] [Google Scholar]

[bib16] 16.Almada P., Pereira P.M., Culley S., Caillol G., Boroni-Rueda F., Dix C.L., Charras G., Baum B., Laine R.F., Leterrier C., Henriques R. Automating multimodal microscopy with NanoJ-Fluidics. Nat. Commun. 2019;10:1223. doi: 10.1038/s41467-019-09231-9. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib17] 17.Pinkard H., Stuurman N., Corbin K., Vale R., Krummel M.F. Micro-Magellan: open-source, sample-adaptive, acquisition software for optical microscopy. Nat. Methods. 2016;13:807–809. doi: 10.1038/nmeth.3991. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib18] 18.Edelstein A., Amodaj N., Hoover K., Vale R., Stuurman N. Computer control of microscopes using μManager. Curr. Protoc. Mol. Biol. 2010;Chapter 14:Unit14.20. doi: 10.1002/0471142727.mb1420s92. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib19] 19.Pinkard H., Stuurman N., Ivanov I.E., Anthony N.M., Ouyang W., Li B., Yang B., Tsuchida M.A., Chhun B., Zhang G., et al. Pycro-Manager: open-source software for customized and reproducible microscope control. Nat. Methods. 2021;18:226–228. doi: 10.1038/s41592-021-01087-6. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib20] 20.Tosi S., Lladó A., Bardia L., Rebollo E., Godo A., Stockinger P., Colombelli J. AutoScanJ: a suite of ImageJ scripts for intelligent microscopy. Front. Bioinform. 2021;1:627626. doi: 10.3389/fbinf.2021.627626. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib21] 21.Fox Z.R., Fletcher S., Fraisse A., Aditya C., Sosa-Carrillo S., Petit J., Gilles S., Bertaux F., Ruess J., Batt G. Enabling reactive microscopy with MicroMator. Nat. Commun. 2022;13:2199. doi: 10.1038/s41467-022-29888-z. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib22] 22.Conrad C., Wünsche A., Tan T.H., Bulkescher J., Sieckmann F., Verissimo F., Edelstein A., Walter T., Liebel U., Pepperkok R., Ellenberg J. Micropilot: automation of fluorescence microscopy–based imaging for systems biology. Nat. Methods. 2011;8:246–249. doi: 10.1038/nmeth.1558. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib23] 23.Goldberg I.G., Allan C., Burel J.-M., Creager D., Falconi A., Hochheiser H., Johnston J., Mellen J., Sorger P.K., Swedlow J.R. The Open Microscopy Environment (OME) Data Model and XML file: open tools for informatics and quantitative analysis in biological imaging. Genome Biol. 2005;6:R47. doi: 10.1186/gb-2005-6-5-r47. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib24] 24.Linkert M., Rueden C.T., Allan C., Burel J.M., Moore W., Patterson A., Loranger B., Moore J., Neves C., Macdonald D., et al. Metadata matters: access to image data in the real world. J. Cell Biol. 2010;189:777–782. doi: 10.1083/jcb.201004104. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib25] 25.Carpenter A.E., Jones T.R., Lamprecht M.R., Clarke C., Kang I.H., Friman O., Guertin D.A., Chang J.H., Lindquist R.A., Moffat J., et al. CellProfiler: image analysis software for identifying and quantifying cell phenotypes. Genome Biol. 2006;7:R100. doi: 10.1186/gb-2006-7-10-r100. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib26] 26.Schneider C.A., Rasband W.S., Eliceiri K.W. NIH Image to ImageJ: 25 years of image analysis. Nat. Methods. 2012;9:671–675. doi: 10.1038/nmeth.2089. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib27] 27.Allan C., Burel J.-M., Moore J., Blackburn C., Linkert M., Loynton S., MacDonald D., Moore W.J., Neves C., Patterson A., et al. OMERO: flexible, model-driven data management for experimental biology. Nat. Methods. 2012;9:245–253. doi: 10.1038/nmeth.1896. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib28] 28.Steiner G.E., Ecker R.C., Kramer G., Stockenhuber F., Marberger M.J. Automated data acquisition by confocal laser scanning microscopy and image analysis of triple stained immunofluorescent leukocytes in tissue. J. Immunol. Methods. 2000;237:39–50. doi: 10.1016/s0022-1759(99)00240-9. [DOI] [PubMed] [Google Scholar]

[bib29] 29.Ecker R.C., Steiner G.E. Microscopy-based multicolor tissue cytometry at the single-cell level. Cytometry A. 2004;59:182–190. doi: 10.1002/cyto.a.20052. [DOI] [PubMed] [Google Scholar]

[bib30] 30.Händel N., Brockel A., Heindl M., Klein E., Uhlig H.H. Cell-cell-neighborhood relations in tissue sections—a quantitative model for tissue cytometry. Cytometry A. 2009;75:356–361. doi: 10.1002/cyto.a.20705. [DOI] [PubMed] [Google Scholar]

[bib31] 31.Barteneva N.S., Fasler-Kan E., Vorobjev I.A. Imaging flow cytometry. J. Histochem. Cytochem. 2012;60:723–733. doi: 10.1369/0022155412453052. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib32] 32.LaBelle C.A., Massaro A., Cortés-Llanos B., Sims C.E., Allbritton N.L. Image-based live cell sorting. Trends Biotechnol. 2021;39:613–623. doi: 10.1016/j.tibtech.2020.10.006. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib33] 33.Schraivogel D., Kuhn T.M., Rauscher B., Rodríguez-Martínez M., Paulsen M., Owsley K., Middlebrook A., Tischer C., Ramasz B., Ordoñez-Rueda D., et al. High-speed fluorescence image–enabled cell sorting. Science. 2022;375:315–320. doi: 10.1126/science.abj3013. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib34] 34.Rosqvist R., Forsberg A., Wolf-Watz H. Intracellular targeting of the Yersinia YopE cytotoxin in mammalian cells induces actin microfilament disruption. Infect. Immun. 1991;59:4562–4569. doi: 10.1128/iai.59.12.4562-4569.1991. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib35] 35.McKinnon K.M. Flow cytometry: an overview. Curr. Protoc. Immunol. 2018;120:5.1.1–5.1.11. doi: 10.1002/cpim.40. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib36] 36.Sommer C., Hoefler R., Samwer M., Gerlich D.W. A deep learning and novelty detection framework for rapid phenotyping in high-content screening. Mol. Biol. Cell. 2017;28:3428–3436. doi: 10.1091/mbc.e17-05-0333. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib37] 37.Caicedo J.C., Cooper S., Heigwer F., Warchal S., Qiu P., Molnar C., Vasilevich A.S., Barry J.D., Bansal H.S., Kraus O., et al. Data-analysis strategies for image-based cell profiling. Nat. Methods. 2017;14:849–863. doi: 10.1038/nmeth.4397. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib38] 38.Hu C., He S., Lee Y.J., He Y., Kong E.M., Li H., Anastasio M.A., Popescu G. Live-dead assay on unlabeled cells using phase imaging with computational specificity. Nat. Commun. 2022;13:713. doi: 10.1038/s41467-022-28214-x. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib39] 39.Godinez W.J., Hossain I., Lazic S.E., Davies J.W., Zhang X. A multi-scale convolutional neural network for phenotyping high-content cellular images. Bioinformatics. 2017;33:2010–2019. doi: 10.1093/bioinformatics/btx069. [DOI] [PubMed] [Google Scholar]

[bib40] 40.Buggenthin F., Buettner F., Hoppe P.S., Endele M., Kroiss M., Strasser M., Schwarzfischer M., Loeffler D., Kokkaliaris K.D., Hilsenbeck O., et al. Prospective identification of hematopoietic lineage choice by deep learning. Nat. Methods. 2017;14:403–406. doi: 10.1038/nmeth.4182. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib41] 41.Christiansen E.M., Yang S.J., Ando D.M., Javaherian A., Skibinski G., Lipnick S., Mount E., O’Neil A., Shah K., Lee A.K., et al. Silico labeling: predicting fluorescent labels in unlabeled images. Cell. 2018;173:792–803.e19. doi: 10.1016/j.cell.2018.03.040. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib42] 42.Röst H.L., Rosenberger G., Navarro P., Gillet L., Miladinović S.M., Schubert O.T., Wolski W., Collins B.C., Malmström J., Malmström L., Aebersold R. OpenSWATH enables automated, targeted analysis of data-independent acquisition MS data. Nat. Biotechnol. 2014;32:219–223. doi: 10.1038/nbt.2841. [DOI] [PubMed] [Google Scholar]

[bib43] 43.Edelstein A.D., Tsuchida M.A., Amodaj N., Pinkard H., Vale R.D., Stuurman N. Advanced methods of microscope control using μManager software. J. Biol. Methods. 2014;1:e10. doi: 10.14440/jbm.2014.36. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib44] 44.Shafqat-Abbasi H., Kowalewski J.M., Kiss A., Gong X., Hernandez-Varas P., Berge U., Jafari-Mamaghani M., Lock J.G., Strömblad S. An analysis toolbox to explore mesenchymal migration heterogeneity reveals adaptive switching between distinct modes. Elife. 2016;5:e11384. doi: 10.7554/elife.11384. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib45] 45.Munkres J. Algorithms for the assignment and transportation problems. J. Soc. Ind. Appl. Math. 1957;5:32–38. doi: 10.1137/0105003. [DOI] [Google Scholar]

[bib46] 46.Danisch S., Krumbiegel J. Makie.jl: flexible high-performance data visualization for Julia. J. Open Source Softw. 2021;6:3349. doi: 10.21105/joss.03349. [DOI] [Google Scholar]

PERMALINK

Data-driven microscopy allows for automated context-specific acquisition of high-fidelity image data

Oscar André

Johannes Kumra Ahnlide

Nils Norlin

Vinay Swaminathan

Pontus Nordenfelt

Summary

Graphical abstract

Highlights

Motivation

Introduction

Results

DDM as an approach for automated targeted image acquisition of relevant data

Figure 1.

DDM enables automated high-fidelity sampling of targeted multi-labeled subpopulations

Figure 2.

DDM allows for the identification and acquisition of bacteria-cell interactions

Figure 3.

DDM allows for the characterization and high-spatiotemporal imaging of migratory subpopulations

Figure 4.

Discussion

Limitations of the study

STAR★Methods

Key resources table

Resource availability

Lead contact

Materials availability

Experimental model and subject details

Cell culture

Bacterial preparation

Method details

Live fluorescence microscopy

DDM framework

Image analysis

Transfection plugin

Infection plugin

Migration plugin

Automated data-dependent microscopy

Quantification and statistical analysis

Additional resources

Acknowledgments

Author contributions

Declaration of interests

Footnotes

Supplemental information

Data and code availability

References

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases