A beginner’s guide to rigor and reproducibility in fluorescence imaging experiments

Abstract

Fluorescence light microscopy is an indispensable approach for the investigation of cell biological mechanisms. With the development of cutting-edge tools such as genetically encoded fluorescent proteins and superresolution methods, light microscopy is more powerful than ever at providing insight into a broad range of phenomena, from bacterial fission to cancer metastasis. However, as with all experimental approaches, care must be taken to ensure reliable and reproducible data collection, analysis, and reporting. Each step of every imaging experiment, from design to execution to communication to data management, should be critically assessed for bias, rigor, and reproducibility. This Perspective provides a basic “best practices” guide for designing and executing fluorescence imaging experiments, with the goal of introducing researchers to concepts that will help empower them to acquire images with rigor.

INTRODUCTION

One of the most commonly used and powerful tools in cell and developmental biology is the light microscope. Since the invention of the microscope and ongoing development of new dyes and tools, numerous studies have used light microscopy to uncover mechanisms of cell division and mitosis (Mitchison and Salmon, 2001), dramatic cell movements, such as those during Drosophila gastrulation (Tomer et al., 2012), and even smartphone-based diagnostic tools in medicine (Breslauer et al., 2009). Like any powerful tool, microscopes can provide insights into biological phenomena, but they can also be used incorrectly. This Perspective will describe common obstacles to reproducibility in fluorescence microscopy experiments and suggestions to overcome them to obtain rigorous quantitative data. Geared toward novice microscopists, this Perspective is not meant to provide a comprehensive survey of imaging modalities, the theory behind optics, or specific imaging techniques. For those seeking a more in-depth discussion, these topics are well-covered elsewhere (e.g., Lichtman and Conchello, 2005; Pawley, 2006; Waters, 2009; Thorn, 2016). Using a generalized experimental workflow (Figure 1), we address common sources of bias and obstacles in image analysis and processing as well as data presentation. The goal here is to provide a basic checklist of items to consider so that one may avoid the most common pitfalls in fluorescence microscopy experiments and communicate scientific discoveries more effectively.

FIGURE 1:

Imaging experiment workflow schematic.

BIAS IN EXPERIMENTAL DESIGN

Experimenter bias is a major obstacle to reproducibility, as it results in potentially skewed data acquisition, analysis, and conclusions (Munafò et al., 2017). The National Institutes of Health recognizes this and has prioritized rigor and reproducibility for all of its grant submissions (https://grants.nih.gov/reproducibility/index.htm). When planning an imaging experiment, there are a number of opportunities to introduce bias, starting with selecting the area of interest. The choice of a “representative” area or region of interest (ROI) has the most direct impact on reproducibility, as this contains the raw data that will be analyzed. There are a number of ways to minimize bias during acquisition. For example, when imaging a multiwell plate of cells, targeting a predetermined number of locations randomly within a well can help guard against unintentional bias. Many acquisition software programs enable users to capture images in fixed or random locations within a well in a reproducible manner. Another approach is to increase the sampling area by tiling across the entire well or dish, omitting any selection bias by comprehensively surveying the entire sample. On the other hand, when imaging a whole organism or tissue, it is important to consider whether the same areas need to be imaged for consistency, or whether many different areas should be surveyed. Additionally, researchers can blind themselves by labeling samples with codes so that the sample’s identity remains unknown during imaging acquisition. Finally, collaborations can help mitigate bias by increasing the number of individuals acquiring the data as long as the group agrees on the same methodology from the beginning, before starting the experiment. Similarly, collaborations where each researcher has a specific task (e.g., one person prepares the sample, another acquires the images, a third performs the processing and analysis) is less likely to result in biased results, especially when combined with blinding (Munafò et al., 2017). In any collaboration, the pipeline and expectations should be laid out and agreed upon before any experiments occur, otherwise, these preventative steps might not help in reducing bias.

Another contributor to bias is post hoc acquisition or analysis. This occurs when an unforeseen new result causes the data to be collected and/or processed in a way to bolster the new conclusion. The best way to avoid this kind of bias is to establish a rigorous acquisition and analysis pipeline before the experiment. One can use a preliminary set of imaging sessions to determine the specifics of the imaging and analysis workflow, such as how ROIs will be selected and acquired, which controls are necessary, how/if the images will be processed, how many samples are needed for sufficient statistical power, and what kind of analysis will be done (Figure 1 and Table 1). These preliminary data tests should be used to create the pipeline but should not be reused for analysis beyond the initial establishment of parameters for the experiment. Preliminary trials can also help to generate new ideas and hypotheses that can be tested with new samples, before the “official” data collection begins. Every data set, whether preliminary or not, should be acquired and analyzed consistently, both within individual experiments and across multiple biological and/or technical replicates.

TABLE 1:

Checklist of parameters to consider in image acquisition.

Controls

Waters, 2009; Schneckenburger et al., 2012; Cole, 2014; Dailey et al., 2014; Icha et al., 2017

Check for autofluorescence using a “no transfection,” a “no antibody,” or a “no dye” control Control for antibody specificity with a no primary antibody control Reduce or account for bleed-through signal Account for or reduce photobleaching Monitor cell health and phototoxicity

Hardware/software calibration

Model and Burkhardt, 2001; Zwier et al., 2004; Wolf et al., 2007; Waters, 2009; Cole et al., 2011, 2013; Stack et al., 2011

Confirm that the light source is fully aligned and evenly illuminating the sample Choose an appropriate objective lens (e.g., chromatically corrected) Minimize changes in filter sets to prevent shifts due to mechanical changes Check overlay and registration of channels in the software (use Tetraspeck beads) Monitor/control for lamp and/or laser intensity Double-check software settings/metadata before analysis: scaling, acquisition parameters, timestamps, etc. Use the correct size coverslip (most microscope objectives use no. 1.5 coverslips which are 0.17 mm thick), or use an objective lens with a correction collar adjusted to the thickness of your sample

Image quality

Lichtman and Conchello 2005; Waters, 2009; Lacoste et al., 2010; Wilson, 2011; Spring, 2013; Dailey et al., 2014

Avoid saturation Increase dynamic range to obtain best contrast Use the Shannon-Nyquist criterion for optimal spatial and temporal sampling Increase signal by ○ using bright, stable fluorophores ○ choosing a high NA objective ○ increasing light intensity (but watch out for photobleaching and phototoxicity) ○ choosing the optimal filter sets ○ matching refractive indices ○ using lowest magnification objective Decrease noise by ○ using cell culture media without phenol red or riboflavin ○ decreasing detector gain ○ eliminating ambient light ○ averaging frames on a laser scanning confocal

Acquisition consistency

Frigault et al., 2009; Lacoste et al., 2010; Cole, 2014; Icha et al., 2017

Monitor factors such as temperature, carbon dioxide concentration, humidity, pH, and air circulation, where applicable Maintain consistent focus/Z-depth, light intensity, sample preparation from session to session

Note that there is an important difference between post hoc analysis (which should be avoided) and adjusting one’s imaging protocol to improve specific parameters, such as boosting signal-to-noise ratios, and obtaining clearer images (which should be encouraged). The former biases acquisition to look for a specific desired result, whereas the latter utilizes improved methodology to obtain better overall data quality.

A quick aside about statistical analysis: From designing an experiment with enough statistical power to understanding which test(s) should be applied, biologists need to understand how to use statistics, either on their own or by consulting with a statistician (Klaus, 2015). Because statistical analysis is beyond the scope of the present discussion, a simple recommendation is to collaborate with a researcher trained in biostatistics. Not only will it be useful to have an expert to perform statistical analysis, but a statistician may further guard against bias if s/he is not invested in a particular outcome.

RIGOR IN IMAGE ACQUISITION

For rigor and reproducibility in imaging, researchers should be aware of the proper controls, hardware/software calibration, how to obtain quantitative images, and how to maintain consistency between imaging sessions. However, there is not a “one size fits all” image acquisition protocol due to the enormous variety of samples (cells, tissues, organisms), imaging modalities, microscope configurations, acquisition software packages, and experimental questions, to name a few. In addition, a reasonable understanding of the math and physics underlying image formation, fluorescence, and sampling that is required for rigorous quantitative image acquisition is beyond the scope of this Perspective.

To provide general guidance, Table 1 is a practical checklist of items to consider during an imaging experiment, and includes references covering these topics (Model and Burkhardt, 2001; Zwier et al., 2004; Lichtman and Conchello, 2005; Wolf et al., 2007; Frigault et al., 2009; Waters, 2009; Lacoste et al., 2010; Cole et al., 2011, 2013, 2014; Stack et al., 2011; Wilson, 2011; Schneckenburger et al., 2012; Spring, 2013; Dailey et al., 2014; Icha et al., 2017). Selecting the proper controls is key, not only for specific experimental conditions but also for technical imaging parameters, such as adjusting for background or autofluorescence and environmental factors. It is also imperative to consider image resolution and quality, which can be addressed by modifying the acquisition settings in the microscope software. The Shannon–Nyquist Criterion is a crucial example, as this can drastically affect the spatial and temporal resolution of the image (Figure 2A). In our example, the Zen imaging software calculated the Shannon–Nyquist sampling rate based on the objective parameters and wavelength of light, and determined that 3488 × 3488 pixels was the optimal image resolution. As a general rule, the proper Shannon–­Nyquist sampling rate will be at least twice the spatial or temporal resolution limit (Shannon, 1998).

FIGURE 2:

Proper sampling is critical for obtaining highly resolved data. A Xenopus laevis stage 9 (7 hpf) embryo stained for tubulin and histone H3 with Alexa Fluor 488 and 568 was imaged on a Zeiss LSM 780 confocal with a 20×/1.0 NA water immersion objective. (A) Top panel shows the full field of view with no zoom; scale bar = 50 μm. Bottom panel shows an inset region of interest (ROI) as marked by the white box; scale bar = 20 μm. Images on the left were imaged at 512 pixels × 512 pixels. Images on the right were imaged using Shannon-Nyquist sampling at 3488 × 3488. The bottom-left image is an example of undersampling or aliasing. (B) Left panel shows the raw image acquired. Right panel shows an enhanced image, where the brightness of the entire image has been increased for both channels to allow for increased visibility.

COMMON OBSTACLES DURING IMAGE PROCESSING/ANALYSIS

Because image processing and analysis pipelines are highly variable, the discussion below will remain generalized but practical. The most common misstep during image processing is improper manipulation, either through nonuniform or selective processing, or by changing raw values indiscriminately. Filters, thresholding, and masks can be extremely useful during processing; however, any changes should be applied uniformly to the entire field of view (FOV) and to all images of an experiment in the same manner. The exact processing steps should be documented in detail (more discussion below).

A common mistake made by researchers is changing the raw values of an image. Each pixel has an intensity value that can be plotted in a histogram. The range of intensity values is defined by the bit depth, often displayed as a grayscale image and referred to as “levels of gray.” Most microscopy images contain either 8-, 12-, or 16-bit pixels, where 8-bit pixels have 2⁸ = 256 and 16-bit pixels have 2¹⁶ = 65,538 shades of gray (Spring, 2013). Bit depth is set at acquisition by the detector (camera or photomultiplier tube [PMT]), but can be altered postacquisition. As a general rule of thumb, increasing the bit depth of an image during postacquisition processing does not result in higher quality images. Therefore, it is imperative to set the desired bit depth during acquisition if quantitative imaging is a priority.

Raw pixel values can be changed in a number of ways. Some examples include saving an altered image after changing the display curve, using nonquantitative image software like Photoshop to apply display curves, or exporting the raw file to a compressed version (e.g., jpeg; Cromey, 2013). Although tools in image acquisition and analysis software are helpful in image enhancement, it is important to perform analysis with the original raw file (or processed versions of the raw file), to save in the original format (if possible), and to regularly check the histogram after any changes to the file. The raw, acquired image does not need to look “pretty” and ready for presentation—these aspects can be enhanced and/or added after the image has been analyzed and processed for its quantitative data (Figure 2B). Afterward, when preparing figures for publication or presentation, it is acceptable to use graphics software programs like Adobe Photoshop or Illustrator to adjust the qualitative aspects, such as brightness and contrast. Ideally, the control and experimental images are treated and adjusted identically. Moreover, these enhancements should still reflect any quantitative data taken from the images, be properly documented in the figure legend or methods, and be done only in order to help the viewer understand the experimental results.

When using software designed for scientific image processing and quantification, understanding what is being done when various filters or macros are applied is imperative. Most software packages have reference materials that can be extremely useful. For instance, ImageJ/FIJI has an extensive forum and user documentation that allows researchers of all levels to research and understand all of the plugins (http://forum.imagej.net/). Commercial software packages typically have online tutorials and/or come with remote technical support. Finally, look for resources within your department, core facility, and institution. Very likely, there are a number of experts who can assist and/or teach you how to process and analyze images. In many cases, an expert collaborator is the best option.

CLARITY IN DATA PRESENTATION AND COMMUNICATION

Methods

Executing a rigorous imaging experiment is only half of the equation; the other half is communicating your methods to the scientific community. A thorough and transparent Methods section is the key to reproducible science. Keeping detailed notes on image acquisition, processing, and analysis parameters is essential. There are also tools available online to encourage protocol sharing and feedback, such as journals dedicated to publishing protocols or open science websites, like protocols.io (www.protocols.io/). Fortunately, many journals have removed word limits from the Methods section, so that researchers can be as thorough as possible. In addition, new tools, such as research resource identifiers (RRIDs; https://scicrunch.org/resources) aid in rigor and reproducibility by systematically assigning unique identifiers to antibodies, organisms, cell lines, and tools, including microscopes. It is a streamlined way for materials to be traced and for researchers to speak the same “methods language.”

Despite these improvements, challenges remain, mostly rooted in tradition or lack of awareness. First, many researchers accidentally leave out intrinsic knowledge or “secret sauce” steps from published protocols. These are often steps a researcher performs almost absentmindedly, such as a heat-treating fetal bovine serum (FBS) for blocking antibodies or using fresh fixative for each experiment, but may be critical to the success of the experiment. One solution is to have another member of the lab follow a written protocol and try to reproduce the results. Any missing steps should become apparent during this process. Second, there is often insufficient detailed documentation of acquisition, processing, analysis, and statistical parameters or how-to steps, resulting in an experiment that cannot be repeated. Finally, it is common to fall down the “black hole” of references—that is, Methods from one paper is cited, which cites another paper, which cites another. It is good practice to cite the primary source of the methods, not a secondary or tertiary source. Even better, cite the primary source but also detail your protocol so that others can immediately understand what was done without searching through another study. Table 2 shows a checklist of items to include in the Methods section.

TABLE 2:

Checklist of items to include in a detailed Methods section.

□ Microscope (Make, Model) □ Inverted or upright □ Objective: manufacturer, magnification, numerical aperture, immersion, correction (if any) □ Light source(s) □ Filters, dichroic mirrors, beam splitters □ Additional hardware used, e.g., stage motor, incubation chamber and set-up, Piezo stage or objective □ Acquisition software and settings □ Confocal, Multiphoton systems: ○ Detector type (PMT, NDD, etc.) ○ Laser line(s) used ○ Pixel resolution ○ PMT gain □ Widefield systems (with cameras): Camera manufacturer, model Note if binning, gain was used □ Separate, detailed transfection and antibody staining protocols, including constructs used, antibody manufacturer(s) and ­concentrations □ For multi-channel acquisitions, note whether channels were acquired simultaneously or sequentially □ Strategy for selecting regions of interest (if any) □ Image processing steps (if any) □ Data analysis pipeline, including statistical methods (if any) □ Software used to make figures, including processing steps, if any (inversion, brightness & contrast, maximum intensity ­projection, etc.)

In addition to asking a lab member or third party to validate protocols, there are other steps that can help assure the reproducibility of imaging experiments. One can ask a microscopist—either an expert in your lab or another lab, or core facility staff—to review the Methods section and the imaging experiments performed. S/he will be able to point out the important aspects that need to be documented. Another resource is the metadata embedded in your raw image file, which includes most of the important imaging parameters. Most acquisition software programs automatically create and embed metadata with each image, but it is good practice to confirm which parameters are saved with your specific software program. Finally, if multistep processing or analysis is used, it is advisable to make use of macros or custom code to track image processing. One can also save multiple versions of images with the processing details in the file name to track changes and/or use simple “README” text files to log processing steps.

Figures

Communicating results in an effective and truthful manner through figures can also influence reproducibility. Below are a few suggestions on how to ethically present images for publication (Casadio, 2015). Similar to processing, any image adjustment (e.g., brightness/contrast) should be applied to the entire image. Adding or removing select objects (e.g., cells, dust) from the image is inappropriate and unethical. Data handling/exporting can also interfere with image quality and is discussed in further detail below.

In terms of presentation, there are some general guidelines to follow regarding color choice for fluorescence images (Johnson, 2012). Contrast should be considered first and foremost. Grayscale is the easiest to see, whereas blue-violet is the worst because the human eye has fewer receptors designed for that range (Mollon, 1982). Another argument for grayscale is that human perception of color is generally nonlinear, so conveying intensity information using a pseudocolored image may not be as effective (Pokorny and Smith, 1970; Welland et al., 2006). One should avoid red/green color combinations out of consideration for color-blind colleagues. Magenta, green, cyan, yellow is a safe and increasingly popular color palette for multichannel images. Another option with a low-intensity fluorescence image is to consider inversion, so that the signal is gray/black on a white background (Johnson, 2012). This is especially useful for talks or poster presentations where overhead lighting cannot be dimmed. In addition, color coding (a.k.a. “heat maps”) can be used to convey information more readily, such as intensity differences, saturation, and depth (in Z). Finally, when presenting a multichannel image, it is good practice to show each channel in its own panel (in grayscale), with the final panel being a merged, multicolor image (Figure 3). Of course, each figure should contain at least one scale bar. Whenever possible, the scale bar should remain consistent within a figure panel or set of images, although it is occasionally necessary to change the scale when presenting a specific magnified ROI.

FIGURE 3:

For multichannel images, present each channel individually in grayscale and use color for merged images. Bovine pulmonary artery endothelial (BPAE) cells were stained with DAPI (4′,6-diamidino-2-phenylindole) to mark DNA and Alexa Fluor 488 phalloidin to mark actin microfilaments (Thermo Fisher; FluoCells Prepared Slide #1) and imaged on a Zeiss LSM 880 confocal with a 63×/1.4 NA oil immersion objective. Scale bar = 20 μm.

There are two major challenges to image data presentation in a publication format: displaying three-dimensional (3D) data and time-lapse images. For 3D data sets, some choose to mash the z-planes together into a maximum intensity projection (MIP). Although a projection can be a compact way to present 3D results, depth information is lost (Figure 4A). Also, depending on the algorithm used to make the MIP, it could be an inaccurate representation of the signal intensity (i.e., if the pixels from all planes were summed together). To better represent Z dimensionality, depth color coding can be used to indicate where each pixel is in a z-stack (Figure 4B). Alternatively, volume rendering via imaging software (e.g., Bitplane Imaris, Arivis Vision4D) can be used to introduce volume projections, highlights, and shadows to a 2D image (Figure 4C). For time-lapse images, one can present key time points as individual panels. However, it may be more effective to employ tools like kymographs to analyze and display particle movement (Chiba et al., 2014).

FIGURE 4:

Alternative forms of representing three-dimensional data. An unstained Drosophila melanogaster eye (Carolina Biological Supply Company) was imaged on a Zeiss LSM 880 confocal microscope with a 20×/1.0 NA water immersion objective. Autofluorescence in the GFP channel was detected following excitation with the 488 nm argon laser. A 60-μm-thick z-stack (89 slices) was acquired. (A) A maximum intensity projection of the z-stack rendered by Zen software (RRID:SCR_013672). (B) Depth color coding of the z-stack (Zen), with blue indicating zero depth and red representing the deepest section (∼60 μm). (C) MIP of the z-stack tilted inside of a volume box, generated using Bitplane Imaris software (RRID:SCR_007370). Scale bars = 50 μm.

PRACTICAL TIPS FOR DATA MANAGEMENT

Acquiring images using modern light microscopes is almost exclusively done with software programs. The rate of data acquisition has increased dramatically over the past few years, as camera resolution and speed have improved, as well as the development of new imaging technologies such as lightsheet and superresolution techniques that require substantially more computational power (Swedlow, 2012). As a result, proper data and file management are critical to rigor and reproducibility.

A classic data handling problem is disorganization. Disorderly or inconsistent file naming, sorting, and storage result in multiple issues, ranging from confusion and time lost looking for data to lack of transparency. This is problematic not just for the person performing the research, but for others hoping to repeat the experiments. One common example of this is when a researcher moves on, leaving behind unclear notes (if any at all). His/her colleagues are left with little to no direction on how to repeat an imaging experiment, resulting in wasted time and money (in terms of reagents, microscope usage, and work hours). The following recommendations are based on an online resource (Kubow, 2016) and personal experience; they are common-sense, but not exhaustive, and should be implemented and practiced regularly. First, there should be a consistent system for file naming and organization that is easy to follow. For example, each experiment should be given an individual folder that is labeled with the date of the experiment or image acquisition as well as a short description of the experiment. It can also be helpful to organize experiment folders by project or type of assay, such as live versus fixed imaging. Individual image files should be named and numbered according to the experimental conditions, as well as any additional identifiers necessary such as genotype, labeled proteins, and sample number. A consistent naming system is also important in maintaining file organization when taking multiple images from the same sample at different magnifications. Second, using resources like an electronic lab notebook, spreadsheet, and/or lab database accessible to all lab members will help everyone bring their note-keeping up to a consistent standard and make experiments trackable (or at the very least, legible). Third, text README files can be used and stored within folders to make annotations regarding the content of the folder, acquisition parameters used, and processing steps (if any). Additional details not recorded in the file naming can also be noted here. These text files can be cross-referenced to a lab notebook or spreadsheet for clarity.

Good data management from the beginning of each imaging experiment will pay dividends at the publication stage. Increasingly, journals are asking authors to post or share complete raw data sets as a condition of publication. Raw data sets should also be maintained in the unfortunate event of fraud accusations. OMERO from the Open Microscopy Environment (www.openmicroscopy.org/) and the Image Data Resource (IDR; https://idr.openmicroscopy.org/about/) are sites that serve as a central repository from acquisition to publication and can help in image data management (Williams et al., 2017). Tools like OMERO and IDR are critical in aiding scientists toward transparency and reproducibility.

CONCLUSION

Light microscopy is a powerful tool in cell biology. To use it properly, it is valuable to understand how every step of an imaging experiment, from experimental design to presentation and file management, plays an integral role toward faithful observation and reporting of biological phenomena as well as toward reproducibility. The guidelines presented here are by no means exhaustive. Every imaging experiment comes with its own challenges, in which case it is best to consult with published literature, an expert in your field, and/or microscopy core staff at your institution. Simple adjustments and careful note taking can easily increase rigor in imaging methods, a goal that every person who uses a fluorescence microscope for his/her research can attain.

Abbreviations used:

2D

two dimensional

3D

three dimensional

BPAE

bovine pulmonary artery endothelial

DAPI

4′,6-diamidino-2-phenylindole

FBS

fetal bovine serum

FOV

field of view

IDR

Image Data Resource

LSM

laser scanning microscope

MIP

maximum intensity projection

NA

numerical aperature

NDD

non-descanned detector

OMERO

Open Microscopy Environment image software

PMT

photomultiplier tubes

ROI

region of interest

RRID

research resource identifiers