Overview of Image Analysis, Image Importing, and Image Processing Using Freeware

Quantitative image analysis turns microscopic data that might otherwise be merely descriptive into reliable mechanistic information. Such reliable structural information is a significant part of any biological investigation. “Structural information” refers to the patterns, shapes, contours, and relationships of objects made visible by their mass, as by staining, or by an electronic/computational technique applied to a biological specimen. Reliability of interpretation often depends on the statistical analysis of multiple images of the same structure. Great strides have been made in recent years in the ability to analyze multiple images quantitatively, due to the ease of acquiring images in digital formats and exporting them to various software packages. The types of analysis most often used involve: (1) elimination of background noise; (2) averaging of multiple specimens by superimposition to determine structural regions in common; (3) morphometric analysis of common measurable features obtained from many similar specimens of the same structure; (4) analysis of expression patterns of gene products over time or after an experimental treatment; (5) subcellular or tissue localization of gene products; and (6) quantitative measurements of rates of movements of cells or subcellular structures or molecules. Image analysis is also used to compare various quantitative aspects of biochemical data when such data are obtained in a visual format, such as a gel or blot. While this unit will focus on the analysis of microscopic material, some of the same tools and programs can also be useful when applied to images of biochemical data. The Commentary section of this unit will present an overview of the types of analysis that have been useful in the past as well as comments on emerging applications (see Background Information).

To maintain the value of the unit for a wide variety of investigations, the focus will be on basic techniques common to all biological applications for extracting reliable information from digital images, rather than on the equipment and software used to do this. Brief step-by-step protocols for importing data and for the first steps of data manipulation in the most common analytical applications, using two different freeware applications, NIH image (see Basic Protocol) and ImageJ (see Alternate Protocol), are provided.

BASIC PROTOCOL

IMAGE PROCESSING USING NIH IMAGE 1.62

The following is a protocol to import and process digital images for analysis or for production of figures using NIH Image 1.62, a freeware application available from the NIH at http://rsb.info.nih.gov/nih-image/. NIH Image is an image-processing program in the public domain designed for Macintosh computers. Digital images acquired in any format can be imported into NIH Image 1.62 if they are saved in a TIFF format. Once imported, NIH Image can be used to measure area, mean, centroid, and perimeter of user-defined regions of interest. The program includes tools for automated particle analysis and for measuring particle path lengths and angles. Density calibrations as compared to standards with user-specified units are also possible. Analyses can be printed, exported to text files, or copied to the clipboard for export to other compatible software.

NIH Image is most useful for black-and-white images. It can also be used to process such images by averaging to reduce background, by adjusting contrast and brightness, by smoothing pixelation, and by cutting and pasting to arrange a gallery of similar specimens for comparisons. NIH Image provides useful tools for editing and transforming digital video images in black and white obtained from confocal microscopy. NIH Image can also be used to transform analog video to digital sequences for archiving. Some of these applications require additional hardware, including the appropriate computer ports to plug in the analog device.

This protocol covers the step-by-step process used to import various types of digital data into NIH Image to process and edit video sequences. NIH Image is capable of many other processes, and detailed instructions for these can be obtained from the NIH Image Manual at http://rsb.info.nih.gov/nih-image/manual/.

Materials

Color-capable Macintosh computer with ≥2 Mb of free RAM and the following programs:

• MacOS v. 7 or later, or OS X capable of running programs in Classic mode

• Web browser, e.g., Netscape or MS Internet Explorer

• Stuffit Expander

• Adobe Photoshop

Digital images and image sequences: these can be translated from a wide variety of file types, although TIFF formats are easiest to work with; importing from many other document types is handled through an Input-Output Macro that downloads with NIH Image

Download freeware

• 1. Go to the Web site http://rsb.info.nih.gov/nih-image/, download NIH Image 1.62, and decompress (“Unstuff”) the file onto the desktop.

• 2. Restart the computer.

• 3. On the desktop, highlight the icon for NIH Image application (a black microscope).

• 4. Under the File menu, select Get Info, then select Memory from the submenu which appears. Set Minimum Size to 75,000, and set Preferred Size to 100,000.

  Smaller memory allocations will work for images that use less RAM, but the author of this unit has found that some operations can fail, incomplete loading of digital sequences can occur, or sequences can be loaded in the wrong order. For very large digital sequences, even more memory may be needed.

• 5. Open the application by double-clicking on the icon.

Import or open the image to process or analyze

To open Bio-Rad confocal files, electron micrographs from a scanner and some other specific formats

• 6a. Use the Input/Output Macro as follows. Choose Special from the menu bar at the top, then select Load Macros from the resulting submenu from the NIH Image folder. When the dialog box appears, open the Macros folder that downloaded automatically when NIH Image was loaded and which will be found inside the NIH Image folder. Scroll down within the Macros folder and highlight Input/Output macros.

  Under Special, a list of file types that can be imported will now be seen. These include TIFF files, electron-microscopic images from Siemens, and confocal files from Bio-Rad.

• 7a. To import Bio-Rad confocal digital sequences or single frames, select Import BioRad 600 Z Series. In the dialog box, select the folder containing the Bio-Rad image sequence to be opened and scroll down in the window to find the specific file. Select Custom 512 and “8-bit” (the Bio-Rad will give a single icon for the entire image sequence).

• 8a. To import files from a different confocal, e.g., Zeiss images saved as TIFF sequences, select Open All; in this case, all the images in a sequence must be placed into a single folder.

To open TIFF files

• 6b. First, attempt to open the file by selecting Open from the File menu.

• 7b. If the file cannot be opened as in step 6b, select Import from the File menu. In the pop-up menu that appears, select TIFF and Custom 512 × 512 (or other image pixel size as necessary under the Set button).

  It is possible to open all the files in a folder by selecting Open All.

• 8b. To work with Metamorph or Zeiss images, save them in the TIFF format in the Zeiss LSM program by selecting Export from the File menu in that program. Open the saved TIFF as in step 6b or 7b, above.

  It is also possible to download the program Browser for free from the Zeiss Web site. This smaller program has only some of the applications included in the full Zeiss LSM program, but can help with saving images in alternative formats.

  For images captured in two or more channels, it is better to save each channel separately and then work with them in ImageJ (see Alternate Protocol).

Manipulate the images

• 9. Upon opening the file, allow the image sequence to load onto the screen.

• 10. To play through the sequence, open the Stacks menu and select Animate. Click the mouse on the image to stop the playback. To toggle through the images in the forward direction, click on the “greater-than” key (>) without pressing the Shift key. To toggle through in the reverse direction, click on the “less-than” key (<) without pressing the Shift key.

  The keyboard numbers can control the speed, with 1 being the slowest and 9 being the fastest.

• 11. To work with individual frames, open the Stacks menu and select Stacks to Windows, which will cause the images in the sequence to be posted on the screen individually.

  Using this view, it is possible to delete an image or process any image in the stack.

  IMPORTANT NOTE: Be careful to put the images back in order on the screen before making them into a stack (see step 12).

• 12. To put a series of individual images into a sequence, open the Stacks menu and select Windows to Stacks. Make the sequence as short as desired by deleting images. Add letters/arrows to the first image in a sequence, or to each image in the windows.

• 13. To crop or process an entire sequence, load the macro Stacks (Special>Load Macros) menu and select Stacks from the Macro folder in the popup window; a new set of commands appears in the scroll-down menu under Special.

  Most of these are self-explanatory. The Crop command is one of the most useful, as it will make it possible to crop all images in the stack to exactly the same size.

• 14. To create a series of cropped images to present as still frames for publication as a figure, or for analysis of a printout, use the rectangular tool from the tool bar and draw the region of interest (ROI) on each frame of the sequence when it is in the stack mode on the screen. For example, if every sixth image is desired, draw the square on the first image, copy the ROI and paste into a new file (typically, at this stage, the author uses Adobe Photoshop to create the new file to paste in the copied image). Without moving the rectangular tool, toggle five frames, and copy the same ROI from the sixth frame. Continue until all images in the series have been pasted into the new still-frame document.

  Most digital cameras and confocals produce images with 72-bit resolution. This does not produce attractive images for publication. To improve the bit mapping in Photoshop, open the Image menu and select Image Size. A resolution of 400 is usually more than enough to produce a smooth figure. Photoshop’s Gaussian Blur option (selected by opening the Filter menu and selecting Gaussian from the Blur submenu that appears) equalizes pixels at a radius that can be selected by the user. This will smooth the sharp edges of adjacent pixels where they differ by large contrast in gray scale between pixel edges.

• 15. Perform frame averaging by merging digital images.

  Background subtraction is best performed during image capture. However, some frame averaging can be done by merging digital images. Frame averaging decreases random variations in pixel intensities.

  Merging digital images after capture is time-consuming. The author uses Adobe Photoshop; in that program, open two or more frames to be averaged. Paste one image on top of the other. Open the Layers palette by selecting Show Layers from the Window menu. In the Layers palette, set opacity at 50% for each of two images, 25% for each of four images, etc. One can adjust the opacity levels to check that overlay is exact. It is best to align a stable object common to all frames. Select Merge Visible from the Layer menu at the top of the screen to compile the image.

• 16. Save the new sequence in NIH Image with a different name by opening the File menu and selecting Save As.

  If the QuickTime radio button is selected in the Save As dialog box, encoded information may be lost when transporting back into TIFF. QuickTime format is very useful as it can be linked into Powerpoint for presentations or posted on the Web. In the Save As QuickTime pop-up menu, select Video and Color under Compressor, and High or better on the Image Quality slider bar. The optimum speed depends on how many images are in the stack and how fast the change is of the object under study. Generally, a 10- to 100-fold increase in the speed of capture is useful. For images captured every 10 sec, the author uses 6 to 10 frames/sec. Marking the video with a “keyframe” at regular intervals allows one to do manipulations of the sequence in Adobe Premiere, a movie-editing software package, or in Flash by Dreamweaver. Every 10 frames is often a useful frequency.

ALTERNATE PROTOCOL

IMAGE PROCESSING USING ImageJ

ImageJ is a more sophisticated image processing system than NIH Image (see Basic Protocol) and can handle color images and superimposition of digital sequences. ImageJ can be downloaded from the NIH Web site at http://rsb.info.nih.gov/ij/.

ImageJ has additional capture macros that can accept data directly from a camera, such as the Hamamatsu Orca 12-bit camera, and from Scine frame grabbers. It also has TWAIN for direct download from most image scanners. It can calculate surface area and line distances, measure angles, and obtain radial profile plots. ImageJ can identify edges and measure changes in the Analyze>Measure descriptors menu. Useful macros include Combine Images to Create Mosaic, which allows digital background subtraction, and Image Calculator Plus, which can combine images of different sizes in different formats.

The ImageJ Web site lists a number of other sites where additional image processing freeware is available. An interesting site is http://bigwww.epfl.ch/thevenaz/differentials/. The Image Differentials plug-in available at that site allows one to perform several differentiation operations on images seen as continuous functions, even though they are stored as discrete pixels. The general principle is to construct the continuously defined function

$$f(x,y)=\mathrm{\sum }k,lc[k,l]\phi (x-k,y-l)$$

As for NIH Image (see Basic Protocol), memory has to be set at a high value (at least 100,000) if digital sequences of 100 or more frames are to be processed. A caveat is that Java allocates memory from the Systems heap, which means that too much memory allocated to ImageJ can reduce the amount of memory available for loading images, but without enough memory, one can run into error messages when processing QuickTime image series. It is therefore necessary to monitor memory under the Apple menu when running ImageJ and make alterations as necessary according to the computer and image sequence with which one is working.

Materials

Color-capable Macintosh computer or PC with 64 Mb (256 Mb recommended) of free RAM and the following programs:

• MacOS v. 7.6 or higher or Windows 95 or higher with Java Runtime Environment 1.1.3

• MRJ 2.1 (download from http://www.apple.com/java/)

• QuickTime for Java (installed with Mac OS 9.04 or higher, or can be installed from QuickTime 1 “custom” Install menu)

1. Once ImageJ is loaded, open the application and open files from within it. If images have been saved as QuickTime files from the original sequences, open by selecting Import from the File menu. To open sequences saved as TIFF, first put all the images into one folder, then open from inside ImageJ by selecting Import from the File menu, then selecting Image Sequence and opening a file within the folder, which will in turn open the whole sequence.

2. Become familiar with the most commonly used image processing procedures in ImageJ (main menu items listed first, then submenus):

   • Image >Adjust > Brightness/Contrast

   • Process >Subtract Background (this requires loading of a background image that is devoid of any structures of interest)

   • Process > Filters > Gaussian Blur

   • Analyze >Measure

3. Rotate digital sequences by selecting Rotate from the Image menu for export to QuickTime, which can then be played in Powerpoint presentations or uploaded onto the Web in the orientation that makes most sense in comparison to other image sequences or to a model/diagram.

   In ImageJ, if a processing routine is applied to a digital sequence, one will be stuck with the results (i.e., this is destructive editing). Therefore, one should always keep a file of the original data separate from the sequence being processed, and save often to preserve desired changes in case subsequent processing is not useful.

4. Save the new image sequences by selecting Save As from the File menu. For a TIFF series, select Image Sequence from the submenu that appears. For QuickTime, select options from the popup menu:

   Compressor:	Video Color
   Quality:	Medium to High
   Timing:	Frames per Second 6–10 Keyframe every 10

   Further processing and analysis can be performed in ever increasingly complex software applications.

COMMENTARY

Background Information

Image analysis to determine the structure of molecules and particles

Prior to the advent of digital image processing, various manual manipulations were required to extract quantitative information from structural data. The term “structural data” in this context refers to photographic or digital images of biological specimens, ranging from whole animals to tiny single molecules. For example, diffraction patterns can be obtained by passing laser light through film containing images of subcellular structures displaying repetitive patterns resembling crystalline arrays, such as the catalase crystal in the peroxisome (Bearer and Orci, 1986), the acetylcholine receptor channel (Unwin, 2002), and the protein coat of viruses (DeRosier and Klug, 1972). Reconstruction of the “ideal” crystalline structure is obtained by solving the diffraction pattern in reciprocal space. Another approach superimposes multiple electron microscopic images of the same structure, which “averages” the image, decreasing the intensity of nonrepeating structures, including those of unrelated background particles or of the movable parts within a fluid region of the structure. This technique has commonly been used in the analysis of electron micrographs. Multiple exposures of the same photographic paper with different images or rotations of the same image superimposed upon itself identifies repeating structures and internal symmetries within a structure. This technique was used to discover the eight-fold symmetry of the endothelial fenestral diaphragm, responsible for the passage of nutrients from blood to tissue in human organs (Bearer and Orci, 1985).

Statistical analysis of biological specimens, also known as “morphometrics,” involves compiling measurements of multiple specimens of the same structure and then mathematically deriving the average size and other statistical values such as standard deviation within the population studied. Such mathematical compilations rely on the acquisition of a large number of measurements from multiple specimens, which until very recently was performed by hand. However, only relatively small sample sizes are easily analyzed by hand, since this requires a large time investment.

Recent advances in structural determination of complex cellular components not amenable to crystallization include tomographic reconstruction of electron-microscopic images. In this approach, images are captured from thick sections of biological specimens embedded in either plastic or vitreous ice using an intermediate-voltage electron microscope. The section is photographed at various angles of tilt, and the 3-D structure assembled digitally. The structure of the neuronal synapse is yielding to this approach (Harlow et al., 1998), which was recently cited as one of the ten top scientific advances in 2002 by the journal Science (News and Editorial Staff, 2002).

New technology exploiting digital image processing includes laser scanning and two-photon confocal microscopy, which allows elimination of the out-of-focus light from fluorescently labeled specimens. Confocal microscopy allows optical sectioning and thereby the acquisition of sharp images of objects deep within thick specimens (White et al., 1987). In confocal microscopy, the image is filtered before capture. Confocal microscopy relies on monochromatic laser excitation of fluorescent tags, and collects only in-focus emission using a pinhole to block out-of-focus emitted light. In two-photon confocal imaging, a recent advance, excitation and collection occurs only from the same optical section of interest, thus prolonging the length of time that a living biological specimen can be observed without laser damage (Kaech et al., 2001; Meyer et al., 2003). New computational approaches (Meta and Multi-Tracker options in the Zeiss 510 LSM package) narrow the bandwidth of detected emitted light, thus providing the possibility of imaging multiple fluorochromes with the same excitation and even only slightly differing emission spectra in the same specimen. An alternative way to eliminate out-of-focus light is deconvolution, which involves digital processing of a series of images obtained by focusing up and down in the same microscopic field. In this approach, out-of-focus light is eliminated digitally after image capture (Hiraoka et al., 1990). Deconvolution requires a motorized stage to collect a series of images in the z-plane (Kam et al., 2001). These images are then processed digitally to extract in-focus structure and detail and eliminate fuzz from out-of-focus or autofluorescent emissions. The equipment is less costly than that used for confocal, but collection times are longer, and thus deconvolution is not as suitable for living specimens undergoing rapid changes. Both of these approaches allow visualization of structures by fluorescence microscopy that would not be resolved by conventional microscopy. Useful texts on microscopy include Matsumoto (1993) and Inoué and Spring (1997). Microscopy courses at the Marine Biological Laboratory in Woods Hole, Massachusetts (http://www.mbl.edu/) are also excellent for viewing demonstrations of the latest equipment.

Video microscopy, pioneered by Robert Allen in the late 1970s using Nomarski optics (differential interference contrast or DIC), is capable of detecting structures that are theoretically below the resolution of the light microscope if lenses with high numerical apertures are used (Allen et al., 1979; Hayden and Allen, 1984; Brady et al., 1985). Such lenses have narrow focal planes, and often have short working distances as well; thus, they have not yet proven useful for imaging objects deep within thick specimens. However, like confocal microscopy, video imaging with either DIC or fluorescent tags can be performed on live specimens. Analysis of such sequence data can provide information about structural changes over time, such as the rates of changes in cellular surface contours (Zhang et al., 2002), the velocities of movement of intracellular particles such as viruses (Bearer et al., 2000), and even the dynamic behavior of purified cytoskeletal filaments during interactions with motors or severing factors (Vale et al., 1985; Bearer, 1991). In this case, microscopy becomes an assay for enzymatic activity.

An Open Microscopy Environment (OME) is being developed as an informatics solution for storage and analysis of optical microscope data (Swerdlow et al., 2003). This public resource hopes to provide image analysis automation, modeling tools, and large sets of data for mining biological material. The goal is for data to be stored as an XML-encoded standard file which can be uploaded and processed by any software. The URL for this project http://www.openmicroscopy.org.

Image analysis to determine clues of function from subcellular location and behavior

In addition to providing information about the exact measurements of the “ideal” for a particular structure, image analysis is also useful to discover expression patterns, location within cells, and dynamic behaviors of molecules both large (proteins and RNA) and small (ATP and calcium ions). The expression patterns and behaviors of proteins inside cells detected by microscopy complement those detected by biochemical and molecular approaches, and readily provide functional information about a protein or gene product.

Because the genome projects have identified a large number of novel genes whose function is not easily predicted from DNA sequence, functional clues derived from structural analysis have become increasingly fashionable. Morphological data are being collected in high-throughput screening assays to identify key proteins from large data sets. Such key proteins can then be selected for analysis by more time-consuming approaches in subsequent experiments.

Strategies using localization to derive functional information about individual proteins and thereby select interesting proteins from large data sets were originally developed using antibodies to localize the protein (Bearer and Alberts, 1988; Miller et al., 1989). Antibody staining provides evidence concerning the type of cell within a tissue that expresses the protein and the subcellular location of that protein in cells. In conjunction with other stains, including antibodies, against other cellular proteins whose function is known, the contribution of the unknown protein to a particular aspect of cellular behavior can be inferred.

In addition to antibody staining, several other methods have been developed to detect gene expression. In situ hybridization (ISH) to detect mRNA expression speeds up the analysis because it uses the DNA as a probe and bypasses antibody production, which is time-consuming and not always reliable. However, results from ISH do not produce information about protein function, only about relative expression between cells in a tissue.

Another method, transfection and overexpression of fusions of the target protein with tags such as green fluorescent protein (GFP), allows observation of the dynamic behavior of a protein in the living cell. This method has most commonly been applied to individual proteins once they are determined to be of interest and is not a commonly used screen, as construction of the GFP fusion takes time. Another powerful approach is to label a protein whose function is known with GFP and then observe the effect of overexpression, deletion, or mutation of the unknown protein on the GFP-labeled structures. Finally, labeling of a protein within a larger complex of proteins allows microscopic tracking of the complex within cells, a strategy used to study the movement of viruses within cells (Bearer et al., 2000).

Questions that microscopic analyses usually address include: (1) the timing of expression changes after stimulation, (2) the types of cells within a multicellular tissue in which the gene product is expressed, and (3) the location of the gene product within the cell and its association with other cellular structures of known function. The function of an unknown protein or small molecule can be assessed using substrate cells that are transfected with some other protein whose function is already known, and monitoring the effect of the novel protein on the behavior of the known protein. This approach has proven very useful in identifying proteins whose function affects the secretory pathway, such as Golgi markers (Lippincott-Schwartz, 2001; Kreitzer et al., 2003) or the actin cytoskeleton (Westphal et al., 1997; Fischer et al., 1998).

While all of these types of structural expression analysis have in common the use of the microscope to obtain images for further analysis, the exact method and interpretation of results differ depending on the question being addressed. Microscope assays can be divided into two subsets: those acquired from living specimens in real time or by time lapse, and those acquired from fixed specimens. These assays use a variety of different microscope technologies and experimental methods. This is a fast-moving area of research, and specific applications and equipment are likely to have evolved between the writing of this unit and its appearance in print.

Critical Parameters

Image analysis is only as good as the images being analyzed. It is often a fruitless waste of time to use digital image processing to improve poor images. Therefore, before embarking on an analytical or processing project, images should be carefully scrutinized for: (1) focus; (2) good signal-to-noise ratios; (3) reproducibility; and (4) background contamination. Artifacts to be avoided are those from: (1) laser or other light damage; (2) aggregation of fluorochrome marker; (3) nonphysiologic localizations of fusion proteins; (4) movement of the stage during collection of video images; and (5) other effects of the experiment on the cell that might damage it or perturb its normal processes.

Scientists engaged in monitor-based digital processing tasks should ensure that the monitor’s colors, brightness, and contrast are set appropriately. Instructions for this come with the Adobe Photoshop package, and must be tailored to the individual monitor being used. Printing of processed images requires that the printer be coordinated with the monitor as well. This is dependent on the manufacturer’s specifications for the particular printer and monitor being used. Monitor-based processing requires a low-light work area and a maximum-resolution screen.

Time Considerations

Digital image processing and analysis are usually more time-consuming than expected. The better the original image(s), the less time is required for processing. The type of data being extracted also define the length of time that must be invested. If software routines are available, then applying them can be fast once a sequence of steps is defined. It usually takes a graduate student a week to learn a routine, then a half-day to a day to make a processed QuickTime movie from a preselected, excellent series of images.