Motion-enhanced, differential interference contrast (MEDIC) microscopy of moving vesicles in live cells: VE-DIC updated

Summary

Video-enhanced DIC microscopy (VE-DIC) with background subtraction has made visible many structures and processes in living cells. In VE-DIC, the background image is stored manually by defocusing the microscope before images are acquired. We have updated and improved VE-DIC by adding automatic generation of the background image as a rolling average of the incoming image stream. Subtraction of this continuously updated 12-bit background image from the incoming 12-bit image stream provides a flat background which allows the contrast of moving objects, such as vesicles, to be strongly enhanced while suppressing stationary features such as the overall cell shape. We call our method MEDIC, for motion-enhanced DIC. By carrying out background subtraction with 12-bit images, the number of graylevels in the moving vesicles can be maximized and a single look-up table can be applied to the entire image, enhancing the contrast of all vesicles simultaneously. Contrast is increased by as much as a factor of 13. The method is illustrated with raw, background, and MEDIC images of moving vesicles within a neurite of a live PC12 cell and a live chick motorneuron.

Introduction

It has been known since 1981 that the contrast in DIC images of live cells, microtubules, and small vesicles can be enhanced by background subtraction of images from a video camera(Allen et al., 1981, Inoue, 1981). Contrast enhancement was initially achieved by extending the range of the gain and offset controls of the camera; the extended offset was used for background subtraction and the gain control was used to expand a narrow section of the grayscale. However, it was quickly found that no single offset value was optimal for all parts of the field of view, because of non-uniform illumination, mottle, or, for thick specimens, variations in the intensity across the specimen. To solve this limitation, a background image is acquired, digitized, stored, and subtracted from the digitized image stream (Allen & Allen, 1983). For thin specimens, the background image is obtained by shifting the slide to a cell-free zone. For thick specimens, a suitable background image is acquired by defocusing the microscope. This system, known as video-enhanced DIC or VE-DIC, has been commercially available as the Hamamatsu Argus processor. The background image stored by the Argus processor needs to be updated periodically. VE-DIC has recently been reviewed (Salmon & Tran, 2007).

With the image-processing scheme we call MEDIC, the background image is generated automatically and updated after every frame. The MEDIC images are generated in real-time at the frame rate of the camera by a Matrox PCI image-processing board. This frees the microscopist from the task of generating a background image. We show that MEDIC can enhance the contrast of moving vesicles in live cells by a factor of 13.

Flow chart for MEDIC

The image-processing steps which make up MEDIC are shown in Figure 1. A stream of images from a 12-bit CCD camera is loaded alternately into Buffers A and B of the image-processing board to increase the time available for processing. Once frame n with intensities I_n is loaded, it is sent to the end of a 9-frame queue for background image generation. This queue then contains frames n, n-1, n-2,…n-8. The 12-bit average of frames n-1 to n-8 is computed; this is a 12-bit rolling average background image. This background image is subtracted from frame n to create the background-subtracted image. An offset is added to make all intensities positive.

Figure 1.

Flow chart for MEDIC microscopy.

Image display generally requires 8-bit intensities (256 gray levels) rather than the 12 bits (4096 gray levels) provided by most cooled CCD cameras. After subtracting the 12-bit background image from the 12-bit raw image, the 3 or 4 most-significant bits (bits 1–3 or 1–4) of the background-subtracted image are usually zero. This allows us to use bits 4–11 or 5–12 when converting to 8 bits.

To illustrate how this works, consider a vesicle whose maximum and minimum intensities are 2016 and 1984 moving in a region where the mean intensity is 2000. The peak-to-trough intensity difference in the raw data, which define the vesicle, then equals 2016−1984 = 32 gray levels. The vesicle contrast C_v is

$$C_v=\frac{{(I_{max}-I_{min})}_v}{I_B}=\frac{32}{2000}=.016$$ (1)

An object with such low contrast would probably not be detectable by the eye.

Background subtraction of 2000 places the maximum and minimum of the vesicle at +16 and −16. Negative intensities cannot be correctly displayed, however, so we add an offset of 128, half the 8-bit grayscale which the monitor can display, to all intensities. The maximum and minimum graylevels of our vesicle are then 144 and 112. The 12-bit intensities are converted to 8 bits by ignoring bits 1:4 and keeping bits 5:12. This retains the superior gray-level information provided by the 12-bit camera. The contrast C_v becomes

$$C_v=\frac{{(I_{max}-I_{min})}_v}{I_B}=\frac{32}{128}=0.29$$ (2)

An object with such high contrast can be readily observed by eye. Background subtraction has enhanced the contrast by a factor of 0.29/0.016 = 18. This illustrates the major advantage of background subtraction.

The order of the steps in MEDIC (Figure 1) is critical to the quality of the images produced. If the conversion to 8 bits were carried out prior to baseline subtraction, the maximum and minimum intensity of the vesicle would be 126 and 124; the background would be 125. Subtraction of the background followed by addition of an offset of 128 to keep numbers positive would give the vesicle a maximum intensity of 129 and a minimum of 127. The contrast would not be enhanced and the vesicle would exhibit only 3 graylevels rather than the 33 graylevels of the MEDIC method. This example shows that conversion to 8 bits prior to background subtraction would generate a markedly inferior image. The additional noise is called digitization noise (Holst, 1998).

After background subtraction, offset, and conversion to 8 bits, aggressive contrast enhancement can be applied to the entire image because all vesicles have the same midpoint intensity. In our case, this midpoint intensity was 128. We have used the 8-bit look-up table (LUT) shown in Figure 2 to enhance contrast; the desired shape was provided by splicing the error function (erf) to its negative at the midpoint of the LUT. The slope at the midpoint was adjusted by the user; a slope of 3 was often optimal.

Figure 2.

Lookup table for contrast enhancement. The slope is 3 at the midpoint. With an offset of 128, the contrast of low-contrast vesicles is enhanced 3-fold.

After enhancement by the LUT, the MEDIC image is displayed on a gray-scale monitor for immediate viewing by the microscopist (Fig 1). Simultaneously, the MEDIC image is sent to a 400-image queue in host ram. When the microscopist observes a noteworthy event, depressing any key activates transfer of the last 400 MEDIC images from host ram to a hard drive for archival storage (Fig 1).

Simulation of MEDIC

Figure 3 show snapshots of a MatLab simulation of the major steps in MEDIC. We generated a movie of 50 frames in which six small “vesicles” of identical size and contrast moved upward at constant velocity along the axis of a cylindrically shaped “cell”. To simulate the differential effects of DIC, the vesicles and the cell were each simulated as the derivative of a Gaussian in the x-direction (along the rows of the image matrix) multiplied by a Gaussian in the y-direction (columns). The width of the Gaussian was 10 pixels for vesicles and 100 pixels for the cell. Poisson noise of physically realistic magnitude (20,000 electrons/full well) was added. The vesicle velocity was set to 10 pixel/frame.

Figure 3.

Simulation of motion-enhanced background subtraction in DIC microscopy for 6 identical vesicles embedded in a cylinder of cytoplasm in the presence of shot noise. The shear direction of the DIC Wollaston prisms is along the x-direction. Top frame: raw image, 4096 gray levels (12-bits). Only the 8 most significant bits (1,2,3,…8) were used for display. Middle frame: background image generated by averaging the 8 previous frames. Only the 8 most-significant bits of the 12-bit image were used for display. Bottom frame: background-subtracted image. The MEDIC image has 256 gray levels, taken as bits 4, 5, 6…11 from the background-subtracted image, after adding a constant to keep values positive.

The top panel of Figure 3 shows a single frame with the 6 “vesicles” inside the “cell”. The contrast of the 6 vesicles varied depending on their location inside the cell; only 4 of the 6 vesicles were apparent to the eye. The middle panel of Figure 3 shows the background image. It was an average of the previous 8 frames. The bottom panel shows the MEDIC image, computed by subtracting the background image from the raw image. The six vesicles were easily seen in the MEDIC image because of the enhanced contrast and flat background. A faint “ghost trail” was visible below each vesicle. This arises from the barely detectable streak in the background image. The contrast of the ghost trail could be reduced by a factor of 2 by increasing the number of frames used to generate the in-focus background image from 8 to 16. Such ghost trails are rarely noticeable in MEDIC data acquired with live cells, because of noise.

Figure 4 shows linescans through the vesicles in the raw, background, and MEDIC images. All six vesicles could be detected in the linescan of the raw image (top panel). The vesicles did not contribute perceptibly to the linescan of the background image. The linescan of the MEDIC image (bottom frame) showed the 6 vesicles with excellent contrast on a flat background. Noise levels were different for different vesicles because shot noise was largely controlled by the intensity of the “cell”.

Figure 4.

Linescans at various steps of MEDIC. Top panel: line-scan through the centers of the 6 vesicles in the raw and background images. Bottom panel: line-scan through the centers of the 6 vesicles in the background-subtracted image. Note the change in the number of graylevels from 4096 to 512 between the top and bottom panels.

MEDIC with live-cell data

Comparable images and linescans at different stages of the MEDIC algorithm for a vesicle moving in a live PC12 cell are shown in Figures 5 and 6. The images were acquired with a Nikon E600 FN microscope equipped with a Nikon 60X water-immersion objective (NA 1.0), condenser (NA 0.9), and DIC optics. Images were digitized at 8.3 fps by a 12-bit, cooled Hamamatsu C4742-95 ORCA ER CCD camera with 6.4 × 6.4 μm² pixels. A Matrox Genesis board housed in an IBM Intellistation, controlled by Genesis Native Language and C++ software, was used to carry out MEDIC processing and display the MEDIC images on an Image Systems M17 gray-scale monitor in a darkened room. The size of the processed image was reduced to 512 × 512 pixels² to ensure that MEDIC processing for one frame was completed before the next frame arrived from the camera. The time between an event in the specimen and the appearance of the event on the screen was thus 2 frames, 1 for acquisition of the image by the digital camera and 1 for MEDIC processing. In our system, at 8.3 frames/s, the combined lag was 0.24s.

Figure 5.

Raw, background, and background-subtracted DIC images of a vesicle within a neurite of a live PC12 cell. Left: large-scale, raw DIC image, 4096 graylevels. The cell body begins at the bottom edge. The image size is 8.5 μm × 48 μm. The rectangle marks the AOI containing a vesicle. Right side, top panel: magnified raw image of the AOI marked on left-hand image. The image has 4096 graylevels (12 bits) in the computer; the 8 most-significant bits are displayed. Right side, middle panel: magnified background image, 4096 gray levels, most-significant bits displayed. Right side, bottom panel: The MEDIC image obtained by subtracting the background image from the raw image, adding a constant to keep values positive, retaining bits 4,5,6,…11, and applying an LUT with slope 3. Objective: Nikon 60X/NA 1.0 water-immersion. Condenser: NA 0.9. Temperature: 35–37C. Camera: Hamamatsu Orca ER. Scale: 1 pixel = 0.107 μm in all images. The bar on the right-hand images is 2 μm long.

Figure 6.

Linescans across the PC12 neurite at the location of the vesicle (previous figure). Top panel: linescans of the 12-bit raw image (solid line ——) and 12-bit background image (dashes - - -). Bottom panel: linescan of the 8-bit MEDIC image. Scalebar: 2 μm.

PC12 cells were grown on the surface of poly-lysine-treated coverslips. They extend long neurites following treatment with NGF (Greene et al., 1998, Hill et al., 2004). We saved 12-bit raw DIC images and MEDIC images for those neurites which showed fast vesicle transport. The Supplementary Materials include several videos for PC12 cells and for chick motorneurons obtained with MEDIC and with normal DIC at the same time. For the purposes of this publication, we replayed the MEDIC video stream and picked out an 80 × 50 pixel² area of interest (AOI) which contained a moving vesicle. We noted the frame number at which the vesicle was centered in the AOI. The left panel of Figure 5 shows a 48-μm-long section of a PC12 neurite in Frame n, with the AOI marked on it. On the right side of Figure 5 are shown expanded images of this AOI in the raw, background, and MEDIC images. The vesicle was essentially undetectable in the raw image but was easily observed in the MEDIC image. A ghost trail was visible above the vesicle, because this particular vesicle moved retrograde (from the top to the bottom in this image).

Movies of raw DIC and MEDIC images for live PC12 cells and chick motorneurons can be viewed in the Supplementary Materials.

Figure 6 shows linescans along the same frame and same row through the raw image, the background image, and the MEDIC image. The vesicle is centered near pixel 40 on the x-axis of all three images. Note that the raw and background images have 4096 graylevels (12 bits) but the MEDIC image has only 256 graylevels. Figure 6 shows that the intensity variation generated by the cell, which is the “background” on which one attempts to discern vesicles, varies by ± 1700 graylevels in the raw image but only ±25 graylevels in the MEDIC images. The edges of the cell, which are prominent in the raw image, are almost undetectable in the MEDIC image.

To measure the contrast benefit of MEDIC quantitatively, we computed the contrast of the vesicle in the raw image and the MEDIC image, using the linescans of Fig 6. Contrast was defined as ΔI/I, where ΔI is the difference between the maximum and minimum intensity defining the vesicle and I is the mean intensity. The contrast of this particular vesicle was 220/1800 or 0.12 in the raw image and (225 − 60)/128 = 1.29 in the MEDIC image. MEDIC thus improved the contrast of this vesicle by a factor of 10.7.

Similar calculations of contrast were determined for several vesicles in PC12 cells and in chick motorneurons (Table 1). To prepare this table, the coordinates of each vesicle and its contrast were first determined in the MEDIC image. Knowing where the vesicle was in the MEDIC image, we then measured the contrast in the raw image from the difference between linescans in the raw and background images at that location (Fig. 6, top panel). Most of the vesicles seen in MEDIC were not directly observable in the raw images.

Table 1.

Contrast enhancement by MEDIC for individual vesicles in neurites of PC12 cells and chick motorneurons

Vesicle	Contrast		Ratio CMEDIC/Craw
	raw image	MEDIC image
PC12-1	.52	1.9	3.7
PC12-2	.22	1.2	5.3
PC12-3s	.11	0.84	7.6
PC12-4	.12	1.29	10.8
PC12-5	.074	0.98	13.2
PC12-6	.15	2.0	13.3
Chick-1	.41	1.9	4.6
Chick-2	.16	1.2	7.5
Chick-3	.16	1.3	8.1
Chick-4	.092	0.86	9.3
Chick-5	.11	1.3	11.4
Chick-6	.13	1.6	12.3

The table shows that contrast enhancement by a factor of 13 is achievable with MEDIC. Videos of raw and MEDIC images, acquired simultaneously, are available in the Supplementary Materials. Occasionally, vesicles could be observed in the raw images. Such vesicles sometimes became saturated after passing through an aggressive LUT in MEDIC. Vesicle PC12-1 in Table 1 is an example.

Normally, only the 8-bit MEDIC images were saved to a hard drive for later processing to determine the number, velocity, and size of vesicles in live cells (Hill et al., 2004, Chisena et al., 2007, Macosko et al., 2007). In some cases, the raw 12-bit images were also saved for archival purposes.

Discussion

The primary benefit of MEDIC is real-time contrast enhancement and background flattening, which allows the microscopist to see, in real time, many vesicles which are not observable in the raw images of the same live cells in real time. For the vesicle shown in Figures 5 and 6, the contrast enhancement delivered by MEDIC was more than a factor of 10. Thus MEDIC allowed us to observe vesicle movement in real time before saving images. Once a suitable cell or dish was found, videos containing 100–400 MEDIC images were saved to the hard-drive. The Supplemental Materials show examples of such videos. The raw DIC images can also be saved.

At a later time, vesicles visible in the stored MEDIC videos can be tracked by one of the many off-line tracking programs currently available for use with image stacks. We use a subpixel pattern-matching algorithm to determine vesicle velocities. Because MEDIC flattens the background and increases the contrast of vesicles dramatically, many vesicles could be tracked by computer in PC12 cells (Hill et al., 2004) and in chick motorneurons (Macosko et al., 2007). These vesicles move at 300 – 2500 nm/s and have a radius of 100–500 nm.

In order to use MEDIC to reveal small vesicles moving for significant distances, e.g. 20 μm in 25 s, the vesicles must remain in focus as they move. The depth of focus of high magnification, high numerical aperture DIC is less than 1 μm (Murphy, 2001). This means that vesicles are best observed if the specimen is thin in the region of observation. The cellular zone within which the motion occurs must also be adherent to the coverslip or slide, for the same reason. However, slow movement of the cell or cellular process in the xy plane, e.g. crawling, does not preclude the observation of fast-moving vesicles because the MEDIC background image is constantly updated.

Users of MEDIC also need to be aware that moving vesicles can exhibit a faint “ghost trails” if the observed vesicle is large, has high contrast in the raw DIC image, or moves slowly relative to its length. Assume an object of intensity I₀ and width w₀ (pixels) at position x_n (pixels) in frame n, moving at speed v (pixels/frame). Suppose the background image is an average of N frames. Assume that there is a gap of p frames between the primary frame and the first background frame and a decrement of q frames between subsequent background frames, so that the background is generated by averaging frames n-p-q, n-p-2q, n-p-3q....n-p-Nq. The ghost position x_g, width w_g, and intensity I_g are then given by

$$x_g=x_n-(p+(N+1)q/2)v$$ [1]

$$w_g=w_0+(N-1)qv$$ [2]

$$I_g=I_0\frac{1}{\left[1+\frac{(N-1)qv}{w_o}\right]}$$ [3]

In our system, with a 60× objective and 6.4 × 6.4 (μm)² CCD pixels, the magnification at the digital image level is 9.36 pixels/μm. The width w of a typical vesicle is 2 pixels in the image (214 nm in the specimen); a typical vesicle velocity v=1.13 pixels/frame (1 μm/s at the specimen, 8.3 frames/s camera). We have set p=0, q=1 and N=8 frames for MEDIC processing (Fig 1). Under these conditions,

$$\begin{array}{l}{x}_g=x_n-5.1\text{pixel}\\ w_g=9.9\text{pixel}\\ I_g=0.20I_o\end{array}$$

This ghost has 1/5 the contrast of the object and is shifted 2.5 particle widths away from the object. It also has reversed shading. Under these conditions, the ghost is easily distinguished from the object. The ghost can be moved further away from the object by increasing p, q, or N. The contrast of the ghost can be reduced by increasing q or N. The price paid for increasing p, q, and N is that the background image is not as timely. In our program, with p=0, q=1 and N=8 the images in the background queue were all acquired within 1 s of the current image. Larger values of p, q, and N may be appropriate for objects moving more slowly than our vesicles. However, if v = 0, the position and intensity of the ghost are the same as those of the object for all values of p, q, and N, so the object is not observable with MEDIC.

The MEDIC images in Figs 5 and 6, and in the Supplementary Videos, were obtained with a 5-year-old Matrox Genesis image-processing board. Such boards are widely used in machine vision. The Genesis board can compute 700 × 700 pixels² MEDIC images at the frame rate of our camera (8.3 frames/s). A newer version of this board and software, Matrox Odyssey, is about 8 times faster than the Genesis board. The Odyssey board should be able to compute 700 × 700 MEDIC images at 60 frames/s or 1024 × 768 images at 40 frames/s. Similar boards and software are available from Dalsa and other vendors. Code for doing MEDIC on the Genesis board is available from the authors on request.

MEDIC retained all or most of the gray levels which defined the vesicle in the raw image. In our system, both the raw images and the background image were 12-bit. Conversion from 12 to 8 bits was done in a way that retained the maximum number of graylevels in the moving particles. If this conversion had instead been done by simple division, the number of gray-scale increments defining the vesicle would have been severely reduced, adding digitization noise to the image noise.

MEDIC does not reduce shot noise or read noise present in the raw DIC image. If the images are digitized by a scientific-grade cooled CCD camera, shot noise will predominate. If the intensity is I at a particular pixel, the root-mean-squared (rms) shot noise is proportional to $\sqrt{I}$. The rms shot noise in the MEDIC background image is reduced to $\sqrt{I/8}$ if 8 frames are averaged. Thus rms shot noise in the background-subtracted image is $\sqrt{I+I/8}$, which is only 6% greater than $\sqrt{I}$. This is a small price to pay for the contrast enhancement provided by MEDIC (Table 1).

We have until now used MEDIC only with DIC microscopy. However, real-time background subtraction with rolling average generation of the background image may also be useful for enhancing the contrast of actively transported fluorescently labeled vesicles and organelles in live cells.