Rapid high resolution imaging with a dual-channel scanning technique

Abstract

A spatial shift between channels in a dual-beam raster scan imaging system introduces a temporal separation between images from the two channels that can be much shorter than the frame rate of the system. The technique is demonstrated by measuring the velocity of erythrocytes in the retinal capillaries. We used an Adaptive Optics Scanning Laser Ophthalmoscope and introduced a temporal separation between imaging channels of 4.7 ms. We imaged three subjects and measured changing capillary blood flow velocity at the pulse rate. Since the time shift between channels is easily and continuously adjustable, this method can be used to measure rapidly changing events in any raster scan system with little added complexity. © 2015 Optical Society of America

Scanning imaging systems are powerful tools for real time imaging of living systems, since they allow precise control of confocality and admit the use of sensitive point detectors such as photomultiplier tubes and avalanche photodiodes. However their temporal resolution is typically limited by the speed at which a beam can be scanned. In retinal imaging most scanning systems [1, 2] use mechanical scanners and the highest speed of these typically generates a line scan rate on the order of 16 kHz. If one wishes to image tissue with a moderately wide field of view orthogonal to the fast scan (say 500 pixels), then the maximum frame rate is determined by the number of lines and the line rate, and in the example above would be about 32 Hz. While this can be increased by bidirectional scanning, further increases in speed require fewer lines. In our application the goal is to study the velocity of cells moving through the retinal capillaries, but over a relatively wide field of view (several hundred microns). While red blood cells (rbcs) can be imaged, their large number and velocity can cause errors in position determination in consecutive frames due to aliasing.

A number of methods are available for measuring blood velocity. To date techniques that are based on Doppler velocimetry velocimetry [3, 4] including variations based on Optical Coherence Tomography (OCT) [5, 6] have been improved and demonstrated by many groups and have been most useful for relatively large retinal vessels (> 50 microns). Also blood flow can be related to the modulation of the speckle signal [7] but this technique tends to be useful only for large vessels as well. Direct imaging techniques have typically used either dye injection [8–12] or externally labeled leukocytes re-injected into the blood stream [12, 13]. Currently Adaptive Optics techniques (AO) are able to provide higher resolution images of flow in small vessels without the need of contrast agents. Using an Adaptive Optics confocal Scanning Laser Ophthalmoscope (AOSLO) the velocity and pulsatility of the large moving features thought to be leukocytes [14], that can be observed moving through the capillaries of the macular region was determined either from the change in position of two consecutive frames of a video [14, 15] or from the change of intensity in a spatiotemporal plot [16, 17]. Also, AO has allowed the measurement of the erythrocyte velocities in small retinal regions either by using high frame rate non-confocal imaging [18], or by shrinking temporarily the field of view to a single scan line [19].

Recently the use of an AOSLO with multiply scattered light detection [20, 21] has allowed improved imaging of the vasculature of the human retina. With this technique rbcs can be directly imaged while travelling through the capillaries of the human eye [22, 23]. The determination of rbcs velocity is desirable since they should provide a better estimate of tissue perfusion and vascular autoregulation than leukocytes, which are much larger than the capillary lumen and thus travel at a different velocity. While the single file flow of rbcs through capillaries simplifies velocity measures, their abundance introduces the issue of velocity aliasing for low frame rate systems, since the same cell cannot be identified between two consecutive frames. To solve this problem we developed a dual-channel AOSLO to image the retina with two close but different wavelengths. By introducing small angular shifts between the beams at the pupil, the same area of the retina is sampled at different times and this produces temporal offsets much smaller than the frame rate of the raster scan itself. Figure 1 shows the general principle of the approach. In the current study we introduced a separation of about 4.7 ms between channels and measured the change in position of the red blood cells during this interval to calculate velocity. Because the signal to noise ratio in our images is relatively low we calculated the average velocity over a 10 ms window and along a capillary segment.

Fig. 1.

(a) Region of the average image created from the dual channel videos recorded in one of the subjects. The image of stationary features was used to calculate the spatial shift between channels, L. (b) Schematic representation of a stationary feature and a cell moving in the vertical direction. The time difference between the images of a stationary feature in both channels is the time difference between lines, t_L, multiplied by the vertical shift between channels, L. However, the time difference for a moving feature depends on both the vertical shift between channels, L, and the vertical shift between images, DL. Note that when measuring velocity with a non-zero vertical component the time difference will differ for features moving upward vs. downward.

The approach is versatile. The angular deviation between the beams can be controlled along either the fast scan or slow scan directions. Thus, the technique can be used over a wide range of time scales. Because both beams use the same scanning mirrors, there is no need to perform complex corrections for image distortions between channels.

The Indiana Adaptive Optics Scanning Laser Ophthalmoscope has been described in detail in previous publications [24, 25]. Essentially a laser is focused on the retina and two scanners are optically conjugated with the pupil of the eye to create a raster. A resonant scanner with an oscillation frequency of 15.1 kHz and a programmable slow scan analog galvanometer were used for horizontal and vertical scanning respectively. The light coming back from the eye was detected by an avalanche photodiode (APD) and sequentially sampled at 30 MHz, and images of 780×520 pixels were collected with a frame rate of 27.97 fps. Two deformable mirrors in a woofer-tweeter configuration are optically conjugated with the pupil of the eye to correct the optical aberrations.

Two imaging wavelengths, 785 and 820 nm, (pupil size 7.5 mm, power at the cornea 100 μW each, safe time >1 hr), were aligned using a 50/50 beamsplitter and the light coming back from the retina was separated with a dichroic mirror with a low pass cutoff at 805 nm. Band pass filters were added in front of each APD to ensure that only light from the design wavelength was collected. The angle between the two beams was changed by placing a mirror with Vernier controls (Thorlabs VM1) at a pupil conjugate plane prior to combining the beams and the focus of one of the channels was manually adjusted by moving the tip of the optical fiber axially (Thorlabs SM1Z) to focus both wavelengths at the same retinal plane. For the current study the two rasters were offset 71 lines vertically to introduce a temporal shift between them of 4.7 ms (71 lines * 15100⁻¹ seconds/line). While less separation would introduce a smaller temporal shift (i.e. more temporal resolution) we found that this amount was acceptable for measuring the velocity of the red blood cells in the capillaries.

To enhance the visualization of red blood cells, multiply scattered light was collected with a 10× Airy disk diameter confocal aperture offset at least 6× Airy disk diameters from the retinal conjugate position [22]. This ensured that the central lobe of the adaptive optics corrected point spread function, where most of the directly backscattered light is concentrated, was not collected by the APD.

For this study we obtained videos of 100 frames of an area of 1.3×1.3 degrees (around 380×380 pm at the retina) producing an image with 9 arcsec per pixel (around 0.68 μm per pixel at the retina). The eye movements in the raw video were corrected with software written in MATLAB [24]. Though the contrast in the multiply scattered images of blood vessels is usually lower than the contrast in the images of the cone photoreceptor mosaic, the software was able to align the video frames without modification. Since both channels are formed by identical scanning elements, both beams pass through the eye following essentially the same optical path and are detected at the same time, the frame movements calculated to compensate the eye movements in one of the channels were saved and applied to the second channel, ensuring that temporal registration was not altered by the alignment software.

To quantify flow we first semi-automatically detected individual cells in one of the videos and then computed the distance traveled during the 4.7 ms. This was performed by first smoothing the images with a Gaussian filter and normalizing the result, and then by statistically detecting changes in the intensity arising from moving cells. To normalize the images we first corrected for overall changes in intensity (which can arise from small variations in the tear film or patient position). We did this by dividing each frame of the aligned video by its mean. We next normalized intensities on a pixel by pixel basis by dividing each pixel of each frame by the average of that pixel’s intensity across all frames. An example of the output of the normalization can be seen in Figure 2.

Fig. 2.

(a) Erythrocytes travelling through the foveal capillaries imaged with the AOSLO collecting multiply scattered light. The capillaries on the upper left of the image are those surrounding the foveal avascular zone. (b) Normalized video to emphasize moving features (see Visualization 1). The images correspond to a subset of one of the 380×380 μm, 3-second videos of 28 fps collected in one of the subjects.

For one of the two channels a region with a capillary was manually selected and cells were then automatically detected in each frame. To detect the cells in a given frame, we computed the mean and standard deviation for each pixel across the entire normalized video. We then produced a binary image for each frame with white pixels when the intensity was above the average plus N times the standard deviation of the pixel intensity over the whole video. The value of N was adjusted interactively for each video, and was typically between 0.3 and 0.6. A morphological “open” operator was applied [26] to this binary image and the center of the remaining components was used as an estimate of the location of the detected cells in each frame. The result was a set of locations within the selected blood vessel.

In principle once a cell is detected in one channel it should be possible to determine the location of the same cell, now having moved in the intervening 4.7 ms. However direct comparison was noisy so we averaged across all cells, within the selected capillary segment for each video frame. We did this by obtaining a ROI around each detected cell, and then averaging the ROI’s. For the main channel this produced an “average cell image” in the center of the average ROI, see Figure 3 (a). For the second channel, the one with the temporal shift, the same ROI’s, after correcting for the offset between the channels, were averaged producing an image of the average cell displaced from the center of the ROI, see Figure 3 (b). Since we are measuring the average displacement of all the cells, the region for analysis was selected to ensure that the capillary was approximately straight so that all the cells traveled in approximately the same direction.

Fig. 3.

Average cell image in a capillary of one of the subjects imaged for channel 1, where the detected cell is located (a) and the same region in channel 2 which is 4.7 ms later (b). Units in the image axis are μm. Absolute value of the X and Y displacement between images (c) velocity of the cell (d) calculated when averaging over 3 consecutive frames. (see Visualization 2).

Stationary objects will be seen in these average images in the same position in both channels while moving particles will appear displaced. For our application we used vertical displacements to measure the time difference between the images. As can be observed in Figure 1, while in the case of stationary objects the time difference between channels is set by the spatial shift between rasters, in moving objects, the displacement includes both the distance traveled by the moving object and the time difference produced by the sequential motion of the scan [16]. This is depicted in Figure 1 by taking into account the actual number of lines.

To obtain an average velocity over the whole video (about 3 seconds), the images of all the cells detected in the selected capillary through all the frames of the videos were averaged. Using the average image of the cells detected in a single frame, one velocity measurement can be performed every 30 ms. In practice we computed the velocity averaged over three consecutive frames (a 10 Hz effective rate).

To test this approach we imaged three young normal subjects (age range 25 to 31 years) with the AOSLO and the velocity was determined in 10 foveal and parafoveal (maximum eccentricity 4.5 degrees) capillaries in each subject. All research was approved by the Indiana University Institutional Review Board, and adhered to the tenets of the Declaration of Helsinki.

The imaging produced clear views of moving cells within the capillaries (Visualization 1). The semiautomatic software detected an average of 7 cells in each frame of the video. In addition, the pulsatile motion of the average displaced cell over time was evident (Figure 3). Average velocities calculated with the average image of all the cells detected in the whole 3 second video were calculated for all the capillaries imaged and ranged from 0.30 to 2.26 mm/s, with a mean value of 1.14 mm/s. The image of the average cell in the channel where the red blood cells were detected was always a compact bright spot with dark features in the direction of the pinhole offset, while the average image in the second channel often appeared more dispersed, indicating either that a change in velocity during the time window used to average had taken place or there were velocity variations between individual cells (Figure 3).

In 22 out of 30 videos, the velocity could be calculated by grouping the cells found in 3 consecutive frames. In the other 8 the average image of the displaced cells did not show a clear maximum and thus we did not determine its location. The change in position of the cell in the channel with the temporal offset caused by a change in velocity can be observed in Visualization 2. The calculated velocities are plotted in Figure 3 for one of the subjects. Here the changes in velocity at approximately 1 Hz (the cardiac cycle) is clear. This pulsatility was observed in all 22 capillaries and in videos where several capillaries were measured, the pulsatility could be compared across capillaries (see Figure 4). Pulsatility, defined as (v_max-v_min)/v_mean ranged from 0.48 to 1.28, with a mean value of 0.79.

Fig. 4.

(a) Standard error map [24] of the capillaries of a retinal area 300 um above the fovea. (b) Velocities of the three capillaries marked with arrows. When the velocity was measured in several capillaries in the same video, the change in velocity was approximately in phase across capillaries.

While the frequency of pulsatile velocity changes was similar across capillaries, the measured velocities were not. The average minimum measured velocity ranged from 0.26 to 1.99 mm/s, mean 0.82 mm/s and the average maximum ranged from 0.71 to 3.98 mm/s, mean 1.88 mm/s. Capillaries with “slow” and “fast” velocities were measured in all the three subjects.

The ability to use two wavelengths to generate a sequence of two images with very brief temporal intervals between them allows a scanning system to image with a high effective frame rate. This separation between beams does not need to be based on wavelength, for instance polarization could be useful in some applications. In our current application we validated this dual-channel approach by measuring blood velocity in retinal capillaries where we could observe a change of velocity at the approximate rate of the heartbeat. While this technique can be best used to measure velocities tangential to the imaging plane, foveal and parafoveal capillaries are known to be stratified.

We measured a range of retinal capillary velocities larger than has been reported previously. Our minimum and average velocities are comparable to those measured on erythrocytes using a high frame rate fundus illumination system [18], or on leukocytes using either the entoptic phenomenon [27–30], labeling with fluorescein [12,13] or direct imaging with AOSLO [14–17]. However, our peak velocities are larger than the ones found with those techniques. The only technique which has found velocities comparable to our peak velocities used intravenous fluorescein [8–11]. In general we expect blood flow velocity for erythrocytes to be faster than that of leukocytes due to their smaller size and the small cross sectional area of the capillaries. For erythrocytes we measured velocities that spanned the range of previous studies, and in the case of the subject in Figure 3, it is clear that even within the same region, we can get a variety of measures across capillaries, and because multiple capillaries are measured within the same video segment, we can rule out physiological shifts, a strength of this approach relative to smaller field techniques [18].

Since the slowest velocities found in erythrocytes are similar to those found in leukocytes and the fastest velocities are higher, it is no surprise that the pulsatility values we measured, are slightly higher than the ones found in techniques based on detecting leukocytes [15, 17, 28].

In a few videos we were able to measure velocities in each frame (typically capillaries with a long distance between branches). In these capillaries we found slightly higher peak velocities, surely due to the smaller time window (approximately 30 ms) compared to averaging three consecutive frames (90 ms window). This suggests that the period of peak velocity is brief and the longer integration window drops the estimated velocity.

The precision in the velocity measurements when using this technique depends on the accuracy in locating the center of the cells in the first channel as well as the centroid calculation of the average cell in the second channel. While a full error analysis would require additional data we estimated a maximum of a 2 pixel error in the displacement estimation which would produce a maximum of 0.4 mm/s error for the temporal separation we introduced. We believe the error is less than this on average, although periods with rapidly changing velocity could produce this larger error. Due to the low signal to noise ratio we had to average several cells and that forced us to study only rectilinear capillary segments, i.e. not curved, where we assumed that all the cells travel with similar velocity. Even if the average image is formed with a single frame, the cells on top of the frame are imaged at a different time than those in the bottom, thus, the data do not represent the instantaneous velocity of a single cell but the temporal average of the velocity of several cells. Despite the averaging, the temporal and spatial resolution was sufficient to detect and characterize a changing rbc flow velocity occurring at the approximate pulse rate.

An advantage of our technique is that the timing is readily adjusted. While we have concentrated on the line time difference, in principle very rapid intervals could be measured by displacements in the direction of the fast scan. Here though care will be necessary to account for any scan distortion (in our case sinusoidal distortion from the resonant scanner). We tested this versatility by decreasing the separation and measuring larger blood vessels near the optic nerve head. Here high velocities could be measured. However, the main prob lem is that when studying flow in larger vessels the velocity varies from the center to the periphery of the vessel [5, 6, 19], and because many cells are imaged simultaneously, care must be taken to focus the two beams identically to ensure that the same cells are detected. If this is achieved, the automated detection will produce a streak that arises from many blood cells, all moving with differing velocities. While in principle the furthest distance moved should represent an estimate of the centerline velocity, in practice other techniques, such as the line scan [19] or Doppler based methods [3–6] are probably better for these larger vessels.

In summary, by using two channels and separating spatially 71 lines the areas imaged with our AOSLO we have introduced a temporal separation of 4.7 ms between the two channels. To produce this short interval with a conventional approach would require a frame rate of 210 fps. To achieve the same frame rate with a single channel AOSLO, a small area of the retina should be imaged (about 50 μm if only 71 lines are collected). Though by imaging a small area with a higher frame rate each cell could be followed for a longer time, the eye movements would complicate the image alignment and only a small area could be studied in each video. The benefit of the dual channel technique is that a relatively large area can be studied, allowing direct comparisons over space. To our knowledge this is the first study reporting pulsatile velocities in the red blood cells travelling through the retinal capillaries.