Eliminating the scattering ambiguity in multifocal, multimodal multiphoton imaging systems

Abstract

Four images of Drosophila Melanogaster antennal lobe structure labeled with red fluorescent protein. The images are separated axially by 7 μm in depth, and were all acquired simultaneously from a single-element detector.

1 Introduction

The first demonstration of a real-time, video rate multiphoton microscope was performed with a system that extended the single focal point excitation source to a line cursor geometry [1]. This design incorporated an elegant bilateral scanning system that could be used to effectively discriminate against scattering in one dimension – the direction orthogonal to the line or slit length. With the line cursor geometry, rapid image rates are readily achieved – the line sweeps out a two-dimensional field in the focal plane with a single axis scan, and the multiphoton signal is simultaneously mapped to a two-dimensional detector such as a charge-coupled-device (CCD) camera.

Notably, by moving to a line cursor geometry the image resolution is compromised. The beam now features both a high numerical aperture (NA) and low NA component. The high NA dimension maintains tight lateral resolution, and the low NA dimension determines the axial (or z-axis) resolution. Consequently, the sectioning is not as high as is typically achieved with a traditional point focus for example.

In order to address this reduction in axial resolution, while maintaining high frame rates, multifocal multiphoton systems were developed. In this geometry, multiple diffraction limited focal spots are created through any number of mechanisms. Microlens arrays were among the very first devices used to split the ultrafast laser into a large number of diffraction-limited focal spots. This array of foci is then rapidly scanned and the multiphoton signal simultaneously mapped to a two-dimensional detector. The basic strategy to help sustain high frame rates was to pack the focal plane with as many focal spots as could be reasonably achieved for any given laser source (i.e., ultimately limited by available laser power). What became immediately apparent, however, was as the density of focal spots was increased the axial resolution degraded – the focal spots interfere and the effective NA is lowered. Fortunately, when dealing with multiphoton imaging there is an additional design degree of freedom available that can be used to address this issue – the laser pulsewidth. If the individual focal spots are delayed in time by several times their pulse duration, they cannot interfere and the resolution achieved with a field of focal spots is identical to that of a single diffraction-limited laser focus [2–4].

While the resolution issue has been effectively addressed, the utility of multifocal systems has primarily only been exploited in thin, relatively nonscattering specimens. Since two-dimensional detectors must be incorporated, scattered signal light contributes to background and inhibits deep imaging or imaging through highly scattering media. Several effective solutions have now been demonstrated for mitigating the scattering issue when using multifocal imaging systems. For example, by descanning the multiphoton signal, the penetration depth where images with reasonable signal-to-noise can be achieved is significantly extended [5, 6]. Our approach to addressing this issue has been to increase the time-delay between foci from picoseconds to nanoseconds and move from integrated detection to photon-counting. The implications of this paradigm shift are that, for the first time, single element detection can be employed with multifocal systems and the scattering ambiguity is entirely eliminated – scattered signal photons are now contributing positively to the image, improving the signal to noise just as in a traditional single focal spot multiphoton imaging system.

This approach has enabled multiple focal planes to be imaged in a parallel process, and hence entire volumes can now be captured instantaneously. Figure 1 is an example of an image volume captured in this manner. Four different focal planes, separated in depth by seven micrometers are captured simultaneously. In this case the signal is two-photon fluorescence from the antennal lobe structure of Drosophila Melanogaster genetically engineered to express red fluorescent protein. Notably, the image contours presented in all images are not arbitrary units – they are given in actual photon counts.

Figure 1.

Four images of Drosophila melanogaster lobe structure labeled with red fluorescent protein. The images are separated axially by 7 μm in depth, and were all acquired simultaneously. The field of view is 250 μm by 250 μm, and the number of photon counts which determine the intensity contours is denoted on the right hand side of each image by the scale bar. The excitation NA was 0.65, and the excitation wavelength was 1030 nm from a Yb:KGW laser.

The utility of this imaging method has recently been further verified by Cheng et al., which uses this type of multifocal approach to measure neuronal activity from multiple focal planes inside a living mouse brain by recording two-photon excitation fluorescence (TPEF) from probes sensitive to calcium activity [7].

To date we have been able to image up to six focal planes, in multiple modalities (e.g., second and third harmonic generation imaging, along with TPEF) simultaneously [8]. In addition, we have added remote focusing capability to the individual planes, providing real-time repositioning of individual focal plane depths [9]. The remote focusing can be used synchronously with the x-y scanners, in fact enabling the focal plane to be rotated or tilted to facilitate more selective viewing of features of interest. In this work, we describe a number of new advancements that further push the capabilities of this new imaging technology. We demonstrate for the first time:

1. a video-rate, photon counting, multimodal multifocal imaging system that includes remote focusing,

2. simultaneous image capture using single element detection, of multiple modalities, without the need for selective spectral filters,

3. the application of programmable gratings for multiphoton image deconvolution,

4. a multimodal, multiphoton line cursor excitation system that only requires single element detection.

2 Video-rate multifocal multimodal photon counting imaging

2.1 Experimental setup

We have previously demonstrated multifocal photon counting imaging with a single-element detector without working at video-rates [8–12]. By working with our multi-beam Yb:KGW laser[8] we can readily image multiple depths without translating the sample or the objective [9]. This system provides us with six time-delayed beams without consuming valuable lab space to construct an optical multiplexer. In this work we expand upon that system in order to add true video-rate imaging, allowing us to capture four depths simultaneously at 30 Hz.

However, performing this imaging at video-rates introduces significant additional challenges. One such difficulty is in rastering the beam rapidly such that the region of interest can be captured at a 30 Hz frame rate. This issue can be resolved by utilizing a polygonal mirror in place of the traditional fast-axis galvanometric scan mirror [13], as shown in the system schematic in Figure 2. For this work we have implemented the scan system using a Lincoln Laser MC-5 polygonal mirror and a GSI Group, Inc. SC2000 scan mirror, synchronizing the two by generating the polygonal mirror frequency control signal phase-locked to the scan mirror movement using an Arduino.

Figure 2.

Four beam video-rate imaging system layout. L1: 400 mm Lens, L2: 100 mm Lens, L3: 40 mm Lens, L4: 40 mm Lens, L5: 35 mm Lens, L6: 200 mm Lens, SM: Scan Mirror, PM: Polygonal Mirror, OBJ: Objective

Another challenge in constructing a rapid imaging system is in implementing electronics that can communicate all of the collected data back to a computer in a timely fashion. To address this issue we transitioned our hardware platform from an Altera DE2 board to an Altera DE3 system, providing us with access to USB 2.0 data-rates. In our previous work all of our data was collected using LabVIEW™ and then rendered using either MATLAB^® or gnuplot. Due to the significant overhead incurred when processing data using LabVIEW™, we migrated our data collection over to the open-source μManager software [14]. By transitioning to the DE3 platform and by writing a custom data collection plugin for μManager we were then able to improve our data-rate from ~ 1 MiB/s to roughly 20 MiB/s (a frame rate of 320 Hz for 8-bit 256 × 256 images). This migration also gave us the ability to have integrated real-time rendering provided by ImageJ, dramatically improving the usability of the system for live data collection.

2.2 Imaging results

Images were collected utilizing a 0.2 microsecond pixel dwell time in order to facilitate video-rate imaging at 30 Hz. This dwell time corresponds to a maximum possible signal of 3.72 counts per pixel per beam for our 18.6 MHz repetition rate. Data corresponding to each of our individual four beams was collected simultaneously as 256 × 256 4-bit images, resulting in a total acquisition of a 256 × 1024 frame at 30 Hz (requiring a sustained data rate of 3.75 MiB/s). The resulting image (Figure 3) represents a summation of 900 individual frames (corresponding to 30 seconds of data collection).

Figure 3.

Simultaneous multi-depth images of a sample of corn starch granules. Images were summed from a source data set composed of 900 frames of 256 × 1024 4-bit data collected at 30 Hz.

3 Simultaneous single-element multimodal detection

3.1 Experimental setup

The optical configuration for our single-element multi-modal detection is the same as that discussed previously for our video-rate system. However, our field programmable gate array (FPGA) designs required a major upgrade in order to separate harmonic data from fluorescence data. Our previous designs relied on a clocking signal from the laser to enable a different beam-specific counter depending on which beam is active at the time. Separating out the harmonic data involved added an internally delayed version of this timing signal to enable and disable an entirely separate set of counters meant specifically for the harmonic data (Figure 4).

Figure 4.

Timing of the FPGA counter bins indicating where photons detected by the PMT will be deposited. H1-H4 represent the harmonic bins for each of the four beams and F1-F4 indicate the fluorescence bins.

It is worth mentioning that, depending on the fluorophore, a significant number of fluorescence counts may appear in the harmonic data channel. This effect can be evaluated by utilizing a time-correlated single photon counting (TCSPC) system to measure the fluorescence response of the fluorophore being utilized. For this work we utilized a bin size for the harmonic data of approximately 1.15 nanoseconds (corresponding to a delay through 6 gates of the Stratix III FPGA used by the DE3 board). In this work we are utilizing 10 μm fluorescent microspheres (Invitrogen F8831) as a fluorophore and corn starch for a second harmonic generation (SHG) signal source. Examining both of these specimens with our TCSPC, we calculated that roughly half of our fluorescence counts should fall within the harmonic bin (Figure 5).

Figure 5.

Time-correlated single photon counting traces for Invitrogen F8831 fluorescent microspheres and for corn starch granules.

3.2 Imaging results

By imaging a mixed sample containing both fluorescent microspheres and corn starch we successfully demonstrated the ability of our photon counting electronics to differentiate between a harmonic response and fluorescence response. Figure 6a shows data collected with a 520 nm (SHG) interference filter in place to screen out the fluorescence channel, Figure 6b displays data collected with a BG39 filter exhibiting both the fluorescence and the SHG data, and Figure 6c shows a composite image of the data overlaying both channels of data from Figure 6b. This data is collected in the same manner as the video-rate data discussed previously, except that an additional harmonic channel is also returned (requiring a sustained data rate of 7.5 MiB/s).

Figure 6.

Images of a sample of fluorescent beads mixed with corn starch granules. Images were summed from a source data set composed of 900 frames of 512 × 1024 4-bit data collected at 30 Hz and are from one representative beam of the data (a 256 × 512 region). (a) and (b) show the fluorescence channel on the left and the harmonic channel on the right, (c) shows these two channels overlapped in a false color image.

5 Single element detection using a line cursor excitation source

As stated in the introduction, the first real-time multi-photon imaging system incorporated a line cursor geometry [1]. This system used a 100 kHz chirped pulse amplification Ti:Al₂O₃ laser system as the excitation source and utilized a two-dimensional detector. Here, we demonstrate that it is possible through spatial modulation of the beam to make the line cursor excitation compatible with single element detection, and that imaging can be done using a Ti:Al₂O₃ oscillator system. While these results are preliminary, they point the way to a new paradigm for rapid volumetric multiphoton imaging that will ultimately be quite straight-forward for the biological community to implement.

The system schematic is shown in Figure 7. The Ti:Al₂O₃ oscillator is an extended cavity system operating at 23 MHz with a pulse duration of 70 fs and pulse energies up to 20 nJ. The energy and repetition rate are a good combination for use with line cursor geometries as described here. The output of the laser is up collimated to 10 mm, and the line cursor is created at an intermediate plane by a 10 mm focal length cylindrical lens. At the focal plane of the cylindrical lens, we place the spatial modulator. The modulator is a printed mask that is rotated at 100 Hz. Each point along the length of the focused beam is modulated in intensity at a rate of 18 kHz, with a bandwidth of several kilohertz. The spatial modulation rate is in fact chirped – sequential points are modulated at slightly higher frequencies as a function of spatial position [15]. In this manner spatial location is selectively encoded, and the image is recovered through straightforward Fourier transform techniques of the time-varying detected signal – be it from TPEF or the actual beam itself.

Figure 7.

Spatially modulated, line cursor excitation multiphoton microscope schematic. The excitation course is an extended cavity Ti:Al₂O₃ oscillator operating at 23 MHz.

After the mask, the beam is directed through a single axis scanner, and relayed to the specimen plane. The excitation optic is 0.75 NA objective. The TPEF signal is collected in the epi direction by a photomultiplier tube, and the fundamental is collected in transmission by a Si photodiode. The scan time is adjusted as appropriate for the signal levels coming from the specimen. In this case our multimodal detection is a nonlinear signal (TPEF) and a linear signal (the light from the excitation beam).

Figure 8 shows images obtained with the system and the mask used to modulate the beam. The specimen was a simple wire test grid immersed in Rhodamine 6G. The upper left image is the TPEF signal, and the lower left image is created with the transmitted signal from the laser beam. The lateral image resolution is 10 μm, and is substantially below the diffraction-limited resolution of the 0.75 NA objective. In this case, the image resolution is fixed by the mask. For this demonstration, the pattern shown in figure z was printed on a simple CD and only consists of low spatial frequencies. The time varying diffraction pattern created by rotating the mask results in substantially under filling the objective, and hence severely limits the image resolution (and hence the number of pixels within the image). This is in no way a fundamental limitation. We are presently fabricating new masks that will result in a time-varying diffraction pattern that will completely fill the objective and produce images that are limited in resolution by the excitation objective NA. Finally, the nonuniformity in intensity of the line cursor is also evident in the images – especially so in the TPEF image given its nonlinear dependence. This is due to the nature of the Gaussian beam and can ultimately be accounted for in post image processing – only raw data is shown here.

Figure 8.

Upper left image – TPEF image of test grid. Lower left image – image of test grid created from fundamental beam. Each pixel is xx μm. Approximately 9 nJ per pulse, as measured in front of the microscope, was used to create these images. Right – image of mask used to create the spatially chirped intensity modulation across the line cursor.

6 Conclusion

Implementing photon counting imaging with a single element detector presents clear advantages for eliminating the scattering ambiguity in traditional multifocal, multimodal multiphoton imaging systems. Besides permitting video-rate multi-depth imaging within highly scattering tissue, this technique can also be extended to simultaneously capturing fluorescent and harmonic data with one single-element detector. This technique can also be utilized with a programmable grating, permitting image deconvolution within scattering media. Additionally, the technique can be combined with spatio-temporal focusing in order to permit single-element detection from a line cursor excitation system without degradation to the effective numerical aperture.