Super-resolution retinal imaging using optically reassigned scanning laser ophthalmoscopy

Abstract

Super-resolution optical microscopy techniques have enabled the discovery and visualization of numerous phenomena in physics, chemistry and biology^1–3. However, the highest resolution super-resolution techniques depend on nonlinear fluorescence phenomena and are thus inaccessible to the myriad applications that require reflective imaging^4,5. One promising super-resolution technique is optical reassignment⁶, which so far has only shown potential for fluorescence imaging at low speeds. Here, we present novel advances in optical reassignment to adapt it for any scanning microscopy, including reflective imaging, and enable an order of magnitude faster image acquisition than previous optical reassignment techniques. We utilized these advances to implement optically reassigned scanning laser ophthalmoscopy, an in vivo super-resolution human retinal imaging device not reliant on confocal gating. Using this instrument, we achieved high-resolution imaging of living human retinal cone photoreceptor cells (determined by minimum foveal eccentricity) without adaptive optics or chemical dilation of the eye⁷.

One of the most accessible organs for studying neuronal behaviour in vivo is the eye, whose retina contains both sensory neurons (photoreceptors) and interneurons. The primary techniques for imaging individual retinal neurons such as the rod and cone photoreceptors and ganglion cells currently employ high-speed adaptive optics (AO) reflective imaging^8–11. Imaging of these cells in vivo with AO scanning light ophthalmoscopy (SLO) has enabled a new understanding of retinal properties such as colour vision¹² and the progression of blinding diseases such age-related macular degeneration and retinitis pigmentosa¹³. However, AO systems are complex and physically large and require expensive, specialised hardware^9,14. Moreover, imaging of the photoreceptors usually requires pharmacological dilation of subjects’ pupils to achieve high resolution. Thus, a non-AO system which maintains high-speed reflective imaging and achieves cellular resolution would be highly desirable.

SLO systems that utilise coherent illumination have a lower resolution than a widefield incoherent microscope¹⁵. If the SLO system is designed as a confocal microscope and utilises an infinitesimally small confocal pinhole, the resolution of an SLO can be improved to equal that of a widefield incoherent microscope, resulting in a doubling of the spatial frequency bandwidth¹⁵ and a $\sqrt{2}$ decrease in the full width at half maximum (FWHM) of the point spread function (PSF)¹⁶. However, using a small pinhole severely diminishes the system’s signal-to-noise ratio (SNR)¹⁷. This tradeoff between resolution and SNR is one of the fundamental drawbacks of confocal microscopy (CM)¹⁸. Several researchers have successfully used small pinholes or detectors to image the retina with adaptive optics^19–24. However, these systems usually require substantial averaging to reveal the additional detail contained in the images, indicating the need for a system capable of acquiring high-resolution images without SNR loss.

An alternative to confocal imaging which has recently been described for obtaining enhanced resolution is the photon or pixel reassignment technique^6,25–30. As in confocal imaging, photon reassignment utilises point scanning and detection but can obtain the complete resolution enhancement of confocal without loss of SNR (see Supp. Mat. III). This is achieved by assigning each detected photon to the position on the sample from which it most likely originated rather than at the centre of the illuminated spot, as assumed in traditional confocal imaging. In a system with identical and radially symmetric detection and illumination PSFs, it has been shown that photons detected a distance 2d from the centre of the illumination point most likely originated from a scatterer a distance d from the illumination point^25,26,28. The reassignment can thus be performed by shifting the detected photons d closer to the illumination point for every value of d, which is equivalent to minifying the returned PSF by a factor 2. Fig. 1 conceptually illustrates resolution improvement without signal loss in photon reassignment.

Figure 1|.

Simulation of reassignment procedure and resulting resolution enhancement. All images have dimensions of 80 px × 80 px and are centreed at (0,0). Scale bars measure 20 px. a, Sample consisting of a single point scatterer in the centre. Colored x’s indicate sample illumination positions. b, System intensity point spread function (PSF) of width σ = 10 px. Intensity range is 0 to 1. c, Detected light after applying reassignment when the sample is illuminated at the orange x. Intensity range is 0 to 1.3. d, Image reconstruction using only the reassigned detected light from the four illumination locations indicated in a. Intensity range is 0 to 1.7. e, Image reconstruction using all reassigned scan points. Intensity range is 0 to 1250. f, Cross-section of system PSF (blue, equivalent to widefield imaging), reassigned reconstruction (red), and CM simulation with 0.25 Airy disc diameter pinhole (green). The WF equivalent simulation has width σ = 10 px; the pixel-reassigned reconstruction and CM simulation both have width σ = 7.1 px.

An alternative method of performing photon reassignment is optical reassignment (OR) or re-scan confocal microscopy^{6,16,27,29–31}. In OR, the photon reassignment and image reconstruction are achieved in hardware. This has been performed by re-scanning the de-scanned light returning from the sample onto a camera with the same scan angles but an optically half-sized PSF^6,30. The super-resolved image is then obtained by integrating the signal on the camera over the full scan. This provides a real-time view of the super-resolved image without post-processing and only requires a camera with a frame rate matching that of the scan pattern. Because OR decouples the detection speed from the scan speed, it can utilise low bandwidth and high QE CCD and CMOS cameras, as opposed to high-bandwidth, low QE photomultiplier tubes, for an additional SNR improvement over confocal microscopy (see Supp. Mat. III). However, to our knowledge, OR has so far only been demonstrated in microscopy for fluorescence or multiphoton imaging^6,27,29,30 and reflected light imaging has not been attempted. To account for the substantial differences in coherent reflective microscopy and fluorescence microscopy, we have also developed a complete theoretical treatment of coherent reflective OR, presented in Supp. Mat. IV. There, we show that the bandwidth of the ORSLO transfer function is equal to that of an incoherent widefield imaging system. In addition, due to OR’s requirement of rescanning the descanned light in exact synchronisation with the illumination scan pattern, OR imaging speeds have been limited to frame rates of approximately 1 Hz or less given 512 lines per frame²⁹.

In this Letter, we describe conceptual advances enabling OR in any scanning system and imaging an order of magnitude faster than previous OR implementations. Using these advances, we constructed the first optically re-assigned scanning laser ophthalmoscope (ORSLO), the first in vivo super-resolution human retinal imaging device not reliant on confocal gating. A schematic of our optical system is shown in Fig. 2. To enable quantitative performance characterisation, the system could be configured either as a system with widefield equivalent (WFE) SLO resolution or as an ORSLO with $\sqrt{2}$ lateral resolution improvement by flipping mirrors FM1 and FM2. When the mirrors are flipped up, the light passes through lens L6A so that the collimated beam size on the backside of a custom-coated double-sided resonant scanner is the same (M = 1) as the beam size on the front of the scanner, producing WFESLO. When the mirrors are flipped down, the light passes through lens L6B and the beam size is increased by a factor of 2 (M = 2), producing ORSLO. This novel design is in contrast to previous versions of OR which relied on galvanometric scanners²⁹ due to their ease of synchronisation. Resonant scanners can achieve higher speeds than galvanometric scanners but it is difficult to synchronise two resonant scanners. Thus, our double-sided resonant scanner design guarantees synchronisation between the sample scan and re-scan sub-systems while allowing for an order of magnitude increase in imaging speed. Raw images were processed as described in Methods and Supp. Mat. I.

Figure 2|.

ORSLO schematic. All optical components are labeled and described in the legend. When flip mirrors are in the up position, lens L6A is used, as indicated by the dotted red lines, producing M =1 and widefield equivalent (WFE) resolution SLO. When flip mirrors are in the down position, lens L6B is used, as indicated by the solid red lines, producing M = 2 and super-resolved ORSLO.

To experimentally verify the resolution improvement of OR using this system, we imaged a 1951 USAF test target, located in the intermediate focal plane between lenses L3 and L4 in the schematic in Fig. 2. Imaging results for the test target are shown in Fig. 3. Resolutions were determined in terms of the FWHM and the resolution improvement factor was determined to be 1.41 for both the horizontal and vertical directions, respectively, identical to the expected resolution improvement of $\sqrt{2}$ or 1.41. The underswings in the PSF resulted from the combination of the non-zero reflection from the second surface of the test chart and the system PSF, which is mostly Gaussian but due to the iris had Bessel-like oscillations. Nevertheless, the data was fit to a Gaussian to capture the FWHM of the central peak. The underswings have an asymmetric appearance primarily due to the size of the error bars. The error bars are larger on the right-hand side (within the bright square) because there is more shot noise from brighter regions.

Figure 3|.

Widefield vs ORSLO imaging of a 1951 USAF test target. a, WFESLO image of test target. b, ORSLO image of same test target. Red and blue boxes indicate the locations where the horizontal and vertical point spread functions were calculated. c, Inset from a showing Group 5 Elements 1 and 2. d, Inset from a showing Group 5 square and indicating locations of PSF measurement. e, Inset from b showing Group 5 Elements 1 and 2. f, Inset from b showing group 5 square and indicating locations of PSF measurement. g-j, The corresponding point spread functions with resolutions indicated in terms of FWHM. Error bars indicate standard deviation of PSFs across all rows/columns.

Single frames acquired from three adult human subjects are shown in Fig. 4. Fig. 4a–c shows three single WFESLO frames from three different subjects with a 0.7° × 0.7° field of view (FOV) at 2.6°, 1.5°, and 1.0° eccentricities, respectively. Fig. 4d–f shows ORSLO frames from the same location and Fig. 4g–i shows the ORSLO frames with histograms matched (ORSLO + HM) to the WFESLO frames for fair comparison. To quantify the resolution improvement of OR, we excerpted 0.25° FOV images from each subject (at 2.5°, 1.4°, and 0.5°, respectively) and calculated the radial power spectra. The 2D spatial frequency spectrum from images of photoreceptors has been observed to contain an annulus of peak intensity (“Yellott’s ring”³²) whose radius corresponds to the density of the photoreceptors³³. Fig. 4j–l contains colour-coded plots of the radial spatial frequency spectrum for each of the imaging modes and eccentricities, along with previously reported measurements of cone density in subjects of similar ages obtained from histology (black vertical line)³⁴.

Figure 4|.

Single-frame comparisons of WFE and ORSLO. Scale bars are 0.25°. a-c, WFESLO single frames from 2.6° eccentricity in subject 01, 1.5° eccentricity in subject 02, and 1.0° eccentricity in subject 03, respectively. d-f, ORSLO single frames at the same location as a-c. g-i, ORSLO frames from g-i histogram matched to the single frames from WFESLO. j-l, Radial power spectra of 0.25° excerpts indicated in a-i, compared to a histological estimate of cone density at the same eccentricity (indicated by black vertical line and cones/mm²)²³. Excerpts are at 2.5°, 1.4°, and 0.5° eccentricity, respectively.

At 2.5°, both the ORSLO and WFESLO spectra exhibit a peak near the histological data, with the ORSLO peak exhibiting higher contrast. At 1.5°, a peak in the ORSLO spectrum is well-defined near the histological measurement, whereas the WFESLO data has a lower contrast peak. At 0.5°, there is a peak in the ORSLO spectrum near the histological measurement but no peak in the WFESLO spectrum, indicating that the cones at this location were not fully resolved for the WFESLO configuration but were for the ORSLO configuration. Additional location-matched comparison images from these subjects are shown in Supp. Mat. II.

Fig. 5 shows large field of view image mosaics in WFESLO and ORSLO configurations, each consisting of 43 single frames. Fig. 5a and 5b show 4.5° × 4.5° mosaics acquired in WFESLO and ORSLO configurations, respectively. The location of the fovea is indicated by a white star. Enlarged excerpts with a 0.25° × 0.25° FOV from 4.0°, 2.4°, and 0.9° eccentricity are shown in Fig. 5c–h. We again calculated the radial power spectra of these excerpts and compared them to histological measurements of cone density³⁴ as shown in Fig. 5i–k. A similar pattern is seen here as in Fig. 4. Overall, ORSLO consistently resolved cones across all measured eccentricities and well into the foveal avascular zone while WFESLO clearly resolved cones only in the 4° eccentricity excerpt.

Figure 5|.

WFESLO vs ORSLO retinal image mosaics. a, b, Retinal mosaics from subject 04 comparing WFESLO imaging and ORSLO imaging, respectively. Scale bars are 0.5°. c-h, excerpts from three locations in a and b, at 3.6°, 2.2°, and 0.7°, respectively. i-k, Power spectra of the excerpts and comparisons to histology²³.

The data shown here indicates a clear improvement in imaging resolution using OR, implemented entirely with commercially available optics. The visualisation of the cone photoreceptor mosaic at 0.5° eccentricity is a substantial improvement over techniques not utilising hardware AO, which were not able to quantitatively verify cone imaging closer than 3.1° eccentricity^17,35. Additionally, these images were acquired and displayed in real time at 16 frames per second (fps), in contrast to other resolution enhancement techniques such as pixel-based reassignment²⁶, which require offline processing. We have also verified, as shown in Supp. Mat. V, that this resolution and contrast enhancement cannot be obtained by deconvolution alone.

Our extension of OR to reflective imaging enables high-speed OR enhancement of any coherent or incoherent scanning microscopy technique. This technology could be widely implemented, enhancing the resolution and SNR of myriad imaging systems. When used in ophthalmic imaging with or without AO enhancement, the enhanced resolution and SNR of OR may permit robust photoreceptor imaging in subjects with diseases such as age-related macular degeneration, allowing for earlier diagnosis and enhanced prognosis of retinal disease^36,37 and better understanding of neuronal behaviour in vivo.

In conclusion, we have developed ORSLO, a high-speed optically super-resolved device for imaging samples under constant motion, such as the living eye, at resolutions beyond that of an unmodified SLO. We have demonstrated a clear lateral resolution improvement by imaging a USAF 1951 test chart and shown both qualitative and quantitative improvements when imaging the human retina. This additional improvement in retinal imaging has allowed ORSLO to obtain images of the cone photoreceptor mosaic closer to the fovea than any previous non-AO system. We expect ORSLO to have a significant impact on the development of future retinal imaging systems, enabling a new understanding of retinal structure and disease. With further developments, our high-speed OR technique can be extended to other imaging applications, including those already employing AO, to examine a broad range of dynamic systems at unprecedented resolutions.

Methods

System design

In the ORSLO system, depicted in Fig. 1, light from a 770 nm superluminescent diode (SLD) with a 15 nm bandwidth (InPhenix Inc., Livermore, CA) was directed through a polarisation-maintaining single mode fibre (PM780-HP, ThorLabs, Newton, NJ) into the system. An off-axis parabolic mirror (OAP) (RC04APC, ThorLabs, Newton, NJ) was then used to create a collimated Gaussian beam with a 2.8 mm 1/e² diameter. An iris aperture following a polarisation beam splitter was used to truncate the Gaussian beam to 1.4 mm in diameter, to create a beam that closely approximates a flat-top profile. For the fast-axis scanner, we used an 8 kHz resonant scanner (Cambridge Technologies, Cambridge, MA) that had a reflective coating on both sides of the mirror substrate to enable both scanning and de-scanning of light from the sample on one side, and re-scanning of light onto a camera on the other side. Light from the resonant scanner was imaged onto a galvanometer scanner (G1) by a 1:1 telescope (lenses L1 and L2).

The slow-axis galvanometer scanners used for scanning light on both the sample and camera were identical scanner models (GSV002, ThorLabs, Newton, NJ), driven with the same sample clock to ensure synchronisation. The telescope between the galvanometer scanners and the subject’s eye (lenses L3 and L4) had a transverse magnification of 2 to generate an ocular pupil beam size of 2.8 mm. This pupil beam size was chosen to maximise lateral resolution given typical human ocular aberrations³⁸.

To minimise the contribution of lens reflection artifacts, polarisation gating was applied with the use of a linear polariser, polarisation beam splitter, and quarter wave plate. A pinhole with a 1.5 Airy disc diameter (200 μm) was used to confocally reject out-of-focus backscattered light. This provided depth sectioning without altering lateral resolution in order to fairly test the ORSLO setup.

In order to better utilise the imaging area of our camera without affecting the resolution of the ORSLO setup, we angularly magnified the scan range of the resonant scanner by a factor of 2 using a telescope with a transverse magnification of ½ (using lenses L7 and L8) and doubled the scan range of G2 relative to G1 to match the optically magnified scan range from the resonant scanner. Finally, light was re-scanned across a 50 mm camera lens (86-614, Edmund Optics, Barrington, NJ) and imaged onto a 2D CMOS camera (GS3-U3-32S4M, FLIR, Richmond, BC, Canada), which had 2048 × 1536 pixels and a global shutter that was synced to the frame rate of the scan pattern on the sample (16 fps).

Human subjects research

The use of our experimental set-up for in vivo imaging of adults was approved by the Duke University Health System Institutional Review Board and adhered to the tenets of the Declaration of Helsinki. Informed consent was obtained from all subjects. Imaging sessions with this system lasted 15-30 minutes per subject. The optical power incident on the subject cornea was 0.54 mW or less, which is within the most conservative limits of the ANSI Z136.1 standard³⁹ for the 774 ± 5 nm source used.

In this study, four adult subjects without known retinal pathology were imaged. Data collected from this research were stored and managed in compliance with guidelines from the Health Insurance Portability and Accountability Act.

Image Processing

Images from the ORSLO system were recorded as binary data files. The files were read by custom scripts in MATLAB and a previously measured background image (specific to the system configuration) was subtracted from each frame. The image was then manually cropped to retain only the ORSLO field of view and corrected for variable intensity (see Supp. Fig. 1). After processing, images were linearly contrast adjusted to occupy the full 0-255 range.

Image mosaics were created by one of two methods. For foveal and parafoveal imaging, images were manually overlapped and combined. For imaging at more distant positions, the MosaicJ plugin for ImageJ⁴⁰ was used.

Test chart resolution measurement

We quantified the system resolution in test chart images by first extracting horizontal and vertical edges of the Group 4 square, depicted by the red and blue boxes in Fig. 3. We then differentiated each image across the edge and fit each row or column to a Gaussian. Using the fits, the rows or columns were registered and averaged. The FWHM of a Gaussian fit to an averaged row or column was used as the system resolution.

As per Supp. Mat. Section IV, the ORSLO system does not have a traditional PSF due to the combination of coherent and incoherent imaging. However, this measurement of the apparent PSF of a sharp edge serves as a useful metric for assessing the resolution and contrast enhancement of the system.

Cone density estimation

Radially-integrated spatial frequency power spectra were created from 0.25° FOV subimages using custom MATLAB scripts. Each subimage was zero-padded, Fourier transformed, and shifted so that the DC term was in the centre of the image. The magnitude of each pixel was calculated, then radially summed to produce a power spectrum in terms of radial frequency (cycles per degree). The study performed by Curcio et al.³⁴ measured the number of cones/mm² in histological samples of the retina taken by seven subjects of ages 27-44. For comparison to our power spectra, we first determined the average histological cones/mm² measurement at the eccentricity where the spectrum was obtained. We then converted that density to linear frequency by the method of Cooper et al.³³ for an eye with an anatomically average axial length of 24 mm³⁴. Our subjects were all emmetropes, indicating that their eye lengths are close to average^41,42.

Data Availability

The data that support the findings of this study are available from the corresponding author on reasonable request.

Code Availability

The code used in this study is available at http://github.com/tdubose/ORSLO.