Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Sep 1.
Published in final edited form as: Opt Lett. 2020 Sep 1;45(17):4940–4943. doi: 10.1364/OL.402392

Multimodal handheld adaptive optics scanning laser ophthalmoscope

Kristen Hagan 1, Theodore DuBose 1, David Cunefare 1, Gar Waterman 1, Jongwan Park 1, Corey Simmerer 1, Anthony N Kuo 2,1, Ryan P McNabb 2, Joseph A Izatt 1,2, Sina Farsiu 1,2,*
PMCID: PMC7792463  NIHMSID: NIHMS1655580  PMID: 32870897

Abstract

Non-confocal adaptive optics scanning laser ophthalmoscopy (AOSLO) has enhanced the study of human retinal photoreceptors by providing complementary information to standard confocal AOSLO images. Previously, we developed the first confocal handheld AOSLO (HAOSLO) capable of in vivo cone photoreceptor imaging in supine and non-cooperative patients. Here, we introduce the first multimodal (M-) HAOSLO for confocal and non-confocal split-detection (SD) imaging to allow for more comprehensive patient data collection. Aside from its unprecedented miniature size and weight, M-HAOSLO is also the first system to perform sensorless wavefront corrected SD imaging of cone photoreceptors.

The integration of adaptive optics (AO) technology into ophthalmic imaging systems has allowed for the dynamic compensation of ocular and system aberrations. Using wavefront sensor or sensorless approaches, AO-enhanced retinal imaging systems have achieved near diffraction limited performance, enabling the in vivo visualization of retinal cellular and sub-cellular features [1–8]. In particular, the confocal AO scanning laser ophthalmoscope (AOSLO) has enabled reliable rod and cone photoreceptor imaging, presenting the potential use for AOSLO images in investigations of retinal disease biomarkers. However, conventional AOSLO systems span large optical tables, often require subject stabilization mechanisms, and cannot accomodate a variety of subject positions during imaging sessions. To address these limitations and extend AO imaging to non-cooperative patient populations, we previously developed the first confocal handheld AOSLO (HAOSLO) system that was able to resolve individual cone photoreceptors in neonates and supine adults while maintaining a compact form factor [9].

As shown in multiple recent studies, confocal AOSLO imaging may result in ambiguity in visualization and analysis of retinal photoreceptors [10, 11]. To address this critical problem, non-confocal offset aperture AOSLO imaging has been demonstrated to enhance the visualization of retinal structures in cooperative adult subjects [6, 12–16]. Specifically, studies have shown that more reliable identification of cone photoreceptors and their inner segments is possible by integrating the information obtained from confocal AOSLO with the complementary non-confocal split-detection (SD) AOSLO images [10, 11]. Similar to differential phase microscopy techniques, non-confocal SD AOSLO utilizes two offset channels to collect differentially propagating photons scattered from the cones, thereby forming an image featuring cone inner segments [10]. However, the incorporation of non-confocal SD channels complicates AOSLO system designs. Thus, despite being shown to enhance visualization of a variety of anatomic and pathological features, incorporating non-confocal channels into AOSLO setups has only been demonstrated in tabletop systems.

Here, we extend our HAOSLO probe to demonstrate the first use of SD imaging in a compact, portable handheld system (216 g) and to expand multimodal (confocal and non-confocal) AOSLO imaging to more flexible patient imaging scenarios. We adapted the collection channel of our previous confocal HAOSLO probe [9] to enable simultaneous three channel detection. The system schematic in Fig. 1a displays the polarization-gated illumination and collection channels of this lens-based system. A 774±5nm superluminescent diode (IPSDD0702C, Inphenix Inc.) illuminated the retina, and a custom fiber bundle (Gulf Photonics Group) collected confocal and two channels of laterally offset non-confocal light from the retina. The trifurcated fiber bundle contained 19 multimode fibers (MMFs) with 50 μm core diameters (Fig. 1a, inset) arranged in a tight hexagonal pattern. The offset fibers were symmetrically divided into two sets of 8 MMFs and routed to two trans-impedance amplifiers (HCA-40M-100K-C, Electro Optical Components Inc.) and photomultiplier tubes (H7422–40, Hamamatsu) while the central MMF was routed to an avalanche photodiode (C12703, Hamamatsu) for confocal signal detection.

Fig. 1. M-HAOSLO Imaging System:

Fig. 1.

Open in a new tab

(a) System schematic detailing custom fiber bundle, enabling three channel (confocal and dual-channel non-confocal) light collection. ADD: Airy disk diameter, APD: avalanche photodiode, DM: deformable mirror, FM: fold mirror, LP: linear polarizer, MMF: multimode fiber, PBS: polarizing beam splitter, PC: polarization controller, PM: polarization-maintaining, PMT: photomultiplier tube, QWP: quarter wave plate, SLD: superluminescent diode, SMF: single mode fiber, TIA: transimpedance amplifier, and TFB: trifurcated fiber bundle. End-on view of fiber bundle that displays packing of 19 MMFs with 50 μm core diameter. (b) Rendering of probe displaying lightweight and compact skeleton design with the beam path shown in red. Dimensions: 12.3 cm x 5.3 cm x 14.4 cm. (c) Photograph of enclosed probe ready for handheld operation.

A microelectromechanical systems (MEMS) mirror (A5L2.1, Mirrorcle Technologies) scanned the beam across a 1.0°×1.0° field of view (FOV) on the retina. The mirror operated in resonance at 1704 Hz on the fast axis, while the slow axis was set to 6.8 Hz to achieve a frame rate of 6.8 Hz. HAOSLO’s theoretical full-width-at-half-maximum spot diameter of 2.3 μm was maintained from [9]. Lens reflections in the confocal channel were suppressed via confocal and polarization gating using a polarizing beam splitter (1000:1 extinction ratio, PBS052, Thorlabs, Inc.) [9]. To further suppress lens reflections in the non-confocal images, we added a linear polarizer (10000:1 extinction ratio, LPNIR050, Thorlabs, Inc.) in the collection path. An achromatic focusing lens with a 40 mm effective focal length (EFL) (45–790, Edmund Optics) created an effective confocal pinhole of 2.18 Airy disk diameters (ADDs) and non-confocal channels with collection up to 13.54 ADDs.

A compact deformable mirror (DM) (DM69, ALPAO) [9] was used as the adaptive component allowing for dynamic wavefront correction. Unlike previously described SD imaging systems [10, 15, 16], optical aberrations were estimated and corrected without a wavefront sensor by using our previously described stochastic Zernike gradient descent (SZGD) algorithm [9] until the mean intensity of the acquired confocal image was maximized.

Before each imaging session, a model eye, consisting of a 25 mm EFL achromatic lens (49–786, Edmund Optics) and a paper retina imaging target, was used to calibrate for system aberrations. The SZGD algorithm optimized the DM shape until the confocal image intensity was maximized. The collection fiber bundle was translated along three axes to further optimize the confocal image. This DM shape was saved as the initial state for human imaging.

Unlike conventional tabletop AO systems with dedicated stabilization mechanisms, M-HAOSLO introduces handheld motion from the operator resulting in substantial inter- and intra-frame motion. To correct for this, we developed a custom image processing pipeline for images collected in handheld mode. After manual reference frame selection, each additional confocal frame with a resolved cone mosaic and corresponding FOV was registered using control point-based strip registration. The control points were manually selected cone photoreceptors. Then, local SIFT based registration was performed to enhance the registration results and improve the final averaged images [17]. Corresponding SD images were co-registered using the confocal registration parameters. For the stabilized imaging sessions (see below), in place of our semi-automatic method, an automatic cross-correlation strip registration method was used [18].

We imaged the right eye of four healthy adult subjects that were either mildly hyperopic or emmetropic. Dilation was performed using 0.5% tropicamide and 5% phenylephrine. The optical power incident on the subject cornea was 0.48 mW or less, within the most conservative limits of the ANSI Z136.1–2014 power standard for 8 hours of exposure [19]. Subjects were consented under a Duke University Hospital System Institutional Review Board approved protocol and adhered to the tenets of the Declaration of Helsinki. Imaging sessions lasted under 1 hour.

Each imaging session consisted of three stages—stabilized chinrest mode, upright handheld mode, and supine handheld mode. First, the probe was mounted on a converted slit lamp table with chin and forehead rest, providing basic subject stabilization, to serve as a benchmark test of M-HAOSLO and demonstrate its ability to be used as a tabletop system. An LED fixation target, placed on the opposing wall and visible to the non-dilated left eye, allowed for imaging approximately 3–5° temporal to the fovea. Subject alignment consisted of initial coarse defocus correction followed by enabling the SZGD algorithm for higher order aberration correction. Following wavefront correction, confocal and two non-confocal channel images were collected using LabVIEW software modified from [9]. Each SD frame was calculated in post-processing as the difference of the two offset channel images divided by their sum [10]. Following stabilized imaging, the probe was used in handheld operation mode to image the subject in an upright, sitting position. The LED fixation target remained on the wall, but the exact retinal eccentricity of the collected frames was not known due to the involuntary head motion of the volunteer. The change in position (and image FOV) necessitated the re-optimization of the wavefront during imaging. The final stage of imaging was the handheld use of M-HAOSLO in a supine position. The subjects transitioned to a fully reclined chair and fixated on an LED placed on the ceiling.

During a stabilized imaging session, we investigated confocal and SD image quality during SZGD convergence in a healthy emmetrope (Fig. 2). After initial defocus correction, the sensorless algorithm was started. Until plateauing after 8 seconds, we observed that the confocal mean intensity increased (Fig. 2a) as the non-confocal mean intensity decreased (Fig. 2b), suggesting that backcoupling of singly scattered light into the pinhole improved as the DM shape optimized. At various times, we acquired between 20–80 high resolution frames to create 5 frame averaged confocal and SD images. Following methods from [20], we analyzed the radially averaged image spatial frequencies and observed that Yellott’s ring (a key imaging biomarker of cone density [20, 21]) became sharper with improved contrast over time (Fig. 2f–h, o–q). This is revealed in the increasing spectral sharpness [22] and spectral peak heights in the radial cross section plots (Fig. 2i–k, r–t) resulting in improved identification of the cone modal frequency. Thus, the confocal-only optimization metric resulted in joint improvement of confocal and SD images.

Fig. 2. Optimization of confocal and SD images during sensorless wavefront correction:

Fig. 2.

Open in a new tab

(a) Mean intensity of confocal channel and (b) non-confocal channels as DM converged to the optimal shape in a healthy emmetrope, subject 3. Nine trials were taken in which convergence was stopped at various time points. Error bars display standard error in intensity value. Orange, purple, and green indicate t= 0, t= 5.66, and t= 21.16 s, respectively. (c-e) Confocal images averaged from 5 frames each and their (l-n) corresponding SD images, respectively. To the right of the images, the log10 magnitudes of the Discrete Fourier Transform are displayed in (f-h) and (o-q) revealing Yellott’s ring. Adjusted average radial cross sections of Fourier transformed images are shown in (i-k) and (r-t) for confocal and SD images calculated as in [20], respectively. The location and height of the spectral peak corresponding to the modal cone spatial frequency, Cf (pixels−1) is denoted by a red line. Metrics of detected cone frequency, spectral peak height, and spectral sharpness (the sum of squared intensity values divided by the sum of intensity values [22]) are shown.

Stabilized imaging results displaying the cone mosaic are shown in Fig. 3. Subject 1 (S1) was a 26-year-old hyperope, and subject 2 (S2) was a 31-year-old emmetrope. Results from subject 3 (S3), a 24-year-old emmetrope, are shown in Fig. 4 with M-HAOSLO used in upright handheld mode. Despite added motion from the subject (non-stabilized head) and imager (hand), which reduced the number of frames collected from the same FOV, the cone mosaic in confocal images and cone inner segments in SD images were clearly visualized. Figure 5 shows images from three supine subjects. Due to the more stable supine position, more images with corresponding FOVs were captured, enabling the creation of a mosaic from S4, a 24-year-old emmetrope (Fig. 5c–d). S1 and S3 results are shown in Fig. 5a–b and 5e–f, respectively.

Fig. 3. Tabletop imaging:

Fig. 3.

Open in a new tab

M-HAOSLO images acquired in tabletop mode on dilated, stabilized adult volunteers (S1-S2). (a) Averaged confocal image using 16 frames from S1. (b) Co-registered and averaged SD image. (c) Averaged confocal image using 18 frames from S2. (d) Co-registered and averaged SD image. Scale bar, 0.15°.

Fig. 4. Upright handheld imaging:

Fig. 4.

Open in a new tab

M-HAOSLO images acquired in handheld operation mode on dilated adult volunteer (S3) sitting in an upright position. (a) Averaged confocal image using 4 frames. (b) Co-registered and averaged SD image. (c) Averaged confocal image using 5 frames. (d) Co-registered and averaged SD image. Scale bar, 0.15°.

Fig. 5. Supine handheld imaging:

Fig. 5.

Open in a new tab

M-HAOSLO images acquired in handheld operation mode on supine dilated adult volunteers. (a) Averaged confocal image using 4 frames from S1. (b) Co-registered and averaged SD image from S1. (c) Four mosaiced, averaged confocal images using 6–8 frames each from S4. (d) Four mosaiced, co-registered and averaged SD images from S4. (e) Averaged confocal image using 4 frames from S3. (f) Co-registered and averaged SD image from S3. Scale bars for each image pair, 0.25°.

We achieved the first demonstration of a compact and handheld AOSLO capable of simultaneous non-confocal SD and confocal imaging of cone photoreceptors. Although M-HAOSLO’s compact lens-based design and sensorless wavefront estimation is not expected to achieve the image quality of tabletop SD AOSLO systems, M-HAOSLO presents the potential of reaching previously excluded patient populations, such as bedridden adults or children. The diagnosis and treatment of these populations could benefit from the more comprehensive data collection provided by M-HAOSLO. Also, to the best of our knowledge, this is the first demonstration of wavefront sensorless AO to collect SD images in the human eye. Thus, M-HAOSLO serves as a platform for SD imaging based on wavefront estimation rather than wavefront measurement. Beyond this pilot study, further investigation on subjects with a compromised cone mosaic and direct comparison to measurements from tabletop AOSLOs would allow for investigation of SZGD’s robustness for AOSLO imaging. In such cases, a joint optimization approach utilizing both confocal and non-confocal channel information may produce optimal results. A future direction of this work is to investigate the visualization of retinal pigment epithelial cells using dark-field imaging, which is expected to be most successful in the foveal region [23].

Acknowledgment.

Funding. National Institutes of Health (NIH) (R21 EY027086, R21 EY029804, P30EY05722); 2018 Unrestricted Grant from Research to Prevent Blindness (RPB); National Science Foundation Graduate Research Fellowship Program (1644868).

Footnotes

Disclosures. K. Hagan, T. DuBose, J. A. Izatt, and S. Farsiu are inventors on a provisional patent related to this work.

REFERENCES

  • 1.Liang J, Williams DR, and Miller DT, “Supernormal vision and high-resolution retinal imaging through adaptive optics,” JOSA A 14, 2884–2892 (1997). [DOI] [PubMed] [Google Scholar]
  • 2.Roorda A, Romero-Borja F, Donnelly III WJ, Queener H, Hebert TJ, and Campbell MC, “Adaptive optics scanning laser ophthalmoscopy,” Optics express 10, 405–412 (2002). [DOI] [PubMed] [Google Scholar]
  • 3.Hofer H, Sredar N, Queener H, Li C, and Porter J, “Wavefront sensorless adaptive optics ophthalmoscopy in the human eye,” Optics Express 19, 14160–14171 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Zawadzki RJ, Jones SM, Pilli S, Balderas-Mata S, Kim DY, Olivier SS, and Werner JS, “Integrated adaptive optics optical coherence tomography and adaptive optics scanning laser ophthalmoscope system for simultaneous cellular resolution in vivo retinal imaging,” Biomedical Optics Express 2, 1674–1686 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Dubra A, Sulai Y, Norris JL, Cooper RF, Dubis AM, Williams DR, and Carroll J, “Noninvasive imaging of the human rod photoreceptor mosaic using a confocal adaptive optics scanning ophthalmoscope,” Biomedical optics express 2, 1864–1876 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Rossi EA, Granger CE, Sharma R, Yang Q, Saito K, Schwarz C, Walters S, Nozato K, Zhang J, Kawakami T, Fischer W, Latchney LR, Hunter JJ, Chung MM, and Williams DR, “Imaging individual neurons in the retinal ganglion cell layer of the living eye,” Proc Natl Acad Sci U S A 114, 586–591 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Wong KSK, Jian Y, Cua M, Bonora S, Zawadzki RJ, and Sarunic MV, “In vivo imaging of human photoreceptor mosaic with wavefront sensorless adaptive optics optical coherence tomography,” Biomed. Opt. Express 6, 580–590 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Liu Z, Kurokawa K, Zhang F, Lee JJ, and Miller DT, “Imaging and quantifying ganglion cells and other transparent neurons in the living human retina,” Proceedings of the National Academy of Sciences 114, 12803–12808 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.DuBose T, Nankivil D, LaRocca F, Waterman G, Hagan K, Polans J, Keller B, Tran-Viet D, Vajzovic L, Kuo AN, Toth CA, Izatt JA, and Farsiu S, “Handheld adaptive optics scanning laser ophthalmoscope,” Optica 5, 1027–1036 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Scoles D, Sulai YN, Langlo CS, Fishman GA, Curcio CA, Carroll J, and Dubra A, “In vivo imaging of human cone photoreceptor inner segments,” Invest Ophthalmol Vis Sci 55, 4244–4251 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Cunefare D, Huckenpahler AL, Patterson EJ, Dubra A, Carroll J, and Farsiu S, “RAC-CNN: multimodal deep learning based automatic detection and classification of rod and cone photoreceptors in adaptive optics scanning light ophthalmoscope images,” Biomedical Optics Express 10, 3815–3832 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Elsner AE, Miura M, Burns SA, Beausencourt E, Kunze C, Kelley LM, Walker JP, Wing GL, Raskauskas PA, Fletcher DC, Zhou Q, and Dreher AW, “Multiply scattered light tomography and confocal imaging: detecting neovascularization in age-related macular degeneration,” Opt. Express 7, 95–106. [DOI] [PubMed] [Google Scholar]
  • 13.Chui TY, Vannasdale DA, and Burns SA, “The use of forward scatter to improve retinal vascular imaging with an adaptive optics scanning laser ophthalmoscope,” Biomed Opt Express 3, 2537–2549 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Guevara-Torres A, Williams DR, and Schallek JB, “Origin of cell contrast in offset aperture adaptive optics ophthalmoscopy,” Opt Lett 45, 840–843 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Sulai YN, Scoles D, Harvey Z, and Dubra A, “Visualization of retinal vascular structure and perfusion with a nonconfocal adaptive optics scanning light ophthalmoscope,” JOSA A 31, 569–579 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Mozaffari S, Jaedicke V, LaRocca F, Tiruveedhula P, and Roorda A, “Versatile multi-detector scheme for adaptive optics scanning laser ophthalmoscopy,” Biomedical optics express 9, 5477–5488 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Liu C, Yuen J, and Torralba A, “Sift flow: Dense correspondence across scenes and its applications,” IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 978–994 (2010). [DOI] [PubMed] [Google Scholar]
  • 18.Dubra A and Harvey Z, “Registration of 2D images from fast scanning ophthalmic instruments,” in International Workshop on Biomedical Image Registration, (Springer, 2010), 60–71. [Google Scholar]
  • 19.A. N. S. Institute, “ANSI Z136.1 American National Standard for Safe Use of Lasers,” (Laser Institute of America, 2014). [Google Scholar]
  • 20.Cunefare D, Cooper RF, Higgins B, Katz DF, Dubra A, Carroll J, and Farsiu S, “Automatic detection of cone photoreceptors in split detector adaptive optics scanning light ophthalmoscope images,” Biomedical Optics Express 7, 2036–2050 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Cooper RF, Langlo CS, Dubra A, and Carroll J, “Automatic detection of modal spacing (Yellott’s ring) in adaptive optics scanning light ophthalmoscope images,” Ophthalmic and Physiological Optics 33, 540–549 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Fienup JR and Miller JJ, “Aberration correction by maximizing generalized sharpness metrics,” J. Opt. Soc. Am. A 20, 609–620 (2003). [DOI] [PubMed] [Google Scholar]
  • 23.Tam J, Liu J, Dubra A, and Fariss R, “In Vivo Imaging of the Human Retinal Pigment Epithelial Mosaic Using Adaptive Optics Enhanced Indocyanine Green Ophthalmoscopy,” Invest Ophthalmol Vis Sci 57, 4376–4384 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES