Accuracy of the Heidelberg Spectralis in the alignment between near-infrared image and tomographic scan in a model eye: a multicenter study

Abstract

Purpose

To evaluate temporal changes and predictors of accuracy in the alignment between simultaneous near-infrared image and optical coherence tomography (OCT) scan on the Heidelberg Spectralis using a model eye.

Design

Laboratory investigation.

Methods

After calibrating the device, six sites performed weekly testing of the alignment for 12 weeks using a model eye. The maximum error was compared to multiple variables to evaluate predictors of inaccurate alignment. Variables included the number of weekly scanned patients, total number of OCT scans and B-scans performed, room temperature and its variation, and working time of the scanning laser. A 4-week extension study was subsequently performed to analyze short-term changes in the alignment.

Results

The average maximum error in the alignment was 15±6 µm; the greatest error was 35 µm. The error increased significantly at week 1 (p=0.01), specifically after the second imaging study (p<0.05), reached a maximum after the eighth patient (p<0.001), and then it varied randomly overtime. Predictors for inaccurate alignment were temperature variation and scans per patient (p<0.001). For each 1 unit of increase in temperature variation, the estimated increase in maximum error was 1.26 µm. For the average number of scans per patient, each increase of 1 unit increased the error of 0.34 µm.

Conclusion

Overall, the accuracy of the Heidelberg Spectralis was excellent. The greatest error happened in the first week after calibration, and specifically after the second imaging study. To improve the accuracy, room temperature should be kept stable and unnecessary scans should be avoided. The alignment of the device does not need to be checked on a regular basis in the clinical setting, but should be checked after every other patient for more precise research purposes.

Introduction

Spectral-domain optical coherence tomography (SD-OCT) has emerged as the gold-standard non-invasive technique to visualize fine retinal details and to evaluate retinal structural changes. High axial scanning speeds and axial resolution of 5 µm to 7 µm allow histological-like cross-sectional images of the posterior structures.¹ Many SD-OCT devices are commercially available, with different features including eye-tracking, image averaging, 3-dimensional imaging, and choroid enhancement.

The Heidelberg Spectralis (Heidelberg Engineering, Heidelberg, Germany) is a widely used confocal scanning laser ophthalmoscope (cSLO). The device simultaneously performs and correlates SD-OCT scans with multiple imaging modalities such as near-infrared (NIR), fundus autofluorescence, red free, fluorescein angiography, and indocyanine green angiography. Its unique ability to co-localize posterior structures on bi-dimensional fundus images and on cross-sectional scans helps ophthalmologists in the diagnosis and the management of many ocular disorders for both clinical care and research. The platform of the device simultaneously images the eye with two beams of light; one beam captures an image of the retina and maps over 1,000 points to track eye movement. Using the mapped image as a reference, the second beam is directed to the desired location despite blinks or saccadic eye movements. The eye-tracking dual-beam technology (TruTrack™ Active Eye Tracking software, Heidelberg Engineering, Heidelberg, Germany) mitigates eye motion artifact and ensures point-to-point correlations between the OCT scan and fundus images. The eye-tracking technology also permits precise scanning of the same location over consecutive visits, and has been proven to provide repeatable macular thickness measurements.² The infrared light (830 nm) of the cSLO is largely invisible to the patient, thus NIR is the best-tolerated and most commonly used imaging modality while acquiring simultaneous OCT scans. Moreover, this wavelength does not necessitate pupil dilation in order to obtain good-quality OCT scans at any arbitrary location of the posterior pole.

The correlation of bi-dimensional fundus images with cross-sectional images allows precise evaluation of posterior structures, as well as analysis of structural changes over micrometric areas of interest. Therefore, precise alignment between the NIR image and the OCT scan is necessary to avoid errors when performing micro-structural analysis of the retinal anatomy. The purpose of this multicenter study was to evaluate temporal changes in the alignment between the simultaneous NIR image and OCT scan when using the Heidelberg Spectralis, which would further assess the inter-instrument variability and determine predictors of inaccurate alignment.

Methods

Six ophthalmological sites were involved in this multicenter study beginning in March of 2012. Sites included (1) Jacobs Retina Center and (2) Hamilton Glaucoma Center at the Shiley Eye Center (University of California San Diego, La Jolla, California, USA); (3) Fondazione IRCCS Ca' Granda-Ospedale Maggiore Policlinico and (4) Luigi Sacco Hospital (University of Milan, Milan, Italy); (5) Azienda Ospedaliera Sant’Anna (Como, Italy); (6) Department of Ophthalmology (Ludwig-Maximilians-University, Munich, Germany).

Before starting the study, all sites checked the alignment of their own device and then performed a calibration procedure, using a dedicated alignment/calibration tool as model eye, and a dedicated software developed by Heidelberg Engineering (Heidelberg, Germany). The tool consisted of a lens and a three-dimensional target. The target was an inverse square pyramid with a slight rotational offset at each step. The software was written to use the 3-dimensional nature of the target to find proper alignment between the cSLO image and the OCT image. The procedure to check the alignment required approximately 60 seconds, and the calibration procedure required between 2 to 3 minutes. The alignment procedure was repeated every week for a total of 12 weeks. This check was done after the last scheduled patient was scanned, and the printout of the procedure was collected. There was no direct contact with patients’ eyes during the alignment/calibration procedure.

All sites collected daily data about turn-on and turn-off times of the laser of the devices, as well as lowest and highest temperature in the area where the Heidelberg Spectralis was located. Using the Heidelberg Eye Explorer software (Heyex, Heidelberg Engineering, Heidelberg, Germany), the number of weekly scanned patients, total number of OCT scans performed, and total number of B-scans performed was exported and collected retrospectively. The average number of weekly scans performed per patient was calculated; each scan (including line, circle, star, or raster scans) was considered to count as 1 unit. The average number of weekly B-scans performed per patient was calculated as well; line and circle scans consist of 1 B-scan, star scan consists of 6 B-scans, and raster scans consist of multiple B-scans according to the settings chosen by the operator. Patient identifying information was not recorded and their imaging results were not analyzed. During the 3-month study, sites were asked not to delete any OCT scans acquired on the device, nor to turn off the laser of the device until scanning the last scheduled patient for each day.

After completing the 12-week study, a 4-week extension study was performed at the Jacobs Retina Center (site 1) to analyze the short-term (daily) change in the alignment. Calibration was performed before scanning the first patient of the week for each of the four weeks. The alignment of the device was then re-checked right after completing the image acquisition of each of these patients. During the following days of the week, the alignment was checked only after scanning the last patient of the day. The same protocol was repeated for all 4 weeks.

Statistical analyses were performed applying the Generalized Estimating Equations (GEE) test using SAS statistical software version 9.2 (SAS Inc, Cary, North Carolina, USA). A p-value <0.05 was considered to be statistically significant.

Results

A summary of all results of the 12-week study is presented in Table 1. After the calibration procedure at baseline, the mean horizontal error among the 6 sites was 0.54 ± 0.31 pixels, equivalent to 3.10 ± 1.80 µm; the mean vertical error was 0.69 ± 0.12 pixels, equivalent to 3.93 ± 0.68 µm. During the 12-week study, the mean horizontal error among the 6 sites was 2.43 ± 1.20 pixels, equivalent to 13.90 ± 6.89 µm; the mean vertical error was 2.46 ± 1.14 pixels, equivalent to 14.08 ± 6.55 µm. The average maximum error among all measurements was 2.60 ± 1.01 pixels (range, 0.41 – 6.07 pixels), equivalent to 15.35 ± 6.29 µm (range, 2.35 – 34.78 µm).

Table 1.

Summary of results during the 12-week study to evaluate the alignment between near-infrared image and tomographic scan on the Heidelberg Spectralis.

	Sites
	(1) Jacobs Retina Center	(2) Hamilton Glaucoma Center	(3) Ospedale Maggiore Policlinico	(4) Luigi Sacco Hospital	(5) Azienda Ospedaliera Sant’Anna	(6) Department of Ophthalmology Munich
Horizontal error (µm ± SD)	18.79 ± 5.39	16.87 ± 1.73	15.99 ± 8.42	8.16 ± 4.35	15.26 ± 7.52	13.74 ± 3.24
Vertical error (µm ± SD)	18.39 ± 4.92	17.21 ± 1.53	17.76 ± 7.96	8.30 ± 4.73	14.42 ± 6.48	13.51 ± 2.66
Maximum error* (µm ± SD)	18.96 ± 5.30	17.42 ± 1.70	17.63 ± 7.95	8.72 ± 4.61	15.35 ± 7.45	14.03 ± 3.12
Lowest temperature (°C ± SD)	18.75 ± 0.45	20.08 ± 0.79	20.17 ± 2.17	20.17 ± 1.53	21.33 ± 0.65	19.6 ± 1.31
Highest temperature (°C ± SD)	23.25 ± 1.14	23.5 ± 0.52	25.3 ± 1.37	25.25 ± 1.71	25.00 ± 0.43	26.08 ± 0.79
Temperature excursion (°C ± SD)	4.25 ± 1.06	2.92 ± 0.79	4.42 ± 2.11	3.67 ± 0.49	3.42 ± 0.90	6.00 ± 1.28
Working time (hours ± SD)	26.06 ± 4.80	43.57 ± 10.31	33.47 ± 7.26	41.65 ± 7.24	44.88 ± 6.91	39.10 ± 5.10
Scanned patients (No ± SD)	40.50 ± 7.86	15.25 ± 7.28	140.50 ± 30.99	155.58 ± 49.38	62.25 ± 7.75	24.58 ± 7.60
OCT scans (No ± SD)	874.92 ± 278.99	176.42 ± 88.69	997.58 ± 221.44	626.67 ± 215.68	380.75 ± 63.10	64.33 ± 18.20
Scans per patient (No ± SD)	21.27 ± 3.45	11.35 ± 1.09	7.15 ± 0.57	4.03 ± 0.43	6.11 ± 0.57	2.65 ± 0.42
B-scans (No ± SD)	4,374 ± 999	6,029 ± 2,739	7,662 ± 1,94	11,402 ± 3,845	4,927 ± 820	n/a
B-scans per patient (No ± SD)	109.64 ± 28.91	409.26 ± 70.76	54.26 ± 3.90	72.46 ± 6.76	79.02 ± 7.15	n/a

SD, standard deviation; OCT, optical coherence tomography; No, number; n/a, not available.

^*Maximum error = the highest value between vertical and horizontal errors.

Analysis of variance showed that the maximum error (i.e., the highest value between vertical and horizontal errors) increased significantly from baseline to week 1 (p=0.01) but no significant changes were noted between week 1 and the following weeks. Figure 1 shows the change of mean horizontal and vertical errors, over time. Multiple comparison analysis using Tukey’s Honestly Significant Difference test showed that the maximum error was significantly lower in site #4 than all the others (p<0.01 for sites # 1, 2, and 3; p<0.05 for site # 5) except for site #6 (p=0.183).

Figure 1.

Change of mean horizontal and vertical errors in the alignment between simultaneous near-infrared image and optical coherence tomography scan on Heidelberg Spectralis, during the 12-week study. Vertical bars indicate the standard deviation of the measurements.

After adjusting for site, univariate regression analysis indicated that the maximum error was correlated with the highest variation in temperature (p<0.01), lowest temperature (p<0.001), and scans per patient (p<0.01). Multivariate analysis showed that the model with the best regression to predict the maximum error was determined by temperature variation and scans per patient (p<0.001, R² = 0.19). Univariate parameter estimates showed that for each 1 unit (°C) of increase in temperature, the estimated increase in maximum error was 0.22 pixels (equivalent to 1.26 µm). For the average number of scans per patient, each increase of 1 unit increased the estimated error of 0.06 pixels (equivalent to 0.34 µm).

In the 4-week extension study, analysis of variance showed that the mean error in alignment was stable after the first patient of day 1 (p=0.181), but increased significantly from baseline after scanning the second patient (p<0.05). Moreover, the level of significance further increased from baseline to the fourth patient (p<0.01) and to the seventh patient (p<0.001) of day 1. The maximum error was noted after scanning the eighth patient out of an average of 12 patients, and was equal to 24.17 ± 4.22 µm. The difference in alignment between the second and eighth patient represented a trend but not a statistically significant change (p=0.066). No significant changes were noted when comparing the mean error after the first day to the mean errors of the following days of each week. Figure 2 shows the change of mean error from the baseline calibration to the last scanned patient of day 1, and from day 1 to day 5 of the week. These means are based on averaged results from all results of 4 weeks.

Figure 2.

Change of mean error in the alignment between simultaneous near-infrared image and optical coherence tomography scan on Heidelberg Spectralis during day 1 (Dashed Line), and from day 1 to day 5. The means are based on averaged results from all results of 4 weeks. Vertical bars indicate the standard deviation of the measurements.

Discussion

The present study showed that, overall, the alignment between simultaneous NIR images and OCT scans of the Heidelberg Spectralis is excellent. The mean error in the alignment was 13.90 µm (horizontal) and 14.08 µm (vertical), and the mean maximum error was 15.35 µm. Interestingly, it increased significantly at week 1 compared to baseline (p=0.01), then varied randomly without significant changes during the 12-week follow-up period. During the first day after calibration, the mean error increased significantly after the second scanned patient compared to baseline (p<0.05), and it gradually rose over the day reaching a maximum after the eighth patient. The mean error was then stable for the remainder of the week. Despite the small average maximum error, the maximum error was greater than 20 µm in 13 out of the 72 total measurements of the 12-week period (18.1%), reaching a maximum of 34.78 µm. These results may have a significant impact on clinical evaluation of small retinal structures that are commonly imaged, such as drusen, microaneurysms, or focal abnormalities of the outer retinal layers. A small drusen is considered to have a diameter of <63 µm³, while a small microaneurysm may have a diameter of 50 µm or less.⁴ Therefore, in case of inaccurate alignment, a small drusen or microaneurysm seen on NIR image may not be accurately located on the OCT image, or vice versa. Figure 3 shows a vertical error of 85 µm between the NIR image and the OCT scan. This error was assessed manually using the internal caliper of the cSLO device.

Figure 3.

Example of vertical misalignment between near-infrared image (Left) and optical coherence tomography scan (Right) on Heidelberg Spectralis. An error of 85 µm was manually measured on the tomographic scan after positioning the caliper at the edge of a large retinal vessel (Magnified Square).

The current study showed that factors influencing the alignment are temperature variation and the number of scans performed per patient. Large increases in temperature of the area where the Heidelberg Spectralis is stored may cause some components to expand and, therefore, affect the alignment between the NIR image and the OCT image. Performing multiple similar OCT scans per patient, or shifting from a scan type to a different one (e.g. linear, circle, or raster), also increases the error in the alignment. This result may be explained with the longer time of use of the laser for a single patient in case of multiple OCT scans. While shifting from one scan to another, the operator must check and modify the settings of the scanning procedure (e.g. location of the scan and of the fixation target, number of B-scans, image averaging, imaging modalities such as enhanced depth imaging or combined depth imaging^{5, 6}) and also has to align the eye to the scanning laser. Such modifications may cause the alignment between simultaneous NIR images and OCT scans to be affected. However, we can’t exclude that other unknown variables may influence the final result of the alignment; this statement should be considered as a limitation of the study.

The Heidelberg Spectralis is a widely used SD-OCT device, in both clinical and study settings. Many investigators have used and are currently using this device in various original studies, as well as in randomized clinical trials. This device has been certified by many reading centers as a diagnostic tool to aid in baseline and follow-up multimodal evaluation of study patients. The simultaneous multimodal imaging feature helps, for example to evaluate absence of retinal damage after subthreshold laser photocoagulation⁷, to document microaneurysms outcome after navigated focal laser photocoagulation⁸, in the difficult diagnosis of occult maculopathies^9–11, and also to localize precisely transplanted stem cells in the subretinal space for different macular pathologies.¹² As a matter of fact, it is extremely important to ensure accurate alignment between simultaneous NIR image and OCT scans, otherwise these micrometric analyses of different retinal structures may not be reliable. With the arrival of eye-tracking technology in other commercial SD-OCT instruments, it might be necessary to assess the alignment accuracy between the two imaging modalities – fundus imaging and OCT scanning – of these devices. At the onset of this study, the other devices were not yet available.

In conclusion, the present study represents a collaboration between several sites that currently use the Heidelberg Spectralis for clinical care and research. The results of this study suggest that the alignment of the device does not need to be checked on a regular basis in the clinical setting. However, for research purposes, a re-calibration of the device should be performed every other patient, since an extreme precision of the scanning feature is necessary to obtain a completely reliable evaluation of micrometric retinal structures. To obtain the highest precision in the alignment between simultaneous NIR image and OCT scan, a stable temperature should be maintained in the area where the device is kept, and users should avoid acquiring unnecessary scans of patients. The results of this study may be useful for ophthalmologists and investigators who use the Heidelberg Spectralis with expectations of high-accuracy and best-quality multimodal assessment of posterior structures.