Prospective, Longitudinal Pilot Study

Abstract

Purpose

To evaluate longitudinally the performance of the Notal Vision Home OCT (NVHO), comprising a spectral-domain OCT device for patient self-imaging at home, telemedicine infrastructure for automated data upload, and deep learning algorithm for automated OCT evaluation. The aims were to study the system’s performance in daily image acquisition and automated analysis and to characterize the dynamics of retinal fluid exudation in neovascular age-related macular degeneration (nAMD).

Design

Pilot prospective, observational longitudinal study.

Participants

Four individuals (mean age, 73.8 years) with nAMD (one or both eyes) undergoing anti–vascular endothelial growth factor therapy in routine clinical practice.

Methods

The participants performed daily self-imaging at home with the NVHO for 1 month. The macular cube scans were uploaded automatically to the Notal Health Cloud. They underwent evaluation separately by the Notal OCT Analyzer (NOA) and human expert graders for fluid presence, segmentation, and volume.

Main Outcome Measures

Daily self-imaging completion, image quality, acquisition time, agreement between automated and human grading of retinal fluid, and temporal dynamics of fluid volume.

Results

Of 240 self-imaging attempts initiated, the number successfully completed was 211 (87.9%). Of these, 97.6% had satisfactory quality. For fluid presence, the NOA agreed with human grading in 94.7% of cases. From a subset of 24 scans with fluid, for agreement between NOA and human fluid volume measurements, the correlation coefficient was 0.996 and mean absolute difference was 1.5 nl (vs. 0.995 and 1.2 nl, respectively, for interhuman agreement). Graphic plots of fluid volume revealed wide variation in the dynamics of fluid exudation and treatment response.

Conclusions

The participants could perform daily self-imaging at home and generate macular cube scans of satisfactory quality. Automated quantitative OCT analysis achieved high agreement with human grading. Daily self-imaging with automated OCT analysis permitted detailed characterization of the dynamics of fluid exudation and revealed wide variation between eyes. Metrics describing these dynamics may become important disease biomarkers. Home OCT telemedicine systems represent an alternative paradigm of disease monitoring; they may allow highly personalized retreatment decisions, with fewer unnecessary injections and clinic visits.

Spectral-domain (SD) OCT is essential for informing retreatment decisions in patients with neovascular age-related macular degeneration (nAMD) and other exudative macular conditions.1, 2, 3, 4, 5, 6 However, the current paradigm of requiring patients to attend a retinal clinic in person for each OCT scan, with or without an anti–VEGF injection on the same day, has limitations. First, this arrangement places a substantial burden on patients, caregivers, and physicians in terms of time and expenditure because of frequent clinic visits.7, 8, 9, 10, 11 For patients and caregivers, this may contribute to the high rates of treatment discontinuation observed in routine clinical  practice.¹² Second, all of the individuals concerned may have increased risk of acquiring or transmitting severe acute respiratory syndrome coronavirus 2 or other transmissible infections, which is particularly relevant for the age group affected by nAMD.^13,14 Third, physicians are required to infer current and recent disease activity by evaluating an OCT scan at a single moment in time (with any comparison limited to other individual time points typically at least 1 month apart), usually without the assistance of automated software tools to help identify or quantify retinal fluid.¹⁵

The pharmacodynamics of intravitreous anti-VEGF therapy mean that the effects of an injection wear off rapidly over time in a way that is highly variable between eyes and, to some extent, drugs.¹⁶ Therefore, injections usually are administered according to 1 of 3 regimens: fixed frequency, pro re nata (PRN), or treat and extend. The aim of the PRN regimen, compared with the fixed-frequency regimen, is to decrease the number of potentially unnecessary injections, whereas the aim of the treat-and-extend regimen is to decrease the number of potentially unnecessary clinic visits.17, 18, 19, 20 However, all 3 approaches have important limitations. Both fixed-frequency and PRN regimens require a clinic visit every month (or 2 months in some fixed-frequency cases).¹⁹ Fixed-frequency treatment leads to unnecessary injections in many eyes, whereas PRN treatment sometimes has been associated with worse visual outcomes.^17,21,22 The treat-and-extend regimen also leads to unnecessary injections in many eyes (particularly in the first year) and can become unwieldy if both eyes have nAMD.^17,18,23 All 3 approaches share additional problems: difficulties around handling missed visits, distinguishing between nonresponders and short-term responders,²⁴ and assessing quiescence. Overall, all of the approaches may suffer from some compromise among suboptimal visual outcomes,^15,25,26 excess risk resulting from unnecessary injections and visits, and increased burden on patients, caregivers, and physicians.7, 8, 9, 10, 11

The possibility of SD OCT assessment in the home setting represents an alternative paradigm of disease monitoring, because this would permit very frequent, even daily, imaging. In addition to the safety, convenience, and flexibility of this approach, it would allow retreatment decisions truly to be personalized (using information such as retinal fluid type, quantity, temporal dynamics, and other parameters). Ideally, this paradigm would avoid both unnecessary injections and unnecessary clinic visits; clinic visits would occur only when genuinely required for retreatment or other clinical reasons. Additional advantages include very frequent monitoring of fellow eyes for conversion to nAMD and of eyes with nonexudative nAMD for conversion to exudative disease, easier assessments of quiescence and reactivation, and large quantities of data for use in clinical research. In addition, this approach may become even more attractive with the advent of longer-lasting anti-VEGF drugs and delivery systems, such as the Port Delivery System (Genentech, Inc).27, 28, 29 Owing to the quantity of imaging data generated, this approach would lend itself well to the OCT assessments being partially automated and quantitative, which has been an area of substantial progress in recent years.¹⁵

The Notal Vision Home OCT (NVHO) system (Notal Vision, Inc) represents an integrated system comprising (1) a home SD OCT device for individual patient self-imaging on a frequent basis (the Notal Home OCT^30,31), engineered so that it can be manufactured in large quantities at an appropriate cost and (2) a dedicated remote diagnostic clinic to support and monitor compliance in patients undergoing home monitoring, including (3) a deep learning-based algorithm for automated and quantitative evaluation of the NVHO scans (the Notal OCT Analyzer [NOA]; Lally DR, et al. Invest Ophthalmol Vis Sci. 61 [ARVO E-Abstract 2571], 2020) and (4) a telemedicine infrastructure to enable secure transmission and storage of the personal health information.

The aim of this pilot observational study was to recruit a small number of participants with nAMD and to study them longitudinally for 1 month with daily self-imaging at home using the NVHO system. Specifically, the objectives were to evaluate the performance of the NVHO system in daily image acquisition and automated analysis and to characterize the temporal dynamics of retinal fluid exudation.

Methods

The Home Retinal Imaging Using Notal Vision Home OCT study was a prospective, observational clinical study with a study period of 1 month. Institutional review board approval was obtained at each clinical site (Tel Aviv Medical Center, Tel Aviv, Israel, and Assuta Medical Center, Tel Aviv, Israel), and written informed consent for the research was obtained from all study participants. The research was conducted under the tenets of the Declaration of Helsinki. The study was registered at ClinicalTrials.gov (identifier, NCT04241198). The sponsor was Notal Vision Ltd.

Notal Vision Home OCT Device and Notal OCT Analyzer

The NVHO system is based on an SD OCT device designed for home use, so that individual patients can perform OCT self-imaging on a frequent basis. The device has been described previously.^30,31 In brief, it is a compact device designed for commercial use (Supplemental Fig 1 available at https://www.ophthalmologyscience.org/). The user can adjust the device height and select which eye will be scanned. User head positioning and visual fixation are guided by a novel proprietary automatic feedback system, which gives prompts to ensure correct head positioning and asks users to look at a blinking fixation target during scan acquisition (Supplemental Figs 2 and 3 available at https://www.ophthalmologyscience.org/). During a user’s first interaction with the device, it performs a one-off automatic calibration procedure to personalize imaging to the user’s refractive error and axial length. The NVHO scan comprises a horizontal raster of 88 B-scans across an area of 3 × 3 mm (10° × 10° field of view), centered at the eye’s point of fixation. The specifications include a central wavelength of 830 nm, scan speed of 10 000 A-scans per second, and 500 A-scans per B-scan. At the end of each self-imaging session, the data are transmitted automatically to the Notal Health Cloud via an inbuilt cellular modem. From the raw data, the cube scans are reconstructed and available for remote review by a physician or other qualified health care professional in a web-based viewer. The cube scans also are analyzed by the NOA, and the results are added to the longitudinal report for that user.

The NOA is an artificial intelligence-based software application that performs fully automated detection and quantification of retinal fluid (both intraretinal fluid [IRF] and subretinal fluid [SRF]) from NVHO macular cube scans. In addition, it generates (1) annotated B-scans, with areas of retinal fluid shown color-coded according to fluid type; (2) en face maps of fluid thickness (separately by IRF and SRF); and (3) ranking of B-scans by order of largest to smallest fluid area (Fig 1). The development and validation of the NOA software for analysis of data from other OCT models have been described previously.^15,32, 33, 34 The validation of the NOA software version for analysis of the NVHO output also has been described previously (Lally DR, et al. Invest Ophthalmol Vis Sci. 61 [ARVO E-Abstract 2571], 2020)³⁵ and was examined again in this study.

Figure 1.

Comparison of retinal fluid detection by the Notal OCT Analyzer (NOA) versus human expert grading: retinal fluid thickness maps with ETDRS grid overlaid (top) and representative B-scans (bottom): (A) intraretinal fluid (IRF) and (B) subretinal fluid (SRF). Notal OCT Analyzer and human gradings of retinal fluid thickness are shown spatially on the thickness maps, with black representing no or minimal fluid thickness and yellow and white representing severe fluid thickness. On the B-scan pairs (with the macular location shown by the blue line on the thickness maps), the left B-scan is the original and the right B-scan is overlaid with the retinal fluid segmentation (shown in red for intraretinal fluid and orange for subretinal fluid).

The NVHO device, like other commercial OCT devices, automatically calculates a manufacturer signal quality index (MSI). This is designed to provide the reviewing physician with an objective, quantitative indication of image quality for clinical interpretation. In brief, an MSI value for each B-scan is derived from the OCT signal intensity and noise characteristics, and the MSI value for the entire cube scan is calculated as the mean MSI of all constituent B-scans. The index ranges from 0 (no visible signal) to 7 (highest quality signal).

Study Population

The study recruited patients with age-related macular degeneration from 2 retinal clinics, the Tel Aviv Medical Center and the Assuta Medical Center. The inclusion criteria comprised: ability and willingness to give informed consent, study eye(s) diagnosed with age-related macular degeneration, study eye(s) with Snellen visual acuity of 20/320 or better, at least one eye with active exudative nAMD undergoing anti-VEGF treatment, and ability to undergo OCT imaging.

Study Procedures

For eligible patients, the following procedures were performed at a screening visit in the retinal clinic. Patient demographic and clinical data were recorded, including age, sex, best-corrected visual acuity, biomicroscopy findings, and diagnosis in the study eye(s), particularly the presence or absence of nAMD, as well as the presence or absence of media opacity that might preclude OCT imaging. Macular SD OCT scans were acquired from both eyes using the Spectralis device (Heidelberg Engineering), comprising a 6 × 6-mm cube (with at least 49 B-scans, standard horizontal orientation, covering a 20 × 20° area), and fluid status at enrollment (as determined by the investigator) was recorded.

For each eligible participant, an NVHO device and a printed setup guide were delivered to the individual’s home. A tutorial video for mandatory review on the device’s external screen explains how to operate the device and perform initial calibration. The participants also were able to call a remote support service for assistance during the installation and training process and throughout the study period. The participants were asked to perform self-imaging using the device on each study eye daily for 1 month. The imaging data were uploaded automatically to the Notal Health Cloud. At the end of the study, the participants were asked to attend a study exit visit at the retinal clinic. Patient clinical data were recorded, including best-corrected visual acuity and biomicroscopy findings, and macular SD OCT cube scans were acquired again using a commercial device.

Because this was an observational study, the participants continued under normal standard-of-care management by their retinal specialist during the study period. Study eyes were able to receive anti-VEGF therapy or other treatment as clinically indicated during the study. The observational data from the home OCT device were not used to inform clinical care and were not provided to the treating retinal specialist.

Home OCT Grading for Retinal Fluid Presence and Severity by the Notal OCT Analyzer and by Human Expert Graders

For each study eye on each study day, the number of home OCT scans successfully completed was recorded. For each scan, the duration of the self-imaging acquisition and the MSI were recorded. The home OCT scans underwent evaluation separately by the NOA and human expert graders. For each eligible scan, the NOA performed automated segmentation and calculation of the following metrics: (1) IRF volume, (2) SRF volume, and (3) total retinal fluid volume (i.e., sum of IRF and SRF). The NOA used these data to create graphs of fluid volume over time. In addition, the NOA generated (1) annotated B-scans, with areas of retinal fluid shown color-coded according to fluid type; (2) en face maps of fluid thickness (separately by IRF and SRF); and (3) ranking of B-scans by order of greatest to smallest fluid area.

Each home OCT cube scan was evaluated independently by 2 human expert graders, who recorded the presence or absence of IRF and SRF in each scan; cases of disagreement between the 2 expert graders were adjudicated by a third expert grader. These grades provided the ground truth labels for retinal fluid presence or absence for comparison with NOA automated grading. Of the home OCT scans with retinal fluid present, a subset of 24 cube scans was chosen randomly such that each study eye contributed 3 scans. For these 24 cube scans, each B-scan was evaluated independently by 2 human graders experienced in manual segmentation (different from the graders mentioned above). The 2 graders performed manual segmentation separately for IRF and SRF on each B-scan for all 24 cube scans. Fluid volume measurements for each grader were calculated using spline interpolation over the segmented fluid areas from all B-scans for each cube scan. The mean of the 2 volume measurements (1 from each grader) provided the ground truth labels for retinal fluid volumes for comparison with NOA automated grading.

Outcome Measures and Statistical Analysis

Several outcome measures were used. The first was daily self-imaging completion, that is, the proportion of OCT scans successfully performed, transmitted, and received by the Notal Health Cloud, from a maximum of 1 scan per study eye per day during the study period. Alongside this, the time taken for self-imaging, that is, the mean OCT scan acquisition time, and the MSI data were analyzed.

The second outcome measure was the level of agreement between the NOA and the human expert grading for the presence or absence of retinal fluid on the home OCT scans. Agreement was calculated as the proportion of OCT scans with the same fluid determination by the NOA and the human grader. In addition, for eyes whose fluid status changed during the study period (i.e., from absent to present or from present to absent, according to human grading), the time to fluid status change was compared between the NOA and human grading.

The third outcome measure was the level of agreement between the NOA and the human expert grading for the segmentation of retinal fluid on the subset of 24 scans. Agreement was assessed by the Dice coefficient (at the level of pixels). Related to this, the fourth outcome measure was the level of agreement between the NOA and the human expert grading for the retinal fluid volume measurements on the same subset of 24 scans. Agreement was assessed by the Pearson correlation coefficient and mean absolute differences. Intragroup agreement (i.e., between the 2 human graders) and intergroup agreement (i.e., between the NOA and the human graders) were compared numerically.

The fifth outcome measure was evaluation of the temporal dynamics of retinal fluid exudation during the study period. The retinal fluid volumes computed by the NOA were plotted graphically over time for each study eye. Multiple quantitative metrics to describe fluid dynamics (i.e., relating to the severity and timing of fluid presence) were developed and applied to the plots of fluid volume over time. These included metrics related to fluid volume (in nanoliters), interval (in days), cumulative retinal fluid load (considered as area under the receiver operating characteristic curve [AUC] in nanoliter-days), and slopes (change in nanoliters/day). These metrics can refer to 1 cycle of treatment (i.e., the period between 2 consecutive anti-VEGF injections) or to any desired period. For example, time with retinal fluid present could be expressed as 60 days with retinal fluid present or 45 days with IRF present during 1 year of anti-VEGF treatment. Similarly, the AUC, which has the advantage of weighting days with fluid present according to the daily fluid volume, could be expressed as 4000 nl-days of fluid presence or 2000 nl-days of SRF presence during 1 year (either overall or spatially by macular region using cumulative fluid thickness maps). In addition, each of the metrics can be applied to total retinal fluid, IRF only, or SRF only.

Statistical analysis was performed using Matlab software (MathWorks).

Results

Study Population

Of 6 patients enrolled at the retinal clinic for potential participation in the study, 4 met the criteria for beginning study participation at home. Of the other 2 patients, 1 contracted COVID-19 before receiving the NVHO device and decided not to proceed with the study, and the other received the device, but did not complete device calibration and decided not to proceed. Hence, the study population comprised 8 eyes of 4 participants. The characteristics of the study participants and eyes are shown in Table 1. Mean age was 73.8 years (range, 69–80 years). Two of the participants had nAMD in both eyes and the other 2 participants had nAMD in 1 eye and intermediate age-related macular degeneration in the fellow eye.

Table 1.

Characteristics of the Study Participants and Eyes

Participant No.	Age at Baseline (yrs)	Sex	Eye	Snellen Best-corrected Visual Acuity at Baseline	Age-Related Macular Degeneration Status	Number of Anti–VEGF Injections before Study Period	Time since Last Anti–VEGF Injection before Baseline (days)	Anti–VEGF drug used during study period
1	80	Male	Right	20/40	Neovascular	>10	18	Aflibercept
			Left	20/40	Intermediate	N/A	N/A	N/A
2	69	Female	Right	20/25	Neovascular	5	61	Aflibercept
			Left	20/40	Neovascular	8	46	Aflibercept
3	76	Female	Right	20/20	Neovascular	>10	6	N/A
			Left	20/133	Neovascular	>10	41	N/A
4	70	Female	Right	20/40	Intermediate	N/A	N/A	N/A
			Left	20/40	Neovascular	9	10	N/A
5	85	Male	Right	20/25	Neovascular	30	43	Ranibizumab
			Left	20/40	Neovascular	5	29	Ranibizumab

N/A = not applicable; VEGF = vascular endothelial growth factor.

In addition, after the end of the 1-month study period for all 4 participants, 1 additional participant with nAMD in both eyes (Table 1) was enrolled for a longer study period of 3 months. The data of this additional participant were prespecified to contribute to analyses of the fifth outcome measure only (i.e., temporal dynamics of retinal fluid), not to those of the other 4 outcomes.

Daily Completion of Self-Imaging

For the outcome measure of daily self-imaging completion, the results are shown in Table 2. All 8 eyes completed study participation to the prespecified duration of 1 month (specifically, 29 days). During this period, of a possible maximum of 232 days of self-imaging (i.e., all 8 eyes with at least 1 scan completed per day for 29 days), the total number of days with scans acquired successfully was 198 (85.3%). Per eye, of 29 days, this corresponded to a mean of 24.8 days (standard deviation, 3.8 days).

Table 2.

Results: Outcome Measure of Daily Self-Imaging Scan Completion

Participant No.	Eye	Study Period (days)	Days with at Least 1 Completed OCT Scan	Total No. of Scans Completed	Imaging Duration per Scan (sec)	Manufacturer Signal Quality Index∗	Scans with Manufacturer Signal Quality Index of ≥2∗
1	Right	29	21 (70)	24	38.0 ± 8.2	3.0 ± 0.6	23 (96)
	Left	29	23 (76)	26	39.3 ± 10.3	3.5 ± 1.0	23 (88)
2	Right	29	18 (66)	18	51.9 ± 20.9	2.9 ± 0.3	18 (100)
	Left	29	27 (90)	28	45.5 ± 19.8	3.5 ± 0.5	28 (100)
3	Right	29	25 (83)	28	41.3 ± 18.7	4.7 ± 1.1	27 (96)
	Left	29	27 (90)	30	44.3 ± 15.1	5.7 ± 1.1	30 (100)
4	Right	29	28 (96)	28	35.1 ± 8.1	6.6 ± 0.4	28 (100)
	Left	29	29 (97)	29	33.6 ± 4.9	5.7 ± 0.4	29 (100)
Mean		29	24.8 (83.5)	26.4	41.1	4.5	27 (97.6)

Data are presented as no. (%) or mean ± standard deviation, unless otherwise indicated.

^∗Range: 0 = no visible signal to 7 = highest quality signal; MSI ≥ 2 is the recommended threshold for satisfactory image quality.

The total number of scans successfully acquired was greater than this since, on some study days, some participants performed self-imaging more than once. Specifically, the participants attempted self-imaging 240 times and successfully completed self-imaging 211 times (87.9%). Hence, the total number of scans successfully acquired during the study period was 211. Per eye, this corresponded to a mean of 26.4 complete scans (standard deviation, 3.9). The mean self-image acquisition time was 41.1 seconds (standard deviation, 13.4 seconds). The mean MSI was 4.5 (standard deviation, 1.3); 97.6% of the scans showed an MSI of 2 or more (the recommended threshold for satisfactory image quality).

Agreement between the Notal OCT Analyzer versus Manual Grading for the Presence of Retinal Fluid

For the analysis of retinal fluid presence, 211 cube scans were eligible. Of these, regarding the ground truth human grading, 11 (5.2%) required adjudication for the presence of retinal fluid; an additional 18 (8.5%) required adjudication only for the fluid type(s) present. Of the 211 scans, the number with retinal fluid present was 93. Of these 93 scans, 49 showed IRF present and 44 showed SRF present (with no scans containing both IRF and SRF). Of the 211 scans, the proportion for which the NOA grading agreed with the human grading was 94.7%. The equivalent values for IRF and SRF were 97.6% and 93.3%, respectively. Of the 93 scans with retinal fluid present, 93.5% were graded by the NOA as having fluid present. Of the 118 scans with no retinal fluid, 95.7% were graded by the NOA as having no fluid.

Five eyes demonstrated a fluid status change during the study period; of these, 4 showed a change from absence to presence and the other 1 eye showed a change from presence to absence. The time differences between the NOA and the human grading recording a status change were 0 days in 2 eyes and –3 days (i.e., NOA ahead of grader), –2 days, and +3 days (i.e., grader ahead of NOA) in each of 3 eyes.

Agreement between the Notal OCT Analyzer versus Manual Grading for the Segmentation of Retinal Fluid on B-Scans

For the analysis of retinal fluid segmentation, 2024 B-scans were eligible (with each study eye contributing 3 cube scans). Of the 2024 B-scans, the number with retinal fluid present (according to human grading) was 434; the equivalent numbers for IRF and SRF were 109 and 338, respectively. For the segmentation of all retinal fluid (i.e., IRF and SRF considered together), the median Dice coefficients for agreement between the NOA segmentation and human segmentation (i.e., intergroup agreement) were 72.6% (NOA vs. human grader 1) and 73.5% (NOA vs. human grader 2; Table 3; Fig 2). The equivalent value for agreement between human graders 1 and 2 (i.e., intragroup agreement) was 71.2%. For IRF, the equivalent median values were 75.9%, 77.8%, and 72.4%, respectively. For SRF, the equivalent median values were 70.3%, 70.9%, and 70.6%, respectively.

Table 3.

Agreement between the Notal OCT Analyzer and 2 Human Expert Graders for the Segmentation of Retinal Fluid at the Pixel Level on B-Scans

Grader	Median Dice Coefficient
	Total Retinal Fluid (%)			Intraretinal Fluid (%)			Subretinal Fluid (%)
	Grader 1	Grader 2	Notal OCT Analyzer	Grader 1	Grader 2	Notal OCT Analyzer	Grader 1	Grader 2	Notal OCT Analyzer
Grader 1	—	71.2	72.6	—	72.4	75.9	—	70.6	70.3
Grader 2	71.2	—	73.5	72.4	—	77.8	70.6	—	70.9
Notal OCT Analyzer	72.6	73.5	—	75.9	77.8	—	70.3	70.9	—

— = not applicable.

Figure 2.

Graphs showing the results of agreement between the Notal OCT Analyzer (NOA) and human expert grading and between the 2 human expert graders for the segmentation of retinal fluid at the pixel level: Dice coefficient, separately for (A) total retinal fluid, (B) intraretinal fluid, and (C) subretinal fluid.

Notal OCT Analyzer versus Manual Grading for the Assessment of Retinal Fluid Volumes

For the analysis of retinal fluid volumes, the same 24 cube scans were analyzed. The median total retinal fluid volume (according to human grader 1) was 4.7 nl (interquartile range, 1.3–11.0); the equivalent volumes for IRF and SRF were 0.0 nl (interquartile range, 0.0–0.8) and 1.8 nl (interquartile range, 0.1–7.5), respectively. For total retinal fluid, the Pearson correlation coefficient for agreement between the volumes assessed by the NOA and by human grading was 0.996 (Table 4; Fig 3). The correlation coefficient for agreement between the 2 human graders was 0.995. For IRF, the equivalent values were 0.978 and 0.986, respectively. For SRF, the equivalent values were 0.997 and 0.997, respectively. In Table 4, the correlation coefficients also are provided for pairwise comparison between all 3 graders (i.e., the NOA and each of the 2 human graders).

Table 4.

Agreement between the Notal OCT Analyzer and 2 Human Expert Graders for the Assessment of Retinal Fluid Volume

Grader	Pearson’s r Value
	Total Retinal Fluid			Intraretinal Fluid			Subretinal Fluid
	Grader 1	Grader 2	Notal OCT Analyzer	Grader 1	Grader 2	Notal OCT Analyzer	Grader 1	Grader 2	Notal OCT Analyzer
Grader 1	—	0.995	0.996	—	0.986	0.967	—	0.997	0.997
Grader 2	0.995	—	0.993	0.986	—	0.984	0.997	—	0.997
NOA	0.996	0.993	—	0.967	0.984	—	0.997	0.997	—

— = not applicable.

Figure 3.

Graph showing the results of agreement between the Notal OCT Analyzer (NOA) and human expert grading (defined as the mean value from 2 graders) for retinal fluid volume measurements (n = 24; R = 0.996).

For total retinal fluid, the mean absolute difference for agreement between the volumes assessed by the NOA and by human grading (i.e., the mean of the 2 graders) was 1.5 nl (standard deviation, 2.1 nl). The mean absolute difference for agreement between the 2 human graders was 1.2 nl (standard deviation, 1.5 nl). For IRF, the equivalent values were 0.3 nl (standard deviation, 0.7 nl) and 0.3 nl (standard deviation, 0.7 nl), respectively. For SRF, the equivalent values were 1.2 nl (standard deviation, 2.1 nl) and 0.9 nl (standard deviation, 1.4 nl), respectively.

Temporal Dynamics of Retinal Fluid

The temporal dynamics of retinal fluid for representative study eyes are shown in Figures 4 and 5, Supplemental Figures 4 and 5, (available at https://www.ophthalmologyscience.org/) and, in the form of video files (Videos 1-10, available at https://www.ophthalmologyscience.org/).

Figure 4.

Retinal fluid volume-time curves for participant 2's left eye. A, Intraretinal fluid (IRF) and subretinal fluid (SRF) volume-time curves from Notal OCT Analyzer segmentation of daily home OCT self-images. The eye received intravitreal anti–vascular endothelial growth factor injections of aflibercept 7 weeks before the beginning of the study period and on day 10 of the study period. No SRF was detected during the first week, followed by the appearance of SRF that increased slowly in quantity over 2 days, from 0 to 3 nl, then increased rapidly over 1 day to 48 nl. After the injection, the SRF volume continued to increase for 1 day (to 137 nl) before leveling off and declining gradually to complete resolution over the ensuing 7 days. Over the next 11 days and until the end of the study period, SRF remained absent. No IRF was identified throughout the study period. B, Representative example of the cumulative SRF load metric represented by area under the receiver operating characteristic curve (AUC). The AUC values (in nanoliter-days) are shown for different phases of the peri-injection period: 28.4 nl-days before the injection and 504.9 nl-days after the injection, with a total AUC of 533.3 nl-days during the 12-day period. Cumulative fluid thickness map during the same period, illustrating spatially the macula’s fluid exposure in nanoliter-days over 12 days (with Early Treatment Diabetic Retinopathy Study grid overlaid).

Figure 5.

Retinal fluid volume-time curves for study participant 5. A, Right eye: subretinal fluid (SRF) volume-time curve from Notal OCT Analyzer (NOA) segmentation of daily home OCT self-images. The eye received an intravitreal anti–VEGF injection of ranibizumab 6 weeks before the beginning of the study period. No SRF was identified during the first 3 weeks of the study period. This was followed by the appearance of SRF that increased in volume over 2 weeks, from 0 to 36 nl, first slowly and then more rapidly. After an intravitreal anti-VEGF injection of ranibizumab at this point, the volume of SRF decreased rapidly to 0 over 3 days. The SRF remained absent over the next 5 weeks. Again, this was followed by the appearance of SRF that increased in volume over 5 days, from 0 to 6 nl. As before, after an intravitreal anti-VEGF injection of ranibizumab, the SRF volume decreased rapidly to 0 over 1 day and remained absent until the end of the study period. Also shown are the NOA SRF segmentation (orange) and the NOA SRF thickness maps from 2 time points, at the time of each anti-VEGF injection, when SRF volume was greatest. B, Left eye: SRF volume-time curve from NOA segmentation of daily home OCT self-images. The eye received an intravitreal anti-VEGF injection of bevacizumab 4 weeks before the beginning of the study period. The SRF was present at a volume of approximately 50 nl during the first 5 days of the study period. After an intravitreal anti-VEGF injection of ranibizumab at this point, the SRF volume decreased rapidly over 3 days to a minimum of 11 nl. The volume of SRF increased slowly and gradually over the next 25 days, from 11 to approximately 32 nl. Again, after an intravitreal anti-VEGF injection of ranibizumab, the SRF volume decreased rapidly over 4 days to a minimum of 11 nl. The volume of SRF increased very slowly and gradually over the next 5 weeks, much more slowly than before, from 11 to approximately 20 nl. After another intravitreal anti-VEGF injection of ranibizumab, the SRF volume decreased rapidly over 4 days, this time to the point of complete resolution, before the end of the study period. Also shown are the NOA SRF segmentation (orange) and the NOA SRF thickness maps from 2 time points, at the time of each anti-VEGF injection. C, Comparison of retinal fluid volume-time curves for the right eye (OD) and left eye (OS). Representative examples of the cumulative retinal fluid load metric represented by area under the receiver operating characteristic curve (AUC; shown in blue). Despite relatively similar fluid volumes at the individual time points of anti-VEGF injection days, the AUC of cumulative retinal fluid load in the intervening period between 2 retinal clinic visits differed very substantially between the 2 eyes of the same participant (53.7 nl-days vs. 812.5 nl-days). Without frequent home OCT imaging, this large difference would remain hidden and might account for differential visual and anatomic outcomes over time.

Discussion

Main Findings and Implications

In this prospective, longitudinal study, all 4 participants who began self-imaging successfully completed the study period. The self-imaging completion rate was 85%, with the other 15% representing an occasional missed day for most participants, rather than any participants with large missing periods. A very high proportion of the scans showed satisfactory image quality.

An important test of the integrated system was the level of agreement between the NOA and human expert grading. Agreement on the presence of any retinal fluid was high; it was very high at 97% for IRF, which is considered particularly important for visual outcomes and re-treatment decisions.^15,36 When disagreement was present, it related to very small quantities of fluid, as observed in a previous study of the NOA.³² The level of agreement also was high for the volume measurements (informed by the segmentation of fluid), which represent the outcome of particular interest. The results in Figure 3 suggest that the NOA reports accurate volume measurements across a wide range of fluid conditions. However, these observations are tentative, because the number of eyes was small, although the number of OCT scans was relatively large. Agreement on the segmentation of fluid itself, at the pixel level, was lower, but still relatively high. Importantly, this was true for both NOA–human and human–human agreement, which demonstrates that this dataset was challenging for the segmentation task. In particular, the quantities of fluid were very small for most eyes (Fig 3). Pixels at fluid–tissue interfaces are the most likely ones to generate disagreement between graders. In scans with small fluid volumes, the proportion of pixels at fluid–tissue interfaces is much greater; in scans with large fluid volumes, the proportion of these pixels is much lower. Hence, higher Dice coefficients would be expected in a dataset with larger fluid volumes. Overall, a comparison of NOA–human and human–human agreement provides the most important understanding of automated performance. In this study, for both segmentation and quantification, these were extremely similar, which meant that the NOA results agreed with those of human experts to the same level that the human experts agreed with each other.

The graphical plots of fluid volume over time revealed various key pathophysiologic events in some eyes (Figs 4 and 5). In routine clinical care, these events typically must be inferred, rather than observed directly. For example, in several eyes (e.g., participant 3's right eye and participant 4's left eye), daily self-imaging captured the sequence of fluid absence followed by recurrent exudation. Other eyes (e.g., participant 2's right and left eyes and participant 5's right eye) demonstrated recurrent exudation followed by fluid resolution after an injection. In 1 study eye (participant 5's right eye; Fig 5), over a sequence of 3 injections, fluid resolution was incomplete after the first 2 injections, but was complete after the third injection. One study eye (participant 3's left eye; Supplemental Fig 5 available at https://www.ophthalmologyscience.org/) showed atrophic degenerative inner retinal cysts, with small and stable fluid volumes. The combination of daily self-imaging and volumetric assessment may assist in making the distinction between exudative and nonexudative pathologic features and therefore may support treatment decisions.¹⁵ Overall, daily self-imaging combined with automated quantitative analysis of the scans provided highly granular data that permitted very detailed characterization of the temporal dynamics of retinal fluid exudation and treatment response. The fluid volume signatures observed in the peri-injection period have characteristics that lend themselves well to mathematical analysis (Fig 4B).

Heterogeneity in the Dynamics of Retinal Fluid Exudation and Treatment Response

Daily home OCT imaging revealed a high degree of heterogeneity in terms of the dynamics of fluid exudation and resolution. For example, even in this small sample, the interval between anti-VEGF injection and recurrent exudation varied from 3 to 9 weeks. This reveals what is assumed for many eyes receiving a treat-and-extend regimen, but must typically be inferred from single time points of OCT imaging. Substantial variation in time to reactivation was observed even within individual eyes; for example, participant 5's right eye showed recurrent exudation at 9 weeks after injection in 1 treatment cycle, but at 5 weeks in the next treatment cycle (Fig 5). If replicated in a larger dataset, observations like this would demonstrate the difficulties of defining a re-treatment interval without repeat OCT imaging in the interim, as in the treat-and-extend approach.

Substantial variation also was observed for the slopes of fluid accumulation over time. Some eyes (e.g., participant 3's right eye and participant 5's left eye) demonstrated relatively monotonic increases in retinal fluid. However, more eyes (e.g., participant 1's right eye, participant 2's left eye, and participant 5's right eye) experienced nonlinear increases in retinal fluid volume, where slower increases were followed by faster increases.

Some variation was seen in the time taken for retinal fluid to resolve after an injection. In most cases (e.g., participant 2's right eye and participant 5's right and left eyes), it took 3 to 4 days for retinal fluid to resolve. However, for one of the same eyes (participant 5's right eye) in the next treatment cycle, a smaller volume of fluid took only 1 day to resolve (Fig 5). For another eye (participant 2's left eye) with a relatively large volume of fluid, the fluid took 9 days to resolve (including a maximum fluid volume actually reached 2 days after injection; Fig 4). Indeed, the AUC of cumulative retinal fluid load was particularly high for this eye, given the combination of high volume and long duration (Fig 4B). These characteristics remain hidden during routine clinical care with OCT images obtained several weeks apart. For comparison, in a recent study of 48 eyes with nAMD based on weekly SD OCT imaging for 4 weeks, the time after injection to maximum central retinal thickness reduction varied widely, between 1 and 4 weeks, but was most common at 3 and 4 weeks.²⁴ Our study showed that the AUC of cumulative retinal fluid load during 1 treatment cycle can differ very markedly between 2 eyes of the same patient, despite very similar OCT scans at clinic visits, as shown for participants 2 and 5 (Fig 5). Together with other characteristics of fluid exudation, differences in the cumulative fluid load in different retinal compartments may help to account for some unexplained variation in visual outcomes between eyes with similar OCT scans at clinic visits, and so could provide an important novel biomarker for disease activity, treatment response, and treatment outcome prediction.

Comparison with Literature

We are not aware of other prospective longitudinal studies of daily home OCT imaging, although other home OCT devices have been reported in the literature.37, 38, 39 The self-examination low-cost full-field OCT device (Visotec GmbH) is designed for use in a home setting.³⁷ Unlike the NVHO, it is based on time-domain technology and acquires en face images sequentially at different depths. The prototype device has an imaging field of 4.5 × 1.4 mm, an axial resolution of 12 μm, and a horizontal resolution of approximately 17 μm. It suffers from more artifact and a worse signal-to-noise ratio than typical commercial SD OCT devices.^37,39 In a pilot cross-sectional study, authors reported initial experiences with this device.³⁷ Patients did not use the device to self-image unsupervised in the home environment; rather, the imaging was performed in a laboratory setting. Of the 51 patients, 94% were able to acquire a retinal image and 76% acquired scans that met the authors’ quality scoring criteria. In the scans of eyes with nAMD, the binary presence of features such as IRF and SRF could be detected, although IRF is more difficult to detect with this technology.

A sparse OCT device also has been developed for potential use in the home setting, with the prototype device named MIMO_02.³⁸ The scanning density is lower than that of typical commercial SD OCT devices. The imaging field is 3.8 × 3.8 mm, with a resolution of between 50 × 50 pixels and 150 × 150 pixels and a depth resolution of 2048 pixels over 4.2 mm. A cross-sectional feasibility study using the prototype device was reported.³⁷ Again, this was not a self-imaging study in the home environment; rather, the imaging was performed in a laboratory setting and an engineer operated the device for some steps. The study included comparison of the accuracy of central retinal thickness measurements between the prototype device and a commercial SD OCT device (Spectralis), but the potential detection of other important biomarkers such as IRF and SRF was not presented.

Strengths, Limitations, and Future Directions

The strengths of this study include its prospective longitudinal design, with daily self-imaging taking place in the home environment. We are not aware of previous studies reporting data obtained from an integrated system comprising a home OCT device, telemedicine infrastructure enabling transmission to a cloud-based system, and a deep learning algorithm for automated evaluation. Additional strengths include comparison with human expert graders on multiple levels and the wide range of outcome measures.

As a pilot study, this report is limited by the small number of participants and by its single-country setting; hence, relatively limited statistical analysis was possible. Any observations relating to general acceptability of the approach and heterogeneity of fluid exudation between eyes are limited by the small number of eyes. Because this was an observational study, where the data were not used to inform clinical care, important future directions include integrating this system into the management of exudative retinal diseases. A trial comparing visual and patient-reported outcomes of home OCT-assisted versus standard-of-care treatment would provide very valuable information.

In conclusion, this pilot prospective study recruited individuals with nAMD who were followed up longitudinally with daily self-imaging at home. The participants generally were able to perform daily self-imaging, with almost all scans of satisfactory quality. The integrated system showed high agreement with human expert grading for retinal fluid presence and quantity. Daily self-imaging provided highly granular temporal data, which permitted very detailed characterization of the dynamics of retinal fluid exudation and resolution. Overall, a high degree of heterogeneity was observed between eyes for signatures of both fluid exudation and treatment response. Multiple metrics to describe these temporal features may become important biomarkers in future clinical care and research. Home OCT systems may enable alternative paradigms of disease management. Home monitoring would allow retreatment decisions to be truly personalized. Such paradigms could avoid both unnecessary injections and unnecessary clinic visits. These considerations are crucial in the context of a global pandemic and will become more important with the advent of longer-acting anti-VEGF drugs and drug-delivery systems. Larger prospective trials are required to determine whether home OCT systems can contribute to improved visual outcomes and quality of life.

Manuscript no. D-21-00081.

Supplementary Data

Video 1

VidClip of participant 1, right eye.

Video 2

VidClip of participant 1, left eye.

Video 3

VidClip of participant 2, right eye.

Video 4

VidClip of participant 2, left eye.

Video 5

VidClip of participant 3, right eye.

Video 6

VidClip of participant 3, left eye.

Video 7

VidClip of participant 4, right eye.

Video 8

VidClip of participant 4, left eye.

Video 9

VidClip of participant 5, right eye.

Video 10

VidClip of participant 5, left eye.