Skip to main content
PMC home page
Neurology® Neuroimmunology & Neuroinflammation logoLink to Neurology® Neuroimmunology & Neuroinflammation
. 2018 Mar 13;5(3):e449. doi: 10.1212/NXI.0000000000000449

Multicenter reliability of semiautomatic retinal layer segmentation using OCT

Timm Oberwahrenbrock 1,*, Ghislaine L Traber 1,*, Sebastian Lukas 1, Iñigo Gabilondo 1, Rachel Nolan 1, Christopher Songster 1, Lisanne Balk 1, Axel Petzold 1, Friedemann Paul 1, Pablo Villoslada 1, Alexander U Brandt 1, Ari J Green 1,**, Sven Schippling 1,**,✉
PMCID: PMC5852947  PMID: 29552598

Abstract

Objective

To evaluate the inter-rater reliability of semiautomated segmentation of spectral domain optical coherence tomography (OCT) macular volume scans.

Methods

Macular OCT volume scans of left eyes from 17 subjects (8 patients with MS and 9 healthy controls) were automatically segmented by Heidelberg Eye Explorer (v1.9.3.0) beta-software (Spectralis Viewing Module v6.0.0.7), followed by manual correction by 5 experienced operators from 5 different academic centers. The mean thicknesses within a 6-mm area around the fovea were computed for the retinal nerve fiber layer, ganglion cell layer (GCL), inner plexiform layer (IPL), inner nuclear layer, outer plexiform layer (OPL), and outer nuclear layer (ONL). Intraclass correlation coefficients (ICCs) were calculated for mean layer thickness values. Spatial distribution of ICC values for the segmented volume scans was investigated using heat maps.

Results

Agreement between raters was good (ICC > 0.84) for all retinal layers, particularly inner retinal layers showed excellent agreement across raters (ICC > 0.96). Spatial distribution of ICC showed highest values in the perimacular area, whereas the ICCs were poorer for the foveola and the more peripheral macular area. The automated segmentation of the OPL and ONL required the most correction and showed the least agreement, whereas differences were less prominent for the remaining layers.

Conclusions

Automated segmentation with manual correction of macular OCT scans is highly reliable when performed by experienced raters and can thus be applied in multicenter settings. Reliability can be improved by restricting analysis to the perimacular area and compound segmentation of GCL and IPL.

Optical coherence tomography (OCT) can be used to demonstrate retinal atrophy in patients with MS.1–3 The technique has become a valuable outcome measure in neuroprotective clinical trials.4,5 There is pathologic evidence of significant retinal injury in MS,6 which was confirmed by OCT7,8 including demonstration of progressive retinal neurodegeneration.3,9 Macular volume scans allow quantification of retinal layers such as the ganglion cell layer (GCL) and inner nuclear layer (INL). These may be predictive of functional or clinical outcomes, such as visual acuity in the case of GCL,10,11 or relapse rate and overall disability.12,13

Although further longitudinal investigations are needed, OCT is already used in clinical trials. The capacity to assess the effects of investigational compounds on respective retinal layers could enhance their diagnostic and prognostic value.4 This underscores the necessity for validated segmentation algorithms, allowing reliable postprocessing in international, multicenter clinical trial settings. Currently, fully manual, semiautomatic, and fully automatic intraretinal segmentation are used in OCT studies. Fully automated and manual segmentation studies have been performed on spectral domain (SD) OCT platforms such as the Heidelberg Engineering Spectralis and Carl Zeiss Meditec Cirrus machines, but all with a limited number of raters and in single centers.14 Although these studies have suggested reasonable within-patient reproducibility and general agreement between devices, reliability across centers and raters remained unclear.

Here, we tested the multicenter inter-rater reliability of semiautomatic intraretinal segmentation comparing 5 independent raters from 5 academic OCT centers in Europe and the United States.

Methods

Participants

The centers in Barcelona, Berlin, and San Francisco each provided OCT data sets from 3 patients with MS and 3 age- (maximal difference of 3 years) and sex-matched healthy controls from their center's research database in March 2013. Data were anonymously shared between the centers using the devices built-in export function. Demographic data of participants were not retained in this process.

Standard protocol approvals, registrations, and patient consents

All data were originally obtained from studies approved by the respective local ethics committees, and all participants provided written informed consent according to the Declaration of Helsinki. Inclusion of fully anonymized OCT data in this study did not need additional permission.

Optical coherence tomography

All OCTs were recorded with Spectralis SD-OCT (Heidelberg Engineering, Inc, Heidelberg, Germany). All participants were examined with the OCT's built-in macular volume scan (25 vertical B-scans and 1024 A-scans, scan angle = 20° × 20°, automatic real-time function [ART] = 49). For quality control (QC) of all OCT scans, the quality criteria as defined by the OSCAR-IB study were used.15 OCT scans of the left eyes were arbitrarily selected to enter the analysis. One scan had to be excluded because of file corruption (the file could not be opened). All other scans passed QC, leading to a total analysis of 17 scans.

Semiautomatic intraretinal segmentation

All B-scans were automatically presegmented using segmentation beta-software (Spectralis Viewing Module version 6.0.0.7) of the Heidelberg Eye Explorer (version 1.9.3.0) provided by the manufacturer, which is almost identical to the final release version of the segmentation algorithm. Five operators from different medical centers (Hospital Clínic de Barcelona; NeuroCure Clinical Research Center Berlin; University of California, San Francisco; University Hospital Zürich; and VU University Medical Center, Amsterdam) manually corrected the 25 segmentation lines of each of the 17 presegmented B-scans. The segmentation lines indicate the following retinal layers: retinal nerve fiber layer, GCL, inner plexiform layer (IPL), INL, outer plexiform layer (OPL), and outer nuclear layer (ONL). The ONL includes the inner segments of the photoreceptors (figure 1A). As an additional parameter, the compound thickness of the GCL and IPL was calculated (ganglion cell inner plexiform layer [GCIP]). The mean thickness of the individual retinal layers was computed in an area of 6 mm around the fovea as defined by the Early Treatment Diabetic Retinopathy Study (ETDRS) (figure 1B).16

Figure 1. Example OCT scan.

Figure 1

Open in a new tab

(A) Optical coherence tomography B-scan with segmentation lines generated from the mean of all 5 raters with color-coded SD between raters (1 pixel ≙ 3.78 μm). Shown are the retinal nerve fiber layer (RNFL), ganglion cell layer (GCL), inner plexiform layer (IPL), inner nuclear layer (INL), outer plexiform layer (OPL), and outer nuclear layer (ONL). For analysis, the GCL and IPL were also segmented as a compound layer (GCIP). In the example B-scan, the segmentation lines with higher levels of disagreement between raters (indicated by higher SDs) are the IPL, the area of the fovea, and segmentation lines crossing blood vessel artifacts. (B) Example thickness map of the GCIP on the fundus image. The mean of all thickness values within the 6-mm ETDRS circle (white circle) was used for analysis. GCIP = ganglion cell inner plexiform layer.

Statistical analysis

Intraclass correlation coefficients (ICCs) with the degree of consistency among measurements (norm-referenced reliability) were used to analyze the level of agreement between individual grader-corrected segmentation results. Local differences across the volume scans were analyzed by calculating the ICC for each A-scan of all layers and visualizing the results as color-coded ICC heat maps. Correlations between the mean layer thicknesses and SD of corrected scans from different centers were analyzed using the Spearman rho. Quantitative OCT analysis and statistical analysis were performed using MATLAB 2012b (Mathworks, Ismaning, Germany) and R (version 3.3.3).

Results

First, we investigated whether intraretinal segmentation produced similar results on a group level between the different centers (figure 2). This was the case for RNFL, GCL, IPL, and GCIP, but not for INL, OPL, and ONL.

Figure 2. Intraretinal layer thickness measurements.

Figure 2

Open in a new tab

Boxplots for different retinal layer thicknesses of OCT scans from all 17 subjects corrected by all 5 raters (Hospital Clínic de Barcelona [HCB]; NeuroCure Clinical Research Center [NCRC], Berlin; University of California, San Francisco [UCSF]; University Hospital Zürich [USZ]; and VU University Medical Center [VUMC], Amsterdam). The dots represent outliers. GCIP = ganglion cell inner plexiform layer; GCL = ganglion cell layer; INL = inner nuclear layer; IPL = inner nuclear layer; ONL = outer nuclear layer; OPL = outer plexiform layer; RNFL = retinal nerve fiber layer.

On an individual measurement level, overall reliability between raters was excellent for RNFL, GCL, IPL, GCIP, and ONL (all ICCs > 0.96) and good for INL and OPL (all ICCs > 0.85). Reliability was similar when patients with MS and controls were analyzed separately (table).

Table.

Intraretinal segmentation inter-rater reliability

graphic file with name NEURIMMINFL2017014902t1.jpg

Open in a new tab

To identify the image areas of disagreement between the centers, we calculated intercenter reliability as an ICC for every A-scan. Here, the perimacular area showed the highest reliability, while the scan's borders and the fovea were subject to higher disagreement between graders (figure 3). RNFL, GCL, GCIP, and ONL showed high local agreement. Of note, the temporal RNFL (figure 3A) and the peri-foveal area in the ONL (figure 3G) showed weaker agreement. The latter most likely reflects Henle fibers.16,17 IPL, INL, and OPL showed weak local agreement (figure 3). In addition, on single B-scans, areas with blood vessels showed higher disagreement between centers (figure 1A). The intercenter variabilities for the different retinal layers were not correlated with the respective mean layer thicknesses (all p > 0.05).

Figure 3. Local intraclass correlation coefficients.

Figure 3

Open in a new tab

Color-coded interclass correlation coefficients (ICCs) for the manually corrected intraretinal layer segmentation across all positions in macular volume scans (1024 A-scans and 25 B-scans). Seven different intraretinal layers were assessed (A-G): retinal nerve fiber layer (RNFL), ganglion cell layer (GCL), inner plexiform layer (IPL), combined GCL and IPL (GCIP), inner nuclear layer (INL), outer plexiform layer (OPL), and outer nuclear layer (ONL). The central foveal scan and peripheral areas outside the ETDRS macular volume are predilection sites of disagreement. GCIP = ganglion cell inner plexiform layer.

Finally, we analyzed whether segmentation results differed after manual correction in comparison with uncorrected segmentation directly derived from the software (figure 4). After manual correction, the outer retinal layers showed the most pronounced differences in comparison with purely automated segmentation and the broadest distribution of differences (range OPL = 8.8 μm; range ONL = 9.4 μm). The inner retinal layer thicknesses differed less profoundly (range RNFL = 1.9 μm, range GCL = 3.4 μm, range IPL = 3.8 μm, and range GCIP = 2.4 μm). The differences between manual and automatic segmentation were centered on zero for the inner layers. By contrast, the manual corrected segmentation volume tended to be lower in outer layers (OPL and ONL).

Figure 4. Differences between semiautomatic and fully automatic segmentation.

Figure 4

Open in a new tab

Boxplots show the differences between automated and manually corrected segmentation for the retinal nerve fiber layer (RNFL), ganglion cell layer (GCL), inner plexiform layer (IPL), combined ganglion cell and inner plexiform layer (GCIP), inner nuclear layer (INL), outer plexiform layer (OPL), and outer nuclear layer (ONL). Manual correction had the biggest influence on the outer layers (difference of up to ca. −10 μm) and less pronounced influence on the inner layers. The stars represent outliers.

Discussion

We analyzed the inter-rater reliability of semiautomatic intraretinal segmentation of macular volume B-scans from OCT. Color-coded heat maps illustrating ICCs between segmentation results from 5 different experienced, academic OCT readers provide a fast overview of the agreement between raters and help to rapidly identify areas of disagreement. Furthermore, analysis on a single B-scan level as illustrated in figure 1A visualizes local deviations between segmentation lines of different raters revealing areas of higher inter-rater variability. An automated segmentation algorithm for Spectralis OCT macular volume scans was recently made available, and a version almost identical to it has been applied in our study. We aimed to determine the inter-rater reliability of manually corrected segmentation of retinal layers compared with results from automated segmentation.

Mean layer thickness when corrected manually showed only little variation between graders in the case of the innermost layers RNFL, GCL and IPL, and ONL. Consistency of results could even be improved when GCL and IPL were segmented as a compound layer (GCIP; figures 2 and 3). The heat maps again emphasize the much better inter-rater reliability of RNFL, GCIP, and ONL compared with INL and OPL. Spatial reliability analysis using heat maps revealed the fovea and the scan periphery as sites of greatest disagreement. Segmentation variation at the fovea is most probably due to thinning of the innermost retinal layers at the foveal pit. Whereas in the scan periphery (outreaching the ETDRS macular volume), major sources of disagreement were induced by retinal vessel artifacts (figure 1A). Comparing manually corrected with automated segmentation results revealed that inner retinal layers (RNFL, GCL, IPL, and GCIP) were accurately identified by automatic segmentation with little spread of thickness differences, whereas the outer retinal layers (INL, OPL, and ONL) showed stronger disagreement and improved with manual correction.

Accurate quantification of retinal layer integrity is relevant in the context of degenerative diseases involving the retina such as MS, especially for understanding the extent and timing of injury to neurons beyond the site of the presumed initial injury. For neurodegenerative diseases, OCT allows evaluation of injury to partially segregated neuronal and axonal compartments and is at a resolution 2–3 orders of magnitude better than what can be achieved via MRI. OCT's quantification also has a tremendous advantage, given its ease and immediacy. However, for multicenter clinical trials, quality criteria and standardized correction methods are required, and a standardized training of expert OCT readers needs to be established. The methods used here are far less labor intensive than those required for brain MRI. In addition, the afferent visual pathway is an increasingly recognized model that allows quantifying MS and optic neuritis (ON) related neuro-axonal damage through solid structure-function investigations.18–22 Therefore, measures of retinal atrophy assessed by OCT have gained significant interest as a powerful outcome tool to monitor neurodegeneration and protective therapies.23 Recently, RNFL thickness quantified by OCT proved its value as a primary outcome measure in patients with acute ON treated with erythropoietin vs placebo in addition to steroids,4 while a number of other trials are ongoing. However, there has been increasing interest in measuring deeper retinal layers such as the GCIP that do not exhibit the acute swelling seen in the RNFL in ON.24 Results from OCT macular B-scan segmentation (GCIP thickness at month 1) proved to be predictive of visual outcomes at month 6 following MS associated ON.25 Given the increasing use of OCT in multicenter clinical trials, the inter-rater reliability of macular multilayer segmentation along with OCT quality reading15,26 will be key to reduce noise and subsequently the sample size required for therapeutic clinical trials. Quality criteria have not yet been proposed for macular volume scans, as it remains unclear how the built-in algorithms perform. With respect to segmentation of macular volume B-scans, the results of our study suggest that restriction to the inner sections of the ETDRS grid and inner retinal layers (including some layers improved with manual correction such as INL) should reduce noise. The inner ETDRS grid has also been shown to have good retest reliability.16 Fortuitously, these areas and layers also have been suggested to demonstrate the most pronounced changes over the course of the disease. In addition to its potential use as an end point in clinical trials, recent OCT studies on disease mechanisms provided evidence of retinal pathology and neuro-axonal damage appearing as early as in the stage of non-ON clinically isolated syndromes.6,27

For the Cirrus HD-OCT (Carl Zeiss Meditec, Dublin, CA, USA), single-center inter-rater correlation coefficients of 0.99 for GCL + IPL and RNFL + GCL + IPL, 0.91–0.94 for INL + OPL, and 0.92–0.93 for ONL + photoreceptor segments have been reported in patients with MS using prototype segmentation software.7

Regarding Spectralis SD-OCT scans, Hood et al.28 reported single-center inter-rater correlations between 2 graders using MATLAB-derived segmentation software for one horizontal B-scan in 30 patients with retinitis pigmentosa and 23 control subjects. Correlation coefficients were 0.95–0.96 for RNFL and GCL + IPL. Correlation coefficients were not reported for other retinal layers. Seigo et al.14 assessed inter-rater reliability for fully manual MATLAB-derived segmentation of the central horizontal macular B-scan traversing the fovea acquired with Spectralis SD-OCT in 52 patients with MS and 30 control subjects. GCIP, ONL, and ONL + photoreceptor segment thickness measures demonstrated excellent inter-rater reproducibility with correlation coefficients of 0.94–0.95, whereas the correlation coefficient for the RNFL was only 0.64 and for the INL 0.51.

A limitation of our study is the rather small sample size. However, given that 25 B-scans per subject were segmented, the analysis is based on 425 B-Scans from 17 subjects.

To conclude, automated segmentation with manual correction by trained raters of Spectralis macular OCT scans is highly reliable and can thus be applied in multicenter settings. Reliability can be improved by restricting analysis to the perimacular area and by compound segmentation of GCL and IPL. The foveal pit and the scan periphery with retinal vessel artifacts were the sites of greatest disagreement.

Intraretinal layer thicknesses might serve as important outcomes for trials in MS or MS ON in the future, and established methods for optimizing the inter-rater reliability of retinal layer segmentation will be crucial. When fully automated segmentation becomes broadly available, the approaches presented in this study could easily be adapted to longitudinal OCT data to compare segmentation results across follow-up visits. This may help to reliably identify segmentation errors and discriminate them from true degenerative changes.

Acknowledgment

This work was developed under the collaboration of the IMSVISUAL network (imsvisual.org).

Glossary

ETDRS

Early Treatment Diabetic Retinopathy Study

GCIP

ganglion cell inner plexiform layer

GCL

ganglion cell layer

ICC

intraclass correlation coefficient

INL

inner nuclear layer

IPL

inner plexiform layer

OCT

optical coherence tomography

ON

optic neuritis

ONL

outer nuclear layer

OPL

outer plexiform layer

QC

quality control

RNFL

retinal nerve fiber layer

SD

spectral domain

Author contributions

TO: drafting/revising the manuscript, analysis and interpretation of data, acquisition of data, and statistical analysis. GLT, SL, IG, RN, CS, and LB: drafting/revising the manuscript, interpretation of data, and acquisition of data. AP, FP, and PV: drafting/revising the manuscript and interpretation of data. AUB: drafting/revising the manuscript, analysis and interpretation of data, statistical analysis, and study concept. AG and SS: drafting/revising the manuscript, interpretation of data, and study concept.

Study funding

No targeted funding reported.

Disclosure

T. Oberwahrenbrock received speaker honoraria from Teva and Bayer and received research support from the Federal Ministry of Education and Research (BMBF). G.L. Traber received travel funding from Santhera. S. Lukas received travel funding and/or speaker honoraria from Genzyme and Bayer and has been employed by Heidelberg Engineering GmbH. I. Gabilondo is a reviewer editor of Neuro-Ophthalmology, part of the journal Frontiers in Neurology; and received research support from the Carlos III Health Institute (JR/00008, PI/16/00005). R. Nolan and C. Songster report no disclosures. L. Balk received research support from Teva. A. Petzold served on the scientific advisory board of optical coherence tomography in MS; served on the editorial board of Neuro-Ophthalmology; holds a patent for Neurofilaments; consulted for Optical Coherence Tomography Quality Control Reading for Passos study, Novartis; and received research support from OCTiMS study, Novartis, NIH, Biomedical Research Centre, based at Moorfields Eye Hospital, NHS Foundation Trust, and UCL Institute of Ophthalmology. F. Paul served on the scientific advisory board of Novartis OCTIMS study and MedImmune; received travel funding and speaker honoraria from Bayer, Novartis, Biogen, Teva, Sanofi-Aventis/Genzyme, Merck Serono, Alexion, Chugai, MedImmune, and Shire; is an academic editor of PLoS One and an associate editor of Neurology® Neuroimmunology & Neuroinflammation; consulted for Sanofi-Aventis/Genzyme, Biogen, MedImmune, Shire, and Alexion; and received research support from Bayer, Novartis, Biogen, Teva, Sanofi-Aventis/Genzyme, Alexion, Merck Serono, German Research Council, Werth Stiftung of the City of Cologne, German Ministry of Education and Research, Arthur Arnstein Stiftung Berlin, EU FP7 Framework Program, Arthur Arnstein Foundation Berlin, Guthy Jackson Charitable Foundation, and the US National Multiple Sclerosis Society. P. Villoslada is on the editorial board of Neurology & Therapy, Curr There Op Neurol, Mult Scl & Demyel Dis, and Systems Medicine; holds patents for Methylthioadenosine for the treatment of MS, Agnostic neurotrophic compounds for the treatment of brain disease, Gene signature pattern as a biomarker for MS, and Algorithm for reconstructing white matter tracts from diffusion images; received research support from Novartis, Instituto de Salud Carlos III, European Commission, National MS Society, NIH, Fundacion Maraton TV3, and GAEM Foundation; holds stock or stock options in Bionure, Spire Bioventures, Mint-Labs and Health Engineering, and Roche; and provided expert testimony for Synthon. A.U. Brandt has a patent pending for Perceptive visual computing based postural control analysis, Multiple sclerosis biomarker, Perceptive sleep motion analysis, and Fovea morphometry; consulted for Motognosis; is on the executive board of IMSVISUAL; received research support from Novartis, Biogen, BMWi, BMBF, and the Guthy-Jackson Charitable Foundation; and holds stock or stock options in Motognosis. A.J. Green served on the scientific advisory board of MedImmune, Novartis, OCTIMS, Inception 5 Biosciences, and Bionure; is an associate editor of JAMA Neurology; was an editorial board member of Neurology; holds a patent for Remyelination molecules and pathways; consulted for Inception 5 Sciences; received research support from Novartis Pharma OCTIMs, Inception Sciences SRA, NINDS, NIA, National MS Society, Sherak Foundation, and Hilton Foundation; holds stock or stock options in Inception 5; and served as an expert witness at Mylan v Teva Pharma. S. Schippling served on the scientific advisory board of Bayer, Biogen, Merck Serono, Novartis, Sanofi-Genzyme, and Teva; received travel funding and speaker honoraria from Bayer, Biogen, Merck Serono, Sanofi-Genzyme, and Teva; is an associate editor of Frontiers in Neurology; holds a patent for Therapeutic vaccination in PML using VP1 and IL7; received research support from Sanofi-Genzyme, Novartis, University of Zurich, Betty and David Koetser Foundation for Brain Research, and Swiss Multiple Sclerosis Society. Full disclosure form information provided by the authors is available with the full text of this article at Neurology.org/NN.

References

  • 1.Petzold A, de Boer JF, Schippling S, et al. . Optical coherence tomography in multiple sclerosis: a systematic review and meta-analysis. Lancet Neurol 2010;9:921–932. [DOI] [PubMed] [Google Scholar]
  • 2.Oberwahrenbrock T, Schippling S, Ringelstein M, et al. . Retinal damage in multiple sclerosis disease subtypes measured by high-resolution optical coherence tomography. Mult Scler Int 2012;2012:530305. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Balk LJ, Cruz-Herranz A, Albrecht P, et al. . Timing of retinal neuronal and axonal loss in MS: a longitudinal OCT study. J Neurol 2016;263:1323–1331. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Sühs KW, Hein K, Sättler MB, et al. . A randomized, double-blind, phase 2 study of erythropoietin in optic neuritis. Ann Neurol 2012;72:199–210. [DOI] [PubMed] [Google Scholar]
  • 5.Raftopoulos R, Hickman SJ, Toosy A, et al. . Phenytoin for neuroprotection in patients with acute optic neuritis: a randomised, placebo-controlled, phase 2 trial. Lancet Neurol 2016;15:259–269. [DOI] [PubMed] [Google Scholar]
  • 6.Green AJ, McQuaid S, Hauser SL, Allen IV, Lyness R. Ocular pathology in multiple sclerosis: retinal atrophy and inflammation irrespective of disease duration. Brain J Neurol 2010;133:1591–1601. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Saidha S, Syc SB, Ibrahim MA, et al. . Primary retinal pathology in multiple sclerosis as detected by optical coherence tomography. Brain 2011;134:518–533. [DOI] [PubMed] [Google Scholar]
  • 8.Gelfand JM, Nolan R, Schwartz DM, Graves J, Green AJ. Microcystic macular oedema in multiple sclerosis is associated with disease severity. Brain 2012;135:1786–1793. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Saidha S, Al-Louzi O, Ratchford JN, et al. . Optical coherence tomography reflects brain atrophy in MS: a four year study. Ann Neurol 2015;78:801–813. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Walter SD, Ishikawa H, Galetta KM, et al. . Ganglion cell loss in relation to visual disability in multiple sclerosis. Ophthalmology 2012;119:1250–1257. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Sanchez-Dalmau B, Martinez-Lapiscina EH, Torres-Torres R, et al. . Early retinal atrophy predicts long-term visual impairment after acute optic neuritis. Mult Scler:1352458517718628. Epub 2017 Jun 1. [DOI] [PubMed]
  • 12.Saidha S, Sotirchos ES, Ibrahim MA, et al. . Microcystic macular oedema, thickness of the inner nuclear layer of the retina, and disease characteristics in multiple sclerosis: a retrospective study. Lancet Neurol 2012;11:963–972. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Martinez-Lapiscina EH, Arnow S, Wilson JA, et al. . Retinal thickness measured with optical coherence tomography and risk of disability worsening in multiple sclerosis: a cohort study. Lancet Neurol 2016;15:574–584. [DOI] [PubMed] [Google Scholar]
  • 14.Seigo M, Sotirchos E, Newsome S, et al. . In vivo assessment of retinal neuronal layers in multiple sclerosis with manual and automated optical coherence tomography segmentation techniques. J Neurol 2012;259:2119–2130. [DOI] [PubMed] [Google Scholar]
  • 15.Tewarie P, Balk L, Costello F, et al. . The OSCAR-IB consensus criteria for retinal OCT quality assessment. PLoS One 2012;7:e34823. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Oberwahrenbrock T, Weinhold M, Mikolajczak J, et al. . Reliability of intra-retinal layer thickness estimates. PLoS One 2015;10:e0137316. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Ouyang Y, Walsh AC, Keane PA, Heussen FM, Pappuru RKR, Sadda SR. Different phenotypes of the appearance of the outer plexiform layer on optical coherence tomography. Graefes Arch Clin Exp Ophthalmol 2013;251:2311–2317. [DOI] [PubMed] [Google Scholar]
  • 18.Pfueller CF, Brandt AU, Schubert F, et al. . Metabolic changes in the visual cortex are linked to retinal nerve fiber layer thinning in multiple sclerosis. PLoS One 2011;6:e18019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Bock M, Brandt AU, Kuchenbecker J, et al. . Impairment of contrast visual acuity as a functional correlate of retinal nerve fibre layer thinning and total macular volume reduction in multiple sclerosis. Br J Ophthalmol 2012;96:62–67. [DOI] [PubMed] [Google Scholar]
  • 20.Costello F. The afferent visual pathway: designing a structural-functional paradigm of multiple sclerosis. ISRN Neurol 2013;2013:1–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Galetta SL, Villoslada P, Levin N, et al. . Acute optic neuritis: Unmet clinical needs and model for new therapies. Neurol Neuroimmunol Neuroinflamm 2015;2:e135 doi: 10.1212/NXI.0000000000000135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Petzold A, Wattjes MP, Costello F, et al. . The investigation of acute optic neuritis: a review and proposed protocol. Nat Rev Neurol 2014;10:447–458. [DOI] [PubMed] [Google Scholar]
  • 23.Barkhof F, Calabresi PA, Miller DH, Reingold SC. Imaging outcomes for neuroprotection and repair in multiple sclerosis trials. Nat Rev Neurol 2009;5:256–266. [DOI] [PubMed] [Google Scholar]
  • 24.Syc SB, Saidha S, Newsome SD, et al. . Optical coherence tomography segmentation reveals ganglion cell layer pathology after optic neuritis. Brain 2012;135:521–533. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Gabilondo I, Martínez-Lapiscina EH, Fraga-Pumar E, et al. . Dynamics of retinal injury after acute optic neuritis. Ann Neurol 2015;77:517–528. [DOI] [PubMed] [Google Scholar]
  • 26.Schippling S, Balk LJ, Costello F, et al. . Quality control for retinal OCT in multiple sclerosis: validation of the OSCAR-IB criteria. Mult Scler J 2015;21:163–170. [DOI] [PubMed] [Google Scholar]
  • 27.Oberwahrenbrock T, Ringelstein M, Jentschke S, et al. . Retinal ganglion cell and inner plexiform layer thinning in clinically isolated syndrome. Mult Scler 2013;19:1887–1895.Available at: msj.sagepub.com/content/early/2013/05/23/1352458513489757. Accessed June 4, 2013. [DOI] [PubMed] [Google Scholar]
  • 28.Hood DC, Cho J, Raza AS, Dale EA, Wang M. Reliability of a computer-aided manual procedure for segmenting optical coherence tomography scans. Optom Vis Sci 2011;88:113–123. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Neurology® Neuroimmunology & Neuroinflammation are provided here courtesy of American Academy of Neurology

RESOURCES