Fully automated detection, segmentation, and analysis of in vivo RPE single cells

Abstract

Objective

To develop a fully automated method of retinal pigmented epithelium (RPE) cells detection, segmentation and analysis based on in vivo cellular resolution images obtained with the transscleral optical phase imaging method (TOPI).

Methods

Fourteen TOPI–RPE images from 11 healthy individuals were analysed. The developed image processing method encompassed image filtering and normalisation, detection and removal of blood vessels, cell detection and cell membrane segmentation. The produced measures were cellular density of RPE layer, cell area, number of neighbouring cells, eccentricity, circularity and solidity. In addition, we proposed coefficient of variation (CV) of RPE cellular membrane (CMD_CV) and the solidity of the RPE cell membrane-shape as new metrics for the assessment of RPE single cells.

Results

The observed median cellular density of the RPE layer was 3743 cells/µm² (interquartile rate (IQR) 1687), with a median observed RPE cell area of 193 µm² (IQR 141). The mean number of neighbouring cells was 5.22 (standard deviation (SD) 0.05) per RPE cell. The mean RPE cell eccentricity was 0.67 (SD 0.02), median circularity 0.83 (IQR 0.01), and median solidity 0.92 (IQR 0.00). The median CMD_CV was 0.19 (IQR 0.02). The method is characterised by a median image processing and analysis time of 48 sec (IQR 12) per image.

Conclusions

The present study provides the first fully automated quantitative assessment of human RPE single cells in vivo. The method provides a baseline for future research in the field of clinical ophthalmology, enabling characterisation and diagnostics of retinal diseases at the single-cell level.

Introduction

The retina is the vitreal-most ten-layered light-sensitive nervous tissue membrane of the eye. Its role is to convert the received light stimuli into nerve impulses and send them with the optic nerve to the visual centres of the brain. The retinal pigmented epithelium (RPE) is the scleral-most monolayer of pigmented retinal cells.

Although they are located outside of the neurosensory retina, RPE cells play some crucial roles, such as light absorption, epithelial transport and maintenance of the visual cycle [1–5]. Some RPE cell morphology characteristics, namely cell density, number of neighbours, eccentricity, and form factor, are postulated to differ depending on cell maturation and condition [2, 6–9]. Some other studies report RPE cell loss caused by diseases of the eye and aging [10–13].

Although several diagnostic imaging modalities allow for in vivo assessment of the human eye (e.g. optical coherence tomography (OCT) [14–16], scanning laser ophthalmoscopy (SLO) [17, 18], and fundus autofluorescence [19, 20]) these methods do not allow for the diagnosis of retinal diseases at their early stage because the minuscule changes in RPE cell morphology cannot be detected. Furthermore, RPE layer in vivo imaging at the single-cell level is challenging due to several factors, namely, the low contrast between neighbouring cells, motion artefacts, retinal layer non-linearity, and difficulties with the image’s focal point identification.

Transscleral optical phase imaging (TOPI), proposed in 2017, is a novel non-invasive, in vivo, high-resolution retinal imaging modality. The use of both adaptive optics and oblique illumination of the retina enhances the contrast of RPE cells [2, 9]. The resultant superior imaging resolution enables discerning single RPE cells’ cellular membranes.

In this paper, we present a novel fully automated method of RPE cell detection, segmentation and analysis at both layer-level and single- cell level for the TOPI-obtained retinal images.

Materials and methods

Study population

The study population included 11 healthy individuals (4 women and 7 men), from which 14 TOPI–RPE images of the left eye were obtained. The mean age of the examined individuals was 29 years (standard deviation (SD) 8). The conducted study adheres to the tenets of the Declaration of Helsinki. The study was approved by the Ethical Committee of the Swiss Department of Health on research involving human subjects (CER-VD N°2017–00976). Informed consent was obtained from all the participants.

TOPI image acquisition

TOPI relies on high-angle oblique illumination of the retina, combined with a flood illumination adaptive optics fundus camera, to enhance cell contrast and correct for ocular aberrations. Transscleral illumination of the retina was performed using two near-infrared light-emitting diodes (wavelength, λ = 810 nm) located on the nasal and temporal side of the eye. The acquired images encompass a field-of-view of 4.4° × 4.4°. More detailed information on the used TOPI setup has been published previously by our group [2].

A single TOPI-obtained RPE layer image is characterised by a low signal-to-noise ratio (SNR). Therefore, prior to image analysis, the SNR is first increased by acquiring several raw images (around 100 per acquisition), then registered, and averaged into a single TOPI image. Image acquisition and registration were performed following the protocol described in detail by Laforest et al. [2]. The acquired image stacks were exported as tif files. These images are characterised by a black border, resultant from the registration padding process. To remove such padding, first, the image gradient is calculated. Second, vertical and horizontal borders that presented an average local intensity gradient value below 10% of the image’s gradient SD were removed, cropping the image to the final size. The eccentricities from the fovea of the imaged retinal areas ranged between 2.5° and 13.44° with a consistent field-of-view of 5°. The final TOPI–RPE images were exported as 1975 by 1906 pixels and with a digital sampling between 0.73 µm and 1.0 µm per pixel. Supplementary Table 1 presents detailed information on imaging parameters, registered images and imaged volunteers.

Image processing

The fully automated TOPI-obtained RPE image processing and analysis is divided into four stages. First, the images are normalised in terms of contrast/attenuation, unevenness of the RPE layer and noise, and any out-of-focus (OoF) areas are discarded. Second, the shadow of retinal vasculature present in the innermost (vitreal-most) retinal layers is detected and removed from the final image. Third, cells are individually detected and segmented. Finally, the fourth and last step consists of characterising the RPE layer in general and single RPE cells (Fig. 1).

Fig. 1. TOPI–RPE image processing diagram, from acquisition to single-cell masks.

a. Filtering and normalization, b. Vessel detection and removal, c. Cell detection, d. Cell segmentation, e. Single cell & Colony metrics. (For the colour version, please refer to the online version of the article). Suba: subroutine a. Subb: subroutine b. VOoF Vessel and out-of-focus mask.

Image filtering and normalisation

In order to adjust for the unevenness of the RPE layer background, flat-field correction with a two-dimensional Gaussian smoothing kernel (σ 10 pixels) was applied. Subsequently, to clean the image from noise, Butterworth highpass filtering (B_hpf) and Gaussian filtering (Gauss) were performed. The first-order Butterworth filter used a cut-off radius of 50 pixels. The Gaussian filter used a smoothing kernel with a σ of 20 pixels (Fig. 1a). To prevent the filtering out of essential RPE morphology, both in the spatial and frequency domain, the filter sizes, thresholds, and values implemented throughout the image processing and analysis methodology, were obtained experimentally and based on previously published literature in the assessment of ex vivo and in vivo morphology of RPE cells. The usual RPE cell size varies between 10 and 14 µm (14–19 pixels with a digital sampling of 0.73 µm) [7, 21, 22].

A lower RPE cell edge contrast characterises the OoF areas compared with the in-focus part of the image. To remove the defocused areas, we implemented the channel-prior method [23] with adaptive gamma correction, which increases the remaining image (deH) contrast. The method allows for single-image enhancement without a priori knowledge of its’ quality or high contrast standard images. Moreover, the channel-prior method produces a distance map (Dist) (an estimation of the haze thickness at each pixel), necessary for the vascular outline approximation.

Detection and removal of blood vessels

The detection of blood vessels is performed by using the four previously obtained images (B_hpf, Gauss, deH, and Dist) (Fig. 1b). Each image is subjected to Subroutine A (SubA). SubA begins with square-shaping the image, and its quadtree decomposition (QuaD) returns a sparse matrix subsequently reconstructed as a block-map. The QuaD threshold is applied at 3*SD of the image. QuaD is a common methodology in several fields, including image processing, being used from multiresolution decomposition and analysis [24–26], to compression [27, 28] and machine learning [29, 30]. Application of QuaD for RPE cells segmentation is a novel approach developed specifically for this project. The QuaD square blocks of ≥8 pixels and ≤10% of the original image size are included in the subsequent image processing. After inverting (image complement), the obtained square blocks maps, small and interconnected structures at their external borders are discarded using morphological filtering (erosion with a discoid element of 4-pixel radius) followed by dilation with the same discoid element. Finally, the last step of SubA is reshaping of the resultant mask to the original’s image size. Supplementary Fig. 1, Block B, presents a more in-depth depiction of the process, where Gauss image is an example input.

The OoF mask obtained during image filtering and normalisation stage is summed with the binary mean of SubA(B_hpf), SubA(_Gauss), SubA(deH), and SubA(Dist), forming the vessel-OoF mask (VOoF). VOoF mask is used to eliminate the infravascular RPE cells from further image processing.

Cell detection

Cell centre detection is based on the method proposed by Khamidakh et al. [31], henceforth named Subroutine B (SubB). In case the distance between adjacent cellular centres is ≤10 pixels, the individual cells are detected as the same cell. We applied SubB to B_hpf, to the contrast-limited adaptive histogram equalised B_hpf, and to the highpass filtered ($\frac{1}{8}$of the original image sized kernel) B_hpf. One more time, cellular centres within ≤10 pixels are fused. Finally, cellular centres in the distance of ≤10 pixels from the image border are removed to prevent the inclusion of non-fully-imaged cells in the image analysis (Fig. 1c). Supplementary Fig. 1, Block C, presents a more in-depth graphical demonstration of the cell detection method, with B_hpf as its input.

Cell membrane segmentation

Detection of the cellular membrane at the single-cell level begins with convolving B_hpf with a discoid structuring element (radius of 4 pixels). The resultant blurring of the image removes any possible local salt-and-pepper noise that might occur during the transformation from the Fourier to the spatial domain. Then, the image is convolved with a star-shaped mask (size 7 pixels). The convolution enhances local vertical, horizontal, and diagonal edges in the image. The final filter is a 7 × 7-pixel Mexican hat. With these three filtering stages followed by zero-crossing in the spatial domain, a binary mask representing the cellular membrane is developed. Finally, the mask is skeletonised and cleaned from sporadic branches, while single pixels are discarded. The inverted mask is convolved with a discoid structuring element (radius of 4 pixels) and re-inverted (Fig. 1d). Such a procedure improves the separation of the cells and prevents their possible overlapping. An example of the process of the cell membrane segmentation method is presented in Supplementary Fig. 1, Block D, with the B_hpf as input. Cellular masks not corresponding with respective cellular centres identified during the “Cell detection” step are subsequently classified as artefacts and discarded from further analysis.

Supplementary Fig. 2 presents an example of the outputs and intermediate results obtained throughout the segmentation and analysis process.

Data analysis

Cells with area or centre overlapping with the VOoF mask were discarded from the analysis of cellular characteristics.

Using the previously created cellular masks and the original TOPI-obtained image, morphological and neighbourhood characteristics of individual RPE cells were assessed (Fig. 1e). MATLAB regionprops function was used to obtain basic morphological characteristics of RPE cells (area, centroid and weighted centroid, eccentricity, solidity, intensity, and circularity). In addition, assessed characteristics included the CV of RPE cellular membrane (CMD_CV) [32, 33], number of neighbouring cells and the cellular density of the RPE layer. To decrease the possible risk of assessment bias, RPE cells immediately adjacent to the VOoF mask were discarded from the number of neighbours’ evaluation. A descriptive analysis was conducted for each image. Supplementary Table 2 presents the full list of the assessed metrics, along with their definitions and formulas.

The normality of variables was assessed with the Shapiro–Wilk’s test (p > 0.10) and histogram skewness (skewness −0.5–0.5).

The image processing pipeline and the underlying algorithms were developed and tested, as well as data management, on a DELL workstation (DELL XPS 13 9380, Windows 10, 64 bits, 2 1.80 GHz, 16.0 GB RAM) equipped with the MATLAB (version R2019, with Bioinformatics Toolbox^TM, Financial Toolbox^TM and Statistics and Machine Learning Toolbox^TM). Image registration was performed with ImageJ 1.52 with a modified macro from Laforest et al. [2], with the plugins TurboReg [34] and Template Matching [35]. For boxplots generation and statistical analysis, we used R studio 1.2.1335 with gmodels, e1071, readxl, and xlsx packages.

Results

Figure 2 presents the resultant analysis maps for an example sample (number 6), where the RPE coverage was 88%.

Fig. 2. Sample 6: Analysis results, with final metrics maps.

CMD_CV: coefficient of variation of RPE cellular membrane mask (For the colour version, please refer to the online version of the article).

Characteristics of the RPE layer

In all obtained image samples, the algorithm analysed a significant image area (median 90%, interquartile rate (IQR) 9%). The discarded parts of the image areas were either blood vessels or blurred/hazed. After discarding cells belonging to the VOoF areas (example in Supplementary Fig. 2—VOoF subpanel), the mean number of cells analysed per sample was 6864 (SD 869). The median observed cellular density of the RPE layer was 3743 cells/µm² (IQR 1687) (Table 1, Supplementary Table 3 and 4, and Supplementary Fig. 3).

Table 1.

Summary of the RPE layer and single-cell level characteristics extracted from the 14 analysed samples.

	Number of cells	Area (µm2)	Circularity (0–1)	Eccentricity (0–1)	Solidity (0–1)	Intensity (0–1)	Centroid shift (pixels)	CMDCV	Number of neighbours	Colony density (cells/mm2)	Cell coverage (%)	Analysis time (s)
Mean	6864	243	0.83	0.67	0.92	0.42	0.29	0.19	5.22	3249	91	52
SD	869	82	0.01	0.02	0.00	0.06	0.06	0.01	0.05	902	7	10
Min	5012	175	0.81	0.64	0.91	0.31	0.19	0.17	5.12	1797	74	42
Max	8452	400	0.84	0.70	0.92	0.51	0.39	0.20	5.31	4237	100	75
Median	6984	193	0.83	0.67	0.92	0.44	0.29	0.19	5.21	3743	90	48
IQR	995	141	0.01	0.02	0.00	0.04	0.08	0.02	0.06	1687	9	12
Shapiro–Wilk’s p value	0.98a	0.00	0.01	0.67a	0.00	0.02	0.69a	0.03	0.94a	0.01	0.21	0.08
Skewness	−0.31a	0.67	−0.45	−0.11a	−1.25	−0.77	0.14a	−0.48	−0.06a	−0.45	−0.80	0.84

CMD_CV: coefficient of variation of RPE cellular membrane mask.

^aValues that fulfil the requirements for normality.

Characteristics of RPE single cells

The median observed RPE cell area was 193 µm² (IQR 141). The mean number of neighbouring cells was 5.22 (SD 0.05) per RPE cell. The mean RPE cell eccentricity was 0.67 (SD 0.02). The RPE cell circularity was at a median 0.83 (IQR 0.01). The median CMD_CV, denoting the distance of each cellular membrane-depicting pixel to the cellular centroid, was 0.19 (IQR 0.02). The median solidity of the RPE cells was 0.92 (IQR 0.00). The median RPE cell normalised image intensity value was 0.44 (IQR 0.04). The mean pixel distance between morphology-based and intensity-based RPE cellular centroids was 0.29 pixels (SD 0.06) (Table 1, Supplementary Table 3 and 4, and Supplementary Fig. 3).

Performance of the image processing algorithm

The median image processing time was 48 s (IQR 12) per image. It included digital image transformations, image segmentation and analysis, rendering and saving metrics, figures and graphs. The most time-consuming stage was image processing and analysis pertaining to single-cell detection and cellular membrane identification, accounting for 65% of the total processing time. The second most tedious stage was metrics calculation (23% of the total processing time) (Table 1, Supplementary Table 3 and 4, and Supplementary Fig. 3).

Discussion

Comparison of the results with the literature

The aim of the presented research was the development of an algorithm enabling automated segmentation and analysis of in vivo TOPI-imaged RPE cells at the single-cell level.

To the knowledge of the authors, fully automated and user-independent in vivo RPE single-cell imaging, segmentation and analysis has not been achieved before, primarily because high-resolution RPE images were not available.

The most akin method presented in the literature was a protocol for the automated segmentation of RPE cells images obtained with adaptive optics SLO, developed by Rangel-Fonseca et al. [36, 37]. Some other authors presented semi-automatic protocols for the detection and segmentation of RPE cells based on the localisation of cone cell centres [38, 39], non-specific for the actual underlying RPE cells layer. Furthermore, unlike ours, none of the three previous methods proposes a fully automated system of vascular outline elimination from the image processing and analysis. Finally, our method discards the OoF region automatically.

The observed characteristics of the RPE layer and RPE single cells are comparable with previously published literature (Fig. 3 and Supplementary Table 3); however, the published research data on RPE statistics are all based on ex vivo or in vitro observations. It is thus interesting to compare the results between human in vivo and ex vivo/in vitro.

Fig. 3.

Spider graph comparison between literature and study obtained parameter ranges. (For the colour version, please refer to the online version of the article).

The observed median cellular density (3743 cells/µm²) was within the range reported previously in the literature (3000–5500 cells/µm²) [37, 40, 41]. Similarly, the median observed RPE cell area at 193 µm² is within the reported studies [21] (140–840 µm²). The mean number of neighbouring cells was 5.22. Since mature, confluent RPE cell colonies or layers, are commonly described in the literature as hexagonal cobblestone conformation, denoting six neighbours per RPE cell [6–8, 38, 42–44] (Supplementary Table 3), this result is somehow unanticipated. Further studies in healthy subjects and different retinal eccentricities need to be undertaken.

The observed mean eccentricity at 0.67 was 27% higher than in the literature [2]. The median circularity at 0.83 was in line with the previously published circularity of 0.74 and matching well with the theoretically ideal circularity of 0.84 shown by Bhatia et al. [7](Supplementary Table 3). To our knowledge, both CMD_CV and cell solidity were used for the first to characterise RPE cells. Hence, a comparison of our findings with previously published literature was not possible.

Strengths and limitations of the study

The presented image processing method is dependent on the high quality of the TOPI-obtained retinal images. Albeit all images are normalised and filtered, imaging is still dependent on various parameters, both intrinsic and extrinsic to the TOPI setup. Conditional external factors to the TOPI setup include morphological heterogeneity of eye dependent on visual impairment, age and illumination conditions.

The primary strength of the study is the fully automated analysis of the TOPI images, which allows for reproducibility and replicability of the results. Another strength of the presented image processing algorithm is a short analysis time, acceptable both for research and for clinical purposes (48 s per image). All 14 samples (from 11 individuals) were able to be analysed and provide metrics both at the cell and colony level.

The small sample size can somewhat decrease the generalisability of the findings regarding RPE cells characteristics. Moreover, as the whole study population was composed of healthy volunteers, the performance of the developed image processing pipeline on patients with degenerative diseases cannot yet be assessed.

Significance of the findings and suggestions for future research

To our knowledge, this work is the first fully automated image processing pipeline developed for high resolution TOPI-obtained RPE images both at layer-level and single-cell levels.

The presented results are a step towards the possible implementation of automated RPE cell analysis in clinical practice for diagnostics of several retinal diseases. The use of non-invasive, real-time and fully automated evaluation of the retinal cells can facilitate the work in both ophthalmological research and practice. Furthermore, the RPE cells’ assessment method proposed in this paper is time-efficient (median of 48 s per image). Finally, the analysis of the retinal cells’ morphology both on layer-level and single-cell level opens new paths for eye health assessment and follow-up.

In the future, associations between different morphological features of the retinal cells and the type of vision impairment should be assessed. Cross-sectional studies with larger study populations should be conducted to validate and assess the performance of the developed method in adapting to the variability of the in vivo RPE layer. Furthermore, to assess the system and analysis pipeline utility in the follow-up and surveillance measure, prospective cohort studies need to be undertaken.

Conclusions

In this study, a new fully automated image processing method for segmentation and analysis of high-resolution TOPI-imaged RPE cells was presented. It is the first report on the RPE single cells in vivo characteristics. The present study lays the groundwork for future research in the field of clinical ophthalmology, enabling characterisation and diagnostics of retinal diseases on the single-cell level.

Summary

What was known before

• Single cell RPE analysis is inexistent or extremely time-consuming.

• Hard to image and analyse in vivo RPE cells.

What this study adds

• First fully automated image processing method for segmentation and analysis of transscleral optical phase images of in vivo RPE cells.

• Analysis at the RPE layer and single-cell level.

• New metrics in the analysis of RPE cells.

Author contributions

FLCS: Study concept and design, analysis and interpretation of data, drafting the paper, critical revision of the paper for important intellectual content, statistical analysis, and technical and material support; TL: Acquisition of data and critical revision of the paper for important intellectual content; MK: Acquisition of data and critical revision of the paper for important intellectual content; LK: Critical revision of the paper for important intellectual content; FBC: Critical revision of the paper for important intellectual content and study supervision; CM: Critical revision of the paper for important intellectual content, obtained funding and study supervision.

Data availability

The image processing codes and datasets generated and or analysed during the current study are available from the corresponding author on reasonable request and subject to the ethical approvals in place and material transfer agreements.

Compliance with ethical standards

Conflict of interest

The authors TL, MK, FBC and CM are involved in a company (EarlySight SA) aiming at commercialising the TOPI technology.

Supplementary information

The online version of this article (10.1038/s41433-020-1036-4) contains supplementary material, which is available to authorized users.