Prevalence and Severity of Artifacts in Optical Coherence Tomographic Angiograms

This cross-sectional study evaluates operational- and patient-related artifacts present in optical coherence tomography angiograms of eyes with diabetic retinopathy.

Key Points

Question

What is the frequency of artifacts in optical coherence tomographic angiography images, and how are these artifacts identified?

Findings

In this cross-sectional study that included 406 optical coherence tomography angiography images of eyes with diabetic retinopathy, the prevalence of artifacts was 53.5%, with shadow, defocus, and movement seen as the most common artifacts. Proprietary quality indices commonly used for identifying unreliable images had a high sensitivity (99%) but low specificity (37% to 41%).

Meaning

Results of this study suggest that knowledge of optical coherence tomographic angiography artifacts may be important in accurately interpreting these images for patient care and in clinical trials.

Abstract

Importance

Artifacts can affect optical coherence tomographic angiography (OCTA) images and may be associated with misinterpretation of OCT scans in both clinical trials and clinical settings.

Objectives

To identify the prevalence and type of artifacts in OCTA images associated with quantitative output and to analyze the role of proprietary quality indices in establishing image reliability.

Design, Setting, and Participants

This cross-sectional study evaluated baseline OCTA images acquired in multicenter clinical trials and submitted to the Fundus Photograph Reading Center in Madison, Wisconsin, between January 1, 2016, and December 31, 2018. Images were captured using the 3 mm × 3 mm and/or 6 mm × 6 mm scan protocol with commercially available OCTA systems. Artifacts, including decentration, segmentation error, movement, blink, refraction shift, defocus, shadow, Z offset, tilt, and projection, were given a severity grade based on involvement of cross-sectional OCT and area of OCT grid affected.

Main Outcomes and Measures

Prevalence and severity of OCTA artifacts and area under the receiver operating characteristic curve (AUC) of quality indices with image reliability.

Results

A total of 406 OCTA images from 234 eyes were included in this study, of which 221 (54.4%) were 6 mm × 6 mm scans and 185 (45.6%) were 3 mm × 3 mm scans. At least 1 artifact was documented in 395 images (97.3%). Severe artifacts associated with the reliability of quantitative outputs were found in 217 images (53.5%). Shadow (26.9% [109 images]), defocus (20.9% [85 images]), and movement (16.0% [65 images]) were the 3 most prevalent artifacts. Prevalence of artifacts did not vary with the imaging system used or with the scan protocol; however, the type of artifacts varied. Commercially recommended quality index thresholds had an AUC of 0.80 to 0.83, sensitivity of 97% to 99%, and specificity of 37% to 41% for reliable images.

Conclusions and Relevance

Findings from this study suggest that artifacts associated with quantitative outputs on commercially available OCTA devices are highly prevalent and that identifying common artifacts may require correlation with the angiogram and cross-sectional OCT scans. Knowledge of these artifacts and their implications for OCTA indices appears to be warranted for more accurate interpretation of OCTA images.

Introduction

The recent introduction of optical coherence tomographic angiography (OCTA) has enhanced understanding about retinal vascular morphologic structures. With computer-generated algorithms for identifying flow in retinal tissue, this commercially available technology allows for both qualitative and quantitative assessments of blood flow in various retinal layers.^1,2,3 Moreover, OCTA provides visualization of the deep capillary plexus, which previously could not be seen with fundus fluorescein angiography.^1,4 The ability of OCTA to image retinal vasculature with quantitative output has generated research in multiple areas, such as diabetic retinopathy, in which OCTA has been used for preclinical detection and disease staging.^4,5,6 However, artifacts may have implications for the quantitative outputs of commercially available OCTA, challenging the interpretability and reproducibility of proposed surrogate end points for clinical trials.^7,8

Artifacts are an inherent challenge for any clinical imaging modality. Specifically, previously described OCTA artifacts included defocus, shadow, movement, segmentation error, tilt, and projection.^9,10,11,12 Advances in the OCTA algorithm have improved the artifacts related to projection and motion.¹³ However, many artifacts such as defocus and shadow remain, which can substantially change the quantitative outputs of OCTA.^14,15 The occurrence and severity of artifacts in spectral-domain optical coherence tomography (SD-OCT) have been shown to increase with the pathological condition and to vary between OCT devices.^16,17,18,19 To date, limited reports exist on the prevalence of OCTA artifacts among patients with diabetic retinopathy.^8,20 Understanding the prevalence, severity, and origin of OCTA artifacts since the development of projection-artifact removal algorithms may help guide improvements in OCTA. To date, no objective scoring system of OCTA images is available that incorporates multiple artifacts and demonstrates the use of algorithm-generated quality indices.

The aim of the present study was to assess the prevalence and severity of technical-related and patient-related artifacts on OCTA images, which were acquired for clinical trials and analyzed at the Fundus Photograph Reading Center in Madison, Wisconsin.

Methods

This cross-sectional study analyzed OCTA scans of untreated eyes with diabetic retinopathy, which were submitted by clinical trials for analysis to the Fundus Photograph Reading Center in Madison, Wisconsin, between January 1, 2016, and December 31, 2018. Certification of a clinical site technician or photographer was required before the acquisition of these OCTA scans. This certification process involved training the technician on a detailed protocol for scan parameters and reviewing sample images acquired according to the protocol on deidentified, non–study participants. The research adhered to the principles of the Declaration of Helsinki²¹ and the Health Insurance Portability and Accountability Act of 1996. Institutional review board approval was granted by the University of Wisconsin. Written informed consent was obtained from all study participants.

Imaging Protocol

Images were submitted by multiple clinical sites that used commercially available OCTA systems (either the CIRRUS HD-OCT 5000, with the AngioPlex module, version 10.0.0.13424 or the Optovue Avanti RTVue XR, version 2018.0.0.5). Submissions included scans in the size of 3 mm × 3 mm and/or 6 mm × 6 mm. If paired 3 mm × 3 mm and 6 mm × 6 mm scans were submitted for an eye, both scans were graded. The submitted Optovue scans comprised 304 A-scans by 304 B-scans for both the 3 mm × 3 mm cube and 6 mm × 6 mm cube as well as 400 A-scans by 400 B-scans for high-definition 6 mm × 6 mm. The Zeiss submissions included 245 A-scans by 245 B-scans for 3 mm × 3 mm and 350 A-scans by 350 B-scans for 6 mm × 6 mm.

Image Assessment

The OCTA scans were reviewed in their respective proprietary software (Optovue review software, version 2018.0.0.5, and Zeiss AngioPlex, version 10.0.0.13424). Proprietary quality index was documented from Optovue images using signal quality (SQ 1 to 10) and signal strength index (SSI 1 to 100). For Zeiss images, image quality was noted using signal strength (SS 1 to 10). The images were analyzed independently for presence and severity of artifacts by 2 trained and certified graders (I.C.H. and M.S.K.). When a discrepancy arose between the 2 graders, another senior grader (J.W.P.) served as the adjudicator. Images were assessed for the following artifacts: decentration, segmentation error, movement, blink, refraction shift, defocus, shadow, Z offset, tilt, and projection. The graders reviewed the B-scans and angiograms using the cross lines available in the software for colocalizing enface images with B-scans.

Each artifact was graded on a severity scale of 0 to 3. A 0 indicated that no artifact was present. Mild severity, graded as 1, indicated that less than 1% of the B-scans within the OCT grid had an artifact present. Moderate severity, graded as 2, indicated that 10% of the B-scans of the outer subfield, but less than 10% of the inner or center subfield, showed an artifact. A grade of 3 meant that the artifact occurred in more than 10% of the OCT B-scans within the inner or center subfield of an OCT grid. This severity scale is similar to guidelines used for SD-OCT evaluation. Artifacts were categorized as being primarily technically related (operator, device, or algorithm dependent), patient-related, or both (Figure 1 and Figure 2; eFigure 1 in the Supplement). Following are descriptions of the artifacts and their severity (eTable 1 in the Supplement).

Figure 1. Patient-Related Artifacts .

Hot colors on the vascular density map are higher density. Shadow artifact (top panels): A vitreous floater obscures vessels in the superior field (arrowhead). Movement artifact (bottom panels): Eye movement has resulted in repeated fovea and decreased vascular density (arrowheads show movement lines).

Figure 2. Technical-Related Artifacts.

Decentration artifact (top panels): Fovea is shifted superiorly with loss of more than 10% of inner subfield as shown by the arrowhead. Tilt artifact (bottom panels): Symmetric decrease in vascular density on temporal half of the angiogram and map (arrowheads) is shown.

Decentration occurs when the scan is not centered on the macula and the OCT grid is repositioned. For this study, decentration artifact was considered severe if the central or inner subfields were outside the OCT grid after repositioning.

Segmentation error artifact occurs when the algorithm-generated segmentation line is classified as an error if the line deviates by more than 50% of the thickness of the pertinent plexus. A grade of 3 was given by our graders if more than 10% of the inner or center subfield was affected and the segmentation error artifact could not be manually corrected.

Eye movement artifact is defined as 1 or more of the following: thin vertical or horizontal white lines over the angiogram in conjunction with interruption, displacement, doubling or ghosting of vessels, and/or quilting defect. Eye movement can result in missing areas of the retina as well as duplicated areas of the retina in the scan.

Defocus artifact refers to the decrease in reflective intensity of the entire B-scan and global loss of small capillary vessels on angiogram. The severity of defocus artifact was graded according to the sharpness of vascular plexus on the angiogram, compared with standard images in the grading protocol, and the reflectivity of retinal layers on cross-sectional OCT.

Blink artifact is identified when consecutive B-scans are signal-void (black) and the angiogram has a corresponding horizontal black band indicating missing scans. A refraction shift artifact is a change in reflective intensity between contiguous OCT scans owing to blinking and a change in refractive index on the corneal surface. A shadow artifact has decreased intensity of retinal layers in isolated areas, often owing to vitreous floaters or corneal opacities.²²

Z offset artifact is characterized by a cross-sectional OCT scan vertically displaced in the OCT window owing to a faulty head placement (also termed out of window).²⁰ The severity of Z offset error was determined by the number of B-scans within the OCT grid showing a portion of the retina displaced out of the cross-sectional OCT window.

Tilt artifact is identified by only one-half of the cross-sectional OCT scan being in focus; this artifact occurs because of a severe angle of incidence, head placement, and/or high myopia. Severe tilt artifact was ascertained by the presence of the Henle fiber layer on the cross-sectional OCT along with unilateral vessel dropout on the angiogram. Mild and moderate artifacts were identified by the presence of the Henle fiber layer without asymmetric vascular dropout on the angiogram.

The projection artifact is identified by assessing the deep, avascular, and choriocapillaris layers for artifactual vessels projected from superficial layers.¹³ In this study, the commercial algorithm used for severity analysis had the ability to reduce the effect of superficial vessel shadowing.

Statistical Analysis

All data were analyzed with descriptive statistics (ie, percentages) and χ² on Microsoft Excel (Microsoft Corp). All tests were 2-sided, and no adjustment was made for multiple analysis. Receiver operating curves were generated and sensitivity analysis was performed using R software, version 3.5.1 (R Project for Statistical Computing). Two-sided P < .05 was considered statistically significant. Data analysis was performed from March 26, 2019, to April 19, 2019.

Results

A total of 406 OCTA images from 234 eyes were included in this study, including 221 (54.4%) 6 mm × 6 mm scans and 185 (45.6%) 3 mm × 3 mm scans. Of the 234 eyes, 172 (73.5%) had paired 3 mm × 3 mm and 6 mm × 6 mm images. The cumulative prevalence of any artifact was 97% (n = 395), and the individual prevalence was as follows: eye movement, 93.1% (378); defocus, 74.9% (304); shadow, 62.3% (253); tilt, 50.5% (205); Z offset, 43.8% (178); refraction shift, 31.8% (129); segmentation error, 24.6% (100); decentration, 21.4% (87); and projection, 6.7% (27). Artifacts previously described as blink, stretch, edge duplication, or loss of signal were present in less than 1% of all images. Only 11 images (2.7%) had no artifacts present. Figure 3 represents the distribution of mild or moderate and severe artifacts across all eyes. As shown in the Table, mild or moderate eye movement was the most prevalent artifact, present in 77.1% (313) of the 406 images.

Figure 3. Percentage of No, Mild or Moderate, and Severe Artifacts in All Images (n = 406).

Table. Overall Prevalence of Artifacts and Severity Between Scan Sizes.

Artifact	Mild or Moderate			Severe
	No. (%)		P Value	No. (%)		P Value
	3 mm × 3 mm (n = 185)	6 mm × 6 mm (n = 221)		3 mm × 3 mm (n = 185)	6 mm × 6 mm (n = 221)
Movement	133 (71.9)	180 (81.4)	.02	43 (23.2)	22 (9.9)	<.001
Defocus	90 (48.6)	129 (58.4)	.051	30 (16.2)	55 (24.9)	.03
Shadow	40 (21.6)	106 (48.0)	<.001	39 (21.1)	68 (30.8)	.03
Tilt	75 (40.5)	105 (47.5)	.16	1 (0.5)	24 (10.9)	<.001
Z offset	43 (23.2)	120 (54.3)	<.001	14 (7.6)	1 (0.5)	<.001
Refraction shift	54 (29.2)	62 (28.1)	.80	6 (3.2)	7 (3.2)	.97
Segmentation error	29 (15.7)	61 (27.6)	.004	3 (1.6)	7 (3.2)	.32
Decentration	15 (8.1)	36 (16.3)	.01	32 (17.3)	4 (1.8)	<.001
Projection	6 (3.2)	21 (9.5)	.01	0	0	NA
Total	178 (96.2)	216 (97.7)	.37	105 (56.8)	112 (50.7)	.22

Abbreviation: NA, not applicable.

Of the 406 OCTA images, 217 (53.5%) had severe artifacts that appeared to be associated with unreliable vascular density (VD) or foveal avascular zone (FAZ). Of these 217 images, 129 (59.4%) had only 1 severe artifact and 88 (40.6%) were graded as having more than 1 severe artifact in the same image. The most prevalent severe artifacts were shadow (26.9% [109]), defocus (20.9% [85]), and movement (16.0% [65]). Segmentation error artifact occurred only when accompanied by other severe artifacts. Projection artifact never appeared to be associated with ungradable images either alone or in conjunction with other artifacts. This finding was ascertained from examining the deep and superficial plexus angiograms independently; vessels present in the superficial plexus were not present in the deep plexus of any image. Percentage agreement between graders on the presence of severe artifacts was 89.4% (363).

No difference was found in the overall prevalence of severe artifacts between 3 mm × 3 mm scans and 6 mm × 6 mm scans (56.8% vs 50.7%; χ² = 1.495; P = .22). However, the type and severity of artifacts varied by the size of the scans (Table). Severe decentration (32 [17.3%] vs 4 [1.8%]; P < .001), movement (43 [23.2%] vs 22 [9.9%]; P < .001), and Z offset (14 [7.6%] vs 1 [0.5%]; P < .001) artifacts were more prevalent in 3 mm × 3 mm scans compared with the 6 mm × 6 mm scans. In contrast, severe defocus (30 [16.2%] vs 55 [24.9%]; P = .03), shadow (39 [21.1%] vs 68 [30.8%]; P = .03), and tilt (1 [0.5%] vs 24 [10.9%]; P < .001) artifacts were more prevalent in 6 mm × 6 mm scans compared with the 3 mm × 3 mm scans. Paired 3 mm × 3 mm and 6 mm × 6 mm scans were available for 172 eyes. Of these, 68 eyes (39.5%) were graded as unreliable in both scans, and 46 eyes (26.7%) had 1 reliable and 1 unreliable scan. Among paired scans, 95 (55.2%) of the 3 mm × 3 mm scans were unreliable, and 87 (50.6%) of the 6 mm × 6 mm scans were unreliable.

The prevalence of artifacts in images was comparable between the Zeiss and Optovue OCTA systems. However, variations in the type of artifacts were found between the 2 systems. Of the 187 OCTA images captured on the Zeiss system, 91 (48.6%) had at least 1 severe artifact, with shadow (48 [25.7%]) and movement (37 [19.8%]) accounting for most of the artifacts. Of the 219 Optovue images, 126 (57.5%) had at least 1 severe artifact, with defocus (63 [28.7%]) and shadow (59 [26.9%]) being the most common artifacts.

The mean (SD) SSI on Optovue OCTA was 59.37 (11.97), and the mean (SD) SQ was 6.39 (2.08). On Zeiss OCTA, the mean (SD) SQ was 7.81 (1.94). Receiver operating characteristic curves were generated from signal indices of both the Optovue and Zeiss systems. Signal indices for sensitivity to reliable images and specificity to unreliable images had an area under the receiver operating characteristic curve of 0.80 for Optovue SSI, 0.83 for Optovue SQ, and 0.81 for Zeiss SQ (Figure 4). Setting the quality thresholds below the mean resulted in an Optovue SSI of 55, sensitivity of 97%, and a specificity of 46%; an Optovue SQ of 6, sensitivity of 99%, and specificity of 41%; and a Zeiss SQ of 7, sensitivity of 99%, and specificity of 37% (eTable 2 in the Supplement).

Figure 4. Receiver Operating Characteristic (ROC) Curves of Signal Strength Index (SSI) or Signal Quality (SQ) With Sensitivity to Reliable Optical Coherence Tomographic Angiography Images and Specificity to Unreliable Images.

The ROC curves were generated from signal indices of both Optovue system (A and B) and Zeiss system (C). The area under the receiver operating curve was 0.80 for A, 0.83 for B, and 0.81 for C.

Discussion

The prevalence and severity of imaging artifacts have been well described on time-domain OCT and SD-OCT. Frequency of artifacts on SD-OCT has been reported as high as 85% in eyes with retinal disease, and substantial variations exist between pathologies and imaging devices.^16,22,23 To date, minimal discussion has occurred on the quantitative implications of various artifacts given that OCTA is a newer technology. Most publications on OCTA have focused on the reproducibility of its quantitative measures, such as FAZ and VD, but have given little attention to its artifacts.^24,25,26 Few small studies have addressed the prevalence of artifacts associated with the reliability of quantitative outputs such as VD and FAZ.^8,20,27 The present study suggests that the prevalence of artifacts associated with the reliability of quantitative outputs from commercially available OCTA may be as high as 53.5% in patients with diabetic retinopathy. OCTA imaging is increasingly being used in clinical trials. The ease of performing a dye-free angiogram to identify vascular changes, such as ischemia or neovascularization, is appealing. Until the system hardware and software are upgraded to enable the acquisition of better-quality images, OCTA assessments may need to be limited to exploratory outcomes in clinical trials.

Understanding the type of artifacts in OCTA may allow for improvements in image quality through algorithm manipulation, hardware upgrade, and technician training. The 3 most prevalent severe artifacts were shadow, defocus, and movement and can be reduced by technician training. Coaching patients before imaging, and improving the motion detection algorithms may reduce motion artifacts. In addition, training technicians to clear vitreous obstruction or increasing the power of OCTA devices may reduce the shadow artifact. Technician-related artifacts, such as defocus, may be easier to ameliorate by making technicians aware of global vascular dropout and poor reflectivity, which may improve reliability. The 3 mm × 3 mm scans appeared to be more prone to technician-related artifacts, such as decentration and Z offset, which could also be improved with training. Increasing the size of the acquisition window for the 3 mm × 3 mm scan would provide more room for centering the scan and grid placement. Segmentation error and projection artifacts were not severe enough to change the VD and FAZ outputs available on commercially available devices, a testament to the improvement of projection artifact removal algorithms.^9,13

Mild or moderate artifacts on OCTA are also frequently present in diabetic eyes. Mild or moderate eye movement was the most prevalent artifact, present in 77.1% of the 406 images. Say et al¹⁰ reviewed 65 images of normal control eyes and found blink lines in 88% of OCTA images. Enders et al²⁰ showed projection artifact in 100% of 16 OCTA images of eyes with diabetic retinopathy. These artifacts were rarely seen in the present study, probably owing to newer versions of imaging system software and projection artifact removal updates.¹³ We also did not see substantial stretch lines, edge duplication, or signal loss, which may be a result of improving software computation. Nevertheless, we noted that only 11 (2.7%) of the 406 images had no artifacts.

We found a substantial association of less known artifacts, such as refraction shift and tilt, with quantitative outputs. Previous studies described projection artifact in the Henle fiber layer when tilt was present but did not demonstrate quantitative output changes, and such research was published prior to projection artifact removal software.¹¹ We noted projection in the Henle fiber layer from the superficial plexus on either imaging device, and we also recorded a change of 13.1% of the foveal VD owing to tilt alone in a healthy eye (eFigure 2 in the Supplement). Although decentration and refraction shift are more easily noticed, tilt artifact is not as obvious. Technicians and clinicians should be trained to look for tilt artifact given its prevalence.

Limited research exists on the association between artifacts and quantitative outputs of OCTA. Yu et al¹⁵ showed that cataract removal significantly changes VD on swept-source OCTA, which has historically had lower prevalence of artifacts than SD-OCT.²⁸ Tomlinson et al¹⁴ also recently showed that defocus artifact by –3 diopters can change VD by 10.4%. However, these studies had small sample sizes, with the latter being only a single case. Although previous studies have assessed the repeatability of OCTA measurements in healthy eyes and their estimated value in diabetic retinopathy diagnosis, these studies have not addressed the implications that highly prevalent severe artifacts like shadow and defocus may have for these outputs.^4,26 If the prevalence of severe artifacts is as high as reported in the present study, and if further research corroborates the degree of association that artifacts have with quantitative outputs, clinicians should be cautious of making clinical decisions based on OCTA quantitative outputs in eyes with diabetic retinopathy.

Quality indices are an imperfect tool for discerning reliable and unreliable images. Previously used signal thresholds of SSI 55 on the Optovue software or SQ 7 on the Zeiss software provide high sensitivity (97% to 99%) for identifying reliable images.¹⁴ However, specificity at these thresholds remains relatively low (37% to 46%); thus, up to 63% of unreliable images that surpass this threshold may not be identified. Severe artifacts often are associated with OCTA measurements despite high signal indices, suggesting that signal indices cannot be used as sole indicators of image quality and that improvements are needed in algorithm calculation of signal indices. Consequently, we recommend that OCTA reliability assessment should include both the signal indices and a review of the angiogram and cross-sectional scan for common artifacts.

Strengths and Limitations

This study has multiple strengths and limitations. The OCTA images we obtained for this study included scans submitted by clinical trials and taken by certified technicians who adhered to quality guidelines and stringent protocol requirements. The true prevalence of artifacts in a real-world situation may be higher than what we have documented in this study. In addition, the evaluation of artifacts was performed by graders with extensive training on clear qualitative definitions of artifacts. Images for this study were taken between 2016 and 2018, when OCTA technology was still new to most clinical sites. The prevalence of artifacts may decrease as technicians become more experienced with OCTA. In addition, only patients with diabetic retinopathy were included. This study was not designed to address the magnitude of change associated with artifacts. Nevertheless, we believe the large sample size and objective classifications of severity provide useful information to help guide the improvement of OCTA imaging.

Conclusions

This study found that artifacts are common in OCTA images and can result in misinterpretation of images. Knowledge of the artifacts and their association with OCTA images, along with working toward improving the artifacts, may be necessary before these images can be used reliably for clinical purposes and as a clinical trial end point.

Supplement.

eTable 1. Description and Scoring System for Artifacts Seen in OCTA Images

eTable 2. Sensitivity and Specificity of Image Quality Indices

eFigure 1. Artifacts Due to a Combination of Patient Related and Operator Related Issues

eFigure 2. Change in Vascular Density With Defocus Artifact