Ability of novice clinicians to interpret high resolution optical coherence tomography for ocular surface lesions

Abstract

Purpose

To assess the ability of novice clinicians to use a commercially available high resolution anterior segment optical coherence tomography (HR-OCT) device to diagnose various lesions of the ocular surface and cornea.

Methods

Cross-sectional study. Twenty six black and white HR-OCT images were projected and clinicians were asked to determine whether the lesions represented ocular surface squamous neoplasia (OSSN) or another ocular surface pathology. A 20-minute instructional lecture was given on HR-OCT interpretation and the same 26 images were shown. The clinicians were asked to repeat their assessment of the lesions. Thirty four novice clinicians at the Bascom Palmer Eye Institute, Miami FL participated. A commercially available device (RTVue, Optovue, Fremont, CA) was specifically chosen for this study.

Results

The mean frequency of correct identification of the 26 lesions was 70% (SD 15%) prior to instruction and after a short lecture, the frequency of correct identification improved to 84% (SD 9%), p=0.002. Novice clinicians were more accurate in correctly determining that a lesion was not an OSSN (ruling it out as a diagnosis) over determining that a lesion was an OSSN (p=0.001). Some lesions (both OSSN and not OSSN), however, were more difficult to interpret than others.

Conclusion

This study demonstrated that all levels of novice clinicians can quickly improve diagnostic accuracy with a commercially available HR-OCT after a short training session.

Introduction

Ocular surface squamous neoplasia (OSSN) is a broad term that encompasses a spectrum of epithelial squamous malignancies from dysplasia to squamous cell carcinoma (SCC). Conjunctival intraepithelial neoplastic lesions (CIN) range from mild, moderate, severe dysplasia to carcinoma in situ.¹ Clinically, OSSN can have a plethora of presentations ranging from the classic papillary or leukoplakic lesions with characteristic tufts of abnormal blood vessels straddling the limbus to less obvious appearances such as opalescence on the cornea or chronic conjunctivitis. The wide range of appearances, and its coexistence in the setting of complex ocular surface disease such as limbal stem cell deficiency, ocular cicatricial pemphigoid or pterygia, can make diagnosis difficult in some situations.²

The gold standard for the diagnosis of OSSN remains histopathologic evaluation after an incisional or excisional biopsy.³ However, in the past three decades, technological advancements have allowed for alternative methods of diagnosis including impression cytology, confocal microscopy, and high-resolution anterior segment optical coherence tomography (HR-OCT).^{2, 4-10}

In particular, high-resolution optical coherence tomography (HR-OCT) can provide an “optical biopsy” of tissue, aiding in the diagnosis and follow-up of corneal and conjunctival lesions. ^{2, 4, 7-10} Several HR-OCT machines have been used to evaluate lesions including a custom build machine^{2, 4, 8, 10} and the commercially available RTVue.⁹ OSSN has been found to have a characteristic appearance on HR-OCT, with a thickened, hyper-reflective epithelium and an abrupt transition between the normal and affected epithelium.^{2, 4} Using epithelial thickness value of 142 microns or greater, Kieval et al were able achieve 100% specificity and 94% sensitivity for differentiating between OSSN and pterygia on a custom built machine.⁴ Similarly, using epithelial thickness value of 120 microns, Nanji et al were likewise able to achieve 100% specificity and 100% sensitivity for differentiating between OSSN and pterygia on the commercially available RTVue.⁹ In both of these studies, two ocular surface oncology experts reviewed all images to assess the presence or absence of OSSN by HR-OCT and the results were confirmed with biopsy.

A remaining question is the general applicability of this technology to other eye care professionals. The purpose of this study was to determine whether novice clinicians at various levels of training could accurately use this technology to discern between OSSN versus other lesions of the cornea and conjunctiva on a commercially available device and study the impact of a short teaching lecture on the accuracy of diagnosis.

Materials and Methods

Study Population

Thirty four novice clinicians at the Bascom Palmer Eye Institute. Level of training included optometry students and residents (n=12), first, second, and third year ophthalmology residents (n=5, 5, 4, respectively), and ophthalmology fellows (n=6). Two clinicians did not disclose their level of training. This study was exempt from IRB review as participation was voluntary and no identifying information was collected from the participants.

Process

Twenty six HR-OCT images were printed on paper, 1 image per sheet, and given to all participants. Of note, no color photographs of the lesions were provided. All lesions had a biopsy proven diagnosis. Twelve lesions were biopsy proven to be OSSN and 14 were non OSSN lesions consisting of anterior basement membrane dystrophy (1), pterygia (3), nevus (2), primary acquired melanosis (2), lymphoma (2), lymphoid hyperplasia (1), melanoma (2), and Salzmann's nodular degeneration (1).

Participants were first asked to identify whether or not a lesion was or was not an OSSN. The next question was to identify what the lesion was, if not an OSSN. After all had recorded their answers, a 20-minute lecture (CLK) was given on the characteristics of all lesions described above. After the lecture, participants were asked to re-review the images and asked to again identify whether the lesions were or were not an OSSN, and re-identify the lesions. These responses were recorded and answer sheets were collected anonymously. Grading of the answer sheets was done by an independent individual with no affiliation to the study.

Statistical Analysis

All statistical analyses were performed using SPSS Version 22 (SPSS, Chicago, IL, USA) statistical package. Descriptive analyses were first performed and reported. Analysis of variance (ANOVA) statistics were used to evaluate performance by level of training. Improvement in diagnosing lesions on HR-OCT after the lecture (compared to before) was evaluated using paired t-test methodology. A p-value less than 0.05 was considered significant in the analysis.

Results

Ability of novice clinicians to correctly diagnose ocular surface lesions by HR-OCT

Overall, the initial mean frequency of correct identification of the 26 lesions was 70% (standard deviation (SD) 15%, minimum 35%, maximum 100%). This improved to 84% after the single 20 minute lecture. (p<0.002).

In the 12 lesions that were pathologically diagnosed as OSSN, the mean frequency of correct identification was 62% (SD 22%) pre-lecture and 76% (SD 15%) post lecture, p<0.0005. (Table 1) In the 14 lesions that were pathologically determined not to be OSSN, the mean frequency of correctly determining that the lesion was not an OSSN (ruling it out) was 76% (SD 15%) pre-lecture and 89% (SD 12%) post lecture, p<0.0005. Novice clinicians were more accurate in correctly determining that a lesion was not an OSSN over determining that a lesion was an OSSN (Chi square comparison of pre- and post-test results p=0.001).

Table 1. Ability of novice clinicians to correctly diagnose or rule out ocular surface squamous neoplasia (OSSN) by HR-OCT alone.

	Pre-test	Post-test	P-value*
Correctly diagnose OSSN	62% (22%)	72% (15%)	<0.0005
Correctly rule out OSSN	76% (15%)	89% (12%)	<0.0005

^*paired t test comparison of post and pre-test results by lesion (OSSN and not OSSN)

OSSN lesions

Within the 12 OSSN lesions, some images were deemed by the oncology experts (AG, CLK) to be classic for OSSN, with the images clearly depicting hyper-reflective, thickened epithelium and an abrupt transition from normal to abnormal (Figure 1) and others were more challenging. Three of the images were deemed easy/classic (25%), 3 as intermediate (25%), and 6 as challenging (50%) to interpret. In the easy example (Figure 1) depicting the classic features of OSSN, 79% of novice clinicians correctly identified the lesion as OSSN pre-lecture and 100% did so post-lecture. In other less classic images thought to be of intermediate difficulty, however, the lesions depicted most but not all features of OSSN. For example, in Figure 2, the abrupt transition is slightly more subtle than it was in Figure 1 and in this case, 68% correctly identified the lesion as OSSN pre-lecture and 74% did so post-lecture. Some lesions were deemed as challenging to interpret by CLK and AG, (Figure 3) and in this case, the frequency of identification was lower, with only 55% of clinicians correctly identifying the lesion as OSSN pre-lecture and 71% post-lecture.

Figure 1.

A classic, easy image of an OSSN on HR-OCT. All features of OSSN (epithelial thickening, hyperreflectivity, and abrupt transition on both sides of the lesion (white arrows) from normal to abnormal epithelium) are seen.

Figure 2.

An OSSN on HR-OCT with intermediate difficulty to interpret (some but not all classic findings). Note shadowing of inferior epithelial border (asterisk) and that the transition from normal to abnormal epithelium is not as distinct as in Figure 1. The transition from normal to lesion is rather distinct on the corneal side (white arrow), but less defined on the conjunctival side (black arrow).

Figure 3.

A challenging image of OSSN on HR-OCT. Note the thickened multifocal areas of hyperreflective epithelium which has variable thickness in different areas (white arrows). There is no distinct transition zone as all of the corneal epithelium is involved to some degree.

Non OSSN lesions

Within the 14 lesions that were not OSSN, 3 were thought to be easy (21%) to rule out by oncology experts (CLK, AG), 5 intermediate (36%), and 6 challenging (43%). For example, novice clinicians were easily able to discern the image of map dot fingerprint basement membrane dystrophy (Figure 4) as being distinct from OSSN (97% correctly pre-lecture, 100% correctly post-lecture). Others were less clearly differentiated. For example, 53% correctly identified a pterygium lesion (Figure 5) as not being OSSN pre-lecture and 88% did so after the lecture. In a similar manner, 41% correctly identified conjunctival melanoma (Figure 6) as not being OSSN pre-lecture and 53% did so after the lecture.

Figure 4.

An easy, classic appearance of map dot fingerprint basement membrane dystrophy on HR-OCT. Note hyperreflective, multi-focal sub-epithelial changes (arrows), with normal overlying low epithelial reflectivity eminating from Bowman's layer.

Figure 5.

Pterygium on HR-OCT. Note hyperreflective, sub-epithelial changes (black arrow). The overlying epithelium is thin, but in some areas may be hypo-reflective (white arrow).

Figure 6.

Malignant melanoma on HR-OCT. Note hyperreflective, sub-epithelial changes (asterisk) with vascular channels and normal or slightly thickened, hyper-reflective overlying epithelium (arrows).

Evaluation by level of training

There were significant differences in frequency of correct identification by level of training. On the pre-test, optometrists were less likely to correctly differentiate OSSN from non-OSSN compared to the other 3 groups of first, second, third year residents and fellows. (Chi square comparison of pre-test results between groups, p<0.0005).

Improvement in ocular surface lesions diagnosis by HR-OCT after the lecture

As above, there was significant improvement in test scores after the lecture. Post-lecture, the frequency of correct identification improved to 84% (SD 9%), p=0.002 in our novice clinicians. Interestingly, after the lecture, there was no longer a significant difference in accuracy by level of training (p>0.05 for all subgroup comparisons by least significant difference (LSD) analysis).

Discussion

High-resolution optical coherence tomography is a powerful tool in the evaluation of the ocular surface and can provide an “optical biopsy” of tissue, aiding in the diagnosis of corneal and conjunctival lesions. There are now several commercially available devices that are able to evaluate the ocular surface. While these devices are already in the physician's office and are used to evaluate for posterior segment pathology, they are under-utilized in the diagnosis and management of ocular surface diseases. Our study demonstrated that the commercially available RTVue HR-OCT helped novice physicians identify various lesions of the ocular surface with a relatively short learning course. In fact, our study demonstrated a statistically significant improvement in the accuracy of diagnosis of ocular surface lesions after a single 20-minute lecture. Seventy percent of novice clinicians (SD 15%) were able to use the HR-OCT alone to correctly identify 26 lesions as being or not being OSSN before the lecture and this improved to 84% (SD 9%), p=0.002. This suggests that minimal training can quickly allow all levels of clinicians to utilize this technology to evaluate corneal and conjunctival lesions.

We found, however, that not all lesions were easy to assess. The most challenging was diagnosing OSSN when not all classic OCT features were present. Specifically, when classic features of OSSN by HR-OCT were present, such as thickened, hyper-reflective epithelium and a clear transition point of normal and abnormal epithelium, our novices were best able to diagnose OSSN on HR-OCT. However, lesions that had a more subtle transition point between normal and abnormal epithelium, more indistinct epithelium to sub-epithelial junctions, and a thinner hyper-reflective epithelium were more difficult for the novices to accurately diagnose by HR-OCT alone. In addition, our novice readers had difficulty differentiating malignant melanoma from OSSN.

Overall, however, the result for our novice volunteers interpreting HR-OCT images with accuracy of 84% (SD 9%) compares favorably to other technologies used to aid in the diagnosis of OSSN. Impression cytology has been reported have a 70-80% sensitivity in diagnosing OSSN^{11, 12}, while a more recent study reported a 39% sensitivity and a 67% specificity using confocal microscopy.¹³

Limitations to this study include a relatively small sample size and that only selected diagnoses were used in each pathology group. Another limitation of this study is that we displayed the same set of images before and after the learning session. As such, it is possible that the improvement in scores was due to learning bias and not only based on knowledge gained from the lecture. Future studies can discriminate between these two possibilities by displaying different image sets of comparable difficulty before and after the learning session.

The trainees had variable pre-lecture experience in evaluating OCT images as some had spent time in clinics where this technology is currently being used. In this study we purposely did not provide a color image of the lesion, something that clinically the physician would have seen. This was to test for the pure OCT evaluation ability. Future studies will need to replicate and expand on our findings, with the inclusion of a larger sample of physicians, a broader range of lesions, a wider range of training levels, and evaluate the effect of adding a clinical photograph.

This is the first study to demonstrate that novice clinicians can utilize commercially available HR-OCT technology, to differentiate OSSN from non-OSSN conjunctival lesions. This study is important as it demonstrated that all levels of novice clinicians can quickly improve diagnostic accuracy with HR-OCT after a short training session. As more clinicians learn to use this technology, they can more fully utilize the capabilities of their existing devices as a helpful adjunct in the management of ocular surface lesions.