Development of a scale for grading pre-plus and plus disease using retinal images: a pilot study

Abstract

When screening infants for retinopathy of prematurity, evaluating posterior pole vascular characteristics (ie, the presence or absence of pre-plus or plus disease) is subjective and prone to error. ROPtool, a semiautomated computer program, measures retinal vessel tortuosity and dilation. In this pilot study, we created an abbreviated pictorial scale of varying vascular tortuosity and dilation based on expert perception of vascular characteristics. We used ROPtool to evaluate the experts’ ability to arrange these images in order of increasing vascular tortuosity and dilation. ROPtool values confirmed successful arrangement of images in order of increasing vascular tortuosity and dilation. This pictorial scale could serve as a reference to decrease subjectivity when diagnosing pre-plus or plus disease.

Accurate classification of retinopathy of prematurity (ROP) is important to guide treatment to reduce risks of retinal detachment and blindness.^1,2 Plus and pre-plus disease are critical components of this classification. A standard photograph demonstrates the minimum arterial tortuosity and venous dilation required to diagnose plus disease.^2,3 Pre-plus is defined as posterior pole vascular abnormalities “insufficient for the diagnosis of plus disease but that demonstrate more arterial tortuosity and more venous dilatation than normal.”³ Plus disease is almost invariably present in treatment-requiring ROP, and pre-plus disease has been associated with eventual progression to treatment-requiring ROP.⁴ The diagnosis of plus and pre-plus disease is largely subjective,⁵ with experts demonstrating different thresholds of vascular abnormalities to trigger pre-plus and plus disease, likely accounting for their high disagreement rates.⁶

The objectives of our study were (1) to create an abbreviated pictorial scale to standardize the grading of pre-plus and plus disease in retinal images based on expert perception of images’ vascular tortuosity and dilation and (2) to evaluate how well the ordering of these images correlates with ROPtool measures of vascular tortuosity and dilation.

Materials and Methods

This study was approved by the Duke Health System Institutional Review Board and complied with regulations of the US Health Insurance Portability and Accountability Act of 1996. Using previously collected narrow-field (45°) retinal images of infants screened for ROP, we selected high quality color images representing a range of vascular dilation and tortuosity. We used these images to create an abbreviated pictorial scale for grading pre-plus and plus disease. The images were ordered based on degree of vascular tortuosity and dilation (Figure 1), based on a consensus of 3 fellowship-trained pediatric ophthalmologists (DKW, SFF, SGP) examining the images together.

FIG. 1.

Abbreviated pictorial scale of increasing vascular dilation and tortuosity created by consensus of 3 fellowship-trained pediatric ophthalmologists. Judgment of vascular dilation increases from left to right. Judgment of vascular tortuosity increases from bottom to top. The ROPtool measurements of both tortuosity index and dilation index are listed for each image.⁷ ROPtool measures tortuosity as the length of the traced vessel divided by a smooth curve generated from points spaced about 40 pixels apart on that vessel and dilation as the average vessel width over its length divided by the area of the optic nerve. The tortuosity and dilation indices are calculated by multiplying the raw tortuosity or dilation values by an adjustment factor so that indices of 10 represent the minimum abnormality necessary for plus disease, as defined by the standard photograph.

The images were then analyzed by a nonophthalmologist (DAN) for vascular characteristics of tortuosity and dilation using ROPtool (v2.1.8, FocusROP, Trumbauersville, PA). ROPtool measures tortuosity as the length of the traced vessel divided by a smooth curve generated from points spaced about 40 pixels apart on that vessel and dilation as the average vessel width over its length divided by the optic nerve’s area.⁷ The tortuosity and dilation indices are calculated by multiplying the raw values by an adjustment factor so that indices of 10 represent the minimum abnormality necessary for plus disease, as defined by the standard photograph.

We compared the arrangement of our pictorial scale to the ROPtool measures of tortuosity index and dilation index from the images. Specifically, we determined whether each index increased from bottom left to upper right, whether tortuosity index increased within each level of dilation, and whether dilation index increased within each level of tortuosity.

Results

Ten images were selected and arranged by increasing vascular dilation and tortuosity (Figure 1). Overall, the ROPtool measure of tortuosity index increased from the lowest to the highest level of perceived level of tortuosity, with overall values ranging from 1.5 to 13.3. Compared to the perceived level of tortuosity, tortuosity index increased within each level of vascular dilation in 10 of 10 (100%) images.

Overall, the ROPtool measure of dilation index increased from the lowest to the highest level of perceived level of dilation, with overall values ranging from 8.4 to 12.5. Compared to the perceived level of dilation, dilation index increased within each level of perceived vessel tortuosity in 9 of 10 images (90%).

Discussion

Overall, our abbreviated pictorial scale of vascular tortuosity and dilation correlated well with the ROPtool measures of tortuosity and dilation index. Whereas our scale correlated well with the ROPtool measure of tortuosity index at each dilation level, it correlated, but not as consistently, with the ROPtool measure of dilation index at each level of tortuosity. This discrepancy may derive from the fact that ROPtool can more accurately measure tortuosity versus dilation because (1) accurate measurement of dilation relies on ROPtool identifying blood vessel edges, and even mild image blur reduces accuracy, leading to over- or under estimation of dilation⁸; (2) ROPtool uses optic nerve area to calculate dilation, thus variation in optic nerve size can affect the calculation of dilation; and (3) as disease worsens, the range of tortuosity changes exceeds that of dilation. Thus, it may be easier to put tortuosity in the correct order compared to dilation.

Improving the accuracy of plus disease diagnosis could help ensure that treatment for severe ROP is timely and uniform among ROP examiners. One study concluded that diagnostic variation among experts is due to systematic biases, with some experts tending to overcall and others tending to undercall.⁶ The development of a continuous severity score of vascular tortuosity and dilation would more accurately demonstrate the range of vascular abnormalities in ROP.⁶

Our pictorial scale is limited by the number and range of vascular characteristics captured in the images available. A strength of the scale is that it was created with narrow-field images, with a field of view similar to that of indirect ophthalmoscopy. Most published retinal photographs depicting pre-plus and plus disease³ have a field of view wider than that of indirect ophthalmoscopy using a 20 D or 28 D lens. This difference may be one factor leading to discrepancies in diagnosing pre-plus and plus disease. By definition, both pre-plus and plus are diagnosed by considering only the posterior pole vessels.^2,3 Used at the bedside, our pictorial scale may help to improve the accuracy of plus and pre-plus diagnosis, as examiners identify the image that most closely resembles their clinical examination findings.

Our pictorial scale using narrow-field images shows good agreement between automated measurement and experienced examiner opinion. Referring to this scale when examining infants or images could improve the accuracy of plus and pre-plus diagnosis. Further validation should be performed having ROP examiners of all levels use our scale to categorize pre-plus and plus disease. This future work will determine whether our scale as currently configured can help to make pre-plus and plus disease diagnoses more uniform or whether a more comprehensive pictorial scale is needed.