Interexaminer agreement of gonioscopy examinations in dogs using the European College of Veterinary Ophthalmologists Hereditary Eye Diseases grading scheme

Abstract

Objective

Prospective observational study with preregistered study protocol to assess interexaminer agreement using the 2022 European College of Veterinary Ophthalmologists Hereditary Eye Disease (ECVO‐HED) gonioscopy grading scheme.

Animals Studied

Sixty client‐owned dogs presented for gonioscopy as part of the required certification process prior to breeding.

Procedure

Two ECVO diplomates with comparable clinical experience performed gonioscopy with a Koeppe gonioscopy lens and slit‐lamp biomicroscope at 10x magnification on all eyes in a randomized fashion.

Results

One hundred and eighteen of 120 eyes (60 dogs) were included. In 110/118 eyes (93%), both examiners provided the same breeding recommendation (yes/no). This translated into an agreement on the final breeding recommendation in 58/60 dogs (97%). Examiners disagreed on pectinate ligament abnormality (PLA) grading in 19/118 eyes and iridocorneal angle width (ICAW) grading in 9/118 eyes. PLA grading disagreement was mostly observed between neighboring grades. Disagreement between PLA and ICAW grading sometimes occurred within the same eye accounting for an observed disagreement in iridocorneal angle abnormality (ICAA) grading in 19/118 eyes in total. Cohen's kappa was Κ = 0.62 (95% TI 0.34–0.89), whereas maximum kappa was maxΚ = 0.82 (95% TI 0.59–1). Mixed model analysis suggested no significant examiner influence on breeding recommendations (OR 0.17; 95% CI 0.02–1.12, p = .09).

Conclusions

Although examiners disagreed on ICAA grading in 19/118 eyes, this resulted in a different breeding recommendation (yes/no) in eight eyes and in two dogs only. Therefore, the use of the 2022 ECVO‐HED gonioscopy grading scheme seems to result in examiners providing the same breeding recommendations in most cases.

1. INTRODUCTION

The European College of Veterinary Ophthalmologists Hereditary Eye Disease (ECVO‐HED) scheme is used uniformly across the European Union and in various neighboring countries by ECVO Diplomates and European Eye Scheme Examiners (ESE) to perform, score, and register the findings of eye examinations in breeding dogs. It provides guidelines to issue an eye examination certificate which is requested by many breed clubs to obtain approval for breeding. Breeding advice is derived from the ECVO‐HED manual. ¹

The ECVO‐HED manual provides a list of dog breeds with special concern for primary angle‐closure glaucoma (PACG), in which gonioscopy is strongly recommended prior to breeding. ² Enough scientific information is available to include PACG on the list of known or presumed hereditary eye diseases (KP‐HED) for the listed breeds. ³ , ⁴ , ⁵ , ⁶ , ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ Specific references for every breed can be found on the ECVO website. ²⁰ PACG is the most common form of primary glaucoma in dogs. ²¹ The disease is typically bilateral, with progression of the second eye to the onset of clinically overt glaucoma mostly within 1 year of the initial diagnosis in the first eye. ²² , ²³ If medical and surgical treatment is unsuccessful, blind and painful eyes may require surgical removal to provide an adequate level of comfort to the animal.

Gonioscopy, an examination technique using special gonioscopy lenses and a focal light source, allows the assessment of the anterior face of the iridocorneal angle (ICA) in dogs. ²⁴ The anterior face of the ICA consists of moderately interlacing fine strands of collagen fibers lined by a single cell layer of iris epithelial or corneal endothelial cells termed the pectinate ligament (PL). These fibrous pillars connect the iris base to the deep limbal pigment band, spanning the ciliary cleft. ²⁵ , ²⁶ Considerable variations in length, thickness, and pigmentation of the PL have been observed between different breeds, individuals, and quadrants of the same eye. ²⁴ , ²⁷ , ²⁸ Thickened and wide fibers, blending into solid sheets of tissue, with or without normal interfiber openings, have been classified as pectinate ligament dysplasia (PLD). ¹³ Evidence that the pectinate ligament conformation may change during life is substantial. ¹¹ , ¹⁴ , ²⁹ , ³⁰ The former terminology dysplasia has therefore been changed to pectinate ligament abnormality (PLA) and iridocorneal angle abnormality (ICAA). ³¹ , ³² , ³³ At the same time, gonioscopy allows the evaluation of variation in the ICA width (ICAW). ¹¹ , ¹⁶

Gonioscopy alone only evaluates the anterior face of the ICA, is merely one method of evaluating the ICA, and is limited in terms of predicting the risk for the development of glaucoma. ⁵ , ⁹ , ³⁴ , ³⁵ With the identification of susceptibility genes or loci, genetic testing has become available for primary open‐angle glaucoma (POAG) and PACG in a small subset of susceptible breeds. ⁶ , ³⁶ , ³⁷ , ³⁸ , ³⁹ , ⁴⁰ , ⁴¹ , ⁴² , ⁴³

High‐resolution ultrasound (HRUS) allows for a deeper assessment of the aqueous outflow structures including the ciliary cleft. ⁴⁴ Since HRUS has demonstrated poor interobserver reliability ⁴⁵ and is not routinely used as a screening tool prior to breeding, but gonioscopy is, this study is focused on the interexaminer agreement of gonioscopy examinations.

Several authors have described ICA narrowness with or without PLA as a possible risk factor in the development of PACG. ⁴ , ⁸ , ⁹ , ¹¹ , ¹² , ¹⁶ , ⁴⁶ Equally, PLA can contribute to PACG in dogs independent of ICAW abnormalities. ⁵ , ⁸ , ¹¹ , ¹² , ¹³ In some cases, a relationship between both entities can be seen. ¹¹ In the ECVO‐HED 2022 gonioscopy scheme, PLA and ICAW are scored as two separate parameters along the complete circumference of the ICA. The grades of these two parameters together provide the final ICAA grade, which determines the individual recommendation for breeding. ³¹ , ³²

Gonioscopy has often been judged as a subjective method to evaluate these findings. ¹¹ , ¹⁴ , ¹⁵ , ¹⁶ , ²⁹ , ⁴⁷ Through the years, several schemes have been applied to limit subjectivity and increase consistency between different examiners. ¹³ , ¹⁴ , ¹⁵ , ¹⁶ At present, the ECVO, the American College of Veterinary Ophthalmologists (ACVO), and the British Veterinary Association/Kennel Club/International Sheep Dog Society (BVA/KC/ISDS) use different schemes to grade and interpret gonioscopy results.

Interexaminer agreement is the extent to which two examiners assign the same score to the same subject. It is often quantified by using the observed agreement, which is calculated in percent, and can range from 0% to 100% of cases where examiners agreed. A certain amount of agreement will be reached simply by chance. Cohen's kappa (Κ) accounts for this expected chance agreement. ⁴⁸ In a perfectly balanced data set, in which the prevalence of the condition that is being scored approaches 50%, Κ ranges from 0 to 1. For a specific, disbalanced data set, Κ is reduced nonlinearly and can decline rapidly. This decline of Κ is not proportional to a decline in agreement between examiners, particularly when the disbalance is due to a very low or very high prevalence of the scored condition (e.g., a low number of dogs with severe ICAA in the study population). ⁴⁹ Hence, a low prevalence can result in a low Κ despite a high level of agreement between examiners. ⁵⁰ Such a change in Κ is unpredictable, which makes the interpretation of Κ difficult and setting‐dependent. To place Κ in perspective and gain a better understanding of the agreement between examiners, Dunn et al. suggested the use of maximum kappa (maxΚ) in addition to Κ. maxΚ represents the greatest achievable agreement between examiners for a specific data set. ⁵¹ , ⁵² Grading is influenced by individual tendencies (e.g., some examiners may be stricter than others). maxΚ accounts for those tendencies. When the difference between maxΚ and Κ is high, individual tendencies are a relevant source of disagreement. This information is useful to determine if variables such as the training received by, or the level of experience of the examiners might further affect interexaminer agreement.

Different gonioscopy schemes have been formally assessed regarding their level of agreement by Oliver et al., using Cohen's kappa. ⁴⁷ The application of a binary scheme (affected vs. nonaffected) yielded very good results. A scheme with three categories and with a large intermediate bandwidth also yielded a good level of agreement. Contrarily, the use of more complex grading systems only demonstrated a fair to moderate level of agreement, which was considered unacceptable. ⁴⁷ Currently, there is no study assessing the widely used ECVO‐HED gonioscopy scheme regarding its interexaminer agreement.

In this study, we aim to explore the interexaminer agreement of gonioscopy examinations in dogs using the ECVO‐HED 2022 gonioscopy scheme. We hypothesize that two equally experienced ECVO diplomates using the scheme will obtain a consistent ICAA grading and final breeding recommendation (yes/no).

2. MATERIALS AND METHODS

This is an observational prospective study, which was preregistered on https://preclinicaltrials.eu, where the study protocol details are available under registration number PCTE0000388. The study was granted full approval by the Federal Food Safety and Veterinary Office of Switzerland.

2.1. Study population

Convenience sampling was used. Client‐owned breeding dogs presented to the ophthalmology section at the Vetsuisse Faculty, University of Zurich from April to October 2023 were eligible for prospective enrollment into the study. Dogs between 1 year and 10 years of age were included if they needed a gonioscopy examination as a mandatory test for breeding approval, required by the respective breed club in Switzerland. Study participants were recruited by contacting various breed clubs, who informed their members via newsletter and email about the study aims and participation. A study handout was sent via email to interested owners prior to enrollment of their dogs into the study. The document is included as Appendix S2 (https://osf.io/rdbt9/?view_only=e1b1f6e711ca428fa5e6f950e9224c22) on the open science framework (https://www.osf.io/), which can be freely accessed. This document was signed by the owner to declare written consent prior to participation in the study. If examination of only one eye by two examiners was possible, the dog was still enrolled, but only the data for this one eye were evaluated. Animals that would have required sedation to prevent more than gentle manual fixation and to prevent repeated placement of the goniolens, in other words, to allow the gonioscopy examination by two examiners, were excluded from study participation.

2.2. Study flow

Each enrolled dog underwent gonioscopy, consecutively performed by two certified ECVO diplomates, each possessing approximately 20 years of experience in the field (AR and SP). Both examiners have regularly performed gonioscopy examinations throughout their 20‐year careers since they both perform gonioscopy not only as part of breeding exams but as part of the routine clinical workup in suspected primary glaucoma cases or cataract workups as well. To address the potential impact of patient compliance on gonioscopy results, both examiners were provided an equal opportunity to conduct initial examinations on patients. The sequence of examiners performing the first gonioscopy on the first eye was generated using R Studio. Either the right eye or the left eye was examined first, according to personal preference. Throughout the entire study, the examiners remained masked from each other's results to minimize observer bias, maintaining the independence of the gonioscopy results. As a result of masking, it was impossible to discuss the gonioscopy results from both examiners prior to presenting the results to the clients. Therefore, the study protocol dictated that the clients were consistently provided with the examination results of examiner 1.

2.3. Direct gonioscopy examination

After instillation of one drop of 0.4% oxybuprocaine (Novesin®, OmniVision GmbH), a Koeppe gonioscopy lens (Ocular Instruments Inc.®, Bellevue, WA 98004, United States) with a diameter of either 14 mm (OKES‐14), 17 mm (OKS), or 19 mm (OKL) was placed onto the eye with a carbomer coupling gel (Methocel 2% OmniVision GmbH® or AJL Visc 1.4%, AJL Ophthalmic). Selection of the lens size was dependent on the corneal diameter and size of the individual patient's lid fissure. Once a certain lens size was chosen by the clinician and proved to be a good fit for the patient, the lens size was not changed between examiners for the examination of that particular patient. The animal was gently manually restrained by a trained technician and examined by both examiners in a randomized order using a handheld slit‐lamp biomicroscope (Kowa; SL‐17; SL‐19) at 10× magnification. When the examiner who was assigned to go first had finished the first eye, the second examiner would examine the same eye without replacing the gonioscopy lens. The other eye was subsequently examined with the second examiner going first, leaving the gonioscopy lens in place for the first examiner to perform the examination on that eye. Efforts were made to avoid repeated goniolens placement. Both examiners evaluated the iridocorneal angle along its 360° circumference for pectinate ligament abnormalities and iridocorneal angle width. These two parameters, PLA and ICAW, were graded separately. The findings were assigned to the ECVO‐HED 2022 gonioscopy scheme separately by each examiner.

2.4. Pectinate ligament grading according to the ECVO‐HED 2022 gonioscopy scheme

According to the ECVO‐HED guidelines, fibrae latae are defined as fibers with a confluent (broad) base and shortened thin insertions at the cornea or as thicker than normal fibers consisting of a fusion of less than five pectinate ligament fibers. Laminae are defined as plates or sheets of continuous tissue, with or without flow holes, consisting of a fusion of more than five pectinate ligament fibers (Figure 1). PLA is categorized as unaffected, affected (mild), affected (moderate), or affected (severe), depending on the severity of and the amount of angle circumference affected by the abnormality. The division into grades is summarized in Table 1. To be able to perform the data analysis in this study, we assigned numerical scores 0–3 to the ECVO scheme grades (Table 1). The ICAW is evaluated by comparison of the length of the pectinate ligament (A) and the distance from the origin of the pectinate ligament to the inner corneal surface (B) (Figure 2). The ICAW is graded based on this ratio A/B and has been divided into three different grades, which were modified from the original publication by Ekesten and Narfström ¹⁶ as summarized in Table 2. To be able to perform the data analysis in this study, we assigned numerical scores 0–2 to the ECVO scheme grades (Table 2). The breeding recommendation results from the combination of PLA and ICA width, expressed as the ICA abnormality (ICAA) score, categorized as unaffected, affected (mild), affected (moderate), or affected (severe) (Table 3).

FIGURE 1.

ECVO‐HED 2022 gonioscopy scheme—pectinate ligament abnormality. Drawings adapted from ECVO‐HED scheme, original images not used. (A) Normal pectinate ligament depicted as thin fibers from origin at iris base to its insertion at the deep limbal pigment band, (B) fibrae latae depicted as broad‐based fibers with shortened thin insertions or thick fibers (<5 fibers), (C) laminae depicted as sheets of tissue (>5 fibers) with or without flow holes. ECVO‐HED, European College of Veterinary Ophthalmologists Hereditary Eye Disease.

TABLE 1.

ECVO‐HED 2022 gonioscopy scheme—PLA grading.

PLA	Grade	Breeding
0%–50% FL	Unaffected (0)	Yes
>50%–100% FL and/or <25% LA	Affected (mild) (1)	Optional
25%–50% LA	Affected (moderate) (2)	Optional
>50% LA	Affected (severe) (3)	No

Note: The PLA grade depends on the type and extent of the abnormality (FL vs. LA) across the 360° circumference of the ICA. In brackets are the numerical study scores assigned to the ECVO scheme grades. The grade is then translated together with the ICAW grade into the overall ICA abnormality grade which results in a final breeding recommendation.

Abbreviations: breeding, breeding recommendation; ECVO‐HED, European College of Veterinary Ophthalmologists Hereditary Eye Disease; FL, fibrae latae; ICA, iridocorneal angle; ICAW, iridocorneal angle width; LA, laminae; PLA, pectinate ligament abnormality.

FIGURE 2.

ECVO‐HED 2022 gonioscopy scheme—iridocorneal angle width. Drawings adapted from ECVO‐HED scheme, original images not used. (A) A ≥ ⅓ of B equals ICA open, (B) A < ⅓ of B equals ICA narrow, (C) A—pectinate ligament—not visible equals ICA closed. ICA, iridocorneal angle.

TABLE 2.

ECVO‐HED 2022 gonioscopy scheme—ICAW grading.

ICAW	Grade	Breeding
A ≥ ⅓ of B (open)	Unaffected (0)	Yes
A < ⅓ of B (narrow)	Affected (moderate) (1)	Optional
A—PL—not visible (closed)	Affected (severe) (2)	No

Note: The ICAW is determined by the ratio of A/B. A is the length of the pectinate ligament, determined by the distance from origin at iris base to its insertion at the deep limbal pigment band. B is the distance from the origin of the PL at the iris base to the inner corneal surface. In brackets are the numerical study scores assigned to the ECVO scheme grades. The grade is then translated together with the PLA grade into the overall ICA abnormality grade which results in a final breeding recommendation.

Abbreviations: breeding, breeding recommendation; grade, grading category; ICA, iridocorneal angle; ICAW, iridocorneal angle width; PL, pectinate ligament.

TABLE 3.

ECVO‐HED manual recommendations regarding gonioscopic findings in dogs.

ICAA grade	Status	ECVO recommendation
Normal (0)	Unaffected	Breeding
Mild (1), Moderate (2)	Affected	Breeding optional
Severe (3)		Breeding not recommended

Note: In brackets are the numerical study scores assigned to the ECVO scheme grades. Breeding: there are no restrictions when breeding with the animal. Breeding optional: If these dogs are used for breeding, they should be bred with unaffected individuals. Breeding not recommended: It is recommended not to breed with the animal.

Abbreviations: ECVO, European College of Veterinary Ophthalmologists; ICAA, iridocorneal angle abnormality.

2.5. Review of ECVO‐HED 2022 gonioscopy scheme material

Prior to conducting the study, the authors reviewed the ECVO‐HED 2022 guidelines and definitions available on the ECVO website (https://www.ecvo.eu/hereditary‐eye‐diseases/ecvo‐manual.html). The material was commented on and discussed in the group for a more comprehensive understanding of how to grade certain ICA findings. Key discussion points included the differentiation between fibrae latae and laminae as well as open, narrow, and closed angles.

2.6. Statistical analyses

Descriptive statistics were performed to summarize the collected data and explore differences in scoring between the two examiners. Based on the study by McHugh et al., we defined an observed agreement of 80% as acceptable, although no clear cutoff values for acceptable versus unacceptable agreement in health research exist. ⁵³ The analysis was assisted by graphs and tables to identify areas where differences in scoring between the two examiners were the most profound.

Cohen's kappa (Κ) and maxΚ were used to quantify interexaminer agreement. To allow for the analysis, ICAA scores from the ECVO‐HED 2022 scheme were transformed into a binomial variable. “Breeding—yes” included individuals with normal, mild, and moderately affected ICAA, while “Breeding—no” included individuals with severely affected ICAA. The breeding recommendations of the ECVO‐HED manual are depicted in Table 3. It is important to note that the applied dichotomy does not exist in practice, but that the “Breeding—yes” category includes animals for which the ECVO‐HED manual would provide the recommendations “Breeding” or “Breeding optional,” and that the “Breeding—no” category includes animals for which the ECVO‐HED manual would provide the recommendation “Breeding not recommended.” The final breeding recommendation is derived from the assigned PLA and ICAW grades. After having been scored independently, the more severe of these two (PLA and ICAW) grades determines the ICAA grade for the eye. For example, if the PLA of an eye has been graded as “affected (mild),” and the ICAW of that same eye as “affected (moderate),” the final ICAA grade for that eye would be “affected (moderate).” The most severely affected of the two eyes, which is the one with the higher ICAA grade, then determines the final breeding recommendation for an individual dog.

The selection of eyes for the calculation of Κ and maxΚ was performed as follows. According to our calculated sample size for the study, 56 eyes would be needed to provide a statistical power of 80% when calculating Κ. We therefore randomly selected 56 eyes, restricted to one eye per dog, from the complete data set, and calculated the Κ value and the corresponding maxΚ for these 56 eyes. To use all the available information from our data set of 118 eyes, this random sampling of 56 eyes was repeated 100 times, always calculating the Κ value and the corresponding maxΚ per random sample of 56 eyes (one eye per dog). The means of Κ and maxΚ were then calculated for these 100 random samples of 56 pooled eyes and assisted with 95% tolerance intervals (TI).

Finally, mixed effect logistic regression (mixed model) was used to assess whether the examiner has an impact on the breeding recommendations. This can be considered an alternative method to quantify interexaminer agreement. The use of the mixed model allowed us to account for the correlation between the two eyes of the same individual (random intercept). We also considered additional variables such as patient age and the order in which the eyes were examined. All analyses were performed in R version 4.0.5 (2021‐03031). ⁵⁴ The following packages were used for the analysis and data plotting: readxl, dplyr, ggplot2, gtsummary, DescTools, lme4, and irr. ⁵⁵ , ⁵⁶ , ⁵⁷ , ⁵⁸ , ⁵⁹ , ⁶⁰ , ⁶¹

2.7. Sample size calculation

The sample size calculation was performed on the wnarifin.github.io “kappa (2 raters)” platform and based on a binary variable with Cohen's kappa statistic, resulting in an n of 56 dogs in total. The statistical assumptions included: minimum acceptable kappa = 0.6, expected kappa = 0.9, proportion of outcome = 0.5, significance level = 0.05, and power = 80%, with no study drop‐out expected.

2.8. Deviations from the preregistered protocol

During the manuscript preparation, there were changes applied to the preregistered protocol. Change 1: Dogs presented to the clinic as primary glaucoma patients were excluded from the study population. Change 2: According to the predefined group allocations, normal and mild ICAA would be assigned into one group vs. moderate and severe ICAA into another group to allow for statistical analysis with Cohen's kappa. Since this allocation did not match the recommendations regarding “breeding—yes” vs. “breeding—no” in the ECVO‐HED 2022 manual, the groups were divided as described in Section 2.6.

3. RESULTS

A total 118 eyes of 60 dogs were examined. Two eyes (of two dogs) were excluded from further analyses since a lack of compliance made a complete examination of both eyes by both examiners impossible. The eye that was completely examined by both examiners was included in the statistical analyses. The most common breeds in the study population were the golden retriever (n = 14) and flat‐coated retriever (n = 13) (Table S1). The median age of the dogs was 4.4 years (IQR 2.0–7.4). Both examiners examined an equal number of eyes first (n = 59). While examiner 1 (Ex1) had the tendency to start with the left eye (OS, n = 37), this was not the case for examiner 2 (Ex2) who preferred to examine the right eye first (OD, n = 36).

3.1. Descriptive statistics

Both examiners agreed on the breeding recommendation in 110 of 118 eyes (93%; breeding—yes: n = 103; breeding—no: n = 7) and disagreed in 8 of 118 eyes. Ex1 classified 13 eyes (7 OS and 6 OD) as “breeding—no,” whereas Ex2 assigned 9 eyes (2 OS and 7 OD) to the same category (Figure 3D,H). Disagreement between the two examiners was more common in OS.

FIGURE 3.

Iridocorneal angle grading results and breeding recommendations. The bar plots represent the number of eyes in each grading category. Ex1, Examiner 1; Ex2, Examiner 2; angle abnormality; grade, grading category; OD, right eye; OS, left eye. The final breeding recommendations (green: “breeding—yes”; red: “breeding—no”) are presented in graphs D and H. Corresponding numbers can be found in Table S2.

The main differences between the examiners in PLA examination results were observed in the categories mild and moderate PLA for OD (Figure 3A) and in the categories mild and severe PLA for OS (Figure 3E). Ex1 scored 10 eyes as narrow in the category ICA width for OD, whereas Ex2 assigned only 7 eyes to the same category (Figure 3B). A similar situation was observed in the left eye, where Ex1 scored 10 eyes as narrow, whereas Ex2 assigned only 6 eyes to the same category (Figure 3F).

Differences in ICAA grading were pronounced in mild ICAA where the PLA and ICAW results of Ex1 assigned 2 right eyes and 3 left eyes into this category, while the results of Ex2 assigned 7 right eyes and 10 left eyes into the same category. Another discrepancy was seen in ICAA severe grades where the assessment of Ex1 classified 7 left eyes as ICAA severe, while only 2 eyes ended up in the same category with Ex2 (Figure 3C,G). This reflects the disagreement in the breeding recommendation for left eyes since the ICAA grade results in the breeding recommendation.

Examiners disagreed in PLA grading in 19/118 eyes and in ICAW grading in 9/118 eyes. Figure 4 and Table 4 assist with visualizing the discrepancies between examiners determining where these were the most pronounced. Different PLA grading affected 4 dogs bilaterally and 11 dogs unilaterally, accounting for a total of 19 eyes with observed disagreement. Discrepancies were observed between neighboring grades in 17/19 eyes: severe (grade 3) vs. moderate (grade 2) in 6/19 eyes, moderate (grade 2) vs. mild (grade 1) in 5/19 eyes, and mild (grade 1) vs. normal (grade 0) in 6/19 eyes. The observed difference in PLA grading crossed two grades in 2/19 eyes: severe (grade 3) vs. mild (grade 1). Different gradings of ICA width affected 3 dogs bilaterally and 3 dogs unilaterally, accounting for a total of 9 eyes with observed disagreement. Discrepancies were only observed between neighboring grades: narrow ICA (grade 1) vs. open ICA (grade 0) in 8/9 eyes and closed ICA (grade 2) vs. narrow ICA (grade 1) in 1/9 eyes. Disagreement of PLA and ICAW occurred within the same eye in seven eyes accounting for an observed disagreement in PLA and ICAW combined for 21/118 eyes. This translated into a different ICAA grading in 19/118 eyes only because in two eyes, the disagreement did not result in a different ICAA grade.

FIGURE 4.

PLA and ICAW grading disagreement: Direct examiner comparison. Each differently colored line represents a dog where examiners disagreed on the grading category. Dots mark the grading category of each eye. In some cases, examiners disagreed in both eyes of the same dog, whereas in others, they disagreed in only one eye. The line connecting the two dots indicates the trend of disagreement between examiners. Ex 1, Examiner 1; Ex 2, Examiner 2; ICAW, iridocorneal angle width; OD, right eye; OS, left eye; PLA, pectinate ligament abnormality.

TABLE 4.

PLA and ICAW grading disagreement: Grading category comparison.

	OD	OS	Total eyes
PLA difference
3–2	1	5	6
3–1	0	2	2
2–1	2	3	5
1–0	3	3	6
Total eyes	6	13	19
ICAW difference
2–1	0	1	1
1–0	3	5	8
Total eyes	3	6	9

Note: Observed differences in eyes for which examiners disagreed. For example, Ex1 scored PLA grade 3 for OD but Ex2 scored PLA grade 2 for the same eye. The difference between scored grades is depicted in Figure 4, corresponding numbers are shown in this table.

Abbreviations: ICAW, iridocorneal angle width; OD, right eye; OS, left eye; PLA, pectinate ligament abnormality.

Disagreement in the breeding recommendation for 8/118 eyes (7%) translated to a different breeding recommendation for 2/60 dogs (3%) only. To arrive at the final breeding recommendation for a dog, only the most severely affected eye counts, that is, the eye with the worst ICAA score. The most severely affected eye was not necessarily the one on which the examiners disagreed, which explains why disagreement in the breeding recommendation for eight eyes translated to a different breeding recommendation for two dogs.

3.2. Cohen's Kappa

The mean Κ was 0.62 (95% TI from 0.34 to 0.89). According to Landis and Koch, in a balanced data set in which Κ would range from 0 to 1, Κ = 0.62 could be interpreted as a substantial level of agreement. ⁶² Because our data were derived from a clinical study in which the prevalence of severe ICAA was suspected to be low (resulting in a very unbalanced data set), a kappa value of 1 is very difficult to reach. The authors used the suggestion by Dunn et al. and calculated maxΚ to assess how individual tendencies affected the agreement between examiners. ⁵¹ The mean maxΚ was 0.82 (95% TI from 0.59 to 1), indicating a slightly higher interexaminer agreement (comparing Κ = 0.62 to a maximal possible agreement of 0.82 instead of 1) when individual tendencies are removed.

3.3. Mixed model

The mixed model analysis revealed that the eye examined by Ex2 had an odds ratio of 0.17 to be classified into the “breeding—no” category compared to the same eye being examined by Ex1, that is, eyes examined by Ex2 were less likely to be classified as “breeding—no” than eyes examined by Ex1. This, however, was not a statistically significant difference (OR = 0.17, 95% CI: 0.02–1.12, p‐value 0.09).

Neither the dogs' age nor the sequence of examiners contributed significantly to improvement of the model performance, therefore, these variables were excluded. This implies that these variables did not substantially influence breeding recommendations for the dogs included in this study.

4. DISCUSSION

This is the first study to formally assess the interexaminer agreement of the revised 2022 ECVO‐HED scheme for gonioscopy in dogs. We can conclude that in a clinical setting with low prevalence of severe ICAA changes, two experienced ophthalmologists using the ECVO‐HED scheme reached a subjectively high agreement of 93% (110/118 eyes), resulting in the same final breeding recommendation in 58/60 (97%) dogs.

Disagreement between examiners in PLA grading was observed between neighboring grades in 17/19 eyes, affecting all categories. The differentiation between fibrae latae and laminae depends on the number of coalescing filaments. Counting blended fibers during gonioscopy is not particularly easy and a subjective estimate has to be made. The percentage of ICA circumference affected by either fibrae latae or laminae is also subjectively estimated by the examiners. Cutoff values between PLA scores have been defined at 50% fibrae latae and at 25% and 50% laminae (Table 1). If the truly affected percentage would lie close to one of these cutoff values, the estimated affected percentages of both examiners could be very similar but still fall on opposite sides of the cutoff value, resulting in the assignment of a different score. Both these factors may have accounted for some of the observed differences in PLA grading. Recording the exact estimate of PLA in % of the ICA would be useful to monitor progression over time and could potentially be implemented into the data system for ECVO‐HED records besides the scored categories. At the same time, giving an exact estimate will inevitably increase the complexity of the grading system and likely result in a lower interexaminer agreement. ⁴⁷ In two eyes (two dogs), one examiner assigned a PLA grade of “mild,” whereas the other examiner assigned a PLA grade of “severe.” One of these dogs was scored as ICAW narrow by both examiners. The presence of a narrow ICA may have complicated the differentiation of PLA in this individual. The second dog had only one eye included in the statistical analysis due to compliance issues.

In 8/9 eyes for which ICAW grading differences between examiners occurred, disagreement was observed between the neighboring ICAW grading categories “open” (unaffected) and “narrow” (affected moderate). During gonioscopy, the gonioscopy lens should be placed such that the entire ICA width from the iris base to the inner corneal surface is visualized. When in doubt, the lens should be repositioned and, if indicated, dynamic gonioscopy should be performed to differentiate appositional angle closure from angle closure with peripheral anterior synechiae. ¹⁶ , ⁶³ Dynamic gonioscopy uses an indirect goniolens (e.g., Karickhoff or Volk diagnostic lens, Posner gonioprism) and opens the ICA by cautiously applying asymmetric pressure on the lens, allowing for better visualization of a narrow angle. ⁶⁴ Still, estimating the distance from the origin of the pectinate ligament to the inner corneal surface (Figure 2, Table 2) remains subjective and challenging, which may lead to differences in scoring. For example, the presence of broad‐based pectinate ligament fibers should not be confused with narrowness of the angle. This can be overcome by applying an imaginary horizontal line along the iris base. ⁶⁵ Using the ECVO scheme, a narrow ICA always results in a moderately affected ICAA grade, but the presence of broad‐based pectinate ligament fibers does not necessarily, depending on the percentage of angle circumference affected. For all practical purposes, the ECVO advice is “breeder option” in either of these two scenarios, and the recommendation would be to breed these dogs with gonioscopically unaffected dogs. Another important consideration is the following: by definition of the ECVO‐HED gonioscopy scheme, the pectinate ligament is not visible with a closed ICA, and therefore, only the ICAW can be scored, but not the PLA. This might not be intuitive to every examiner and has been discussed by the ECVO‐HED committee. Besides “ICAW closed,” the angle can alternatively be graded as “PLA severely affected,” and both options translate into the same recommendation against breeding with the animal. ⁶⁶

As a result, although the examiners disagreed on final ICAA grading in 19/118 eyes (16%), they gave a different breeding recommendation in 8/118 eyes (7%) only. This breeding recommendation is related to the allocation of the study groups. ICAA grades normal, mild, and moderate being classified as breeding—yes and ICAA grade severe as breeding—no. We chose this division because, in line with the ECVO manual, most breed clubs would allow mildly and moderately affected dogs to be bred with unaffected dogs. ¹ Since the most severely affected eye accounts for the final breeding recommendation of the dog, the translation into the final breeding recommendation for the individual animal resulted in a disagreement of 2/60 (3%) dogs. This further decreases the observed difference between examiners. One of the two dogs with discrepancies in the final breeding recommendation was a flat‐coated retriever which had only one eye included in the study due to compliance issues. Here, one examiner graded PLA as “affected (mild),” whereas the other examiner graded PLA as “affected (severe).” The other dog was a Border Terrier which had its ICAW graded as “narrow” by both examiners. Here, one examiner scored 25%–50% laminae in both eyes, grading PLA as “affected (moderate),” whereas the other examiner scored >50% laminae in both eyes, grading PLA as “affected (severe).” As discussed before, both examiners may have scored the extent of the laminae close to 50% ICA circumference, but still falling on opposite sides of the 50% cutoff value, which decisively affected the final breeding recommendation for this dog. Examiners assigned grades according to the ECVO‐HED grading scheme, but no exact estimates of PLA in percent affecting the ICA were recorded. Unfortunately, this precludes a detailed assessment in these cases and the suggested explanation remains an assumption. Reduced patient compliance and a narrow ICA can possibly make the distinction between fibrae latae and laminae and a consistent estimation of the percentage of affected ICA circumference challenging.

Relying only on Cohen's Kappa was insufficient for our study because it did not allow us to reflect on areas where disagreement between examiners occurred. Therefore, we complemented our analysis with descriptive statistics and the use of a mixed model. By doing so, the authors hoped to strengthen the analysis and identify possible areas of the ECVO‐HED 2022 scheme that might lead to inconsistent scoring. The authors would like to emphasize that a high level of agreement indicated by Cohen's kappa does not demonstrate that the examiners scored correctly, it merely indicates that the examiners scored consistently. ⁴⁸ Unfortunately, at present, there is no validated gold standard method to identify and grade ICA pathology, which can be used to definitively classify cases. This not only precludes evaluating whether obtained scores are correct but also prevents the calculation of the prevalence of severe ICAA. This is important, since Κ values heavily depend on the prevalence of the scored condition, in this case, severe ICAA, and not only on the observed agreement between examiners. ⁵² The mixed model analysis confirmed that neither the dogs' age, the sequence of examiners, or the examiner itself substantially influenced breeding recommendations.

Prior studies have assessed interexaminer agreement using two‐, three‐, and four‐tiered grading schemes. Two raters using a two‐tiered grading system classifying dogs to be either unaffected (0% PLD) or affected (any PLD) showed an agreement of 96%, with a Κ = 0.83 (95% CI = 0.61–0.99). This was interpreted as a reasonably strong level of agreement. ²⁹ Further assessment of different grading schemes in the UK showed that a three‐tiered scoring system including large cutoff values without estimating ICA width yielded a good level of agreement (Κ = 0.68), whereas four‐tiered schemes demonstrated fair to moderate levels of agreement, which was considered unacceptable by the authors. ⁴⁷ Older grading systems consisted of even more categories, including eight grades of PLA and/or five degrees of ICA width. ¹³ , ¹⁶ The finer the discriminations that need to be made, the more unlikely it will be for examiners to consistently agree on score assignments and grading. At the same time, clear cutoff values facilitate agreement between examiners. ⁵³ Also, simplification of grading schemes leads to a loss of collected patient information. The ECVO‐HED grading scheme includes four different grades of PLA and three different grades of ICA width, resulting in different final ICAA grades. The Κ = 0.62 that we arrived at in our study is therefore consistent with the more complex grading system employed in the ECVO‐HED grading scheme. Knowing that Κ is specific to the study setting and largely depends on the prevalence of the scored condition, Κ values, and therefore, interexaminer agreement, cannot be directly compared between studies.

This study does not come without limitations, which the authors would like to emphasize at this point. The examinations were performed by two boarded ECVO diplomates with 20 years of experience in the field, who are currently both working at the same institution. The agreement might be lower between examiners from different institutions and with different levels of experience. Due to differences between populations in the prevalence of the scored condition, the agreement between examiners may also differ depending on the dog population that is studied. Finally, the level of compliance of the individual patient clearly affects the thoroughness of the examination that is performed.

Prior to conducting this study, the available HED documentation on the ECVO website was reviewed by all clinicians participating in the study, which may have increased the level of agreement between examiners. This is not an undesired effect and underlines the importance of HED training sessions for scheme examiners during ECVO and national meetings.

5. CONCLUSIONS

Although examiners disagreed on the ICAA grading in 19/118 eyes (16%), this resulted in a different breeding recommendation in 8/118 eyes (7%), and 2/60 dogs (3%) only. Therefore, the use of the 2022 ECVO‐HED gonioscopy grading scheme seems to result in examiners providing consistent breeding recommendations in the majority of cases. Most scoring inconsistencies between the two examiners involved neighboring PLA and ICAW grades. Reducing the complexity of the scheme may improve consistency. On the other hand, a less detailed scoring system will inevitably result in a loss of collected data and preclude monitoring of progression over time.

AUTHOR CONTRIBUTIONS

Niklas Holz: Conceptualization; investigation; project administration; visualization; writing – original draft; writing – review and editing. Malwina Ewa Kowalska: Data curation; formal analysis; methodology; visualization; writing – original draft; writing – review and editing. Simon Anton Pot: Conceptualization; investigation; supervision; writing – original draft; writing – review and editing. Antonella Rampazzo: Conceptualization; investigation; supervision; writing – original draft; writing – review and editing.

CONFLICT OF INTEREST STATEMENT

The authors have no conflicts of interest to declare.

ETHICAL APPROVAL STATEMENT

This study was approved by the Federal Food Safety and Veterinary Office of Switzerland. National no. 35599; Cantonal no. ZH025/2023.

PRECLINICAL TRIALS REGISTRATION

This study was preregistered on https://preclinicaltrials.eu under the registration number PCTE0000388.

Supporting information

Appendix S1

Appendix S2

Holz N, Kowalska ME, Pot SA, Rampazzo A. Interexaminer agreement of gonioscopy examinations in dogs using the European College of Veterinary Ophthalmologists Hereditary Eye Diseases grading scheme. Vet Ophthalmol. 2026;29:e13292. doi: 10.1111/vop.13292

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are openly available and can be accessed via https://osf.io/rdbt9/?view_only=e1b1f6e711ca428fa5e6f950e9224c22.