Harmonizing the Classification of Age-related Macular Degeneration in the Three Continent AMD Consortium

Abstract

Purpose

To describe methods to harmonize the classification of age-related macular degeneration (AMD) phenotypes across four population-based cohort studies: the Beaver Dam Eye Study (BDES), Blue Mountains Eye Study (BMES), Los Angeles Latino Eye Study (LALES), and Rotterdam Study (RS).

Methods

AMD grading protocols, definitions of categories, and grading forms from each study were compared to determine whether there were systematic differences in AMD severity definitions and lesion categorization among the three grading centers. Each center graded the same set of 60 images using their respective systems to determine presence and severity of AMD lesions. A common five-step AMD severity scale and definitions of lesion measurement cutpoints and early and late AMD were developed from this exercise.

Results

Applying this severity scale changed the age-sex adjusted prevalence of early AMD from 18.7% to 20.3% in BDES, from 4.7% to 14.4% in BMES, from 14.1% to 15.8% in LALES, and from 7.5% to 17.1% in RS. Age-sex adjusted prevalences of late AMD remained unchanged. Comparison of each center’s grades of the 60 images converted to the consortium scale showed that exact agreement of AMD severity among centers varied from 61.0% to 81.4%, and one-step agreement varied from 84.7% to 98.3%.

Conclusion

Harmonization of AMD classification reduced categorical differences in phenotypic definitions across the studies, resulted in a new 5-step AMD severity scale, and enhanced similarity of AMD prevalence among four cohorts. Despite harmonization it may still be difficult to remove systematic differences in grading, if present.

Introduction

In epidemiologic studies, the power to detect associations of risk factors for infrequent outcomes is often low. Many investigators have sought to minimize this problem by pooling data from several studies or by performing meta-analyses. This approach has commonly been taken in discovering genetic associations and in examining the interactions of genetic, host, and environmental factors for age-related macular degeneration (AMD) and other ocular traits and diseases.^1-8 Synchronization or harmonization of the disease phenotypes and risk factors/indicators might reduce heterogeneity secondary to methodological differences among studies.^9,10

Data from the Beaver Dam Eye Study (BDES), the Blue Mountains Eye Study (BMES), and the Rotterdam Study (RS) were previously pooled to examine the relationships of risk factors to the prevalence and five-year incidence of late AMD.^11-13 From this established collaboration, the Three Continent AMD Consortium, consisting of four large population-based studies (the BDES, the BMES, the Los Angeles Latino Eye Study [LALES], and the RS), was formed in 2009 to examine the effects of gene × environment and gene × host interactions on the incidence of early and late AMD and the progression of AMD. The purpose of this paper is to describe the methods used to harmonize the AMD phenotype and to define the resulting AMD severity scale for use in analyses of data from the four cohorts.

Materials and Methods

Brief Descriptions of Populations, Photography and Grading

The dates of the baseline and follow-up examinations, the number of participants at each examination, the number with gradable fundus photographs, the cameras used, the multistep grading procedures, and the definitions of early AMD used by each study are presented in Table 1. The cohorts and the methods used to examine them have been described in detail elsewhere (Figure 1 and Online Supplement parts A-L, available at http://informahealthcare.com/journal/ope).^14-40 Approval for the BDES was granted by the Institutional Review Board at the University of Wisconsin. Ethics committee approval for the BMES was provided by the Western Sydney Area Health Service Human Research Ethics Committee. The LALES was approved by the University of Southern California Health Sciences Campus and Cedars-Sinai Medical Center Institutional Review Boards. The Medical Ethics Committee of the Erasmus Medical Center approved the RS. Informed consent was obtained from all participants and the tenets of the Declaration of Helsinki were observed by all four studies.

TABLE 1.

Characteristics of participating studies and their grading methods

Study	Examination periods	N participants (N with gradable photos*)	Angle and camera type/model used	Fields photographed	Grading processes	Definition of early AMD
BDES	1988-1990	4926 (4778)	30° film Zeiss FF4	ETDRS 1,2 (stereo); modified 3 (nonstereo)	Preliminary, Detail, Edit, Adjudication, Longitudinal Review	Any drusen size or type in the presence of pigmentary abnormalities or soft indistinct drusen ≥125 μm diameter
	1993-1995	3722 (3556)	30° film Zeiss FF4
	1998-2000	2962 (2831)	30° film Zeiss FF4
	2003-2005	2375 (2260)	30° film Zeiss FF4
	2008-2010	1913 (1790)	30° film Zeiss FF4
BMES	1992-1994	3654 (3583)	30° film Zeiss FF3	ETDRS 1,2 (stereo) and four other fields	Detail, Longitudinal Review	Soft distinct drusen ≥125 μm diameter in the presence of pigmentary abnormalities or soft indistinct drusen ≥125 μm diameter
	1997-1999	2334 (2307)	30° film Zeiss FF3
	2002-2004	1952 (1689)	30° film Zeiss FF3
	2007-2010	1149 (1050)	40° digital Canon CF-60 DSi and Canon EoS 1DS Mark II
LALES	2000-2003	6357 (5875)	30° film Zeiss FF450+	ETDRS 1,2 (stereo); modified 3 (nonstereo)	Preliminary, Detail, Edit, Adjudication, Longitudinal Review	Any drusen size or type in the presence of pigmentary abnormalities or soft indistinct drusen ≥125 μm diameter
	2004-2008	4658 (3908)	30° film Zeiss FF450+
	2010-date		30° digital Topcon TRC-50DX
RS	1989-1993	6780 (6405)	35° film Topcon TRV-50VT	ETDRS 1, 2 (stereo)	Detail, Longitudinal Review	Soft distinct drusen ≥63 μm diameter in the presence of pigmentary abnormalities or soft indistinct drusen ≥125 μm diameter
	1993-1995	6315 (4932)	35° film Topcon TRV-50VT
	1997-1999	4797 (3625)	35° film Topcon TRV-50VT
	2000-2004	3550 (2662)	35° digital Topcon TRC 50EX and Sony DXC-950P
	2009-2011	1658 (1447)	35° digital Topcon TRC 50EX and Sony DXC-950P

AMD, age-related macular degeneration; BDES, Beaver Dam Eye Study; BMES, Blue Mountains Eye Study; ETDRS, Early Treatment of Diabetic Retinopathy Study; LALES, Los Angeles Latino Eye Study; RS, Rotterdam Study.

^*In at least one eye.

Figure 1.

A. Grids defining the 9 macular subfields (not to scale). B. Measuring tools for lesion size and area of involvement (not to scale). Circles represent fractions of the total area of the center (C), inner (I), and outer (O) subfields of the grid in part A, and the 3 crossed lines each measure 1000 μm in length.²² CC, center circle; II, inner inferior; IN, inner nasal; IS, inner superior; IT, inner temporal; OI, outer inferior; ON, outer nasal; OS, outer superior; OT, outer temporal.

Grading Comparisons and Steps Used to Harmonize AMD Severity

In order to examine the comparability of AMD lesion grading protocols and definitions of AMD among the studies, grading forms, previous publications, and data dictionaries from each study were collected by the University of Wisconsin Ocular Epidemiology Group (UWOEG) which grades fundus photographs for BDES and LALES. We compared the conventions of the lesion-specific data assessed by each of the three grading centers (UWOEG, BMES grading team, RS grading team), i.e., whether the study graded early lesions in the presence of late AMD, how lesion locations in the macula were graded (globally or by subfield), and if there was agreement in the definitions of specific lesions and their severities as well as early and late AMD. The differences in grading protocols and the absence or presence of AMD lesions were discussed during regular teleconferences to determine whether there were systematic differences in lesion identification among the gradings performed at the three grading centers. Differences in the definitions of early and late AMD, approaches described in the grading protocols among the grading centers, and specific differences in categorizing lesions are presented in Tables 1 and 2. To ensure that each study used the same definitions for late AMD, senior investigators at each grading center (RK and BEKK of the UWOEG, PM and JJW of the BMES, JRV and CCWK of the RS) were provided with either film or digital copies of stereoscopic pairs of photographs of any eye that had been graded as having late AMD at any visit for review and discussion. Each senior investigative team independently reviewed these images and held teleconferences with the other teams to discuss disagreements in lesion classification until a consensus was reached for each case among all four studies.

TABLE 2.

Differences in grading among the four studies for specific age-related macular degeneration lesions

Lesion	Differences	Resolution
Drusen Area	RS used % area in center/inner/outer rings of grid; BDES, BMES, and LALES used area within grid using μm2 cutpoints	Reassigning drusen area in RS after application of mathematical algorithm based on regrading 60 eyes
Drusen Size/Type/Area, Pigmentary Abnormalities	BMES did not grade in presence of late AMD	Analyses excluded eyes with late AMD
Intermediate Drusen	BMES categorized drusen 63 to 125 μm in diameter as intermediate size drusen while this was not a category in BDES, LALES, or RS	Drusen type was not included in the AMD severity scale
Geographic Atrophy	Number of characteristics defining its presence varied	Used 2 characteristics (shape, edge definition, total absence of retinal pigment epithelium, and/or visibility of choroidal vessels) in addition to size

BDES, Beaver Dam Eye Study; BMES, Blue Mountains Eye Study; LALES, Los Angeles Latino Eye Study; RS, Rotterdam Study.

To assess lesion-specific definitional differences among the three grading centers, the UWOEG digitized a set of stereoscopic images of 60 eyes with lesions characteristic of the range of severity of AMD selected from BDES participants, then reprinted the images on film and sent identical copies to the UWOEG, BMES, and RS grading teams. These images were graded twice, more than six months apart, by the UWOEG: one time using the BDES protocols and another time using the LALES protocols. The image set had a balanced distribution of lesion characteristics considered to be typical of AMD: varying drusen size, type, and area, increased retinal pigment, retinal pigment epithelium (RPE) depigmentation, geographic atrophy, RPE detachment/sensory serous retinal detachment, subretinal hemorrhage, or subretinal fibrous scars. The 60 images were graded independently by each center using their respective forms and approaches. Grading results were sent to the UWOEG to examine their comparability and evaluate reasons for disagreements in the gradings of the 60 eyes among the centers.

A spreadsheet was constructed with the specific AMD lesions (e.g., drusen size, type, and area, pigmentary abnormalities, geographic atrophy, exudative AMD) as well as the final severity scores of the 60 images graded by each of the centers. These data were used to create common definitions of the presence and absence of each lesion based on the common lesion cutpoints (e.g., drusen size >125 μm in diameter) across studies.

A Three Continent AMD Consortium severity scale was developed based on these harmonized cutpoints defining each early AMD lesion (Table 3). This scale allows for the common definitions of prevalence and incidence of AMD to be used in subsequent pooled data or meta-analyses conducted by the Consortium. The scale has five categories of AMD severity numbered from 10 to 50, where level 10 represents no AMD and level 50 represents late AMD. Levels 20, 30, and 40 represent mild, moderate, and severe stages of early AMD, respectively. An AMD severity scale score was assigned to each eye based on lesion severity as graded by each study’s grading protocol, i.e., each image had four grades, one from each study group.

TABLE 3.

Three Continent AMD Consortium age-related macular degeneration severity scale

Level	Label	Description
10	No AMD	No, questionable, small, or intermediate sized drusen (<125 μm in diameter) only, regardless of area of involvement, and no pigmentary abnormalities (defined as increased retinal pigment or RPE depigmentation present) OR No definite drusen with any pigmentary abnormality.
20	Mild early AMD	Small to intermediate sized drusen (<125 μm in diameter), regardless of area of involvement, with any pigmentary abnormality OR Large drusen (≥125 μm in diameter) with drusen area <331,820 μm2 (equivalent to O-2 circle, defined as a circle with diameter of 650 μm) and no pigmentary abnormalities.
30	Moderate early AMD	Large drusen (≥125 μm in diameter) with drusen area <331,820 μm2 and any pigmentary abnormality OR Large drusen (≥125 μm in diameter) with drusen area ≥331,820 μm2, with or without increased retinal pigment but no RPE depigmentation.
40	Severe early AMD	Large drusen (≥125 μm in diameter) with drusen area ≥331,820 μm2 and RPE depigmentation present, with or without increased retinal pigment.
50	Late AMD	Pure geographic atrophy in the absence of exudative macular degeneration OR Exudative macular degeneration with or without geographic atrophy present.

AMD, age-related macular degeneration; RPE, retinal pigment epithelium.

To evaluate grader variability, we then compared the consortium scale score assigned based on each study’s grading scheme to the score that was assigned based on each of the other studies’ grading schemes. Weighted kappa statistics were calculated using the Fleiss-Cohen weighting method,⁴¹ which was also used by the Age-Related Eye Diseases Study for grading quality control comparisons.⁴² Using the new harmonized definitions, the age-sex adjusted prevalence of each AMD global lesion, and early and late AMD, were compared to the original AMD prevalence estimates reported by each study, based on their previous definitions. Age was categorized as <55 years, 55-64 years, 65-74 years and ≥75 years and the proportion of participants in each age-sex category was used to weight the age-sex adjusted prevalence rates.

Results

Differences among Grading Conventions

The differences among the studies’ grading conventions for early AMD lesions are summarized in Table 2.

Drusen Size

The RS graded drusen size at the first three visits using the same conventions as the other studies, but was the only study to discard this variable at the fourth and fifth visits. Drusen size in RS was therefore defined by type (hard/soft <125 μm diameter and soft distinct/indistinct ≥125 μm diameter). In order to construct a consistent drusen size variable that would be consistent for all four studies at all visits, we selected 125 μm as the cutpoint for presence/absence of large drusen. The drusen size variable used in all consortium analyses has three levels: no drusen, drusen with diameter <125 μm, or drusen with diameter ≥125 μm. These cutpoints were then used to define drusen size for each cohort in all phases of the study.

Drusen Area

The RS graded drusen area as the percentage of the center, inner, or outer rings of the grid involving drusen while the other studies graded the area of drusen within the grid using square micrometer cutpoints (<125 μm², <250 μm², <350 μm², <650 μm², <1/2 disc area (DA), ≥1/2 DA). To resolve this difference, the RS was asked to regrade drusen area in the 60 sample eyes using the same cutpoints as used by the other studies. Based on the results of these gradings, the RS developed a mathematical algorithm which added the center, inner, and outer values together to assign a drusen area value in μm² to each eye based on the percent involvement cutpoint used in the RS grading system.

Algorithm used in RS

Drusen area (%) was graded in the RS for the central circle, inner circle and outer circle for each eye separately. To assign a drusen area per eye in μm² we calculated the affected area in each circle using the formula for the area of a circle (= π × r²).

$$\begin{array}{cc}\hfill \text{Area central circle}& =\pi \times 500\times 500\hfill \\ \hfill \text{Area inner circle}& =\pi \times 1500\times 1500-\text{Area central circle}\hfill \\ \hfill \text{Area outer circle}& =\pi \times 3000\times 3000-\text{Area inner circle}-\text{Area central circle}\hfill \end{array}$$

We then multiplied the percent drusen area by the area of the corresponding circle.

Omissions

The BMES did not grade drusen and pigmentary abnormalities if an eye had signs of late AMD while the other two grading centers did grade drusen and pigmentary abnormalities irrespective of the presence of other lesions. In the harmonization process the analysts agreed to exclude eyes with late AMD from consortium meta-analyses of drusen or pigmentary abnormalities.

Drusen Type

The BMES labeled drusen with a diameter between 63 μm and 125 μm as intermediate drusen, while the other studies did not use a comparable intermediate drusen descriptor. This difference was resolved by not including drusen type as part of the Consortium AMD severity scale definition.

Lesion Characteristics

There were differences among the grading centers in definitions for geographic atrophy, e.g., sharp edge, lack of RPE, visible choroidal vessels, and circular shape, and for the minimum size required to grade a lesion as being pure geographic atrophy. These differences were resolved by defining the presence of geographic atrophy as an area of RPE atrophy greater than or equal to that of a circle 350 μm in diameter, and presence of at least two of the defining characteristics described above.

Presence and Severity of AMD Categories

The presence and severity of early AMD was defined differently by each study using the AMD severity scale developed from their individual study’s grading scheme. In the BDES, early AMD was defined as presence of definite drusen of any size along with pigmentary abnormalities (Table 1). However, the RS required that drusen be at least 63 μm in diameter and the BMES at least 125 μm in diameter, combined with the presence of pigmentary abnormalities, to be graded as early AMD. For example, an eye with hard distinct drusen <63 μm in diameter and a pigmentary abnormality would be classified as not having early AMD by the BMES and RS but as having early AMD by the BDES. This difference would be expected to lead to a lower prevalence of early AMD in the BMES and RS compared to the BDES. The levels of severity of early AMD also differed in scales in which drusen area was used to define early AMD (Online Supplement sections E, G, and L). To resolve these differences, mild early AMD cases that were originally included as early AMD in the BDES and the LALES but as no AMD in the BMES or RS were coded as level 20 on the harmonized scale.

Harmonized Lesion Definitions and Grading Scale

Harmonized consortium definitions of the presence and absence of AMD lesions were agreed upon (Table 4) and used to develop a 5-step AMD consortium severity scale (Table 3). A single “worse eye” summary variable was created for each participant for both the AMD level and each individual lesion. All the studies agreed to use these person-level “worse eye” variables as the phenotype in all analyses. When a lesion or AMD severity level was ungradable in one eye it was assumed to have the same severity as the fellow eye. If any lesion used in defining the AMD scale was ungradable in both eyes, then the individual was not assigned an AMD consortium scale score. A participant with any particular lesion ungradable would still be considered for analyses for any other individual lesions that were gradable.

TABLE 4.

Definitions of age-related macular degeneration lesions used globally across all four studies

Lesion	Global Definition
Large drusen size	≥ 125 pm in diameter
Large drusen area	≥ 650 pm in diameter
Increased pigment	Any AMD related increased pigment
RPE depigmentation	Any AMD related RPE depigmentation
Geographic atrophy	Area of atrophy ≥350 μm in diameter and presence of at least 2 of these features: sharp edge, lack of RPE, visible choroidal vessels, and circular shape
Exudative AMD	Presence of any of the following: pigment epithelial detachment and/or retinal detachment, subretinal hemorrhage, subretinal scar, subretinal new vessels, treatment for exudative lesion

AMD, age-related macular degeneration; RPE, retinal pigment epithelium.

We were able to assign a consortium scale score to 59 out of the 60 sample eyes graded using the BMES protocol; however, one eye was determined by the BMES grading center to have ungradable drusen size, type, and area. We were able to assign a consortium scale score to all 60 eyes graded using the BDES, LALES, and RS protocols (i.e., no lesions were determined to be ungradable in any of the test eyes).

Using the new harmonized Three Continent AMD Consortium severity scale, the exact grading agreement of the 60 eyes between centers varied from 61.0% between the BMES versus RS and RS versus LALES to 81.4% between the BMES and the LALES, and the within-one-step agreement varied from 84.7% between the RS and LALES to 98.3% between the BMES and the BDES (Figure 2). Weighted kappa scores varied from 0.66 to 0.86, indicating moderate to substantial levels of agreement among the grading centers.

Figure 2.

Agreement between each pair of grading centers for grading of age-related macular degeneration using the Three Continent AMD Consortium Age-Related Macular Degeneration Severity Scale after harmonization. A. Beaver Dam Eye Study (BDES) vs. Blue Mountains Eye Study (BMES); B. BDES vs. Rotterdam Study (RS); C. BMES vs. RS; D. BDES vs. Los Angeles Latino Eye Study (LALES); E. BMES vs. LALES; F. RS vs. LALES. CG, cannot grade; CI, confidence interval.

Comparisons of new and previous estimates of the prevalence of early and late AMD in each study population are presented in Table 5. The prevalence of early AMD in each of the four studies had greater similarity among themselves using the harmonized definition than for each study’s original definition. The largest changes in prevalence reflect changes in the cutpoints for drusen size and area after harmonization. As expected, the prevalence of late AMD did not change (Table 5).

TABLE 5.

The age-sex adjusted prevalence of early and late age-related macular degeneration in the four cohorts using the individual three grading center original definitions and the Three Continent AMD Consortium global definition

	BDES		BMES		RS		LALES
Lesion/Level	Original Definition %	Consortium Definition %	Original Definition %	Consortium Definition %	Original Definition %	Consortium Definition %	Original Definition %	Consortium Definition %
Early AMD	18.7	20.3	4.0	12.8	7.6	17.4	14.1	15.8
Large drusen	15.9	15.8	5.4	5.4	8.9	12.9	14.8	14.8
Large drusen area	6.7	4.0	NA	7.4	NA	10.6	3.0	3.0
Increased retinal pigment	12.4	12.4	9.1	9.1	7.5	7.5	4.7	4.7
RPE depigmentation	7.3	7.3	5.6	5.6	7.2	7.2	1.9	1.9
Late AMD	1.8	1.8	1.8	1.8	1.6	1.6	0.2	0.3
Geographic atrophy	0.6	0.6	0.6	0.6	0.7	0.7	0.2	0.2
Neovascular AMD	1.2	1.2	1.2	1.2	1.0	1.0	0.1	0.1

AMD, age-related macular degeneration; BDES, Beaver Dam Eye Study; BMES, Blue Mountains Eye Study; LALES, Los Angeles Latino Eye Study; NA, not applicable; RPE, retinal pigment epithelium; RS, Rotterdam Study.

Discussion

The Three Continent AMD Consortium, consisting of four large population-based cohorts totaling 20,655 persons at baseline examinations with gradable fundus photographs for AMD in at least one eye and varying follow-up periods from approximately 8 to 20 years, was formed to examine genetic and environmental interactions in relation to the incidence of AMD. When the three grading centers began, each study used modifications of the Wisconsin Age-Related Maculopathy Grading System to grade and define lesions of AMD. Those modifications led to several systematic differences in the way the photographs were graded as well as in the criteria to classify early AMD. There were differences in prevalence of early AMD across the studies that were hypothesized to be due, at least in part, to those grading/classification differences. These differences made harmonizing AMD phenotypes across the studies essential for collaborative research projects. We defined common lesion thresholds across the studies, which resulted in an AMD scale that differs somewhat from previous schemes used by our groups and others. Our harmonization effort resulted in more comparable estimates of the prevalence of early AMD in our studies (Table 5). While the harmonization is assumed to have reduced the variability among the studies and to have increased study power for pooled data and meta-analyses to detect associations, there are no data or examples available yet to support this assumption.

Harmonization of the classification by changing definitions can reduce some differences in phenotypes among different study cohorts. However, it does not change differences in the estimated prevalence of specific lesions; we believe that some of these differences arise from systematic variation in the image quality, the grading process itself, and other specifics of examination procedures (e.g., methods of capturing and assessing AMD such as cameras, graders, grading protocols, number of gradings, edits, and adjudication)⁴³ although we cannot rule out the possibility of true differences among the studies arising from true differences in prevalence of lesions.

A limitation of the harmonization process is that any eye with late AMD not identified as such by the original cohort was not discussed amongst the investigators of these studies. Nevertheless, all four cohorts followed up with participants more than once, and because of the longitudinal review processes in place for each study would likely result in the detection of late AMD cases missed at a given visit at subsequent visits. We therefore expect that the number of late AMD cases missed would be very small and that those missed would most likely be individuals who died before follow-up or did not return for follow-up visits.

Misclassification of pure geographic atrophy may occur because only color images were used to identify exudative AMD and small RPE detachments may occur and collapse, leaving only atrophic lesions during the relatively long time interval between two visits. Because the time intervals were similar and RPE detachments were treated similarly among the studies, we do not think this resulted in systematic differences in their estimates of the prevalence of the two types of late AMD, exudative and pure geographic atrophy.

Conclusions

The harmonization provided an opportunity to reduce categorical differences in the estimates of early AMD and early AMD lesions with the goal of reducing variability in AMD phenotype definitions prior to data pooling and/or meta-analyses. This process will likely increase the power to detect genetic, host, and environmental risk factors, especially as we aim to use meta-analyses or pool data from multiple studies. Harmonization is, of course, a post-hoc attempt to achieve consistency of disease detection and classification. While not perfect compared to the use of the same techniques in identifying affected and non-affected subjects, it is a practical approach to decreasing phenotypic heterogeneity across studies. Because early AMD can be defined differently by different studies, absence of harmonization may lead to a failure to detect associations of risk factors in pooled or meta-analyses. If early AMD is to be used as an endpoint in analyses requiring data from multiple studies, harmonization is strongly recommended.