A Novel AMD Severity Scoring System Leveraging the AREDS Studies and Routine Clinical Electronic Medical Records

Abstract

Objective:

To develop a novel severity scoring system for age-related macular degeneration (AMD) that can generate distinct scores for exudative and non-exudative AMD, and that can be applied to routine clinical features captured in electronic medical record (EMR) data, where complete data on features included in traditional scoring systems is rare.

Design:

Retrospective cohort study with external validation.

Participants:

Data from participants in the Age-Related Eye Disease Study (AREDS), AREDS2, and the Eye Adult Changes in Thought study (Eye ACT).

Methods:

Severity score models were developed for non-exudative (“dry”) AMD (AMD-D) and exudative (“wet”) AMD (AMD-W) based on confirmatory factor analysis (CFA) of data from AREDS and AREDS2. Models were applied to an independent cohort of the Eye ACT study whose longitudinal ophthalmic clinical data were extracted from an EMR capturing routine care, using natural language processing-based text mining algorithms.

Main Outcome Measures:

Trajectories of AMD-D and AMD-W scores in the Eye ACT cohort and relationship with age and the onset of the first anti-vascular endothelial growth factor (antiVEGF) treatment.

Results:

In the Eye ACT cohort, AMD-D and AMD-W scores showed a moderately positive correlation (Pearson 0.702, 95% CI: 0.699–0.704). In 4,412 eyes from 2,248 participants in Eye ACT, which never received antiVEGF, AMD-D scores increased slightly before the age of 80, followed by a steeper increase through age 90. In 220 eyes of 171 Eye ACT participants, which received antiVEGF, most showed a pattern of gradually increasing AMD-W scores in the weeks or months prior to the antiVEGF treatment.

Conclusions:

The CFA-based scoring system enabled detailed assessments of both non-exudative and exudative AMD severity using features collected in a routine clinical setting with ubiquitous missing data, where standard AREDS scoring is not possible.

Precis:

We have developed and validated a novel scoring system for age-related macular degeneration (AMD) severity using confirmatory factor analysis of landmark AMD trial datasets and applied it to routine clinical electronic medical record data.

INTRODUCTION

Age related-macular degeneration (AMD), a progressive, degenerative condition of the retina, is the leading cause of blindness in industrialized countries and affects nearly 200 million older adults worldwide.¹ AMD is classified into early, intermediate, and late stages where both non-exudative/atrophic (“dry”) and exudative/neovascular (“wet”) forms can lead to severe vision loss and permanent blindness. While the exudative forms have been treated with frequent intravitreal injections of vascular endothelial growth factor inhibitors (antiVEGF) over a decade, the treatment for non-exudative AMD is rather nascent.² Non-exudative AMD encompasses over 90% of all AMD, thus extensive research is ongoing to develop better treatment and prevention approaches.³

The ability to characterize AMD-related phenotypes and accurately assess disease severity is critical for accelerating research in AMD. Multiple severity scales have been developed over the years for research and clinical use based on the identification of features from fundus photography.^4–9 Both the Age-Related Eye Disease Studies (AREDS)¹⁰ and AREDS2¹¹ landmark trials have provided the foundation of our current understanding of the natural history and risk factors of AMD progression. Indeed, the AREDS severity scoring system is one of the most frequently used severity scoring systems in AMD research studies.⁷ Frequently known as the “nine-step” system, the AREDS severity scoring system includes 9 scores for non-advanced, non-exudative AMD, and additional scores of 10 and 11 for central geographic atrophy (GA) and exudative AMD, respectively.^7,12

These scoring systems work well in research settings designed to capture all of the AREDS measure indicators in a standardized way. However, while some elements of AMD severity are also documented in routine clinical care, it is rare that all elements of the AREDS measures are captured. So although electronic medical records (EMRs) contain a wealth of clinical data relevant to AMD severity, missing data poses a challenge in applying a uniform scoring system designed for studies that have extensive and complete data captured at each time point such as AREDS. Furthermore, the standard AREDS AMD severity score only includes a single score for exudative AMD, and there is no existing granular scoring system for assessing the later stages of AMD severity.

We developed a novel AMD severity scoring system leveraging the extensive retinal feature data incorporated in the standard AREDS AMD severity score from the AREDS and AREDS2 datasets. We generated separate scores in a continuous scale for non-exudative AMD and exudative AMD severity using confirmatory factor analysis (CFA), which we term CFA-based severity hereafter. We then applied the CFA-based scoring system to data from a longitudinal cohort consisting of EMR from routine clinical care of Eye Adult Changes in Thought (Eye ACT) study participants. Potential benefits of the CFA-based scoring system include its applicability to routine EMR data where missing data are common, and its ability to provide granular scores in both non-exudative and exudative AMD. Our scoring system has broad potential applications when applied to large-scale EMR datasets. These include screening for clinical trials, monitoring individual patient trajectories using the EMR data from routine clinical settings.

METHODS

Study cohorts

AREDS.

AREDS and AREDS2 were randomized clinical trials collecting longitudinal general visual measures and AMD-related eye features using color fundus photography, conducted from 1992 to 2005 and from 2006 to 2012, respectively.^10,11 The AREDS study included 4,757 participants aged 55–80 years with no AMD to those with early and intermediate AMD in both eyes or in one eye with advanced AMD in the fellow eye but 20/30 vision or better in at least one eye. Participants were enrolled from 11 sites nationwide. AREDS2 enrolled 4,203 participants (including 112 AREDS participants) aged 55–80 from 82 clinical sites with intermediate both eyes or in one eye with advanced AMD in the fellow eye. AREDS2 participants were followed for five years. Due to the stringent inclusion criteria and longitudinal nature of these studies, the AREDS/2 datasets include extensive clinical characterization of intermediate and advanced AMD disease features in both eyes of participants over multiple visits.

Eye ACT.

The Eye ACT study acquires additional data on participants in the Adult Changes in Thought (ACT) study, a prospective, longitudinal study embedded in the Kaiser Permanente Washington (KPWA) health care system, which follows a cohort of older adults prospectively until incident dementia or death.¹³ The ACT study collects demographic, cognitive, and health information from ACT participants (a total of 5,763 to date) directly at their baseline and biennial follow-up study visits, and gathers additional medical information by linking to participants’ in-network longitudinal medical records.¹⁴ The Eye ACT study acquires detailed ophthalmic clinical information on these same participants from eye encounters, optometric and ophthalmologic clinic visits from the EMR via a natural language processing-based text mining algorithm.¹⁵ in short, the text mining algorithm extracts all eye-related clinical records and identifies a priori relevant clinical features present in these visit records on ACT study participants dating back to 2005.¹⁶ These visit records include the findings from the clinical exam, (i.e. visual acuity, intraocular pressure, slit lamp exam findings, and dilated exam findings), as well as the clinician’s assessment and plan. The text mining algorithm, implemented in Python,¹⁷ employs pattern recognition on all sections of optometry and ophthalmology notes to identify mentions of target eye-related features along with contextual information including eye laterality (OD, OS, OU, or unknown), negation status (e.g., negated, hypothetical, etc.), note section information (e.g., “macula”, “OCT” interpretation), and dates (differentiate current findings from historical data copied from a previous encounter). Heuristics are then applied to evaluate this preliminary output and determine the most relevant information for each eye-related feature. (Supplemental Methods 1) The text mining algorithm is evaluated and performance is improved through an ongoing validation iterative process (Supplemental Methods 1, Supplemental Table S1). As of April 2023, Eye ACT had abstracted eye-related health records from 4,037 ACT participants during 75,548 clinical eye encounters between January 1^st, 2005 through the earliest of a participant’s disenrollment from ACT, death, or October 31^st, 2022.¹⁵ This study was approved by the Institutional Review Board of Kaiser Permanente Washington Health Research Institute and conducted in accordance with the Declaration of Helsinki.

Development of CFA-based AMD-severity score models

We developed two continuous severity scoring systems using clinical features observed from the most recent visit of 8,848 AREDS/AREDS2 participants. The most recent visit was chosen to capture the widest range of AMD severity. Analyses were conducted at the eye level. We developed separate models for nonexudative (AMD-D) and exudative (AMD-W) AMD severity using a CFA framework.¹⁸

CFA models assume that AMD severity can be represented by latent severity scores based on observation of clinical features. In short, the latent severity scores are unmeasured random variables that represent non-exudative and exudative AMD severity. Each clinical feature is an observed characteristic (e.g., drusen, hemorrhage) from individual eyes. These clinical features are treated as indicators of an underlying severity level. The model parameterizes each clinical feature with two parameters – a difficulty level and discrimination. Consider a dichotomous (present/absent) indicator such as the presence of subretinal fluid (SRF). The difficulty level for the SRF indicates the severity of AMD where 50% of eyes are expected to have SRF. Eyes with higher AMD severity would be expected to have <50% with SRF, while eyes with AMD severity higher than that level would be expected to have >50% with SRF. The discrimination parameter quantifies the steepness of the relationship near the difficulty level; features with higher discrimination parameters will have tighter relationships near the difficulty level, and those with lower discrimination parameters will have looser relationships.

These CFA models are also known as item response theory (IRT) models.^19–21 These parameters can be used to describe the performance of each feature (item) with item information curves, which plot the independent contribution of each item to overall measurement precision in estimating severity scores.^22–25 Further information about these models and the performance of each item can be found in Supplemental Methods 2 and Supplemental Figure S1. We selected retinal features to include in our scoring system based on their availability and detail in the AREDS/AREDS2 and Eye ACT datasets (Table 2). Two ophthalmic experts (EC, CL) specified the eye features considered for AMD-D and AMD-W.^26,27 We used abacus plots to visually represent the relationship between CFA-based severity scores, after rescaled to 0–10 range, and all possible combinations of clinical features, making it easier to interpret how different feature patterns correspond to severity levels based on the AREDS/AREDS2 dataset.

Table 2.

Subclinical ophthalmic characteristics for the age-related macular degeneration (AMD) confirmatory factor analysis (CFA)-based severity score system in AREDS/AREDS2 and Eye ACT.

Characteristics	Definition
Subretinal fluid	Presence of subretinal fluid (0 - no presence; 1 - presence)
Subretinal hemorrhage	Presence of subretinal hemorrhage (0 - no presence; 1 - presence)
Scar	Presence of scar (0 - no presence; 1 - presence)
Non-drusenoid PED	Presence of non-drusenoid PED (0 - no presence; 1 - presence)
Pigmentary changes	Presence of pigmentary change, including both hypo and hyperpigmentation (0 - no presence; 1 - presence)
Geographic atrophy	Presence of geographic atrophy (0 - no presence; 1 - presence)
Drusen	A 4-category composite variable on the presence and size of hard drusen and the presence of soft drusen. (0 - no presence of either hard drusen or soft drusen, 1 - only moderate hard drusena, 2 - moderate hard plus any soft drusen, 3 - large hard drusenb)

^aHard drusen size ≥ 1/24 disk diameter (~63 μm) and < 1/12 disk diameter (~125 μm)

^bHard drusen size ≥ 1/12 disk diameter (~125 μm)

ACT, Adult Changes in Thought; PED, pigment epithelial detachment

For AMD-D, we modeled severity based on drusen characteristics (hard drusen size, presence of soft drusen), geographic atrophy (GA), and pigmentation changes. For AMD-W, we modeled severity based on SRF, SRH, scars, pigmentation changes, and non-drusenoid pigment epithelial detachment (PED) (Table 2). The models were built using datasets comprising data from 17,618 and 15,235 eyes for AMD-W and AMD-D, respectively, from each individual’s most recent AREDS/AREDS2 visit. We estimated both difficulty and discrimination parameters for each feature and assessed model fit using the comparative fit index (CFI) and the root mean square error of approximation (RMSEA).^28–30 A good model fit was defined as CFI >0.95 and RMSEA <0.08.^29–31

The models were fit using the package mirt in R 4.0.2 with a full information maximum likelihood approach.³² Based on the estimated difficulties and discriminations, a severity score for a given combination of features was estimated using Bayesian approach via the expected a posteriori estimate.^33–35 Estimated severity scores for each eye were rescaled from 0 (least severe) to 10 (most severe).

Comparisons between CFA-based scores and AREDS scores in the AREDS/AREDS2 held-out test set

To understand correlation between CFA-based severity scores and traditional AREDS scores, we applied the CFA-based severity system to an AREDS/AREDS2 held-out test set consisting of all observations in the AREDS/AREDS2 dataset except for the most recent visit data. We evaluated relationships between CFA-based scores and AREDS scores in the held-out test set, and Pearson’s correlation was used to assess the alignment between AMD-D and AREDS scores 1–10. Detailed methods are found in Supplemental Methods 3.

Application of CFA-based severity system to Eye ACT data from routine clinical care

We applied the CFA-based severity system to text mining-extracted data from routine clinical care in the Eye ACT cohort to calculate the AMD-D and AMD-W severity scores for 8,074 eyes on 4,037 Eye ACT participants. We inspected longitudinal trajectories of AMD-D and AMD-W scores as described below along with their non-parametric mean trajectories and 95% confidence bands using a generalized additive model with cubic splines.³⁶

For the AMD-D assessment, we restricted to 4,412 eyes from 2,248 participants that had at least one AMD-D score within 3 years of age 80 and that had never received antiVEGF. We then examined longitudinal trajectories by computing change scores (i.e., differences relative to the score at age 80) for the eyes across time. For the AMD-W evaluation, we restricted to the 220 eyes from 171 participants that were clinically diagnosed with AMD (based on International Classification of Diseases (ICD)-9 or 10 codes) and treated with antiVEGF (Current Procedural Terminology [CPT] code 67028). Then, using the score at 1^st antiVEGF treatment as the reference, we examined longitudinal trajectories by computing change scores (i.e., differences relative to the score at first treatment) for the eyes across time around treatment. To provide further context and interpretation to use of initial antiVEGF treatment as a reference point, we also summarized the time between the first ICD code diagnosis of exudative AMD and the first CPT code for antiVEGF treatment.

RESULTS

Characteristics of AREDS/2 and Eye ACT study participants are shown in Table 3. The 17,696 individual eyes of 8,848 participants in the AREDS/2 study were followed for a median of 5.4 years (Interquartile range [IQR]: 4.7, 10.3) with a median of 6 visits (IQR: 5, 10). The 8,074 individual eyes of 4,037 participants in Eye ACT had longer available follow-up on average, with a median of 10.1 years (IQR: 4.9, 14.8) and had more encounters on average with eye care providers with a median of 14 visits (IQR: 6, 25).

Table 3.

Baseline characteristics and data availability of Eye Adult Changes in Thought (Eye ACT) and AREDS/AREDS2 participants in the analytic set.

	AREDS/AREDS2	Eye ACT
Total number of participants	8,848	4,037
Total number of eyes	17,696	8,074
Mean follow-up time in years (SD)*	7.0 (3.6)	9.6 (5.6)
Median follow-up time in years (IQR)*	5.4 (4.7, 10.3)	10.1 (4.9, 14.8)
Mean number of eye encounters (SD)*	7.1 (3.1)	18.7 (17.8)
Median number of eye encounters (IQR)*	6 (5, 10)	14 (6, 25)
Mean age in years at the baseline eye encounter (SD)	71.1 (6.7)	74.1 (9.8)
Median age in years at the baseline eye encounter (IQR)	71.1 (66.3, 72.0)	74.2 (66.0, 81.8)
Sex (Male)	3,864 (43.7%)	1,665 (41.2%)
Race and ethnicity
White, non-Hispanic	8,435 (95.3%)	3,550 (88.2%)
Black, non-Hispanic	230 (2.6%)	126 (3.1%)
Hispanic	96 (1.1%)	59 (1.5%)
Asian and Pacific Islander	45 (0.5%)	149 (3.7%)
Others	42 (0.5%)	139 (3.5%)
Missing	–	14
Holding a Bachelor’s or higher degree	3,243 (37.0%)	2,159 (53.5%)
Missing	81
≥1 APOE ε4 allele	–	855 (27.0%)
Missing	–	866
Smoking history (Yes)**	4,975 (56.2%)	694 (17.2%)
Diabetes history (Yes)***	927 (10.5%)	643 (15.9%)
Hypertension history (Yes)***	4,240 (48.0%)	2258 (56.0%)
Missing	6

^*IQR, interquartile range; SD, standard deviation

^**Smoking history was self-reported by participants in AREDS/AREDS2 and collected from electronic medical records and ACT baseline survey in Eye ACT.

^***Diabetes and hypertension history was self-reported by participants in AREDS/AREDS2 and collected from EMR based on International Classification of Diseases 9/10 codes in Eye ACT.

The CFA-based severity scores for AMD subtypes

The mappings of all combinations of non-exudative AMD clinical features to CFA-based AMD-D scores and exudative AMD features to CFA-based AMD-W scores are illustrated with abacus plots. Unlike the traditional AREDS “nine-step” score, there were 16 possible unique score values in the AMD-D model and 32 possible unique scores in the AMD-W model. Both the AMD-D (CFI=0.99 and RMSEA=0.021) and AMD-W (CFI=0.97 and RMSEA=0.019) scores demonstrated good fit statistics.(Table 4)

Table 4.

The estimated difficulty and discrimination parameters of non-exudative and exudative severity scores on individual subclinical ophthalmic features and goodness of fit measures of the confirmatory factor analysis-based models (AMD-D and AMD-W) using the training set in AREDS/AREDS2 cohorts.

	AMD-D severity score		AMD-W severity score
	Estimated discrimination	Estimated difficultya	Estimated discrimination	Estimated difficultya
Subretinal fluid	--	--	2.30	1.77
Subretinal hemorrhage	--	--	1.88	2.21
Scar	--	--	2.20	1.67
Non-drusenoid PED	--	--	1.12	2.63
Pigmentation changes	3.20	−0.07	0.59	−0.22
Geographic atrophy	1.02	1.72	--	--
Drusen – category 1b	1.21	−0.88	--	--
Drusen – category 2b		−0.65	--	--
Drusen – category 3b		−0.27	--	--
Model goodness of fit assessmentc
Comparative Fit Index (CFI); higher is better	0.99		0.97
Root mean square error of approximation (RMSEA); lower is better	0.021		0.019

^aDifficulty was calculated as the difficulty parameter (b_m) rescaled by the reciprocal of the discrimination parameter (a_m) following the notations in Supplemental Method 2.

^bDrusen: category 1 - only moderate hard drusen; category 2 - moderate hard plus any soft drusen; category 3 - large hard drusen. (Please see Table 2 for more details)

^cModel goodness of fit was obtained on data without missing values (N=15,172 for AMD-D and N=15,193 for AMD-W scores).

PED, pigment epithelial detachment

The abacus plot in Figure 2 (panel A) shows how different combinations of retinal features correspond to varying levels of non-exudative AMD severity, as captured by the AMD-D score. For example, the presence of a moderate-size hard drusen alone—or in combination with a soft drusen but without other non-exudative AMD features—is associated with low severity (AMD-D score <2.13). At moderate severity levels (AMD-D score: 2.23–6), we observe more frequent co-occurrence of features such as large-size hard drusen, geographic atrophy (GA), and pigmentation changes. These may appear individually or in combinations—for example, moderate-size or larger drusen with GA, or with pigmentation changes. Higher AMD-D scores (>6) reflect the co-occurrence of at least two hallmark features, such as pigmentation changes and GA, with or without large-size drusen, signaling more severe non-exudative AMD.

Figure 2. All possible combinations of subclinical ophthalmic features and the corresponding rescaled confirmatory factor analysis (CFA)-based non-exudative age-related macular degeneration (AMD-D) (panel a) and exudative (AMD-W) (panels b,c) severity scores.

A value of 0 on the x-axis represents the least severe status and 10 the most severe status. The dots that are lined up vertically at each severity score value indicate the corresponding manifesting features. This figure demonstrates the relationship between each unique combination of subclinical ophthalmic features and confirmatory factor analysis AMD-D and AMD-W severity scores based on the estimated parameters inferred using AREDS/AREDS2. In addition, it shows the relative severity of each unique feature combination in relation to others. PED, pigment epithelial detachment.

For exudative AMD, the presence of either pigmentation changes or non-drusenoid PED alone is associated with low severity (AMD-W score <4.3). (Figure 2, panels B,C) Moderate severity (AMD-W score: 4.3–6.95) is typically characterized by either the presence of a single feature strongly indicative of exudative AMD—such as subretinal fluid, scar, or hemorrhage—with or without pigmentation changes and/or non-drusenoid PED, or by the co-occurrence of pigmentation changes and non-drusenoid PED. Severe exudative AMD (AMD-W score >6.95) corresponds to the co-occurrence of at least two hallmark exudative features—specifically subretinal fluid, SRH, and scar.

The total information curves shown in Figure S3 indicate that the estimated AMD-D severity score is most precise for individuals of moderate severity level of non-exudative AMD, primarily attributed to the presence/absence of pigmentation changes. The presence of GA provided the most information at the high end of the non-exudative AMD severity continuum compared to the mild to moderate stages of non-exudative AMD, as its corresponding information curve was higher with worsening non-exudative AMD severity.(Figure S3) AMD-W (Figure S4) provide the most information for individuals with moderate to highly severe exudative AMD, a consequence of the joint contribution of SRF, SRH, and scar.

Comparison between CFA-based severity scores and AREDS scores in the AREDS/AREDS2 held-out test set

There was a general alignment (Pearson correlation of 0.877, 95% CI 0.876–0.879) between AMD-D scores and AREDS severity scoring system scores 1–10 (exclusively dry AMD scores). (Figure 5) Additional results are found in Supplemental Results.

Figure 5. Comparison of the rescaled confirmatory factor analysis (CFA)-based AMD-D (panel a) and AMD-W (panel b) severity scores to the AREDS scores in the AREDS/AREDS2 held-out test set.

(No. of visits = 98,461 in panel a and 106,195 in panel b). Box plots (red) were shown to demonstrate the distribution of CFA-based AMD-D and AMD-W scores stratified by the AREDS scores.

CFA-based severity scores and patterns of change in the Eye ACT cohort

On the review of clinical encounters from 8,074 eyes of 4,037 Eye ACT participants where missing data were more frequent, we were able to generate AMD-D and AMD-W severity scores on 151,096 individual eye visit records with a median score of 0 (IQR: 0, 5.07 for AMD-D and 0, 2.10 for AMD-W). These lower levels of AMD severity in Eye ACT are expected as both AREDS and AREDS2 enrolled people with AMD, while ACT’s participants are from a community-based cohort not selected based on AMD. The joint distribution of the two AMD severity scores for 151,096 eye-visit records (Table S5) showed that 68.7% (n=103,875) had a value of 0 for both scores. Among visits documenting at least one feature (n=45,784), 91.8% (n=42,005) had either pigmentation changes or at least two AMD-D or AMD-W features, suggesting at least moderate to severe AMD for at least one score. There was moderate positive correlation between the AMD-D and AMD-W scores in the Eye ACT cohort (Pearson 0.702, 95% CI: 0.699–0.704).

Figure 6 shows the pattern of AMD-D scores of 4,412 eyes from 2,248 participants, which never received antiVEGF treatments, in relation to severity at age 80. For the majority of eyes, the AMD-D scores either remained stable or showed an increasing trend. The fitted mean curve (in yellow) demonstrated a minor increase in AMD-D scores before the age of 80 on average, followed by a steeper increase through age 90 years.

Figure 6. Trajectories of non-exudative age-related macular degeneration (AMD-D) severity score by age in Eye ACT.

For each eye, the difference in rescaled AMD-D severity score measured at each eligible clinical encounter compared to the rescaled AMD-D score at age 80 years old, as approximated by the rescaled score at the age closest to 80 within 77 to 83 years data from 4,412 eyes from 2,248 participants that never received antiVEGF and with at least one AMD-D score between age 77–83 years in the Eye Adult Changes in Thought (Eye ACT) cohort. The trajectories of scores for each individual eye are marked as gray curves. The golden curve and yellow band represented the smoothed mean curve on rescaled AMD-D score change and the corresponding 95% confidence band.

Figure 7 shows AMD-W score trajectories of 220 eyes from 171 participants, which received an initial antiVEGF treatment and had an available AMD-W score. The majority of treated eyes showed an increasing pattern of AMD-W scores months prior to the antiVEGF treatment. There were various post-antiVEGF patterns with the majority remaining unchanged due to the ceiling effect of the score or exhibiting a slight increase. The fitted mean curve of the CFA-based scores had a steeper rise in the window of time before the 1^st antiVEGF treatment followed by a plateau after the treatment. For the analysis of time between first ICD code diagnosis of exudative AMD and first antiVEGF treatment, 52% of participants had antiVEGF on the same day as the first ICD code diagnosis of exudative AMD, >70% of participants received their first antiVEGF within 7 days of initial ICD code diagnosis, and >80% had their first antiVEGF within 35 days of ICD code diagnosis.

Figure 7. Trajectories of exudative age-related macular degeneration (AMD-W) severity scores relative to the score at 1^st vascular endothelial growth factor inhibitor (antiVEGF) therapy in Eye ACT.

For each eye, the difference in the rescaled AMD-W score measured at each eligible clinical encounters compared to the rescaled score at the time of 1st antiVEGF treatment on 220 eyes from 171 participants that received antiVEGF treatment and had a valid AMD-W score at the time of 1st antiVEGF treatment in the Eye Adult Changes in Thought (Eye ACT) cohort. The trajectories of score change on individual eyes are marked as gray curves. The golden curve and yellow band represented the smoothed mean curve on rescaled AMD-W score change and the corresponding 95% confidence band.

DISCUSSION

We utilized a CFA framework to develop novel severity models using AMD clinical features collected in the AREDS/AREDS2 cohorts and applied this approach to a routine clinical EMR database. Rarely are all elements included in standard AREDS scores available from EMR datasets from routine clinical care. However, our approach was able to generate scores for the Eye ACT cohort when any subset of features for scoring severity were available from the EMR. When we applied the CFA-based scoring system to the routine EMR data collected in the Eye ACT study cohort, CFA-based scores showed increasing AMD-D scores with older age and a sharp rise in the AMD-W scores several weeks to months before the 1st antiVEGF treatment.. Several AMD severity scoring systems have been described in the literature, primarily involving 3-field fundus photographs and more recently with optical coherence tomography (OCT). The Wisconsin scoring system was one of the earliest methods and developed for use in two large population studies, the Beaver Dam Eye Study and the Framingham Eye Study. The scoring system assesses multiple characteristics of AMD (drusen size/area, retinal pigment epithelium [RPE] degeneration, increased pigment, GA, etc.) in fundus photos and relies on a grid overlay to identify the macular subfields with disease involvement.⁴ Data from AREDS led to the development of a 9-step severity scale for AMD in 2005, which expanded upon the Wisconsin scale and became the current operational gold standard for disease severity assessment. Drusen characteristics, pigmentary abnormalities, and features indicative of neovascular AMD were assessed and scored in zones and globally.⁷ A simplified “risk factor” version of this scale was also developed for clinical use, enabling physicians to quickly estimate the 5-year risk of developing advanced disease.⁸ Additional scoring systems have been developed since, such as the Beckman Initiative for Macular Research Classification, also based upon the AREDS AMD scale, which categorizes patients as having early, intermediate or late AMD based on the progression of drusen of increasing size, subsequent pigmentary abnormalities, and the development of exudative AMD and/or GA.³⁷ All these scoring systems are based on a limited number of discrete severity categories defined by the presence/absence of individual disease features, and none of them provide a granular severity scale for exudative AMD or simultaneous scoring for both types of AMD. In addition, many of the features used in these scoring systems may not be routinely captured in the clinical setting, and therefore it is difficult to apply them retrospectively to clinical datasets due to missingness.

Unlike traditional scoring methods that categorize AMD severity with pre-defined criteria, the CFA-based system treats severity as a “latent variable,” inferred from multiple subclinical ophthalmic features. The model estimates severity based on the presence or absence of these features recorded in the EMR. The weighting of each feature on overall severity is estimated from the data, as opposed to assigning a priori weights to each feature. This has the advantage of not assuming that each feature has equal relevance to severity. Our findings from the AREDS/AREDS2 data suggest that the CFA models fit the data very well.

These flexible CFA models also are quite adept at handling missing data. Once difficulty and discrimination parameters for a feature are estimated, they can be used regardless of what other data are available. This characteristic enables ascertaining underlying levels of constructs (severity) for eyes only with subsets of features observed. Since it is uncommon to have all of the elements needed for the “nine-step” AREDS score in routine clinical care, some approach that is adept at addressing missing data is needed if we are to study longitudinal relationships with AMD severity using AREDS scores. We illustrate this strength of the proposed CFA-based severity model in this work.

A notable strength of our approach is that our scores separate non-exudative and exudative AMD as separate phenomena quantified by the AMD-D and AMD-W severity scores as these two etiologies may differ mechanistically. For non-exudative AMD, our findings suggest that equal unit weighting of all of the AREDS features is not well supported by the data. And for exudative AMD, our findings suggest that there is substantial heterogeneity in the severity of exudative AMD, all of which is obscured in the standard AREDS score with a single point value.

We found not surprisingly that severity of exudative and non-exudative AMD were correlated with each other. In the Eye ACT study dataset, the AMD-D scores behaved as we had expected with aging. Scores were higher on average with age until they appeared to plateau, consistent to what we observe clinically.³⁹ It is important to note that CFA-based scores are not meant to be interpreted as a chronological scale, but they provide a scoring system that integrates various combinations of features that occur in the process of developing advanced AMD and how they contribute to overall severity. As previously reported by Klein et al, the appearance of retinal risk features before the onset of central GA is known to vary significantly.⁴⁰ The CFA-based scoring system provided ranking of the retinal risk features and their combinations and quantified varying amount of information contributed by individual features to the overall severity along the AMD severity continuum.

Understanding trends in exudative AMD scores may be more challenging as these depend highly on treatment received and the response. However, our results suggest a wide range of severity can be quantified in patients with an AREDS score of 11.^7,12. Furthermore, we were somewhat surprised that we could detect signals of impending conversion from non-exudative to exudative AMD using EMR data alone, as evidenced by the sharp rise in AMD-W scores even 6 months prior to the initial antiVEGF treatment. Although some of this trend could be due to a potential delay between the clinical diagnosis of exudative AMD and patients receiving their first antiVEGF treatment, our analysis found that the majority of patients (>70%) received their first antiVEGF within a week of their first ICD code diagnosis of exudative AMD in Eye ACT.

Our results suggest that even a few variables from routine clinical EMR data may be valuable in providing insights into AMD natural history, especially when analyzed over multiple clinical visits. Exploring the longitudinal relationship between CFA-based severity scores and visual outcomes extracted from the EMR would be an interesting next step for exploring the utility of the CFA-based scoring system in predicting disease progression and functional vision loss. In addition, future work could incorporate longitudinal fellow-eye interactions to enhance predictive modeling for AMD progression.

Undoubtedly, retinal imaging is powerful and provides novel methods of characterizing severity. For example, the Classification of Atrophy Meetings (CAM) program developed a system for classification of AMD based on OCT imaging to better characterize progression from intermediate AMD to GA.⁴¹ In doing so, CAM defined new image-based markers or stages of AMD, including incomplete RPE and outer retinal atrophy (iRORA) without neovascularization, which was thought to be a precursor to GA,⁴¹ while GA itself was defined as complete RPE and outer retinal atrophy (cRORA).⁴² While these results with the newer OCT modality are exciting, being able to leverage decades of previously unused clinical data as we have done here may also be beneficial for studying AMD-associated outcomes. In addition, as more recently described biomarkers and/or interpretations of OCT imaging become more frequently noted in the new data sources, the flexible CFA approach we took to AMD-D and AMD-W could be extended to include new features or biomarkers, such as reticular pseudodrusen or other imaging related biomarkers. Recent advances in large language models may provide effective methods for ingesting archived EMR data providing unparalleled longitudinal data for future studies.

There are many potential applications for CFA-based severity scores. The CFA-based system can be integrated into EMR and the scores can be automatically generated at each clinical encounter. These scores may prove valuable for clinicians to assess patients’ disease progression. In addition, the scoring system’s reliance on information from the clinical features collected in EMR could provide more efficient monitoring of subtle changes in disease course, especially for exudative AMD. This method of analyzing EMR clinical data from all previous visits to identify severity trends could inform the physician of possible disease progression or improvement. For clinical trials, these detailed continuous severity scores could be useful for identification and screening of participants.

Furthermore, our CFA-based model could potentially be applied to other ophthalmic diseases. CFA/IRT models have been used in educational testing for decades, since the introduction of IRT.⁴³ In 1967, Samejima introduced the graded response model that was able to address ordinal data such as the AMD severity data considered here.²¹ In her later discussion of the graded response model,³⁵ Samejima points to its early adoption in estimating skeletal maturity from expert ratings of thirty-four features of the knee on radiographs.⁴⁴ More recent examples include periodontal literature⁴⁵ and knee arthroplasty outcomes.⁴⁶ These models seem particularly useful for a common situation in which it may be useful to incorporate disparate indicators of an unobserved latent severity level in a single composite. In this instance, we applied this CFA approach to quantify both exudative and non-exudative AMD severity, and the models appear to work well, providing additional clinical insights beyond what can be realized with the standard AREDS severity scoring system score. Similar approaches may be applied to develop severity models of other eye diseases.

Our study has several limitations. The CFA scoring system approach is dependent on what clinicians have included in their clinical notes. If the AMD features collected in AREDS/AREDS2 studies were never present or very rarely present in the Eye ACT EMR (e.g., distinction between hypo- and hyperpigmentation, type of PED), these features were excluded from consideration when developing the scoring system. In addition, the distinction between soft and hard drusen and the size of soft drusen were rarely available in the EMR data. Therefore, soft drusen were only evaluated in a binary (presence/absence) manner as part of the composite drusen variable. We also relied on a text mining algorithm to extract the Eye ACT from the EMR, but its validation metrics were reassuring. Despite these limitations, we intended this to be a proof-of-concept study relying on a large clinical EMR database as opposed to rigorous clinical trial data, and our results support the potential usefulness of such an approach. Our current models could be applied to other existing EMR databases and provide useful scoring systems in datasets for which AREDS scoring would not be suitable due to missing data.

AMD is a complex disease that requires careful management to prevent vision loss. The severity scoring system described here, developed from multiple clinical observations over time from a large cohort of well-characterized participants, provides clinicians and researchers with an additional and more nuanced, continuous scoring system to characterize AMD severity and monitor progression. This scoring system enables retrospective analysis in existing datasets from routine care where AREDS scoring may not be applicable due to likely large amounts of missing data. This work also demonstrates how accessible longitudinal clinical EMR data may be used for granular disease characterization, which may be applicable to other diseases beyond AMD.

Abbreviations:

ACT

Adult Changes in Thought

AMD

age-related macular degeneration

AMD-D

non-exudative AMD

AMD-W

exudative AMD

antiVEGF

vascular endothelial growth factor inhibitors

AREDS

Age-Related Eye Disease Study

CAM

Classification of Atrophy Meetings

CFA

confirmatory factor analysis

CFI

comparative fit index

CI

confidence interval

cRORA

complete retinal pigment epithelium and outer retinal atrophy

EMR

electronic medical records

GA

geographic atrophy

ICD

International Classification of Diseases

iRORA

incomplete retinal pigment epithelium and outer retinal atrophy

IQR

interquartile range

IRT

item response theory

OCT

optical coherence tomography

PED

pigment epithelial detachment

RMSEA

root mean square error of approximation

RPE

retinal pigment epithelium

Data sharing:

Data from this analysis cannot be made publicly available for ethical and legal reasons. In order to replicate our findings, a researcher may need access to personal health identifiers (PHI) Version 8/6/2024 2 including dates of birth and death, dates of diagnoses, and ages over 89. These are required variables for the analysis, and we cannot publicly release this information without IRB approval and a Data Use Agreement with interested researchers. However, the datasets used and/or analyzed in the current study are available upon reasonable request and execution of appropriate human subjects review and data sharing agreements by following the process described on the Adult Changes in Thought (ACT) website: actagingresearch.org.”