Application of the Sight Outcomes Research Collaborative Ophthalmology Data Repository for Triaging Patients With Glaucoma and Clinic Appointments During Pandemics Such as COVID-19

Key Points

Question

During a pandemic, can researchers use large data sets to help ophthalmology clinics identify upcoming glaucoma appointments to safely postpone and prioritize appointments for rescheduling during ramp-up (reopening) periods?

Findings

In this cross-sectional study, an algorithm was developed that considered glaucoma severity and progression risk plus the morbidity risk from potential coronavirus disease 2019 exposure for 1034 upcoming glaucoma patient appointments. It identified patients whose appointments could safely get postponed and facilitated prioritization of appointments for rescheduling.

Meaning

These findings suggest that researchers can leverage big data to triage ophthalmic clinic appointments, balancing the glaucoma progression risk against the morbidity risk from coronavirus disease 2019 exposure during ophthalmic care.

This cross-sectional study used an algorithm for patients with glaucoma that considered glaucoma severity and progression risk vs high-risk features for COVID-19 morbidity to assess which patients can safely postpone appointments during the COVID-19 pandemic.

Abstract

Importance

During the coronavirus disease 2019 (COVID-19) pandemic, eye care professionals caring for patients with sight-threatening diseases, such as glaucoma, have had to determine whether some patient appointments could safely get postponed, weighing the risk that the patient’s glaucoma could worsen during the interim vs the morbidity risk of acquiring COVID-19 while seeking ophthalmic care. They also need to prioritize appointment rescheduling during the ramp-up phase (when pandemic-associated service reductions are eased).

Objective

To describe a flexible and scalable scoring algorithm for patients with glaucoma that considers glaucoma severity and progression risk vs the presence of high-risk features for morbidity from COVID-19, using information from a large data repository.

Design, Setting, and Participants

In this cross-sectional study, patients with upcoming clinic appointments for glaucoma from March 16, 2020, to April 16, 2020, at an academic institution enrolled in the Sight Outcomes Research Collaborative (SOURCE) Ophthalmology Electronic Health Record Data Repository were identified. A risk stratification tool was developed that calculated a glaucoma severity and progression risk score and a COVID-19 morbidity risk score. These scores were summed to determine a total score for each patient.

Main Outcomes and Measures

Total scores and percentages of clinic appointments recommended for rescheduling.

Results

Among the 1034 patients with upcoming clinic appointments for glaucoma, the mean (SD) age was 66.7 (14.6) years. There were 575 women (55.6%), 733 White individuals (71%), and 160 Black individuals (15.5%). The mean (SD) glaucoma severity and progression risk score was 4.0 (14.4) points, the mean (SD) COVID-19 morbidity risk score was 27.2 (16.1) points, and the mean (SD) total score was 31.2 (21.4) points. During pandemic-associated reductions in services, using total score thresholds of 0, 25, and 50 points would identify 970 appointments (93.8%), 668 appointments (64.6%), and 275 appointments (26.6%), respectively, for postponement and rescheduling. The algorithm-generated total scores also helped prioritize appointment rescheduling during the ramp-up phase.

Conclusions and Relevance

A tool that considers the risk of underlying ophthalmic disease progression from delayed care receipt and the morbidity risk from COVID-19 exposure was developed and implemented, facilitating the triage of upcoming ophthalmic appointments. Comparable approaches for other ophthalmic and nonophthalmic care during the COVID-19 pandemic and similar crises may be created using this methodology.

Introduction

Coronavirus disease 2019 (COVID-19) was characterized as a pandemic in March 2020.¹ Given the rapidity and extent of the spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in the US, a number of public health agencies including the US Centers for Disease Control and Prevention (CDC) recommended physical distancing measures, including staying home as much as possible, to reduce the risk of virus acquisition and transmission.² Following guidance from the CDC, on March 18, 2020, the American Academy of Ophthalmology recommended that ophthalmic practices limit care to only patients with urgent or emergency ocular conditions.³ Therefore, a need quickly arose for eye care professionals to distinguish urgent from nonurgent eye care and weigh the risks and benefits associated with decisions to proceed with or postpone care for patients with previously scheduled appointments. Patients with chronic ocular diseases, such as glaucoma, risked experiencing irreversible visual acuity or visual field loss from disease progression if they were unable to receive timely medical care, yet a disproportionately large number of patients with glaucoma are also of older age and have medical comorbidities that predispose them to morbidity and mortality from COVID-19.⁴ Furthermore, in-person appointments would increase risk of COVID-19 transmission to other patients, clinic staff, and clinicians^5,6 and may injudiciously consume scarce personal protective equipment.

We describe an algorithm we developed to assist eye care professionals with determining for which patients the individual risk balance favored keeping their previously scheduled appointments vs having their appointments postponed. This algorithm was developed to be easy to implement, given the urgency of the situation; scalable, so that it could assist clinicians with making determinations for large numbers of patients; and flexible enough to accommodate changes in guidance from the CDC or other agencies, which was anticipated, given the rapidly changing environment. The scoring algorithm we developed could also be used to prioritize rescheduling patient appointments that had been postponed or canceled, in times when local, state, and federal authorities ease restrictions and professional societies encourage care expansion.⁷

Methods

Data Source

Data were derived from the Sight Outcomes Research Collaborative (SOURCE) Ophthalmology Data Repository, which captures electronic health record (EHR) data of all patients receiving any eye care at academic medical centers participating in this consortium. SOURCE captures information on patient demographics, diagnoses identified based on International Statistical Classification of Diseases and Related Health Problems, Tenth Revision billing codes, and structured and unstructured (free-text) data from all clinical encounters (eg, clinic visits, operative reports), as well as results from ocular diagnostic tests. Researchers have published other studies⁸ using data from SOURCE. Given the time-sensitive nature of this study, our analyses focused on a subset of patients diagnosed with glaucoma in SOURCE who were receiving care at the University of Michigan Kellogg Eye Center, Ann Arbor. The University of Michigan institutional review board approved this study and waived the need for informed consent because the primary use of the scoring algorithm we developed was to assist with appointment triaging rather than research.

Inclusion and Exclusion Criteria

All patients with Kellogg Eye Center Glaucoma Service clinic appointments from March 16, 2020, to April 16, 2020, were identified. This included patients with visits scheduled with ophthalmologists and optometrists on the service. Persons younger than 18 years and patients who had never previously been seen at the institution were excluded.

Capturing Risk Factors for Morbidity and Mortality From COVID-19 in the Electronic Health Record

Persons at greatest risk for morbidity and mortality from COVID-19 are individuals of older age, pregnant women, and those with chronic medical illnesses.⁴ Older age was defined as 65 years or older. Greater statistical weight was assigned to persons 80 years or older compared with those aged 65 to 79 years, consistent with data that demonstrate worse prognosis with advancing age. Women who were pregnant or had recently given birth and may have been breastfeeding were considered high risk, too. The Charlson Comorbidity Index (CCI), a validated measure of overall health,⁹ was used to quantify the medical comorbidities of each patient.

Capturing Glaucoma Severity and Risk for Progression in the Electronic Health Record

The SOURCE repository was searched to identify selected clinical characteristics of patients with glaucoma that could help identify patients who require closer monitoring, either because of greater disease severity or higher risk of progression. These factors included incisional intraocular surgery in the past 3 months, records of high (≥30 mm Hg) or low (<6 mm Hg) intraocular pressure (IOP) in the past year, considerable visual field loss in 1 or both eyes, and monocularity. Considerable visual field loss was defined as the last recorded mean deviation on the Humphrey Field Analyzer (HFA) (Carl Zeiss Meditec) of −12 dB or worse in 1 or both eyes. Monocularity was defined as a best-recorded visual acuity at the most recent ocular visit of 20/40 or better in 1 eye and 20/80 or worse in the contralateral eye.

Determining Scores

For each patient, a COVID-19 morbidity risk score (CS) and glaucoma severity and progression risk score (GS) were determined by assigning point values to the noted characteristics and calculating the sum (Table). A value of 0 was used when data for selected characteristics were missing. Because the system was intended to provide guidance regarding which patients’ appointments could safely get postponed and rescheduled, positive point values were treated as favoring rescheduling the appointment and negative point values as favoring retaining the previously scheduled appointment. A more negative GS represented a greater likelihood of attaining benefit from the patient attending the previously scheduled clinic appointment. A more positive CS represented an increased risk for morbidity or mortality from COVID-19 and thus a greater benefit from sheltering in place and postponing a previously scheduled clinic appointment. Next, a total score (TS) was determined to capture the competing risks conferred by GS and CS by taking this sum: total score = (COVID-19 morbidity risk score) + (glaucoma severity progression risk score).

Table. Glaucoma Severity and Progression Risk Score and COVID-19 Morbidity Risk Score Point Values^a.

Characteristic	Points
Glaucoma severity and progression risk score
Intraocular surgery within the past 3 mo	−25
Monocularityb	−10
More than 50% of recorded IOPs <6 mm Hg or >30 mm Hg in the past 12 mo	−10
Mean deviation
<−12 dB in both eyes	−10
<−12 dB in 1 eye and ≥−12 dB in the contralateral eye	−5
>−6 dB in both eyes	10
All IOPs in both eyes in the past 12 mo ≥6 mm Hg and ≤30 mm Hgc	10
COVID-19 risk score
Pregnant or breastfeeding	25
Age, y
≥80	25
65-79	10
Charlson Comorbidity Index
>4	30
3-4	20
1-2	10

Abbreviation: IOP, intraocular pressure.

^aPoint values assigned to each characteristic, for the glaucoma severity and progression risk score and the coronavirus disease 2019 morbidity risk score.

^bDefined as a best-recorded visual acuity score of 20/40 or better in 1 eye and 20/80 or worse in the contralateral eye.

^cA patient must have 1 or more IOP measurement recorded over the past 12 months to be eligible for this.

Thresholds for Recommending Appointment Postponement and Rescheduling

The aim was to translate these scores into actionable information for clinicians. Therefore, the percentage of appointments that would be recommended to be postponed during the period associated with pandemic-induced reductions in services based on TS thresholds of 0 or more points, 25 or more points, and 50 or more points were determined.

Validation

The glaucoma service faculty were required to review the EHRs of all patients with clinic appointments from March 16, 2020, to May 5, 2020, and assign each patient to 1 of 3 categories: tier 1 (urgent; patients who should retain their existing appointments or, for those who canceled their appointments, those with follow-up required within 3 weeks of commencement of ramp-up [reopening after reductions in service]), tier 2 (semiurgent; patients with postponed or canceled appointments requiring follow-up care within 3 months of ramp-up commencement), and tier 3 (patients with postponed or canceled appointments whose follow-up appointments could safely get deferred beyond 3 months of ramp-up commencement). To assign the most appropriate tier for each patient, clinicians were provided with our algorithm scores along with other pertinent information, such as the checkout health records from the last clinic appointment. We assessed how well our algorithm’s TS aligned with the clincians’ tier assignments for each patient.

Participant characteristics were summarized for the sample using means and SDs for continuous variables and frequencies and percentages for categorical variables. No statistical comparisons involving P values were performed. All statistical analyses were performed using SAS version 9.4 (SAS Institute).

Results

We identified 1091 patients with upcoming glaucoma clinic appointments between March 16, 2020, and April 16, 2020. After excluding 57 patients with no prior records of eye care at the institution, 1034 eligible patients remained. The mean (SD) age of these patients was 66.7 (14.6) years. There were 575 women (55.6%), and the racial/ethnic composition included 733 White individuals (71%), 160 Black individuals (15.5%), 90 individuals of Asian or Pacific Islander background (8.8%), and 48 individuals (4.6%) classified as being of other or unknown races/ethnicities.

Among the group of patients with upcoming appointments, the CS ranged from 0 to 55 points. The mean (SD) CS was 27.2 (16.1) points. The GS ranged from −50 to 20 points. The mean (SD) GS was 4.0 (14.4) points. The TS ranged from −50 to 75 points. The mean (SD) TS was 31.2 (21.4) points (Figure 1).

Figure 1. Distribution of Risk Scores.

Different triage thresholds (0, 25, and 50) were assessed to demonstrate varying degrees of caution regarding the adverse outcomes associated with increased risk of COVID-19 exposure and different levels of risk tolerance for glaucoma progression. In this sample, 970 of the 1034 patients (93.8% of the scheduled appointments) would be identified for postponing and rescheduling if a TS threshold of 0 points were used. The TS thresholds of 25 and 50 points would identify 668 patients (64.6% of appointments) and 275 patients (26.6% of appointments) for postponing and rescheduling, respectively (Figures 2 and 3).

Figure 2. Coronavirus Disease 2019 (COVID-19) Morbidity Risk Score vs Glaucoma Severity and Progression Risk Score.

Dot sizes indicate the number of patients with a given combination of COVID-19 morbidity risk score and glaucoma severity/progression risk score. Shaded areas capture the patients whose appointments should get postponed and rescheduled based on different total score thresholds.

Figure 3. Number of Patient Appointments to Postpone Based on Different Total Score Thresholds.

A and D, Distribution of patients at a threshold of 0. B and E, Distribution of patients at a threshold of 25. C and F, Distribution of patients at a threshold of 50.

Specific Examples

The following examples demonstrate how the scoring system could be applied to specific patients. The contributions of each subcomponent to the CS and GS at the varying triage appointment thresholds for the entire sample are shown in eFigures 1 and 2 in the Supplement.

Positive TS

This patient was in her ninth decade of life, with several chronic medical diseases (CCI, 11), monocular vision, no recent intraocular surgery, a mean deviation on HFA of −2.0 dB in the right eye and −8.9 dB in the left eye, and 2 previous IOP measurements within the preceding year, neither of which were less than 6 mm Hg or more than 30 mm Hg in either eye. She would have had a GS of 0, a CS of 55, and a TS of 55.

Negative TS

We compared a patient in the fifth decade of life with no chronic medical illnesses (CCI, 0) who had monocular vision, a history of intraocular surgery 2 months ago, a mean deviation on HFA of −0.6 dB in the right eye and −24.5 dB in the left eye, and 5 recorded IOP measurements in the preceding year, all of which were more than 30 mm Hg. This patient would have had a GS of −50, a CS of 0, and a TS of −50.

Neutral TS

A third patient was in the ninth decade of life with a few chronic medical diseases (CCI, 4), monocular vision, recent intraocular surgery, and a mean deviation on HFA of −25.1 dB in the right eye and −22.5 dB in the left eye, with 6 recorded IOP measurements in the preceding year, 1 of which was less than 6 mm Hg in 1 eye. The patient would have a GS of −45, a CS of 45, and a TS of 0.

Validation

There were 3030 patients with glaucoma clinic appointments scheduled from March 16, 2020, to May 8, 2020. Among the 2958 patients with a TS of 0 or more, 2371 of these patients (80.2%) were assigned to tiers 2 or 3, indicating that after reviewing their EHRs, the glaucoma specialist determined their follow-up could safely get delayed for 3 or more months after ramp-up commencement. Likewise, 1749 of 2119 patients (82.5%) with a TS of 25 or more and 776 of 901 patients (86.1%) with a TS of 50 or more were also classified as being in tiers 2 or 3. There were 72 patients with a TS of less than 0; 55 of their corresponding appointments (76.4%) were recommended to be kept, or if the appointment was canceled, the clinician indicated the patient required urgent care within the first 3 weeks of ramp-up activities (tier 1).

Discussion

This report describes the development of a scoring algorithm intended to provide guidance to eye care professionals regarding which clinic appointments to postpone or keep by considering the competing risks of morbidity resulting from glaucoma progression as a result of delayed care vs the potential risk of acquiring COVID-19 when seeking eye care services. As a decision support tool, it was intended to provide additional information to clinicians who would integrate these scores with clinical judgement and other considerations, including risk of viral transmission to other individuals and personal protective equipment conservation, to make decisions regarding which upcoming appointments could safely get triaged. This approach was made to be easy to implement, revise, and scale, given the rapid spread of SARS-CoV-2; the similarly rapid progression in scientific understanding of the epidemiology of COVID-19 transmission; and the expectation of evolving institutional, organizational, and governmental guidelines. The information necessary to quickly calculate these scores for all patients with upcoming appointments was readily accessible in the SOURCE repository.

An advantage to this approach may be its flexibility to permit ophthalmic practices and health systems to adjust the CS and GS scoring schemes or the TS threshold as the situation changes. For example, as practices begin the process of ramping up care when it becomes safe to do so, these scores can provide information to assist with prioritizing which patients should be seen sooner than others to help clinicians offer care to patients who most need it. This approach would complement other efforts, such as maintaining social distancing in clinics, minimizing clinic wait times, and increasing clinical efficiency to permit clinicians to safely care for as many patients as possible.^10,11

When we validated our scoring algorithm against glaucoma specialists’ review of patients’ EHRs to assign them a tier for follow-up care, we found that among the 72 patients whose TSs were less than 0 (indicating severe glaucoma or high risk of disease progression with few risk factors for morbidity from COVID-19), the glaucoma specialists felt most of these appointments (76.4%) either should not get postponed, or if they had been canceled, that the patient required rescheduling within the first 3 weeks of ramp-up activities. To permit sufficient social distancing during ramp-up activities, some stable patients were assigned to tier 2 or 3 despite possessing TSs less than 0. Among the 1749 patients whose TSs were 25 or more (indicating either the patient’s glaucoma was not overly severe or the patient had several risk factors for morbidity from COVID-19), the glaucoma specialists felt 82.5% of those appointments could safely get delayed 3 or more months. Overall, the findings of this validation exercise demonstrate that the TSs from our algorithm aligned with glaucoma specialists’ preferences for appointment triaging for most patients. Feedback from the clinicians during the validation exercise was that they found that not only was the TS helpful, but the GS and CS were as well, when they were trying to assign their patients to the proper tier. We encourage readers to not rely exclusively on the scores of our algorithm but also consider patient-specific factors that may influence follow-up, such as travel distance, caregiver’s risk of exposure to SARS-CoV-2 when escorting patients to clinic appointments, and whether the patient resides in a skilled nursing facility. We view the use of these scores as a useful guide for appointment postponement and rescheduling, which can be particularly helpful for busy clinicians when there are many hundreds or thousands of patient appointments that have been affected.

Another potential advantage is that this approach is scalable. While, in the interest of time, this system was pilot tested on patients with glaucoma at the University of Michigan, it may be possible to rapidly deploy this scheme at other academic centers in the SOURCE consortium, because our code quickly calculates scores for every patient. For practices that are not a part of SOURCE, it may be possible to work with local information technology staff to gather the necessary data elements from the EHRs to permit these calculations on large numbers of patients with upcoming appointments.

Limitations

Information, such as short-term complications and long-term sequelae from COVID-19, was unknown at the time of conducting our analyses, and even now it remains incompletely understood; yet decisions for postponing, retaining, or rescheduling appointments had to be made amidst this uncertainty. We assigned higher weights to many of the COVID-19 risk factors relative to the glaucoma severity and progression risk factors to reflect that morbidity from SARS-CoV-2 is likely to be more serious than visual acuity or visual field loss from glaucoma progression because of delayed care receipt; this assumption and others will require additional validation in the future, especially as we learn more about COVID-19 epidemiology and risk factors. Furthermore, additional research is needed to explore whether our prioritization of clinic appointments, based in part on this scoring algorithm, resulted in improved outcomes (reduced glaucoma progression and COVID-19 morbidity) compared with approaches used by other eye care professionals to triage clinic appointments. Second, the system piloted in this study focused specifically on patients with glaucoma and may not be generalizable to patients with other diseases. Applying this approach to others who do not have glaucoma may lead to misleading recommendations. For patients without glaucoma, it may be more useful to prioritize clinic appointments based solely on the CS and not the TS or devise new algorithms to capture severity of those particular conditions. Third, a subset of patients (n = 57) could not perform HFA testing and were either monitored with Goldmann perimetry or unable to perform perimetry. This could have resulted in an underestimation of the TS for these patients if they possessed substantial visual field loss that was not captured by the mean deviation values generated from the HFA. Finally, while many patients in this study also receive nonocular care at the University of Michigan (permitting us to obtain a near-complete capture of their overall health characteristics), for those who do not, our EHR may not have captured all of the chronic diseases the patient possesses. This may have resulted in an underestimation of their CS and therefore an overestimation of their TS. This is particularly problematic for prioritizing appointments for patients who are new to the health system.

Conclusions

A scalable risk stratification tool was developed using information from a large data repository to provide information for clinicians and health systems to triage upcoming clinic appointments for glaucoma care, considering the risk to the patient of morbidity from exposure to COVID-19 against the risk of underlying glaucoma progression that could result from delayed receipt of care. While our approach is tailored to patients with glaucoma, we believe similar methods may be developed to help offer guidance for appointment rescheduling for other ophthalmic and nonophthalmic conditions during this pandemic and similar situations in the future.

Supplement.

eFigure 1. Distribution of Characteristics Comprising the COVID-19 Morbidity Risk Score, by triage threshold.

eFigure 2. Distribution of Characteristics Comprising the Glaucoma Severity/Progression Risk Score, by triage threshold.