The American Academy of Ophthalmology IRIS Registry (Intelligent Research In Sight): current and future state of big data analytics

Abstract

Purpose of review

To describe the drivers, development, and current state of the American Academy of Ophthalmology IRIS Registry (Intelligent Research In Sight), and analytics involving deidentified aggregate IRIS Registry data.

Recent findings

The IRIS Registry has a core mission of quality improvement and reporting. In addition, analytic projects performed to date have included characterizing patient populations and diseases, incidence, and prevalence; clinical outcomes and complications; risk factors and effect modifiers; practice patterns; and trends over time. Pipeline projects include application of artificial intelligence and machine learning approaches for predictive modeling and analytics, disease mapping, detecting patterns and identifying cohorts, and optimizing treatment based on patient-specific characteristics.

Summary

The IRIS Registry is the nation’s largest single specialty clinical registry, with unique data elements specific to ophthalmology. It offers a wealth of opportunities involving big data analytics, including traditional inferential statistics as well as machine learning and artificial intelligence approaches scalable on massive amounts of data.

INTELLIGENT RESEARCH IN SIGHT REGISTRY BACKGROUND AND DEVELOPMENT

For the American Academy of Ophthalmology, the road to quality improvement has been a journey of developing preferred practice patterns, evidence-based assessments of new technology and quality measures [1]. The fundamental key to quality improvement is measurement. In 1880, this quote has largely been attributed to Lord Kelvin: “If you can’t measure It, you can’t improve it.”

To support its measurement goals, the Academy initiated it first clinical data registry in 1996, known as the National Eyecare Outcomes Network (NEON). NEON was developed to collect a common dataset on cataract patients, including postoperative outcomes and patient functional status and symptoms [2]. At its peak, NEON included 249 physicians and 17,876 cataract surgery patients. However, NEON was discontinued because of the lack of participation limited by necessity of manual entry of data, and the lack of demand or incentives for performance measures [3].

The American Academy of Ophthalmology created its first 6 quality measures with the advent of the Physician Quality Reporting Initiative (later the Physician Quality Reporting System, or PQRS), a Centers for Medicare and Medicaid Services program to drive value-based purchasing by rewarding the reporting of data. In 2008, the Academy developed 2 cataract surgery outcome measures. The Cataracts Measures Group, including 2 patient-reported outcome measures, was accepted in 2012 in the system now known as PQRS. The Academy has since developed 43 quality measures in total, including 30 outcome measures. In March 2010, the Academy initiated the Ophthalmic Patient Outcomes Database to collect data for PQRS measures only, including the Cataracts Measures Group with two patient-reported outcome measures for a limited number of patients. Practices entered their data manually and viewed their data prior to submission, allowing for correction of errors and omissions. This database was discontinued after 2013.

In April 2011, the Academy started exploring a new registry with a hearing on electronic health records (EHRs) and Data Registries at the Academy MidYear Forum, which brings together ophthalmologist members, committee chairs, state, and subspecialty leaders across the country. A strategic planning session that invited key individuals from other specialty societies was held with the Board of Trustees (Board) in June 2011, and the Board approved investigation of the merits and logistics of registry development in September 2011. In December 2011, the Clinical Registry Task Force was assembled and met weekly from January to March 2012 to coalesce on the registry objectives and development, and to hear from other society representatives about their registry experiences. This Task Force delivered a report to the Academy Chief Executive Officer in March 2012 and then consulted with potential registry vendors and other Academy leaders through August 2012.

The Task Force evaluated the current registries in place and in development, learning from the longstanding inpatient procedure registries of the Society of Thoracic Surgeons [4] and the American College of Cardiology (ACC) [5] and the outpatient registry of the ACC [6]. These two professional registries had a strong history of clinical improvement, many inputs for data but workflow relegated to nonphysician staff for largely manual input on the inpatient side, and large expenses that were offset largely by hospital/facility fees and staffing for data input or by industry support. Ophthalmology inherently differed in several key respects, including an outpatient setting; lack of support from large facilities, hospitals, and their staff; and need to rely on broad adoption of EHR systems, because manual entry and interference with physician workflow would render the registry unusable and undesirable. The Task Force recommendations were to define a broad and inclusive mission to embrace not only process and outcome quality improvement and professional improvement, but also data analytics and scientific advancement. This would greatly enhance the adoption of a registry and provide a rigorous foundation for its growth.

The core business of the registry, in terms of data integration, curation, quality measure calculation and reporting, was recommended to be most efficiently performed by an external vendor, rather than creating an in-house entity with all the requisite professional experience. Another critical component was creation of a measure development group to ensure that development would be synchronized with data available in EHRs to minimize the burden on practices and include all ophthalmic subspecialties. This was modelled after the ACC’s outpatient EHR-based registry whereby systems integration software was utilized to collect data from individual practices [6].

The Task Force delivered its recommendations to the Academy Board in September 2012. The summary recommendation was that the ophthalmic clinical registry may represent a seminal change in how ophthalmologists improve performance and outcomes while shortening the timeline for the dissemination of important clinical knowledge, expanding research opportunities, and facilitating drug and device surveillance. To do so would require broad input from specialists, a viable business plan, and minimal or no adverse impact on physician workflow. The timing was optimal because the Academy could capitalize on a confluence of lessons learned by other societies on the leading edge of clinical registries, new technology that extracted data from different EHRs to eliminate manual data entry (systems integrator), and federal incentives for quality reporting. The Task Force recommendations were approved by the Board, and the Academy launched the IRIS Registry (Intelligent Research in Sight) in early 2014. The development and maintenance of the IRIS Registry was funded by the Academy, with no charge to Academy members for participation.

A perfect storm of quality incentives, broader EHR adoption, and a concerted information campaign accelerated the embrace of the IRIS Registry by Academy members. The Medicare Meaningful Use Program was created in 2009 as part of the Health Information Technology for Economic and Clinical Health Act to incentive the purchase of EHRs, starting in 2011, with a maximum of $44,000 per physician, and to penalize those without EHRs. In 2014, the PQRS program increased the requirement for quality measure reporting, adding to the incentives to adopt an EHR for electronic reporting. In 2017, PQRS was replaced by the Quality Payment Program (QPP), authorized by the Medicare Access and CHIP Reauthorization Act of 2015 (MACRA). The QPP streamlined the different programs into one program known as the Merit Based Incentive Payments System (MIPS) that rewards clinicians for value, rather than just volume, and to improve care received by Medicare beneficiaries as well as to lower costs to the Medicare program. The MIPS program has 4 performance categories: Quality (6 measures); Promoting Interoperability (evolution of the Meaningful Use program); Improvement Activities; and Cost [7].

The IRIS Registry has grown to be the nation’s largest single specialty clinical registry. This centralized data repository and reporting program collects EHR data from individual practice EHR databases or accesses files from cloud-based EHRs and performs quality measure calculation for reporting the MIPS Quality category [8]. The IRIS Registry aggregate data is de-identified for privacy purposes, and the practices retain ownership of their own data. Participants access reports on their performance to help pinpoint opportunities to improve the quality of their care, thereby advancing care delivery and patient outcomes. In aggregate, improvement on aggregate quality measures was demonstrated over the first three years using the IRIS Registry database [9]. Currently, as of January 1, 2022, the IRIS Registry database includes 422.99 million visits from 71.90 million patients.

GOALS AND DIRECTION: QUALITY AND REPORTING

The initial mission of the IRIS Registry is to develop a registry of ophthalmologic ambulatory encounters with little impact on office workflow that captures the essential data elements for the following: continuous quality improvement efforts, Maintenance of Certification activities, enhanced patient care outcomes, and pay for performance programs. The benefits to the profession are as follows: benchmarking performance against peers around the country that would drive improvement in patient outcomes, meeting requirements for practice improvement needed for maintenance of certification and maintenance of licensure, reporting quality as required by the federal government and third party payers, thereby avoiding financial penalties, providing an infrastructure for drug and device surveillance, enabling data analytics to address disparities in care and outcomes and to answer the question of value of ophthalmology services. Additional, IRIS Registry data can be used to perform observational studies of practice patterns, disparities of access to care and outcomes, comparative effectiveness of treatment outcomes, risk factors for adverse events, and natural history of disease.

GOALS AND DIRECTION: ANALYTIC MISSION

Beyond its core mission of quality reporting and quality improvement, the IRIS Registry is also able to compellingly contribute to analytics at the population level. It offers a unique combination of massive scale—the largest clinical data registry in the world—as well as specialty-specific clinical depth, including visual outcomes and intraocular pressure. Although other data sets such as administrative insurance billing claims also represent national populations, the IRIS Registry includes additional critical ophthalmic variables, including visual acuity. Furthermore, the IRIS Registry’s size cannot be matched in sources such as surveys and observational studies, clinical trials, and other single and multicenter analyses. These features make it optimal for population health big data analytics, including rare diseases as well as eye-specific outcomes.

The Academy has partnered with Verana Health (San Francisco, CA) for IRIS Registry data curation and analytics, including housing IRIS Registry records, hosting quality dashboards for participating practices, and supporting research on aggregate data. Clinical data obtained directly from EHRs are curated and made accessible for research analyses.

The dataset is available for population-level analytics through multiple mechanisms: Academy-sponsored analyses; individual investigator project grants through Research to Prevent Blindness, the Hoskins Center IRIS Registry Fund, Knights Templar Eye Foundation Pediatric Ophthalmology Fund, and the American Glaucoma Society; and the IRIS Registry Analytic Center Consortium sites. The Hoskins Center and Knights Templar Eye Foundation grants are particularly targeted to ophthalmologists in private practice, and the Analytic Center Consortium is made up of academic medical centers with domain expertise in big data analyses. In both cases, eligibility requires actively contributing data to the IRIS Registry. Four Consortium sites were initially selected by the Academy in 2019 (Massachusetts Eye and Ear Infirmary, Stanford University, Wills Eye Hospital/Thomas Jefferson University, and University of Washington), with periodic opportunities for additional sites to apply. Additional sites are actively being onboarded as of 2022, and Consortium sites may also collaborate with investigators outside their institution [10].

BIG DATA: SCALE, SCOPE, AND REGISTRY RESEARCH

In its current version, the IRIS Registry aggregate, deidentified dataset includes patient demographics (including detailed race/ethnicity), provider specialty/subspecialty and practice 5-digit zipcode, insurance coverage, diagnoses and procedures (including laterality), medications, visual acuity, intraocular pressure, and cup-to-disc ratio. Several dataset versions have been used in analyses thus far, including an internal, deidentified version for Academy and Verana analyses, a deidentified version (called “Rome”) made available to Consortium sites initially, and the current deidentified research version made available in 2021 (“Chicago”). Going forward, all research will utilize the Chicago version, including work by the Academy, Consortium sites, and project grant recipients. Analyses are performed in a secure online cloud computing environment.

Analytic projects performed to date have included characterizing patient populations and diseases, incidence, and prevalence; clinical outcomes and complications; risk factors and effect modifiers; practice patterns; and trends over time. For example, investigators have characterized the population of patients receiving minimally-invasive glaucoma surgery, determined incidence of endophthalmitis after cataract surgery (and found no increase in endophthalmitis among patients undergoing immediate sequential bilateral cataract surgery versus delayed sequential surgery), reported visual outcomes among age-related macular degeneration patients based on specific vascular endothelial growth factor inhibitor treatment agent, described treatment patterns for myopic choroidal neovascularization and diabetic macular edema, and evaluated risk factors for retained lens fragments after cataract surgery [11^■–13^■,14,15,16^■,17^■].

Beyond these epidemiologic questions, other possibilities for IRIS Registry projects include application of artificial intelligence and machine learning approaches for predictive modeling and analytics, disease mapping, detecting patterns and identifying cohorts, and optimizing treatment based on patient-specific characteristics. Given rapid growth of the IRIS Registry—reflecting the number of patients and visits as well as the amount and complexity of granular data for each patient encounter—automated, scalable Big Data approaches will be increasingly important for research going forward. In addition, the IRIS Registry infrastructure for collecting real world data directly from EHRs may enable other work such as surveillance monitoring of ophthalmic treatments and/or conducting registry-embedded clinical trials at lower cost and greater efficiency than traditional methods [18,19^■].

STRENGTHS, LIMITATIONS, AND FUTURE DIRECTIONS

The IRIS Registry dataset does possess some key limitations, in particular the absence of systemic health and healthcare data, lack of imaging data or narrative free text (e.g., visit notes), incomplete surgical records, and inherent constraints of real-world data such as data entry errors, missing data, and variability in documentation habits from a wide range of sources [20^■■]. Surgical records are frequently missing or incomplete since the Registry ingests data from outpatient electronic health records, and affiliated surgery centers may not have an EHR or use a different platform which is not integrated with the IRIS Registry. The IRIS Registry’s time horizon is also limited; although the dataset includes retroactive ingestion of historic EHR data as new practices are onboarded, the data by definition does not predate EHR adoption. The earliest robust records begin in 2013, with a more limited dataset in these years.

Despite these constraints, the large scale of the IRIS Registry, its diverse ophthalmology patient population with detailed patient-level demographic data, and inclusion of critical ophthalmic outcome metrics not available in administrative billing claim datasets make it an invaluable source for population-level eye care analytics. A future state where the Registry includes additional data such as curated and deidentified free text, imaging records, and/or linked systemic health data will dramatically expand its potential. Even in its present form, however, the growing size and scope of the IRIS Registry offers a wealth of opportunities.

KEY POINTS.

• The American Academy of Ophthalmology IRIS Registry(Intelligent Research In Sight), is the nation’s largest single specialty clinical registry, includes critical ophthalmology outcome metrics, and is uniquely suited to big data population health analytics in ophthalmology.

• Analytic projects performed to date using the IRIS Registry have included characterizing patient populations and diseases, incidence, and prevalence; clinical outcomes and complications; risk factors and effect modifiers; practice patterns; and trends over time—with potential to apply artificial intelligence and machine learning approaches for predictive modeling and other analytics as the dataset grows in scale and complexity.

• Current limitations of IRIS Registry analytics include absence of systemic health and healthcare data, lack of imaging data or narrative free text (e.g., visit notes), incomplete surgical records, and inherent constraints of real-world data such as data entry errors, missing data, and variability in documentation habits from a wide range of sources.