Predictive Modeling of Risk Factors and Complications of Cataract Surgery

Abstract

Purpose

To quantify the relationship between aggregated preoperative risk factors and cataract surgery complications, as well as to build a model predicting outcomes on an individual-level—given a constellation of demographic, baseline, preoperative, and intraoperative patient characteristics.

Setting

Stanford Hospital and Clinics between 1994 and 2013.

Design

Retrospective cohort study

Methods

Patients age 40 or older who received cataract surgery between 1994 and 2013. Risk factors, complications, and demographic information were extracted from the Electronic Health Record (EHR), based on International Classification of Diseases, 9^th edition (ICD-9) codes, Current Procedural Terminology (CPT) codes, drug prescription information, and text data mining using natural language processing. We used a bootstrapped least absolute shrinkage and selection operator (LASSO) model to identify highly-predictive variables. We built random forest classifiers for each complication to create predictive models.

Results

Our data corroborated existing literature on postoperative complications—including the association of intraoperative complications, complex cataract surgery, black race, and/or prior eye surgery with an increased risk of any postoperative complications. We also found a number of other, less well-described risk factors, including systemic diabetes mellitus, young age (<60 years old), and hyperopia as risk factors for complex cataract surgery and intra- and post-operative complications. Our predictive models based on aggregated outperformed existing published models.

Conclusions

The constellations of risk factors and complications described here can guide new avenues of research and provide specific, personalized risk assessment for a patient considering cataract surgery. The predictive capacity of our models can enable risk stratification of patients, which has utility as a teaching tool as well as informing quality/value-based reimbursements.

Introduction

Cataracts are the most common cause of reversible visual impairment in the United States, affecting over 20 million people, with approximately 3 million surgeries performed annually at a cost of $3.4 billion to Centers for Medicare and Medicaid Services (CMS)—the largest single procedure source of CMS expenditures.[1, 2] Rates of surgery continue to rise; between 1990 and 2010, the incidence of cataract surgery increased between 2.5- and 6.5-fold, varying by region in the United States.[3–5] Although a safe and effective surgery, rare complications add up to substantial numbers due to the sheer volume of cataract surgeries performed annually.[6] These complications can have significant functional and financial impact. Direct clinical costs of managing complications range from $400 to $6000 for an episode of care[7] and indirect costs are even greater.

Previous epidemiologic studies have quantified the incidence of a number of intraoperative and postoperative complications of cataract surgery.[8–11] However, much of the existing literature focuses on the relationship between risk factors and post-operative visual acuity.[12–15] Studies examining the effect of risk factors on complications have tended to look at aggregated outcomes (e.g. any intraoperative complication).[12, 16, 17] Studies exploring specific outcomes have tended to focus either on the relationship between preoperative risk factors and intraoperative complications,[10, 18–25] or on the relationship between intraoperative and postoperative complications.[26–28] Few studies have examined relationships between preoperative risk factors and postoperative complications, or explored the combined effects of a large number of risk factors and complications simultaneously.

The aims of our study are two-fold: 1) to quantify the relationship between aggregated pre-operative risk factors and complications in cataract surgery (as recorded in the electronic health record, or EHR); and 2) to build a model to predict the occurrence of these outcomes on an individual level given a constellation of demographic, baseline, preoperative and intraoperative characteristics for a given patient. We selected cataract surgery as a high volume and well-studied procedure (with accepted risk factors, complications, and results) to validate our models.

A framework to identify risk factors among patient characteristics and their combined effects may guide new avenues of research and help quantify individual risk for a patient considering cataract surgery. We envision a predictive model that can be valuable in case-specific risk stratification—with utility for patient selection, as a teaching tool, and potentially informing risk adjustment for anticipated implementation of value-based reimbursement.

Methods

Data Source and Study Population

The patient population was drawn from the Stanford Translational Research Integrated Database Environment (STRIDE), which contains data from the EHR for 1.8 million patients seen at the Stanford Hospital and Clinics and Stanford Children’s Health Network between 1994 and 2013. Extraction and processing of the datasets was approved by the Stanford Institutional Review Board.

Cohort and Feature Selection

We utilized a retrospective cohort study design. The cohort was defined to include all patients who had a Current Procedural Terminology (CPT) code for cataract surgery (either 66982 or 66984) and who were at least 40 years old at the time of surgery. To account for the fact that many patients may be referred to Stanford to receive cataract surgery and have this surgery as their only encounter in our database, we limited our cohort to patients who had at least one year of records prior to their surgery. This step helps ensure we capture as many pre-operative risk factors as possible. Since most patients would return to the provider who performed their surgery in the case of any postoperative complications, we did not impose any such restriction on the follow-up records after cataract surgery.

In total, there were 5,549 unique patients accounting for 8,382 unique surgeries, since patients could each have up to two surgeries—one surgery per eye occurring at least 30 days apart (Figure 1). Potential risk factors and complications were determined by an expert clinician (Table 1). These characteristics, as well as key demographic information and the Charlson comorbditiy score to serve as a proxy for the patient’s overall health[29], were extracted from the EHR using International Classification of Diseases, 9^th edition (ICD-9) codes, CPT codes, drug prescription information, and text data mining via natural language processing methods (described previously).[30] A full list of all codes used is provided in Appendix Tables S1 and S2.

Figure 1.

Schematic outline of the methodology for this paper. We first created a cohort consisting of 5,549 unique patients who had undergone cataract surgery and had sufficient records. We then used the 8.382 distinct surgeries to identify the relationship between risk factors and complications using multivariate regression and LASSO feature selection. Finally, we split the 8.382 surgeries into a training set consisting of 85% of the data and a test set of 15% of the data. Using the training set, we created and tuned random forest predictive models and then ran them on the individuals in the test set to characterize the performance of our predictive models.

Table 1.

Prevalence of risk factors and complications in study population.

Preoperative Risk Factors	N	(%)1	Complication	N	(%)*
Age (by half-decade)			Intraoperative Complications
40–45 years			Complex surgery3	768	9
46–50 years			Posterior capsular tear	35	0
51–55 years			Dropped nucleus or retained lens material	76	1
56–60 years			Unplanned anterior vitrectomy	128	2
61–65 years			Intraoperative Floppy Iris Syndrome (IFIS) or iris damage	49	1
66–70 years			Suprachoroidal Hemorrhage	1	0
71–75 years			Any Intraoperative Complication4	214	3
76–80 years
81–85 years			Short-term Postoperative Complications (within 90 days)
≥86 years			Endophthalmitis	82	1
Gender			Glaucoma or ocular hypertension	175	2
Male	3464	(41)	Pars plana vitrectomy/lensectomy	107	1
Female	4918	(59)	Retinal tear/detachment	107	1
Race/Ethnicity			Corneal edema	81	1
Caucasian	5715	(68)	Wound leak	39	0
Asian	948	(11)	Intraocular lens subluxation	13	0
Hispanic	298	(4)
Black	245	(3)	Long-term Postoperative Complications (within 1 year)
Other/Unknown	1176	(14)	Posterior capsule opacification/YAG capsulotomy	406	5
Myopia	936	11	Cystoid macular edema	160	2
Hyperopia	362	4	Post-cataract refractive surgery	277	3
Diabetes mellitus	2750	33	Intraocular lens exchange or repositioning, or secondary intraocular lens	123	2
Age-related macular degeneration	844	10
Glaucoma	2003	24	Aggregated Postoperative Complications
History of uveitis (or posterior synechiae)	244	3	Any lens-related complication5	896	11
History of eye trauma	172	2	Any postoperative complication6	692	8
History of retinal tear/detachment	1158	14	Any posterior segment complications7	237	3
History of steroid use (systemic or ocular)	563	7	Any anterior segment complication8	233	3
History of alpha-blocker use	436	5	Any intraocular lens malfunction9	116	1
Fuch's endothelial dystrophy	91	1
Prior refractive surgery	389	5
Prior vitrectomy	486	6
Other prior intraocular surgery	367	4
Intravitreal injection	248	3
Pseudoexfoliation	995	12
Corneal scarring or significant corneal arcus	381	5
Corneal Edema	80	1
Nystagmus	192	2
Lens Subluxation	85	1
Surgery on both eyes2	2829	51

¹⁾Out of 5549 unique individuals

²⁾Not considered as a risk factor per se, but included in the model to adjust for cumulative increase in risk from bilateral surgery

³⁾Though not by definition a complication, we included complex cataract surgery (Current Procedural Terminology, or CPT, code 66982) in the model, as compared to routine cataract surgery (CPT code 66984).

⁴⁾Includes all intraoperative complications except Complicated Surgery

⁵⁾Includes posterior capsule opacification/YAG capsulotomy, intraocular lens (IOL) dislocation, IOL repair, secondary IOL placement, and post-cataract refractive surgery

⁶⁾Includes all postoperative complications listed above (short-term and long-term)

⁷⁾Includes endophthalmitis (or vitreous tap with intravitreal antibiotics in the immediate postoperative period), retinal tear/detachment, pars plana vitrectomy/lensectomy, and cystoid macular edema

⁸⁾Includes corneal edema, wound leak/dehiscence, and glaucoma or ocular hypertension

⁹⁾Includes IOL dislocation, IOL repair, and secondary IOL placement

To reduce the possibility of misidentifying a historical event or preexisting condition as a surgical complication, we excluded instances where a complication was identified in the medical record preoperatively. That is, a complication was “counted” only if it was incident—first noted during or after the surgery.

Complications were classified as: 1) Intraoperative, 2) Short-term postoperative, or 3) Long-term postoperative. Intraoperative complications included those most likely to occur during cataract surgery, and we included instances mentioned within 5 days of the date of the surgery (to account for notes or codes that may have been inputted in the immediate post-operative period). Short-term postoperative complications were defined as occurring within 90 days of the cataract surgery, and long-term postoperative complications defined as occurring within 365 days of surgery. In addition to exploring individual complications, we also aggregated related outcomes (e.g. all lens-related complications or all complications occurring in the posterior segment of the eye) to improve our statistical power and identify trends among associated conditions.

Statistical Analyses

In order to identify and quantify relationships across a spectrum of risk factors and complications, we built multiple multivariate logistic regression models. We created a separate model for each complication, including the same variables in each model (risk factors plus key demographic information)—except in cases where a particular complication could also be a risk factor (e.g. glaucoma/ocular hypertension was not counted as a complication if it was present preoperatively). Intraoperative complications were included as risk factors for postoperative complications (e.g. posterior capsule tear as a risk factor for endophthalmitis). To account for the increased exposure to risk among patients who have had two distinct surgeries (up to twice the cumulative risk for intra- or post-operative complications as compared to a patient with unilateral surgery), we controlled for one surgery versus two distinct surgeries as a binary variable in our models.

To assess and quantify the importance of each variable, we calculated odds ratios (ORs) and 95% confidence intervals (CIs) for each variable/outcome pair. We also calculated p-values quantifying the significance of the relationship between each variable/outcome pair. Unlike other methods examining the relationships of a very large number of m variables on n outcomes using m × n univariate regressions (e.g. in genomic analysis), we only performed n—where n = 20 outcomes—unique multivariate regressions. Thus, we did not feel that it was necessary to use a method, like the Bonferroni correction, to account for multiple comparisons.

To identify the most important variables, we used a bootstrapped least absolute shrinkage and selection operator (LASSO) model, which utilizes a penalized logistic regression in order to shrink some of the regression coefficients to zero, eventually performing variable selection.[31] The tuning parameter (λ) was optimized through 10-fold cross-validation and the subset of variables selected in the model was recorded. We repeated this process through 100 bootstrapped iterations. Those variables that were present in ≥90% of the bootstrapped LASSO models were then input back into a multivariate logistic regression model and ORs, 95% CIs and p-values were calculated as above. Results were displayed graphically through bubble plots of statistically significant values (p<0.05), with axes organized through hierarchical clustering to show similarity between groups of risk factors and complications.

To create a predictive model, we used the random forest classification method.[31] We left out 15% of the study population to serve as a test set and used the other 85% to train and tune our model. Since there was a great deal of class imbalance, even for our most prevalent outcomes (>15:1 ratio between unaffected and affected individuals), we downsampled the majority class for each complication in the training set, such that there was a 1:1 ratio of unaffected to affected individuals. The test set data was not downsampled and was a representative sample of our population used to determine the model’s performance. In order to achieve the best predictive performance, we used 5-fold cross validation of the training set to determine the optimal number of trees and the optimal p-value cutoff for determining whether to call an individual “positive” or “negative” for a given predicted outcome. Then, we built a model for each individual outcome. After tuning the characteristics of the model, it was run on the held out test set. The predictions of the model were then compared to the known results from our test set. Relevant statistics describing the outcomes are reported, including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and the area-under-the-curve (AUC) describing the receiver operating curve (ROC). All statistical analyses were conducted in R version 3.0.2.[32]

Results

Patient Population and Demographics

Demographic characteristics of our population are shown in Table 1. Our population was predominantly female (59%, vs. 41% males), consistent with other large studies examining cataract surgery.[3, 33] The mean age at the time of surgery was 74.4 years (SD=10.9); this age distribution is also similar to other studies.[3, 8] Our population was predominantly white (68%), with Asians making up the next largest racial/ethnic group (11%).

Quantifying Risk Factors

When evaluating intraoperative complications, we found, as expected, that the feature-selecting LASSO model (Figures 2A and 2B) produced a more parsimonious subset of the risk factors found in the full multivariate regression model (Supplementary Figures S1A and S1B). That is, while all of the risk factors from the LASSO model were also found in the multivariate regression, the regression model also included many more features, which were excluded by the LASSO model. In most instances, the LASSO model de-emphasized potentially protective factors. Similarly, risk factors that weakly increased risk of complications in the multivariate model were dropped out or deemphasized during feature-selection in the LASSO model (e.g., glaucoma was found to be a weak risk factor for any intraoperative complication in the multivariate model, but was not selected in the LASSO model). Being on alpha-adrenergic antagonists, having a pre-existing diagnosis of diabetes and being relatively young (between 51–55 years) were all risk factors significantly associated with intraoperative complications.

Figure 2.

Bubbles represent statistically-significant (p<0.05) results of multivariate regression of variables identified by a bootstrapped least absolute shrinkage and selection operator (LASSO) model. The size of the bubble corresponds to the inverse of the 95% confidence interval (CI)—that is, 1/(95%CI)—such that results with a larger variance are shown as smaller bubbles and results with a smaller variance are shown as larger bubbles. The color of the bubbles is related to the odds ratio (OR), with blue indicating an OR <1 and red indicating an OR>1 (the maximum value of the OR was capped at 2 in the graphical representation). Axes are organized through complete linkage hierarchical clustering to show similarity between groups of risk factors and complications. A) Intraoperative complications from multivariate regression on risk factors selected through bootstrapped LASSO model, clustered in 3 groups for each axis. B) Postoperative complications from multivariate regression on risk factors selected through bootstrapped LASSO model, clustered in 5 groups for each axis.

The LASSO model for postoperative complications behaved similarly to the model for intraoperative complications, with risk-reducing and weakly-associated features from the full multivariate regression being de-emphasized or dropped.

The greater number of evaluated postoperative complications allowed additional patterns to emerge. Younger age (<60 years old), prior anterior vitrectomy or refractive surgery, history of age-related macular degeneration (ARMD), and complex cataract surgery were all risk factors associated with postoperative complications.

The calculated odds ratios, 95% CIs, and p-values for intraoperative and postoperative the LASSO regressions are in Appendix Table S3.

Predictive Modeling

The results of the random forest predictive models for intraoperative and postoperative complications and evaluated on the test set data are presented in Table 2. In general, the predictive models tended to have very high negative predictive value (NPV >0.95) with moderate sensitivity (>67%) and moderate AUC (>0.65). Predictive models for relatively prevalent complications (like posterior capsule opacification/YAG capsulotomy) and predictive models for aggregated outcomes (like any posterior segment complication or any anterior segment complication) tended to have better reliability than models for other individual complications. However, several less-prevalent complications also had good predictive value—e.g., intraocular lens subluxation (sensitivity 100%, AUC 0.79) and endophthalmitis (sensitivity 75%, AUC 0.70).

Discussion

Identifying Known Associations

A number of previous studies have looked at the effect of risk factors on visual outcomes following cataract surgery.[12–15] Risk factors commonly associated with worse visual outcomes include: age-related macular degeneration (ARMD), diabetic retinopathy, corneal opacity/pathology, older age, female sex, previous vitrectomy, previous retinal detachment surgery, alpha-blockers, complex surgery, and intraoperative complications.[12–15] We found similar trends in our LASSO models, including ARMD, corneal pathology, alpha-blocker medications, and complex surgery each as recurring risk factors for a number of complications. For example, we find an increased risk of intraoperative complications associated with preoperative use of alpha-blockers such as tamsulosin—a relationship which has reported extensively in the literature.[34]

Our methods allowed us to examine complications and patient/surgical characteristics (potential risk factors) individually and in aggregate, confirming recognized associations as well as identifying possible new ones. Many existing studies of predictors for cataract surgery complications have evaluated risk factors for aggregated complications. [16, 21, 26, 34, 35] Our findings were consistent with the literature describing risk factors for any postoperative complication, including: complex cataract surgery, any intraoperative complication, black race and previous eye surgery (including keratorefractive surgery). [17, 26]

Research on individual complications is more limited, with studies tending to concentrate on intraoperative complications, such as posterior capsule tear[13, 24, 25, 36–38] and intraoperative floppy iris syndrome (IFIS).[39–43] Postoperative endophthalmitis and retinal tear/detachment are also well-studied[16, 28, 44–49]—and our analysis also found complex cataract surgery to be associated with higher risk for endophthalmitis—however, other post-operative complications have been less extensively described in the literature.[17, 35, 50–53]

Novel Associations

In addition to confirming known associations, we also found a number of novel risk factors. Prior studies have demonstrated that diabetic retinopathy increases the risk of poor visual acuity outcomes and surgical complications.[14–16, 25] However, our study indicates a significant association between systemic diabetes mellitus itself (irrespective of diabetic retinopathy) and a number of complications, including posterior capsule tear, endophthalmitis, and cystoid macular edema. Additionally, while young age (<60 years old) has previously been linked to endophthalmitis[28] and retinal detachment/tear,[27, 48] our work found it as a risk factor for many other intra- and post-operative complications.

We also found other factors which appeared to increase the risk for several intra- and post-operative complications and are clinically plausible. Previous vitrectomy appeared to increase the risk for a number of complications (including cystoid macular edema and dropped nucleus or retained lens fragments), and hyperopia appeared to increase risk of complex cataract surgery. Steroids have long been known to increase the risk for glaucoma[54]—a finding which was confirmed in our data—but our study suggests that alpha-antagonist drugs may also increase the risk for glaucoma/ocular hypertension after cataract surgery.

Results using these methods can be used to guide future research and studies to explore the nature and physiological underpinnings behind observed associations. Additionally, awareness of the potential effects of these risk factors can help guide clinical practice. Knowing that a patient has one or more of these risk factors may make a surgeon more cognizant of potential intraoperative complications. Similarly, patients with one or more associated risk factors might be given more specific warnings or precautions, and/or be followed more closely postoperatively to help prevent complications. Availability of large national empiric clinical data (such as the predominantly EHR-based American Academy of Ophthalmology Intelligent Research in Sight, or IRIS, data registry[55]) will provide the opportunity to apply these methods with greater statistical power—to more accurately determine risk and identify clinical associations.

Predictive Modeling for Cataract Surgery

The very high NPV of our predictive models suggests potential utility for preoperative screening. Our models overall outperformed existing published models, [26] with our AUCs ranging from approximately 0.63–0.84. One such model, which relies on labor-intensive questionnaires, achieved an AUC of 0.627 for predicting any postoperative complication, compared to our AUC of 0.68 for the same outcome using a model independent of human input.[26] Other studies have developed systems to predict visual function/acuity after cataract with similar performance to our complication-specific models.[56–58] Complication risk can thus be reliably predicted from preoperative data alone, and results used for patient selection, counseling, and risk adjustment.

Predictive tools are already being used in other disciplines to help patients make informed decisions about treatments based on their own personal data and risk factors (for example, an online risk calculator to help patients decide between Coronary Artery Bypass Graft surgery and Percutaneous Coronary Intervention).[59] And, in ophthalmology, a prognosis calculator is available for uveal melanoma.[60] Although cataract surgery is routine and safe, it remains a significant event for patients, who may value personalized information. Thus, one potentially valuable outcome of this work may be the development of a predictive tool for assessing individualized risk of complications from cataract surgery. Additionally, these models could be used in teaching hospital settings to evaluate risk in a structured way, selecting appropriate teaching cases and assisting in training of resident surgeons.[61–63] Also, these models may be useful for evidence-based risk-adjustment of surgeon quality metrics for value-based payments (e.g., Medicare value-based payment modifier beginning in 2015).[16, 63, 64] In this setting, false positives may be better tolerated than false negatives for an acceptable quality benchmarking system.

Not only valuable at the aggregate level, a growing body of literature suggests that site-specific models may be more accurate than generalized clinical guidelines in predicting outcomes.[65–67] Rather than treating differences in demographics, patient comorbidities, and institutional characteristics as confounders to be controlled for a given population, it may be more effective to incorporate these factors into models—improving predictive value for that specific population. The methods used in our study are portable and can be used to develop local predictive models in other systems.

Limitations

Readers should consider some limitations when interpreting our findings. Since our results were derived from a single clinical site, our study population may not be representative of the general population. While the text-analysis techniques used in this study have been shown to be quite accurate in detecting negated terms (97% accuracy), drug mentions (93% accuracy) and disease conditions (86% accuracy),[68–71] it is possible that some conditions or procedures were not adequately captured in our analysis (IFIS, for example, is almost certainly underreported in our sample since there it lacks specific codes and short acronyms are difficult to detect and classify through text-analysis). Furthermore, in addition to free text analysis, we also used ICD-9 and CPT codes—which have inherent limitations in accuracy. Finally, we stress that predictive accuracy for identifying associations should not be interpreted as evidence of causality.

Conclusions

We have constructed a series of models with good reliability for predicting cataract surgery complications, and identified several novel clinical associations. Although our results are derived from a single clinical site, the methods may be disseminated for broader application (including site-specific local analysis) and guide new avenues for research. The potential to directly integrate these predictive tools into EHRs may enable personalized medicine and decision-making at the point of care[72]—for patient counseling and as a teaching tool—as well as informed risk adjustment for value-based payment programs.

Supplementary Material

Fig. S1A. Figure S1.

Bubbles represent statistically-significant (p<0.05) results of multivariate regression. The size of the bubble corresponds to the inverse of the 95% confidence interval (CI)—that is, 1/[95%CI]—such that results with a larger variance are shown as smaller bubbles and results with a smaller variance are shown as larger bubbles. The color of the bubbles is related to the odds ratio (OR), with blue indicating an OR <1 and red indicating an OR>1 (the maximum value of the OR was capped at 2 in the graphical representation). Axes are organized through complete linkage hierarchical clustering to show similarity between groups of risk factors and complications. A) Intraoperative complications from multivariate regression on all risk factors, clustered in 3 groups for each axis. B) Postoperative complications from multivariate regressions on all risk factors, clustered in 5 groups for each axis.

What Was Known

• While cataract surgery is generally a very safe surgery, severe complications do exist. Given the large volume of cataract surgeries each year, even relatively infrequent complications occur in significant numbers

• Pre-existing conditions are known to predispose patients to intraoperative and postoperative complications. Risk factors commonly associated with generally poor surgical outcomes include: age-related macular degeneration (ARMD), diabetic retinopathy, corneal opacity/pathology, older age, female sex, previous vitrectomy, previous retinal detachment surgery, alpha-blockers, complex surgery, and intraoperative complications.

What This Paper Adds

• In addition to confirming existing risk factor-complication relationships, we also identify novel associations, such as a significant association between systemic diabetes mellitus itself (irrespective of diabetic retinopathy) and a number of complications, including posterior capsule tear, endophthalmitis, and cystoid macular edema.

• We have also created models for predicting cataract surgery complications on an individual level, which outperform existing published models.