Abstract
Objective
To develop and validate a predictive model to estimate the probability of being nonadherent to topical glaucoma medications.
Design
Prospective cohort study.
Participants
Patients being treated with once-daily prostaglandin eye drops.
Methods
A predictive model for nonadherence was developed from the Travatan Dosing Aid (TDA) study (n = 196) using stepwise logistic regression. The performance of the TDA-derived model was assessed using a separate cohort of subjects from the Automated Dosing Reminder Study (ADRS; n = 407). The assessment was based on regression coefficients, discrimination, and calibration. We also developed a scoring system from the TDA-derived model to simplify the estimation of risk for clinical use.
Main Outcome Measures
Usage of drops was monitored electronically for 3 months in both studies. Adherence was calculated as the percentage of days on which a dose was taken within 4 hours of the average dosing time for that patient. Nonadherence was defined as taking ≤75% prescribed doses within a window starting 2 weeks after the baseline visit until 2 weeks before the follow-up visit.
Results
Six factors, including younger age, black race, worse general health status, shorter duration of glaucoma medication therapy, lower self-reported adherence, and admitting to not following doctors’ orders, were associated with being nonadherent and were included in the predictive model. The coefficients for the TDA-derived and the ADRS-derived predictive models were similar. The risk scoring system developed from the TDA study had good discrimination (area under the receiver operating characteristic curve of 0.80) and calibration (Hosmer-Lemeshow goodness-of-fit test, P = 0.102) when applied to the ADRS population.
Conclusions
The TDA-derived predictive model for nonadherence performed well in an independent population. A risk scoring system was developed using demographic data and patient responses to 4 questions to provide an estimate of the probability of being nonadherent.
Poor adherence to topical medication is a commonly observed problem. The nonadherence rate has been reported to range from 5% to 80% across 34 studies of patients with glaucoma.1–3 A recent study found that patients who failed to take topical medications as prescribed were more likely to have worse visual outcomes.4
Studies have shown that physicians cannot predict who is poorly adherent and that patient self-report is poorly correlated with actual medication use.1,5 Physicians would therefore benefit from tools that can help identify at-risk patients.
Several studies have identified risk factors for poor adherence with topical glaucoma medications. Poorer adherers were more likely to be ≤50 or ≥80 years of age and to be African American.2 Factors such as disease severity, medication cost, complexity of the dosing regimen, forgetfulness, inconvenience, and side effects have also been associated with nonadherence.2,6,7 Although these study results have provided a better understanding of the relevant risk factors involved in nonadherence, clinicians are still faced with the challenge of integrating these factors into a global assessment of the risk for a particular patient.
A model integrating these risk factors could help to estimate an individual’s probability of being nonadherent. We therefore present the development of a predictive model and assess its performance in an independent population of glaucoma patients.
Methods
We developed the predictive model using data from the Travatan Dosing Aid (TDA) study and validated it in patients who participated in the Automated Dosing Reminder Study (ADRS). Both study protocols had 2 phases. Phase 1 was an observational cohort study to assess patients’ adherence to once-daily prostaglandin therapy. Phase 2 was a controlled study to improve adherence for those found to have low adherence in phase 1. Similar research protocols were used in each study and are summarized in Table 1 (available at http://aaojournal.org).1 The protocol of each study was approved by the institutional review boards of the participating institutes and adhered to the Declaration of Helsinki.
Table 1.
Protocols of the Travatan Dosing Aid (TDA) study and the Automated Dosing Reminder Study (ADRS)
| TDA | ADRS | |
|---|---|---|
| Clinic site | Glaucoma services at the Wilmer Eye Institute and the Scheie Eye Institute | Glaucoma service at the Wilmer Eye Institute |
| Enrollment period | Aug 2006 – June 2007 | Jan 2009 – May 2010 |
| Study design | Prospective clinical study | Prospective clinical study |
| Phase I: Determine non-adherent patients | Phase I: Determine non-adherent patients | |
| Phase II: Interventions to improve adherence | Phase II: Interventions to improve adherence | |
| Intervention | Combination of:
|
Telecommunication-based intervention with automated reminder phone calls once per day |
| Prostaglandin eye drops | Travoprost (free of charge) | Travoprost, Latanoprost or Bimatoprost (not free of charge) |
| Latanoprost or Bimatoprost were switched to Travoprost during the study | ||
| Dose monitor | The Travatan Dosing Aid (DA) | Medication Event Monitoring System (MEMS) |
| A built-in memory chip records the time and date when the lever was depressed | A built-in memory chip records the time and date when the cap was opened | |
| Length of follow up | 3 months for each phase | 3 months for each phase |
| Adherence rate calculation | The percentage of days on which a dose was taken within 4 hours of the average dosing time | The percentage of days on which a dose was taken within 4 hours of the average dosing time |
| Definition of non-adherent | Took 75% or fewer prescribed doses within a window starting 2 weeks after the baseline visit until 2 weeks before the follow-up visit. | Took 75% or fewer prescribed doses within a window starting 2 weeks after the baseline visit until 2 weeks before the follow-up visit. |
Eligibility Criteria
Patients had to be ≥18 years old, be using a prostaglandin agent once per day (with or without other topical medications), and have no expectation of surgery within 6 months. They also had to agree to have their medication use monitored and be willing to receive dosing reminders, should they be included in the second phase of the study. Patients willing to participate provided informed consent.
Baseline Visit
At the initial visit, demographic and clinical information were recorded, including age, sex, self-reported race, highest level of education, glaucoma diagnosis, duration of medication treatment of glaucoma, ocular medications and dosages, intraocular pressure, and cup-to-disc ratio for each eye.
Patients were also administered a number of questionnaires including a self-assessment of general health and the Center for Epidemiologic Studies Depression Scale.8 They also answered the following questions regarding medication adherence:
Over the past month, what percentage of your drops do you think you took correctly?
How many years have you been using drops for glaucoma?
In the past 2 weeks, how frequently did you use your glaucoma drops?
Patients also responded to a series of statements using a Likert scale response of strongly disagree, somewhat disagree, neither agree nor disagree, somewhat agree, and strongly agree to the following statements:
Some days I don’t take all of my glaucoma medicines.
When I don’t take my glaucoma medicine, the main reason(s) is/are:
I’m the sort of person who follows doctors’ orders exactly.
Using glaucoma medications every single day may cause long-term side effects and problems.
I have an easy time remembering to take my prostaglandin drops once per day.
I don’t like the idea of using glaucoma drops.
Monitoring and Assessment of Adherence
The TDA (Alcon, Fort Worth, TX) is the electronic monitor used in the TDA study. A bottle of travoprost was placed in the device and a lever was used to administer a drop. A built-in memory chip recorded the time and date each time the lever was depressed. The TDA can only provide data on the use of travoprost because no other medication bottles fit in the device. Consenting patients in the TDA study who were receiving latanoprost or bimatoprost were switched to travoprost during the study. Sufficient travoprost bottles for the study period were provided free of charge to participants.
Participants in the ADRS were given a clear plastic medication bottle with an electronic cap (Medication Event Monitoring System, AARDEX Group Ltd, Silon, Switzerland) that recorded the date and time every time the bottle was opened. Subjects were instructed to keep their prostaglandin medication in the provided bottle and otherwise use it as they would normally. Patients in the ADRS therefore continued using latanoprost, bimatoprost, or travoprost as prescribed; medications were not provided.
Participants were instructed to use their medications with the monitor every day until the 3-month follow-up visit. Patients were called 1 week after enrollment to assess their ability to continue using the monitor safely and effectively. After 3 months, the patient returned the monitor and the dosing data were downloaded to a computer for analysis.
Adherence was calculated as the percentage of days on which a dose was taken within 4 hours of the average dosing time for that patient. The determination of adherence was based on doses taken within a window starting 2 weeks after the baseline visit and ending 2 weeks before the 3-month follow-up. This was done to minimize any effect of a clinic visit on adherence. If the adherence was ≤75%, the subject was determined to be “nonadherent.” This definition of nonadherence is empirical and is based on our prior work in this area.1
Statistical Analysis
We developed the predictive model from 196 patients who successfully completed the initial 3-month adherence assessment in the TDA study. In the univariate logistic regression, 10 predictors with P<0.2 were included as candidate variables. We used backward stepwise selection to determine which candidate variables were associated with nonadherence (P<0.2 to be removed from the model.) Six predictors were identified and included in the final multivariate model: age, race, self-reported general health, duration of glaucoma medication therapy, self-reported adherence during the past month, and the patient’s answer to whether he follows doctors’ orders exactly.
We validated the TDA-derived predictive model in the ADRS cohort. The performance of the predictive model was assessed using 3 criteria: Equality of regression coefficients, discrimination, and calibration.
As a first step, we tested whether the regression coefficients (log of the odds ratio) of the TDA-derived predictive models were similar to the coefficients obtained when the model was developed using the ADRS population. For this purpose, the multivariate logistic regression functions were derived using the same 6 predictors identified as predictive in the TDA study but using only data from the ADRS cohort. For each predictor, the regression coefficients for the TDA- and ADRS-derived models were compared using a z statistic (z = (βTDA − βADRS)/(SE2TDA + SE2ADRS)1/2), where βTDA and βADRS are the regression coefficients for the TDA- and ADRS-derived models, respectively, and SETDA and SEADRS are the standard errors of the coefficients.
To further evaluate whether odds ratios were correctly specified in the TDA-derived models in relation to the ADRS population, we fit a regression model to the ADRS data set using the coefficients derived from the TDA study.
We assessed discrimination by determining the ability of the predictive model to separate patients who adhered to the prostaglandin therapy from those who did not. This was summarized by calculating the area under the receiver operating characteristic curve (AUROC). Agreement between predicted and observed risk quintile in the validation population was assessed using the Hosmer-Lemeshow goodness-of-fit statistic.
We constructed a simplified risk scoring system in which we assigned points to each predictor by dividing the β coefficient (log odds ratio) in the final TDA model by the sum of coefficients, multiplying by 100, and rounding to the nearest integer. To calculate the predicted risk for an individual patient, the scores for each predictor are summed and the total score is then converted to a predicted risk for being nonadherent.
Statistical analyses were performed using STATA version 12.0 (StataCorp LP, College Station, TX).
Results
At the 3-month follow-up, 196 patients (70%) in the TDA study and 407 (83%) in the ADRS completed the adherence assessment. Those who completed the study were more likely to be male, to be white, and to have a higher level of education than those who did not.1
Baseline demographic and clinical characteristics of participants in the 2 studies were similar except for a higher proportion of African Americans in the TDA. Patients in the TDA study had a lower adherence rate than those in the ADRS (71% vs 87%; Table 2).
Table 2.
Baseline Characteristics of Competed Patients in the Travatan Dosing Aid (TDA) Study and Automated Dosing Reminder Study (ADRS)
| TDA (Derivation), n = 196 |
ADRS (Validation), n = 407 |
|
|---|---|---|
| Age, yrs, mean (SD) | 64.7 (12.2) | 69.1 (12.3) |
| Female | 82 (41.8) | 207 (50.9) |
| Race | ||
| Black | 90 (45.9) | 138 (33.9) |
| White | 100 (51.0) | 241 (59.2) |
| Other | 6 (3.1) | 28 (6.9) |
| Highest education | ||
| Primary school | 18 (9.3) | 36 (8.9) |
| High school | 49 (25.3) | 117 (28.8) |
| College | 77 (39.7) | 117 (28.8) |
| Post college | 50 (25.8) | 136 (33.5) |
| Self-reported general health | ||
| Excellent | 51 (26.0) | 90 (22.3) |
| Good | 116 (59.2) | 247 (61.1) |
| Fair | 26 (13.3) | 61 (15.1) |
| Poor | 3 (1.5) | 5 (1.2) |
| Glaucoma diagnosis | ||
| Primary OAG | 154 (83.2) | 270 (66.3) |
| Primary ACG | 4 (2.2) | 23 (5.7) |
| Secondary glaucoma | 5 (2.7) | 24 (5.9) |
| OAG suspect | 8 (4.3) | 44 (10.8) |
| OHTN | 10 (5.4) | 20 (12.3) |
| Other | 4 (2.2) | 26 (6.4) |
| Duration of treatment (yrs), mean (SD) | 8.8 (9.1) | 9.7 (9.3) |
| Use of other glaucoma medications | ||
| Prostaglandin only | 92 (46.9) | 143 (35.1) |
| Second agent | 45 (23.0) | 103 (25.3) |
| Three agents or more | 59 (30.1) | 161 (39.6) |
| IOP at enrollment (treated eyes), mean (SD) | 17.0 (4.7) | 15.9 (4.4) |
| Cup-to-disc ratio (treated eyes), mean (SD) | 0.71 (0.20) | 0.71 (0.19) |
| Monitored adherence, mean (SD) | 70.9 (23.9) | 87.1 (19.2) |
| Nonadherent rate | 87 (44.4) | 70 (17.2) |
| Self-reported adherence (%), mean (SD) | 0.95 (0.14) | 95.6 (9.5) |
ACG = angle closure glaucoma; IOP = intraocular pressure; OAG = open angle glaucoma; OHTN = ocular hypertension; SD = standard deviation.
Data presented as n (%) unless otherwise indicated.
Developing a Predictive Model Using the TDA Study
In the TDA study, 87 patients (44%) had an adherence rate of <75% and were classified as being “nonadherent.” The average adherence recorded was 88% in the adherent group, whereas it was about 50% in the nonadherent group. Patients in the nonadherent group were younger, more likely to be African American, less likely to report “excellent” health, and had a lower educational level than those in the adherent group. The nonadherent group had also been using glaucoma medications for a shorter time (Table 3, available at http://aaojournal.org). Nonadherent patients reported a lower adherence rate over the past month (93% vs 97%) and were more likely to admit missing some drops over the past 2 weeks or not taking drops for various reasons (e.g., forgot, inconvenient). The nonadherent group was also less likely to agree that they were the kind of person who followed doctors’ orders exactly. There were no differences between adherent and nonadherent patients in their agreement with the following statements: (1) eye drops can cause problems; (2) remembering my drops is easy; and (3) I don’t like the idea of using eye drops (Table 4; available at http://aaojournal.org).
Table 3.
Baseline characteristics of adherent and non-adherent patients in the Travatan Dosing Aid (TDA) Study (derivation cohort)
| Characteristics | Adherent (n = 109) |
Non-adherent (n = 87) |
p-value |
|---|---|---|---|
| Age ≤50 years, n(%) | 7 (6.4) | 16 (18.4) | 0.013 |
| Female, n (%) | 43 (39.5) | 39 (44.8) | 0.469 |
| Race, n (%) | |||
| Black | 37 (33.9) | 53 (60.9) | <0.001 |
| White | 68 (62.4) | 32 (36.8) | |
| Other | 4 (3.7) | 2 (2.3) | |
| Education high school or less, n (%) | 33 (30.3) | 34 (40.0) | 0.173 |
| Self reported general health, n (%) | |||
| Excellent | 35 (32.1) | 16 (18.4) | 0.093 |
| Good | 61 (56.0) | 55 (63.2) | |
| Fair | 11 (10.1) | 15 (17.2) | |
| Poor | 2 (1.8) | 1 (1.2) | |
| Depression score (CES-D10), mean ± SD | 4.1±5.5 | 5.0±5.0 | 0.219 |
| Glaucoma Diagnosis, n (%) | |||
| Glaucoma | 92 (88.5) | 71 (87.7) | 0.521 |
| Suspect, OHTN or other | 12 (11.5) | 10 (12.4) | |
| Duration of treatment (year), mean ± SD | 10.0±10.2 | 7.0±7.1 | 0.020 |
| Use of other glaucoma medications, n (%) | |||
| Prostaglandin only | 52 (47.7) | 40 (46.0) | 0.982 |
| Second agent | 25 (22.9) | 20 (23.0) | |
| Three agents or more | 32 (29.4) | 27 (31.0) | |
| IOP at enrollment (mmHg), mean ± SD | 16.8±4.6 | 17.3±4.9 | 0.468 |
| Cup-to-disc ratio (treated eyes), mean ± SD | 0.71±0.20 | 0.72±0.21 | 0.767 |
SD, standard deviation; OHTN, ocular hypertension; IOP, intra-ocular pressure.
Table 4.
Responses to questions about glaucoma therapy and adherence with comparison between adherent and nonadherent patients in the Travatan Dosing Aid (TDA) study (derivation cohort)
| Adherent (n = 109) |
Non-adherent (n = 87) |
p-value | |
|---|---|---|---|
| Measured adherence (%), mean ± SD | 88.0 (5.7) | 49.6 (20.6) | <0.001 |
| Reported adherence last month (%), mean ± SD | 96.7 (11.7) | 92.9 (17.1) | 0.071 |
| Past 2-week glaucoma drop use, n(%) | |||
| Every day | 86 (79.6) | 54 (65.1) | 0.053 |
| All but one or two | 17 (15.7) | 19 (22.9) | |
| Not for > 2 days | 5 (4.6) | 10 (12.1) | |
| Some days I forgot to take drops, n(%) | |||
| Strongly/Somewhat agree | 79 (73.8) | 48 (57.8) | 0.029 |
| Neither/Somewhat/Strongly disagree | 28 (26.2) | 35 (42.2) | |
| Always take eye drops, n(%) | 59 (54.1) | 27 (31.0) | 0.001 |
| Any reasons not taking drops (forgot, inconvenient, too expensive, side effects, irritates the eyes, don’t need it) | 50 (45.9) | 60 (69.0) | |
| Follows doctors orders, n(%) | |||
| Strongly/Somewhat agree | 102 (93.6) | 73 (83.9) | 0.037 |
| Neither/Somewhat/Strongly disagree | 7 (6.4) | 14 (16.1) | |
| Drops cause problems, n(%) | |||
| Strongly/Somewhat agree | 85 (78.7) | 66 (75.9) | 0.731 |
| Neither/Somewhat/Strongly disagree | 23 (21.3) | 21 (24.1) | |
| Remembering is easy, n(%) | |||
| Strongly/Somewhat agree | 94 (87.0) | 69 (83.1) | 0.537 |
| Neither/Somewhat/Strongly disagree | 14 (13.0) | 14 (16.9) | |
| I don’t like drops, n(%) | |||
| Strongly/Somewhat agree | 81 (75.7) | 64 (73.6) | 0.743 |
| Neither/Somewhat/Strongly disagree | 26 (24.3) | 23 (26.4) |
SD, Standard deviation
Six predictors of nonadherence were identified using stepwise selection: age, race, length of glaucoma treatment, general health, reported adherence, and compliance with doctors’ orders (Table 5). This prediction model performed well in the derivation cohort from the TDA study with an AUROC of 0.75 (95% confidence interval [CI], 0.68–0.82) and a nonsignificant Hosmer-Lemeshow goodness-of-fit statistic (P = 0.467; Fig 1A, B).
Table 5.
Logistic Regression of Factors and Responses Associated with Nonadherence in the Travatan Dosing Aid (TDA) Study (Derivation Cohort)
| Univariate Odds Ratio (95% CI) |
P | Stepwise Multivariate Odds Ratio (95% CI) |
P | |
|---|---|---|---|---|
| Age ≤50 years | 3.28 (1.28–8.39) | 0.013 | 2.75 (0.89–8.44) | 0.078 |
| Race | ||||
| White and other (reference) | 1 | 1 | ||
| Black | 3.03 (1.69–5.45) | <0.001 | 3.27 (1.64–6.49) | 0.001 |
| Education | ||||
| College and postgraduate (reference) | 1 | |||
| High school | 1.21 (0.62–2.36) | 0.567 | ||
| Primary school | 2.98 (1.05–8.45) | 0.040 | — | |
| Self-reported general health | ||||
| Excellent (reference) | 1 | 1 | ||
| Good | 1.97 (0.98–3.95) | 0.055 | 2.32 (0.99–5.45) | 0.053 |
| Poor or fair | 2.69 (1.05–6.90) | 0.039 | 2.94 (0.96–8.98) | 0.058 |
| Duration of treatment (yrs) | ||||
| >10 (reference) | 1 | 1 | ||
| 1–10 | 2.48 (1.21–5.08) | 0.013 | 2.32 (1.04–5.17) | 0.041 |
| ≤1 | 2.98 (1.24–7.19) | 0.015 | 1.77 (0.65–4.77) | 0.262 |
| Reported adherence ≤95% | 2.80 (1.40–5.59) | 0.004 | 2.58 (1.17–5.71) | 0.019 |
| Glaucoma drop use in past 2 weeks | ||||
| Everyday (reference) | 1 | — | ||
| All but 1 or 2 | 1.78 (0.85–3.72) | 0.125 | ||
| Not for >2 days | 3.19 (1.03–9.82) | 0.044 | ||
| Some days I forgot to take drops | ||||
| Strongly disagree (reference) | 1 | — | ||
| Somewhat disagree/neither | 3.75 (1.06–13.22) | 0.040 | ||
| Somewhat/strongly agree | 2.34 (1.25–4.39) | 0.008 | ||
| Reasons why don’t take eye drops | ||||
| Always take it | 1 | — | ||
| Any reasons (forgot, inconvenient, side effects, don’t need it, don’t want it, etc.) | 2.62 (1.45–4.73) | 0.001 | ||
| Follows doctor’s orders exactly | ||||
| Strongly/somewhat agree (reference) | 1 | 1 | ||
| Neither/somewhat/strongly disagree | 2.79 (1.07–7.26) | 0.035 | 3.18 (0.85–12.01) | 0.087 |
CI = confidence interval.
Figure 1.
A, The receiver operating characteristics (ROC) curve of discriminating people who adhered versus who didn’t in the derivation cohort (Travatan Dosing Aid [TDA] Study). The area under the ROC is 0.75 (95% confidence interval, 0.68–0.82). B, Comparison of predicted versus observed nonadherent rate for the derivation cohort (TDA). Each circle represents a quantile of predicted risk. The Hosmer-Lemeshow goodness-of-fit statistic is 2.54 (P = 0.467).
Validation of the Predictive Model in the ADRS Study
The coefficients of the predictive model (log odds ratios) in the 2 studies were similar for age, race, health, and compliance with doctors’ orders (Table 6). The log odds ratios of being nonadherent were greater in the ADRS if patients had a shorter length of glaucoma therapy and reported a lower adherence rate (P<0.1).
Table 6.
Comparison of the Regression Coefficients (Log Odds Ratios) for Predicting Nonadherence in the Travatan Dosing Aid (TDA) Study and the Automated Dosing Reminder Study (ADRS)
| TDA Study (Derivation Cohort), Coefficients (95% CI) |
ADRS (Validation Cohort), Coefficients (95% CI) |
|
|---|---|---|
| Age ≤50 years | 1.01 (−0.11–2.13) | 0.70 (−0.34–1.74) |
| Black (ref: white and other) | 1.18 (0.50–1.87) | 0.95 (0.35–1.55) |
| Self-reported general health (ref: excellent) | ||
| Good | 0.84 (−0.01–1.70) | 0.84 (−0.02–1.69) |
| Poor or fair | 1.08 (−0.38–2.19) | 0.95 (−0.06–1.97) |
| Duration of treatment (year) (ref: >10) | ||
| 1–10 | 0.84 (0.04–1.64) | 0.77 (0.01–1.54) |
| ≤1 | 0.57 (−0.42–1.56) | 1.68 (0.65–2.72) |
| Reported adherence ≤95% | 0.95 (0.16–1.74) | 1.89 (1.28–2.50) |
| Follows doctor’s orders exactly (ref: strongly/somewhat agree), neither/somewhat/strongly disagree | 1.16 (−0.17–2.49) | 1.44 (−0.08–2.97) |
CI = confidence interval; ref = reference.
When applied to the ADRS cohort, the TDA-derived predictive model had an AUROC of 0.80 (95% CI, 0.75– 0.86), whereas that of the ADRS-derived model was 0.83 (95% CI, 0.78–0.88). The AUROCs between the 2 models were not significantly different (P = 0.08; Fig 1A). The discrimination ability for the ADRS-derived model is likely an overestimate because it is based on the same cohort from which the model was derived.
The TDA-derived model fit to the validation cohort from the ADRS showed good calibration based on the Hosmer-Lemeshow goodness-of-fit statistic (P = 0.1). The predictive model tended to overestimate probabilities in the lower-risk quintiles, with a difference of 3.6% to 8.5% between predicted and observed probabilities (Fig 2B).
Figure 2.
A, The receiver operating characteristics (ROC) curve of discriminating people who adhered versus who didn’t in the validation cohort (Automated Dosing Reminder Study [ADRS]). The area under the ROC (AUROC) is 0.80 (95% confidence interval, 0.75– 0.86) for the Travatan Dosing Aid Study (TDA)–derived model and 0.83 (95% confidence interval, 0.78–0.88) for the ADRS-derived model (P = 0.078 between the 2 curves). B, Comparison of predicted versus observed nonadherent rate for the TDA-derived model in the validation cohort (ADRS). Each circle represents a quantile of predicted risk. The Hosmer-Lemeshow goodness-of-fit statistic is 9.91 (P = 0.102).
Risk Scoring System
To calculate the predicted probability for an individual patient, the scores for each predictor were summed and then converted to a predicted probability using a 7-step procedure (Table 7). For instance, a 60-year-old (0 point) African-American patient (19 points) who has been treated for glaucoma for 5 years (14 points) reported that he had excellent general health (0 point), only missed 2 drops in the past month (15 points), and admitted to “somewhat” following doctors’ orders (0 point). The total score is 48, which corresponds with a predicted probability of being nonadherent of 28% to 41%.
Table 7.
Scoring System for Estimation of the Probability of Being Nonadherent to Once-Daily Prostaglandin Therapy
| Step | Risk Score |
|---|---|
| 1. Age (yrs) | |
| ≤50 | 16 |
| >50 | 0 |
| 2. Race | |
| White and other | 0 |
| Black | 19 |
| 3. Self-reported general health | |
| Excellent | 0 |
| Good | 14 |
| Poor or fair | 17 |
| 4. Duration of glaucoma medication (yrs) | |
| ≤1 | 9 |
| 1–10 | 14 |
| >10 | 0 |
| 5. Reported adherence over the past 1 month (%) | |
| ≤95 | 15 |
| >95 | 0 |
| 6. Follow doctor’s orders exactly? | |
| Strongly or somewhat agree | 0 |
| Neither/somewhat/strongly disagree | 19 |
| Sum of score, steps 1–6 | 0–100 |
| 7. Total score | Probability of being nonadherent (%) |
| 0–10 | 3–5 |
| 11–20 | 6–10 |
| 21–30 | 11–17 |
| 31–40 | 18–27 |
| 41–50 | 28–41 |
| 51–60 | 42–57 |
| 61–70 | 50–70 |
| 71–80 | 71–82 |
| 81–90 | 83–89 |
| 91–100 | 90–94 |
To calculate the predicted probability for an individual patient, the scores for each predictor are calculated on steps 1–6 and then summed to give a total score, which is converted to a predicted probability of being nonadherent (step 7).
Discussion
We have developed and validated a 6-variable predictive model for estimating the probability of being nonadherent. The estimate of the magnitude of the predictors identified by the TDA study was similar to that found in the ADRS. When the TDA-derived model was applied to an independent population studied as part of the ADRS, the model had good discrimination and calibration, suggesting good generalizability.
In both the TDA and ADRS studies, black race, a lower self-reported adherence rate, and shorter duration of glaucoma treatment were significantly associated with an increased risk of being nonadherent. Similar findings have been reported previously in studies using a variety of data sources.7,9–14 Black race may appear as a risk factor if other predictors such as educational attainment were associated with race. However, in the TDA study, race remained associated with poor adherence even after adjustment in multivariate analyses for education, income, and other variables.7 This finding suggests that race may somehow be an independent predictor for nonadherence or that the true underlying predictors are not being measured. Patients who had been taking medications for >10 years were more likely to take their medications regularly. It is possible that patients with a longer duration of glaucoma treatment had better knowledge of glaucoma treatments and were more aware of what eye drops do and how eye drops should be used. They may also have more advanced glaucoma damage and be more motivated to take medications to prevent progression.
In addition to these predictors of poor adherence, patients who perceived that they were not in excellent health were also less likely to take their eye drops regularly. These patients may have inherent difficulties taking any medications, not just glaucoma eye drops.15,16 Sicker patients may also pay more attention to their general health status instead of any glaucoma-induced vision loss. Patients who admitted to less than ideal use of medications indeed had lower adherence rates. Many reported that they forgot or that it was inconvenient to take drops. Very few reported nonadherence due to cost, side effects, or other medical conditions.
The TDA-derived predictive model had a reasonably good discrimination, with an AUROC of 0.75 in the derivation cohort and 0.80 in validation. An AUROC of 0.75 indicates that, in 75% of the cases, the model allocated a higher predicted probability for a subject who actually is nonadherent than for a patient who is adherent. This level of performance is similar to what was found when risk models were developed using the Ocular Hypertension Treatment Study to estimate the risk of conversion from ocular hypertension to glaucoma.17
We developed a strategy to identify nonadherence by combining clinical factors and answers to some simple questions (Chang DS, Quigley HA, Friedman DS, et al. Toward a predictive model for detecting nonadherence to glaucoma medications: The ADRS. Poster presented at the American Glaucoma Society Meeting, March 3, 2011, Dana Point, CA). Patients who are non-European or have no family history of glaucoma are identified as being more likely to be nonadherent.8 Those patients are then considered to be nonadherent if 2 of the following 3 conditions are positive: (1) answering yes to the statement, “Some days I forget to take 1 of my doses of glaucoma medications”; (2) indicating they take <95% of their drops; and (3) not knowing the names of all of their glaucoma medications. Using this strategy allowed us to identify 69% of nonadherent patients, with a false-positive rate of 14%. Compared with not using any approach, this strategy enabled providers to identify nonadherence 5 times as often. However, some variables (family history of glaucoma and medication naming) were assessed in only the ADRS study, which made it impossible to validate this strategy in the TDA study.7
Use of prediction models is most appropriate for individuals who resemble subjects from the original studies. For populations with different demographic or clinical characteristics, application of this model may still be helpful; however, validation of its application is required. In addition, it is possible that factors other than those studied may be related to the risk of being nonadherent in some patients. As more studies are carried out on different populations, it is likely that the predictive capabilities of this model will be enhanced by recalibration or inclusion of other variables.
Because the prevalence of nonadherence was higher in the TDA study than in the ADRS (44% vs 17%), we calibrated the intercept of the final model by averaging the values from the TDA- and the ADRS-derived models. The higher prevalence of nonadherence in the TDA study may be explained by its younger population, with a higher proportion of African Americans (46% vs 34%) compared with the ADRS. Patients who participated in the TDA study received free medication, which may, paradoxically, have resulted in poorer adherence because of selection bias of the participants in the study.9 It is also possible that the clinicians who carried out both of these studies changed their behaviors and office routines in a way that improved adherence in the 2 years between the TDA study and the ADRS and that these changes resulted in real increases in adherence in their patients.
Poor adherence with therapy remains an important barrier to providing optimal care to glaucoma patients. The predictive model developed herein would provide an easy tool to help clinicians to better identify patients who may not take medications as prescribed. Finding such patients is an important step before intervening to improve their adherence. The multifaceted intervention utilized in the TDA study (educational video, review of barriers to using drops, regular phone call reminders, and audible and visible alarms) significantly increased adherence with glaucoma medications.18 In the ADRS study, we also found that daily, automated phone call reminders helped patients take their medications more regularly (data not shown).
In conclusion, we created a predictive model in one population and applied it to another. Using multiple performance measures, we found that the model performed well. We also developed a simple scoring system that can be used to estimate the probability of being nonadherent to once-daily prostaglandin therapy. Incorporating this into clinical practice could help physicians to better identify which patients are less likely to be adherent to their medications.
Acknowledgments
Funded by the Microsoft BeWell Fund. The funding organization had no role in the design or conduct of this research.
Footnotes
Financial Disclosure(s):
The authors have no commercial or proprietary interest in any materials discussed in this article.
References
- 1.Okeke CO, Quigley HA, Jampel HD, et al. Adherence with topical glaucoma medication monitored electronically: the Travatan Dosing Aid study. Ophthalmology. 2009;116:191–199. doi: 10.1016/j.ophtha.2008.09.004. [DOI] [PubMed] [Google Scholar]
- 2.Friedman DS, Quigley HA, Gelb L, et al. Using pharmacy claims data to study adherence to glaucoma medications: methodology and findings of the Glaucoma Adherence and Persistency Study (GAPS) Invest Ophthalmol Vis Sci. 2007;48:5052–5057. doi: 10.1167/iovs.07-0290. [DOI] [PubMed] [Google Scholar]
- 3.Olthoff CM, Schouten JS, van de Borne BW, Webers CA. Noncompliance with ocular hypotensive treatment in patients with glaucoma or ocular hypertension: an evidence-based review. Ophthalmology. 2005;112:953–961. doi: 10.1016/j.ophtha.2004.12.035. [DOI] [PubMed] [Google Scholar]
- 4.Sleath B, Blalock S, Covert D, et al. The relationship between glaucoma medication adherence, eye drop technique, and visual field defect severity. Ophthalmology. 2011;118:2398–2402. doi: 10.1016/j.ophtha.2011.05.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kass MA, Gordon M, Meltzer DW. Can ophthalmologists correctly identify patients defaulting from pilocarpine therapy? Am J Ophthalmol. 1986;101:524–530. doi: 10.1016/0002-9394(86)90940-2. [DOI] [PubMed] [Google Scholar]
- 6.Sleath B, Robin AL, Covert D, et al. Patient-reported behavior and problems in using glaucoma medications. Ophthalmology. 2006;113:431–436. doi: 10.1016/j.ophtha.2005.10.034. [DOI] [PubMed] [Google Scholar]
- 7.Friedman DS, Okeke C, Jampel H, et al. Risk factors for poor adherence to eyedrops in electronically monitored patients with glaucoma. Ophthalmology. 2009;116:1097–1105. doi: 10.1016/j.ophtha.2009.01.021. [DOI] [PubMed] [Google Scholar]
- 8.Ross CE, Mirowsky J. Components of depressed mood in married men and women. The Center for Epidemiologic Studies’ Depression Scale. Am J Epidemiol. 1984;119:997–1004. doi: 10.1093/oxfordjournals.aje.a113819. [DOI] [PubMed] [Google Scholar]
- 9.Friedman DS, Hahn SR, Gelb L, et al. Doctor-patient communication, health-related beliefs, and adherence in glaucoma: results from the Glaucoma Adherence and Persistency Study. Ophthalmology. 2008;115:1320–1327. doi: 10.1016/j.ophtha.2007.11.023. [DOI] [PubMed] [Google Scholar]
- 10.Dreer LE, Girkin C, Mansberger SL. Determinants of medication adherence to topical glaucoma therapy. J Glaucoma. 2012;21:234–240. doi: 10.1097/IJG.0b013e31821dac86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Kholdebarin R, Campbell RJ, Jin YP, Buys YM. Multicenter study of compliance and drop administration in glaucoma. Can J Ophthalmol. 2008;43:454–461. doi: 10.1139/i08-076. [DOI] [PubMed] [Google Scholar]
- 12.Djafari F, Lesk MR, Harasymowycz PJ, et al. Determinants of adherence to glaucoma medical therapy in a long-term patient population. J Glaucoma. 2009;18:238–243. doi: 10.1097/IJG.0b013e3181815421. [DOI] [PubMed] [Google Scholar]
- 13.Kass MA, Meltzer DW, Gordon M, et al. Compliance with topical pilocarpine treatment. Am J Ophthalmol. 1986;101:515–523. doi: 10.1016/0002-9394(86)90939-6. [DOI] [PubMed] [Google Scholar]
- 14.Yang Y, Thumula V, Pace PF, et al. Predictors of medication nonadherence among patients with diabetes in Medicare Part D programs: a retrospective cohort study. Clin Ther. 2009;31:2178–2188. doi: 10.1016/j.clinthera.2009.10.002. [DOI] [PubMed] [Google Scholar]
- 15.Kennedy J, Tuleu I, Mackay K. Unfilled prescriptions of Medicare beneficiaries: prevalence, reasons, and types of medicines prescribed. J Manag Care Pharm. 2008;14:553–560. doi: 10.18553/jmcp.2008.14.6.553. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Vandenbroeck S, De Geest S, Dobbels F, et al. Prevalence and correlates of self-reported nonadherence with eye drop treatment: the Belgian Compliance Study in Ophthalmology (BCSO) J Glaucoma. 2011;20:414–421. doi: 10.1097/IJG.0b013e3181f7b10e. [DOI] [PubMed] [Google Scholar]
- 17.Medeiros FA, Weinreb RN, Sample PA, et al. Validation of a predictive model to estimate the risk of conversion from ocular hypertension to glaucoma. Arch Ophthalmol. 2005;123:1351–1360. doi: 10.1001/archopht.123.10.1351. [DOI] [PubMed] [Google Scholar]
- 18.Okeke CO, Quigley HA, Jampel HD, et al. Interventions improve poor adherence with once daily glaucoma medications in electronically monitored patients. Ophthalmology. 2009;116:2289–2293. doi: 10.1016/j.ophtha.2009.05.026. [DOI] [PMC free article] [PubMed] [Google Scholar]