A novel eye drop application monitor to assess patient compliance with a prescribed regimen: a pilot study

Abstract

Purpose

To assess the ability of a novel imaging device to allow physicians to personalize therapeutic regimens based on objective patient drop administration data.

Methods

A novel imaging system was used to record video of the drop technique of subjects in clinic (n=25) or at home (n=17) for 1 week. Video assessment by a reading center was compared with patient reporting and their prescribed regimen with respect to how many drops were applied and how many landed in the eye.

Results

Reading center assessment of both drops dispensed and drops landing in the eye was significantly different from the prescribed regimen in the clinic (P_d=0.005, P_i<0.001, respectively) and at-home arms (P_d=0.003, P_i<0.001, respectively).

Conclusions

This imaging system is a powerful tool to help physicians tailor patient therapy more accurately, to help researchers evaluate new drop therapies with objective rather than subjective data, and to potentially facilitate better patient training for improved drug delivery.

Introduction

Eye drop medications, both prescription and over the counter, are a mainstay of therapy for treating ocular disorders such as glaucoma. Eye drops are a preferred method of treatment because they are effective, non-invasive, and, in theory, easy to use. However, it is well-known that compliance and getting the eye drop in the eye are significant problems.^{1, 2, 3, 4, 5, 6, 7} Even in cases where a doctor has asked patients direct questions regarding eye drop regimen compliance, and electronic monitoring has been employed, drug delivery failure is still an issue.^{8, 9, 10, 11, 12, 13} Several devices have been developed to help patients adhere to and monitor eye drop regimens,^{12, 13, 14, 15} although none of these techniques for monitoring compliance are meant to determine whether the drops get into the eye; rather they measure if a patient attempts to administer the eye drops.

A recent study using video monitoring found that out of the subjects claiming not to miss the eye when applying drops, nearly one-third actually missed; and out of all the subjects, approximately one-third could not get a drop onto the eye at all.⁵ Another recent study found that only about 9% of patients who use eye drops are able to properly self-administer them.⁷ Thus, even if patients are reminded about taking their eye drop medication, there still exists a significant number of patients who fail to achieve eye drop delivery that follows their prescribed eye drop regimen. Some patients waste copious amounts of eye drops trying to get the medication into their eyes only to end up with too many or too little drops in their eye.¹⁶ Thus, even if the eye drops do make it into a patient's eye, there is no way to know if the correct dosage was used. This makes it hard for physicians to determine whether their prescribed drug regimen is not working due to an ineffective drug or due to the drop not making it into the eye, which in turn can impact future therapeutic choices. Although direct observation can provide additional information, it is not always an option for patients taking eye drop medications; and even if a direct observer is available to attend all applications of the medication, direct observation is not a reliable method of detection as it is very difficult to determine whether an eye drop has landed in the eye fully or on the lid in real time. If the patient or observer blinks, it can be hard to tell if drop(s) that end up on the lid are excess that overflowed the tear reservoir, or if the drop did not get into the eye at all. We tested a novel, portable, reusable, and inexpensive device, the Eye Drop Application Monitor (EDAM), which can allow physicians to directly monitor patient compliance and even provide information on how to improve a subjects drop-delivery technique (Figure 1).

Figure 1.

The Eye Drop Administration Monitor (EDAM). Shows actual images of (a) the EDAM alone and with the lights off, (b) attached to a drop bottle with lights on, and (c) connected to the monitor/recording device.

Rather than relying on indirect compliance measurements such as bottle weight and timers/alarms, the EDAM uses a video monitoring system, which records the time of application, as well as the actual administration itself; therefore, physicians and patients can see when, and how much of the eye drop actually was applied during a self-administrating procedure and how much made it into the eye. This information can then be remotely transferred to a computer for easy viewing and transmission to a reading center, thereby allowing physicians to easily monitor the compliance of their patients. It also allows the patient to review what they are doing wrong, and for a physician, nurse, or ophthalmic assistant to make specific recommendation on how to correct and improve the drop delivery or alter the patient's therapeutic regimen. Furthermore, this device is to benefit not only patients using prescribed eye drops but also to enhance data collection accuracy for clinical trial(s) that test the efficacy of an eye drop therapy.

This work was initiated as a pilot and exploratory study, as it was desirable to allow flexibility to try to identify issues that may develop prior to undertaking a longer or larger study. The goal was to see if the EDAM is easy to use, if it captures the desired data, and to learn what if any issues occur at home with patient's home drop installation.

Materials and methods

Inclusion/exclusion criteria

Subjects were randomly recruited to each phase of this study on a first come first consented basis in the course of their normal clinical visitation schedule at the Retina Health Center or Konowal Vision Center. For the portion of the study done in clinic, the first 25 patients who successfully completed the first phase were included in the data analysis. However, only 17 subjects successfully completed the at-home phase and data analysis through 31 August 2014; after a device design was achieved that eliminated image acquisition problems that occurred with earlier versions, such as lid or patient's fingers blocking the images of the drop application, the study was discontinued. At that point, a new trial assessing at-home drop compliance in cataract surgery patients was initiated with the new model. The primary inclusion criteria for study patients who participate in the at-home testing was that they had to be starting or receiving at least one prescription eye drop and that it had to be prescribed for use at least once a day. Only those who could self-consent were enrolled in the study. All subjects in the trial signed informed consent forms, and Institutional Review Board approval for this study was obtained from the Lee Memorial Health System. This work was HIPAA compliant and adhered to the tenets of the Declaration of Helsinki, and we certify that all applicable Institutional and Governmental regulations concerning the ethical use of human volunteers were followed during this research.

Study design

This was a non-randomized prospective pilot study. There were two phases to this study: one phase took place in the clinic; the other took place at the subject's home. In the clinic phase, subjects were equipped with the EDAM, trained to use the device, and asked to dispense one drop of artificial tears into each eye. Following each application attempt, patients were asked to write down how many drops were dispensed and if the drop landed in the eye, outside the eye, or half in and half out. The patient was then asked to repeat the process for a total of 10 times. Accuracy of the written logs was compared with logs created by a technician at the reading center after viewing the application procedure in slow motion.

For the second phase of the trial, subjects, and their designated caregivers, if applicable, were asked to use the EDAM at home for all drop applications for 5–14 days. At the end of the trial period, subjects were asked to return to the clinic to allow the study personnel to download all images acquired by the EDAM. Again, the accumulated videos were reviewed by the reading center and compared with subject logs.

Statistical analysis

The number of drops dispensed and the number of drops that landed in the eye were compared between the subject, the prescribed regimen, and the reading center for the in-clinic phase and at-home phases. A subject's prescribed at-home regimen varied between individuals, with some using only one drop of one medication per day, and others using as many as six drops from three different medications per day; this led to large SD between subject's drops dispensed and drops in, but a paired Student's t-test was used to account for this variation. To normalize the variation in the number of drops used, variability from the prescribed regimen was evaluated as a percentage that each subject was off from their prescribed regimen and the reading center.

Results

Patient demographics

Thirty seven patients were enrolled in the clinic phase of this study, and 38 patients were enrolled in the at-home phase, although only 25 and 17 subjects, respectively, successfully completed the trial. Ten of these subjects successfully participated in both phases. Of those who successfully completed the clinic phase, 8 were women and 17 were men, while in the at-home phase, 6 were women and 11 were men. The average age of the in-clinic subjects was 78 with a range of 61–95, while the average at-home subject age was 70 with a range of 23–89. All the in-clinic phase patients used eye drops prior to the study.

In-clinic phase

The prescribed regimen was compared with the logs created by the subject and the reading center for both the in-clinic and at-home phases (Table 1).

Table 1. Drops in and drops dispensed for in clinic and at-home arms as reported by subjects and the reading center compared with the prescribed regimen.

In clinic
Drops dispensed				Drops in
Subject #	Subject	Reading center	Prescribed regimen	Subject #	Subject	Reading center	Prescribed regimen
1	20	45	20	1	17	10	20
2	20	20	20	2	19	20	20
3	16	21	20	3	6	10	20
4	27	24	20	4	23	18	20
5	17	20	20	5	14	20	20
6	20	27	20	6	18	4	20
7	20	19	20	7	17	10	20
8	18	24	20	8	12	16	20
9	20	20	20	9	19	15	20
10	16	20	20	10	13	13	20
11	20	20	20	11	8	9	20
12	20	26	20	12	19	16	20
13	20	20	20	13	15	13	20
14	20	23	20	14	20	13	20
15	23	21	20	15	22	12	20
16	21	28	20	16	15	3	20
17	20	20	20	17	7	8	20
18	22	47	20	18	14	4	20
19	16	21	20	19	13	10	20
20	21	25	20	20	13	19	20
21	19	28	20	21	19	23	20
22	20	22	20	22	10	11	20
23	25	26	20	23	21	19	20
24	18	19	20	24	15	17	20
25	19	24	20	25	15	17	20
Average	19.92	24.4	20	Average	15.36	13.2	20

At home
Drops dispensed				At-home drops dispensed ratios
Subject #	Subject	Reading center	Prescribed regimen	Subject #	Subject/reading center	Subject/prescribed regimen	Reading center/prescribed regimen
1	54	34	26	1	159%	208%	131%
2	7	7	7	2	100%	100%	100%
3	8	9	6	3	89%	133%	150%
4	48	61	54	4	79%	89%	113%
5	95	104	84	5	91%	113%	124%
6	43	55	42	6	78%	102%	131%
7	14	17	14	7	82%	100%	121%
8	16	14	14	8	114%	114%	100%
9	15	17	13	9	88%	115%	131%
10	7	15	7	10	47%	100%	214%
11	7	7	7	11	100%	100%	100%
12	14	16	14	12	88%	100%	114%
13	17	34	14	13	50%	121%	243%
14	14	10	10	14	140%	140%	100%
15	32	34	32	15	94%	100%	106%
16	105	102	104	16	103%	101%	98%
17	75	90	76	17	83%	99%	118%
Average	33.59	36.82	30.82	Average	93%	114%	129%

Drops in				At-home drops in ratios
1	52	21	26	1	248%	200%	81%
2	6	5	7	2	120%	86%	71%

At home
Drops in				At-home drops in ratios
Subject #	Subject	Reading center	Prescribed regimen	Subject #	Subject/reading center	Subject/prescribed regimen	Reading center/prescribed regimen
3	5	4	6	3	125%	83%	67%
4	45	47	54	4	96%	83%	87%
5	86	78	84	5	110%	102%	93%
6	42	34	42	6	124%	100%	81%
7	12	10	14	7	120%	86%	71%
8	15	12	14	8	125%	107%	86%
9	9	10	13	9	90%	69%	77%
10	2	10	7	10	20%	29%	143%
11	6	7	7	11	86%	86%	100%
12	12	10	14	12	120%	86%	71%
13	12	0	14	13	NA	86%	0%
14	10	9	10	14	111%	100%	90%
15	27	30	32	15	90%	84%	94%
16	104	101	104	16	103%	100%	97%
17	71	70	76	17	101%	93%	92%
Average	30.35	26.94	30.82	Average	112%	93%	88%

In clinic actual drop comparisons		At-home actual drop comparisons
Drops dispensed	P-value	Drops dispensed	P-value
Subject vs reading center	0.003451	Subject vs reading center	0.149988
Subject vs prescribed regimen	0.876579	Subject vs prescribed regimen	0.137618
Reading center vs prescribed regimen	0.004906	Reading center vs prescribed regimen	0.002654
Drops in		Drops in
Subject vs reading center	0.068492	Subject vs reading center	0.115668
Subject vs prescribed regimen	0.000031	Subject vs prescribed regimen	0.795148
Reading center vs prescribed regimen	0.000002	Reading center vs prescribed regimen	0.000597

Comparative analyses were done using a Student's t-test.

Thirty seven subjects were enrolled in the clinic phase of the trial, though data from only 25 subjects was used. Six subjects were not used for analysis because they did not follow the trial directions and held the device incorrectly, resulting in videos that did not capture the drop application; training on where/how to position the recording head as well as a design modification has reduced the problem. Five subjects filled out the data log incorrectly, making their data unusable. One subject was dropped from the analysis because the device broke during use so they were unable to complete the study requirements.

Drops dispensed

Subjects were asked to dispense a total of 20 drops (10 in each eye) in their physician's office. In comparison, the reading center noted an average of 24±7.1 drops dispensed (P=0.005, Figure 2a), while the subject log only noted 20±2.5 drop (P=0.88). Thus, the subjects thought they were dispensing the correct amount of drops, when in reality they were dispensing significantly more drops than prescribed (P=0.003).

Figure 2.

In-clinic comparison of the number of Drops dispensed and how many landed in the eye. (a) Shows the log book comparison between the actual number of drops dispensed for each subject during the in-clinic phase and the average number of drops dispensed. Bars denote SD, **P<0.005 (n=25, Student's paired t-test). (b) Shows the log book comparison between the actual number of drops that landed in the eye for each subject during the in-clinic phase and the average number of drops that landed in the eye. Bars denote standard deviation; ***P<0.001 (n=25, Student's paired t-test).

Drops in

In comparison to the prescribed regimen, subjects believed they delivered an average of 15±4.54 drops (P<0.001, Figure 2b) to their eyes, while the reading center noted only an average of 13±3.7 drops (P<0.001) were delivered to their eyes. Although there was no statistically significant difference between the number of drops landing in the eye logged by the subject and the reading center (P=0.07), there was a trend toward significance.

At-home phase

Results were analyzed for 17 of the study subjects. Subject's prescribed regimens varied by subject, and ranged from one drop in one eye per day to eight drops in each eye per day.

Thirty eight subjects were enrolled in the at-home phase, though only 17 provided usable data. Seven subjects did not follow the trial directions and either held the device incorrectly, forgot to use the device, or used it incorrectly, resulting in videos that did not capture the drop applications (poorly angled cameras viewing only the eyelid or cheek, did not turn camera on, or placed their hand or fingers in front of the camera). Design modifications were implemented in order to overcome this difficulty. Seven subjects filled out their log incorrectly, and three subjects forgot to fill out the log completely. Additional patient contact has been initiated to remind subjects to fill out their log and how to review any issues with the use of the device, although this problem persists, underscoring the difficulty of obtaining accurate subject reporting. Four subjects claimed to have issues with the device malfunctioning, although a malfunction could only be found in one of the devices.

Drops dispensed

Compared with the prescribed regimen, with an average of 31 drops (range of 7–104), the reading center noted an average of 37 (range 7–104, P=0.003, paired t-test, Figure 3a), while the subject log only noted 34 drops (range 7–105, P=0.14, paired t-test). Differences between at-home subject and reading center logs trended toward, but did not reach statistical significance (P=0.15, paired t-test). Perhaps more relevant in this case is the average percent that each subject over or under dosed. On average, subjects reported dispensing less than they actually did; this led them to believe that they were over dispensing their medication by only 14% when in reality it was 29% (Table 1). Thus, although subjects thought they were only dispensing a slightly incorrect amount of medication, they were underestimating their error rate and actually dispensing significantly more drops than prescribed, indicating that they waste a significant amount of their medication.

Figure 3.

At-home comparison of drops dispensed and drops that landed in the eye. (a) Shows the log book comparison between the actual number of drops that landed in the eye for each subject during the at-home phase and the average number of drops that landed in the eye. Bars denote SEM; ***P<0.001 (n=17, Student's paired t-test). (b) Shows the ratios between the subject or reading center and the prescribed regimen for each subject, and shows the average of these ratios. Bars denote SEM.

Drops in

Compared with the average prescribed regimen of 31 drops, the reading center noted an average of 27 (range 0–101) drops landing in the eye, while the subject log noted 30 (range 2–104) drops (P=0.8, paired t-test, Figure 3b). Although the reading center logs and the prescribed regimen were significantly different (P<0.001, paired t-test), the subject logs were not significantly different from the prescribed regimen (P=0.8, paired t-test) or the reading center logs (P=0.11, paired t-test) although the latter approached statistical significance. On average, subjects believed that they got more drops in than they actually did (Table 1). This resulted in subjects believing they were on average 7% off from their prescribed regimen, when in fact they were 17.6% off (Subject 13 was dropped from the Table 1 average as they did not get any drops in their eyes, resulting in a ratio with a denominator of 0. When this is included the average is 17.6%.). Thus, similar to the drops dispensed assessment, at-home subjects were on average about twice as non-adherent as they believed when reporting drops landing in the eye. Again, this indicates that subjects believed they were only slightly under-dosing themselves, leading them to think that the difference from the prescribed regimen was not significant, when in reality it was.

Drops delivered properly

During the in-clinic phase only one subject was able to deliver all of their 20 drops properly (one drop per attempt, which went in, without contacting their lashes and/or cornea with the dropper bottle). Only 15/25 (60%) of the clinic phase subjects were able to apply drops without touching their eyeball, eyelid/lash, or adnexa. On average, 28% of the attempts to deliver an eye drop resulted in contact (Figure 4) in-clinic phase. In the at-home phase, none of the subjects were able to perform proper drop applications for every attempt during their regimen. Only 5/17 (29%) of at-home subjects were able to apply their drops without ever touching their eye, eyelid/lash, or adnexa. An average of 19% of attempts to deliver an eye drop resulted in contact (Figure 4).

Figure 4.

Percentage of patients who properly dispensed their eye drops. Bar graph shows what percent of in-clinic and at-home subjects could properly dispense their eye drops, what percent used more than one drop, and what percent contacted/contaminated the eye drop bottle tip.

Discussion

This pilot trial has both confirmed previous reports on difficulties that patients experience when administering eye drops, and has further shown that the experimental imaging device can provide a unique insight for physicians and patients alike. By reviewing the video captured by the EDAM, it was clear that few patients are able to properly and consistently apply the drops; most have issues either getting the drops in their eyes, applying the correct amount of drops, touching the bottle to the eye or adnexa or some combination of the above. Our results demonstrate a variety of possible ways a patient can incorrectly apply their eye drops and is consistent with previous literature that has found that <10% of patients can properly apply their own eye drops. We found only one subject in-clinic (and zero at-home subjects) was able to properly dispense 20 drops and have them all land in their eye without touching their eyelash, eyelid, or adnexa with the dropper; no at-home subjects were able to do this. This underscores the magnitude of the problem patients face with drop installation and potential benefit of a device, which can help physicians and patients understand the problem so that it can be corrected. This could include implementation of alternative surgical therapeutic options or, potentially, attempts at restraining the patient.

The in-clinic and at-home results showed many similarities. Interestingly, it was noted during the at-home phase that subjects with poor drop technique generally drastically underreported how many drops they dispensed and drastically over reported how many they got in their eyes. Thus, although the average represents a range of patients with differing drop techniques, it was clear that some subjects had very good drop technique and others did not, so even with the relatively small pilot trial, significant differences were found. However, none of the subjects reported the EDAM impeding their drop attempt; all tolerated the device very well.

One issue that ophthalmologists face in the clinic is that many of their patients are chronically running out of their prescribed eye drop medication before they are scheduled for a refill. This results in loss of regimen adherence as patients are only prescribed a certain amount of medication per month and it can be difficult and costly to refill, which can contribute to poor clinical outcomes. The EDAM has shown that in-clinic and at-home subjects believed that they were dispensing the correct amount of drops in-clinic and 14% more than the prescribed regimen at home, when in fact they were dispensing about 22% and 29% more, respectively, thus clearly showing why patients run out of their prescribed medication before their allotted time. The Centers for Medicare and Medicaid Services guidelines state that patients should receive a refill at 70% of predicted days of use in order to have enough drops to comply with their prescribed regimen;¹⁷ using the subjective data, 3 out of 17 at-home exceeded this number, and using the EDAM system it was found that 6 out of 17 exceeded it (P=0.24, chi square). Of those six, three required 50% or more drops to achieve what they believe was their prescribed regimen. During the in-clinic phase, only one out of 25 subjects (4%) thought that they over dispensed their drops by 30% or more, when in fact 7 did (28%), a significant difference of opinion (P=0.02, chi square). Of these seven subjects, two dispensed >50% of the requested drops. Although some of this difficulty may be due to the device, it likely (based on our data and clinical experience of patients complaining) explains why patients are running out of drops before they are allowed by their insurance to refill their prescription. Alternatively, one could argue that patients in this study may have spent more time and effort trying to get their drops in correctly during the study because they knew that they were being recorded, in which case the number of patients running out of drops may be much higher than we found. In either case, our data support the notion that current insurance guidelines are not providing enough medication to allow a significant number of patients the ability to comply with their prescribed regimen. It may be possible that for problem patients, the EDAM system can be used to determine what the issues are with their drop delivery, and to train them to overcome their difficulties. A study to answer this question is currently being planned and should commence shortly.

The results from the EDAM pilot study also highlight its potential to provide information that may be useful in clinical research studies. For example, if subjects are routinely misrepresenting their actual applied dosage, this can have serious effects on the results of a clinical trial and one can see how large differences in perceived versus actual drug administration would skew the results of a drug trial. This may also result in over reporting of adverse side effects, which would not occur or occur less frequently at the prescribed doses. Currently, although drug companies can track the amount of medication dispensed by a subject in a clinical trial, there is no objective way for them to know how much is actually being delivered to or around the subject's eye. Along with more accurate compliance tracking, the EDAM system ensures the data collected reflects the efficacy of the drop and not the patient's or an observer's perceived successful or unsuccessful drop application. It can provide the ability to intervene early if a patient is non-compliant and/or not applying the drops correctly, provide a better understanding of side effects, and determine if they are due to over dosing, miss-dosing, or sustaining contact/trauma from the bottle. Finally, the EDAM may help improve our understanding of how drop bottle shape and tip shape can affect a patient's technique and compliance. This should help eye drop bottle designers understand patient issues better and ultimately lead to better and more efficient drop bottle and/or drop delivery designs and techniques.

In conclusion, this study has found that there are a number of issues with eye drop delivery that impact patients, their physicians, and makers of eye drop medications. Most importantly, subjects are not adhering to their prescribed regimen and there exists a significant difference between subject perceived adherence and actual adherence at home. This results in subjects incorrectly reporting their adherence, making it extremely difficult for physicians to determine the cause of ‘treatment failure'. Is the prescribed regimen not working due to incorrect dosing or incorrect medication? Is surgery a better option for certain patients? Are insurers providing enough drops per month for patients with difficulty getting drops? And can a subject's drop technique be amended through personalized training and education to improve their technique? Until now, objective data to answer these questions has been unavailable. Advances in technology have changed that, and the current pilot study has shown that the EDAM is the first device that can do this in an outpatient setting. Hopefully, by shedding light on these issues the EDAM system will help physicians better understand the problems so they can better treat their patients.

AME, HW, and RLA have a financial interest in the EDAM device. The remaining authors declare no conflict of interest.