The utility of an electronic adherence assessment device in type 2 diabetes mellitus: a pilot study of single medication

Abstract

Objectives:

The primary objective of this pilot study was to determine if the Medication Event Monitoring System (MEMS) is capable of providing meaningful estimates of compliance within the indigenous Qatari population. The secondary objective was to highlight any specific problems which might be associated with the use of MEMS within this population.

Method:

A sample of adult diabetic Qatari patients attending an outpatient diabetic clinic were administered a Knowledge, Attitude, and Practices (KAP) questionnaire and then dispensed one of their regular medications in a MEMS^®-fitted bottle. Data contained in the MEMS^® were downloaded after the patients returned for a refill and adherence was estimated using 2 methods: pill count and MEMS^® data.

Results:

A total of 54 patients agreed to participate in this pilot study. Adherence to daily doses was 67.7% and with regimen 13.7%. No correlation was found between adherence assessed by pill count and MEMS^®. The association between KAP and adherence was generally poor. A number of other issues and challenges in the use of MEMS^® that could affect its utility were noted and will be discussed.

Conclusions:

Our results revealed problems associated with the use of MEMS^® that could affect its usefulness in assessing adherence in this part of the world. Some issues identified in this pilot study included retrieving the MEMS^®, registering extra opening of MEMS^®, desire to hoard medicine by taking doses at different frequency than recorded in MEMS^®. All these issues could be closely associated with the attitudes and practices of the patients, as demonstrated by our KAP analysis and correlations.

So far there is no accepted “Gold Standard” to objectively assess adherence with drug therapy. Strategies to assess adherence include self-reporting, pill-counting, and the use of the Medication Event Monitoring System, or MEMS^®. To our knowledge, the latter method had not previously been tested in the Gulf country region. There is little doubt that pharmacotherapeutic (or pharmacological) agents remain the most used treatment modality for the management of disease,¹ which means that successful pharmacological treatment of illness (especially chronic illness) would be significantly influenced by the extent to which patients comply with (or adhere to) their medication regimens. Adherence with medication therapy was defined as the degree or extent of conformity to the recommendations made regarding day-to-day treatment by the provider with respect to the timing, dosage, and frequency with prescribed medication.² The problem of inadequate patient prescription adherence is common to all areas of medicine.^2–5 However, the assumption by many healthcare providers that nonadherence can easily be detected by the treating physician had not been substantiated.⁶ On the contrary, results of recent studies showed that physicians could not predict adherence with any more accuracy than if they were guessing.⁷ Several factors that might lead to poor adherence were reported in the literature. Lack of patient information about their disease or its treatment, adverse effects of prescribed drugs, the patient’s dissatisfaction with their health condition, the personality of the patient, the disease, and the treatment (cost and complexity).^7,8–10 Assessing adherence presents an ongoing challenge. Although several methods are used to estimate and quantify adherence, accurate measurement continues to be difficult, especially when studying populations with unique cultural traditions and geographical situations.

Attempts to measure nonadherence more objectively have stimulated the development of a wide spectrum of methodologies and provided a considerable body of descriptive and analytic literature. These strategies fall into three main categories: measuring biological serum levels of the medication to check if the doses prescribed were actually taken by the patient, using data derived from dispensing records, and gauging information directly from the patient to infer the degree of adherence.^11–14

The later method is operationalized using three basic techniques: self-reporting,^a pill-counting, and the use of electronic adherence assessment devices. One such device is the Medication Event Monitoring System¹⁵ (MEMS^®). Using biological assays to measure the concentration of a drug or its metabolites is intrusive, burdensome to the patient, and often costly.¹⁶ Drug and food interactions, physiological differences, and the half-life of the drugs all complicate the measurement and may diminish reliability. Estimating adherence based on data derived from dispensing records is faced with concerns related to data completeness and data records reliability. Patient self-assessed adherence to drug therapy relies on interviewing patients to assess their knowledge of the medications and the dosing schedule provides little information and is liable to be affected by interviewer bias.^2,17 Counting the remaining number of pills of a patient who returns for a refill presents an easy method for assessing adherence, but is not free of problems. For example, Grymonpre and coworkers suggested that pill count may underestimate patient adherence in older populations, because pill counts are often based upon the date a prescription is filled, patients who get prescriptions refilled prior to their first one running out and then combining pills into a single (and possibly nonoriginal) bottle presents complications.

The MEMS^® is an electronic innovation that brings modern technology and an element of objectivity to the assessment of patient adherence. This is an alternative approach to the study of adherence and – as with all methods – there are aspects of the technique which may potentially to introduce bias and misunderstanding into the interpretation of results.^b For example, it cannot distinguish a missed dose from one doubled at the next bottle opening (so that patient diaries have been advocated¹⁴ as a suitable adjunct to method). Clearly, one fundamental weakness of the MEMS^® is also its inability to distinguish between cap openings in which a dose is or is not removed. However, in spite of these shortcomings, a growing number of investigators are taking a MEMS^® path to the study of adherence although the analysis and interpretation of the results have stimulated a wide variety of approaches.

However, although this technique has been used successfully within Western populations for the last 10 years, to our knowledge it had not been applied to any population in the Arabian Gulf or the wider Middle East. Given the significant number of potential pitfalls which are associated with MEMS^®, we decided that – prior to embarking on our full study (aimed at estimating the prevalence of adherence among Qataris, in a variety of circumstances) – it was first necessary to validate the technique within the Arabian Gulf. Therefore the two primary objectives in this pilot study were: (a) to investigate the capability of MEMS^® for accurately estimating adherence with drug therapy in the context of type 2 diabetes mellitus (T2DM) within the Qatari population of the Arabian Gulf by comparing parallel estimates of pill counts and indications from the questionnaires, and (b) to assess the value of the questionnaires in suggesting possible reasons for any lack of adherence. We were also interested in identifying any specific local factors which might compromise the practicality of applying the MEMS-based technique (and associated questionnaires) to this population.

Methods

This study was approved by the Institute Review Board in Hamad Medical Corporation (Doha, Qatar) in October 2008.

Study protocol

All adult Qatari patients with a confirmed diagnosis of T2DM and who had scheduled visits at the Endocrinology Outpatient Clinic at Hamad General Hospital were identified prior to their appointment and offered the opportunity to enroll in the study by the physician if they were prescribed the oral anti-diabetic medication metformin. Trained research assistants discussed the consent process with the patients; the patients were informed that the purpose of the study was to assess the usability of new medication containers which were intended to improve the efficiency of the treatment of diabetes at the hospital. Those who consented were administered a Diabetes Habits and Beliefs Questionnaire (DHBQ). Any other medications prescribed to the patient were packaged in their usual containers.

Before leaving the pharmacy, all enrolled patients were asked to: (i) to take all future doses of metformin only from the serially numbered MEMS^® bottle (containing one month’s supply), (ii) to return the MEMS^® bottle to the outpatient pharmacy in one month’s time (in order to obtain a refill), (iii) to only to take the prescribed dose each time the bottle was opened, and (iv) to return the bottle with all remaining pills (if any). During the follow-up appointment, the medication bottle and MEMS^® were retrieved and a short MEMS^® satisfaction questionnaire was administered to assess the patient’s experience in using the MEMS^®. Data contained in the MEMS^® were then downloaded using MEMS^® software and the number of remaining pills was documented. Arabic versions of all questionnaires were used throughout this study.

Questionnaires

This 51-item DHBQ questionnaire was adapted from the Diabetes Time Management Questionnaire created by Gafarian et al (1999) which had been specifically developed to assess beliefs, general time management skills, and areas specifically relevant to adherence to diabetes therapy regimens.¹⁸ The DHBQ also contained questions adapted from the Hospital Anxiety and Depression Scale, an instrument that was originally developed as a self-assessment scale for detecting states of depression and anxiety in an outpatient setting.¹⁹ During questionnaire development, the translated version of the DHBQ was tested for its face validity and cultural adaptability within a small group of Qatari diabetic patients not taking part in the study. In addition to basic demographics information (age, marital status, highest education, and employment category) the DHBQ Questionnaire contained the following 47 items:

• (8) questions: diabetes general knowledge

• (14) questions: diabetes-related activities (practice)

• (6) questions: time-management (habits)

• (9) questions: attitude to living with diabetes

• (10) questions: depression status (mood)

Each of these questions was rated on a 7-point Likert scale, and the overall score for each category was obtained by summing the individual responses.

The time since diagnosis (TSD) with diabetes was retrieved from each patient’s medical record shortly after the face-to-face interviews. The MEMS^® User Satisfaction Questionnaire comprised three questions: (i) How easy is it for the patient to use the MEMS, (ii) Did the patient use the MEMS as their only source for metformin, and (iii) Did the patient ever open the container and not take a dose.

Medication event Monitoring systems (MEMS^®)

The Medication Event Monitoring Systems (MEMS^®; APREX, Division of AARDEX, Union City, CA) utilizes drug packaging with electronic circuitry to compile ambulant patient medication dosing histories.²⁰ Each monitor records information pertaining to the times that the medication container is opened and closed (medication events). The MEMS^® is simply a standard medication container bottle whose cap is fitted with a microprocessor that records every bottle opening. These real-time data are stored on the MEMS^® and can later be uploaded to a computer. In this study only one medication (metformin) was monitored using MEMS^® because it was felt that it would be impractical and burdensome to use multiple MEMS^® and prior research has demonstrated that monitoring one medication with the MEMS^® provides a valid indicator that patients took all of their medications.²¹ Metformin was selected as the medication to be investigated because it can have different dosing frequencies (once, twice, or thrice daily regimens) and because it is frequently prescribed to T2DM patients. Data for analyses were exported from the MEMS^® database into the JMP statistics package (SAS Institute, Cary, NL, USA). The events were then aggregated into sets of same-day events and displayed in day-of-year order for each study participant. The number of opening events per day was recorded for all days within a 30-day window beginning at the first patient event of the trial. The total number of events per patient was also recorded and zeros were inserted for days which had no recorded opening events. Any opening within 15 minutes of the previous opening was ignored.

Measurements of adherence

Three independent measures of adherence were obtained: (i) MEMS^® dosage adherence (MEMSd) which is the percent number of bottle openings divided by the total number of doses prescribed, (ii) MEMS^® regimen adherence rate (MEMSr) which is the percentage of days in which the dose regimen (measured as bottle openings) was executed as prescribed,²² (iii) Pill count was reexpressed as a percentage: (1– (N^o._returned/N^o._prescribed)) × 100%. A threshold value of acceptable adherence of 80% was used for all three measures.

Statistical analysis

The data were analyzed using JMP (version 5) and SPSS^® (version 17); scientific plots and figures were produced using Origin (Microcal v5.1). Two-group comparisons were assessed using t tests (where permissible); otherwise the nonparametric Mann–Whitney test was used. Differences between multiple groups were assessed using one-way analysis of variance. Correlations and intercorrelations between questionnaire scores and adherence estimates were estimated using simple pairwise correlation coefficients and Spearman’s nonparametric correlation coefficient. Statistical significance was taken throughout to be P < 0.05.

Results

Fifty-four patients agreed to participate, of whom the majority (61%) were female. The median age was 51 years (mean 50, SD 9.6), and most patients were married. Because of logistical problems (mainly related to a limited budget assigned for purchasing of equipment in the pilot study), only 37 out of the total patients enrolled in this study were issued their medication in MEMS^® containers (68%). The responses to the MEMS^® satisfaction questions (administered at the second visit when the patients returned the MEMS^®) showed that 90% of the patients who were administered the questionnaire found the bottle with the MEMS^® cap easy to handle, and 86% of patients said they used the MEMS^® as the only source for their prescribed metformin. Furthermore, 94% of patients said that they opened the container only to take a dose of their medicine. However, only 57% of the total number of patients who were handed MEMS^® responded to this questionnaire.

MEMS^® retrieval

Out of 37 patients provided their medication in MEMS-capped bottles, 13 (35%) patients returned their medicine bottles later than the refill date. The mean number of late days was 13 (SD = 17.6). Six patients (22%) failed to return the MEMS^® despite multiple follow-up calls. One patient admitted that she threw the bottle filled with MEMS^® in the trash bin.

Adherence results

Thirty seven MEMS® units were handed out to participants with the prescribed metformin tablets but adherence was assessed for only the 27 patients who returned their units. Quantification for the degree of adherence for these patients was computed as MEMSd (% number of bottle openings divided by the total number of doses prescribed), and MEMSr (% days in which the dose regimen was taken, measured as bottle openings by the total number of days for which the medication was prescribed).

The number of times the MEMS^® was opened ranged between 9 and 155 per patient (Median = 70; Mean = 68.63; SD = 39.34; SEM = 7.57). The number of days when the MEMS^® was used by the patients ranged from 2 to 42 per patient (Mean = 26.52; SD = 11.22; SEM = 2.16; Median = 29). Overall adherence with daily doses (MEMSd) was 67.7% (SE = 6.9; Median = 82.2%). Overall adherence with regimen was 13.7% (SE = 1.9; Median = 15.0%). Frequency of distribution of adherence as percentage for MEMSd and MEMSs is illustrated in Figures 1 and 2.

Figure 1.

Frequency of distribution of compliance with daily dose (n = 27).

Figure 2.

Frequency of distribution of compliance with regimen (n = 27).

MEMS^®-measured adherence: correlations

Table 1 summarizes correlations between adherence (MEMSd and MEMSr) with the five domains (Knowledge, Attitude, Practices, Habits, and Depression) as measured using the respective questionnaires. The only significant correlation was observed between adherence with practices in respect to diabetes time management, suggesting that as time management improves, so does adherence with the antidiabetic medication.

Table 1.

Correlations between compliance with KAP domains¹

MEMS®compliance	Knowledge	Practice-D	Practice-G (habits)	Attitude	HADS
MEMSd2	0.081	0.361	0.153	0.158	−0.172
	P = 0.687	P = 0.082	P = 0.445	P = 0.430	P = 0.391
MEMSr3	−0.042	0.564	0.219	0.004	0.014
	P = 0.836	P = 0.004	P = 0.270	P = 0.981	P = 0.943

Notes:

¹Correlations measured as Pearson correlation coefficients, P < 0.05 indicates statistical significance;

²MEMSd: compliance measured as a factor of daily dosage;

³MEMSr: compliance measured as a factor of treatment regimen.

Pill count

Pill counts were performed when each MEMS^® unit was returned. The range of pills returned was 0–78, with a mean of 6.74 (SD = 16.19, SE = 3.12, Median = 0). Adherence was estimated using the pill count method, and it was calculated to be 91.3% (SE = 18.6; Median = 100.0%). There was no significant correlation between the adherence as assessed by pill count and any of the other variables.

Discussion

Poor adherence with drug therapy is a significant, widespread problem among all levels of patients and across disease states, age groups, and other patient group characteristics.²² In a review of studies published in the period from 2002 until 2008, Cramer and colleagues looked at the problem of poor adherence among patients with diabetes and other cardiovascular diseases, and found that the average 12-month adherence rate was 63% and that it was similar across therapeutic classes.²³ They concluded that good adherence had a positive effect on outcome in 73% of the studies examining clinical outcomes. With this and other similar reports in mind, and with consideration to the fact that diabetes, specifically T2DM, is on the rise globally and in Qatar, our intention was to conduct a fact-finding study looking at one of the methods for objectively assessing adherence with T2DM in Qatar. The available methods for estimating adherence all have their limitations, and logistical challenges for conducting such a study were unknown.

The main objective of this pilot study was to examine the usefulness or utility of the MEMS^® in an adherence study and to identify any issues that affect its usage in such context.

Only 37 patients were issued their medication in MEMS-fitted bottles. This was partly because a proportion of patients failed to return MEMS^® in time for re-cycling. We assessed patients’ satisfaction with MEMS^®, a new medicine bottle with which the patient was unfamiliar. Based on a short questionnaire, the majority of patients found it easy to use. The bottles fitted with the MEMS^® differ in terms of size, looks, and the way it has to be opened. Despite these differences, it was clear that patients did not find it difficult to handle. However, the MEMS^® satisfaction questionnaire was administered to a little more than half of the participants.

MEMS^® retrieval has proved to be a challenge and an issue of potential impact in adherence studies. First, the researchers had to limit the amount of information related to the MEMS^® to the minimum to avoid compromising the sensitivity of the measured outcome of adherence through a Hawthorne effect.²⁴ As a result, a portion of patients must have dealt with the MEMS-fitted containers as they would with any other medicine bottle, and either delayed returning them or lost them. As a result, 22% of the MEMS^® units were not returned, and 35% were returned late, some after several reminders. The problem was compounded by the fact that, at over US$100.00 per 1 unit, MEMS^® is a costly commodity to lose in bulk. To minimize the potential for the occurrence of this problem one has to reexamine how MEMS^® was issued and the quality of communication the pharmacist and/or the research assistant have had with the patient. As part of the methodology, four pieces of advice were given to the patient during the dispensing process: that the patient uses the issued medication bottle as the only source for their metformin; that the patient returns the medication bottle to the outpatient pharmacy in one month’s time at the date for the refill; that the patient opens the bottle only to take the prescribed dose; and that the patient returns the bottle (with MEMS^® cap on) with all remaining pills (if any). Apparently some of these instructions were either not fully comprehended or were not perceived as important by some of the participants. Additionally, as we have seen in the rest of the results, there was evidence that patients opened the MEMS^® repeatedly at intervals too close to each other to be counted as an event to take a dose (number of openings ranged from 9 to 155 per patient from the date of dispensing until the date when MEMS^® was returned). This, as we shall discuss further below, must have had an impact on the reliability of the MEMS^® in the circumstances of measuring adherence. To minimize potential for loss or delay of returning of the MEMS^® units, a focused dialog with the patient (at the time of dispensing) where the patient clearly commits to returning the MEMS^® on time and using it properly needs to be conducted by appropriately trained individuals. This must be combined with scheduled reminder phone calls (prior to the refill date and preferably at weekly intervals).

The use of MEMS^® presented us with several challenges in relation to interpretation of results and devising methods for optimum usage in clinical trial environments. We have already alluded to the problem of retrieval of the costly MEMS^® units and provided suggestions on how to minimize potential loss of the units and their stored data. The second group of challenges was related to other issues, including quality and interpretation of data.

MEMS^® estimated adherence was computed using two approaches. The first was through assessing adherence with the doses per day (MEMSd). This method considered whether the patient took the daily doses as prescribed (ie, once, twice, or three times a day). A patient might be prescribed metformin twice a day, but s/he took it only once a day. The estimated adherence MEMSd in this case would be 50% (ie, ½*100%). The result of adherence with dosage (MEMSd) had been found to be 67.7%. This is around 20% less than the 80% cut-off point for an acceptable adherence. The figure of 80% is commonly used and clinically relevant cutoff point which had been considered to have reasonable balance between sensitivity and specificity in studies of adherence in patients with cardiovascular diseases.²⁵

However, adherence with dosage regimen (ie, adherence in respect of the days where the regimen was 100% followed, or MEMSr) gave adherence levels of only 13.7%. The interpretation of this poor result was most probably due to the ‘zero tolerance’ approach this assessment method follows. To clarify, consider the patient above who was prescribed a dose to be taken twice a day but took it only once a day. His or her estimated MEMSd was 50%, but his or her MEMSr in this case will be zero. Because s/he did not completely comply with the daily regimen of two per day, s/he basically had no score. We have noticed repeated occurrences of this behavior, supported by the statistics, where some patients started taking regular doses (documented by MEMS^® opening events) that were not in line with the prescribed dose (ie, patients prescribed one tablet three times a day but they took one tablet twice a day every day). This had a big impact on the adherence as a factor of the regimen.

There are at least two possibilities for this poor MEMSr phenomenon. The first is a practice of intentional nonadherence. This had been discussed in previous publications, and Johnson referred to it as the intentional decision to miss medications.²⁶ This is where patients decide to take a drug at intervals or at dosages that do not correspond with prescribing instructions. The patient might be exercising a sort of personal autonomy or self-rule, reflected in this sort of medication-taking behavior which in turn might be due to previous experience with a specific medication (adverse effects, effectiveness at a specific dose, and so on). The second possibility is what could be termed intentional prescribing error, where some physicians might be verbally telling the patient to take a specific dose at a specific frequency (eg, one tablet once per day), but prescribing another dosage frequency (one tablet three times per day). The reason for this is usually to get the pharmacy to issue more stock of medication to the patient so s/he does not have to come frequently for refills. If this is the case, we would be registering in our study the prescribed dose as the benchmark to estimate adherence, but the patient would open the MEMS^® at a rate that is less than the registered dose (based on his agreement with the prescribing physician), and the net result would be poor adherence (both with dose and regimen). However, we have no solid evidence for this apart from clinical experience, in addition to oral and informal communications with pharmacists and patients outside the frame of this project.

Another observation from the MEMS^® data was the number of times the MEMS^® was opened by the patient (range: 9–155 times per patient) and the number of days when the MEMS^® was used by the patients (range: 2 to 42 per patient). The maximum expected total number of opening per patient in a month should have been 90 times (calculated as one tablet three times a day for one month). The large number of openings that occurred meant that patients opened the container more that they should have. We cannot know when a dose was taken and when it was not in these events. Also the range of the number of days when MEMS^® was used provides an interesting figure of 2–42. While the 42 days simply meant a delay in returning the unit (and probably also an indication of poor adherence), the 2 day figure meant either the medication was used scarcely or it was used, but from another source. Both possibilities indicate violation to our agreement with patients at the beginning of the study.

Because of these irregularities in the use of MEMS^®, it was not surprising that correlation of adherence with other patient-related variables were inconsistent with intuition or logic. Based on the statistical operations used, no association of significance was observed with adherence as measured by the MEMS^® and age, years since diagnosis (YSD), gender, or education status. When correlated with KAP variables, adherence only showed an association with practices (time management and habits). This last set of correlations made some sense, as one would expect those patients who are organized and adhere to certain daily routines would also exhibit better adherence with their medications.

Adherence as measured from pill count data was over 90%. This basically meant that most of the patients returned no or very few tablets at the time of their refill. However, this outcome cannot be taken at face value. First, correlations with adherence as measured by pill count and as measured by MEMS^® were insignificant. One would expect that these two methods correlate strongly since the doses taken from the MEMS-fitted containers would automatically be counted as an event supporting adherence, and will be missing when tablets are counted, again supporting adherence. In the absence of positive correlation between the two methods, the fact that the majority of patients returned empty bottles could not be taken as strong evidence of adherence. The possible reason for the empty containers could be due to patients emptying their medications in other containers and bringing the MEMS-fitted bottles back, not thinking this would matter. This is another demonstration of the importance of a strong emphasis on the correct methods participants in this study should follow.

The data generated by MEMS^® and the observations related to it in this pilot study presented us with more question than answers. We could not, for example, attribute MEMS^® cap opening events to doses taken with any confidence, and the number of tablets taken at each event remains an assumption. However, the findings and observations will at least help in recommending a number of pre-requisites that can maximize the expected benefits of the MEMS^® if it is to be used to measure adherence in future clinical trials. An important direct outcome of this study would be the cost-savings that it will make when we use the experience we gained into a future adherence study. We could use the information and lessons learnt in this project to minimize potential loss of MEMS^®, improve the participant’s adherence to the study protocols, and maximize the quality of information gained. One of the major limitations of this study is the small sample size, thus results may not be generalizable. However, the study is a pilot; as such it achieved its objective by providing data that will help in designing larger studies in which the challenges and lessons identified in the current pilot project would be addressed. We have also not attempted to assess, through correlation statistics, any association that might have existed between the number of medications taken by patients and other patient and regimen-related factors (apart from the KAP domains) with adherence, where our intention in this pilot study had been specifically directed towards understanding how the electronic device used to assess adherence might be applied to the local population rather than assessing adherence as an end point. Therefore, the factors which we were primarily concerned with here, and which are reported in this manuscript, may not have been previously reported in this region.

Conclusion

The utility of MEMS^® in assessing adherence remains elusive. Results gained in this study provide strong evidence that for the MEMS^® to generate valid, reliable, and useful data in assessing adherence, it has to be supported by a strong, focused, and structured method for patient orientation and follow-up. This is especially true because of the inability of the study team to explain the real objective of the study to the patient (ie, to assess adherence). This limitation presents real threats to any similar study, not least the retrieval of the MEMS^®, registering extra opening of MEMS^®, desire to hoard medicine by taking doses at different frequency than recorded in MEMS^®. The overall outcome of this pilot study was considered to be achieved, and the application of improved methodology in future research is now possible largely because of the insights and lessons this project has provided us with.