Skip to main content
PMC home page
Therapeutics and Clinical Risk Management logoLink to Therapeutics and Clinical Risk Management
. 2007 Aug;3(4):685–690.

How should we measure medication adherence in clinical trials and practice?

Jeannie K Lee 1, Karen A Grace 1, Terri G Foster 1, Monica J Crawley 1, Goldina I Erowele 1, Hazel J Sun 1, Phuong T Turner 1, Lance E Sullenberger 1, Allen J Taylor 1
PMCID: PMC2374934  PMID: 18472991

Abstract

Objective

To determine if simple adherence measures, such as twenty-four hour recall and refill history, are accurate for routine use, compared to more time-consuming measures such as pill counts.

Design

Randomized, double-blind, placebo-controlled trial.

Setting

Walter Reed Army Medical Center, a tertiary medical center in Washington.

Patients

Men and women >30 years old with known coronary heart disease and taking a statin medication.

Intervention

Clinical pharmacists met with patients for adherence assessments.

Main outcome measures

Adherence was measured by pill counts, twenty-four hour recall by patient, and refill history per computer record. Temporal changes in these adherence measures were assessed using general linear models for repeated measures.

Results

Adherence was consistently greater for the experimental agent than for the statin therapy (n = 148). Mean pill count adherence for statin drug was 78.7 ± 25.2% compared to 93.5 ± 11.6% (P < 0.001) for the study agent. Refill history and twenty-four hour recall inaccurately measured adherence when compared to pill counts. Adherence, as determined by pill count, for both experimental (P = 0.029) and statin therapy (P = 0.015) showed significant variability across time in general linear models. Neither refill history nor twenty-four hour recall was sensitive to temporal changes.

Conclusions

Twenty-four hour recall and refill history inaccurately measure medication adherence for both clinical trial and clinical practice pharmacotherapies. Further, these measures are insensitive to changes in adherence. For a single or multiple assessments across time, pill count more accurately measures medication adherence. Pill count should be the standard for monitoring medication adherence for both clinical trials and clinical practice.

Keywords: adherence, hyperlipidemia, niacin, pharmacist, pill count, simvastatin

Introduction

Adherence to prescribed medications, both in research trials and in clinical practice, is crucial to the success of the pharmacologic interventions. However, for both settings there is no accepted standard method of assessing adherence, both as a cross sectional measurement and for measurements across time (Gordis 1984; Mattson et al 1984; Norell 1984; Cramer et al 1989; Rudd et al 1989; Avorn et al 1998; Farmer 1999; Liu et al 2001; Benner et al 2002; Garber et al 2004). The latter is crucial to understanding the success of pharmacy adherence programs. Frequently reported methods of measuring medication adherence include patient self-report, the use of electronic databases for prescription claims or refill history, and the use of pill counts. In general, pill counts are regarded as being more accurate than self-report or refill history (Inui et al 1981; Stewart 1987; Botelho et al 1992; Choo et al 1999; Hamilton 2003). However, pill counts are tedious and difficult to administer, thus an understanding is needed of the quantitative disagreement with easier methods such as twenty-four hour recall and refill history.

Objective

We compared pill count, twenty-four hour recall and refill history, and their temporal relationships for both an experimental and clinical drug in a prospective, blinded clinical trial. Our objective was to determine if simple adherence measures, such as twenty-four hour recall and refill history, are sufficiently accurate for routine use, compared to more time-consuming measures such as pill counts.

Methods

ARBITER 2 was a prospective double-blind, randomized clinical trial comparing the efficacy of extended-release prescription niacin (Niaspan®, Kos Pharmaceuticals Inc., Cranbury, NJ) and matching placebo on the rate of atherosclerosis progression. The study was conducted from December 2001 to May 2004. By study design, all patients had known coronary heart disease and were required to already be taking a statin medication prior to study entry and continue taking the statin medication throughout the duration of the study. The trial was conducted at Walter Reed Army Medical Center, a tertiary medical center, and the protocol was approved by the institution’s Department of Clinical Investigation. All volunteers provided written, informed consent.

A dedicated research pharmacy, which manages all medications provided for approved protocols, dispensed the study medication. Statin therapy was dispensed by the main outpatient pharmacy of the facility along with the remainder of the participant’s medications. The majority (93%) of patients in the study used simvastatin, the primary statin provided by the Department of Defense, with the remainder prescribed atorvastatin. All participants met with one of the six clinical pharmacists, per study protocol, for planned adherence assessments of their study drug and statin medication. After an initial meeting at 30 days, adherence assessments were performed at 90, 180, 270, and 365 days for the 148 study patients. Of these, 14 participants (9.5%) forgot to bring their statin bottles to their study visit, (but did partake in study drug pill counts). These subjects are excluded from some analyses where noted. Participants were instructed to bring their medication bottles to each visit. Pill counts were calculated as the number of pills taken (the number of pills dispensed – the number of pills counted). The number of pills expected to have been taken was calculated by multiplying the daily dose (1/2, 1 or 2 tablets) by the number of days since the date dispensed. We a priori and rigorously defined successful adherence on pill counts as 85–100% of the pills taken during each follow-up period (Krueger et al 2003). Adherence per drug use by twenty-four hour recall was assessed based on the patient’s verbal response (from memory or with an aid of a personal medication list) of all of the chronic medications taken (including the study and statin medications) within 24 hours prior to each pharmacy visit. We considered the patients to be adherent according to the twenty-four hour recall if they were able to report taking all of their chronic medications. Adherence per refill history was determined by assessing the electronic pharmacy record system (Composite Health Care System, CHCS) to quantify the number of pills dispensed relative to time. Adherence was noted if all of the patient’s chronic medication refills were consistent with the number of days dispensed by the pharmacy, always within 90-days. All patients received statin and study medications from the military health care system.

The 148 study participants were seen and interviewed in a Pharmacy Outcomes Clinic at Walter Reed Army Medical Center throughout the study duration (up to 1 year). Pharmacists working in the clinic received training in each of the adherence assessment and documentation. Data were collected on a custom data collection form including written reminders on adherence assessment methods and adherence definitions. A data dictionary was collectively reviewed and discussed among the participating pharmacists in order to minimize the potential for inter-rater variability.

At the conclusion of each visit, the remaining study medication was taken and returned to the research pharmacy and a new drug supply was provided. This ensured that the exact start and end date of the pill count assessment was clearly documented. Participants were responsible for refilling their statin medication through the main outpatient pharmacy on schedule.

Mean pill counts for statin and study drug were computed and expressed as the mean ± standard deviation. Means were compared using paired t-tests or t-tests for independent groups, as appropriate. Temporal changes in pill count adherence were assessed using general linear models during a monitoring phase of up to 1 year. All analyses were conducted using SPSS statistical software (v 13.0; Chicago, IL). A P value ≤0.05 was considered statistically significant.

Results

A total of 148 subjects participated in pill counts of the study medication and assessment provided by the pharmacy intervention team. One-hundred thirty four subjects participated in pill counts of the statin medication. The mean age was 67 ± 10 years, 92% were men. All had a history of prior coronary heart disease (by study design) and had a high prevalence of cardiovascular risk factors including hypertension in 74.3% and diabetes mellitus in 27.7%. Most subjects, 86.3%, had a complex pharmacy regimen, defined as taking greater than 5 daily medications and/or 12 or more medication doses per day. 32.9% of the participants were taking multiple daily doses, defined as medications taken 3 or more times per day.

Adherence was consistently greater for the experimental agent (n = 148) than for the statin therapy (n = 134). The mean pill count for statin drug at 90, 180, 270 and 365 days was 78.7 ± 25.2% (n = 133; note the difference of 1 patient’s missing pill count for this measure) compared to 93.5 ± 11.6% (n = 148) (P < 0.001) for the study agent. This differential was consistently observed across the 12 month observation period (Figure 1). Despite this within group stability or persistence, there was a substantial degree of intra-subject variability in pill counts. Adherence for both experimental (P = 0.029) and statin therapy (P = 0.015) showed significant variability across time in general linear models.

Figure 1.

Figure 1

Open in a new tab

Temporal pattern of pill-count adherence across 12 months for both statin (Panel A) and study drug (Panel B).

Twenty-four hour recall and refill history were insensitive to changes in pill counts over time. To assess this, we compared the maximum differential between separate pill counts for subjects who either were or were not always adherent by recall or refill history as defined under Methods. This analysis was limited to subjects in whom at least 3 of the 4 possible pill counts were completed (statin, n = 85; study drug, n = 126) in order to provide a more stable estimate of pill count differential. The proportion of subjects who were always adherent according to recall and refill history was 58.8% and 60.1%, respectively. The maximum differential in pill counts across the various time points (eg, 90 days vs. 180 days) in adherent subjects was similar to that observed in subjects considered nonadherent when categorized either by twenty-four hour recall (Figure 2A) or refill history (Figure 2B) [P = NS for all comparisons]. Among participants who were always adherent by refill history, the maximum differential of pill counts to statin agent was 43.6 ± 28.0% vs 36.8 ± 20.4% in those who were not always refill adherent (P = NS).

Figure 2.

Figure 2

Open in a new tab

Maximum differential in pill-counts expressed as the greatest difference in pill counts between 90, 180, 270 and 365 day assessments in groups of patients categorized as adherent or nonadherent for 24-hour recall (Panel A) and refill history (Panel B).

The mean number of pill counts completed per patient (maximum of 4 possible pill counts occurring at 90, 180, 270, and 365 days) was 3.4 ± 0.9. The relationship for adherence assessments using pill counts compared to either twenty-four hour recall or refill history were different for study drug and statin therapy. Both twenty-four hour recall and refill history underestimated adherence for the study drug, and overestimated adherence for statin agent, compared to the pill count assessment (Table 1).

Table 1.

Mean pill counts vs 24-hour recall and refill history adherence assessments across 12 months

N* Study agent P values N(#) Statin P values
Pill count 24-hour recall Pill count 24-hour recall
Pill count vs 24-hour recall 147 93.5 ± 11.6 85.7 ± 20.2 <0.001 134 78.3 ± 25.6 85.8 ± 21.3 0.009
Pill count Refill history Pill count Refill history
Pill count vs Refill history (Mean ± SD) 148 93.5 ± 11.6 81.9 ± 24.9 <0.001 133 78.7 ± 25.2 82.3 ± 25.2 0.225
Open in a new tab
*

Missing one twenty-four hour recall data.

#

Missing one statin pill count data.

Discussion

Understanding adherence to pharmacotherapies is crucial in clinical practice and research studies to ensure optimal clinical outcomes and valid study results. Research in this field has been limited by the lack of a true gold standard, recognizing that each method has strengths and weaknesses (Evans et al 1983; Mattson et al 1984; Craig 1985; Wright 1993; Grymonpre et al 1998; Choo et al 1999; Farmer 1999; Schwed et al 1999). Methods used in clinical practice are typically simple assessments of medication refill history or a patient-recall assessment (Fletcher et al 1979; Stewart 1987; Christensen et al 1997; Grymonpre et al 1998). These methods are easy to perform; however, they are more crude and their accuracy is limited. Although some suggest that the patient-recall and refill history assessments are accurate enough, especially when performed in combination (Mattson et al 1984; DiMatteo et al 2002; Farley et al 2003), these methods are generally regarded to substantially overestimate medication adherence (Haynes et al 1980; Inui et al 1981; Stewart 1987; Grymonpre et al 1998; Farmer 1999; Shalansky et al 2004). Additionally, the ability of the patient-recall and refill history to detect changes in adherence is unknown. In comparison, pill counts are laborious and rely upon the assumption that medications missing from the pill bottle were taken (Norell 1984; Rudd et al 1988; Cramer et al 1989; Farmer 1999). They also rely upon accurate reporting dates for starting prescriptions but can be more precise when carefully performed as in this study.

This study compared the longitudinal relationships among different methods of measuring medication adherence to both a study drug (placebo or extended-release prescription niacin) and the “control” clinical medication, their statin prescription. Our analysis has shown important differences in adherence between study and clinical medications, differences in the ability of twenty-four hour recall and refill history to detect nonadherence, and insensitivity of twenty-four hour recall and refill history to detect temporal changes in adherence. Pill count assessment not only showed the differences in patient adherence to study agent and clinical therapy, but reported the rate of adherence (from which we can detect nonadherence) and the changes in adherence across time (intra-subject variability). The observed differences in study and clinical medications are of interest. Although this study cannot provide the cause of this observation, it is likely that study participants were influenced by the intervention of the clinical pharmacists providing their study medication and observing their medication taking behavior.

Strengths of our study, performed within a cardiovascular disease population prescribed multiple additional pharmacologic agents, include its overall low attrition rate (Taylor et al 2004), quantitative data, inclusion of multiple drugs and manners of adherence assessments, and its longitudinal design. Even though nonadherence has been shown to be common in clinical practice, patients in this study were generally adherent despite complex medical regimens.

Limitations

The generalizability of our findings is limited by the application in a population of patients consenting to a clinical trial, who may more closely follow health advice and be interested in their health. Because there were six clinical pharmacists assessing the medication adherence for this study, there is a potential for inter-rater variability even though we instituted a standardized training for them. Since the clinical agent, statin, was not provided by the research pharmacy, some forgot to bring the bottles for pill count, hence the 14 subjects excluded from some analysis. Also, the scope of this study was limited by our inability to include other types of adherence measures such as electronic prescription bottles and drug blood levels (direct measures of adherence).

Conclusion

We found that twenty-four hour recall and refill history inaccurately assess medication adherence compared to pill counts for both clinical trial and clinical practice pharmacotherapies. Further, twenty-four hour recall and refill history are insensitive to temporal changes in adherence when assessed longitudinally. From these data, we conclude that pill count is a superior method of medication adherence assessment compared to twenty-four hour recall and refill history in both clinical practice and long term medication studies. Pill counts should serve as the standard for monitoring patient adherence for both experimental and clinical drugs.

Disclosure

The opinions or assertions herein are the private views of the authors and are not to be construed as reflecting the views of the Department of the Army or the Department of Defense. There are no conflicts of interest or financial interest of the authors to disclose.

Funding support

Partial funding for this study was provided by Kos pharmaceuticals Inc., Dranbury NJ in the form of a research grant administered by the Henry M. Jackson Foundation for the Advancement of Military Medicine.

Previous presentations

Abstract and poster presented at the Annual Scientific Sessions of the American Heart Association, Orlando, Florida, November 2003.

References

  1. Avorn J, Monette J, Lacour A, et al. Persistence of use of lipid-lowering medications: a cross-national study. JAMA. 1998;279:1458–62. doi: 10.1001/jama.279.18.1458. [DOI] [PubMed] [Google Scholar]
  2. Benner JS, Glynn RJ, et al. Long-term persistence in use of statin therapy in elderly patients. JAMA. 2002;288:455–61. doi: 10.1001/jama.288.4.455. [DOI] [PubMed] [Google Scholar]
  3. Botelho RJ, Dudrak R. Home assessment of adherence to long-term medication in the elderly. J Fam Pract. 1992;35:61–5. [PubMed] [Google Scholar]
  4. Choo PW, Rand CS, Inui TS, et al. Validation of patient reports, automated pharmacy records, and pill counts with electronic monitoring of adherence to antihypertensive therapy. Med Care. 1999;37:846–57. doi: 10.1097/00005650-199909000-00002. [DOI] [PubMed] [Google Scholar]
  5. Christensen DB, Williams B, Goldberg HI, et al. Assessing compliance to antihypertensive medications using computer-based pharmacy records. Med Care. 1997;35:1164–70. doi: 10.1097/00005650-199711000-00008. [DOI] [PubMed] [Google Scholar]
  6. Craig HM. Accuracy of indirect measures of medication compliance in hypertension. Res Nurs Health. 1985;8:61–6. doi: 10.1002/nur.4770080112. [DOI] [PubMed] [Google Scholar]
  7. Cramer JA, Mattson RH, Prevey ML, et al. How often is medication taken as prescribed? A novel assessment technique. JAMA. 1989;261:3273–7. [PubMed] [Google Scholar]
  8. DiMatteo MR, Giordani PJ, Lepper HS, et al. Patient adherence and medical treatment outcomes: a meta-analysis. Med Care. 2002;40:794–811. doi: 10.1097/00005650-200209000-00009. [DOI] [PubMed] [Google Scholar]
  9. Evans L, Spelman M. The problem of non-compliance with drug therapy. Drugs. 1983;25:63–76. doi: 10.2165/00003495-198325010-00004. [DOI] [PubMed] [Google Scholar]
  10. Farley J, Hines S, Musk A, et al. Assessment of adherence to antiviral therapy in HIV-infected children using the Medication Event Monitoring System, pharmacy refill, provider assessment, caregiver self-report, and appointment keeping. J Acquir Immune Defic Syndr. 2003;33:211–18. doi: 10.1097/00126334-200306010-00016. [DOI] [PubMed] [Google Scholar]
  11. Farmer KC. Methods for measuring and monitoring medication regimen adherence in clinical trials and clinical practice. Clin Ther. 1999;21:1074–90. doi: 10.1016/S0149-2918(99)80026-5. [DOI] [PubMed] [Google Scholar]
  12. Fletcher SW, Pappius EM, Harper SJ. Measurement of medication compliance in a clinical setting. Comparison of three methods in patients prescribed digoxin. Arch Intern Med. 1979;139:635–8. [PubMed] [Google Scholar]
  13. Garber MC, Nau DP, Erickson SR, et al. The concordance of self-report with other measures of medication adherence: a summary of the literature. Med Care. 2004;42:649–52. doi: 10.1097/01.mlr.0000129496.05898.02. [DOI] [PubMed] [Google Scholar]
  14. Gordis L. General concepts for use of markers in clinical trials. Control Clin Trials. 1984;5:481–7. doi: 10.1016/0197-2456(84)90008-4. [DOI] [PubMed] [Google Scholar]
  15. Grymonpre RE, Didur CD, Montgomery PR, et al. Pill count, self-report, and pharmacy claims data to measure medication adherence in the elderly. Ann Pharmacother. 1998;32:749–54. doi: 10.1345/aph.17423. [DOI] [PubMed] [Google Scholar]
  16. Hamilton GA. Measuring adherence in a hypertension clinical trial. Eur J Cardiovasc Nurs. 2003;2:219–28. doi: 10.1016/S1474-5151(03)00058-6. [DOI] [PubMed] [Google Scholar]
  17. Haynes RB, Taylor DW, Sackett DL, et al. Can simple clinical measurements detect patient noncompliance? Hypertension. 1980;2:757–64. doi: 10.1161/01.hyp.2.6.757. [DOI] [PubMed] [Google Scholar]
  18. Inui TS, Carter WB, Pecoraro RE. Screening for noncompliance among patients with hypertension: is self-report the best available measure? Med Care. 1981;19:1061–4. doi: 10.1097/00005650-198110000-00008. [DOI] [PubMed] [Google Scholar]
  19. Krueger KP, Felkey BG, Berger BA. Improving adherence and persistence: a review and assessment of interventions and description of steps toward a national adherence initiative. J Am Pharm Assoc (Wash DC) 2003;43:668–78. doi: 10.1331/154434503322642598. [DOI] [PubMed] [Google Scholar]
  20. Liu H, Golin CE, Miller LG, et al. A comparison study of multiple measures of adherence to HIV protease inhibitors. Ann Intern Med. 2001;134:968–77. doi: 10.7326/0003-4819-134-10-200105150-00011. [DOI] [PubMed] [Google Scholar]
  21. Mattson ME, Friedman LM. Issues in medication adherence assessment in clinical trials of the National Heart, Lung, and Blood Institute. Control Clin Trials. 1984;5:488–96. doi: 10.1016/0197-2456(84)90009-6. [DOI] [PubMed] [Google Scholar]
  22. Norell SE. Methods in assessing drug compliance. Acta Med Scand Suppl. 1984;683:35–40. doi: 10.1111/j.0954-6820.1984.tb08712.x. [DOI] [PubMed] [Google Scholar]
  23. Rudd P, Byyny RL, Zachary V, et al. Pill count measures of compliance in a drug trial: variability and suitability. Am J Hypertens. 1988;1:309–12. doi: 10.1093/ajh/1.3.309. [DOI] [PubMed] [Google Scholar]
  24. Schwed A, Fallab CL, Burnier M, et al. Electronic monitoring of compliance to lipid-lowering therapy in clinical practice. J Clin Pharmacol. 1999;39:402–9. doi: 10.1177/00912709922007976. [DOI] [PubMed] [Google Scholar]
  25. Shalansky SJ, Levy AR, Ignaszewski AP. Self-reported Morisky score for identifying nonadherence with cardiovascular medications. Ann Pharmacother. 2004;38:1363–8. doi: 10.1345/aph.1E071. [DOI] [PubMed] [Google Scholar]
  26. Stewart M. The validity of an interview to assess a patient’s drug taking. Am J Prev Med. 1987;3:95–100. [PubMed] [Google Scholar]
  27. Taylor AJ, Sullenberger LE, Lee HJ, et al. Arterial Biology for the Investigation of the Treatment Effects of Reducing Cholesterol (ARBITER) 2: a double-blind, placebo-controlled study of extended-release niacin on atherosclerosis progression in secondary prevention patients treated with statins. Circulation. 2004;110:3512–17. doi: 10.1161/01.CIR.0000148955.19792.8D. [DOI] [PubMed] [Google Scholar]
  28. Wright EC. Non-compliance – or how many aunts has Matilda? Lancet. 1993 Sep 10;342:909–13. doi: 10.1016/0140-6736(93)91951-h. [DOI] [PubMed] [Google Scholar]

Articles from Therapeutics and Clinical Risk Management are provided here courtesy of Dove Press

RESOURCES