Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2010 Sep 1.
Published in final edited form as: AIDS Behav. 2008 Dec 16;14(4):748–754. doi: 10.1007/s10461-008-9502-1

LACK OF ASSOCIATION BETWEEN RETROSPECTIVELY COLLECTED PHARMACY REFILL DATA AND ELECTRONIC DRUG MONITORING OF ANTIRETROVIRAL ADHERENCE

Trisha Acri 1, Thomas R TenHave 2, Jennifer C Chapman 2,3, Hillary R Bogner 2,4, Robert Gross 2,3
PMCID: PMC2891597  NIHMSID: NIHMS162188  PMID: 19089608

Abstract

Antiretroviral medication refill adherence has not been compared directly to electronic drug monitoring (EDM) in any identifiable published study. We retrospectively studied adults with undetectable HIV titers on highly active antiretroviral therapy. We used Pearson correlation coefficients and receiver operating characteristic curves to relate the two adherence measures, and we used the Wilcoxon rank-sum test to assess the relation between adherence and viral load. In sixty-five subjects, the majority of whom were African American and male with median age of 44 years, pharmacy refill adherence was difficult to collect retrospectively, was not significantly correlated with EDM adherence, and was not significantly related to viral load. Ninety-day supply pharmacy refill adherence correctly classified 95% EDM adherence maximally at 94 days between refills, and the measure was most sensitive for non-adherence at <90 days. Reassessment of the relation between pharmacy refill data and EDM would be warranted when pharmacy refill data is collected as soon as feasible from sources with complete data capture.

Keywords: Adherence, HIV-1, Antiretroviral therapy, pharmacy, measurement

INTRODUCTION

The impact of HIV infection and AIDS on morbidity and mortality has greatly declined since the introduction of highly active antiretroviral therapy (HAART) (Centers for Disease Control and Prevention 2006). The success of these regimens on an individual patient’s health status depends upon relatively high levels of adherence to therapy (Bangsberg et al. 2000; Bangsberg et al. 2001; Garcia de Olalla et al. 2002; Gross et al. 2001; Sethi et al. 2003; Wood et al. 2003). Lower rates of adherence are associated with increasing HIV viral load (Gross et al. 2001), treatment resistance (Bangsberg et al. 2000; Sethi et al. 2003), progression of disease (Bangsberg et al. 2001), and death (Garcia de Olalla et al. 2002; Wood et al. 2003).

An efficient, easy method of measuring adherence is required to improve monitoring of adherence in clinical practice. An analysis by Gross et al. showed that non-adherence occurs up to 3 months prior to breakthrough of viral load (Gross et al. 2008). Clinicians are unable to accurately assess whether individual patients are or are not adherent (Bangsberg et al. 2001; Gross et al. 2002; Miller et al. 2002), and patients’ self reports to their physicians are also likely to be in error (Berg and Arnsten 2006; Simoni et al. 2006). Therefore, accurate, sensitive measures of adherence would give clinicians objective data to use to discuss individual non-adherence risk factors without causing patients shame or stigma.

Current objective methods of measuring adherence include electronic drug monitoring of pill-taking behavior, biologic measures, and pill counts. None of these is considered a gold standard, and the appropriate method for measurement depends upon the intended purpose (Chesney 2006).

One of the most valid methods of measuring adherence that is used frequently in clinical trials and smaller observational studies is electronic drug monitoring (EDM). This method utilizes a medicine bottle cap with an enclosed computer chip to record each time the bottle is opened (Berg and Arnsten 2006; Chesney 2006; Gross et al. 2002; Liu et al. 2001; Miller and Hays 2000). The EDM pill caps are able to monitor adherence continuously. Some consider this method to be the gold standard of adherence measurement for assessing the validity of other measures (Berg and Arnsten 2006; Chesney 2006).

Unannounced pill counts are another objective measure that can be used in the clinical setting (Bangsberg et al. 2000; Bangsberg et al. 2001); however, they may be less feasible. Pill counts can also be conducted over the telephone (Kalichman et al. 2007), but these may be time and resource intensive.

Pharmacy refill adherence has been validated as a measure of antiretroviral adherence related to viral load (Gross et al. 2005; Grossberg et al. 2004). This method has been used to measure adherence in other chronic diseases including arthritis (Deyo et al. 1981) and hypertension (Choo et al. 1999; McNabb et al. 2003; Rudd et al. 1990; Steiner et al. 1991; Steiner and Prochazka 1997). Pharmacy refill adherence has not been compared directly to other valid, objective adherence measures, such as EDM, over the same time period of antiretroviral adherence in any published study that we could locate. We hypothesized that percent adherence derived from the number of days between pharmacy fills would be related to EDM adherence with the rationale that the pharmacy method might supplant EDM in clinical settings.

METHODS

Participants

The study was conducted as a retrospective cohort study utilizing data previously collected for a prospective cohort study. For the present study, all subjects who participated in the original study were eligible for inclusion. The original study was designed to determine how much adherence is required for maintenance of HIV suppression (Gross et al. 2008) and also to determine the factors that predict non-adherence to antiretroviral therapy (Holmes et al. 2007). Eligible study subjects in the original study were adults on a stable highly active antiretroviral regimen including 600 milligrams of the non-nucleoside analog reverse transcriptase inhibitor efavirenz once daily. Subjects were required to have HIV viral titers below the level of quantification with ultrasensitive viral assays (i.e. <75 viral copies/ml with the Chiron bDNA 3.0 or the Roche ultrasensitive assay) at initial enrollment into the study. In the parent study, patients were recruited and followed between 2000 and 2004, and efavirenz adherence was assessed with EDM over 12 months.

In the original study, patients were recruited prospectively from several HIV clinics in Philadelphia, Pennsylvania: Hospital of the University of Pennsylvania, the Philadelphia Veterans’ Administration Medical Center, Presbyterian Hospital, the Jonathon Lax Treatment Center at Philadelphia Fight, Temple University Hospital, and Hahnemann Hospital. All patients were then evaluated at the General Clinical Research Center of the University of Pennsylvania Medical Center.

For the present study, the dates on which refills of medication were obtained and the number of tablets given with each refill were collected retrospectively from the pharmacies of each patient in the parent study. More specifically, we repeatedly attempted to contact each subject from the parent study by telephone in order to obtain information pertaining to the pharmacies used during participation in the study. Between 2004 and 2006, we then contacted each local pharmacy to obtain dates of refills of medication obtained during the time period in which the subject used the EDM cap in the parent study. When the local chain pharmacy no longer retained the data, we contacted the company’s central data repository to obtain pharmacy history for the individual subject. All subjects signed informed consent prior to participation in the parent study.

Measures

Adherence from pharmacy refill data

For the primary analysis, the duration of interest was the time during which the patient had a 90-day supply of medication in his possession, as this time period of pharmacy refill adherence has previously been associated with viral suppression (Grossberg et al. 2004). The pharmacy refill measure of adherence assumes that previously obtained fills are used prior to obtaining a refill and that no other supply of medicine is available (Choo et al. 1999; Deyo et al. 1981; Gross et al. 2005; Grossberg et al. 2004; McNabb et al. 2003; Miller and Hays 2000; Rudd et al. 1990; Steiner et al. 1991; Steiner and Prochazka 1997). For patients who received a 30-day supply of medication with each refill, the time period in which a subject possessed a 90-day supply of medication was marked by 4 refills of medication. Specifically, the 90-day supply time period includes an initial fill of 30 tablets and three refills, since the last refill is used only as a reference date to close the 90-day supply time period. We then defined percentage adherence as number of pills dispensed in the first three fills divided by the number of days between the first and last fills × 100% (Grossberg et al. 2004). As an example, a subject who received refills on January 1, February 2, March 5, and April 11 with a 30-day supply of tablets in each would be 90% adherent with only 90 tablets dispensed in this 100-day time period. Percentage adherence can be greater than 100% by this definition if refills are obtained prior to the expected refill date.

By this method, it was possible over the duration of the one year study to have at most 4 separate 90-day supply time periods of adherence for each subject in the primary analysis. For those patients who received a 90-day supply with each refill, the 90-day supply time period of interest was marked by 2 refills (an initial 90 tablet fill and one refill). If the refill dates were not available because the pharmacy did not have refill data for the subject during the study, we assumed pharmacy refill adherence was zero percent.

Adherence from EDM

Since EDM data are continuous, every day of adherence is captured. Therefore, we used the same dates on which each patient obtained refills as the reference frame for computing the percentage adherence. For those subjects for whom pharmacy refill adherence was zero because refill dates were not available, we used the dates on which viral load draws were obtained every three months as the endpoints for calculating the EDM adherence. Percentage adherence was defined as number of doses taken/number of doses prescribed × 100%. The number of doses taken was calculated by generating a sum of the number of times the pill bottle was opened (a surrogate for the number of doses taken by the patient) in each block of time of interest. The quantity removed from the monitored bottle was assumed to be the correct dose.

For our primary analysis, the adherence time periods were defined by the dates of pharmacy refills over the time in which a subject possessed a 90-day supply of medication, less than 30 days preceding the viral loads drawn at 3, 6, 9, and 12 months of observation in the parent study. If no pharmacy refills were obtained, then the adherence time periods were defined by the dates upon which viral loads were drawn (approximately every 3 months). If no viral loads were obtained, then the 90-day supply time periods were framed by the dates on which pharmacy refills were obtained beginning from the first fill after use of the EDM cap began. If the viral load dates were missing and no pharmacy refills were obtained, then the end of the time period was defined as 90 days prior to next viral load. For example, if the viral load date was missing at the 3 month follow up visit, then the end of the first time period was considered to be 90 days prior to the date on which the viral load was drawn at the 6 month follow up visit. We did not account for instructions to stop medication that a patient received from the physician, nor for periods of time in which the patient had access to another source of medication, for example during hospitalization.

Data Analysis

We calculated the Pearson correlation coefficient between pharmacy refill adherence and EDM adherence for each individual at different time points. Confidence intervals were constructed based on Fisher’s Z transformation, using a design effect adjustment to account for the within-person correlation due to multiple time points of adherence over time for each patient. Because the correlation treats both measures of adherence as outcomes, we calculated the average of the two within-subject correlations for the EDM and pharmacy measures of adherence. The average of the two within-subject correlations was then used in a design effect computation (1+rho*(k-1)) where rho is this average within-subject correlation and k is the average number of observations per subject. This design effect was then used to adjust the Fishers Z transform-based confidence interval for the correlation between the EDM and the pharmacy refill data (Diggle et al. 1994).

We also generated receiver operating characteristic (ROC) curves (Hanley and McNeil 1982) to determine the ability of the pharmacy refill measure to discriminate good and poor adherence over the time in which a subject had a 90 day supply of medication, as defined by 95% adherence measured using EDM. ROC curves plot the sensitivity and specificity of a screening test (pharmacy refill adherence) when it is compared to a reference standard (EDM adherence). Cut points on the curve can be chosen based on the sensitivity or specificity needed depending upon the planned use of the screening test to discriminate between two outcomes, i.e., good and poor adherence by the reference standard. In our study, two different cut points on the ROC curve were chosen: the first was selected to decrease the misclassification of non-adherence or adherence status, and the second was chosen to maximize the sensitivity of the pharmacy refill adherence measure to find non-adherence.

Finally, we used the two-sample Wilcoxon rank-sum test to determine the relationship between breakthrough of viral load to levels greater than 1000 viral copies/ml and each of the two measures of adherence in separate analyses.

RESULTS

A total of 117 subjects from the parent study were eligible for this study. Of these, 34 were lost to follow up and could not be contacted for consent to participate in this study. In addition, there were 10 subjects for whom complete pharmacy data could not be obtained. Finally, 6 subjects were excluded because they had not obtained at least a 90-day supply of medication, and 2 subjects were excluded because the refills did not occur within 30 days of viral load draws during the parent study. Therefore, 65 subjects were included in our primary analysis.

Demographic data on the enrolled subjects were as follows: forty-nine (75%) subjects were male; median age was 44 years (range 28–70). Forty-one (63%) subjects were African American, seventeen (26%) were White, three (5%) were Hispanic, and four (6%) were of other race. Nine (14%) subjects used injection drugs, twenty-eight (43%) were men who have sex with men, and six (9%) had a blood transfusion prior to 1985. Fifty-two (80%) of the subjects had at least a high school education. Forty-one (63%) had an income of less than $15,000. The median CD4 cell count of the sample participants was 438 (interquartile range (IQR) 257–561 cells/ml) and 54 (83%) of the subjects had been diagnosed with AIDS. Thirty-seven (57%) of the subjects were antiretroviral naïve prior to the currently studied regimen. The demographic profile of the subjects who were excluded from the analysis due to missing data or loss to follow up was not significantly different from the enrolled subjects except that the excluded subjects were more likely to have a history of injection drug use.

In the primary 90-day supply analysis, there were 168 time points of data where estimates of both EDM and pharmacy adherence were available from 65 participants. Of these, 90/168 (54%) had 95% or greater adherence by EDM. In addition, 64/168 (38%) had 95% or greater adherence by the pharmacy refill estimate of adherence, and 68/168 (40%) had 0% adherence by pharmacy refill adherence.

The Pearson correlation coefficient between pharmacy refill adherence and EDM adherence was 0.022 (95% confidence interval −0.19 to 0.23). Because EDM adherence and pharmacy refill adherence were repeatedly measured for each subject over one year’s time, we adjusted for the resulting clustering on both the EDM and pharmacy refill adherence measures by computing the within-subject correlations for each measure: EDM adherence (0.50) and pharmacy refill adherence (0.00). The average of these within-subject correlations (0.25) was used to adjust the 95% confidence interval around the point estimate.

After excluding the 68 time points where the pharmacy refill estimate was zero, if we assume that all episodes of missing pharmacy data were due to loss of data by error at the pharmacy rather than non-adherence, the correlation was 0.049 (95% confidence interval of −0.20 to 0.30). This confidence interval accounts for an average within-subject correlation of 0.93 given the within-subject correlations of 0.86 and 0.99 for EDM and pharmacy refill adherence, respectively. A visual plot demonstrating the lack of direct relationship between EDM and pharmacy adherence is shown in Figure 1. A secondary analysis ignoring timing of refills in relation to viral load including 67 subjects and 192 time points of adherence data showed similar results.

Figure 1.

Figure 1

Open in a new tab

Relationship between pharmacy refill adherence and EDM adherence

Figure 2 shows the ROC curve for the sensitivity and specificity of the relationship between 95% EDM adherence and the time to obtaining a refill when a subject had a 90-day supply of medication in their possession. Maximum correct classification of EDM adherence status (67.5% of subjects correctly classified as adherent or non-adherent) occurred when the number of days between refills for a 90-day supply was 94 days (area under the curve 0.688, 95% CI 0.591–0.786). Maximum sensitivity for non-adherence occurred with less than 90 days between refills. Similar results were seen in an analysis using a 30-day supply time period where the highest sensitivity for non-adherence was seen with a cut point for obtaining a refill less than 30 days after the original fill. Maximum correct classification (63.9%) was seen if the refill was not obtained by 32 days (data not shown, area under ROC curve 0.626, 95% CI 0.542–0.710).

Figure 2.

Figure 2

Open in a new tab

Receiver operating characteristic curve for the sensitivity and specificity of the relationship between the number of days between refills of medication and 95% EDM adherence

In the viral load analyses using the two-sample Wilcoxon rank-sum test, there were 3 subjects with breakthrough of viral load to greater than 1000 viral copies/ml. The median EDM adherence was 25.8% (IQR 24.2–77.2%) with viral load breakthrough and 95.5% (IQR 84.9–100.0%) without breakthrough, and EDM adherence was significantly related to viral load (z=2.476, p<0.05). The median pharmacy refill adherence was 0% (IQR 0–73.2%) with viral load breakthrough and 89.1% (IQR 0–98.9%) without breakthrough, but this difference was not statistically significant (z=1.437, p>0.1).

DISCUSSION

In our analysis, pharmacy refill adherence correlated poorly with EDM despite the two measures’ previously reported significant correlations with viral load (Gross et al. 2001; Gross et al. 2008; Grossberg et al. 2004). Our results suggest that retrospective collection of pharmacy refill data from commercial pharmacies does not result in a measure of adherence which correlates with EDM adherence. Pharmacy refill adherence also was not significantly related to viral load, perhaps in part because only 3 subjects with complete data had breakthrough of viral load.

Our results show that a higher percentage of subjects had 95% or greater adherence by EDM than by the pharmacy refill estimate of adherence. This is likely to be primarily due to the number of subjects for whom pharmacy refill data were not available from the pharmacies they reported using during the study. Our analysis assumes that all patients for whom pharmacy data were unavailable were non-adherent. In fact, there are multiple other possible explanations for missing pharmacy data. It is possible that data were lost by the pharmacy, or that patients used a pharmacy different from those they recalled, obtained medication from alternate sources, kept stockpiles of medication at home, or received medication from another person.

This analysis demonstrated that obtaining pharmacy refill data retrospectively was technically difficult. Thirty-four subjects from the parent study could not be relocated and were lost to follow up. In addition, we were unable to obtain complete pharmacy data on 10 subjects due to poor subject recall of pharmacies used during the study, subjects’ use of multiple pharmacies, loss of pharmacy records with closure of pharmacy or change of ownership, and the need for pharmacy-specific consent forms.

Comparison of our experience to prior work by the group (Graham et al. 2007) suggests that collection of pharmacy data was easier when data were collected closer in time to patients’ receipt of refills from the pharmacy. In the group’s previous study, obtaining pharmacy data was much easier with direct clinician to pharmacist contact (Graham et al. 2007). During the current retrospective study, local pharmacists suggested collection of data within one year of the date of refill because data are more likely to be sent to central repositories after this time.

We used ROC curves to compare the pharmacy refill adherence measure to suboptimal adherence (95% or less) as defined by EDM. Because we did not know whether or not subjects had an extra supply of medication at home, they may have returned to the pharmacy “late for a refill” without missing any doses. Therefore, it was important to test various cut points in the number of days late for a refill to define non-adherence. The results of the ROC analysis help to determine the ability of the pharmacy refill measure to correctly classify individuals as adherent or non-adherent as defined by the EDM measure.

In our first ROC analysis, our goal was to maximize correct classification by defining the number of days between refills at which the greatest percentage of subjects were correctly defined as adherent or non-adherent with 95% EDM adherence as the reference standard. The results suggest that a patient who is 4 days late to obtain a refill (taking 94 days to get back to the pharmacy when he has a 90-day supply of medication in his possession) can be defined as non-adherent in reference to 95% EDM adherence. In our second ROC analysis, we chose a cut point to increase sensitivity for non-adherence in order to decrease missed non-adherence. The second ROC result suggests that a patient who returns to the pharmacy even 1 day late may be non-adherent.

Adherence was measured repeatedly for each subject over one year, and we found a moderate to high within-subject correlation of 0.25–0.93. These results suggest that an individual’s past adherence is predictive of future adherence, though adherence is a dynamic phenomenon (Gross et al. 2006) warranting ongoing real time monitoring.

Our results are limited by the small sample size of subjects with complete data available for analysis due to both subject loss to follow up and lack of pharmacy data. We had difficulty locating some patients and obtaining pharmacy refill data retrospectively from some commercial pharmacies. It is possible that a relationship between pharmacy refill and EDM adherence or viral load would be evident with a larger number of subjects or more complete data availability.

In addition, very few subjects for whom complete data were available had breakthrough of viral load. Subjects who were lost to follow up may have been less adherent than those who were included in our analysis. If more subjects with poor adherence or viral load breakthrough were included in the analysis, we might have found a relationship between pharmacy refill adherence and viral load.

Other factors may have been responsible for the lack of pharmacy data available for retrieval. For example, patients may have obtained medication from a friend or from a pharmacy they did not recall using, or they may have stockpiled doses by obtaining refills sooner than necessary and using the extra doses in the next time period. As our goal was to assess the validity of pharmacy refill adherence absent other information, we did not ask patients about these behaviors. Future studies should attempt to better elucidate some of the behaviors that may mask adherence in individual patients.

Pharmacy refill adherence has previously been found to be related to viral load outcomes. This analysis using retrospectively-collected pharmacy data did not find a relationship to either EDM adherence or viral load breakthrough. Our experience with use of this measure suggests that data should be collected as close as possible to the time each subject refills his prescription. Future work using pharmacy refill data should ensure that the data are collected as soon as feasible and, if possible, from sources with longstanding history of complete or near complete data capture (e.g., the Veterans’ Administration). Reassessment of the relation between pharmacy refill data and EDM would then be warranted.

ACKNOWLEDGEMENTS

This research was supported by an NIH Health Research Services Administration grant (TA). The parent study was supported by funding from Bristol-Myers Squibb and K08 MH001584 (RG).

REFERENCES

  1. Bangsberg DR, Hecht FM, Charlebois ED, Zolopa AR, Holodniy M, Sheiner L, Bamberger JD, Chesney MA, Moss A. Adherence to protease inhibitors, HIV-1 viral load, and development of drug resistance in an indigent population. AIDS. 2000;14:357–366. doi: 10.1097/00002030-200003100-00008. [DOI] [PubMed] [Google Scholar]
  2. Bangsberg DR, Perry S, Charlebois ED, Clark RA, Roberston M, Zolopa AR, Moss A. Non-adherence to highly active antiretroviral therapy predicts progression to AIDS. AIDS. 2001;11:1181–1183. doi: 10.1097/00002030-200106150-00015. [DOI] [PubMed] [Google Scholar]
  3. Bangsberg D, Hecht F, Clague H, Charlebois ED, Ciccarone D, Chesney M, Moss A. Provider assessment of adherence to HIV antiretroviral therapy. Journal of Acquired Immune Deficiency Syndromes. 2001;26:435–442. doi: 10.1097/00126334-200104150-00005. [DOI] [PubMed] [Google Scholar]
  4. Berg KM, Arnsten JH. Practical and conceptual challenges in measuring antiretroviral adherence. Journal of Acquired Immune Deficiency Syndromes. 2006;43:S79–S87. doi: 10.1097/01.qai.0000248337.97814.66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Centers for Disease Control and Prevention. Epidemiology of HIV/AIDS—United States, 1981–2005. MMWR: Morbidity and Mortality Weekly Report. 2006;55:589–592. [PubMed] [Google Scholar]
  6. Chesney MA. The elusive gold standard. Future perspectives for HIV adherence assessment and intervention. Journal of Acquired Immune Deficiency Syndromes. 2006;43:S149–S155. doi: 10.1097/01.qai.0000243112.91293.26. [DOI] [PubMed] [Google Scholar]
  7. Choo PW, Rand CS, Inui TS, Lee ML, Cain E, Cordeiro-Breault M, Canning C, Platt R. Validation of patient reports, automated pharmacy records, and pill counts with electronic monitoring of adherence to antihypertensive therapy. Medical Care. 1999;37:846–857. doi: 10.1097/00005650-199909000-00002. [DOI] [PubMed] [Google Scholar]
  8. Deyo RA, Inui TS, Sullivan B. Noncompliance with arthritis drugs: magnitude, correlates, and clinical implications. Journal of Rheumatology. 1981;8:931–936. [PubMed] [Google Scholar]
  9. Diggle PJ, Liang K-Y, Zeger SL. Analysis of longitudinal data. New York: Oxford University Press; 1994. [Google Scholar]
  10. Garcia de Olalla P, Knobel H, Carmona A, Guelar A, Lopez-Colomes JL, Cayla JA. Impact of adherence and highly active antiretroviral therapy on survival in HIV-infected patients. Journal of Acquired Immune Deficiency Syndromes. 2002;30:105–110. doi: 10.1097/00042560-200205010-00014. [DOI] [PubMed] [Google Scholar]
  11. Graham J, Bennett IM, Holmes WC, Gross R. Medication beliefs as mediators of the health literacy-antiretroviral adherence relationship in HIV-infected individuals. AIDS & Behavior. 2007;11:385–392. doi: 10.1007/s10461-006-9164-9. [DOI] [PubMed] [Google Scholar]
  12. Gross R, Bilker WB, Friedman HM, Strom BL. Effect of adherence to newly initiated antiretroviral therapy on plasma viral load. AIDS. 2001;15:2109–2117. doi: 10.1097/00002030-200111090-00006. [DOI] [PubMed] [Google Scholar]
  13. Gross R, Bilker WB, Friedman HM, Coyne JC, Strom BL. Provider inaccuracy in assessing adherence and outcomes with newly initiated antiretroviral therapy. AIDS. 2002;16:1835–1837. doi: 10.1097/00002030-200209060-00021. [DOI] [PubMed] [Google Scholar]
  14. Gross R, Zhang Y, Grossberg R. Medication refill logistics and refill adherence in HIV. Pharmacoepidemiology & Drug Safety. 2005;14:789–793. doi: 10.1002/pds.1109. [DOI] [PubMed] [Google Scholar]
  15. Gross R, Yip B, Re VL, Wood E, Alexander CS, Harrigan PR, Bangsberg DR, Montaner JSG, Hogg RS. A simple, dynamic measure of antiretroviral therapy predicts failure to maintain HIV-1 suppression. Journal of Infectious Diseases. 2006;194:1108–1114. doi: 10.1086/507680. [DOI] [PubMed] [Google Scholar]
  16. Gross R, Bilker WB, Wang H, Chapman J. How long is the window of opportunity between adherence failure and virologic failure on efavirenz-based HAART? HIV Clinical Trials. 2008;9:202–206. doi: 10.1310/hct0903-202. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Grossberg R, Zhang Y, Gross R. A time-to-prescription-refill measure of antiretroviral adherence predicted changes in viral load in HIV. Journal of Clinical Epidemiology. 2004;57:1107–1110. doi: 10.1016/j.jclinepi.2004.04.002. [DOI] [PubMed] [Google Scholar]
  18. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36. doi: 10.1148/radiology.143.1.7063747. [DOI] [PubMed] [Google Scholar]
  19. Holmes WC, Bilker WB, Wang H, Chapman J, Gross R. HIV/AIDS-specific quality of life and adherence to antiretroviral therapy over time. Journal of Acquired Immune Deficiency Syndromes. 2007;46:323–327. doi: 10.1097/QAI.0b013e31815724fe. [DOI] [PubMed] [Google Scholar]
  20. Kalichman SC, Amaral CM, Stearns H, White D, Flanagan J, Pope H, Cherry C, Cain D, Eaton L, Kalichman MO. Adherence to antiretroviral therapy assessed by unannounced pill counts conducted by telephone. Journal of General Internal Medicine. 2007;22:1003–1006. doi: 10.1007/s11606-007-0171-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Liu H, Golin CE, Miller LG, Hays RD, Beck CK, Sanandaji S, Christian J, Maldonado T, Duran D, Kaplan AH, Wenger NS. A comparison study of multiple measures of adherence to HIV protease inhibitors. Annals of Internal Medicine. 2001;134:968–977. doi: 10.7326/0003-4819-134-10-200105150-00011. [DOI] [PubMed] [Google Scholar]
  22. McNabb JJ, Nicolau DP, Stoner JA, Ross J. Patterns of adherence to antiretroviral medications: the value of electronic monitoring. AIDS. 2003;17:1763–1767. doi: 10.1097/00002030-200308150-00005. [DOI] [PubMed] [Google Scholar]
  23. Miller LG, Liu H, Hays RD, Golin CE, Beck CK, Asch SM, Ma Y, Kaplan AH, Wenger NS. How well do clinicians estimate patients’ adherence to combination antiretroviral therapy? Journal of General Internal Medicine. 2002;17:1–11. doi: 10.1046/j.1525-1497.2002.09004.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Miller L, Hays R. Measuring Adherence to Antiretroviral Medications in Clinical Trials. HIV Clinical Trials. 2000;1:36–46. doi: 10.1310/enxw-95pb-5ngw-1f40. [DOI] [PubMed] [Google Scholar]
  25. Nieuwkerk PT, Oort FJ. Self-reported adherence to antiretroviral therapy for HIV-1 infection and virologic treatment response: a meta-analysis. Journal of Acquired Immune Deficiency Syndromes. 2005;38:445–448. doi: 10.1097/01.qai.0000147522.34369.12. [DOI] [PubMed] [Google Scholar]
  26. Rudd P, Ahmed S, Zachary V, Barton C, Bonduelle D. Improved compliance measures: applications in an ambulatory hypertensive drug trial. Clinical Pharmacology & Therapeutics. 1990;48:676–685. doi: 10.1038/clpt.1990.211. [DOI] [PubMed] [Google Scholar]
  27. Sethi AK, Celentano DD, Gange SJ, Moore RD, Gallant JE. Association between adherence to antiretroviral therapy and human immunodeficiency virus drug resistance. Clinical Infectious Diseases. 2003;37:1112–1118. doi: 10.1086/378301. [DOI] [PubMed] [Google Scholar]
  28. Simoni JM, Kurth AE, Pearson CR, Pantalone DW, Merrill JO, Frick PA. Self-report measures of antiretroviral therapy adherence: A review with recommendations for HIV research and clinical management. AIDS & Behavior. 2006;10:227–245. doi: 10.1007/s10461-006-9078-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Steiner JF, Fihn SD, Blair B, Inui TS. Appropriate reductions in compliance among well-controlled hypertensive patients. Journal of Clinical Epidemiology. 1991;44:1361–1371. doi: 10.1016/0895-4356(91)90097-s. [DOI] [PubMed] [Google Scholar]
  30. Steiner JF, Prochazka AV. The assessment of refill compliance using pharmacy records: methods, validity, and applications. Journal of Clinical Epidemiology. 1997;50:105–116. doi: 10.1016/s0895-4356(96)00268-5. [DOI] [PubMed] [Google Scholar]
  31. Wood E, Hogg RS, Yip B, Harrigan PR, O'Shaughnessy MV, Montaner JS. Effect of medication adherence on survival of HIV-infected adults who start highly active antiretroviral therapy when the CD4+ cell count is 0.200 to 0.350 × 10(9) cells/L. Annals of Internal Medicine. 2003;139:810–816. doi: 10.7326/0003-4819-139-10-200311180-00008. [DOI] [PubMed] [Google Scholar]

RESOURCES