Comparison of pharmacy-based measures of adherence to antiretroviral therapy as predictors of virological failure

INTRODUCTION

Adherence to highly active antiretroviral therapy (HAART) is a key determinant of clinical outcomes in HIV infection, with sustained high levels of adherence the strongest known predictor of viral suppression [1–4]. Standardized, routine, and cost-effective adherence monitoring is thus necessary to identify patients who would benefit from targeted support in order to prevent poor treatment outcomes.

Several methods are commonly employed to assess individual degree of adherence in clinical settings. However, there is currently no consensus on a standard measure for routine use [5]. While direct assessments of adherence, including electronic monitoring (MEMS), are typically more accurate than indirect measures, these methods are also costly and impractical in resource-limited settings with high HIV disease burden [6]. Pharmacy-based adherence measures, such as prescription refill data, are simple and objective methods for assessing compliance, and employ information that is often already routinely collected for medical or pharmacy records [7].

Adherence indicators calculated from pharmacy refill data can take a variety of forms. Commonly used measures assess timing of medication acquisition or enumerate doses available/taken, both with the intent of identifying gaps in treatment [8,9]. In most cases, dichotomous or categorical measures are created from continuous indices, using a variety of cutoffs which may or may not have clinical significance [9,10]. Assessment and reporting of adherence to HAART using pharmacy refill data have not been standardized, making comparison across time, between clinics, and between study populations difficult.

The purpose of this study was to compare pharmacy record-derived indicators of adherence to HAART based on ability to predict viral load suppression in resource-limited settings.

METHODS

Study Population

We analyzed data from the Themba Lethu Clinic (TLC), an observational cohort of adult patients initiating HAART in Johannesburg South Africa [11]. The clinic provides medication and HIV-related medical care free of charge to patients. Included in our analysis were treatment-naïve men and women initiating HAART between April 1, 2004 and July 31, 2011. Individuals were followed until they died, transferred care to another facility, or were lost to follow-up. Patients still in care at the end of follow up (September 30, 2011) were administratively censored. Women starting HAART while pregnant were excluded due to potential fundamental differences in overall health and motivation for treatment initiation [12].

Definitions

Clinic patients attend scheduled pharmacy appointments in order to obtain prescription refills for antiretroviral drugs. An electronic data management system maintains records of what drugs are dispensed, as well as the scheduled and actual dates of pharmacy attendance for each refill. Refill appointments are scheduled based on standard 28 or 56 day cycles. To avoid missed doses due to unforeseen short delays in attending scheduled visits, two and four extra pills are dispensed respectively for each 28- and 56-day refill cycle. Visits scheduled within one week of a standard 28 or 56-day refill cycle were included in this analysis, with the assumption that enough pills were dispensed to allow for complete pill coverage between visits. Visits scheduled outside of this window were excluded.

Based on differences between scheduled and actual dates of each pharmacy visit, eight different adherence measures were derived, each assessing timing of clinic attendance, medication coverage between visits, or a combination of both. Adherence measures based on clinic attendance included (1) a simple binary indicator of presenting on or before the scheduled appointment date vs. attending after the scheduled visit, (2) presenting on or before the median number of days late among all delayed visits vs. attending earlier (3) and presenting more or less than 30 days late. Measures based on pill coverage included (4) a binary indicator of having pills for all days between pharmacy visits vs. not, as well as having medication for (5) 90% or more and (6) 80% or more of days between refills. Finally, a composite categorical indicator combining attendance and pill coverage categorized visits as (8) either on time, late with complete pill coverage, or late without sufficient coverage. Binary indicators created from the number of days late were based upon the two extremes of attending exactly on time, attending late enough to miss an entire month of coverage, and an intermediate value reflective of the study population (median days late). Thresholds applied to the continuous value for percentage of days with pill coverage compared perfect adherent behavior (100%) with other frequently used cut-points for defining adherence (80%, 90%) [13].

Additionally, at a subset of clinic visits self-reported adherence was assessed. Questions about adherence were administered at the discretion of the clinic nurse and participants were selected without a specific algorithm. Specifically, patients were asked to evaluate the number of prescribed doses taken on time in the week prior to the current visit using the following scale: “All” (≥ 90%), “Most” (60–90%), “About half” (30–60%), “A few” (10–30%) or “None” (<10%). For the purposes of this analysis, we dichotomized self-reported response into patients taking 90% or more of doses on time and those taking less than 90%, i.e. “All” vs. other categories.

Virological failure was defined as a hybrid measure of failure to achieve suppression of plasma HIV to ≤400 copies/mL within six months of initiating HAART or a viral load above 400 copies/mL after previously documented viral load suppression ≤400 copies/mL [14]. To increase temporal association and predictive value of the adherence measure, only pharmacy refill visits corresponding to the two months directly prior to each viral load assessment were eligible for inclusion in the analysis. Viral load measurements without corresponding refill data for the two months immediately prior to the assessment were also excluded.

Because all eligible visits for individual patients were included, a single patient could contribute multiple data points and could “fail” multiple times if more than one viral load assessment met the criteria for virological failure.

Statistical Methods

Baseline characteristics of individual patients reported at the time of HAART initiation were described using standard descriptive statistics. We used generalized estimating equations (GEE) with a binomial distribution, logit link function, and independent correlation matrix to measure the association between each of the adherence indicators and virological failure while accounting for within-individual correlation. The odds ratios (ORs) and 95% confidence intervals (CIs) calculated with robust standard errors were examined to identify which measures of adherence most strongly associated with virological failure. We used c- (or concordance) statistics, defined as the area under the under the receiver operating characteristic curve, to identify the measures of adherence best able to classify a case of virological failure or success [15].

Sensitivity and specificity, as well as positive and negative predictive values, were calculated in order to further quantify the ability of each indicator to correctly classify patients as adherent or non-adherent, with virological status serving as the indicator of true adherent behavior. In addition to looking at the association between viral load and adherence for all eligible visits, we performed a stratified analysis by refill schedule (28 vs. 56-day refill cycles).

Sensitivity Analyses

We evaluated the association between several additional adherence thresholds and virological failure in order to validate the results of our primary analysis, including picking up a refill 10, 15, 20 and 25 days late, as well as having estimated 95%, 70%, and 60% pill coverage between refills. In order to examine the association between virological failure and adherence measured over a longer period of time, we also performed a post hoc analysis repeating the above analyses using adherence assessments from four consecutive months of follow-up, rather than two months. Additional post hoc analyses comparing performance of adherence indicators in early and late phases of treatment were performed; specifically, each of the indicators was assessed at the first viral load measurement after completing 6 months of HAART, as well as in cohorts restricted to patients who had achieved viral suppression by 6 months, patients on first- line regimens, and patients on regimens that were second-line or later.

RESULTS

A total of 8,695 adults contributed 29,937eligible visits. The median age at HAART initiation was 37 years and 63% (N=5505) of those starting treatment were women. The mean baseline CD4 count was 103 cells/mm³ (standard deviation: 74 cells/mm³) with 32% having 50 cells/mm³ or fewer (Table I). Of the 29,937 viral load assessments, 14% (N=4,095) indicated virological failure, either due to failure to suppress to 400 copies/mL or less by six months of treatment (N=1,259), or rebounding to over 400 copies/mL after successful suppression (N=2,836).

Table I.

Characteristics of 8,695 HIV-positive patients at time of HAART initiation in Johannesburg, South Africa.

Demographics	All patients a
Age, years	37 (9)
Female	63.3
Unemployed	49.9
Clinical
Body mass indexkg/m2	22.6 (5.3)
WHO stage III or IV	42.0
Prevalent tuberculosis	17.9
Laboratory
CD4 count cells/mm3	103 (74)
CD4 category cells/mm3
≤ 50	32.0
51–100	20.9
101–200	36.3
201–350	10.9
Viral load b log 10copies/ml	5.6 (6.2)
Viral load category c copies/ml
401–10,000	17.4
> 10,000	82.6

^aCategorical variables are expressed as % total; continuous variables are expressed as mean (standard deviation).

^bViral load at baseline was missing in 6761 (78%) patients.

^cThose with viral loads <400 copies/ml at baseline were presumed to not be treatment naïve and were excluded from the analysis

Adherence was high regardless of measure applied, with 84% of eligible visits occurring on or before the scheduled pharmacy visit date, and 88% occurring before pills from the last refill ran out. Among visits occurring late, the median time of actual attendance was five days after the scheduled visit. Most late visits occurred within several days of the scheduled visit, but 15% (N=737) occurred more than 30 days after the scheduled refill. Accounting for extra doses dispensed with each refill, only 20% of those showing up late did not have enough pills to cover at least 80% of days between the two visits.

Independent of pharmacy refill schedule, all measures demonstrated increased probability of virological failure with lower adherence (Table II). Not surprisingly, more extreme classifications of non-adherence showed stronger associations with virological failure, including a gap in treatment of 30 days or more (OR 2.56; 95% CI:2.16, 3.03)and having less than 80% pill coverage in the two months prior to viral load assessment (OR 1.89; 95% CI: 1.62, 2.20). However, simple binary measures of coming on time (OR 1.27, 95% CI: 1.16, 1.38) or having enough pills between visits (OR 1.26; 95% CI: 1.15, 1.39) also showed an association with virological failure.

Table II.

Associations between measures of adherence and virological failure for all eligible pharmacy visits (two 28-day cycles, and a single 56-day cycle), among 8,695 HIV-positive patients in Johannesburg, South Africa.

	All visits (n=29,937)			Visits with self reported adherence data (n=18,082)
	All visits	Adherence based on 2 visits	Adherence based on single visit
	OR (95% CI)	OR (95% CI)	OR (95% CI)	OR (95% CI)
Pharmacy attendance
Picked up prescription refill on or before scheduled date	1.	1.	1.	1.
Picked up prescription refill late	1.27 (1.16, 1.38)	1.17 (1.04, 1.33)	1.22 (1.09, 1.38)	1.42 (1.27, 1.60)
Picked up refill fewer than 5 days after scheduled date	1.	1.	1.	1.
Picked up prescription refill more than 5 days late+	1.38 (1.24, 1.53)	1.22 (1.04, 1.43)	1.46 (1.26, 1.68)	1.32 (1.15, 1.52)
Picked up refill fewer than 30 days after scheduled date	1.	1.	1.	1.
Picked up more than 30 days late	2.56 (2.16, 3.03)	1.85 (1.46, 2.34)	3.16 (2.52, 3.97)	1.70 (1.32, 2.18)
Percentage of Days Covered
Had 100% coverage between pharmacy visits	1.	1.	1.	1.
Had < 100% coverage between pharmacy visits	1.26 (1.15, 1.39)	1.13 (0.98, 1.30)	1.32 (1.16, 1.51)	1.17 (1.03, 1.34)
Had ≥ 90% coverage between pharmacy visits	1.	1.	1.	1.
Had < 90% coverage between pharmacy visits	1.71 (1.50, 1.96)	1.34 (1.10, 1.62)	2.00 (1.66, 2.39)	1.34 (1.08, 1.67)
Had ≥ 80% coverage between pharmacy visits	1.	1.	1.	1.
Had < 80% coverage between pharmacy visits	1.89 (1.62, 2.20)	1.49 (1.20, 1.86)	2.17 (1.77, 2.67)	1.34 (1.11, 1.61)
Combination attendance and pill coverage
Came on time, had enough pills	1.	1.	1.	1.
Came late, had enough pills	1.23 (1.62, 2.20)	1.15 (0.94, 1.42)	1.45 (1.24, 1.68)	0.88 (0.72, 1.07)
Came late, did not have enough pills	1.39 (1.24, 1.55)	1.26 (1.07, 1.48)	1.57 (1.25,1.98)	1.34 (1.16,1.56)
Self-reported adherence
Took ≥ 90% of prescribed doses in the week before assessment				1.
Took < 90% of prescribed visit				1.14 (1.02,1.28)

The c-statistics, with a potential range of 0.5 to 1.0, were low for all of the assessed measures, ranging from 0.506 to 0.521 (data not shown). All of the indicators showed very low sensitivity (Se), particularly those for the most non-adherent behaviors of coming 30 days or more late (Se: 5%) and having less than 80% pill coverage between visits (Se: 6%) (Table III). The binary indicators for being on time (Se: 19%) and having complete pill coverage (Se: 14%) were associated with increased, but still low sensitivity while maintaining moderate specificity (Sp: 84% and Sp: 89%, respectively). Self-reported adherence assessment also performed comparably with these two indicators (Se: 13%; Sp: 88%).

Table III.

Test characteristics for adherence measures identifying patients with virological failure.

Indicator	Sensitivity (95% CI)	Specificity (95% CI)	PPVa (95% CI)	NPVb (95% CI)
Came after scheduled refill date	0.19 (0.18,0.20)	0.84 (0.83, 0.85)	0.18 (0.17, 0.19)	0.86 (0.85, 0.86)
Came 5 or more days late	0.12 (0.11, 0.13)	0.91 (0.91, 0.92)	0.19 (0.18, 0.21)	0.85 (0.85, 0.86)
Came more than 30 days late	0.05 (0.04, 0.06)	0.98 (0.98, 0.98)	0.30 (0.27, 0.34)	0.85 (0.85, 0.86)
Less than 100% pill coverage (pill count)	0.14 (0.13, 0.15)	0.89 (0.88, 0.89)	0.18 (0.17, 0.19)	0.85 (0.84, 0.86)
Less than 90% coverage (pill count)	0.07 (0.06, 0.08)	0.96 (0.95, 0.96)	0.23 (0.20, 0.25)	0.85 (0.85, 0.86)
Less than 80% coverage (pill count)	0.06 (0.05, 0.06)	0.97 (0.97, 0.97)	0.24 (0.22, 0.27)	0.85 (0.85, 0.86)
Fewer than 90% of doses taken (self-report)	0.13 (0.12, 0.14)	0.88 (0.88, 0.89)	0.19 (0.17, 0.21)	0.83 (0.82, 0.84)

^aPositive predictive value

^bNegative predictive value; the NPV and PPV only apply to this population or one with an identical prevalence

When the relationship between adherence and virological failure was examined by refill schedule (28 days vs. 56 days), non-adherence between visits was more strongly associated with virological failure among those attending the clinic every 2 months; that is, if only one visit to the pharmacy was required in the two months prior to the viral load assessment, rather than two (Table II). This held true for all pharmacy-based measures of adherence, although refill schedule seemed to have less of an impact on the indicators for coming on time and having complete pill coverage between visits.

Self-reported adherence assessment was performed at 60% of eligible visits. Those experiencing virological failure were more likely to be asked for self-assessment than those achieving virological suppression (17% vs. 13%, p=0.01). Among those assessed, 85% (n= 15,434) indicated taking >90% of their prescribed doses in the week prior to the appointment. Rate of virological failure was similar in those reporting suboptimal adherence (19%) and those reporting taking 90% of more of the prescribed doses (17%). Self-reported adherence showed a slightly weaker association with virological failure compared to pharmacy-based measures (OR 1.14; 95% CI: 1.02, 1.28). When each of the adherence measures of interest was examined within the subgroup of visits where self-reported adherence was also reported, the associations were comparable to what was seen in the analysis of all visits.

Sensitivity Analyses

Thresholds of 10, 15, 20 and 25 days late, as well as 95%, 70% and 60% pill coverage between refills, applied to the same 2 month adherence measures used in the primary analysis, demonstrated the same general trend of increasing magnitude of the odds ratio for thresholds capturing more extreme non-adherent behavior (Supplemental Table I).

When we evaluated each of our indicators from the main analysis using adherence estimated over 4 months rather than 2 months, the number of eligible visits was reduced by nearly 30% (N=21,640). For nearly all indicators, there was a modest increase in the association between non-adherence and virological failure when adherence was tracked over four months rather than two (Supplemental Table I). Effect estimates were less precise due to the reduced number of eligible pharmacy observations.

Four sensitivity analyses attempted to assess differences in indicator performance in early and later stages of treatment (Supplemental Table II). When the adherence indicators were evaluated using only a single viral load assessment, the first after completing 6 months of HAART, the odds ratios were modestly reduced in magnitude, particularly for the indicators including more severe cases of non-adherence (> 30 days late for refill, <80% pill coverage between refills). Also assessing indicator performance earlier in treatment, an analysis restricted to patients taking first-line regimens indicated a stronger association between non-adherence and virological failure when compared to both the main analysis and to an analysis restricted to patients on second-line or later HAART regimens. When adherence indicators were assessed across the entire treatment period, but only among patients who successfully achieved initial viral load suppression within 6 months of starting HAART, the odds ratios were attenuated compared to the main.

DISCUSSION

Adherence was high in the study cohort, with nearly 90% of pharmacy appointments occurring before medication ran out, consistent with observations in similar settings [16,17]. Each of the adherence indicators derived from pharmacy refill data indicated a significant association between non-adherence and virological failure. This includes the strictest definitions of adherence, picking up a refill on or before the scheduled date and having 100% pill coverage, although the associations were more modest than thresholds allowing for some degree of non-perfect adherence. The magnitude of the associations between non-adherence and virological failure increased as the binary thresholds for non-adherence decreased, with attending a pharmacy appointment more than 30 days late and having less than 80% pill coverage between pharmacy visits producing the strongest associations.

Additional thresholds for adherence indicators were evaluated as a post hoc sensitivity analysis (attending pharmacy refill appointment 10,15,20 and 25 days late, and pills coverage of 95%, 70% and 60%). These thresholds demonstrated a similar trend of increasing odds ratios as adherence definitions were restricted to a greater number of missed days or doses. Predictive abilities for these indicators were similar to those evaluated in the primary analysis.

The association between self-reported adherence assessment and virological failure was slightly weaker than that of comparable measures derived from refill data, in agreement with other studies which suggest self-reported adherence is generally less accurate than pharmacy based measures [7, 18]. Overall agreement between pharmacy-based and self-reported measures, however, was relatively high. Conclusions about the value of self-report in monitoring adherence in our data are limited by the fact that self-assessments were collected in a limited and non-random subset of individuals in our cohort.

Two additional analyses provided insight into adherence patterns across the treatment period. We found that non-adherence, using any definition, was more likely to occur when an individual had to return to the pharmacy every month to pick up prescriptions. However, the association between measured adherence and virological outcomes was stronger when prescriptions were refilled every two months. In this clinical setting, more frequent pharmacy visits are scheduled for the first few months after treatment initiation, as well as for individuals with known barriers to treatment response and those switching regimens. It may be the case that those scheduled to come more frequently for pharmacy refills have higher rates of drug mutations or issues with drug absorption or dosing that are driving response to treatment more than degree of adherence. Association between viral load and adherence was also stronger among individuals on their first HAART regimen. This may be due to the presence of fewer drug mutations earlier in treatment [19]. Our sensitivity analyses revealed only slight differences in performance of the adherence indicators when applied only at the first test for virological response after six months on HAART, or when applied only to the follow-up of patients who experienced viral load suppression within six months of starting HAART.

Virological response to treatment may be dependent on sustained high levels of adherence over many months or patterns of adherence over the duration of the treatment period. Because we restricted our analysis to refills that fit either a standard 28- or 56-day cycle, and most individuals had at least one refill period that did not fall into a standard schedule, we were unable to look at cumulative adherence over a period of many months or across the full history of treatment with HAART. Extending the period over which adherence was assessed from 2 to 4 months did increase the ability of almost all of the calculated adherence measures to predict virological outcomes. This is in agreement with previous studies which have indicated that pharmacy-based measures applied over shorter durations of adherence assessment are typically less accurate when predicting virological outcomes compared to adherence estimated over longer periods of time [7]. We also observed an overall decrease in adherence when looking over a longer period of follow-up, as maintaining high levels of adherence was now dependent on more trips to the pharmacy for refills. The increases in the magnitude of the association between non-adherence and virological failure were relatively modest, however, and power was reduced through the exclusion of nearly 30% of refill visits.

While regression results indicated associations between virological failure and each of the calculated indicators, as well as for self-report, all adherence measures demonstrated weak ability to predict virological failure. No c-statistics in the main analysis exceeded 0.528, which is essentially the equivalent of random classification (0.5), driven by very low sensitivity of all measures. Further, 81% of all virological failure events occurred in those with complete pill coverage in the prior two months. This lack of predictive ability may reflect the crudeness of our adherence measurement, in which pharmacy refill does not directly correspond to taking doses as prescribed [20,21]. In our study setting there are few resources for obtaining ARVs outside of the clinic pharmacy. This limits misclassification of non-adherent behavior among those attending pharmacy visits late, as it is unlikely they actually procured their medication from another source. The observed behavior, however, also represents the upper limit of potential adherence, in that if a patient does not possess drugs, they cannot take them; while mere possession of drugs does not guarantee drug intake, but only enables it. In agreement with our data, these measures are generally regarded as less sensitive but more specific [7]. Other contributing factors to virological response, including resistance and dosing issues, are not accounted for when strictly looking at adherence [22,23].

Because adherence measures alone fail to identify most cases of virological failure, ongoing and consistent viral load monitoring should remain a key component of clinical evaluation, regardless of adherence data available. The adherence measures assessed in this analysis, however, still have value for clinical care and research. The high specificity of the assessed indicators suggests that patients identified as non-adherent using refill data are likely correctly classified and truly in need of appropriate intervention. This is particularly true in the more extreme cases, where individuals show up many days or weeks late for their prescription refills. In addition to being at higher risk for negative virological outcomes, interventions for these individuals with long gaps in treatment coverage are critical for long-term retention in care. Additionally, adherence can be assessed more frequently than viral load. It may also be the case that the effects of sub-optimal adherence observed over a short period of time (2 months in our analysis) only become evident later in follow-up. Therefore, a classification of non-adherence during follow-up can provide an opportunity for early intervention even in the absence of corresponding virological data.

While we recognize that our study setting, which used scheduled refill appointments and had extensive electronic pharmacy records, may not be generalizable to most resource-limited settings, some insights from our analyses may be applicable across HAART treatment settings. Our findings suggest that pharmacy-based or self-reported measures indicating non-adherence should taken seriously and should be followed-up with appropriate interventions. Reports of adequate adherence, on the other hand, may require skepticism and evaluation by other means. The results of our analysis highlight some of the limitations of pharmacy-based measures for clinical assessment of patients on HAART, primarily the low sensitivity, regardless of threshold applied, of adherence measures classifying viral load. What is also suggested from our results, however, is the clinical value of highly specific data to identify non-adherent patients. Further, our analysis demonstrates that the threshold chosen to assess adherence using pharmacy data impacts the observed link between adherence and clinical outcomes, which is valuable for both clinical and research questions.