Correlation between pill counts and biologic effects in an HIV-1 prevention clinical trial: implications for measuring adherence

Introduction

In a randomized, placebo-controlled clinical trial of a daily self-administered regimen, adherence is a critical determinant of effectiveness. For the primary intention-to-treat analysis of a clinical trial adherence is typically not considered, since if the intervention arm is more effective than the placebo arm, adherence was sufficient for a biologic effect to be observed. However, one potential explanation for lack of efficacy in an intention-to-treat analysis is poor adherence. Secondary analyses examining trends in effectiveness by adherence levels can be supportive of biologic effects of study medications, which may help inform whether the drug truly has limited or no biologic effect or whether suboptimal adherence has limited the effect, which could be modified by new formulations with more sustained delivery or longer half-lives. These important secondary analyses depend on accurate measurement of adherence. Understanding the effect of adherence to daily prevention strategies in HIV-1 prevention clinical trials is a topic of considerable recent attention.

Adherence to a self-administered daily pill is a behavioral outcome, and like many behaviors, is difficult or impossible to measure directly (1, 2). Studies often use multiple strategies to triangulate adherence, ranging from relatively inexpensive methods like participant self-reported adherence and clinic-collected pill counts of unused study product to more resource-intensive strategies such as unannounced home visits or daily electronic monitoring devices. Because of their convenience, self-report and pill count adherence measures are often collected on all subjects at potentially all study visits, but can be inaccurate due to social desirability bias with self-report and potential for manipulation of pill counts(3, 4). In large trials, more expensive methods for measuring adherence are feasible in only a subset of participants and visits.

Biologic markers of treatment effect have been very useful in defining the strengths and limitations of various adherence measurement strategies (5, 6). Biomarkers of clinical effects have also been used to validate adherence measures of treatment efficacy (7-10), including viral load suppression and progression to AIDS in treatment of HIV-1 (11), and blood pressure in treatment for hypertension (4). In this study we used two clinical biomarkers to examine accuracy of pill counts as a measure of adherence in a randomized trial of acyclovir to reduce HIV-1 infectiousness.

Methods

Population and Procedures

The Partners in Prevention HSV/HIV Transmission Study was a randomized, double-blind, placebo-controlled clinical trial of acyclovir herpes simplex virus type 2 (HSV-2) suppressive therapy, conducted at 14 sites in 7 African countries (Botswana, Kenya, Rwanda, South Africa, Tanzania, Uganda, and Zambia). A total of 3408 HIV-1 seropositive persons who were also seropositive for HSV-2 were enrolled, along with their HIV-1 seronegative sexual partners. Subsequent testing identified 27 ineligible participants (3 were not HIV-1 seropositive, 24 were not HSV-2 seropositive), thus 3381 are included in this analysis. The HIV-1 infected partner received twice-daily acyclovir 400 mg or matching placebo. A complete description of the study is available elsewhere(12). As reported previously, acyclovir had no effect on prevention of HIV-1 transmission to the HIV-1 uninfected partner, in spite of an average 0.25 log₁₀ copies/mL reduction in plasma HIV-1 RNA concentrations and 73% reduction in incident genital ulcer disease (GUD) due to HSV-2 in the HIV-1-infected partners (13), biologic effects that have been replicated in other studies (14-19). At baseline, none of the HIV-1 infected partners were taking combination antiretroviral therapy and none were eligible for therapy under national treatment guidelines of their country of residence. Clinical and immunologic status was monitored during the study and subjects were referred for antiretroviral therapy if they experienced clinical disease progression or CD4 count decline (20).

Plasma HIV-1 RNA concentrations and HSV-2 GUD assessment

HIV-1 infected partners were seen monthly for provision of study drug (acyclovir or placebo). GUD was assessed by physical exam every three months. Ulceration consistent with genital herpes was swabbed and the specimens tested by polymerase chain reaction (PCR) for HSV-2 (21, 22); samples with ≥150 copies of HSV DNA/mL were considered positive.

Plasma HIV-1 RNA measurements (COBAS AmpliPrep/COBAS TaqMan HIV-1 RNA assay, version 1.0, Roche Diagnostics, Indianapolis, IN) were performed on samples collected at baseline, at months 3, 6 and 12 and at the final study visit. The limit of quantification was 240 copies/mL.

The study protocol was approved by the University of Washington Human Subjects Review Committee and ethical review committees at each of the collaborating organizations. All participants provided written informed consent.

Clinic-based pill count adherence

One bottle of 80 pills was dispensed at each study visit for twice-daily dosing for one month, providing sufficient extra pills to allow visit windows of up to 40 days. Each bottle was individually numbered, and participants returned bottles for pill counts at each study visit. Clinic staff recorded the identifying number of each bottle at the dispensing visit, and the number of pills remaining in that numbered bottle when the participant returned the bottle at a subsequent visit. Monthly adherence was computed by comparing the number of pills not returned for each bottle with the expected dosing for the elapsed days between visits. At some visits, participants did not return pill bottles; and on occasion >1 bottle was dispensed to a participant to cover a planned absence.

To assess the association between pill counts and biologic response in the present study, only precisely measured adherence by pill count was used: i.e., an adherence value was calculated only for study intervals when a single bottle was dispensed and returned at adjacent visits. At visits where a participant did not return a bottle, missed a visit, returned >1 bottles, or had multiple bottles in circulation, pill count adherence was reported as incalculable, thus pill count adherence was calculated only when the participant had sufficient pills for the visit interval.

Assessment of adherence and viral load effect included all post-baseline visit months prior to a viral load measure (i.e. months 3, 6, 12, and exit visit). Assessment of adherence and GUD included all post-baseline quarterly visits where genital exams were performed. Visits with protocol-initiated drug interruptions and visits following initiation of antiretroviral treatment were excluded from the analyses.

Statistical Methods

Our goal was to compare the biologic effect between acyclovir and placebo groups across various adherence categories. For each visit, pill count adherence was classified as 0-<50%, 50-<80%, 80-<95%, 95-<102%, ≥102%, or incalculable. “Very high” adherence of 95-<102% was considered consistent with full adherence to daily dosing between visits, given the uncertainty of timing of dosing of last dose relative to the time of the clinic visit, and was thus used as the comparison group. “Over-adherence” (>102%) corresponds to fewer pills returned than expected.

For the plasma viral load biomarker, the primary parameter of interest was the change in log₁₀ plasma HIV-1 RNA between acyclovir and placebo arms within each adherence level. Changes in log₁₀ plasma HIV-1 RNA were modeled using linear mixed effects fitting an interaction between adherence and study arm, adjusted for baseline log₁₀ plasma HIV-1 RNA, assuming compound symmetry for repeated measures. For the HSV-2 GUD biomarker, the primary parameter of interest was incidence of HSV-2+ GUD based on positive HSV-2 DNA PCR from a genital lesion, indicated by odds ratios, between arms within each adherence level. Changes in HSV-2+ GUD were modeled using logistic regression with a generalized estimating equation approach assuming independent correlation, fitting an interaction between adherence and study arm.

Change in plasma HIV-1 RNA and GUD for the same levels of adherence to placebo and acyclovir was used to determine the biologic effect of acyclovir. Randomization assures balance by arm in baseline covariates, but this balance is not assured in adherence groups determined post-randomization. To examine potential confounding of the relationship between study arm and plasma HIV-1 RNA or GUD within each adherence group, we tested for significant interactions between placebo and acyclovir arms and covariates chosen a priori for potential association with adherence (age, marital status, gender, and time-dependent covariates of CD4 count, any outside partner, any unprotected sex with study partner). In a linear regression model predicting percent pill-count adherence using data from all visits and accounting for repeated measures, we tested for significant interaction between each covariate and study arm.

Results

Of the 3,381 eligible discordant couples enrolled in the study, in 1,097 (32%) the male partner was HIV-1 infected. Median age of the HIV-1-infected partner was 32 (IQR 27-38) with median CD4 count at enrollment of 462 cells/mm³ (IQR 347-631). Retention was very high: 2,956 (93% of expected) of HIV-1-infected partners completed the 12 month visit and 1,265 (92% of expected) the 24 month visit, and the median number of monthly visits was 20 (IQR 15-24).

Pill count adherence

Of the 59,115 scheduled on-drug dispensing visits recorded among 3,381 participants, monthly adherence could be calculated in 52,635 (89%). The 11% of visits with incalculable adherence occur in almost equal part due to bottles not returned (5.9%) and irregular patterns of study drug bottle dispensation and return at the next monthly visit (5.1%) (Table I). Only 0.3% of visits had low pill count adherence (defined as up to 50% pills taken) (Table II), and 3% of visits had moderate adherence of 50%-80%. A majority (70%) of visits had high (80-<95%) or very high (95-<102%) adherence. Over-adherence (≥102%) was measured in 15% of visits, when fewer pills were returned than expected. There was no statistically significant difference between acyclovir and placebo arms in pill count adherence (p = 0.37), or in proportion of visits with incalculable adherence (p = 0.73).

Table I.

Pill count data and estimation of adherence

All study visits	Number of visits	% of visits included
Scheduled visits	61,279
Protocol specified treatment interruption	2,164
Visits assessed for pill count completeness	59,115	100.0%
Calculable pill count adherence	52,635	89.0%
Incalculable adherence	6,480	11.0%
Missing pill count dataa	3,487	5.9%
Multiple bottles in circulation	2,993	5.1%
Pills returned at non adjacent visitsb	904	1.5%
Multiple bottles returned at a visit	1,924	3.3%
Multiple bottles dispensedc	165	0.3%

^aBottles forgotten or lost

^bThe participant has more than one pill bottle as a result of not returning a bottle at a previous visit or having multiple bottles dispensed at a visit.

^cMultiple bottles were dispensed in situations where the participant would be unable to come to the clinic and would otherwise be without study drug.

Table II.

Pill count adherence for all monthly visits

Pill count adherence		Acyclovir		Placebo		Total
		N	%	N	%
Low	0-<50%	91	0.3%	100	0.3%	191
Medium	50-<80%	975	3%	927	3%	1,910
High	80-<95%	4,166	14%	4,421	15%	8,652
Very high	95-<102%	16,756	56%	15,891	54%	32,923
Over-adherence	≥102%	4,475	15%	4,833	16%	9,357
Incalculablea		3,229	11%	3,251	11%	6,528
Total		29,692		29,423		59,561

^aIncalculable as a result of unreturned bottles or multiple bottles in circulation

Pill count adherence and viral load

The relationship of adherence to plasma viral load was assessed across 11,405 visits with plasma HIV-1 RNA measurements from 3,309 participants. At high (80-<95%) and very high (95-<102%) adherence, a statistically significant greater reduction in log₁₀ plasma HIV-1 RNA was observed in acyclovir compared to placebo arm participants: −0.26 (95% confidence interval (CI) −0.32, −0.21), p < 0.001 and −0.29 (95% CI −0.33, −0.25), p < 0.001, respectively (Figure 1). Compared to the very high adherence group, both the ‘over-adherence‘ (≥102%) group and the incalculable group had significantly smaller changes in viral load, consistent with poorer compliance to taking the medication (t-test for interaction compared to the very high adherence group of t = 3.21, p = 0.001 and t = 2.29, p = 0.02, respectively, Figure 1). Low and moderate adherence were also associated with smaller plasma HIV-1 RNA change.

Figure 1.

Effect of Acyclovir on plasma viral load reduction by pill count adherence category

Pill count adherence and HSV-2+ GUD

Genital ulcers were observed at 356 of 19,944 quarterly visits with clinical assessment of ulcers, and 219 of these tested positive for HSV-2 DNA: 46 in the acyclovir arm and 173 in the placebo arm. Figure 2 shows the change in odds of occurrence of HSV-2+ GUD, comparing acyclovir to placebo, within each adherence group. The pattern of effectiveness in reduction in HSV-2+ GUD by pill count adherence paralleled that seen with plasma HIV-1 RNA, with the biggest effects seen in the high and very high adherence groups, and less effectiveness in all other groups. In the high and very high adherence groups, acyclovir reduced the odds of incident HSV-2+ GUD by 93% (95% CI: 68-93%, p = 0.003) and 84% (95% CI: 73-91%, p < 0.001), respectively. Significantly less reduction in incident HSV-2 GUD was seen in the over-adherence and incalculable adherence groups, with reduction in odds of 62% (95% CI: 21-82%, p = 0.01)and 49% (95% CI: 10-76%, p = 0.09 . The Chi-squared test for interaction compared to the very high adherence group had p = 0.04 and p=0.01, respectively.

Figure 2.

Effect of Acyclovir on reduction in genital ulcer disease by pill count adherence category

We identified two potential confounders of the relationship between study arm and plasma HIV-1 RNA or GUD within adherence groups: in models predicting percent adherence we found statistically significant interactions between unprotected sex and arm ( p = 0.02 for interaction) and sexual partners other than the enrolled HIV-1 uninfected partner and arm (p = 0.008). No significant interactions were found with age, marital status, gender, baseline GUD and CD4 count (time dependent). We tested for confounding of the biologic effects of acyclovir on plasma HIV-1 RNA and GUD for each adherence group by adjusting for both unprotected sex and outside partners, and found the estimates were not substantively affected.

Discussion

We analyzed pill counts of returned, unused study drug from nearly 60,000 visits in an efficacy trial of acyclovir suppression among HIV-1/HSV-2 co-infected persons for prevention of HIV-1 transmission. Adherence, as measured by monthly pill counts of returned study drug, showed a dose-dependent association with acyclovir biologic effects, indicating pill counts reflected differing levels of adherence. Apparent ‘over-adherence’ by pill counts, with fewer pills returned than expected, potentially indicated low adherence, as evidenced by the smaller biologic effect in plasma viral load reduction and incident genital herpes.

Pill counts are commonly used as a quantitative measure of adherence in clinical trials; study staff actively track pill bottles due to the regulatory requirement to track dispensing of investigational products and the need to ensure a supply of blinded study product to each participant. Among participants who are compliant with bottle returns, the number of pills remaining in the bottle reliably measures pills not taken; pills not returned are assumed to have been taken as instructed. Because clinic-based pill count measures of adherence can be assessed on all participants and at all visits, this measure is widely utilized in many clinical trial settings.

Clinic-based pill counts are logistically challenging to implement in large clinical trials with frequent study visits. Pill tracking methods need to accommodate unscheduled visits, and be adjusted for drug interruptions, additional bottles dispensed to allow for travel, replacement drugs for loss or expiration, and participants forgetting to return bottles. Although the majority of clinical trials report secondary analyses by adherence groups, there is seldom reporting of the completeness or quality of pill count data or the percent of visits excluded in these secondary analyses as a result of missing or unreliable adherence data. In our study, adherence by clinic-based pill count was calculable in 89% of visits, higher than in previous HIV-1 prevention trials (6, 23, 24), even using the very stringent criteria for definition of ‘calculable’ counts in this analysis. The high quality ascertainment of pill counts in the Partners in Prevention HSV/HIV Transmission Study reflected high participant retention and careful monitoring of individually numbered bottles, which led to minimal loss of data from missed and late visits and high compliance with pill bottle return.

Clinic-based pill counts can lead to inflated estimates of adherence. A significant challenge to getting unbiased adherence measure from pill counts is ensuring that study sites perform adherence counseling independent of pill count assessments, as there is a risk of inadvertent social desirability bias if participants learn to not return unused pills to avoid counseling about missed doses. Pill counts performed at unannounced home visits or during random phone calls, electronic medication monitoring and biologic markers of drug ingestion provide more objective, yet expensive measures of adherence (2, 25, 26), These also have limitations in interpretation, for example, opening a pill container with electronic monitoring does not ensure the pills were taken and laboratory measurements of drug levels offer only periodic assessment and can be influenced by a “white coat effect” of dosing just prior to a study visit (3).

Clinic-based pill counts are likely an accurate measure of adherence where adherence is less than perfect, if participants physically return pills that have not been taken. In our study we demonstrated that decreased adherence to daily acyclovir resulted in decreased reduction in plasma viral load and HSV-2+ GUD, for those with <80% pill count adherence. Similar biologic reductions in plasma viral load and GUD incidence observed for the categories with 80-<95% and 95-<102% adherence may reflect limited power to distinguish biologic effects at the upper ranges of adherence or potential misclassification, if participants removed unused pills for travel, doubled doses, or did not return unused pills. Others have reported that calculated clinic-based pill adherence over-estimates true adherence (4, 10), and a clear weakness of clinic-based pill count adherence is the inability to discriminate between highly adherent people and those discarding pills who appear adherent. Reliability of pill count data may differ by risk population: recently HIV prevention studies of daily pre-exposure prophylaxis have reported different levels of inconsistency between pill count adherence and an objective biologic measure of detectable drug levels in plasma, although all showed overestimation of adherence. In MSM, pill count adherence was >85%, but only 50% of plasma samples had drug levels consistent with daily dosing(6); in a population of women in Africa, pills count adherence was >86% but plasma drug levels were present in only 38%(27) of controls; whereas among HIV discordant couples, pill count adherence was > 95% but 80% of plasma samples had detectable drug levels(28).

Our finding of reduced biologic effects on GUD and plasma viral load when adherence data was incalculable suggests that missing pill count adherence data among those returning for visits (as distinct from those missing visits) likely indicates poor adherence, as others have reported(3). Moreover, biologic effects in our data suggest that ‘over-adherence’ by pill count data also indicates likely low levels of pill taking; similar findings have been reported from hypertension studies (4). Over-adherence can result from a variety of behaviors, including participants not returning unused pills perhaps in part due to social desirability bias, repackaging the pills (for example into a pill box as an adherence aide), taking more pills than instructed, and losing pills(1).

We reported measures of the difference in biologic outcome in plasma viral load and incidence of HSV-2+ GUD, for visits grouped by adherence level. The Partners in Prevention HSV/HIV Transmission Study was double-blinded with identical-appearing placebo pills. Given the lack of side effects with acyclovir compared to placebo (29, 30), unmasking of arms is unlikely thus it is credible that subgroups with similar pill-count adherence shared similar characteristics and confounding of biologic effects within adherence groups was likely minimal. Other limitations were that plasma HIV-1 RNA was measured at 4 time points in the study and GUD only quarterly, whereas pill counts were done monthly. In addition, GUD swabs were only collected when ulcers were found by visual inspection on clinical exam, likely leading to under-assessment of HSV-2+ reactivation in this population of HSV-2 seropositive adults.

Pill counts of returned study drug in a clinical trial, when rigorously implemented, are a useful indicator of less than perfect adherence. Our assessment of the agreement between pill count adherence and biologic evidence of drug activity, within the context of a placebo-controlled randomized clinical trial, indicated missing pill counts and measured ‘over-adherence’ also likely indicate poor adherence. Clinical trials should carefully implement procedures for tracking returned study product, and report the proportion of visits with over-adherence, and missing or incalculable adherence data, to provide a clear interpretation of effectiveness data and the association of efficacy with adherence.