The Combination of Pill Count and Self-Reported Adherence is a Strong Predictor of First-Line ART Failure for Adults in South Africa

Abstract

Background

Suboptimal adherence to antiretroviral therapy (ART) is a strong predictor of virologic failure (VF) among people with HIV. Various methods such as patient self-report, pill counts and pharmacy refills have been utilized to monitor adherence. However, there are limited data on the accuracy of combining methods to better predict VF in routine clinical settings. We examined various methods to assess adherence including pill count, medication possession ratio (MPR), and self-reported adherence in order to determine which was most highly associated with VF after ≥ 6 months on ART.

Methods

We conducted a secondary analysis of data from a case-control study. At enrollment, pharmacy refill data were collected retrospectively from the medical chart, pill counts were completed to derive a pill count adherence ratio (PCAR) and a self-report questionnaire was administered to all participants. Parametric smooth splines and receiver operator characteristic (ROC) analyses were carried out to assess the accuracy of the adherence methods.

Results

458 patients were enrolled from October 2010 to June 2012. Of these, 158 (34.50%) experienced VF (cases) and 300 (65.50%) were controls. The median (IQR) PCAR was 1.10 (0.99–1.14) for cases and 1.13 (1.08–1.18) for controls (p<0.0001). The median MPR was 1.00 (0.97–1.07) for cases and 1.03 (0.96–1.07) for controls (p=0.83). Combination of PCAR and self-reported questions was highly associated with VF.

Conclusion

In this setting, a combination of pill count adherence and self-report adherence questions had the highest diagnostic accuracy for VF. Further validation of this simple, low-cost combination is warranted in large prospective studies.

INTRODUCTION

Suppression of HIV replication requires optimal exposure to antiretroviral (ARV) therapy (ART). ART adherence has been shown to be among the strongest predictors of virologic failure (VF) [1, 2], development of drug resistance [3] and ultimately disease progression [4] and death [5–7].

Various approaches exist to assess medication adherence. There is no established gold standard method for measuring adherence and each method has advantages and disadvantages [8, 9]. Electronic monitoring (e.g., MEMs caps) [10, 11] and directly observed therapy (DOT) [12] are highly accurate measures of adherence used in clinical trials, although each lack feasibility in a routine clinical setting [13–15]. On the other hand, self-reported questionnaire responses, pill counts, and pharmacy refill metrics, such as the medication possession ratio (MPR) [16] and proportion of days covered (PDC) [17], are low-cost alternatives but have been variably effective at estimating adherence. Therapeutic drug monitoring, which measures plasma, hair or red blood cell concentrations of ARVs, has been used to determine ART exposure but presently lacks standardization, is costly and is not available in most clinical settings.

There are limited studies evaluating the accuracy of simple, low-cost ART adherence monitoring tools such as self-report questionnaires, pill count and pharmacy refill, used individually or in combination, to predict treatment success or failure. The Risk Factors for Virological Failure (RFVF) Study was a case-control study conducted at the McCord Hospital (MCH) in Durban, South Africa with the express goal of identifying the prevalence of HIV drug resistance mutations and risk factors for VF after first-line ART [18]. In this substudy of the RFVF, we sought to identify which measurement of adherence was most highly associated with VF and determine risk factors associated with poor adherence. We examined participant responses to an adherence questionnaire, pharmacy refill data and unannounced pill counts.

METHODOLOGY

Study Site and Design

MCH is a referral center for ART within the province of KwaZulu-Natal. Treatment and care at MCH have been supported by the South African government and the President’s Emergency Plan for AIDS Relief [19]. Viral load (VL) monitoring is routinely performed 5 months after initiating first-line ART and every 6 months thereafter. The study design and inclusion criteria for the RFVF study were described elsewhere [18]. In brief, the present data are derived from a case-control study. Cases were defined as patients with newly identified VF (VL > 1000 copies/mL) and controls (matched 2:1) were defined as participants with VL ≤ 1000 copies/mL after ≥ 5 months of a first ART regimen, respectively.

Data Collection

All study participants underwent a single, semi-structured interview that consisted of a questionnaire, a neurocognitive assessment, and a pill count at enrollment. The demographics, pharmacy refill and laboratory data were obtained retrospectively from the electronic medical record. All of the data were stored and abstracted from Redcap electronic data capture tools hosted at Emory University [20]. All statistical analyses were performed in SAS (SAS Institute, Version 9.3, Cary, NC).

Adherence Metrics

The objective of this analysis was to identify which adherence metric or combination of metrics most accurately predicts VF in the RFVF study. The three adherence metrics that we investigated in this paper were MPR, pill count adherence ratio (PCAR) and self-reported adherence. The precise definition of these measures plays a prominent role in the success or failure of these methods and is described in detail below.

Both MPR and PCAR are composite measures based on several derived variables. At study enrollment, participants presented their ARV bottles and all unused pills were counted. The unused pills are defined as pill count at enrollment (C). Cases were identified as having VF within 1–2 weeks of a visit to the clinic. These participants were then notified and enrolled into the study if they agreed to participate within 1–2 weeks from that date. Their enrollment date was therefore 2–3 weeks from the most recent claim (last refill). Controls were selected within the same week as the cases. Their date of enrollment corresponded to a claim date, in most situations, with their last refill 28 days prior to enrollment. A graphical illustration of the pharmacy refill claims is provided in Supplemental Fig. (1).

We defined the length of the eligible refill window (W) as the number of days between the earliest refill date in the 6 months prior to the enrollment date and the enrollment date. A single day of supply (P) of ARVs is defined as all of the pills necessary to be taken in one day and the total number of days of supply (D) is defined as the sum of all days of supply dispensed from the pharmacy during W. Expected pills remaining (E) was then calculated as the difference of D and W which must be converted to pills using the pill per day quotient (Q).

• C: pill count on the date of enrollment

• P: all of the pills necessary to be taken in one day (a single day of supply)

• W: number of days in eligible refill window (refill window length)

• D: total number of days of supply

• E: expected pills remaining; E = (D − D) × Q,

• Q: pill per day quotient; Q = pill/dose × dose/day.

MPR is a proxy measure for access to care, describes an individual’s ability to pick-up refills and is used as a method to quantify medication use [21]. One of the disadvantages of MPR is that there are more than 4 different published measures using this term [22]. In the RFVF study, we defined the MPR as the ratio of D divided by the number of days in the study interval. In order to standardize the length of the observation window in the denominator, the number of days in the study interval was fixed at 180 days for all participants, so MPR is defined,

$$\mathit{MPR}=\frac{D}{180}.$$

If a pharmacy refill occurred sufficiently close to the enrollment date, D in the numerator of MPR was adjusted by subtracting those days of supply that extended beyond the enrollment date [23–25]. A smaller MPR is less adherent while an MPR of greater than or equal to 100% is perfectly adherence [26]. An MPR ≥ 80% (at least 144 out of 180 days) is a common threshold for dichotomizing medication adherence [27, 28].

PCAR is a measure of how well a participant follows the prescription schedule and a continuous metric of adherence patterns in a clinical setting [29, 30]. Intuitively, we perceive the ratio [31, 32] as the fraction of pills prescribed that were actually taken during the follow-up period:

$$\mathit{PCAR}=\frac{[\mathit{Total}\mathit{Doses}\mathit{Dispensed}]-[\mathit{Total}\mathit{Doses}\mathit{Missed}]}{\mathit{Total}\mathit{Doses}\mathit{Dispensed}}$$

In the RFVF study, the total pills missed or skipped are inferred from a corrected pill count observed on the enrollment date. The corrected pill count (C − E) is computed by subtracting expected pills remaining from the pill count observed on the date of enrollment. Thus, the general formula for PCAR is one minus the proportion of pills missed during the eligible refill window [33].

$$\mathit{PCAR}=\{\begin{array}{c}1-\frac{C-E}{D\times Q},E>0\\ 1-\frac{C-E}{D\times Q-E},E\le 0\end{array}$$

When E > 0, (D × Q) indicated total number of pills dispensed. However, we noticed that for E ≤ 0 (equivalently, D ≤ W), (D×Q − E = W×Q) was used as denominator since for larger refill window, the total number of pills dispensed over the window should be considered as the window length times pill per day quotient.

Self-reported adherence is a common measure in clinical studies because it is inexpensive to collect. This fact makes it particularly attractive in resource-limited settings [34]. At enrollment, participants were asked a series of questions (RFVF Questionnaire) including some modified from the AIDS Clinical Trials Group (ACTG) adherence questionnaire [9] (See Supplemental Table 2). In the RFVF study, the research coordinator, who was blinded to the study assignment (case or control) and a trained HIV counselor, conducted the interview that contained information regarding ART adherence. The possible factors that were considered to have an association with VF were included in the model [9, 35].

ARV Hierarchy

We calculated PCAR based on just one ARV according to the “ARV hierarchy” (Supplemental Table 1). This “ARV hierarchy” was based upon pill burden (dose and frequency) and frequency of being prescribed. For each participant, we chose the ARV that ranked highest in the “ARV hierarchy”. For instance, if a participant’s regimen consisted of lamivudine (3TC), stavudine (d4T), and lopinavir/ritonavir (LPV/r), we chose 3TC to calculate the PCAR since 3TC ranked highest.

Diagnostic Accuracy of Statistical Models

In order to assess the fitness of the adherence metrics in relationship to VF, seven statistical models were constructed from different permutations of three sets of covariates: PCAR, MPR, and self-report questions. We fitted the statistical model via logistic regression and then computed the receiver operating characteristic (ROC) curve using the observed VF status and predicted probabilities from the regression fit. The area under the ROC curve (AUC) is then calculated using standard methods [36–38]. The diagnostic accuracy of the seven models was evaluated by comparing the AUCs of corresponding models and the model with the largest AUC achieved the best sensitivity and specificity among the models considered. We used the Akaike information criterion (AIC) to evaluate goodness-of-fit of the statistical model. Of the seven models, the one with smallest value of AIC was preferred [39–41].

Model Selection for Risk Factors

All variables from the questionnaire and case report form (CRF) as well as the explanatory covariates were analyzed to determine their association with PCAR and MPR in univariate analyses. Only significant (p<0.05) and epidemiologically meaningful factors were further analyzed. Several logistic regression multivariable (MV) models and general linear models (GLM) were constructed utilizing a stepwise variable selection procedure by domain and then overall to generate final models. Model 1 (baseline factors) was aimed at identifying the risk factors present at the initiation of ART that were most associated with PCAR and MPR. Model 2 attempted to assess the association of all time-updated variables with PCAR and MPR.

RESULTS

Descriptive Analysis

Between October 2010 and June 2012, 458 individuals receiving first-line ART were enrolled into the RFVF study (158 cases and 300 controls). The cohort demographics and adherence metrics are summarized in Table 1. Over 35% of the participants were male and the median age was 38.4 years (IQR, 33.2–45.2). The median (IQR) of CD4 count at enrollment for cases was 206 cells/μL (108–340), which was lower than that for controls 359 cells/μL (240–484). The median (IQR) of recent HIV RNA VL was 17,138 (2,974–74,056) copies/mL. The median VL for controls was not quantifiable since the majority of VL were undetectable. The average pill count (C) for cases was 16.13 pills (SD, 12.40), higher than that for controls 11.94 pills (SD, 9.66) (p<0.01). The median (IQR) of refill window length (W) was 164 days (154–172) for cases and 166 days (166–168.5) for controls (p<0.01). The median (IQR) of total number of days of supply (D) for cases was 180 days (195–210) and 210 days (180–210) for controls (p<0.0001). The median (IQR) PCAR was 1.10 (0.99–1.14) for cases and 1.13 (1.08–1.18) for controls (p<0.0001). The median MPR was 1.00 (0.97–1.07) for cases and 1.03 (0.96–1.07) for controls.

Table 1.

Selected characteristics of variables of interest.

Variables	Overall (n=458)	Control (n=300)	Case (n=158)	p Value£
Age
Mean ± SD (N)	39.6 ± 9.0 (458)	40.9 ± 9.1 (300)	37.1 ± 8.4 (158)	<0.0001*
Median [Q1–Q3]	38.4 [33.2 – 45.2]	39.4 [34.8 – 46.6]	36.6 [31.2 – 41.7]
Gender
Male N (%)	162 (35.37%)	87 (29.00%)	75 (47.47%)	<0.0001*
Female N (%)	296 (64.63%)	213 (71.00%)	83 (52.53%)
Race
Black N (%)	452 (98.91%)	295 (98.66%)	157 (99.37%)	0.66
Colored N (%)	5 (1.09%)	4 (1.34%)	1 (0.63%)
Enrollment CD4 Count in cells/μL
Mean ± SD (N)	334.4 ± 210.4 (456)	383.0 ± 207.5 (299)	241.7 ± 183.5 (157)	<0.0001*
Median [Q1–Q3]	300.5 [183.5 – 448.0]	359.0 [240.0 – 484.0]	206.0 [108.0 – 340.0]
Enrollment HIV RNA Viral Load copies/ mL for Cases
Mean ± SD (N)	___	___	95236 ± 196760 (158)	___
Median [Q1–Q3]	___	___	17138 [2974–74056]	___
Pill Count at Enrollment (C)
Mean ± SD (N)	13.33 ± 10.81 (431)	11.94 ± 9.66 (288)	16.13 ± 12.40 (143)	0.0006*
Median [Q1–Q3]	11 [5 – 18]	10 [5 – 16]	13 [7 – 24]
Refill Window Length (W)
Mean ± SD (N)	163.1 ± 19.2 (454)	165.2 ± 12.8 (300)	160.5 ± 22.5 (154)	0.0052*
Median [Q1–Q3]	168 [159 – 170]	166 [166 – 168.5]	164 [154 – 172]
Total Number of Days of Supply (D)
Mean ± SD (N)	197.0 ± 27.6 (455)	201.2 ± 24.1 (300)	189.1 ± 32.0 (155)	<0.0001*
Median [Q1–Q3]	210 [180 – 210]	210 [180 – 210]	180 [195 – 210]
Medication Possession Ratio (MPR)
Mean ± SD (N)	1.00 ± 0.12 (458)	1.00 ± 0.09 (300)	0.99 ± 0.16 (155)	0.83
Median [Q1–Q3]	1.03 [0.96 – 1.07]	1.03 [0.96 – 1.07]	1.00 [0.97 – 1.07]
≥ Overall Median (%)	47.0	49.3	42.6	0.17
≥0.90 (%)	87.3	89.3	83.5	0.10
≥0.80 (%)	95.9	96.7	94.3	0.23
≥0.70 (%)	97.6	99.0	94.9	0.010*
QUARTILE				0.2665†
Highest Quartile (%)	14.3	11.0	20.7
Upper Middle (%)	32.8	38.3	21.9
Lower Middle (%)	28.4	25.3	34.2
Lowest Quartile (%)	24.6	25.3	23.2
Pill Count Adherence Ratio (PCAR)
Mean ± SD (N)	1.10 ± 0.11 (429)	1.12 ± 0.10 (288)	1. 07 ± 0.10 (141)	<0.0001*
Median [Q1–Q3]	1.12 [1.05 – 1.17]	1.13 [1.08 – 1.18]	1.10 [0.99 – 1.14]
≥ Overall Median (%)	51.3	56.3	41.1	0.0033*
≥0.90 (%)	90.2	92.7	85.4	0.020*
≥0.80 (%)	92.4	94.7	88.0	0.015*
≥0.70 (%)	93.4	96.0	88.6	0.0045*
QUARTILE				0.0008*†
Highest Quartile (%)	23.8	29.9	11.4
Upper Middle (%)	27.5	26.4	29.8
Lower Middle (%)	23.8	24.0	23.4
Lowest Quartile (%)	24.9	19.8	35.5

^£P values obtained using Wilcoxon test for Refill Window Length, Total Number of Days of Supply, MPR and PCAR, using univariate logistic regression for Age, Gender, Race and Pill Count.

^†P values obtained using Kolmogorov-Smirnov (K-S) test.

^*P values ≤0.05.

Table 2 displays the raw pill count (C), MPR and PCAR based on the priority ARV according to the “ARV hierarchy”. Most participants received an EFV-containing regimen (overall: 82.93%, case: 79.62%, control: 84.67%). The second and third most prevalent regimens were a 3TC-containing regimen (overall: 9.85%, case: 12.10%, control: 8.67%) and TDF-containing regimen (overall: 6.56%, case: 8.28%, control: 5.67%). The average MPR was 1.03 (SD: 0.09) and 0.94 (SD: 0.16) for controls and cases, respectively, receiving a 3TC-containing regimen (p<0.05). The MPR was not statistically different by VF status for participants receiving EFV- and TDF-containing regimens. Controls were more adherent than cases for participants on a 3TC-containing regimen (p<0.01) as well as on an EFV-containing regimen (p<0.01) but PCAR was not statistically different among cases and controls on a TDF-containing regimen.

Table 2.

Pill count, medication possession ration and pill count adherence ratio by antiretroviral.

Variables Mean ± SD (N)	Overall (n=458)	Control (n=300)	Case (n=158)	p Value£
Pill Count at Enrollment (C)
3TC	24.62 ± 18.00(42)	23.58 ± 16.59(24)	26.00 ± 20.12(18)	0.6720
EFV	11.97 ± 8.76(358)	10.88 ± 7.95(245)	14.33 ± 9.95 (113)	0.0014*
FTC	15.67 ± 18.90(3)	15.67 ± 18.90(3)	NA (0)	-
TDF	13.54 ± 10.25(28)	10.00 ± 6.69(16)	18.25 ± 12.44(12)	0.0540
Medication Possession Ratio (MPR)
3TC	0.99 ± 0.13(45)	1.03 ± 0.09(26)	0.94 ± 0.16(19)	0.0360*
EFV	1.00 ± 0.11(379)	1.00 ± 0.10(254)	1.00 ± 0.13(125)	0.8201
FTC	1.02 ± 0.08(3)	1.02 ± 0.08(3)	NA (0)	-
TDF	0.99 ± 0.20(30)	1.01 ± 0.08(17)	0.96 ± 0.30(13)	0.5160
Pill Count Adherence Ratio (PCAR)
3TC	1.09 ± 0.13(42)	1.14 ± 0.09(24)	1.02 ± 0.15 (18)	0.0087*
EFV	1.10 ± 0.10(357)	1.11 ± 0.10(245)	1.08 ± 0.09 (112)	0.0027*
FTC	1.16 ± 0.09(3)	1.16 ± 0.09(3)	NA (0)	-
TDF	1.11 ± 0.10(27)	1.13 ± 0.08(16)	1.08 ± 0.13(11)	0.1979

^£P values obtained using two sample t test.

^*P values ≤ 0.05.

3TC: Lamivudine, EFV: Efavirenz, FTC: Emtricitabine, TDF: Tenofovir Disoproxil Fumarate.

Parametric Splines

In Fig. (1), we summarized the unadjusted probability of VF as a function of two adherence metrics, PCAR and MPR. Note that the observed range of PCAR and MPR is different and explains the discrepancy in the length of the curves. Except the sparse distributed outliers, the probability of VF was lowest when PCAR was within 1.20 to 1.40. This meant that, based on our algorithm, lower PCAR was crudely associated with a higher probability of VF. Interestingly, MPR showed a similar overall trend. Here, the probability of VF was lowest if MPR was in the interval 0.70–1.00. The probability of VF was at least 0.5 when MPR was less than 0.6.

Fig. 1.

Parametric splines for estimated probability.

ROC Analysis

Besides the two composite metrics, PCAR and MPR, all the participants were asked questions concerning pill use. We selected 13 self-report questions found to be marginally significant in univariate analyses (See Supplemental Table 2). Frequencies of the self-reported adherence questions are displayed in Table 3.

Table 3.

Frequencies of self-reported adherence.

Variables N (%)	Control (n=300)	Case (n=158)	p Value£
1. Took at least 1 dose more than 1 hour late in the last month	10(3.33)	14(8.86)	0.4034
2. Missed at least 1 dose in the last month	9(3.00)	16(10.13)	0.0166*
3. Took at least 1 dose more than 1 hour late in the last week	4(1.33)	9(5.70)	0.0144*
4. Missed at least 1 dose in the last week	1(0.33)	11(6.96)	0.0148*
5. Used a cell phone to remember to take medications	273(91.00)	134(84.81)	0.0475*
6. Used TV/radio to remember to take medications	16(5.33)	22(13.92)	0.0022*
7. Used a cell phone to remember to come for a drug collection appointment	0(0.00)	3(1.90)	0.9846
8. Missed at least 1 dose because was away from home	9(3.00)	18(11.39)	0.0005*
9. Missed at least 1 dose because was busy with other things	8(2.67)	21(13.29)	0.0002*
10. Missed at least 1 dose because fell asleep through the dose time	2(0.67)	9(5.70)	0.0098*
11. Missed at least 1 dose because ran out of pills	0(0.00)	4(2.53)	0.9805
12. Missed at least 1 dose because forgot to take pills	4(1.33)	8(5.06)	0.0209*
13. Missed at least 1 dose because wanted to avoid side effects	0(0.00)	3(1.90)	0.9824

^£P values obtained using univariate logistic regression.

^*P values ≤ 0.05.

The logistic regression consists of 7 models:

1. A model including self-reported questions (QS) only.

2. A model including adherence (PCAR) only.

3. A model including medication possession ratio (MPR) only.

4. A model including the combination of PCAR and QS.

5. A model including the combination of MPR and QS.

6. A model including the combination of PCAR and MPR.

7. The full model including the combination of PCAR, MPR and QS.

The ROC curves for all 7 statistical models are displayed in Fig. (2). When PCAR, self-reported questions, or MPR were modeled separately, none performed satisfactorily. MPR performed the worst by itself, and self-report performed the best by itself. We found the best model to be one that combined PCAR and self-report (AUC: 0.7161; AIC: 502.38) and performed almost as well as the full model (AUC: 0.7592; AIC: 481.35) in its diagnostic accuracy of VF status. Among the 13 self-reported questions, the strongest risk factors were: missed at least 1 dose in the last week (p=0.0148), took at least 1 dose more than 1 hour late in the last week (p=0.0144), used media to remember to take medications (p=0.0022), missed at least 1 dose because was away from home (p=0.0005), missed at least 1 dose because was busy with other things (p=0.0002), missed at least 1 dose because fell asleep through dose time (p=0.0098) (See Table 3).

Fig. 2.

ROC AUCs for ART adherence measurement methods.

Risk Factors for PCAR and MPR

PCAR

Baseline Risk Factors

The PCAR was dichotomized by greater than or less than median PCAR (1.12). The significant risk factor for a lower PCAR (< 1.12) in the logistic model was the use of stavudine (d4T) in the current ART regimen (p=0.0371 OR 1.960). Additional risk factors included age (per 5 year increase, OR 1.094), gender (male vs female, OR 0.984), less than three pre-ARV education sessions (OR 2.008) and use of a personal vehicle to travel to clinic (OR 1.391). The ROC AUC for baseline risk factors was 0.6108. In a GLM (PCAR treated as a continuous outcome), the risk factors for a lower PCAR were younger age (p=0.0312) and the use of d4T in the current ART regimen (p=0.0335).

Overall Risk Factors

In a full MV logistic model that included all domains, the absence of lipodystrophy (p=0.0097), the use of d4T in the current ART regimen (p=0.0599), not being pleased with their clinic experience (p=0.0608) and symptoms of diarrhea (p=0.0594) were associated with lower PCAR. Additional risk factors included age (per 5 year increase, OR 1.052), gender (male vs female, OR 1.090), and less than three pre-ARV education sessions (OR 1.693). The ROC AUC for this model was 0.6525. In a GLM, the risk factors for lower PCAR were the absence of lipodystrophy (p=0.0307), younger age (p=0.0359) and the use of d4T in the current ART regimen (p=0.0498).

MPR

Baseline Risk Factors

The MPR was dichotomized using median MPR (1.03). The significant risk factors for lower MPR (< 1.03) in the logistic model included the use of d4T in the current ART regimen (p=0.0194) and self-pay for clinic medications (p=0.0159). The ROC AUC for baseline risk factors was 0.6386. In a GLM (treating MPR as a continuous outcome), the risk factors for lower MPR were younger age (p=0.0270), having at least one family member living with HIV (p=0.0342), where started ARVs (p=0.0025) and took ethambutol in the 6 months prior to enrollment (p=0.0284).

Overall Risk Factors

In a full MV logistic model that included all domains, the use of d4T in the current ART regimen (p=0.0164), self-pay for clinic medications (p=0.0148), and symptoms of sadness (p=0.0394) were associated with lower MPR. Additional risk factors included not always practicing safe sex (OR 1.623). The ROC AUC was 0.6451. In a GLM, the significant risk factors for lower MPR were the number of pre-ARV education training session received (p=0.0016), what clinic ARVs were initiated (p=0.0113) and took ethambutol in the 6 months prior to enrollment (p=0.0210).

DISCUSSION

Adherence metrics and smooth splines in our study showed the PCAR performed well in association with VF. In addition, the MPR was another effective quantitative method to measure adherence. We found that the larger the MPR, the greater the likelihood that the participant had VF, demonstrating it to be a valid adherence measure [42, 43]. Higher adherence is associated with a lower probability of failing ART, hence taking pills and adhering to ART is essential to long term survival for individuals living with HIV and can result in better clinical outcomes in resource limited settings [2, 7, 44] or populations with low health literacy [45]. Moreover, due to the sexual transmission of HIV in communities, adherence to ART is not only an issue of central importance to clinicians and patients, but also for public health [9, 46]. In order to improve adherence, stressing the importance of following dosing schedule and explanation of the adverse effects to participants at each visit is crucial [47]. Better methods to effectively monitor pharmacy pick-ups should be explored in future studies. As well, the health belief model argues that participants should be motivated to adhere with medications by addressing their cultural beliefs and perceptions on illness [48].

ROC analyses provided a different view of the accuracy of the methods used to measure adherence in the RFVF study [49]. The combination of PCAR and self-reported questions was a better tool in the logistic model with better diagnostic accuracy and model fit [50]. As the self-reported method is inexpensive to implement and PCAR is relatively straightforward to calculate, these findings are important to further the study of pharmacy refill data [51, 52].

An interesting aspect of this study is that the average PCAR and MPR were higher than 1.0 in both cases and controls based on our algorithm. It is known that participants will accumulate 2 extra days of supply with each refill and thus it may account for a ratio being greater than 1. In addition, self-reported questions and pill count measurements tend to overestimate adherence [34]. But it is not clear if the “over-adherence” is a surrogate for VF. One explanation for “over-adherence” may be “pill dumping”. For instance, MPR >1.30 could signify pill dumping. This is where participants pour out their pills just prior to a pill count in order to appear “adherent” to providers [53]. Having a family member with HIV was associated with VF in the RFVF study [18]. This indicates that another key factor could be sharing pills with family members and partners. Therefore, besides computing quantitative ratios, analysis of self-reported adherence could provide additional information.

There are several strengths of this analysis including comparing several quantitative methods to measure adherence. Although a gold standard for adherence does not exist, our findings suggest PCAR combined with selected self-report adherence were superior to other tool combinations in identifying non-adherent participants [54] and tend to be an accurate measure to evaluate the time frame [22, 29]. Unlike many previous studies in which the measures were dichotomized as good versus poor adherence or self-report adherence, our PCAR is more sensitive [55, 56].

Our study had a few limitations. First, because these data come from a case-control study, we cannot say definitively that low adherence causes virologic failure. Some degree of non-adherence is not uncommon in ART and it is important to assess whether the non-adherence is sufficient to influence virological outcome [57]. At the same time, this secondary analysis coupled with a growing body of scientific literature [56, 58–61] does make a compelling argument. Second, the relatively short period of pharmacy refills examined (6 months) for the 458 participants could suggest that not all participants who have difficulties consistently adhering to ART will be detected [62]. Furthermore, the self-reported questionnaire could introduce a reporting bias into the results since some people may not completely recollect more remote events or may not be completely forthcoming with their responses [63, 64]. Finally, it would be necessary to understand how much VF was defined by patterns of poor adherence using the PDC [65].

An interesting direction for future research in resource-limited settings would be to determine the extent to which patterns in the repeated discrepancies between the observed pharmacy refill pick-ups and the expected pharmacy refill pick-ups over a six-month period can be summarized and modeled. We partially examined these discrepancies via generalized estimating equations and a time-averaged marginal model but found no differences between cases and controls (p=0.3848). The time-averaged approach would be more useful if, for example, cases tended to pick-up medications ahead of the expected date whereas controls tended to pick-up a few days later than expected. But the interesting patterns of pharmacy refills may be more subtle than that. One hypothesis is that long- and short-delays in refill pick-up may be more or less likely to lead to virologic failure depending on the regimen. Long gaps of efavirenz, say, may lead to virologic failure whereas short misses tend to be more easily tolerated due to the long half-life of the drug concentration in the blood. Although we contend that such a pattern analysis would be interesting, it was somewhat tangential to the main objective of finding an optimal combination of adherence measures that was highly correlated with virologic failure in our study, and therefore we did not pursue it beyond the aforementioned marginal models.

In conclusion, PCAR and self-reported questionnaires were feasible and accurate in discriminating cases from controls in this setting. It will be important to validate these results in large prospective studies as well as to further refine and optimize the algorithm of the ratio.