Abstract
Purpose
The purpose of this study was to determine whether diary-driven adjustment of Medication Event Monitoring System (MEMS) data based on supplementary information strengthens the relationship between measured antiretroviral medication adherence and plasma HIV viral load (VL).
Methods
HIV+ adolescents on antiretroviral treatment were monitored with MEMS for 30 days preceding a VL measurement. The primary outcome was VL ≥400 copies/ml. Handwritten diaries were used to comprehensively record deviations from recommended use (bottle opened but dose not taken or bottle not opened and dose taken). Data were adjusted (“cleaned”) based on diary events. Data were “capped” at the prescribed number of doses/day. Receiver operator characteristic analysis compared the relationships between 1) Raw MEMS data, 2) diary-cleaned, 3) capped, or 4) cleaned and capped MEMS data and VL.
Results
Over 30 days preceding VL measurements, 273 adolescents had 465 diary events. Capping resulted in fewer patients classified as 95% adherent (65.2%) compared to raw data (71.4%), p<0.001. Adherence was highly associated with VL (OR 1.05, p<0.001). The area under the ROC curve for continuous adherence compared to VL was 0.89 (95% CI: 0.82 – 0.95). Neither diary-cleaning, capping, nor cleaning and capping MEMS data significantly altered the association between adherence and VL (p = 0.14, 0.40, and 0.19, respectively).
Conclusion
MEMS data-cleaning based on diary entries did not affect the adherence-VL relationship.
Keywords: adherence, electronic monitoring, MEMS, self-report, HIV
Introduction
Electronic drug monitoring with devices such as Medication Event Monitoring System caps (MEMS) is considered a “gold standard” for measuring adherence.1 Each time a MEMS cap is opened, a microelectronic monitor records the date and time as a presumed dose.2,3 MEMS monitoring is subject to limitations including misclassification of adherence due to patients opening bottles without taking doses and patients taking out multiple doses at a time to avoid having to carry the monitoring devices with them. MEMS data underestimates adherence when patients remove one or more doses to take at a later time.4,5 Conversely, MEMS-data may overestimate adherence when the bottle is opened but no medication is taken, as happens with pharmacy refills or “curiosity openings.”2 Cleaning data based on patient-reports of “irregular” MEMS use would be expected to provide more accurate measures of adherence if all reports are true, but could decrease accuracy if patients report taking doses that were not actually taken.2
For people infected with the human immunodeficiency virus (HIV), excellent adherence to antiretroviral therapy (ART) is critical to achieving virologic suppression, which in turn is associated with survival. HIV-infected adolescents are especially vulnerable to poor adherence, poor virologic outcomes, and HIV-related mortality compared to younger children and adults.6–8 At the same time, more perinatally HIV-infected children are surviving into adolescence than ever before.9 In 2012, the World Health Organization (WHO) reported that approximately 2.1 million adolescents worldwide are infected with HIV, of whom more than 80% live in sub-Saharan Africa.10–12 Poor adherence may result in treatment failure and the development of viral resistance, a problem that is particularly devastating in resource-limited settings with few treatment options.13 Adherence monitoring is important both for research and for clinical care. In research, accurate monitoring allows for a clearer understanding of the mechanisms of treatment failure. In clinical care, an understanding of adherence patterns can allow for interventions to improve outcomes.
Adherence values calculated from MEMS data have been shown to better correlate with HIV-RNA levels compared to other measures.3 However, little is known about whether the accuracy of MEMS data for HIV treatment is improved or hindered by comprehensive data-cleaning strategies. Social desirability bias may lead patients to intentionally misuse MEMS caps. For example, study participants using MEMS have described “cheating,” that is, opening their MEMS caps without taking a dose, out of fear of health care provider disapproval of poor adherence.14 In general, adolescents and children are more susceptible to social desirability bias than adults.15–17 Consequently, adolescents may be more at risk of such deception.
MEMS data can be adjusted in various ways, including adding or removing electronically captured “doses” based on patient reports or by capping the number of doses to limit factitious “doses.” However, the effect of implementing different data-cleaning strategies on the accuracy of MEMS data capture is unknown. We undertook this study to clarify whether a comprehensive data cleaning strategy based on self-reported MEMS use irregularities and/or capping of MEMS data at prescribed doses/day could improve MEMS adherence accuracy among HIV-infected adolescents.
Methods
We included participants from an on-going prospective cohort study examining antiretroviral therapy (ART) adherence among 300 HIV-infected adolescents (10 – 19 years) with confirmed HIV infection who are being treated at the Botswana-Baylor Children’s Clinical Centre of Excellence in Gaborone, Botswana.18 All patients had been on treatment for at least 6 months prior to the start of monitoring. To increase the likelihood that adherence to therapy would be associated with virologic response, we excluded 1) patients with genotypic evidence of high level resistance to their current regimen and 2) patients awaiting resistance tests. Eligible participants had at least one plasma HIV viral load (VL) drawn between 61 days after initiation of monitoring and the date of their Month 6 visit. Subjects must have had MEMS data for the 30 days preceding their most recent (month 6 or month 3) VL.
MEMS Data and Diaries
All participants were given a MEMS® 6 monitor (MeadWestvaco/AARDEX, Richmond, VA USA)8 and a MEMS diary (Appendix I) and were instructed to only open the bottle with the MEMS cap at the times of dose-taking and pharmacy refills. If the cap was opened and a dose was not taken or the cap was not opened and a dose was taken, the participants were instructed to record the date, time, and a brief description of the event in the MEMS diary in their primary language (Setswana or English). The staff at the study site was blinded to the MEMS data which were uploaded directly to the off-site study team (MedAmigo software, MWV, Richmond, VA USA) 3-monthly. MEMS diaries were uploaded into a secure web-based database.19
Events recorded in the MEMS diaries were coded as excluded events (E), inserted events (N), or uninterpretable events (U). Excluded events were those in which the monitored bottle was opened but no pills were removed and/or taken, while inserted events were those in which the MEMS bottle was not opened but pills were taken. Un-interpretable events, which do not affect data cleaning, are events that are not recorded clearly or completely enough to be coded as excluded or inserted (e.g. event recorded without a date).
Exposure and Outcome Variables
Mean percent adherence was calculated for each individual patient for 30 days prior to the VL drawn on their Month 6 visit date. If no VL was drawn on at the Month 6 visit date, the last VL before the Month 6 visit date was used as long as it was at least 61 days after study entry. Only VLs drawn at least 61 days following commencement of MEMS monitoring were included in the analyses to account for a potential Hawthorne Effect. The Hawthorne effect suggests that individuals in a study behave differently due to their sensitivity to being observed.20 A three-month study by Deschamps et al. proposed that the MEMS caps themselves may function as a short-term intervention, significantly improving average adherence rates of adult patients for approximately 40 days. After this time, the “intervention effect” is thought to decline until adherence rates plateau at levels more indicative of pre-study behavior.21 In this study, the inclusion of MEMS monitors might have amplified the effect, despite blinding of the patients, parents, and study team members with patient-contact to the MEMS data. If a patient with poor adherence and a detectable VL pre-study had excellent adherence at the beginning of the study due to the Hawthorne Effect, adherence may have been a worse predictor of VL if the virus did not have adequate time to reach undetectable levels with short-term improved adherence.
A detectable VL was defined as ≥ 400copies/mL. Raw percent adherence values included all events recorded in the MedAmigo database in the specified interval (Equation 1).
Equation 1.
Raw Percent Adherence calculation
Diary-cleaned percent adherence values were generated by subtracting excluded events from and adding inserted events to the total number of MedAmigo events (Equation 2).
Equation 2.
Diary-cleaned Percent Adherence calculation
Data were also cleaned by “capping” data at prescribed doses per day, where a “day” is defined as 3:00 AM – 2:59 AM local time (e.g. for a patient following a twice daily regimen, total doses per day could not exceed 2). Capped mean percent adherence could not exceed 100% (Equation 3).
Equation 3.
Capped Percent Adherence calculation
“Cleaned-capped” percent adherence values were calculated in the same manner as diary-cleaned percent adherence except the numerator could not exceed prescribed doses/day. Adherence was defined as the mean percent adherence in the 30 days prior to the VL measurement and was dichotomized at ≥95% = “optimal” versus <95% = “suboptimal.”22
Statistical Analyses
Normality was determined graphically and using a Shapiro-Wilk test. For nonparametric data, Wilcoxon Signed Rank Test determined whether there was a significant difference between the percent adherence values calculated from raw MEMS data compared to those calculated from cleaned and/or capped MEMS data. T-tests were planned for parametric data. Kruskal-Wallis test followed by Dunn’s test was used for multiple comparisons between adherence measures for non-parametric data. Cohen’s kappa measured agreement between dichotomized (adherent vs. non-adherent) raw and cleaned and/or capped adherence variables. McNemar’s test assessed the difference in proportion of patients classified as adherent or non-adherent before and after data cleaning and/or capping.
The sensitivity and specificity for correctly classifying “optimally adherent” vs. “sub optimally-adherent” patients as virologically suppressed/not virologically suppressed was calculated. As a secondary analysis, a lower cut-point (≥90% = “optimal” versus <90% = “suboptimal”) was compared. Pearson’s chi-squared tests assessed the relationship between dichotomized adherence variables and VL. Spearman’s test was planned for parametric data. Logistic regression was used to assess the odds of virologic failure based on adherence. ROC analyses were used to examine the relationship between continuous adherence and VL. The AUC of cleaned and/or capped adherence measures were compared to the AUC of the raw adherence measure.1
All analyses were performed with a two-sided alpha-level of 0.05 using STATA/SE 12.1 software23. The study had 80% power to detect a difference of 3% or more between groups. The study was approved by Institutional Review Boards at University of Pennsylvania, Baylor College of Medicine, and the Health Research Development Committee (HRDC) of Botswana.
Results
Three hundred patients were enrolled in the source cohort. The final sample size for this sub-study was 273 patients (Figure 1). Confirmed or suspected non-suppressible virus status resulted in the exclusion of 3 and 8 patients, respectively. Two patients were excluded due to unusual circumstances that resulted in lack of MEMS data for the monitored period. Fourteen patients were excluded because a VL was not available during the required time period. Table 1 summarizes participant characteristics.
Figure 1.
Sample size flow chart (VL = viral load; GART = genotypic antiretroviral resistance test; MEMS = medication event monitoring system)
Table 1.
Demographic and baseline characteristics of study participants
| Full Cohort (N = 300) | Eligible Participants (N = 273) | Adherenta (N = 195) | Non-Adherent (N = 78) | P-valueb | |
|---|---|---|---|---|---|
| Age (years) | 13.4 (11.8 – 15.6) | 13.2 (11.8 – 15.5) | 12.9 (11.5 – 14.8) | 14.5 (12.6 – 16.9) | < 0.01 |
| Time on antiretroviral therapy (years) | 7.5 (5.3 – 8.8) | 7.5 (5.3 – 8.7) | 7.5 (4.6 – 8.7) | 7.9 (6.4 – 8.8) | 0.29 |
|
| |||||
| Gender | |||||
| Female | 158 (53%) | 143 (52%) | 105 (54%) | 38 (49%) | 0.90 |
| Orphan | 40 (13%) | 36 (13%) | 27 (14%) | 9 (12%) | 0.97 |
| Attending School | 279 (93%) | 254 (93%) | 186 (95%) | 68 (87%) | 0.12 |
|
| |||||
| Mode of HIV Acquisition | > 0.99 | ||||
| Perinatal | 233 (78%) | 211 (77%) | 149 (76%) | 62 (79%) | |
| Breastfeeding | 51 (17%) | 48 (18%) | 36 (18%) | 12 (15%) | |
| Blood Transfusion | 6 (2%) | 5 (2%) | 2 (1%) | 3 (4%) | |
| Rape | 3 (1%) | 3 (1%) | 3 (2%) | 0 | |
| Unknown | 7 (2%) | 6 (2%) | 5 (3%) | 1 (1%) | |
|
| |||||
| Disclosure Statusc | < 0.01 | ||||
| Disclosed | 195 (65%) | 176 (64%) | 121 (62%) | 55 (71%) | |
| Non-disclosed | 105 (35%) | 97 (36%) | 74 (38%) | 23 (29%) | |
|
| |||||
| HAARTd Category | 0.15 | ||||
| NNRTI-based | 150 (50%) | 135 (49%) | 90 (46%) | 30 (38%) | |
| PI-based | 150 (50%) | 138 (51%) | 105 (54%) | 48 (62%) | |
|
| |||||
| Pre-HAART WHO Clinical Stage | > 0.99 | ||||
| Asymptomatic (1) | 24 (8%) | 20 (7%) | 16 (8%) | 4 (5%) | |
| Mild (2) | 35 (12%) | 32 (12%) | 24 (12%) | 8 (10%) | |
| Advanced (3) | 136 (45%) | 125 (46%) | 89 (46%) | 36 (46%) | |
| Severe (4) | 102 (34%) | 93 (34%) | 64 (33%) | 29 (37%) | |
| Unavailable | 3 (1%) | 3 (1%) | 2 (1%) | 1 (1%) | |
|
| |||||
| Pre-HAART WHO Immunodeficiency Classification | 0.97 | ||||
| None (1) | 19 (6%) | 17 (6%) | 11 (6%) | 6 (8%) | |
| Mild (2) | 76 (25%) | 70 (26%) | 53 (27%) | 17 (22%) | |
| Advanced (3) | 67 (22%) | 59 (22%) | 46 (24%) | 13 (17%) | |
| Severe (4) | 132 (44%) | 121 (44%) | 82 (42%) | 39 (50%) | |
| Unavailable | 6 (2%) | 6 (2%) | 3 (2%) | 3 (4%) | |
|
| |||||
| WHO Clinical Staging Classification at Study Entry | > 0.99 | ||||
| Asymptomatic (1) | 267 (89%) | 242 (89%) | 175 (90%) | 67 (86%) | |
| Mild (2) | 4 (1%) | 4 (1%) | 3 (2%) | 1 (1%) | |
| Advanced (3) | 17 (6%) | 16 (6%) | 11 (6%) | 5 (6%) | |
| Severe (4) | 12 (4%) | 11 (4%) | 6 (3%) | 5 (6%) | |
Classification based on raw data;
Comparison of all 4 groups, Kruskal-Wallis for continuous variables and χ2 for discrete variables;
defined as whether or not the adolescent had been made aware of his/her own HIV infection by the family;
Highly active antiretroviral therapy (2-drugs in the nucleoside reverse transcriptase inhibitor family + one non-nucleoside reverse transcriptase inhibitor (NNRTI) or one protease inhibitor (PI)
Data Cleaning Based on Diary Entries
Data cleaning required approximately 10 minutes per diary after translation of comments made in the local Setswana language and uploading of diaries into the electronic system. This resulted in over 100 hours spent cleaning per 3 month monitoring interval. Of the total events (n = 1156) coded for Month 3 diaries, 479 (41%) were un-interpretable events. Of the remaining events, 611 (90%) were excluded events and 66 (10%) were inserted. Of the total events (n=1154) coded for Month 6 diaries, 347 (30%) were un-interpretable events. Of the remaining events, 718 (89%) were excluded events and 89 (11%) were inserted.
Comparing Continuous Adherence Variables
Descriptive statistics for all adherence measures are summarized in Table 2. Continuous adherence variables were left-skewed (raw: MO = 101.6, cleaned: MO = 100.0, capped: MO = 100.0, clean-capped: MO = 100.0). Calculated median (IQR) percent adherence was 101.6 (91.9 – 103.2) using raw data, 100.0 (90.3 – 101.6) using diary-cleaned data, 98.4 (87.1 – 100) using capped data, and 100 (90.3 – 100) using data that was both diary-cleaned and capped. Raw data were statistically significantly different from diary-cleaned (p < 0.001), capped (p < 0.001), and clean-capped data (p < 0.001). However, capped data were not significantly different from clean-capped data (p = 0.06). The absolute difference between the raw median percent adherence and the cleaned, capped, and clean-capped percent adherence values was small, ranging from 1.6% to 3.2%.
Table 2.
Comparison of percent adherence values calculated from raw and adjusted medication event monitoring system (MEMS) data
| Adherence Variables | Mean (SD) % | Median (IQR) % | P-valuea | Multiple Comparisonb |
|---|---|---|---|---|
| Raw | 91.2 (22.5) | 101.6 (91.9–103.2) | A | |
| Cleaned | 89.5 (22.5) | 100.0 (90.3–101.6) | < 0.001 | B |
| Capped | 87.7 (22.1) | 98.4 (87.1–100) | < 0.001 | C |
| Clean-capped | 88.7 (22.0) | 100 (90.3–100) | < 0.001 | C |
Wilcoxon signed-rank test,
Kruskal-Wallis/Dunn’s test for all comparisons between raw, cleaned, capped, and clean-capped percent adherence values. All pair-wise comparisons occur simultaneously. Letters indicate significant differences. A- raw is different from cleaned, capped and clean-capped. B-cleaned is different from capped. C-capped is not different from clean-capped.
Comparing Dichotomous Adherence Variables
When optimal adherence was dichotomized at 95%, diary-cleaned data agreed 100% with clean-capped data. Thus, for dichotomized adherence analyses, clean-capped data results are not reported separately. Raw dichotomized adherence data had high agreement24 with dichotomized diary-cleaned (kappa = 0.92, 95% CI: 0.87 – 0.97) and capped (kappa = 0.86, 95% CI: 0.79 – 0.92) data. Diary-cleaning did not cause a significant change in the proportion of patients classified as adherent or non-adherent (p = 0.10). However, capped data classified significantly fewer (65%) patients to be classified as adherent compared to raw data (71%, p < 0.001) (Table 3).
Table 3.
Changes in adherence classification from using raw and adjusted medication event monitoring system (MEMS) data, where ≥95% adherence is “adherent” and <95% adherence is “non-adherent”
| Variable | Adherent | Non-Adherent | Test statistica | P-valuea |
|---|---|---|---|---|
| Raw | 195 | 78 | ||
| Cleaned | 190 | 83 | 2.78 | 0.10 |
| Capped | 178 | 95 | ||
| (Clean-capped)* | 17.00 | < 0.001 |
McNemar’s Test tests the null hypothesis that the number of patients classified as adherent and non-adherent using raw data is the same as the number of patient classified as adherent and non-adherent using the cleaned, capped, and clean-capped data
Note: results of capped and cleaned-capped identical
Comparing Adherence Variables to VL
There was a highly significant association between dichotomized adherence and VL for all adherence measures with the odds of virologic failure with adherence <95% vs. ≥95% ranging from 19 (95% CI: 6–65) with capped data to 27 (95% CI: 8–94) with raw data. Considering adherence as a continuous variable resulted in an odds ratio of 1.05 (p<0.0001) for all measures of adherence. The area under the ROC curve (AUROC) for continuous raw adherence compared to VL was 0.89 (95% CI: 0.82 – 0.95). The AUROC for continuous diary-cleaned and capped adherence compared to VL was 0.90 (95% CI: 0.84 – 0.96) and 0.89 (95% CI: 0.82 – 0.96), respectively. Neither cleaning (p = 0.14), capping (p = 0.40), nor clean-capping (p = 0.19) significantly altered the adherence-VL relationship (Figure 2). The percent of patients for whom adherence <95% was associated with lack of virologic suppression was 88.5% (69.8–97.6) for all adherence measures. Those patients for whom the classification (“optimal adherence” vs. “not optimal adherence” at the 95% cut-point) varied based on data cleaning and/or capping were all virologically suppressed. The percent of patients for whom adherence ≥95% was associated with virologic suppression for raw, diary-cleaned, capped, and clean-capped data was 78.1% (95% CI: 72.3–83.1), 76.0% (95% CI: 70.2–81.2), 71.1% (95% CI: 65.0–76.7), and 76.0% (95% CI: 70.2–81.2) respectively. Lowering the “optimal adherence” cut-point to 90% lowered the percent for whom suboptimal adherence (<90%) was associated with lack of virologic suppression for raw data to 66.7% (95% CI: 44.7–84.4) and to 79.2% (57.9–92.9) for all diary-cleaned, capped, and clean-capped data. Sensitivity and specificity of ROC analyses at 95%, 90% and 80% cut-points are displayed in Table 4.
Figure 2.
Receiver operating characteristic (ROC) curves exhibiting the relationship between raw, diary-cleaned, capped, and clean-capped medication event monitoring system (MEMS) data and viral load (VL)
Table 4.
| Adherence Measure | Cut-point | % Sensitivity | % Specificity | Correctly Classified (n = 273) | % Correctly Classified |
|---|---|---|---|---|---|
| Raw | 95% | 78.1 | 88.5 | 216 | 79.0 |
| Clean | 76.0 | 88.5 | 211 | 77.2 | |
| Capped | 71.1 | 88.5 | 199 | 72.8 | |
| Clean-capped | 76.0 | 88.5 | 211 | 77.2 | |
|
| |||||
| Raw | 90% | 84.2 | 76.9 | 228 | 83.5 |
| Clean | 82.1 | 84.6 | 225 | 82.4 | |
| Capped | 80.1 | 84.6 | 220 | 80.5 | |
| Clean-capped | 82.1 | 84.6 | 225 | 82.4 | |
|
| |||||
| Raw | 80% | 89.0 | 69.2 | 238 | 87.1 |
| Clean | 88.2 | 69.2 | 236 | 86.4 | |
| Capped | 87.4 | 69.2 | 234 | 85.7 | |
| Clean-capped | 88.2 | 69.2 | 236 | 86.4 | |
Discussion
Although some studies attempt to account for MEMS data irregularities through tools such as in-depth interviews and patient diaries, data cleaning methodology is not standard across studies and sometimes cleaning does not occur at all.25,26 A study by Bangberg et al. that found that data from an electronic monitor that had been “adjusted” by patient self-report led to less misclassification compared to unadjusted data.27 However, the Bangsberg study compared adjusted electronic monitoring results to unannounced pill counts, not viral load. Pill counts, like MEMS data, can be altered by patients seeking to hide non-adherence. Viral loads are more objective measures of successful treatment.
Not Enough Juice for the Squeeze
This study found no significant advantage to cleaning data obtained from MEMS bottles, despite approximately 100 hours dedicated to this process for every 3-month interval for a cohort of 300 patients. At the same time, concerns that the use of diaries might defeat the objective nature of MEMS were not justified by our results. Incorporating subjective data into the mechanically-collected dataset did not significantly weaken the relationship between adherence and VL although the data were collected from adolescents, the age group most susceptible to biasing study results towards socially-desirable responses.15,16 While we had some concern that incorporating diary data could actually weaken the adherence:VL relationship due to attempts to make adherence appear better than it actually was, patient “cheating” appeared to be minimal. Diary cleaning and capping would be expected to be most useful in situations in which patients are “misusing” their MEMS caps (i.e. taking out multiple doses to be taken at multiple time points with a single bottle opening or opening the bottle when they are not taking doses).
In all cases, adherence was significantly associated with VL. With an AUC near 0.89 for the comparison of raw MEMS data to virologic failure, there was little room for improvement. MEMS alone appears to have captured nearly true adherence. All adherence measures showed comparable sensitivity and specificity at the 95% adherence threshold. The specificity of the test (i.e. correctly predicting an adherent patient to have an undetectable VL) is of particular clinical importance in resource-limited settings due to the relative difficulty of obtaining VL tests.28 If virologic failure can be effectively ruled out based on adherence, researchers and clinicians can more rationally prioritize which patients would benefit most from having a VL drawn. The more conservative estimate of who is adherent, derived with the simple step of capping data, may be preferable to reduce the risk of misclassifying patients with borderline adherence.
Clinical versus Statistical Significance
Although there were several statistically significant differences between the raw data and the cleaned and/or capped data, the clinical significance of these differences is marginal, at best. Near perfect antiretroviral adherence should always be the goal, yet viral suppression may be achieved at adherence rates much lower than the conventional 95% benchmark.29 In our study, lowering the adherence threshold to 90% did not improve the ability to discriminate those without virologic suppression. Recent studies have shown that patterns of adherence and non-adherence may be even more important than average adherence for predicting virologic outcome. For example, Oyugi et al. found that treatment interruptions that exceeded two days were significantly associated with non-nucleoside reverse transcriptase inhibitor drug resistance.30 Parienti et al. found that, for patients with <80% mean adherence, the number of consecutive days of treatment interruption was significantly associated with virologic rebound (p < 0.02) while average adherence was not (p = 0.65).31 These studies demonstrate that adherence ought to be thought of in a more sophisticated way than strict dichotomous thresholds. Thus, relatively small changes in the calculated mean adherence resulting from cleaning and/or capping the raw MEMS data are unlikely to be of clinical importance.
Study Limitations
These results may only be generalized to similar cohorts (HIV+ adolescents living in a resource-limited setting). However, since adolescents are generally more-likely than adults to give socially-desirable rather than true answers,15–17 the fact that cleaning and capping data from adolescents didn’t appreciably improve or worsen MEMS adherence values is meaningful. Eight patients were excluded due to pending unavailability of resistance test results and 14 patients were excluded simply due to lack of VL in the appropriate time interval. Moreover, a notable number of events (approximately 40% at the 3 month study visit and 30% at the 6 month study visit%) were considered un-interpretable and were excluded from the analysis because they lacked sufficient detail to allow for data cleaning. Examples of unverified events include those that had illegible dates or explanations and those that described “extra” bottle openings that did not correspond with openings recorded by the MEMS caps. Collecting reliable diary data requires a high level of commitment from study participants.5 Many study participants provided extremely detailed information (e.g. “bottle opened at 06:31 and dose not taken until 06:43”). However, for other participants there was repeated reporting of unverifiable information (e.g. reported openings that did not correspond with MEMS openings) even with repeated coaching from the research team.
Conclusion
Our data suggest that adherence researchers should cap MEMS data at prescribed doses per day as this method does not weaken the adherence-outcome relationship but may confer improved specificity. Cleaning data with patient diaries, however, provides no clear advantage and it is a time-consuming endeavor. By eliminating routine diary-cleaning from protocols, researchers and participants can save time and effort without sacrificing data accuracy.
Key Points.
Modifying MEMS data with prospectively-collected self-reports of deviations from recommended MEMS cap use did not improve the relationship between adherence and HIV viral load relative to unadjusted data
Capping MEMS data at prescribed doses per day classified significantly fewer patients as adherent compared to unadjusted data, but did not strengthen the relationship between adherence and viral load outcome.
Efforts to “improve” microelectronic adherence data capture with supplemental data may not improve data quality.
Acknowledgments
Sponsors of Research and Grant Number(s): This project was funded in part through NIH career development award (Lowenthal) K23 MH095669 and through a CDC-PEPFAR Public Health Effectiveness Study Grant. Dr. Gross receives funding from the Penn Center for AIDS research (P30 AI 045008). Part of Jessica Eby’s work was funded through the Children’s Hospital of Philadelphia Research Institute Summer Scholars Program (CRISSP).
Footnotes
Conflict of Interest Statement: The authors have no conflict of interest to declare.
Statement about Prior Postings and Presentation: A poster related to this work was presented at the CRISSP poster session. No other publications or presentations of these data have preceded this manuscript submission.
References
- 1.DeLong E, DeLong D, Clarke-Pearson D. Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach. Biometrics. 1988;44:837. doi: 10.2307/2531595. [DOI] [PubMed] [Google Scholar]
- 2.Berg K, Arnsten J. Practical and Conceptual Challenges in Measuring Antiretroviral Adherence. JAIDS Journal of Acquired Immune Deficiency Syndromes. 2006;43:S79–S87. doi: 10.1097/01.qai.0000248337.97814.66. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Arnsten J, Demas P, Farzadegan H, et al. Antiretroviral Therapy Adherence and Viral Suppression in HIV-Infected Drug Users: Comparison of Self-Report and Electronic Monitoring. Clinical Infectious Diseases. 2001;33:1417–1423. doi: 10.1086/323201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Lyimo R, van den Boogaard J, Msoka E, et al. Measuring adherence to antiretroviral therapy in northern Tanzania: feasibility and acceptability of the Medication Event Monitoring System. BMC Public Health. 2011;11:92. doi: 10.1186/1471-2458-11-92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Bolger N, Davis A, Rafaeli E. Diary Methods: Capturing Life as it is Lived. Annu Rev Psychol. 2003;54:579–616. doi: 10.1146/annurev.psych.54.101601.145030. [DOI] [PubMed] [Google Scholar]
- 6.Lamb M, Fayorsey R, Nuwagaba-Biribonwoha H, et al. High attrition before and after ART initiation among youth (15–24 years of age) enrolled in HIV care. AIDS. 2014;28:559–568. doi: 10.1097/qad.0000000000000054. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Nachega J, Hislop M, Nguyen H, et al. Antiretroviral Therapy Adherence, Virologic and Immunologic Outcomes in Adolescents Compared With Adults in Southern Africa. JAIDS Journal of Acquired Immune Deficiency Syndromes. 2009;51:65–71. doi: 10.1097/qai.0b013e318199072e. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Cramer J. How Often Is Medication Taken as Prescribed? JAMA. 1989;261:3273. doi: 10.1001/jama.1989.03420220087032. [DOI] [PubMed] [Google Scholar]
- 9.Mofenson L, Cotton M. The challenges of success: adolescents with perinatal HIV infection. Journal of the International AIDS Society. 2013:16. doi: 10.7448/ias.16.1.18650. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.WHO. HIV and adolescents: HIV testing and counselling, treatment and care for adolescents living with HIV: policy brief. Geneva: WHO Press; 2013. [Accessed March 7, 2015]. pp. 1–12. Available at: http://apps.who.int/iris/bitstream/10665/94561/1/9789241506526_eng.pdf?ua=1. [Google Scholar]
- 11.UNAIDS. AIDS by the numbers. Geneva: UNAIDS; 2013. [Accessed March 7, 2015]. pp. 1–12. Available at: http://www.unaids.org/sites/default/files/media_asset/JC2571_AIDS_by_the_numbers_en_1.pdf. [Google Scholar]
- 12.UNICEF. Progress for children: A report card on adolescents. New York: United Nations Children’s Fund (UNICEF); 2012. [Accessed March 7, 2015]. pp. 1–56. Available at: http://www.unicef.org/publications/files/Progress_for_Children_-_No._10_EN_04232012.pdf. [Google Scholar]
- 13.Madi D, Bhaskaran U, Ramapuram J, Rao S, Mahalingam S, Achappa B. Adherence to antiretroviral therapy among people living with HIV. North Am J Med Sci. 2013;5:220. doi: 10.4103/1947-2714.109196. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Safren S, Kumarasamy N, Hosseinipour M, et al. Perceptions About the Acceptability of Assessments of HIV Medication Adherence in Lilongwe, Malawi and Chennai, India. AIDS and Behavior. 2006;10:443–450. doi: 10.1007/s10461-006-9094-6. [DOI] [PubMed] [Google Scholar]
- 15.Brown M, Kodadek S. The use of lie scales in psychometric measures of children. Res Nurs Health. 1987;10:87–92. doi: 10.1002/nur.4770100204. [DOI] [PubMed] [Google Scholar]
- 16.Mwamwenda T. Age Differences in Social Desirability. Psychological Reports. 1995;76:825–826. doi: 10.2466/pr0.1995.76.3.825. [DOI] [PubMed] [Google Scholar]
- 17.Mabe P, Treiber F. Social desirability response tendencies in psychiatric inpatient children. J Clin Psychol. 1989;45:194–201. doi: 10.1002/1097-4679(198903)45:2<194::aid-jclp2270450204>3.0.co;2-2. [DOI] [PubMed] [Google Scholar]
- 18.Botswana National HIV & AIDS Treatment Guidelines. Gabarone: Botswana Ministry of Health; 2012. [Accessed March 7, 2015]. pp. 1–197. Available at: http://www.med.upenn.edu/botswana/user_documents/BotsNatHIV-AIDSTreatGuideWEB22-05-2012.pdf. [Google Scholar]
- 19.Harris PA, et al. Research electronic data capture (REDCap) - A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377–381. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Fernald D, Coombs L, DeAlleaume L, West D, Parnes B. An Assessment of the Hawthorne Effect in Practice-based Research. The Journal of the American Board of Family Medicine. 2012;25:83–86. doi: 10.3122/jabfm.2012.01.110019. [DOI] [PubMed] [Google Scholar]
- 21.Deschamps A, Wijngaerden E, Denhaerynck K, Geest S, Vandamme A. Use of Electronic Monitoring Induces a 40-Day Intervention Effect in HIV Patients. JAIDS Journal of Acquired Immune Deficiency Syndromes. 2006;43:247–248. doi: 10.1097/01.qai.0000246034.86135.89. [DOI] [PubMed] [Google Scholar]
- 22.Gross R, Yip B, Re V, III, et al. A Simple, Dynamic Measure of Antiretroviral Therapy Adherence Predicts Failure to Maintain HIV-1 Suppression. The Journal of Infectious Diseases. 2006;194:1108–1114. doi: 10.1086/507680. [DOI] [PubMed] [Google Scholar]
- 23.StataCorp. Stata Statistical Software: Release 12. StataCorp LP; 2011. [Google Scholar]
- 24.Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Family medicine. 2005;37:360–363. [PubMed] [Google Scholar]
- 25.Fennie K, Bova C, Williams A. Adjusting and Censoring Electronic Monitoring Device Data. JAIDS Journal of Acquired Immune Deficiency Syndromes. 2006;43:S88–S95. doi: 10.1097/01.qai.0000248336.97814.2f. [DOI] [PubMed] [Google Scholar]
- 26.Pearson C, Simoni J, Hoff P, Kurth A, Martin D. Assessing Antiretroviral Adherence via Electronic Drug Monitoring and Self-Report: An Examination of Key Methodological Issues. AIDS and Behavior. 2006;11:161–173. doi: 10.1007/s10461-006-9133-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Bangsberg DR, Hecht FM, Charlebois ED, Chesney M, Moss A. Comparing Objective Measures of Adherence to HIV Antiretroviral Therapy: Electronic Medication Monitors and Unannounced Pill Counts. AIDS and Behavior. 2001;5:275–281. doi: 10.1023/A:1011396711486. [DOI] [Google Scholar]
- 28.Rawizza H, Chaplin B, Meloni S, et al. Immunologic Criteria Are Poor Predictors of Virologic Outcome: Implications for HIV Treatment Monitoring in Resource-Limited Settings. Clinical Infectious Diseases. 2011;53:1283–1290. doi: 10.1093/cid/cir729. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Bangsberg D. Less Than 95% Adherence to Nonnucleoside Reverse-Transcriptase Inhibitor Therapy Can Lead to Viral Suppression. Clinical Infectious Diseases. 2006;43:939–941. doi: 10.1086/507526. [DOI] [PubMed] [Google Scholar]
- 30.Oyugi J, Byakika-Tusiime J, Ragland K, et al. Treatment interruptions predict resistance in HIV-positive individuals purchasing fixed-dose combination antiretroviral therapy in Kampala, Uganda. AIDS. 2007;21:965–971. doi: 10.1097/qad.0b013e32802e6bfa. [DOI] [PubMed] [Google Scholar]
- 31.Parienti J, Das-Douglas M, Massari V, et al. Not All Missed Doses Are the Same: Sustained NNRTI Treatment Interruptions Predict HIV Rebound at Low-to-Moderate Adherence Levels. PLoS ONE. 2008;3:e2783. doi: 10.1371/journal.pone.0002783. [DOI] [PMC free article] [PubMed] [Google Scholar]