Feasibility and Acceptability of Real-time Antiretroviral Adherence Monitoring among Depressed Women living with HIV in the Deep South of the US

Abstract

This study presents feasibility and acceptability data on the use of a real-time wireless electronic adherence monitor (EAM), among African American women living with HIV with co-occurring depression, residing in remote areas of the Southeastern United States. EAM and self-report ART adherence was monitored over an average of 14.8 weeks among 25 participants who were recruited at four HIV clinics in Alabama. Intra-class correlation showed a low degree of concordance between EAM and self-report (ICC = 0.33, 95% bootstrap CI: 0.13, 0.59). 83% of data collected via EAM was transmitted in real-time. Due to technological failures, 11.4% were not transmitted in real-time, but were later recovered, and 5.7% were lost entirely. Acceptability was examined through surveys and qualitative interviews. Results suggest that EAM monitoring is acceptable and feasible in a rural US setting; however, technological difficulties, such as loss of connectivity may impede the device’s usefulness for just-in-time adherence interventions.

INTRODUCTION

Within the United States (US), the Deep South (defined as Alabama, Georgia, Louisiana, Mississippi, and South Carolina) bears a disproportionate burden of new HIV infections and HIV-related deaths and African American women comprise 69% of all new HIV diagnoses among women (1, 2). Furthermore, depression has been identified as one of the strongest predictors of suboptimal adherence to antiretroviral therapy (ART) and engagement in care (3–5). African American women living with HIV (WLWH) who experience co-morbid depressive disorders represent a population highly vulnerable to suboptimal adherence (3, 5–8). With HIV viral load suppression dependent on optimal ART medication adherence, the need for adherence interventions is particularly pressing among African American WLWH and depression, residing in rural areas of the Deep South (1).

ART adherence monitoring interventions have been critical in achieving viral load suppression and optimal health outcomes among people living with HIV (PLWH) (9, 10). However, traditional means of adherence monitoring have numerous limitations. Self-report measures of ART adherence and pill counts are prone to overestimation if social desirability results in over reporting of dosing or disposal of unused pills (11–13). Further, assessing adherence with pharmacy refill data may also overestimate adherence as it does not reflect doses taken, only the number of doses available (14). To increase accuracy, researchers are increasingly looking to electronic adherence monitoring (EAM) devices. Traditional EAM devices record each opening with a time and date stamp, thus EAMs provide a detailed dose to dose record of patient adherence. However, data from traditional EAM devices must be manually downloaded from the device during study/clinic visits, informing researchers and clinicians of non-adherence well after it has occurred. Because virologic rebound may occur within 48 hours of treatment interruption, early detection of adherence lapses is needed to improve HIV-related outcomes (15). Technological advances, which allow for real-time medication monitoring and intervention, provide the opportunity to detect and intervene upon adherence lapses before virologic rebound of drug resistance occurs (16).

Internationally, the real-time wireless transmission of adherence data has been successfully utilized in resource-limited settings for remote/long-distance monitoring (17–21). Experience with real-time ART adherence monitoring in the US, to date, has been limited to two studies with HIV negative participants and one that included a text message based adherence intervention which precluded the authors from examining the acceptability and feasibility of the devices separately from the challenges encountered in delivering the intervention (25–27). Barriers and disadvantages that have been identified in the US context include the underestimation of adherence due to device non-use, the overestimation of adherence due to curiosity openings, and discomfort with electronic monitoring (25–27). Despite these barriers, reviews of the literature conclude that utilizing cellular signals to monitor adherence and deliver real-time adherence reminders is a useful tool for increasing ART adherence, particularly in resource limited settings (20, 23, 24). No studies to date have examined the acceptability and feasibility of EAMs in rural areas of the Deep South where HIV infections are concentrated and often constrained by limited resources (28). This manuscript addresses this gap in literature by presenting data on the feasibility, validity, and acceptability of EAMs among African American WLWH, suffering from depression and living in rural areas of the US Deep South.

METHODS

Data for this investigation were extracted from a larger parent study, Tele-C.A.R.E., which utilized a 2-arm randomized controlled design comparing a culturally adapted cognitive-behavioral-therapy intervention described elsewhere (29), to an attention-matched control group receiving supportive psychotherapy. The goal of the parent study was to reduce depression and increase ART adherence. The current analysis includes all participants from the Tele-C.A.R. E. study and data from an additional 3 participants enrolled in a feasibility/pilot study that was conducted to ensure feasibility and acceptability of the Tele-C.A.R. E. intervention.

Each participant was provided a Wisepill™ 2G (Model RT200) pillbox that held one week’s worth of medication and that tracked when participants opened and closed the device. To ensure comfort with the device and reduce curiosity openings, participants were instructed on how to open, fill, and charge the device at baseline. Additionally, the intervention included guided exercises related to participants’ adherence goals, which included the development of strategies to integrate the Wisepill™ pillbox into their daily routine.

Study Population

Study participants (N=25) included WLWH recruited through comprehensive HIV care clinics located in four rural communities throughout Alabama. Based on Tele-C.A.R.E. study inclusion criteria, participants in this study include those who 1) self-identified as African-American, 2) had been prescribed ARTs for at least two months, 3) were English-language proficient, 4) were over the age of 18, and 5) met the clinical definition of depression per the M.I.N.I. diagnostic instrument (30). Participants with active or untreated mental health related disorders (other than depression) and those who had participated in cognitive behavioral therapy or an intensive adherence intervention in the past year were excluded from study participation. On average, participants took 14.8 weeks to complete the parent study intervention. Two participants were temporarily lost to follow up, though both participants were later reengaged and completed the parent study. This period of disengagement resulted in missing data for medication adherence and thus the period of disengagement was excluded from the current analysis.

Technological Feasibility Measures

Two types of data messages, hereinafter referred to as “events”, were recorded and wirelessly transmitted by the EAM device: medication events and control events. The EAM dispenser records a data message to a central webserver every time the dispenser is opened. Data messages indicating the opening of the device are categorized as “medication events” and serve as a proxy measure of ART adherence. The second type of data message is the control event (referred to as the device heartbeat) and contains information about battery life and signal connectivity.

Given sufficient battery and cellular connectivity, the EAM device immediately transfers event data to the server. However, if the device battery is low (but not dead) or if there is no network access at the time of the data recording, event messages (medication events and control events) are temporarily stored within the device’s internal memory and can be transmitted to the server once battery power is restored and/or connectivity is reestablished. Events that were not delivered in real-time but transmitted later are considered “delayed events.” Once data has been transmitted to the server, (in real time or delayed) it becomes immediately accessible to research staff through a secure Internet interface. An expected event for which no data exists, (delayed or real-time) is referred to as a “lost event.” Lost events represent technological failures such that events are neither recorded in temporary memory nor relayed in real time.

Comparison of Adherence Measures

Participants chose a daily two-hour target window for optimal ART intake during their baseline visit, with the target time adjusted as needed at weekly visits. Three participants were prescribed medications that were dosed twice daily and thus were asked to identify the dose they found most difficult to take. This is the dose that was recorded for adherence measures.

Self-Report: Data collected included whether participants took their ART each day, and if so, what time they took it. Prior to each telemedicine-delivered psychotherapy session, the study coordinator printed out an adherence report from the Wisepill database and compared recorded events against participant’s self-reported events in a timeline follow-back manner. Using a calendar, participants provided a retrospective account of their ART adherence since their last session. Participants were also asked to report any openings of the Wisepill™ device not related to their adherence (i.e. refilling box or accidental openings).

Self-reported timely adherence is defined as a participant reporting that she had taken her ART medication within the 2-hour target window. Daily adherence is defined as taking ART medication within the 24-hour period around their target time, regardless of whether or not it was taken during the 2-hour target window. If the participant reported not taking any ARTs during the 24-hour period, this is counted as a missed dose.

EAM: As with self-report measures, EAM timely adherence is defined as a medication event within the 2-hour target window and EAM daily adherence is defined as a medication event within the 24-hour period around their target time, regardless of whether or not the medication event occurred during the 2-hour target window. An EAM reported missed ART dose is defined as the absence of a reported medication event (real-time or delayed), in the presence of a control event that indicated proper device functionality.

Acceptability and Feasibility Measures

During the six-month follow-up visit, participants completed an adapted 20 item feasibility and acceptability questionnaire (31), and were interviewed, using a qualitative semi-structured guide to gather more information about their experiences and attitudes towards all aspects of the parent study: counselling sessions, HIV adherence counselling sessions, adherence monitoring with the real-time wireless EAM device, and the use of Tele-Medicine for delivery of counselling services. Three items of the feasibility and acceptability questionnaire specifically assessed feasibility and acceptability of the EAM device. These three items asked participants to indicate the extent to which they agreed with the following statements, using Likert scale responses (completely agree - completely disagree): “Using the WisePill device was easy,” “Using the WisePill device helped me to take my medication,” and “Refilling the WisePill with my medication was easy.” Similarly, structured interview guides covered topics related to all aspects of the parent study. Specific to feasibility and acceptability of the EAM device, qualitative interview topics included ease or difficulty of use of the EAM device; the effects of the EAM device on adherence; and perceived benefits and drawbacks of using the EAM device to monitor ART adherence.

Analyses

Descriptive statistics were used to summarize the demographic and background characteristics of participants. Summary statistics were computed for self-report adherence, EAM adherence per participant, and technological feasibility measures described above. To assess concordance between a real-time wireless EAM and self-report adherence measures, we estimated intra-class correlations with 95% bootstrap confidence intervals (1000 resamples). These analyses were conducted in R 3.2.0 (25). Two generalized linear mixed models were used to estimate and test differences in proportion of adherence (timely and daily, respectively) between the EAM and self-report. The modeling approach allowed for the inclusion of a covariance structure among repeated measurements on the same individuals.

Semi-structured interviews were audio recorded and transcribed verbatim. A thematic analysis, informed by a general inductive methodology (32), of the data was conducted using NVivo 10 software. The data analysis process began with a familiarization with the data and topical coding, which included listening to recordings and reading transcripts in their entirety and the application of broad codes which represented aspects of the parent study (counselling sessions, HIV adherence counselling sessions, EAM device, and the use Tele-Medicine for delivery of counselling services) to each interview excerpt. To assess acceptability and feasibility of the EAM device, all text segments coded “EAM device” were then independently coded by two investigators using open coding methods, allowing codes to emerge from the data. Finally, codes were organized into themes and subthemes and the finalized coding structure was applied to all transcripts. Throughout the coding process, the two coders regularly met to discuss codes and discrepancies and resolved by consensus opinion or by the creation of new, mutually agreeable, codes/themes.

Ethical Review

This study was approved by the Institutional Review Board at the University of Alabama at Birmingham.

RESULTS

Participant characteristics are presented in Table I. All 25 participants were African American women who were living in primarily rural areas of Alabama. The median participant age was 45.7 years (range 25.1–65.7 years), with the majority (68%) of participants living without a partner (i.e., single, divorced, separated, or widowed) and 80% of participants reported a high school education or more. Over half of the sample (52%) reported an annual income of less than $10,000. Eighty-eight percent of participants were virally suppressed at baseline, defined as plasma HIV RNA viral load less than 200 copies/ml. Eight percent of participants had enrollment baseline CD4 counts less than 200 cells/μL. The mean time since HIV diagnosis was 8.1±7 years and the mean time on ART medications was 6.5±5.7 years. Eighteen participants (72%) had previously been diagnosed with depression by a healthcare professional, with a mean depression symptoms score of 36.7±9.0 across the entire sample.

Table I.

Demographics (N=25)

Variable	N/Meana	%/SDa
Age (years)	45.7	11.9
Education
< High school	5	20%
High school graduate/received GED	6	24%
Some college/Technical training	11	44%
College graduate	3	12%
Marital Status
Single	7	28%
Married/cohabitating	8	32%
Separated/divorced/widowed	10	40%
Household income <$10,000	13	52%
Years since HIV diagnosis	8.1	7
Years on ART	6.5	5.7
CD4 countb
<200 cells/µl	2	8%
HIV viral loadb
<200 copies/ml	22	88%
Prior Depression Diagnosis	18	72%
Center for Epidemiologic Studies –Depression (CES-D) scale score	36.7	9.0

^aMean and standard deviations presented for continuous variables, frequency and percent presented for categorical variables

^bMeasured at baseline

Technological Feasibility of EAM Use

During the study period, a total of 2,566 events were expected for all participants combined, assuming one dosage per day. A summary of observed technical problems is presented in Table II. Approximately 83% (2,129) of EAM data was recorded via wireless cellular network in real time. Loss of real-time events (n=437) included 253 losses attributed to signal lapses and 184 losses attributed to dead battery; however, the majority of these events (n=292; 67%) were classified as delayed events because the event data was retrieved once connectivity was restored or the device battery was charged again. Complete technical failures resulted in a loss of 145 events (5.7%).

Table II:

Technical Problems Encountered during Real Time Adherence Monitoring

	Delayed Eventsa	Lost Eventsb	Total	% of Expected Events (n=2,566)
Battery failures	95	89	184	7.2%
Signal failure	197	56	253	10.0%
Total Number Events	292	145	437	17.0%
% of Expected Events	11.4%	5.7%	17.0%

^aEvents that were not delivered in real-time but transmitted later.

^bExpected events for which no data exists, delayed or real-time

Comparison of self-report and EAM adherence measures

Descriptive statistics for overall EAM and self-report adherence, over the intervention period, are reported in Table III. Median daily adherence reported by EAMs across all participants was 88.7%. Median timely adherence reported by EAMs across all participants was 67.1%. A wide range of individual adherence was observed with both daily (min: 30%, max 100%) and timely adherence (min: 18%, max: 96.5%) as measured by EAMs. Median daily adherence by self-report across all participants was much higher, 98.7% (min: 72.7%, max: 100%). Median timely adherence by self-report was 73.4% (min: 38.8%, max 98.8%). A low degree of concordance was observed between ART adherence reported by EAMs and self-report; (Daily: ICC = 0.33, 95% bootstrap CI: 0.13, 0.59, Timely: ICC = 0.72, 95% bootstrap CI: 0.54, 0.84). Median adherence levels by each measure are shown in Fig. 1. The generalized linear mixed models (not shown) indicated a significant difference in adherence between the two measures, with self-report consistently greater than with EAM adherence. In timely adherence the average difference was estimated at 10% (SE=1.2%, p<.001), and for daily adherence the average difference was estimated at 11.5% (SE=1%, p<.001).

Table III:

Descriptive Statistics for Real-time wireless Electronic Adherence Monitoring (EAM) and Self-report Adherence (N=25)

	Daily Adherence		Timely Adherence
	EAM	Self-Report	EAM	Self-Report
Mean Adherence  (Standard Deviation)	83.8% (±18.6)	94.0% (±8.2)	64.5% (±21.7)	73.6% (±15.6)
Median Adherence     (Range)	88.7% (30–100)	98.7% (72.7–100)	67.1% (18–96.5)	73.4% (38.8 −98.8)

Fig. 1.

Comparison of EAM reported median adherence to self-reported adherence using intra-class correlations (ICC) with 95% bootstrap confidence

Acceptability

Per the acceptability and feasibility 3-item questionnaire, 92% percent of participants reported that their EAM was easy to use. Equally, 92% indicated that their EAM device helped them to take their medications. Ninety-six percent of participants reported that refilling the device with medications was easy.

In addition, qualitative data was collected from 23 of the 25 participants in the form of an in-depth exit interview. Qualitative responses, summarized below, reveal the nuances of participants’ experiences with their EAM device.

Theme 1: Improving Adherence Behavior

Numerous participants reported finding that their EAM device helped to improve their adherence through 1) serving as a reminder and 2) providing a sense of accountability.

Sub-Theme: Improving Adherence Behavior as a Reminder

Participants described the visual presence of the box served as a reminder. Further, the slots in the device made it easy to verify if they had taken their medication for the day, as they were able to easily identify if the slot for the day was empty, thus avoiding missing doses or double dosing.

Because I use it always. I know it’s there. It’s just a normal routine as you would do, and if you forget taking your medicine, you look at the box, oh, I’ll take my medicine, it’s right there. So it’s a reminder to me to make sure I take my medicine… I took it. It kept me on point, that little box. Yes, it did. It kept me on my toes…It was helpful. Age: 45 years old; Time on ART: 1 year

Well, it helped in the fact that you had your slots, and it kept you—’cause I would be real absent minded. If it were not for the slots, I probably would forget which day I took what or if I took one this day. Age: 45 years old; Time on ART: less than 1 year

Sub-Theme: Sense of Accountability

Participants reported that knowing that their adherence was being monitored, provided them with a sense of accountability and thus a motivation to be more adherent. Further, participants reported that having a set timeframe to take their medicine made them more accountable and made fitting dosing into their routine easier.

Like I said, it helping me monitor and keep—because being that it was sending a signal, I wanted to take—make sure I took my medicine on time. Sometimes I know I fell short, but it made me wanna take my medicine… Then for me, like I said, it send(s) the signal to make sure I’m staying up with my meds, which is that’s a positive note for me, helping me. Age: 42 years old; Time on ART: 5 years

Because when I first started taking my medication, I would take it anytime. Sometimes I take it early in the morning, and sometime I take it late at night, whenever. Once I stayed in my timeframe, between 6:00 a.m. and 8:00 a.m. everything started working better. Age: 45 years old; Time on ART: 1 year

Theme 2: Drawback of EAM Monitoring

Participants were also asked what hindered the incorporation of the EAM device into their daily activities. Three general themes were identified: 1) routine interruptions, 2) difficulty keeping the battery of the device charged, and 3) storage capacity.

Sub-Theme: Routine Interruption

Some participants indicated that the use of the EAM device was challenging because it was not part of their established routine.

I’ve taken it so long without the box that the box was a challenge. Age: 58 years old; Time on ART: 14 years

It was a little hard at first. Being so—everywhere, and not having the structure, and having my regimen set up. Being off that regimen for so long, it definitely was a challenge for me… I had to—this is another thing I had to get used to doin’, too, because I never had to use a pill box to take my medication. I’d just taken it… I find it easier to take it out of my prescription bottle. Age: 55 years old; Time on ART: 1 year

Sub-Theme: Difficulties in Charging the Device

Remembering to charge the EAM device was a challenge for some participants. This assertion was supported by the interruption in real-time data transmission.

I know one time I made a mistake and forgot to charge it. I don’t know why I forgot, I just forgot to charge it Age: 42 years old; Time on ART less than 1 year

Sub-Theme Storage Capacity

Participants reported wishing the device was larger to allow for more medication storage.

The only thing I just—I got so many medicine… I’m on about 15 pills. Age: 41 years old; Time on ART: less than 1 year

The only thing about the Wisepill™ is the reason why I couldn’t use it—well, I did use it some, but I really didn’t because I have eight to nine pills to take [laughter]—they can’t fit all in the box…I had too many pills to put in the box. My pills weren’t small. They’re kind of big, so you can only get about three—maybe three in there. How I started doin’ was—I’d just take the Prezista, Norvir, and Epzicom, put it in the box, and just put a big supply of ‘em, but I had to reach in my purse, so that got confusion’… Other than that, it probably would be a good idea for somebody who didn’t have as many pills as I did. Age: 27 years old; Time on ART: 3 years

DISCUSSION

This proof of concept study provides evidence that the use of an EAM, a real-time adherence-monitoring device, is technically feasible in a population of African American women who are diagnosed with HIV infection and depression while residing in remote rural areas of the Deep South. In this population of psychosocially disadvantaged women, we found a low degree of concordance between EAM and self-report adherence. This finding is in agreement with a large body of research citing mHealth adherence measurements to be significantly lower than self-report data (33–36). Generally, self-report measures of adherence tend to overestimate adherence when compared to other methods due to social desirability and memory biases (37). The tendency to overestimate adherence through self-report may make real-time adherence monitoring particularly useful though prior research suggests that EAM may underestimate adherence due to non-use of the device (22, 25).

Using quantitative survey data and qualitative interviews, participants expressed both positive and negative impressions of their experience with the EAM device. Overall, the participants indicated that their experience with the EAM device was positive. Similar to recent studies, acceptability was generally high and many participants found that the device itself conveyed a sense of support and accountability (38). Similar to the only other known US based feasibility study of the Wisepill™ device, participants reported feeling like the device helped them to remember to take their medications and that real-time monitoring motivated better adherence by providing a sense of accountability (27). Participant feedback of the device mirrors feedback reported in previous studies such as the need for modification of the devices’ design to include more storage capacity (18, 21). Though previous studies on the acceptability of the device in international settings have garnered mixed concerns over monitoring (18, 27, 38), our participants reported that knowing they were being monitored helped them to adhere to their medication regimen.

The majority of events were retrieved in real-time (83%). For the current study, delayed transmission was not a hindrance as the device was used only to monitor adherence, not to implement a real-time intervention. However, the loss of real-time monitoring due to technological difficulties may impede the device’s usefulness for just-in-time adherence interventions. In the case of signal lapses, the openings are recorded, but are not received until the device regains signal connectivity. As noted in previous studies (39), this back-storage capability minimized the loss of data by backing up the data whenever connectivity interruptions did occur, allowing the recovery of the majority of data collected during signal disruptions.

Technical failures may be in part due to the underlying technology used in the 2G EAM device used. Due to the advancement of network technology in the US, many US cellular companies began to shut down 2G towers during the time of data collection, limiting our selection of cell network providers. Since data collection, EAM devices have been equipped with 3G technology; however, most US cellular providers are now operating on 4G networks and are moving into 5G networks. The rapid advancement of technology should be considered carefully when adopting EAM for US based studies.

Qualitative data presents the technological based difficulties participants had with the EAM device, mainly in forgetting to charge it. Importantly, battery losses are more likely to result in a complete loss of data than signal lapses. While data loss due to forgetting to charge the device could be seen as a participant level error, it is important that interventions reduce participant burden while simultaneously maximizing data quality if significant advancements in the use of real-time monitoring devices for adherence interventions are to be made. Since data collection for the current study, the EAM device battery capacity has been updated and the current EAM battery life has been extended to 6 months (40, 41). While the improved development of future devices with better battery life is one important solution, researchers can also be proactive in their manner of handling technological challenges such as recharging batteries at the time of clinic visits. One such solution to minimize data loss due to low battery, implemented by Evans and colleagues in South Africa (19), is to send an SMS reminder to recharge the battery whenever low battery power was detected. These findings echo those of previous studies which found connectivity and battery life (17, 25) to be persistently challenging with EAM devices. Though the newest Wisepill™ devices have attempted to address these issues we recommend critical consideration of possible solutions to these technological challenges before implementing adherence interventions using EAMs in rural US settings. This study is not without limitations, particularly as it relates to the effect of the device on adherence. While depression and other cognitive impairments are known to affect adherence and could possibly affect the use of the device, we believe this is doubtful as each participant received CBT therapy that specifically focused on their adherence to ARTs. Furthermore, previous research indicates that traditional pillboxes may lead to greater adherence among PLWH; thus, the device itself could be considered an intervention. The impact of the device on adherence is outside the scope of the current study. Future research should attempt to disentangle the barriers and facilitators of EAM device usage, and their impact on adherence, compared to traditional pillboxes.

CONCLUSION

Real-time monitoring devices are particularly valuable in assisting populations with known difficulties in adherence as real-time monitoring provides the opportunity for real-time interventions. While minor technological aspects of the device require monitoring, the EAM device demonstrates potential for real-time interventions in research and clinical settings. EAM adherence monitoring seems to be acceptable and feasible for use in rural US settings; however, the loss of real-time monitoring data due to technological difficulties may limit the device’s usefulness for just-in-time adherence interventions if technological gaps between the device and network technologies exist. The potential for adding EAM to longitudinal ART adherence protocols in the US should be considered to evaluate its power to predict clinical outcomes.