Development and Usability of a Smartphone Application for Tracking Antiretroviral Medication Refill Data for Human Immunodeficiency Virus

Summary

Background:

Adherence to antiretroviral medication leads to HIV suppression and decreased morbidity and mortality. In resource-limited settings, the dependence on paper medical charts and unstable electronic health records creates a challenge to monitoring medication adherence. A pharmacy-based strategy that utilizes existing cellular phone infrastructure may lead to a more stable system to monitor adherence.

Objectives:

To develop and evaluate the usability of a smartphone-based software application (app) for tracking antiretroviral medication refill data in a resource-limited setting.

Methods:

A pharmacy-based smartphone app for tracking HIV medication adherence was developed through a multi-step rapid prototyping process. The usability of the app was assessed during the daily activities of pharmacy dispensers at HIV clinics in and around Gaborone, Botswana using a validated computer usability survey.

Results:

The study demonstrated the effective development of and favorable end-user responses to a pharmacy-based HIV medication adherence app. End users had suggestions for minor changes to improve the app’s functionality.

Conclusions:

In resource-limited settings where electronic health record support is limited, such a system was feasible and appealing. In the future, this system may allow for improved HIV medication adherence tracking and be applied to medications beyond antiretrovirals.

1. Background and Significance

1.1. The Importance of Adherence in HIV Treatment

The treatment of human immunodeficiency virus (HIV) involves multiple antiretroviral (ARV) medications taken in combination. Adherence to ARVs is critical for viral suppression as well as length and quality of life [1, 2, 3]. Despite these benefits, adherence is a challenge. Socioeconomic barriers to adherence may include poverty, homelessness, poor health literacy, geographic isolation, lack of provider availability, and inadequate insurance [4, 5]. Other clinical conditions such as psychiatric illness and substance abuse also contribute [4, 5].

Adherence challenges may be worse in resource-limited regions. For example, areas of Sub-Saharan Africa have a dearth of trained providers, making basic provision of HIV treatment difficult and limiting adherence and retention programs [6, 7]. Weak healthcare infrastructures unable to provide critical pharmacy, laboratory or medical record support may also affect adherence in these settings [6, 7].

In both the developed and resource-limited settings, measuring adherence may identify patients at risk for defaulting on their ARV regimens which may, in turn, allow providers to target the subset of individuals in need of extra attention and resources.

1.2. Measuring ARV Adherence

A number of strategies have been used to facilitate measurement of ARV regimen adherence. Methods such as in-office pill counts, patient self-reporting, physician estimation of adherence, drug level monitoring, and electronic bottle monitoring have all been used [8, 9, 10, 11]. In addition to these patient and provider-based tools, pharmacy-based approaches have also shown promise. Specifically, drug refill data have proven to be indicative not only of adherence but also of retention in care for HIV patients [12].

New technologies have also been leveraged to improve adherence. Data suggest that smartphone-based tools can be useful components of adherence strategies as they allow direct access to patients outside their provider’s office [13, 14, 15]. In these studies smartphone applications (“apps”) on patient phones provided conduits for one-way communication from provider to patient. These apps provided reminders to patients throughout the day to take their medications at the appropriate time. In one study, a two-way app-based discussion between patients and providers had beneficial effects [16]. Here, smartphone reminders facilitated conversations between patient and nurses concerning lagging ARV adherence. However, this intervention looked at the outcome of subjective self-reported adherence without the incorporation of objective data, such as pharmacy refill records.

1.3. The Botswana Healthcare System

With the explosion of cellular capabilities in resource-limited settings, smartphones may play a critical role in optimizing care for HIV-positive patients, particularly when these technologies are combined with pharmacy-based adherence strategies. The Sub-Saharan African country of Botswana may serve as such a setting. Approximately 98% of all health centers in Botswana are operated by the government [17], and since 2001 these government facilities have provided free ARVs to all of its HIV-positive citizens [18]. By the end of 2014, 247,947 patients were on ARVs in the public sector compared to 18,723 patients receiving ARVs through private sector Medical Aid [19].

In Botswana, all public sector ARV management and prescription is performed through infectious disease care clinics [20]. These specialized HIV clinics are associated with tertiary and secondary hospitals as well as with stand-alone primary care clinics. From a supply chain standpoint, ARVs are distributed to these HIV clinics nationwide through the government’s Central Medical Stores [21, 22]. Distribution is based on each HIV clinic’s utilization [21].

Availing patients of these medications is reliant on pairing every HIV clinic with an on-site pharmacy. Once a patient has seen an HIV provider, that patient then must go to the clinic’s on-site pharmacy to obtain the free government-provided ARVs [20]. At the pharmacy, specially-trained HIV pharmacists and technicians counsel each patient on medication adherence and are considered critical to the entire team of providers, including doctors, nurses, and social workers. As such, pharmacists and technicians in Botswana could serve as the frontline of recognizing failures in treatment adherence [20].

1.4. Technical Limitations to Healthcare Provision in Botswana

In Botswana, the technical challenges to monitoring patient adherence are two-fold – patient data management and supply chain maintenance. Regarding patient data, two electronic databases comprise the bulk of the HIV National Data Warehouse – the Integrated Patient Management System (IPMS) and the Patient Information Management System (PIMS) II [19, 22]. IPMS is an electronic health record and ordering system that is only available at the country’s large hospitals and associated clinics [22]. PIMS II is available at all IPMS sites and all other public sector sites that prescribe ARVs [22]. The lack of integration of these systems has been problematic in Botswana’s desire to create a centralized electronic health record. As of 2015, the government of Botswana recognized that the National Data Warehouse had not been updated with available patient data due to a lack of funds [19]. The government’s inability to expand current infrastructure for patient data entry has further hampered data availability for clinical uses, such as improving adherence [21]. Furthermore this fragmentation of the electronic healthcare record has led to ongoing dependence on the use of paper charts, which patients carry with them [23].

Beyond these data collection issues, the government of Botswana has faced problems related to the supply chain of medications, in particular ARVs. In 2009 the Botswana Ministry of Health requested assistance in managing its Central Medical Stores. As a result, the United States Agency for International Development part-improve the logistics of drug procurement and national distribution [22]. In 2012 the management of these logistics was returned to the Ministry of Health, which in turn awarded it to a sole-source contractor. Since that time a number of issues related to the supply chain have been identified, including inadequate performance by the contractor, staffing of the Central Medical Stores by inexperienced personnel, and a poorly-functioning medical informatics infrastructure [21, 23].

1.5. Measuring App Usability

A smartphone-based medication tracking system may provide the support that circumvents these technical challenges. This could take advantage of the existing reliable smartphone network. A smartphone application that monitors pharmacy refill data in HIV patients could be effective in monitoring adherence and communicating data to appropriate stakeholders. However, when creating such an app, end-user usability is of primary importance.

When applied to interfacing with technology, usability is the subjective or objective measure of mental workload for a given task in a realistic environment [24, 25]. Subjective measures of usability evaluate perceived ease of use [26] and are particularly relevant for evaluating a tool without disruption of the primary task [27]. For the evaluation of a tool incorporated into clinical practice such as a pharmacy-based smartphone app, non-disruption of care is critical.

2. Methods

2.1. Overview of App Development

We developed a smartphone app through a rapid prototyping systems development life-cycle approach to construct the system architecture, design and implementation specifications (Figure 1). The system was adapted from methods developed by the Agency for Healthcare Research and Quality [28]. Through this method, a simplified prototype of the app was initially built to perform a few major functions of the final system, so that it could be evaluated, and iteratively refined as additional complexity was added incrementally, to achieve all of the desired functionality. This method provided developers with a systematic structure. The prototype implementation plan defined the phases of the project that would be most appropriate for this task. This approach employed incrementally-developed system prototypes (i.e., “straw-man” models) such that all developers provided input in the development process and had access to increasingly complex, concrete representations of the final system. Importantly, the system design could be altered in response to requirements and to participant responses, as they inevitably changed during development.

Figure 1.

Rapid prototyping process.

2.2. The Development Team

Throughout each stage of development, the app was reviewed by a multi-disciplinary team of experts in the field of HIV treatment and research, including in Botswana. The team of prototype evaluators included physician-researchers, pharmacists, adherence interventionists, and clinical research coordinators.

2.3. Gathering App Requirements and Development of Initial Prototype

The rapid prototyping life-cycle began with gathering requirements for the app, through discussions amongst the development team, in sufficient depth to facilitate the initial app system specifications (Figure 1, Box 1). Information from “straw-man” use scenarios were collected via hypothetical “think-aloud protocols,” in which members described a typical use-session and identified facilitators and barriers they might foresee. This information was used to design the system architecture. We then developed an initial prototype based on system requirements and capabilities identified by the development team (Figure 1, Box 2). The prototype described the technical specifications and architecture of the system.

The app prototype was reviewed with development team members involved in the initial app design and then other team members with similar roles who were not involved in the requirement gathering (Figure 1, Box 3). Feedback from these reviews focused on acceptability and identification of any needed changes, additions, or deletions.

2.4. Prototype Revision/Completion

The prototype was then revised based on stakeholder feedback (Figure 1, Box 4). The prototyping process was then repeated and iterated through the 4 steps until the final design was created. Acceptability of the final design was established as part of the initial and ongoing requirements gathering phases of the cycle. The design was considered complete when all members of the development team agreed that further specification was unnecessary and that the app could feasibly be implemented.

2.5. Usability Assessment

2.5.1. Study Site and Population

Once the app was created, its subjective usability was assessed using a psychometric questionnaire in a population of pharmacists and pharmacy technicians in Botswana. The study was performed at six HIV clinics in the greater Gaborone area: Bontleng, Phase II, Broadhurst Traditional Area, Broadhurst 3, Princess Marina Hospital Infectious Diseases Care Clinic, and Old Naledi.

All available adult pharmacists and pharmacy technicians at the six study sites were eligible for inclusion and were invited to participate. Only dispensers for adult patients were recruited given the difference in the infrastructure of the local children’s clinics. Pharmacists and technicians were recruited under the auspices of the Greater Gaborone District Health Management Team Pharmacy Head and the Princess Marina Hospital Chief Pharmacist. Participants provided informed consent for their involvement in the study. We targeted a minimum of 10 users to assess usability based upon the standard 10±2 rule for usability evaluations [29].

2.5.2. App Training and Use

The pharmacy staff were provided with a smartphone, loaded with the app, during two 4-hour periods – one morning session and one afternoon session, conducted on different days. During the first session, the goals of the study were explained and informed consent was obtained. After use of the app was explained the remaining time of approximately three hours was spent observing the dispensers using the app during the course of a typical clinical session. The second session was devoted use of the app during a clinical session. At the end of the second session, dispensers were surveyed to assess the usability of the app.

2.5.3. Usability Questionnaire

Usability and demographic data were determined using a self-administered questionnaire with three sections. The first section of the questionnaire used questions based on the validated IBM Computer System Usability Questionnaire (CSUQ) form [27] and included questions on usability and subject reaction. The CSUQ is a psychometric questionnaire developed for assessing the subjective usability of computers in the workplace. This section contained general questions regarding the app’s usability. Responses were based on a Likert scale, ranging from 1 to 7, with 1 representing “Not at all” and 7 representing “Extremely“. The second section contained questions assessing user reactions specific to dispensing medications and adherence. Responses in this section were also based on a Likert scale with response values identical to the scale used in the first section. For all questions, space was provided for free responses by participants. In both sections, we assessed median scores for each scaled question as a measurement of the respondents’ satisfaction. We collected the free responses from these sections and reported those that best reflected overall participant impressions. The last section contained general demographic questions. Medians and interquartile ranges of response values were assessed using Stata (version 14.0, StataCorp, College Station, TX) software.

3. Ethics

All participants provided written informed consent. Oversight of the project was provided by the Republic of Botswana Ministry of Health’s Health Research and Development Committee research ethics board. Approval was likewise obtained from the University of Pennsylvania Institutional Review Board.

4. Results

4.1. App Design

The principal functions chosen for the app were management of prescription medications, collection of pharmacy refill data, and ability to query for automated adherence calculations. To simplify use, the development team determined that the app’s functions would be divided into the basic components of data entry and retrieval. Figures 2 and 3 display screenshots of the app, demonstrating its two major tabs – one for entering patient data (“Enter records”) and one for retrieving data (“Retrieve records”). Under the Enter Records tab there are nine fields for data entry. These include “Enter patient omang [National Identity Card number]”, “Enter refill date”, “Select a drug” (dropdown menu), “Enter tablets per day”, “Enter number lost”, “Enter number returned”, “Enter new number of tablets dispensed”, “Enter your (pharmacist’s) omang”, and “Select the current clinic location” (dropdown menu). Under the “Retrieve records” tab an individual patient may be looked up by national identity number. Once a patient’s record has been retrieved, data on each prescribed drug may be reviewed. For each drug, data generated automatically by the app include a trend of pharmacy visit dates, number of pills dispensed on each date as well as lost/returned pill counts. Adherence was calculated based on an algorithm run at the server using the following validated formula: (days’ supply of medications dispensed-days’ supply of doses returned)/days between prescription refills and expressed as a percentage [8, 30]. Once calculated, adherence rates for the preceding months could be displayed as shown in Figure 4.

Figure 2.

Record entry tab. This tab includes fields for patient ID numbers and pertinent medication refill data.

Figure 3.

Record retrieval tab. This tab includes fields for patient ID numbers and prescribed medications. Refill and adherence data is retrieved for elected medications.

Figure 4.

Medication adherence. Adherence data is calculated automatically and displayed as trends.

4.2. Usability

We enrolled 14 dispensers who were practicing pharmacists or pharmacy technicians at the local sites. Table 1 summarizes general and demographic data about the participants. Most subjects were female and native Setswana speakers. Of the 13 participants who responded regarding education level, all had received some degree of post-secondary school training and on average had been dispensing medications for longer than six years. A majority of the respondents used some form of personal computing device at home, although fewer owned a smartphone.

Table 1.

Demographic/general data of participating pharmacists.

Sex	Female 9 (69%)	Male 4 (31%)
Number of years post-secondary education completed	1–3	9 (70%)
	3–6	3 (23%)
	unknown	1 (8%)
Mean number of years dispensing medications (Standard Deviation)	6.6 (5.5)
Mean length of stay in current job (Standard Deviation)	4.7 (5.0)
Is Setswana your native language?	Yes 10 (71%)	No 4 (29%)
Use a desktop, laptop or tablet at home?	Yes 11 (92%)	No 1 (8%)
Own a smartphone?	Yes 8 (62%)	No 5 (38%)
What do you do when a patient is late for refills (most common course of action)? *	Counsel on adherence	12 (92%)
	Refer to site counselor	4 (31%)
	Refer to head matron	0 (0%)
	Dispense without comment	0 (0%)
What do you think your colleagues do when a patient is late for refills (most common course of action)? **	Counsel on adherence	11 (79%)
	Refer to site counselor	4 (29%)
	Refer to head matron	0 (0%)
	Dispense without comment	0 (0%)
What is the purpose of adherence counseling?	Instill fear of treatment failure and death	Yes 1 (7%)
	Say that non-adherence is reasonable behavior	Yes 0 (0%)
	Gather information from patient	Yes 2 (14%)
	Explain how adherence can benefit patient	Yes 14 (100%)
If you were provided adherence data by this tool at each refill, how much would it affect your interactions with patients?	A lot	10 (72%)
	Somewhat	3 (21%)
	Not at all	1 (7%)
If the app were adopted do you think it would change your work burden?	A lot	10 (77%)
	Somewhat	3 (23%)
	Not at all	0 (0%)

^*Three respondents indicated two answers

^**One respondent indicated two answers

The majority of participants believed that both they and their colleagues normally counsel their non-adherent patients during dispensing visits. All 14 respondents identified explaining the benefits of adherence as a purpose of adherence counseling.

Tables 2 and 3 summarize dispensers’ impressions of the app regarding patient interactions and work burden, as well as how usable the app was in their day-to-day workflow. Table 2 reflects data from the general usability section of the questionnaire. For that section, all questions except 9 and 18 scored a median of 5 or higher. Participants felt less positively that the system gave sufficient error messages for fixing problems (question 9) and that it had all the functions they expect (question 18). Table 3 reflects the data from the user reaction section of the questionnaire. On all but one item participants reported satisfaction of a median of 5 or greater.

Table 2.

General usability questions.

Question	Median (IQR)
1. I am satisfied with how easy it is to use this system.	5 (4, 6)
2. It is simple to use this system.	6 (5, 7)
3. I can effectively complete my work using this system.	5 (4, 6)
4. I am able to complete my work quickly using this system.	5.5 (4, 6)
5. I am able to efficiently complete my work using this system.	5 (3, 6)
6. I feel comfortable using this system.	5 (5, 6)
7. It is easy to learn to use this system.	6 (6, 7)
8. I believe I became productive quickly using this system.	5 (4, 5)
9. The system gives error messages that clearly tell me how to fix the problems.	4 (1.5, 5.5)
10. Whenever I make a mistake using the system, I recover easily and quickly.	5 (5, 6)
11. The information (such as on-screen messages) provided with this system is clear.	5 (5, 6)
12. It is easy to find the information I need.	5 (4, 6)
13. The information provided with the system is easy to understand.	6 (5, 7)
14. The information is effective in helping me complete my work.	5 (4, 6)
15. The organization of information on the system screens is clear.	6 (6, 7)
16. The interface of this system is pleasant.	5 (5, 6)
17. I like using the interface of this system.	5 (5, 7)
18. The system has all the functions and capabilities I expect it to have.	4 (3, 5)
19. Overall, I am satisfied with this system.	5 (4, 6)

Scale: 1 “Not at all”, 2 “No”, 3 “Not very much”, 4 “Somewhat ”, 5 “Yes” 6 “Very much so”, 7 “Extremely so”.

Table 3.

User reaction questions.

Question	Median (IQR)
1. I feel confident I can use the system when dispensing.	6 (5, 6)
2. Using the system when dispensing is under my control.	5 (5, 6)
3. People who are important to me think I should use the system when dispensing.	4 (3, 5)
4. My coworkers think I should use the system when dispensing.	5 (4, 6)
5. Patients are likely to benefit from my using the system when dispensing.	5 (5, 6)
6. Providers will likely track medication adherence better due to my using the system when dispensing.	5 (3, 6)
7. Patients will likely receive better treatment due to my using the system when dispensing.	6 (4, 6)
8. My using the system is beneficial.	5 (5, 6)
9. My using the system is a good thing.	5 (5, 6)
10. I intend to use the system if implemented permanently.	6 (5, 7)

Scale: 1 “Not at all”, 2 “No”, 3 “Not very much”, 4 “Somewhat ”, 5 “Yes” 6 “Very much so”, 7 “Extremely so”.

Table 4 displays a representative sample of subject comments and general themes of those comments. On review of the subjective comments in the general usability and user reaction sections of the survey as well as the qualitative section of the survey, three main themes were noted: 1) Desire for Technical Improvements, 2) Issues with Manual Documentation, and 3) Concerns Over the Cellular Network/Security.

Table 4.

Perspectives of participating pharmacists.

Themes	Subject Comments
Desire for technical improvements	“It is very easy to use and it is fast but it does not have some of the information needed for adherence calculation”
	“I am satisfied [,] it’s just that it does not calculate and give the results of compliance immediately”
	“… I wouldn’t say the system is quick until some of the things are upgraded e.g. catering for more than one medication…”
Issues with manual documentation	“If at all the app is going to replace manual documentation, it is an advantage because it will be more fast”
	“It is quick. Convenient for the patient (since they will be taking drugs everywhere)”
	“Yes it is effective BUT I still have to write paper work to indicate the refill date of the patient…”
Concerns over the cellular network/security	“Network problems can make it slow. If you have already confirmed data submission it is not easy to edit the information”
	“…some patients will not easily get convinced of assurance on privacy”
	“If the system and the server crash or gets hacked, information will be lost”

In regard to the first theme, subjects were generally pleased with the app, but were interested in ongoing technical improvements. In the usability section of the survey, one pharmacist stated, “It is very easy to use and it is fast,” followed by the erroneous comment that “… it does not have some of the information needed for adherence calculation”. Another pharmacist noted similar satisfaction but would prefer more rapid calculation of adherence. Other comments noted that, while the app improved overall efficiency, further upgrades could improve its performance.

In addition to a desire for ongoing technical improvements, issues were raised regarding manual documentation. Subjects noted satisfaction with the app’s use of electronic as opposed to manual, which would allow for patients to have their records available regardless of travel (“[patients] will be taking drugs everywhere”). While the possibility of improved efficiency with electronic documentation was noted, one subject did cite a concern about the redundancy of dual electronic/manual documentation.

Finally, subjects raised concerns related to the cellular network and security. One subject indicated that the network may be slow in regard to data transmission and retrieval. Two subjects raised concerns over security. While both subjects felt that patient privacy was an issue, one subject also felt that protection against data loss and the need for back-up should be a priority.

5. Discussion

To test our smartphone app in a real-world setting, we involved HIV pharmacists and pharmacy technicians in and around Gaborone to use the app during typical work-days. Participants in our study generally responded positively to the usability of the software, deeming the app to be acceptable in terms of workflow, ease of use, and proper patient care. They felt the user interface was pleasant and easy to understand, and were able to incorporate the app into their usual routine, quickly becoming productive and efficient with it. Furthermore, the pharmacists and technicians felt that mistakes in using the app were easy to correct and error messages were helpful in the process of correction. In regard to patient care, study participants felt that patients would not only benefit from our app, but would experience better care should this system be put into place. Subjects did express the need for technical improvements. Some had difficulty with the adherence calculation component of the app. These concerns provide important data, which will assist in future enhancements of the app.

While our app may become a unique component of an adherence strategy not addressed in the current literature, there are limitations to our technology. As this is a provider-oriented app, it may provide patients will less agency over improving their own adherence. Providers will need to take care to use the app in a collaborative fashion with patients. From the provider’s perspective, the technology will still need further technical improvements. As noted in the survey data, the participants wished for more functions and capabilities than are currently provided by the app. Improvements will need to be made to optimize the app for generalized use. Furthermore, participants expressed concerns about patient perceptions of privacy. When applying this app to patient records, care will need to be taken regarding privacy. Moreover, care will need to be taken to explain security measures to participating patients and that this app represents a collaboration between patient and provider.

The novel aspects of this study’s smartphone app are twofold. First, it has the potential to allow pharmacists, who work closely with HIV providers, to better participate in improving patient adherence. While dose reminders to patients in the forms of text messaging and “alarm” apps may be helpful, they do not facilitate discussion between patients and the HIV treatment team. A real-time assessment of adherence, as provided by this app, could facilitate a discussion of adherence at the time of ARV dispensation.

Furthermore, this app allows for mobile access to a national pharmaceutical database. This may be particularly important in a setting such as Botswana, where the population is geographically mobile and still dependent on paper records given unstable local access to electronic healthcare records [23, 31]. Even at remote clinical sites in Botswana, cellular coverage is readily available [32]. Through its ability to access central pharmacy data via the cellular network, our smartphone application could circumvent the current data challenges.

Through these improvements in patient care, this tool has the potential to capture data that might otherwise be lost. For example, patients who misplace their paper records or whose electronic records cannot be accessed due to infrastructure limitations can be provided with their correct medications at the correct dosages as recovered via the smartphone app. Furthermore, medication adherence can be properly assessed based upon prescription refill data that may otherwise have been lost or may be unavailable.

Such redundancy would not only benefit individual patients, but may also help the healthcare system as a whole. A consistent and intact record of pharmacy data, by site, will improve the accuracy of nationwide drug utilization data. As of last year, Central Medical Stores distribution centers reported a stock-out rate for all medications to be 26% [19]. The prescription tracking system created by our application may allow the Central Medical Stores to more strategically distribute medications around the country, preventing stock-outs. If our app is implemented, individual patients as well as the healthcare system as a whole may benefit. Pharmacists and technicians may be able to provide focused interventions, assisting patients who have difficulties adhering to their ARV regimens. While this project’s primary goal is focused on HIV care, the monitoring of non-ARV prescription may lead to improved resource distribution for other medical conditions treated with pharmaco-therapy.