Usability and Feasibility of a Tablet-Based Decision-Support and Integrated Record-Keeping (DESIRE) Tool in the Nurse Management of Hypertension in Rural Western Kenya

Abstract

Background

Mobile health (mHealth) applications have recently proliferated, especially in low- and middle-income countries, complementing task-redistribution strategies with clinical decision support. Relatively few studies address usability and feasibility issues that may impact success or failure of implementation, and few have been conducted for non-communicable diseases such as hypertension.

Objective

To conduct iterative usability and feasibility testing of a tablet-based Decision Support and Integrated Record-keeping (DESIRE) tool, a technology intended to assist rural clinicians taking care of hypertension patients at the community level in a resource-limited setting in western Kenya.

Methods

Usability testing consisted of “think aloud” exercises and “mock patient encounters” with five nurses, as well as one focus group discussion. Feasibility testing consisted of semi-structured interviews of five nurses and two members of the implementation team, and one focus group discussion with nurses. Content analysis was performed using both deductive codes and significant inductive codes. Critical incidents were identified and ranked according to severity. A cause-of-error analysis was used to develop corresponding design change suggestions.

Results

Fifty-seven critical incidents were identified in usability testing, 21 of which were unique. The cause-of-error analysis yielded 23 design change suggestions. Feasibility themes included barriers to implementation along both human and technical axes, facilitators to implementation, provider issues, patient issues and feature requests.

Conclusions

This participatory, iterative human-centered design process revealed previously unaddressed usability and feasibility issues affecting the implementation of the DESIRE tool in western Kenya. In addition to well-known technical issues, we highlight the importance of human factors that can impact implementation of mHealth interventions.

Introduction

The number and type of mobile health (mHealth) applications, developed for a wide variety of health interventions, have proliferated over the past several years. This is particularly true in low- and middle-income countries (LMICs), where mHealth solutions have been proposed to address the huge shortfalls in qualified health professionals by complementing a task-redistribution strategy with clinical decision support.[1] While there has been a subsequent surge in the literature following the rapid growth in mHealth interventions, most studies have been focused on the outcomes of interventions. Relatively few have explained what components contribute to successful outcomes and what aspects lead to failed implementation, despite the recognition that lack of attention to usability and feasibility issues has been shown to increase overall costs and delay successful implementation.[2,3] In addition, feedback for iterative system improvement is relatively lacking.[4] A recent systematic review of mHealth literature demonstrated that only 14% of studies reported on user assessment of technology.[5]

Of the mHealth usability and feasibility studies that have been published, very few have been conducted in sub-Saharan Africa (SSA).[5] Technical challenges that have been reported include lack of broadband internet access, inconsistent or limited network availability, transmission error detection and management, transmission of large files, ensuring patient privacy during wireless transmission, information security, phone security and sharing, and usability issues.[1,6] Administrative challenges include lack of business model, insufficient strategic leadership, absence of a change management plan, difficult learning environment, limited buy-in from practitioners, high staff turnover, and limited eReadiness.[1,7]

While the majority of the mHealth literature has involved infectious and maternal health issues, cardiovascular disease (CVD) is the leading cause of mortality in the world, with 80% of CVD deaths occurring in LMICs.[8] Hypertension, a major risk factor for CVD, contributes significantly to the burden of CVD in SSA. However, awareness, treatment, and prevention of hypertension remain very low throughout SSA.[9,10] In order to properly address the human resource challenge of managing CVD and hypertension in LMICs, task-redistribution of CVD care from physicians to nurses has been proposed.[11] In this context, mHealth interventions to support hypertension and CVD care delivery have been developed and are being evaluated.[12,13]

For example, the Academic Model Providing Access to Healthcare (AMPATH), based in western Kenya, has recently expanded its clinical scope of work from human immunodeficiency virus (HIV) to address non-communicable diseases, including an integrated diabetes and hypertension care program. To supplement its task redistribution strategy, AMPATH has developed a tablet-based Decision Support and Integrated Record-keeping (DESIRE) tool to record patient data and assist rural clinicians with clinical decision-making. From the outset of the project, it was recognized that an iterative process of usability and feasibility assessment would be critical for the potential successful implementation and scale-up of the project. Thus, this usability and feasibility study was conducted to assess the barriers and facilitators to implementing the DESIRE tool.

Methods

Usability and feasibility testing of the DESIRE tool is one component of a larger implementation research project evaluating different aspects of nurse management of hypertension in western Kenya.[14] Institutional review boards at the Icahn School of Medicine at Mount Sinai (New York) and Moi University College of Health Sciences (Kenya) have approved the protocol.

Setting

AMPATH is a collaboration among Moi University College of Health Sciences, Moi Teaching and Referral Hospital, and a consortium of North American universities and medical centers, founded in western Kenya in 2001.[15] The partnership works to deliver health services through the Kenyan Ministry of Health facilities, and has established a human immunodeficiency virus (HIV) care system in western Kenya that serves over 100,000 patients. In partnership with the Government of Kenya, AMPATH is expanding its clinical scope of work to address comprehensive primary care, including hypertension.

Currently in Kenya, only physicians are authorized to manage hypertension. [16] Traditionally, the nurse's role in hypertension care is to enhance self-management strategies by educating and counseling the patient about medication adherence and lifestyle modification.[17] In general, if a patient is found to be hypertensive, they would require referral to a hospital to be further evaluated and managed by a physician. AMPATH has a Memorandum of Understanding with the Kenyan Ministry of Public Health and Sanitation and the Ministry of Medical Services to test and evaluate innovative approaches to chronic disease management, including task redistribution of hypertension care to rural nurses.

Thus, the cornerstones of AMPATH's chronic disease management (CDM) program include geographic decentralization of clinical services for enhanced access, improved supply line of essential medicines, and task redistribution of basic clinical care to rural nurses.[18] Individuals with elevated blood pressure during community-based testing are referred to AMPATH's CDM program, which has developed clinical algorithms derived from the World Health Organization [19] and Joint National Commission 7 [20] guidelines for hypertension management using drugs available in the Kenyan national formulary (Appendix A).[21] Nurses based at rural dispensaries (on average one dispensary per 10,000 individuals) have been trained to manage uncomplicated hypertension according to the CDM algorithms.

Existing AMPATH decision support and electronic data capture systems, primarily designed to address the challenges of HIV care in a resource-limited setting, were modified to create the DESIRE tool, which provides both record-keeping and clinical decision support capability for rural clinicians.[22–24] The tool was developed using the Open Data Kit (ODK) platform, a java-based software system for Android (Google, Inc. Mountain View, CA USA) that was created in part by members of the AMPATH Informatics team.[25]

Nurses have been provided with Huawei IDEOS Tablets (Huawei Device Co., Ltd.) running the Android operating system and loaded with the DESIRE tool. DESIRE has three core functionalities: 1) a data entry and validation; 2) branching logic featuring decision support, alerts, and reminders; and 3) the ability to retrieve and display historical data derived from the electronic AMPATH Medical Record System (AMRS) stored on a central data server (Figure 1). The DESIRE decision support provides nurses with the appropriate prompts and reminders to guide the nurse through the AMPATH hypertension management algorithm (Appendix A). Near the end of the encounter, the software utilizes clinical data entered by the nurse and, based on the algorithm and programmed logic, presents a patient-specific clinical decision support recommendation to the nurse regarding patient management. In addition, an image of the algorithm flowchart is displayed for the nurse to view.

Figure 1.

A schematic overview of the DESIRE tool and its three core functionalities.

The AMPATH Medical Record System (AMRS) server holds AMPATH's medical records in a central electronic database, including individuals with elevated blood pressure at the time of community-based testing. Relevant clinical data is stored locally on the tablet and the DESIRE software allows for local offline work that can be later synced to the AMRS server when network connectivity is available. The tablets have been equipped with a cellular modem and SIM card to connect with the AMRS server through the cellular network, in order to sync medical records between the nurse tablets and the AMRS database. Data security features of the system include user authentication, automatic timeouts, encryption of data on the device and a secure https-based data transmission system.

Participants

Participants for this study included the rural clinicians (nurses and clinical officers) involved in the CDM clinical program. Eligibility criteria included attending a training of the DESIRE tool and use of the DESIRE tool in clinical work for a minimum of four weeks. Table 1 provides details of the number of participants who were invited and who participated. A total of 12 nurses and one clinical officer participated in the various components of the study; some participants were involved in more than one component.

Table 1.

Summary of eligible individuals who were invited and who participated in the usability and feasibility research activities.

Testing	Participant Type	Invited	Participated
Usability Testing	Nurses and 1 clinical officer	7	7
Usability Focus Group Discussion	Nurses	8	8
Feasibility Semi Structured Interview	Nurses	6	6
Feasibility Semi Structured Interview	CDM implementation staff	2	2
Feasibility Focus Group Discussion	Nurses	7	5

Usability Testing

Usability is defined as the extent to which a product can be utilized by specified users to achieve specified goals with effectiveness, efficiency and satisfaction, in a specified context of use.[26] Five nurses, each equipped with an IDEOS tablet were subject to one “think aloud” exercise and one “mock patient encounter,” prior to using the DESIRE tool in the clinical setting. It has been demonstrated that using five subjects can detect over 80% of usability problems.[27]

During the “think aloud” exercise, each nurse was asked to speak out loud what s/he was seeing, thinking, doing and feeling as s/he used the DESIRE tool while “managing” a paper-based case scenario (Appendix B). Both concurrent (during decision-making) and retrospective (after decision-making) protocols were used. During “mock patient encounters,” a research team member conducted a role-play as a mock patient using a set of prepared case scenarios (Appendix B), and the nurse was instructed to examine, diagnose and treat the mock patient using the DESIRE tool. During both of these exercises, the research team member made observations regarding breaks in the workflow, as well as errors made by the nurses or in the output of the DESIRE tool.

One focus group discussion with eight nurses was also conducted, to understand the perception of the DESIRE tool and obtain recommendations for improvement. This focus group discussion was arranged after the nurses had used the DESIRE tool for at least one month, in order to allow them time to use the tool in their respective clinical settings. A structured discussion guide was utilized after pilot-testing (Appendix C). The think aloud exercises, mock patient encounters, and focus group discussion were audio-recorded, transcribed, and translated into English.

Feasibility Testing

A feasibility study in the context of technology development is an analytical tool used to determine if parts of a project can perform together to create a technically and economically viable concept.[28] Feasibility testing was conducted after each nurse had used the DESIRE tool for a minimum of three months, in order to assess the way in which the tool interacted with the nurse's work environment and workflow. To evaluate feasibility, we conducted semi-structured interviews of five nurses, semi-structured interviews of two members of the implementation team, and a focus group discussion with five dispensary nurses. Structured question guides were utilized for all interviews and discussions (Appendices D & E). The interviews and focus group discussion were audio-recorded, transcribed, and translated into English.

Analysis

Using Atlas.TI (usability transcripts) and Nvivo 10 (feasibility transcripts), content analysis [29] of the transcripts was performed independently by two members of the research team; disagreements in coding were resolved by discussion or by consultation with a third investigator. For the usability analysis, deductive (a priori) codes included: effectiveness (task completion), efficiency (time requirements for task completion), and user satisfaction (ease of use, ease of learning, error minimization, and recall capacity).[30–32] Critical incidents, defined as “incidents that significantly impact the usability of DESIRE,” were identified and ranked according to severity.[33–35] A cause-of-error analysis was used to develop corresponding design change suggestions.[36] The transcripts containing each critical incident were reviewed for factors that contributed to the occurrence of the critical incident. The researchers developed design change suggestions based on the participant's actions and thoughts spoken aloud preceding, during and following each incident, along with assessment of the user interface.

For the feasibility analysis, deductive codes included acceptability and infrastructure. Components of acceptability included: a) confidence using the DESIRE tool on a day-to-day basis in the work setting, b) likelihood of recommending the DESIRE tool to one's peers, c) value added by the tool to the workflow, d) impact on the practitioner-patient relationship, and e) empowerment of the end-users. Infrastructure assessment included technical, logistical, human, and cultural barriers to the deployment of the DESIRE tool. For both usability and feasibility transcripts, significant inductive (emerging) codes were also identified.

Results

Usability

Fifty-seven critical incidents were identified between the think aloud exercises and mock patient encounters, 21 of which were unique. (Table 2)

Table 2.

Critical incidents, categorized into task failures and serious problems, with corresponding frequencies and design change suggestions.

	Critical Incidents	#	Design Change Suggestions
Task Failure	Difficulty selecting form	7	Remove extraneous steps for selecting form
	Unable to locate program icon	1	1) Lock ODK icon to home screen 2) Delete all other icons on home screen
	Inability to activate onscreen keyboard	1	Default keyboard to open when text box selected
	Unable to save form	1	Reword save and exit form
	Use of hardware button exits form	1	Disable hardware buttons during encounter
Serious Problem	Field pre-filled with incorrect data	7	Correct programing error
	Uncertain advancing page when field is not applicable	6	Add “not applicable” box to check before advancing
	Difficulty recalling information during encounter	5	Add summary page that can be accessed during encounter
	Confusion of previous and current vitals field	4	1) Load previous vitals from server 2) Enlarge header text
	Confusion of previous and current encounter date field	4	Do not default previous encounter date to current date
	Unable to locate symbol on keyboard	3	Move “.” button to number keyboard
	Difficulty changing date of referral	3	Help box with instructions
	Forced to schedule follow-up when none indicated	3	Only display follow-up page when indicated by algorithm
	Confusion with screen orientation	3	Lock screen orientation
	Confusion between device and ODK password	2	Synchronize passwords or alter length of passwords
	Confusion converting between days and weeks for last menstrual period	1	Provide both days and weeks on last menstrual period page
	Wrong encounter type selected	1	Reword encounter type options
	Uncertain if return date is for referral or return to clinic	1	Reword return to clinic appointment field
	Confusion with add group feature	1	Reword add group feature notification
	Participant attempts to minimize keyboard before advancing	1	None
	Uncertainty of referral form	1	Reword referral form

The cause-of-error analysis of the critical incidents resulted in 23 design change suggestions. A severity ranking of the critical incidents found five incidents resulting in task failure. The most frequently cited task failure was “Difficulty Selecting Form”. In this critical incident, nurses were unable to initiate an encounter in DESIRE due to confusion about how to proceed with selecting an encounter form (Figure 2).

Figure 2.

Screen shot of the “fill patient forms” screen that resulted in inability of the user to select the proper form and advance to the next screen, leading to task failure.

In order to correctly advance to the next screen a user must first select “fill patient forms”, which brings up a pop-up menu, then select “Dispensary Form (323)”, then select “Fill Form”. During the think aloud exercises, nurses frequently asked the observers how to proceed when reaching the “fill patient forms” button. Several mock patient encounters resulted in task failure because nurses were unable to execute this sequence and the encounter was unable to proceed. Based on these observations, research team members generated several candidate solutions to this task failure. Taking into consideration potential limitations related to programing and the need to accommodate multiple forms, we ultimately arrived at a design change suggestion to “Remove Extraneous Steps for Selecting Form.”

Sixteen critical incidents were found to be serious problems. The most frequently identified serious problem was “Field Pre-filled with Incorrect Data.” Due to a programing error, fields for vitals and demographic information were prepopulated with incorrect data during the think aloud exercises and mock patient encounters. Nurses were found to compensate for this error by manually deleting the prepopulated data and inputting the correct information. This was noted to be a serious problem because, while it did not result in termination of the encounter, it interrupted the nurse's workflow and had the potential to introduce incorrect data into the medical record. The proposed solution to this problem was to “correct the programing error” by eliminating this bug.

Feasibility

Five themes were found to emerge during the analysis of the feasibility focus group and interviews. The themes were barriers to implementation, facilitators to implementation, provider issues, patient issues and feature requests.

Barriers to Implementation

The barriers that were identified in feasibility testing corresponded to several levels of the program's technical and human axes (Figure 3; Table 3).

Figure 3.

Barriers to implementation categorized into the Technical Axis and Human Axis. Frequency of barriers coded noted in parentheses.

Table 3.

Barriers to implentation, categorized into technical and human axes, with corresponding proposed solutions.

	Barrier to Implementation	Proposed Solution
Technical Axis	Conflicting modules on server	• Close collaboration with other teams using server
	Server internet connectivity failure	• Improve server internet connection and uptime
	Lack of reliable network coverage	• Work with network provider to ensure wireless coverage • Allow application to work in both online and offline modes
	Lack of data credit	• Regular schedule for data bundle transfer
	Intermittent hardware crashing	• Replace defective hardware and investigate cause
	Inability to create new patient records	• Possible biometric use to prevent duplication of records
	Usability issues	• (See Usability Design Change Suggestions)
Human Axis	Procurement delays	• Streamline procurement process
	Redundant paperwork For MOH reporting	• Program DESIRE to auto-generate reports
	Lack of IT expertise	• Focus on recruiting, training and retaining IT staff
	Mentors do not use DESIRE tool during mentorship sessions	• Train mentors to use DESIRE and encourage use with mentees
	Heavy nurse workload	• Ensure adequate staffing
	Difficulty learning DESIRE during initial use	• Improve training • Increase training Frequency
	Stigma of HIV services at the same site	• Community outreach and sensitization
	Interruption in patient-provider relationship while using DESIRE	• Nurse communication training

The most frequently cited barrier to implementation of DESIRE was “Cellular Network Issues”. At the outset of the project, the implementation team discovered that several of the proposed implementation sites did not have reliable wireless data coverage. Some sites were determined to lack 3G access and network speed was severely hampered by use of the slower 2G data network. Throughout the implementation period, the team consulted with the program's network provider to improve coverage to clinic sites.

Despite those attempts to improve coverage, four out of six nurses reported difficulty with network access at their sites. Other providers cited intermittent network availability. Disruption of connectivity to the system's server also affected network uptime. The feasibility focus group conducted two months after the initial interviews identified a resolution of network availability issues. However, we discovered that airtime credit for data bundles was felt to be insufficient, which disrupted nurses’ ability to download and upload data to the server via the wireless network.

A frequently cited barrier to implementation was identified as “Server Problems”. Throughout the development and implementation of the project, several issues involving the AMRS server negatively impacted DESIRE testing and rollout. Several different programs at AMPATH utilize the AMRS server. Modules from different programs running simultaneously on the same server resulted in inability for the DESIRE tool to connect with the server

Due the large number of patient records in the AMRS system, the DESIRE tool was programmed to create a cohort of patients consisting of only those living within each respective dispensary's catchment area. In order to reduce the number of duplicated or incomplete patient records and ensure data quality, the implementation team restricted the ability of nurses to create new patient records. This was based on the assumption that the majority of hypertensive patients would be referred by the home-based testing counselor within the dispensary's catchment area. However, a substantial number of patients had incorrect geographic information in the record, chose to seek care at a dispensary outside of their geographic locale, or presented at the local dispensary without having had community-based testing but with elevated blood pressure during the dispensary visit. These factors often led to an incomplete or incorrect patient cohort on a nurse's device. Since nurses were not allowed to create new records for these patients, they resorted to using paper-based forms and medical records. Nurses repeatedly stated that the inability to create new electronic records severely limited the utility of DESIRE.

In addition to barriers to implementation located along the technical axis, barriers were found to exist along the human axis. Administrative issues served as barriers to the uptake and utilization of the DESIRE tool. Various administrative requirements and regulations involved with the process of tablet procurement led to delays that resulted in a substantial time lag between training and implementation. This led to the need for repeat training and may have also affected morale of the dispensary nurses. In addition, transportation to the more remote clinical sites was identified as a barrier to rollout and implementation. However, this barrier was addressed during implementation by the procurement of a dedicated vehicle for the CDM team.

Clinical mentorship by physicians and clinical officers were included in AMPATH's CDM program to enhance nurses’ ability and confidence to manage hypertension. However, the clinical mentors did not use the tablets during site visits and instead managed patients with paper forms. Nurses reported that having mentors who did not use DESIRE hampered their ability to learn to use the tablet. Nurses requested that their clinical mentors use DESIRE during their mentorship visits.

Nurses frequently stated that the dispensaries where they work are extremely busy with many patients and various other competing obligations, including management of other clinical conditions, which have affected the implementation of DESIRE. In addition to using the tablet to manage hypertension, they are required to submit monthly reports on paper to the Kenyan Ministry of Health . The administrative obligation of reporting to both the CDM project and Ministry of Health was referred to by one nurse as “double work.”

Facilitators to Implementation

Several technical factors proved to be facilitators to DESIRE's implementation. Electricity was reported as reliable in all of the clinics. Nurses did not experience any issues with charging the tablets or problems with tablet battery life. Only one hardware problem was noted during the initial rollout, in which a tablet could not be turned on, which was addressed by replacing the tablet. No subsequent hardware issues have been reported. Program managers found the experience of their colleagues in the development and implementation of other mHealth programs to be a helpful facilitator. They were able to consult with colleagues in other AMPATH programs using mobile technology to help troubleshoot when technical issues arose.

DESIRE was found to have several key advantages over paper charts. Overall, five out of the six nurses interviewed preferred using the tablet to the paper form. Nurses reported that, in contrast to using a paper chart, DESIRE helped to organize their files and provided a helpful offsite backup. In addition, nurses frequently stated that using DESIRE enhanced patient privacy. In contrast to paper forms that they believe could be inappropriately read or stolen, DESIRE's secure password and storage in a safe instilled a greater feeling of health information security:

Participant:.... it's confidential

Moderator: confidentiality is priority, okay, than those files that you have all over the room, anybody can come in

Participant: even the cleaner can sink into the papers (laugh)

Provider-specific Issues

An important factor in the introduction of a new technology is the learning curve required for its adoption. Prior to receiving the device, many nurses felt that using a handheld device would be more time-consuming and confusing than managing hypertensive patients with the previous method of a paper form.

Participant : I thought it was going to be a lot of work.

Moderator: In what way?

Same participant: Like now you are now coming to write to the tablet, go through the tablet, like when you are seeing a patient then you swipe the tablet, the queue is outside, so I was thinking maybe it will be a lot of work for me.

Upon initial use, most nurses found DESIRE to be more time-intensive and challenging than the paper form. Despite an initial training, the first few attempts to use the device proved confusing. Nurses had difficulty navigating the DESIRE interface and entering data. However, the nurses reported being able to quickly learn to use DESIRE with continued use. Overall, nurses found DESIRE to be initially challenging to learn but, after one month of use generally preferred it to paper forms and would strongly recommend it to their colleagues.

Participant : okay just like she has said, I thought it would really give a lot of hard time, going through the paper work then the tablet, but with time we've found it easier to use the tablet than the paper work.

Participant: I think it's not a complicated thing; yes it's easy to use it.

Participant: with time it will be easier to use the tablet rather than paper work.

Several trainings were held to teach nurses how to use DESIRE. Various delays in implementation, as stated above, led to trainings taking place far in advance of the actual DESIRE rollout, which led to decreased skill, confidence, and morale. However, trainings were identified as a major source of motivation for the participants. Nurses requested continual trainings with DESIRE and preferred using mock patients to simulate use instead of theoretical instruction. Nurses also endorsed peer learning as a critical component of learning to use DESIRE.

Participant:... there are those who are sharp with the tablet they can show some of us things...So maybe during break you find someone they explain to you.

Nurse confidence with the DESIRE was impacted by many factors. As anticipated, nurses’ previous experience with smartphones was found to relate to confidence in learning and using DESIRE. Several of the barriers to implementation detailed above, including network and server problems, decreased nurses’ confidence in the device. In addition, lack of feedback to the user by the software application, specifically the lack of notification that data successfully synced with the server, also decreased confidence with the device. However, nurses found that continued use with the system led to increased confidence. Participants approximated that it took three to five encounters with patients in a clinical setting to instill confidence.

Participants’ responses regarding the time required to use DESIRE for a clinical encounter varied. All nurses stated that initially, using DESIRE to treat hypertensive patients was slower than using a paper form. Reasons for DESIRE slowing down an encounter included the requirement that every part of the form be completed, as well as unfamiliarity with the interface. DESIRE was found to speed up clinical encounters by reducing the amount of time needed to locate previous documentation and by pre-populating demographic data. Estimates for the amount of time required for a typical DESIRE encounter ranged from five to 20 minutes while using a paper form ranged from three to thirty minutes. During the initial feasibility interviews four out of six participants found the tablet to be slower than the paper form with the remaining two finding it faster. This did not correlate to the number of patients the nurse had seen while using DESIRE. During the follow-up focus group, participants came to a consensus that using DESIRE was the same or faster than the paper form.

Participant: I think when I use the tablet am able to trace that patient faster than going to the files, looking for the files, feeding the files.

Participant: I think it has assisted a lot, because with the tablet, it's much easier and it makes work easier rather than the paperwork, because if I record now the patient on this one, I don't need to go on another sheet of paper when that patient comes, I just look for the name on the tablet and then continue.

In addition to impacting encounter speed, DESIRE also altered nurses’ workflow. Prior to DESIRE implementation, a patient assistant recorded patients’ vitals and demographic information on a paper form. However, since the vitals need to be entered directly in DESIRE, nurses have adapted their workflow in different ways. Some nurses have trained the patient assistants to input vitals into DESIRE while others record information on paper and then input data into the system after the encounter.

Nurses found DESIRE's decision support to be helpful in the treatment of hypertensive patients. The decision support was perceived as making the encounter easier for the nurse, as well as improving the overall quality of care. Nurses found the decision support to be empowering by providing recommendations for treatment and justification for referral. In addition, several nurses stated that the requirement to complete all screens in a sequential order helped to ensure that key clinical observations were not missed.

Participant: ... as time goes by after using it, it build[s] confidence in me because when you are using it, it leads you to the answer, like if the patient is hypertensive, like the pressure is this, if you insert like type the blood pressure for that day, it will automatically tell you whether to refer or manage, so confidence.

However, the assessment of decision support was not universally positive. In one instance, a nurse stated that she had encountered a recommendation with which she disagreed. She was uncertain how to proceed when her clinical judgment had conflicted with the clinical decision support.

Patient-specific Issues

The DESIRE tool and its ability to provide decision support may impact the relationship between patients and the nurses. While not a universal sentiment, several nurses felt that, during the initial period of implementation, the focus on the tablet may have negatively impacted the patient-provider relationship. Nurses expressed concerns that they spent more time focusing on the screen which interfered with picking up on non-verbal clues from the patients.

Participant: the relationship now has reduced simply because we concentrate too much in the gadget. The patient likes to talk to you directly but you, you just concentrate in the gadget.

Nurses who were concerned about the device negatively impacting their relationship with patients felt that this barrier could be surmounted with continued use and increased familiarity with the device. In fact, some non-hypertensive patients have inquired as to why they are not treated with the device.

Participant: they are even saying TB[Tuberculosis] is chronic, why don't we have our tablet

Nurses felt that patients viewed the device as futuristic and improving patient care. While the device has pre-programed decision support and does not have telemedicine features, some patients believe that the device connects in real-time with the tertiary care hospital and that they are receiving advice from a physician. One nurse has observed patients traveling from outside their catchment area to her facility for treatment because the clinic has the DESIRE tool.

Participant: in fact they are thinking, they are saying we are connecting to Referral [the tertiary hospital], that's what they think. And feedback is coming so the management is from daktari [doctor] from Referral (laugh) [...] this tablet is sending information to daktari in Referral and he is sending back feedback so it's like he is being seen by doctor in Referral (laugh)

The DESIRE tool was designed to treat hypertensive and diabetic patients, and its algorithms are not designed to integrate other co-morbid conditions into hypertensive management. Several of the hypertensive patients identified at clinics also are HIV-positive. Currently, patients who have both HIV and hypertension are not treated with DESIRE, but are instead treated with paper forms by clinical officers.

Feature Requests

Throughout feasibility testing, several additional features of DESIRE were requested by nurses or identified by the implementation team. One of the most frequently requested features was a summary screen that nurses could consult during the encounter, since this information was only accessible by swiping backwards through previously completed screens. Nurses stated that several times during encounters they would wish to review the patient's vitals or symptoms, but were unable to quickly refer back to the information. Nurses also requested the ability to see the trend of vitals and medications over time to better understand a patient's clinical course.

Participant: when theses clients come, we can only manage to see the previous blood pressure, but now I can't be able to compare with the very initial, so we were requesting if there can be any way I may see or the first one was like this, this one is the previous one and this is today's. It will assist me to know whether we are progressing or its fluctuating high, down, high, down, that was our request.

Several nurses reported interest in treating more patients with DESIRE. Nurses asked that DESIRE's decision support be expanded to treat patients with comorbid conditions such as HIV. They also asked for a decision support tool for use in the antenatal setting. They requested the ability to create new records for patients without previous AMRS records. Furthermore they requested that the DESIRE tool be programed to generate reports for the MOH. Additional requests included confirmation that a sync with the server was successful, a free text field to make notes and the ability to search the internet from the device for clinical information.

Discussion

This study assessed the usability and feasibility of implementing a tablet-based integrated record-keeping and decision support tool to assist in task redistribution of chronic disease management in a low-resource LMIC setting. Fifty-seven critical usability incidents were identified, and many of these were found to be due to user interface problems or confusion in wording. A cause-of-error analysis yielded 23 design change suggestions, including rewording text, removing extraneous steps from processes, standardizing responses across questions and changing tablet defaults to reduce confusion. Feasibility testing identified five domains that impact the acceptability and infrastructure of the DESIRE tool: barriers to implementation, facilitators to implementation, provider issues, patient issues and feature requests. Overall, the most frequently identified barriers to implementation were network and server issues; however, human factors such as administrative issues and nurse concerns also featured prominently. To the best of our knowledge, this is the first report of usability and feasibility testing for mHealth interventions in SSA aimed at non-communicable diseases such as hypertension.

Our results support the findings of usability studies that have been conducted for other disease entities in SSA. For instance, a usability study of ODK Collect for population HIV surveillance by AMPATH identified issues with font size, lack of intuitive navigation, slow GPS, and coordinate acquisition lack of location hierarchy. The study resulted in design changes to the user interface, improved logic, the ability to save forms in progress, and disallowed non-sensible values.[25] Similarly, a usability study of a mobile health application for Ghanaian midwives found difficulty operating a small onscreen keyboard, selecting buttons on a smartphone screen, a safety risk in lack of application password, and difficulty adding new patients to the system.[37] Another study of an mHealth application for tuberculosis treatment in rural South Africa found poor uptake among healthcare workers despite high reported acceptability, largely due to difficulty uploading patient information.[38]

Similar to the usability findings, we identified technical factors affecting feasibility that are in line with what has been previously reported in other settings. These include lack of broadband internet access, inconsistent or limited network availability, transmission error detection and management, ensuring patient privacy during wireless transmission, phone security and sharing, and information security.[1] Our study identified several of these challenges including inconsistent or limited network availability, transmission error detection and learning environment, but did not detect concerns regarding privacy, security, or interoperability.

However, we also identified a variety of human factors that impact feasibility, which have been less thoroughly explored in the literature. A frequently cited barrier to implementation was the heavy workload of the rurally based nurses. Task-redistribution of hypertension care to nurses creates more obligations for nurses. This additional responsibility comes along with increased administrative needs including reporting data to the ministry of health. A handheld device in the midst of this work setting may feel like an extra burden in addition to an already increasing workload. By aligning tablet-based functionality with ministry of health reporting requirements, it may be possible to address the workload issues of nurses while simultaneously improving device use and care quality.

Another human factor was identified as the stigma associated with HIV programs run by AMPATH, the implementing organization. Nurses believe that hypertensive patients may avoid seeking care if they could be falsely identified as having HIV, and HIV patients may avoid care because of a loss of anonymity. This calls for the need of careful system-level planning, as well as community outreach, on top of addressing the full gamut of technical issues.

The introduction of a new technology into the clinical encounter has the potential to profoundly impact the patient-provider relationship. Some nurses felt that the initial use of the DESIRE tool slowed down the encounter and forced them to focus on the device instead of the patient, missing important non-verbal clues. However, nurses felt that with acclimation, the tool allowed them to provide better technical care to their patients. According to the nurses, patients felt that they were receiving better care with the use of a “futuristic technology,” which may have partially reflected an inaccurate perception of the actual features of the device with some thinking that they were receiving a direct telemedicine service connected to a referral hospital. Indeed, obtaining patients’ direct perceptions and interpretation of experiences of the DESIRE tool would also be critical to understanding long-term community acceptance or rejection of the intervention.

In summary, identifying usability and feasibility issues are important not only to improve the overall user experience and deliver superior patient care but may also be critical for programmatic success. The device users proposed several feature requests to improve workflow and reduce the need for redundant reporting. Most of the design change suggestions and feature requests have been incorporated into updated versions of the tool that AMPATH has subsequently developed.

Limitations

While this study was designed to capture many of the potential issues regarding the usability and feasibility of the DESIRE tool, there are several limitations to our approach. The study was qualitative and therefore hypothesis-generating rather than hypothesis-testing. In addition, cost information was outside of the scope of this study and therefore was not evaluated at this time. Overall, the ability to conduct the study was pegged to the implementation of a new technology in a resource-limited setting. Delays in implementation for the CDM program resulted in variability of the duration and intensity of nurse use of DESIRE prior to testing. The clinical algorithms were designed using the then-current JNC 7 guidelines and are currently being revised in light of the recent publication of JNC 8. In addition, the possibility of non-compliance of the nurses with the decision support recommendations of the DESIRE tool in clinical practice was not evaluated in this study. Finally, as stated above, direct patient perceptions were not included in this study.

Conclusion

As we have demonstrated, the use of this participatory, iterative, human-centered design process has revealed previously unaddressed usability and feasibility issues affecting the implementation of the DESIRE tool. The critical incidents identified resulted in substantive design and development change suggestions, highlighting the importance of usability testing as part of implementing mHealth systems in LMICs. Feasibility testing identified barriers that were both unique to this specific setting, as well as barriers that have relevance for similar programs in low-resource settings worldwide. In addition to confirming the presence of previously reported technical issues, we highlight the importance of human factors that can impact an mHealth intervention's implementation success. Our study thus illustrates the need to identify usability and feasibility issues, not only to improve the overall user experience and deliver superior patient care, but also to ensure overall programmatic success in similar low-resource settings.

Highlights.

• We report usability and feasibility testing of a mHealth intervention in Kenya

• Usability testing revealed critical incidents and generated design changes

• Feasibility testing identified human and technical barriers to implementation

Summary Table.

What Was Already Known on the Topic

• Relatively few usability and feasibility studies have been conducted of mHealth interventions in sub-Saharan Africa, especially related to non-communicable diseases.

• Technical challenges that have been previously identified include: lack of broadband internet access, inconsistent or limited network availability, transmission error detection and management, transmission of large files, ensuring patient privacy during wireless transmission, information security, phone security and sharing, and usability issues.

• Administrative challenges that have been previously reported include: lack of business model, insufficient strategic leadership, absence of a change management plan, difficult learning environment, limited buy-in from practitioners, high staff turnover, and limited eReadiness.

What This Study Added to Our Knowledge

• To the best of our knowledge, this is the first report of usability and feasibility testing for mHealth interventions in sub-Saharan Africa aimed at non-communicable diseases such as hypertension.

• Usability testing revealed critical incidents impacting intervention success, and a cause-or-error analysis generated corresponding design changes.

• Feasibility testing confirmed the importance of technical factors that may impact the success of an mHealth intervention.

• In addition, we also identified a variety of human factors that impact feasibility, which may have relevance for similar programs in low-resource settings worldwide.

Abbreviations

AMPATH

Academic Model Providing Access to Healthcare

AMRS

AMPATH Medical Record System

CDM

Chronic Disease Management

CVD

cardiovascular disease

DESIRE

Decision Support and Integrated Record-keeping

HIV

human immunodeficiency virus

LMICs

low- and middle-income countries

mHealth

mobile health

ODK

Open Data Kit

SSA

sub-Saharan Africa