Converting and expanding mobile support tools for tuberculosis treatment support: Design recommendations from domain and design experts

Abstract

Background:

Tuberculosis (TB) remains one of the top ten causes of death globally despite it being largely treatable. Poor adherence to treatment directly contributes to poor outcomes, such as, prolonged infectivity and the development of drug resistance. Mobile phone-based interventions have the potential to improve treatment outcomes.

Objective:

The purpose of this study was to solicit design and domain expert feedback of a previously developed TB support intervention converted to a mobile application.

Methods:

We used prototyping in iterative cycles that included integrating findings from prior formative research with endusers and soliciting feedback from design and content experts. In this project, we used low-fidelity prototype evaluation to inform the design of high-fidelity prototypes for further testing and iterative refinement.

Results:

We received 12 survey results. Overall, the participants agreed that the functions would be easy to learn/use. Recommendations for improvement included: simplify the reporting by offering broad categories; split complex screens to be more intuitive and user friendly; modify feedback graphics to display data more clearly; incorporate instructions for each task/function to guide users and collapse the information once users had viewed it; display navigation icons on each screen and add a main menu button; have medication tracker be homepage and limit redundancies. Several potential functionalities were suggested, such as adding a notes/journal and a social feature. We were able to easily incorporate recommendations and feedback into the high-fidelity prototypes and continue testing and refinement. After we came to a stable prototype through testing, we gave the interactive prototype to our developers to program a base functioning model.

Conclusion:

The proposed design recommendations provide valuable insight to inform initial conversion of an interactive intervention to customize patient support, which include a smartphone app and a direct drug metabolite test reengineered for home use. We argue that iteratively developing low- and high-fidelity prototypes with content and design experts to guide initial programming of a functional beta app paves the way to better explore further refinement needs and recommendations with endusers rather than using hypothetical scenarios.

INTRODUCTION

TB remains one of the top ten causes of death globally.[1] The over 4,500 deaths per day associated with TB is recognized as unacceptable given that the majority of the deaths are preventable.[2, 3] Medication adherence to TB treatment is known to be complex and challenging.[4] For example, TB patients may face stigma associated with the disease largely associated with poverty, travel burden to healthcare facilities, lack of understanding about the disease and its treatment or low health literacy in general, and unmanaged medication side effects.[5–7] Additionally, health care systems often struggle to identify the patients non-adhering and return them to treatment due to human and financial resource limitations.[8, 9] Failure to complete the long course of treatment undermines the health care provided and increases the risk for poor outcomes for the individual and the community, including continued transmission, treatment failure, and the development, propagation of drug resistance, and increased costs.[10] If efforts go unchanged, it is estimated that by 2050, multidrug-resistant TB could kill as many as 2.5 million people per year and cost the global economy up to US$16.7 trillion.[11] To address the global health emergency that TB represents, the WHO End TB Strategy has set goals to reduce deaths and incidence levels by 95 and 90 percent by 2035, respectively, relative to 2015 levels.[12] To meet these goals there is substantial interest in the use of mobile health (mHealth) interventions to address challenges to treatment adherence, provide patient-centered care, and support healthcare systems.[13, 14]

The proliferation and access to mobile phones and nearly ubiquitous mobile network coverage has helped fuel the enthusiasm for mHealth interventions in low- and middle-income countries where the heavies burden of TB is felt.[15] mHealth interventions emerged as simple text message reminders[16–18] with some study results supporting their impact on improving health outcomes.[19] However, limitations to texting based intervention were noted, such as, uncertainty if messages were read, difficult linking the message to other resources, and limited character count to provide context for the message,[18] as well inconclusive efficacy data.[20–22] As mobile phone technology has advanced there has been a transition to smartphone based interventions built using mobile applications (apps).[23] mHealth apps allow for multiple functions and sophisticated features, such as, patient initiated treatment reminders, chat functions, and tailored education about taking medications as prescribed.[23] App-based interventions have been increasingly tested to support individuals with chronic conditions, such as, asthma,[24] cardiovascular disease,[25] diabetes, mental health,[26] and for smoking cessation.[27] Despite the growing number of health care related apps, estimated at over 325,000,[28] there have been limited patient focused apps rigorously developed and evaluated to support individuals with active TB.[29]

We are using an iterative design process to develop a mobile intervention consisting of a TB support app and direct metabolite test.[30] The goal of the app was to provide patient-centered support to individuals and a mechanism by which a TB team can readily identify individuals struggling to adhere to treatment and in turn, improve treatment outcomes. In this manuscript, we describe a study that solicited design and domain expert feedback on initial low- and high-prototypes of the app.

MATERIALS AND METHODS

To facilitate a user-centered design, we used prototyping in iterative cycles that included integrating findings from prior formative research with endusers with TB [31–33] and soliciting feedback from design and content experts as a first step. In this project, we used low-fidelity prototype evaluation to inform the design of high-fidelity prototypes for further testing and iterative refinement with a new cohort of patients. This prototype process helps solidify high level concepts, identify and solve major usability problems, and minimize cost and time for programming rewriting to design a product more closely representing the final version.[34] The participants in this study were students or graduates in Human Centered Design and Engineering at the University of Washington with experience building or evaluating mobile applications (N=9) and TB experts (N=3) from Argentina, a country with a high TB burden and low treatment success rates.[35] We developed an online survey study to gather input on the usability, design, and content of the low fidelity app mockups.

Previously, we elicited user requirements and information needs through the rigorous development and testing of TextTB, a texting-based intervention developed with patients and experts and guided by the Information, Motivation, Behavioral Skills (IMB) Model.[31–33] The IMB model asserts that to effectively self-manage treatment appropriate information about the disease, treatment, personal and social motivation, and adherence behavioral skills to achieve sustained adherence behavior and targeted health outcomes are needed.[36, 37] The IMB model formed the basis of our approach to guide interviews, intervention content development and messages to be used in the app. Prior work was also conducted to understand clinical workflow and identify health provider challenges, such as tracking and bringing patients back to treatment.[31, 33] In summary, between 15 to 60 minutes per day was required for the treatment supporter to review participant reports and respond to questions or issues.[31] This time could be broken up into smaller short time intervals between other work responsibilities. During testing of the texting-based intervention, patient began using a free app for communicating, WhatsApp, in place of text messaging thus the impetus for converting and expanding the interactive patient support intervention to meet changing mobile phone preferences.[31] Using the base functionalities of the TextTB intervention developed with patients and experts, we designed low-fidelity sketches by drawing ideas and designs of the communication, education, and reporting functionalities and added a direct-adherence test component. After analyzing recommendations and establishing basic layout and functionality, we designed an interactive high-fidelity prototype using the web-based interactive prototyping software Axure (Axure Share Software Solutions Inc.). This interactive design consisted of screen design images (wireframes) with established flow and interactivity that imitates the look, feel, and navigation of a functioning app (e.g., clicking on functionality icons such as information led to the screens with education, images and further hyperlinks). This software provided the appearance of using a functional app and let reviewers add comments and recommendations directly to, for example, an icon or screen. Next, the research team and TB experts reviewed the high-high fidelity application to identify potential problems, simplify, and reorder the flow of the functionalities. A research team member, who is an expert in human-center design, was able to rapidly integrate the recommendations into the high-fidelity prototype and share with programmers. Figure 1 shows a general overview of the methodology.

Figure 1.

Iterative development process

Recruitment.

Design expert participants were invited in an email announcement through a university listserv of the University of Washington Human Center Design and Engineering school. Of the 14 design trainees or graduates who responded with interest to evaluate the mockups, 9 completed the survey. Their experience in designing applications ranged from one year or less (n=1), 1–2 years (n=6) and 3–5 five years (n=2). The domain experts included a Regional TB director/pulmonologist (with over 30 years of TB management experience) and two primary care physicians involved in TB research (each with over 20 years of practice and 10 years of TB research focus).

Instrument.

The survey included screenshots of the mockups and questions related to the proposed app’s functionalities: medication adherence administration, side effect monitoring, interactive communication, direct drug metabolite testing, and a general section for specific design questions and were guided by the Mobile Application Rating Scale (MARS).[38] Although the MARS tool is intended for the evaluation of developed applications, we found the tool and its domains helpful in guiding some of the questions for evaluation at the early design stage. The survey questions focused on usability, clarity of instructions, and requests for design suggestions, such as, for layout, fonts, and color scheme. Other questions inquired about functionalities that could be added, or ways to reduce potential application usage fatigue or improve photo quality for the metabolite testing. The majority of the questions were open ended to allow the participants to freely state their opinions and to provide detailed answers.[39, 40]

The online survey instrument was created in REDcap, a secure web-based application for building and managing online surveys or databases (https://www.project-redcap.org). The survey was emailed to those who offered to participate along with instructions and was intended to take approximately 30 minutes to complete. The survey was translated to Spanish for the domain experts. Participants who completed the survey received a $15USD gift card as a token of appreciation for their time.

Data analysis

Open-ended survey questions were analyzed using descriptive analysis[41] using Nvivo (QRS International) software. Findings were grouped into the four MARS dimensions: Engagement (interest/entertainment, customizability, interactivity – alerts, feedback, reminders), Functionality (ease to learn, buttons and menu, navigation, flow logic, gestural design), Aesthetics (layout, graphic design, overall visual appeal, color scheme, stylistic consistency), and Information (quality and quantity of information, visual explanation, credible sources).[38]

Ethical consideration

This study was reviewed by the Institutional Review Board at the University of Washington and was determined to qualify for exempt status (STUDY00003401).

RESULTS

We received 12 survey results. Overall, the participants agreed that the functions would be easy to learn/use. The survey results provided several recommendations for improvement. Table 1 provides a summary of the feedback and recommendations followed by more detail of the findings within each domain.

Table 1.

Summary list of recommendations to improve design

To improve
	Simplify the reporting by offering broad categories
	Split complex screens to be more intuitive and user friendly
	Modify feedback graphics to display data more clearly
	Incorporate instructions for each task/function to guide users
	Collapse the information once users had viewed it
	Display navigation icons on each screen
	Add a main menu button
	Limit redundancies
Additional functionality
	Medication tracker be homepage
	Notes/journal
	Social feature

Engagement

The interactivity could be improved by supporting audio or chatbot communication options with healthcare professionals and adding feedback from healthcare professionals and/or caseworker. Additional feedback could include adding a descriptive summary to the visual summary of data and a calendar with days marked according to reporting (e.g., green if reported taking medication, red if did not report). Interactive communication must always use an encouraging tone and not place personal judgment on the user. To support entertainment/interest it was suggested to include gamification strategies such as ways to earn points or rewards (virtual or real) for users when they use the app daily to buy wallpapers or ringtones so that the user can change the way the app looks.

Functionality

Overall the participants agreed that the functions to enter/self-report daily medication administration, report side-effects of medication and take the TB drug adherence test appear easy to learn/use. The function to communicate with a treatment supporter received the most neutral ratings (some suggestions for improvement described in Engagement). To improve the functionality, it was recommended to simplify redundant language by, where possible, collapsing information once the user has viewed it and minimize notifications to help reduce potential user fatigue. Examples of simplification statement are: “Patients shouldn’t feel like they are wasting time” and “the app should become something they do extremely quickly or barely thinking about it during their daily life routine.” To improve navigation the complex screens should be split and could have a progress bar. The navigation buttons need to be on each screen and a main menu button added. The spin control button was recommended to be avoided. The time selection for reporting medication administration should be customized to the local preferences, a 24-hour clock and allow option to select a broad time of day (e.g., morning) rather than the specific time. It was recommended that the medication tracker be the home screen. A common suggestion was to add a notes/journaling function, and self-selecting alarm, and social feature. Ideally the app will have the functionality to analyze the test results and request that the user complete the test on a random basis.

Aesthetics

To improve the graphics in the app mockups it was suggested to display the graphic feedback more clearly. The displays for medication self-reporting in a line graph, the pie chart display of test results, and the star on the medication tracking progress screenshot (to depict historic percentage of reporting) were considered not easy to understand. Suggestions included converting to bar or dot graph. The working “nearly always” in the medication tracking progress feedback text, was considered vague and not allowing for realistic/actionable feedback. To improve the visual appeal participants recommended white or a light-colored solid background and avoiding bright or flashy colors that might strain the eyes. It was suggested to make sure the background passes W3C accessibility guidelines (w3c.org), an international community to develop Web standards. Regarding the layout, the font should be San Serif font (e.g., Helvetica, Calibri, Verdana, or Geneva) and at least 14-point font size for easy readability. Limiting the use of red unless trying to point out something very important and/or urgent and using italics sparingly was recommended. Adequate line spacing between words were recommended to make the screens not seem so cluttered and easier to read.

Information

To improve information quality suggestions highlighted the need for additional instructions and guidance on how to use the app (e.g., how to report taking medication, side effects, take test). For side effect reporting, users must be able to select all that apply and visual information (e.g., images) could be added to differentiate between some side effects (e.g., rash and hives). Content experts indicated that non-technical terms should be used for the side effects (e.g., redness of the skin rather than rash). Further guidance should be added on how to respond if experiencing a potential side effect, for example an alert, “If you are having problems taking your medication, please go to your health care center or send a message through the app for a care coordinator to contact you.” Information was requested on which symptoms require health professional evaluation, and actions or suggestions to take to help prevent or relieve a symptom or side effect. An alert was recommended to notify users that the tool is not for emergency use and interactions would only take place during clinical hours of operation. Other additional information recommended was regarding what is acceptable while on the medications, such as, diet, amounts of exercise, and/or foods that should be avoided. Additional explanation was recommended by both design and content experts on what a positive and negative test result means for the direct metabolite test and suggested that a positive test could be understood as a bad thing. To guide the image capture for the test strip, it was recommended to either add a translucent overlay or an outline of the test strip, so the user would know where to take the photo.

Review of app in production

When reviewing mPOWEr, the app in production being adapted for TB, participants indicated that the scroll bar to select the severity of various symptoms (e.g., pain level, temperature) could be used for the TB support app for symptom reporting. Other suggestions were to allow users to upload pictures and explanations of their symptoms that would allow them to create a diary, similar to the one in mPOWEr. One of the participants also commented on how mPOWEr has all their features separated into other subheadings, which allows for ease of use and access.

Because prototype development was relatively simple and had a foundation in prior enduser research, we were able to easily incorporate recommendations and feedback into the high-fidelity prototypes and continue testing and refinement. Figure 2 shows an evolution of major revisions based on survey feedback and testing. After we came to a stable prototype through testing, we gave the interactive prototype to our developers to program the base functioning model.

Figure 2.

Low-fidelity mockups and high-fidelity prototypes. Circles highlight issues identified and where Note: there were other versions, but these represent major revisions after feedback.

DISCUSSION

In this paper, we describe how we used low- and high-fidelity prototyping to refine the functionality and design of a mobile application to support individuals in active TB treatment who receive self-administered therapy. Because TB patients often report physical challenges and social isolation during treatment[42, 43] a primary goal for the refinement and expansion of this intervention is to improve adherence and treatment success by providing interactive support to help minimize the extent of social isolation, resolve issues timely, and improve health literacy about the disease and its treatment. The intervention also aims to increase patient engagement in self-management and development of skills to maintain adherence by capitalizing on the wide adoption of smartphones globally and their advanced functionalities.[44, 45] Given the ongoing burden of TB globally, mHealth tools have presented hope to improve TB treatment outcomes and more easily provide a way for over-burdened healthcare systems to monitor and track individuals in treatment and provide extra support where needed.[13, 14] However, limited research has been conducted to develop and evaluate patient focused TB apps.[29] We chose to start by building on prior research with patients and experts,[31–33] using an interactive approach to respond to changing needs and phone use of patients. We found the iterative approach to be useful because we were able to quickly get feedback and test and refine design and content ideas. It was also essential to have subject matter experts involved in the design and development and iterative testing. Several actionable recommendations were identified, and a more refined high-fidelity prototype produced to inform a functional base model production. In addition, the IMB theoretical model was useful in guiding the interpretations and design strategy and IMB based messages were incorporated into the low-and high-fidelity designs. Similarly, the theoretical model has served as a guide for other app development.[30, 36]

Our study demonstrates the value of iterative design with design and content expert feedback. Recommendations and suggestions covered a comprehensive set of considerations pertaining to usability and design, engagement, and additional functionalities to provide support as well as the overall user experience. This approach allows for a more effective implementation avoiding the cost of designing a system in silo and then deploying it only to find out that it does not meet significant needs or standards. For example, to maintain engagement with the app, experts suggested motivational language, positive encouraging feedback, simplification of the reporting process, and gamification options. Maintaining interest in using an app routinely over a long period is a recognized challenge that must be maintained in the forefront of design.[24] Forgetfulness is a commonly cited reason for non-adherence, thus offering reminders or follow-up may help mitigate this barrier.[24] Adding pictures or further information/definition of symptoms was suggested to allow users to distinguish between different side effects and better understand if what they might be experiencing is a normal or abnormal side effect of the treatment. A journal function to self-track treatment progress and side effects was suggested as a way for users to record how they are feeling. Logged accounts of self-reported side effects could potentially be shared with their treatment provider to track trends in side effects as well as adherence. A social forum for patients to interact with others was suggested and has been recommended in mobile apps for other conditions.[46, 47] Researchers have cautioned that ongoing moderation by a healthcare professional could be needed to ensure accuracy of information or shared recommendations could.[48]

One of the main challenges identified in producing high-fidelity prototypes is that it appears to be a final product; however, there may be performance issues or complex functionality not detectable that could affect usability or satisfaction. [34] Therefore, it is important to recognize that high-fidelity testing is not a final stage and further usability evaluation is needed. Various methods are available for evaluation and usability testing with no one being designated as the best.[30, 34, 49] Each project may benefit from different evaluation strategies during development. Guidance for development in global health mobile interventions have been described.[50, 51]

Limitations

While the survey approach followed in this study has considerable benefits given the limited understanding in this area and the exploratory nature of this work, it also has limitations. The sample size was small and the first conversion step was not conducted with patients. Rather, the research team opted to build on prior similar work developing an interactive intervention with patients to establish base functionalities with which to begin the conversion process. Thus, this research was considered a preliminary stage. This process allowed for the planning of focus groups and field-testing with a new cohort of patients using a beta app and taking the urine metabolite test daily. We anticipate better discussions after field-testing as participants can get a sense of what it would be like using the tools rather than simply supplying hypothetical situations. Next steps are to build a fully functional app and improve the home metabolite test and assess the extent to which the recommendations proposed impact usability, user acceptance in practice in a new subset of patients initiating treatment. We will also identify further refinement needs based on a large sample and longer study duration. In addition, the assessments were conducted online through survey with low-fidelity and later with high-fidelity software. Ideally, usability testing should be done within the intended setting while the user is confronted with usual tasks.^[52] This iterative approach is accompanied by understanding that further refinement will be needed. The follow up assessments with a new cohort of patients will help to identify further needs and recommendations for refinement prior to pilot testing.

CONCLUSIONS

The work reported here reflects a multidisciplinary collaborative effort to develop tools to customize patient support, which include a smartphone app and a direct drug metabolite test reengineered for home use, to achieve increased rates of TB treatment success. Convenient patient-centered strategies are needed to help maintain individuals in treatment to ensure treatment completion. We argue that iteratively developing a low- and high-fidelity prototypes with content and design experts to guide initial programming of a functional beta app paves the way to better explore further refinement needs and recommendations with endusers rather than using hypothetical scenarios.