Abstract
Background
The majority of new cancer cases are expected to be diagnosed in low- and middle-income countries (LMICs) by 2025, and 65% of cancer deaths currently occur in LMICs. Treatment adherence, patient monitoring, and follow-up are essential to cancer care but are often not possible in these settings. Out Patient (OP) Care, a smartphone application (app) developed to fill this gap, texts appointment reminders to patients and electronically stores medical records confidentially.
Objectives
This study aims to present the development of this app and evaluate its usability and feasibility as defined by provider and patient experiences in the context of a multidisciplinary cancer clinic in Gaborone, Botswana.
Methods
OP Care was piloted at a multidisciplinary team gynecologic oncology clinic in Gaborone, Botswana. The app was developed through an iterative process with feedback from clinic staff and physicians. The usability was evaluated using a cross-sectional survey. All staff members in the gynecologic oncology clinic, which typically consists of one doctor and four nurses, as well as a portion of the staff in the Princess Marina Hospital general oncology clinics in LMICs to remind oncology ward used the app. All providers using the app were surveyed, along with all patients who attended the gynecologic oncology clinic during the 3-week survey period. Staff demographics, reactions, and opinions on usability, as well as patients’ reactions to the appointment reminders were collected. Agreement to the ease-of-usability statements was recorded on a 1 (not at all) to 7 (extremely so) scale. Primary outcomes were the app’s usability and the feasibility of text reminders from the patient’s perspective.
Results
Nine staff and 15 patients were surveyed. Staff included three doctors and six nurses and encompassed all of the staff in the gynecologic oncology clinic as well as a portion of the general oncology ward. All surveyed staff owned a smartphone and used a computer at home. Most (78%) staff did not feel that OP Care would increase their work burden and were willing to use the app if implemented permanently (median: 6; interquartile range [IQR]: 1). Seventeen out of the nineteen usability questions, such as “I feel comfortable using this system,” scored a median of 6, corresponding to “very much so.” Patients reported that the reminder text messages were helpful (median: 6; IQR: 1) and preferred the text reminders to be in Setswana (median: 7; IQR: 1).
Conclusion
High usability scores indicate that the app can be scaled up to usage in this clinic and others. Although patients appreciate OP Care, the option for call and text reminders in Setswana is indicated.
Keywords: medical information, smartphone, mobile applications, electronic medical records, medical oncology
Introduction
Cancer burden in low- and middle-income countries (LMICs) continues to rise, with the majority of cancers by 2025 predicted to be diagnosed in LMICs.1 Although the total incidence of cancer in these countries is lower than that of high-income countries, the mortality rate in LMICs is disproportionately high, with 65% of cancer deaths occurring in less developed parts of the world.2 With this growing burden, it is critical that LMICs are equipped to fight cancer. When treating cancer, in addition to time of diagnosis, adherence to treatment and monitoring after treatment are critical for improving outcomes.
Adherence to treatment and follow-up of patients posttreatment are essential to monitor for recurrences, progression, and quality-of-life outcomes.3,4 Without these capabilities, life-saving chemotherapy or radiation may be delayed in diagnosed cancer patients, and recurrences in previously cured patients may be missed until they become too advanced to treat.5 Lack of centralized methods of patient tracking, retention, and treatment record-keeping is a challenge in many LMIC oncology clinics.6 Due to a multitude of factors including time missed from work, transportation, and finances, patients may not adhere to treatment and become lost to follow-up.5 In Botswana as well as many other LMICs, the patient’s medical records exist only with the patient in hard copy,7 making it difficult to deduce their staging and previous treatments if papers are misplaced. Providers may be unable to make informed treatment decisions, which can result in either overtreatment or inadequate therapy. Even in cases where an electronic medical record (EMR) is present, computers are often in limited supply, are out of date, and run on poor internet access.8 This makes regular recording of and access to patient health records almost impossible, especially at point of care.
Practically, this means that providers need a readily accessible, safe, and convenient place to store patients’ treatment information, relevant health records, and scheduling, as has been done in other specialties.9 Clinics also require a method of reminding patients of appointments, as well as a way of reaching out to patients who do miss appointments to ensure they receive proper care.9
Smartphone applications (apps) are at the forefront of health care innovation in low-resource settings, making them an obvious choice to tackle the aforementioned challenges. In Botswana, a pharmacy-based smartphone app to track treatment adherence in human immunodeficiency virus (HIV) patients has already been successfully piloted.10 Another app in Brazil used to support shared decision-making about thromboembolic prophylaxis in atrial fibrillation was successfully piloted on low-income patients with low educational attainment.11 Although text message reminders have been successfully implemented in several LMICs to increase patient attendance to follow-up appointments, they have not been paired with an effective tool to track these patients and store their records.12–15 The Out Patient (OP) Care smartphone app was developed by ONE BCG Pty Ltd from physicians and health care providers for oncology clinics in LMICs to remind patients of their appointments, track the progress and treatment decisions of oncology patients, and store medical records.
Objectives
In this study, we present the development of OP Care and evaluate its usability and feasibility as defined by provider and patient experiences in the context of a multidisciplinary cancer clinic in Botswana.
Methods
Development of OP Care
Before starting development on the app, data were gathered on cancer types, subtypes, staging of cancer presented to the clinic, and treatment intent from specialists and the American Joint Committee on Cancer Staging Manual. After background data were provided to developers, the app was developed over the course of 5 months, using feedback from physicians and health care providers at the Multidisciplinary Team (MDT) Gynecologic Oncology clinic. Multiple iterations of the app were tested to optimize its functionality before it was officially piloted. After piloting the app in the MDT Gynecologic Oncology clinic, feedback about its usage was sent to developers, initially weekly and later as needed. As feedback is given, the app continues to be updated. We tested using version 2.3.11. Version 2.4.13 is currently in use.
App Design
The OP Care app was developed to address the issues described previously. It is used by health care providers to store patient demographic and clinical data, make appointments, and automatically send text message reminders to patients about their upcoming appointments. Patients do not use the app themselves and therefore do not need smartphones to benefit from the text message reminders that the app can send to any phone with text capabilities. They are informed of the fact that their personal and health information will be securely stored on this app with their permission and that they will receive automated text message reminders about their upcoming appointments. Further details regarding the design of the app are presented in the following sections.
Security
The app stores patients’ protected health information, and therefore contains two levels of security (i.e., user’s email address and password). Users must register and be granted access to the information stored in the app, allowing for tight control of its use. No patient data are stored on the mobile device on which the app is being used. Once a session times out or is ended, the data are deleted and no written data persists on the device. Therefore, if a phone or tablet is lost or stolen, patient data are not accessible. Patient information is encrypted and stored on cloud servers, which are in a secure location, and physical access is restricted. The information is further secured with the use of Secure Sockets Layer (SSL) certificate technology. SSL implemented in conjunction with firewall restrictions ensures that all data transmission between the mobile device and the server are encrypted and cannot be interrupted while in transit from a network. Being a mobile app, it makes it convenient and portable while also being safe.
Patient Profiles
In the app, providers “enroll” patients by creating profiles for them where they can enter patients’ identification information including name, date of birth, national identification number, and picture. In effect, the app functions as an EMR if one is not readily available. It can hold the patient’s cancer diagnosis, stage, treatment plan or previously completed treatment, HIV status, and previous and future appointment dates. There is additional room for miscellaneous notes that do not fall in the aforementioned categories. The app also has the ability to take pictures of patients’ paper records and store them in their profile. This bypasses the often-impossible task of scanning these records, serving as backup storage for hard-copy records. To allow providers to more easily search for patients falling in different groups, patients’ profiles can be tagged with standardized tags specific to the clinic’s needs (e.g., referred for surgery, referred for colonoscopy).
Text Message Reminders
The primary function of the app is as a reminder system for patients, sending automated text messages 1 week and 2 days before their appointments. The text messages include a phone number to call in case they have any questions, making it readily accessible on the patient’s phone. Furthermore, it gives providers the ability to track their patients, monitor their progress, and ensure that no patient is lost to follow-up.
Scheduling
Additionally, the app gives providers the ability to schedule patients, check them in when they arrive, and complete the appointment when they leave. This allows providers to anticipate their workload for a given day and to call and reschedule patients who do not show up. Keeping track of which patients often miss appointments can also be used for a long term to identify potential barriers to accessing care.
Clinic Workflow with OP Care
Once a patient is enrolled and any additional medical information specific to that visit is added, their follow-up dates are entered in the app and the appointment is “completed.” The patient receives a text welcoming them to the OP Care system and then receives the reminder texts 1 week and 2 days before their appointment. On the day of their appointment, their profile appears in the “Incoming” tab, and they are “checked in” when they arrive to the clinic. Their information is then updated based on their new visit, and the process repeats.
Study Site and Population
The OP Care app was piloted in a MDT Gynecologic Oncology clinic and the oncology ward. The Princess Marina Hospital (PMH) oncology ward provides treatment and follow-up care to all oncology patients, whereas the (blinded) clinic specializes in treating gynecologic cancers and following up patients after completion of treatment.
All patients seen in the clinic since the app was introduced were enrolled in the app at the beginning of their clinic visit once they consented. This amounted to approximately 50 patients per week over 20 weeks before this study took place. To this end, all of the staff that attended the patients (four nurses and two doctors) in the MDT Gynecologic Oncology clinic were trained in using the app. In the Princess Marina Hospital (PMH) oncology ward, all patients receiving chemotherapy and all new patients were enrolled at the beginning of their visit once they consented. This amounted to approximately 20 patients per week over 2 weeks before this study took place. The staff who attend these patients were therefore trained in using the app (two nurses and one doctor).
Patients were only excluded if they did not consent, and there were no dropouts. The workflow in the PMH oncology ward made surveying those patients difficult due to patient load and lack of available staff; therefore, only patients from the MDT Gynecologic Oncology clinic, the app was piloted in th clinic were surveyed for the purposes of this report. All patients who presented to the clinic during the 2-day survey period and had been previously enrolled in OP Care were consented and surveyed regarding their opinions about the effectiveness of the OP Care appointment text message reminders in reducing missed appointments. This amounted to 15 patients. There were no dropouts. Further detail regarding the content of this survey is presented in the Staff and Patient Questionnaires section.
Application Training and Use
OP Care was introduced in the MDT Gynecologic Oncology clinic in February 2018. The developers of the app first trained the clinic oncology coordinator, who then trained the remaining staff. The training period lasted 5 days, after which staff used the app independently. Any questions the staff had about using the app following the training period were directed to the oncology clinic coordinator. After its introduction, all new patients were enrolled to the best of the staff’s ability, amounting to approximately 10 patients per week. Once the staff became comfortable with usability, they expanded to enrolling follow-up patients as well, for a total of approximately 50 patients per week. In June 2018, after its success in the MDT Gynecologic Oncology clinic, the app was piloted in the PMH oncology ward. The relevant staff were trained by two MDT Gynecologic Oncology clinic staff members who were proficient in the use of the app over the course of 2 weeks.
Staff and Patient Questionnaires
In June 2018, the staff in the MDT Gynecologic Oncology clinic and the oncology ward who had been using OP Care were surveyed to determine the usability and feasibility of the app. The written questionnaire had three sections. The first section collected background and demographic information, the second assessed their reactions to using the app, and the third assessed user satisfaction with regard to usability. The third section comprised the validated IBM Computer System Usability Questionnaire (CSUQ), which uses subjective measures to assess ease of usability and user satisfaction.16 The usability of a software is a measure of the mental workload required for a given task and can be further defined by how easy the software is to learn, how efficiently it can be used, how easily it can be relearned, how often errors occur, and how pleasantly the software is designed.17 For the second and third sections of the questionnaire, responses were recorded on a Likert scale ranging from 1 to 7 based on the participant’s agreement with the statement. The numbers on the scale corresponded to the following responses: not at all (1), no (2), not very much (3), somewhat (4), yes (5), very much so (6), and extremely so (7). Participants were also given the opportunity to write additional comments after each question in the second and third sections of the survey.
Patients in the MDT Gynecologic Oncology clinic were surveyed using a different questionnaire to assess their opinions regarding the app’s goal of reducing missed appointments using reminder text messages. The questionnaire was developed collaboratively by research staff and used the same Likert scale used for the staff survey, ranging from 1 to 7. Although the questionnaires for both staff and patients were developed in English, the one for the patients was translated to Setswana by the clinic oncology coordinator, who is a native Setswana speaker and also fluent in English, before being administered. While staff filled out the questionnaire themselves, patients’ surveys were administered orally by a native Setswana speaker, and their responses were recorded by the administrator.
Both staff and patient questionnaires contained room for optional comments after each question. Patient responses to these sections were translated to English for analysis (see Appendix A for the surveys administered).
Data Analysis
Quantitative analysis of the background and demographics section of the staff survey was performed by calculating simple proportions and frequencies of number of responders to each statement. The summative nature of the Likert scale used for the second and third sections of the staff survey allowed for easier interpretation of the small sample size.16 Responses were analyzed using the median and interquartile range (IQR) for each question, which were calculated to reflect the general opinion of the providers.
Because these surveys took place only a few months after OP Care had been introduced into the clinic, a quantitative analysis of the number of missed appointments over time could not be performed. Therefore, a subjective evaluation of patients’ reactions to the text message reminders was performed instead. Because this survey used the same Likert scale as the staff surveys, it was also analyzed using the median and IQR of the responses.
Free-text comments produced by both staff and patients were grouped by a common theme to convey the main ideas expressed by each group. Comments were first grouped by origin (i.e., whether they came from a staff survey or a patient survey). Within these overarching groups, each individual comment was treated as a single meaning unit given its brevity (with each one being one sentence or less) and the fact that each comment was produced in response to a single close-ended survey question (Appendix A). Two independent members of the research staff read all of these free-text comments and categorized them based on recurring themes they saw among the comments. For example, if several different patients expressed a similar idea in response to a given survey question, these comments would be grouped together. Themes were identified independently by each of two researchers and then reconciled, and representative comments for each theme were chosen to report. This method of analysis and reporting was adapted from a similarly conducted study investigating the usability of a smartphone app tracking antiretroviral medication adherence in Botswana.10
Ethical Considerations
Patients provided informed consent before enrollment in the app. The project was approved by the University of Pennsylvania, the Ministry of Health and Wellness in Botswana, and Princess Marina Hospital Institutional Review Board (file number: 820159).
Results
Health Care Provider Experience
One doctor and four nurses in the MDT Gynecologic Oncology clinic and two doctors and two nurses in the PMH oncology ward used OP Care and were surveyed, for a total of nine staff members. Table 1 contains responses from the first section of the staff questionnaire. Seven (78%) were female, and the same percentage were native Setswana speakers. Eight (89%) have practiced oncology for 2 to 5 years. All participants used either a desktop, a laptop, or a tablet at home, or a combination of them, and owned smartphones. Before OP Care was introduced, five (56%) providers indicated that they rescheduled patients’ appointments if patients forgot to bring hard copies of their medical records to the appointment. Seven (78%) providers did not feel that using OP Care would increase their work burden.
Table 1.
Staff background and demographics
| Sex | Female | 7 (77.8%) |
| Male | 2 (22.2%) | |
| Occupation | Doctor | 3 (33.3%) |
| Nurse | 6 (66.7%) | |
| Number of years of postsecondary education completed | 1–3 | 0 (0%) |
| 3–6 | 2 (22.2%) | |
| 6 or more | 7 (77.8%) | |
| Number of years practicing in the oncology field | <2 | 1 (11.1%) |
| 2–5 | 6 (66.7%) | |
| >5 | 2 (22.2%) | |
| Number of years in the current job | <2 | 1 (11.1%) |
| 2–5 | 5 (55.6%) | |
| >5 | 3 (33.3%) | |
| Is Setswana your native language? | Yes | 7 (77.8%) |
| No | 2 (22.2%) | |
| Do you use a desktop, laptop, or tablet at home? (Check all that apply) | Desktop | 2 (22.2%) |
| Laptop | 7 (77.8%) | |
| Tablet | 6 (66.7%) | |
| Do you own a smartphone? | Yes | 9 (100%) |
| What did you do when a patient forgot their records before OP Care was introduced? (Check all that apply) | Try to complete appointment without records | 5 (55.6%) |
| Create new records | 3 (33.3%) | |
| Reschedule appointment | 5 (55.6%) | |
| Counsel the patient on the importance of bringing records | 7 (77.8%) | |
| Try to trace with other oncology services to retrieve information | 1 (11.1%) | |
| Confirm pathology using IPMS | 1 (11.1%) | |
| What did you do when a patient came back after a missed appointment before OP Care was introduced? (Check all that apply) | Ask the patient why they missed the appointment | 7 (77.8%) |
| Counsel the patient on the importance of coming to appointments | 8 (88.9%) | |
| Do not acknowledge the missed appointment | 0 (0%) | |
| Make a new appointment | 1 (11.1%) | |
| If it’s clinic day, the patient can queue up and be seen if they have all the cards | 1 (11.1%) | |
| If this application were adopted, do you think it would increase your work burden? | Definitely | 1 (11.1%) |
| Somewhat | 1 (11.1%) | |
| Not at all | 7 (77.8%) | |
| Unsure | 0 (0%) |
Abbreviations: IPMS, Integrated Patient Management System; OP, Out Patient.
In the second section of the questionnaire, providers rated how strongly they agreed with different statements about the app on the Likert scale detailed previously, with 1 corresponding to “not at all” and 7 to “extremely so.” (Table 2). Statements describing the app as beneficial to patient care, such as “Patients are likely to benefit from my using this application,” were rated highly by providers, with a median score of 6 (“Very much so”) and an IQR of 1. Statements regarding patients forgetting appointments, including “Most patients find it difficult to remember their appointments,” were generally supported by providers (median score 4 “Somewhat”; IQR: 1). The median score to agreeing to the statement “My coworkers would be able to easily operate this application” was 5, which corresponded to “Yes” on the scale, with an IQR of 1. Agreeing to the statement “I intend to use the app if implemented permanently” scored a median of 6 (“Very much so”) and an IQR of 1.
Table 2.
Staff reactions to the usefulness of OP Care
| Reaction | Median (IQR) |
|---|---|
| 1. I feel confident in using the application when seeing patients | 6 (5–6) |
| 2. Using the application when seeing patients is under my control | 5 (4–7) |
| 3. My coworkers will be willing to use this application | 4.5 (4–5.5) |
| 4. My coworkers would be able to easily operate this application | 5 (5–6) |
| 5. Most patients find it difficult to remember their appointments | 4 (4–5) |
| 6. Most patients find it difficult to remember to get a blood test before their appointment | 4 (4–5) |
| 7. Patients are likely to benefit from my using this application | 6 (6–7) |
| 8. Patients will likely receive better treatment due to my using this application | 6 (5–6) |
| 9. My using the application is beneficial | 6 (5–7) |
| 10. I intend to use the application if implemented permanently | 6 (6–7) |
Abbreviations: IQR, interquartile range; OP, Out Patient.
In the third section of the staff questionnaire, as shown in Table 3, statements about the usability of the app were rated by providers. These statements included “Overall, I am satisfied with how easy it is to use this system” (median: 6; IQR: 1) and “I feel comfortable using this system” (median: 6; IQR: 1). Both “It was easy to learn to use this system” and “The system gives error messages that clearly tell me how to fix the problems” were not rated as highly by providers as the other statements, with a median rating of 5 (“Yes”) and an IQR of 3.
Table 3.
Staff reactions to using OP Care
| Reaction | Median (IQR) |
|---|---|
| 1. Overall, I am satisfied with how easy it is to use this system | 6 (6–7) |
| 2. It is simple to use this system | 6 (5–7) |
| 3. I can effectively work using this system | 6 (5–7) |
| 4. I am able to work quickly using this system | 6 (4–6) |
| 5. I am able to efficiently work using this system | 6 (5–6) |
| 6. I feel comfortable using this system | 6 (5–6) |
| 7. It was easy to learn to use this system. | 5 (3–6) |
| 8. I believe I became productive quickly using this system | 6 (5–6) |
| 9. The system gives error messages that clearly tell me how to fix the problems | 5 (4–7) |
| 10. Whenever I make a mistake using the system, I recover easily and quickly | 6 (5–6) |
| 11. The information (such as on-screen messages) provided with this system is clear | 6 (5–7) |
| 12. It is easy to find the information I need | 6 (5–6) |
| 13. The information provided with the system is easy to understand | 6 (5–6) |
| 14. The information is effective in helping me complete my work | 6 (5–6) |
| 15. The organization of information on the system screens is clear | 6 (5–7) |
| 16. The interface of this system is pleasant | 6 (5–7) |
| 17. I like using the interface of this system | 6 (6–7) |
| 18. This system has all the functions and capabilities I expect it to have | 6 (5–6) |
| 19. Overall, I am satisfied with this system | 6 (5–6) |
Abbreviations: IQR, interquartile range; OP, Out Patient.
Table 4 contains the results of the qualitative analysis from the open-ended comments provided by the staff. Staff comments were grouped together by common ideas, and themes were extrapolated by research staff. The main themes that arose from this analysis, as well as representative comments reflecting these themes, are presented. These main themes were as follows: (1) appreciation for the app’s functions, (2) suggestions for improvement, and (3) workflow concerns. Appreciation for the app’s functions included the EMR function as well as the patient reminders. One staff member felt that the information stored in OP Care “minimized the time of trying to get information on the patient,” and another felt that the app “helps the patient stay updated on his appointment.” Suggestions for improvement included the addition of other functions such as language choice and the option to discharge oncology patients on the app. Workflow concerns encompassed issues with a slow network, as well as fears of increased workload when using OP Care in addition to the old system of registering patients in a physical register, with two staff members describing these dual responsibilities as being “tedious.”
Table 4.
Staff’s free-response comments
| Theme | Comments |
|---|---|
| Appreciation for application’s functions | “…the application gathers a lot of information about the patient in case the patient turns up without his notes.” “Helps in monitoring number of incoming patients…” “OP Care is good as it gives you the summary of a patient you might have met before. This minimizes the time of trying to get information on the patient.” “…there is a reminder so there is no way they would miss their appointment.” “…this helps the patient stay updated on his appointment.” |
| Suggestions for improvement | “I would suggest that we could language choice for SMS messages to patient.” “Option for discharging patients for oncology care” |
| Workflow concerns | “With an increase in the amount of patients, handling the app, interacting with the patient, and documenting becomes tedious.” “If I have to do the interaction with the patient, updating the register and updating the app becomes tedious and slow.” “Network sometimes slow.” |
Abbreviation: OP, Out Patient.
Patient Experience
A total of 15 patients in the MDT Gynecologic Oncology clinic were also surveyed about their reactions to the text message appointment reminders (Table 5). Although responses were recorded on the same Likert scale mentioned previously, some had a very wide IQR, making the data difficult to interpret. In those cases, responses were divided into two categories. The first category corresponded to an overall answer of “No” and comprised responses from 1 to 4 on the scale. The second corresponded to an overall answer of “Yes” and referred to responses from 5 to 7. Based on these categories, 33% of patients disagreed with the statement “I would prefer receiving a call instead of a text reminder,” while 67% agreed. Patients were divided on having the time of their appointment in the text reminder as helpful. The statement “I would prefer to receive the text reminders in Setswana” was rated a median of 7 (“Very much so”), with an IQR of 1. Both “This system helps my doctor do his/her job” and “I would like my doctor to continue using this system” were rated a median of 6 (“Very much so”), with an IQR 1 and 2, respectively.
Table 5.
Patient reactions to text message reminders
| Reaction | Median (IQR) |
|---|---|
| No, 0 (0%) | |
| 1. I would prefer receiving a call instead of a text reminder | 5 (2–7) 5 “No”a 10 “Yes” |
| 2. I found it difficult to remember my appointments before OP Care was introduced | 2 (2–5) |
| 3. The number of text reminders I receive from this system is appropriate | 5 (3–7) 4 “No”a 11 “Yes” |
| 4. The timing of the text reminder is appropriate (seven days before appointment and two days before appointment) | 6 (5–6) |
| 5. I would prefer to receive the text reminders in Setswana | 7 (6–7) |
| 6. I would prefer to receive the text reminders in English | 4 (2–5) |
| 7. The time listed in the text reminder (e.g., 8:00 a.m.) is helpful | 5 (2–7) 7 “No”a 8 “Yes” |
| 8. The text messages I receive from this system keep me from missing my appointments | 7 (6–7) |
| 9. This system helps my doctor do his/her job | 6 (5–6) |
| 10. This system takes too much time and makes my appointment too long | 2 (2–2) |
| 11. I would like my doctor to continue using this system | 6 (5–7) |
Abbreviations: IQR, interquartile range; OP, Out Patient.
For responses with an IQR of 4 or higher, answers were also divided into two categories. The first corresponded to “No” and comprised responses of 1 to 4 on the scale, and the second corresponded to “Yes” and comprised 5 to 7 on the scale.
Table 6 illustrates the main themes that arose from the review of patient comments as well as some of the comments that reflected those themes. General themes that were found from this group were as follows: (1) preference for phone call reminders, (2) pre-OP Care method of remembering appointments, (3) positive attitudes toward the app, and (4) design improvements. “Pre-OP Care method of remembering appointments” focuses on the opinions of some patients that they were able to easily remember appointments without needing the kind of reminders provided by OP Care. These patients reported they would “refer to [their] medical records” to remind themselves of their appointments. “Positive attitudes toward application” encompassed appreciation for the appointment reminders and the improvement of clinic workflow due to the app. One patient reported that the apps “reminds me of when to come for my appointment,” and another felt it “hastens the work process.” “Design improvements” include the preference for phone call reminders over text messages, use of Setswana instead of English, and removal of specific appointment times in the reminders. Patients commented on the fact that the text message reminders specified an appointment time of 8:00 am, but due to the first-come, first-serve nature of the clinic, they were often not seen at this specific time. They felt this should be remedied to be more accurate.
Table 6.
Patient’s free-response comments
| Theme | Comments |
|---|---|
| Preference for phone call reminders | “I don’t always check my messages and messages can be missed (can come in without hearing them).” “It’s possible that I might not see the message, but when the phone rings, I can hear it and pick up.” “The caller can confirm that they spoke to the patient and delivered the message.” |
| Pre-OP Care method of remembering appointments | “I check from my medical records. I share the appointment date with my family so that they can remind me.” “I refer to my medical records for the appointment date.” “I always refer to my medical records” |
| Positive attitudes toward application | “It helps me understand my condition better as well as how I will be assisted.” “It hastens the work processes and helps me to remember my visits/appointments It can even be used to cancel my appointment in the event that there is no clinic.” “It helps health workers to know which days to expect patients.” “It reminds me of when to come for my appointment.” “I can miss a call, but will be able to retrieve a message any time I check (later).” |
| Design improvements | “I forget my cell phone at times.” “I don’t understand English.” “Even though we are given 8 am, we are often consulted much later after staff meetings or attending to other work-related activities before seeing patients.” |
Abbreviation: OP, Out Patient.
Discussion
In this study, we evaluated the development, usability, and feasibility of a new smartphone app to improve patient loss to follow-up and patient adherence in an oncology clinic in Botswana. The results of the questionnaires indicate overall positive reactions from both staff and patients. Before OP Care was introduced, most providers rescheduled appointments if patients forgot to bring hard copies of their medical records and providers did not have an alternative method of accessing patient records to use for the appointment. This suggests the importance of an app like OP Care that stores patient records and relevant data for provider use during appointments.
To our knowledge, only one other app-based system has been piloted in Botswana. This pharmacy-based smartphone app tracks treatment adherence in HIV patients.10 Of pharmacy staff surveyed about usability of that app, around one-third did not own a smartphone compared with our study where all staff surveyed reported the use of a smartphone. The majority of usability statements in the previous app study scored a median response of 5 on a Likert scale of 7, and the majority in this study scored a median of 6.10 The pharmacy-based app study used the same IBM CSUQ statements and Likert scale for responses used in this study.10 Furthermore, despite their relative comfort with technology, the staff in this study did not score questions related to the ease of learning the app and error messages as high as other questions. This indicates that the error messages currently in place are not sufficient to make the app easy to learn. However, in regard to the remaining aspects of usability covered by the survey, the app scored high.
Patients had an overall positive response rate to the usability of this app. However, the wide range of responses for questions regarding preference for phone call reminders instead of text messages indicates that further information is needed to properly adjust the app. Potential reasons for the heterogeneity in preference for phone calls are differences in patient ages, education levels, and cell phone literacy. Having an appointment time listed in the reminders also had a wide range of responses. All reminders included an 8:00 a.m. appointment time as patients are seen on a first-come, first-serve basis by the provider. Although some patients felt that the time motivated them to reach the clinic early in the morning, others felt it was misrepresentative. A potential solution for this may be to set expectations of clinic flow when appointments are made. Most patients noted wanting communication with them to be in Setswana.
Text message appointment reminders have already improved patient follow-up in clinics in several LMICs including China, Malaysia, Brazil, and India.12–15 However, this successful service has not yet been paired with the mobile patient-tracking and record-keeping tools provided by OP Care to improve treatment adherence and follow-up specifically in LMICs. Continued use of mobile apps to solve health care problems in LMIC is supported by the success of other apps worldwide. In Brazil, an app has been successfully piloted to educate low-income patients with low educational attainment about thromboembolic prophylaxis in atrial fibrillation.11 Although the usability of this app was tested on patients rather than providers, its success indicates that the use of mobile apps at point of care is feasible and should be pursued.
Both staff and patients felt that this app is an asset to the health care experience and improves patient care. Expanding the app’s use will also increase its capabilities by providing smoother communication between clinics, allowing for an easier transition for patients who are seen by multiple providers. After adjustments are made to the app, the goal is to expand its use to the remaining oncology centers in Botswana (Francistown, Maun, and Serowe). Giving providers an easy to use tool with which they can track patients’ treatment, follow-up appointments, and medical records will allow them to streamline care delivery and fill the current gaps in oncology in many LMIC. An important consideration as the app expands is ensuring the safety of patients’ data, as the information stored is of a very sensitive nature. Limiting the use of the app to only the relevant staff and continuing to use the existing safety features in place, including a two-step authentication will help ensure that patient data are protected.
A limitation of the study was that a small number of participants were surveyed. The number of providers was constrained by the number of staff who were using the app. Including more staff from other departments with little to no prior experience with smartphones would help enhance the applicability of our survey results to a more diverse pool of people. The number of patients surveyed was limited to those who had been previously enrolled in the app and then returned again for an appointment during the time that the surveys were being administered. The sample of patients was therefore a convenience sample and may be biased toward patients who remember their appointments anyway because they are more likely to come to their appointments. Additionally, out of all of the questionnaires administered, only the IBM CSUQ was a validated research tool. We acknowledge that all staff included in this study own smartphones and some type of computer system at home. Although the staff interviewed felt that the app was easy to use, this may not be applicable if OP Care was expanded to setting in which providers and staff do not regularly use smartphones or computers, such as community health workers in more rural areas.
Further research is needed to provide quantitative evidence that the number of missed appointments has decreased since the introduction of the OP Care text message reminders. This would be done by measuring the number of appointments missed by patients after OP Care was introduced and analyzing these data for significant change over time. More data are also needed to assess how often patients forget their records and how often OP Care is used to make up for this.
Conclusions
The smartphone app OP Care was introduced in the Gynecological Oncology MDT Clinic and the Oncology Ward at Princess Marina Hospital in Gaborone, Botswana, to address issues with treatment adherence and posttreatment monitoring. With the increasing incidence rate of cancer in LMICs, as well as the disproportionately high mortality rate, filling in the gaps in oncology in these countries is vital. OP Care allows for patients’ identification information and relevant medical records to be stored in a reliable, portable, and secure system.
After introducing the app, the reactions of the staff and their opinions about the app’s usability, as well as the reaction of patients to its use were assessed using questionnaires. Overall, the usability portion of the questionnaire indicated that the staff members satisfied with the app’s ease of use. The staff members were willing to use the app long term and feel that it improves quality of care. Patients also feel that the app benefits them and would like their doctor to continue using it. Some technical issues such as effective error messages and the language of patients’ reminder text messages should be addressed. However, based on these data, the app does meet its goals of usability and benefit to patients.
Funding
None.
Footnotes
Conflict of Interest
G. D. is the CEO of ONE BCG, the company that developed the application that was implemented in this study. S. P. S. is also an employee of ONE BCG. This application was developed and is distributed at no cost to consumers, with only social goals in mind. The company gains no profit from this application.
References
- 1.Farmer P, Frenk J, Knaul FM, et al. Expansion of cancer care and control in countries of low and middle income: a call to action. Lancet 2010;376(9747):1186–1193 [DOI] [PubMed] [Google Scholar]
- 2.International Agency for Research on Cancer. Cancer fact sheets: all cancers excluding non-melanoma skin cancer. Available at: https://gco.iarc.fr/today/data/pdf/fact-sheets/cancers/cancer-fact-sheets-29.pdf. Accessed June 24, 2018
- 3.Sankaranarayanan R, Boffetta P. Research on cancer prevention, detection and management in low- and medium-income countries. Ann Oncol 2010;21(10):1935–1943 [DOI] [PubMed] [Google Scholar]
- 4.Dickinson R, Hall S, Sinclair JE, Bond C, Murchie P. Using technology to deliver cancer follow-up: a systematic review. BMC Cancer 2014;14:311. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Cazap E, Magrath I, Kingham TP, Elzawawy A. Structural barriers to diagnosis and treatment of cancer in low- and middle-income countries: the urgent need for scaling up. J Clin Oncol 2016;34(01):14–19 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Fundytus A, Sullivan R, Vanderpuye V, et al. Delivery of global cancer care: an international study of medical oncology workload. J Glob Oncol 2018;4(04):1–11 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Randall TC, Ghebre R. Challenges in prevention and care delivery for women with cervical cancer in Sub-Saharan Africa. Front Oncol 2016;6:160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Fritz F, Tilahun B, Dugas M. Success criteria for electronic medical record implementations in low-resource settings: a systematic review. J Am Med Inform Assoc 2015;22(02):479–488 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Alamo ST, Wagner GJ, Sunday P, et al. Electronic medical records and same day patient tracing improves clinic efficiency and adherence to appointments in a community based HIV/AIDS care program, in Uganda. AIDS Behav 2012;16(02):368–374 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Coppock D, Zambo D, Moyo D, et al. Development and usability of a smartphone application for tracking antiretroviral medication refill data for human immunodeficiency virus. Methods Inf Med 2017;56(05):351–359 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Stephan LS, Almeida ED, Guimarães RB, et al. Oral anticoagulation in atrial fibrillation: development and evaluation of a mobile health application to support shared decision-making. Arq Bras Cardiol 2018;110(01):7–15 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Chen ZW, Fang LZ, Chen LY, Dai HL. Comparison of an SMS text messaging and phone reminder to improve attendance at a health promotion center: a randomized controlled trial. J Zhejiang Univ Sci B 2008;9(01):34–38 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Liew SM, Tong SF, Lee VK, Ng CJ, Leong KC, Teng CL. Text messaging reminders to reduce non-attendance in chronic disease follow-up: a clinical trial. Br J Gen Pract 2009;59(569):916–920 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.da Costa TM, Salomão PL, Martha AS, Pisa IT, Sigulem D. The impact of short message service text messages sent as appointment reminders to patients’ cell phones at out-patient clinics in São Paulo, Brazil. Int J Med Inform 2010;79(01):65–70 [DOI] [PubMed] [Google Scholar]
- 15.Prasad S, Anand R. Use of mobile telephone short message service as a reminder: the effect on patient attendance. Int Dent J 2012;62(01):21–26 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Lewis JR. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum Comput Interact 1993;7(01):57–78 [Google Scholar]
- 17.Bevan N, Kirakowski J, Maissel J. What is usability? Proceedings of the Fourth International Conference on Human Computer Interaction, Stuttgart, September 1991 [Google Scholar]