Usability Testing of Tablet-based Cognitive Behavioral Intervention Application to Improve a Simple Walking Activity for Older Adults with Arthritis Fatigue

Abstract

The purpose of this study was to test usability of a tablet-based cognitive behavioral intervention (“Tab-CBI”) application. Tab-CBI was designed to improve a simple walking activity for older adults with arthritis fatigue. Two phases of usability evaluation were conducted: phase I with 5 older adults to identify any initial design issues and phase II with 10 older adults to examine if Tab-CBI was seamlessly integrated into their daily living. Participants perceived that Tab-CBI was easy, light, and intuitive to use and helped them walking. They reported two key benefit as a motivator to make more active in walking and improved confidence of self-management of fatigue. They referred video recordings and videoconferencing communications as two preferable features. There were also comments for improvement, for example, a challenge for holding a tablet when videoconferencing communications. Participants comments were incorporated into a final version of Tab-CBI to prepare for a future pilot study.

Introduction

Arthritis is a leading cause of long-term disability in older adults in which fatigue, a prevalent symptom of arthritis, occurs in up to 80% of patients. ^1,2 Arthritis fatigue is a major concern, as symptoms are disabling, far-reaching, and associated with significant decline in daily function and greater healthcare utilization. ^3,4 Despite its high prevalence and debilitating impact on health outcomes, effective methods for managing arthritis fatigue remain limited. Medications used to treat arthritis (e.g., inflammation medications) have limited effectiveness on reducing fatigue. ⁵ Despite the known benefits of physical activity-based interventions (e.g., pool-based therapy, dynamic strength training, yoga, low impact aerobics), barriers exist as these interventions are resource-intensive and costly, requiring highly structured trained personnel and high facility utilization. ^5-8 In addition, a critical weakness of these interventions is participants’ lack of long-term engagement in and adherence to the interventions. ^9,10 As a result, the beneficial effect of the interventions largely disappear after the intervention is complete. ^10,11

To effectively manage arthritis fatigue, our team developed a tablet-based cognitive behavioral intervention (“Tab-CBI”) to facilitate a simple walking activity. Simple walking demonstrates significant reductions in fatigue, is easy to learn, and requires minimal professional guidance, equipment, or facility access. ^6,11 A behavioral change strategy, known as cognitive behavioral therapy (CBT), was used to improve older adults’ long-term engagement in and adherence to the simple walking intervention. CBT focuses on promoting long-term behavioral changes through identifying links between a patient’s thoughts and feelings that drive behaviors, and cognitively restructuring these links. ¹² The Tab-CBI application includes three specific features: (a) four weekly CBT-based learning modules to encourage a simple walking activity, (b) weekly to-do tasks, and (c) a HIPAA compliant videoconferencing tool for patient education and follow-up. Tab-CBI is launched in a mini-tablet computer (Model: Samsung Galaxy Tab-A) which can store sizable educational materials and is easy to carry. The minimum requirements of the tablets are Android version 5.1 Lollipop or higher, 1.5 GB RAM, 8 GB ROM with Wi-Fi and Bluetooth capabilities. Details about the development of Tab-CBI can be found in our previous study. ¹³

Usable technologies (e.g., devices, systems) must be effective, efficient, and satisfying to use according to the International Human Computer Interaction standards. ¹⁴ The goal of usability is to develop technologies that end users perceive as easy, effective, and enjoyable to use, thus promoting acceptance and use of the technologies being introduced. ¹⁵ To develop such usable technologies, the technology development process should iteratively refine the technologies, starting with a low-fidelity prototype to test the interface layout, progressing to a high-fidelity prototype to provide users an accurate “look and feel” of the final product in their daily lives. ¹⁶

To establish usability of Tab-CBI, we evaluated older adults’ usability experience in two phases to identify any initial design layout issues and to examine if Tab-CBI was seamlessly integrated into their daily life. We used a cognitive walkthrough with a think-aloud technique. A cognitive walkthrough is a usability method in which a task-specific approach to identify facilitators and barriers in completing key specific tasks of a technology is used. ¹⁷ A think aloud is a commonly used usability method where subjects verbalize their thoughts about performing actions. ¹⁸ The think-aloud method allows researchers to determine why and how the participants are interacting with the system in a particular way and to identify problems within the system design. ¹⁶ This method has been successfully used to test usability of tablet-based systems ^19,20 and videoconferencing systems in the past. ^21,22 This article describes the results of the two phases of usability findings.

Material and Methods

Study design and participants

This was a descriptive exploratory study using a qualitative method. Usability testing was conducted in two phases where each phase had a different objective. Phase I involved a focus group interview with five (5) older adults to identify problems with the initial design of Tab-CBI. Phase II was a field test with ten (10) participants to explore if the Tab-CBI seamlessly fit into participants’ daily routine. Phase II had a larger sample size to account for the increasing complexity of the tasks the participants were asked to complete. ²³

A convenience sample of older adults with arthritis fatigue was recruited from two senior centers, arthritis support groups, and council of aging (COA) centers across Western Massachusetts and Vermont. Inclusion criteria were (a) diagnosis of osteoarthritis or rheumatoid arthritis, and (b) presence of greater than minimal fatigue scoring ≥17 on the PROMIS Fatigue Short Form v1.0 Fatigue-8a (PROMIS F-SF). ²⁴ Exclusion criteria were (a) currently engaging in regular exercise; (b) non-ambulatory or having a condition that would limit the ability to walk (e.g., foot deformity, lower extremity joint surgery in past 6 months, stroke, severe chronic obstructive pulmonary disease, etc.); (c) overt delirium, dementia, or any conditions indicating deteriorating cognitive status as determined by the Mini-Cog; ²⁵ and (d) no WIFI capability. Of the eighteen (18) older adults that the research team contacted, sixteen (16) met the inclusion criteria and one (1) of the sixteen refused, leaving fifteen (15) older adults in the study.

Measures

Participant’s fatigue was measured using the PROMIS F-SF. ²⁴ The PROMIS F-SF consists of eight items that measure both the experience of fatigue (e.g., severity) and the interference (e.g., physical, emotional, and social) of fatigue on daily activities over the past week. Response options are on a 5-point Likert scale, ranging from 1 = never to 5 = always. Scores can range from 8 to 40, with higher scores indicating greater fatigue. The PROMIS F-SF demonstrates good reliability with Cronbach’s alphas of .72 to .88. ²⁶ The scale also shows good validity across ethnically/racially diverse populations ²⁶; concurrent validity was supported by moderate to high correlations between the PROMIS F-SF and the Multidimensional Fatigue Symptom Inventory-Short Form (r = .70 to .85) and between the PROMIS F-SF and the Brief Fatigue Inventory (r = .60 to .85). Discriminant validity was supported by moderate correlations with other measures of constructs that are related, but not alike, such as the Perceived Stress Scale (r = 62) and the Center for Epidemiological Studies-Depression (r =.64).

Cognitive status was determined by the Mini-Cog. ²⁵ The Mini-Cog consists of two components: a 3-item recall test for memory and a clock drawing test. The scale shows a high level of sensitivity (76%) and specificity (89%) ²⁷ and is not adversely influenced by age, language, or education. ²⁸

Procedures

Consenting participants were recruited and screened for inclusion and exclusion criteria. In Phase I the interviews were conducted in private settings at the convenience of the participant. The interview began with the research assistant (RA)’s verbal instructions, which were uniformly presented according to a PI-prepared package. The package included instructions and explanations about the interview purpose and procedure, Tab-CBI, the “think-aloud” method, and usability questions. Then, the RA demonstrated several examples of the think-aloud method to show participants what would be expected. After learning how to operate a tablet, participants performed the scenarios capturing key user tasks of Tab-CBI (e.g., connecting to and communicating with a provider, and retrieving educational modules) and identified if Tab-CBI was easy to use and navigate. The RA observed and documented participants’ interactions with Tab-CBI in field notes. Interviews were audio recorded, and the recordings were transcribed verbatim in a MS Word file. The data from the RA’s field notes and the tape-recorded interviews were analyzed for themes and patterns related to use of Tab-CBI and were used to help revise the prototype.

In Phase II, participants received the RA’s instructions about the study purpose, procedure, and Tab-CBI, along with training of how to operate it. Then participants received a tablet and an accelerometer and were asked to perform key tasks (visiting weekly learning modules, communicating with a provider using a videoconferencing tool, taking self-assessment quizzes, and recording daily steps taken and syncing the accelerometer to the tablet) for 2 weeks of the study period. Since the participants’ videoconferencing experience was one of the key details sought from the usability test, we ensured that participants had a brief weekly videoconferencing meeting with the RA. During the videoconferencing meeting, the RA collected participants’ feedback on their use of Tab-CBI (e.g., overall impression, likes, dislikes, and any comments to improve Tab-CBI) using the user experience interview guide. The feedback was used to refine the prototype, resulting in a final version of Tab-CBI.

Results

Participants’ Demographics

The study sample consisted of 15 older adults, ages 58-81 with a mean age of 65.3 years (SD=5.65). Participants were mostly female (n=11, 73.3%); White (n=13, 86.7%); married (n=10, 66.7%); living with a spouse (n=8, 53.3%), and had a high school or higher degree education (n=14, 93.3%). The majority of participants had experience using a tablet (n=11, 73.3%), desktop or laptop (n=10, 66.7%), and smartphone (n=14, 93.3%), and used such devices (e.g., computer, tablet or smartphone) several times per day (n=13, 86.7%) and for more than 2 years (n=14, 93.3%). There were thirteen (87%) participants with a diagnosis of rheumatoid arthritis and two (13%) with osteoarthritis. Participants’ mean PROMIS Fatigue SF-8a score was 27.8 (SD=4.77) indicating greater than minimal fatigue.

Phase I Findings

Participants made 53 comments consisting of 29 positive comments and 24 negative comments regarding the improvement of the Tab-CBI method. Overall, participants perceived the Tab-CBI as easy to use and an excellent way to structure self-care. Examples of such comments are, “Using the tablet can become a habit- this is helpful to promote everyday walking activities,” “Goal setting are helpful in getting out and walking” and “Ease of using the self-assessment quizzes is good.” They indicated that the video recordings and videoconferencing tool as the two most favorable features of Tab-CBI because there was “no reading involved and no need of reading glasses” and “[I] don’t have to leave home to communicate with a provider.”

Participants’ comments for improvement were categorized into three groups; design and layout, content of educational modules, and key functions of Tab-CBI (e.g., videoconferencing communication, taking electronic outcome surveys and self-assessment quizzes, playing video recordings, and setting SMART goals) based on the potential design solutions to be implemented. The majority of participants’ comments are on the design and layout (n=8, 33.3%), followed by content (n=6, 25%), function (n=6, 25%), and others (n=4, 16.7%). Key comments included; (1) a list of summary points in each learning module did not stand out because they were written in the same font size as other text (design and layout), (2) several icons were too small for those with visibility and/or hand/finger tremor issues (design and layout), (3) the intended messages for several cognitive behavioral model examples in the video recordings were difficult to grasp (content of educational modules), (4) the videoconferencing tool was difficult to operate (key functions of Tab-CBI), (5) after playing all of the video recordings in a private YouTube channel, a list of irrelevant video clips popped up which was distracting (key functions of Tab-CBI), and (6) when the participant chose the wrong answer in the self-assessment quizzes, they did not have any explanation as to why the answers were wrong, but were only asked to “try again” (key functions of Tab-CBI), and (7) all of the participants were unable to physically put on or take off the accelerometer band around their wrist because the silicone wrist band that the accelerometer came with was too stiff for those with weak wrist and hand strength (other). Participants’ comments provided an actionable list of changes for refining the Tab-CBI application and were incorporated into a next version of Tab-CBI to prepare for the phase II usability test. Table A shows examples of how our team addressed the comments to refine the Tab-CBI application.

Table A.

Examples of participants comments for improvement and the relevant changes made

Participants comments	How the comments were addressed
The summary points do not stand out against the text	The summary points were made in a larger font than the text.
Icons are small for those with visibility or hand/finger tremor issues	Since Tab-CBI adopted the off-the-shelf software to build the application, icons are not a customizable feature of the software. Instead, we revised the training materials to include how to enlarge icons using finger movements.
Unclear examples of cognitive behavioral model in video recordings	The script for the videos was reviewed and revised to ensure that the examples were written using easy and simple terms, and understandable for older adults. We also added a brief statement to explain what each example is about, for example, “this is an example of a way to broadening your thinking”.
A videoconferencing tool is difficult to operate	The training materials were refined to provide step by step instruction on how to use a videoconferencing tool.
The YouTube’s up next videos feature is distracting	The default “up next video” feature on YouTube was disabled.
In self-assessment quizzes, there was no explanation why the answer was wrong when choosing a wrong answer	We added an explanation of what was wrong with the answer whenever a wrong answer was chosen.
Unable to physically put an accelerometer wrist band on.	The silicon bands were replaced with stainless-steel metal bands with a magnetic clasp which is easy to get on and off as well as easy to adjust.

Phase II Findings

Participants’ data were organized with the following interview questions:

What do you like about Tab-CBI?

Five key topics emerged from the participants comments. First, participants reported that Tab-CBI was “portable enough, handy, easy to carry, and perfect size.” Second, participants agreed that the learning materials were easy to read and understand and the video recordings were the right length. Third, the Tab-CBI application was user friendly and self-explanatory; the icons were clearly labeled. Fourth, participants liked the video recordings because they were motivational. Participants said, “Continuing to use it makes me aware of my activity” and “[Tab-CBI] made me keep tracking my activity/exercise.” Fifth, the reminder tools, for example, to alert charging a tablet and syncing the accelerometer to the tablet were good visual cues. One participant said, “Without this [reminder], I would keep forgetting, because of this, I am on track.”

What challenges have you had using the Tab-CBI application at home?

Several themes emerged regarding the challenges of using Tab-CBI at home. Although the majority of participants agreed the amount of information was adequate, there was a great deal of information with too many details to read and digest which made them overwhelmed. One participant said, “the video is a great resource but could be shortened.” The second comment was about the inconvenience of carrying a tablet on top of a cell phone. Despite the small and light (5 in width } 8 in height, 1.1 1bs.) size of a tablet, participants said, “ [I am] tired of carrying extra piece of device on top of phone.” The last and the most frequently cited comment was a need for a stand to hold the tablet. The participants frequently commented that a major symptom of arthritis was pain and swelling in the hands and wrists. Therefore, it was difficult to watch videos, read the text in the learning modules, or have a video conferencing meeting by holding the tablet in one hand.

What changes would you recommend?

There were several suggestions made to improve the Tab-CBI application. First, participants wanted to have journals within the tablet to leave notes on their Tab-CBI activities. For example, “today there was a heavy rain all day, so I did not go out for walking” or “[I] was sick today and stayed in bed.” Participants also suggested having a lightweight stand to hold the tablet, especially for those with decreased hand strength and hand/wrist discomfort because of arthritis. Several participants preferred a phone to be the Tab-CBI platform rather than a mini-tablet because of the inconvenience of carrying two devices. Lastly, there is one additional comment on group learning with Tab-CBI. A group of participants recommended learning in a group setting rather than in an individual setting and said, “I do better in a group setting, peer instructions, or sharing, than on my own.” While the literature suggests that group activity has advantages over solo activity with regards to self-improvement, more testing is needed to determine if Tab-CBI group settings has any such advantages over individual use of Tab-CBI.

What are the benefits of using the Tab-CBI application?

Tab-CBI has been identified to be both a motivator and a way to improve self-care confidence in participants. The majority of participants referred to Tab-CBI as a motivator that increased their motivational level to increase walking and physical activity. Tab-CBI was designed to improve self-management skills of problem-solving and goal-setting. The participants said, “Seeing where I was at each day motivated me to set a new goal and do a little more next time,” “[Tab-CBI] is an incentive, especially during a time needing to accomplish a walk goal,” “it made me more aware of moving –I can do more than I thought –a fun toy to inspire.” The second theme of self-care confidence was illustrated through participant reports of improved sense of control over their walking activity and fatigue symptoms. Participants said, “it keeps reminding me I can do something to make a difference,” “it is hard on my own, and this gives me a way to do it by myself.”

Based on the participants’ comments, appropriate modifications were made to Tab-CBI. For example, the script for video recordings was reviewed and redundancies were removed to make it succinct. All video recordings were shortened to 3 minutes or less in length. The team also integrated a stainless-steel stand for participant use. The stand securely held the tablet with the ability to rotate or tilt its angle (see Figure A). The team also made a place for participants to leave notes within Tab-CBI. For the next step, our team is conducting a pilot study to examine the effect of Tab-CBI on walking and fatigue as well as on self-confidence and quality of life.

Figure A.

Tablet-holding stand. This stainless-steel stand holds the tablet securely with an ability to rotate or tilt its angle.

Discussion

To develop technology that is accepted and used by end users, we conducted user-centered evaluation of Tab-CBI in two phases. The goal of a user-centered usability test is to develop technologies that end users perceive as easy, effective, enjoyable to use, and fit seamlessly into their daily routines, thus promoting acceptance and use of the technologies. ²⁹ Overall, older adults in the study perceived Tab-CBI as easy, light, intuitive to use and helpful for improving their walking. They also made several comments for improvement such as difficulties holding the tablet and a need for a space for journal writing, which were incorporated into the final version of Tab-CBI.

The usability findings highlight several issues regarding the use of Tab-CBI for older adults. First, the majority of participants liked the video recordings because no reading was involved. Considering older adults’ age-related vision impairment, video recordings, rather than or in addition to text-based educational materials, are an essential consideration for nurse educators and researchers in designing instruction materials for older adults. Second, the common message from all participants was that they liked the videoconferencing tool as a key benefit of Tab-CBI. Tab-CBI includes a videoconferencing tool when communicating with patients for education and follow-ups. A videoconferencing tool is well suited to meet the needs of older adults with arthritis who often have difficulties traveling to places outside the home because of their physical and/or functional impairments associated with arthritis. Tab-CBI provides the opportunity for regular communication and education with the provider while eliminating additional travel constraints such as cost, time and the impact of weather. ³⁰ However, operating a videoconferencing tool may be a challenging task for older adults as shown in our usability findings. Many of the participants in our study had difficulty in using the tool’s user interface (e.g., starting it, stopping it, increasing/decreasing the volume, or adjusting the size of the screen). The findings emphasize the importance of carefully developed user training materials whereby paying particular attention to step-by-step instructions is vital.

There were two comments we would want to highlight although they were not incorporated into the refinement of Tab-CBI. These comments addressed the platform used and the structure of how the education was provided, which would significantly change the study as designed and were beyond the scope and resources of this project. However, the comments provided meaningful insight into the future direction of Tab-CBI research.

Several participants reported inconvenience due to carrying an extra device in addition to a mobile phone and suggested a phone as the potential platform for Tab-CBI instead of a mini-tablet. The study purpose was to develop a tablet-based application; therefore, we were not able to incorporate this suggestion into the refinement. A recent study from the Pew Research Center found that about half (53%) of older adults ages 65 and above owned a smart phone. ³¹ The fact that older adults own more smartphones than tablet computers (32%) indicates a smartphone-based application might improve widespread acceptance and dissemination of Tab-CBI.

Participants also provided feedback regarding the structure of the education, recommending group learning rather than individual learning. Participants stated they would do better in a group setting rather than in an individual setting. Previous studies demonstrate a CBT group approach is beneficial in achieving behavioral changes. In a group setting, mutual peer reinforcement by encouraging one another to meet their goals is often far more powerful than reinforcement by a provider alone. ³² These findings together with our usability comments warrant further research to explore the effect of group learning as an effective learning strategy.

The videoconferencing tool was well accepted by older adults because it improves their access to the intervention, while closely resembling face-to-face interactions between a patient and a clinician, making it an effective alternative to the traditional in-person education approach. ^30,36,37 Our findings of the end-user interest and engagement in the video conferencing tool is consistent with these findings. There are, however, concerns and barriers associated with tablet research such as technology-related issues (e.g., too much and too complex technology or feelings of inadequate instructions and guidance). ³⁸ Older adults tend to be concerned about the complexity of technological devices and lack of confidence to complete the tablet tasks successfully. ³⁹ Our study participants also reported similar concerns that the videoconferencing tool was difficult to operate, and that they were afraid they might break the device. These previous findings, along with our findings, suggest the importance of formal technology training courses that introduce technology into older adults’ lives in an accessible way and assist them in keeping up-to-date with technological advances and trends. Research has found that formal computer training of older adults reduced their anxiety and increased their confidence of using technology. ^38,39 With increased confidence, older people can enjoy the benefits of using new technologies. Ensuring older adult participants have adequate training with technological inventions must be considered in future research or education.

There were several limitations in this study. First, participants were not diverse; they were mostly White, female, and highly educated. Therefore, sample representativeness may be limited. Additionally, there may be differences between those who agreed to participate in the study and those who declined; the results are not likely to reflect the perceptions and experiences of those who did not participate.

Conclusions and Nursing Implications

Alleviating fatigue for older adults with arthritis is important to improve health outcomes, maximize quality of life and reduce unnecessary healthcare costs. Our team developed a tablet-based cognitive behavioral intervention (Tab-CBI) application to improve self-management of a simple walking activity to decrease arthritis fatigue. Our team conducted two phases of usability testing for Tab-CBI. We found that older adults in this study perceived Tab-CBI as easy to use and helpful in improving their walking. They reported a key benefit of Tab-CBI as a motivator and that it helped improve feelings of self-confidence. Participants preferred the video recordings of learning materials and videoconferencing communication. There were also comments for improvement, which were addressed and incorporated into a final version of Tab-CBI. Additional feedback regarding the technological platform and the structure of how the education is provided will be considered and examined in future studies. Currently, research is in progress to conduct a pilot field study that examines the effect of Tab-CBI on walking and fatigue as well as self-confidence and quality of life.

Use of technology for older adults to promote health outcomes and quality of life continue to grow and Tab-CBI is one such application of technology designed specifically for older adults with arthritis fatigue. In order for technology to be widely accepted and successfully used, usability of the technology needs to be tested with end users in mind. This is referred to as a user-centered approach.²⁹ Our usability study demonstrated the process of obtaining older adults’ feedback of Tab-CBI and incorporating those comments when refining Tab-CBI. Nurse educators, clinicians, and researchers may find the user-centered approach helpful in developing and evaluating technology in the future.

Highlights.

1. Arthritis fatigue is associated with declines in daily function

2. Current activity-based interventions (yoga, aerobic) are resource rich and costly

3. A tablet-based cognitive behavioral intervention (Tab-CBI) improves simple walking

4. Usability participants perceived that Tab-CBI was easy, light and intuitive to use

5. Participants feedback was incorporated into a final version of Tab-CBI