MS Energize: Field trial of an app for self-management of fatigue for people with multiple sclerosis

Abstract

Multiple sclerosis (MS) is a lifelong neurological condition affecting around 2.2 million people worldwide. There are a wide range of symptoms, with fatigue reported as one of the most troublesome. MS Energize—or MS Energise in UK English regions—is an iPhone app focused on self-management of fatigue for people with MS. Based on cognitive-behavioral therapy principles, the app covers MS fatigue, how to use energy effectively, how behavior, thoughts and emotions interact and impact on MS fatigue, as well as the potential effects of bodily and environmental factors. MS Energize provides education, interactive tasks, and supports application of the principles into a user's day-to-day life.

We field tested the usability and perceived usefulness of MS Energize with 11 people with longstanding multiple sclerosis in New Zealand and the United Kingdom. Participants used the app over a period of five to six weeks after which they rated the usability of the app and participated in an in-depth qualitative interview.

We developed four main themes through our thematic analysis. 1. Validation of participants' own experiences of living with MS fatigue. 2. The personal cost in engaging with such an intervention. 3. Reframing experiences and adding to knowledge. 4. That the app was generally a good idea. Field testers' feedback also identified usability issues that could be addressed. In particular, the amount of text-based content in the app contributed to the app itself being fatiguing. This field-testing process has highlighted the value of the app while also guiding our roadmap for further developments to enhance usability and usefulness. The next step is further refinement of components of MS Energize in preparation for a trial of its clinical and cost effectiveness.

Highlights

• •Field testing assessed usability and perceived usefulness of a fatigue self-management app for people with multiple sclerosis (MS).
• •Feedback from field testers reinforced the value people with MS perceive for a tool such as MS Energize.
• •Findings highlighted support may be important early in diagnosis, when conventional ways of fatigue management delivery are often unavailable.
• •MS energize could potentially fill a gap in service delivery.
• •Field testing feedback will guide further developments to enhance usability and usefulness.

1. Introduction

Multiple sclerosis (MS) is a long term, unpredictable, incurable, demyelinating disease of the central nervous system, leading to significant impairment, health burden and disability. It affects around 2.2 million people worldwide (Feigin et al., 2017). Fatigue is one of the most commonly reported symptoms of MS (Barin et al., 2018; Krupp et al., 2010). MS fatigue differs from tiredness experienced by healthy people in both severity and impact. It has a profound effect on all spheres of life (Krupp et al., 2010; Wood et al., 2013) both for the person with MS and their relatives (Bogosian et al., 2011) and is one of the key precipitants of early retirement (Schiavolin et al., 2013; Simmons et al., 2010). Between 65 and 97% of people with MS experience significant levels of fatigue and 50–60% describe it as their most disabling symptom (Amato and Portaccio, 2012). Effectiveness of pharmacological management of MS fatigue is equivocal, with most trials failing to demonstrate consistent change in fatigue (Amato and Portaccio, 2012; Asano and Finlayson, 2014). Other approaches such as energy conservation or exercise have also been disappointing, measuring only short-term reductions in fatigue and using a wide range of outcome measures (Blikman et al., 2013, Blikman et al., 2017; Heine and de Groot, 2016).

1.1. Management approaches for MS fatigue

Given the relationships between physical and psychological aspects of MS fatigue (van Kessel and Moss-Morris, 2006) it is no surprise that greater attention is being paid to psychological approaches which impact upon fatigue. Most notably we have seen the development of cognitive behavioral therapy (CBT) based interventions for MS fatigue management. Such approaches have been developed, trialled and found effective for MS-related fatigue by members of our team (Thomas et al., 2010, Thomas et al., 2013, Thomas et al., 2014; van Kessel et al., 2008; see also review in van den Akker et al., 2016). Despite promising findings of CBT approaches for fatigue management delivered by health professionals, these approaches are relatively costly, usually require travel, may be difficult to access by people who are unwell or have difficulties with mobility, and are not widely available. This is true internationally (Rhodes et al., 2014). Cost-effective solutions are required that alter the service delivery model to increase the availability of more universal support for managing this debilitating symptom. There is a need to find approaches that deliver evidence-based cognitive behavioral interventions in innovative formats that do not involve the significant health delivery costs often associated with consultations with health professionals. Although app development is costly, the cost to the user is small and generally not ongoing (Muoio, 2018).

1.2. mHealth interventions

Mobile health interventions can empower people to take charge of their symptoms without health professional input, can be used when it is most timely and relevant, and support individuals to change health behaviors and improve disease management. They can be designed to be used in both a self-help style and as an adjunct to therapy (de la Vega et al., 2014; Knight-Agarwal et al., 2015). They can be individualized through evidence-based algorithms, and can incorporate in-built mechanisms to ensure a person works through materials at a pace slow enough to maximize outcomes and offer opportunities to embed the intervention into everyday life (Free et al., 2013). Evidence suggests that people with neurological conditions are able to use mobile health technologies with no special training (Babbage, 2014), that they can be used by people with physical impairments, and specifically by people with MS (Giunti et al., 2018a; Marrie et al., 2019). A small survey (n = 51) carried out by members of the research team in New Zealand (NZ) suggested that 80% of people with MS surveyed use a smartphone (van Kessel et al., 2017).

We aimed to develop a comprehensive cognitive behavioral intervention for fatigue through an interactive smartphone application (app) that people with MS could use to self-manage their fatigue. MS Energize—or MS Energise in UK English regions—is an app designed to enable a person with MS to learn about MS-related fatigue, with interactive activities to consolidate their learning, and tools to support them in making changes in their daily lives. The current paper describes the stage of obtaining feedback from people with MS on the interface, layout, navigation, usability and content of the app after they field tested the full MS Energize app over multiple weeks in a naturalistic setting.

2. Material and methods

2.1. Participants

Participants were six women and five men, most of whom had lived with multiple sclerosis for many years, recruited from New Zealand and the United Kingdom—see Table 1. (Note in Table 1, and throughout this manuscript, pseudonyms are used to refer to participants.) Participants ranged in age from 41 to 59 years. Six were of New Zealand European descent, one was New Zealand Māori, one was South African mixed ethnic background, and three were White British. Some participants were previously known to the two teams as they had taken part in earlier parts of this research programme or other research projects at the university, while six participants were not previously known.

Table 1.

Participant demographics.

Pseudonym	Gender	Age	Years living with MS	Ethnicity
Kathy	Female	46	>10	NZ European
George	Male	59	>10	NZ European
Heera	Female	55	>10	NZ Māori
Amy	Female	45	10	NZ European
Alison	Female	48	>10	NZ European
Grace	Female	41	3	South African mixed ethnic background
David	Male	59	>10	NZ European
Felicity	Female	56	>10	NZ European
Steve	Male	42	8	White British
Tony	Male	42	8	White British
Kevin	Male	46	10	White British

2.2. Equipment

2.2.1. MS Energize app

MS Energize was originally conceived by PK, KVK and DB. MS Energize is a native iOS app developed as part of the current research program. The app consists of a series of seven inter-linked core topics (see Fig. 1a):

• •MS fatigue—explaining fatigue.
• •Behavior—how what you do affects MS fatigue.
• •Thoughts—how thoughts can influence MS fatigue.
• •Emotions—emotions and their impact.
• •Body—potential effects of bodily factors.
• •Environment—the impact of external factors.
• •Future—how to maintain progress and deal with setbacks.

Fig. 1.

Sample screenshots from MS Energize.

All these modules draw on CBT principles. The topics are designed to be initially worked through in a sequential order (see Fig. 1a). Each topic contains education (1b, c), interactive tasks (1d–f), personalized feedback (1f, g) and app-supported real-world actions. Activities and actions include knowledge quizzes (1d), vignettes, sleep (1g) and activity diaries and goal setting (1h). MS Energize media include audio and video (1c) that present both men and women with MS who are drawn from different cultural and ethnic backgrounds: two females and one male, two identifying as NZ European (one of these originally from England) and one NZ Māori. These are accompanied by text in lay person's language and some terms that are familiar to people with MS (1b). The app provides visual summaries and encouragement on program progress and accomplishments using built-in algorithms. Standardized measurement tools in the app allow evaluation of changes in fatigue severity and impact (1e). Progress is staged to prevent counter-therapeutic haste and overload, while providing time for the person to evaluate their fatigue levels and how they impact on daily life over time.

MS Energize was coded in Objective-C and Swift using the Xcode Integrated Development Environment. DB was architect and lead developer of MS Energize, writing approximately half the code, while our external development partner MEA Mobile contributed to design, prototyping, and completed the remainder of the coding. Elsewhere (paper in preparation) we describe the design, development and usability testing process that preceded this field trial.

2.2.2. Smartphone handsets

At the time of this study, MS Energize required a minimum of iOS 9.1. It could therefore be run on the iPhone 4S and all newer iPhones, and on the iPod Touch fifth-generation. When participants had a compatible iPhone we installed MS Energize on their own phone for the study. If a participant did not own a compatible iPhone we loaned them an iPhone 6S for the duration of the study. All participants were familiar with smartphone technology and all owned a smartphone, even if they were loaned a phone for the study.

2.2.3. The System Usability Scale

The System Usability Scale (SUS; Brooke, 1996) is a 10-item measure for the assessment of user perceptions of the usability of a software system. Despite being brief, it has consistently been shown to have good reliability and validity (see Lewis, 2018, for a comprehensive review). It has become the most widely used standardized survey measure (Lewis, 2018), including demonstrating value in the evaluation of mHealth applications (see Zapata et al., 2015, for examples of the use of the SUS and discussion of other usability assessment methods in mHealth). The standard scoring method places scores on a scale from 0 to 100 (see Lewis, 2018). The Sauro–Lewis curved grading scale converts this score to a normative percentile score, and provides letter grades for usability ranging from A to F, where A represents an above-average user experience and F indicates a poor one (Lewis, 2018; Sauro and Lewis, 2016).

2.3. Procedure

Ethical approval was obtained for New Zealand participants through the Auckland University of Technology Ethics Committee and for UK participants through the Bournemouth University Research Ethics Committee. Our research team comprised members from both NZ and the UK, and we recruited in both locations for greater breadth of data. People who had a diagnosis of MS and self-reported fatigue were eligible for the study. A convenience sampling approach was utilized. Participants were recruited through prior contact from a national survey or other project work, or through a local branch of the MS Society in New Zealand or the UK. From this general outreach, 11 people approached the team to participate. None dropped out.

MS Energize was installed onto participants' phones. Participants were asked to try to use the app regularly, to proceed through modules according to their intuition and previous experiences of app and phone use. A team member contacted participants between 2 and 5 days later to check for any early technical issues they may have had, then participants were contacted again 5–6 weeks after the initial meeting. At this point, field testers completed the SUS to capture their high level impressions of using MS Energize. A member of the research team (ST or JD) then conducted an in-depth qualitative interview, using an interview guide (see Appendix 1), with each field tester to further understand their experience of using the app. The interviewers' professional backgrounds were psychology and pharmacy/public health. Interviews, lasting between 20 and 40 min, took place in people's homes or an MS clinic (with no one else present) and were audio recorded and transcribed.

2.4. Data analyses

Quantitative ratings from the post-testing SUS questionnaire were analysed using descriptive statistics to provide a quantitative summary of field tester reactions to using MS Energize.

Transcripts were analysed using thematic analysis, following the guidelines by Terry et al. (2017), that are based on the approach of Braun and Clarke (2006). Themes were both derived in advance (since the team wished to address particular themes as part of usability testing) and derived from the data. Three team members (DB, JD, AS) spent time becoming familiar with the interview transcripts and undertook initial data coding. In a data analysis working session attended by these three authors plus KvK, who also had prior access to the interview transcripts, the codes and associated proto-themes were examined, and reworked through discussing their individual coding work together. On the basis of this analysis session, DB and JD revised the proto-themes, which were subsequently discussed again between this initial group of four authors. From here, DB and JD prepared candidate themes and theme definitions that were circulated to all authors, whose further feedback and clarifications were incorporated. This rigorous review process led to the refinement of the core themes and subthemes presented below.

Data saturation was not considered relevant for this usability study. Pragmatically, we are reporting on usability data with a small group of people. From a pragmatic usability testing perspective our view is it would be wasteful of participant time to continue to collect a larger sample of participants to strive for ‘saturation’ when the purpose of such work is to identify the most prominent usability issues with a technology in order to fix them—following on in the future with additional participants to test the improved version of the technology, rather than strive to achieve a saturation of insights on the original technology. Additionally, data saturation is in any case a highly contested concept that many qualitative researchers reject. Arguments against the idea of saturation include that it implies there is a true set of themes that will ‘emerge’ from the data if sufficient raw source material is mined. We rather take the approach of Terry et al. (2017), arguing that themes and insights are actively constructed by researchers through their engagement with the data. From such a perspective, there is no fixed set of latent themes to be identified in a dataset, and ‘saturation’ is thus from this perspective not a theoretically meaningful concept.

Participant checking was not carried out. Our participants have a long- term condition where fatigue is a primary presenting symptom. For this research, we felt it would be an unreasonable burden on participants to add a participant checking stage to the current process. We also note that Thomas (2017) demonstrated there is little evidence participant checks increase the quality of qualitative research.

3. Results

3.1. System Usability Scale ratings

At the completion of field testing, participants rated MS Energize with a median total score of 75 on the SUS (mean score 72.3, range 65 to 90 plus one outlier score of 32.5). On the Sauro–Lewis curved grading scale (Lewis, 2018; Sauro and Lewis, 2016), this is a B, with the score falling in the 70–79th percentile.

Participant ratings on individual SUS items are presented in Fig. 2. Nearly all participants (n = 10) reported they were able to start using the app without learning a lot (Q10), that they felt confident using the app (Q9), and that they would not need the support of a technical person to use the app (Q4). Most participants also felt that it was easy to use (Q3, Q8, n = 9) and they imagined most people would learn to use the app quickly (Q7, n = 8). A majority (n = 8) also reported the app functions were well integrated (Q5) and the app was consistent (Q6), though some participants disagreed with this (n = 3). We saw much more mixed feedback however on the level of complexity in the app, with over half of participants being either neutral on this (n = 4) or strongly endorsing that the app was ‘unnecessarily complex’ (n = 2). Just 3 of our 11 participants endorsed that they ‘would like to use this app frequently’—the majority of participants (n = 5) were neutral on this.

Fig. 2.

Ratings of MS Energize on System Usability Scale items.

3.2. Thematic analysis

Thematic analysis of the interview data generated four themes: validation; personal cost; reframing my experience and adding to my knowledge; a good idea—for someone. Pseudonyms are used when presenting participants' quotes.

3.2.1. Validation

The first theme reflected findings that MS Energize validated participants' own experiences of living with MS fatigue, and included the two subthemes discussed below.

3.2.1.1. “Made me feel more normal”—relating to real people

When an app user listened to or watched a person living with MS giving a brief description of their experiences in a short video, there were two reactions. There was a sense of being able to relate to what was being shared and a validation of their own experiences.

“It just makes me feel more of a human or something to relate to somebody else who's having the same kind of experiences.”

Alison

Seeing someone in a similar situation had a normalizing effect. Field testers described living with MS fatigue, acknowledging that it is a difficult thing to explain to people who can't see it. As a result, they may feel that they are perceived as “a bit nutty really”.

“Especially listening to [neurologist]. Because he was explaining things and it was good to hear somebody at that level explaining that what we were going through, some of the fatigue things that we were going through, that they are real.”

Steve

Engaging with the content in the app resonated with people's own experiences of fatigue. This was something widely appreciated amongst field testers.

3.2.1.2. Someone doing something

Several field testers endorsed the decision to create the app. They felt it was positive that the app had been developed and that there was something concrete available to use. This was another form of validation, affirming that MS Energize was addressing a legitimate concern.

“I liked that it existed, yay. That's really good and really positive that, I really like just that about it.”

Kathy

“I think it's great the more things that we can get, you know to support people with MS, I think it's fantastic.”

Amy

These comments are consistent with our observation that people with MS are often relatively under-served compared to people with other long-term health conditions. This underlying enthusiasm also appeared to reflect a desire to see a greater range and depth of services and intervention options available to people with MS.

3.2.2. Personal cost

The second theme referred to the idea that there was a personal cost in engaging with such an intervention, yet it also had the potential to reframe individuals' own experiences and add to their knowledge. This theme included three sub-themes.

3.2.2.1. Using the app could be fatiguing

Some users found the app required significant cognitive effort. The large amount of content in the app was commonly noted. While the information was acknowledged as appropriate and accurate, it was simply too much to take in for some people.

“I had to try and remember exactly what had happened that day at what time and how I'd felt, how fatigued I'd been at that time and it was quite a lot of brain work… I found that quite hard.”

Alison

It appeared that the text-heavy presentation of the Learn and Interact sections within each module of the app were experienced as particularly fatiguing.

“There's too much to read when you're going through all the instructions, it's too texty and if you're doing that on a phone screen, yeah, that actually contributes quite badly to your fatigue.”

Tony

Not everyone felt this way, however. Some field testers reported that they found the app straightforward to engage with and they were able to fit this into their routine.

“You could stop whenever you wanted to… Go through the first two bits, have a rest, you can then quite easily come back, just carry on... you could stop at any time.”

George

One suggestion made by a field tester was to include guidance about time of day to use the app or to consider energy levels first.

“Sometimes I'd pick the worst time to go through it. I'd probably do it, you know, on the bus on my way home from work when I was really tired and fatigued... Make sure it's not the end of the day because you're reading stuff and you really don't want to be doing that after you've done a full day of work.”

Steve

This participant highlights that we did not sufficiently take into account that fatigue is felt most strongly at the end of a day when considering our app design and our intended patterns of use, such as asking users to review their activity at the end of a day. App interactions should be designed to work well at other times of the day, not anticipating the end of the day as the primary time users will engage with the app.

3.2.2.2. Focusing on fatigue could prompt negative thoughts

While concentrating on the app itself was described as fatiguing, the app content was of course focused on the issue of fatigue. It described potentially fatiguing situations, asked a user to engage with scenarios that involved a hypothetical person experiencing a range of reactions to their MS symptoms, and also proposed activities to support the user to manage fatigue in their day-to-day life. Many of these practical elements provided tools which required the user to record aspects of their experiences and fatigue. This was not always experienced as positive:

“I found that my experience of my, the reason I wouldn't keep using the app is my experience of my fatigue while I was putting so much thought into it was actually worse.”

Kathy

Some activities were specifically designed to engage users in identifying unhelpful thinking styles in fictional characters described in various situations, as part of the process of learning to recognize and challenge such thinking. Paradoxically, the reports of some field testers indicated that engaging with these stories instead prompted unhelpful thoughts in them about their own fatigue.

“For example there was one of the quizzes… where you were to, you have an option of how this person might feel in the situation, it was a person with MS who had to cancel on going out with a friend for a meal because of the fatigue and … the app's correct answer was that she felt guilty. And I found that really frustrating, I think that's an inappropriate feeling to have… So it's that kind of putting the idea in my mind that I should feel guilty in that circumstance.”

Kathy

Some stories in the app, such as the one Kathy describes here, were illustrating unhelpful reactions that a person might have to a situation. The story itself prompted unhelpful thoughts for Kathy that led to her distress. This suggests that Kathy did not challenge her own unhelpful thought, something that is encouraged as part of the app. This is a salutary lesson regarding the need to carefully scrutinize process when taking clinical techniques that are evidence-based and tried-and-tested in face-to-face therapy and converting them to a self-directed platform. In a face-to-face therapeutic session the clinician can identify and address unexpected client distress, and explicitly reflect with the client on potentially unhelpful thinking. The surprising reactions observed in some of our participants give us pause for thought about whether similar effects are also created in other contexts that present therapeutic ideas to end users but where there is no direct clinician oversight.

3.2.2.3. Challenging existing strategies

Field testers had their own strategies for coping with MS fatigue—unsurprising given most were many years post initial diagnosis. In places, field testers reported the information in the app seemed to contradict or exclude their own strategy. This was a challenge for some.

“I just had those questions about where my experience differed slightly about what I thought the app was saying to me, but that was just something that related to me personally.”

Alison

Within the app content, users are told that each person's experience varies. It may be helpful to add additional acknowledgement that people may also have found other strategies useful, and that these could be legitimate, even if they weren't included in the app. However, at times, users described concrete benefits from considering the differences between their past knowledge and experience, and what was presented in the app.

When the content didn't make sense there was also a perception of wasted energy and time. A participant described watching an animation that she didn't understand the point of.

“One particular one I was really like, I really had no idea what they're trying to get at. And others, some of the other ones I kind of saw what you were getting at but actually watching then took time that I didn't really feel that I needed to take time to do.”

Amy

Our participants described a continual sense of valuing the limited amount of energy available to them and their need to use it wisely. This included not wasting it on reading or concentrating on what was felt to be irrelevant or material they perceived to conflict with what they already knew about their MS.

3.2.3. Reframing my experience and adding to my knowledge

The third theme focused on the idea that some people who had been living with MS fatigue for many years discovered new information in the app or new ways of managing their fatigue.

“I hadn't expected to see two batteries, I just hadn't thought about it like that and I was still in the process of kind of figuring out how that related to my experience over the years in relation to those two batteries and just thinking, is that helpful to me?... I think it was just that I hadn't thought of that idea before and I needed a lot more time to think about it in relation to my years of experience about that, yeah.”

Alison

Testers who discovered new insights described their decisions to try and think about things differently. After years of developing their own strategies to cope with fatigue, it was a challenge to be presented with new information, yet participants were open to changing their perspectives.

“One of the things I like to know is understand what's going on. So to know about my fatigue more, to know that diet, exercise or daily activities can have an impact on my fatigue. That every afternoon I have a nana nap is because that helps with my well-being. And it was interesting around nana naps because they talked about function and stuff and I thought, well no, I have a nana nap but I thought OK maybe I need to think about it differently because I actually don't sleep. I have a nana nap but I don't have a rest. It's my body's not functioning but my brain's still working, so I learnt a bit from that guy that was in the video. And I think the information's good, I think it taught me. I liked it.”

Heera

Some field testers were prompted to re-think some of their management strategies even though they stated that they felt well aware of the issues surrounding MS fatigue and how to manage it.

“I feel as though I'm quite aware of the issues that this surfaces, and while I might not be 100% in managing them all, I don't know that I would use the tool… What was helpful was the little pages. I think that was a really good learning exercise for me… I might return to it to go, oh what was that issue about emotions? Unhelpful thoughts versus helpful thoughts… If I'm sharp with my wife I'll reflect on that and apologize because it's taught me to do those things.”

David

Some enjoyed being challenged and reflected on what they knew and viewed the field testing as an opportunity to check that they were still following best practice recommendations.

“I liked the fact that it challenged me to think about what I had learnt already about fatigue and myself and how I managed it. And to just try and figure out if that was still true and whether any of the new information on the app I could use to my advantage or not... it just made me review everything I knew really and how I was using it in my life, and whether I'd got into habits that weren't helpful to me and whether I didn't know something that could be helpful to me, so try and be open to the new ideas the app presented to me if there were some.”

Alison

Whether users were directed to rethink strategies or chose to view it as a learning exercise, most agreed they had spent some time in reflection and consideration of the ways they were managing their fatigue, which they viewed positively.

3.2.4. A good idea—for someone

The fourth theme highlighted that field testers generally viewed the app as a good idea, and this theme contained two sub themes.

3.2.4.1. Hitting the mark

Field testers described the app as being appropriate, recognizable and something they could relate to. From the orange color of the app (MS Society color in the UK) to the inclusion of explanations from a neurologist via video clips, people identified with it.

“When they described certain things it made sense to me so I felt like I was in the right place, and it was all things that were relevant to me and I could identify with, so yeah.”

Felicity

Participants noted the content felt relevant to their lives and situations. Reflecting on their journey, and how they had learned to make changes in thinking patterns over the years, users affirmed that the content was in line with what they had previously been taught and found helpful.

“To getting to understand that it's OK, it's not you, it's the MS that's causing it and what you can do is change your thinking away, these things, and what's going on in your mind, change that, which is changing a lot of your normal behavior, a lot of the emotions. That was very, very good in the app.”

George

3.2.4.2. A good idea if you're new

Most field testers found the app to be comprehensive in its coverage of techniques and advice for managing fatigue and indicated that for a person newly diagnosed with MS it would be a very valuable resource.

“People who are newly experiencing MS fatigue, definitely. Yep, there was a lot of stuff that I would have gotten out of it that I've had to learn for myself over time or by picking up little bits from here and there; having it all together in one resource? Really, really valuable, really good.”

Kathy

Users who had lived with MS for many years described the process of learning about the condition and having many questions at the beginning of their journey. Their feedback was that the app was a resource with much helpful information in one place that would be likely to have been valuable to them when they were first coming to understand MS.

4. Discussion and conclusion

Participants provided much positive feedback about MS Energize, and gave it reasonable ratings for usability. These findings are consistent with previous research suggesting people with MS can use mobile health technologies (Babbage, 2014; Giunti et al., 2018a; Marrie et al., 2019). However, participants in the current study were least positive about whether MS Energize is a tool they would wish to continue to use on an ongoing basis. Almost all our field testers had many years' experience of living with multiple sclerosis. Many participants suggested that MS Energize would be particularly useful for people who were newly diagnosed with MS, rather than for themselves. Given this advice, the absence of participants who were more recently diagnosed with MS is, therefore, a particular limitation of the current study. People who have lived with MS for a long time may have had good insights into what would have been useful to them when they were first diagnosed. It is also possible that participants sought to identify a group for whom the app is likely to be useful, because they wished to see a tool like this benefit people. We cannot assume, however, that newly diagnosed people will respond positively to this tool, and this is an important area for further study.

Field testers highlighted the need to continue to examine how to evolve our presentation of the learning material in the app. In particular, the quantity of text-based content presented challenges both in terms of complexity and in its fatiguing effects. We have identified a range of ways to further improve this tool, encompassing changes aimed at enhancing the usability of specific components as well as decreasing the overall cognitive load. In particular, we plan to provide alternatives to reading text-based content. MS Energize already supports the in-built text-to-speech facilities of the iOS platform: selecting text, tapping, selecting “Speak”, as well as support for the system-wide VoiceOver accessibility framework. However, our participants who provided feedback on the volume of text in the app did not mention these technologies, and follow-up discussions we had with two of these participants (beyond the interviews reported here) indicated they had not used these functions and indeed were unaware their smartphones had such a capability. In an upcoming version of the tool, we intend to provide options for users to have text content read out on demand through a button tap, as well as a setting to read out text automatically as it is displayed. Initially we will use the built-in system text-to-speech voice, but we also see value in providing an option to download high quality recordings of an actual human reading out the content. Alongside these, we will examine expanding our use of video to further replace text content.

Our research had a number of limitations. One Maori and six white New Zealand participants closely reflects the proportion of Maori to the general population of New Zealand, but in other respects our sample is not as representative as would be ideal with 9 of our 11 participants being white. Our focus in this field trial was on users' qualitative feedback to using the app. However, additional insights might have been gained if we had collected detailed use data that identified which areas of the app each field tester engaged with, the depth of their engagement with those app sections, the number of times and duration that they engaged with each section during the field testing period. This would have provided an additional lens for understanding feedback on usability—for instance, did issues such as fatigue or usability issues affect engagement with some areas of the app more than others. Another approach that might have been more useful for providing these kinds of understanding would have been to field test each app component separately, rather than requesting our field testers work through an entire broad intervention largely independently during the trial.

User centered design techniques have often proved valuable in other mHealth projects including those for people with MS (D'hooghe et al., 2018; Giunti et al., 2018b), as part of accelerating translation of new interventions into practice (Mohr et al., 2013, Mohr et al., 2017). We see the value in further iterative co-design and usability testing of each of the interactive tools in MS Energize (e.g., the Thought Record, Sleep Diary, Activity Planner) to make these more usable and valuable for people with MS.

Accessing any therapeutic intervention for MS-related fatigue will have some cost in terms of fatigue—including, for face-to-face interventions, the practical costs of tasks like travel to and from appointments (Rhodes et al., 2014). The fatiguing effects of engaging with a mobile app-based delivery may be less than this. A mobile app also provides opportunities for digesting content in short sessions, the length of which is under the user's individual control. However, our user feedback highlights the importance of not discounting the fatiguing effects even of engaging with a mobile app—which will likely not be seen in contrast to the alternative face-to-face delivery, but rather weighed up in comparison to simply living life without engaging with that intervention.

Feedback from our field testers reinforced the value people with MS perceive for a tool such as MS Energize. MS Energize is ambitious—aiming to present the content that might otherwise be covered in around eight face-to-face 1 h sessions with a cognitive behavioral therapist, plus the tools to support a person to undertake the ‘homework’ that would normally occur between these sessions. As discussed here, there is much work to ensure that tools such as MS Energize are as usable and useful as possible. We believe that the effective use of smartphones and other similar technologies provides an opportunity to provide health interventions in contexts where they would otherwise be unavailable or unaffordable. Delivering these interventions at the time, place and pace that an end user wants to access them also has the potential to support integration into people's day-to-day activities and thus ultimately support them to achieve long term outcomes that they value.

Funding

This research was supported by funding from Auckland University of Technology, New Zealand;AUT Ventures, New Zealand; Kiwinet, New Zealand; Bournemouth University, United Kingdom.

Declaration of competing interest

The software package that is the focus of this paper has been released through the iOS app store. Kersten, van Kessel and Babbage are the inventors of the product, and along with Auckland University of Technology and Bournemouth University receive royalty payments associated with sales of the software.

Appendix 1. Indicative questions

Part 1:

• 1.What are your first impressions of the app?
• 2.What did you like about using MS Energise? Why?
• 3.What didn't you like about MS Energise? Why?
• 4.
  What was helpful about the app?

  • a.
    What wasn't helpful?
• 5.
  Was the app easy to use?

  • a.
    Were the instructions clear to follow?
  • b.
    Did it seem logical?
• 6.
  Were you able to understand the content of the app?

  • a.
    Language used?
• 7.
  Did everything work as expected?

  • a.
    Did anything not work, or seem surprising?

Part 2:

• 1.Did you become tired/fatigued by using the app?
• 2.In what way, if any, did your MS symptoms affect your ability to use the app?
• 3.
  Is there value in the app?

  • a.
    What is the value of the app for you?
• 4.Is anything missing from the app that you think should be there?
• 5.Would you recommend the app to others?
• 6.
  Would you use the app?

  • a.
    If yes, what would be a reasonable price to pay for it?