Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Mar 12.
Published in final edited form as: Psychiatr Rehabil J. 2013 Sep 9;36(4):289–296. doi: 10.1037/prj0000019

Development and Usability Testing of FOCUS: A Smartphone System for Self-Management of Schizophrenia

Dror Ben-Zeev 1, Susan M Kaiser 2, Christopher J Brenner 3, Mark Begale 4, Jennifer Duffecy 5, David C Mohr 6
PMCID: PMC4357360  NIHMSID: NIHMS571345  PMID: 24015913

Abstract

Objective

Mobile Health (mHealth) approaches can support the rehabilitation of individuals with psychiatric conditions. In the current article, we describe the development of a smartphone illness self-management system for people with schizophrenia.

Methods

The research was conducted with consumers and practitioners at a community-based rehabilitation agency. Stage 1: 904 individuals with schizophrenia or schizoaffective disorder completed a survey reporting on their current use of mobile devices and interest in mHealth services. Eight practitioners completed a survey examining their attitudes and expectations from an mHealth intervention, and identified needs and potential obstacles. Stage 2: A multidisciplinary team incorporated consumer and practitioner input and employed design principles for the development of e-resources for people with schizophrenia to produce an mHealth intervention. Stage 3: 12 consumers participated in laboratory usability sessions. They performed tasks involved in operating the new system, and provided “think aloud” commentary and post-session usability ratings.

Results

570 (63%) of survey respondents reported owning a mobile device and many expressed interest in receiving mHealth services. Most practitioners believed that consumers could learn to use and would benefit from an mHealth intervention. In response, we developed a smartphone system that targets medication adherence, mood regulation, sleep, social functioning, and coping with symptoms. Usability testing revealed several design vulnerabilities, and the system was adapted to address consumer needs and preferences accordingly.

Conclusions and Implications for Practice

Through a comprehensive development process, we produced an mHealth illness self-management intervention that is likely to be used successfully, and is ready for deployment and systemic evaluation in real-world conditions.

Keywords: Mobile Health (mHealth), serious mental illness, technologies, illness-management

Schizophrenia is the most severe and costly of psychiatric illnesses (Murray & Lopez, 1996; Wu et al., 2005). Antipsychotic medications alone may be insufficient for effective long-term management of the illness; many individuals report experiencing persistent symptoms even in the context of ongoing pharmacological care (Kane, 1996), medications have not been shown to be effective in treating social, occupational, and cognitive impairments (Buchanan et al., 2009), and there is evidence suggesting that the majority of people with schizophrenia stop taking their medications as prescribed over time (Valenstein et al., 2006).

Illness Self-Management for Schizophrenia

The need for additional treatments for schizophrenia has propelled the development of several evidence-based psychosocial interventions that can be offered as adjunctive to antipsychotic medications (Mueser, Deavers, Penn, & Cassisi, 2013). Illness-management is one approach that was designed to engage individuals with schizophrenia as active agents in their own treatment by teaching them to self-monitor their ongoing clinical status, avoid high-risk stressors, stay on track with their medications, and use various coping and compensatory strategies when symptoms and other problems associated with their psychiatric condition emerge (Mueser et al., 2006). Evidence from more than 40 randomized controlled studies suggests that illness management strategies can effectively enhance people’s knowledge of their illness, help them take their medications, mitigate the severity and distress associated with persistent symptoms, and reduce symptomatic relapse and hospitalizations (Mueser et al., 2002).

Illness management can play an important role in the functional rehabilitation of individuals with schizophrenia, but it is rarely available for a variety of reasons. Chief among them are the lack of clinicians who are trained in these interventions and limited funding for psychosocial treatments in most mental health settings (Drake, Bond, & Essock, 2009). Moreover, the availability of resources does not necessarily ensure their quality; studies with individuals receiving routine mental health services outside of tightly controlled research studies show that even when psychosocial treatments are offered, they often do not meet evidence-based guidelines and standards (Mojtabai et al., 2009). Clearly, there is a need for intervention delivery models that can overcome these obstacles and increase the accessibility and utilization of high-quality illness-management strategies for schizophrenia.

Mobile Health

Clinical researchers have begun harnessing advanced technologies to develop highly innovative approaches that hold great promise for enhancing the accessibility and quality of care available to people with schizophrenia, their family members, and supports (Álvarez-Jimenez et al., 2012; Ben-Zeev et al., 2012). Mobile technologies such as cellular phones and smartphones are particularly well-suited to serve as platforms for psychiatric illness self-management as they are ubiquitous, carried on the person, typically turned on, and often have impressive computational capacities and connection to the Internet (Proudfoot, 2013). Thus, in addition to their originally intended commercial uses, mobile devices can also facilitate access to self-monitoring resources, time-sensitive health information, prompts, reminders, and personalized illness-management tools in real-time and in real-world environments (Harrison et al., 2011; Luxton, McCann, Bush, Mishkind, & Reger, 2011). The enthusiasm for leveraging mobile technologies in support of health care has led to the emergence of a novel interdisciplinary field called Mobile Health (mHealth). In just a few years since its inception, mHealth has seen phenomenal growth. More than 10,000 mHealth applications are available for download on popular online app stores (with more added daily); federal, foundation, and private sector organizations are funneling resources to support the development of new mobile technology-based treatments, and industry and consumers are pushing for broad mHealth implementation and scale up (Tomlinson, Rotheram-Borus, Swartz, & Tsai, 2013).

User-Centered Intervention Development

The rapid development of mobile technology and wireless infrastructure has outpaced the science. Many mHealth systems are being made available to the public with insufficient scientific effort devoted to their design, development, and evaluation (Nilsen et al., 2012). The quality of any new system (e.g., usability, potency, safety) requires a thorough understanding of the context of its proposed deployment. If applications are produced with inadequate consideration of the needs of their intended users, they will be difficult to learn, will be misused or underutilized, and will ultimately fail to accomplish their objectives (Maguire, 2001). User-centered approaches are particularly important when developing technological systems for people with severe psychiatric disabilities, as they often have a set of unique characteristics (e.g., cognitive impairments, salient symptoms, limited literacy) that may greatly affect the manner in which they can engage in technology-based services. Our research team set out to create an mHealth intervention that would be suitable for, and usable by, individuals with serious mental illness. In the current article, we describe the staged approach that led to the development of FOCUS, a smartphone system specifically designed to support self-management of illness among people with schizophrenia.

Methods and Results

Study Site and Participants

All stages of the project were conducted with individuals receiving care and practitioners at a large psychiatric rehabilitation agency in Chicago. The organization serves over 3,000 individuals annually and offers a wide range of services (e.g., vocational rehabilitation, psychotherapy, illness-management training, pharmacotherapy) and treatment models (e.g., residential facilities, community-based case management, drop-in centers). This effort was part of an ongoing academic–community partnership designed to foster collaborations that can lead to the development of practices that are effective in real-world conditions (Frounfelker et al., 2012).

Stage I: Needs Assessment: mHealth Survey for Individuals With Serious Mental Illness

Procedures

Direct service staff surveyed more than half of the individuals receiving care at the time, in the context of an initiative to explore mHealth options for treatment provision in community settings. Respondents answered questions about their ownership and use of mobile technologies, payment methods, and interest in future services. Survey information was combined with data from the electronic medical record including demographic and historical self-report, DSM–IV diagnosis determined by agency clinical staff, and ongoing status (e.g., annual income, hospitalizations). Findings from the complete sample were discussed elsewhere (Ben-Zeev, Davis, Kaiser, Krzsos, & Drake, 2013), but here we specifically report on findings from a subset of individuals living with psychotic disorders, as they represent the population of interest for the current project.

Results

The sample consisted of 904 individuals with schizophrenia or schizoaffective disorder with a mean age of 47; 68% were men; 61% were African American, 37% Caucasian, and 5% Hispanic. Approximately 34% had less than a high school diploma/GED, 38% had a high school diploma/GED, and 28% had some post-high school education. The majority (74%) had an income of $10,000 or less annually. We found that 63% of respondents owned a mobile device (i.e., smartphone, mobile phone, or bidirectional texting device for the hearing impaired). Of mobile device users, 91% used it for talking, 31% for texting, and 13% for access to the Internet. Daily use was reported by 58% of users. Approximately 22% reported using a month-to-month plan, 9% used prepaid phone cards, 10% had a long-term contract with a mobile carrier, 25% reported using “government minutes” (i.e., the federal “Lifeline” program that helps low-income individuals pay for mobile phone service in the United States), and the rest were not sure about their payment methods or did not answer the question. Many respondents indicated they would be interested in receiving mHealth services delivered via their mobile device, including reminders about taking their medications or appointments (44%), regular check-ins with practitioners (38%), and psychoeducation and information about treatments and services (31%).

Practitioner Input

Procedures

We met with a group of eight practitioners specifically recruited to represent a range of specializations and service models (e.g., community-based treatment team, vocational rehabilitation, residential care, young adult program). First, participants were asked to complete a brief survey in which they rated (1 = Strongly Agree; 2 = Agree; 3 = Neutral; 4 = Disagree; 5 = Strongly Disagree) a series of statements pertaining to the potential use of mobile devices for services among individuals receiving care. Upon completion of the survey, practitioners engaged in a facilitated group discussion led by two of the study authors (DBZ, SK). Specifically, they were instructed to consider how an mHealth intervention could be of greatest utility to users, to identify a clinical population and treatment modality that would benefit from an mHealth resource, and to suggest points for consideration in the development of a mobile system intended for individuals with severe psychiatric disabilities.

Results

Practitioner survey responses are presented in Table 1. All practitioners believed that individuals will be able to learn to use a mobile device. The majority believed that they will respond to treatment prompts multiple times a day, but will experience difficulty operating an mHealth device. Over half believed individuals are likely to sell the device, and two practitioners believed they are likely to break it. Only one practitioner believed that individuals are likely to experience an increase in symptoms as a result of engaging in the mHealth intervention.

Table 1.

Practitioner Survey Responses

Statement Response Frequency (%)
“Individuals will be able to learn how to use the device properly.” Strongly agree 2 (25)
Agree 6 (75)
“Individuals will experience difficulty operating the device.” Agree 6 (75)
Neutral 2 (25)
“Individuals will respond to prompts from the device multiple times a day.” Agree 6 (75)
Neutral 1 (12.5)
Disagree 1 (12.5)
“Individuals are likely to sell the device.” Strongly agree 1 (12.5)
Agree 5 (62.5)
Neutral 2 (25)
“Individuals are likely to break the device.” Strongly agree 1 (12.5)
Agree 1 (12.5)
Neutral 3 (37.5)
Disagree 3 (37.5)
“Individuals are likely to experience an increase in symptoms as a result of being prompted by the device to answer personal questions.” Agree 1 (12.5)
Disagree 6 (75)
Strongly disagree 1 (12.5)
Open in a new tab

In the subsequent facilitated group discussion practitioners stated that an effective mHealth tool could be beneficial to individuals with severe psychiatric disabilities and facilitate services to “people who are hard to reach.” Specifically, they saw value in an mHealth platform that (a) individuals would use for regular self-monitoring of symptoms and functioning that could be accessed remotely by practitioners, (b) would have functions that could support and expand services (e.g., medication and appointment reminders, prompts to practice illness-management coping skills), and (c) would give individuals tools they could access wherever and whenever they wanted. The group identified three local treatment groups they thought would benefit most from an mHealth service: young adults engaged in Dialectical Behavior Therapy, people attempting to quit using substances, and adults with severe psychiatric conditions engaged in illness self-management training. The group suggested that the mobile system deliver interventions in a visually engaging manner and recommended incorporating additional elements beyond simple text messaging (e.g., sound, images, colors). They emphasized the importance of developing a system designed to be usable by people with low literacy levels and/or cognitive impairments.

Discussion

We found that the majority of individuals with schizophrenia and schizoaffective disorder were already using mobile technology and many are open to receiving services via mobile devices. The penetration of mobile technologies among people living with psychotic disorders was approximately 20% lower than in the general population during the time of the study (Smith, 2011). The practitioners we met with saw potential value in a mobile technology-based resource that would expand the range and potency of existing services, as well as provide new tools that would allow individuals to self-monitor and manage their illness in their own environment. Practitioners did not believe engagement in an mHealth-based treatment would have iatrogenic effects. However, they cautioned us that some individuals would likely have difficulty initially learning to use a new mobile device. A limitation worth mentioning is that the number of practitioners we met with was relatively small, and their comments may not have reflected the views of all those working at the mental health agency, or other organizations. Based on the client and practitioner data we collected, we concluded that deploying existing mHealth resources that were developed for the general population may prove problematic for our intended users, as they may be less familiar with the steps needed to download/update software or remotely initiate a text-messaging service, diagnose communication delays linked to poor wireless reception, or distinguish between communications that originate from the mHealth intervention and information “blasts” sent from wireless companies or marketers. Therefore, we decided to develop a new mHealth resource specifically designed for people with schizophrenia.

Stage II: Developing an mHealth Intervention

Selecting a mobile medium

When developing a novel mHealth intervention there is an abundance of contemporary mobile platforms and systems to choose from, including dedicated mobile monitoring clinical tools (Myin-Germeys, Birchwood, & Kwapil, 2011), sensor packages (e.g., accelerometer wristbands, wearable patches applied directly to the skin), laptop computers and tablets, basic mobile phones with call/text capabilities, and smartphones. We elected to develop a smartphone-based intervention system to minimize potential technical and functional points of failure:

  1. We wanted to use a device that would not appear atypical or attract unwanted attention if used in public. Smartphones are widely used and rapidly dropping in costs—more adults in the United States now own smartphones than basic cell phones (Rainie, 2012). They are easily carried, socially acceptable, and concealable.

  2. We wanted individuals to integrate the new mHealth resource into their routine, with the device readily accessible to help provide illness self-management strategies in the moment, as real-life challenges arise. There is emerging evidence that people with schizophrenia are willing to use smartphones for clinical self-monitoring in daily life (Palmier-Claus et al., 2012). Smartphones have call, text, video, camera, music, and gaming capabilities that make them highly engaging. These functions increase the likelihood that the device will hold some appeal beyond solely serving as a therapeutic instrument and will be carried regularly.

  3. We wanted to develop an intervention that would be intuitive and easy to navigate. To date, many mHealth interventions for management of chronic health conditions used Short Message Service (SMS, or text-messaging) that can be delivered to basic and less costly mobile phones. However, people with schizophrenia and impaired cognitive functioning may have difficulties negotiating the various steps involved in receiving, responding to, and sending text messages (Granholm, Ben-Zeev, Link, Bradshaw, & Holden, 2012). Smartphones allow for text messaging as well as other forms of verbal and nonverbal communication of information (Luxton et al., 2011). They have large touchscreens that allow for clear/large displays and easier manipulation for people who have dexterity or fine motor impairment. Although smartphones are much more sophisticated than basic mobile phones on the back-end, this computational complexity enables a much simpler and intuitive interface on the user-end. Recent research suggests that when given the option, people with schizophrenia prefer to use smartphones for self-monitoring and find them easier to use (Ainsworth et al., 2013).

  4. Finally, individuals with schizophrenia often have limited resources, and we wanted to design a tool that could be available to users independent of the quality of their wireless service or whether they had access to a data plan with a commercial carrier. We decided to design a smartphone application that could be installed on the device itself rather than a web-based application that would require the phone’s Internet connectivity functions or intervention messages sent from a remote server. An application that is “native” to the phone could be used anytime, unhampered by poor reception, connection speed, availability of call minutes, or functioning of a remote server.

Conceptual framework for the intervention

The FOCUS system is grounded in two theoretical models: the cognitive model of psychosis and the stress–vulnerability model of schizophrenia. The cognitive model of psychosis (Freeman, Garety, Kuipers, Fowler, & Bebbington, 2002; Garety, Kuipers, Fowler, Freeman, & Bebbington, 2001) proposes that, in the context of illness-related neurobiological vulnerabilities, maladaptive thinking styles (e.g., a tendency to “jump to conclusions,” cognitive inflexibility) and distorted schemas about the self (e.g., I am incompetent, I am bad), others (e.g., people are hostile), and the world (e.g., the world is dangerous) contribute to the emergence of firmly held dysfunctional beliefs (e.g., delusions of persecution, reference, control) and fuel the interpretation of anomalous sensory experiences (e.g., auditory hallucinations with derogatory content). The model emphasizes the important role of social interactions and suggests social isolation is especially detrimental, as it contributes to sustainment of dysfunctional beliefs that go unchecked and unchallenged over time. It also suggests distress and dysfunction associated with experiencing symptoms are largely linked to the meaning and interpretation of symptoms (e.g., voices are of divine origin) and beliefs regarding anticipated consequences (e.g., I will never be able to work). The stress-vulnerability model (Liberman et al., 1986; Zubin & Spring, 1977) posits that the course and outcomes of schizophrenia are determined by the interplay of biological vulnerabilities (e.g., predisposition for the illness), stress, and coping. To improve illness outcomes, illness-management strategies for schizophrenia based on this model aim to interrupt the cyclical relationship between stress (e.g., fatigue, interpersonal conflict, poor medication adherence) and vulnerability that often lead to symptomatic relapse and illness exacerbation (Mueser et al., 2006).

Designing the mHealth platform

We assembled a multidisciplinary team composed of individuals with complementary sets of expertise (Kumar, Nilsen, Pavel, & Srivastava, 2013), including illness-management for severe psychiatric conditions, behavioral intervention technologies, telemedicine, smartphone software programming, and public health. Practitioners and consumers of psychosocial services were consulted throughout the development process. Together, we outlined a blueprint for a mobile illness self-management system designed for smartphones running the Android operating system.

The FOCUS system is composed of several applications that deploy an array of adapted psychosocial intervention techniques targeting five general domains: medication adherence (behavioral tailoring, psychoeducation about illness, and motivational interviewing strategies), mood regulation (behavioral activation for depression, mindfulness and relaxation strategies for anxiety), sleep (sleep hygiene strategies, health and wellness psychoeducation), social functioning (anger management, activity scheduling, and cognitive restructuring for persecutory ideation), and coping with persistent auditory hallucinations (distraction, relaxation, guided hypothesis testing). Our intention was to structure an intervention framework that would allow users to select the areas they would like to focus on from a menu that could be populated with additional content and treatment targets in the future with relative ease (e.g., diabetes management, smoking cessation).

Suitability for people with schizophrenia

Individuals with schizophrenia may have some unique difficulties in navigating electronic-based interventions and resources. For example, Brunette and colleagues (2012) have shown that people with schizophrenia had problems navigating web-sites designed for the general public (Brunette et al., 2012). To ensure that FOCUS will be usable by individuals who may have illness-related impairments, the system was developed in accordance with design principles for developing e-resources that are user-friendly for people with schizophrenia: minimal steps are required to access content; memory aids (e.g., “continue” button on the bottom of screens) are available to facilitate navigation; screens avoid distracting and superfluous elements; simple screen arrangements, sentence composition, and concrete wording minimize the need for abstract thinking; interface is organized using a simple geometry; content is worded at a low reading level, and a minimal amount of total text is used to convey meaning (Rotondi et al., 2007).

Visual displays (e.g., photographs, cartoons, images of post-it notes with hand-drawn messages) were interwoven with written content to enhance user engagement. Photos with images of people were specifically selected to include a mix of ethnicities, ages, and genders. FOCUS does not require significant working memory load, as all question and response options appear on the same screen, with no need to scroll/scan/encode/recall. The navigation is simple; the system is operated via touchscreen icons on the face of the smartphone. Once an icon is tapped, the system immediately takes the user to the next screen, so there are no prolonged intervals or delays in viewing content.

Overview of the FOCUS System and Features

System-initiated illness self-management

Once FOCUS is downloaded to a smartphone and activated, individuals are prompted to engage with the system daily by an auditory signal and large visual notification that appears on the smartphone home screen, requesting the user check-in. Agreeing will launch a brief self-assessment of one’s status in the target domain (e.g., “How well did you sleep last night?”) with multiple-choice touchscreen response options followed by feedback; if individuals report no difficulties, the system provides positive reinforcement in a summary statement. If any of the assessment targets are “flagged,” FOCUS encourages the user to engage in illness self-management strategies directly linked to the nature of the difficulties they endorsed. Illness-management interventions are structured as interactive question–answer exchanges, suggestions, and visual aids. User responses determine branching (i.e., what follow-up interventions they will receive). For example, a social intervention may focus on anger management, challenging persecutory ideation, or concrete social skills training, depending on one’s response to the assessment questions.

On-demand resources

In addition to responding to FOCUS prompts, individuals are able to initiate guided illness self-management skills as needed. The smartphone has several touch-screen buttons on the intervention selection screen, each representing a different target domain (see Figure 1). Pressing on a button immediately launches an illness self-management assessment and tailored intervention for that particular target. If participants prefer to access self-management content without having to go through a series of questions and responses, they have the option of using a “Quick Tips” function. By pressing the Quick Tips icon on the smartphone’s homescreen, users are taken to a touchscreen menu where they can select a domain to receive immediate (untailored) self-management affirmations in the form of brief text with an accompanying image (e.g., “Some medications can have side effects, and it may take some time before your body gets used to it”; “You can’t control other people’s behavior, only how you respond to it”).

Figure 1.

Figure 1

Open in a new tab

The FOCUS intervention selection screen.

Summary dashboard

Users have the option to authorize automated transmission of their data (e.g., frequency of engagement with the system, content of self-assessments, on-demand tools selected) to a secure server, where it is processed and displayed in the form of a dashboard. All data is time/date stamped and can be viewed via secure web page. Users can grant practitioners access to their FOCUS dashboard, with the intention that the information is used to supplement face-to-face services and inform ongoing clinical monitoring.

Stage III: Usability Testing

Procedures

Twelve individuals diagnosed with schizophrenia or schizoaffective disorders were recruited to participate in 2-hr individual laboratory-based usability testing. First, participants provided demographic information and reported on their experience with mobile technology. Next, they were asked to perform a series of tasks involved in the operation of the smart-phone (e.g., using the phone’s touchscreen buttons, “swiping” screens, using the slide-out keyboard) and the intervention system (e.g., finding the FOCUS icon and launching the intervention, selecting treatment modules, responding to multiple choice assessment questions, progressing through different intervention screens, returning to the homescreen), all in the presence of research staff. Two facilitators were present for each usability session: one administered the session and the other transcribed. Participants were asked to “think aloud” (i.e., provide continuous commentary) while operating the system, a common approach to usability testing that enables evaluation of the ease with which a system is learned and provides information about design problems (see Table 2) (Brinck, Gergle, & Wood, 2001). Individuals provided feedback on the FOCUS interface (i.e., layout, color, graphics, text size, color, readability) and engaged in all treatment modules. The facilitators documented spontaneous comments and observed participants performing basic phone functions, completing self-report items, receiving system feedback, engaging in follow-up intervention sequences and accessing on-demand resources. At the end of each session, participants were administered a brief questionnaire asking them to rate statements focusing on their confidence in their ability to use the system (“I am confident that I will be able to use this system”), satisfaction (“I am satisfied with how easy it is to use the system”), perceived helpfulness (“Overall, how helpful do you think something like this will be for you”), and concerns (“Do you have any concerns about using something like this?”). Finally, participants were given a list of possible names for the system and asked to rate their preference.

Table 2.

Selected Think-Aloud Comments

“I like it …. People helpin’ people—an app. If you feel depressed—an app. Got something to talk to your caseworker about.”
“This button is too small.”
“Too many words. Just go right to the point.”
(In response to sleep hygiene suggestion to “shut off” before going to sleep) “‘Shutting off’ what? What does this mean?”
“You can go into the computer and choose what you want to work on.”
“I like the picture of the cloud … these pictures are cool … positive.”
(Commenting after voices intervention) “Good suggestion. I never tried that before.”
“This should have a time management button too.”
“[This is] more helpful than a manual. I think technology is much greater than a manual—more straightforward.”
“[It will] help me relax … like sleeping and medicines. [I] like the idea that you can choose what to work on.”
“It’s too light. Might fall out of pocket and screen might crack.”
“Need to buy a case with a belt clip that snaps on.”
“You can see it, with glasses or without.”
“[I] thought it was going to be hard to use but it’s not.”
Open in a new tab

Usability sessions were conducted in two cycles. The first was completed with seven participants using a web-version of FOCUS (the system was accessed using the smartphone Internet browser). We used this strategy for early testing using low-fidelity web-based versions to optimize the intervention prior to programming the native application. The second cycle was conducted with five participants using the native application installed on the smart-phone.

Results

Participants had a mean age of 45; 67% were men, 75% were African American, 8% were Caucasian, and 17% were Hispanic. Approximately 50% had less than a high school diploma/GED, 17% had a high school diploma/GED, and 33% had some post-high school education. Seventy-five percent of the 12 participants currently owned and used a mobile device. Only one individual owned and used a smartphone.

All participants were able to learn to use the device and the illness self-management system following a brief tutorial. In the first cycle, usability testing revealed that several participants had difficulty understanding abbreviations (e.g., “meds”) and longer words (e.g., “transition,” “environment”), text was too lengthy, the font was too small, and touchscreen buttons were too close and sensitive to touch, resulting in users making unintended selections or inadvertently skipping screens. All participants commented that they saw value in the images and visual aids. In preparation for the second cycle of usability testing, abbreviations were removed, the amount of text was reduced, wording was simplified to a fourth-grade reading level, font size was increased, the space between buttons as well as button diameter was enlarged, the sensitivity of touchscreen buttons was modified to require longer touch-and-release, and the number of screens with images and visual aids was doubled. In the second cycle of testing, the number of usability issues was greatly reduced; participants did not demonstrate or express any difficulties in operating the smartphone device and the adapted system; think-aloud comments were either neutral (“so now I go here”), positive (“looks beautiful”), or focused on specific intervention content (e.g., “I can’t do that, don’t have my pills with me”; “this makes me feel energetic”). One participant expressed concern about using an expensive-looking device in his neighborhood. Another was concerned the smartphone was “too light” and may fall out of his pocket and break. He stated that a belt clip attachment would be helpful.

All 12 participants felt confident they would be able to use the system (eight “strongly agreed,” four “agreed”). The majority of participants were satisfied with how easy it was to use (eight “strongly agreed,” two “agreed,” one “neutral,” one “strongly disagreed”) and viewed the system as helpful (nine “very helpful,” one “somewhat helpful,” one “neutral,” one “somewhat unhelpful”). Several participants actually engaged in system-suggested coping strategies while in the room (e.g., relaxation) during the usability session. Finally, most participants selected FOCUS as their preferred name for the system.

Discussion

Usability testing demonstrated that even individuals with no prior experience with smartphones can learn to use a mobile illness self-management system quickly. Staged iterative usability testing was essential for discovering intervention enhancement needs (e.g., more visual aids, larger buttons), for resolving lingering design and content flaws (e.g., touchscreen sensitivity, remove abbreviations), and for identifying unanticipated challenges that will need to be addressed before deployment (e.g., concerns about carrying or damaging an expensive device). Overall, participants saw value in the FOCUS system, were confident they could use it for self-management of illness, and expressed interest in doing so given the opportunity.

More and more researchers are trying their hand at mHealth and related technology-supported, illness-management approaches. We drew from other technology-based intervention development frameworks (e.g., Whittaker, Merry, Dorey, & Maddison, 2012; Välamäki et al., 2008), adding new pieces and revising elements that were less suitable for our clinical population or study objectives. Detailed descriptions of intervention development are rarely seen in the scientific literature, as most investigative teams only report on the findings of systemic evaluations of the efficacy or effectiveness of the final product (e.g., Randomized Controlled Trials). In order to propel the field forward, there is a need for descriptions of the intervention development strategy, including important decision-making intersections at every stage, and explanations for how information gathered from intended end-users was used to guide the process (Fjeldsoe, Miller, O’Brien, & Marshall, 2012). Insights from the development of one intervention can inform emerging ones, and help guide investigators as to which strategies would be most helpful.

Conclusion

Innovative technologies can play an important role in supporting self-management of illness, well beyond the confines and limitations of the physical clinic or treatment center (Ben-Zeev et al., 2012; Marsch & Ben-Zeev, 2012). For behavioral intervention technologies to be viable, they need to be developed with the needs, characteristics, and preferences of their intended treatment populations in mind. Only at the end of a comprehensive user-centered development process were we able to produce an mHealth system that is more likely to be used successfully by people with schizophrenia and is ready for testing in real-world conditions (Figure 2).

Figure 2.

Figure 2

Open in a new tab

Staged development of the mHealth intervention.

Acknowledgments

This research was supported by the National Institute of Mental Health, Award R34MH100195, and a Career Development Award to Dror Ben-Zeev from the Dartmouth SYNERGY Center for Clinical and Translational Science.

Contributor Information

Dror Ben-Zeev, Dartmouth College.

Susan M. Kaiser, Thresholds-Dartmouth Research Center, Chicago, Illinois

Christopher J. Brenner, Thresholds-Dartmouth Research Center, Chicago, Illinois

Mark Begale, Northwestern University.

Jennifer Duffecy, Northwestern University.

David C. Mohr, Northwestern University

References

  1. Ainsworth J, Palmier-Claus JE, Machin M, Barrowclough C, Dunn G, Rogers A, Lewis S. A comparison of two delivery modalities of a mobile phone-based assessment for serious mental illness: Native smartphone application vs text-messaging only implementations. Journal of Medical Internet Research. 2013;15:e60. doi: 10.2196/jmir.2328. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Álvarez-Jimenez M, Gleeson JF, Bendall S, Lederman R, Wadley G, Killackey E, McGorry PD. Internet-based interventions for psychosis: A sneak-peek into the future. Psychiatric Clinics of North America. 2012;35:735–747. doi: 10.1016/j.psc.2012.06.011. [DOI] [PubMed] [Google Scholar]
  3. Ben-Zeev D, Davis K, Kaiser S, Krzos I, Drake RE. Mobile technologies among people with serious mental illness: Opportunities for future services. Administration and Policy in Mental Health and Mental Health Services Research. 2013;40:340–343. doi: 10.1007/s10488-012-0424-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Ben-Zeev D, Drake RE, Corrigan PW, Rotondi AJ, Nilsen W, Depp C. Using contemporary technologies in the assessment and treatment of serious mental illness. American Journal of Psychiatric Rehabilitation. 2012;15:357–376. doi: 10.1080/15487768.2012.733295. [DOI] [Google Scholar]
  5. Brinck T, Gergle D, Wood SD. Usability for the Web: Designing Web sites that work. Burlington, MA: Morgan Kaufmann; 2001. [Google Scholar]
  6. Brunette MF, Ferron JC, Devitt T, Geiger P, Martin WM, Pratt S, McHugo GJ. Do smoking cessation websites meet the needs of smokers with severe mental illnesses? Health Education Research. 2012;27:183–190. doi: 10.1093/her/cyr092. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Buchanan RW, Kreyenbuhl J, Kelly DL, Noel JM, Boggs DL, Fischer BA, Keller W. The 2009 schizophrenia PORT psychopharmacological treatment recommendations and summary statements. Schizophrenia Bulletin. 2010;36:71–93. doi: 10.1093/schbul/sbp116. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Drake RE, Bond GR, Essock SM. Implementing evidence-based practices for people with schizophrenia. Schizophrenia Bulletin. 2009;35:704–713. doi: 10.1093/schbul/sbp041. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Fjeldsoe BS, Miller YD, O’Brien JL, Marshall AL. Iterative development of MobileMums: A physical activity intervention for women with young children. International Journal of Behavioral Nutrition and Physical Activity. 2012;9:151. doi: 10.1186/1479-5868-9-151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Freeman D, Garety PA, Kuipers E, Fowler D, Bebbington PE. A cognitive model of persecutory delusions. British Journal of Clinical Psychology. 2002;41:331–347. doi: 10.1348/014466502760387461. [DOI] [PubMed] [Google Scholar]
  11. Frounfelker RL, Ben-Zeev D, Kaiser SM, O’Neill S, Reedy W, Drake RE. Partnering with mental health providers: A guide for services researchers. Journal of Mental Health. 2012;21:469–477. doi: 10.3109/09638237.2012.705923. [DOI] [PubMed] [Google Scholar]
  12. Garety PA, Kuipers E, Fowler D, Freeman D, Bebbington PE. A cognitive model of the positive symptoms of psychosis. Psychological Medicine. 2001;31:189–195. doi: 10.1017/S0033291701003312. [DOI] [PubMed] [Google Scholar]
  13. Granholm E, Ben-Zeev D, Link PC, Bradshaw KR, Holden JL. Mobile assessment and treatment for schizophrenia (MATS): A pilot trial of an interactive text-messaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophrenia Bulletin. 2012;38:414–425. doi: 10.1093/schbul/sbr155. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Harrison V, Proudfoot J, Wee PP, Parker G, Pavlovic DH, Manicavasagar V. Mobile mental health: Review of the emerging field and proof of concept study. Journal of Mental Health (Abingdon, England) 2011;20:509–524. doi: 10.3109/09638237.2011.608746. [DOI] [PubMed] [Google Scholar]
  15. Kane JM. Treatment-resistant schizophrenic patients. The Journal of Clinical Psychiatry. 1996;57:35–40. [PubMed] [Google Scholar]
  16. Kumar S, Nilsen W, Pavel M, Srivastava M. Mobile Health: Revolutionizing healthcare through transdisciplinary research. Computer. 2013;46:28–35. doi: 10.1109/MC.2012.392. [DOI] [Google Scholar]
  17. Liberman RP, Mueser KT, Wallace CJ, Jacobs HE, Eckman T, Massel HK. Training skills in the psychiatrically disabled: Learning coping and competence. Schizophrenia Bulletin. 1986;12:631–647. doi: 10.1093/schbul/12.4.631. [DOI] [PubMed] [Google Scholar]
  18. Luxton DD, McCann RA, Bush NE, Mishkind MC, Reger GM. mHealth for mental health: Integrating smartphone technology in behavioral healthcare. Professional Psychology: Research and Practice. 2011;42:505–512. doi: 10.1037/a0024485. [DOI] [Google Scholar]
  19. Maguire M. Methods to support human-centered design. International Journal of Human-Computer Studies. 2001;55:587–634. doi: 10.1006/ijhc.2001.0503. [DOI] [Google Scholar]
  20. Marsch LA, Ben-Zeev D. Technology-based assessments and interventions targeting psychiatric and substance use disorders: Innovations and opportunities. Journal of Dual Diagnosis. 2012;8:259–261. doi: 10.1080/15504263.2012.723308. [DOI] [Google Scholar]
  21. Mojtabai R, Fochtmann L, Chang S, Kotov R, Craig TJ, Bromet E. Unmet need for mental health care in schizophrenia: An overview of literature and new data from a first-admission study. Schizophrenia Bulletin. 2009;35:679–695. doi: 10.1093/schbul/sbp045. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Mueser KT, Corrigan PW, Hilton DW, Tanzman B, Schaub A, Gingerich S, Herz MI. Illness management and recovery: A review of the research. Psychiatric Services. 2002;53:1272–1284. doi: 10.1176/appi.ps.53.10.1272. [DOI] [PubMed] [Google Scholar]
  23. Mueser KT, Deavers F, Penn DL, Cassisi JE. Psychosocial treatments for schizophrenia. Annual Review of Clinical Psychology. 2013;9:465–497. doi: 10.1146/annurev-clinpsy-050212-185620. [DOI] [PubMed] [Google Scholar]
  24. Mueser KT, Meyer PS, Penn DL, Clancy R, Clancy DM, Salyers MP. The illness management and recovery program: Rationale, development, and preliminary findings. Schizophrenia Bulletin. 2006;32:S32–S43. doi: 10.1093/schbul/sbl022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Murray CJL, Lopez AD, editors. The global burden of disease and injury series, Vol. 1: A comprehensive assessment of mortality and disability from diseases, injuries, and risk factors in 1990 and projected to 2020. Cambridge, MA: Harvard University Press; 1996. [Google Scholar]
  26. Myin-Germeys I, Birchwood M, Kwapil T. From environment to therapy in psychosis: A real-world momentary assessment approach. Schizophrenia Bulletin. 2011;37:244–247. doi: 10.1093/schbul/sbq164. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Nilsen W, Kumar S, Shar A, Varoquiers C, Wiley T, Riley WT, Atienza AA. Advancing the science of mHealth. Journal of Health Communication. 2012;17:5–10. doi: 10.1080/10810730.2012.677394. [DOI] [PubMed] [Google Scholar]
  28. Palmier-Claus JE, Ainsworth J, Machin M, Barrowclough C, Dunn G, Barkus E, Lewis SW. The feasibility and validity of ambulatory self-report of psychotic symptoms using a smartphone software application. BMC Psychiatry. 2012;12:172. doi: 10.1186/1471-244X-12-172. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Proudfoot J. The future is in our hands: The role of mobile phones in the prevention and management of mental disorders. The Australian and New Zealand Journal of Psychiatry. 2013;47:111–113. doi: 10.1177/0004867412471441. [DOI] [PubMed] [Google Scholar]
  30. Rainie L. Pew Internet & American Life Project: Two-thirds of young adults and those with higher income are smartphone owners. 2012 Retrieved from http://www.pewinternet.org/~/media/Files/Reports/2012/PIP_Smartphones_Sept12%209%2010%2012.pdf.
  31. Rotondi AJ, Sinkule J, Haas GL, Spring MB, Litschge CM, Newhill CE, Anderson CM. Designing websites for persons with cognitive deficits: Design and usability of a psychoeducational intervention for persons with severe mental illness. Psychological Services. 2007;4:202–224. doi: 10.1037/1541-1559.4.3.202. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Smith A. Pew Internet & American Life Project: Americans and their cell phones. 2011 Retrieved from http://pewinternet.org/Reports/2011/Cell-Phones.aspx.
  33. Tomlinson M, Rotheram-Borus M, Swartz L, Tsai AC. Scaling up mHealth: Where is the evidence? PLOS Medicine. 2013;10:e1001382. doi: 10.1371/journal.pmed.1001382. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Valenstein M, Ganoczy D, McCarthy JF, Kim HM, Lee TA, Blow FC. Antipsychotic adherence over time among patients receiving treatment for schizophrenia: A retrospective review. Journal of Clinical Psychiatry. 2006;67:1542–1550. doi: 10.4088/JCP.v67n1008. [DOI] [PubMed] [Google Scholar]
  35. Välimäki M, Anttila M, Hatonen H, Koivunen M, Jakobsson T, Pitkanen A, Kuosmanen L. Design and development process of patient-centered computer-based support system for patients with schizophrenia spectrum psychosis. Informatics for Health & Social Care. 2008;33:113–123. doi: 10.1080/17538150802127207. [DOI] [PubMed] [Google Scholar]
  36. Whittaker R, Merry S, Dorey E, Maddison R. A development and evaluation process for mHealth interventions: Examples from New Zealand. Journal of Health Communication. 2012;17(Suppl 1):11–21. doi: 10.1080/10810730.2011.649103. [DOI] [PubMed] [Google Scholar]
  37. Wu EQ, Birnbaum HG, Shi L, Ball DE, Kessler RC, Moulis M, Aggarwal J. The economic burden of schizophrenia in the United States in 2002. Journal of Clinical Psychiatry. 2005;66:1122–1129. doi: 10.4088/JCP.v66n0906. [DOI] [PubMed] [Google Scholar]
  38. Zubin J, Spring B. Vulnerability—A new view of schizophrenia. Journal of Abnormal Psychology. 1977;86:103–126. doi: 10.1037/0021-843X.86.2.103. [DOI] [PubMed] [Google Scholar]

RESOURCES