Mobile, Social, and Wearable Computing and the Evolution of Psychological Practice

Abstract

Psychological assessment and intervention are extending from the clinic into daily life. Multiple forces are at play: Advances in mobile technology, constrained clinical care, and consumer demand for contextualized, nonstigmatizing, and low-cost alternatives are beginning to change the face of psychological assessment and interventions. Mobile, social, and wearable technologies are now enabling individuals to measure themselves and to integrate myriad forms of help and entertainment. The massive data sets generated by self-tracking of mood and passive sensing of voice, activity, and physiology may eventually reorganize taxonomies of mental health concerns. Compelling mobile therapies will also emerge, involving contextually appropriate, entertaining, and dynamic feedback to provide help in the context of daily life. The efficacy of such applications will be tested through citizen science as well as clinical trials. This article reviews technical advances that can be applied to enhance assessment and intervention and dramatically increase access to psychotherapy. It is recommended that, in addition to exploring clinically oriented products, practitioners should support patients’ use of direct-to-consumer applications in ways that align with therapeutic objectives.

In the last decade, the mobile phone has taken on greater emotional and practical significance in people’s lives. Technical advances, in particular the integration of social networking, wearable and embedded sensors, photography, and diverse applications, have transformed the mobile phone into a platform for self-expression, social learning, and role exploration. The emotional bond with the phone has been explored by neuroscientists and ethnographers as well as market researchers. At the neuronal level, people respond to the iPhone sound as they do to a loved one (Lindstrom, 2011), and in many cultures the phone is experienced as an extension of the self (e.g., Ito, Daisuke & Matuda, 2005). Although the intense attachment to the phone is a poor substitute for interpersonal relationships, it can be leveraged as a “transitional object” in the service of psychological growth (Turkle, 2007). Mobile phones thus have great potential to enhance psychological services. Their affordability and near ubiquity will make it easier to scale interventions and to enrich assessment with contextual data about functioning in daily life. These approaches are increasingly necessary as the demand for services cannot be met through the traditional model of individual level therapy (Kazdin & Blase, 2011). Mobile therapies also offer potential alternatives to widely prescribed psychotropic medications, the efficacy and risks of which are under increasing scrutiny (Turner et al., 2008; Whitaker, 2011). Clinicians can make use of mobile technologies in a variety of ways. In addition to clinically oriented tools, a large body of consumer-oriented products can be applied to enhance therapy and assessment. These consumer products may reduce the stigma associated with mental health care and lower barriers to seeking treatment or completing exercises in between sessions. Here we discuss trends in mobile, social, and wearable computing that have implications for clinical practice.

Advances and Perils in Mobile, Social, and Wearable Computing

Mobile Applications

The pool of direct to consumer mobile phone apps (applications) related to emotional health is rapidly growing. By the summer of 2012, it is estimated that there will be more than 13,000 health apps intended for use by consumers in Apple’s AppStore (Dolan, 2011). Of the 9,000 consumer health apps that are currently available, approximately 6% are categorized as relating to mental health, 11% to stress management, 4% to sleep and 2%, to smoking cessation (Dolan, 2011). As noted by Luxton, McCann, Bush, Mishkind, and Reger (2011), more than 200 apps related to behavioral health appeared on the Blackberry market in 2011. One of the most common components of apps related to emotional well-being is mood tracking, which although relatively simple, holds significant potential for aiding emotional awareness and resilience. The trends accumulated from frequent monitoring can help individuals understand emotional patterns and situational triggers (Morris et al., 2010) and enhance communication between patient and therapist (Aguilera & Muñoz, 2011). The experience sampling method, easily enabled by mobile phones, reduces biases in patients’ retrospective report and in the snapshot impressions formed by clinicians (Csikszentmihalyi & Larson, 1987). Most of the apps that offer psychological guidance are based on positive psychology, such as the Live Happy application inspired by Sonja Lyubormirksy’s research (2008). An excellent index of mental health apps was compiled by Luxton et al. (2011), but new apps are developed and released at a rapid pace. The scale and scope of psychologically oriented applications is impressive. The most radical mental health intervention from mobile phones, however, may lie in their affordances for social media.

Social Media

Facebook and other social media, it can be argued, were the most large-scale and influential psychological interventions of the last decade. These social networking tools are largely accessed through mobile phones and are therefore relevant to the discussion of mobile technology and mental health. Facebook, while gathering extensive data, addresses needs for social connectedness of more than 900 million individuals. Twitter, LinkedIn, and other social media offer related value. The loneliness that these sites may help counteract is a core element of affective disorders and a major risk factor for many physical and mental illnesses (Hawkley & Cacioppo, 2010). This potential psychosocial benefit is independent of online support communities and forums specifically focused on mental health. Active engagement on Facebook has been associated with less loneliness and greater feelings of social connectedness (Burke, Kraut, & Marlow, 2011), echoing previous research on social capital (Ellison, Steinfield, & Lampe, 2007). Another mechanism through which social network applications may offer mental health benefit is emotional contagion, that is, the spread of positive affect. Semantic analyses of Facebook status updates indicates that that affective tone spreads within networks (Kramer, 2012). Thus the influence of social media on mental health depends on how and with whom one interacts online: specifically, the extent to which one shares content and forms friendships with people who express positive emotion.

Social media can also pose significant problems for psychological wellbeing. Privacy concerns are highlighted in examples such as the recent suicide of a college student whose homosexual activity was broadcast in his roommate’s Twitter feed and webcam (Zernike, 2012). The psychosocial downsides of social media, from damaging social comparison, parental distraction, to a loss of control over one’s life story, are discussed with great nuance by Sherry Turkle (2011). Turkle suggests that the boasting and very simplistic commentary in typical status updates may actually preclude meaningful intimacy and expression. To this discussion, Kate Crawford (2010) raises the concern of the “compulsory sharing”: In opting out of this broadcasting, one risks invisibility and irrelevance. Facebook communication is of course very different from face to face interaction with a trusted friend or a support group. By expressing vulnerability on these sites, for example, one may inadvertently jeopardize social support (Newman et al., 2011). However, in comparison with a support group defined by common suffering, Facebook and other social network platforms offer a broader set of role models and sources of inspiration. These in-network role models may be especially influential because of shared friends and cultural factors. It is possible that hybrid approaches such as Google Plus, which allows users to define social “circles” for different types of sharing, will allow people to safely express vulnerability with a small group while learning from a broad range of influential role models.

Wearable Sensing

In addition to monitoring via self-report apps and social media, a variety of new apps use data from wearable sensors to enable passive tracking of physiological responses, movement, and location. A burst of such products that integrate wearable sensing and mobile applications for health and wellbeing are now hitting the market. An example is Jawbone UP, a bracelet which senses activity and physical states from a wrist band and offers feedback on exercise, diet, and sleep. An array of similar products integrate accelerometry (built in to most smartphones), temperature, and pulse. Others, such as Zeo and Neurosky, use head worn sensors to discern brain activity related to sleep cycles and engagement, respectively. Posture sensors intended for reduction of back pain are also relevant because they could be integrated with pain management interventions. The validity of the data from some of these low-cost products is debatable—a source of significant frustration for some end users and researchers. More precise and costly tools, such as Affectiva’s sensors of skin conductance and facial recognition, are emerging. As the materials and research advance, consumer products are bound to improve and their relevance to clinical practice will increase as well.

The seeds of these products are found in decades of research at the intersection of computer science, design, and psychology. Interdisciplinary fields such Ubiquitous Computing, Pervasive Computing, Persuasive Technology, Affective Technology, and Human Computer Interaction emerged to share and cultivate this research. The “sociometer” worn microphone that measured conversational turn-taking (Choudhury, 2003), signal processing of affect in voice and speech (e.g., Scherer, 2003; Sung, Marci, & Pentland, 2005), a steering wheel that senses stress from galvanic skin response (Healey & Picard, 2005), mobile therapy activated by a wireless electrocardiogram (Morris & Guilak, 2009), a photographic journal for diabetics (Frost & Smith, 2002), and Gordon Bell’s digital life chronicle (2001) are examples of early research projects that paved the way for today’s products and ongoing innovation.

Smartphones, and in particular the iPhone, have, as Rachel Hinman (2011) from Nokia research aptly stated, provided “a gateway drug for ubiquitous computing.” Much of the wearable and environmental sensing that was considered futuristic only five years ago have become practical or at least easily imaginable tools for daily life. Location and orientation sensing provide step by step directions, wearable sensors can track behavior and physiology, scanning and object recognition allow immediate purchasing and product research, and facial recognition software permits cross referencing of individuals’ online profiles (Acquisti, Gross, & Stutzman, 2011). Smart phones, particularly the iPhone and Android app stores, have transformed these concepts into tangible experiences and opened the door for further innovation. These technologies are quickly influencing physical and mental health care practices.

Technical Convergence That Will Enhance Clinical Practice

Mobile applications, social media, and wearable sensing are evolving in ways that will make them increasingly relevant to clinical practice. First, the sensing of behavior, physiology, and context will be seamlessly embedded into all aspects of daily life, fulfilling Weiser’s early vision for ubiquitous computing (1991). Self-reporting of emotional states will complement the nonspecific physiological data collected by sensors that are either worn or embedded within mobile devices. Sensing of mood, whether from speech, heart rate variability, or neuronal patterns, will become increasingly smart, passive, and continuous. Researchers are now able to characterize the affect of someone talking on a mobile phone by analyzing hundreds of speech qualities (Chang, Fisher, & Canny, 2012). Sensing of social engagement is similarly embedded: In an extension of Choudhury’s sociometer research (2003), the BeWell mobile application monitors conversational turn-taking and reflects wellness data with a dynamic visualization underwater life (Lane, Choudhury, Campbell et al., 2011). Ongoing research with machine learning will combine many indicators of psychological states from voice, expression, movement, and other behaviors that are observable by one’s devices. This embedded emotional profiling will lead to more nuanced assessment and therapies that are tailored and adaptive.

Where logical, the sensing and feedback will occur on the devices people are already using. Embedded approaches such as Azumio’s stress apps, which use the camera of the phone to measure pulse, are far more convenient than the previous era of dedicated biofeedback devices such as Stress Eraser, a handheld that senses respiration rate, or Heart Math, which relies on a finger worn oximeter that plugs into the P.C. Similarly, products such as Philips’ new Vital Signs Camera, which uses the iPad’s camera to assess heart and breathing rate via small changes in facial coloration and chest movement, will likely be more appealing than bulky peripherals. Following an “embedded assessment” approach, feedback in future scenarios will be continuously recalibrated to an individual’s state and his or her responses to feedback in different situations (Morris & Guilak, 2009).

The utility of these technologies will be aided by greater integration across apps, services, and devices. Today, consumers can easily try out many different apps for help with various aspects of their lives. Unfortunately, the consumer typically does not benefit from knowledge exchange across these apps. One flips between a menstrual cycle tracker, a sleep monitor, and a diet coach, for example, without useful cross referencing. For the most part, these aggregated data are available only to analytics companies, such as Google. In the future, people will expect data from different apps to interact and work on their behalf. For example, one’s mood and banking apps might, in combination, spark insights about whether one shops to avoid certain feelings or whether one can actually buy happiness (e.g., through purchasing experiences versus objects). This cross functionality will add to the value provided by any given application and could guide therapeutic recommendations. Advances in data visualization will help individuals and clinicians investigate and act upon trends.

Tomorrow’s systems will facilitate controlled sharing. The social life of very personal data, such as mood, is an intriguing development. It is difficult to predict exactly how this will evolve, but it is clear that people will want to share data from different aspects of their lives with different groups of people, that their criteria for sharing will be dynamic, and that sharing preferences will vary considerably across individuals. The opportunity to share self-tracking data on sites such as PatientsLikeMe has contributed to an impressive bank of public health data that is changing the way patients evaluate their medical options. Many people now use social media to exchange personal health data and informally share advice. Next-generation therapies may involve a layering of crowd-sourced solutions on clinical suggestions. Increasingly, patients expect that, when they provide data about their symptoms or treatment responses, they will have access to it for personal investigation, sharing with others, and learning from the anonymized data of peers.

Shared data will facilitate “Citizen Science” and empower health consumers. Those at the frontiers of self-tracking, for example those in the growing Quantified Self and Participatory Medicine movements, are using off the shelf technologies to assess themselves and uncover paths to optimizing wellbeing. Two platforms for self-experimentation are the Personal Analytics Companion (www.pacoapp.com) and FUNF (http://funf.media.mit.edu). The “Open mHealth” (http://openmhealth.org) movement brings together such as building blocks, to make it easier for others to develop new tools. Aggregation of data across applications, individual investigations, and studies would create a much-needed data bank to help identify disease markers and profiles. Such an aggregation of data would allow people to examine the relationships between different aspects of their health and behavior in light of population trends.

Implications for Clinicians and Researchers

The advances in mobile technology described above—particularly in embeddedness, contextual awareness, sociality, and analytics—will make mobile technology increasingly relevant to mental health care. To begin, they will provide clinicians with a more contextualized understanding of patients’ struggles and an opportunity to tailor treatment accordingly. Rich sets of population data will eventually allow clinical researchers to refine diagnostic systems by examining clusters of symptoms and treatment responses.

These advances will also allow clinicians to offer mobile therapies as either adjuncts or substitutes to psychotherapy, addressing the need for affordable, nonstigmatizing, and effective treatment (Kazdin & Blase, 2011). Cognitive– behavioral therapy is particularly amenable to mobile interventions, given its emphasis on self-monitoring and in situ experimentation with alternative coping strategies. Preliminary studies of mobile therapy based on CBT show promise (Burns et al., 2011; Morris et al., 2010). Individuals used mobile therapies creatively to increase self-awareness, cope with diverse stressors, and empathize with others (Morris et al., 2010). The critical question moving forward will not be whether these tools are better than today’s interventions, but how they can complement one another to enhance to make more psychologically intelligent applications and more sophisticated therapies.

Given the central role of mobile, social, and wearable computing in people’s lives, clinicians should stay abreast of developments and look for ways to make use of these technologies. One option is to use and evaluate products designed for clinicians. Such systems will allow clinicians to easily view trends in patients’ data between sessions and to tailor therapies accordingly. For example, the Mood 24/7 app (Foreman, Hall, Bone, & Kaplin, 2011) allows individuals to track and instantly share moods with providers via text messaging. Mobile systems will continue to illuminate the contextual factors associated with distress and help patients apply specific therapeutic techniques according to social circumstance, location, activity, and other situational variables. Early field trials of mobile therapy as an adjunct to CBT show promise for enhancing therapy and shortening treatment protocols (E. Gorenstein, personal communication, 2009). The use of text messaging as an adjunct to CBT has also received initial positive feedback from patients in a public clinic setting (Aguilera & Muñoz, 2011). The growing integration of mobile technology into mental health treatment is evidenced by clinical trials of mobile interventions for severe mental illnesses such as schizophrenia (Depp et al., 2010), borderline personality disorder (Rizvi, Dimeff, Skutch, Carroll, & Linehan, 2011), and substance abuse (Gustafson, Shaw, Isham, Baker, Boyle, & Levy, 2010). By staying informed of such studies and relevant technology advances, clinicians will be able to guide patients toward the most appropriate, empirically validated applications and incorporate those tools to enhance treatment.

A second path of involvement is for clinicians to integrate the tools patients are already using—whether they relate to moods, diet, music, or general social networking—into the therapy, treating them as they would a narrative or their own impressions of a patient. The clinical dialogue can motivate self-tracking and help the patient align an app’s prompting with therapeutic objectives. The therapist need not have expertise with or endorse the apps; the value lies in discussing how patients are using the tools, and what insights and obstacles arise as they try to make changes in their lives. Discussion about the patterns that patients see in their moods, thoughts and behaviors may help overcome the biases in retrospective report and allow both therapist and patient to develop a better understanding of situational challenges and coping strategies. Insights about these patterns can guide the therapy and patients’ use of applications. Given the innovation in mobile applications and eagerness of consumers to experiment with them, clinicians should explore their role in therapy and assessment. This is particularly true for basic but onerous aspects of therapy such as self-monitoring.

The stages of the therapeutic alliance may shed light on the possibilities for how people can optimally engage with their technologies over time, whether these are direct to consumer applications or products delivered via clinicians. Clinicians can help patients become more sophisticated in their use of tools and give input to developers about features that would enhance therapy or assessment. The qualities of a successful therapeutic relationship— trust, empathy, collaborative investigation— can inform the interaction design and the capabilities of these technologies. The ability to trend data and personalize feedback goes a long way toward establishing rapport. A deeper challenge is responding in ways that raise insight and ability to recognize choices. Siri, the highly publicized listening capability on Apple’s iPhone 4S, doesn’t yet fulfill this need. Technology should be able not just to hear and fulfill demands, but to offer interpretations. Demands such as, “Where is the nearest Starbucks?” or “Tell my friend I’m running late” could conceivably be met with questions about whether a third cup of coffee or making a friend wait aligns with one’s values and long-term goals. Such interpretation would obviously need to be done artfully, lest it erode patience and mental health. The qualities required to build an alliance and raise awareness should be integrated with a variety of other psychological principles, to help people make sustained changes in their lives. The integration of such technology into a therapeutic relationship could improve the alliance by making a patient feel cared for, even if prompts or messages from a therapist or therapeutic agent are automated (Aguilera & Muñoz, 2011).

There are also risks, of course, to bringing technology into the therapeutic encounter. Privacy is a significant concern both to individuals as well as health systems. It is critical that patients understand who has access to their mobile data, how frequently it is monitored, and whether a clinician will intervene between sessions because of negative mood patterns, thoughts, or behaviors tracked on mobile application. Another issue pertains to the “digital divide” and the possibility that lack of access to advanced technologies among low income, elderly, or rural populations could increase disparities in mental health. Further research is essential to illuminate the potential hazards and benefits of integrating mobile technology into clinical practice.

Summary

Mobile phones and all that they interact with—including apps, wearable sensors, and social media—are deeply affecting every aspect of life. They help one navigate not only places and purchases, but also social identities and psychological transitions. Many apps invite self-tracking of emotional and physical health, and participation in citizen science, in which individuals contribute their data for self-investigation and to enable population studies. Mobile technology is beginning to influence psychological assessment and intervention. Therapeutic intervention based on tracking will become increasingly sophisticated, offering therapeutic value to many who do not have access to therapy and enhancing therapy for those who are in treatment. Wearable sensors will facilitate self-tracking, identify contextual variables associated with distress, and help tailor mobile therapy to one’s situation. Social networking applications, largely accessed through mobile phones, pose both promise and risk for mental health, as people share very personal data in search of support, role modeling, and insights about how they relate to others. As consumers demand more nuanced expression and control over their data, applications will emerge that are increasingly valuable for enhancing assessment and psychotherapy. Clinicians who stay abreast of technology advances and associated research will be able to help their patients reap the most benefit from these tools.

Biographies

Margaret E. Morris received her PhD in clinical psychology from the University of New Mexico. She is a senior researcher at Intel. She studies the relationships that people form with technology and develops mobile and social applications to explore how technology can invite emotional self-awareness, behavioral change, and social connectedness.

Adrian Aguilera received his PhD in clinical psychology from the University of California, Los Angeles. He is an Assistant Professor in the School of Social Welfare at the University of California, Berkeley. His area of research focuses on using information technology to improve mental health services for underserved populations.