Pilot Study of Aurora, a Social, Mobile-Phone-Based Emotion Sharing and Recording System

Abstract

Background:

Emotion is a ubiquitous aspect of humanity that governs behavior in a number of ways and is linked inextricably with health. Pausing to evaluate one’s emotional state in the face of decisions and reflecting on past patterns of emotion have been shown to improve behaviors. Further, social expression of emotion has been shown to directly improve health outcomes. While the virtual reality research community does not ignore emotion on the whole, there does exist a need to explore what roles emotional awareness and emotion sharing can play in this domain.

Methods:

A mobile-phone-based social emotion recording and sharing system, Aurora, was developed to provide individuals with a means to pause and evaluate their emotional state, reflect on past emotions, share emotions with others, and participate in socially supportive activities with peers. A study was conducted with 65 subjects to evaluate Aurora as a tool to encourage emotional reflection and awareness as well as social sharing of emotion.

Results:

Users of Aurora reported an increased comfort in socially expressing emotion and were encouraged to share emotions, even with strangers. Subjects also reported liking reflecting on their emotional state and found it valuable. Subjects’ behavior also suggested that the system encouraged individuals to reach out to one another in acts of social support.

Conclusions:

The Aurora system offers a tool for encouraging emotional awareness, emotion sharing, and socially supportive behavior. Such a tool could be impactful in numerous health settings where emotion is considered to be an important indicator of or influence on outcome, such as for weight loss, alcohol cessation, or cancer sufferers.

Introduction

The link between emotion and health is inextricable. Emotion impacts clinical outcomes in a broad range of conditions and settings¹ and governs key health behaviors such as eating habits,² exercise,³ and substance abuse.⁴ Emotion is also a fundamental component of social support,⁵ another key contributor to health outcomes.^6–10 As such, dealing with emotion is a focal point in health-related research, going well beyond simply attempting to induce positive emotional states.¹ This article focuses on three such aspects of emotion and related constructs, namely, emotional awareness and reflection, emotion sharing, and social support.

First, emotional awareness can play a crucial role in improving health and health-related behavior. There is a highly automatic nature to everyday behavior, even termed “mindless,” as many decisions are executed then forgotten almost instantaneously. For example, the typical person believes they make between 20 and 30 food-related decisions a day when in fact they make between 200 and 300.¹¹ That emotion can govern such behavior, as has already been said, compounds the problem. As such, the ability to pause mid-behavior to step back and examine one’s actions rather than succumbing to inertia can lead to better overall health decisions.¹² Further, reflecting on one’s past emotional state and emotional behavior is a key means for learning more appropriate behaviors.¹³

An equally important aspect of the emotion–health linkage is the social sharing of emotion, i.e., individuals expressing their emotions to one another. Emotion sharing is important to day-to-day existence,¹⁴ but even more so when dealing with stress and anxiety that might be related to changing behaviors or dealing with health-related issues.¹⁵ Individuals expressing emotion through writing experience benefits in the areas of physical health, psychological well-being, physiological status, and general functioning.¹⁶ Expression of emotion has also been shown to lead to reductions in negative affect that are known to facilitate improvements in many areas of health and health-related behavior.^1,14,17

Finally, numerous studies have established the health benefits of social support. Social connectedness has been shown to be an important contributor to health.⁹ Conversely, isolation has been shown to be extremely deleterious both physically and emotionally.¹⁰ Social support appears to be particularly relevant for individuals facing difficult health challenges such as quitting smoking,⁶ diet and exercise related to weight loss and maintenance,⁷ and even surviving and coping with cancer.⁸

Each of these aspects of the emotion–health linkage is difficult or impossible for a person to leverage on their own. A wide array of technologies has shown promise in supporting these aspects to promote healthy behavior, and these technologies can be described in the context of the virtuality continuum, with real environments at one end and fully virtual environments at the opposite end.¹⁸ At one end of the spectrum, virtual reality tends to focus on reconstructing the physical world (reproduction fidelity), with attention to the value of interpersonal affective communication still in its infancy—although work by Riva and colleagues demonstrates the circular interaction in virtual reality between emotions and the extent of presence.¹⁹ Further work speaks to the effectiveness of virtual reality for evoking and experiencing affect. While virtual reality work in controlled settings can sway understandings and attitudes in, say, exercise behaviors,²⁰ this work toward the less immersive end of the virtuality continuum highlights the value of emotional and social dimensions in technological health support. Nothing suggests that these dimensions will not remain valuable and effective throughout the continuum.

In our day-to-day existence, portable technologies are an effective way of fully engaging with people.²¹ Mobile phones are increasingly ubiquitous and important aspects of modern existence that are always with their owners and rarely switched off.²² Because of this, mobile phones are excellent vehicles for behavior change. Further, the fact that mobile phones are inherently social makes them ideal vehicles for delivering and seeking support.²¹ All this contributes to the potential for mobile technology to help shape interpretation and action in a variety of health-related contexts.²³

This pilot study in healthy individuals presents Aurora, a mobile-phone-based emotion recording and sharing application. It demonstrates that Aurora leverages the ubiquitous and social nature of mobile phones to engage people in emotional awareness and reflection, emotion sharing, and social support. Assuming that Aurora engages at-risk individuals in the same manner, a more formal study is likely to demonstrate the improved clinical outcomes associated with these three target behaviors.

The Aurora System

Methods

Aurora is a mobile phone application developed by the research team at Cornell University to encourage and facilitate the sharing of emotion. At present, there are native versions of the application for the Android and iOS platforms, and a mobile-phone-optimized Web-based version. This application is currently available for research purposes only but will be publicly available via app stores and the Web and free to use for research purposes.

In Aurora, a user logs in to the system via a typical log-in screen, and the system immediately prompts her to update her emotional status, i.e., post to the system (Figure 1A). The process consists of tapping the empty photo box, selecting a single photo to represent how the user is feeling from the gallery displayed (Figure 1B) and optionally adding a textual note (Figure 1C). The gallery initially displays 72 photos from which to choose. Should one of the initial 72 photos not suffice in representing a user’s emotion, tapping a “more photos” button refreshes the view, providing 72 new photos. Tapping a photo selects it and returns the user to the update screen with their choice of image, where tapping the “update status” button submits the post.

Figure 1.

Aurora user interface: (A) the user begins at the update screen; (B) the photo-selection screen scrolls to display 72 images and a “more photos” button; (C) after selecting a photo, the user can type a note then click “update status” to post; (D) the group mood view depicts recent posts made by group members and basic instructions as well as messages such as “status updated”; and (E) clicking a photo in the group view displays more detail about the post.

The group home screen (Figure 1D) is an aggregate view of the 15 most recent posts made by members of the user’s group. This feature imitates the popular “news feed” and similar features of Twitter and Facebook. A user can also choose to view the most recent posts listed by group member or a retrospective view of all of the posts made by a given user, her/himself included. In each of these views, tapping on any photo displays a status window (Figure 1E) detailing who shared the emotion, when they shared it, and the note, if any, that accompanied the post. Aurora provides one-to-one private messaging: users can send private messages to any other user in their group directly from the status window of one of their posts (Figure 1E) or from a simple inbox; recipients of new messages are explicitly notified within Aurora.

The design of Aurora is based on work in affective computing,²⁴ which suggests that, because of the complexity of human emotion, computer systems should simply act as a medium through which to convey emotion.²⁵ Aurora uses photographs as this medium given that they can be emotionally charged and have a certain emotional legibility,²⁶ can be a means of expressing and exchanging emotion and have private meaning based on prior experience,²⁷ and can be used for sharing experiences²⁸ and norms.²⁹ It is hoped that this tension between emotional legibility and private meaning offers users a range of interpretive flexibility,³⁰ from images with more specific emotional content (such as an expressive human face) to something more ambiguous (a drop of water rippling in a glass). In interpretively flexible systems, meaning is negotiated between the user, designer, and the computational intelligence of the system itself,³¹ leading to a better overall experience for the user.

In selecting photos to use with Aurora, Lang’s International Affective Picture System, an archive of photos representing a validated instrument for evoking a variety of emotional responses,²⁶ was considered. However, it was neither purposed for nor examined from the standpoint of subjects selecting images that they feel represent their current emotional state. Therefore, a custom set was developed using the online photo-sharing service Flickr. The Flickr user community has tagged many of the photos on the site with words of affect. Russell’s Circumplex Model of Affect³² served as the basis in selecting a set of 36 affective tags. For each tag, as many as 250 Creative Commons-licensed images were downloaded (taking advantage of recent photos sorted by Flickr’s internal “interestingness metric” in an attempt to locate photos that might best give voice to users’ emotions); this resulted in approximately 9000 photos. After removing images with obviously inappropriate or explicit content, 7714 photos remained for use in the system.

Procedure

For this study, approved by the Cornell University Institutional Review Board for Human Participants Research, 65 subjects used Aurora for a period of 7 days. Subjects were placed in groups ranging in size from 10 to 17 subjects: one group was strangers aged 18–55 years; one group consisted of members of a research group including undergraduate students, graduate students, and faculty; two groups consisted of college classmates; and a final group consisted of high school students from a single group at a university summer school program (see Table 1). The group of strangers was the initial test group gathered from colleagues and friends of friends of the research team who were unlikely to know one another. The college student groups were recruited through classes (not taught by members of the research team), the research team was assembled through the lab director, and the high school students were approached through their summer school instructor. Subjects were brought to the research laboratory where they were consented, given a pre-study questionnaire, and taught to use the Aurora system. Subjects were asked to use Aurora whenever they liked over the course of the following week. Users returned to the lab for a post-study questionnaire one week later. Three undergraduate research assistants were trained to code posts for the presence of emotional content.

Table 1.

Participant Characteristics by Group

Group	N	Number of males	Age range (years)
Strangers	10	5	18–55
Research group	17	7	18–55
College classmates 1	11	5	18–22
College classmates 2	12	5	18–22
High school classmates	15	10	16–18

When assessing users’ experience with Aurora, well-understood problems with simply using postexperiment questionnaires and interviews became apparent, namely, that subjects often incorrectly report on past events due to poor memory or even their current state during post-test.^33,34 Aurora users’ experience was therefore assessed via ecological momentary assessment, a data-collection method for assessing subjects in context at opportune moments.³³ Immediately after posting, a user could be randomly prompted to take an in-application survey with questions pertaining directly to that post. The questions were short (e.g., Where are you? What are you doing?) and were answered with dropdown menu choices before returning users to the system.

Results

In all, subjects made 507 posts to the system—the median number of posts per user was five, or just fewer than once per day. The most common subjects of photos selected were people (38%), animals (23%), and nature and nature scenes (12%) (see Figure 2 for sample posts). In spite of the broad age range (18–55 years), there was no discernable difference in usage patterns based on age. This is likely linked to the fact that the subjects uniformly reported being frequent users of mobile apps, the mobile Web, and social networking services such as Facebook. After randomly selected posts, subjects chose from lists regarding where they were and what they were doing; Table 2 depicts the responses to each of these questions. Overall, subjects enjoyed using Aurora—all but one subject reported enjoying using Aurora, and 79% of subjects stated that they would use Aurora again in the future.

Figure 2.

Sample Aurora posts.

Table 2.

Top Four Responses to Questions in the In-System Survey Administered after Random Posts

Where are you?
At home	45%
At a restaurant, dining hall, etc.	19%
In class, lab, studio, etc.	15%
At the library	11%
What are you doing?
Studying	19%
About to fall asleep	17%
Eating	13%
In transit	11%

Emotional Awareness

In response to the random assessments following posts, 81% of the responses indicated that subjects had thought about their emotional state in the process of posting. When later asked if using Aurora made them think more about their emotions than they normally would, 86% percent responded that they did in fact pay more attention to their emotions while using Aurora. Further, 36% even anticipated thinking more about their emotions in the future after having used Aurora.

Analysis of the qualitative data suggested that emotional awareness was an important aspect of subjects’ experience with Aurora as well. For example, “I enjoyed the challenge of having to take a step back and figure out what my current mood was and then having to break it down even further into just a photo. It was a fun exercise.” Female, 18–25.

“I never had the patience to keep a journal, but this program made me analyze my mood from time to time.” Female, 15–17.

Emotion Sharing

Prior to the study, 78% of subjects reported that they shared emotions with members of their group only occasionally, while the remainder reported that they had never shared emotions with the group. No respondent indicated that they regularly shared emotions with their group before using Aurora. During the study, however, a substantial amount of emotion sharing occurring within the system (see Figures 2B, 2C, 2D for posts with emotional content). Coding found that 71% of the posts contained obvious emotional content (coder agreement was 88%).

The post-study questionnaire asked subjects if they were more comfortable sharing emotions in Aurora than in other ways such as face-to-face or over the phone, and 71% stated that they were somewhat more comfortable, 14% said they were much more comfortable, and only 14% said they were no more comfortable. Further, every respondent indicated that they were either very comfortable or somewhat comfortable sharing emotions with their group in Aurora (50% each) while none reported feeling uncomfortable.

Finally, using Aurora influenced the stated emotion-sharing behaviors of some subjects even outside the system; 65% indicated that they were either somewhat more or much more comfortable sharing emotions with their group because they had used Aurora. Further, 86% of respondents stated that, after they stopped using Aurora, they would share emotions with their group more frequently than they did prior to the study.

Social Support

During the study, 35 private messages discussing 26 different posts were sent by 14 users and received by 18 users. A total of 20 users (31%) experienced explicit social support within Aurora. The content of these messages included queries for more details (“So who you loved by?” or “Why do you think it stinks?”) and reflections on posts (“It’s not that bad lucky you arnt at home all depressed”). The content of messages was somewhat lightweight from an emotional support perspective, but then again, the subjects in our system were not under any known particular duress. Furthermore, interviews suggest that many colocated Aurora users discussed posts with each other outside the application in a supportive matter.

Discussion

This study finds Aurora to increase emotional awareness, encourage emotion sharing, and encourage socially supportive behaviors (see Table 3 for survey results). The health implications of these findings, while not implicit, are clear, as each of these three emotional behaviors have been linked to significant positive outcomes in health in a range of domains, including nutrition, exercise, and weight loss;^2,3,7 substance abuse and cessation;^4,6 and cancer.⁸

Table 3.

Summary of Survey Results

Survey item	Percent responding
Emotional Awareness
Thought about emotional state while posting	81%
Anticipate more future self-reflection on emotions having used Aurora	36%
Paid more attention to emotions while using Aurora	86%
Emotion Sharing
Before Aurora, shared emotions with a group member only occasionally	78%
Posts in Aurora containing obvious emotional content	71%
Very comfortable sharing emotions in Aurora	50%
Somewhat comfortable sharing emotions in Aurora	50%
Somewhat or much more comfortable sharing emotions using Aurora than in other ways	76%
After using Aurora, somewhat or much more comfortable sharing emotions	65%
After using Aurora, will share emotions more frequently than prior to Aurora	86%
Social Support
Experienced explicit social support in Aurora	31%

Perhaps the most interesting aspect of these findings is the fact that Aurora is able to widely encourage emotional contemplation and sharing, even among strangers. There is evidence of a possible cause—users feel more comfortable sharing emotions in Aurora than they do through other mediums such as face-to-face. Several factors may contribute to this. First, Aurora is a system in which emotion sharing is the primary function. This may remove much, or all, of the negative stigma associated with sharing serious emotions in other online environ-ments (e.g., Facebook). A user of Aurora would likely maintain a different group of contacts in Aurora than they would in other domains, perhaps even a group of people with whom they would feel more comfortable sharing emotions, particularly in contexts such as disease or losing weight. Another factor could be a product of the medium; self-disclosure online has been found to occur more frequently than in face-to-face encounters.³⁵ Also, the mobile aspect provided people with an opportunity to share what might not have otherwise existed, evidenced by one subject who stated, “I enjoyed using this program because it made me feel like I had some place to express myself and share my feelings even if that wasn’t the case in real life at that moment.”

It is also interesting to consider the role of emotional contagion³⁶ in a social system such as Aurora. Given that emotion spreads through social networks³⁷ and has been demonstrated to do so in online contexts,³⁸ the possibility exists that negative or positive emotions experienced by one or two individuals could significantly impact the entire group. On one hand, this suggests an opportunity for a situation where improvements in health and emotion contribute to a ratcheting effect of positivity, possibly even bolstered by positive affect interventions;¹ on the other, it warns of a scenario that should be prepared for in which individuals spiral downward together.

There are several limitations to this work. First, further controlled experimentation is warranted to verify the findings presented here. Second, the study was conducted in healthy populations, not clinical target populations. While there is certainly value in encouraging healthy behavior in the general population, ultimately, such applications must be tested in unhealthy or at-risk populations. The most obvious limitation of this work is that it has not reported on actual health or health behavior outcomes. Yet this work does provide strong evidence to warrant future research along these lines, such as examining the use of Aurora specifically for a variety of target behaviors such as those mentioned earlier. In fact, Aurora is currently deployed in a study involving cancer patients and anxiety and in another study examining emotional eating.

Finally, Aurora could be tailored for use in specific contexts. For example, people in a dieting group might be asked to log in and post whenever they felt like lapsing or even be intelligently interrupted by the system at likely moments of weakness. In such a scenario, the pause to think about one’s emotional state at that moment might be enough to help them get past the urge. Further, they could immediately receive messages of encouragement from other members of their group to persuade them to stay on track. Such examples would capitalize on all key aspects of the system as a means for encouraging more constructive emotional behavior.

Conclusions

This article has presented Aurora, a mobile-phone-based emotion recording and sharing application. Evaluation has found that the use of Aurora encourages people to be more aware of their emotions, more comfortable, and more likely to share emotion and leads them to engage in socially supportive behavior. This represents a significant finding, given that countless studies have found that these three elements of emotion have profound impact on many aspects of health and health-related behavior. This research should pave the way for future studies that, on a grander scale, may find links between use of mobile emotion sharing systems and better outcomes for health. Further, researchers working on technologies across the virtuality continuum should take note of these findings and consider how similar constructs could be ported into their domain.