Bringing an Ecological Perspective to the Study of Aging and Recognition of Emotional Facial Expressions: Past, Current, and Future Methods

Abstract

Older adults perform worse on traditional tests of emotion recognition accuracy than do young adults. In this paper, we review descriptive research to date on age differences in emotion recognition from facial expressions, as well as the primary theoretical frameworks that have been offered to explain these patterns. We propose that this is an area of inquiry that would benefit from an ecological approach in which contextual elements are more explicitly considered and reflected in experimental methods. Use of dynamic displays and examination of specific cues to accuracy, for example, may reveal more nuanced age-related patterns and may suggest heretofore unexplored underlying mechanisms.

Older adults do not perform as well as young adults on traditional emotion recognition tasks. When presented with an image of a static face posing a particular emotional expression, older adults are less likely than young to name the emotion expected from the task. In this paper, we attempt to deconstruct this apparently straight-forward finding, by considering the methods and conceptual frameworks that have been used in work on aging and emotion recognition to date. Our goal is to argue that current conclusions about the emotion recognition accuracy of older adults may be constrained by the methods used thus far to test performance. A more ecological approach to examining age differences in emotion recognition accuracy could potentially yield different, or at least more complex, conclusions about the effects of age on these abilities.

Nonverbal behavior researchers have developed a number of methods for assessing accurate decoding of facial expressions of emotion (see Hall, Bernieri, & Carney, 2005, for a summary). Using similar methods, the existence of age-related decrements in emotion recognition accuracy for facial expressions has been documented in a number of individual studies (e.g., Calder et al., 2003; Isaacowitz et al., 2007; Keightley, Winocur, Burianova, Hongwanishkul, & Grady, 2006; Malatesta, Izard, Culver, & Nicolich, 1987; McDowell, Harrison, & Demaree, 1994; Mill, Allik, Realo, & Valk, 2009; Moreno, Borod, Welkowitz, & Alpert, 1993; N. A. Murphy & Isaacowitz, 2010; Orgeta, 2010; Orgeta & Phillips, 2008; Oscar-Berman, Hancock, Mildworf, Hutner, & Weber, 1990; Phillips, MacLean, & Allen, 2002; Ruffman, Halberstadt, & Murray, 2009a; Sullivan & Ruffman, 2004a; Sullivan, Ruffman, & Hutton, 2007; A. Suzuki, Hoshino, Shigemasu, & Kawamura, 2007; Wong, Cronin-Golomb, & Neargarder, 2005), as well as in a recent meta-analysis of the literature (Ruffman, Henry, Livingstone, & Phillips, 2008). There appear to be fairly ubiquitous negative age effects on performance across emotions, with some suggestion that certain discrete emotions show larger age differences than others and that negative emotions overall show larger age deficits than positive ones. Specifically, older adults are typically worse than young adults at correctly recognizing fearful and sad faces (Calder et al., 2003; Keightley et al., 2006; Orgeta & Phillips, 2008; Ruffman et al., 2008; Wong et al., 2005), and sometimes also angry faces (Calder et al., 2003; Orgeta & Phillips, 2008; Phillips et al., 2002; Ruffman et al., 2008; Sullivan & Ruffman, 2004a; Wong et al., 2005), and neutral faces (McDowell et al., 1994), but are often equal to (Orgeta & Phillips, 2008; Phillips et al., 2002) or better than young adults at recognizing disgusted faces (Calder et al., 2003; A. Suzuki et al., 2007; Wong et al., 2005). Smaller age deficits (Ruffman et al., 2008), age equivalence (McDowell et al., 1994; N. A. Murphy & Isaacowitz, 2010; Orgeta & Phillips, 2008; Phillips et al., 2002; Sullivan & Ruffman, 2004a), or even an advantage for older adults (Moreno et al., 1993; N. A. Murphy, Lehrfeld, & Isaacowitz, 2010), has been found for happy and surprise recognition.

Beyond facial expressions, older adults have also been found to be less accurate at identifying the intended emotion in lexical stimuli (Grunwald et al., 1999; Isaacowitz et al., 2007), emotional prosody of voices (Brosgole & Weisman, 1995; Paulmann, Pell, & Kotz, 2008; Ryan, Murray, & Ruffman, 2010), and bodily expressions (Ruffman et al., 2009a; Ruffman, Sullivan, & Dittrich, 2009b). The discrete emotions older adults have trouble identifying varies by modality. For example, in one study no age differences in recognizing fear from lexical stimuli was found, but older adults were worse than their younger counterparts at recognizing fearful faces (Isaacowitz et al., 2007). The lion’s share of research in this area has focused on facial expressions of emotion, and is the focus of this paper. A facial expression is a gesture made with facial muscles. Below, we review the primary methods that have been used to examine age differences in facial expression emotion recognition accuracy, along with the conceptual frameworks that accompany each method (whether explicitly or implicitly). In most cases, samples of younger and older individuals are compared and age differences in performance are examined; at the end of the paper we critically consider this approach.

Descriptive Methods

The first large category of methods can be broadly classified as “Descriptive.” Consistent with basic strategies used in nonverbal emotion perception research (Knapp, 1972; Hall et al., 2005), descriptive methods simply assess the emotion recognition performance of younger and older adults to some set of target emotional stimuli, and then try to identify plausible causes of the observed pattern of age effects. So, when studies find older adults worse at emotion recognition overall, they posit an underlying explanatory mechanism for those findings. The best example of this can be found in Ruffman et al.’s (2008) meta-analysis: first the meta-analytic results are reported, then the pattern of results is considered to support a “neuropsychological” perspective in which underlying age-related changed in neural processes could dictate behavioral changes in emotion recognition ability: “In sum, structural changes in gray or white matter as well as changes in neurotransmitters might be related to older adults’ recognition difficulties” (p. 874).

The brain-based explanation is that age-related differences in emotion recognition are tied to structural and functional changes in the neural substrates that are important for recognizing emotions. For example, the amygdala has been linked to the recognition of fear (Sato, Kochiyama, Yoshikawa, Naito, & Matsumura, 2004) and sadness to some extent (Adolphs & Tranel, 2004). The orbitofrontal cortex appears to be important for the recognition of anger (F. C. Murphy, Nimmo-Smith, & Lawrence, 2003). It is unclear, however, to what extent these regions are affected by normal aging. Some studies report relative preservation of the amygdala and the orbitofrontal cortex with age (e.g., Good et al., 2001; Grieve, Clark, Williams, Peduto, & Gordon, 2005; Salat, Kaye, & Janowsky, 2001) while others indicate that there is an age-related decrease in the volume or activation of these regions (e.g., Cerf-Ducastel & Murphy, 2003; Jack et al., 1997; Y. Suzuki et al., 2001). Some research has interpreted these age differences in amygdala activation as evidence for an age-related shift in the response of the amygdala based on the type of emotional stimuli presented (Gunning-Dixon et al., 2003; Mather et al., 2004). Interestingly, behavioral findings indicating age-related maintenance of disgust recognition are supported by the preservation with age of the insula, which appears to be important for disgust recognition (Calder et al., 2003).

In contrast to behavioral studies in which participants are shown a face and asked to identify the emotion expressed on it, much of the relevant neuroimaging research has examined neural responses when participants passively view (rather than attempt to identify) facial expressions of emotion. There is also some evidence that there are distinct neural components that are responsive to specific emotions, irrespective of the modality of emotional information. For example, the hippocampus is activated for anger and fear of both faces and pictures from the International Affective Picture System (IAPS) and the ventromedial prefrontal cortex is activated for anger faces and IAPS pictures (Britton, Taylor, Sudheimer, & Liberzon, 2006). Despite some promising connections between behavioral and brain data in the context of aging and emotion recognition, recent neuropsychological evidence cautions against over-interpreting these connections between studies identifying specific brain regions for recognizing discrete emotions and age-related changes in these brain regions because we do not know to what extent the aging brains studied were atypical (atypical cognitive decline or preclinical dementia) rather than representative of normal aging (Burgmans et al., 2009).

A second descriptive approach focuses not on the neuropsychological abilities of the older adult but instead on the motivation they bring to the task. Socioemotional selectivity theory (Carstensen, Isaacowitz, & Charles, 1999) argues that time limitations lead individuals to prioritize goals related to regulating and optimizing emotional states. While increasing age is not the only life context associated with limited time perspective, older adults do appear to have a more limited view of the future and to prioritize emotional goals (Carstensen, 2006). Recently, researchers have wondered whether older adults’ prioritization of emotional goals might be revealed in how they process information; this has led to some findings of what have been termed “age-related positivity effects” (e.g., Carstensen & Mikels, 2005) such that older adults show preferential processing of positive relative to negative information. This is in contrast to the standard negative preference shown by young adults in many tasks (e.g., Rozin & Royzman, 2001).

Though not every study has found support for systematic differences between young and older adults in emotional processing (e.g., Murphy & Isaacowitz, 2008), age-related positivity effects have been identified primarily in attention to, and recall and recognition memory for, emotional images. Given that this could have implications for emotion recognition as well, several studies have attempted to discern whether age differences in emotion recognition accuracy could also reflect positivity effects. For example, Williams et al. (2006) investigated age differences in an emotion recognition task along with the underlying neural mechanisms. A cross-sectional sample spanning seven decades completed a forced-choice emotion recognition task to identify the emotion of happy, fearful, and neutral facial expressions. Younger age groups were more accurate than older age groups at recognizing fear, while the youngest age group (12-19 years) was less accurate than the older age groups at recognizing happiness. Functional neuroimaging data showed a decrease in the medial prefrontal cortex in response to happy faces and an increase in response to fearful faces, suggesting greater controlled processing for negative emotions and reductions in controlled processing for positive emotions.

However, findings from other studies suggest limits to the utility of the motivational account to fully describe patterns of age differences in emotion recognition accuracy. The meta-analysis described above did indeed find age-related deficits in recognition of positive as well as negative emotional expressions, though the effects were smaller for positive expressions (Ruffman et al., 2008). We have noted elsewhere (Isaacowitz et al., 2007) that many tasks assessing recognition accuracy for positive emotions are constrained by ceiling effects (it is relatively easy to correctly identify a happy face when that is the only positive response option). It is possible that one explanation for the apparent findings from the Williams et al. (2006) study above suggesting age changes in controlled processing of positive stimuli has to do with the relative ease of the task. When the recognition task is made more difficult, age effects emerge even for positive emotions (Isaacowitz et al., 2007; Orgeta, 2010). This suggests that positivity effects match only some, but not all, of the descriptive findings on age differences in emotion recognition accuracy.

While such descriptive approaches are logical early steps in trying to match patterns of findings and possible underlying mechanisms, their limitations are also readily apparent: in most studies, no actual mechanisms are specifically tested. So for example, studies that frame their patterns of findings in terms of structural brain changes and/or changes in neurotransmitters do not actually assess anything about the brain or neurotransmitters or directly relate accuracy changes to such processes. Studies positing motivational mechanisms similarly do not assess anything about motivation directly. Thus, the descriptive approach primarily involves collecting of accuracy data and thought experiments as to what mechanisms might plausibly yield such patterns.

Age Differences in Mechanisms

Not surprisingly, then, other research has tried to more directly assess potential causal mechanisms: we refer to this as the “Mechanism-Based” approach. In this approach, researchers will still evaluate age differences on an emotion recognition accuracy task, but will also assess age differences on some conceptually interesting additional task(s) that might relate to accuracy, either within the same task and/or with the same participants.

The first plausible mechanism that has been suggested and tested in this type of design is general age-related cognitive decline. In this line of reasoning, age-related declines in emotion recognition accuracy could just be one specific behavioral manifestation of well-known general decrements in perceptual (Schneider & Pichora-Fuller, 2000) and cognitive functioning that happen with age. For example, it is well-documented that older adults exhibit slower reaction times, reduced working memory capacity, declines in executive functioning, and increased difficulty in inhibiting irrelevant information (Bäckman, Small, & Wahlin, 2001; Birren & Fisher, 1995; Salthouse, 1996; West, 1996; Zacks, Hasher, & Li, 2000). It is logical to explore the possibility that age effects on emotion recognition accuracy could just be reduced to these more general age-related changes.

However, findings from studies that have pursued this explanation have generally not found that age-related perceptual and cognitive change can fully account for age differences in emotion recognition accuracy (Keightley et al., 2006; Sullivan & Ruffman, 2004b). For example, age-related decrements in recognizing sadness and fear were found after controlling for individual differences in visual perception of faces, face processing, and fluid intelligence (Sullivan & Ruffman, 2004b). On the other hand, one study found that the overall main effect of age on emotion recognition was no longer significant once memory was entered as a covariate (MacPherson, Phillips, & Delia Sala, 2002). This study, however, also found an age by emotion interaction such that age differences were only significant for identifying sad faces. While the findings suggest that the overall main effect of age in recognizing emotions was accounted for by age differences in general memory performance, age differences in sad recognition may have been much stronger than overall age differences (which included happy, disgust, and surprise recognition, emotions that typically show little or even opposite age effects), and it was this more robust negative age effect (and thus the primary effect of interest) that could not be explained by the general memory covariate.

The second plausible mechanism targeted in this literature has been faulty gaze patterns in older adults leading to worse emotional recognition accuracy. While older adults have been found to show different gaze patterns to emotional faces than young adults in passive viewing tasks (e.g., Isaacowitz, Wadlinger, Goren, & Wilson, 2006), the studies we discuss below address to what extent older adults look at emotional faces in ways that make it hard for them to be accurate when specifically asked to recognize the emotion expressed on the face. These studies use observational eye tracking, recording where young and older adults’ gaze fixates when either viewing faces or identifying emotional facial expressions. To the extent that gaze patterns lead to inaccuracy they may be considered “faulty”, though this not meant to imply any age-related change in the functioning of gaze patterns; in fact, research has shown age maintenance in fixational control (Kosnik, Kline, Fikre, & Sekuler, 1987). The idea that gaze patterns could lead to age-related inaccuracies in emotion recognition has been bolstered by research on emotion recognition among young individuals with autism; they have been found to show deficits in emotion recognition that can be linked to faulty gaze preferences, and may be improved by forced shifting of gaze onto more salient facial features (Neumann, Spezio, Piven, & Adolphs, 2006; Spezio, Adolphs, Hurley, & Piven, 2007). Specifically, individuals with autism seem to avert gaze from the eyes, and make errors in identifying emotions like fear which involve substantial expression around the eyes. Directing gaze to the eyes seemed to improve fear recognition at least in one patient with amygdala damage (Adolphs et al, 2005).

Two studies provided some support for the idea that older adults might show different gaze patterns toward emotional faces in the context of an emotion recognition task, and that these differences might lead to accuracy differences. In one study (Wong et al., 2005), researchers found a correlation between accurate recognition and fixation to the top half of faces, leading them to suggest that older adults might make errors by focusing too little on the top of faces and too much on the bottom half. In a similar vein, another study found that more fixation to the mouth region of faces correlated with lower accuracy in emotion recognition specifically for older adults attempting recognize negative emotional faces (Sullivan et al., 2007). Together, these studies suggest that age differences in fixation to faces parallel age differences in accuracy and might therefore be a mechanism underlying age-related deficits: if (some) emotions require attention to the eye region of faces, and older adults look less at the eyes, that could lead them to some inaccurate responses. Various possibilities exist as to why older adults may look away from the eyes, from overreliance on the mouth as a cue due to lipreading, to beliefs about the accuracy of eye cues; these remain to be tested in future research. However, it has been shown that the top of the face is important for identifying anger, fear, and sadness, while disgust and happiness are best identified from the lower half of the face (Calder, Young, Keane, & Dean, 2000). And it is the recognition of anger, fear, and sadness, the “eye emotions”, are the emotions which show the largest age differences (Ruffman et al., 2008). This pattern could be interpreted as suggesting that older adults display attunements in their gaze patterns to facial cues that are most informative about happiness, and less so concerning most negative emotions.

One limitation of those two studies is that neither examined whether age differences in gaze could account for age differences in emotion recognition accuracy; instead they simply correlated fixation with performance. It might be that a correlation between these two constructs results from a third variable altogether, such as motivation or anxiety. Or, it could be that fixation differences account for part of, or all of, the age differences. To try to distinguish among these various possibilities, Murphy and Isaacowitz (2010) measured eye gaze during a recognition accuracy task and directly tested whether gaze patterns could statistically account for age differences in accuracy. While the usual age differences were found in accuracy, and some age differences were found in fixation to the faces (with younger adults looking at the eyes more than older adults), analyses trying to link fixation and accuracy told a different story: For anger and sadness, controlling for a whole host of predictors, critically including a number of different measures of fixation to those face types, failed to eliminate the significant age difference in accuracy. In other words, any differences between young and old in patterns of looking at the faces could not fully account for differences between them in recognition accuracy.

The most promising results from the Mechanism-Based approach may arise from a study linking the experience of negative emotions with recognition specifically of sadness. One study statistically eliminated age differences in sadness recognition by controlling for measures like reported anxiety and depression (A. Suzuki et al., 2007). These results are suggestive, but have not yet been replicated nor generalized to age differences in recognition of other emotions beyond sadness. Because depression is also often an indication of poor physical health (Holahan et al., 2010), future work should also investigate whether there is a relationship between health and emotion recognition.

Recently, links between emotion mimicry and emotion recognition have led researchers to investigate bodily responses of the perceiver when attempting to recognize emotions. Empirical work on embodied cognition suggests that perception of an emotional expression involves simulation of that emotion (Halberstadt, Winkielman, Niedenthal, & Dalle, 2009; Niedenthal, Winkielman, Mondillon, & Vermeulen, 2009). In one study, electromyography during anger recognition revealed that older adults’ corrugator responses to angry expressions was related to their reduced anger recognition (Bailey, Henry, & Nangle, 2009). It has yet to be determined whether corrugator response differences among older adults are the cause or the result of reduced anger recognition and whether these findings generalize to the other facial expressions older adults have difficulty identifying (e.g., fear and sadness).

The “Mechanism-Based” approach has generated a number of interesting studies and plausible mechanisms, and some individual studies have accounted for emotion-specific age differences, but no study has identified one or more mechanism(s) that could statistically account for overall age deficits in emotion recognition accuracy. Even using a laundry list of possible predictors - ranging from ability to recognize gender from faces to general indicators of fluid and crystallized intelligence (Keightley et al., 2006), to the mood of the perceiver doing the recognition task (A. Suzuki et al., 2007) - has not satisfactorily accounted for age differences in performance. Despite the plethora of studies on the topic of emotion recognition and aging, we do not have a clear picture of the nature of age effects and what could underlie such effects. This suggests that a different approach is needed for the future.

Integrating Context into the Study of Aging and Emotion Recognition: An Ecological Approach

Where can one look for help in increasing the ability to directly account for age differences in emotion recognition accuracy? In the remainder of this paper, we make an argument for integrating context, loosely defined, into the study of emotion recognition and aging as a way of increasing ecological validity and thereby improving our understanding of age differences (and similarities) in how emotions are identified from facial expressions. Context has been a key construct in aging research, though much of the research on context effects in cognitive aging come from studies of intelligence (Colonia-Willner, 1998) and memory performance (Hess, 2005). For example, age differences in recall for paired associates are eliminated when the pairs of words are related, suggesting that older adults are able to use the passive environmental support in the stimuli to boost their recall performance (Smith, Park, Earles, Shaw, & Whiting, 1998). One possible explanation for this finding is that older adults may develop strategies that rely on cues in the environment to compensate for age-related losses in cognitive and sensorimotor abilities. Context effects have also reduced age differences in social information processing. In an impression formation task, young adults were better at recalling inconsistent information about the target than older adults. However, when the task was placed in a more meaningful context by asking participants to evaluate the target person’s suitability for a job, the age differences in person memory were attenuated (Hess, Follett, & McGee, 1998). Thus, placing a task in a meaningful context can both reduce age differences in performance and also illuminate mediating and moderating factors for such age differences.

The study of emotion recognition and aging to date, we believe, has been largely devoid of context. We consider context to include both the ways in which older adults do emotion recognition in their everyday lives (e.g., who they do it with, when they do it), as well as the context that may be available to them in order to clarify emotional expressions in particular situations (such as situational cues, body movements, etc.). Ignoring such context has constrained researchers’ ability to understand the effects of aging on emotion recognition ability. One possibility is that the perceptual ability to recognize facial expressions of emotion from still photographs of strangers does decline with age but that in daily life older adults are able to effectively compensate for this perceptual decline by factoring in cues from other channels, such as vocal, historical, or situational cues. Integrating these types of context has the potential to lead researchers toward methods that better describe and ultimately identify mechanisms concerning age-related differences in emotion recognition. Consider the case of distinguishing between a fearful and a surprised face, two emotions that are often confused with each other (e.g., Isaacowitz et al., 2007), likely because they both display a widening of the eyes (Ekman, 2003). Perhaps the pattern recognition abilities needed to distinguish between fearful and surprised faces declines with age, and so older adults rely more on the situational cues, such as the context of a surprise party vs. a burglar in the home, than young adults to determine fear or surprise. It may be that integrating these contextual cues takes more effort than simply “reading out” a facial expression, thus older adults may not always be motivated to exert this effort.

One direction is suggested by considering a critical limitation of the research described above: namely that all the studies, as well as the conceptual frameworks used to generate the methods and to interpret the findings, focus on the perceiver who is attempting to decode an emotional expression external to themselves. This approach is a logical first step, but it misses other key players in the behavioral transaction: the target itself, and the context in which the perceiver is trying to make a judgment about the target, which may carry with it certain affordances, or opportunities for action and interaction (Zebrowitz & Montepare, 2006). An ecological approach turns our attention toward methods that could a) identify tasks where older adults are not quite so impaired in their accuracy and b) provide a wider array of plausible mechanisms to explain cases in which older adults are genuinely impaired. In addition to raising the need to consider the perceiver, the target, and how they are contextualized, an ecological approach also raises a key functional question: do older adults show functional problems in their everyday life consistent with observed lab performance?

Right away we face a serious caveat from the ecological approach: namely, that older adults do not appear to show serious interpersonal deficits. All available evidence points to the conclusion that older adults tend on average to have good, satisfying social relationships (e.g., Birditt, Fingerman, & Almeida, 2005; Carstensen, Graff, Levenson, & Gottman, 1996; Lansford, Sherman, & Antonucci, 1998; Levenson, Carstensen, & Gottman, 1994). Given that accurate recognition of emotions in others is important for successful relationships and social adjustment (Carton, Kessler, & Pape, 1999; Engelberg & Sjöberg, 2004), it is difficult to reconcile older adults’ less accurate emotion recognition abilities with their interpersonal relationship success and savvy. How can older adults be impaired at recognizing emotions yet still have strong, satisfying relationships? Past findings could be overestimates of the age differences in accuracy, and/or the magnitude of the differences may have been overstated. For example, in Isaacowitz et al. (2007, p.156) we conclude: “on average, even older adults correctly classified 82% of the stimuli (compared with 88% among young adults).”

The logical implication of this is that some component(s) of the studies to date that suggest age-related deficits do not translate into significant functional problems for older adults. Why might this be the case? An important possibility is that emotion recognition tasks to date have not tapped into how older adults actually do their (usually successful) emotion recognition. For example, do they use cues from channels other than facial expression? Do they use some cues from faces that are not available in static faces? Is their performance somehow boosted because they generally recognize expressions in familiar social partners rather than strangers? While these are conceptual points broadly concerning how older adults might use the larger “context” of emotional expression to accurately decode the feelings of others, they nonetheless have methodological implications for how studies should be done to elucidate exactly what age deficits exist, and why.

Therefore, we next turn to review the small body of literature that suggests components of emotional expressions that older adults may use to successfully identify these expressions in real-world emotional transactions, but that are not available for them to use in most traditional emotion recognition tasks. These findings may hold some clues as to why older adults do poorly on these tasks but not in everyday life, and suggest aspects of the transaction between perceiver and target that are critical for accurate real-world identification.

Dynamic Displays

One obvious methodological limitation of most research in this area is the use of still photographs as targets. Clearly, most real-world emotion recognition does not involve looking at pictures and deciding what the person in the picture is feeling; instead, most recognition happens in an ambulatory environment in which both parties (the perceiver and the target) are in motion. Not surprisingly, then, research that uses dynamic emotional stimuli rather than static stimuli finds more accurate performance (for all emotions except happiness, though that may of course be subject to ceiling effects in performance), at least among younger adults (Ambadar, Schooler, & Cohn, 2005; Bould & Morris, 2008; Lederman et al., 2007). Motivation may also be a factor when interpreting static facial expressions versus dynamic facial expressions, especially for older adults, because the task of identifying an emotion from a static photo may seem so artificial and far-removed from daily life that older adults may not be sufficiently motivated to perform well.

Another truism is that older adults have had more of a lifetime of experience recognizing emotions in others. However, this perhaps mundane observation may have important methodological implications: The dynamic nature of the expressions is something that older adults may have experience with, more so than static images, because that is what they observe in the stream of their everyday experience. Thus, tasks using dynamic stimuli may tap more into this accumulated experience. This would then raise questions about what aspects of the unfolding expressions in time are used for accurate identification.

From still photographs, happy and surprised faces can be accurately recognized more quickly than disgust, fear, anger, or sadness (Calvo & Nummenmaa, 2009). In terms of the limited data on dynamic facial expressions of emotion, there do seem to be emotion-specific velocities which are perceived as most natural, with surprise and fear as relatively fast emotional expressions and disgust and sadness perceived more optimally as slow displays (Hoffmann, Traue, Bachmayr, & Kessler, 2006). However, while dynamic and first-last displays (a display with a neutral image shown for 500 ms followed by and image of the target facial expression) afford better recognition than static displays, they were not different from each other. This may suggest that a sensitivity to change in general, rather than emotion-specific temporal cues, is responsible for improved accuracy in dynamic versus static displays (Ambadar et al., 2005), at least among young adults.

Another limitation of the current research is that it typically uses photos of strangers as targets. However, research with young adults suggests that people are better at recognizing the emotional expressions of a friend than a stranger (Sternglanz & DePaulo, 2004), perhaps because they are more motivated to interpret the emotional expressions of their friends than strangers (Fingerman, Miller, & Charles, 2008; Thibault, Bourgeois, & Hess, 2006). Perhaps older adults are adept at recognizing the facial expressions of their familiar social partners, which allows them to be successful in interpersonal relationships. This hypothesis remains to be tested.

What other aspects of emotion expression might also benefit from experience but are not featured in traditional emotion recognition tasks? Another has to do with the nature of the expressions themselves. In the Ekman picture set, and other static emotional picture sets, the faces show prototypic emotional expressions (some might say caricatured; e.g., Barrett & Kensinger, 2010). Real-world expressions may be more ambiguous (Rozin & Cohen, 2003) and may tend to be evaluated not in terms of big categories like “what emotion is X expressing?” but rather may frequently involve more subtle distinctions such as “is X smiling because she is actually happy?” or “are those tears of joy or sadness?” In other words, real-world emotion recognition may involve subtle distinctions rather than prototypical differences, and these distinctions may be what older individuals have accumulated experience doing.

One series of studies has considered some of these two factors and examined whether older adults would show better, worse, or equal performance as their younger counterparts at identifying expressions that are dynamic and represent subtle distinctions concerning target expressions. A neuropsychological explanation would likely posit that older adults should do worse, while an experience explanation would posit that older adults would do better than young adults. And because a positive emotion was studied, socioemotional selectivity theory would not expect age-related decline in accuracy.

In these studies (N. A. Murphy et al., 2010), younger and older adults were presented with video stimuli of targets going from a neutral expression to a smile. These target videos were culled from longer videos; smiles had been extracted by raters and facial criteria were used to assess which smiles were genuine (in other words, expressed actual enjoyment, a Duchenne smile) and which were posed. The critical clue in deciphering a posed smile from a spontaneous smile is the contraction of the orbicularis oculi muscles (i.e., the muscles around the eyes that create “crows’ feet” when contracted; Ekman, Davidson, & Friesen, 1990). Participants were shown these extracted smile videos and asked to judge whether they expressed genuine enjoyment or were posed.

In the first study, older and younger viewers were shown only young adult targets: both groups did fairly well on this task, and there were no significant differences in accuracy between the age groups. In a follow-up study, both older and younger targets were included. This mixture of target stimuli seems to have improved the performance of the older individuals (perhaps somehow tapping into their experience base more), because in this version, the older adults actually outperformed the young adults in terms of accuracy. This was true even though, overall, the older targets were more difficult for all subjects to accurately identify, regardless of the subject age. Finding a potential age-related improvement stands in stark contrast to the bulk of the previously-reviewed research on aging and emotion recognition, with the exception of a few studies that have found older adults to perform better than the young in identification of disgust (Calder et al., 2003; A. Suzuki et al., 2007) and one single study that did find older adults outperforming young at happy recognition from still photos (Moreno et al., 1993). No other study using still images, or a less subtle distinction (such as “what emotion is this expressing?”) has found age-related improvement in identification of happiness, even though happiness is usually thought to be the easiest emotional expression to accurately identify (Adolphs & Tranel, 2004).

Cues to Accuracy

Transcending the methodological constraints imposed by traditional emotion recognition tasks, and finding paradigms that more closely mimic the experience in recognizing others’ emotions that older individuals may have accumulated will help researchers achieve a description of aging and emotion recognition that is more ecologically valid. However, even that will not give an indication as to how older adults do their accurate emotion recognition, and whether the way they do it is different from how young adults do it. To go beyond description and actually uncover mechanisms that are used in successful emotion recognition in the context of aging will require aging researchers to borrow concepts and methods from the broader field of nonverbal behavior. Below, we propose adopting some of the tenets of the Brunswikian framework for understanding how adults of different ages do and do not use cues in their attempts at emotion recognition. That is, researchers should consider cues to accuracy, as in Brunswik’s (1955) lens model, in order to identify the ecological validity of different cues and investigate the cues which people use (across age and separately by age group). The lens model considers both valid and invalid cues that are available when making the judgment and also which of these cues people tend to utilize.

One important warning about a purely empirically-driven approach to investigating cues used in accurate emotion recognition is provided by Zebrowitz and Collins (1997). They suggest using conceptually-derived groups of cues rather than a kitchen-sink approach to every possible cue. This raises the question of what cues or categories of cues might one conceptually suggest as being particularly salient and/or effective in older adults’ attempts to recognize emotional expressions in others. Below, we provide some speculation as to what clusters of cues might be most promising to evaluate in future studies using a conceptually-driven Brunswikian approach.

One potential set of cues on which older adults may rely involves aspects of the context that serve to disambiguate facial expressions that may not be clear to older adult perceivers. That is, while older adults may have deficits in recognizing emotion in an impoverished task in which only one modality of information (e.g., static facial expressions) is presented, the redundancy that occurs across channels in everyday life may result in no or little age-related loss in emotion recognition in daily life. Older adults may be able to draw on contextual cues to help them interpret the situation. In other words, adding contextual information to the typical emotion recognition task may attenuate age differences in emotion recognition while at the same time increasing the ecological validity of the task. For example, while recent research has suggested that even young adults will use contextual information in their judgment of facial expressions (Aviezer et al., 2008), the effects of context on judgments may be even more pronounced for older individuals (Noh & Isaacowitz, in prep). This would be consistent with studies from the cognitive aging literature suggesting that older adults rely on context as a compensatory strategy (Hess, 2005; Li, Lindenberger, Freund, & Baltes, 2001; Smith et al., 1998), and also that they may be more likely to outsource certain types of processing to external cues (Spieler, Mayr, & LaGrone, 2006).

It is also possible, following from literature on socioemotional selectivity theory and positivity effects described above (e.g., Carstensen, 2006), that older adults are especially likely to hone in on positive cues in their environment, though we are not aware of any studies that have directly investigated this. For example, symbols associated with happiness, such as smiling faces, might be especially likely to be used as cues by older adults as they try to identify emotional expressions, though this remains for future research to investigate.

A final set of cues are idiosyncratic ones that have proven useful to the perceiver in the past in decoding emotions, perhaps especially with frequent social partners. These may be based on structural aspects of the faces of their primary social partners and/or ways that particular individuals in their social groups tend to display expressions. For example, some individuals wrinkle their nose in disgust, while others primarily create a disapproving frown with their mouth. That is, not all individuals express disgust with the prototypical nose wrinkle. Once these idiosyncratic cues are learned about a social partner (such as their children or spouses), perceivers may be able to achieve accuracy with those particular social partners which may not necessarily translate to others.

Another reason idiosyncratic cues may render familiar partners easier to read than strangers is that there are developmental changes even within a person in how their emotions are expressed. Physiological changes that accompany age such as decreased levels of collagen lead to changes in the outward appearance of older adults’ faces. In particular, wrinkles, sagging jowls, and drooping around the eyes contribute to morphological age-related changes in appearance (Hooyman & Kiyak, 1996). Even when an older adult is in a relaxed position, their facial features may convey emotions such as sadness or anger due to these physical changes in the face. Only a handful of studies have examined age-related differences in the facial expression of emotions. Some studies have shown relatively little age-related changes in the facial expression of emotions (e.g., Levenson, Carstensen, Friesen, & Ekman, 1991), others have found that the emotional expressions of older adults are less accurately identified by young, middle-aged, and older others (Borod et al., 2004; Malatesta et al., 1987), suggesting that age-related changes in emotional expression render the emotion recognition task more difficult when using older adult target faces as stimuli. In addition, neutral facial expressions of older adults were rated as more intense than neutral expressions of younger adults (Borod et al., 2004).

The possibility of an own-age effect in emotion recognition has been tested in a handful of studies. These studies have examined whether each age group is better at recognizing the facial expressions of their same-age peers, perhaps due to age-relevance or recent experience/practice with such facial expressions. Indeed, many individuals report having more social encounters with peers of their own age (e.g., Ebner & Johnson, 2009). Malatesta and colleagues (1987) presented videos of facial expressions of young, middle-aged, and older women to young, middle-aged, and older judges. Overall they found that older targets led to the most errors for all ages of judges. Consistent with this finding, Ebner and Johnson (2009) found that young facial expressions in static photos were more accurately identified by both young and older adults than facial expressions of older adults, with no evidence for an own-age bias. More recently, older adults were found to be just as sensitive to deciphering between posed and genuine smiles of young and older targets (N. A. Murphy et al., 2010) -- again, no evidence of an own-age bias. Interestingly, an eye tracking study found that older adults use a similar scanning pattern to young adults when viewing older faces, but not young faces, suggesting that in-group and out-group membership may influence how older adults process certain stimuli (Firestone, Turk-Browne, & Ryan, 2007), but whether these strategy differences translate to differential accuracy for young and old target faces has yet to be shown.

Considering potentially idiosyncratic cue use brings up a problem with how older adults make use of cues generally. That is, there may be individual differences that moderate either what cues older adults use, and/or whether the use of particular cues leads to accurate emotion recognition for them. In our work on attention-emotion regulation links in aging, we have found that attentional ability moderates whether the display of positive gaze preferences leads to positive mood outcomes (e.g., Isaacowitz, Toner, & Neupert, 2009; Noh, Lohani, & Isaacowitz, in press). Individual differences in attentional ability by definition have to do with efficiency of cue use, whether it is a temporal cue (the alerting network), a spatial cue (orienting network), or an incorrect cue (the conflict/executive control network). Therefore, it may be that older adults with good abilities on some or all of those networks can use cues for accurate emotion recognition, whereas those with poorer abilities cannot. Or, it may be that certain attentional networks correlate with the use of particular types of cues and not others.

Conclusions: What is the future of the study of aging and emotion recognition?

The context of emotion recognition may not be the same for all perceivers regardless of age. Ultimately, an ecological approach that takes context seriously would suggest that a key hypothesis for future work may need to be that the pathway to successful emotion recognition accuracy may be fundamentally different for younger and older adults. For example, it is assumed in recent work that young adults’ accurate emotion recognition happens relatively automatically and results not from careful processing but instead from more of a gut feeling (Tracy & Robins, 2008). This may or may not be the case for older adults. In recent work in our lab, we attempted to increase participants’ motivation to do well on the task, in an attempt to improve older adults’ recognition accuracy. Specifically, participants were told, “Once you have completed the emotion recognition task, we will go back through the test together, face by face, and you will explain to me why you chose the emotion you chose for each face”. While the manipulation did not seem to improve older adults’ accuracy, it did have an impact on younger adults’ accuracy. However, the impact was in the wrong direction: the instructions actually disrupted the performance of the younger adults, making them less accurate in their recognition of emotions as compared to young adults who did not receive the motivation manipulation.

Such findings are suggestive that accurate emotion recognition may be automatic for young adults but not older adults. If that were the case, it would have some interesting implications: for example, it might be that older adults can accurately recognize emotions but that it takes cognitive resources for them to do so. Thus, they may not do well on such tasks when they have any simultaneous demands on their thinking; even coming to a lab may consume resources for older adults (due to other cognitive testing during the session, lack of familiarity with the environment, etc.) that may detract from their ability to perform well on a standard emotion recognition task. Another possibility is that older adults can do accurate emotion recognition automatically, but only when the tasks reflect the kind of recognition they have experience doing, such as using dynamic emotional displays and/or recognizing emotional expressions in their close social partners.

One possible fundamental difference in the processing of emotional displays of emotion is whether emotion recognition is a gut feeling for young adults, but older adults need to devote more processing resources in order to achieve accuracy. This would suggest that older adults need to be motivated to devote their resources to processing emotional expressions, while young adults do not need to be so motivated. On the other hand, perhaps if the task were closer to what older adults actually do in daily life, such as deciphering someone’s feelings from a video clip, they would perform this task automatically.

It may be that these fundamental differences in automatic versus more controlled processing of faces results in the typical age differences found in traditional emotion recognition research, but may not amount to practical differences in everyday life. There may be compensatory processes that ensure that small decrements in recognition accuracy do not reach the level of everyday functioning (such as by interacting only with familiar partners, avoiding surprising emotional situations, etc.). By bringing a more ecological approach to the study of age differences in emotion recognition, we hope to spur the field to discover whether there are age differences in recognizing facial expressions in everyday life, and if so, what may be the consequences of such differences. At the same time, such work could be useful to the field of nonverbal behavior generally in providing for a model for the integration of context more centrally into experimental methods as well as into theoretical models.

Now that we have made a case that age is important for nonverbal researchers to consider, we have to raise one concern about doing so: namely, that age is generally a carrier variable in psychological research, not generating changes by itself but instead serving as a proxy for other underlying mechanisms. Beyond the usual limitations of cross-sectional study design for the investigation of age-related processes, the typical “extreme age group” design in which individuals 18-22 or so are compared to individuals 60 and above (as used in many of the studies described above; e.g., Keightley et al., 2006; McDowell et al., 1994; N. A. Murphy & Isaacowitz, 2010; Orgeta & Phillips, 2008; Phillips et al., 2002; Ruffman et al., 2009a; Sullivan et al., 2007; A. Suzuki et al., 2007; although some studies include a middle-aged or continuous age sample, cf. Isaacowitz et al., 2007; Malatesta et al., 1987; Mill et al., 2009; Moreno et al., 1993; Williams et al., 2009) may obscure how age differences in a process like emotion recognition actually involve continuous shifts over time in proximal processes. There are numerous candidates for such proximal processes that may be important but have not yet been featured in research on aging and emotion recognition even though they covary with age: though studies have examined general cognitive functioning (as described above), none have investigated the role of executive control ability specifically, or pattern recognition, for example.

The study of aging and emotion recognition is exciting, and should not be viewed as just another domain of clear and certain age-related decrement. Nor should it be seen as an area in which the mechanisms are clearly delineated. Neuropsychological and motivational hypotheses remain to be fully explored, and there is room not only for new methods but also for more nuanced conceptual frameworks that could generate additional mechanism-relevant hypotheses that can be tested with these improved methods. Much is left to be done. We hope that future work will move beyond descriptive or mechanism-based approaches in order to provide a complete view of the transaction among a perceiver, a target, and the cues that link one to the other. While we do not expect that such work will necessarily find age-related advantages in emotion recognition, we do anticipate that this more contextualized, ecological perspective will reveal strengths in addition to weaknesses in how older adults attempt to accurately identify emotional expressions in their environment.