Figure 1.

An illustration of the hierarchical and multifaceted nature of understanding other people’s behavior in emotional terms. We use fear as an example, but note that the distinctions we make can be applied to any emotion category. To simplify, we demonstrated how a fearful facial expression might be identified at three levels of abstraction. At the lowest level are the different elements of the stimulus (e.g., facial and bodily actions, object of gaze) that in combination can be understood in terms of relatively higher-level behavioral categories, such as fear. Using our capacity for reasoning about the causes of behavior (e.g., using a Theory of Mind), even higher levels of understanding become possible: We understand not just that the person is expressing fear, but why they are expressing it. To organize the literature review shown in the main text, we compartmentalize component processes as contributing primarily to either early detection of emotion-relevant cues; intermediate categorization of the behavior in terms of emotion and action categories; and, finally, further inferences that serve to build a coherent narrative that identifies the cause(s) of the emotion. In actuality, processing at these three levels is typically concurrent and involves substantial feedback, not shown in this figure.