Identifying identity and attributing value to attributes: reconsidering mechanisms of preference decisions

Abstract

Although the orbitofrontal cortex (OFC) robustly encodes value during preference decisions, it also encodes multiple non-value features of choice options. The role of this information, and its relationship to the options’ overall value remain open questions. In this opinion, we attempt to disentangle oft-studied categories of option information – identity and attributes – in the context of both classic theories of economic choice and contradicting evidence of choice biases in multi-attribute decisions. In doing so, we aim to set forth considerations for understanding the wide variety of decision-relevant information encoded by the OFC during preference decisions.

Every day we make simple decisions that require us to weigh multiple facets of the options we choose between. For instance, when deciding what to order for lunch, it may be that a pasta dish tastes good, but makes you sleepy for the afternoon. Perhaps the sandwich shop is also delicious but often sends the wrong order. Larger decisions such as buying a house or finding a new job can involve many features, and a decision-maker might use any number of strategies and sources of information to choose which is better for them. Although a good deal of research has focused on this type of complex decision, the neural computations that allow us to weigh multiple features remain unknown. In this opinion, we reassess the role of non-value information in preference decisions. We analyze which features might be considered an option’s identity or constituent attributes, and how these relate to evolving ideas of the functional role that the OFC plays in value-based choice. But first, we start by outlining a widely held view of how neural representations of value relate to decision-making.

Integrated value and decision-making

Research on economic decisions has emphasized the concepts of integrated value and expected utility that incorporate weighted evaluations of all choice-relevant features into a single scalar variable. Foundational theories of value-based decision-making posit a sequence of operations in which different sources of information that relate to an option’s “goodness” are first combined to compute an integrated value that can then be compared to the value of other options (Figure 1A) (1–3). Integrated values are neutral with respect to the identity of an option, and therefore solve the problem of how to compare unalike options by permitting comparison on a common value scale.

Figure 1. Schematic of two hypotheses for multi-attribute decisions.

A) Decision-making on the basis of integrated value. Each option (A and B) consists of two attributes that are integrated to compute an overall value, and option comparison occurs between these values. B) Decision-making on the basis of attributes. Each option (A and B) again consists of two attributes, but comparison occurs between like attributes without computing the overall option value. Note that we do not posit that these are the only two hypotheses for the process of decision-making, and that the term attribute may refer to a wide variety of option aspects, including quality, time delay, cost, and taste, et cetera, all of which are disparate information sources that may engage different pathways and processing mechanisms. (Created with BioRender.com)

Integrated value can be estimated as a relative quantity by testing subjects’ preferences in a series of choices that systematically vary features of each option (Figure 2). For example, if a subject is indifferent between 3 units of option B and 1 of option A, one can conclude that the value of option B is approximately 3 times the value of A. Using this approach, it has been shown that signals in the OFC commonly vary with integrated value. That is, when tested against a range of other task-related variables, integrated value tends be the best predictor of neural responses following either a reward cue or the reward itself (4), leading to the conclusion that OFC encodes integrated values. While most studies have focused on areas 11 and 13 of OFC in macaques, similar value-related signals are widely distributed in ventral frontal cortex, including ventromedial prefrontal cortex (4–7). These quantities are encoded by the activity of single neurons (4–6), populations (8, 9), field potentials (9–11), and fMRI BOLD responses (12, 13), and meet several criteria for true value coding. They integrate multiple attributes, including quantity, quality, constituent composition, cost, and probability (4, 9, 14–16), and respond to both positive and negative outcomes on a continuous scale (17–20). They are independent of sensory qualities, so that different stimuli predicting the same outcome are often coded similarly (5, 6, 21), as are stimuli predicting qualitatively different outcomes that are equally preferred (22, 23). For example, in a task where monkeys made preference decisions, we found that the majority of OFC neurons encoded equally preferred primary and secondary outcomes (fruit juice and a visual representation of earned reward respectively) as a common-currency value, and did not distinguish stimuli that predicted equally preferred rewards of different types (9, 10, 24). In addition, OFC value signals change to reflect new preferences when associations or motivations change (6, 25–27). Thus, there is sound evidence that integrated value is, indeed, encoded by the OFC.

Figure 2. Example of a neuron encoding integrated value.

The left panel shows patterns of binary choices, fit with sigmoid curves, between two options (A and B), in which each option is a different flavor of juice offered to monkeys in different quantities (x-axis), with different probabilities (X, X”). The flex of the sigmoid determined the indifference point, or the quantities at which A* and B* were chosen with equal probability. The X-axis denotes the ratio of juice B to juice A. The right panel shows mean firing rates of a neuron encoding the value of the chosen option. Activity increased with the subjective value of the chosen option, regardless of whether the animal chose juice A (circles) or juice B (diamonds), and quantitatively reflected the trade-off between juice type, amount, and probability, as measured by the subject’s choice patterns. Adapted from Raghuraman and Padoa-Schioppa [14].

Based on correlations with choice behavior, we (9) and others (28, 29) have argued that integrated values in OFC can serve as inputs to the decision-making process. For example, representations of the chosen option’s value are stronger when subjects make faster choices (9), suggesting a relationship between value signaling and choice behavior. Other models detail how comparisons might occur among OFC neurons encoding the value of different choice options, or “offers” (30), which are intermingled in a population that encodes the value of the offer that was chosen (14, 28, 31–34). The responses of neurons encoding offer values integrate information about the offer, including reward quantity and probability (14), consistent with an integrated value signal that could be compared to arrive at a choice (30). This idea that integrated values in OFC directly contribute to decisions is bolstered by recent evidence suggesting causal links between OFC activity and optimal choice behavior in monkeys (35).

Departing from this view, however, are a number of observations that either pose unanswered questions or stand in contrast to the notion that decisions are made by comparing the integrated values of different options (36–42) (see (43) for a summary). To support these empirical findings, theoretical frameworks have been proposed in which decisions do not rely on the computation of integrated values. For instance, one idea dubbed decision by sampling suggests that simple memory retrieval and comparison among features could underlie complex decisions (44, 45). Related models propose that evidence accumulates in favor of either option as features are sampled sequentially (36, 43, 46–48). Other views suggest that decision processes take place in multiple domains, which may include but are not limited to integrated values, with competition between options occurring in parallel and choice emerging through a distributed consensus (49–52). Indeed, recent results in rodents found that OFC inactivation did not affect economic choices, suggesting either a species difference or that decision-making processes elsewhere in the brain are also relevant (53, 54). A full discussion of alternative models is beyond our current scope, as we do not aim to endorse any particular theory, but excellent recent reviews can be found in (43, 49, 55). Instead, our goal is to revisit prominent perspectives on OFC’s role in decision-making in light of results suggesting that choices may depend, at least to some extent, on information other than integrated values.

Attribute effects in decision-making

One of the main sources of evidence that decisions depend on information other than integrated values comes from multi-alternative choice tasks. A suite of bias effects known as context or decoy effects crop up repeatedly, if not reliably, when subjects choose between more than two options (Figure 3). Given a rational decision-maker, the presence of a third, less desirable option (a decoy) should have no effect on choices between two better options, a principle outlined in axioms of economic theory stating that optimal decisions should be independent of irrelevant alternatives (56–59). However, decoys do have reproducible effects on choice behavior, and notably the direction of these influences depend on the similarity between the decoy’s attributes and those of the other options. In the classic example, these attributes consist of an item’s quality and cost, but similar effects occur with time delays, monetary value, probability, and perceptual stimuli, across a variety of species (60–68). For example, in the attraction effect, a decoy with similar but slightly lower magnitude and probability to one of two equally preferred options (option 1) will tend to influence choice toward that option (1), despite the fact that the other (2) is equally as good and the decoy is objectively worse. Indeed, the mere presence of a third option can affect choice in an attribute-specific manner (50), meaning these biases cannot easily be explained if decisions depend only on integrated values. Thus, curious inconsistencies in choice behavior have become acutely important in understanding basic processes involved in making a decision.

Figure 3. Summary of context (decoy) effects.

Options 1 and 2 are denoted by solid circles, and lie along a line of equal preference. While option 1 is higher in attribute A and lower in attribute B, and option 2 is lower in attribute A and higher in attribute B, they are nevertheless selected equally in binary choice. The presence of any of the decoys displayed (smaller red options), however, will increase the choice frequency of Option 1 (red). dc = decoy which produces the compromise effect, da = decoy which produces the attraction effect, ds = decoy which produces the similarity effect. The specific attribute arrangement of the decoy is essential for influencing the direction of the effects. (Created with BioRender.com)

What qualifies as an attribute?

The finding that, under some circumstances, attribute information can enter into a decision raises the more proximal question of what, if anything, differentiates an option’s identity from its attributes. Historically, there has been overlap between the qualities described by these terms. For instance, probability (or risk) tends to be considered an attribute, while features like taste could be considered either an attribute or a unique identity. In some instances, the term attribute has been used to mean any facet of an option used to guide choice – for example, a stimulus and an action that each predict unique probabilities have been considered two attributes of a final probability of obtaining reward (50). More commonly, the term attribute refers to aspects of an option that directly relate to its goodness (such as the amount of juice). In this way, the term is sometimes used to refer to stimuli communicating components comprising the option, and sometimes the qualities of the option themselves. Likewise, identity can also refer to a stimulus (e.g., a juice-predicting picture) or a quality of the reward (e.g., the juice) (25, 69).

On one hand, it seems reasonable to argue that parametrically varying features such as amount, probability, delay, or cost, constitute attributes of an outcome, whereas categorical features such as the taste of a juice or the odor of a food are identities. However, the latter can be translated to a parametric space, for instance by varying sucrose concentration or mixing odorants in different proportions. In this case identity is less clear-cut, as a subject would not only need to consider whether an option is sweet, but how sweet it is. Another view distinguishes features extrinsic to the option (e.g., quantity, cost, or probability) and those that are intrinsic to it (e.g., taste or quality), and suggests that these are interpreted differently or engage different neural circuits (70). This distinction maps nicely onto the concept of identity (intrinsic qualities) and attributes (extrinsic qualities). However, the example of sweetness and similar situations, in which an intrinsic quality varies across instances of an option, still provide an ambiguous case. Perhaps the most parsimonious view is that the differentiation between identity and attribute depends on how the decision-maker uses available information to categorize options. Indeed, in a task with a large stimulus space, integrated value coding in primate OFC was surprisingly rare compared to cost type, which was the key feature that distinguished groups of trials and was likely used by the subjects to understand the choices at hand (71).

A further complication is that sometimes attributes have unique relationships to each other. For instance, ketchup may be a positive when paired with french fries, but negative when paired with ice cream. A recent study examined this question and the role of the vmPFC in making configuration-dependent evaluations (72). Participants were asked to learn about abstract, multi-attribute objects (“Fribbles” (73)), whose values were indicated either by the sum of their component attributes or by the specific combination of them. Participants with vmPFC damage were selectively impaired when value was configuration-dependent, suggesting that understanding non-linear attribute relationships enlists different processes than simply combining them. This echoes similar results showing unique involvement of vmPFC in social situations like combining desirable attributes in spouses (74) and political candidates (75), or combining affective attributes of artwork with perceptual information (76). Thus, choices may be encoded differently when options need to be evaluated on the basis of holistic configurations, rather than the sum of individual attributes. In addition, across identities, attributes, and attribute configurations, there is evidence that different brain processes are engaged when making decisions based on different types of feature information. How a choice is approached by the decision-maker, whether determined by task design or by individual strategies, may, in turn, change how these features are perceived and ultimately understood.

Attribute and identity coding in OFC

One approach to gain insight into the role of attribute evaluation in preference decisions is detailed investigation of decision variables represented in the brain. Given its well-known role in evaluation and decision-making, OFC is a candidate region that could provide such insights. In OFC, an array of task correlates beyond integrated values have been reported, yet their role in decision-making is unknown. For instance, early comparisons of OFC neurons recorded from rodents and non-human primates identified a consistent difference in how option features are coded. Whereas integrated value signals are common in primate OFC, in rodents, neurons tend to encode information related to the value of outcomes in a manner that is more specific than an integrated value signal (77). For instance, neurons signal unique stimulus-outcome associations (78), or qualities of only one type of reward, such as the amount of chocolate but not vanilla milk (79). This is not to say that specific outcome signals are not also found in primate OFC. Indeed, the offer values described above are defined by varying amounts of a particular flavor of juice, so that flavor could be interpreted as the outcome’s identity. Similarly, we have also found a small number of neurons that selectively responded to the amount of primary or secondary rewards, though these were relatively few compared to the number coding nonspecific values (9, 10).

In humans, fMRI studies have also found that OFC encodes identity-specific information. For instance, BOLD signals in OFC differentiate foods that are equally preferred (69). Although identity-general value codes were found in neighboring ventromedial prefrontal cortex (vmPFC), there did not appear to be a serial relationship whereby the general value signal was ‘built up’ from more specific outcome information, as proposed elsewhere (70). Instead, it was suggested that general value and identity-specific information may serve different roles, with the former supporting undifferentiated appetitive behaviors such as approach, and the latter contributing to more nuanced organization of context-specific behaviors and task representations (80, 81). A follow-up study used satiety to devalue specific foods and found changes in functional connectivity between an OFC region encoding identity-specific information and vmPFC encoding general value (25). That connectivity fluctuated depending on motivational state supports the notion that identity information in OFC is not always a precursor to general value signals, and depending on the circumstances may serve a parallel function. Consistent with this, lesions of OFC diminish effects of outcome identity on conditioned behavior while leaving intact responses to general value (82).

Beyond identity, option attributes are also encoded by OFC (8). In monkeys, a subset of OFC neurons encode single components of “bundled” options that consist of two different juice rewards, and these responses change when the animal becomes sated on the relevant component (16). Evidence from human fMRI suggests that beliefs about the constitutive nutrient attributes of evaluated food items are encoded in lateral OFC, and can be isolated from the overall subjective value of the food (74). In this case, connectivity analyses suggest that attribute information may contribute to an overall value signal, but unlike integrated value, attributes are only encoded at the time of food evaluation, not when offering a monetary bid for the food or receiving feedback about the food’s value, a distinction that could relate attribute signals to an early step in decision-making, such as information sampling. In line with this possibility, subjects performing a different multi-attribute task in which they sequentially uncovered attributes of different choice options tended to favor an attribute-based search pattern, uncovering the same attribute of different options in sequence (83). In contrast, patients with vmPFC damage that encroached on OFC used a search strategy that uncovered multiple attributes of one option, potentially indicating that the ventral frontal cortex plays a role in both acquiring and using information about attributes when making decisions. Taken together, there is strong evidence that information beyond integrated value, including option attributes and identities, is represented by OFC activity, leaving the door open for investigation into how these signals might be used in decision-making processes.

Conclusions

Although value signals are common in OFC, features of options such as identities and attributes are also represented, and some evidence suggests that these may not necessarily contribute to the construction of integrated values. This, combined with behavioral evidence that decision computation can depend on an option’s features, opens the possibility that attribute and identity signals in OFC inform choices directly. Which information is encoded and how it is parsed likely depends on aspects of the task and perspectives of the decision-maker. Thus, it may be that value-based choices can occur in the domain of attributes or integrated values, depending on factors like attention or previous experience. Recent work in both OFC coding and multi-attribute choice have revealed attentional modulations (84–89) (but see (60)), leading to the possibility that attention to options or attributes could drive neural responses in OFC. Alternatively, the degree to which an integrated value is well-learned or constructed online may determine the need for OFC to encode unique features (25, 69). As decisions become more predictable, an overall task structure, or schema, that generalizes across specific instances of those decisions, can emerge to more efficiently guide behavior, a process that has recently been shown to involve OFC (80). Taken together, OFC appears to represent information that is most useful for the completion of a task at hand (81), providing a potential explanation for the confusing overlap of option identity and attribute, as well as why integrated value signals are not always seen (71). Importantly, we emphasize that there are fruitful avenues beyond integrated value that will help us better understand the processes underlying decision-making in OFC. In this way, instances where value isn’t represented may be just as illuminating as those where it is. The capacity of the brain to solve the problems placed before it is immense, and observations from one decision-making context reveal only one version of how OFC can use information to make a choice. Ultimately, the demands on and whims of the decision-maker crucially influence the choice process, and there is much to be gained by integrating these perspectives into interpretation and theory.

Highlights.

• Integrated value has repeatedly been shown to be encoded by the orbitofrontal cortex

• Non-value information may play a critical role in preference decisions

• The terms identity and attribute have overlapped in the decision-making field

• How identities and attributes are used by OFC may crucially relate to task demands