Context-Sensitive Computational Mechanisms of Decision Making

Abstract

Real-world information is primarily sensory in nature, and understandably people attach value to the sensory information to prepare for appropriate behavioral responses. This review presents research from value-based, perceptual, and social decision-making domains, so far studied using isolated paradigms and their corresponding computational models. For example, in perceptual decision making, the sensory evidence accumulation rather than value computation becomes central to choice behavior. Furthermore, we identify cross-linkages between the perceptual and value-based domains to help us better understand generic processes pertaining to individual decision making. The purpose of this review is 2-fold. First, we identify the need for integrated study of different domains of decision making. Second, given that both our perception and valuation are influenced by the surrounding context, we suggest the integration of different types of information in decision making could be done by studying contextual influences in decision making. Future research needs to attempt toward a system-level understanding of various subprocesses involved in decision making.

Introduction

Decision making is the process of choosing an option among several alternatives. Day-to-day life involves many decision-making scenarios that range from making decisions at the level of an individual (eg, purchases in a supermarket), group (eg, collaborative decision making in companies), or an organization (eg, recruitment of personnel). In individual decision making, choice behavior would reveal subjective and relative preferences of an individual among the multiple alternatives available. It is therefore imperative to understand the mechanisms involved how people arrive at such decisions. Consider an example when you are asked to choose between 1 apple versus 2 apples, which can be considered a value judgment based on the quantity of the reward. Compare this with a situation where you are asked to choose between 1 apple and 1 orange. In such scenarios, a more rewarding option is not necessarily more (or less) in quantity, instead is a different type of reward altogether. Hence, to understand a decision-making phenomenon, we need to understand the computation of subjective values of the available alternatives. Decision-making scenarios encountered in real-life situations are often also repetitive in nature. Thus, understanding the dynamics of value computation over time can help us better understand how people make decisions the way they do and how these decisions can be improved.

Decision making forms a key link between sensation and action in which decision transformation maps sensory evidence onto an action selection followed by its execution. Thus, studying decision making has potential to open a window to study cognition in general.¹ Real-world information is primarily sensory in nature, and understandably people attach value to the sensory information to prepare for appropriate behavioral responses. How does the perceptual judgment of alternatives in such scenarios differ from value-based preferences?² The value computation step distinguishes a simple perceptual discrimination task from those that involve value-based decision making.³ Theoretical accounts have suggested 2 distinct systems for valuation and action selection.⁴ At a neuronal level, decision-related signals can be easily distinguished from sensory signals, but it is more difficult to isolate these from the action signals.⁵ The selected action that reveals the choice made and the cognitive process that precedes the choice can be termed as a decision.⁶ Choices are formed through the computations that are performed when samples of information (either sensory or values) are integrated toward selecting one of several alternatives.⁷ Thus, choices form a key measure to study individual preferences.

As pointed out by Marr,⁸ to understand the mechanisms of any cognitive phenomenon, it becomes imperative to investigate at multiple levels covering the computational, algorithmic, and architectural implementations. Accordingly, the purpose of this review is 2-fold. First, we identify the need for integrated study of different domains of decision making, so far studied in an isolated fashion.⁹ Second, we review and suggest that the integration of different types of information in decision making could be done by studying contextual influences in decision making.

In the next 2 sections, we further review literature from value-based and perceptual decision-making domains and identify cross-linkages between the 2 domains to help us better understand decision-making processes in general.

Value-based decision making

As noted above, valuation of alternatives is central to theoretical, empirical, and computational approaches of decision making. For optimal decision making, subjective evaluation of available alternatives is necessary prior to making a choice. A general mechanism for decision making can be understood by the notion of “common currency” where the comparisons of various competing alternatives can be made over the same relative scale.¹⁰ Value-based decision making is mostly investigated in the context of “free-choice” or preference tasks allowing subjects to choose freely among the various options available. The various alternatives provided to the participants vary in magnitude or reward probability and then the choice behavior is studied with reference to these manipulations.

Valuation has also been central to classical and modern economic theories that provide a rich account of human decision-making behavior and subjective preferences. Studies of decision making started with describing outcomes of people’s choices when faced with uncertain or risky decision-making scenarios, in which the choice outcomes are probabilistic in nature and are drawn from a known set of distributions.⁶ The underlying principle used to explain people’s choices was the maximization of the expected monetary value of the available alternatives. The expected value is given by the sum of payoffs of a particular state multiplied by its respective probability of occurrence. Bernoulli¹¹ demonstrated that expected value is not the maximization of monetary values, but it depends on other factors such as people’s perception of the likelihood of winning over the chosen alternative. Kahneman and Tversky¹² showed how in reality it is difficult to formulate a model that takes into account both normative and descriptive theories. Prospect theory was, thus, proposed as a descriptive theory. They also show how in real life people violate the normative accounts of maximization. The fundamental idea behind prospect theory is that the utility function is governed by the gains and losses compared with a relative reference point. The utility function looks like an asymmetric S-curve around the relative reference point. The value function plotted above the reference line is typically a concave-shaped curve reflecting the potential gains and is convexly shaped for values below the reference line denoting potential losses. The asymmetry of the S-shaped curve denotes people behaving as risk-averse in the gains domain and risk-seeking in the loss domain.

Reward prediction error and reinforcement learning

Economic theories have suggested that choice behavior is determined by the values attached to different alternatives in decision-making scenarios. The complementary perspective on how these values are acquired is provided by reward-related learning theories.¹³ Seminal work by Schultz et al¹⁴ showed phasic activity of dopaminergic midbrain neurons corresponding to prediction error which elicits the difference between actual and expected rewards. This process is similar to the computational reinforcement learning (RL) algorithms which model how an agent learns the values of states and actions.¹⁵ The agent’s task is to learn an optimal policy that maximizes expected sum of rewards with future rewards discounted exponentially by their delay. Hence, an agent would begin without any knowledge of the environment and would learn or sample through experience over time discovering the most rewarding outcomes using explore or exploit strategies. The Temporal Difference learning, gives the differences in predictions over successive time steps to model the learning process. The discounted sum of all future rewards is computed by an exponential weighting such that reinforcement at a distant time step becomes less important. This can be expressed as follows:

$$V\left(s_t\right)\leftarrow V\left(s_t\right)+\alpha \left(R-V\left(s_t\right)\right)$$

where $s_t$ is the state visited at time t, R is the reward received after transitioning and $\alpha$ is the learning rate parameter.

Q-learning is a modified version of TD learning, where the values of state-action pairs are learned directly instead of updating values of each state with respect to the previously experienced rewards. Q-learning can be formally expressed as follows:

$$Q\left(s_t,a_t\right)\leftarrow Q\left(s_t,a_t\right)+\alpha \left(R-Q\left(s_t,a_t\right)\right)$$

where, $Q(s_t,a_t)$ is the value of the current state-action pair, $\alpha$ is the learning rate and R is the reward received on trial t.

Another class of models, also based on the RL framework, known as the actor-critic model consists of 2 modules—an actor and a critic. The critic module is responsible for calculating prediction error, and the actor module is responsible for using the prediction error to update the action values and select an optimal choice from the available alternatives. Daw et al¹⁶ proposed 2 distinct types of RL mechanisms that could explain the difference between habitual and goal-directed behavior during learning and decision making. In model-free learning, the agent learns through reward prediction error signals which are insensitive to the model of the environment, ie, learning is based on actual experience of rewards or punishments. However, in a model-based learning, the underlying structure of the surrounding environment (eg, the reward and transition probabilities of a given state-action pair to a new state) helps to predict future outcomes of action sequences.

Current Models of Decision Making

The process of decision making involves at least 2 steps—we need first to represent the various alternatives and then compute the value of these alternatives. The valuation step in decision making depends on environmental factors such as context and time pressure. Contextual knowledge enables an individual to make optimal choices that must be either provided beforehand or learned through experience over time. The value computation can be considered to be either a bottom-up process or a top-down process. The bottom-up process of valuation is dependent on frequency and quantity of reinforcement (reward) of the available alternatives. The top-down process takes into account environmental factors such as expected outcomes or cognitive and emotional states.⁶ This has been proposed earlier in terms of visual processing mechanisms, where “low-level information extraction of visual properties is followed by higher level decision process of evaluating goals and expectations of the participant to prepare for an appropriate behavioral response.”^17(p454)

Gold and Stocker¹⁸ recently reviewed visual decision making bringing together a holistic perspective combining events that occur long before decision making to postdecision processes. More specificially, statistical regularities and perceptual learning provide contextual information in addition to top-down and bottom-up factors that influence a decision. The decision process is described in terms of evidence accumulation and deliberation culminating in commitment of choice. Postdecision evaluative processes inform future decisions.

Doya¹⁹ proposed that a decision-making process is composed of 4 steps: first is the accumulation of evidence to recognize the present state, second step involves action evaluation in terms of rewards and punishments of the available options, third step involves choosing one of the options based on cost and benefits calculated in the previous step, and the last step of the process is to reevaluate the option according to the outcome and update the rewards or losses associated with the outcome.

Rangel et al²⁰ in one of their reviews on value-based decision making propose the following multistage framework for understanding the processing steps involved in value-based decision making. These components of the framework are not necessarily separate brain regions; rather, they have overlapping brain structures involved. The framework has 3 components. First, they divide the process of decision making into 5 basic subcomponents—(1) representation, (2) valuation, (3) action selection, (4) outcome evaluation, and (5) learning. Representation includes identifying internal and external states such as hunger level and threat levels, respectively, and also a potential course of action associated with each of these states. Valuation step is responsible for assigning a value to each action given the internal and external states. This step results in associating the rewards (or losses) associated with each action. The action selection step is the act of comparing the valuation of each action and making a choice. After a choice is made, the next step is outcome evaluation, where the brain measures the desirability of the current outcome associated with the action taken in the previous step. Finally, the representation, valuation, and action selection are updated to make better future decisions; this step is called the learning step. On the basis of human and animal behavioral evidence, the second component of framework of Rangel et al shows that there are 3 types of valuation systems: (1) Pavlovian, (2) habitual, and (3) goal-directed systems. The Pavlovian valuation systems are “hard-wired” responses to a small set of environmental stimuli. Habitual valuation systems learn and assign values on the basis of past experiences or a trial and error method. Goal-directed systems assign values to actions by calculating an action outcome mapping and reward associated with each of these mappings. Goal-directed systems incorporate modulating variables that affect the decision-making process. Some of these modulating variables are risk and uncertainty and delay discounting.

In another review, Kable and Glimcher²¹ using primate and human neurophysiology studies, in addition to neuroeconomic models, formulated another multistage decision-making framework. This framework included 2 major steps: (1) valuation and (2) choice. During the valuation stage, different factors of each available alternative are taken into consideration, and a subjective value is assigned to each of the options available. The valuation circuits are seen to be primarily located in the ventromedial prefrontal cortex (vmPFC) and striatum. The choice stage has comparisons being made of the subjective values from the previous step to decide on the best alternative. Neural representation of choice circuits is found in lateral prefrontal and parietal cortex. The valuation and choice stages are similar to valuation and action selection step from Rangel’s framework. The other components from Rangel’s framework, i.e., representation, outcome evaluation, and learning, are not explicitly identified in Kable and Glimcher’s framework, but the processes performed in these stages are implied in the model.

There is a growing consensus that key brain systems associated with the subjective valuation network include the ventromedial and dorsolateral part of the prefrontal cortex, posterior cingulate cortex, and striatum.²¹ The subjective value network converts the different competing alternatives into a common currency to facilitate a decision. The valuation system is proposed to be subserved by vmPFC that bridges the sensory information with choice execution. It receives input from the dopaminergic systems that have been shown to encode reward prediction error (see Schultz et al,²² for a recent review). More recently, the lateral intraparietal cortex has been shown to play a key role in the transformation of value (Figure 1).²³

Figure 1.

Neural model of decision making.

Adapted from Levy and Glimcher.¹⁰

The domain of value-based decision making has identified computation of value as a common currency for guiding the choice process. The next section reviews perceptual decision making, where the sensory evidence accumulation rather than value computation becomes central to choice behavior.

Perceptual decision making

Consider a scenario where on a clear sunny morning you are commuting from home to work. You are effortlessly able to see most information, for example, the shops on the sides of the road, signboards to turn left or right, or pedestrians waiting to cross the street. However, during foggy weather, the sensory information is noisy and hence to reach at a particular decision, for example, taking a particular turn or to stop at a traffic light, you take longer to accumulate the incomplete or ambiguous sensory information from the surroundings. This type of decision making where you make categorical judgments over the accumulated sensory evidence is referred to as perceptual decision making. The accumulated sensory information is then translated to guide us how we behave in the world.

In laboratory settings, perceptual decisions are investigated by simulating the uncertain environments through controlled tasks, for example, by asking participants to discriminate the direction of a noisy patch of random dots or a Gabor patch consisting of alternating gratings with certain orientations. However, in real life, there are various other factors that appear simultaneously with the visual stimulus of interest which may or may not influence the decision-making process. Some of these factors are the difficulty of the task presented, prior probability of the event occurrence or knowing the outcome of the task. Because perceptual decisions are easier to control and quantify as compared with other decisions, perceptual decision-making domain has been studied extensively using various behavioral, computational, and neural approaches (see Hanks and Summerfield,²⁴ for a recent review). We now discuss various experimental approaches used to understand the process of perceptual decision making in humans and primates. Most commonly used approaches to investigate perceptual decision making are based on different visual discrimination tasks where the participant has to identify a certain stimuli or pattern from a noisy environment. A random dot motion task is perhaps the most widely used task to investigate perceptual decision mechanisms. In this paradigm, the participants are asked to identify the motion of coherently moving dots among a patch of noisy dots moving in random motion. In nonhuman primate studies, the identification of direction is usually revealed by eye movements. In human participants, the direction identification response is usually recorded as a key press corresponding to a particular direction of motion. Typically, such studies use a 2-alternative forced choice (2AFC) setup where the minority coherent dots move in either the left or the right direction. Random dot motion task has 2 variants. The first is where subjects are allowed to report the direction of the coherently moving dots as soon as they recognize the motion of the dots. This is known as the reaction time variant. Such an experimental setup not only shows when the participant decided on the stimulus direction but also shows how much evidence was required to reach that particular decision. This is a good setup for investigating the speed-accuracy trade-off in perception. The second variant of the task is where the stimulus is displayed for a fixed amount of time usually for 1 or 2 seconds.

Several computational models have been used that characterize performance and explain how humans and animals arrive at a decision when faced with a noisy environment. Signal detection theory (SDT) offers a simple explanation of how stimulus information is accumulated over time to arrive at a decision. It is a mathematical procedure that can quantify between the signal (stimuli of interest) and noise (background stimuli of no interest). The main advantage of using SDT over other popular mathematical frameworks such as information theory and game theory approaches is that it can specify how a single observation leads to a single response. SDT can also be applied to both behavioral and neuronal data. There is also a notion that representation of sensory evidence further gives rise to something called a decision variable over which a decision rule is applied.¹

The SDT suggests that a decision is reached on the basis of a single sample of information, whereas another prominent framework of perceptual decision making is sequential sampling models (SSMs) that assume multiple samples are integrated over time until a decision boundary is reached.²⁵ Sequential sampling models of decision making can be seen as a time domain extension of the SDT. All SSMs repeatedly sample and accumulate evidence over time to reach a decision. Sequential sampling models are separated into 2 classes: diffusion models²⁶ that assume relative evidence accumulation over time and race models that assume independent evidence accumulation and response commitment once the first accumulator crosses a boundary.²⁷

Diffusion Decision Models

Diffusion models are used to model fast 2-choice decisions (see Ratcliff et al,²⁶ for a recent review). These models assume accumulation of noisy information over time toward the decision criteria representing the 2 available choices. The 2 choices are represented as upper and lower boundaries. Once the accumulated evidence crosses a threshold toward 1 of the 2 choice decision boundaries, the corresponding option is chosen. This assumption not only gives the choice distributions but also gives the response time distributions which is the time taken by the model to reach a decision boundary. The speed with which the evidence accumulation process approaches 1 of the 2 decision boundaries represents the relative evidence toward 1 of the 2 boundaries and is termed as drift rate (v). Due to noise in each trial of the drift process, the time taken to reach a particular boundary would vary across trials. If such a consistent variation is observed over different conditions, the drift rate reflects task difficulty with smaller drift rates representing more difficult tasks. When participants’ performances are compared, drift is a measure for individual cognitive or perceptual speed of information processing. The distance between the 2 boundaries is called threshold and is denoted by “a.” Threshold per trial gives the amount of evidence to be accumulated until a response can be executed. A lower threshold leads to not only faster responses but also increased noise influence on judgments making the decisions impulsive and more error prone. A higher threshold leads to careful responses (slower more accurate but skewed RT distributions). Different studies have shown that the parameter “a” is sensitive to speed versus accuracy instructions. In addition, slowing in response times is shown to be attributed to age-related changes which can be partially explained by the conservative response styles. Response time per trial does not solely comprise the decision making but also includes perception, movement initiation, and execution which clubbed together form the nondecision time parameter (t) of the drift diffusion process. The drift diffusion model also includes a bias parameter z, which represents the starting point of the drift process relative to the 2 boundaries. Bias parameter is responsible for the starting point of response time distributions for each trial. Difference in bias parameter across conditions can reflect choices encountered with different payoff matrices (Figure 2). For example, the starting point moves toward a response threshold when the corresponding response leads to greater rewards (see Voss et al,²⁸ for a review).

Figure 2.

Schematic diagram of a drift diffusion model.

There are also other models that aim to understand perceptual decision making which are inspired by the underlying biologically phenomenon such as inhibition mechanisms of neurons and information decay over time. Leaky competing accumulator (LCA) model is one such leaky diffusion model. The LCA is a connectionist network model of decision making. Preferences in this model are based on the sequential evaluation of advantages and disadvantages of each prospect.²⁹ Another model similar to the LCA is the linear ballistic accumulator (LBA). The LBA has 2 accumulators as in the LCA but assumes a noisy environment for longer timescales but a noise-free environment during the evidence accumulation process.³⁰

As described in the previous 2 sections, research in perceptual decision making has mainly relied on experimental paradigms and computational models that are different from those of value-based decision making. Most of the decisions pertaining to social situations involve more than one individual, which forms yet another domain of study in decision-making research, expanded in the following section.

Social decision making

Traditionally, decision making has been investigated under the light of individual decision making. The experiments designed are such that participants are asked to choose between different available options of monetary gambles or goods. Participants’ choices in such settings only depend on their individual preferences and valuation. However, in real-life scenarios, the decision making by individuals is not done in isolation but is an outcome of complex social interactions which sometimes also depends on the choices of other individuals. Scientists in recent years have begun investigating social decisions using approaches from game theory which is a part of experimental economics. Game-theoretic constructs are useful to investigate situations that involve conflict between short-term rewards and more distant, but potentially larger, rewards. For example, to keep long-term benefits of a sustained cooperative relationship would I be willing to go through the immediate costs attached to altruism (see Rilling and Sanfey,³¹ for a review). A variety of neuroscientific methods are also being used to understand a more detailed picture of psychological and neural correlates underlying social decision making.

Experimental games such as the ultimatum game and trust games have been widely used to study strategic interactions during fair versus unfair behaviors. In an ultimatum game, 2 players are given an opportunity to divide a given some of money among themselves. One of the players is called as proposer who decides on what should be the proportion of dividing the money. The other player called the responder can either accept or reject the proposer’s offer of the division. If the responder accepts the offer, then the money is split as proposed. In case the responder decides to reject the offer, both proposer and responder end up with receiving nothing. According to Nash equilibrium prediction, if people make choices purely out of self-motivation, then any offer made by the proposer should always be accepted by the responder, and the proposer should always offer the smallest nonzero amount. However, it is seen that offers below 20% split are rejected half of the time³² possibly owing to the feeling of being mistreated. Thus, participant’s choices are motivated not only by self-interest but also by other factors. Neuroscientific methods can help unraveling these complex decision-making interactions. Trust game is useful to study reciprocity which is another essential element of social interaction. Trust game involves 2 participants: one player is the investor and the other is the trustee. The investor decides to how much money has to be invested together with the trustee. The amount that the investor decides is than multiplied by some factor and the new increased amount is then at the disposal of the investor to return (or not) to the investor. According to game theory predictions, trustee out of his rational and self-motivated interests would never honor the investor’s trust and would end up returning zero money to the investor and the investor knowing this would never invest anything in the first place. However, in real-life scenarios, it is seen that trustees do transfer some amount of money and the trust is reciprocated.³³ Prisoner’s dilemma is another experimental game used for investigating reciprocity. This game helps us better understand competition and cooperation in complex social settings. In this game, both players decide to choose whether to trust the other player or not without knowing each other’s intent. In prisoner’s dilemma, the highest payoff is achieved when one of the partners cooperates and the other defects. If both participants decide to cooperate, a moderate payoff is achieved and when both decide to defect the participants receive the least payoff.

The paradigms mentioned above involving social decision making extend individual decisions to highly complex social scenarios. Thus, in addition to the valuation of multiple alternatives, the decisions in social interactions also depend on choices of others, which affect both the individual as well as others.³¹

How context-dependent perception affects valuation and choice

In visual perception domain, this has been long known that context of surrounding objects in the field of view affects the perception of the target object. Perceptual attributes of an object such as perceived size, shape, and color of objects are liable to change due to changes in the context despite the actual physical attributes being constant at all times. For example, Ebbinghaus illusion (or Titchener circles) shows 2 objects which look equal in size, but when the same objects are surrounded by other objects, the target objects are perceived to be of different sizes.

However, in the field of value-based decision making, these ideas of context as modulators have not been considered by the standard economic theories. The expected utility theory and optimal foraging theory³⁴ assume that irrelevant information (or context) does not affect the decision outcome. This also means that the value calculation step is strictly performed only on relevant alternatives and contextual information such as temporal history and available resources do not bias decision making. Contrary to the above-mentioned theories, a number of studies have shown that context strongly affects choice behavior violating the assumption of normative theories that humans are rational decision makers. In multialternative choice, inclusion of other options can alter choices among a fixed set of options.^35,36 The attraction effect is the enhancement of the preference for one of the options by introducing a similar but inferior decoy option. The similarity effect increases the probability of selecting the dissimilar option, and the compromise effect increases the probability of selecting the third option that is intermediate to the 2 original options.³⁷ Studies, for example, show that context can lead to a bias in subjective valuation and hence lead to a change in preferences³⁸; choice behavior was also shown to be biased when the same alternatives were differently worded (framing effect³⁹).

The decision field theory⁴⁰ was based on the idea of sequential sampling of information that is accumulated over time to make a decision in uncertain environments. Kőszegi and Rabin⁴¹ developed a reference-dependent model of preferences. They assume that a person’s reference point is determined by the context of the rational expectations about outcomes from recent past. They illustrated that the endowment effect (ie, people ascribe more value to things merely because they own them) observed in laboratory settings disappears in real market due to the reference-dependent expectation of sellers and buyers in the context of trade. Stewart et al⁴² proposed theory of decision by sampling in which valuation of alternatives depends on both the immediate context and from memory (ie, outcomes from previous decisions). This suggests that people do not use internal scales for value but rather constitute simple cognitive processes such as comparison of relative rank of an alternative. Therefore, subjective values are constructed online depending on both the immediate context and also from memory. More recently, Bayesian models have been proposed that explain choice behavior in terms of incentive values reflecting relative values of rewards in a given context.^43,44 The theory suggests that value of an option corresponds to a precision-weighted prediction error. The predictions are based on the expectations of the reward. This theory could explain contextual effects of options those that are presented in the past as well as options currently available.

Examples from both perceptual and value-based domains suggest that decision processes are dependent on the specific context a decision maker is facing while encountering these choices. Context faced by a decision maker could, for example, be the history of reward experiences or could also depend on the relative value of other possible alternatives being presented. Breiter et al⁴⁵ showed 3 kinds of prospects in which participants could win or lose money (good: US $10, US $2.50, US $0; intermediate: US $2.50, US $0, −US $1.50; and bad: US $0, −US $1.50, −US $6). The outcome of US $0 on a good prospect will be experienced as a loss and the same outcome in a bad prospect would be experienced as a win. Accordingly, responses in amygdala and nucleus accumbens were found to be context dependent. Similar finding of context dependence across a number of brain regions was reported by Nieuwenhuis et al⁴⁶ who compared win and loss gambles involving a common outcome of winning or losing nothing (Figure 3).

Figure 3.

Context gambles.

Adapted from Nieuwenhuis et al.⁴⁶

Contexts can be manifested in 2 different ways, i.e., spatial context and temporal context. Spatial context refers to the value of other simultaneously available options within a trial, whereas temporal context is the outcomes experienced over time or across trials.⁴⁷ A study by Tremblay and Schultz⁴⁸ elucidates temporal context manipulation in the orbitofrontal neurons. A monkey was presented with 3 different cues that yielded 3 different rewards—cereal, apple, and raisins. In one block, the pair of rewards available were a cereal or an apple and the rigorous firing rate of neurons for apple indicated a higher preference for apple. However, in another block, when the less preferred reward of cereal was replaced with a more preferred reward, raisin, the neurons fired rigorously for raisins implicating neuronal coding of preferred choice. The neuronal firing for the previously higher preferred choice (apple) reduced when paired with a different alternative. This indicates that the neuronal activity encodes a relative preference over time in the context of other available rewards. Similarly, Padoa-Schioppa⁴⁹ demonstrated that neuronal firing in orbitofrontal cortex adapted to different ranges of reward magnitude when compared with no reward. Earlier findings by Tobler et al⁵⁰ and Kobayashi et al⁵¹ suggest that dopaminergic neurons and orbitofrontal neurons adapt to the range of input reward magnitudes. This phenomenon can be explained using the efficient coding hypothesis.

The efficient coding hypothesis is inspired by information coding theory used for communication networks. Scientists suggested that neurons use a similar process to efficiently encode information.^52,53 The efficient coding hypothesis states that a group of neurons adjusts the spiking rates to encode maximum information for efficient resource utilization, similar to communication systems which attempt to transfer information in fewest possible bits. A large body of literature in visual neuroscience has shown that neurons in the sensory circuits adapt to the properties of the surroundings. Similar to the visual information transfer system in the brain, some researchers now suggest that the decision-making circuitry in the brain also follows the efficient coding hypothesis.⁵³ For example, a monkey is presented with 2 choices—one drop of juice versus a full jug of juice. In such a case, it is easier for the monkey to make a decision as in the former case the spiking of neurons would be far less than the latter. Now consider an example, where the monkey has a choice between a jug of juice which is almost full versus a jug that is completely full. In this case, if neurons were to encode exact information, then it would become quite difficult for the monkey to make a choice when let us say one neuron fires 80 spikes a second versus the other one which fires at 100 spikes a second. Scientists now propose that brain avoids this problem by encoding the subjective value information by the relative difference between the 2 available choices. In the above example, this would lead to the neurons encoding the jug almost full with a low firing rate as it is currently the worst choice of the 2 when compared with the other more rewarding option. It now again becomes easy for the monkey to make a choice between the 2 options. Neuronal adaptation, therefore, can have additive (shift) and multiplicative (gain) normalization that scales the reward values relative to other available alternatives (ie, context). Indeed, normalization has been proposed to form the basis for context-dependent decision making.⁵⁴

Context modulation is one such efficient coding hypothesis which is involved in both decision and sensory system circuitry.⁵⁵ As such, in the current literature, there is no specific definition for context or its modulation. Broadly context modulation can be defined by the interaction between 2 different kinds of inputs: first consists of the feed-forward connections from the earlier areas in the preprocessing stream and second consists of the modulatory system that controls the system response to the driving inputs. A theory of context modulation would also explain the optimality of behavior contrasting with the earlier notions of independence of irrelevant alternatives axiom.

Recent studies by Rigoli et al⁴³ manipulated context in value-based decisions and found a role for hippocampus and dopaminergic midbrain mediating the corresponding neural adaptations. Their manipulation involved partially overlapping high- and low-value contexts. The reward values common to both the contexts were used to identify the corresponding context-sensitive neural mechanisms.

Toward a Domain-Generic Understanding of Decision Making

For decades, research in decision making has focused on using paradigms and computational models that individually focus on investigating perceptual, value-based, or social decision-making scenarios in isolation. However, in real life, decision-making scenarios rarely occur as isolated scenarios belonging to 1 of the 3 decision-making domains. In real-life scenarios, we do not always encounter choices which are more or less rewarding, but we also come across alternatives or rewards which are of different types. Furthermore, perceptual decisions can be ultimately seen as reward driven and in the same vein economic decisions might require perceptual valuation of available options.⁹ Recent studies have shown supportive neural and behavioral evidence in monkey studies^23,56,57 for investigating perceptual and economic decisions in an integrated way.

Given that contextual effects are apparent both in sensory perception and valuation, the current literature has a gap in understanding contextual influences in decision making. As elaborated earlier, the subcomponents of decision making involve both sensory encoding (representation) and valuation before choice execution. It is therefore imperative that we need to understand which of the subcomponents of decision processes are modulated by contextual information. We propose that it is necessary to identify not only the existence of context-induced bias but also the uses of contextual information to understand which stages of the decision-making process are influenced. Specifically, integration of simultaneously available contextual information can be distinguished from experience-based or temporal context. These context-sensitive mechanisms are similar to widely known simultaneous and successive contrast effects in vision. Contextual influences in valuation have been proposed to involve the spatial and temporal contexts in line with the efficient coding hypothesis.⁵³ According to the general agreement of a Bayesian observer, individuals incorporate experience in terms of a prior. Therefore, studying the subprocesses involved in decision making involves not only the valuation of alternatives and incorporation of simultaneously available contextual information but also the prior by considering the temporal context of the decision-making scenarios. There also is a need to establish the role of common currency hypothesis in decisions that do not explicitly invoke valuation processes such as in perceptual judgment. A key question then is to what extent the cognitive processes of decision making are similar or different across different domains—perceptual and economic (value-based). This can be studied using experimental paradigms that integrate these 2 domains. A relatively new line of investigation is studying the integration of reward values with perceptual decisions.^9,23,57–60

Akin to the perceptual decisions, we can consider that the valuation process also has an accumulation of evidence. Recently, there have been successful attempts to use the drift diffusion model to investigate value-based decisions.⁶¹ Although this is intuitive in perceptual decision making, there is little explanation on how such an evidence accumulation of valuation might take place. Value-based decision making has so far been studied majorly by prespecifying the probabilities and associated outcome values. Chawla and Miyapuram⁶² studied value-based decisions in an experience-based, repetitive decision-making setup. Participants have to repeatedly sample from 2 cards left or right that would reveal a corresponding outcome. We here investigated how past experience shapes expectations. This experimental approach is also interestingly suitable to study a variety of reinforcement types. Hence, this experimental approach can be used for domain-generic studies of decision making. Using meta-analysis of brain imaging studies, we investigated the neural mechanisms involved in different types of decision making.^63-65 If a single system exists that supports computation of a (common) decision variable, then we should find overlapping regions of brain activation. However, if the value and perceptual decisions involve different processes, then we would observe involvement of different sets of brain activations. According to the common currency coding hypothesis, the neural substrates of decision making would involve a common circuit across different domains of decision making.¹⁰ One challenge in verifying this hypothesis is that different paradigms are used for different domains of decision making and therefore cannot be studied in single studies. Although there are very few studies that study the cross-linkages between different decision-making domains, we found common or domain-generic, as well as domain-specific, brain activations.⁶⁴ The common network revealed by the conjunction analysis comprises basal ganglia (caudate, putamen, pallidum), insula, together with inferior frontal region, supplementary motor area, and thalamus. These regions could form the common currency evaluation and choice execution networks in the brain. The inferior parietal area appears to be involved in both perceptual and value-based decision making, but not necessarily in social decision making. Anterior cingulate cortex and medial prefrontal cortex show selectivity toward social and value-based decision making. It further remains open to future work to identify the specific cognitive and neural mechanisms supporting the temporal dynamics corresponding to the different stages of decision making.

Conclusions, Limitations, and Future Research

Through this review, we propose that models of decision making should not only consider the evaluation of simultaneously available alternatives but also include explicitly the context/prior information gained from experience. In other words, we can speculate a multistage decision model that should segregate stimulus encoding, choice or action selection, and execution processes from that of computation of a decision variable. Furthermore, the outcome evaluation should be integrated back to update the prior for making future decisions.¹⁹ We can arrive at 2 kinds of models for decision making. First, a cognitive model needs to specify the various stages and their role in decision making. For example, previous literature has not attempted to tease apart whether top-down information influences stimulus encoding or choice execution stage. Future work needs to reconcile the decision process, i.e., computation decision variable with the parallel accounts of a common currency of valuation models. Second, a neural model can specify which regions of the brain participate in these different stages of decision making. Neural computation is an extremely complex phenomenon to be understood in terms of spatial segregation in the brain regions subserving various functions over time or in parallel. It might be possible that over different timescales, multiple functions are performed by a network of overlapping brain regions (see also Schultz et al, 2017). The study of decision making as presented in this review reflects a small subset of experiments suitable for a laboratory environment. Other research in the field of decision making has already addressed questions pertaining to the ecological validity of laboratory-based research.⁶⁶ Behavior measures such as choice made and response times are measures of the outcomes of a decision process. Computational modeling approaches are suitable for simulating the decision process resulting in behavioral outcomes. The computational modeling approach inherently has a crucial limitation that it can match the experimental observations but cannot make claims about the mechanisms underlying the cognitive phenomenon being investigated, which in our case was decision making. For ease of scientific investigations and rigor, cognitive phenomena are studied largely in isolation. By taking an example of decision making, this review appeals the need for a holistic and a system-level understanding of a cognitive phenomenon.