Modernizing conceptions of valuation and cognitive control deployment in adolescent risk taking

Abstract

Heightened risk taking in adolescence has long been attributed to valuation systems overwhelming the deployment of cognitive control. However, this explanation of why adolescents engage in risk taking is insufficient given increasing evidence that risk-taking behavior can be strategic and involve elevated cognitive control. We argue that applying the Expected Value of Control (EVC; Shenhav, Botvinick, & Cohen, 2013) computational model to adolescent risk taking can clarify under what conditions control is elevated or diminished during risky decision making. Through this lens, we review research examining when adolescent risk taking might be due to—rather than a failure of—effective cognitive control and suggest compelling ways to test such hypotheses. This effort can not only resolve when risk taking arises from an immaturity of the control system itself versus differences in what adolescents value relative to adults, but also identify promising avenues for channeling cognitive control towards adaptive outcomes in adolescence.

Risk-taking behaviors spike in adolescence relative to childhood and adulthood. Pervasive assumptions of why adolescents take risks have led to a mischaracterization of adolescents’ reduced capacity to exert cognitive control to accomplish goal-directed behaviors, resulting in many social, legal, and educational policies that place limits on the potential of adolescent decision making. There is growing evidence of the positive and adaptive benefits of adolescent risk taking for social growth and learning (Telzer, 2016), which suggests some risk behaviors may be strategic (Maslowsky, Owotomo, Huntley, & Keating, 2019) and involve a flexible implementation of cognitive control rather than failed cognitive control (Crone & Dahl, 2012). Despite the known involvement of cognitive control and valuation processes in adolescent risk taking, current theories fail to sufficiently delineate how these processes dynamically interact across various forms of risk behaviors.

In this paper, we argue that applying the Expected Value of Control (EVC) computational model (Shenhav et al., 2013) to adolescent risk taking can clarify how valuation and cognitive control processes are involved in risky decision making. We first describe the EVC model and map hypotheses about how various combinations of control and valuation computations can account for different types of adolescent risk taking. We then review preliminary research suggesting that certain forms of adolescent risk-taking behavior require elevated cognitive control. Finally, we offer future directions to test how cognitive control and valuation mechanisms may uniquely shape risky decision making during adolescence. Through this lens, we argue that adolescent risk taking can be adaptive, require control, or both, all of which challenge existing conceptions of the role of cognitive control in adolescent risk behavior.

Why Do We Assume Risk-Taking Behavior is a Failure of Cognitive Control?

Dual-process (e.g., dual systems, imbalance) theories, which pit fast, hot, impulsive processes against slow, cold, controlled processes, have been applied extensively to understand adolescent risk taking. Based on findings from developmental neuroscience research, these theories (e.g., Casey, Getz, & Galvan, 2008; Steinberg et al., 2008) propose that adolescents demonstrate heightened subcortical sensitivity (i.e., ventral striatum; VS) to socially rewarding contexts during a developmental period when the cognitive control system (i.e., prefrontal cortex; PFC) is still maturing. The imbalance between heightened reward sensitivity and immature cognitive control is thought to bias adolescents towards risky decisions. This perspective suggests that an “unchecked” subcortical system is maladaptive, rendering adolescents ineffective at engaging cognitive control, particularly in socioemotionally “hot” contexts.

Despite their appeal and utility as a heuristic tool for understanding brain development, dual-process theories are limited in explaining the diversity of adolescent risk taking (Pfeifer & Allen, 2012, 2015). By pitting control systems against valuation systems, such theories assume that risk taking occurs because of an internal failure of cognitive control. Dual-process theories therefore are prone to conflating process with outcome, inferring the former from the latter. Moreover, these theories do not describe how control is engaged nor do they consider that risk-taking behaviors may be adaptive and arise from elevated control. For instance, if a risk-taking behavior is not habitual (e.g., trying a drug for the first time), the enactment of cognitive control may be necessary to override the habitual response of avoiding drugs to engage in the behavior that may be valued (e.g., social approval from the peer group). This propensity to seek risk via the enactment of cognitive control might underlie a range of exploratory behaviors that promote learning in adolescence. Furthermore, risk-taking behaviors do not always involve elevated control, such as when risky behavior becomes so routine that it gradually demands less or no cognitive effort (e.g., those who often take risks to explore unknown aspects of their environment may require less cognitive control to do so over time). To refine our understanding of the role of cognitive control in adolescent risk taking, it is vital to clarify under what conditions, and for which individuals, cognitive control is elevated or diminished across different types of risk-taking behaviors.

Modernizing Conceptions of Cognitive Control in Adolescent Risk Taking

A Brief Introduction to EVC

The EVC model offers a promising reconceptualization of the role of cognitive control in adolescent risk taking by modeling what computations are involved in control itself and under what conditions control is engaged (Shenhav et al., 2013). Cognitive control is an internal process that supports the deployment of behavior that is advantageous towards achieving some goal, but is taxing in that it must override engaging less effortful behavior (e.g., habits). EVC defines what must be evaluated to engage cognitive control, a process that carries an inherent cost (Inzlicht, Schmeichel, & Macrae, 2014; Shenhav et al., 2017). Specifically, the dorsal anterior cingulate cortex (dACC) is proposed to estimate the expected value associated with allocating control to one task over another by integrating information about one’s environmental and internal states (e.g., motivation), as well as the relative values of the reward(s) one will garner if enacting various behaviors based on input from cortico-subcortical circuits (e.g., ventromedial PFC, VS) that project to the dACC (Shenhav et al., 2013). This expected value determines 1) whether it is worth investing control in a given task; 2) how much control should be invested in a given task; and 3) when several candidate control signals are in contention, which is the most worthwhile (Shenhav et al., 2013). Control is engaged when its exertion is estimated to garner more reward than engaging behaviors requiring less or no control. The EVC mechanism is considered a “meta-controller” system because it evaluates the cost of control itself by evaluating the computational resources involved in control. It does so through estimating the costs of different signal identities (i.e., targeted response) and the intensity of such signals in order to deploy the optimal control signal that maximizes reward. As such, EVC is a particular instance of a value-based control system (Berkman et al., 2016), but is distinguishable from other value-based controllers because it requires a unique computational and neural architecture to evaluate when the computational cost of deploying control is beneficial (Shenhav, 2017). Given its added specificity in formalizing relations between control and valuation computations in the brain, the EVC model offers a new suite of computationally precise, testable hypotheses that can be used to specify the boundary conditions by which various risk behaviors arise from elevated control rather than a lack thereof, and how value can modulate the flexible deployment of cognitive control.

EVC is motivated by the larger literature on how adolescents learn to maximize the rewards they gain through adaptive decision making. Adolescents can learn which action is most rewarding through trial and error (i.e., model-free learning) or by forming a larger cognitive model of potential actions and their consequences (i.e., model-based learning) (Daw, Niv, & Dayan, 2005), the latter of which is particularly important for facilitating flexible behavior. Compared to adults, adolescents show enhanced reinforcement learning (e.g., faster and more optimal choices; Davidow et al., 2016) and are better able to learn objectively from feedback when confronted with biased and inaccurate information (Decker, Lourenco, Doll, & Hartley, 2015). Adolescent-specific benefits in reinforcement learning are associated with elevated reward sensitivity in dopaminergic systems (e.g., VS) (Davidow et al., 2016; Decker et al., 2015), which are important for assigning value to potential responses (i.e., expected value) and updating them based on feedback or experience (i.e., prediction error). Indeed, adolescents are better than adults at maximizing the rewards associated with risky choices via tracking changes in the expected value of different choices (Barkley-Levenson & Galván, 2014), which is likely influenced by their bias toward encoding experienced outcomes that are better—rather than worse—than expected (i.e., positive prediction errors; Cohen et al., 2010; Davidow et al., 2016). Although previous research suggests elevated reward sensitivity during adolescence can facilitate flexible, goal-directed behaviors, the internal processes that are computed before a decision is made are still conflated with the enacted (risk) behavior.

Situating Theories of Adolescent Risk Taking within EVC

The EVC model can enrich our understanding of why adolescents engage in risk taking by specifying the conditions under which cognitive control is attenuated or elevated in risk taking, and how valuation modulates the engagement of cognitive control. EVC does not conflate risk behavior (e.g., choosing an uncertain outcome with a potential loss) (Figure 1A) with internal processes of valuation and control (Shenhav, 2017). Control is engaged when the effortful risk behavior is estimated to garner more reward than a less effortful or habitual response (Figure 1B). As such, risk behavior could result from elevated control if engaging in that risk behavior requires more effort than enacting non-risky behavior (see Figure 1C for examples of control-contingent risk taking). This contrasts with the oft-discussed scenario captured by dual-process models, in which risk behavior that is non-effortful (e.g., talking to a peer while driving) emotionally overwhelms control systems that could deploy non-risky behaviors (e.g., maintaining focus on driving in the presence of peers; Chein, Albert, O’Brien, Uckert, & Steinberg, 2011).

Figure 1. The Expected Value of Control (EVC) model applied to adolescent risk taking.

(A) Reconceptualizing the role of cognitive control in adolescent risk-taking behaviors using the Expected Value of Control (EVC) model. The deployment of cognitive control during risk taking depends on how effortful the risk behavior is relative to others that might be enacted, such that control is required when the behavior is more effortful and overrides a habitual response. Here, we depict how the riskiness of a behavior is orthogonal to how effortful it is. Two examples of adolescent risk-taking behaviors differ in terms of how effortful they are to deploy: acting impulsively requires little control, whereas choosing to strategically explore one’s environment is effortful and thus requires elevated control.

(B) Using EVC to understand how value guides a flexible deployment of cognitive control. In the EVC model, valuation is part of—rather than separate from—the evaluation process of when and how much control should be engaged in a given risk context. EVC is computed for each response, which comprises the estimated future reward one will receive after the response minus the estimated costs of deploying control to engage the response. If the estimated future reward for a risky behavior, including the estimated cost of deploying control to engage that behavior, is greater than a less effortful non-risky behavior, control is deployed and the risk behavior is enacted. Here, the thickness of the arrows reflects the strength of control signals, with larger signals overriding smaller signals.

(C) Mapping two examples of control-contingent risk-taking behaviors onto the EVC model. Risk-taking behavior that requires control is engaged when the value of such effortful risk-taking is greater than that of a more habitual response, which can arise from 1) top down changes in valuation (e.g., social norms) or 2) bottom-up changes in valuation (e.g., propensity to seek novel experiences).

It is vital to acknowledge how the relationship between valuation and cognitive control systems as characterized by EVC differs fundamentally from current models of adolescent risk taking. In EVC, valuation is part of the evaluation process of when to engage control and how much control should be invested, which can arise from top-down (e.g., requiring language or memory to inform what one values; top-down value differential) or bottom-up (bottom-up value differential) propensities to estimate greater reward from certain choices over others (Figure 1B). This contrasts with current models of adolescent risk taking in which cognitive control processes are separate from—and must subdue—bottom-up valuation systems (e.g., Strang, Chein, & Steinberg, 2013).

Control-Contingent Risk-Taking Behavior in Adolescence

The EVC model provides a new conceptual framework for testing the conditions in which control is necessary for adolescent risk taking and how value can guide a flexible deployment of cognitive control. One example of control-contingent risk taking that is common during adolescence is exploring unknown aspects of an environment, which involves more effort than acting on the basis of one’s current knowledge (exploration vs. exploitation) (Daw, O’Doherty, Dayan, Seymour, & Dolan, 2006). Compared to adults, adolescents have a greater propensity to explore the unknown aspects of their environments, which is motivated by the added value of acquiring new information, learning about the consequences of alternative behaviors, or determining the causal structure of their environment (Crone & Dahl, 2012; Gopnik et al., 2017; Kayser, Op De Macks, Dahl, & Frank, 2016; Nussenbaum et al., 2019). For example, adolescents are more risk seeking than adults when risks are unknown, but are more risk averse than adults when risks are known (Tymula et al., 2012), suggesting that adolescents strategically engage in exploratory behavior when there are more unknown risks in the environment. In fact, adolescents are better than adults at drawing on the information they acquire from exploring their environments to improve future choices (Davidow et al., 2016; Decker et al., 2015), particularly in social domains (Gopnik et al., 2017) or when incentivized by larger rewards (Teslovich et al., 2014). In contrast, older individuals pursue a narrower hypothesis search when exploring their environment and tend to maintain these beliefs even in the face of evidence inconsistent with their hypothesis (Gopnik et al., 2017; Nussenbaum et al., 2019).

There are many individual differences in estimating the benefits and costs of exploring one’s environment. One factor influencing how much control to engage in risky exploratory behavior is prior knowledge. When prior knowledge is low for a given domain, exploratory behavior could be engaged because the potential information gain outweighs the small knowledge one already has for the safer alternative behavior. For example, adolescents may be more willing to engage in risky exploratory behavior if they have less extensive prior knowledge about a particular social domain. A major advantage of the EVC model is its ability to integrate prior knowledge of how cognitive control brings about reward (e.g., in the past, did the adolescent get more reward by exploring over exploiting?), arbitrate between competing control-contingent signals (e.g., should the adolescent explore environment A vs. B?), and select the control-contingent response that maximizes the EVC (e.g., adolescent explores environment A).

Improving an Understanding of How Adolescents Enact Risky Behavior using EVC

The need for, and intensity of, control during exploration also depends on whether adolescents use random or directed exploratory strategies. Random exploration requires less control than directed exploration, because the former implements a simple algorithm to randomly sample from recently unchosen options, whereas the latter must take into account ongoing estimates of the informational value of unchosen options, and select the behavior which maximizes information gain (Somerville et al., 2017). Moreover, directed exploration is sensitive to the amount of information one can pursue (e.g., how much longer one has to solve a problem), which is hypothesized to engage other computationally intensive decision algorithms (Gershman, 2017; Somerville et al., 2017). Emerging evidence suggests that directed exploration does not emerge until adolescence, perhaps due to ongoing maturation of corticostriatal reward circuitry that constrains the use of more sophisticated exploratory strategies (Decker, Otto, Daw, & Hartley, 2016; Somerville et al., 2017).

EVC can guide efforts to delineate under what conditions directed and random exploration in adolescent risky behavior engages cognitive control relative to pursuing safe behavior that require less control. For instance, engaging in directed exploration even under the risk of an aversive outcome may ultimately be optimal for acquiring new information, a strategy that is likely exaggerated in adolescents relative to adults. EVC can formalize to what extent adolescents opt for these behaviors in such environments, and outline neural dynamics that would distinguish this control-contingent risky behavior (exploration) relative to exploiting knowledge about a safe, but potentially less rewarding, option. Along similar lines, using a random exploration strategy might be more or less effortful depending on how automatic a competing, non-risky behavior is. Importantly, because the deployment of cognitive control in risk taking is comprised of complex estimations about the level of reward, risk, and effort involved in one candidate response relative to others, it is imperative to formalize these properties on a continuum rather than a dichotomy in order to appropriately capture individual differences in the control demands associated with various risky behaviors.

Although we have focused on exploration as an instance of risky behavior that may require elevated control, adolescents can, under the right motivational incentives, engage in other types of control-contingent risk behaviors (Crone & Dahl, 2012; Pfeifer & Berkman, 2018). Adolescents can more flexibly exert cognitive control, for instance, when social rewards can be accrued by engaging cognitive control (Gopnik et al., 2017; Hardin et al., 2009). Thus, control-contingent risk behaviors unique to adolescence may be particularly evident in contexts wherein social rewards can only be optimized if one risks exposing oneself to a large loss. Unlike adults that have a Pavlovian bias to avoid large risks even when doing so is maladaptive (Huys et al., 2012), adolescents may be less susceptible to these biases and more adept at deploying control appropriately even in high-risk environments (Decker et al., 2015), especially in the context of social rewards. One speculative but interesting example of this could be adolescents’ decision to take risks to benefit their peers (i.e., prosocial risk taking; e.g., standing up to a bully) (Do, Guassi Moreira, & Telzer, 2016), which may involve elevated control to face an immediate loss (e.g., physical harm or reputational costs) in order to ultimately accrue a larger gain (e.g., peer acceptance).

Although the type and weight of value inputs that influence the EVC estimation of candidate responses may differ in adolescents compared to adults, we speculate that the iterative process by which the need for control is signaled (i.e., control signal monitoring and specification) is similar across development and relies on a common set of mechanisms that is anchored in the dACC (Shenhav et al., 2013). However, we hypothesize that the process by which specified control signals are implemented (i.e., control regulation) shows robust age-related changes from adolescence into adulthood, perhaps due to ongoing maturation of corticostriatal circuitry that constrain the integration of value, goals, and actions (Casey, Heller, Gee, & Cohen, 2019; Davidow, Insel, & Somerville, 2018). Unpacking the components of cognitive control detailed in EVC will likely play a key role in explaining the diversity of risk-taking behaviors during adolescence.

Improving Tasks Designed to Measure Cognitive Control using EVC

Currently, many experimental paradigms designed to investigate adolescent risk taking are not optimized for understanding how cognitive control is deployed. Although traditional risk-taking paradigms, such as the Balloon Analog Risk Task (Lejuez, Aklin, Zvolensky, & Pedulla, 2003) or Columbia Card Task (Figner, Mackinlay, Wilkening, & Weber, 2006) have formalized important sub-components of adolescent risk taking (e.g., expected value), they conflate decision processes with the outcome of risk taking and cannot distinguish when risk taking is habitual (no control is needed to exploit known rewards) or effortful (control is required to explore the environment for potentially greater rewards). To test EVC in adolescence, future experimental paradigms should pit putative habitual behaviors against those known to require elevated control and effort to infer how cognitive control is deployed in risky behavior (e.g., Cohen, Dunbar, & McClelland, 1990). Such tasks are enriched by defining when and under what conditions risky behavior is beneficial to the adolescent, so as not only to identify when control is engaged, but also when it is adaptive (e.g., Kayser et al., 2016). Indeed, there could be circumstances in which adolescents engage too much control during risk taking. For instance, adolescents relative to adults might engage unnecessarily elaborative model-based reasoning (which requires elevated control) in social evaluation contexts to impress peers, which in turn increases risky behavior. Another promising direction is to build on recent work (e.g., Somerville et al., 2017) to provide clear incentives (monetary or social rewards) for effortful and non-effortful behaviors. Experimental tasks informed by the EVC model can better measure the degree to which value differences influence the strength and timing of cognitive control deployment. This effort can help inform the enterprise to channel cognitive control towards adaptive behaviors.

Conclusions

Despite pervasive assumptions regarding the failure of cognitive control in explanations of adolescent risk taking, certain instances of adolescent risk taking may require elevated cognitive control. We argue that applying the EVC computational model (Shenhav et al., 2013) to the domain of adolescent risk taking can help elucidate when risky behaviors result from elevated or diminished control. Control-contingent risk behaviors may increase in adolescence due to top-down or bottom-up changes in valuation that are associated with social (e.g., peer norms) or biological (e.g., puberty) markers of development. Ultimately, a better understanding of how control mechanisms are involved in adolescent risk taking can help us redirect adolescent risky behavior away from maladaptive outcomes (e.g., substance use) and towards adaptive outcomes (e.g., learning).