A Scale-Free Systems Theory of Motivation and Addiction

Abstract

Scale-free organizations, characterized by uneven distributions of linkages between nodal elements, describe the structure and function of many life-based complex systems developing under evolutionary pressures. We explore motivated behavior as a scale-free map toward a comprehensive translational theory of addiction. Motivational and behavioral repertoires are reframed as link and nodal element sets, respectively, comprising a scale-free structure. These sets are generated by semi-independent information-processing streams within cortical-striatal circuits that cooperatively provide decision-making and sequential processing functions necessary for traversing maps of motivational links connecting behavioral nodes. Dopamine modulation of cortical-striatal plasticity serves a central-hierarchical mechanism for survival-adaptive sculpting and development of motivational-behavioral repertoires by guiding a scale-free design. Drug-induced dopamine activity promotes drug-taking as a highly connected behavioral hub at the expense of natural-adaptive motivational links and behavioral nodes. Conceptualizing addiction as pathological alteration of scale-free motivational-behavioral repertoires unifies neurobiological, neurocomputational and behavioral research while addressing addiction vulnerability in adolescence and psychiatric illness. This model may inform integrative research in defining more effective prevention and treatment strategies for addiction.

1. Introduction

Rapid gains in understanding the neurobiology of addiction (Nestler, 2004; Volkow, 2004b) have been possible in large part because the disease model is particularly amenable to scientific investigation. In contrast to other psychiatric disorders, the primary inciting agent (the drug) is tangible, and core clinical phenomenology (drug self-administration) can be objectively measured in animals or humans. Nonetheless, addictions remain the leading cause of preventable medical morbidity and mortality in the United States (Services, 1994; NORC, 1999) and rank among the most difficult to treat and highly stigmatized mental disorders. Thus, addiction research arguably holds the greatest but most unrealized promise in utilizing a leading edge of psychiatric neuroscience to address a tremendous public health problem.

As in all areas of translational neuroscience, addiction research faces tremendous obstacles of biophysical complexity. The brain expresses up to 10-fold more genes than other organ systems (Yu and Rasenick, 2004) and collects, processes, stores, and manufactures neural-encoded information—a product that is considerably elusive to biological investigation. In parallel to this complexity, large volumes of data concerning the genetic, cellular, neural circuit, developmental, behavioral-clinical, and epidemiological aspects of addiction have been collected but remain poorly integrated. Paramount to a deeper and more comprehensive understanding of addictive disease, translational theories that bind and understand the causal relationships between these observational domains are needed (Volkow, 2005).

Non-biological fields such as mathematics, engineering, physics and computer science have significant potential for addressing the extreme complexities of addiction. These fields routinely develop and employ models of structure and process that integrate and organize seemingly disparate data as a means to ‘see the forest through the trees’. This paper applies one such mathematical approach to addiction: the study of scale-free structural maps. The realization that many complex biological and psychosocial systems conform to a scale-free organization, in which a few nodal elements within a structure are richly interconnected, while most have few connection links, has already achieved broad attention and applicability in biomedical science (Khanin and Wit, 2006). Here, we explore how re-defining motivated behavior as a complex scale-free structure provides translational scaffolding that bridges neurobiological and behavioral levels of observation in the pathology of addiction. At the core of this application, a novel perspective emerges on how cortical-striatal circuits, as modulated by dopamine (DA) neurotransmission, organize and adapt motivated behavior via the management and revision of sequential neural representations. According to a scale-free organizational plan, such information management would entail decision-making and habit formation capacities as features of motivational processing increasingly implicated in addictive disease (Bechara, 2003; Bickel and Johnson, 2003; Yin and Knowlton, 2006). This neuroinformatic aspect of the scale-free application incorporates contemporary views of addiction as a disease of motivational process (Kalivas and Volkow, 2005) and dopamine-mediated learning (Hyman, 2005) into a broader model that links neurobiological, developmental, behavioral, clinical and epidemiological data in the study of addiction.

2. Scale-free networks: Structure, properties and natural examples

Complex networks defined by nodal elements and their connections are ubiquitous in nature (Kitano, 2002; Fewell, 2003; Jasny, 2003). Molecules in a volume of gas may be considered nodal elements making random electrostatic contact with other molecules per unit time. In the brain, neurons are related via axodendritic connections (Laughlin, 2003), while individuals are connected by social relationships (Fewell, 2003). On larger scales, cities can represent nodes interconnected by highways or flight routes (Jasny, 2003).

Until recently, it was often assumed that natural complex networks are defined by random distributions of connections (Erdos and Renyi, 1960; Bollobas, 1985; Amaral et al., 2000). In such networks, a randomly sampled node has the greatest probability of having a total number of connections k equal to the mean number of connections per node in the network. Very few nodes have substantially smaller or larger numbers of connections than this mean. The connection structure of these ‘random’ or ‘exponential’ networks are depicted by a probability distribution revealing the bell shaped Poisson distribution (Barabasi and Bonabeau, 2003) where the proportion of nodes bearing significantly fewer or greater connections than the mean connections/node in the network declines exponentially as e^-k.

As indicated in the work of Barabasi and colleagues, real complex networks are often not best defined by a random connection structure (Barabasi and Bonabeau, 2003). Instead, they may be better classified as scale-free, where most nodes have a relatively low number of connections, but an increasingly exclusive minority have an increasingly high degree of connectivity (Figure 1) (Barabasi and Albert, 1999). In this architecture, the probability (frequency) of identifying nodes having k connections follows a power law distribution k^-γ, where γ is usually identified as 2<γ <3 in real-world systems (Barabasi and Albert, 1999; Barabasi and Bonabeau, 2003). In scale-free as opposed to random networks, a small but significant number of nodes (termed ‘hubs’) will have numbers of connections far exceeding the limited scale determined by a small range around the mean connections/node in the network (hence the term ‘scale-free’). These networks are also ‘free-of-scale’ in the sense that regardless of what observational scale the network is being viewed (i.e. whether you are looking at smaller or larger portions, or the whole network), examination of sub-regions will reveal self-similarity, or constancy, in the overall hierarchical organization. That is, any reasonably sized sub-region will have the same scale-free connection distribution as that of the whole network (Barabasi et al., 2001).

Figure 1.

Frequency distribution of nodes by number of connections/node in scale-free vs. random networks.

Different connection structures of random vs. scale-free networks are associated with different functional properties. First, scale-free networks are more efficient in terms of pathways of interactions between non-directly connected nodes. In a random network, the mean number of connections traversed in going from any one node to any other (‘network diameter’) is much larger than for a scale-free network with a similar number of nodal elements (Barabasi and Bonabeau, 2003; Cohen and Havlin, 2003; Kim et al., 2003). This property emerges from the ability of the few highly connected hubs of the scale-free network, which are essentially non-existent in a random network, to act as way stations between much larger collections of elements that are not mutually interconnected (Shargel et al., 2003). Second, scale-free networks are more robust to, or tolerant of, the elimination of connections associated with the random loss of nodes (Barabasi and Bonabeau, 2003; Shargel et al., 2003). That is, if nodes (and their associated connections) are randomly eliminated, the overall scale-free structure tends to be self-maintaining, and network diameter does not substantially decrease with significant proportions of nodal losses. In contrast, random networks are more susceptible to random nodal losses, where network diameters are readily increased and subsets of nodes are more susceptible to becoming completely isolated from the rest of the network. Third, and conversely, scale-free networks are more highly vulnerable to non-random, targeted elimination of the fewer highly interconnected nodes, where the loss of these hubs can rapidly produce a catastrophic breakdown in global network connectivity (Kim et al., 2003; Shargel et al., 2003).

In investigations spanning the microscopic to macroscopic continuum and diverse scientific fields, a variety of naturally occurring complex networks have been described as scale-free. Within the cell, several molecular-based forms of scale-free networks exist (Bray, 2003). Metabolic pathways involve networks of enzymes that are linked by common substrates; most enzymes interact with only a few substrates, but a vital minority interacts with a very large number (Jeong et al., 2000; McAdams, 2003). Networks of proteins and their interactions have similarly been described as scale-free (Jeong et al., 2001; Qian, 2001; Bray, 2003). The known universe of protein structures is definable as a scale-free system where protein families are related by common structural folds: the number of families with increasing numbers of structural folds diminishes with a power law (Koonin et al., 2002). On still higher levels, functional networks of the brain may be co-activated in a scale-free manner (Eguiluz et al., 2005). Animal and human social networks also conform to scale-free structures (Fewell, 2003). For instance, in dolphin societies, most dolphins interact with only a few peers, while a few interact with many (Lusseau, 2003). The same has been observed for human social interaction or communication including linkages defined by sexual contacts (Liljeros et al., 2001; Schneeberger et al., 2004), motion-picture acting relationships (Albert and Barabasi, 2000), scientific publication co-authorship (Bilke and Peterson, 2001) internet links (Albert et al., 1999) and sequential word usage in human language (Ferrer et al., 2001).

The applicability of scale-free models to diverse real-world phenomena appears related to two major issues. First, it is possible to apply the structure of nodes and their connections to a wide variety of objects and their relationships. Increasingly powerful methods of data gathering and analysis facilitate investigations into the degree to which specific systems conform to scale-free architectures (Barabasi and Bonabeau, 2003). Second, studies of both artificial and real-world scale-free networks indicate a fundamental theme of their formation: they are self-organizing in a manner consistent with processes identified in natural evolution (Gibson and Honeycutt, 2002; Koonin et al., 2002; Aldana and Cluzel, 2003). Therefore, complex natural systems, particularly those composed of elements that result from biological processes, commonly conform to scale-free architectures (Wolf et al., 2002).

Unlike random networks, the formation of scale-free networks requires both the addition of new nodes over time and their preferential attachment to other more highly connected nodes in the network (Barabasi and Albert, 1999; Jeong et al., 2003). Thus in a generic scale-free network, the probability (π) that a newly introduced node will form a connection with another node i in the network is

$$\Pi \left({\mathrm{k}}_{\mathrm{i}}\right)=\frac{{\mathrm{k}}_{\mathrm{i}}}{\Sigma {\mathrm{k}}_{\mathrm{j}}}$$

where k_i is the number of already existing connections associated with node i, and Σ k_j is the total number of connections already present in the network (Barabasi and Albert, 1999). Under these conditions, the status of the more highly connected hubs of the network will be maintained or accentuated with network growth analogous to the ‘rich-get-richer’ concept (Barabasi and Bonabeau, 2003). Although variations on this general formula account for real-world possibilities such as competition between nodes for connections (Bianconi and Barabasi, 2001), the core aspect of preferential attachment as most succinctly expressed above appears to be necessary for scale-free system formation.

Preferential attachment has two major implications as a developmental and evolutionary process. First, it allows objects serving as nodes in a given evolving network (e.g. enzymes in a metabolic network of an evolving unicellular organism) to gain an ever-increasing diversity of connections (e.g., exert enzymatic effects on an increasing number of substrates). In this manner, greater adaptive and organizational efficiency occurs as functionally vital network objects (hubs) evolve to take on new roles (Jeong et al., 2000; Jeong et al., 2001; Ravasz et al., 2002). Such interpretations of the scale-free organization of molecular constituents of various species have been proposed to indicate the evolutionary age of different molecular substrates within or across species, and the degree of evolutionary relatedness between different species (Jeong et al., 2001; Podani et al., 2001; Koonin et al., 2002). Second, the developmental organizing principals of scale-free systems endow them with functional robustness to random degradation as a natural pressure in evolution (Aldana and Cluzel, 2003; Barabasi and Bonabeau, 2003). Indeed, just as many biological and other real-world scale-free networks show connection structures defined by a power law distribution k^-γ, where γ often empirically ranges between 2 and 3 (Barabasi and Albert, 1999; Jeong et al., 2000; Barabasi and Bonabeau, 2003), simulated scale-free systems using values of γ ranging beyond these limits show loss of network robustness to random degradation, suggesting that evolutionary forces determining survival fitness bound this parameter (Aldana and Cluzel, 2003; Cohen and Havlin, 2003; Kim et al., 2003).

3. Scale-free maps as an organizational structure of motivated behavior

3.1 Motivational-behavioral repertoires

Based on contemporary perspectives in behavioral neuroscience (Berridge, 2004; Wise, 2004) we define motivation as neural processes that determine the prioritization, ordering and performance of specific behaviors required to achieve a specific goal(s). Motivation entails multiple, non-mutually-exclusive constructs. One is decision-making, in which multiple options of specific behaviors, their sequences, or their associated goals may be pursued. A second is will, or intent, strategy or plan. This component is experienced subjectively prior to or during the performance of goal-oriented behavior, but is only objectively confirmed upon performance of the goal-oriented behavior. From an evolutionary perspective, motivational structures underlying behaviors promoting survival and reproduction are selected. From a behavioral economic perspective (Glimcher and Rustichini, 2004), time (e.g. lifespan of the individual) and physical capability constrain the collection of motivations and related behaviors to a large but finite resource collection for any one individual—the motivational behavioral repertoire (Bickel and Marsch, 2000; 2001).

3.2 Development of motivational-behavioral repertoires as scale-free maps

As suggested by Berridge, re-conceptualizing motivation is a vital process for interpreting new neurobiological data in terms of observable behavior (Berridge, 2004). In this section, we synthesize an earlier iteration in this process discussed by behaviorists in the 1960′s, with modern concepts of motivation as defined above. Here, motivation and behavior are couched in terms of discrete units that are inter-related in a complex but finite system comprising an individual’s motivational-behavior repertoire.

In work describing social behavior of rats in the early 1960′s, Grant and Mackintosh characterized a diverse but finite repertoire of discrete, recurring behavioral motifs displayed in sequences by rats during social interactions (Grant, 1963; Grant and Mackintosh, 1963). These behavioral repertoires have been studied as mechanisms of socio-affective communication relating to fear, aggression, territorial bargaining, sexual attachment, and dominance hierarchies (Takahashi, 1990; Sajdyk and Shekhar, 1997; Panksepp, 1998a; Insel, 2003; Sheehan et al., 2004). In an effort to characterize a system of rat social behavior based on hundreds of empirical observations, Grant labeled discrete, identifiable sets of social behavior and plotted them as nodes in a network (Grant, 1963). The connections in this network represent the highest empirical probabilities that one social behavior would follow another in a temporal sequence (Figure 2A, reprinted with permission). Although scale-free topologies would be described 35 years after Grant’s work, his data are organized in a way that is amenable to evaluation as a scale-free system. We have re-assessed these data as a frequency histogram of behavioral nodes categorized by the number of connections per node (Figure 2B). Connections to or from behavioral nodes were analyzed separately to account for the temporal flow of behavior. Although the small number of nodes (23) precludes definitive characterization (simulated and empirically-defined scale-free systems usually encompass hundreds to thousands of nodes), the connectivity architecture of Grants’ social behavioral repertoires is suggestive of a scale-free like organization. Most behaviors are sequentially connected to only 1 or 2 others, and a more limited but significant set of behavioral nodes serve as hubs. A curve fitting approximation over the nodes with k ranging from 1 to 5 connections/node gives a connection probability distribution P(k)=k^-γ where γ=1.71 for connections to nodes and γ=1.88 for connections from nodes. Both of these values approach the typical range of 2<γ ≤ 3 observed in much larger, empirically verified scale-free systems. In addition, some of the most highly connected hubs in this system (e.g. ‘retreat’) may represent the most vital nodes in the social behavioral repertoire from a survival standpoint.

Figure 2.

(A) Pathway diagram showing the most probable occurrences and sequential flow of social behaviors in the rat (Reprinted by permission, (Grant, 1963)). U.P.-Upright Posture; O.U.P - Offensive Upright Posture; D.U.P-Defensive Upright Posture; S.P.-Sideways Posture; O.S.P.-Offensive Sideways Posture; D.S.P.-Defensive Sideways Posture; Agg. Post.-Aggressive Posture; Sub. Post. -Submissive Posture; Dist. Amb.-Distance Ambivalence; Disp. Act.-Displacement Activities. (B) Frequency distribution of behavioral nodes (social behaviors) by numbers of connections to, or from nodes, approximates a scale-free organization.

While not intended as a proof that complex behavioral data conforms to a scale-free organization, this re-examination of Grant’s data illustrates two important points. First, it shows a concrete approach for mapping behavioral data in terms of a complex network that may be assessed for scale-free motifs. Second, to the extent that Grant’s social data might be organized as a scale-free structure, social behavior could represent a local neighborhood embedded within a much larger system encompassing the entire behavioral repertoire (including non-social behavior) that is itself scale-free. This notion follows from the property of self-similarity of scale-free systems in which appropriately sized samples of sub-regions within the greater system are similarly scale-free. These concepts lead to the central thesis of this manuscript. Specifically, we postulate that the very large but finite set of behaviors encompassing the entire behavioral repertoire of the individual can be understood as a complex map of nodes (discrete observable behaviors) connected by links implied by the temporal sequencing of discrete behaviors. Motivation is formalized as the connection pathways linking discrete behavioral nodes, and the entire set of links and/or pathways in this network encompass the individual’s motivational repertoire. Motivational-behavioral repertoires are then theorized to follow scale-free organizational motifs in which motivational pathways are distributed between behaviors approximating a non-random power law distribution P(k)=k^-γ, and γ may approach or lie within the range (2,3).

Understanding behavioral repertoires as hierarchical organizations of discrete behavioral units was of general interest to leading experimentalists and theorists in the 1960′s (Lorenz, 1965; Skinner, 1966). A central dilemma of this work was how one might define a ‘behavioral unit’ or develop a conventional language for a system of behavioral units. For example, is an individual conducting a ‘recreational behavior’ as a behavioral unit, or is she ‘playing golf’? When playing golf is she linking the unit behaviors of ‘club selection’, ‘swinging at the ball’, ‘walking to the ball’, etc., or is she linking even more elemental behaviors such as the numerous cognitive and motor steps of a single swing at the ball? McFarland addresses this issue in work proposing a hierarchical mapping of discrete behavioral activities according to motivational factors. He suggests that an experimentally viable system of discrete behavioral units can be arrived at without the need for adherence to universal criteria concerning functionality, or scale of classification, as long as the system is internally consistent and defined by mutually exclusive and discretely observable categories of behaviors (McFarland and Sibly, 1975). Once such a system is adopted, it may also be viewed in terms of behavioral evolution. According to Skinner, “The entire repertoire of an individual or species must exist prior to ontogenic or phylogenic selection, but only in the form of minimal units. Both phylogenic and ontological contingencies ‘shape’ complex forms of behavior from relatively undifferentiated material.” (Skinner, 1966).

Based on these ideas, we can conceptualize the development of motivational-behavioral repertoires as the growth of a scale-free like system, as exemplified here around three general behavioral domains (Figure 3). From ontological or phylogenic perspectives, spheres of behavior encompassing 1) neurovegetative/homeostatic (e.g. eating), 2) social (e.g. mating), and 3) motor/instrumental (e.g. ambulation) are core requirements of survival in animals and are genetically encoded. They manifest at birth as a small set of primitive forms but are elaborated upon, grow and diversify through development across the lifespan (Figure 3A). The notion that these behaviors emerge according to a scale-free developmental principal is consistent with contemporary theories of psychomotor development as a self-organizing process (Sporns and Edelman, 1993; Thelen, 1995).

Figure 3.

Conceptual schematic of the ontological development of motivational-behavioral repertoires according to a scale-free organizing principal. Functional spheres and emerging behavioral nodes that may be classified as closely related to these functional spheres are correspondingly color-coded. Only a small number of the much larger number of nodes comprising real behavioral repertoires are depicted here for conceptual clarity. In the childhood and adult systems, each graphically represented node may also be viewed as representing a local scale-free neighborhood of many interrelated motivational links and behavioral nodes. Fundamental survival-dependent spheres of function (A) emerge in the form of the behavioral repertoire of the newborn (B). Into and throughout childhood (C), the emergence of numerous new skills and behaviors are hierarchically clustered around key behavioral hubs that subserve fundamental survival functions. In adolescence/adulthood (D), new behavioral nodes form from integrative events between behavioral nodes of childhood origin, and motivational repertoires are re-aligned according to the exploration and mastery of adult social, sexual and occupational roles.

In the newborn, fundamental spheres of behavior manifest early as discrete behaviors typical of the limited and undifferentiated, but developmentally appropriate behavioral repertoire of the infant (Figure 3B). These few discrete behavioral nodes are interconnected by links representing the sequential progression of one discrete behavior (e.g. nursing) to another (e.g. sleep). Infantile motivation may be conceptualized as the links or pathway of links between nodes (e.g. crying followed by nursing may be viewed as hunger motivation). Certain behaviors (e.g. random limb movements, babbling) might seem meaningless and random. Yet these behaviors represent the buds, or initial scaffolding, of multiple and more elaborate behaviors that emerge during the substantial expansion of behavioral repertoires throughout childhood (Frossberg, 1999).

Into and through childhood (Figure 3C), as forebrain circuits develop (Hughlings-Jackson, 1958; Gallistel, 1980), the emergence of thousands of new skills and behaviors creates a system of sufficient numbers of behavioral nodes whereby hierarchical clustering can occur. Such structuring reflects the self-organizing nature of psychomotor development, produced by an interaction between neurobiological-developmental events and multiple repetitions of sensory-motor interactions with the environment that favor the emergence of the most functional behaviors (Sporns and Edelman, 1993; Thelen, 1995). In the terminology of Piaget behavior becomes increasingly differentiated: emerging sets of behavioral nodes are often the ontological and functional descendents of infantile behavioral nodes (Yates, 1996). Many new behavioral nodes may also represent the fusion of already developed behavioral nodes that had previously occurred separately. Hierarchical structuring within the system emerges as certain behavioral nodes are performed more frequently than others by virtue of being linked with a greater variety of other functionally vital behavioral nodes. These hubs are critical elements within the behavioral repertoire because they represent core elements in a local neighborhood of related behaviors directly mediating survival (e.g. eating lunch), or because they link functionally different neighborhoods and/or spheres of behavior (e.g. eating lunch with friends links neurovegetative and social spheres). As in the newborn, the links between behavioral nodes represent the temporal progression from one activity to another, and motivational repertoires entail pathways of such links. Although many behaviors and the motivational pathways between them may not have an immediate functional/survival consequence (e.g. play, elementary school reading and writing), they represent the foundation upon which the adult motivational-behavioral structure will be built. The daily activities of the child may be conceptualized as a traversing of the nodal network of behaviors along motivational pathways, much of which will include play and learning behavior within the social and motor/instrumental spheres (Panksepp, 1998b).

Through adolescence and young adulthood, the structure of the motivational-behavioral repertoire continues to develop according to trends set forth through childhood (Figure 3D). However, many of the previously discrete behavioral nodes of childhood, such as those involved in play and learning become reorganized and integrated in preparation for the competent performance of adult roles of family life, child rearing and occupation-related behaviors. These changes correlate with a host of neurodevelopmental events involving primary motivational circuits, supporting cognitive and limbic areas, and the frontal cortex in particular (Chambers et al., 2003). A robust area of development in adolescence is in the domain of sexuality, in which newly differentiated nodes, or new nodes formed by the integration of earlier ones within the neurovegetative, social or motor spheres, become an increasingly organized cluster of motivational pathways (Gorski, 1999).

In the adult, key psychological or behavioral motifs relevant to contemporary theories of motivation and addiction may be viewed from a structural perspective (Hyman, 2005; Kalivas and Volkow, 2005). Decision-making is the neurocompuational process that determines which inter-nodal connections or series of connections (motivational pathways) the individual will take in moving across behavioral nodes. In habit formation, a series of behavioral nodes become sequentially linked in an increasingly invariant sequence (i.e. erosion of the decision-making component of the process), such that behavioral node A leads to node B with increasing invariance. This process would essentially be the same as the integrative process described above whereby new behavioral nodes are created by the fusion of previously independent behavioral nodes (i.e. habitual behavior as acquisition of a locked-in sequence of behavior).

Understanding motivational-behavioral repertoires as scale-free like systems thus provides a new concept for understanding the functional organization and development of behavior as a biophysical product subject to evolutionary pressures. Structurally, discrete behaviors are organized hierarchically whereby certain behaviors (e.g. eating, crying, sexual behavior) operate as hubs within local behavioral sets (functional clusters) crucial to survival. As in scale-free systems, this organizational structure would: a) allow the efficient execution of behavioral sequences or strategies to achieve survival goals by virtue of minimizing motivational-behavioral network diameters; and, b) be tolerant to random degradation (e.g. varieties of mental or physical illness that eliminate subsets of behavioral nodes do not completely impair the survival of the individual). Developmentally, newly introduced behavioral nodes, whether formed de novo or by differentiating or integrating events involving previously existing nodes, would show preferential linkage attachment with already existing behavioral nodes serving as hubs for survival-dependent behavioral clusters. Many survival-dependent behavioral hubs (e.g. eating, crying, sexual behavior) would both instantiate behaviors that are the most conserved across species while representing the most early formed nodes in the behavioral evolution of the species or in the development of the individual.

4. Neurobiological instantiation of scale-free motivational-behavioral repertoires

If motivational-behavioral repertoires of individuals are structured and develop according to a scale-free organization, how might the design and function of the relevant brain systems generate this organization? The proposed behavioral level theory makes three major predictions that can be explored in terms of available neuroscientific data. First, nodes and connection links are the two distinct types of elements in any scale-free system. They serve differing but complimentary roles, are described by different parameters, and in real world systems are physically instantiated by different types of observables. Given these differences, we anticipate the existence of semi-independent brain systems that manage information processing giving rise to these nodal (behavioral) vs. connection (motivational) elements of the scale free-system. Second, the execution of motivated behavior would be synonymous to traversing a scale-free map of motivational links and behavioral nodes in which different pathways can be pursued. Therefore, these same neural substrates should also be interactively dedicated to functions of decision-making and/or sequencing. Third, given the evolutionary and developmentally adaptive self-organizing attributes of scale-free systems, we should be able to define an additional neural sub-system that interacts with those described above, that governs the behavioral and neural systems plasticity of motivational-behavioral repertoires according to a scale-free organizational motif.

Next we review how current neurobiological data address these three predictions. In prefacing the following sections, it is important to make clear that our aim is not to show that the physical substrates of the brain itself are necessarily organized in some scale-free structural manner. Rather, we aim to show how these substrates, however non scale-free in themselves, may be built and integrated to process and manage information that generates a scale-free mapping of motivated behavior, enacted (or navigated by) the organism. Thus while it is certainly true that independent scale-free organizations may be nested within others, intervening levels of physical system organization may be decidedly non-scale free, and yet serve as objects within, or substrates that generate, larger scale-free-like structures.

4.1 Semi-segregated circuits process nodal (behavioral) vs. connection (motivational) information

The primary behavioral output system of the brain is localized to its anterior half and comprises frontal cortical, subcortical-striatal, and midbrain components (Chambers et al., 2003). Although neural circuits external to this primary system, but connected with it, including the hypothalamus, lateral septum, amygdala and hippocampus also process motivational/behavioral information (Sewards and Sewards, 2003; Sheehan et al., 2004), this cortical-striatal assembly is believed to be the major neural platform that organizes, executes and adapts flexible, goal directed behavioral output (Haber, 2003; David et al., 2005; Kelley et al., 2005).

The principal architectural feature of the cortical-striatal assembly is defined by cortical-striato-thalamo-cortical loops: serial polysynaptic projection pathways from the frontal cortex to the striatum, through the globus pallidus, to the thalamus and back to the cortex (Alexander et al., 1986; Masterman and Cummings, 1997; Swanson, 2000). Generally, these cortical-striatal circuits utilize excitatory (glutamatergic) neurotransmission in/out of the cortex, inhibitory (GABAergic) transmission between subcortical stations, and are modulated by dopamine (DA) afferents originating in the midbrain.

Accumulating evidence supports more detailed elaborations on this architectural plan in two major ways. First, at the level of the striatum, interactive ‘direct’ (Caudate-Putamen (CA-PU) →globus pallidus interna→thalamus) and ‘indirect’ (CA-PU→globus pallidus externa→subthalamic nucleus→globus pallidus interna→thalamus) pathways may compete neuroinformatically as a mechanism in support of decision-making or behavioral response selection (Graybiel et al., 2000; Rubchinsky et al., 2003). Second, and most pertinent here, not one but many parallel, anatomically segregated cortical-striatal loops serve differential aspects or hierarchical levels of behavioral control including motivational processing (Cummings, 1993; Rolls and Treves, 1998b; Kolomiets et al., 2001; Chambers et al., 2003).

Consistent with a design reflecting the need for semi-independent information processing streams for pre-motor planning vs. execution of concrete motor behavior, these parallel circuits follow a functional-anatomical plan along an anterior-ventral to posterior-dorsal gradient (Alexander et al., 1990; Weinberger, 1993; Masterman and Cummings, 1997; Haber and McFarland, 1999). Loops traversing at the anterior-ventral extent of the cortical-striatal assembly, whose glutamatergic fibers originate in the orbital, medial and anterior cingulate regions of the prefrontal cortex (PfC), project into the ventral striatum (Nucleus Accumbens (NAc)) (Heimer and Wilson, 1975). This ventral cortical-striatal module mediates pre-motor output processing including executive-decision making functions that incorporate emotional, homeostatic, and contextual memory information guiding the flow of thought and motivation (Weinberger, 1993; Groenewegen et al., 1999; Chambers et al., 2003). Meanwhile, loops residing at the posterior-dorsal extent of the cortical-striatal assembly, including supplementary motor areas and the motor-sensory strip of the cortex, project into the dorsal striatum (CA-PU) and handle processing that most directly informs concrete motor output.

Transitional zones between the ventral and dorsal cortical-striatal modules entail mixed functional and biological features typical of the ventral or dorsal territories, and/or they serve associative functions between these zones (Zahm and Brob, 1992; Joel, 2001). For example, the ventral striatal NAc is composed of a ventral-medial ‘shell’ region and a more dorsal-lateral ‘core’. The ‘core’ is anatomically juxtaposed to the CA-PU (dorsal striatum), and is associated with a mixture of ventral and dorsal striatal-like functions, protein expression profiles and extrinsic connectivity patterns (De Olmos and Heimer, 1999; Kelley, 2004). In the CA-PU, islands of neurons termed ‘striosomes’, which share some functional, histochemical and connectivity characteristics with neurons in the NAc shell, are suspended in the ‘matrix’ of the dorsal striatum, seemingly separated from related ventral striatal territories (Haber and McFarland, 1999; Joel and Weiner, 2000; Grande et al., 2003).

At the level of the midbrain, the semi-segregated nature of the ventral vs. dorsal cortical-striatal modules becomes even more apparent. The ventral tegmental area (VTA) contains DA cell bodies projecting predominantly into the NAc and associated PfC (Kalivas, 1993; Haber and McFarland, 1999). Conversely, the substantia nigra (SN) projects predominantly into the CA-PU (Kalivas, 1993; Haber and McFarland, 1999). Projections into these midbrain areas are similarly segregated: while VTA receives reciprocating input from both NAc (GABAergic projections) and PfC (glutamatergic projections), the SN receives inputs predominantly from subcortical regions (Sesack et al., 1989; Kalivas, 1993; Celada et al., 1999).

In sum, available evidence suggests that while the behavioral output system of the brain follows one overriding architectural plan, there are semi-segregated information processing streams within this design. Two closely related, but semi-segregated brain modules generate and process information relevant to the two categories of elements in the proposed scale-free structure of motivated-behavior: 1) a dorsal cortical-striatal module serving as a neural representational processing stream for generating behavioral repertoires as the nodes; and, 2) a ventral cortical-striatal module that processes motivational representations as the connections. In the following sections we explore how the functional anatomy and neuroinformatics of the dorsal (4.2) and ventral (4.3) cortical-striatal modules might allow trajectories across a scale-free map of motivated behavior via their sequencing and decision making functions, and how the structure of the scale-free map might change in habit learning.

4.2 The dorsal cortical-striatal module: Neuroinformatics of behavioral nodes

The function of the dorsal cortical-striatal module is clinically evident from a survey of neurological diseases affecting its extrapyramidal (dorsal striatum) vs. its pyramidal (projections from motor cortex to spinal motor neurons) components. Lesions of the motor cortex can completely abolish movement in the contralateral side of the body, while pathology in spinal motor neuron tracts weaken or destroy motor activity in specific muscle groups. However, strokes and other pathologies of the striatum such as Parkinson and Huntington disease typically impair the flow and sequencing of behavior (Pryse-Phillips and Murray, 1985). These observations suggest the striatum is specialized for maintaining the sequential flow of behavioral output, and for storing reproducible patterns of behavioral programs. These aspects implicate the dorsal striatum as a key substrate that allows individuals to perform sets of reproducible behaviors instantiating nodes in a scale-free like system.

4.2.1 The dorsal striatum stably sequences motor representations comprising behavioral nodes

A more detailed understanding of the specialized sequencing role of the striatum has been elaborated upon by functional and anatomical comparisons with the cortex. The striatum appears to be optimally designed for the reliable generation of serial neural representations required for the performance of stable-habitual motor programs (James, 1890; Jog et al., 1999). This functional allocation to the dorsal striatum would largely free other brain regions (e.g. frontal cortex) from having to participate in stable habitual behavioral functions that need not be performed with conscious awareness. By autonomously executing well-learned motor programs, the dorsal striatum provides the capacity for daydreaming about playing golf while actually driving home from work.

Comparisons between the cortex and striatum on the neural network level, suggest how the neuroinformatics of this sequencing role may take place. The frontal cortex and its sub-divisions, including motor areas, is fundamentally organized as a sheet of excitatory glutamatergic neurons (intercalated with local inhibitory GABAergic neurons), projecting to each other both within and across functional subdivisions (Sporns et al., 2002). This excitatory and highly associative architecture of the frontal cortex is thought to allow for the generation of high-order, polymodal forms of cognition such as imagination and daydreaming. In contrast, the CA-PU (and its downstream target, the globus pallidus) is an almost exclusively inhibitory network ensemble, whose principal projection neurons (GABAergic medium spiny neurons) comprise the vast majority of the neuronal population. Principal projection neurons of the dorsal striatum are collaterally interconnected, either directly with other medium spiny neurons or via acetylcholine-bearing striatal inter-neurons (Plenz, 2003), and send inhibitory afferents to downstream striatal stations (e.g. globus pallidus) (Groves et al., 1995). Moreover, while the net firing rate of any one CA-PU medium spiny neuron is determined from a combination of excitatory glutamatergic afferent activity from the cortex and thalamus, and inhibitory input from other striatal regions, the summated inhibitory input tone and low basal firing rates of these neurons create conditions where most neurons fire at low rates and only a minority fire phasically at any one moment (Groves et al., 1995). As suggested by Rolles and Treves, these inhibitory design attributes of the striatum may have specifically evolved to allow for competitive emergence of neural representations coding for differential movements in a behavioral sequence and/or provide a relatively stable network environment for the sequential generation of reproducible firing pattern sequences (Rolls and Treves, 1998b).

On the neuroinformatic level, neurons in somatotopic regions of the CA-PU corresponding to specific body parts in the homunculus of the motor cortex increase their firing in phase with movements of those body parts (Crutcher and Alexander, 1990; Kimura, 1990; Flaherty and Graybiel, 1994; Carelli et al., 1997). Corresponding to the specialized motor sequencing role of the striatum, functionally specific neuronal ensembles within the striatum are proposed to undergo sequential transitions between distinct firing patterns; e.g., A→B→C→D→... (Pennartz et al., 1994; O’Donnell et al., 1999; Gilles and Arbuthnott, 2000). Each of these firing patterns (A, B, C...) would be defined by the different firing rates across an array of neurons of an ensemble, corresponding to each sequential step of the motor sequence. For the appropriate and autonomous execution of reproducible motor sequences, previous firing patterns in a striatal ensemble should influence subsequent firing patterns occurring within the same ensemble, such that extrinsic projections to the dorsal striatum can remain relatively uninvolved (Berns and Sejnowski, 1998). For example, one may have a general routine in the morning of shutting off the alarm clock, getting out of bed, brushing teeth, disrobing, showering, dressing, eating breakfast, going to work (striatal network firing patterns progress through A→ B→ C → D → E →F→ G→ H ...). However, it is not appropriate to drive to work immediately after disrobing (firing pattern D goes immediately to H), even if one is thinking about going to work at that moment. While illustrating the adaptive importance of why habitual behaviors should be outsourced to the dorsal striatum semi-independently from extrinsic processing, this example also depicts a neurocomputational capacity for autonomously generating reproducible representational sequences that may be readily supported by striatal neuroanatomy (Wickens and Arbuthnott, 1993; Berns and Sejnowski, 1998; Rolls and Treves, 1998a).

Features of striatal networks share similarities with artificial neural networks characterized by inhibitory projections and recurrent inhibitory collateral connections where large numbers of differential firing patterns can be relatively sparsely represented and reproducibly emergent (Wickens and Arbuthnott, 1993; Arbib, 1998; Berns and Sejnowski, 1998). In these systems, the imposition of an input stimulus produces an initial firing pattern, A, that temporally and autonomously evolves to B, C, and D, without the need for further extrinsic input and as determined by the state of the connectivity structure contained within the network. Eventually, the network settles into a final stable firing pattern X, but not before it transitions between A→B→C→D→ ... leading to X. Once reaching the stable firing pattern X, the system requires a new bout of afferent input stimulation before cycling through additional sets of intermediate firing-pattern representations. Notably, different initial firing patterns other than A may initiate entirely different cascades of subsequent firing patterns consistent with alternative behavioral sequencing. Regardless, such autonomously generated firing pattern states would be transiently and sequentially transmitted in some form to distal striatal-thalamo-cortical stations finally encoding concrete motor output via the pyramidal system.

The results of in vivo, multi-neuronal striatal recordings from rats learning habitual motor tasks are generally consistent with this model (Graybiel, 1998; Jog et al., 1999), in which reproducible temporal sequences of firing patterns are engrained with behavioral practice (Barnes et al., 2005). During learning of a specific motor sequence, neurons within somatotopically specific motor regions of the CA-PU show motor- or task-event-specific increases in firing above their usual relatively sparse firing rates. In the early stages of learning, these increases in firing are frequently observed at crucial intermediate steps of the motor sequence, possibly corresponding to greater extra-dorsal striatal involvement (extrinsic stimulation of dorsal striatal neurons), in attempting to configure the initiation or ordering of a particularly desirable representational sequence. As the desirable motor sequence becomes well-learned, fewer striatal neurons show intermediate motor-phase-specific increases in firing. This feature may correspond to the establishment of a more reliable and relatively sparsely encoded representational sequence that evolves autonomously without the need for extra-striatal input at critical junctures. Observed increases in the proportion of striatal neurons showing increased firing rates in phase at the start of the well-learned motor task may represent extra-striatal stimulation required to initiate the particular habitual striatal representational sequence, optimal for efficient completion of the whole task.

In sum, various lines of brain research suggest the striatal component of the dorsal cortical-striatal assembly generates and manages neural firing representations that encode for observable behavioral programs and their stable sequential organization, instantiating behavioral nodes in a scale-free map. Next we explore how the striatal component of the ventral cortical-striatal assembly contributes to the motivational-behavioral map by generating motivational links connecting behavioral nodes. In particular, we find that the ventral cortical-striatal module, in cooperation with midbrain sources of DA, would serve as a key source of extrinsic input to the dorsal cortical-striatal module in choosing, configuring, and altering behavioral representational sequences generated by the dorsal striatum.

4.3 The ventral cortical-striatal module: Neuroinformatics of motivational links

Given the anatomical-design motifs shared between the dorsal and ventral cortical-striatal modules, the ventral module would be expected to entail some neuroinformatic capacity for reproducible firing pattern generation and sequencing. However, structural-anatomical features unique to the ventral module suggest its sequencing function is not specifically designed for performing stable habitual motor programming. As previously mentioned, the connectivity of the ventral striatum contrasts from the dorsal striatum in several major ways. First, the NAc receives glutamatergic input from the limbic PfC rather than more dorsal frontal cortical areas. Second, the NAc receives DA afferent fibers predominantly from the VTA, rather than from the SN. Third, and most relevant here, the ventral cortical-striatal module is broadly and directly interactive with limbic circuits across all of its cortical, striatal and midbrain levels.

Unlike the motor strip or the CA-PU, both the PfC and NAc receive significant glutamatergic inputs from the basolateral nucleus of the amygdala (BLA) and the ventral hippocampal formation (Kelley and Domesick, 1982; Groenewegen et al., 1987; Groenewegen et al., 1999; Baxter and Murray, 2002). Unlike the dorsal-striatal associated SN, the VTA sends DA afferents to the limbic PfC, BLA and ventral hippocampus. Within the NAc itself, medium spiny neurons receive VTA-DA afferents in functionally relevant proximity to reception zones of glutamtergic afferents from the PfC, BLA and ventral hippocampus (O’ Donnell et al., 1999). Similarly, the PfC receives convergent glutamatergic projections from the BLA and ventral hippocampus in proximity to VTA-DA afferents, while the ventral hippocampus and BLA are themselves reciprocally interconnected via glutamatergic projections (Mulder et al., 1998; Gurden et al., 1999; Pitkanen et al., 2000; Ishikawa and Nakamura, 2003).

4.3.1 The ventral striatum flexibly sequences neural representations of motivations as links connecting behavioral nodes

Given substantial data defining the emotional and cognitive functions of the amygdala, hippocampus and PfC, their inputs to the NAc implicate the NAc as a key brain substrate where emotional, and cognitive information processing directs behavioral output (Mogenson et al., 1980). Targeted brain stimulation and neuronal recording studies have examined how glutamatergic input from distributed prefrontal cortical-temporal limbic regions and VTA-DA input functionally integrate to control NAc firing patterns (Yang and Mogenson, 1984; Yang and Mogensen, 1985; O’Donnell and Grace, 1994; Jay et al., 1995; O’Donnell and Grace, 1995; Finch, 1996; Mulder et al., 1997). Together, these studies indicate that an individual medium spiny neuron of the NAc will fire at low or high frequency as determined by an integrative function of: a) local inhibitory GABAergic input from neighboring NAc neurons; b) afferent glutamatergic excitatory control from the PfC, amygdala and ventral hippocampus; and c) DA influence from the VTA. Firing patterns across ensembles of NAc neurons, as generated by the integration of signal patterns from PfC executive, amygdalar affective, and hippocampal contextual memory centers, may thus represent the computational building blocks of motivational states (Pennartz et al., 1994; O’Donnell et al., 1999).

While entailing capacity for firing pattern sequence generation, the integration of distributed limbic inputs within the NAc provides a computational platform for the more flexible capacity of decision-making capacity guiding motivational control (McFarland and Sibly, 1975; Christakou et al., 2004; Spinella, 2004). In the NAc, sequential ensemble firing patterns 1→2→3→ etc. would encode motivational information leading to behavioral destinations corresponding to reward acquisition, on a scale-free map of motivated behavior (Figure 4). Consistent with this framework, instrumental learning produced by differential types of motivational rewards (i.e. food vs. drug) corresponds to the emergence of differential and potentially mutually exclusive firing pattern arrays across medium spiny neurons of the NAc (Deadwyler et al., 2004). After extensive self-administration of an addictive drug, in which drug-seeking is installed as a major motivation, NAc firing patterns associated with drug intake become more robustly expressed (Hollander and Carelli, 2005). In this way, motivational firing pattern representations emerging sequentially in the ventral cortical-striatal module provide alternative pathways of progression (links) between concrete behavioral programs (nodes), which are in turn composed of habitual motor sequences as encoded by the dorsal cortical-striatal module. In many cases (Figure 4) discrete behavioral steps, directly encoded by the dorsal striatal-cortical module can be shared between larger behavioral programs (generated by larger strings of neural representational sequences within the dorsal module) via the choreographing-like function of the ventral cortical-striatal module in configureing sequence changes in the dorsal module. Interestingly, this understanding of the ventral dorsal-striatal module casts a new perspective on the nature of motivation: rather than directly being about receiving gratification from reward-goals, motivation serves to orient and sequence behavior programs that allow the greatest probability of achieving reward-goals. In this way, behavioral sequences themselves become the most direct ‘goals’ of the representational sequences of the ventral cortical striatal module, and appropriately configured strings of behavioral sequences best secure behavioral rewards.

Figure 4.

Traversing a local neighborhood of the scale-free motivational-behavioral repertoire corresponds to simultaneous behavioral (gold) and motivational (green) phenomena transcending behavioral, neurocircuit, and neuroinformatic/neurophysiological levels. On the Observable Behavioral Level, performance of alternative non-habitual sequences of behaviors (i and ii) could have one discrete behavior program in common (node C). On the Scale-free Organizational level, discrete behavioral programs (gold) are interlinked by differing motivational pathways (green). On the Neuroanatomical Level, semi-segregated information processing streams of the dorsal (gold) and ventral (green) cortical-striatal modules generate firing pattern sequences representing behavioral vs. motivational information respectively. On the Neuroinformatic Level, when performing either behavioral series i or ii, the progression of firing patterns in the dorsal cortical-striatal module (gold) is configured by neural representations in the ventral cortical-striatal module (green) by virtue of inter-modular connectivity at cortical (glutamate), and striatal/ midbrain (GABA) stations and in cooperation with midbrain DA-cell firing. The temporal progression of medium spiny neuronal ensemble firing patterns in the ventral cortical-striatal module incorporates motivational processing and decision-making as informed by pre-frontal cortical, amygdalar and ventral hippocampal glutamatergic afferents. Alternate behavioral sequences i and ii may have behavior sub-programs (C) in common by virtue of the flexible configuring influence of the ventral upon the dorsal module.

4.4 Interfacing ventral and dorsal cortical-striatal modules: Links configure nodes as decision-making leads to habit

Traversing a scale-free motivational-behavior map as explored here requires that neural representations in the ventral cortical-striatal module co-configure with those in the dorsal module. That is to say, emerging sequences of firing patterns in the ventral module would play a major role in activating, initiating or guiding the flow of firing pattern sequences in the dorsal module. To illustrate the concept of co-configuration, Figure 4 shows a local neighborhood of a motivational-behavioral repertoire where discrete behavioral nodes (denoted alphabetically: A,B,..I) are interconnected via motivational links (denoted numerically: 1,2.., 14). For different non-habitual behavioral sequences ((i) vs. (ii)) that might include some action steps in common (behavioral node C), the ventral and dorsal cortical-striatal modules are functionally engaged such that a given sequential firing pattern code in the ventral system assists with initiating or guiding the more stable firing pattern sequences in the dorsal system. In pathway (i), the individual performs observable behaviors (...A→C→E→F...), which are correlated with a sequential series of firing patterns within the dorsal cortical-striatal module. In configuring this sequence, the ventral module produces a sequential series of neural ensemble firing patterns (...→2→6→12→...). In performing an alternative behavioral sequence (ii) (...→D→C→G→...), the ventral module produces a different neural representational sequence (...→3→8→...). The non-habitual nature of these behavioral sequences is shown by the presence of multiple decision points (e.g. different motivational link options) associated with each behavioral node. At any one behavioral node, only a small subset of the entire repertoire of motivational links (ventral striatal firing patterns) become potentially available for full representation, upon which computational processes within the ventral cortical-striatal module (i.e. the decision-making process) select a specific ventral striatal firing pattern that will lead to or configure a specific behavioral activity (dorsal striatal firing pattern).

In the formation of habitual behaviors (Figure 5), as a sequence of behaviors (→A→C→F→) is repetitively performed, dorsal cortical-striatal modular representational sequences are generated in an increasing invariant manner such that behavior A will lead to F via C with increasing probability (Barnes et al., 2005). In this case, motivational sequences represented in the ventral system (→2→7→) become increasingly dominant while the other codes for motivational links associated with behavioral node C (3, 5, 6, 8, 9, 10) are formed more rarely to the point of extinction. Upon full acquisition of the habit, behavioral nodes A, C, F have collapsed into one behavioral node (as in the fusion process of motivational-behavioral repertoire development described in section 3.2), and the participation of the ventral cortical-striatal module representations (2, 7) in configuring the A, C, F sequence of the dorsal module is no longer required.

Figure 5.

In habit formation, a sequential series of behavioral actions (nodes A, C, F) become enacted in sequence with increasing (near 100%) probability. This corresponds to structural changes of the motivational-behavioral repertoire where certain motivational links (2, 3, 5, 6, 7, 8, 9, 10) are obliterated. Neuroinformatically, this would entail the loss of a significant likelihood that firing pattern representations (3, 5, 6, 8, 9, 10) would emerge in the ventral cortical-striatal module while connections (2,7) occur with near 100% probability. Eventually, the temporal sequencing of representations (A →C→ F) occurs essentially autonomously in the dorsal module, without the need for ventral cortical-striatal modular configuring input. Neurobiologically, these changes are mediated by neuroplastic events within local networks of the cortical-striatal module, facilitated by DA, glutamatergic and GABAergic interactions.

Mounting neurobiolgical data on the function and anatomy of the cortical-striatal assembly are consistent with this relationship between the ventral and dorsal modules (Letchworth et al., 2001; Gerdeman et al., 2003; Everitt and Robbins, 2005; Yin and Knowlton, 2006). While the acquisition or performance of flexible instrumental goal-directed actions (non-habitual behavior) and decision-making is most closely associated with functionality of the NAc, associated PfC and temporal limbic regions (Balleine and Killcross, 1994; Balleine and Dickinson, 1998; Corbit et al., 2001), the control of more invariant stimulus-response behaviors (habitual behavior) is most closely associated with the CA-PU and associated cortex (Fernandez-Ruiz et al., 2001; Yin et al., 2004; Yin and Knowlton, 2006). Meanwhile, the re-configuration, interruption, inhibition, or shifting of well-learned habitual motor program representations in the dorsal cortico-striatal module may require the re-intervention of the ventral cortical-striatal module (Graybiel, 1998; Matsubara et al., 2004). Consistent with this idea, recent work has examined simultaneous neuronal recording in dorsal striatum and hippocampus in rats learning alternative spatial-behavioral tasks (Johnson and Redish, 2005; Johnson et al., 2005). While the evolution of striatal neuronal firing patterns appears to be more closely linked with longer-term configuration and/or stabilization of specific firing pattern sequences during learning (corresponding to habit formation), hippocampal neurons appear to be more variably active allowing greater behavioral flexibility during early stages of learning, in which decision-making processes predominate. This study provides additional evidence contrasting the neuroinformatic roles of the ventral vs. dorsal cortical-striatal modules since the hippocampus directly and robustly projects only to the ventral module, both at the levels of the mPfC and NAc, while receiving DA afferents only from the VTA. As will be discussed in section 4.5.2, habit formation likely involves DA mediated neuroplastic processes that alter the representational repertoires of both the ventral and dorsal cortical-striatal modules, and the ways in which they are functionally co-configured (Everitt and Robbins, 2005).

Anatomical pathways allowing co-configuration likely involves multiple connectivity routes in the brain. Given the contiguous structure of the frontal cortex and striatum as a whole, it is possible that information contained within one functional corticostriatal loop influences adjacent loops at the level of the cortex, striatum or other subcortical stations (Groves et al., 1995; Joel, 2001; Yin and Knowlton, 2006). For example, striatal transition zones anatomically positioned between the ventral and dorsal territories receive converging cortical projections from both prefrontal cortical areas and more caudal-dorsal motor areas (Haber et al., 2006). Also, thalamic afferents back to the cortex are integrated into cortical layers in ways indicative not only of the thalamus’ role in serving as the final relay of cortical-striatal-thalamo-cortical loops, but also suggestive of its regulation of intra-cortical information flow, possibly between pre-frontal and dorsal motor regions (McFarland and Haber, 2002). Evidence supporting the maintenance of functional and anatomical segregation of coherent information streams across cortical-striatal loops is not totally consistent with this possibility as a dominant mechanism, nor would it account for the directionality or hierarchical control of motivational-behavioral information processing along a ventral to dorsal gradient (Haber, 2003). On the other hand, the existence of bi-directional communication between parallel cortical-striatal loop stations in some regions, and only unidirectional projections in others, suggests a neural circuit basis for the hierarchical configuring influence of the ventral cortical-striatal module over the dorsal (Haber et al., 2000; Haber, 2003). Using injections of retrograde and anterograde axonal tract tracers into the striatum and midbrain, information processing streams through corticostriatal loops appear to advance in a ventral to dorsal gradient when the connectivity patterns between the NAc and CA-PU and the DA cell populations of the VTA and SN are all taken into consideration (Haber et al., 2000). Although the SN is the major source of DA for the dorsal striatum, striatal inputs to the SN arise predominantly from the ventral striatum. Given the central role of DA in mediating behavioral flow and long-term motivational changes (see the next section), evidence for ‘spiraling’ cortical-striatal-mesolimbic pathways are suggestive of the capacity of the ventral cortical-striatal module to exert configuring influence over motor representations in the dorsal module. These findings suggest a neurobiological basis for motivational links connecting behavioral nodes as a structural concept of motivational-behavioral repertoires.

4.5 Dopamine dynamics facilitate navigation and alteration of the scale-free map of the motivational-repertoire

The execution of motivated behavior is highly sensitive to environmental change, particularly with respect to survival-oriented behavior. Such flexibility is intrinsic even to developmentally static scale-free systems (Bar-Yam and Epstein, 2004), and may confer significant evolutionary advantage (Willeboordse, 2006). When vast numbers of possible motivational links and behavioral nodes are interconnected, complex adaptive behavior is not reflexive but involves many decision points that dictate different behavioral strategies or motivational pathways, depending on environmental conditions. However, environmental changes could be so drastic that the current motivational-behavioral repertoire lacks the necessary composition of behavioral nodes or motivational connections to adaptively cope. Under such circumstances, it is advantageous for the structure of motivational-behavioral repertoires to remain plastic throughout adult life, much as it has undergone developmental change phylogenically and ontologically. We posit that not just any random structural revision will result from this plasticity. Because of the optimal structural and functional properties of the scale-free design for conveying survival fitness, these plastic events will render a new structure that is altered from the last, but nonetheless remains scale-free.

In this section, we discuss how neurobiological and computational investigations support a role for DA in creating and maintaining a scale-free structure of motivational-behavioral repertoires. We consider DA first as a transmitter system with immediate consequences, and second, as facilitating long-term (neuroplastic) changes in the cortical-striatal assembly. Both categories of DA function, one facilitating action and the other facilitating action-memory, are rooted in the modulatory effects of DA on information processing in glutamatergic/GABAergic cortical-striatal pathways. While both the acute and long-term aspects of DA function are important in cortical-striatal functioning, the long-term, neuroplastic effects of DA are hypothesized to be particularly important in organizing scale-free structures of motivational-behavioral repertoires in health and in addiction.

4.5.1 Acute role of dopamine: facilitating trajectories across the motivational-behavioral repertoire

The immediate role of DA in subcortical brain regions is suggested by Parkinson disease and the action of dopamine blocking drugs. Insufficient DA signaling in the dorsal striatum produces characteristic loss of the flow of motor output associated with bradykinesia, rigidity and tremor. These states suggest an immediate role of DA release in ‘gating’ motor program representations within the striatum (Wan and Swerdlow, 1996; Kotter and Wickens, 1998; Murer et al., 2002; O’Donnell, 2003). Although no consistent definition for such gating has been uniformly adopted, it implies that DA outflow is important in: (a) allowing striatal firing patterns to be coherently and unambiguously represented so that they may be ‘read’ by downstream motor structures for concrete behavioral implementation; and/or (b) facilitating the sequential progression of coherent firing pattern representations in the striatum, corresponding to the performance of a complete motor activity.

Evidence in support of these possibilities arises from anatomical and physiological studies of the striatum in animal models. As described in section 4.2, a major contribution to the net firing rate of a striatal medium spiny neuron is determined by the combination of excitatory glutamatergic afferent activity from the cortex and thalamus and local inhibitory input from other striatal regions. However, superimposed on these conditions, phasic DA release to a striatal neuron can be either inhibitory or excitatory, depending on the combination of concurrent excitatory/inhibitory tone to that particular neuron (West et al., 2003; David et al., 2005). For striatal neurons receiving a certain intensity of summated excitatory input, DA promotes entry of that neuron into a physiological ‘up’ state (depolarized), during which phasic firing of that striatal neuron may occur (Nicola et al., 2000). For striatal neurons more weakly excited by cortical afferents, the same DA efflux would maintain that neuron in the more hyperpolarized ‘down’ state in which striatal neuron action potential generation cannot occur (O’ Donnell, 2003). These data suggest that by modulating the firing rates of many individual neurons in an ensemble to extremes (of not firing vs. firing rapidly), DA sharpens the signal to noise ratio of a given representation across the striatal ensemble (O’ Donnell, 2003). Creation of such high-contrast, yet sparsely represented firing patterns in the striatum may be critical for computational processing of motor representations in downstream basal ganglia structures. For instance, in animal models of Parkinson disease involving chemical destruction of DA afferents to the striatum, medium spiny neurons show abnormal firing responses to glutamatergic input (Calabrisi et al., 1993) and their downstream pallidal/subthalamic targets show abnormally high firing frequencies or oscillatory bursting activity (Murer et al., 1997; Tseng et al., 2001).

The presence of high-contrast firing patterns in the striatum, as facilitated by DA input to medium spiny neurons, may also be required to maintain the autonomous progression of coherent firing patterns (e.g. A→B→C→D→) corresponding to sequential components of a motor program (see section 4.2). For instance, without the DA-mediated effect of enhancing the impact of excitatory input to some striatal neurons but not others, ambiguous and perhaps noisy firing patterns may emerge across the striatal ensemble (e.g. A_corrupted). Such corrupted representations may not easily evolve autonomously into the next desired coherent firing pattern (e.g. B) or any other patterns interpretable by the brain. Notably, for extremely well-learned motor programs, the autonomous evolution of the corresponding representational sequences may show relatively high tolerance to such degradation and/or blocking, especially since the performance of these ‘habitual’ sequences presumably requires less ongoing cortical/ventral cortical-striatal modular input requiring DA-mediated transformation. Thus, with disease progression, Parkinson disease patients are known to have increasing difficulty in initiating or shifting between motor routines, particularly when they are substantially different or non-habitual (Fama and Sullivan, 2002). Moreover, increased cortical involvement in configuring and facilitating striatal representational sequences required for motor programming may serve as a short-term compensatory mechanism for the progressive loss of DA influx (Sabatini et al., 2000; Bezard et al., 2003).

Analogous to the situation in the dorsal striatum where SN-DA release gates concrete motor behavior, the immediate role of VTA-DA release in the ventral striatum is viewed as gating of motivation (O Donnell et al., 1999; Salamone et al., 2003; West et al., 2003). Generally, VTA-DA neuronal firing, and/or increases in NAc DA levels, occur in association with four categories of immediate stimuli: 1) natural rewards (including food, sex, power, or survival-related resources) (Spanagel and Weiss, 1999; Kelley, 2004); 2) novelty or unexpected events (Ljungberg et al., 1992; Bardo et al., 1996; Panksepp, 1998c; Volkow, 2004b; Lisman and Grace, 2005); 3) stressful or aversive stimuli (Piazza and Le Moal, 1996; Finlay and Zigmond, 1997); and 4) intake of addictive drugs (including nicotine, alcohol, cocaine, amphetamine, opiates, cannabinoids, etc) (Wise, 1990; Koob, 1992; Self and Nestler, 1998; Wise, 1998; Gerdeman et al., 2003; Rodd et al., 2004; Volkow, 2004b). These observations most clearly associate ventral striatal DA release in the stimulus-induced gating of motivation, rather than specifically with pleasurable affective states, as was once widely believed (Robinson and Berridge, 1993; Di Chiara, 2002; Berridge, 2004; Volkow, 2004b; Wise, 2004). For instance, natural rewards stimulate motivational programming necessary for sustaining life and reproduction. In the case of novelty, curiosity, or the motivation to explore the unknown or learn new skills, should represent motivational programming of high importance; novelty-related behaviors can lead to the identification of new opportunities for natural rewards or new methods for acquiring them. DA signaling in response to unexpected, stressful or aversive stimuli, although often occurring during un-pleasurable affective states, may gate motivational programs and behavioral action required to evade threats to survival.

The action of addictive drugs in increasing DA efflux in the ventral striatum is distinct from that of natural motivational stimuli in two major respects. First, the DA stimulating properties of abused drugs use are generally not associable with survival-oriented behavior, in keeping with views of drug addiction as a pathological ursurpation of natural, adaptive motivational mechanisms (Orford, 2001; Newlin, 2002; Robinson and Berridge, 2003). Second, the abilities of drugs to stimulate DA release are often substantially greater than those of natural stimuli, although initial drug use is generally not as motivationally powerful as many natural motivational stimuli (Volkow, 2004b). Acquisition of drug addiction in humans, or the progression of behavioral changes in animal models of addiction, typically requires multiple drug doses over time. Thus, the dopaminergic properties of addictive drugs illustrate a functional feature of DA release in the ventral striatum not readily apparent from studies that examine its immediate role. DA release not only serves an immediate functional role in gating pre-motor programs (or motor programs in the dorsal striatum), but it also sculpts the future motivational agenda of the individual as a long-term neuroplastic agent.

4.5.2 Long-term role of dopamine: sculpting the motivational-behavioral repertoire according to a scale-free plan

The role of DA in neuroplasticity, learning and memory has been extensively described (Kelley, 1999; Berke and Hyman, 2000; Hyman and Malenka, 2001; Nestler, 2001; Waelti et al., 2001; Jay, 2003; Robinson and Berridge, 2003). As reviewed in these works and discussed briefly here, an emerging picture suggests that brain plasticity allowing for alterations in motivation and behavior are not necessarily limited to the DA system per se, but actually occur as far more distributed changes across the whole cortical-striatal assembly, as facilitated by ongoing DA transmission.

Briefly, long term potentiation (LTP), long-term depression (LTD), or their variants are have been extensively studied as key cellular processes allowing for changes in synaptic strength and thus changes in network learning and memory (Wang et al., 1997; Munakata and Pfaffy, 2004). While most of the available evidence focuses on LTP/LTD occurring at glutamatergic synapses, it is thought that on a neural network level, if not on the level of individual synapses, alterations in GABAergic transmission assume an equally vital role as those in excitatory transmission in the overall changes in plasticity observed in real neural networks (Gaiarsa et al., 2002).

Cortico-striato-thalamo-cortical loops are composed predominantly of excitatory/inhibitory transmission streams (section 4.1) and accumulating data depicts DA as a key neuroplastic agent within this anatomy. It is within the cortical-striatal assembly, and at the level of the striatum in particular, where glutamatergic and GABAergic systems interface most richly in combination with DA afferents from the midbrain. DA influences LTP, LTD, or their variants involving excitatory synapses in the frontal cortex, striatum, and midbrain (Mulder et al., 1997; Gurden et al., 1999; Thomas et al., 2000). These events may occur intracellularly via a number of interactions between protein cascades responsive to DA receptor occupancy and those that mediate LTP and other synaptic neuroadaptations in response to glutamatergic activity (Horger et al., 1999; Berke and Hyman, 2000; Centonze et al., 2001; Hyman and Malenka, 2001). They may also involve DA-mediated regulation of presynaptic glutamatergic release or post-synaptic sensitivity to glutamate (Snyder et al., 1998; Chase and Oh, 2000; Bamford et al., 2004). Investigations on the capacity of repeated doses of addictive (or other DA-releasing drugs) to exert chronic changes in neural systems in the striatum and /or the VTA have identified alterations in neurotransmitter efflux, neuronal protein/receptor expression, intracellular signaling, neural firing properties and neuronal dendritic morphology (Self and Nestler, 1995; Nicola et al., 1996; Nestler et al., 2001; Robinson et al., 2001; Gerdeman et al., 2003; Kalivas et al., 2003). At the level of the NAc, differential temporal patterns of DA influx not only impact the relative contributions of prefrontal cortical vs. limbic (hippocampal) input to the NAc (Goto and Grace, 2005a), but they also modulate opposing forms of neuroplastic change at prefrontal cortical vs. hippocampal glutamatergic synapses onto NAc neurons (Goto and Grace, 2005b). Together, these findings corroborate growing evidence that long-term behavioral changes associated with chronic drug-induced DA discharge requires the participation of, and neuroadaptative changes at glutamatergic synapses within the cortical-striatal assembly (Vanderschuren and Kalivas, 2000; Everitt and Wolf, 2002).

Because DA serves as an agent of neural network connection change primarily involving excitatory/inhibitory synapses within cortical-striatal modules, it would be expected to influence, but not itself instantiate alterations in behavioral nodes or motivational links in a scale-free system of motivated behavior. Instead, DA appears to operate as a supervisory guidance signal for the self-organizing aspects of distributed neuroplastic processes and behavioral programming. Such centralized control of distributed plasticity may confer protection against failure under certain unexpected environmental conditions, while allowing cooperative changes to take place across entire neuronal ensembles or networks (Seeley, 2002). Thus, while acute DA efflux may facilitate the temporal evolution, or motoric implementation of firing patterns generated by a striatal ensemble (e.g. A→B→X →C→ Y→ D→) during the immediate performance of a motor program, patterns of variable DA efflux in combination with changes in glutamatergic input to the striatum over repeated executions of the motor program would cause the entire sequence to evolve over trials (e.g. A→B→X→C→Y→D→ at trial 10 becomes A → B→C→J →D→ at trial 100) (Barnes et al., 2005). In this manner, DA acts as a sculptor of motivational and behavioral repertoires associated with reward-related learning, habit formation (section 4.4), or the addictive process (Everitt and Robbins, 2005; Goto and Grace, 2005b).

Research combining computational and neurobehavioral approaches provides a framework for understanding how DA-related neuroplasticity within the cortical-striatal assembly may guide the organization of motivated behavior so that it entails the evolutionary fitness of scale-free systems. A branch of computational neuroscience developing over the last two decades examines the implementation of reinforcement learning in adaptive artificial neural networks. Although initially designed for solving control and decision problems in robotics, these computational models have been shown to accurately simulate animal behavior in reward-conditioned learning including classical conditioning, water-maze learning, and foraging behavior (Dayan and Abbott, 2001). Within this family of artificial reinforcement learning systems, neural network connection plasticity is governed by a Hebbian-like incremental synaptic weight change rule that is modulated by a supervisory learning signal δ (Barto, 1998; Dayan and Abbott, 2001). Computationally, δ reflects the difference between an internal representation of the reward topography (that evolves with learning over repeated reward exposure trials) and that of the actual external environment (Montague and Berns, 2002). Because δ is embedded into local self-organizing synaptic learning mechanisms as an internally-generated surrogate of the reward topography of the outside world (Montague et al., 2004; Schultz, 2004), it serves a computational and neuroplastic role analogous to the effects of DA on glutamatergic synaptic plasticity as suggested by neurobehavioral research.

This neuroinformatic interpretation of DA has been most clearly and elegantly demonstrated in studies of natural-reward conditioned learning in monkeys (Waelti et al., 2001). As monkeys learn to associate a neutral stimulus to the onset of a natural reward, their maximal rates of DA cell firing shifts from the time of reward onset to the timing of the neutral stimulus onset. Using the temporal-difference form of reinforcement learning, which encodes a temporal (rather than spatial) version of the δ-signal as an internally generated representation of the expectation of reward delivery (Montague et al., 1996; Schultz et al., 1997), the temporal evolution of the peak amplitude of the δ-signal in the artificial network tracks precisely with that of dopamine cell firing of monkeys.

In other applications of reinforcement learning, the δ-signal reflects an internal mapping of the external reward topography in spatial rather than temporal terms. Such simulations have been shown to accurately reproduce core aspects of foraging behavior, including the Matching Law. Observational research in several animal species (insects, rodents, primates, humans) has shown that given an array of possible action choices leading to rewards, animals learn to allocate a fraction of their total choices to each action in proportion to the relative magnitude of reward most likely available from each action choice (Herrnstein, 1961; Rachlin and Laibson, 1997; Sugrue et al., 2004). This Matching Law is expressed as

$$\frac{{\mathrm{C}}_{\mathrm{i}}}{\Sigma {\mathrm{C}}_{\mathrm{j}}}=\frac{<{\mathrm{R}}_{\mathrm{i}}>}{\Sigma <{\mathrm{R}}_{\mathrm{j}}>}$$

where C_i is the number of choices of action i giving a mean reward <R_i > from an array of j possible choices that contains a total reward of Σ <R_j >. This mathematical statement formalizes the empirical observation that animals appear to have an evolutionarily conserved capacity to learn the stochastic reward topography of their environment, and to act on such learning by allocating effort toward goals according to their relative reward values.

Given the direct correspondence of the δ-signal in temporal difference learning with DA system function as described above, DA system dynamics may also guide cortical-striatal based reinforcement learning consistent with empirical accounts of the Matching Law (Preuschoff et al., 2006). Notably, the Matching Law also identifies with the generic version of the preferential attachment rule initially described by Barabasi as a fundamental law of scale-free system formation (Barabasi and Albert, 1999). Repeating this rule as first presented in section 2, a scale-free system as opposed to a random one will be formed if, when a new node is added, that node connects to an existing node i with probability

$$\Pi \left({\mathrm{k}}_{\mathrm{i}}\right)=\frac{{\mathrm{k}}_{\mathrm{i}}}{\Sigma {\mathrm{k}}_{\mathrm{j}}}$$

where k_i is the number of already existing connections associated with node i and Σ k_j is the total number of connections already present in the network (Barabasi and Albert, 1999).

Although the similarities between the right side of Barabasi’s equation and the Matching Law equality appear simple, they suggest a relationship bearing important implications for understanding the role of DA in the self-organizing structure of motivational-behavioral repertoires. Just as animals learn to allocate motivational effort according to the relative reward provided by each goal, so might new behavioral nodes in an evolving motivational-behavioral repertoire develop preferential attachment via motivational pathways with other more highly connected behavioral hubs that most efficiently mediate survival-oriented behavior. Interestingly, studies of foraging behavior in several species demonstrate that animals utilize a scale-free-like topographic search strategy to most efficiently acquire the greatest resources (Viswanathan et al., 1999; Reynolds, 2005). For instance, when searching sparse and randomly placed rewards, flight segments of birds (as a measure of effort) are distributed according to a scale-free power law k^-γ, where γ is most optimal near a value of 2 (Viswanathan et al., 1996; Viswanathan et al., 1999).

Although speculative, these lines of evidence suggest that analogous to DA dynamics in sculpting learning processes underlying the Matching Law, DA signaling may drive and maintain the most efficient mapping of behavioral organization onto the frequently changing temporal-spatial topology of survival-dependent resources in the external world. In terms of the proposed model, DA dynamics may represent a central neural substrate-mechanism for guiding the structural organization of motivational links and behavioral nodes toward a scale-free topology.

5. Addiction as a disease of altered scale-free structures of motivated-behavior

Having explored the application of scale-free systems as a brain and behavioral theory of motivated behavior, we now apply this concept as a translational model of addiction. At the core of this application, we interpret the DA activity shared among nearly all classes of addictive drugs as producing a process whereby drug-seeking and drug-taking become installed as increasingly well-connected behavioral nodes in the global systems of motivational-behavioral repertoires. Similarly, in temporal difference learning where the δ-signal is ‘exogenously’ augmented in modeling the DA-effects of addictive drugs, action choices leading to drug intake eventually dominate over motivations leading to natural reward goal acquisition (Redish, 2004).

The formal psychiatric diagnostic language of DSM-IV considers two main types of chronic substance use disorders: substance abuse and substance dependence (DSM-IV-TR, 2000). The term ‘addiction’ is not explicitly used in the DSM-IV-TR, although the term is widely used to describe substance use disorders of a chronic and interfering nature (Maddux, 2000). Core elements of addiction have been described as including craving, impaired control, compulsive engagement, and continued performance despite adverse consequences (Shaffer, 1999; Holden, 2001). Temporally, the process of addiction typically follows three stages of progression from (i) substance use and experimentation to (ii) substance abuse, to (iii) substance dependence, with this last state generally considered the equivalent of addiction. The central feature progressing through these clinical stages is the increase in instances and persistence of drug use and drug seeking at the expense of other normal, healthy or adaptive behaviors. In the criteria for substance abuse, this feature is expressed in a relatively mild form in which substance use results in sporadic compromise of work obligations, family relationships, societal obligations, risk of bodily harm, and/or legal trouble. In substance dependence, this feature has progressed substantially. Signs of somatic adaptation (tolerance and withdrawal) and medical compromise resulting from chronic drug use may occur in parallel with a markedly diminished ability to stop or consistently reduce substance use. Drug-seeking and drug-taking become major preoccupations and behaviors that compromise or eliminating adaptive social, occupational or recreational activities.

Representing the progression of addictive disease in behavioral economic terms, the large but finite repertoire of the individual’s behaviors may be visualized graphically, where the amount of time or motivational effort allocated to drug seeking/taking consumes an increasingly large portion (Figure 6). Since motivational-behavioral repertoires of individuals are large but finite, the increasing proportion of behavior devoted to drug-seeking/taking will necessarily result in the reduction or elimination of other adaptive behaviors previously present in the system.

Figure 6.

Allocation of the behavioral repertoire in the addiction process. Drug-seeking and drug-taking increasingly dominate the behavioral economy.

Addiction progression can be further schematized as a changing scale-free map of the motivational-behavioral repertoire (Figure 7). In this framework, drug-taking in substance use/experimentation is introduced as a behavioral node of the system. With progression of the addiction process, the drug-taking behavioral node becomes increasingly connected with other behavioral nodes in a scale-free-like developmental manner. However, addiction differs from the natural or typical development of a scale free-system where newly added nodes preferentially grow connections with already present hubs. In addiction, the other existing behavioral nodes, or those that might be introduced after the drug-taking node, may become preferentially attached to the drug-taking node, even though the drug-taking node is not itself yet highly connected at early stages. This drug-induced aberration in the organizational evolution of the scale-free system is expected by the abilities of abused drugs to release DA within the cortical-striatal assembly (Volkow, 2004b). In this model, addictive drugs usurp the role of the DA system to sculpt motivational-behavioral repertoires by forming and maintaining motivational links according to a scale-free organization.

Figure 7.

Progressive alterations in the scale-free structure of motivational-behavioral repertoires in the addiction process. Substance use/experimentation introduces the drug-taking behavioral node (single white node, upper left) in association with pharmacological stimulation of the DA system. DA is implicated in the building and maintenance motivational-behavioral repertoires according to a scale-free structure, achieving the most efficient mapping of behavioral organization onto the adaptive survival-dependent reward topography of the external world. In substance abuse, repeated bouts of drug-induced DA-stimulation produce a growing number of motivational links to the drug-taking node. In substance dependence, the drug-taking node becomes a highly connected behavioral hub in the scale-free system at the expense of many other adaptive natural motivations and behaviors formerly present in the system.

In the fully progressed state of addiction, the drug-taking node has become a highly connected hub in the network, rivaling and surpassing degrees of connectivity associated with other adaptive or vital behavioral nodes in the network. In this state, the multiple connection pathways to the drug-taking node represent increased proportions of the total motivational repertoire devoted to drug-taking, which may be observed objectively as drug-seeking or communicated by the individual as craving. As the motivational repertoire may also be thought of as significant probabilities of sequential progression from one behavioral node to the next, an increasing connectivity to the drug-taking hub will result in a net increase in the overall probability of drug-seeking or performing the drug-taking behavior. Corresponding to this progression of connectivity, the perceptions, thoughts or feelings usually associated with the performance of one behavior and its sequential progression to drug-taking can become ‘relapse-triggers’ or ‘drug-craving cues’ as they presage the drug-seeking connectivity path to the drug-taking node.

6. Clinical and Research implications

As a translational theory, this model explains several key clinical-epidemiological aspects of addiction including its chronicity, resistance to treatment (robustness of established disorder), and its onset in vulnerable populations including adolescents/young adults and in the mentally ill (Chambers et al., 2001; Chambers et al., 2003; Bickel and Potenza, 2006).

6.1 Chronicity and treatment resistance of addiction

The scale-free model of addiction holds that once the drug-taking node becomes a highly connected hub in the scale-free motivational-behavioral repertoire, the system remains robust to perturbation associated with random changes in specific behavioral nodes or their associated connections. Thus, the effects of time and current treatments for drug addiction may not efficiently influence the established scale-free structure surrounding the drug-taking node in highly addicted individuals. That is, the resiliency of the scale-free network of addiction would be expected to be robust to interventions (Bickel and Potenza, 2006). This view may address how psychotherapeutic modalities for addiction work while explaining the lack of efficacy of many medication treatments. For instance, motivational enhancement therapy or group therapy may work by strengthening natural-adaptive motivational pathways in the motivational-behavioral structure while the drug-seeking pathways degrade with time. However, successful pharmacological manipulations of cortical-striatal circuits would need to specifically target and ablate the motivational representational sequences surrounding the highly connected drug-taking hub but without adversely influencing the motivational pathways interconnecting the adaptive-behavioral nodes, or the endogenous capacity of the cortical-striatal/DA system to organize new adaptive connections or behavioral nodes. While most candidate drug treatments for addiction do target components of cortical-striatal circuits, how they influence the neuroinformatics of these circuits for specifically combating addiction-related motivation remains unclear. A challenge is that both the natural-adaptive and addictive-drug-related motivational links of the proposed scale-free system are biophysically instantiated by alternate sets of firing pattern representations mounted by the same neurobiolgical substrates. Nonetheless, new progress in the treatment pharmacology of addiction is underway as investigations test anti-craving drugs active at GABAergic and glutamatergic receptor systems of the cortico-striatal assembly (Kalivas and Volkow, 2005; O Brien, 2005), components that appear to be most directly involved in generating the neural representations and plasticity of the motivational-behavior repertoire.

6.2 Adolescence as a critical period of addiction vulnerability

Adolescence is a period of rapid acquisition of new skills and behaviors necessary for adult roles corresponding to developmental changes in the ventral cortical-striatal module (Chambers et al., 2003). During this active growth phase of the motivational-behavioral repertoire, the DA system may have a particularly potent influence on the relatively highly plastic prefrontal cortex and ventral striatum in organizing the addition of many new behavioral nodes and motivational links according to a scale-free topology. This situation may entail a more rapid installment of drug-taking behavior as a highly connected behavioral hub, as reinforced by a heightened sensitivity to the DA-active properties of addictive drugs in adolescence. Perhaps more ominously, studies of the development of natural and artificial scale-free systems indicate that once a highly connected node becomes established in the network, it tends to collect additional links associated with the subsequent introduction of other nodes; the longer a hub is present, the more highly connected it becomes (Barabasi and Bonabeau, 2003). In this manner, the introduction of a drug-taking behavioral node earlier in development (i.e., during adolescence) will accumulate greater motivational salience (connection links) with subsequently introduced behavioral nodes leading to an increased dominance of the drug-taking hub in adulthood. This model corresponds to clinical data indicating that earlier onset substance use in adolescence predicts greater addiction severity and involvement of greater number of substances (Taioli and Wynder, 1991 546; Kandel et al., 1992; Anthony and Petronis, 1995).

6.3 Addiction vulnerability in the mentally ill

Abnormalities of scale-free motivational-behavioral repertoires inherent to varieties of mental illness before drug exposure, may invite an accelerated process of addiction upon drug exposure. Nearly all forms of major mental illness including schizophrenia, bipolar disorder, major depression, post-traumatic stress disorder, antisocial and borderline personality disorder entail increased vulnerability to acquiring substance use disorders involving all of the major illicit and legal addictive drug groups (O Brien et al., 2004). Post-mortem and in-vivo imaging analyses spanning these disorders indicate that one or more cortico-limbic substrates that are intrinsic to (or provide glutamatergic inputs to) the ventral cortical-striatal assembly, including the prefrontal cortex, hippocampal formation, and amygdala, are morphologically or functionally disordered in these conditions (Weinberger et al., 1994; Callicott and Weinberger, 1999; Charney et al., 1999; Bremner et al., 2000; Driessen et al., 2000; Raine et al., 2000; Benes and Beretta, 2001; Chambers et al., 2001; Blumberg et al., 2003; Volkow, 2004a). Moreover, animal models involving disruptions of these brain areas induce both psychiatric syndromes and signs of addiction vulnerability (Wolf et al., 1995; Schmelzeis and Mittleman, 1996; Weissenborn et al., 1997; Chambers and Self, 2002; Holmes et al., 2002; Chambers et al., 2004; Chambers and Taylor, 2004; Chambers et al., 2005). In particular, direct damage to the NAc itself produces a robustly impulsive phenotype (Cardinal et al., 2001). Because these brain centers are responsible for the formation of specific firing patterns in the ventral cortico-striatal module, their dysfunction may: a) cause aberrant jumping between firing patterns comprising the motivational repertoire consistent with clinical constructs of impulsivity/poor decision making; and/or b) they may limit the set of firing patterns comprising the motivational repertoire that can be generated in the ventral module, consistent with clinical constructs of compulsivity. In either case, the DA effects of addictive drugs may provide an abnormally robust impetus to restructuring motivational-behavioral repertoires in a scale-free manner surrounding a drug-taking node relative to the capacity for endogenous DA to reliably direct scale-free motivational structuring around natural adaptive reward-related behavioral nodes. Consistent with these notions, recent computational work modeling DA with the δ-signal in temporal-difference learning suggests that extinction from drug-seeking requires the ability of neural networks to partition memory of contextual state space into two domains (i.e. create two closely-related but different memory traces), one in which drug learning has occurred, and another in which extinction learning can occur (Redish, 2005). In this way, memory of drug use is not erased in abstinence from drug use— rather, it is out-competed by new memories underlying extinction learning. Such partitioning may require the integrity of hippocampal communication to the ventral cortical-striatal module allowing the generation of new motivational firing patterns or sequences. Disruption of this connectivity in mental illness may impair generation of, switching into, or maintenance of drug extinction behavior, leading to higher probability of relapse in mental illness.

6.4 Scale-free model concept in light of behavioral sensitization and current theoretical models of addiction

Behavioral sensitization, in the form of progressive increases in locomotor activity after repeated non-increasing drug dosing, has long been utilized in addiction research; while addictive drugs of divergent psychoactive profiles can produce the effect, non-addictive drugs do not (Babbini and Davis, 1972; Pierce and Kalivas, 1997; Miller et al., 2001; Fish et al., 2002). However, theoretical uncertainty remains about how sensitization relates to the core instrumental processes of addiction, while self-administration has gained ‘gold-standard’ status as a method-model for addiction (Piazza et al., 1989; De Vries et al., 2002). Shedding some light on the possible relationship between behavioral sensitization and addiction, the incentive-sensitization model of addiction proposed by Robinson and Berridge proposes that addiction is essentially an incremental increase in the motivational-enhancing properties of addictive drug-taking (Robinson and Berridge, 1993). The scale-free model is consistent with this view and elaborates on it, while providing a more direct a priori explanation for why addictive drugs might produce behavioral as well as motivational sensitization. According to a scale-free interpretation, animals placed into a locomotor testing environment will display a complex mapping of behaviors. Certain behaviors (e.g. exploratory locomotion) occur more frequently because they are most adaptive in that setting. These behaviors may be considered as hubs in a scale-free like structure of behavior emerging in the experimental context. Because the addictive drug is administered extrinsically in a non-instrumental fashion, its DA agonist effects will occur fairly non-specifically during the execution of many behavioral nodes and their sequences. Acutely, the activational effects of DA will cause more rapid progression across the map of behavioral nodes, causing those behaviors that naturally occur most frequently to occur even more within a finite time (e.g. as with acute locomotor activation of psychostimulants). Chronically, repeated DA exposure will induce neuroplastic changes that exacerbate the already naturally present scale-free system, favoring more permanent increases in the execution frequency of behavioral hubs like exploratory locomotion.

The scale-free model is also consistent with other aspects of Robinson’s and Berridge’s model, including the distinction between drug liking and drug wanting. Consistent with incentive-sensitization, the scale-free model emphasizes the predominant role of motivational-behavioral computations over affective ones in the chronic addiction process. Thus, motivational computations are not functionally obligated to, and often do not associate subjective experiences of pleasure with motivated behavior. More directly, the immediate responsibility of motivation is to configure behaviors, as goals unto themselves, for increasing the probability of reward acquisition. In this way, the scale-free model may be viewed as differing from the Hedonic-set point theory proposed by Koob, which more directly links addiction with drug-induced homeostatic responses in affective representations (e.g. pleasure) that influence motivational processes (Koob and Le Moal, 1997). However, the proposed manner in which addictive drugs take advantage of the natural role of DA in maintaining a scale-free organization of motivated-behavior is reminiscent of Koob’s concept for homeostatic dysregulation in addiction. But rather than hedonic affective states becoming increasingly dependent on drug use at the expense of natural pleasures, the scale-free model proposes that drug use builds in motivational linkages around the drug taking node at the expense of natural motivations, given the finite economy of the motivational-behavioral repertoire.

6.5 Testable predictions and future directions

Although the proposed model is supported by existing data, empirical gaps warrant examination through hypothesis-driven experimentation. On the behavioral level, the theory predicts the existence of scale-free like organizational maps, heretofore unexplored in both human and laboratory animal behavior. While current evidence suggests that motivated behavior at a group level conforms to a scale-free organization (e.g. mammalian social, communication networks (Albert et al., 1999; Albert and Barabasi, 2000; Liljeros et al., 2001; Lusseau, 2003; Schneeberger et al., 2004)), confirmation of scale-free like development and organization of motivational-behavioral repertoires within individuals is needed. In current behavioral research in general, and in animal modeling of addiction in particular, dependent measures of behavior typically examine only increases or decreases in one or two discrete behaviors (e.g. lever pressing) in highly constrained environments. Emerging technologies for behavioral research such as position or activity-dependent remote telemetry allows for automated assessment of the frequency and sequential organization of many behaviors emitted by one or small groups of subjects (Chen et al., 2005; Clever Systems, 2005). Application of these methods in realistic and complex behavioral settings should facilitate data gathering surpassing the quality and quantitative resolution of Grant’s work (section 3.2) for describing organizations of motivated behavior. These advanced methods will also allow testing of the developmental aspects of the proposed theory. Examining the normal evolution of behavioral repertoires from prenatal to adult ages can assess scale-free like growth of motivated behavior.

In higher order animals, including humans, defining scale-free like behavioral structures is expected to be considerably more challenging, owing to the extreme complexity of behavioral repertoires observed in these species. Moreover, the manner in which regularity of behavior occurs as a result of biological, circadian, or cultural/environmental rhythms (e.g. the work week vs. the weekend) would be expected to obscure or complicate assessment of systems of behavior, or create some regions of the motivational-behavioral repertoire that are less scale-free than others. At this level, and in contrast to lower order laboratory animals, experimental conditions, or windows of data gathering, will likely need to be artificially constrained or more carefully selected, rather than expanded. However, consistent with approaches used with laboratory animals, testing of the scale-free model of motivation and addiction in humans will require greater emphasis on objective measures of behavior, non-verbal performance, or drug intake, as opposed to the use of self-reporting and historical rating scales.

Testing the neurobiological dimensions of the model can involve probing of components of cortical-striatal or DA-related circuits, to see how interventions effect the development, enactment, or environmentally-induced modifications of motivational-behavioral repertoires, corresponding to or diverging from a scale-free plan. For instance, does functional manipulation or impairment of DA-bearing VTA neurons during a limited phase of pre-adult development alter or curtail the structure of adult motivated behavior? How might manipulations to limbic centers that project into the ventral cortical-striatal module (i.e. hippocampus and amygdala), or changes to components of ventral vs. dorsal cortical striatal module itself, alter or ablate discrete behaviors or whole interconnected neighborhoods of the behavioral map? Might regionally specific frontal cortical, ventral striatal or thalamic interventions temporarily or chronically impair the co-configuring relationship between motivational links and behavioral nodes? Which behavioral nodes or motivational links are more susceptible to such interventions, those acquired more recently, those that are more sparsely connected, or both? Which of these interventions, and what of those involving neurotransmitter systems other than DA, glutamate and GABA, may cause motivational-behavioral repertoires to become more random-like in its connectivity structure?

The neuroinformatic aspects of the scale-free model are arguably the most speculative, but perhaps most deserving of further exploration. It is inherently difficult to use neurobiolgical methods alone to understand neuroinformatic phenomena. Moreover, the proposed theory does not require that the underlying neural substrates themselves are literally physically organized according to a scale-free principal. Instead they are proposed to process, co-configure, and sculpt neural firing pattern representations of motivated-behavior so that they may conform to a scale-free organization. Thus, searches for scale-free physical-structural attributes of cortical-striatal neurons, or neural connectivity might be tangential to the proposed theory. However, frequently emerging neurobiolgical-behavioral correlation findings, not well explained by existing theory, may be better understood from the neuroinformatic perspectives of the proposed model. For example, while psychostimulant treatment in rats increases both c-Fos immediate early gene expression in the striatum and locomotor activity, the expected positive relationship between drug induced c-Fos expression and locomotor activity breaks down under various conditions such as with DA-antagonist treatment (Mura et al., 2004), whether the animals have developmental-hippocampal lesions (Lillrank et al., 1996), or depending on which striatal sub-region is examined (Hedou et al., 2002). Rather than viewing psychostimulant-induced c-Fos expression as merely a pharmacological result of receptor activation on the cellular level, the proposed theory would interpret c-Fos expression densities as measures of the way the drug or drug history changes overall cortical-striatal information processing leading to differential behavioral results—c-Fos expression patterns as cumulative-time measures proportional to the extent of variability and/or frequency of recurring representational firing patterns emerging across specific striatal network ensembles. From this perspective, less c-Fos density in particular cortical-striatal compartments, and more in others, might correspond to greater frequency or duration of one particular behavioral program (e.g. locomotion vs. sustained catatonia).

Clearly, for more definitive penetration of the neurobiological-neuroinformatic frontier suggested by the propose model, investigations that wed computational neural network simulations with methodologies of neurobehavioral research will be particularly important. For instance, network simulations and behavioral investigations may be used to test whether DA dynamics, operating in terms of a temporal-difference-like δ-signal, promotes scale-free structuring of learned complex behavior. Network simulations in conjunction with in-vivo polyneuronal and multi-site recording techniques (e.g. sampling ventral and dorsal striatum simultaneously) in behaving animals may also be applied. These techniques could explore whether neural representations sequentially represented in the ventral and dorsal cortical-striatal modules emerge and co-configure in an inter-dependent manner according to a scale-free organizational motif corresponding to the predicted organization of motivated behavior. Already, various studies examining physiological (Stam and de Bruin, 2004 1170; Freeman, 2005 1167; Grinstein and Linkster, 2005 1166) or neuroimaging measures of brain activation states (Eguiluz et al., 2005; Achard et al., 2006) suggest scale-free organizational motifs may describe the functional relationships between distributed cortical brain regions. To the extent that striatal firing patterns are generated by or influence temporally changing neocortical activation states, these data may point to a scale-free organization to temporally emerging striatal firing patterns. Based on these approaches, neuroimaging or physiological studies of motivational neurocircuitry in addicted and non-addicted states could be considered in relationship to the proposed scale-free model (Bjork et al., 2004; Hommer, 2004; Hommer et al., 2004; Knutson et al., 2004; McClure et al., 2004; Reuter et al., 2005).

In summary, the proposed scale-free model has considerable potential as a comprehensive translational theory of motivation and addiction. While conceptually consistent with the incentive-motivational theory of addiction as proposed by Robinson and Berridge, the neuroinformatic aspects of the proposed model, by incorporating neural network-based perspectives on information processing and plasticity, provides a new way to bridge incentive-motivational theory to micro-level research defining the concerted effects of genes (Egan et al., 2001; Hariri et al., 2002) and environment (Caspi et al., 2003) in increasing addiction vulnerability. This follows as neural network simulations can readily model the extent to which network and informatic plasticity is the integrative product of environmentally derived information flowing through neural networks and the molecular-genetic properties of the network’s constituent neurons. The scale-free model also frames incentive-motivational theory within a macro-level conceptual structure that accounts for major clinical level phenomena such as adolescent development and psychiatric illness as addiction vulnerability states. Ultimately, empirical gaps and testable predictions of the model will be most useful if they guide integrative investigations wedding computational, animal behavioral and human studies on cortical-striatal systems functioning and plasticity toward preventing and treating addictions.