How psychological framing affects economic market prices in the lab and field

Abstract

A fundamental debate in social sciences concerns how individual judgments and choices, resulting from psychological mechanisms, are manifested in collective economic behavior. Economists emphasize the capacity of markets to aggregate information distributed among traders into rational equilibrium prices. However, psychologists have identified pervasive and systematic biases in individual judgment that they generally assume will affect collective behavior. In particular, recent studies have found that judged likelihoods of possible events vary systematically with the way the entire event space is partitioned, with probabilities of each of N partitioned events biased toward 1/N. Thus, combining events into a common partition lowers perceived probability, and unpacking events into separate partitions increases their perceived probability. We look for evidence of such bias in various prediction markets, in which prices can be interpreted as probabilities of upcoming events. In two highly controlled experimental studies, we find clear evidence of partition dependence in a 2-h laboratory experiment and a field experiment on National Basketball Association (NBA) and Federation Internationale de Football Association (FIFA World Cup) sports events spanning several weeks. We also find evidence consistent with partition dependence in nonexperimental field data from prediction markets for economic derivatives (guessing the values of important macroeconomic statistics) and horse races. Results in any one of the studies might be explained by a specialized alternative theory, but no alternative theories can explain the results of all four studies. We conclude that psychological biases in individual judgment can affect market prices, and understanding those effects requires combining a variety of methods from psychology and economics.

Different social sciences emphasize fundamentally different views of individual behavior and, consequently, of market misbehavior. When markets produce bad outcomes, as in the recent economic crises, psychologists are inclined to emphasize the roles of individual cognitive mistakes such as optimism, and emotional forces such as greed and fear. Economists are inclined to emphasize that political pressures and poorly designed economic institutions created incentives for rational actors to make decisions that had negative external effects for society. A scientific way to make progress in evaluating these viewpoints is to investigate whether the psychological view could be correct: i.e., Can systematic, psychologically plausible mistakes in individual judgment shift economic market outcomes away from statistically normative standards?

How individual judgment bias affects aggregate economic and political activity is likely to depend on details of underlying psychology and relevant economic and political institutions. For example, it is well known that there can be a “wisdom of crowds,” in which group predictions are better than those of most individuals in the group (1) through averaging of idiosyncratic errors (2). In particular, if the distribution of judgments is symmetrically distributed around the true value, then the average judgment is expected to be close to the true value. However, if judgments of group members are biased in a particular direction, then the group judgment will remain biased (and could even be more strongly biased than individual judgments).

However, aggregate judgments reflected by market prices are likely to be special compared with other kinds of group aggregation. An economic market price is a “group judgment” in which each trader’s opinions are weighted by their trading activity; confidence has to be backed by money. As a result, even if most investors make a judgment mistake, if a small number of well-informed and well-capitalized traders do not make that mistake, then the disproportionate influence of the “smart money” could lead to sensible pricing and ideal capital allocation for the entire market.

This optimistic conclusion about markets is sensitive to assumptions about rationality and trading structures (3). Economic theory shows that whether behavioral biases become smaller or larger due to trading activity depends on whether more rational traders have an incentive to trade against those biases to exploit arbitrage opportunities, or trade with those biases to profit (as in some price bubbles), possibly multiplying the effect of a small bias (4–7). For example, financial money managers with large sums to invest may be evaluated by clients who are sensitive to short-term returns. The clients’ “short horizons” can then limit savvy managers’ willingness to bet aggressively that market mispricing will be rapidly corrected (8, 9), which delays corrections or even magnifies them.

Given these considerations, the simplest test for psychological influences on market activity requires two ingredients: (i) a well-established, robust psychological pattern of bias in individual judgment; and (ii) a type of market in which that pattern would clearly be revealed by observable economic market data such as prices (e.g., 10). Our studies have both of these ingredients. We explore whether a specific judgment bias affects prices in a laboratory experiment and a field experiment, and in two natural prediction markets.

This self-contained series of studies is unique because it shows the effect of one mathematically specified judgment bias across very different levels of judgments and market prices. Previous studies have extrapolated from interpretations of particular experimental data to inspire similar analyses of stock market data (11, 12). However, those studies have not directly linked a set of experimental and field data closely around a single phenomenon.

The psychological pattern we explore is how dividing the set of possible events into a particular “partition” influences the perceived likelihood of those events. For instance, when evaluating the possible closing values of the Dow Jones Index on December 31, one might evaluate the events {<12,000; 12,000–13,000; >13,000} or {<13,000; 13,000–14,000; >14,000}. The tendency for the specific partition of the state space to influence judged probabilities is known as partition dependence (PD).

The markets we explore are prediction markets in which prices reflect an aggregate judgment of the likelihood of a set of possible naturally occurring events. In prediction markets, financial claims are traded that pay a fixed sum of money when a precisely defined event occurs. For example, on September 23, 2012, a $10 claim on the event “Barack Obama to be reelected President in 2012” traded for $7.08. This price implies a collective judgment, 6 wk before the election, that Obama’s reelection has a 71% chance of occurring. Prediction markets provide an ideal domain for a test of psychological bias in market prices because asset values depend only on judged likelihoods of events. Biases in likelihood judgments could therefore be clearly reflected by prediction market prices.

Our market data therefore create an empirical competition between the psychological and economic points of view about whether PD in individual likelihood judgments is also evident in market prices. Roughly speaking, psychology says “yes” and economics says “no.”

Early prediction markets were based on remarkable laboratory experimental evidence that modest amounts of informed trading could lead to prices that “aggregate” information that is dispersed among people (13). Inspired by these stylized laboratory examples, the Iowa Experimental Markets on political events were created in 1992 (14), followed a decade later by other markets on worldwide current events (e.g., Intrade) and internal markets used by companies for forecasting company outcomes (15, 16).

Prices in prediction markets are usually interpreted as a collective probability assessment, or “crowd wisdom” of event likelihood (adjusted for any association between risk taking and judgment that biases the market price away from the mean; SI Appendix, section S2). These predictions are found to be generally at least as accurate, and often more accurate, than those derived from opinion polls or expert judgments (17–19). Prediction markets provide an ideal test of PD because their events are often continuously distributed variables that, for practical reasons, partition the state space into a mutually exclusive and exhaustive set of discrete events (intervals). Previous studies of these markets—and economic theorizing about prediction markets as well—have assumed that the particular way in which the state space is partitioned should not affect prices. However, if individual judgmental biases influence markets, then how partitions are explicitly presented could influence market prices through these individual judgments.

Many studies of individual judgment show that “unpacking” a single interval [I₁ ∪ I₂] into two separate ordered subintervals I₁ and I₂, which are logically equivalent, reliably increases judged probability [i.e., the sum of the judged probabilities P(I₁) + P(I₂) > P(I₁ ∪ I₂)]. Unpacking an interval into its explicit parts seems to draw extra attention to those parts and increase their judged likelihood.

Dependence of judgment on the salient partition of the state space seems to reflect an anchoring of judged probability on a diffuse “ignorance prior” belief at 1/N (when there are N explicitly presented intervals), with insufficient adjustment from the 1/N anchor, due to limited imagination about how different the intervals are. For instance, in the Dow Jones example above, the interval X <= 13,000 is two of three events in the first partition (ignorance prior = 2/3) and one of three events in the second partition (ignorance prior = 1/3). This simple 1/N hypothesis turns out to be a useful tool for making predictions about when judgment and price errors will be small or large.

Previous evidence of PD has been found in judgments by highly trained decision analysts (20), in valuation of hypothetical insurance policies that unpack possible causes of death (21), and is robust to learning (22) and financial motivation (20, 23, 24) in laboratory experiments. PD is also evident in economic field data on allocations of savings to personal investments (10, 25), and allocations of capital to businesses in multidivision firms (26, 27). There is also limited previous evidence of PD in prediction markets (18).

We investigate both whether a PD bias exists in individual judgments about naturally occurring events, and, if so, whether prediction market prices propagate that bias or suppress it. The combination of laboratory experiments, field experiments, and naturally occurring market data that we report also addresses questions in economics about whether psychological laboratory effects generalize to important economic field settings (28–30).

We present four laboratory and field studies using different events, subject populations, and market mechanisms. If a common explanation emerges for results in all of the four, then Occam’s razor should elevate that common explanation over a set of ad hoc idiosyncratic study-specific explanations.

Study 1: A Short Laboratory Experiment

Subjects (n = 192) traded bets linked to the eventual numerical value of financial, sports, and weather events (in balanced order across groups). The full numerical range for each event value was divided into four intervals. For each of two separate trading groups, either the two lowest-valued intervals or the two highest-valued intervals were combined into a single interval (Fig. 1A, lower part). That is, assets linked to two separate intervals I₁ and I₂ were traded by one group, whereas the single interval equal to their union, I₁ ∪ I₂, was traded by another group.

Fig. 1.

Design and evidence of PD in a laboratory experimental prediction market. (A) Time course of a typical experimental session in study 1 (upper part) and construction of assets for the two DAX Index (German stock market) partitions (lower part). The digital option will pay a fixed amount (1€) only if the DAX closes within the specified interval 2 wk in the future. (B) The development over time of price differences of the packed and summed unpacked assets, for the sports assets in study 1. Prices are averaged over all 12 market replications. The difference in the final prices is significant (Kruskal–Wallis, P < 0.001). The slope of the time trend for the difference in prices is −0.0094 (Upper) and −0.0159 (Lower).

Both groups were told about their own partition of intervals and, crucially, they were also told about the other group’s partition. This instruction to both groups is essential because (in theory) PD could result from an inference by traders that the market designer chose partitions that tend toward equal probability; such an inference would coincide with the 1/N heuristic. In our design, however, both groups are told about both partitions. Thus, any inferences that participants draw from the choice of the partitions should be the same in both groups so that any difference in the groups’ behavior cannot readily be attributed to information conveyed by the choice of partitions.

Because winning bets paid 100 cents, holding one asset for every interval was sure to yield 100 cents when the event’s numerical value is determined. To encourage exploitation of pricing mistakes, subjects were allowed at any time to invest 100 cents to buy a portfolio of one share on each event from the experimenter. Thus, if the sum of offered share prices was above 100 cents, low-risk arbitrage was possible. Similarly, arbitrage could be exploited if a set of shares on each event could be bought for less than 100 cents. Such arbitrage was also common; small differences occurred in 85% of markets and were arbitraged in an average of 12.83 s (SI Appendix, section S1.7).

Trading was conducted in two 10-min rounds for each event, using a customized continuous double auction (CDA). The CDA allows subjects to submit either bids to buy or asks to sell, or to accept current bids and asks to make a trade. Hundreds of experiments have shown that this trading mechanism reveals information reliably and rapidly (31) and creates near-maximum gains from trade (32). Subjects also expressed their individual beliefs about event probabilities before and after trading, so we could compare market prices with the distribution of those beliefs and see if market trading changed beliefs from before to after (Fig. 1A).

Average prediction market prices exhibited a large and persistent degree of PD (Fig. 1B). For instance, the average of the last three trade prices was 0.354 for Muenster temperature [20–23.9 °C] and 0.496 for Muenster temperature [≥24 °C] for the first group. However, that average was only 0.707 for Muenster temperature [≥20 °C] for the second group (SI Appendix, section S1.1). The difference of 0.143 (=0.354 + 0.496 − 0.707) is a measure of PD.

Overall, the difference between the average of the last three prices of unpacked assets and the associated packed asset was 0.267, 0.229, and 0.149 for finance, sports, and weather events, respectively. The corresponding differences in individual pretrading judgments were 0.312, 0.261, and 0.278. Posttrading judgment differences were lower than the pretrading differences by 0.055, 0.005, and 0.052. The PD gap in prices is also shrinking slightly across the length of the trading periods (Fig. 1B). These small effects suggest that market trading can modestly reduce PD in prices and postmarket judgments.

Study 2: A Multiweek Field Experiment

We found that the PD effect does shrink a little over the course of the 20-min trading span in the laboratory experiments of study 1. This observation motivates a second study of a prediction market involving sporting events that unfold over about 8 wk (n = 456 participants). The events are American National Basketball Association (NBA) basketball playoffs (2005–2006) and the 2006 soccer World Cup. These prediction markets trade assets over five intervals of total victories by NBA teams, or total goals scored by national teams across the World Cup (in regulation play, excluding shoot-outs). As in study 1, there were two separate trading groups in which assets linked to goal counts of four teams were traded. In each group, one interval was unpacked into two subinterval assets and another interval (unpacked in their counterpart group) was packed into a single asset. The trading mechanism and all other methods were very similar to those in study 1 (SI Appendix), but they were adapted for Web access only and trading was possible for several weeks.

The PD effects are similar in magnitude to those in the laboratory markets. Individual belief judgments summed across unpacked intervals are a median of 0.20 higher in NBA and 0.15 higher in World Cup markets than in comparable packed intervals. There is not much visible convergence over time (Fig. 2A gives results for the most actively traded NBA team, the Dallas Mavericks).

Fig. 2.

Prices and pseudoarbitrage profit opportunities in NBA and World Cup field experimental markets. (A) Price chart for packed and unpacked assets in prediction market trading the number of wins by the Dallas Mavericks, DAL. Numbers at the top of the graph indicate cumulated number of wins. (B) For each NBA playoff team, average levels of partition dependence (PD) arbitrage are positive and typically larger than reverse PD arbitrage (which is often zero). (Reverse) PD arbitrage occurs if a hypothetical profit could be made by buying (selling) the packed event in one market and selling (buying) the unpacked events in the other market. At times where no bids or asks were available, the more conservative from the previous and subsequent price are used, i.e., the price that makes the occurrence of (reverse) PD arbitrage less likely. Levels are set to 0 if no arbitrage opportunity existed. The numbers displayed in the figure are the time-weighted averages of the available PD (or reverse PD) arbitrage levels over the complete trading period for each team generated by the low intervals (I₁ and I₂) and by the high intervals (I₃ and I₄).

Creating an index of the amount of overall PD in these prices is complicated by the fact that liquidity was low. Because there was not always a range of bids and asks at which to trade, what “the price” is at any moment in time is ambiguous. A conservative approach interpolates hypothetical prices by assuming that buyers could always buy at the higher of the last price or the next purchase price (and oppositely for sellers). This approach creates a continuous flow of virtual prices, to measure how much could be hypothetically earned by selling assets on two unpacked events and buying the cheaper packed event (called PD arbitrage) or executing the opposite trade (reverse PD; which should never be profitable if PD exists). The time-weighted average hypothetical arbitrage profit is highly variable among teams, but is much higher in exploiting PD effects than exploiting reverse-PD effects (Fig. 2B and SI Appendix, section S2.6). A less conservative measure using bids and asks, when prices are absent, yields an even stronger conclusion (SI Appendix, section S2.7).

Study 3: Macroeconomic Indicators

Our third dataset is 153 large-scale “economic derivatives” prediction markets for four macroeconomic indicators, created by Goldman Sachs and Deutsche Bank (33). The markets are parimutuel in structure and were conducted 1–2 d before the indicators were released. We focus on only “digital options,” which pay off if the indicator value lies in a specific interval. A full set of digital option market prices for N numerical intervals spanning the entire range reveals an approximate probability distribution for that indicator’s likely value (Fig. 3, Center).

Fig. 3.

Graphical illustration of a mixture model of prediction market prices. The mixture model assumes that the probability distribution observed in the prediction market (Center) is a linear mixture of a 1/N naïve prior distribution with weight λ (Left) and an unbiased distribution reflecting traders’ best guesses with weight 1-λ (Right).

Suppose the observed market prices are equal to the weighted average of a hidden “best guess” (unanchored) price and a 1/N anchor value (Fig. 3). Then a simple statistical procedure can be used to impute the weight on the 1/N anchor that is implicit in observed prices (for details, see SI Appendix). The weights are not precisely estimated—e.g., there is no apparent 1/N effect for jobless claims—but for the other three indicators, the 1/N anchor weight appears to be positive (P < 0.10), and it is strongly positive overall (=0.44, P < 0.01). These results are consistent with a PD bias in the predicted direction. These results suggest that, although the Goldman Sachs–Deutsche Bank markets provided adequate predictions of the underlying macroeconomic indicators, the predictions were distorted by a bias toward equal probabilities for all traded events.

Study 4: Horse Races

The fourth and final dataset uses horse races. In parimutuel horse racing, bettors buy tickets on horses they think will win. The losers’ bets are divided among the winners (after a track takeout). These are prediction markets because the percentage of money bet on each horse is a collective perceived (subjective) probability of that horse winning. Because “long-shot” horses have a low perceived probability, a lot of money is paid out to the relatively few people who bet in the rare event that the long-shot wins (i.e., the rate of return, or “odds,” are high).

Many studies show that these long shots are overbet relative to their actual chances of winning (and favorites with high probability and low odds are relatively underbet). This tendency is called the “favorite–long-shot bias” and is well established across many countries and decades (34). Several different explanations have been offered, including frictions that prevent equilibration (35), interaction of information and timing (36, 37), a desire for positive skewness bets, and overweighting of the low probabilities of long shots winning (34).

The 1/N PD bias is consistent with the favorite–long-shot bias, because the odds of horses with true probabilities below 1/N (the long shots) will be overestimated if their odds are biased in the direction of a 1/N anchor. However, the 1/N bias also makes the stronger prediction that long shots will be more overbet (relative to actual winning probability) when the number of horses (N) in a race is smaller. No current theory of favorite–long-shot bias makes this prediction (SI Appendix, section S4).

Detecting the effect of the number of horses requires comparing perceived market probabilities (from actual market betting odds) to actual winning frequencies in different probability categories, and then comparing those perceived–actual differences across races with different numbers of horses. This has been done with a remarkable sample of all US horse race starts from 1992 to 2001 [6.3 million horse starts (34)]. This analysis shows an orderly pattern in which perceived probabilities of long shots are indeed higher in races with fewer horses, as predicted by the PD effect (Fig. 4). Perceived probability inferred from betting is consistent with placing a weight of 0.12 on a 1/N PD component (much lower than the weight of around 0.40 inferred in study 3).

Fig. 4.

The relation between perceived probability (estimated from aggregate market betting) and actual probability (sample relative frequencies) for races with different numbers of horses. 1/N bias predicts the curve from races with fewer horses will lie above the curve from races with more horses. The graph was created from a lowess-smoothed version (bandwidth = 0.4) of their (Snowberg and Wolfers, 2010) Fig. 1 for races with exactly 6, 8, 10, or 12 horses, and then converting racetrack odds to implied probabilities. (Sample sizes are multiples 0.044, 0.192, 0.145, and 0.053 of the full sample; the minimum is ∼250,000 horse starts.) Regressing perceived probabilities, for different numbers of horses and actual probabilities of 0.02, gives a weight on 1/N of 0.12. Analysis performed by Snowberg and Wolfers, based on data in their 2010 paper. (Reproduced with permission from Snowberg and Wolfers.)

Discussion

A central question that cuts across social sciences is how individual cognition influences collective activity. We focus on whether judged probability of a numerical interval is partition dependent, increasing when the interval is unpacked into equivalent adjacent subintervals. Two experiments and two field datasets measure how much PD is reflected in prediction market prices. All four types of data show substantial and persistent PD in market prices. PD is evident in a short (1-h) laboratory experiment, in a weeks-long field experiment on NBA and World Cup (Federation Internationale de Football Association, FIFA) outcomes, in a parimutuel prediction market forecasting economic indicators, and in a population of US parimutuel horse race markets. The studies span a wide range of structure, time, and trading domains; they are not meant to comprise a step-by-step “bridge” between laboratory and field (as was seminally done in ref. 38).

Combining these data sources is useful for the following reasons: General explanations based on trading frictions (35, 37) might apply to parimutuel markets but do not apply to the CDA laboratory and field markets; explanations based on information conveyed by the choice of partitions in horse race and economic indicator markets are eliminated by laboratory experiments; and explanations based on low financial stakes and trader experience are eliminated by the parimutuel field markets. Occam’s razor therefore favors the only interpretation that applies to all four datasets, which is that PD in individual judgments also influences market prices (cf. refs. 39–41).

Our studies also illustrate the complementarity between laboratory and field methods—an idea that is widely endorsed in experimental economics (29, 42). In this setting, complementarity means that the weakness of one method makes the compensating strength of a different method especially valuable. Studies 1–2 are strong for eliminating perceived information from partitions, but cannot answer questions about PD in natural settings. Studies 3–4 are strong for naturalism but weak on control of partition information.

A boundary argument—that markets always work to completely eliminate judgment errors (28)—is clearly not consistent with all of our data. In addition, although prediction markets do have a good track record of predicting naturally occurring events, compared with other types of prediction such as political polls, there is evidence of modest favorite–long-shot biases even in prediction markets (besides studies 3–4 in this paper), which is consistent with PD (18). Nonetheless, it is certainly useful to ask whether there is broader significance beyond prediction markets, for the impact of judgment biases in other types of economic markets.

Note that PD is just one simple example of “framing,” i.e., how descriptions can direct attention, exaggerating some features of asset value and suppressing others. The prediction markets we used are the simplest markets in which these effects can be shown. More generally, in asset and credit markets there are many ways to describe or package statistical features of possible asset values. These include categorical bond ratings (AAA), coarse categories (43) (bull and bear markets, National Bureau of Economic Research-classified recessions, glamour and value stocks), and emphasis of summary statistics such as 1-y or 5-y historical returns, a stock price’s peak in the last year, or a value-at-risk quantile. Deviations from Black–Scholes pricing of derivative options are also consistent with subjective weighting of the chance of exercise (44) (consistent with possible influence of an {exercise, no exercise} partition). Like explicit partitions, all these types of coarse descriptions compress information. The psychology behind PD suggests such information-compressing descriptions might naturally inhibit attention to the information that is hidden by compression, but that is normatively important.

More research is needed to see how generally shifts of attention based on descriptions affect market outcomes. They are likely to have the least effect in markets dominated by highly sophisticated traders who can ignore distracting descriptions and envision underlying value distributions. However, when there is high uncertainty about value, or when sophisticated traders care about value perceived by others who are less sophisticated (as in many asset markets), it is possible that framing effects could shift attention in a way that is not dampened throughout a financial system, or is in fact multiplied. More studies like ours are needed to gain a deeper understanding of such effects, and their impact and robustness, before prescribing effective remedies.