Choking and Excelling Under Pressure in Experienced Classifiers

Abstract

We extend previous work examining the effects of pressure on category learning to the effects of pressure on categorization performance in highly trained individuals. After extensive training on either a rule-based or information-integration classification task, half of the participants performed the same task on a fifth day while under pressure to earn a monetary bonus ($50) for themselves and a ‘partner.’ Performance of this group was compared to that of a low pressure control group who performed without the pressure manipulation. Pressure caused performance decrements for experienced classifiers performing both rule-based and information-integration tasks relative to the control groups. These results contrast with previous research where inexperienced classifiers choked on rule-based tasks but excelled on information-integration tasks. An additional ‘super pressure’ block of trials was given at the end of the fifth session. Under this type of pressure, participants performing an information-integration task outperformed those performing rule-based tasks. Implications for theories of choking under pressure are discussed.

Introduction

The anecdotal phenomenon of choking under pressure has been the subject of much laboratory research (e.g., Beilock & Carr, 2005; Markman, Maddox, & Worthy, 2006; Beilock, Kulp, Holt, & Carr, 2004; Masters, 1992). Choking under pressure occurs when someone underperforms on a task relative to their normal performance because of an acute stressor. Research has focused on two different explanations for choking. The Distraction hypothesis, which shares some similarities to Processing Efficiency Theory (e.g. Eynseck and Calvo, 1992), proposes that pressure leads to a decrease in available working memory resources, which in turn has a negative impact on the performance of cognitively demanding tasks (Wine, 1971). Alternatively, the Explicit Monitoring hypothesis, which shares some similarities to Reinvestment Theory (e.g. Masters and Maxwell, 2008), proposes that pressure causes increased attention to skill-focused processes which disrupts the performance of proceduralized tasks (Baumeister, 1984).

We test these accounts using highly trained participants. To place this work in context, we first discuss previous research exploring the Distraction and Explicit Monitoring hypotheses. Then, we present data from a study in which a total of 69 participants first received over 2500 trials of classification learning across four separate training sessions, and then were put under pressure in a fifth experimental session.

Previous research on Distraction and Monitoring

There is evidence supporting both the Distraction and Monitoring theories of choking under pressure. Studies supporting the Distraction hypothesis often come from working-memory intensive tasks. For example, Markman et al. (2006) studied category-learning tasks. Participants performed either a rule-based task which has been shown to recruit working memory resources (Ashby, Alfonso-Reese, Turken, & Waldron, 1998; Maddox & Ashby, 2004; Maddox, Filoteo, Hejl, & Ing, 2004; Zeithamova and Maddox, 2006), or an information-integration task which has been shown to recruit a procedural-based learning system which is not working-memory demanding (Maddox & Ashby, 2004; Maddox, Ashby, & Bohil, 2003; Ashby et al., 1998; Decaro, Thomas, & Beilock, 2008). In accordance with the Distraction hypothesis, novice participants choked while performing the rule-based task under pressure, but excelled while performing the information-integration task under pressure.

Further support for the Distraction hypothesis comes from Beilock and Carr (2005), who showed that performance of participants with high working-memory capacity declined more under pressure than did performance of those with low working-memory capacity. Additional studies using cognitively demanding tasks such as math problems also support the Distraction hypothesis (e.g. Ashcraft and Kirk, 2001; Beilock et al. 2004), and there is some evidence that skill-focused training can reduce choking under pressure in motor tasks such as golf-putting (Lewis and Linder, 1997; Beilock and Carr, 2001 Experiment 3).

Support for the Monitoring hypothesis has been demonstrated primarily in motor tasks such as golf-putting (Masters, 1992; Hardy and Mullen and Jones, 1996; Beilock and Carr, 2001 Experiment 4), simulated baseball batting (Gray, 2004), and free throw shooting (Liao and Masters, 2002). A common finding in many of these studies is that attention to skill-focused processes during training disrupts performance while under pressure, whereas training without being instructed to focus on the processes at hand (Liao and Masters, 2002), or training while performing a working-memory demanding task (Masters, 1992; Hardy et al., 1996) leads to improved performance under pressure. The Monitoring view assumes that skill-focused training gives participants explicit knowledge of the motor processes used to complete the task. Under pressure, participants then monitor their performance, and call to mind this explicit information, which has the paradoxical effect of hurting performance.

Of greater relevance to the current study are experiments conducted using expert participants. Gray (2004) had Division 1-A collegiate baseball players with 14–18 years of experience perform a simulated batting task while under pressure. He found direct evidence that performance pressure caused increased attention to skill-focused processes as confirmed by improved performance on a skill-focused task, and that this increased attention caused decrements in performance for expert participants. These results showed that, for expert baseball players, pressure did not cause a decrease in working memory capacity as proposed by the Distraction theory (Wine, 1971), but rather caused an inward shift in attentional focus as proposed by the Monitoring theory (Baumeister, 1984).

Overview of the Current Study

In the present study we examined the effects of pressure on the performance of participants given extensive practice with rule-based or information-integration classification tasks. Sixty-nine participants learned the categories in four one-hour sessions given on consecutive days, performing over 2500 learning trials each. Each participant then completed a fifth experimental session.

A critical difference between rule-based and information-integration category structures is that the optimal strategy for rule-based tasks is explicit (i.e., verbalizable), whereas the optimal strategy for information-integration tasks is implicit (i.e., not verbalizable). For example, for the category structure shown in Figure 1a, the optimal rule is to classify stimuli with relatively low spatial frequency into one category and stimuli with relatively high spatial frequency into the other category. In contrast, there is no verbalizable rule that can be employed to optimally distinguish the categories in the information-integration structure shown in Figure 1b. Here the optimal rule may be verbalized as: ‘if the spatial orientation is greater than the spatial frequency, classify the stimuli into one category, but if the spatial orientation is less than the spatial frequency, classify the stimuli into the other category.’ However, this type of rule is difficult or impossible to implement to solve the task because the relative magnitude of each spatial dimension cannot be easily compared.

Figure 1.

(a) Plot of stimuli from the rule-based category structure. (b) Plot of stimuli from the information-integration category structure.

Dual-Systems Theory of Classification Learning

There is abundant evidence that separate neural systems are recruited when learning to classify stimuli from rule-based and information-integration category structures (Maddox and Ashby, 2004; Ashby, Alfonso-Reese, Turken, and Waldron, 1998; Nomura, Maddox, Filoteo, Ing, Gitelman, Parrish, Mesulam, & Reber, 2007). The COVIS (COmpetition between Verbal and Implicit Systems) model details these separate neural systems (Ashby et al., 1998). COVIS assumes that a frontally-mediated hypothesis-testing system that depends heavily on working memory and executive attention dominates rule-based learning, whereas a basal-ganglia mediated procedural learning system that is sensitive to the timing of the feedback, but is not demanding of working memory dominates information-integration learning (Ashby et al., 1998; for a review see Maddox and Ashby, 2004).

An important component of this paradigm is that rule-based category learning is more analogous to performing working-memory demanding tasks such as solving math problems that have been used previously to study choking under pressure (e.g. Beilock and Carr, 2005). Information-integration category learning is more analogous to learning motor tasks such as golf-putting or baseball batting that have been used previously to study choking under pressure (e.g. Beilock and Carr, 2001; Gray, 2004). Information-integration category learning has been closely related to procedural learning (Ashby, et al, 1998). For example, Ashby and colleagues found that switching the response keys between training and transfer phases in rule-based and information-integration classification tasks impaired information-integration transfer performance, but not rule-based transfer performance (Ashby, Ell, and Waldron, 2003). This suggests that the explicit rule-based system learns abstract category labels, whereas the procedural-based information-integration system learns response positions. While links between information-integration classification learning and motor tasks should be made with caution, there is some evidence that information-integration learning requires a more procedural system.

Distraction and Monitoring theories make contrasting predictions regarding learning under pressure for rule-based and information-integration tasks. The top rows of Table 1 summarize the learning predictions based on a dual-systems theory of classification. Distraction theory states that pressure will reduce the amount of available working memory resources. Because rule-based category learning requires working memory, it will suffer under pressure. However, information-integration category learning does not require working memory, and, in fact, low working memory has been shown to improve information-integration learning (DeCaro et al., 2008; however see Tharp & Pickering, in press). Thus, the Distraction hypothesis predicts excelling under pressure in information-integration learning. The Explicit Monitoring hypothesis makes the opposite predictions with respect to learning. The Explicit Monitoring hypothesis states that pressure causes one to explicitly focus on the processes needed to carry out the task. Because rule-based learning relies on explicit hypothesis-testing, the Explicit Monitoring hypothesis predicts that pressure should improve rule-based learning. In contrast, monitoring of explicit processes is counterproductive to information-integration. Markman et al. (2006) examined the effects of pressure on rule-based and information-integration learning and found choking under pressure for rule-based learning, but excelling under pressure for information-integration learning in support of the Distraction hypothesis.

Table 1.

Predicted performance for participants performing under pressure

	Rule-Based	Information-Integration
Learning (Dual-System)
Distraction	Choking	Excelling
Monitoring	Excelling	Choking
Experienced (Cortical)
Distraction	Excelling	Excelling
Monitoring	Choking*	Choking*
Summary of Results
Pressure	Choking	Choking
Super-Pressure	Choking	Excelling

^*As explained in the Introduction, Monitoring Theory may also predict a greater degree of choking for Rule-Based participants than Information-Integration participants due to a greater amount of explicit knowledge about the category structure for available for Rule-Based participants.

An Alternative Theory of Classification for Experienced Classifiers

Markman et al. (2006) examined classification learning, whereas the current study examines classification performance after extensive training. The neural systems used to learn the two category structures may be different from the systems used to classify stimuli from each structure once the categories are well-learned, and thus the effects of pressure may be different. Ashby and colleagues’ (1998; see also Ashby, Ennis, & Spiering, 2007) proposed a Subcortical Pathways Enable Expertise Development (SPEED) theory of expert classification that assumes that the basal-ganglia mediated procedural learning system used to learn information-integration category structures is gradually replaced by a much faster and more direct cortico-cortical system that involves projections from extrastriate visual cortices directly to motor and premotor areas. It is also reasonable to propose that a similar cortically mediated response system gradually replaces the frontal-based explicit system responsible for learning rule-based categories as expertise develops and knowledge of the categories becomes automated. This view suggests that performance of experienced classifiers will be cortically mediated, regardless of which structure they are learning. The predictions for this view are summarized in the middle rows of Table 1.

If we assume that experienced classifiers use the same systems that have been shown to mediate responding during learning, then Distraction Theory predicts choking under pressure for participants performing rule-based tasks and excelling under pressure for participants performing information-integration tasks, while Explicit Monitoring Theory predicts excelling for participants performing rule-based tasks and choking for participants performing information-integration tasks. In contrast, if we assume that the responses of highly experienced classifiers are mediated by a direct automated system, then Distraction Theory predicts excelling under pressure for participants performing both rule-based and information-integration tasks, while Monitoring theory predicts choking under pressure due to an increase in skill-focused attention that disrupts the automated system. Previous research suggest that Distraction Theory is more appropriate for performance under pressure during learning (e.g. Markman et al., 2006), while Monitoring Theory is more appropriate for performance under pressure for highly experienced participants (e.g. Gray, 2004; Masters, 1992)

Even though this view assumes that experienced classifiers ultimately use the same system to make their classification responses regardless of the category structure, this view predicts that pressure harms rule-based classification performance more than information integration performance. This difference arises because of differences in the amount of explicit knowledge about the categories available to experts classifying stimuli from rule-based or information-integration category structures. Because rule-based tasks are learned via an explicit hypothesis-testing process, the explicit knowledge about the category structure available to participants performing rule-based tasks is much greater than that for participants performing information-integration tasks. Assuming that pressure causes an increase in skill-focused attention which will disrupt the automated system (i.e. Monitoring Theory), the difference in explicit knowledge about the category structures may influence the extent of the observed performance decrements for highly-trained participants performing each task while under pressure. While Monitoring theory predicts performance decrements for participants performing both tasks while under pressure due to disruption of the automated system caused by an increase in skill-focused attention, the decrements may be greater for participants performing rule-based tasks because they have explicit knowledge of the category structure with which they can monitor their performance. In contrast, participants performing information-integration tasks may experience less severe decrements in performance because they have less explicit knowledge of the category structure with which to monitor their performance. This would corroborate the findings of several studies mentioned above for which more explicit knowledge about the process of performing the task during training led to worse performance while under pressure (e.g. Gray, 2004; Masters, 1992; Liao and Masters, 2002).

To test these competing hypotheses we conducted an experiment in which participants were first given extensive training on either rule-based or information-integration classification tasks. After four 1-hour sessions of training on the task, participants completed a fifth session. During the fifth session half of the participants were given a pressure manipulation that has been used in many previous pressure experiments and the other half were simply asked to do their best. We present this as Experiment 1a.

After completing the fifth session which consisted of eight blocks of eighty trials we gave all participants a final 80-trial block and asked them to try to perform perfectly on this block. Participants who had performed Experiment 1a under both high and low pressure were told that if they did not make a single incorrect classification on the final block of trials then they would receive $100 cash. There were two motivations for this novel manipulation. First, we aimed to examine the effects of pressure across a shorter time-span (one block versus eight blocks). We also wanted to explore the effects of increasing the incentives and bonus criterion on performance.

Experiment 1a

In this experiment we examined the effects of pressure on rule-based and information-integration classification performance in highly-trained participants. Each participant received extensive initial training with either a rule-based or an information-integration task. By the end of the fourth training session, all participants were classifying with at least 85% accuracy.

At the beginning of the fifth session, half of the Rule-Based and half of the Information-Integration participants were put under high pressure by giving each the opportunity to earn a monetary bonus ($50) for exceptional performance (95% accuracy). Each participant in the High Pressure condition was informed that earning the bonus was a ‘team effort’. Specifically, they were told that they had been paired with a partner who had already reached the bonus criterion, but in order for either they or their partner to receive the bonus money they both had to reach the bonus criterion on a randomly determined block during the final session. The other half of the Rule-based and Information-Integration participants were put in the Low Pressure condition. These participants were simply told to continue classifying as they had in the previous four sessions. This pressure manipulation is identical to that used in Markman et al’s study of the effects of pressure on classification learning. Thus, we have good reason to believe that this manipulation successfully induces pressure in our participants.

Method

Participants

Participants were 69 members of the University of Texas community who were paid for their participation. The experiment consisted of five sessions on five consecutive weekdays with the constraint that no participant could start on a Tuesday, so that the pressure session would never occur immediately following a weekend. Participants completing all five sessions received $30. Participants were randomly assigned to one of four between-subjects conditions: Rule-Based-Low Pressure (N=17), Rule-Based-High Pressure (N=18), Information-Integration-Low Pressure (N=17) or Information-Integration-High Pressure (N=17). To ensure extensive initial learning, participants who did not reach a minimum of 85% accuracy across all blocks of session four were excluded. After completing the fourth session of the experiment participants who did not reach 85% accuracy level were paid and told that they did not need to complete the fifth session because they did not meet the minimum accuracy criterion. Nine participants failed to meet this performance criterion. Data from a total of 15 participants in each condition were used in the final analysis.

Materials and Stimuli

Participants performed the experiment on a personal computer using Matlab software. The rule-based and information-integration structures are shown in Figures 1a and 1b. Stimuli were Gabor patches varying in the frequency (spacing of bars) and orientation of the grating relative to the x-axis of the computer screen. The experiment used the randomization technique introduced by Ashby and Gott (1988). Forty Category A and 40 Category B stimuli from the rule-based categories were generated by sampling randomly from two bivariate normal distributions. Each random sample (x₁, x₂) was converted to a stimulus by deriving the frequency [f = .25 + (x₁/50)] and the orientation [o = x₂(π/500)]. The stimuli for the information-integration categories were generated by rotating the 80 rule-based stimuli clockwise by 45º around the center of the spatial frequency spatial orientation space and then shifting the spatial frequency and spatial orientation by an amount that resulted in the appropriate d′. The d′ was 4.5 for rule-based categories and 8.0 for information-integration categories. The d′ was increased for the information-integration category structure to equate performance accuracy between the two category structures. The category distribution parameters for both category structures are listed in Table 2. Each stimulus was created using the Matlab psychophysics toolbox. The size of each stimulus was 200 X 200 pixels. Each stimulus was centered in a 650 X 650 pixel stimulus presentation box in the upper left-hand corner of the computer screen. On the right side of the screen was a point meter that kept track of the number of points earned during the current block.

Table 2.

Category Distribution Parameters for the Rule-Based and Information-Integration Category Structures Used in the Experiment

Category Structure	μx	μx	σx2	σy2	covxy
Rule-Based
Category A	280	125	75	9000	0
Category B	320	125	75	9000	0
Information-Integration
Category A	275	150	4538	4538	4351
Category B	324	101	4538	4538	4351

Procedure

Initial Training

Every participant completed four sessions of initial learning. Each session consisted of eight 80-trial blocks. These sessions typically lasted just under an hour. On each trial, a stimulus was presented, the participant generated a categorization response, and corrective feedback was provided. There was no response deadline. Participants were free to take as long as they wished to categorize each stimulus. Feedback was given at the bottom left-hand corner of the screen, centered below the stimulus presentation box. If the participant was correct, the phrase ‘Correct’ was shown, and if the participant was incorrect, the phrase ‘No, that was an (A or B)’ was shown. Two points were added to a point meter on the screen when participants were correct, and zero points were added when incorrect. The counter was reset to zero at the start of each block.

Pressure Manipulation

At the beginning of the fifth session, half of the participants were introduced to the High Pressure manipulation, and half were simply told to continue categorizing (Low Pressure condition). Participants in the High Pressure condition were told that they would receive a monetary bonus of $50 if both they and a (fictional) partner exceeded a performance criterion (95% accuracy over a randomly determined block in the final session), and that neither would receive a bonus if one failed to reach the criterion. The participant was then informed that their partner had already exceeded the criterion, so the bonus for the participant and the partner rested on the participant’s performance. Throughout the course of the fifth session a line on the bonus meter indicated how many points were required to reach the bonus, and a box in the lower left-hand corner of the screen reminded participants in the High Pressure conditions that their partner had reached the bonus criterion. Participants in the High Pressure conditions also heard a ‘ching’ cash register sound when they made a correct classification, and a ‘buzz’ sound when they made an incorrect classification. This type of manipulation has been used in numerous previous studies and has been shown to induce pressure (e.g. Beilock and Carr, 2001; Gray; 2004; Beilock et al., 2004; Beilock and Carr, 2005; Beilock and Decaro, 2007).

Results

We analyzed the data in two ways. First, we examined performance accuracy for the final training and pressure sessions. Next we applied decision bound models (Maddox and Ashby, 1993; Ashby and Maddox, 1993) to determine the specific strategies used to classify the stimuli.

Behavioral Analyses

Training Performance

It is important to ensure that performance was equal across all conditions at the end of training. We conducted a 2 (category structure) X 2 (pressure level) ANOVA on the final session accuracy. Neither the interaction nor the main effects of pressure level or category structure were significant on the final training session accuracy. There were no significant differences between the Low Pressure and High Pressure conditions on final training session accuracy for participants performing a rule-based task, F<1 (M=0.890 for the Low Pressure condition, and M=.903 for the High Pressure condition), or for participants performing an information-integration task, F<1 (M=.906 for the Low Pressure condition, and M=.898 for the High Pressure condition). Thus any pressure-driven performance differences cannot be attributed to differences in initial task difficulty.

Performance Under Pressure

Accuracy Based Analyses

To determine how pressure affected performance, we examined the change in performance from the final training session to the pressure session as a function of category structure (rule-based vs. information-integration) and pressure level (High Pressure vs. Low Pressure). Figure 2 displays these data. Positive values denote improved performance in the pressure session, whereas negative values denote a performance decrement. A 2 (pressure level) X 2 (category structure) ANOVA revealed a main effect of pressure, F(1,58)=9.81, p<.01, η²=.15, with Low Pressure participants (M=.017) showing significantly greater improvement than High Pressure participants (M=−.005).

Figure 2.

Difference in accuracy between the pressure and final pre pressure sessions collapsed across all blocks.

Thus, unlike our previous category learning results (Markman et al., 2006), experienced participants exhibited similar performance decrements under pressure for both rule-based and information-integration category structures; though these decrements were more pronounced for participants given the rule-based task. This finding suggests that classification for experienced classifiers might be mediated by a system that is different from the dual learning systems that mediate rule-based and information-integration category learning. Perhaps an automated cortical system, like that proposed in SPEED, mediates responding for highly experienced participants performing both rule-based and information-integration tasks and is disrupted when the participant is under pressure due to an increase in skill-focused attention to the task. The uniform decrement in performance across category structures then appears to support the Explicit Monitoring hypothesis. This finding is consistent with Gray’s (2004) work examining the performance of expert participants while under pressure where the performance of highly experienced collegiate level baseball players declined under pressure due to an increase in skill-focused attention on the task.

Model-based analyses

The accuracy-based analyses support the proposal that pressure causes performance decrements for participants performing rule-based and information-integration classification tasks. Accuracy analyses are an informative measure of performance, but they provide no information regarding the specific strategies that participants use to classify stimuli. Knowledge of the effects of pressure on strategy use is of fundamental importance. To address this issue, we applied decision bound models (Maddox and Ashby, 1993; Maddox, 1999) to the data from each individual participant on a block by block basis.

Decision bound models assume that participants use a decision bound to separate stimuli into categories with stimuli on one side of the bound being classified into one category and stimuli on the other side of the bound being classified into the other category. The optimal decision bound in the Rule-Based condition is depicted by the vertical line in Figure 1a. Examples of sub-optimal strategies would be to shift the decision bound along the frequency dimension, or to place a decision bound along the orientation dimension. The optimal decision bound in the Information-Integration condition is depicted by the diagonal line in Figure 1b. Sub-optimal strategies might include altering the slope or y-intercept of the optimal decision bound, or setting a decision bound on either the frequency or orientation dimension.

With highly experienced participants, we expect a high proportion of optimal decision bound use. Fits of the models will be useful in determining if pressure causes a shift in classification strategies to a sub-optimal or even to a qualitatively different strategy (e.g. from an optimal frequency-dimension decision bound in the Rule-Based condition to an orientation-dimension decision bound).

We fit seven Decision Bound models separately to the data from each participant on a block-by-block basis. All of the analyses were performed at the individual-participant level because of concerns with modeling aggregate data (e.g., Ashby, Maddox, & Lee, 1994; Estes, 1956; Maddox, 1999; Maddox & Estes, 2004; Smith & Minda, 1998).

Rule-based models

The optimal unidimensional frequency model assumes that the participant uses the optimal criterion along the spatial frequency dimension and applies the rule: “Respond ‘A’ if the spatial frequency is low and ‘B’ if it is high.” This model has one free parameter that represents the variance of internal (perceptual and criterial) noise, and was fit only to data from participants performing rule-based tasks. The generalized unidimensional frequency rule model assumes that the participant uses a criterion along the spatial frequency dimension, but allows the criterion value to be estimated from the data (2 free parameters total). The generalized spatial orientation model assumes that the participant uses a criterion along the spatial orientation dimension, and allows the criterion value to be estimated from the data (2 free parameters total). These two models were fit to data from all participants.

The conjunction models assume that the participant uses a conjunctive rule in which he or she makes separate decisions about the levels of the two dimensions and then selects a response based on the outcome of these two decisions. Two conjunctive rules were examined:

1. “Respond ‘A’ if spatial frequency is low and orientation is large, otherwise respond ‘B,’” and

2. “Respond ‘B’ if spatial frequency is high and orientation is small, otherwise respond ‘A’.”

Both rules partition the perceptual space into four regions. The first assigns one to Category A and three to Category B, and the second assigns three to Category A and one to Category B. The conjunction models have three parameters (a criterion on each dimension, and an internal noise parameter). The conjunction models were fit to data from all participants.

Information-integration models

The general linear classifier model (GLC) assumes that the decision bound between each pair of categories is linear. This produces an information-integration decision strategy because it requires linear integration of perceived frequency and orientation. The GLC has three parameters: the slope and intercept of the linear bound, and an internal noise parameter. The optimal GLC model assumes that the participant uses the linear bound that maximizes accuracy. This model has only one free parameter representing the internal noise. The GLC was fit to all data, and the optimal GLC was fit only to data from Information-Integration participants.

Random Responder Model

Each participant’s data was also fit by a one-parameter random responder model that assumed a fixed probability (estimated by the model) of responding “A” for all the stimuli. This model never provided the best account of any participant’s data and is not discussed further.

Model fitting procedure

Each model was fit separately to the data from each of the eight blocks of trials in sessions four and five for each participant. The model parameters were estimated using maximum log-likelihood (Wickens, 1982), and the goodness-of-fit statistic used was AIC = −2lnL + 2k, where k is the number of free-parameters (Akaike, 1974). The AIC statistic penalizes models with extra free parameters. The best fitting model is the model with the smallest AIC value. For each block for each participant in each session we determined which model provided the best fit to the data.

Results of model fitting

Final Training Session Analysis

We found that the optimal or generalized unidimensional spatial frequency model fit best in every block of trials for Rule-based Low Pressure and High Pressure condition participants. In addition, we found that the optimal or general linear classifier fit best in every block of trials for Information-integration Low Pressure and High Pressure condition participants. Thus, participants used a strategy of the same form as the optimal, but in some cases the strategy was sub-optimal. We compared the proportion of participants best fit by the optimal model in each condition. As expected, there were no significant differences between the Low Pressure and High Pressure conditions for either the rule-based or information-integration category structures, F<1.

Pressure Session Analysis

As with the final learning session, we found that Rule-based and Information-integration participants in the Low Pressure and High Pressure conditions used a strategy of the same form as the optimal, but in some cases the strategy was sub-optimal.

Figure 3a displays the proportion of Information-Integration participants in each condition whose data was best fit by the optimal model in each block. A larger proportion of Information-Integration Low Pressure participants’ data were fit best by the optimal model in seven of eight blocks (p<.05 by sign test). Binomial tests reveal that a significantly higher proportion of data sets from Low Pressure participants were fit best by the optimal model in block 8, p<.05, and a marginally higher proportion were best fit by the optimal model in block 2, p<.10.

Figure 3.

(a) Proportion of Information-Integration participants best fit by the optimal GLC model for each block of the pressure session. (b) Proportion of Rule-Based participants best fit by the optimal unidimensional frequency model for each block of the pressure session.

Figure 3b displays the proportion of Rule-Based participants in each condition whose data was fit best by the optimal model in each block. A larger proportion of Rule-based Low Pressure participants’ data were fit best by the optimal model in 3 of 8 blocks (ns based on a sign test). However, significantly more Low Pressure condition participants were fit best by the optimal model in blocks 4 and 7 (p<.05 by binomial test).

Taken together these findings suggest that pressure does not lead to a qualitative shift in strategy (e.g., from an information-integration strategy to a rule-based strategy), but rather can lead to an increase in the use of a sub-optimal strategy (albeit a strategy of the same form as the optimal).

Noise from Optimal Models

Although pressure did not lead to large qualitative shifts in strategy use, it is likely that pressure leads to more subtle changes. As a first pass, we examined the estimated noise parameter values from fits of the optimal model to each participant’s data. Larger values of the noise parameter reflect greater noise in the perception of each stimulus and/or greater variance in the memory for, and application of, the decision bound.

We subtracted the optimal model noise estimate averaged across all blocks for the pressure session from that averaged across all blocks for the final training session. Positive values indicate less perceptual noise and/or less variance in the memory for, and application of, the decision bound in the pressure session, whereas negative values indicate more perceptual noise and/or greater variance in the memory for, and application of, the decision bound in the pressure session.

Figure 4 displays the noise difference measure for each of the four conditions. A 2 (pressure level) X 2 (category structure) ANOVA revealed a main effect of pressure on this measure with Low Pressure participants yielding a larger (and positive) noise difference (i.e., less noise in the pressure session relative to the final training session) than High Pressure participants who yielded a negative noise difference, F(1,58)=4.21, p<.05, η²=.07. The noise difference were larger for participants performing information-integration tasks than for those performing rule-based tasks, but this was likely due to the greater distance from the decision bounds for the information-integration stimuli than for the rule-based stimuli.

Figure 4.

Difference in criterial noise from the optimal decision bound model between the final training session and the pressure session.

The combined modeling results indicate that pressure caused a decrement in optimal strategy use for both Rule-Based and Information-Integration High Pressure participants. High Pressure participants were more likely to be fit by a sub-optimal model that had the same form as the optimal model. High pressure did not cause a complete shift in classification strategy or a complete collapse in performance. This pattern suggests that the decrements in accuracy for participants under high pressure resulted from a disruption of the execution of the optimal classification strategy that was reflected by increased perceptual and criterial noise.

Experiment 1b

In many domains of expert performance, there is little room for error. Performance by athletes, doctors, soldiers, pilots, students, and other high-stake performers can be disrupted by only small decrements in accuracy or execution. Very often choking involves only the smallest mistake or blunder. A game can be won or lost by a missed free throw, and admission into a prestigious academic institution can be gained or lost by errors on only a few test problems. While our highly experienced classifiers did not show major qualitative drops in performance they did significantly underperform relative to equally experienced peers who were under low pressure. Thus, the amount of pressure placed on students, athletes, and others striving to perform optimally can influence how well they do.

We used a pressure manipulation that is similar to many manipulations that have been shown to induce pressure in previous laboratory studies (e.g. Beilock and Carr, 2001; Gray; 2004; Beilock et al., 2004; Beilock and Carr, 2005; Beilock and Decaro, 2007). This type of pressure manipulation involves a combination of social and monetary incentives. Participants in our experiment were asked to achieve at least 95% accuracy on each block. This amounted to making 76 out of 80 correct classifications. We used this manipulation because it followed directly from previous work studying choking under pressure (e.g. Markman et al., 2006; Beilock and Carr, 2001; Gray; 2004; Beilock et al., 2004; Beilock and Carr, 2005; Beilock and Decaro, 2007). However, we also wanted to examine performance in an extremely high stakes situation. To this end, we added a final ‘super pressure’ block of trials onto the end of the final experimental session in which all participants were given a high monetary incentive ($100 cash) to perform perfectly. This manipulation was added to examine the effects of different levels of pressure on performance.

Method

Participants performed Experiment 1b as an addition to the fifth session of Experiment 1a. After completing eight blocks of 80 trials during the fifth and final session participants were asked to perform one final 80-trial block under ‘super pressure.’ Participants performed the same categorization task they had been performing throughout the entire 5-day experiment. We informed all participants, including those in the Low Pressure condition, that they had the opportunity to earn $100 cash if they did not make a single incorrect classification on the final block of trials. After completing the super pressure block participants were debriefed regarding the first pressure manipulation and given any bonus money they had earned for their performance.

Results

Because performance for many participants quickly dropped off after they made their first incorrect classification during this block (i.e. they gave up) we compared the trial on which they made their first incorrect classification. Figure 5 shows the average first trial missed for each condition. A 2 (Category Structure) X 2 (Pressure Level) ANOVA showed a main effect of category structure, F(1,50)= 4.99, p<.05, η²=.09². Participants performing an information-integration (M=21.30) task went significantly longer before making an incorrect response than participants performing a rule-based task (M=11.44) This finding is consistent with the idea that pressure causes an increase in skill-focused attention that disrupts people’s ability to respond automatically. This increase in monitoring hurt rule-based participants more than information-integration participants because they had explicit knowledge of the category structure during training. The additional knowledge of the category structure eventually disrupted their performance under pressure (e.g. Gray, 2004; Masters, 1992; Liao and Masters, 2002).

Figure 5.

Average first missed on the super pressure block.

General Discussion

This study examined the influence of pressure on experienced participants’ rule-based and information-integration classification performance. We employed two types of pressure manipulations that we call ‘pressure’ and ‘super pressure’. For the pressure manipulation we created a situation where participants in the High Pressure conditions were told that they must reach a very high accuracy criterion (95%) on a randomly chosen block to earn $50 for themselves and their partner who already did his or her part in earning the bonus. Thus the participant was faced with the uncertainty of knowing which block would be chosen to determine whether they receive the bonus, and with the knowledge that failure on his or her own part to reach the bonus criterion would cause a worthy partner to not receive the bonus. A manipulation very similar to this has been used in numerous previous studies and has been shown to induce pressure (e.g. Beilock and Carr, 2001; Gray; 2004; Beilock et al., 2004; Beilock and Carr, 2005; Markman et al., 2006; Beilock and Decaro, 2007).

This type of pressure generally harmed experienced participants performing both a rule-based classification task and an information-integration task. These results are in direct contrast to classification performance under pressure in novices (e.g. Markman et al., 2006). Previous research demonstrated that learning the rule-based task is working-memory intensive, while learning the information-integration task actually benefits from low working-memory capacity (Decaro et al., 2008; however see Tharp & Pickering, in press). Thus, the observation that pressure hurts both tasks suggests that performance of experienced participants involves different processes than performance of participants learning the task. This finding is consistent with the view that expertise involves training of cortico-cortical connections that can swiftly and efficiently handle task performance (e.g. Ashby et al., 2007).

Model-based analyses indicate that the pressure manipulation caused an increase in the use of sub-optimal strategies, albeit strategies of the same form as the optimal strategy. Significantly more blocks of data from participants in the Information-Integration High Pressure condition were fit best by a sub-optimal model relative to model fits from data from the final training session. Noise parameter estimates from fits of the optimal models to the data indicate that pressure caused an increase in perceptual and criterial noise for participants performing both rule-based and information-integration tasks under pressure. These results suggest that the pressure manipulation disrupted the processing in the categorization system that was most optimal for participants performing both types of tasks under pressure.

After all participants had completed five sessions of 640 trials each we employed a super pressure manipulation that was different in many ways from the pressure manipulation. All participants were given a final 80-trial block in which they were told that perfect performance would yield a bonus of $100 cash. Unlike the pressure manipulation, this manipulation had no social component. Participants were trying to earn the large monetary bonus only for themselves. The super pressure manipulation also differed from the pressure manipulation in that there was less uncertainty as to which block would be used as the criterion for the reception of the bonus. For the pressure manipulation participants were told that a randomly selected block would be used to determine if they received the bonus. If they made over 95% correct classifications on this randomly selected block then they would receive $50 for themselves and a partner. Thus, there was some uncertainty as to which block would be used for determination of the bonus. There was no such uncertainty with the super pressure manipulation. Participants knew the specific block that would determine whether they met the bonus criterion on. Finally, the two manipulations differed in that the super pressure manipulation offered absolutely no chance for failure. While the 95% accuracy criterion given in the pressure manipulation offered some small room for mistakes, any inaccuracy in the super pressure manipulation would lead to failure.

We found a different pattern of results between the two pressure manipulations. For the pressure manipulation, participants choked under pressure relative to participants given no pressure regardless of whether they learned the explicit rule-based task or the implicit information-integration task. In contrast, for the super pressure manipulation, participants performing the rule-based task performed much worse than participants performing the information-integration task.

The results of the super pressure manipulation are consistent with the possibility that explicit knowledge obtained during rule-learning disrupt the procedural skill developed over the extensive training period. On this view, the super pressure manipulation caused participants to explicitly monitor their performance, but only Rule-Based participants had explicit knowledge of the category structure that could influence their performance. On this view, Information-Integration participants were more immune to the deleterious effects of monitoring, because they did not have explicit knowledge about the category structure from their early learning. It should be acknowledged, however, that we can only assume that the super pressure manipulation further raised anxiety levels. Future efforts should be made to include manipulation checks (e.g. Houtman and Bakker, 1989).

The acute nature of the super pressure manipulation may have brought out more differences between participants performing rule-based and information-integration tasks than the regular pressure manipulation. The uncertainty about which block’s accuracy score would determine the bonus for the pressure manipulation, as well as the longer time span of the pressure manipulation compared to the super manipulation (i.e. eight blocks for the pressure manipulation compared to one block for the super pressure manipulation) may have reduced some advantages for participants performing information-integration tasks. The super pressure manipulation required only one flawless string of trials to achieve the bonus, whereas the pressure manipulation required consistently high performance throughout eight 80-trial blocks. It may have been more difficult to maintain high performance over such a long time-span.

Future work should investigate different types of pressure manipulations. Many real world events contain pressure situations that are more similar to either our pressure or super pressure manipulation. Exams and sporting events, for example, may have more in common with our pressure manipulation. These events occur over long periods of time. Participants do not necessarily have to perform perfectly, but they must maintain very high performance throughout. Team sporting events involve a social component similar to the one in our pressure manipulation. Teammates depend on each other to reach their common goals. Other real-world events such as surgeries, military missions, and chess matches more closely resemble our super pressure manipulation. These events offer basically no room for failure. The differences that we found in performance between the pressure and super pressure manipulations we used may have implications for differences in the types of pressure people face in real-world situations.

Implication for Theories of Choking

These results suggest that the effects of pressure on experienced classification are more in line with the predictions from the Monitoring theory than from the Distraction Theory. The results are also consistent with a recently proposed theory of experienced information-integration classification (i.e. Ashby et al., 2007), and may extend the theory to account for experienced rule-based classification. The middle rows of Table 1 predict a uniform decrement in performance while under pressure for both rule-based and information-integration category structures. This pattern was obtained for the pressure manipulation. Participants under pressure were less accurate during the pressure session relative to the final pre-pressure session than participants under no pressure. This supports the prediction of Monitoring theory that pressure causes an increase in skill-focused attention leading to lower accuracy.

One puzzling aspect of these data, however, is the performance in the super pressure block. Participants given the rule-based task choked under super pressure, while participants performing the information-integration task showed much smaller performance decrements under super pressure. This pattern was observed during rule-based and information-integration classification learning (Markman et al., 2006) and was interpreted as support for the Distraction theory. It is possible to view these results in a way that is consistent with the Monitoring theory. On this interpretation, pressure decreases performance for Rule-Based participants because they possess explicit knowledge about the task that they acquire during learning. This explicit knowledge disrupts the automated system when under pressure. In further support of this view the first pressure manipulation appears to have caused greater decrements in performance for participants performing rule-based tasks than for those performing information-integration tasks.

Masters and colleagues’ Reinvestment Theory, a refinement of Explicit Monitoring theory (see, Masters and Maxwell, 2008), states that individuals engage in conscious self-regulation of their performance when there is a discrepancy between their current performance level and their desired performance level. This theory could explain why participants performing information-integration classification tasks did not show as much choking on the super pressure block as they did during the first pressure manipulation. The shorter time span to perform under pressure may have given participants less time to ruminate about the discrepancy between their current level of performance and their desired level of performance. Thus, over longer periods of time participants may ‘break’ under pressure and engage in explicit monitoring which leads to choking under pressure.

The Distraction theory is consistent with the pattern of data obtained in the super pressure block, assuming that the same neural systems that mediate classification during learning also mediate classification after the categories are well-learned. However, this view cannot account for the uniform decrements in performance found across both category structures after the first pressure manipulation. The combined pattern of data offers the most support for the Monitoring theory of choking under pressure in highly experienced participants.

One view that is very similar to the Distraction theory is Processing Efficiency theory (Eynseck and Calvo, 1992; Wilson, Smith, Chattington, Ford, and Marple-Horvat, 2006). Like Distraction theory, Processing Efficiency theory states that pressure causes worry that reduces available working memory resources. However, Processing Efficiency theory further predicts that pressure may also stimulate increases in on-task effort, which may partially or totally compensate for reduced performance effectiveness. This increase in on-task effort could be viewed as similar to the proposed mechanism of choking in Explicit Monitoring theory. Thus Processing Efficiency theory could combine both hypothesized mechanisms of choking into one theory. Such a ‘double whammy’ effect of pressure has been proposed elsewhere (e.g. Beilock and Gray, 2007).

Finally, individual differences in performing under pressure should be addressed in future work. Several studies have already investigated dispositional traits such a reinvestment (e.g. Masters, Polman, & Hammond, 1993; Shea, Wulf, Whitacre, and Park, 2001). Participants who score high on the Reinvestment Scale (Masters et al., 1993), tend to choke more when performing motor tasks under pressure. Other individual differences variables such as action orientation (Raab and Johnson, 2004), skill-focused attention (Baumeister, 1984), and internal and external attentional foci (Shea et al., 2001) may also affect performance under pressure.

Much work remains to be done to examine the effects of pressure on performance, particularly for experts. Many of the anecdotal phenomena of choking under pressure involve highly experienced individuals placed in high-stakes situations. This study demonstrated that pressure affects novice and expert participants differently. This study also illuminated the need for further development of models of expertise. The conclusions to be drawn regarding the effects of pressure on expert participants depend upon models of expertise in category-learning. Neuroimaging studies are another avenue for further exploration to help determine the processes compromised by performance pressure.