Figure 5.

Computational models capturing effects of cue-based expectations on pain and confirmation bias on expectation updating. A. Reinforcement learning model (Model 1). Perceptual inference within a trial combines expectations with noxious input to determine perceived pain. The γ parameter controls the relative impact of these two sources. Learning between trials involves updating the expectation for the current cue toward the current perceived pain. Experience-resistant expectations are modeled by assuming different learning rates, the α_c and a parameters, when the direction of prediction error is, respectively, consistent and inconsistent with the cue’s initial low or high pain association. B. Bayesian model (Model 2). Pain perception and expectation are products of Bayesian inference with respect to a generative model of the task environment that extends the classic Kalman filter. Note that arrows in this diagram indicate statistical dependencies in the subject’s generative model, not dynamics of the subject’s state of knowledge as in Figure 5a. Under the generative model, the mean threat level signaled by each cue (μ_c, index c suppressed in figure) drifts randomly from trial to trial, with step size determined by the σ_η parameter. The current threat level (or objectively correct pain level, π_t) on any trial t deviates randomly from μ_t, with standard deviation equal to the σ_ψ parameter. The noxious input (N_t) is a noisy indicator of π_t, with standard deviation equal to the σ_ε parameter. Inference within a trial (not shown) combines the current belief about with the observed value of N_t to estimate the current value of π_t; this estimate is the subject’s experienced level of pain (P_t). Inference across trials combines the beliefs about μ_t and π_t to estimate μ_t+1; this is the subject’s reported expectation the next time this cue is presented (E_t+1). Experience-resistant expectations are modeled by letting cue-pain associations drift in the directions of their initial values between trials, to a degree governed by the β parameter. Model 2 is formally nearly equivalent to Model 1, except that it assumes γ and α adapt from trial to trial to reflect the subject’s current level of uncertainty (Eqs. 3.1, 3.3, 3.6).