From Classical Methods to Generative Models: Tackling the Unreliability of Neuroscientific Measures in Mental Health Research

Abstract

Advances in computational statistics and corresponding shifts in funding initiatives over the past few decades have led to a proliferation of neuroscientific measures being developed in the context of mental health research. Although such measures have undoubtedly deepened our understanding of neural mechanisms underlying cognitive, affective, and behavioral processes associated with various mental health conditions, the clinical utility of such measures remains underwhelming. Recent commentaries point toward the poor reliability of neuroscientific measures to partially explain this lack of clinical translation. Here, we: (1) provide a concise theoretical overview of how unreliability impedes clinical translation of neuroscientific measures; (2) discuss how various modeling principles—including those from hierarchical and structural equation modeling frameworks—can help to improve reliability; and then (3) demonstrate how to combine principles of hierarchical and structural modeling within the generative modeling framework to achieve more reliable, generalizable measures of brain-behavior relationships for use in mental health research.

The Need for Improved Neuroscientific Measures in Mental Health Research

Neuroscientific measures have long held promise for revolutionizing clinical practice (1). This promise remains strong today, evidenced by funding initiatives such as the U.S. National Institute of Mental Health’s Strategic Plan to “define the brain mechanisms underlying complex behaviors” (2) and the U.K. Research and Innovation’s funding priority focuses on “neurosciences and mental health” and “systems approaches to the biosciences” (3). Despite decades of promises backed by millions of dollars in funding, neuroscientific measures still fail to meet basic validity requirements that would make them useful for diagnosis or treatment personalization in a real-world clinical context (4–7). Such failures are likely due to a variety of factors ranging from specifying our theories of how clinical phenomena arise to the statistical methods we use test our theories (8,9). Here, we give attention to the latter, with a particular focus on issues of reliability and measurement error in the context of behavioral and neural data. We then discuss frameworks that have been developed to account for imperfect reliability when making inferences, including classical test theory, hierarchical modeling, and structural equation modeling. Finally, we discuss how generative models can be developed to incorporate principles from these modeling frameworks, and how such models can be used to better understand brain-behavior relationships while accounting for low reliability or measurement error. Throughout, readers can refer to Box 1 for an overview of the terminology we use. Additionally, we recommend that interested readers follow along with our analyses using the companion Rmarkdown notebook located at: https://github.com/Nathaniel-Haines/biopsych-reliability-2022.

Box 1.

Reliability — Any measurement is comprised of the signal or “true score” and noise or measurement error; reliability is the fraction of measurement variance that is not error variance. Reliability is essentially a signal to noise ratio. Types of reliability measures include:

• Internal Reliability — Consistency within a measure (e.g., consistency across items within a self-report measure). Can also be thought of as how “precisely” a parameter is estimated.

• Split-Half Reliability — Consistency within a measure, quantifies extent to which parts of a given measure contribute to the overall measurement (e.g., consistency of performance on first half of a task with performance on second half of a task or consistency of odd compared to even numbered trials within a task).

• Test-Retest Reliability — Consistency of a measure across time or repeated assessments (e.g., stability of task performance across successive administrations).

• Inter-Rater Reliability — Consistency of measurement across individuals (e.g., consistency of scores from multiple clinicians determining a diagnosis).

Measurement Error — All remaining variance in a measure that is beyond the variance attributable to the variable of interest’s “true score”; i.e., measurement noise.

Structural Equation Model — A broad family of statistical models that leverage the covariance structure among variables to understand associations between observed variables and estimate latent (unobserved) variables that reflect the error-free shared variance among a set of predictors.

Hierarchical Model — A statistical model in which measurement units are clustered within groups hierarchically (e.g., people within a population, trials within a person, time points within a self-report construct) and variance is attributed to both the lower-level units and the higher-level group. Similar terms include: multi-level models, random-effect models, and mixed-effect models.

• Hierarchical Shrinkage — Individual “random effect” estimates in hierarchical models are “pooled” or “regressed” toward the group-level mean. Shrinkage occurs through the same mechanism as regression-based true score estimation per classical test theory.

Generative Model — A statistical model of the joint probability distribution $p(y,\theta )$ on observed data $\left(y\right)$ and unobserved parameters $\left(\theta \right)$. Conceptually, generative models can be thought of as probabilistic models that can simulate forward all aspects of both the observed and unobserved data/variables of interest. For example, a trial-level reinforcement learning model with specified parameters can generate/simulate data that resembles the structure of a participant’s actual choices in a decision-making task, with latent parameters such as learning rate, representing unobserved variables that influence behavior. Note: this definition is distinct from the neuroscience construct of “generative model” used in the context of “predictive coding”, “active inference”, and “free energy”.

Bayesian Model — A statistical model within which all uncertainty is quantified by specifying probability distributions on data and parameters of interest. Inference with Bayesian models is based on Bayes’ rule, which is written as $p(\theta \mid y)=\frac{p(y\mid \theta )p\left(\theta \right)}{p\left(y\right)}$. Because $p(y,\theta )=p(y\mid \theta )p\left(\theta \right)$, Bayesian models are all, in principle, generative models. As such, both the theory and practice of developing Bayesian models is highly aligned with the philosophy of generative modeling.

The Reliability Paradox and Brain-Behavior Relationships

The goal of clinical assessment is to permit inferences of individuals’ behavior, performance, and, broadly, outcomes. Yet, there are critical considerations related to measurement reliability that do not receive adequate attention. For example, a number of studies have demonstrated variability in neural results, and thus effects on reliability, depending on features of the analytic pipeline (10–12); assumptions about the hemodynamic response function (13); subject motion, especially when motion is linked to stimuli (14); whether activations or connectivity paths were located in the cortex versus subcortex (15–18); and a variety of other factors (see (19) for review).

The Reliability Paradox (20) has emerged as a particularly important issue affecting reliability in clinical neuroscience—many behavioral measures that produce robust group effects paradoxically show low reliability $\left(\rho \right)$ coefficients¹. Typically, $\rho$ is expressed as a ratio of between- relative to within-unit variability in a measure, where $\rho$ ranges between 0 and 1, and higher or lower values indicate that a measure does a better or worse job of differentiating units within a group. Units within groups could be people within a population, trials within a person, or any number of other similarly structured hierarchical data structures. In the context of the Reliability Paradox, units are individual people, and the group is some population of interest that people are sampled from (but see Zorowitz & Niv (this issue) for a more comprehensive overview of the various conceptualizations of reliability). Therefore, when behavioral task measures have low reliability, it prevents using the measure for individual difference inference. For clinical applications, such inferences include determining whether someone scores above a clinical threshold on some behavioral, neural, or self-report measure.

Since Hedge et al. (20) coined the term, the Reliability Paradox has been observed throughout the psychological and brain sciences, with $\rho =.4–.5$ across a wide range of behavioral tasks measuring self-control, implicit attitudes, learning and decision-making, and more (21–23). Beyond behavioral measures, the Reliability Paradox is also apparent in neuroscience—a recent meta-analysis of 90 fMRI experiments showed an average task-based BOLD response $\rho \approx .4$ across different brain regions and tasks (16). Resting state, task-based, and regional activation/connectivity measures show comparably low reliability (24–26), suggesting that—like behavioral tasks—many neural measures cannot be used for clinical decision-making.

The low reliability of common behavioral and neural measures has led to considerable, justified concerns regarding the use of behavioral tasks and fMRI experiments to develop and test theories of individual differences (16,27–29). The primary issue with low reliability is that, if both behavioral and neural measures are unreliable, statistical power suffers, and the sample sizes required to overcome measurement error to detect significant associations with critical outcomes must increase to compensate. For example, the sample size needed to detect a true medium effect $(r=.3$, with 80% power at $\alpha =.05)$ between two measures with perfect reliabilities $\left({\rho }_{\text{behavior}}={\rho }_{\text{neural}}=1\right)$ is 84. Under a more realistic scenario where ${\rho }_{\text{behavior}}={\rho }_{\text{neural}}=.4–.5$, the necessary sample size increases dramatically to 346-524, which would push scanner costs (assuming $750/hour) to ≈ $259,500 in an optimistic scenario. Given that most clinical neuroscience studies have sample sizes far below these numbers, it is likely the case that many reported findings are spurious or even in the wrong direction compared to true underlying effects (30,31).

A Theoretical Dive into Measurement Error and Reliability

Despite the bleak reality painted above, recent re-analyses and commentaries suggests that low reliability can be overcome. Some commentaries point to different ways of developing measures (7,32), whereas others have used alternative statistical modeling frameworks to address issues arising from the use of traditional measures that otherwise result in unwarranted attenuation of reliability (9,33–35). Throughout this paper, we focus on this latter statistical modeling perspective, providing a concise overview of the key principles along with a brief tutorial that demonstrates these principles in action.

Classical Test Theory

The Classical Test Theory (CTT) perspective views low reliability as arising primarily from measurement error. Measurement error perturbs a “true score”, resulting in the “observed score” that is quantified. In its simplest form, the CTT model follows the form:

$$y_i={\theta }_i+{ϵ}_{i,t}$$ (1)

For person $i,y_i$ is their observed score, ${\theta }_i$ is their true score, and ${ϵ}_{i,t}$ is their measurement error for observation $t$ (36). Typically, we assume that true scores and measurement error are normally distributed such that ${\theta }_i\sim 𝒩(0,{\sigma }_{\theta }^2)$ and ${ϵ}_{i,t}\sim 𝒩(0,{\sigma }_{ϵ}^2)$. Here, ${\sigma }_{\theta }^2$ indicates the variance in true scores across people, and ${\sigma }_{ϵ}^2$ indicates the variance of observed scores around the true scores within people (i.e., the measurement error), where each person is assumed to share the same within-person variance.

The true score $\left({\theta }_i\right)$ is the quantity that we aim to make inferential statements about. Herein lies the role of reliability in individual difference inference in as early as the 1920 ‘s, Kelley showed that true scores could be estimated $({\widehat{\theta }}_l)$ by regressing observed scores $\left(y_i\right)$ toward the group-level mean score $\left(\bar{y}\right)$ in proportion to the reliability $\left(\rho \right)$ of the observed scores (e.g., reliability of person-level measures within a group) (37,38):

$${\widehat{\theta }}_l=\bar{y}+\rho \cdot \left(y_i-\bar{y}\right)$$ (2)

Here, if $\rho =0$, person-level observed scores contain no information to differentiate people. As a result, each person’s estimated true score is simply the group-level average $({\widehat{\theta }}_l=\bar{y})$. Otherwise, if $\rho =1$, person-level observed scores contain no measurement error, so the observed score is equivalent to the true score $({\widehat{\theta }}_l=y_i)$. For values of $\rho$ between 0 and 1, observed scores are regressed toward the group-level average score in proportion to reliability.

Notably, ${\widehat{\theta }}_l$ in equation 2 is now referred to as a James-Stein estimator throughout mathematical statistics (39–42). The discovery of such estimators produced a revolution in statistics due to their counter-intuitive properties—they are mathematically proven to give an equal or better estimate in terms of mean squared error relative to the unbiased sample mean for each person $y_i$². This result is often described as Stein’s Paradox. As outlined by (42), R. A. Fisher had already proved that the sample mean $y_i$ contains all possible information that can be used to inform estimates of the true mean ${\widehat{\theta }}_l$. However, Fisher did not consider that information across multiple sample means could be used to mutually constrain estimation of each individual sample mean. Intuitively, in the absence of further information, the group-level mean is your best estimate of a person’s true score before you can measure that individual. Regression-based true score estimators (or James-Stein estimators) formalize this intuition and extend it to the case where information on the individual is present, yet unreliable (or contaminated with measurement error). As we discuss later, this regression-based estimator ties together traditional CTT and more modern hierarchical models.

To use equation 2, we must first estimate reliability. In CTT, assuming that the observed scores $y_i$ are summaries of multiple conditionally independent measurements (e.g., the mean of multiple response times), reliability $\left(\rho \right)$ is formulated as the ratio of true score variance (or between-person variance) relative to total variance:

$$\rho =\frac{{\sigma }_{\theta }^2}{{\sigma }_{\theta }^2+\raisebox{1ex}{${\sigma }_{ϵ}^2$}\!\left/ \!\raisebox{-1ex}{$n$}\right.}$$ (3)

Here, $n$ is the number of independent measurements (e.g., trials, although in CTT observed scores are treated as single observations, or $n=1$), and $\raisebox{1ex}{${\sigma }_{ϵ}^2$}\!\left/ \!\raisebox{-1ex}{$n$}\right.$ is then the standard error of measurement (assuming shared $n$ and ${\sigma }_{ϵ}^2$ across all $y_i$). In a response time task, ${\sigma }_{\theta }^2$ captures the variance in mean response times across people, and ${\sigma }_{ϵ}^2$ captures the variance of response times across trials within people. Note that this particular definition of $\rho$ is termed average score reliability, also referred to as $\mathrm{I}\mathrm{C}\mathrm{C}(k)$ or $\mathrm{I}\mathrm{C}\mathrm{C}(1,k)$ as described by (43) and (44), respectively. A consequence of this definition is that reliability necessarily decreases as within-person variance $({\sigma }_{ϵ}^2)$ increases, $n$ decreases, or between-person variance $({\sigma }_{\theta }^2)$ decreases.

In addition to being useful for understanding the relationship between observed and true scores, the concept of reliability in CTT can be used to better estimate “true correlations” given observed correlations. For example, in an fMRI study, we may collect behavioral response times in addition to ROI activations across participants and trials. We may then model the relationship between the vector of average response times across people $(\mathit{y})$ and the vector of average ROI activation across people $(\mathit{x})$ to determine if the ROI relates to the behavioral process underlying response times. In doing so, the true correlation of interest $(r_{{\theta }_y,{\theta }_x})$ is attenuated by the (un)reliability of both measures (${\rho }_y$ and ${\rho }_x$), resulting in the following observed correlation:

$$r_{y,x}=r_{{\theta }_y,{\theta }_x}\sqrt{{\rho }_y{\rho }_x}$$ (4)

Because there is often high variability in both behavioral response times and neural ROI measures across trials, ${\sigma }_{{ϵ}_y}^2$ and ${\sigma }_{{ϵ}_x}^2$ are often high relative to between-person variability in either measure (${\sigma }_{{\theta }_y}^2$ and ${\sigma }_{{\theta }_x}^2$). This results in low reliability for both measures $({\rho }_y{\rho }_x\ll 1)$, in the end producing a low observed correlation $(r_{y,x})$ even if the true correlation $(r_{{\theta }_y,{\theta }_x})$ is quite high. To correct an observed correlation for measurement error only requires a rearrangement of equation 4:

$$r_{{\theta }_y,{\theta }_x}=\frac{r_{y,x}}{\sqrt{{\rho }_y{\rho }_x}}$$ (5)

Above, if reliability is perfect for both measures $({\rho }_y{\rho }_x=1)$, then the observed and true correlations are equivalent. Otherwise, to the extent that ${\rho }_y{\rho }_x\to 0$, the observed correlation is increasingly adjusted toward the true correlation. However, this correction (or dissattenuation for unreliability) must be treated with caution—if reliability is not estimated appropriately, estimates of the true correlation are biased and potentially invalidated in response (44,45).

Hierarchical and Structural Latent Variable Models

It is well known that correlations estimated between latent variables are typically greater in magnitude than the same correlations estimated using summed scores of the respective measures—why is this the case? Typically, latent variables are discussed from the perspective of structural equation models (SEMs) wherein latent variables reflect common variance in manifest indicators (i.e. observed data). As a result, latent variables are considered “error free”, or having perfect reliability, averaging across indicator variables to arrive at a more precise estimate of the “true” (i.e. latent) variable. Formally, we can understand the properties of latent variables by focusing on hierarchical models as a specific case (see Box 1), noting that hierarchical models can be viewed as a form of latent variable model within the SEM framework (46–48). Hierarchical models are models in which measurement units are clustered within groups hierarchically (e.g., people within a population), with variance attributed to both the lower-level units and higher-level group. Hierarchical models exert a pooling or shrinkage (a.k.a. regularization or regression to the mean) effect on person-level parameters (which are latent variables). For example, in a hierarchical model where observed response times are assumed to be normally distributed within persons $i$ with mean ${\theta }_i$ and standard deviation ${\sigma }_{ϵ}$, and where variation between persons is also assumed to follow a normal distribution with mean ${\mu }_{\theta }$ and standard deviation ${\sigma }_{\theta }$, the shrinkage factor for each person $i$ is given by (49):

$${\rho }_i=\frac{{\sigma }_{\theta }^2}{{\sigma }_{\theta }^2+\raisebox{1ex}{${\sigma }_{ϵ}^2$}\!\left/ \!\raisebox{-1ex}{$n_i$}\right.}$$ (6)

Here, $\raisebox{1ex}{${\sigma }_{ϵ}^2$}\!\left/ \!\raisebox{-1ex}{$n_i$}\right.$ is the standard error of the mean within person $i$, which depends on the variance in their response times across trials and the number of trials $n_i$ that they underwent in the task. Hierarchical pooling of each person’s observed sample mean response time $(y_i)$ toward the group-level mean $(\mu )$ then follows the equation:

$${\widehat{\theta }}_l=\mu +{\rho }_i\cdot \left(y_i-\mu \right)$$ (7)

where ${\widehat{\theta }}_l$ is the pooled person-level estimate.

Note that equations 6–7 are identical to equations 2–3—hierarchical pooling occurs through the same mechanism as regression-based true score estimation (50). As a result, we can view the “random effects” (or person-level estimates) estimated from hierarchical models as “true score estimates”, which are corrected for unreliability. The correspondence also extends to individual difference correlations. Akin to how the reliability of two measures can be used to disattenuate their observed correlation, models that directly estimate the correlation between latent person-level parameters result in individual difference correlations that are corrected for unreliability (9,33–35). The take-away is that CTT and hierarchical/latent variable models can account for measurement error in the same fundamental way, even to the point of producing identical results when the models are precisely specified. Figure 1 illustrates the effect of disattenuation (or hierarchical pooling).

Figure 1. Hierarchical pooling (or disattenuation) in behavioral data.

The same mathematics underlie both reliability-based disattenuation and the pooling in hierarchical models. Here we demonstrate hierarchical pooling in the context of delay discounting rates $\left(k\right)$ estimated for the same people across multiple sessions (a test-retest design). Gray dots indicate discounting rate estimates using maximum likelihood applied separately to each person, whereas red dots indicate the same estimates derived from a generative model with a multivariate hierarchical structure. Lines connect estimates that belong to the same person, revealing the effect that pooling has on each person. Notice that pooling occurs within each dimension simultaneously, producing generative model estimates with an overall higher correlation than the corresponding maximum likelihood estimates. Figure reprinted (with permission) from (9).

Generative Modeling as a General Framework

Given that classical and latent variable modeling frameworks can both be used to account for reliability, choosing the best one comes down to a question of flexibility. In the context of mental health research, models can often become quite complex. For example, we may use item response theory to estimate latent traits from a self-report, a reinforcement learning model to estimate parameters governing learning dynamics in a behavioral task, or a model of the hemodynamic response function in an fMRI experiment to estimate the effect of task stimuli on neural activity. Further, we may want to combine these models in some way, facilitating estimation of correlations between self-report, behavioral, and neural model parameters. CTT and classical structural equation modeling frameworks simply lack the flexibility to fully specify and estimate parameters from complex models with data spanning multiple modalities (e.g., neural, behavioral, self-report), structures (e.g., two-level, three-level, etc. hierarchies), and timescales (e.g., seconds, trials, days).

The generative modeling framework is well suited to tackle complex models that incorporate multiple data structures (including hierarchical) from multiple different modalities and timescales(9,34,51,52). Formally, we define a generative model as a statistical model of the joint probability distribution $p(y,\theta )$ on given observed data $(y)$ and unobserved parameters $(\theta )$. Conceptually, generative models can be thought of as probabilistic models that can simulate forward all aspects of both the observed and unobserved data/variables of interest. Once a generative model is fully specified, parameters can be estimated using either classical or Bayesian methods. However, Bayesian methods are often preferred for both practical and theoretical reasons. Practically, modern probabilistic programming languages (e.g., Stan, PyMC, JAGS) make specifying and fitting generative models to data relatively simple. Theoretically, Bayesian analysis aligns strongly with the philosophy of generative modeling—we first specify a joint probability distribution that captures our knowledge of the data generating process and our uncertainty in key parameters of interest (i.e. a generative model), and then we use probability theory (i.e. Bayes Rule) to condition our model on observed data and update our representations of uncertainty. Although this does require careful attention to the assumed data-generating process (53,54), the benefit is that Bayesian analysis offers a unified approach for estimating parameters of arbitrarily complex generative models that represent the substantive theories we aim to test while simultaneously accounting for unreliability or measurement error (55,56).

A Primer on Generative Brain–Behavior Modeling

To demonstrate the benefits of generative modeling, we will walk through a conceptual tutorial of joint brain-behavior modeling. Because reinforcement learning (RL) models have played such a large role in understanding a diverse range of mental health conditions (57,58), we focus our example on how to infer the relationship between trial-level prediction errors derived from a reinforcement learning model (for primers on RL models, see (59–61)) and corresponding BOLD signaling in a region of interest. We organized the following sections to: (a) detail the contrived experimental design; (b) describe the assumed behavioral and neural data-generating models, in addition to how they are linked together in the case of the generative model; (c) show how common analytical approaches can be viewed as special cases of the full generative model—cases that make unrealistic assumptions regarding measurement error; and end with (d) simulations to show how the generative approach compares to an approach that ignores reliability.

The Experimental Design

Throughout, we base our analyses on a simple event-related fMRI design wherein people undergo a probabilistic learning task. The task is a two-armed bandit where the true expected value of each option drifts across trials to motivate continued learning. Specifically, outcomes from each option follow a normal distribution with means ${\mu }_1$ and ${\mu }_2$ and standard deviations ${\sigma }_1={\sigma }_2=0.3$. Each option starts on trial 1 with ${\mu }_1={\mu }_2=0$. After an outcome is sampled on a given trial, both means are independently updated according to ${\mu }_{1:2}={\mu }_{1:2}\cdot \gamma +{ϵ}_{t,1:2}$, where $\gamma =0.99$ and ${ϵ}_{t,1:2}\sim 𝒩(0,0.1)$. A high value for $\gamma$ enforces a high autocorrelation between successive values of $\mu$, which makes it possible for people to develop a preference for one option over another as ${\mu }_1$ and ${\mu }_2$ temporarily drift apart. As depicted in Figure 2, we assume that the outcome presentation phase in the task for trial $t$ is jittered such that it occurs $s_t$ seconds after the previous trial $(t-1)$ outcome presentation, where $s_t\sim \mathrm{l}\mathrm{o}\mathrm{g}𝒩(2,0.2)$. Finally, we assume that brain volumes are measured by the fMRI scanner once every 2 seconds $(TR=2\mathrm{s})$. For simplicity, we focus purely on the stimulus outcome—but not choice—presentation phase of the task.

Figure 2. Schematic of the experimental task.

We designed the task used throughout our simulations such that the outcome presentation for each trial $t$ occurred $s_t$ seconds after the previous trial’s outcome presentation. To simplify the task for the purpose of demonstration, we assumed that $s_t\sim \mathrm{l}\mathrm{o}\mathrm{g}𝒩(2,0.2)$ such that the median time between outcome onsets was 2 seconds, with some variability from trial to trial. This design abstracts away the specific timings of each of the various task phases (e.g., fixation jitter time, choice time, and outcome presentation time), yet it retains the core elements necessary for simulations.

The Behavioral Model

We assume learning dynamics governed by delta-rule learning with learning rate $\alpha (0<\alpha <1)$:

$$\begin{gathered} V_t=V_{t-1}+\alpha \cdot P{E}_t \\ P{E}_t=(R_t-V_{t-1}) \end{gathered}$$ (8)

where $V_t$ is the expected value on trial $t$, and $P{E}_t=\left(R_t-V_{t-1}\right)$ is the corresponding prediction error between the actual reward $R_t$ and expected value. Note that $P{E}_t$ is the quantity that we aim to associate with corresponding neural activity.

Next, we assume that choice probabilities follow a softmax (i.e. multinomial logistic) choice function with inverse temperature $\xi (0<\xi <\infty )$ :

$$p_{t,c}=\frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(V_{t,c}\cdot \xi \right)}{\sum _{k=1}^2\mathrm{e}\mathrm{x}\mathrm{p}\left(V_{t,k}\cdot \xi \right)}$$ (9)

where $p_{t,c}$ is the probability of choosing option $c$ on trial $t$. These choice probabilities then give rise to actual choices per a categorical (or multinomial) distribution:

$$y_{t,B}\sim \mathrm{C}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{g}\mathrm{o}\mathrm{r}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{l}\left(\left[p_{t,1},p_{t,2}\right]\right)$$ (10)

Here, $y_{t,B}$ indicates the observed choice for the respective person and trial, where $B$ indicates that this is the behavioral (as opposed to neural) data. Figure 3 demonstrates how expected value and choice probability are represented by the behavior of the model on a trial-by-trial basis.

Figure 3. Behavioral model learning dynamics.

The top row shows changes in the learned expected value for each option as choices are simulated from the behavioral model described by equations 8–10 (where $\alpha =.2$ in the Low Learning Rate panel, $\alpha =.8$ in the High Learning Rate panel, and $\xi =1$ in both cases). The thick transparent lines indicate the true expected value for each option, which drifts with normally distributed noise across trials (note that the trajectory is identical across panels). Points on lines indicate that the given option was chosen and produced a positive prediction error on the given trial, resulting in an increase in learned expected value for the next trial. The bottom row shows how the expected values in the top row translate to choice probabilities. Together, the panels show that the simulated agent learns a noisy representation of the true expected value for each option, which is then passed through the choice rule to produce a slight preference for one option over the other dependent on past experience.

The Neural Model

Because the neural measure in our example is BOLD signaling measured throughout an fMRI scan, the neural model itself takes on the form of a hemodynamic response function (HRF). A detailed walkthrough on the properties of HRFs is outside the scope of the current tutorial (see (62) for details). For our purposes, we will assume that neural responses follow the canonical HRF—a double-gamma function that follows the form:

$$h(s)=\beta h_0(s)=\beta \left(\frac{s^{{\alpha }_1-1}{b}_1^{{\alpha }_1}\mathrm{e}\mathrm{x}\mathrm{p}(-b_{s}s)}{({\alpha }_1-1)!}-c\frac{s^{{\alpha }_2-1}{b}_2^{{\alpha }_2}\mathrm{e}\mathrm{x}\mathrm{p}(-b_{s}s)}{({\alpha }_2-1)!}\right)$$ (11)

Above, $h\left(s\right)$ is the height of the value of the HRF at time $s$ (where $s$ is continuous time in seconds), $h_0\left(s\right)$ can be thought of as the basis function of the HRF, the shape of which is governed by a set of pre-determined parameters as described in SPM12: ${\alpha }_1=6,{\alpha }_2=16,b_1=b_2=1$, and $c=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$6$}\right.$(63). The actual value of the HRF at time $s$ is determined by scaling this basis function by $\beta$, a parameter estimated directly from neural data. Thus, the parameter of interest in equation 11 is $\beta$, which represents the amplitude of the basis HRF in response to the behavioral stimuli assumed to elicit a neural response. We relegate details on the link between behavioral and neural model parameters to a later section.

The HRF described in equation 11 only captures the BOLD response to a single stimulus. To capture the temporal dynamics of the BOLD response over the course of an experiment, individual HRFs specified per equation 11 are assumed to follow in response to each stimulus outcome presentation. If we assume that BOLD responses are themselves time invariant, we can simply shift the HRF in equation 11 to start from each stimulus onset time, and then sum up the heights across HRFs at each timepoint to determine the expected BOLD response. Mathematically, this shifting is done by convolving the HRF with another function positioned at stimulus onset times (e.g., a boxcar or impulse function) (62). The summation across convolved HRFs is then conventionally done using matrix algebra, which provides a convenient way to separate out (and thereby estimate) the neural activation parameters $\beta$. Specifically, we can define a design matrix of convolved HRFs where rows represent trials $t=1,2,3,\dots ,T$ and columns represent TRs $s=0,2,4,\dots ,S$:

$$X=\left[\begin{array}{ccc}{h}_{0,1}\left(0\right)& \cdots & h_{0,1}\left(S\right)\\ ⋮& \ddots & ⋮\\ h_{0,T}\left(0\right)& \cdots & h_{0,T}\left(S\right)\end{array}\right]$$ (12)

Here, $h_{0,1}\left(0\right)$ is the value of the convolved basis HRF for the first trial $(t=1)$ when the first brain volume is taken at the start of the experiment $(s=0)$. Each row is therefore a timeseries of the convolved basis HRF for a given trial over the entire course of the experiment. Then, we can define a row vector of neural activation parameters as:

$$\mathit{\beta }={\left[\begin{array}{c}{\beta }_1\\ {\beta }_2\\ \cdots \\ {\beta }_T\end{array}\right]}^T$$ (13)

where ${\beta }_t$ captures neural activity in response to the stimulus outcome presentation on trial $t$. The scaling and summation of convolved basis HRFs can then be done simultaneously with:

$${\mu }_{\text{Neural}}\left(s\right)=\mathit{\beta }X$$ (14)

where ${\mu }_{\text{Neural}}\left(s\right)$ is the expected value of the BOLD signal at time $s$. Note that the assumed distribution of $\mathit{\beta }$ will become important for linking behavioral and neural models together later. In isolation, however, we can simply assume that weights follow a normal distribution such that $\mathit{\beta }\sim 𝒩({\mu }_{\beta },{\sigma }_{\beta })$.

Notably, we do not observe ${\mu }_{\text{Neural}}\left(s\right)$ directly. Instead, the observed BOLD signal contains measurement error resulting from a variety of factors. For example, the raw timeseries of an ROI or individual voxel is only obtained after the whole brain image has been pre-processed. Such pre-processing pipelines undoubtedly introduce measurement error. Here, we assume that such measurement error is normally distributed as follows:

$$y_{\text{Neural}}(s)\sim 𝒩({\mu }_{\text{Neural}}(s),{\sigma }_{\text{Neural}})$$ (15)

where $y_{\text{Neural}}\left(s\right)$ is the observed BOLD signal at time $s$, and ${\sigma }_{\mathrm{N}\mathrm{e}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{l}}$ captures all sources of measurement error that result in mis-fit between the predicted and observed BOLD signal. Figure 4 depicts HRFs and the resulting observed BOLD signal simulated from the neural model described by equation 11–15.

Figure 4. Shifting and summing up individual HRFs.

Individual HRFs are represented by dotted lines, each with a corresponding onset marked by the point at the start of the HRF. The solid line indicates the total BOLD signal as determined by equations 11–15. To show how variations in HRF amplitude—which capture stimulus effects—translate to variation in the observed BOLD signal, we assumed that $\mathit{\beta }\sim 𝒩(1,.2)$. For visualization purposes, we set ${\sigma }_{\text{Neural}}=0$. HRFs themselves do not appear smooth because we simulated them with a 2 second TR, capturing how they may be observed in an actual fMRI experiment.

Linking Behavioral and Neural Models

In the context of fMRI designs, behavioral and neural models can be combined in a variety of different ways dependent on our theory of the data-generating process (64). In our running example, we assume that behavioral task prediction errors directly give rise to the BOLD signals observed in response to outcome presentations throughout the experiment. From an inferential perspective, we could estimate the relationship between trial-level behavioral model prediction errors and corresponding single-trial neural $\mathit{\beta }$ weights in three different ways for an individual person:

1. Post-hoc model: Fit the models separately, extract trial-level point-estimates of prediction errors $(\widehat{\mathit{P}\mathit{E}})$ and $\mathit{\beta }$ weights $(\widehat{\mathit{\beta }})$, and then fit a regression model predicting $\widehat{\mathit{\beta }}$ with $\widehat{\mathit{P}\mathit{E}}$ (the hat “^” denotes an estimate);

2. Sequential model: Fit the behavioral model, extract trial-level point-estimates of prediction errors $(\widehat{\mathit{P}\mathit{E}})$, and then enter $\widehat{\mathit{P}\mathit{E}}$ as single-trial regressors in the neural model. This model is sometimes referred to as the “model-based regressor approach” throughout model-based neuroscience; or

3. Generative model: Fit the behavioral and neural models simultaneously, where latent trial-level prediction errors are used as single-trial regressors in the neural model

As we describe in the following paragraphs, the generative model is the only one that characterizes all assumed sources of uncertainty, thereby producing a better estimate of the relationship between trial-level prediction errors and corresponding BOLD signaling.

The generative model is specified by modifying the distribution on $\mathit{\beta }$ as follows:

$$\begin{gathered} {\mathit{\mu }}_{\beta }={\beta }_0+{\beta }_{PE}\mathit{P}\mathit{E} \\ \mathit{\beta }\sim 𝒩\left({\mathit{\mu }}_{\beta },{\sigma }_{\beta }\right) \end{gathered}$$ (16)

Here, $\mathit{\beta }$ is the vector of trial-level neural beta weights as described in equations 13–14, $\mathit{P}\mathit{E}$ is a vector of latent prediction errors as defined by equation 8, ${\beta }_0$ is an intercept term capturing background neural activation, ${\beta }_{PE}$ is a slope parameter that captures how strongly the latent prediction errors relate to BOLD responses, and ${\sigma }_{\beta }$ is an error term capturing the variation across $\mathit{\beta }$ conditional on $\mathit{P}\mathit{E}$. Equation 16 can be interpreted in many ways—a structural model between latent prediction errors and beta weights, a hierarchical model on beta weights with prediction errors as a group-level predictor, or a prior distribution on beta weights centered on latent prediction errors.

We can view both the post-hoc and sequential models as special cases of the generative model. Specifically, the sequential model is equivalent to the generative model under the assumption that prediction errors $(\mathit{P}\mathit{E})$ are estimated with perfect precision (i.e. without error), which can only be exactly true if the behavioral model parameters governing learning (just $\alpha$ in our example) are themselves estimated with without error. Similarly, the post-hoc model assumes that both the behavioral and neural parameters ($\mathit{P}\mathit{E}$ and $\mathit{\beta }$) are estimated with perfect precision, which only happens when all parameters contributing to uncertainty in learning or the BOLD response in either model $(\alpha ,{\beta }_0,{\beta }_{PE},\mathit{\beta },{\sigma }_{\beta })$ are estimated without error. Any imprecision in $\mathit{P}\mathit{E}$ or $\mathit{\beta }$ is then propagated to ${\sigma }_{\text{Neural}}$, which has the effect of attenuating the true correlation between $\mathit{P}\mathit{E}$ and $\mathit{\beta }$. Figure 5 demonstrates this attenuation with simulated data by comparing the recovered correlations between the post-hoc, sequential, and generative models. Despite the true correlation being 1 across all simulations, increases in uncertainty $({\sigma }_{\text{nerual}})$ produce attenuated, overconfident estimates for the post-hoc model, but not the sequential or generative models, when learning rates are generally low. However, when learning rates are high and uncertainty $({\sigma }_{\text{nerual}})$ is relatively low, the sequential model underestimates the true correlation. Overall, the sequential model performs comparably to the generative model in many cases—this is good news for previous research linking prediction errors to BOLD responses, much of which has used the sequential approach. The reason is quite nuanced, stemming from the phenomena described by (65), wherein the trial-level prediction error signal can be estimated well even when the learning rate is estimated with high error. We return to this point in the discussion.

Figure 5. Recovered brain-behavior correlation.

Recovered correlation between behavioral model trial-level prediction errors and neural model single-trial $\mathit{\beta }$ weights for the post-hoc, sequential, and generative models. We simulated data by varying ${\sigma }_{\text{Neural}}$ across a grid from 0 to 1. We set the behavioral ($\alpha =.2,\xi =1$ for Low Learning Rate; $\alpha =.8,\xi =1$ for High Learning Rate) and neural $({\beta }_0=0,{\beta }_E=1,{\sigma }_{\beta }=0)$ model parameters constant to isolate the effects of ${\sigma }_{\mathrm{N}\mathrm{e}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{l}}$. These particular parameter settings imply a perfect correlation $(r=1)$ between prediction errors and neural beta weights. After estimating parameters for each approach, we converted the regression model parameters to correlations with the formula: $r=\frac{{\beta }_{PE}}{\raisebox{1ex}{$SD\left(\mathit{\beta }\right)$}\!\left/ \!\raisebox{-1ex}{$SD\left(\mathit{P}\mathit{E}\right)$}\right.}$. Here, $SD\left(\mathit{\beta }\right)$ and $SD\left(\mathit{P}\mathit{E}\right)$ indicate the standard deviation of $\mathit{\beta }$ weights and prediction errors across trials. This formula is iterated over each posterior sample, preserving the full posterior distribution of correlations. Lines then indicate posterior means and intervals indicate 95% highest density intervals. Because we use the empirical standard deviations to convert parameters, the resulting correlation shows a small amount of error, which is why the intervals suggest that $r>1$ in some cases.

Discussion

Reliability is a factor that impacts statistical inference and is a challenge in all domains of psychological and psychiatric research. In the practice of cross-modality research, common approaches to examine associations compound measurement unreliability, while presuming that the units of analysis are perfectly measured. Through theoretical arguments and empirical simulations, we demonstrate how brain-behavior associations can be better estimated using fully generative models rather than models that rely on summary estimates of key quantities of interest. Further, we show how the assumptions underlying standard practices (e.g., the model-based regressor or sequential modeling approach) can be better understood within the generative modeling framework. Future theoretical work might build on the current models by considering variance due to researcher analytic decisions (10–12), individual differences in learning strategies (66), and differences in theoretical accounts of how neural activity relates to behavioral performance (66).

In the context of mental health research, the use of generative models holds potential to greatly improve measurement—both in the form of more precision and in the development of statistical models that better capture our theories of core neurocognitive phenomena (9,33,34,52,64). Our results show that, at minimum, researchers interested in developing neuroscientific measures based on prediction errors for mental health applications should use sequential (a.k.a. model-based regressor) over post-hoc models. However, there are cases wherein even the sequential model underperforms the more general generative model—with high learning rates, in a high neural signal-to-noise context, sequential model estimates of brain-behavior correlations show a bias toward 0. Given that clinical groups and health controls often show differences in learning rates (67), and may also show differences in BOLD signal variability (68), generative models may produce more consistent findings than sequential models when investigating clinical phenomena related to prediction errors. However, the extent to which these performance similarities generalize beyond the simple learning paradigms in the current study remains unknown. In fact, prediction errors may be a special case—they can be estimated well irrespective to learning rate estimation error given a relatively simple task design (65), which explains why sequential and generative models performed similarly in our simulations. Future work should systematically investigate how these results generalize across the learning task designs and associated models used throughout clinical neuroscience and computational psychiatry.

For researchers interested in digging deeper, the generative modeling philosophy and associated framework has continued to become more widely accessible throughout the past decade. For example, we now have: (a) introductory and advanced (69,70) statistics textbooks detailing the approach; (b) software packages that allow simplified linear model syntax-based model specification of custom models (71) in addition to those that offer total flexibility through the use of probabilistic programming languages (72); and (c) open-source online courses designed to cover both general generative modeling principles (73) in addition to those with a particular focus on computational cognitive models relevant for clinical science and computational psychiatry (74). For a more comprehensive tutorial on applying generative models to fMRI data, we highly recommend (62), who include the practical R code and fMRI pre-processing details required to analyze real-world data. We anticipate that generative modeling principles will be increasingly incorporated into graduate programs and research communities, aiding in the translation of clinical science to real-world clinical practice.