Temporal Trade-Offs in Psychophysics

Abstract

Psychophysical techniques typically assume straightforward relationships between manipulations of real-world events, their effects on the brain, and behavioral reports of those effects. However, these relationships can be influenced by many complex, strategic factors that contribute to task performance. Here we discuss several of these factors that share two key features. First, they involve subjects making flexible use of time to process information. Second, this flexibility can reflect the rational regulation of information-processing trade-offs that can play prominent roles in particular temporal epochs: sensitivity to stability versus change for past information, speed versus accuracy for current information, and exploitation versus exploration for future goals. Understanding how subjects manage these trade-offs can be used to help design and interpret psychophysical studies.

Psychophysics quantifies relationships between physical and psychological events with appealing rigor and simplicity. In studies of perception, memory, preference, and other cognitive capacities, the manipulation of a property in the real world (e.g., the brightness of a visual stimulus to be detected, the number of items to be remembered, or the relative values of items to be selected) causes an “impression” in the brain that “must be liable to more or less accidental derangement at every step of its progress” [1, 2]. This noisy impression leads to probabilistic behavioral reports. Traditional methods combine precise measurements of these reports with models of the derangement process to infer the quality of the signals represented in the brain. In many cases, signal quality can be described using a simple scalar value like discrimination threshold.

However, even for the most carefully controlled psychophysical tasks, this simplicity belies the complexity of the decision processes that are used to convert the manipulated property into a measurable behavioral report [3–5]. Subjects must mediate attentional and memory demands, the integration of information from different sources, decision rules, preferences, biases, goals, and other strategic factors. In general, incomplete knowledge of how these factors are managed can affect the interpretation of psychophysical data [6*]. Accordingly, major advances in psychophysical approaches often involve new ways to identify and account effectively for the strategic choices made by subjects. For example, early detection experiments that measured a high probability of detecting a dim visual stimulus potentially conflated a low discrimination threshold with a tendency to report detections even in the absence of a stimulus. Signal detection theory resolved this issue by distinguishing the internal representation of the stimulus from the subject’s decision rule for making a perceptual judgment [3, 4].

Despite such successes, accounting for the different ways that subjects solve tasks remains a fundamental challenge for psychophysics. To help overcome this challenge, here we review recent work describing several strategic factors that share two key characteristics that can affect performance on a broad range of tasks (Figure 1). First, they involve the flexible use of time in processing information to generate responses. Second, this flexibility reflects inherent trade-offs between competing goals. We focus on three trade-offs that can interact and operate over multiple timescales. Each one also has particular relevance to how subjects process information from a certain temporal window: 1) a change-stability trade-off that governs the historical extent of past information that is relevant for future decisions, 2) a speed-accuracy trade-off that governs the duration of the current information-processing window, and 3) an explore-exploit trade-off that governs the future time horizon used for establishing goals. Studying these trade-offs under relatively simple conditions can yield insights into the kinds of complex choices subjects make more generally, which is useful in psychophysics for designing tasks and interpreting data.

Figure 1.

A psychophysical subject thinking about how to use past, present, and future information to solve a task.

Processing past information and a change-stability trade-off

Psychophysical subjects can make flexible use of information from the past to inform current choices. Under some conditions, such as perceptual tasks in which stimulus presentations are explicitly selected to be independent from trial to trial, subjects often make choices that are also largely independent of past choices, stimuli, or outcomes. However, even under these conditions choices and response times (RTs) can show sequential effects across trials. These effects can result from incomplete knowledge of the independent task structure, innate strategies like probability matching, lower-level motor mechanisms, or other factors [7–10]. In addition, many tasks are designed with across-trial dependencies, and subjects can use information from either the recent or more distant past to guide behavior [11, 12].

This flexible processing of historical data is likely governed by many factors, including task demands and memory limitations. In addition, the choice of how far in the past to consider historical information that is used to inform future choices may involve principles of effective inference in dynamic environments [13, 14]. These principles include an inherent trade-off between sensitivity to fundamental changes in a relevant signal and sensitivity to the value of the signal when it is stable but noisy [15**]. For example, consider a process that predicts the probability or magnitude of the next expected reward based on past stochastic rewards. Using a narrow time window that takes into account only the most recent rewards can allow such a process to constantly adjust its predictions based on new information. These adjustments are particularly useful for maintaining accurate predictions when the environment can undergo fundamental change-points. However, these predictions also tend to follow noise in the environment, giving rise to unstable predictions and potentially suboptimal choices during periods of noisy stability. In contrast, wider windows that include information further into the past can average out variability but are relatively insensitive to change-points, which are smoothed over.

Under some conditions, human subjects appear to regulate this trade-off adaptively, using wider windows during periods of expected stability and narrower windows during periods of expected change [13, 15**, 16]. Models of choice behavior that ignore these adaptive adjustments can mischaracterize the resulting choice variability as noise or other factors, resulting in potentially misleading interpretations of the underlying computational mechanisms [6*]. However, taking these adjustments into account can also be challenging because they do not simply depend on the statistical stability of information presented in a given task, but also on the subject’s subjective estimate of that stability. For example, even when statistically stable information is presented, subjects can use a narrow window to process historical information, acting as if they expect some level of instability [15**]. This kind of expectation might contribute to forms of imperfect information integration that have been found under a variety of conditions but are ascribed to other factors such as an urgency to respond [17, 18] or computationally costly or leaky mechanisms [19–21].

Processing incoming information and the speed-accuracy trade-off

Akin to the flexibility that subjects use to process past information, they also can make flexible use of time when processing incoming information. For many perceptual, memory, and other tasks, decision processes are deliberative and therefore take time [7, 22, 23]. This time is often used to improve accuracy by signal averaging over the incoming inputs or by increasing the number of sequential opportunities to make an effective judgment from those inputs [24, 25]. Under these conditions, choice accuracy tends to increase as a function of experimenter-controlled stimulus durations for up to several hundred milliseconds [26, 27]. For RT tasks, in which the subject chooses when to respond and hence effectively controls the duration of the stimulus-driven decision process, longer RTs also typically correspond to higher accuracy [28, 29]. Thus, the temporal window used to process incoming information governs an inverse relationship between decision time and decision accuracy, known as the speed-accuracy trade-off.

Control of this trade-off is flexible and can be affected by instructions, changes in reward contingencies or task timing, and other factors [30–32]. In the absence of an understanding of how this trade-off is managed, measurements of choice accuracy or RT alone provide an incomplete picture of the encoding and use of task-relevant information. For example, studies of perceptual learning, which involves long-lasting, training-induced improvements in perceptual sensitivity, often estimate perceptual thresholds before and after learning but typically do not also measure RTs [33]. Consequently, a measured increase in perceptual sensitivity could, in principle, result from an improvement in how the brain encodes the relevant sensory variable, an increased emphasis on the accuracy of the decision at the expense of speed, or both [34*].

Ambiguity from an uncharacterized speed-accuracy trade-off might also be relevant for tasks with experimenter-controlled stimulus durations. For example, subjects might use decision rules that can terminate the processing of incoming stimulus information at any time during stimulus presentation. Early termination subsequently filters out potentially useful information that could have been used to improve accuracy but frees up processing resources for other uses, such as preparing subsequent actions [35]. In these cases, a lack of RT measurements can make it difficult to assess the temporal window used by the subject to process incoming information. Further complicating this problem, information-processing dynamics within the chosen temporal window might not be uniform or predictable. For example, sniffs govern the timing of olfactory processing [21, 36]. Likewise, expectations about environmental changes that arise from processes governing the use of historical information, as discussed above, can cause the impact of incoming information on a subsequent decision to wax and wane over time [15**, 21, 36]. In these cases, a combination of approaches, including RT measurements, variable stimulus durations, and choice-guided reverse correlation of noisy stimulus features, can help infer both the temporal window and the information-processing dynamics the brain uses to process that incoming information [26, 35, 37, 38].

Processing future goals and the explore-exploit trade-off

Flexible temporal processing windows can also be applied to expectations [39]. To achieve the goal of maximizing reward in the near-term, subjects can exploit the option with the highest expected reward [40]. However, exploitation may neglect the possibility of gaining new information about other options, such as changes in reward contingencies, that are helpful for maximizing reward in the long-term. In contrast, exploring might provide new information but at the cost of missing out on available rewards in the near-term.

This exploration-exploitation trade-off can confound the interpretation of psychophysical data. Exploratory behavior can be modeled by adding stochasticity to the action-selection process [22, 40]. However, evidence for random choice behavior in dynamic environments is mixed, and many task designs conflate factors that could produce similar effects on behavior [41–43**]. For example, for tasks in which past choices govern the information available for the current choice, information-seeking behavior can be difficult to distinguish from reward-seeking behavior. Thus, decoupling information and reward can help identify exploratory aspects of behavior that affect psychophysical performance [43**]. More generally, over-emphasizing stochastic action selection can minimize sensitivity to other processes, such as adaptive inference to maximize near-term reward, that can contribute to behavioral variability [6*]. Conversely, ignoring a tendency to make stochastic choices can result in overestimates of uncertainty or adaptability in the exploitative decision process.

Exploration might also reflect other rational processes. For example, subjects might aim to reduce uncertainty, valuing actions proportional to the uncertainty about their outcomes [44–46], or to satisfy curiosity, valuing actions proportional to the quality of their predictions about the associated outcomes [47, 48]. These more complex strategies can be even more difficult to identify and address in psychophysical data. For example, subjects might appear to seek uncertainty in one context but avoid it in another [49, 50]. To account for these apparently contradictory effects, it can be helpful to identify different possible sources of uncertainty. These sources, which are also central to the change-stability trade-off, include unexpected change-points (uncertainty about the identity of a signal), estimation uncertainty (insufficient samples to provide a reliable estimate of the signal), and noise (intrinsic variability in the signal). Thus, the time window used for establishing goals about future rewards might be modulated by the same factors affecting the processing of past and present information [13–16, 51]. Using task designs that carefully distinguish these and other forms of uncertainty, and possibly the timescales over which they occur, can help to identify their distinct effects on behavior [16, 52*]. Accounting for these effects can, in turn, allow for more precise estimates of how the brain processes other task-relevant variables.

Conclusion

The rigor and simplicity of psychophysical tasks are based on the tenuous assumption that potentially complex strategic factors that affect task performance can be ignored, controlled, or addressed effectively. In this article we highlighted several factors that challenge this assumption and are prevalent across tasks. These factors share the common feature of flexibility in the temporal windows used to process information, as applied to the past, present, and future. This flexibility can make it difficult to infer from behavior the mapping between controlled physical events and the psychological responses they elicit. Nevertheless, because control of this flexibility often involves rational trade-offs in processing information, careful experimental design and analysis can help to account for their effects on behavior.

Highlights.

• The simplicity of psychophysical measurements belies underlying behavioral complexity

• This complexity includes rational trade-offs in using time to process information

• Temporal windows on past information govern a change-stability trade-off

• Temporal windows on incoming information govern a speed-accuracy trade-off

• Temporal windows on future expectations and goals govern an explore-exploit trade-off