Reply to van den Berg and Ma: Robust decision makers are not omniscient

In their letter, van den Berg and Ma (1) expresses three concerns regarding our study (2). First, they question the validity of our modeling work (1). Second, they doubt the novelty of our results (1). Third, they propose a simpler explanation for our data (1). We address these points in reverse order.

With regard to the latter point, van den Berg and Ma (1) might like to look again at our experiment 4. Observers weighted perceptually identical elements differently when they fell centrally or at the extremes of decision space (e.g., purple colors are downweighted in a red/purple task but not a red/blue task). This finding provides clear evidence that downweighting occurs in decision space and is not caused by distortions in perceptual space as they suggest (1).

Second, questioning the novelty of our findings, the work by van den Berg and Ma (1) refers to studies where observers integrate visual cues optimally across dimensions (3, 4). By contrast, our study showed robust averaging within a single task-relevant dimension (2). These two phenomena might share some computational principles; however, the answer awaits additional research.

Finally, they question our modeling work, arguing that the log probability ratio (LPR) model that we have described computes the probabilities incorrectly. Indeed, as we explained in our manuscript, our LPR model estimated the likelihood of each option as if elements were conditionally independent given the response category, an assumption that is incorrect (elements are independent given the distribution that they were sampled from but not conditioned on the response category, because there are six possible distributions for each category) (2). Importantly, van den Berg and Ma note that an optimal observer acknowledging nonindependence will weight all elements equally, unlike our LPR model or human observers.

We thank van den Berg and Ma (1) for their comments, which prompt us to clarify this point and highlight a broader issue at stake here. The correct model that they describe is indeed optimal: it uses the precise structure of our experiment, where elements on each trial were sampled from 1 of 12 possible Gaussians (6 in each response category) (1). However, unlike van den Berg and Ma (1), who have read our Methods section, the participants had no way of knowing this information. Rather, an ideal (but not omniscient) observer who simply computed the stimulus response association through feedback in the simplest possible way would perform exactly as our LPR model—and as the human observers.

Whether human observers solve perceptual or cognitive tasks in a mathematically optimal fashion is important (5), but whether a solution is optimal depends on the problem to be solved. Each optimal model embodies specific assumptions about the quantity that participants are trying to optimize, their prior beliefs, and the information that they can use and learn to solve the task. The work by van den Berg and Ma (1) describes a model that is based on different assumptions than our model and therefore, makes different predictions. Ultimately, however, cognitive scientists are concerned with identifying models that successfully predict human behavior. Although future research may eventually arbitrate against it, the LPR model in the our study constituted one such model (2).