Reply to Wager et al.: Pain and the dACC: The importance of hit rate-adjusted effects and posterior probabilities with fair priors

Over the last half century, lesion and single-unit recording studies across multiple species converge on the dorsal anterior cingulate cortex (dACC) as central to pain processing (1–7). Our response (i) identifies a flaw in Wager et al.’s analysis (8) that underestimates the dACC’s contribution to pain and (ii) presents dACC-wide posterior probability analyses that provide further evidence that pain is a better account of dACC function than executive, conflict, or salience processes. Issues regarding our use of Z-scores, the proper definition of selectivity, and whether one can categorize a neural region in terms of a particular function are addressed elsewhere (https://www.psychologytoday.com/blog/social-brain-social-mind/201601/more-evidence-pain-related-description-dacc).

In Wager et al.’s reply (8), they ask a different question than we did in our original article (9). Wager et al. have a reasonable approach if one were interested in predicting the research topic of a randomly selected study from the Neurosynth database that shows a dACC effect. If, however, the goal was to assess the psychological function of the dACC (i.e., structure-to-function mapping), which was our goal, then correcting for study number disparities across terms is essential (which Neurosynth wisely does by default).

Imagine a database consisting of 100,000 attention studies and 100 pain studies. If a voxel is activated in 1,000 attention studies and all 100 pain studies, we would draw two conclusions. First, a randomly drawn study from the 1,100 with an effect would likely be an attention study. Second, because 100% of the pain studies produced an effect and only 1% of attention studies did, we would also conclude that this voxel is more selective for pain than attention. Hit rates (e.g., the number of pain studies that activate a region divided by the total number of pain studies in Neurosynth) are more important for assessing structure-to-function mapping than the historical tendency to conduct more studies on some topics than others.

We attempted to recreate and then build on Wager et al.’s analysis (8). At Montreal Neurological Institute (MNI) coordinates 0, 10, 34, we observed that 18% of 240 studies with effects were pain studies (Fig. 1A). However, Wager et al.'s analysis ignores the fact that cognitive terms have up to 5 times more studies in Neurosynth’s database than do pain terms (Fig. 1B) and thus can produce the numbers in Fig. 1A despite very low hit rates (Fig. 1C). After correcting for this disparity (Fig. 1D), 34% of the expected activations are from pain studies, which is nearly 5 times more than that from the largest cognitive term (7%) and more than from all cognitive terms combined.

Fig. 1.

(A) We recreated Wager et al.’s analysis (8) of 240 studies with activations within 8 mm of MNI coordinates 0, 10, 34 in the Neurosynth database. Two of us manually coded study titles and then considered these in light of Neurosynth-based term loadings for each study and, when necessary, consulted the original article. (B) The total number of studies in the Neurosynth database per term. (C) Term hit rates computed by dividing the number of studies observed at MNI coordinates 0, 10, 34 for a term by the total number of studies for that term in the Neurosynth database. (D) The number of studies that would be observed within 8 mm of MNI coordinates 0, 10, 34 assuming a fair distribution of studies (400 studies/term). Multiplying each term’s hit rate by 400 studies yields 100 expected effects (e.g., 400 pain studies × 10.0% hit rate = 40 expected pain studies; 400 motor studies × 1.2% hit rate = 5 expected motor studies). Note that this correction could not be applied to the heterogeneous “other” category and thus was kept at the 15% level from A. Altogether, there were a total of 118 total expected effects, of which 34% would be pain effects (i.e., 40/118), nearly 5 times more than the percentage for any cognitive terms, and more than all cognitive terms combined. Pain and emotion are the only terms that exceed the 8% expected by chance.

We also conducted a series of Neurosynth analyses that (i) consider all dACC voxels and (ii) directly compare pain studies against equal numbers of executive, conflict, and salience studies. Across all dACC voxels, the average posterior probability was 0.68 for pain, double those observed for the other terms (0.32) (Fig. 2A). In addition, 86% of dACC voxels had higher posterior probabilities for pain than for each other term (Fig. 2B). This is strong evidence that pain has a stronger structure-to-function relationship with dACC than executive, conflict, or salience processes.

Fig. 2.

(A) We used Neurosynth’s Python-based core tools to run a series of customized analyses with equal numbers of studies per term. We had Neurosynth directly compare, via random sampling, the maximum number of studies in each pairing that was possible while producing equal numbers per term (see B for counts). Thus, we created a fair empirical prior of 0.50 for each term. Posterior probabilities above 0.50 suggest some selectivity for one term over another. Posterior probabilities were computed for each of 1,110 voxels in our dACC mask (created from the Harvard−Oxford probabilistic atlas by selecting voxels at least 35% likely to be in dACC). Across all of the voxels in the dACC mask, the average posterior probability for pain relative to each term is shown above each bar (average = 0.68). This does not imply that 68% of all future dACC activations will be pain effects, but it does indicate a decidedly stronger structure-to-function mapping (i.e., relative selectivity) between dACC and pain than between dACC and these other terms. Shown at the bottom of each bar is the number and percentage of dACC voxels with a higher posterior probability for that term than its alternative. The posterior probability for pain is higher than for all three alternative terms (i.e., pain > executive AND pain > conflict AND pain > salience) in 86% of voxels (956/1,110). Results are similar when pain is compared against fear and autonomic. (B) Voxels with a greater posterior probability for pain are shown in red. Voxels with a greater posterior probability for the other terms are shown in blue. These analyses are based on the standard terms implemented by Neurosynth. Despite Wager et al. using these automated terms in multiple publications of their own and showing they correspond well with manually based categorization (10), they suggest that Neurosynth’s automated term generation may not correspond to the actual processes studied in different papers (8). To assess this, we examined a sample of papers in Neurosynth for the terms pain, executive, conflict, and salience (50 studies per term) and found ≥96% were termed appropriately.