(A) Schematic of experiment and results from Lerner et al. (2011). fMRI participants listened to an intact auditory narrative as well as versions scrambled at the scales of words, sentences and paragraphs. (B) Lower-level regions (e.g. auditory cortex) exhibited responses that were reliable across all stimuli, with little dependence on prior temporal context. By contrast, higher-level regions (e.g. precuneus) exhibited responses that depended at each moment on tens of seconds of prior context in the stimuli. (C) Schematic of the “process memory hierarchy”. Lower-level regions (e.g. sensory regions) exhibit shorter integration timescales, integrating over entities such as phonemes and words. Higher-level regions (e.g. lateral and medial parietal regions) exhibited longer integration timescales, combining information on the scale of entire events (e.g. paragraphs of text). (D) Schematic of predicted data when comparing the representations of brain regions sensitive to temporal context on different scales. The dependent variable is the “intact-scramble correlation”, quantifying the similarity of neural response to the same input in different contexts. (E) Schematic of a signal gain model for explaining the pattern of brain responses shown in panel D. (F) Schematic of hierarchical linear integrator model, HLI. LSS = long scale scramble, MSS = medium scale scramble, FSS = fine scale scramble. HLI = hierarchical linear integrator model.