Figure 5. Architecture, task, and training procedure influence the correlation between representations in DNNs and temporally emerging brain representations.

(a) We created 5 different models: 1) a model trained on object categorization (object DNN; Fig. 1); 2) an untrained model initialized with random weights (untrained DNN) to determine the effect of architecture alone; 3) a model trained on a different real-world task, scene categorization (scene DNN) to investigate the effect of task; and 4,5) a model trained on object categorization with random assignment of image labels (unecological DNN), or spatially smoothed noisy images with random assignment of image labels (noise DNN), to determine the effect of the training operation independent of task constraints. (b) All DNNs had significant representational similarities to human brains (layer-specific analysis in Suppl. Fig. 4). (c) We contrasted the object DNN against all other models (subtraction of corresponding time series shown in (b). Representations in the object DNN were more similar to brain representations than any other model except the scene DNN. Lines above data curves significant time points (n = 15, cluster definition threshold P = 0.05, cluster threshold P = 0.05 Bonferroni corrected by 5 (number of models) in (b), and 4 (number of comparisons in (c)); for onset and peak latencies see Suppl. Table 3a,b). Gray vertical lines indicates image onset.