Fig. 2. models distance metrics.

a Scatter plot of Open AI models (excluding ChatGPT and GPT-4 because their log-scores are not available) as a function of the two main principal components (PC2 and PC3), calculated from an estimation of the KL-Divergence between models (see Methods for details). The size of each datapoint is proportional to the size of the model. Grey datapoints designate models excluded from cognitive analysis, while red ones designate included ones. Note that, while by definition PC1 did explain more variance, in actuality it only captured the separation between AD1 and the remaining models. Thus, while not apparent in the present figure, it should be kept in mind that AD1 was far-removed from CDV2 and all other models along the PC1 axis. b Scatter plots where each datapoint represents an experimental vignette item (i.e., verbal stimuli) for the Cognitive Reflection Test (CRT; cyan, blue) and the Linda/Bill problem (L/B; pink, purple). Points are plotted as a function of the parameters of an exponential function fitted on the log-probability. Light colors (pink, cyan) represent the canonical items of each test as currently known. Dark colors (purple, blue) identify the new items that we created for this study. Results are plotted for davinci-003.