Table 3.
Performance of different models.
| Predicted in (%) | Mean absolute percentage error (%) | Median absolute percentage error (%) | Prediction within 10% of ground truth (%) | Prediction within 1% of ground truth (%) | Prediction equal to ground truth (%) | |
|---|---|---|---|---|---|---|
| Ordered model | 98.0 | 9.2 | 0.0 | 87.1 | 78.2 | 76.9 |
| Accurate entities model | 96.0 | 7.8 | 0.0 | 88.2 | 79.9 | 78.5 |
| Conditional model | 64.7 | 1.7 | 0.0 | 96.9 | 93.8 | 93.8 |
| GPT-4o regular prompt | 99.3 | 1.5 | 0.0 | 96.6 | 90.6 | 88.6 |
| GPT-4o strict prompt | 94.7 | 0.6 | 0.0 | 98.6 | 92.3 | 90.8 |
The “Predicted in” column indicates the percentage of trials for which a prediction could be made. The remaining columns indicate the accuracy of the respective model in predicting the ground truth, ie, how many people were randomized.