Table 2.
Program results
| % Courses passed | Question | ||||
|---|---|---|---|---|---|
| Program | τ = 50% | τ = 60% | τ = 70% | Max | count |
| Engineering | 80.0 | 60.0 | 40.0 | 0.83 | 1,343 |
| Chemistry | 83.3 | 66.7 | 50.0 | 0.85 | 1,417 |
| Life science | 85.7 | 71.4 | 57.1 | 0.85 | 1,477 |
| Physics bachelor | 100.0 | 55.6 | 33.3 | 0.86 | 958 |
| CS bachelor | 91.7 | 66.7 | 50.0 | 0.87 | 1,487 |
| CS master | 100.0 | 83.3 | 50.0 | 0.87 | 1,514 |
| Data science master | 90.0 | 70.0 | 30.0 | 0.86 | 1,576 |
| Physics online | 100.0 | 63.6 | 27.3 | 0.84 | 837 |
| Life science online | 85.7 | 71.4 | 57.1 | 0.75 | 996 |
For each program, the first three columns show the percentage of courses for which GPT-4 surpasses the thresholds τ = 50, 60, 70% correctly answered questions using the majority vote strategy. “Max” represents the proportion of questions in this degree correctly answered by at least one prompting strategy. Program levels are specified as Bachelor, Master, or Online. The first three programs (Engineering, Chemistry, Life Science) are first-year Bachelor course programs.