Table 2.
Performance of GPT-3.5 and GPT-4 by pre-professional competency
| Competency (n) | GPT-3.5 | GPT-4 | p-Value | ||||
|---|---|---|---|---|---|---|---|
| Incorrect, n (%) | Partial, n (%) | Correct, n (%) | Incorrect, n (%) | Partial, n (%) | Correct, n (%) | ||
| Capacity for improvement | 0 (0.0) | 17 (16.8) | 16 (20.5) | 0 (0.0) | 11 (13.8) | 22 (21.4) | 0.28 |
| Cultural competence | 1 (14.3) | 6 (5.9) | 4 (5.1) | 1 (33.3) | 4 (5.0) | 6 (5.8) | 0.84 |
| Ethical responsibility to self and others | 0 (0.0) | 21 (20.8) | 22 (28.2) | 0 (0.0) | 17 (21.3) | 26 (2.5) | 0.76 |
| Reliability and dependability | 1 (14.3) | 13 (12.9) | 9 (11.5) | 1 (33.3) | 11 (13.8) | 11 (10.7) | 1.00 |
| Resilience and adaptability | 0 (0.0) | 10 (9.9) | 1 (1.3) | 0 (0.0) | 5 (6.3) | 6 (5.8) | 0.46 |
| Service orientation | 0 (0.0) | 4 (4.0) | 5 (6.4) | 0 (0.0) | 4 (5.0) | 5 (4.9) | 1.00 |
| Social skills | 3 (42.9) | 11(10.9) | 12 (15.4) | 0 (0.0) | 17 (21.3) | 9 (8.7) | 0.085 |
| Teamwork | 2 (28.6) | 19 (18.8) | 9 (11.5) | 1 (33.3) | 11 (13.8) | 18 (17.5) | 0.25 |
| Total | 7 (4) | 101 (54) | 78 (42) | 3 (2) | 80 (43) | 103 (55) | 0.009 |
Statistically significant relationships (p < 0.05), determined through Fisher’s exact tests, are bolded