Table 4. Comparison of ChatGPT- and CCC-Assigned Milestone Levels (1–5) by ACGME Competency and Word Count.
|
Competency |
Pearson |
Mean difference (ChatGPT-CCC) |
||
|
High |
Low |
High |
Low |
|
|
PC aggregate |
0.89 |
0.85 |
0.48 |
0.34 |
|
MK aggregate |
0.77 |
0.73 |
0.52 |
0.58 |
|
Prof aggregate |
0.75 |
0.59 |
0.69 |
0.67 |
|
ICS aggregate |
0.64 |
0.70 |
0.98 |
0.69 |
|
All aggregate |
0.85 |
0.8 |
0.61 |
0.51 |
Note: Median word count for all residents was 416 with high (n=12) being ≥416 and low (n=12) being <416.
Abbreviations: CCC, Clinical Competency Committee; ACGME, Accreditation Council for Graduate Medical Education; PC, patient care; MK, medical knowledge; Prof, professionalism; ICS, interpersonal and communication skills