Skip to main content
. Author manuscript; available in PMC: 2023 Nov 9.
Published in final edited form as: J Am Coll Surg. 2022 May 11;234(6):1181–1192. doi: 10.1097/XCS.0000000000000190

Table 5.

Variables That Can Be Used as Ground Truth to Validate Artificial Intelligence Methods and Artificial Intelligence— Enabled Metrics for Learning Curves and Personalized Curricula

Variable
Skill assessment
 1. Skill category (eg expert/novice/intermediate; 85%)
 2. Standardized structured rating scales (95%)
 3. Patient outcomes (90%)
Learning curve
 1. Specific operative process measures, such as blood loss, ischemia time, and so forth (90%), but not operative time (77.5%)
 2. Measures of procedure-specific surgical success, such as continence or nonconversion (87.5%)
 3. Postoperative outcomes such as complication; length ofhospital stay (92.5%)
 4. Oncologic outcomes such as surgical margins, number of lymph nodes, and so forth (95%)
 5. Patient-specific outcomes such as survival, patient-reported outcomes such as quality of life, satisfaction, and so forth (82.5%)
Personalized curricula
 1. Standardized milestones such as achieving a certain level of skill (100%)
 2. Surgeon perception of whether learner can be entrusted with specific aspects of care (80%)
 3. Performance in the operating room (100%)
 4. Surgical outcomes in patients (95%)
 5. Error in performing the operation (100%)

Panel agreement shown in parentheses.