Deciding How Much to Trust AI for Teaching and Assessment

Case: The Feedback That Arrived Before You Did

A program director (PD) reviews documentation feedback generated by an AI tool being piloted in the residency program. The system analyzed a postgraduate year 2’s admission note and produced detailed suggestions: tighten the history of present illness, clarify the differential, strengthen the reassessment plan. The feedback is specific, well-organized, and ready to share.

But the PD pauses. The AI didn’t see the patient. It doesn’t know this trainee’s baseline, learning goals, or that she struggles with confidence rather than competence. Should this feedback go directly to the resident? Be reviewed first? Be discarded entirely?

These are new questions facing many medical educators: not whether to use AI for teaching and assessment, but how much autonomy to grant AI, and for which tasks.

From Supervising Residents to Supervising Tools

The first 2 articles in this series addressed AI’s arrival in the clinical learning environment and potential strategies for supervising residents who use AI.^1,2 This installment shifts focus to the AI tools themselves. When programs deploy AI for teaching and assessment tasks, such as generating practice questions, summarizing simulation debriefs, or categorizing resident feedback by milestone, how should educators decide what level of oversight is required?

The instinct to treat all AI outputs with equal skepticism or equal enthusiasm misses the point. A tool that drafts board-style practice questions poses different risks than one summarizing a trainee’s portfolio for the Clinical Competency Committee. Context matters. Stakes matter. Supervision should vary accordingly.

Applying entrustment thinking to AI tools offers a path forward. Just as supervision of trainees is calibrated based on demonstrated competence and task demands, supervision of AI can be calibrated based on the tool’s demonstrated performance and the educational stakes involved.

SCALES-AI: An Entrustment Framework for Educational AI

SCALES-AI (Supervision- and Context-Aligned Levels of Entrustment for AI in Education) provides a systematic approach to this problem.³ Rather than asking whether AI is good or bad, the framework poses a more useful question: For this tool doing this task in this context, what level of autonomy is appropriate?

The SCALES-AI framework links 4 foundational principles to 4 dimensions of trustworthiness. Ability asks whether the tool performs the task accurately and consistently for the intended use. Integrity asks whether the tool is transparent and reliable, whether outputs are reproducible, and whether the reasoning can be traced. Benevolence asks whether the tool serves learners’ and educators’ best interests, or whether it might inadvertently cause harm. Equity asks whether the tool performs fairly across learner populations or risks amplifying bias.

These dimensions inform a 6-level entrustment scale (0-5) that governs how much autonomy an AI tool should have for a specific educational task (Figure).

Figure.

SCALES-AI Entrustment Levels (0-5)

Abbreviation: SCALES-AI, Supervision- and Context-Aligned Levels of Entrustment for AI in Education.

Note: Levels determine appropriate AI autonomy for educational tasks based on demonstrated trustworthiness. Higher levels require less supervision but are reserved for lower-stakes applications. Adapted from Preiksaitis et al.³

At Level 0, the task is simply not appropriate for AI because it requires human judgment, empathy, or nuanced contextual understanding. Final competency determinations and sensitive feedback on professionalism belong here. At Level 1, the AI provides information only while the human retains full responsibility for all use decisions. At Level 2, the AI may act, but a human directly oversees performance with the ability to intervene immediately if needed. Level 3 allows indirect supervision, where AI performs tasks with human review of outputs before implementation or distribution to others. At Level 4, the AI operates autonomously within defined parameters with periodic performance audits. Level 5, reserved for exceptional cases involving narrow and low-stakes tasks, allows full autonomy with continuous monitoring for quality assurance.

High-stakes summative decisions should remain at Levels 0 to 2. Level 5 is reserved for tasks where the consequences of error are minimal and performance has been evaluated extensively.

The Same Tool, Different Levels

A critical insight from SCALES-AI is that the same AI tool might warrant different levels for different tasks or different levels depending on how it is configured.

Consider documentation feedback in the case above. A basic prompt asking the AI to provide feedback on a trainee’s note warrants Level 2: outputs may miss important nuance, so every instance needs faculty review before sharing with the resident. The same model configured with a rubric-specific prompt and retrieval-augmented generation grounded in local documentation guidelines might reach Level 3, for appropriate trainees. Here periodic spot-checks would replace review of every output.

The same approach applies to practice question generation. A general topic prompt warrants Level 3 because questions are useful but require faculty edits before use. A model that has been fine-tuned on board-style questions, for a specific content area, might reach Level 4: operating autonomously within a defined scope while faculty conduct periodic quality audits.

Thus, the entrustment level is not a fixed property of the AI tool. It is a property of the tool-task pairing, in a specific context.

What Should Stay at Level 0

Some educational tasks should not be delegated to AI regardless of how well it performs on other measures. Final competency determinations belong here because decisions about promotion, graduation, and readiness for unsupervised practice require human accountability. Providing feedback on professionalism concerns requires empathy, relationship, and judgment that AI cannot provide. High-stakes decisions that a trainee can potentially and meaningfully contest demands human ownership. Assessments in which the relationship between educator and learner is itself developmental, such as coaching conversations and remediation planning, lose their essential character if automated.

The principle is straightforward: the higher the stakes and the greater the need for human understanding, the lower the appropriate AI level of autonomy. These boundaries should be explicit in program policy so that everyone understands what AI will and will not do.

Levels Are Not Permanent

Entrustment under SCALES-AI is dynamic rather than fixed. A tool’s level may change when the underlying model updates, when the prompt or configuration changes, when the grounding corpus such as guidelines or rubrics is revised, when performance drifts over time, when faculty or trainees report concerns, or when equity checks reveal differential performance across groups. For this reason, SCALES-AI pairs decisions at each level with reevaluation triggers: prespecified conditions that prompt reassessment. This prevents programs from adopting a tool and forgetting to reassess use. Human judgment must stay in the loop as tools evolve. See Box 1 for an example to integrate the SCALES-AI approach. See Box 2 for useful resources.

Box 1. Try This

• Identify one AI tool the program uses or is considering for teaching or assessment. Map it against the 4 dimensions (Ability, Integrity, Benevolence, Equity) and note where evidence is strong and where gaps remain.

• Assign a provisional AI entrustment level, by starting conservatively. A tool can earn more autonomy over time; trust lost is harder to rebuild.

• Define scope limits that specify what the tool is allowed to do and what is explicitly out of bounds.

• Specify reevaluation triggers that will prompt reassessment, such as model updates, quarterly audits, or a threshold number of faculty concerns.

• Name at least one task that stays at Level 0 and make this explicit in program policy. Transparency about what AI will not do builds trust in what it can do well.

Box 2. Useful Resources

Preiksaitis C, Nord G, Davis J, Sangal R, Mackey C, Oh L, Rose C. SCALES-AI: a supervision- and context-aligned entrustment framework for integrating artificial intelligence into emergency medicine education. Preprint. EdArXiv.

Full entrustment scale, worked scenarios, implementation checklist, and interrater calibration template.

Gin BC, O’Sullivan PS, Hauer KE, et al. Entrustment and EPAS for artificial intelligence (AI): a framework to safeguard the use of AI in health professions education. Acad Med. 2025;100(3):264-272. doi:10.1097/ACM. 0000000000005930

Foundational work adapting entrustment concepts to AI in health professions education.

Masters K. Ethical use of artificial intelligence in health professions education: AMEE guide no. 158. Med Teach. 2023;45(6):574-584. doi:10.1080/0142159X.2023.2186203

Framework for transparency, accountability, and learner welfare when deploying educational AI.

Looking Ahead

Entrustment frameworks move past binary thinking. AI is not simply trustworthy or untrustworthy. The question is always: Trustworthy for what, under what conditions, with what oversight? As programs pilot AI tools for teaching and assessment, the goal is not to maximize automation but to match autonomy to demonstrated performance, preserve human judgment where it matters most, and build governance that can adapt as capabilities evolve.

The next article in this series will address educational research: how to design studies involving AI, establish validity evidence for AI-assisted measures, and navigate the reproducibility challenges unique to rapidly changing tools. Let us know about your successes, concerns, and questions about using AI for teaching and assessment in GME.