ABSTRACT
Competency‐based education (CBE) has shifted health professions training away from time‐based progression and numerical grading toward criterion‐referenced, pass/fail judgements of competence. As programmes adopt CBE, educators must design assessments that generate defensible evidence of competence while remaining sustainable at scale. This Clinical Teacher's Toolbox article presents a comparative framework for selecting summative, pass/fail assessments using two complementary lenses: authenticity and feasibility. We examine commonly used classroom‐based assessments, including higher order multiple choice questions, oral examinations (vivas), case‐based documentation write‐ups, chart‐stimulated recall, adaptive virtual case platforms, simulation, objective structured clinical exams (OSCEs), uploaded clinical video demonstrations, simulated patient education videos and emerging virtual or augmented reality approaches. We situate these assessments along Miller's Pyramid and domains of competence. Authenticity is conceptualised as a continuum shaped by task design and the extent to which assessments require integration of knowledge, reasoning, and performance aligned with professional practice. Feasibility reflects faculty workload, feedback capacity, calibration demands, scalability and vulnerability to inappropriate use of artificial intelligence. By integrating these lenses, we demonstrate how combining scalable written assessments with selectively deployed higher authenticity modalities can generate meaningful upstream evidence that supports downstream entrustment decisions. This toolbox offers practical guidance for designing repeatable, defensible and sustainable pass/fail assessment within competency‐based programmes.
1. Introduction
Competency‐based education (CBE) has become a prominent framework across health professions, shifting training away from time‐based progression and numerical grading toward criterion‐referenced, pass/fail judgements of competence. In this model, assessment plays a central role in learner progression, with repeated assessment and feedback intentionally embedded to support development over time.
Despite this shift, assessment practices remain variable. Traditional written examinations, particularly multiple‐choice formats, are still widely used at the course level because they are efficient, scalable and aligned with licensure testing. Although such assessments can contribute useful evidence, they generally provide limited insight into learners' ability to integrate knowledge, reasoning and performance compared with assessments that require explanation, interaction or demonstration.
2. Authentic Assessment
Authentic assessment in CBE is shaped by task design rather than format and involves appropriate challenge, performance or product generation, transfer, metacognition, professional fidelity and feedback opportunities [1]. These elements align closely with Miller's Pyramid, which provides a developmental structure for locating assessment evidence along a continuum from knowledge to observable performance.
3. Miller's Pyramid
Miller's Pyramid progresses from knowledge (“knows”) and applied understanding (“knows how”) to demonstration (“shows how”) and performance in practice (“does”), with entrustment representing a supervision judgement about a specific professional activity [2]. In CBE, assessment design involves intentional movement along this continuum to support increasingly complex demonstrations of competence (Figure 1).
FIGURE 1.
Miller's Pyramid, extended by 10 Cate and adapted here to position classroom‐based assessments as authenticity and feasibility shift. The “knows,” “knows how” and “shows how” levels provide progressively richer evidence of competence, while “does” and “trusted” occur in clinical training, where classroom evidence lays the foundation for entrustment.
4. Formative and Summative Assessments
Within CBE, formative and summative assessments serve complementary purposes: frequent formative assessment supports learning through low‐stakes, feedback‐rich activities, whereas summative assessment informs progression decisions and therefore demands greater reliability and defensibility [3, 4]. Although many assessment modalities may be used formatively, this paper focuses on their use as summative, pass/fail assessments.
5. What Is the Problem?
CBE prioritises observable performance, yet assessing performance at scale is difficult. Highly authentic methods such as simulation provide rich evidence but demand substantial time and resources, limiting routine use [5]. Without deliberate design, programmes may rely on unsustainable approaches or scalable formats that yield insufficient evidence. In pass/fail systems, educators need guidance to balance authenticity and feasibility while supporting defensible progression decisions.
6. Developing the Framework
We focused on assessments commonly used for summative, pass/fail decisions in classroom‐based courses, comparing them along two dimensions: authenticity and feasibility. Informed by our experience teaching and grading within a CBE curriculum and by assessment literature within and beyond CBE, these comparisons are intended as heuristic decision aids rather than validated rankings.
7. Authenticity as a Comparative Lens
In this paper, we treat authenticity as a comparative lens rather than a categorical judgement about assessment quality. Authentic assessment involves realistic tasks requiring learners to apply knowledge, skills and attitudes in ways that reflect professional practice [1, 6]. It also includes learner perceptions and identity formation through genuine explanation and dialogue [7].
Framing authenticity comparatively allows educators to examine how different pass/fail assessment formats generate distinct but complementary evidence. Within CBE, progression decisions draw on multiple sources of evidence, and no single assessment captures all domains of competence [3].
8. Authenticity of Different CBE Assessments
Table S1 in the supporting information situates commonly used pass/fail assessment modalities along an authenticity continuum, aligning each with typical levels of Miller's Pyramid and domains of competence. Competency domains are drawn from the Accreditation Council for Graduate Medical Education; because analogous domains exist across health professions, the table supports adaptation within discipline‐specific frameworks rather than exhaustive cross‐profession mapping.
Assessments higher on Miller's Pyramid generate observable performance evidence aligned with behaviour‐based domains such as patient care and communication. Midlevel formats foreground clinical reasoning and explanation, while lower level assessments primarily sample foundational and applied knowledge that supports performance.
9. Feasibility as a Comparative Lens
Authenticity alone does not determine whether an assessment survives in a CBE curriculum; feasibility often does. Even well‐aligned assessments must be implementable repeatedly, fairly and without exhausting faculty or institutional resources. Feasibility is therefore best understood as a comparative lens for examining how modalities function under real‐world conditions.
Authenticity alone does not determine whether an assessment survives in a CBE curriculum; feasibility often does.
Within CBE, feasibility reflects faculty workload, feedback capacity, scoring consistency, scalability and attention to generative artificial intelligence. Rather than judging feasibility in isolation, educators must consider how approaches balance these factors across courses and over time (Figure 2). Feasibility explains why some highly authentic assessments are used selectively, while others become routine, and why effective programmes rely on combinations rather than a single method [8].
FIGURE 2.
Feasability and authenticity as complementary considerations in assessment selection. Feasibility reflects grading effort, feedback timeliness, faculty alignment, scalability and vulnerability to inappropriate AI use; authenticity reflects Miller's level, domains of competence and EPA alignment.
Feasibility explains why some highly authentic assessments are used selectively, while others become routine.
10. Faculty Grading Effort, Feedback Potential and Time Demand
Table S2 in the supporting information situates the same assessments from Table S1 along a feasibility continuum, comparing faculty grading effort, feedback potential, alignment demands, scalability and vulnerability to inappropriate AI use. Faculty grading effort and feedback potential are closely linked and together shape the feasibility of authentic assessment.
Oral examinations (vivas) illustrate this balance. Vivas allow faculty to probe clinical reasoning in real time, clarify uncertainty and tailor feedback to individual learners, supporting assessment of integration and judgement [9]. However, their one‐to‐one format limits scalability and places substantial demands on faculty time, making them best suited for targeted use to clarify reasoning or resolve uncertainty rather than routine deployment across large cohorts. Brief oral examinations can be combined with other modalities to sample reasoning efficiently while reserving longer vivas for complex cases.
Uploaded clinical video demonstrations occupy a more moderate position on this continuum. Videos of patient education encounters or clinical skills performances shift observation to an asynchronous format, allowing faculty to review submissions and provide targeted feedback without the constraints of real‐time grading. From a learner perspective, the process of recording and reviewing these videos reinforces the skills that are being assessed. Learner‐produced videos promote the giving and receiving of formative feedback and offer a scalable approach for summative assessments [10]. Case‐based documentation write‐ups similarly allow learners to demonstrate clinical reasoning in a format that mirrors practice and remains scalable through structured scoring.
Technology‐enhanced approaches can further extend feedback capacity. Adaptive virtual case platforms can allow learners to navigate branching clinical decisions while generating structured performance data at scale. Virtual and augmented reality platforms support clinical decision‐making while shifting toward more repeatable training methods without simulated patients [11]. However, these efficiencies require upfront faculty investment to design meaningful cases and align automated feedback with performance. Taken together, these approaches highlight a central feasibility principle: combining selective, high‐effort assessments with more scalable modalities preserve educational value without creating unsustainable faculty workload.
11. Faculty Alignment, Scoring Consistency and Scalability
As performance‐based assessments scale, faculty alignment becomes essential. Modalities such as simulation often involve multiple assessors and contextualised judgement, increasing the need for shared standards to ensure defensible pass/fail decisions [5]. Chart‐stimulated recall similarly surfaces clinical reasoning and decision‐making under uncertainty through structured case review and guided reflection [12], further reinforcing the importance of calibration.
Consistency requires explicit rubrics, clear performance expectations and exemplar review as upfront investments that protect fairness. Scalability also adds constraint: simulation and oral examinations require substantial coordination of space, personnel and time, limiting frequent use in large cohorts. In contrast, video‐based assessments allow distributed, asynchronous review while maintaining shared criteria, supporting both alignment and scalability [10].
Consistency requires explicit rubrics, clear performance expectations and exemplar review as upfront investments that protect fairness.
12. AI Vulnerability and Detectability
The emergence of generative artificial intelligence (AI) has intensified scrutiny of assessment design. Increasing cognitive complexity, emphasising applied reasoning, incorporating clinical images or case vignettes and pairing written work with oral or supervised components can enhance authenticity while reducing vulnerability to inappropriate AI use [13]. For example, pairing case‐based multiple choice questions with a brief structured viva enables scalable knowledge assessment while making reasoning visible and mitigating AI‐related risk. Written assessments such as case‐based documentation are similarly vulnerable to AI‐assisted drafting, but pairing them with oral defense or observable performance keeps reasoning visible; mismatches between explanation and performance often surface during targeted questioning.
Pairing written work with oral or supervised components can enhance authenticity while reducing vulnerability to inappropriate AI use.
13. Practical Tips
Designing summative, pass/fail assessment within competency‐based curricula requires intentional planning. Table 1 outlines design principles that apply regardless of modality selection, supporting repeatable, defensible and sustainable pass/fail assessment in competency‐based curricula.
TABLE 1.
Design principles for repeatable, defensible and sustainable pass/fail assessment in competency‐based curricula.
| Design principle | What to do | Why it matters in CBE |
|---|---|---|
| Design for repetition | Choose assessments that can be repeated with minimal added grading time | Learners often require multiple attempts to demonstrate competence |
| Use complementary assessments across the curriculum | Sequence scalable and higher authenticity assessments intentionally across courses | Builds longitudinal, programmatic evidence without overburdening any single course |
| Align with entrustment | Require learners to explain decisions, communicate plans or reflect on uncertainty within assessments | Prepares learners for future supervision and trust decisions |
| Plan feedback deliberately | Choose assessments that allow timely, meaningful feedback | Feedback is central to progression but constrained by faculty time |
| Standardise pass/fail decisions | Use shared rubrics, exemplars and calibration | Improves fairness and consistency across assessors |
| Anticipate faculty assessment workload early | Consider cohort size, scheduling and faculty availability | Feasibility determines sustainability |
| Design with AI in mind | Pair written tasks with brief oral or performance components | Strengthens reasoning authenticity and protects validity |
Table 2 translates these principles into modality‐specific guidance, mapping common assessment goals to aligned assessment formats.
TABLE 2.
From goal to assessment modality: A practical decision guide for competency‐based educators.
| If you need to … | Consider … |
|---|---|
| Measure knowledge at scale | Multiple choice questions (MCQ) |
| Sample higher authenticity performance in large cohorts | Add targeted patient scenarios including videos to MCQs |
| Probe clinical reasoning verbally | Oral examination (viva) |
| Assess evolving decision‐making digitally | Adaptive virtual case platform |
| Evaluate documentation competence | Case‐based documentation write‐up |
| Observe patient communication skills | Simulated patient education video |
| Examine clinical reasoning through structured case review | Chart‐stimulated recall |
| Review recorded clinical skills | Uploaded clinical video demonstration |
| Develop clinical skills without use of standardised patients | Virtual or augmented reality |
| Sample structured performance across stations | Objective structured clinical exam (OSCE) |
| Evaluate performance in highly authentic environments | Simulation |
For example, in a 120‐learner foundational course, faculty might use higher order case‐based multiple choice questions as the primary summative assessment while sampling reasoning through brief, structured vivas for learners near the pass threshold. Selective use of uploaded clinical video demonstrations or simulated patient education tasks can then provide observable performance evidence without requiring full‐cohort simulation.
14. From Classroom Assessment to Clinical Entrustment
Classroom assessments provide essential upstream evidence for clinical entrustment decisions. In clinical training, readiness is operationalised through Entrustable Professional Activities (EPAs), with supervision levels reflecting increasing trust. Although EPAs are formally assessed in clinical settings [2], classroom assessments in courses such as anatomy can be deliberately designed to mirror documentation, oral presentation or diagnostic interpretation, thereby preparing learners for EPA performance [14].
Classroom assessments provide essential upstream evidence for clinical entrustment decisions.
15. Related Assessment Approaches and Scope Considerations
Several commonly used assessment approaches were excluded due to this paper's focus on summative, pass/fail, classroom‐based assessments. Script concordance testing can offer insight into clinical reasoning under uncertainty, but locally developed instruments require rigorous psychometric validation, and the limited availability of widely validated tools across disciplines constrains its defensible use across contexts.
Other approaches, including reflective writing, portfolios and peer feedback, support reflection and practice‐based learning and are highly scalable [10]. However, variability in expertise, potential bias and challenges with consistency limit their appropriateness for summative pass/fail decisions. These approaches are therefore best positioned as formative or contributory elements within a broader programmatic assessment system. Their exclusion reflects a deliberate scope decision, not a judgement of educational value.
16. Limitations and Contextual Considerations
This paper offers a comparative framework to inform assessment design rather than prescribe solutions, and alternative categorisations may suit different disciplines and institutions. Feasibility factors such as faculty workload, resources, technological infrastructure and cohort size vary widely and shape implementation. Alignment with Miller's Pyramid and domains of competence is meant to prompt reflection, recognising that task design and implementation often influence authenticity and feasibility more than format alone. Many assessments also function differently when used formatively or in combination.
Finally, choosing the right assessment is necessary but not sufficient for defensible pass/fail judgements, which also rest on sampling enough performance to generalise, mapping tasks to intended competencies, setting principled cut‐scores, calibrating assessors, documenting decisions, providing structured pathways for learners who do not pass and monitoring how decisions affect learners over time. The framework is best used as one part of a coherent programmatic assessment system, not a substitute for it.
17. Conclusion
Defensible progression in CBE depends on integrating complementary assessments over time. Because no single modality captures competence, educators must combine scalable approaches with selective higher authenticity methods to generate meaningful, sustainable evidence. Using authenticity and feasibility as guiding lenses supports alignment with entrustment expectations and prepares learners for clinical practice.
Using authenticity and feasibility as guiding lenses supports alignment with entrustment expectations and prepares learners for clinical practice.
Author Contributions
Jeb Helms: conceptualization, writing – original draft, writing – review and editing, project administration, investigation. Jessica Bender: conceptualization, investigation, writing – review and editing, writing – original draft. Samantha B. Stempel: validation, visualization, resources, writing – review and editing.
Funding
The authors have nothing to report.
Supporting information
Table S1: Authenticity of common pass/fail assessment modalities, showing each modality's typical Miller's Pyramid level, authenticity to clinical practice and domains of competence. Ratings are comparative rather than absolute and vary with the setting in which an assessment is delivered.
Table S2: Feasibility of common pass/fail assessment modalities, comparing faculty grading effort, feedback potential, alignment demands, scalability and vulnerability to inappropriate AI use. Ratings are comparative rather than absolute and vary with the setting in which an assessment is delivered.
Acknowledgements
The authors have nothing to report.
Data Availability Statement
Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.
References
- 1. Ashford‐Rowe K., Herrington J., and Brown C., “Establishing the Critical Elements That Determine Authentic Assessment,” Assessment and Evaluation in Higher Education 39, no. 2 (2014): 205–222, 10.1080/02602938.2013.819566. [DOI] [Google Scholar]
- 2. ten Cate O., Carraccio C., Damodaran A., et al., “Entrustment Decision Making: Extending Miller's Pyramid,” Academic Medicine 96, no. 2 (2021): 199–204, 10.1097/ACM.0000000000003800. [DOI] [PubMed] [Google Scholar]
- 3. Hawkins R. E., Welcher C. M., Holmboe E. S., et al., “Implementation of Competency‐Based Medical Education: Are We Addressing the Concerns and Challenges?,” Medical Education 49, no. 11 (2015): 1086–1102, 10.1111/medu.12831. [DOI] [PubMed] [Google Scholar]
- 4. Griffiths J., Dalgarno N., Schultz K., Han H., and van Melle E., “Competency‐Based Medical Education Implementation: Are We Transforming the Culture of Assessment?,” Medical Teacher 41, no. 7 (2019): 811–818, 10.1080/0142159X.2019.1584276. [DOI] [PubMed] [Google Scholar]
- 5. Buléon C., Mattatia L., Minehart R. D., et al., “Simulation‐Based Summative Assessment in Healthcare: An Overview of Key Principles for Practice,” Advances in Simulation 7, no. 1 (2022): 42, 10.1186/s41077-022-00238-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Vlachopoulos D. and Makri A., “A Systematic Literature Review on Authentic Assessment in Higher Education: Best Practices for the Development of 21st Century Skills, and Policy Considerations,” Studies in Educational Evaluation 83 (2024): 101425, 10.1016/j.stueduc.2024.101425. [DOI] [Google Scholar]
- 7. Ajjawi R., Tai J., Dollinger M., Dawson P., Boud D., and Bearman M., “From Authentic Assessment to Authenticity in Assessment: Broadening Perspectives,” Assessment and Evaluation in Higher Education 49, no. 4 (2024): 499–510, 10.1080/02602938.2023.2271193. [DOI] [Google Scholar]
- 8. Crawford L., Cofie N., McEwen L., Dagnone D., and Taylor S. W., “Perceptions and Barriers to Competency‐Based Education in Canadian Postgraduate Medical Education,” Journal of Evaluation in Clinical Practice 26, no. 4 (2020): 1124–1131, 10.1111/jep.13371. [DOI] [PubMed] [Google Scholar]
- 9. Abuzied A. I. H. and Nabag W. O. M., “Structured Viva Validity, Reliability, and Acceptability as an Assessment Tool in Health Professions Education: A Systematic Review and Meta‐Analysis,” BMC Medical Education 23 (2023): 531, 10.1186/s12909-023-04524-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Roberts C., Jorm C., Nisbet G., et al., “Video‐Based Peer Assessment of Collaborative Teamwork in a Large‐Scale Interprofessional Learning Activity,” BMC Medical Education 24, no. 1 (2024): 1307, 10.1186/s12909-024-06124-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Tene T., Vique López D. F., Valverde Aguirre P. E., Orna Puente L. M., and Vacacela Gomez C., “Virtual Reality and Augmented Reality in Medical Education: An Umbrella Review,” Frontiers in Digital Health 6 (2024): 1365345, 10.3389/fdgth.2024.1365345. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Mutter M., Kyle J. R., Yecies E., Hamm M., and DiNardo D., “Use of Chart‐Stimulated Recall to Explore Uncertainty in Medical Decision‐Making Among Senior Internal Medicine Residents,” Journal of General Internal Medicine 37, no. 12 (2022): 3114–3120, 10.1007/s11606-022-07396-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Newton P. and Xiromeriti M., “ChatGPT Performance on Multiple Choice Question Examinations in Higher Education. A Pragmatic Scoping Review,” Assessment and Evaluation in Higher Education 49, no. 6 (2024): 781–798, 10.1080/02602938.2023.2299059. [DOI] [Google Scholar]
- 14. Helms J. and Donovan S., “Shifting Anatomy Away From a Time‐Based Model: Competency‐Based Education Insights for Anatomy Educators,” Clinical Anatomy 39, no. 1 (2026): 60–71, 10.1002/ca.70033. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Table S1: Authenticity of common pass/fail assessment modalities, showing each modality's typical Miller's Pyramid level, authenticity to clinical practice and domains of competence. Ratings are comparative rather than absolute and vary with the setting in which an assessment is delivered.
Table S2: Feasibility of common pass/fail assessment modalities, comparing faculty grading effort, feedback potential, alignment demands, scalability and vulnerability to inappropriate AI use. Ratings are comparative rather than absolute and vary with the setting in which an assessment is delivered.
Data Availability Statement
Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.