Abstract
Objective
This study aimed to evaluate the quality of 3 objective structured clinical examinations (OSCEs) administered at a chiropractic college in 2024. The focus was on applying postexamination quality assurance methods suitable for small cohorts to assess examiner consistency, scoring patterns, and how well the exams distinguished between different levels of student performance.
Methods
Data were collected from 3 summative OSCEs: Clinic Entrance, Chiropractic Management, and Introduction to Chiropractic Practice. Borderline regression was used to calculate cut-scores. Additional QA metrics included number of failures, coefficient of determination (R2), intergrade discrimination, and between-group variation.
Results
Cut-scores for most OSCE stations were above the standard 70% threshold, indicating generally strong student performance. The exception was the neurological station in the Clinic Entrance exam, which had a cut-score of 66.7% and a failure rate of 53.1 percent. R2 values ranged from 0.40–0.79, with several stations showing moderate to strong alignment between checklist scores and overall examiner judgments. Between-group variation ranged from 40.9–79.5, suggesting examiner consistency varied across stations. Intergrade discrimination was low overall, indicating limited discrimination between performance levels and some examiner inconsistency.
Conclusion
Evidence-informed analysis of examinations is an important aspect of quality control in teaching. These findings suggest that OSCE quality is constrained more by examiner inconsistency and weak intergrade discrimination than by student performance per se. Targeted calibration of examiners and program-level redesign of station checklists and global rating scales are priorities for quality improvement.
Keywords: Clinical Competence, Checklist, Reproducibility of Results, Benchmarking, Chiropractic
INTRODUCTION
The Objective Structured Clinical Examination (OSCE) is a high-stakes examination based on the performance of clinical skills, including competencies of both practical skills and critical thinking.1 Since its inception in the 1970s2 it has been thoroughly studied and widely adopted by educational institutions in the medical and allied health fields.3 As with any examination, maintaining quality and rigor is vital.4 In the quality assurance of the OSCE there are many metrics available, but smaller cohorts, common in chiropractic colleges,5,6 pose significant issues with common statistical analyses.7 Previous research recommended test battery for small samples of OSCE scores, providing an evidence-based pathway for educators to analyze, review, and improve small-scale OSCEs.8 This follow-up report reviewed these recommended metrics and illustrated their utility with real-world exemplars of their use in an everyday academic environment.
Over the course of 2023 and 2024 the design and implementation of global rating scales (GRSs) was undertaken for various courses at the New Zealand College of Chiropractic, with the aim of using them alongside traditional checklist scoring for practical or OSCE-style examinations. The lead of each course was encouraged to create a GRS specific to their own course and needs. A GRS is a guide, anchored in examinee behavior, that is unlinked to the checklist score and gives an overall, or global, view of the examinee’s performance. For example, a scattergun approach, where the examinee does anything they can possibly think of, abstaining from thinking critically about the case and exam process, may mean the examinee passes if only a traditional checklist is used. This examinee approach, however, does not show clinical utility or excellence in performance, but more so indicates a shallow understanding of the clinical assessment and diagnostic approach.9–11 A GRS provides a counterpoint to a checklist and allows for a more objective review of performance. Checklists and GRSs, when assessed using a borderline regression (BLR) analysis,12,13 and alongside other OSCE quality assurance metrics (cut-scores, number-of-failures, R2, intergrade discrimination [IGD], and between-group variation [BGV]) can reveal issues with examination procedure and implantation such as problems in examiner training, inequality in examiner making, and missed concepts in student learning.14 An example of the GRS for the Management Plan station of an examination is shown in Figure 1.
Figure 1.

An example of GRS categories for a management plan station. GRS, global rating scales.
While this approach of encouraging each course leader to create their own checklist and GRS allowed alignment with local course content, it also introduced heterogeneity in rating scales that may limit cross-station comparability. Given prior concerns about the limited sensitivity of checklists to clinical reasoning and known examiner “hawk–dove” effects, we anticipated that metrics such as R2, IGD, and BGV might reveal tensions between checklist-based scoring, global judgments, and examiner consistency. Based on these concerns, this study aimed to evaluate the reliability and quality of 3 2024 OSCEs using quality assurance metrics suited to small cohorts.
METHODS
Study Design
Three courses with practical OSCE-style components that included both checklists and GRSs were reviewed using the quality assurance metrics from our previous research,8 and recommendations were made to improve examinations.
Deidentified examination scores for 3 2024 OSCE-style examinations in the fourth year of a 5-year chiropractic degree program were provided for analysis: Clinic Entrance (CE), Chiropractic Management (CM), and Introduction to Chiropractic Practice (ICP).
Data Sources
This study was reviewed and approved by the New Zealand College of Chiropractic Research Committee and was an anonymized data-driven study and included no active participants. Further ethical review was sought by the Health and Disabilities Ethics Committee of New Zealand. As the research included no active participants, anonymized data, and was part of course review metrics, it was considered out-of-scope for external ethical review (Health and Disabilities Ethics Committee out of scope reference: 2021 EXP 11582).
Quality Assurance Metrics
Borderline Regression
A borderline regression (BLR) analysis was used to identify cut-score (or pass mark based on the capacity of the cohort) in a way that is reliable for use in small scale OSCEs.12,15 A BLR can be used to infer if the station is too simple or complex. In the BLR method, checklist scores for each level of the GRS are plotted then fitted with a regression line, where the regression line for the borderline group intersects with the Y-axis is taken as the cut-score, or a pass mark dependent on the performance of the cohort as a whole,15,16 an easy station might result in the cut-score/new pass mark going up from 50%–70%, whereas a difficult station’s suggested cut-score might suggest the pass mark dropping from 70%–55%. It is important to note that a BLR analysis resulting in a cut-score higher than the pre-set standard pass mark does not allow us to determine if a cohort is performing well or the examination is too easy.
Consistent with recommended BLR usage in OSCE research,8 GRS categories (eg, 1–4 or 1–5) were treated as approximately interval-scaled predictors. This assumed that performance gaps between adjacent categories were roughly comparable. We recognize this as a pragmatic approximation and interpret regression results descriptively.
R Squared
The R2 coefficient is the proportional change in the dependent (checklist) variable due to change in the independent (GRS) variable. A higher R2 is better.9 An R2 of, for example, 0.95, implies that 95% of GRS variation are accounted for by variation in their checklist scores. A low R2 suggests the need to review the station checklist or design.4
Intergrade Discrimination
Intergrade discrimination (IGD) gives the average increase in checklist mark with each step up on the GRS, essentially the slope of the regression line.17 A flat slope indicates either high examiner variability of marking or a high number of failing students.4
Between-Group Variation
The proportion of total variance, or the variation of marks between groups (BGV) is an estimation of variance in checklist scores that is due to student performance only.18 This metric gives a good indication of the consistency of the examination process and can highlight differences between examiner marking, such as the hawk or dove effect.19 The BGV should be under 30%, with anything over 40% indicating a need for review.4 But the BGV should be viewed in concert with R2. A high BGV with high R2 suggests effects of inconsistent examiner marking, whereas a high BGV with a low R2 suggests a poorly designed checklist.4
Statistical Analysis
Deidentified data for 3 courses were evaluated. Descriptive statistics (unadjusted means, SDs, and counts) were used to describe the characteristics of the study sample. Adjusted R2, regression slopes, and proportion of total variance were created using the stats package from base R software (R Foundation for Statistical Computing). Borderline regression was performed via simple linear regression of OSCE checklist (dependent variable) and GRS scores (independent variable) using the R stats package. Model assumptions were visually inspected via quantile-quantile plots, fitted value and residual plots, and scatter plots and no major departures from linearity/homoscedasticity were noted within the constraints of small sample sizes. Statistical significance was defined as P < .05. All data were presented at 1 or 2 significant digits for ease of reading, but all calculations were performed with unrounded data.
RESULTS
For all examinations and stations, the standard pass mark was set at 70%, with a detailed breakdown of the station marks provided in Supplementary File 1. The CE examination had 6 stations: History, Orthopedic, Neurological, Regional, Spinal, and Report of Findings (ROF). The CM examination had 3 stations—Differential Diagnosis (DDX), Physical Examination, and Management—while the ICP examination was a singular station. The GRS format varied across examinations: the CE examination employed 4 categories (Poor, Pass, Good, Excellent), the CM examination used Poor, Borderline [fail], Good [pass], and Excellent, and the ICP examination used 5 categories (Poor, Fail, Pass, Good, Excellent). Each course lead developed their own GRS specific to the examination content and all stations had a standard pass mark of 70%.
Up to 47 examinees sat the CE, as not all examinees sat all stations. The mean score for the history station was 85.2% (n = 47), orthopedic was 80.5% (n = 47), neurological 68.1% (n = 32), regional 79.5% (n = 47), spinal 81.5% (n = 35), ROF 85% (n = 46), and the adjustment station was 77.7% (n = 32).
Fifty-one students took part in the CM examination with the mean grade being 87.7% for the DDX section, 87.4% for the physical examination station, and 88.1% for the management station. For this examination, a GRS category of “Good” was considered adequate performance. Seventy-eight students took part in the ICP examination with the mean grade being 88.6% for the regional section. A GRS score was missing for 5 examinees scores.
Borderline Regression Results
Borderline regression (BLR) analysis revealed variability in the relationship between the traditional pass mark and the calculated cut-score. In the CE examination, the History station yielded a cut-score of 14.8 (74%), which exceeded the standard pass mark, suggesting a high performing cohort or that a higher threshold could be considered. Similarly, the Orthopedic station cut-score was 15.5 (70.5%), aligning closely with the traditional standard, indicating appropriate difficulty for the cohort, although some examinees were graded “Pass” or “Good” despite failing via the checklist. The Neurological station produced a cut-score of 26 (66.7%), lower than the standard pass mark, indicating that a reduced threshold might be appropriate; however, the high number of “Poor” ratings suggests excessive difficulty. For the Regional station, the cut-score was 14.8 (70.5%), aligning with the standard pass mark, although the distribution between “Pass” and “Good” on the GRS suggests challenges in discrimination. The Spinal station’s cut-score of 14.5 (72.5%) was slightly higher than the pass mark, while the ROF station’s cut-score was 25.3 (72.2%), again slightly higher, however likely influenced by 2 very low-scoring examinees (Fig. 2A).
Figure 2.

BLR plot for CE. (A) History station, (B) Orthopedic station, (C) Neuro station, (D) Regional station, (E) Spinal station, and (F) Report of Findings (ROF) station. BLR, borderline regression; CE, Clinic Entrance.
Within the CM examination, the DDX station’s cut-score of 9.8 (81.8%) was considerably higher than 70%, suggesting either a high performing cohort or a station too simple for the cohorts’ capabilities. The Physical Examination station yielded a cut-score of 22.4 (86.1%), also indicating that the standard was set well below the observed performance level. The Management station’s cut-score of 15.4 (85.5%) suggested a similar pattern. For the ICP examination, the cut-score for the Regional station was 34.1 (77.5%), higher than the standard pass mark. Additionally, no “Poor” scores were recorded, suggesting the cohort performed well or the examination was too easy.
R2 (Coefficient of Determination)
The coefficient of determination (R2) values indicated varying degrees of correlation between checklist scores and GRS ratings. In the CE examination, the History station produced an R2 of 0.71, the Orthopedic station 0.48, the Neuro station 0.59, the Regional station 0.72, the Spinal station 0.63, and the ROF station 0.79. These results suggest moderate to strong associations across most stations, with Orthopedic demonstrating the weakest correlation. In the CM examination, the DDX station had an R2 of 0.66, the Management station 0.59, with the Physical Examination station again showing a weaker alignment between the checklist and the GRS. However, the Physical Examination station was 0.40. Low R2 values, as per the Physical Examination, indicate that checklist variance may not have been captured by the GRS, raising concerns that checklist items and global judgments could be targeting different constructs or that examiner application of the GRS was inconsistent. For the ICP examination, the R2 value for the Regional station was 0.59, indicating a moderate to strong relationship.
Intergrade Discrimination
Intergrade discrimination (IGD) scores, expressed in the context of each station’s total possible score, suggested variable discriminatory capacity across the stations. In the CE examination, the History station’s IGD was 2.0/20, Orthopedic 2.35/22, Neuro 4.77/39, Regional 2.78/21, Spinal 2.02/20, and ROF 4.69/35. These values indicate that, overall, the CE checklist items demonstrated relatively weak discrimination between candidates of differing ability levels, with the Neuro and ROF stations performing slightly better. Additionally, in the Orthopedic station, 6 examinees GRSs classified them as “good” or “pass” when they failed via their checklist score, suggesting the checklist contributed to poor discrimination of student ability. In the CM examination, IGD scores were 2.23/12 for DDX, 2.15/26 for Physical Examination, and 1.99/18 for Management, again indicating generally weak discriminatory performance. For the ICP examination, the IGD for the Regional station was 4.14/44, which, while numerically higher, still represents limited discrimination relative to the station’s maximum possible score.
Many IGD values were small relative to station maximum (eg, 4.77/39 for CE Neuro, 2.15/26 for CM Physical Examination), suggesting that a 1-step change in GRS often corresponded to only a modest change in checklist scores. This weak separation between performance levels implies that the current GRS steps and/or checklist weightings may not support meaningful discrimination between “Poor,” “Borderline,” “Good,” and “Excellent” performance.
Between-Group Variance
Between-group variance (BGV) results pointed towards potential examiner inconsistency in multiple stations, especially when viewed in concert with R2 results. Within the CE examination, BGV values were 71.98 for History, 49.42 for Orthopedic, 60.23 for Neuro, 72.37 for Regional, 64.36 for Spinal, and 79.46 for ROF. These values, particularly those exceeding 60, suggest substantial variability between examiner groups. In the CM examination, the DDX station recorded a BGV of 66.52, Physical Examination 40.92, and Management 60.04, again reflecting moderate to high interexaminer variability. For the ICP examination, the BGV for the Regional station was 59.83, indicating a high degree of potential inconsistency in examiner scoring. BGV values >60% in several stations indicate that much of the observed variance in checklist scores lay between examiner groups rather than within them. When combined with moderate-to-high R2 values, this pattern was consistent with systematic differences in examiner stringency.
Summary of Concerns
Across metrics, a subset of stations emerged as priorities for quality improvement. In the CE Neuro station, a low cut-score (66.7%), high failure rate (53.1%), moderate R2, and elevated BGV together suggest both station difficulty and examiner inconsistency. In the CE ROF and CE Regional stations, relatively high cut-scores combined with BGV >70% raise concerns about unequal standards between examiner groups. In the CM Physical Examination station, low R2 (0.40) and modest IGD highlight potential misalignment between checklist structure and global ratings.
DISCUSSION
Assessing the performance of the examinations overall, the stations seemed to work well, however the major themes from the analysis seem to be that some of the stations may be too easy for the current cohort of students, that there may be some inconsistencies in examiner marking, as evidenced by the R2 and BGV in the BLR analyses, and that checklists may need to be revised to better discriminate between students of different abilities, which may be bolstered by increasing the difficulty of the exam or refining the checklist.
Given that examiner inconsistency seemed to be a common theme, perhaps what is needed for improving quality assurance is a measure of examiner preparedness. If metrics such as these are to be useful, indeed if quality assurance of exams is to be a serious undertaking, not just a box-ticking exercise, should examiner competency be also assessed? Perhaps as part of examiner training, stations should be answered by examiners in isolation and model answers should not be disseminated between examiners until examiner competency or consistency has been shown. It may be that this procedure was too onerous, but previous research does support nonuniformity of OSCE examiners is an ongoing problem in many OSCEs.20 To increase the consistency of examiner marking, research supports the involvement of examiners in the design of the station and the checklist and recommends that examiners be active practitioners in the field.21 Lastly, better interexaminer reliability is found when examiners work in pairs,22 something that may be difficult to achieve in smaller tertiary institutes. While examiner inconsistency is an issue that should be addressed, any potential QA assessor also needs to weigh the possibility that a high standard pass rate may reflect a good teacher or a very capable, prepared cohort and may not necessarily reflect flaws in the teaching or examination process.
Assessing what a BLR tells us about the capability of the examinee or the difficulty of the examination also needs to be explored further. Is it necessary that a certain percentage of examinees must fail? Or does a high cut-score simply represent a well-prepared cohort of examinees rather than a simplistic station. Individual cut-scores need to be examined in the context of their examination, rather than applying blanket rules. For example, if a station can be shown to assess core practitioner competencies in a thorough manner, then a high cut-score should be evidence of a successful teaching and learning environment.
Future Studies
Based on the findings of the study, a 3-step plan was suggested to improve future OSCE quality. Step 1 would be the redesign of checklists and GRS. This would involve all examiners in co-developing program-level GRS frameworks (core domains and anchor wording) rather than fully bespoke course-level scales. Intergrade discrimination results would be used to decide where to increase the spread between categories (eg, improving the distinctions between “Borderline” and “Good”). Step 2 would focus on an examiner calibration strategy. This could consist of recording a small sample of stations which examiners could blind score. After scoring, a group session would be held, where scores and rationales are discussed. An additional consideration could include piloting paired examiners for high-stakes stations, with consensus scores. The final part, Step 3 – Re-audit, would be to re-apply R2, IGD, BGV and BLR metrics to the next OSCE cycle to assess whether these changes positively affect BGV and IGD.
While these QA metrics do shed some light on the quality of the examinations assessed they leave some questions still to be answered. For example, there is limited qualitative insight into why certain stations performed poorly or why examiner inconsistencies may have arisen. Future research could encompass student and examiner interviews or focus groups, which could provide valuable perspectives on these issues and help shape future examinations.
Future studies are planned that assess student preparedness/clinical readiness as they enter their internship, and throughout their remaining studies. It may be possible to combine metrics of clinical readiness with OSCE performance, which may help to inform teaching practices and curricula and potential effects on learning outcomes or clinical readiness.
Strengths
A strength of this OSCE analysis was the use of quality assurance metrics suited to small cohorts. Previous research has shown that, compared with a checklist, the GRS is more sensitive to expertise in examinees,23 has high interstation reliability or internal consistency, and high concurrent validity and construct validity.23,24
Limitations
A limitation of this study that is linked to examiner marking is the construct of determining what comprises a good GRS. We must ask if the GRSs used in this study are appropriate for the courses and examinations they were used. Earlier research has identified certain aspects of importance in GRS (response to the patient’s feelings and needs; degree of coherence in the interaction; verbal and nonverbal expression24), however these aspects do not cover the critical thinking required for a patient-centered diagnostic or management interaction. Additionally, because each course lead developed their own GRS, it cannot be assumed that a rating of “Good” or “Excellent” is comparable across courses. This heterogeneity likely contributed to both between-station differences in BLR slopes and the difficulties in interpreting program-level patterns. A key priority is the development of a validated, centralized GRS framework with shared domains and anchor descriptors.
Other limitations to consider are the distance between GRS categories. While future 3-step plans aim to make GRS categories equidistant, the current GRS categories may not be equidistant, limiting the precision of cut-scores. Additionally, having few “Poor” ratings may have reduced the ability to estimate slopes in the BLR analysis.
CONCLUSION
The review and analysis of 3 separate examinations for a chiropractic-specific degree revealed a generally high performing cohort but difficulties with examiner consistency and discrimination between levels of examinee performance. These results suggest that the quality assurance OSCE metrics proposed can provide useful information when assessing the performance of each OSCE station. For these data, improvements in examiner training, and checklist and GRS design were recommended. However, more research is needed to identify what type of examiner training is best, and what makes for the best GRS, and if further qualitative assessment of the examination process might also improve future exam iterations.
SUPPLEMENTARY FILE
Supplementary material associated with this article can be found in the online version at doi: 10.7899/JCE-25-24.
DECLARATIONS
Competing Interests
The authors declare no competing interests.
Data Availability
Data may be requested from the author upon reasonable request.
Artificial Intelligence
Artificial intelligence was not used to prepare this paper.
Contributorship
Concept development: AEC. Design: AEC. Supervision: AEC, KDWS, MF, IA. Data collection/processing: AEC, KDWS, MF, IA, KH. Analysis/interpretation: AEC, KDWS, MF, IA, KH. Literature search: AEC. Writing: AEC. Critical review: AEC, KDWS, MF, IA, KH.
Funding Statement
Funding No funding was received for this study.
REFERENCES
- 1.Yousuf N, Violato C, Zuberi RW. Standard setting methods for pass/fail decisions on high-stakes objective structured clinical examinations: a validity study. Teach Learn Med. 2015;27(3):280–291. 10.1080/10401334.2015.1044749 [DOI] [PubMed] [Google Scholar]
- 2.Cuschieri A, Gleeson FA, Harden RM, Wood RA. A new approach to a final examination in surgery. Use of the objective structured clinical examination. Ann R Coll Surg Engl. 1979;61(5):400–405. [PMC free article] [PubMed] [Google Scholar]
- 3.Barman A. Critiques on the objective structured clinical examination. Ann Acad Med Singap. 2005;34(8):478–482. [PubMed] [Google Scholar]
- 4.Pell G, Fuller R, Homer M, Roberts T;, International Association for Medical Education . How to measure the quality of the OSCE: A review of metrics – AMEE guide no. 49. Med Teach. 2010;32(10):802–811. 10.3109/0142159X.2010.507716 [DOI] [PubMed] [Google Scholar]
- 5.Hawk C, Rupert RL, Hyland JK, Odhwani A. Implementation of a course on wellness concepts into a chiropractic college curriculum. J Manipulative Physiol Ther. 2005;28(6):423–428. 10.1016/j.jmpt.2005.06.015 [DOI] [PubMed] [Google Scholar]
- 6.Harvey P, Goodell K. Development and evolution of an information literacy course for a doctor of chiropractic program. Comm Inform Literacy. 2008;2(1):52–61. 10.15760/comminfolit.2008.2.1.56 [DOI] [Google Scholar]
- 7.Atilgan H. Sample size for estimation of G and Phi coefficients in generalizability theory. Eurasian J Educ Res. 2013;13(51):215–227. [Google Scholar]
- 8.Cade AE, Mueller N. Measuring the quality of the OSCE in a chiropractic programme: a review of metrics and recommendations. J Chiropr Educ. 2023;38(1):9–16. 10.7899/JCE-22-29 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Regehr G, MacRae H, Reznick RK, Szalay D. Comparing the psychometric properties of checklists and global ratings scales for assessing performance on an OSCE-format examination. Acad Med. 1998;73(9):993–997. 10.1097/00001888-199809000-00020 [DOI] [PubMed] [Google Scholar]
- 10.Pell G, Homer M, Fuller R. Investigating disparity between global grades and checklist scores in OSCEs. Med Teach. 2015;37(12):1106–1113. 10.3109/0142159X.2015.1009425 [DOI] [PubMed] [Google Scholar]
- 11.Read EK, Bell C, Rhind S, Hecker KG. The use of global rating scales for OSCEs in veterinary medicine. PLoS One. 2015;10(3):e0121000. 10.1371/JOURNAL.PONE.0121000 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Moreno-López R, Hope D. Can borderline regression method be used to standard set OSCEs in small cohorts? Eur J Dental Educ. 2022;26(4):686–691. 10.1111/eje.12747 [DOI] [PubMed] [Google Scholar]
- 13.Sim JH, Abdul Aziz YF, Vijayanantha A, Mansor A, Vadivelu J, Hassan H. A closer look at checklist scoring and global rating for four OSCE stations: do the scores correlate well? Educ Med J. 2015;7(2):39–44. 10.5959/eimj.v7i2.341 [DOI] [Google Scholar]
- 14.Hurley KF, Giffin NA, Stewart SA, Bullock GB. Probing the effect of OSCE checklist length on inter-observer reliability and observer accuracy. Med Educ Online. 2015;20(1):1–5. 10.3402/meo.v20.29242 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Wood TJ, Humphrey-Murto SM, Norman GR. Standard setting in a small scale OSCE: a comparison of the modified borderline-group method and the borderline regression method. Adv Health Sci Educ Theory Pract. 2006;11(2):115–122. 10.1007/S10459-005-7853-1 [DOI] [PubMed] [Google Scholar]
- 16.Hejri SM, Jalili M, Muijtjens AMM, Vleuten Van Der CPM. Assessing the reliability of the borderline regression method as a standard setting procedure for objective structured clinical examination. J Res Med Sci. 2013;18(10):887–897. [PMC free article] [PubMed] [Google Scholar]
- 17.Steinhorst RK, Myers RH. Classical and modern regression with applications. J Am Stat Assoc. 1988;83(401):271. 10.2307/2288958 [DOI] [Google Scholar]
- 18.O’Grady KE. Measures of explained variance: cautions and limitations. Psychol Bull. 1982;92(3):766–777. 10.1037/0033-2909.92.3.766 [DOI] [Google Scholar]
- 19.Preusche I, Schmidts M, Wagner-Menghin M. Twelve tips for designing and implementing a structured rater training in OSCEs. Med Teach. 2012;34(5):368–372. 10.3109/0142159X.2012.652705 [DOI] [PubMed] [Google Scholar]
- 20.Reid K, Smallwood D, Collins M, Sutherland R, Dodds A. Taking OSCE examiner training on the road: reaching the masses. Med Educ Online. 2016;21:32389. 10.3402/MEO.V21.32389 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Yeates P, Moreau M, Eva K. Are examiners’ judgments in OSCE-style assessments influenced by contrast effects? Acad Med. 2015;90(7):975–980. 10.1097/ACM.0000000000000650 [DOI] [PubMed] [Google Scholar]
- 22.Mortsiefer A, Karger A, Rotthoff T, Raski B, Pentzek M. Examiner characteristics and interrater reliability in a communication OSCE. Patient Educ Couns. 2017;100(6):1230–1234. 10.1016/J.PEC.2017.01.013 [DOI] [PubMed] [Google Scholar]
- 23.Ilgen JS, Ma IWY, Hatala R, Cook DA. A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment. Med Educ. 2015;49(2):161–173. 10.1111/MEDU.12621 [DOI] [PubMed] [Google Scholar]
- 24.Hodges B, McIlroy JH. Analytic global OSCE ratings are sensitive to level of training. Med Educ. 2003;37(11):1012–1016. 10.1046/J.1365-2923.2003.01674.X [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
Data may be requested from the author upon reasonable request.
