Skip to main content
PMC home page
Annals of Surgery logoLink to Annals of Surgery
. 1995 Dec;222(6):735–742. doi: 10.1097/00000658-199512000-00007

The Objective Structured Clinical Examination. The new gold standard for evaluating postgraduate clinical performance.

D A Sloan 1, M B Donnelly 1, R W Schwartz 1, W E Strodel 1
PMCID: PMC1235022  PMID: 8526580

Abstract

OBJECTIVE: The authors determine the reliability, validity, and usefulness of the Objective Structured Clinical Examination (OSCE) in the evaluation of surgical residents. SUMMARY BACKGROUND DATA: Interest is increasing in using the OSCE as a measurement of clinical competence and as a certification tool. However, concerns exist about the reliability, feasibility, and cost of the OSCE. Experience with the OSCE in postgraduate training programs is limited. METHODS: A comprehensive 38-station OSCE was administered to 56 surgical residents. Residents were grouped into three levels of training; interns, junior residents, and senior residents. The reliability of the examination was assessed by coefficient alpha; its validity, by the construct of experience. Differences between training levels and in performance on the various OSCE problems were determined by a three-way analysis of variance with two repeated measures and the Student-Newman-Keuls post hoc test. Pearson correlations were used to determine the relationship between OSCE and American Board of Surgery In-Training Examination (ABSITE) scores. RESULTS: The reliability of the OSCE was very high (0.91). Performance varied significantly according to level of training (postgraduate year; p < 0.0001). Senior residents performed best, and interns performed worst. The OSCE problems differed significantly in difficulty (p , 0.0001). Overall scores were poor. Important and specific performance deficits were identified at all levels of training. The ABSITE clinical scores, unlike the basic science scores, correlated modestly with the OSCE scores when level of training was held constant. CONCLUSION: The OSCE is a highly reliable and valid clinical examination that provides unique information about the performance of individual residents and the quality of postgraduate training programs.

Full text

PDF
735

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Ansell J. S., Boughton R., Cullen T., Hodges C., Nation E., Peters P., Scardino P. Lack of agreement between subjective ratings of instructors and objective testing of knowledge acquisition in a urological continuing medical education course. J Urol. 1979 Dec;122(6):721–723. doi: 10.1016/s0022-5347(17)56572-6. [DOI] [PubMed] [Google Scholar]
  2. Cohen R., Reznick R. K., Taylor B. R., Provan J., Rothman A. Reliability and validity of the objective structured clinical examination in assessing surgical residents. Am J Surg. 1990 Sep;160(3):302–305. doi: 10.1016/s0002-9610(06)80029-2. [DOI] [PubMed] [Google Scholar]
  3. Endean E. D., Sloan D. A., Veldenz H. C., Donnelly M. B., Schwarcz T. H. Performance of the vascular physical examination by residents and medical students. J Vasc Surg. 1994 Jan;19(1):149–156. doi: 10.1016/s0741-5214(94)70129-6. [DOI] [PubMed] [Google Scholar]
  4. Gleeson F. Defects in postgraduate clinical skills as revealed by the objective structured long examination record (OSLER). Ir Med J. 1992 Mar;85(1):11–14. [PubMed] [Google Scholar]
  5. Goetzl E. J., Cohen P., Downing E., Erat K., Jessiman A. G. Quality of diagnostic examinations in a university hospital outpatient clinic. Ann Intern Med. 1973 Apr;78(4):481–489. doi: 10.7326/0003-4819-78-4-481. [DOI] [PubMed] [Google Scholar]
  6. Grosse M. E., Cruft G. E., Blaisdell F. W. The American Board of Surgery in-training examination. Arch Surg. 1980 May;115(5):654–657. doi: 10.1001/archsurg.1980.01380050076018. [DOI] [PubMed] [Google Scholar]
  7. Harden R. M., Stevenson M., Downie W. W., Wilson G. M. Assessment of clinical competence using objective structured examination. Br Med J. 1975 Feb 22;1(5955):447–451. doi: 10.1136/bmj.1.5955.447. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Irby D. M., Fantel J. I., Milam S. D., Schwarz M. R. Legal guidelines for evaluating and dismissing medical students. N Engl J Med. 1981 Jan 15;304(3):180–184. doi: 10.1056/NEJM198101153040329. [DOI] [PubMed] [Google Scholar]
  9. Joorabchi B. Objective structured clinical examination in a pediatric residency program. Am J Dis Child. 1991 Jul;145(7):757–762. [PubMed] [Google Scholar]
  10. Langdon L. O., Grosso L. J., Day S. C., Norcini J. J., Kimball H. R., Fletcher S. W. A core component of the certification examination in internal medicine. J Gen Intern Med. 1993 Sep;8(9):497–501. doi: 10.1007/BF02600111. [DOI] [PubMed] [Google Scholar]
  11. Lunenfeld E., Weinreb B., Lavi Y., Amiel G. E., Friedman M. Assessment of emergency medicine: a comparison of an experimental objective structured clinical examination with a practical examination. Med Educ. 1991 Jan;25(1):38–44. doi: 10.1111/j.1365-2923.1991.tb00024.x. [DOI] [PubMed] [Google Scholar]
  12. Matsell D. G., Wolfish N. M., Hsu E. Reliability and validity of the objective structured clinical examination in paediatrics. Med Educ. 1991 Jul;25(4):293–299. doi: 10.1111/j.1365-2923.1991.tb00069.x. [DOI] [PubMed] [Google Scholar]
  13. Maxim B. R., Dielman T. E. Dimensionality, internal consistency and interrater reliability of clinical performance ratings. Med Educ. 1987 Mar;21(2):130–137. doi: 10.1111/j.1365-2923.1987.tb00679.x. [DOI] [PubMed] [Google Scholar]
  14. Newble D. I., Swanson D. B. Psychometric characteristics of the objective structured clinical examination. Med Educ. 1988 Jul;22(4):325–334. doi: 10.1111/j.1365-2923.1988.tb00761.x. [DOI] [PubMed] [Google Scholar]
  15. Norman G. R., Tugwell P., Feightner J. W. A comparison of resident performance on real and simulated patients. J Med Educ. 1982 Sep;57(9):708–715. doi: 10.1097/00001888-198209000-00008. [DOI] [PubMed] [Google Scholar]
  16. Petrusa E. R., Blackwell T. A., Ainsworth M. A. Reliability and validity of an objective structured clinical examination for assessing the clinical performance of residents. Arch Intern Med. 1990 Mar;150(3):573–577. [PubMed] [Google Scholar]
  17. Reisner E., Dunnington G., Beard J., Witzke D., Fulginiti J., Rappaport W. A model for the assessment of students' physician-patient interaction skills on the surgical clerkship. Am J Surg. 1991 Sep;162(3):271–273. doi: 10.1016/0002-9610(91)90086-s. [DOI] [PubMed] [Google Scholar]
  18. Reznick R. K., Blackmore D., Cohen R., Baumber J., Rothman A., Smee S., Chalmers A., Poldre P., Birtwhistte R., Walsh P. An objective structured clinical examination for the licentiate of the Medical Council of Canada: from research to reality. Acad Med. 1993 Oct;68(10 Suppl):S4–S6. doi: 10.1097/00001888-199310000-00028. [DOI] [PubMed] [Google Scholar]
  19. Reznick R., Smee S., Rothman A., Chalmers A., Swanson D., Dufresne L., Lacombe G., Baumber J., Poldre P., Levasseur L. An objective structured clinical examination for the licentiate: report of the pilot project of the Medical Council of Canada. Acad Med. 1992 Aug;67(8):487–494. doi: 10.1097/00001888-199208000-00001. [DOI] [PubMed] [Google Scholar]
  20. Roberts J., Norman G. Reliability and learning from the objective structured clinical examination. Med Educ. 1990 May;24(3):219–223. doi: 10.1111/j.1365-2923.1990.tb00004.x. [DOI] [PubMed] [Google Scholar]
  21. Sloan D. A., Donnelly M. B., Johnson S. B., Schwartz R. W., Strodel W. E. Assessing surgical residents' and medical students' interpersonal skills. J Surg Res. 1994 Nov;57(5):613–618. doi: 10.1006/jsre.1994.1190. [DOI] [PubMed] [Google Scholar]
  22. Sloan D. A., Donnelly M. B., Johnson S. B., Schwartz R. W., Strodel W. E. Use of an Objective Structured Clinical Examination (OSCE) to measure improvement in clinical competence during the surgical internship. Surgery. 1993 Aug;114(2):343–351. [PubMed] [Google Scholar]
  23. Sloan D. A., Donnelly M. B., Schwartz R. W., Munch L. C., Wells M. D., Johnson S. B., Strodel W. E. Assessing medical students' and surgery residents' clinical competence in problem solving in surgical oncology. Ann Surg Oncol. 1994 May;1(3):204–212. doi: 10.1007/BF02303525. [DOI] [PubMed] [Google Scholar]
  24. Stillman P. L., Swanson D. B., Smee S., Stillman A. E., Ebert T. H., Emmel V. S., Caslowitz J., Greene H. L., Hamolsky M., Hatem C. Assessing clinical skills of residents with standardized patients. Ann Intern Med. 1986 Nov;105(5):762–771. doi: 10.7326/0003-4819-105-5-762. [DOI] [PubMed] [Google Scholar]
  25. Wray N. P., Friedland J. A. Detection and correction of house staff error in physical diagnosis. JAMA. 1983 Feb 25;249(8):1035–1037. [PubMed] [Google Scholar]

Articles from Annals of Surgery are provided here courtesy of Lippincott, Williams, and Wilkins

RESOURCES