Skip to main content
NCBI home page
Quality & Safety in Health Care logoLink to Quality & Safety in Health Care
. 2003 Dec;12(6):416–420. doi: 10.1136/qhc.12.6.416

A pilot study in ophthalmology of inter-rater reliability in classifying diagnostic errors: an underinvestigated area of medical error

C Margo 1
PMCID: PMC1758028  PMID: 14645756

Abstract

Background: Misdiagnosis is the least studied form of medical error. Before effective strategies to reduce misdiagnosis can be developed, there needs to be a better understanding of the factors that lead to these errors.

Aim: To evaluate the applicability and reliability of three classification systems for misdiagnosis.

Design: Retrospective independent analysis of five cases by clinical experts.

Participants: Three ophthalmologists trained in ocular oncology who devote at least 75% of their practice to ocular oncology.

Main outcome measures: Percentage agreement in determining cause of misdiagnosis.

Results: Participants agreed a misdiagnosis occurred in all cases and the error was graded as serious 14 of 15 times (93%). Inter-rater agreement for root cause varied among the three classification systems from 47% to zero.

Conclusions: Although there was excellent agreement among clinical experts of what constitutes serious misdiagnosis under idealized conditions, there is not a reliable method for categorizing the primary or root cause for these errors. The origins of misdiagnosis are complex, often multifactorial, and more difficult to categorize than other types of medical error. Misdiagnosis is a professional and public healthcare challenge that will require novel strategies to enable it to be successfully studied.

Full Text

The Full Text of this article is available as a PDF (118.8 KB).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Anderson R. E., Hill R. B., Key C. R. The sensitivity and specificity of clinical diagnostics during five decades. Toward an understanding of necessary fallibility. JAMA. 1989 Mar 17;261(11):1610–1617. [PubMed] [Google Scholar]
  2. Bartlett E. E. Physicians' cognitive errors and their liability consequences. J Healthc Risk Manag. 1998 Fall;18(4):62–69. doi: 10.1002/jhrm.5600180407. [DOI] [PubMed] [Google Scholar]
  3. Battles J. B., Kaplan H. S., Van der Schaaf T. W., Shea C. E. The attributes of medical event-reporting systems: experience with a prototype medical event-reporting system for transfusion medicine. Arch Pathol Lab Med. 1998 Mar;122(3):231–238. [PubMed] [Google Scholar]
  4. Battles J. B., Shea C. E. A system of analyzing medical errors to improve GME curricula and programs. Acad Med. 2001 Feb;76(2):125–133. doi: 10.1097/00001888-200102000-00008. [DOI] [PubMed] [Google Scholar]
  5. Bayer-Garner Ilene B., M Fink Louis, Lamps Laura W. Pathologists in a teaching institution assess the value of the autopsy. Arch Pathol Lab Med. 2002 Apr;126(4):442–447. doi: 10.5858/2002-126-0442-PIATIA. [DOI] [PubMed] [Google Scholar]
  6. Benson W. E., Regillo C. D., Vander J. F., Tasman W., Smith A. F., Brown G. C., Ho A. C., McNamara J. A. Patient-initiated second medical opinions: their necessity and economic cost. Retina. 2001;21(6):633–638. doi: 10.1097/00006982-200112000-00011. [DOI] [PubMed] [Google Scholar]
  7. Brennan T. A. The Institute of Medicine report on medical errors--could it do harm? N Engl J Med. 2000 Apr 13;342(15):1123–1125. doi: 10.1056/NEJM200004133421510. [DOI] [PubMed] [Google Scholar]
  8. Burwen D. R., Margo C. E., McNeil M. M., Brown J. M., Tapelband G., Jenkins R. B., Jarvis W. R. A pseudoepidemic of postoperative scleritis due to misdiagnosis. Infect Control Hosp Epidemiol. 1999 Aug;20(8):539–542. doi: 10.1086/501666. [DOI] [PubMed] [Google Scholar]
  9. Epstein Ronald M., Hundert Edward M. Defining and assessing professional competence. JAMA. 2002 Jan 9;287(2):226–235. doi: 10.1001/jama.287.2.226. [DOI] [PubMed] [Google Scholar]
  10. Forkner-Dunn D. J. Commentary: to err is human--but not in health care. Am J Med Qual. 2000 Nov-Dec;15(6):263–266. [PubMed] [Google Scholar]
  11. Friedman S. M., Margo C. E. Choroidal neovascular membranes: reproducibility of angiographic interpretation. Am J Ophthalmol. 2000 Dec;130(6):839–841. doi: 10.1016/s0002-9394(00)00605-x. [DOI] [PubMed] [Google Scholar]
  12. Gorter S., van der Heijde D. M. F. M., van der Linden S., Houben H., Rethans J. J., Scherpbier A. J. J. A., van der Vleuten C. P. M. Psoriatic arthritis: performance of rheumatologists in daily practice. Ann Rheum Dis. 2002 Mar;61(3):219–224. doi: 10.1136/ard.61.3.219. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Hayward R. A., Hofer T. P. Estimating hospital deaths due to medical errors: preventability is in the eye of the reviewer. JAMA. 2001 Jul 25;286(4):415–420. doi: 10.1001/jama.286.4.415. [DOI] [PubMed] [Google Scholar]
  14. Hofer T. P., Kerr E. A., Hayward R. A. What is an error? Eff Clin Pract. 2000 Nov-Dec;3(6):261–269. [PubMed] [Google Scholar]
  15. Kassirer J. P., Kopelman R. I. Cognitive errors in diagnosis: instantiation, classification, and consequences. Am J Med. 1989 Apr;86(4):433–441. doi: 10.1016/0002-9343(89)90342-2. [DOI] [PubMed] [Google Scholar]
  16. Layde Peter M., Cortes Leslie M., Teret Stephen P., Brasel Karen J., Kuhn Evelyn M., Mercy James A., Hargarten Stephen W., Maas Leslie A. Patient safety efforts should focus on medical injuries. JAMA. 2002 Apr 17;287(15):1993–1997. doi: 10.1001/jama.287.15.1993. [DOI] [PubMed] [Google Scholar]
  17. Leape L. L. Error in medicine. JAMA. 1994 Dec 21;272(23):1851–1857. [PubMed] [Google Scholar]
  18. Leape L. L. Institute of Medicine medical error figures are not exaggerated. JAMA. 2000 Jul 5;284(1):95–97. doi: 10.1001/jama.284.1.95. [DOI] [PubMed] [Google Scholar]
  19. Lester H., Tritter J. Q. Medical error: a discussion of the medical construction of error and suggestions for reforms of medical education to decrease error. Med Educ. 2001 Sep;35(9):855–861. doi: 10.1046/j.1365-2923.2001.01003.x. [DOI] [PubMed] [Google Scholar]
  20. Margo C. E. The accuracy of diagnosis of posterior uveal melanoma. Arch Ophthalmol. 1997 Mar;115(3):432–434. doi: 10.1001/archopht.1997.01100150434028. [DOI] [PubMed] [Google Scholar]
  21. McNutt Robert A., Abrams Richard, Arons David C., Patient Safety Committee Patient safety efforts should focus on medical errors. JAMA. 2002 Apr 17;287(15):1997–2001. doi: 10.1001/jama.287.15.1997. [DOI] [PubMed] [Google Scholar]
  22. Shojania Kaveh G., Burton Elizabeth C., McDonald Kathryn M., Goldman Lee. Changes in rates of autopsy-detected diagnostic errors over time: a systematic review. JAMA. 2003 Jun 4;289(21):2849–2856. doi: 10.1001/jama.289.21.2849. [DOI] [PubMed] [Google Scholar]
  23. Weingart S. N., Wilson R. M., Gibberd R. W., Harrison B. Epidemiology of medical error. BMJ. 2000 Mar 18;320(7237):774–777. doi: 10.1136/bmj.320.7237.774. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Quality & safety in health care are provided here courtesy of BMJ Publishing Group

RESOURCES