Abstract
OBJECTIVES: To assess the effects of incomplete data upon the output of a computerized diagnostic decision support system (DSS), to assess the effects of using the system upon the diagnostic opinions of users, and to explore if these effects vary as a function of clinical experience. DESIGN: Experimental pilot study. Four clusters of nine cases each were constructed and equated for case difficulty. Definitive findings were omitted from the case abstracts. Subjects were randomly assigned to one of four clusters and were trained on the DSS prior to use. SUBJECTS: The study involved 16 physicians at three levels of clinical experience (six general internists, four residents in internal medicine, and six fourth-year medical students), from three academic medical centers. PROCEDURE: Each subject worked up nine cases, first without and then with ILIAD consultation. They were asked to offer up to six potential diagnoses and to list up to three steps that should be the next items in the diagnostic workup. Effects of DSS consultation were measured by changes in the position of the correct diagnosis in the lists of differential diagnoses, pre- and post-consultation. RESULTS: The DSS lists of diagnostic possibilities contained the correct diagnosis in 38% of cases, about midway between the levels of accuracy of residents and attending general internists. In over 70% of cases, the DSS output had no effect on the position of the correct diagnosis in the subjects' lists. The system's diagnostic accuracy was unaffected by the clinical experience of the users.
Full Text
The Full Text of this article is available as a PDF (836.4 KB).
Selected References
These references are in PubMed. This may not be the complete list of references from this article.
- Bankowitz R. A., Lave J. R., McNeil M. A. A method for assessing the impact of a computer-based decision support system on health care outcomes. Methods Inf Med. 1992 Feb;31(1):3–10. [PubMed] [Google Scholar]
- Bankowitz R. A., McNeil M. A., Challinor S. M., Miller R. A. Effect of a computer-assisted general medicine diagnostic consultation service on housestaff diagnostic strategy. Methods Inf Med. 1989 Nov;28(4):352–356. [PubMed] [Google Scholar]
- Bankowitz R. A., McNeil M. A., Challinor S. M., Parker R. C., Kapoor W. N., Miller R. A. A computer-assisted medical diagnostic consultation service. Implementation and prospective evaluation of a prototype. Ann Intern Med. 1989 May 15;110(10):824–832. doi: 10.7326/0003-4819-110-10-824. [DOI] [PubMed] [Google Scholar]
- Berner E. S., Jackson J. R., Algina J. Relationships among performance scores of four diagnostic decision support systems. J Am Med Inform Assoc. 1996 May-Jun;3(3):208–215. doi: 10.1136/jamia.1996.96310634. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Berner E. S., Webster G. D., Shugerman A. A., Jackson J. R., Algina J., Baker A. L., Ball E. V., Cobbs C. G., Dennis V. W., Frenkel E. P. Performance of four computer-based diagnostic systems. N Engl J Med. 1994 Jun 23;330(25):1792–1796. doi: 10.1056/NEJM199406233302506. [DOI] [PubMed] [Google Scholar]
- Johnston M. E., Langton K. B., Haynes R. B., Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome. A critical appraisal of research. Ann Intern Med. 1994 Jan 15;120(2):135–142. doi: 10.7326/0003-4819-120-2-199401150-00007. [DOI] [PubMed] [Google Scholar]
- Kassirer J. P. A report card on computer-assisted diagnosis--the grade: C. N Engl J Med. 1994 Jun 23;330(25):1824–1825. doi: 10.1056/NEJM199406233302512. [DOI] [PubMed] [Google Scholar]
- Miller R. A., Masarie F. E., Jr The demise of the "Greek Oracle" model for medical diagnostic systems. Methods Inf Med. 1990 Jan;29(1):1–2. [PubMed] [Google Scholar]
- Miller R. A. Medical diagnostic decision support systems--past, present, and future: a threaded bibliography and brief commentary. J Am Med Inform Assoc. 1994 Jan-Feb;1(1):8–27. doi: 10.1136/jamia.1994.95236141. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miller R. A., Pople H. E., Jr, Myers J. D. Internist-1, an experimental computer-based diagnostic consultant for general internal medicine. N Engl J Med. 1982 Aug 19;307(8):468–476. doi: 10.1056/NEJM198208193070803. [DOI] [PubMed] [Google Scholar]