Abstract
OBJECTIVE: To evaluate the performance of tools for authoring patient database queries. DESIGN: Query by Review, a tool that exploits the training that users have undergone to master a result review system, was compared with AccessMed, a vocabulary browser that supports lexical matching and the traversal of hierarchical and semantic links. Seven subjects (Medical Logic Module authors) were asked to use both tools to gather the vocabulary terms necessary to perform each of eight laboratory queries. MEASUREMENTS: The proportion of queries that were correct; intersubject agreement. RESULTS: Query by Review had better performance than AccessMed (38% correct queries versus 18%, p = 0.002), but both figures were low. Poor intersubject agreement (28% for Query by Review and 21% for AccessMed) corroborated the relatively low performance. Subjects appeared to have trouble distinguishing laboratory tests from laboratory batteries, picking terms relevant to the particular data type required, and using classes in the vocabulary's hierarchy. CONCLUSION: Query by Review, with its more constrained user interface, performed somewhat better than AccessMed, a more general tool. Neither tool achieved adequate performance, however, which points to the difficulty of formulating a query for a clinical database and the need for further work.
Full Text
The Full Text of this article is available as a PDF (1.4 MB).
Selected References
These references are in PubMed. This may not be the complete list of references from this article.
- Barrows R. C., Jr, Cimino J. J., Clayton P. D. Mapping clinically useful terminology to a controlled medical vocabulary. Proc Annu Symp Comput Appl Med Care. 1994:211–215. [PMC free article] [PubMed] [Google Scholar]
- Bell D. S., Pattison-Gordon E., Greenes R. A. Experiments in concept modeling for radiographic image reports. J Am Med Inform Assoc. 1994 May-Jun;1(3):249–262. doi: 10.1136/jamia.1994.95236156. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Campbell K. E., Das A. K., Musen M. A. A logical foundation for representation of clinical data. J Am Med Inform Assoc. 1994 May-Jun;1(3):218–232. doi: 10.1136/jamia.1994.95236154. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cimino J. J., Clayton P. D., Hripcsak G., Johnson S. B. Knowledge-based approaches to the maintenance of a large controlled medical terminology. J Am Med Inform Assoc. 1994 Jan-Feb;1(1):35–50. doi: 10.1136/jamia.1994.95236135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cimino J. J. Use of the Unified Medical Language System in patient care at the Columbia-Presbyterian Medical Center. Methods Inf Med. 1995 Mar;34(1-2):158–164. [PubMed] [Google Scholar]
- Clancey W. J. The learning process in the epistemology of medical information. Methods Inf Med. 1995 Mar;34(1-2):122–130. [PubMed] [Google Scholar]
- Evans D. A., Cimino J. J., Hersh W. R., Huff S. M., Bell D. S. Toward a medical-concept representation language. The Canon Group. J Am Med Inform Assoc. 1994 May-Jun;1(3):207–217. doi: 10.1136/jamia.1994.95236153. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Friedman C., Cimino J. J., Johnson S. B. A schema for representing medical language applied to clinical radiology. J Am Med Inform Assoc. 1994 May-Jun;1(3):233–248. doi: 10.1136/jamia.1994.95236155. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gouveia-Oliveira A., Salgado N. C., Azevedo A. P., Lopes L., Raposo V. D., Almeida I., de Melo F. G. A unified approach to the design of clinical reporting systems. Methods Inf Med. 1994 Dec;33(5):479–487. [PubMed] [Google Scholar]
- Henry S. B., Holzemer W. L., Reilly C. A., Campbell K. E. Terms used by nurses to describe patient problems: can SNOMED III represent nursing concepts in the patient record? J Am Med Inform Assoc. 1994 Jan-Feb;1(1):61–74. doi: 10.1136/jamia.1994.95236137. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hripcsak G., Ludemann P., Pryor T. A., Wigertz O. B., Clayton P. D. Rationale for the Arden Syntax. Comput Biomed Res. 1994 Aug;27(4):291–324. doi: 10.1006/cbmr.1994.1023. [DOI] [PubMed] [Google Scholar]
- Huff S. M., Rocha R. A., Bray B. E., Warner H. R., Haug P. J. An event model of medical information representation. J Am Med Inform Assoc. 1995 Mar-Apr;2(2):116–134. doi: 10.1136/jamia.1995.95261905. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnson S. B., Hripcsak G., Chen J., Clayton P. Accessing the Columbia Clinical Repository. Proc Annu Symp Comput Appl Med Care. 1994:281–285. [PMC free article] [PubMed] [Google Scholar]
- McCray A. T., Nelson S. J. The representation of meaning in the UMLS. Methods Inf Med. 1995 Mar;34(1-2):193–201. [PubMed] [Google Scholar]
- O'Neil M., Payne C., Read J. Read Codes Version 3: a user led terminology. Methods Inf Med. 1995 Mar;34(1-2):187–192. [PubMed] [Google Scholar]
- Patel V. L., Arocha J. F. Cognitive models of clinical reasoning and conceptual representation. Methods Inf Med. 1995 Mar;34(1-2):47–56. [PubMed] [Google Scholar]
- Rothwell D. J. SNOMED-based knowledge representation. Methods Inf Med. 1995 Mar;34(1-2):209–213. [PubMed] [Google Scholar]
- Safran C., Porter D., Lightfoot J., Rury C. D., Underhill L. H., Bleich H. L., Slack W. V. ClinQuery: a system for online searching of data in a teaching hospital. Ann Intern Med. 1989 Nov 1;111(9):751–756. doi: 10.7326/0003-4819-111-9-751. [DOI] [PubMed] [Google Scholar]
- Safran C. Searching for answers on a clinical information system. Methods Inf Med. 1995 Mar;34(1-2):79–84. [PubMed] [Google Scholar]
- Sengupta S., Clayton P. D., Molholt P., Sideli R. V., Cimino J. J., Hripcsak G., Johnson S. B., Allen B., McCormack M., Hill C. IAIMS and sharing. Int J Biomed Comput. 1994 Jan;34(1-4):339–348. doi: 10.1016/0020-7101(94)90034-5. [DOI] [PubMed] [Google Scholar]
- Stead W. W., Hammond W. E. Computer-based medical records: the centerpiece of TMR. MD Comput. 1988 Sep-Oct;5(5):48–62. [PubMed] [Google Scholar]
- Tuttle M. S., Cole W. G., Sheretz D. D., Nelson S. J. Navigating to knowledge. Methods Inf Med. 1995 Mar;34(1-2):214–231. [PubMed] [Google Scholar]
- van Mulligen E. M., Timmers T., van Bemmel J. H. A new architecture for integration of heterogeneous software components. Methods Inf Med. 1993 Aug;32(4):292–301. [PubMed] [Google Scholar]
- van Mulligen E. M., Timmers T., van Bemmel J. H. User evaluation of an integrated medical workstation for clinical data analysis. Methods Inf Med. 1993 Nov;32(5):365–372. [PubMed] [Google Scholar]