Characterizing Information Needs and Cognitive Processes During CIS Use

INTRODUCTION

Medical errors are often associated with inadequate cognitive processing that is based upon impaired access to information.¹ Understanding the information needs of nurses and physicians’ when using a clinical information system (CIS) is difficult largely because there are few systematic attempts made to do so. We collected 15.5 hours of data of nurses and physicians’ CIS interactions in three clinical environments (cardiac ICU, a general medical/surgical nursing, and an ambulatory clinic). We accomplished this through observational, cognitive-based methods (e.g., thinking aloud during interaction) and the video capturing of events with a portable usability laboratory.² We developed a systematic approach to identify and define in-context clinical information needs while using a CIS, and determine how to categorize and code such events. ³

METHODS

The event capturing and subsequent coding methodologies were developed as follows. First, we installed the usability laboratory in three clinical settings. Next we obtained informed consent and videotaped the nurses’ and physicians’ interactions with the CIS while the participants were thinking aloud about the task. The data was analyzed using a systematic coding schema that was iteratively developed for this study. The schema was tested for reliability by three of the authors (LC, MG, JC) who independently coded the events. A subset of 30 events was initially coded and disagreement was resolved through discussion. Inter-rater reliability for the subset was 93.3%. When the remaining events were coded, reliability increased to 97%.

RESULTS

Through our coding schema we attempted to identify the information in the following categories: 1) Was it related to the subject (patient), to the infrastructure of the institution, to the domain in which the clinician was working or to the intersection of these concepts ? [(S) or (D) or (I) or (SD) or (SI) or (DI) or (SDI)]; 2) Was it a root question (who, what, etc.) related to a health disorder or did it require specific information about patient management? [(B)ackground or (F)oreground]; 3) Was it expressed as an explicit information need or was it inferred implicitly from the participant’s actions? [(E) or (I)]; 4) Was an answer obtained, delayed or not sought, or sought yet not found? [(S)uccess, (D)eferred or (F)ailure]; 5) Was it from a computer-based resource, a human, or from a paper source? [(C), (H), or (P)].

To illustrate the method, an example from a resident working on patient planning in Oncology is given below: The resident, who was unfamiliar with the term malacoplakia, stated: "Mala-Mala... Malacoplakia...what is it?…what does '-plakia' stand for?” Almost simultaneously, another colleague looked it up on an adjacent comp uter–but was not successful. In terms of coding this event for the first resident, the information need was domain-specific (D) because they were searching for a domain related definition, a background question (B) because they needed to answer the question “What is…?”, explicit (E) because it was clearly stated, and failed (F) by a human (H) (i.e., the second resident).

Based on this coding methodology, subsequent work develops two additional classifications. The first addresses the context of the information need (Currie, et al, 2003 AMIA in press). For example, where in the CIS did the information need arise: in the laboratory report, in the discharge summary, or outside the CIS? The second addresses the classification of the information need (Allen, et al., 2003 AMIA in press). For example, was the information need that arose generically an issue of navigation (to an other resource) or of functionality (such as the necessity of building something new)?

CONCLUSIONS

Coding of events was iterative and needed reconsideration and revisiting each time the environmental context shifted, for example, from cardiac ICU to ambulatory clinic to achieve consistent inter-rater coding. Our methods captured these shifts in a systematic fashion yielding a rich set of meaningfully coded data that can be extended to include population-specific information needs. Potential Infobutton solutions will follow once the challenge of specific needs is addressed.