The Classification of Clinicians’ Information Needs While Using a Clinical Information System

Abstract

Introduction

Information needs are prevalent in clinical practice. They represent a potential source of medical errors. This study seeks to empirically determine the information needs of clinicians while using a clinical information system (CIS), and characterize those needs. In addition this paper will provide the framework necessary for the development of the solutions to these information needs.

Methods

Clinicians were observed while using a CIS. They were recorded on audiotape and the computer screen recorded on videotape. The types of question during these interactions were recorded. A classification of the questions provided the conceptual and architectural basis for the development of context-sensitive links to information resources, called infobuttons.

Results

There were 154 information needs. The questions were grouped into seven categories. Within these categories we were able to identify eleven specific repeated question patterns, accounting for 72 or 47% of users’ questions.

Discussion

These findings are applicable to a number of settings and can be generalized to other institutions. The proposed infobuttons based on six categories, will be navigational (‘how-to...’ links), cross-reference (‘what is the…’ links), domain knowledge buttons in the areas of laboratory, pharmacy, diagnosis, and definitions/general information. Using these groups we were able to identify eleven patterns of questions.

INTRODUCTION

The Institute of Medicine’s seminal report “To Err is Human” sensitized the health care profession to the magnitude of medical errors.¹ The IOM’s report in part stated: “it (technology) can enhance human performance to the extent that the human plus technology is more powerful than either is alone.” Investigators have long been interested in physician information needs and in the provision of appropriate solutions, including technological ones.^2,3,4,5

Providing information, in a timely manner, to physicians (and indeed other health care providers) is essential for the decision making process. ^6,7 Whenever clinicians encounter data they often require additional information in order to formulate a decision. It has been estimated that in the inpatient setting there are 5 questions per patient discussed. ⁸ Potentially, if these information needs are unanswered at the time a physician is making a clinical decision errors may occur.

A central theme in medicine is how to take the information available to the clinician (memory, text, journals, and colleagues) and apply that information to a specific patient. Now more than ever there is a need for technological solutions (information systems) that seamlessly supplement the information requirements of the health care provider in a timely manner.

With the increased use of clinical information systems (CISs) in clinical practice it is interesting to determine the information needs at that time while the clinician is interacting with the system and how he or she resolves any information need that he may have. It is expected that while using a computer to obtain new data the clinician will have unmet information needs and potentially these needs would be soluble using the same computer. The study of clinician’s information needs in the context of the use of a CIS to our knowledge has not been done before.

We are seeking to develop a set of context-specific functions, called infobuttons (IBs) to anticipate and address the information needs that occur when clinicians use a CIS. ⁹ In order to do so, we are taking an empirical approach to determining what information needs arise, in a given context, based on factors such as user traits, patient traits, CIS task being performed, and specific data that may be triggering the need. One study is using a survey instrument, together with analyses of system log files. ¹⁰ In a parallel study, presented here, we are using direct observation to identify information needs.

This study has entailed the collection of video and audio data from clinicians using the CIS in the normal course of their day. ⁹ We have developed a coding scheme for characterizing the needs that we observe ¹¹ and applied the scheme to observational data. In the present study, we examine the information needs thus identified and attempt to characterize them with respect to the types of IBs that will be needed to address them.

METHODS

The acquisition of users’ information needs while using the CIS (WebCIS)¹² has been carried out with a portable usability laboratory (PUL),¹³ which recorded the interaction on videotape and audiocassettes. The videotape captured the computer screen as the users worked, while the audiocassettes recorded the users’ vocalization of what they were doing and verbalization of what other information they needed at that time to complete their task. A number of health care professionals were recorded in a variety of settings that included the medical-surgical wards, the coronary care unit, and the ambulatory clinics.

Two investigators using the transcribed audiocassettes and videotapes extracted the requisite data, including the users’ demographic data, and the instances of information needs expressed by the users. An information need event was said to have occurred if the user expressed (implicitly or explicitly) the need for additional information to formulate a clinical decision. Subsequently, a third investigator repeated the process, identifying instances of information needs. These results were compared with the initial data to determine the validity of the collected data. The results of this work served as input data for the current project and are described elsewhere, along with a more extensive description of the observational techniques used in this study. ¹⁴

To provide the conceptual and architectural framework for the design for the different types of IB that will eventually be constructed, we characterized the information needs that had been identified in the previous phase of our study. The first step was to review the tapes and at each instance of an information need, identify the context and determine as precisely as we could the specific question the user was asking and the specific information the user was trying to obtain.

Next, the questions were reviewed and a determination of the ‘generic’ question was ascertained for each. A generic question can be thought of as the simplest representation of the information need, as shown below. The generic questions were then examined for commonality of information need; we then speculated on how each would be resolved by some hypothetical IB. Within each category we attempted to identify those questions that occur most frequently.

RESULTS

The prior study produced a total of 15 hours 29 minutes of tape recording. Of the seventy-nine users, thirty-four generated 154 information needs. The work by Currie et al elaborates on the characteristics of clinicians studied. Our analysis of the 154 information needs yielded a scheme consisting of seven classifications into which the questions where placed (Table 1). Table 2 shows examples of these questions and the ways in which they were rendered as generic questions.

Table 1.

Classification of Information Needs

	Classification	Definition	IB Function
Need for resources internal to WebCIS	Navigation	A question about any function in WebCIS (either currently present or could be added at a later date) that that the user is unable to access or use.	The IB link would take the user to a resource that tells him how to access the needed information.
	Cross-Reference	A response or question, which alludes to the need to add an item to WebCIS or give the user access to another resource, which answers the need.	The IB would access the resource that would solve the problem and present the solution to the user.
Need for resources external to WebCIS: e.g. Harrison’s, Up-to-Date, Epocrates, dictionaries etc	Laboratory	Information needs that are related to laboratory domain knowledge.	The IB would provide information about a specific laboratory finding.
	Pharmacy	Information needs that are related to the pharmacy domain knowledge.	The IB would provide information about specific medications.
	Differential Diagnosis	Questions related to alternative causes of a symptom or laboratory or radiology finding.	IB would provide context-specific differential diagnoses.
	Definition/Information	Questions related to the meaning of words. In addition, general information on a topic.	IB would provide a definition or general information about a topic/word.
	Miscellany	Those needs that do not readily translate into patient/institution needs.

Table 2.

Examples of User Question Types, Conversion to Generic Questions, and Classification.

User’s Question	Generic Question	Classification
What is the normal value for this calcium result?	What is the normal value of this <laboratory test>?	Navigation: because normal laboratory values are available in WebCIS, an IB link would take the user to another page informing the users on how to access the information needed.
Since the CT scan of the liver is abnormal what were his liver function test results?	What <laboratory tests> are associated with this particular <radiology reports>?	Cross-reference: because the IB would need to bring information to the user from another part of WebCIS (or some other resource) that cannot normally be reached by simple navigation.
What is this patient’s calculated anion gap?	What is the calculated <anion gap>?	Laboratory
Is there a metered dose preparation of metaproterenol?	What are the different formulations of <medication>?	Pharmacy
The causes of coffee-ground emesis are?	What is the differential diagnosis of <symptom/sign/test result>?	Differential Diagnosis
What does malacoplakia mean?	What does <term> mean?	Definition/Information need

Based upon the classification scheme, we were able to identify different types of information needs; within in each category, we identified recurring patterns of generic questions, as shown in (Table 3). Though each group consisted of a heterogeneous mix we were able to capture eleven types of questions, as shown in Table 4.

Table 3.

Classification of 154 Clinicians’ Information Needs

Classification	Question Types	Generic Questions	Number Fitting Pattern
Navigation	48	3	20
Cross-Reference	27	3	22
Laboratory	19	2	7
Pharmacy	31	1	8
Differential	9	1	8
Diagnosis
Definition/Information	15	1	7
Miscellany	5	0	0
Total	154	11	72

Table 4.

Recurring Generic Questions found in User Questions

Classification	Questions	Generic Question	Description
Navigation	9	What does <term, number, abbreviation> in WebCIS mean?	Abbreviations and WebCIS terms
	6	How do I find the <normal value/trend> for <test>?	Understanding laboratory results
	5	What is the <telephone number/beeper number> for <attending/resident>?	On-line phone directory
Cross Reference	8	What other <tests> are important to know when ordering/reviewing this <medication, report> on this patient?	Correlation of tests, radiology reports, and medications
	5	What patient educational information is available about this <test/disease>?
	9	What inpatient or outpatient medication is this patient on or has been given?
Laboratory	5	How do you calculate <finding> for this patient with these results?	Calculation
	2	How do you perform a <procedure> to obtain this <laboratory test>?	Associated procedures
Pharmacy	8	Is this the correct preparation of <medication> for this patient?	Dosage knowledge and patient orders
Differential Diagnosis	8	What is the differential diagnosis of <sign/symptom/laboratory test/report finding> in this patient?	Differential diagnosis of patient data
Definition/Information	7	What is the definition of this <term>?	Definition of term in report

Figure 1 shows the distribution of successes vs. non-success of attempts to resolve information needs in this study. A success was recorded if the user found a solution to his question whether or not he used WebCIS; a non-success was recorded if there was not a resolution of the information need or the solution to the information need was deferred. As expected, navigation needs tended to be successful; however, 56% of cross-reference needs were not successful, and reviewing the diagram shows a similar tendency for laboratory and differential diagnosis needs.

Figure 1.

Rates of Successful Resolution of Information Needs , by Classification

We also determined whether or not the user was using a computer at the time of the information need. The use of the computer was more prevalent in the laboratory (78%), and definition/info (53%) categories. In the cross-reference category, computers were used 50% of the time. If the users were not using a computer, they were speaking with a colleague or using a paper record at the time of the information need.

DISCUSSION

To our knowledge, this is the first study that categorizes empirical data from the observation of health care providers using a CIS, determining their information needs, and categorizing those needs in to generic question-types. The empiric study of human performance in a number of situations facilitates the creation of systems that are more responsive to the users needs and result in the reduction of conditions that lead to error. Smith’s paper identified and reviewed 13 studies that described the information needs of clinicians.¹⁵ Significantly of the 13 studies identified only three were observational. It is our expectation that the use of an observational study will provide accurate information about clinicians’ needs.

Our study found that there were 154 information needs, from which we could discern eleven patterns that accounted for 72 (47%) of the clinician’s information needs. These patterns will form the basis for our construction of the IBs.

One limitation of our study was that it was performed at one institution. However, we attempted to minimize this effect by using a number of different sites and users within the institution. We believe that the generic information needs we are finding are likely to occur to users of CISs at other institutions.

Additional study is likely to uncover additional information needs. We attempted to obtain a representative population of users to obtain a representative sample of questions. The occurrence of repeating question patterns encourages us that corresponding IBs will address at least a portion of our users’ information needs.

Our study showed that while using a CIS, many of the needs were navigational or ‘how-to’ questions. The IB will provide a link to a Web page that will provide information on how to access information in WebCIS.

In contrast, the cross-reference questions will present more of a development challenge. These questions were ‘What is/are …?’ type questions. They require the cross-linking of information, often in a context-specific manner. We know from our study that people tended to look at laboratory tests if they see an abnormal radiograph; therefore, a sophisticated IB would provide the result only if the report is abnormal.

Solutions to the differential diagnosis questions may come from the use of clinical decision support systems such as DXplain or QMR. ^16,17 In this instance, the IB would pass to the clinical decision support system the context: age, gender, result, symptoms, and signs.

Determining definitions of terms in reports is feasible. The issue with building this IB would be to devise a method to determine which term or combination of terms in a report need to be defined.

The results of this study help us to anticipate the general types of information needs of users of a CIS. Different CISs models will result in different information needs, however we expect these needs will conform to the general information types outlined in this study. For example, users seem to need access to context specific domain knowledge; in addition, they require the ability to link data located in disparate sections of a CIS. Incorporating this knowledge in the design and building of information systems will result in CISs that are more intuitively responsive to the user.

The next stage of the project will be to validate our methods by developing the individual IBs using the classification scheme we devised and the generic questions we developed as a guide. To assess our success (or failure) we will have to keep log file records of the frequency of the IBs’ use and repeat the observational study to determine if these buttons are providing solutions to information needs as we expect.

CONCLUSIONS

Web-based CISs offer powerful opportunities to provide clinicians with diverse information in a timely manner. It is presumed that the availability of information to clinicians at the time they need it will improve patient care, improve clinical outcomes and reduce medical errors. We have shown that information needs while using a CIS can be characterized and formulated to provide the basis of a technological solution.