Impact of a Computer-based Patient Record System on Data Collection, Knowledge Organization, and Reasoning

Abstract

Objective: To assess the effects of a computer-based patient record system on human cognition. Computer-based patient record systems can be considered "cognitive artifacts," which shape the way in which health care workers obtain, organize, and reason with knowledge.

Design: Study 1 compared physicians' organization of clinical information in paper-based and computer-based patient records in a diabetes clinic. Study 2 extended the first study to include analysis of doctor–patient–computer interactions, which were recorded on video in their entirety. In Study 3, physicians' interactions with computer-based records were followed through interviews and automatic logging of cases entered in the computer-based patient record.

Results: Results indicate that exposure to the computer-based patient record was associated with changes in physicians' information gathering and reasoning strategies. Differences were found in the content and organization of information, with paper records having a narrative structure, while the computer-based records were organized into discrete items of information. The differences in knowledge organization had an effect on data gathering strategies, where the nature of doctor-patient dialogue was influenced by the structure of the computer-based patient record system.

Conclusion: Technology has a profound influence in shaping cognitive behavior, and the potential effects of cognition on technology design needs to be explored.

Emerging information technologies, such as the Internet, decision support systems, and computer-based patient records (CPRs), do not merely afford the possibility of enhanced performance but participate in an “intellectual partnership.” In this partnership, the human being and the computer are viewed as dynamically interacting, resulting in distributed performance. This interaction can be understood in terms of learning, involving the division of labor and the development of a subtle interdependence over time.

In considering the impact of this partnership, we can distinguish between “effects with” and “effects of” technology. Effects with refers to changes in intellectual performance while people learn and interact with technologies, whereas effects of refers to enduring changes resulting from human interaction with technology, even when people are away from machines.¹ The enduring effects can result in significant changes in performance.

Numerous cognitive and social challenges are involved in understanding and engineering an effective use of emerging technology in the workplace.² In recent years, cognitive science research has made progress in understanding learning processes and skill acquisition in complex technology-based domains. Advances in the use of information technology are rapidly changing the way we think, reason, make decisions, and interact with others. The CPR can be considered a tool that aids the mind and, as such, a “cognitive artifact.”³ Beyond merely extending human memory, these artifacts or tools affect human reasoning in ways that may be subtle, yet profound.

Issues related to the complex interaction among health care workers and emerging information technologies are rapidly coming to the fore in the field of medical informatics. To date, impressive information technologies have been developed in medicine, ranging from advanced decision support tools to CPR systems. As cognitive artifacts, such systems have the potential to greatly enhance and extend human capabilities by providing health care workers with access to the latest information and assistance in performing complex cognitive tasks, including medical diagnosis and treatment planning.

Although considerable effort has been expended in the development of these technologies, far less work has been devoted to examining their effects on the basic cognitive processes involved in health care. Numerous outcome-based evaluations in medical informatics have focused on assessing the effects of the introduction of information systems on predefined outcome variables,⁴ such as patient mortality and cost of health care. The effects of systems on complex decision processes and human knowledge organization has remained to be more fully explored.⁵ Previously, we have argued for the need for in-depth analysis that focuses on detailing the actual use of such systems by health care workers as they solve complex problems.⁶

Cognitive artifacts that directly interact with users to aid and provide advice in medical reasoning include decision support and expert systems.^7–8 However, an emerging class of information tools may have a more subtle yet equally important effect on user interaction and reasoning in health care. This class includes systems designed to facilitate electronic retrieval and access to information and provide links to other medical information systems. Evidence indicates that in professional domains, such as medicine, such tools may transform human cognition and activity in important, unexpected ways.⁹

The CPR system is one example of such a system.¹⁰ Computer-based patient record systems are designed to allow physicians to directly enter patient data, findings, and notes into a computer system that may be linked to hospital-wide databases and decision support systems. The objectives of implementing such systems include replacing hand-written paper-based records with CPRs to improve access to information and quality of health care decision making. However, use of these systems may also affect, in unanticipated ways, fundamental cognitive processes involved in health care.

In this paper we examine the effects of the use of an advanced CPR system on physician knowledge organization and reasoning. Although CPR systems are likely to cause changes in the nature of medical practice and are increasingly being used in medical practice, the deployment of such computer technologies has often proved more difficult than anticipated.^11–13 In some health care settings, the introduction of CPR systems has been fraught with difficulties, ranging from technical problems in integrating these systems with other information resources to fundamental problems with user interfaces.¹²

In our research we generally consider effects of technology at several levels, from examination of individual cognitive processes to analysis of systems, where effects are considered in the context of distributed cognition (e.g., the effects of the use of systems on decision making in clinical settings). Our research framework, which typically involves both laboratory and naturalistic study of systems, builds on models and conceptual frameworks for understanding human–computer interaction that consider both the individual computer user and distributed aspects of cognition.^2,14 We employ a variety of methods from cognitive science to characterize the skill, reasoning, and problems of subjects of varying levels of expertise as they learn to use and master information technologies in real-world domains.

Problems in Representing Knowledge Using Computerized Patient Records

A fundamental problem in medical cognition is the retrieval of information from memory. For information to be successfully retrieved, it must be organized to facilitate recovery. The complexity of the medical domain exacerbates this problem.

An important aspect of medical cognition is clinical problem solving and, more specifically, diagnosis generation. The focus of much research in medical cognition has fallen on diagnosis, principally because diagnosis is the basis for many medical decisions.¹⁵ Since a diagnosis is is based on the collection of information about the history, symptoms, and signs of a case, a critical component of actual clinical problem solving involves gathering and organizing this data in patient records.

One branch of research into the knowledge organization of CPRs has involved assessing the representation of patient problems using such systems.¹⁶ This line of work has focused on issues of knowledge organization and has considered the content of medical record as belonging to two distinct levels.¹⁷ According to this perspective, the medical record comprises observations and meta-observations. The direct observations consist of what was heard, seen, thought, and done concerning the patient, whereas the meta-observations are part of a higher conceptual level and are directly linked to problem representation (e.g., providing information about how decisions were made by the physician).

The concept of a clinical problem resides in the layer of meta-observations. The evolution of the definition of a problem, the grouping of observations pertinent to that problem, the recording of activity over time, and the subsumption of one problem by another are all meta-observations. Therefore, a problem can be related to a symptom, an abnormality, a treatment, or anything else that has been observed in the case. This is not because “problem” is a superordinate concept subsuming all other medical statements; rather, it is because problems are based on meta-observations about other medical statements.¹⁸ Other studies have indicated that CPRs allow for sufficient completeness and accuracy to facilitate the diagnostic process.¹⁹ However, some studies have shown that aspects of hand-written paper records (including certain types of perceptual and visual cues) that greatly enhance decision making may be lacking in CPRs.²⁰

Despite the potential advantages of CPRs over paper-based patient records and the effort that has been spent developing and deploying such systems, they are still not widely used in routine medical practice. One proposed reason for this is that system designers have not yet learned to represent computerized medical information in a form that seems intuitive to physicians.⁶ Researchers are beginning to explore how physicians adapt when moving from the traditional hand-written paper-based record to a CPR. For example, a recent study examined changes in the extent to which physicians were able to represent patient problems when using a CPR system.⁹ Extended use of the CPR system was found to result in changes in the order and type of patient information requested by physicians when interviewing patients.

This paper reports the results of research on a number of related questions: 1) How do physicians manage information flow when using a CPR system? 2) What are the differences in the way that physicians organize and represent this information using paper-based and CPR systems? 3) Are there long-term, enduring effects of the use of CPR systems on medical knowledge representation and diagnostic reasoning?

A combination of qualitative and quantitative analyses were employed, focused at the levels of both the individual physician's interaction with the system and the on physician–patient interaction. The aim was to characterize changes in cognitive strategies over time as physicians learn the use of this technology, and to identify differences in users' style of interaction with such systems.

The methods and results are discussed in three sections, corresponding to three interrelated studies that were conducted: 1) an initial study of the organization of patient records made before, during, and after exposure to a CPR, including investigation of the effects of a CPR system introduced into a clinic; 2) a study of the processes involved in use of a CPR system during physician–patient interviews; and 3) an analysis of users' perceptions and patterns of CPR use over time. The research design for all three studies was approved by the local institutional review board, participation was made anonymous, and informed consent was obtained from all participants.

Study 1: Knowledge Organization

Study 1 examined the effect of a CPR on the organization of medical knowledge, and involved two phases of research. The first phase consisted of an in-depth analysis of the organization of information in computer-based and paper-based records made by a single subject. This was followed in a second phase by analyses of the content of records made by a group of physicians who were introduced to the use of the CPR in a diabetes clinic.

Phase I: In-depth Analysis of Records Made by a Single Subject

The initial phase of Study 1 involved analysis of the patient records made by an endocrinologist from a metabolic day center at a Montreal hospital, who had expertise in the area of diagnosing and treating diabetes. The physician had been using a CPR system in his practice for more than six months, and his previous paper-based hand-written records were made accessible for comparison.

Previous research²¹ found that an initial in-depth case study involving a small number of subjects allows for detailed analysis of cognitive processes that can lead to identification of critical process variables to be tested in follow-up studies using larger sample sizes. It has been argued²² that such studies can provide insight at the level of theory development leading to generation of specific hypotheses and provide insight into refining research questions for further investigations.⁵ It has also been argued that medical tasks have certain invariant properties, such as diagnostic reasoning, that are amenable to in-depth analysis as a prelude to testing for their generalizability.

Method

The CPR system used in the study is flexible and interactive. It operates on a portable laptop or desktop computer running Microsoft Windows, and includes an electronic pen for data entry. At the core of the organization of information in the system is a medical knowledge base that incorporates industry- standard diagnostic codes, procedures, and medical terminology and contains approximately 100,000 elements. The system has several uses, including entering information about the main reason for consultation and the patient's social profile, family history, and medical history. It allows the physician to enter details about medical management (therapy, instructions, and referral).

The user interface allows the physician to use an electronic pen to point, draw, and write on a pad or directly on the computer screen. (Figure 1▶ shows a sample screen display for a patient record.) The main interface metaphor is the “paper scroll,” where the user can enter data using an electronic pen and can scroll through the record to a desired section or, alternatively, click on a menu bar on the side panel to go directly to that section.

Figure 1.

The main CPR screen display, showing part of a patient record.

The user can also search for a section of the system by entering key words. The data entry method is based on a point-and-click or pick- list technology, where the user clicks on categories on the screen for which values (e.g., blood pressure) can be entered, either by typing in the value, clicking to indicate the presence or absence of a finding (and the level of severity of symptoms), or using the pen to enter a value or note. In addition, the system incorporates software for handwriting recognition to allow for data entry by pen. The system can also display sections of the relevant clinical items to the user; for example, it will adjust the display of information slightly depending on the patient's particular complaint (the system has a number of “filters,” which are basically variations of a general template for the organization of medical findings, based on general medical conditions). Categories of information displayed on the screen can be opened or collapsed by clicking on the triangles in front of the categories (Figure 1▶).

In the CPR the information presented to users is structured in layers, with the first layer providing access to the top-level categories of information about a patient problem. The second layer of information, which is obtained by clicking on a main category heading (e.g., history of present illness, as shown in Figure 1▶), provides access to the rest of the information contained in the medical knowledge base (e.g., details about the patient's symptoms).

The structure of the CPR is based largely on the paper metaphor; that is, the designers of the system structured it according to categories of information they identified as being essential from their experience using traditional paper-based records.²³ During the initial development phase of the system, the designers of the CPR interviewed groups of physicians (who were users of paper records and did not have exposure to CPR systems) to develop a structure for the information to be contained in the CPR. System features, including provision of an electronic pen for data entry and for ordering of information by the system in a sequential linear fashion (allowing users to scroll or “page” through screens of information) all have analogs in the organization of the paper record. The specific categories of information displayed on the screen (and the type of information contained in each category) are as follows:

• Chief complaint, including patient's presenting symptoms

• Past medical history: patient's surgical, medical, obstetric, and psychiatric history and allergies

• Life style: patient's medication, immunization, and habits

• Psychosocial profile: patient's cultural origin, schooling and occupation, social environment, developmental history, and sexual history

• Family history, organized and accessible by disease (e.g., diabetes) or by person (e.g., father)

• History of present illness, organized by symptoms, systems, or disease

• Review of systems, organized by systems (e.g., cardiovascular)

• Physical examination, organized by systems (e.g., skin)

• Diagnoses, organized by category of disease

• Investigation, organized by type of investigation (e.g., hematology)

• Treatment, organized by types, including medications and procedures

The surface structure of the CPR records, as reflected in the top-level categories presented to the user and their order, is given in Table 1▶. Also indicated in the table, for comparison, is the structure and order of information recorded in the physician's hand-written paper-based patient records made prior to the introduction of the CPR. The general nature of information categories in the CPR and paper-based records is similar, but they are not exactly the same and their ordering differs. The structure of the paper-based records made by the subject in Study 1 was based on standards for the content and structure of patient records proposed by the Quebec Ministry of Health.

Table 1 .

Nature and Order of Appearance of Categories of Information in the Paper-based and Computer-based Patient Records

Paper-Based Patient Record	Computer-Based Patient Record
Current medications	Chief complaint
Family history	Personal history
Past history	lifestyle
Personal history	Psychosocial profile
Functional inquiry	Family history
Physical examination	History of present illness
Assessment	Review of systems
Recommendations	Physical examination
	Diagnoses
	Investigation
	Treatment

Procedure

Initially, 20 patient records created by the subject were collected for detailed analysis: ten CPRs and ten paper-based patient records. The computer and paper records were matched for age, sex, and nature of problem to create ten pairs of documents. All the paper records had been created by hand before the physician had any experience with the CPR system. The patients seen by the physician typically had some form of diabetes.

After using the CPR system to enter data about all patients encountered during a six month period, the physician was asked to proceed with his next patient interviews using only hand-written paper records. Our interest was in examining the residual effect of the CPR on knowledge organization by the physician. Five of these paper-based patient records were selected (matched with the previous paper-based and CPR records for type of medical case) and were analyzed to determine whether there were differences in the content and organization of these records, compared with the computer-based records and the previous paper-based records that were created prior to introduction of the CPR.

The information contained in each patient record was broken down into segments, representing individual units of information or concepts. Each segment typically corresponded to the recording of one medical finding, such as the patient's blood pressure or weight. For all records, the number of individual items of information (or segments) belonging to the following categories were identified: chief complaint, past medical history, life style, psychosocial profile, family history, history of present illness, review of systems, physical examination, diagnoses, investigation, and treatment. This involved the experimenter's breaking text and individual entries of information into segments and then classifying each segment. For example, the sentence “Seventy-four-year-old woman, who had a history of polyuria/nycturia and fatigue,”entered by the physician in one of the paper-based records, resulted in the identification of four segments, one item corresponding to the patient's age and three other items corresponding to the patient's history of polyuria, nycturia, and fatigue (classified as belonging to the category history of present illness).

Information recorded in the CPR was likewise identified and coded as belonging to the specific information categories, with one or more lines of text on the screen (as in Figure 1▶) corresponding to an individual unit or segment of information (e.g., “this patient has been suffering for two years”). The coding was conducted independently by two judges, who coded the records for content separately. The degree of agreement was high, and any discrepancies in coding were resolved through subsequent discussion and agreement of the raters.

In this way the text was analyzed by identifying the units that lie under its surface structure.^21,24 We have employed this type of analysis in numerous studies in which transcripts of verbal discourse and texts written by physicians have been analyzed for their content.¹⁵ The approach is particularly applicable to the analysis of patient records, in which individual units of information are readily identifiable as separate entries.

The results of this initial analysis (i.e., a list of individual items of information) were given to an expert physician, who was asked to read each segment of information and judge whether it was critical for making the diagnosis of the patient's primary disease. (The physician was blind to the purpose of the rating.) The percentage of “critical” and “noncritical” information, relative to the total number of segments contained in the records, was calculated. In addition, the number of diagnoses recorded in the paper-based and CPR records (as well as the number of specifications of diagnoses) was tabulated to examine the mean number of diagnoses generated by the physician. Analysis of variance (ANOVA) was used to determine whether significant differences exist in the type of information recorded in the paper-based and CPR records.

Results

Nature and Organization of Information.

The results indicated that the CPRs contain slightly more information critical to the primary diagnosis than did the paper-based records that were made before introduction of the CPR (92.4 percent of the information was critical using the CPR, and 87.2 percent was critical using the pre-CPR paper-based records). The amount of critical diagnostic information seen in the paper-based records that were created after the physician's exposure to the CPR contained the greatest proportion of critical information (98.7 percent).

Table 2▶ gives the means and standard deviations for occurrences of each category of medical information in the CPR and the paper-based records (made before and after CPR exposure). The mean number of information units (i.e., segments) recorded in the pre-CPR paper-based records (mean, 83), the CPR records (mean, 81) and the post-CPR paper-based records (mean, 68) were not significantly different. However, there were differences in the nature of information which was recorded. The CPR records contained significantly more items relating to past medical history than did either the pre-CPR or post-CPR paper-based records (F_2,24 = 6.94, P < 0.005), and lifestyle (F_2,24 = 7.33, P < 0.005). In contrast, the pre-CPR paper-based records contained more items of information about review of systems and psychosocial profile (F_2,24 = 11.79, P <0 .001) than the CPR records (F_2,24 = 4.30, P < 0.05).

Table 2 .

Number of Items of Information, by Category, Entered by a Physician in Ten Paper-based Records before Exposure to the Computer-based Record System (Pre-CPR Paper), in Ten Computer-based Records (CPR), and in Five Paper-based Records (Post-CPR Paper) after Exposure to the System

	Mean (SD)
Category of Information	Pre-CPR Paper		CPR		Post-CPR Paper
1. Chief complaint	2.6	(.84)	3.4	(2.12)	3	(1.22)
2. Past medical history	2.2	(1.69)	4.6*	(1.79)	2.2	(0.45)
3. lifestyle	5.8	(1.75)	8.1*	(1.73)	5	(1.41)
4. Psychosocial profile	3.1	(1.20)	1.1**	(0.74)	2.4	(0.55)
5. Family history	7.4	(2.55)	6.2	(3.39)	6.2	(1.30)
6. History of present illness	11.3	(4.60)	9.6	(3.17)	9.4	(3.21)
7. Review of systems	19.5*	(2.37)	14.5	(5.48)	18.6	(2.51)
8. Physical examination	16.3	(5.96)	18.3	(2.22)	12.4	(3.71)
9. Diagnoses	3.2	(1.40)	7.1**	(3.60)	3	(1.58)
10. Investigation	6.9	(1.97)	3.7	(4.85)	2.4	(2.51)
11. Treatment	4.4	(1.84)	4.8	(2.20)	3.4	(1.14)
Mean entries per record	82.7		81.4		68

Note: One asterisk (*) indicates P < 0.05; two (**), P < 0.001.

In addition, CPR records contained significantly more information items related to diagnoses than did the paper-based records made both before and after exposure to the CPR (F_2,24 = 7.20, P < 0.001). Examination of the information entries for diagnosis indicated that this difference was due to more detailed description in the CPR records of the nature of the main diagnosis. For example, using the CPR, the physician typically entered information indicating to what extent the diabetes was controlled (corresponding to an entry on the computer screen)as well as a specific indication of the manifestations of the diabetes. (For example, for one patient record this included an entry indicating a diagnosis of diabetes type II plus the following three entries: “diabetes with renal manifestation: hyporeninemic hypoaldosteronism,” “diabetes with ophthalmic manifestation,” and “diabetes with neurologic manifestation.”) As shown in Table 3▶, the CPR records contained significantly more of these specifications of diagnoses than did the paper-based records.

Table 3 .

Mean Number (SD) of Specifications of Primary Diagnosis and Different Diagnoses Entered by a Physician in Patient Records

	Pre-CPR Paper		CPRs		Post-CPR Paper
Number of records	10		10		5
Specifications of primary diagnosis	3.2	(1.40)	7.1*	(3.60)	3.0	(1.58)
Different diagnoses	1.8	(0.92)	1.3	(0.95)	1.2	(0.45)

Note: Pre-CPR Paper indicates paper-based records used before the physician was exposed to the computer-based system; CPRs, computer-based records; Post-CPR Paper, paper-based records used after exposure to the CPR system. *P < 0.01

Table 3▶ also shows the number of different diagnoses (i.e., diagnoses belonging to different disease categories) entered in the CPR and paper-based records. The CPR and post-CPR paper-based records contained a smaller list of different diagnoses than did the pre-CPR paper-based record, but this difference was not statistically significant.

Summary.

Analysis of the nature of paper-based and CPR records indicated an emphasis on recording different types of information. Computer-based records were found to contain more information about the patient's past medical history and lifestyle. In addition, CPRs contained more information related to the patient's primary diagnosis than did the paper-based records. This appears to reflect differences in the organization and structure of the paper-based and computer-based records.

Qualitative Characteristics of the Patient Records.

In considering qualitative differences in the patient records made by hand (both before and after exposure to the CPR) and those made using the computer, a number of striking patterns emerge. In a typical paper-based record, the physician recorded information on paper in narrative form, with connected and linked text and sentences. In contrast, in a typical patient record made using the CPR, the chief complaint was separate from the rest of the information and was located at the beginning of the record. Information in the CPR was entered as discrete units of information, was not linked in a narrative form, and followed the structure imposed by the system.

In terms of structure and style, the post-CPR paper records resembled the CPR more than they resembled the pre-CPR paper-based record. Like the information in the CPR, the information in the post-CPR paper record was organized in discrete units that were not linked by connecting narrative. It is interesting that the physician, when interviewed at the end of the study, was not aware of the extent to which exposure to the CPR had affected his style of recording patient data.

Comparison of the structure of the paper-based and CPR records suggests that even limited exposure to the CPR can have an important residual effect on the way a physician organizes information, even when the CPR is no longer used. Furthermore, it may be inferred that the structure of the CPR emphasizes the recording of information related to the patient's primary diagnosis. For example, the patient's chief complaint, which is often linked to the primary diagnosis, appears as the first information category at the beginning of the CPR. The complaint entry also has the effect of causing the system to load subsequent subcategories of information that are related to the complaint. Thus, in the CPR, more information about aspects of a case related to a specific medical problem or diagnosis may be present.

Information is typically recorded in the CPR in a succinct factual style (e.g., medical findings are simply selected by the user as being present or not and are not linked by connecting text). As a consequence, although the CPR contained approximately the same number of information items as the pre-CPR paper-based records, the CPR is shorter (an average of two printed pages per record). The CPRs also contain a more detailed description of the patient's medication and habits than does the pre-CPR paper-based record, corresponding to a detailed screen presented by the system for entering information related to this particular category.

Another striking difference between the CPR and pre-CPR paper-based records is that detailed information about the time course of the evolution of the patient's problem was nearly completely lacking from the CPR records. In contrast, much of the written narrative in the pre-CPR records consisted of discussion of time course, e.g., a detailed discussion of how the patient's problem had progressed.

The post-CPR paper-based records, like the CPR records, also contained very few descriptions of time course (in the five post-CPR records, only a single description of time course of the onset of the patient's condition appeared). Thus, the use of the CPR affected how the physician dealt with describing temporal aspects of a patient's problem, indicating a residual “effects of” use of the CPR when the physician returns to using paper-based hand-written records. In addition, the general format, organization, and style of the post-CPR paper-based records closely resembled those of the information recorded in the CPR system.

Summary.

analysis of the structure of patient records indicated differences between the paper-based records made before and after CPR exposure and the CPR, not only in what information was recorded but also in how the information was organized. Paper-based records made after use of the CPR closely resembled CPR records in structure and style, indicating at least a temporary residual effect of the use of this technology.

Phase II: Use of the CPR in a Diabetes Clinic

A second phase of research was conducted to assess the effects of the use of the CPR system by physicians in a diabetes clinic. This phase involved examination of the content of the paper-based records and CPRs made by the physicians, and considered variables of interest identified from the first phase of Study 1.

Method

Subjects.

Subjects consisted of 16 physicians from a metabolic day center at a large Montreal hospital, which specializes in the diagnosis and treatment of patients with diabetes and endocrine disorders. The CPR system was introduced in the clinic, and each physician underwent a training session in its use and was supplied with the system. The clinic formed an ideal environment for studying the effects of the transition from paper-based patient records to CPRs.

Procedure.

All the CPRs created by the physicians were made accessible for analysis during the period of the study. Six months after introduction of the system, each physician was also asked to create a CPR for one patient case for which they had already created a paper-based record (i.e., they were instructed to record the same information into the CPR). The CPR and paper-based record for individual patients were compared using the methodology described above for the first phase of Study 1, where segments containing individual items of information were identified in the records and then classified according to the information category in which they belonged.

Results

Comparison of CPR and Paper-based Records.

Table 4▶ shows the number of information items recorded by 7 physicians (i.e., of the 16 in the clinic, the 7 physicians who routinely used the CPR) in CPRs and paper-based records. Using the CPR system, significantly more information was entered by the physicians about the patients' chief complaints (F_1,29 = 14.35, .001). Also, the CPRs contained significantly less information about review of systems than the paper-based records (F_1,29 = 10.42, P < 0.003). This is consistent with the findings from the first phase of Study 1, where use of the CPR was associated with a focus on patients' particular complaints, diagnoses, and problems, with less emphasis on information about underlying physiology.

Table 4 .

Number of Items of Information Entered by Seven Physicians in Computer-based Patient Records (CPRs) and the Corresponding Paper-based Records (Paper)

Category of Information	Paper	CPR
1. Chief complaint	10	28**
2. Past medical history	13	13
3. Lifestyle	33	19
4. Psychosocial profile	10	11
5. Family history	7	14
6. History of present illness	55	27
7. Review of systems	52*	8
8. Physical examination	60	55
9. Diagnoses	14	9
10. Investigation	29	17
11. Treatment	21	24
Total	304	225

Note: One asterisk (*) indicates P < 0.05; two (**), P < 0.001.

In comparison of CPRs with the corresponding hand-written records (created for the same patient visits), it was found that the amount of recorded data differed. Most notable are differences in the type of recorded information, with the paper records typically containing more information categories than the CPRs. In general, there was a tendency for CPRs to contain fewer total items of information than the paper-based records.

Summary.

The results extend the findings from the first phase of Study 1, which involved analysis of records from a single physician, to comparison of CPRs and paper-based records made by several physicians. As in the first phase, the results indicate that use of the CPR is associated with the recording of types of information different from the types recorded in hand-written paper records, with a greater emphasis on information about patients' main complaints and diagnoses.

Study 2: Impact of the CPR on Physician–Patient Interaction

Following from the results of Study 1, which indicated potential effects of use of the CPR system on the process of information gathering and recording, a second study was undertaken to investigate this. In Study 2, the interactions of physicians with patients while the physicians used the CPR system were recorded using a methodology described by Kushniruk et al.⁹ This involved recording physician–patient interactions on audio- and videotape while the physicians used the CPR system.

Method

The CPR screen seen by the physician was output to a VCR and recorded on a videotape using a PC–video converter. The actual physician–patient verbal exchanges were input from a microphone into the audio input of the VCR. The result of this was a complete record of what the physicians were actually doing on the computer while interacting with their patients. This methodology for data collection borrows from recent advances in the area of usability testing,^25,26 extended for application in naturalistic settings.²⁷

Previous methodologic frameworks used in the study of physician–patient interaction²⁸ have included methods of discourse analysis for examining the acquisition of knowledge by physicians during interviews. In Study 2 we extended this type of analysis to include physicians' interactions with the CPR during interviews with patients. For 20 interviews by two physicians with patients, physician–patient interactions involving the use of the CPR were recorded. One physician was an intermediate-level user of the CPR system, and one was an expert user. All the video data were transcribed and coded for verbal exchanges between physician and patient as well as for each physician's interaction with the CPR.

Example of a Physician–Patient–CPR Interaction

An excerpt of a coded transcript of physician–patient exchanges is given in Table 5▶. Individual verbal exchanges between the physician and patient are shown in the second column of the table, with the topic of each exchange listed in the first column. Numbers refer to corresponding time on the video tape (i.e., the video counter). Using a methodology refined over the past several years, the transcripts of the physician–patient interactions were linked to the corresponding section of the video tape, facilitating later replay and coding of sections of the tape.²⁷ The physician's interactions with the CPR system are shown in the third column of the table, and the topic of the computer actions in the fourth column.

Table 5 .

Examples of Physician–Patient Interaction Involving a CPR System, Coded by Topic of Discussion and Corresponding Physician Action on the Computer

Topic of Physician– Patient Exchange	Physician–Patient Exchange	Physician's Action on the Computer	Topic of Computer Action
Lifestyle—Diet	1.Physician: “And what else do you have in the morning?”
	Patient: “Nothing else.” 00:08:15
Lifestyle—Diet	2.Physician: “So, just a croissant?”
	Patient: “A croissant but with an orange and something else.
		action: Goes to Lifestyle section, goes to Medication section, opens writing box.	Lifestyle—Medication
		goal: Enter Lifestyle data. 00:08:30
Lifestyle—Diet	3.Physician: “How big a croissant?”	action: Types “humulin,” keyword search window opens.	Lifestyle—Medication
	Patient: “This one”
		subgoal: List medication—humulin.
Lifestyle—Diet	4.Physician: “That's a big croissant!”	action: Clicks on “humulin,” options appear.	Lifestyle—Medication
	Patient: “It comes like this; it's already prepared.”
		action: Clicks on “Humulin 30/70 Suspension,” window closes.
Lifestyle—Diet	5.Physician: “I know, but you have to be careful, you have to be very, very careful. You know what a croissant is made with?”	action: Clicks on the insert button on the menu bar.
		action: Opens writing box on screen under Lifestyle medication.
	goal: Advise patient on diet.	action: Types 35; box closes.
	Patient: “Yeah, but I buy those that are not made with butter.”
Lifestyle—Diet	6.Physician: “What are they made with, what are your ones made with?”	action: Scrolls down screen.	Lifestyle—Medication
		action: Opens writing box on screen under Lifestyle medication.
	Patient: “Margarine.”
Lifestyle—Diet	7.Physician: “Just as fat.”	action: Types “26.”	Lifestyle—Medication
	Patient: “And they say 50% less fat.” 00:09:15
Treatment	8.Physician: “But that is still more fat than you need. Use bread, use ordinary bread.”
	Patient: “Ordinary bread?”

The discussion began with the physician asking the patient about his diet. Between exchanges 2 and 3, the physician began entering into the CPR information about the patient's medication. During the third verbal exchange, the physician entered data about medication at the same time that he was discussing diet with the patient.

In general, our analyses of physicians' interactions with CPRs in actual practice show variation, from use of the system by physicians only when the patient interview is completed (and the patient has left the office) to use of the system while the physician is talking to the patient. In this example, the physician entered information into the CPR while talking to the patient; the information was related to, but different from, the subject of the ongoing verbal discussion. In many cases, data entry into the computer about a particular medical topic was either shortly preceded by or immediately followed by a verbal exchange dealing with that topic.

Results

Users of the CPR can be classified into intermediate and expert users. Here, intermediate level users are defined as those who have used the system for the actual recording of some patient data but are still learning and mastering the system's features. In contrast, our expert users were those who had used the system extensively for entering all their cases for at least two years and who showed familiarity and mastery of all the main features of the system.

Table 6▶ presents a summary of the sections of the CPR that were accessed by two physicians while they dealt with similar medical cases. The first column of the table shows the information categories in the order in which they are presented on the computer screen by the CPR. The expert user accessed sections of the system in an order different from the order of sections presented by the computer, and moved back and forth between sections. From our previous pilot studies, we have found this movement back and forth among different sections of the CPR system is typical of its use by highly experienced users, and results in questioning during the physician–patient interview that is not completely bound by the order of sections on the computer screen.

Table 6 .

Order in which Information in the Computer-based Paper Record Is Accessed by an Intermediate User and by an Expert User for Typical Patient Cases

Categories of Information	Order as Accessed by Intermediate User	Order as Accessed by Expert User
1. Chief complaint	1	1
2. Past medical history	2	3
3 Lifestyle	3	6
4 Psychosocial profile	6	3
5. Family history	7	6
6. History of present illness	8	8
7. Review of systems	10	3
8. Physical examination	11	4
9 Diagnoses	12	8
10. Investigation	–	9
11 Treatment	–	11
		9
		2
		11

Note: Categories of information are shown i the order in which they appear in the CPR.

In contrast, the intermediate-level user, who had been using the CPR system to enter patient cases for several months but had considerably less experience with it than the expert user, followed the order of sections on the screen without deviation when asking the patient questions and entering data. This finding, which held for other intermediate-level users in a related pilot study, is consistent with an effect of technology reported by Kushniruk et al.,⁹ in which the use of such systems is associated with what is termed “screen-driven” behavior. Specifically, novice users were found to ignore the order of categories on the screen, but with experience they became highly guided by the system (i.e., screen-driven), at some point following the system's order entirely.⁹ In this study, expert users returned to a behavior whereby they do not necessarily follow the order of categories on the screen when interviewing patients.

Table 7▶ illustrates the degree to which the topic of verbal exchanges corresponded to actions made by an expert and an intermediate-level user of the CPR. For example, the initial dialogue of the expert user dealt with history of present illness, while his corresponding actions on the CPR also dealt with managing information about history of present illness. The expert user kept the dialogue and computer actions he performed somewhat synchronized and continually used the computer throughout the interview (e.g., for entering data managing screens). Analysis of the video data revealed that in some cases this physician appeared to prepare for an upcoming verbal exchange by performing actions on the computer related to a topic (e.g., selecting a related medical finding) before actually initiating a verbal exchange on that topic, while in other cases, discussion of a topic immediately preceded computer actions related to that topic. In either case, the verbal exchange and the actions on the computer generally dealt with the same topic. In contrast, in only a few instances did the verbal dialogue between the intermediate-level user and the patient match this physician's actions on the computer. Unlike the expert user, the intermediate-level user often had extended periods in the interview during which he did not interact at all with the CPR.

Table 7 .

Match Between Topic of Verbal Dialogue and CPR Actions for an Intermediate User and an Expert User of the Computer-based Patient Record During Two Physician–Patient Interviews

Intermediate User		Expert User
Dialogue Category	CPR Action Category	Dialogue Category	CPR Action Category
Lifestyle—diet	Chief complaint	History of present illness	History of present illness
History of present illness	No action
	No action	Lifestyle—medication	Lifestyle—medication
Review of systems	No action
Treatment		Psychosocial profile
	Past medical history
Lifestyle—diet		History of present illness	History of present illness
Lifestyle—diet
Lifestyle—medication	Lifestyle—medication	Other	Other
History of present illness	History of present illness	Lifestyle—habits	Lifestyle—habits
		Lifestyle—habits	Lifestyle—habits
Treatment
	Review of systems
Treatment
	Physical examination
Lifestyle—diet		Physical examination	Physical examination
	Investigation
Lifestyle—diet		Physical examination	Physical examination
	Treatment
Lifestyle—diet

note: Bold font indicates matches, when the topic of the verbal exchange corresponded to CPR actions made by the physicians.

Summary.

Analyses of physician interaction with the CPR during doctor-patient interviews indicate that nature of questions posed by a physician who is a less experienced CPR user during physician-patient interviews is influenced by the computer's organization of information and typically corresponds to the order in which information is presented on the computer screen. The computer's organization appears to affect the intermediate level user more than the expert.

Study 3: Individual Differences in CPR Use

Method

To explore the nature of individual differences in use of the CPR, semi-structured interviews were conducted with the clinic's staff. The interviews were audiotaped and transcribed for later analysis. Staff were interviewed prior to their introduction to the CPR system, to obtain baseline data on their background with computers and their attitudes toward new information technology. Questions included how often they used computers and how they thought the CPR would affect their daily work practice.

Three months after introduction of the system, follow-up structured interviews were conducted with each subject. They were asked how often they used the CPR, for what type of cases they used it, what the advantages and disadvantages of the system were, what problems they encountered in its use. The transcriptions of the audiotaped interviews were analyzed to identify statements made by the subjects about their work activities, key issues and concerns about the introduction of the CPR in the clinic, and changes in their perceptions of the CPR system.

Data from automated logging and summarization of subjects' use of the CPR were also collected over the course of the study. The data, consisting of a record of the number of cases entered each month by each subject into the CPR, were analyzed in conjunction with the results from the interviews.

Results

Table 8▶ summarizes interviews with seven users of the CPR system, which were conducted before they started to use the CPR system (first interview) and three months after they started to use the system (second interview). All the physicians who were interviewed had some prior experience with computers. This varied from the use of word processors and other common programs, to prior experience with a CPR system. The users who had the most computer experience included subjects 1, 2, 3 and 7, and these subjects were also among the most frequent users of the CPR system.

Table 8 .

Changes in Subjects' Expectations and Perceptions of CPR Use, and General Comments

Prior CPR Use		Prior Computer Experience	Predictions Re: Time (First Interview Data)	Follow-up Re: Time (Second Interview Data)	General Comments
Subject 1	No	Extensive—word processing, graphics, clinical applications	Will be slower than using paper-based records	Generally takes longer, potentially will save time	Learning curve steep, likes ability to customize
Subject 2	Yes	Extensive—word processing, database	Will be slower than using paper-based records	Takes longer	Prefers different features and organization
Subject 3	No	Intermediate—word processing, clinical applications	Will save time and be more efficient	Generally takes longer, but prescriptions are faster	Print-outs clear, access to categories a problem
Subject 4	No	Limited—word processing	Will save time	Takes longer	Frustrated with details of organization
Subject 5	Yes	Limited—word processing, clinical use	Will save a little time	Takes longer (for now)	Access to information is better, learning curve steep
Subject 6	No	Intermediate—research, clinical applications	Will save time and be more efficient	Takes longer	Need for greater flexibility in the system
Subject 7	Yes	Extensive—word processing, CPR use, clinical applications	Easier retrieval	Potentially can save time	Need for integration with other systems

Although all the physicians received training in use of the CPR, a number of physicians in the clinic did not use the technology in their practice. Seven of the 16 physicians who were given the opportunity to use the system did not use it for entering any real patient cases. Comments made (during the training sessions, which were taped) by physicians who did not subsequently use the system indicated concern about the steep learning curve that would be required to adequately master its use and concern that, ultimately, use of the system not only might not save them time but might, in fact, cost them time and thus permit them to see fewer patients. Physicians who saw patients in the diabetes clinic infrequently also questioned the benefit of learning the use the system for only a limited number of patients each week.

Examination of the transcribed interviews with physicians who did make subsequent use of the system indicated that a number of approaches were taken to adapt to the structure of information imposed by the system and that these approaches were based largely on the users' level of prior computer experience.

General Computer Experience

Three subjects, exemplified by Subject 1, had considerable experience in computer use in general as well as in medical applications, but had never used a CPR system before. Subject 1 used the system to enter patient data increasingly over time and eventually used it for all his cases. From the follow-up interview it was clear that he had experimented with different strategies for using the system, finding customization of the record's layout to be a most useful feature, which allowed him to circumvent aspects of the system's information organization that he found restrictive. This user was willing to experiment and learn the features of the system that would best suit his style of practice, while adjusting his style to suit the system when needed.

Prior Experience with CPR Systems

Two physicians had some experience with another CPR system. For example, Subject 2 had used a different CPR system while working at another hospital. However, he had difficulty mapping the CPR system used in the study to his preferred organization. Although this physician said that he was enthusiastic about the possibility of improving his practice using the CPR, he found its categories and organization too restrictive. Unlike Subject 1, he was unwilling to adapt to the organization of the new system and constantly compared it with the other CPR system he had used.

Limited Prior Computer Experience

Three subjects, exemplified by Subject 4, had limited experience with computers and had never used a CPR. Subject 4 found that the system has considerable potential and by the follow-up interview was beginning to use the system more extensively. She attempted to use the CPR during the actual patient interview but found that focusing on the system often interfered with the interview. In discussing the system, this subject focused on particular details of the system's structure and problems she was having at a fine-grained level (e.g., the exact ordering of information on the screen). She said that she had difficulty adjusting her work patterns to the system's organization.

Figure 2▶ shows the mean percentage of patient cases entered into the CPR (out of the total number of patient records created over a 22-month period) by three groups of physicians who used the CPR for entering patient cases. These physicians belonged to three distinct categories, based on their level of usage at the end of the study: 1) physicians with a consistently high level of usage of the CPR, i.e., who entered at least 95 percent of their cases as CPR records during each of the last six months of the study; 2) physicians with an intermediate level of usage; and 3) physicians whose usage at the end of the study had decreased to close to zero.

Figure 2.

Percentage of patient cases entered using the CPR, by three groups of physicians characterized by consistently high CPR usage (close to 100%; diamond shapes), intermediate usage (squares), and low usage (triangles) at the end of the study period.

These three categories of physicians, based on usage statistics, correspond closely to the three general categories described above—i.e., physicians with extensive prior computer experience tended to use the CPR for all their cases after an initial learning period; physicians who also had experience with a previous CPR system tended to use the CPR at an intermediate level; and physicians with limited general computer experience tended either not use the CPR at all or to decrease their usage to close to zero. Physicians who stopped using the system indicated that they experienced continual problems in its use and that ultimately it did not suit their needs and style of interacting with patients.

The perceived advantages and disadvantages of the system were fairly consistent across physicians, despite differences in their style and frequency of use. Physicians noted that the legibility of their patient records had improved with the computer and that certain aspects of their work were facilitated (e.g., ease in creating prescriptions). Disadvantages mentioned by all the subjects included problems in learning to adapt to the structure and organization of the CPR. Several physicians indicated concern about the potential effect of the system on their established work routines.

Physicians cited difficulties in adjusting to use of the CPR while the patient was present during a physician–patient interview. Some said that they would use the system only after the patient had left the room, whereas others attempted to adapt their interview style to include use of the CPR during the interview with the patient.

Although training was conducted in small groups, each consisting of four or five physicians who had similar computing experience, a number of the subjects indicated that individualized training was needed to supplement the classroom style of teaching in the small group. The importance of training is underscored by the fact that, of the seven physicians who did not use the CPR system at all after its introduction, five had not attended the recommended training sessions. They are the subject of an ongoing research project that we are currently undertaking.

Summary.

Periodic interviews were conducted to examine physicians' perceptions of the CPR and its use. Physicians' approaches to interacting with the system and their usage patterns were based to a large extent on their prior experience with computers in general and on their experience with other CPR systems. Usage appears to be greater when specific training is provided. Usage statistics indicated three distinct groups: physicians who adopted use of the CPR for entering all their patient cases, physicians who used the system at an intermediate level, and physicians whose use of the system consistently decreased over time.

Discussion

Use of the CPR described in this paper led to essential changes in the organization of information contained in patient records. There were essential differences in what was recorded in the types of records, even when physicians were asked to enter the same case (i.e., record the same information) into both CPRs and paper-based records.

There was some information loss in the CPR, but it is not clear whether the lost information is relevant to physician decision making about the patient. The relative lack of information about time course in the CPR may have important implications for diagnostic and treatment decisions for conditions in which the chronology of symptoms is critical. An ideal CPR probably should capture this chronology, which is recorded in paper. New technologic approaches to the recording of such information in the CPR should be explored. The effects of the changes we have found on the subsequent comprehension and interpretation of the CPR by the same physician as well as by other physicians is the focus of an ongoing investigation that we are conducting.

A number of interesting findings about the use of the CPR over time emerge from the data in this study. After having some exposure to the CPR, physicians showed at least a temporary residual learning effect. Paper-based records made after physicians' exposure to the CPR closely resembled the computer-based records in their format and organization. This is an example of the effects of technology, in which experience in the use of a technology changes users' reasoning and representation patterns, even in the absence of the technology.¹ Since reasoning is intimately related to the organization of knowledge structures, one can infer that the consistent use of a CPR (over at least one year) has a direct effect on knowledge organization and reasoning patterns in medical decision making.

Study of the physicians who did choose to use the system revealed a number of strategies used by them. Users of one type were characterized by a willingness to adjust to aspects of the CPR that differed from their normal practice while demanding capabilities for customizing the CPR whenever possible to meet their specific styles. This approach was characterized by an interplay between the physician and the system in which the system changed the physician's behavior and system organization was in turn modified by the physician where customization was possible. Users who experienced frustration while learning to use the system appeared more rigid in their insistence that the system support their particular styles of information gathering and recording. In general, the organization of the CPR had an important effect on learnability, and the learners' prior knowledge and exposure to related computer technology greatly affected their learning and their interaction with the system.

We examined one specific CPR in the studies reported here and found that its long-term use varied as physicians began to develop their own personal interaction styles. The findings from the studies show that this technology has an effect on human cognition. Different technologies may affect users in different ways; however, it is essential that we examine and understand the dynamic nature of the interaction between physician and computer if we are to improve computer systems in health care. The studies described here were conducted in a relatively homogeneous diabetic practice with endocrinologists as subjects. We intend to extend the studies to an outpatient clinic in internal medicine that has varied patient and physician populations.

Conclusions

Innovative information technologies are becoming an important part of daily professional activity in domains like medicine. It is essential that the subtle yet potentially profound effects of such systems on fundamental cognitive processes be better understood, particularly as such systems are used increasingly in the making of complex and critical decisions. It is a question of not only how the technology shapes our minds but also how our knowledge about cognition should shape technology. Organized knowledge is fundamental to the development of reasoning and decision making associated with expertise. We have described analyses that focus both on the organization of information in the patient records and on the process of actually using information technology during interactions with patients to acquire and record information. Such an approach is warranted because some of the most enduring effects of information technology may be both complex and unanticipated by both designers and evaluators of systems.²⁹

This research applied a number of methodologic and theoretic approaches, derived from studies in cognitive science and human–computer interaction, to assess the effects of the use of a CPR system on physicians' acquisition and organization of knowledge and their reasoning with it. Whether the effects of such systems are positive or negative, it is essential that attempts be made to assess and characterize these effects.

If the effects of computer systems (including CPRs) used in daily practice on human cognition can be documented and understood, the potential for using this knowledge in the design of future systems is enormous, both to improve the human–computer interaction and to use systems to promote training and teaching and reinforce recommended behaviors and reasoning patterns. To achieve this potential, the design of such systems should take into account the dynamic nature of human–computer interaction. It should also be understood that as we update the technology with new designs, human cognition will also change.