Contrasting Views of Physicians and Nurses about an Inpatient Computer-based Provider Order-entry System

Abstract

Objective: Many hospitals are investing in computer-based provider order-entry (POE) systems, and providers' evaluations have proved important for the success of the systems. The authors assessed how physicians and nurses viewed the effects of one modified commercial POE system on time spent patients, resource utilization, errors with orders, and overall quality of care.

Design: Survey.

Measurements: Opinions of 271 POE users on medicine wards of an urban teaching hospital: 96 medical house officers, 49 attending physicians, 19 clinical fellows with heavy inpatient loads, and 107 nurses.

Results: Responses were received from 85 percent of the sample. Most physicians and nurses agreed that orders were executed faster under POE. About 30 percent of house officers and attendings or fellows, compared with 56 percent of nurses, reported improvement in overall quality of care with POE. Forty-four percent of house officers and 34 percent of attendings/fellows reported that their time with patients decreased, whereas 56 percent of nurses indicated that their time with patients increased (P < 0.001). Sixty percent of house officers and 41 percent of attendings/fellows indicated that order errors increased, whereas 69 percent of nurses indicated a decrease or no change in errors. Although most nurses reported no change in the frequency of ordering tests and medications with POE, 61 percent of house officers reported an increased frequency.

Conclusion: Physicians and nurses had markedly different views about effects of a POE system on patient care, highlighting the need to consider both perspectives when assessing the impact of POE. With this POE system, most nurses saw beneficial effects, whereas many physicians saw negative effects.

Many medical centers are investing substantial resources in the development and installation of computer systems to facilitate some aspects of clinical care. Functions of these systems may include not only storage of medical records but electronic links among departments and sophisticated decision support^1,2,3 in the form of diagnostic tools, error detection, and automated reminders. Computerized systems have been developed to allow electronic entry of orders.^{4,5,6,7,8,9,10} Many early systems focused on entry of medications in pharmacies by pharmacy technicians or clerical staff.^11,12,13 Others had rudimentary ordering systems that were initially not well integrated into the rest of the clinical information system.¹⁴

Modern computerized systems often allow physicians, nurses, or other health care providers to enter orders directly into sophisticated, integrated systems. These computer-based provider order-entry (POE) systems typically are intended to improve staff efficiency, decrease resource use, and enhance patient care. Features of existing POE systems vary considerably but may include specific feedback about care being offered to patients. They can offer unique abilities to track orders, provide access to disease-based information, detect faulty or incompatible sets of orders, communicate among departments, automate requests for tests and consultations, assist with cost control, avoid drug interactions, and offer advice about patient management. They may also be useful for teaching students¹⁵ and house staff, conducting internal studies, and even for achieving national accreditation.¹⁶ To understand some impacts of POE systems, several academic medical centers have conducted studies of their inpatient or outpatient systems.^{8,9,10, 17,18,19,20,21}

Patients, health care providers, hospital staff, educators, administrators, and health care payers have a stake in the impact of medical computer systems. Therefore, evaluations of computerized medical systems must reflect the many dimensions of potential impact of the systems, such as finances, quality of care, use of resources, efficiency of care, and attitudes. Studies of various computer-based clinical decision support systems have been summarized in the medical literature.²² Implementing decision support has been shown to decrease use of hospital resources.²³ More than 20 years ago, McDonald²⁴ showed that physicians receiving computerized suggestions about outpatient care were relatively likely to respond to them. Others have demonstrated that reminders improve compliance with preventive care protocols.^{25, 26} In inpatient studies, computerized display of simple status messages describing actual and predicted lengths of stay for individual patients was associated with shorter lengths of stay,²⁷ and displays of charges revealed trends toward fewer tests ordered,²⁸ decreased lengths of stay,²⁹ and lower total hospital charges.^28,29 At LDS Hospital in Salt Lake City, computerized reminders about perioperative antibiotics have been associated with improved rates of postoperative wound infections.³⁰

Significant cost savings associated with POE have been demonstrated. In one study, Tierney et al.¹⁹ reported that implementation of an inpatient POE system with automated decision support, but without direct electronic links to departments with ancillary services, could reduce costs in one hospital by an estimated $3 million per year, with no significant increase in the number of re-admissions or subsequent outpatient visits. It is important to note that differences among order-entry systems most likely lead to different impacts on hospitals. Order-entry systems that offer no decision support, for example, may be expected to have a smaller impact on hospital charges than those that do offer decision support.

The views and attitudes of physician and nursing staff toward POE are vital determinants of the acceptance, impact, and ultimate success of a given POE system. Although many physicians believe that computers can benefit medical practice,³¹ actual implementation of computer systems, especially for inpatient care, has met negative attitudes and considerable resistance from health care staff, including both physicians and nurses.^5,8 Even in hospitals with advanced systems, initial acceptance of POE has required extensive training of users, vigorous encouragement from administrators,¹⁰ and active support of senior clinical staff.¹⁸ Lee et al.²⁰ reported that overall provider satisfaction with a POE system depended more on user's perceptions about ease of use and provider productivity than on features intended to improve quality of care. Differences between provider types were also noted regarding preferences for specific system features: physicians valued decision support, whereas nurses valued the improved legibility of orders. Although house staff in the study of Lee et al. were generally more satisfied with POE than nurses, Lee et al. did not assess the opinions of clinical fellows or attending physicians, who may have strong feelings and important insights about the usefulness of POE. They also did not question respondents about the effects of POE on specific aspects of clinical care.

It is unclear whether other types of POE systems, such as ones with links to ancillary departments but without sophisticated decision support, uniformly improve staff efficiency and decrease resource utilization without adversely affecting quality of care. To determine how both physicians and nurses view the impact of POE on patient care, we conducted a survey of nurse and physician users of an inpatient POE system at a large, urban teaching hospital six months after the system was implemented on eight medical wards. Our specific aims were to assess the attitudes of medical house officers, attending physicians, clinical fellows, and nurses toward a POE system with direct links to ancillary departments but without sophisticated decision support; determine their views about specific strengths and weaknesses of the system; and elicit opinions from a representative group of users about the effects of the system on overall quality of care and specific aspects of patient care, including resource utilization, time spent with patients, and ordering errors. We hypothesized that a POE system that helps organize nurses' work, requires physicians to enter orders, and does not offer physicians the benefit of reminders or decision support would appeal more to nurses than to physicians and might lead nurses to indicate more strongly than physicians that quality of care was improved.

Methods

Description of the POE System

Our POE system, Ordernet, is based on the INVISION commercial product by Shared Medical Systems Corp. (SMS). The base product provided by the vendor is best described as a standard practice model that provides an array of user-specific interfaces that guide users through the ordering process. It is then incumbent on each medical center to modify these model interfaces to reflect actual local practice. Starting in fiscal year 1993, the basic SMS product was extensively augmented for use at our hospital by staff of The Johns Hopkins Medicine Center for Information Systems, with input from committees representing the departments of medicine, pharmacy, pathology (including laboratory medicine and the blood bank), nursing, radiology, and nutrition. Both physicians and nurses had extensive input into the modifications. The teams worked closely with programming staff to create specialized physician, nursing, and ancillary interfaces. This was an iterative process, evolving through a series of testing and feedback, that allowed the various health care providers to ensure that the order-entry process enabled them to use the system in a way that reinforced practice guidelines and was most efficient in facilitating high-quality patient care.

The system “views” that appear as the system is used depend on the role of the user: There are, in general, two distinct views, one for physicians and physician extenders such as physician assistants, and a second one for nurses. Within the physician view, specific transactions may require countersignature or verification from someone with a higher level of authority, depending on the setting and medical staff bylaws. The details of these views were modified from the original system. Other features that were modified include options for census displays, printing, and online documentation of medications.

Many new features were also developed, including those for ancillary views (laboratory specialists, radiologists, dietitians, and respiratory therapists); cosigning and verifying orders; creating and managing order sets, discharge orders and worksheets, nurses' orders, patient information, consultants' orders “do not resuscitate” orders, and pharmacists' orders (verifying, displaying, printing, and taking verbal orders); adding orders; and features addressing a variety of medication orders, laboratory orders, end-of-session processes, pathology and radiology orders, ICD-9 coding and diagnostic groups, the cardiac station, neurometrics, and the pulmonary laboratory. The system has been supported each year by the equivalent of seven or eight full-time employees and three or four consultants, for a total of about 56 person-years of effort to date. This does not include time spent by trainers, nursing students, physician consultants, and support analysts, who have also worked on development. When implemented, the new system entirely replaced handwritten order sheets, requiring physicians to enter all orders directly into computer terminals located on hospital units.

Potential users include all physicians and nurses in the Department of Medicine, although orders are created and entered primarily by house officers. Clinical fellows and attending physicians use the system primarily to review orders, active medications, and laboratory results and to enter pre- and post-procedure orders. Nurses typically use the system to extract and implement orders but enter some orders directly—for diet and management of intravenous catheters, for example—hence the term “provider order entry,” rather than “physician order entry.”

An electronic light pen or mouse is used to select items from on-screen menus; when ordering medications, users may also type free-text orders. To standardize some orders and decrease the time required to enter them, default or user-modifiable order sets are available, providing fast access to groups of orders that are often based on a diagnosis, critical pathway, or procedure.

The POE network is linked directly to the hospital laboratory, the pharmacy, and the radiology department. The system automatically prints labels for laboratory specimens and prints a running transcript of orders for the nurses and the medical record. There is no interface with billing systems, materials management, or outpatient clinics, and users cannot recall orders from prior admissions or query the clinical database directly. Decision support is limited to identifying common medication interactions and detecting contraindications due to stated allergies.

The POE system was pilot-tested at The Johns Hopkins Hospital on one unit from May 1995 to March 1996, and then was implemented on all eight general internal medicine units in the hospital from April to July 1996. Physicians (primarily residents) who were expected to enter orders into the system received dedicated training sessions about using the system. On the day of implementation on each medical unit, all residents working on those units began to enter all orders electronically for all patients on the units. The system is not installed in the intensive care unit or coronary care unit. The mainframe-based system is used in several hospital buildings, with an average of approximately four workstations on each medical unit and a scheduled downtime of two hours per week. Approximately 21 percent of all hospital beds were covered by the POE system, and users may have moved among units with and without POE during the study period.

Survey Content

For this study, we modified previously developed questionnaires^17,20 to include 45 questions with seven-point Likert scales, rating attributes of the system and users' experience with computers. Attributes rated included preference for continuing to use the POE system or returning to handwritten orders; impact of the POE system on time spent with patients, number of tests ordered, ordering errors, and overall quality of patient care; POE system response time and down-time; ease of use; and overall satisfaction. Five items collected routine demographic information, and five open-ended items asked participants to describe advantages, disadvantages, and desired improvements. Survey responses were confidential; we tracked respondents by users' hospital identification codes.

Targets of Survey and Sampling Strategy

The target population for the survey was nurses and physicians working on the eight general (i.e., nonintensive) internal medicine units that were using the POE system. Participants included all 96 medical housestaff, a sample of attendings and fellows in internal medicine and several medicine subspecialties, all eight unit nurse managers, and a 50 percent random sample of the 198 medical floor registered nurses, stratified by unit (▶).

Table 1.

Distribution of Survey Respondents

Respondent Group	No. Eligible for Survey	No. (%) Responding
House officers:
Intern	32	31 (97)
Junior resident	32	28 (88)
Senior resident	32	27 (84)
Attending physicians:
Cardiology	17	15 (88)
General Internal Medicine	15	14 (93)
Infectious diseases	8	6 (75)
Gastroenterology	5	3 (60)
Other	4	4 (100)
Fellows:
Cardiology	11	9 (82)
Gastroenterology	4	3 (75)
Infectious diseases	4	2 (50)
Nurses:
Staff nurse	99	82 (83)
Nurse Manager	8	7 (88)
Total	271	231 (85)

To focus on attending physicians with substantial exposure to the POE system, we surveyed attending physicians who admitted at least 42 patients in the 12 months preceding the survey (yielding 49 attending physicians). This sample consisted of 17 cardiologists, 8 infectious disease specialists, 5 gastroenterologists, 2 nephrologists, 1 hematologist, 1 rheumatologist, and 15 internists, four of whom were chief residents (n = 49). We also surveyed clinical fellows who spent most of their professional time providing care to patients on the medical wards; these included 11 cardiology fellows, 4 gastroenterology fellows, and 4 infectious disease fellows (n = 19). Floor nurses and nurse managers were considered a single provider type (“nurses”) for analysis. Attendings and fellows were also considered one provider type (“attendings/fellows”), since their interactions with the POE system had similar purposes and expected frequencies.

Survey Administration

In November 1996, about five months after implementation of the POE system, we discussed distribution of the questionnaires with leaders in the departments of nursing and medicine, to gain awareness and acceptance and to spread word of the upcoming survey. We informed participants about the purpose of the study and the need to evaluate attitudes and opinions about the system and its effects on care. Questionnaires were then distributed to each group via interdepartmental mail, personal mailboxes, conferences, and clinics. Persons who did not respond within three weeks were sent a second questionnaire; if there was no response after three more weeks, personal contact was made with nonresponding participants or their supervisors, and a third and final questionnaire was sent.

Analytic Approach

Stata 5.0 software³² was used for all statistical analyses, and an alpha level of 0.05 was used to test the significance of all comparisons. For the Likert-scaled items, differences in ratings of the possible effects of the POE system among the three provider types were assessed by the Kolmogorov-Smirnov test, a nonparametric test of differences in distributions. Respondents with no hands-on POE experience were excluded from the final analyses. To simplify presentation of most results, we collapsed ratings of 1, 2, and 3 into a “low” group and ratings of 5, 6, and 7 into a “high” group, leaving a rating of 4 as “neutral.” The full scale of values from 1 to 7, however, was used for all statistical tests.

We used correlation coefficients to test variables that we believed, based on our perceptions and bivariate analyses, could have a significant relationship to overall satisfaction with the POE system. These variables were prior experience with personal computers, perceived effect of POE on quality of patient care, problem with system response time, inconvenience from downtime, perceived effect of POE on time spent with patients, belief that job was easier under POE, perceived change in ordering errors, belief that new data provided by POE were helpful, perceived speed of order execution, and overall ease of POE use. We have added a general measure of the instrument's reliability by including a Cronbach alpha test of items that reflect an overall assessment of the system. The three items included in this measure concern overall satisfaction with the POE system, frustration with the system, and the desire to continue to use the system (versus returning to handwritten orders).

Results

Response to Survey

Questionnaires were received from 231 (85 percent) of 271 eligible participants, including 86 (90 percent) of the house officers, 56 (82 percent) of the attendings/fellows, and 89 (83 percent) of the nurses. Five nurses and one house officer in the original target sample were ineligible for the study, because of unidentifiable addresses or changes in professional roles. There was no significant difference between groups in the response rates (P > 0.3 by chi-squared test). ▶ gives details of all responding groups.

Characteristics of Respondents

On a seven-point scale (1 indicating “none”; 7, “tremendous amount”), the respondents' mean ratings of their experience with computers was 4.6 for house officers, 5.1 for attendings/fellows, and 3.2 for nurses (P < 0.001 by Kolmogorov-Smirnov test for the difference between physicians and nurses). Ninety-one percent of the respondents indicated that they had experience with our POE system. Those without POE experience were attendings (n = 17) or fellows (n = 3), and their distributions by clinical specialty area were not significantly different from those of participants with experience (P > 0.1 by chi-squared test). Participants without POE experience were excluded from the remainder of the analysis.

All responding house officers, 63 percent of responding attendings/fellows, and 98 percent of responding nurses indicated that they received formal training on POE (P < 0.001 by Fisher exact test for a difference between groups). Users could report one of six categories of frequency of using POE to view or enter orders: more than five times each day, three to five times each day, one or two times each day, one to six times each week, less than once per week, and never. The median reported frequency of use of the POE system to view or enter orders was more than five times daily for house officers, less than three times daily for attendings/fellows, and one to five times daily for nurses (frequency range for all provider types: 0 to more than 5 times daily). About a third of attendings and fellows entered orders less often than once per week.

Opinions about Ease of Use

Reports about the system's ease of use varied by provider type. Users of POE were asked whether they agreed (on a scale from 1 to 7) that “once learned, [the system] is easy to use.” Sixty-three percent of house officers agreed, compared with 37 percent of attendings/fellows and 78 percent of nurses (P < 0.03 by Kolmogorov-Smirnov test). Easy access to help about the system was indicated by 24 percent of house officers and 33 percent of attendings/fellows, compared with 66 percent of nurses (P < 0.01 by Kolmogorov-Smirnov test). Provider types differed in opinions about whether POE made their jobs easier: 37 percent of house officers indicated that the system made their jobs easier rather than harder (15 percent neutral), compared with 6 percent of attendings/fellows (28 percent neutral) and 63 percent of nurses (13 percent neutral) (P < 0.01 by Kolmogorov-Smirnov test).

Strengths and Weaknesses of the POE System

Ranked items showed that many users found specific features or aspects of the POE system to be benefits or barriers (▶). There was general agreement about disadvantages: 89 percent of house officers, 58 percent of attendings/fellows, and 58 percent of nurses complained about problems with downtime, while 74 percent of house officers, 65 percent of attendings/fellows, and 33 percent of nurses indicated problems with system response time. In answering open-ended questions, respondents differed in their opinions by provider type: features preferred by house officers were remote viewing or ordering (57 responses) and laboratory/radiology ordering (16 responses); features preferred by attendings and fellows were remote viewing or ordering (5 responses) and displaying active orders (4 responses); and those preferred by nurses were legibility (42 responses) and viewing active orders (14 responses).

Table 2.

Opinions about Provider Order-entry (POE) Features Held by 50 Percent or More of Respondents

Feature	Respondents (%)
Benefits:
Label printing for specimens	84
Radiology requisitions	80
Ordering remotely	80
Viewing orders remotely	79
Medications displayed with dose times	79
Formulary on-line	64
Orders executed quickly	57
Order sets for admissions	56
Patient data provided by system	52
Barriers:
Downtime	71
Excess work for simple orders	71
Lag time for new admissions*	64
System response time	55

^*Lag time refers to the delay from the time a patient is admitted to the time when POE for that patient is available.

Perceived Effects of POE System on Patient Care

▶ shows respondents' beliefs about perceived effects of the POE system on specific aspects of patient care. House officers (44 percent) and attendings/fellows (34 percent) were more likely than nurses (9 percent) to report that use of POE decreased their time with patients. House officers were also more likely than nurses to indicate that use of POE was associated with more tests and more errors in ordering. Both nurses (63 percent) and house officers (59 percent) indicated that orders were executed faster with POE. Most nurses (56 percent) believed that overall quality of care with POE was better rather than worse (21 percent neutral), in contrast to only 29 percent of house officers (40 percent neutral) and 34 percent of attendings/fellows (23 percent neutral) (P < 0.03 by Kolmogorov-Smirnov test).

Figure 1.

Perceptions by professional groups about the effects of the computer-based provider order-entry system on specific aspects of patient care. Items were rated on a seven-point Likert scale but are grouped here according to low scores (1, 2, or 3), a neutral score (4), or high scores (5, 6, or 7). The white blocks reflect perception of increased time with patients, fewer tests, and fewer errors in ordering; the hatched blocks indicate a rating of 4, reflecting a “neutral” perception; and the solid blocks reflect perception of decreased time with patients, more tests, and more errors in ordering. An asterisk (^*) indicates house officers differ from nurses, P < 0.001 by Kolmogorov-Smirnov. A dagger (†) indicates attendings/fellows differ from nurses, P < 0.001 by Kolmogorov-Smirnov.

Overall, cardiology staff were less satisfied with POE than noncardiology staff and were also less in favor of continuing to use POE (P < 0.001 by Kolmogorov-Smirnov test). This group reported that quality of care was worse with POE than without it; POE was less convenient than handwritten orders; their job was harder under POE; orders were less clear and were executed more slowly; and ordering medications was not easy.

Overall Satisfaction with POE System

Differences in satisfaction with POE were observed: 42 percent of house officers and 34 percent of attendings/fellows gave the system high satisfaction scores (one of the top three ratings), compared with 69 percent of nurses (P ≤ 0.001 by Kolmogorov-Smirnov test). The percentage of respondents in favor continuing use of POE was higher for nurses (75 percent) and house officers (66 percent) than for attendings/fellows (44 percent). In the correlation analysis, the two factors that were most strongly correlated with satisfaction were the belief that POE made the user's job easier and the overall ease of system use (▶). Other variables tested, including belief that POE increased overall quality of patient care, were also significantly correlated with satisfaction (P < 0.05). The Cronbach reliability coefficient for overall satisfaction with the POE system, frustration with the system, and the desire to continue to use the system (versus returning to handwritten orders) was 0.92.

Table 3.

Correlates of Satisfaction with the Provider Order-entry (POE) System

Correlation Coefficient*	Variable
0.79	Belief that POE made user's job easier
0.76	Overall ease of using POE system
0.75	Belief that POE increased overall quality of patient care
0.62	Belief that new data provided by POE was helpful
0.61	Belief that the system has decreased ordering errors
0.58	Belief that the system has increased time spent with patients
0.57	Belief that the system has increased timeliness of order execution
0.45	Belief that system response time was not a problem
0.33	Lack of experience with personal computers
0.27	Belief that downtime was not an inconvenience

^*All correlation coefficients have P < 0.05.

Discussion

Our study evaluated a POE system installed in the Department of Medicine at an academic hospital. We surveyed house officers, attendings/fellows, and nurses. The most striking findings in this study are the different views that these groups had about the effects of POE on patient care. One effect concerns the ordering of laboratory tests and medication: House officers indicated that POE led to increased ordering. Although nurses tended to believe that POE had decreased ordering of laboratory tests and medications, it is important to note that in our hospital, house officers are responsible for initiating most of these orders. Increased ordering under POE, if true, may be due to the selection of items by menu structures or to the presence of order sets, either of which may have prompted physicians to order more than one test at a time. Rapid execution of orders following POE may also increase ordering if house staff attribute execution time to the system and respond by continuing to order yet more tests. These features may have also encouraged physicians to prescribe some medications more frequently. Although this study was not designed to determine the actual number of tests or medications ordered, the reported views of house officers in our study suggest that our POE system may affect resource utilization differently from systems that can inform users about costs or appropriateness of tests or medications at the time of ordering.^19,28 Conveying such information may be a key determinant of the number of tests and medications ordered.

A second effect on patient care concerns errors in ordering. Most house officers and many attendings, fellows, and nurses indicated a belief that POE use was associated with an increased number of errors. The objective studies from other institutions suggest that POE systems, in general, do not have major tangible adverse effects on clinical aspects of care, such as increasing errors or needless laboratory testing. Although users' perceptions may be inaccurate, an increase in errors—if true in this study—could be explained by our system's menu-based approach, whereby users may incorrectly select an order adjacent to a desired one. We did not ask users to explain their responses, so our insights are speculative. Periodic use of keyboard-based free-text entry, a feature not rated highly in another study,²⁰ may change error rates and efficiency, and implementation requires correct interpretation of keyed text.^7,33 Omissions or errors with orders may be decreased with other methods of computer-based monitoring, such as notifying users about potential drug interactions, drug contra-indications due to allergies or other reasons, or the need to check serum drug levels.³⁴ Further investigation is needed to determine whether errors in orders actually increased at our institution, but the perceived increase emphasizes the need to monitor ordering errors as POE systems are deployed and to institute methods of preventing errors. Both physicians and nurses have roles in monitoring, since they may recognize different types of errors according to their unique roles in delivering patient care.

Introducing POE into medical practice may alter the amount of time providers spend with patients. Our respondent groups had markedly different views about this, physicians indicating that POE decreased time spent with patients but nurses indicating increased time. One explanation is that POE is associated with increased time to generate orders,^{16,17,19,20,21} and this may indeed detract from the time physicians could otherwise spend with patients. Nurses, on the other hand, who are freed from interpreting handwritten orders and who are supported by POE's streamlined system of organizing orders, may find more time for direct patient care. The benefits of a POE system do appear to come at the expense of imposing an increased time burden on physicians. To achieve the most appropriate balance, system developers should obtain the views of both physicians and nurses about the impact of their POE system on the time that each type of caregiver is able to spend with patients.

Some demands on time are created by computer hardware itself. Problems with system response time^20,35,36 and downtime have been observed here and elsewhere. In our study, house officers' frequent need for around-the-clock system access to enter new orders and hoc has probably increased their concern about down-time. Failures of the entire system are rare, but failures of local networks or links between departments (e.g., medicine, admissions, pharmacy, pathology) occasionally cause degradation of service. Some institutions have created paper- or computer-based backup systems for downtime.⁷ Provider order-entry requires substantial technical support, maintenance, and protocols for failure.¹⁶

Differences in users' opinions were also seen between cardiology and noncardiology staff. Cardiology staff were more critical than noncardiology staff of the POE system. A higher proportion of cardiology staff reported slow response time and difficulty using POE. We hypothesize that because of either cardiologists' higher workloads or the urgency of many cardiovascular conditions, the increased time required to enter orders with POE may alter the perceptions of cardiologists and may be especially cumbersome. Expanding the use of POE to units with higher acuity of care or greater volumes of orders will require attention, regarding a priori design of systems, to the opinions of both physicians and nurses on those units.

One of the most important findings in this study was that physicians were generally less satisfied than nurses with the system. The best correlates of satisfaction, such as ease of use and perceived impact on making the user's job easier, closely matched those in the study by Lee et al.,²⁰ despite opposite findings about which professional group liked the system more than the other. Our system differs from many others in that it has more extensive links to ancillary departments in the hospital (e.g., with departments of radiology, pharmacy, and pathology) but less sophistication in decision support. These findings highlight two valuable lessons.

First, those who perform order entry but receive few direct benefits from POE are likely to have much less favorable views of the system than those who receive tangible benefits, such as improved efficiency of daily work or the ability to provide better patient care. This is a consistent finding, regardless of whether an individual system tends to provide direct benefits primarily to physicians or primarily to nurses. Benefits to distinct user groups and nonuser groups should be formally assessed. Direct electronic links among departments, for example, may improve overall efficiency and speed of order execution, but they do not seem to enhance the satisfaction of the end users.

Second, although POE systems and medical centers vary widely in structure, POE system developers—which may include hospital staff and faculty—should focus on improving features that directly enhance ease of use and productivity, if they hope to improve satisfaction with using these systems. Barriers such as excess work for simple orders could be studied by conducting focus groups of users; possible solutions might involve decreasing the number of screens and mouse clicks required to create orders or creating a streamlined process for generating common, simple orders. Decreasing lag time for new admissions may necessitate earlier entry of new patients into the system. Features that improve clinical care, such as decision support, may not bolster ease of use at all, thus creating potential disparities between effects of various features and the need to prioritize the importance of features during the design stage. For reasons of technical complexity, commercially available systems may focus more on improved ease of use than on decision support. Creating both effects may require user-specific “phase-in” periods, during which decision support is initially withheld from an individual while the user becomes accustomed to other aspects of the system.

Several additional factors may have contributed to house officers' overall tendency to convey relatively negative impressions of the POE system. House officers' levels of chronic stress and limited time, along with the sudden shift in work patterns brought by the implementation of inpatient POE—however unavoidable—may have led to exaggerated survey responses. Time delays and small obstacles in placing new orders may have created a false impression that errors were being made, more tests were being ordered, and physicians were spending much less time with patients. Others' experiences in this field have shown us that perceptions of a system's performance often differ from reality, and we have thus provided suggestions for how the perceived impact of our system may be either confirmed or refuted. Downtime, although scheduled, was minimal and unlikely to affect opinions in the presence of so many dominant factors already mentioned. The low representation of the system in our hospital is also unlikely to affect opinions, since our study involved only the department of medicine, and POE extended to all adult general medical units in the hospital, as well as to the adult cardiology unit within our department.

Because our system contains a custom-designed user interface, it is possible that some of the observed findings are unique to this system. Users' responses in our study may not reflect opinions of POE users elsewhere, where hospital conditions and POE features such as ancillary links and decision supports may differ. Nevertheless, the results of this study could apply to a number of POE systems that, like ours, provide links to ancillary departments without sophisticated decision support. Although not proved here, the influence of decision support on opinions about systems is probably prominent.

The self-reported nature of data in this study is an important limitation of this study, especially since users' perceptions may not accurately reflect actual performance.¹⁷ Nevertheless, a survey of physician and staff users is one important component in the evaluation of potential effects of the POE system on patient care, from the perspectives of those who are in the best position to see the effects of its use. Small trends indicating that attendings and fellows have more experience with computers than house staff may contradict conventional wisdom about the greater computer-related knowledge of younger generations. The high response rates for all groups, however, argues against a selection bias. Furthermore, there was no significant difference in reported experience between attendings and fellows and house staff. Problems suggested by the survey, such as increased ordering of tests and medications and increased ordering errors, require more detailed, objective investigation. We are currently undertaking an objective analysis of changes in the ordering of laboratory and radiology tests before and after implementation of our POE system.

It should be noted that the survey instrument was not designed to transform sets of individual scales into summative scores to represent particular measures of opinion, clinical care, or use of resources. We have not used the survey for this purpose in our analyses. The correlation coefficients and the high Cronbach alpha coefficient, however, provide some information that supports the construct validity of the individual items included in the questionnaire. Creating other measures for the Cronbach test was not appropriate, because we did not assume that questions within the other broad categories should be linked with each other. Regarding quality of care, for example, there was no reason to assume that opinions about time spent with patients should match opinions about errors in ordering.

Conclusion

We conclude that physicians and nurses have different opinions about a computer-based POE system. In contrast to previous studies, physicians in this study indicate more adverse effects of POE, whereas nurses indicate more benefits. We suggest the need to study the effects of POE systems, whether positive or negative, on the distinct user and nonuser groups during the design stages. We also indicate that clinical decision support, while probably a key aspect of using POE systems to improve quality of care, may require time for users to adjust to its influence on patterns of work and thought. To gain acceptance of POE systems, designers should focus early efforts on ease of use and elimination of barriers.