Understanding the Work of Pediatric Inpatient Medicine Teams: Implications for Information System Requirements

Abstract

Health information systems tend to be designed primarily for data retrieval and data entry, with insufficient attention paid to the larger contexts in which work occurs. As a result, low physician acceptance and satisfaction remain barriers to the successful integration of current informational and decision support systems (e.g., CPOE systems). This paper reports on a qualitative field study of team-based clinical care work and decision making. Our aim is to consider a radical redesign of clinical information systems, one that is built with context and the constraints of work practice in mind. We apply Cognitive Work Analysis (CWA) methods to characterize the work at a large children’s hospital, and analyze these results to consider new designs for clinical information systems. In this study, we report on themes, constraints, and ideas for design, showing how our CWA analyses lead to designs that are very different from current technology information systems.

1. Introduction

Health information technology has been shown to decrease medication errors, resource utilization and costs.¹ However, achieving end-user acceptance and satisfaction especially with computer physician order entry (CPOE) and electronic medical records (EMR) remains a barrier in health organizations including children’s hospitals.² Our goal was to analyze the work and social structure of modern team-based inpatient care teams with the ultimate goal of recommending system requirements to better support in-situ work. To meet this goal, we carried out an in-depth observational study of team-based care, and applied Cognitive Work Analysis (CWA) methods³ to gain insight into work activities and physician decision making.

We identified two central themes to the work of the participants in our study: The first, project management, describes the work that physicians do as they manage the important time, resources and interdisciplinary personnel central to the care of medically complex children in an inpatient setting. Our study site is a teaching hospital and so our second theme, education, is an implicit component of the environment. These themes are incorporated into our analysis products. At the end of our analysis, we propose high-level system requirements and design ideas developed from our two themes and based on the use of a formative systems analysis framework. We find these design ideas to be substantially different from the design of current health care information technology.

2. Setting

Seattle Children’s hospital is a tertiary care facility located in Seattle, Washington serving the larger Washington, Wyoming, Alaska, Montana, Idaho region with 250 beds and 13,482 admissions in 2008. The hospital serves as the base for the University of Washington Pediatric Residency Program training 92 residents in 2008 in addition to pediatric rotations from other residency programs. A typical general pediatrics faculty and housestaff team consists of an attending physician, four interns (1^st year residents) and two senior residents (2^nd and 3^rd year residents, hence-forth referred to as “seniors”) in addition to medical students. The faculty attending rotates every 2 weeks while the housestaff rotate every 4 weeks. An important feature of this healthcare setting is family-centered rounds. The physician team as well as nurses, social workers, pharmacists, nutritionists, and care coordinators join with members of the patient’s family or caregivers to discuss and plan the daily care of the patient. This rounding structure exemplifies the strong institutional commitment to team-based, interdisciplinary, family-centered care.⁴

3. Methods

We performed a qualitative field study on general pediatric inpatient medicine (house) teams. We conducted field observations over five (5) months, including forty-six (46) observation participants (forty physicians and medical students, three care coordinators and three nurses) and gathered supporting documents, protocols and information resources. Our field study included night and weekend observations. We also conducted in-depth semi-structured interviews with eleven (11) housestaff members. There were nine male and two female interview participants. Four of the participants were attending physicians, three seniors, two interns, two medical students. We focused our study on housestaff and attendings assigned to the general pediatric service regardless of their subspeciality.

Our work domain for this research is defined as the general inpatient medicine team at Seattle Children’s. We collected qualitative data on the work of general pediatric inpatient physician teams and used a systems analysis method, Cognitive Work Analysis (CWA), that provides a holistic framework for the analysis of complex socio-technical work domains.³ CWA identifies the constraints workers face and promotes flexibility by permitting workers to navigate constraints to pick successful paths towards goals. To support flexibility and unknown future work, CWA focuses on formative modeling rather than normative (system-defined) or descriptive (current) modeling.

In this paper we will present two primary work products of CWA, work domain analysis and control task analysis. Work domain analysis, the first phase of CWA, develops a representation of the work system functions. In order to develop an accurate model of the system that can cope with novelty and multiple goal paths, we must first describe the context independent of any action. Vicente describes creating the map for which you then can have multiple sets of directions (the actions or tasks) in order to navigate the map and respond to new situations. Control task analysis maps the cognitive states and cognitive processes for a work task, in this case Executing Care Plan, using a decision ladder as a modeling tool. Note that the decision ladder, unlike the work domain analysis is not a model of a cognitive activity.³ Rather it is a template noting general states of knowledge and data processing activities arranged in sequence abstracted from studying decision-making in multiple work domains.⁵ Previous applications of cognitive work analysis to medical domains have focused on investigating whether or not biological systems can be modeled using a framework derived from the study of control systems. Examples of CWA application to biomedical domains include Hajdukiewicz’s work modeling biological functions and Sharp’s work describing the biological functions necessary for neonatal ICU monitoring and Effken’s work in ICU displays.^6–8 More recent applications of CWA in medical domains include characterizing patient falls⁹, inpatient medication management¹⁰ and cardiac care nurses¹¹.

This study was approved by the University of Washington and Seattle Children’s IRB. In what follows, we have changed participants’ names to protect their identity.

4. Results, Theme 1: Project Management

Although attendings, seniors, interns and students have different responsibilities and operate in a hierarchical team structure, a central project management theme emerged from our analysis. All team members coordinated tasks, time and personnel in order to complete their work. For example, an intern who oversees the day-to-day management of the patient might coordinate consults from the nutritionist and social worker to further patient care, whereas seniors manage the availability and work loads of interns to meet the same goal. The Project Management Institute characterizes project management as “managing projects as entities… managing change and transition. And today, as never before, it is value driven. It is about meeting and exceeding customer expectations, about getting the best bang for the buck, creating value, and shortening implementation schedules.”¹² Specifically, the core skills for project management include integration, scope, time, cost, quality, human resources, communications, risk, procurement management. All of these themes emerged similarly in our coded data.¹³ These project management goals also were echoed in our discussions with participants.

From our observation data, we noted that depending on the situation, several team members could execute a particular strategy. The strategy for the goal of patient care is to order a consult, but it is independent of how the team members execute it whether it is the intern who enters the order or the attending who provides a peer-to-peer phone call to obtain the consult. This illustrates one of the principles of CWA: the dimensions of analysis remain independent though affected and constrained by other dimensions. We therefore discuss three sub-themes within project management that synthesize work across dimensions: managing patients, managing the team, and managing personal time.

4.1. Managing Patients

While all team members work towards the goal of patient care, the primary job of an intern is to know and manage the details of the execution of that care. An admitting intern will examine the patient, present the patient during rounds and propose a care plan to the team. The interns enter orders, coordinate internal hospital services and external services such as hospice or transportation. During patient discussions, task completion and temporal questions were very common during rounds or among the team. Interns were asked if procedures, consultations or medications had been ordered and if so, when would they be carried out. Attendings and families frequently ask when the patient can be discharged.

Anne, an intern, illustrates a patient management problem:

We put in an order for imaging and expect[ed] by the end of day I would have some kind of result and I found out it hadn’t been done yet because the schedule was too busy.

In this example, Anne runs into a constraint in the work environment that she can’t control, the imaging department schedule. As a consequence, she has to change strategies and adjust her plan for the patient.

Another component of managing patient care resources is understanding the reasons behind decisions. Paul, a senior, explains how the transfer of knowledge can break down:

I would sometimes go to sleep… and wake up and find that several orders had been entered by the overnight [resident] and I’d say to the person coming on in the morning… we started a new medicine. And they’d say ‘why’ and I’d say well (he) didn’t call me to tell me so I don’t know why.

Paul’s experience illustrates the nature of team-based care in this setting and what is reflected in the Team members dimension of Figure 1: although a single intern manages the patient primarily, other team members manage the patient as well either as the night on-call, during rounds when the non-presenting intern enters orders while the primary intern is presenting, or a result of work redistribution. In addition, because patient care is multidisciplinary, often patient decision are made by consulting services without explanation to the intern.

Figure 1.

Simplified work domain analysis hierarchy, focusing on project management.

4.2. Managing the Team

Seniors are tasked with managing the team, distributing work and being the first point of contact for the housestaff. Although the attending physician is ultimately responsible for the decisions made by the team, senior residents make sure that work is done. As John explains, “My goal… is not to let anything slip through the cracks.”

Steve explains the competing goals of team management:

“Let’s get through rounds. Let’s see as many kids as we can. Let’s teach. Let’s prioritize patient care and then education. Let’s get our post-call intern out.”

The team needs to “get through rounds” and discuss as many patients as they can within the finite amount of time allocated. The seniors decide whether or not the team will visit the bedside or discuss the patient in the team room or even bother discussing the patient at all due to time constraints. They need to provide teaching opportunities either themselves or with the attending and feedback for the interns without depriving the team of the time needed to complete their work. They are required to send the post-call intern home before 1PM (permanent work environment constraint). Throughout the day, they allocate and reallocate patient management and tasks, adjusting for ad hoc work schedule changes such as when team members are sick, called away or overloaded with other patients. These priorities, resident schedules, team workload, which patients are sicker than others, which patients provide better educational opportunities are all constraints in our CWA framework in different dimensions.

4.3. Managing Personal Time

For all the participants we interviewed, efficiency emerged as a central personal theme as well as a perceived institutional goal. All participants expressed goals to maximize efficiency and felt success was measured by their level of efficiency. Often when workloads are high, completing daily required tasks is challenging and secondary priorities such as self-education or time at the bedside are dropped. Sleep quality has been associated with higher stress scores in physicians and our data indicates that sleep and leaving work are also strong motivators for efficiency.¹⁴

Steve explains that he would prefer to have more time to enter orders:

“It would be great if we all had the time… to present and then sit down and put in orders on our kids and really think about it. …it’s a different story when you have a half hour to put orders as opposed to 30 seconds.”

Our data supports other research that shows resident work patterns are erratic and fragmented.¹⁵ To compensate for the lack of time, tasks such as finishing progress notes are carried through rounds as interns attempt to multi-task. Carl, an intern, explains that multitasking during rounds comes at the price of being able to listen and learn about patients that they might end up managing.

“If we finished our notes like in the morning… you (could) listen in on rounds, you (could) put your own input in... There’s patients I’ve never seen before that I’ve taken care of, you know. You like see them through the window but you don’t actually get to go in.”

The above examples illustrate some personal goals of our participants: to have more time to think about orders or to have more time to meet patients. CWA supports analyzing personal goals because within high level team goals such as good patient care, individual team members should be able to choose a path that meets their needs.

5. Results, Theme 2: Education

As a teaching hospital, education is an institutional goal supported formally by seminars, training and conferences. Team members feel responsible to teach each other on an ad hoc basis, forming a strong culture of group teaching. Therefore, it was not surprising that medical education emerged as a clear theme in our analysis of team goals and individual skills. From our strategies analysis, we identified an additional educational sub-theme, work environment education. This theme concerns learning about Seattle Children’s in particular and what procedures, roles, protocols and idiosyncrasies exist in the hospital.

5.1. Medical Education

The need for medical education and support of teaching occurs at all levels of training and situations. We turn back to Anne who is not yet comfortable with putting orders by herself:

I feel hesitant to put any order without double-checking with somebody. Even the most benign orders.

Anne needs support from her senior or attending and we observed that discussing orders frequently led to opportunities for teaching as the senior and intern reviewed orders or decisions together in the team room. It is not always the case, however, that personal teaching can occur when it is needed. During family-centered rounds, it is hospital procedure for an intern to enter orders while another intern presents the patient. Steve describes putting orders in during rounds:

“…you’re ordering meds you’re unfamiliar with all the time. All the time. The orders are flying and you’re not sure what the dosing is… for some random drug they use up on Hem-Onc or something they use on Rheumatology. You’re saying, what’s that? How do you spell it?”

Similar to Anne, Steve needed help with orders, but given the quick pacing and time constraints of rounds, he needed an alternate learning strategy that supported his situation.

Gaps in knowledge are not limited to the housestaff. Joe, an attending whose primary work is in a different pediatric subspecialty, has never practiced general pediatrics and only serves 2 two-week shifts a year. He expressed great anxiety about his rotations:

“Honestly, I feel terrified every time I do it. I feel like a fish out of water for lots of reasons. I don’t keep up with the literature. I don’t practice it day to day. I don’t converse with colleagues about [it].”

Joe is anxious about his lack of general pediatric inpatient knowledge and as a result relies heavily on the housestaff. Even if Joe is an extreme case, the increase in the complexity and severity of hospitalized children was cited by the attendings as reasons for frequent consultations and peer-to-peer education.¹⁶

5.2. Work Environment Education

We observed that lack of knowledge about the work environment such as how to arrange for an interpreter, the hospital-specific nomenclature of orders and the division of labor and responsibilities of hospital units were challenges team members faced. Steve shares a story:

We had a kid we had to start IVIg on for Kawasaki and the kid started to have a reaction... We had three different brains thinking about this, plus the house team, and none of them could tell me with any accuracy what the protocol is for reaction to IVIg. So finally after talking to Cards and ID and my attending, I called Rheumatology who has not ever consulted on this patient or even heard of their name. And I said, here’s the deal and they were able to give me the protocol.

This story illustrates a strategy, albeit a frustrating one, that Steve used to accomplish his goal of finding the protocol. Although familiarity with the work environment increased with experience and time in the hospital, we observed that the same interns on different teams supporting new subspecialties continued to discover gaps in their knowledge about the hospital. In addition, not all residents are part of the in-house residency program. A number of our participants were from family medicine residency programs fulfilling their pediatric inpatient training requirements and had no familiarity with the hospital.

Joe as an attending also expressed his lack of familiarity with the hospital environment:

“I don’t know how it gets done. I don’t know how to put the orders in. I don’t even know how to discharge patients.”

Our participants, especially housestaff, uncovered work environment constraints by trial and error if the information was not readily available from team members. Attendings, if they are in-house hospitalists, can provide guidance and information on hospital resources or protocols, but as in Joe’s case, they may themselves be as uneducated about the work environment and needing support themselves.

6. Analysis

We employed a systems framework, Cognitive Work Analysis (CWA), to analyze our work domain and develop system requirements. CWA contains several analysis techniques to develop a holistic model of work including the tasks (e.g. ordering), workers (physicians, nurses) and roles (attending, seniors, interns). For this paper, we will focus on the first and second phases of CWA, work domain analysis and control task analysis.⁵ Work domain analysis results in a generalized or prototypical model of the work domain without regard to tasks or individuals. Control task analysis uses a decision modeling tool, the decision ladder, that shows how workers move from information processing activities to knowledge states. However, the decision ladder does not assume the workers will move from state to state in a linear fashion and accounts for the ability to jump from any part of the ladder to any other. We also describe how we integrated the two themes we uncovered (patient project management and education) into our analysis. We will first discuss work domain analysis and present a model of the work functions and physical objects in the work domains connected by mean-end linkages. We will then show how we used control task analysis to map clinical tasks in order to identify states of knowledge and entry and exit points that can be supported through information technology.

6.1. Work Domain Analysis

To understand work domain analysis, consider a city map. The map allows for choosing different routes from Point A to Point B based on dynamic, unpredictable factors such as construction, traffic or sight-seeing vs. getting to work. The buildings and one-way streets on the map are “constraints” on the types of routes that can be chosen. Similarly, we aim to create a “map” of our work domain by defining the constraints that shape physician behaviors. Just as the city map is independent from the route eventually chosen, our model is independent of how work is carried out or by whom – task and organizational analysis being separate analysis phases of CWA. Defining a prototypical, generalized model is a key to its utility because workers rely on the regularity of the system in order to choose appropriate actions and detect faults. In other words, navigating using the map relies on the predictability that one-way streets will remain one-way when plotting routes.

We used Rasmussen’s technique known as the abstraction hierarchy to model our work domain.⁵ Figure 1 shows a simplified version of five levels in the hierarchy that are linked through a means-end relationship shown on the left. First, we defined the general activities of the work domain. From our qualitative analysis, we found that project management and education are general activities of the inpatient medicine teams at Seattle Children’s. These serve as the “What” in Figure 1. We then modeled the goals for these general activities, the “Why”, and the physical resources available to fulfill the activities, the “How”. Figure 1 only shows a few examples of goals and resources that are associated with the general function of “project managing patients”; we provide the full table of work domain analysis results in Lin, 2010¹⁷.

Example goals include providing good patient care, efficient patient care and family-centered care that could be measured by family satisfaction, length of stay, financial measurements, the number of patients assigned to teams, and medication (or other) errors. Examples of physical resources in this environment include computerized medical records, online medical resources, medical books, online work schedules, phones and pagers. Linked together, they form a prototypical model of the inpatient medicine work domain by defining the behavior-shaping constraints of physician work. As we discuss later, this work domain analysis, built directly from our observational study, can be used to inform design choices for information systems that better match work processes.

6.2. Control Task Analysis

Whereas work domain analysis focuses on the processes to satisfy goals, that level of analysis says nothing about a temporal breakdown of the tasks and activities that are carried out by the physicians. In CWA analyses, these aspects of work are understood in terms of a control task analysis using the decision ladder template.

A decision ladder maps the cognitive states and cognitive processes for a work task and is the work product of the second phase of CWA, control task analysis; Figure 2 shows our decision ladder for the task of executing a care plan. Note that the decision ladder, unlike the abstraction-decomposition space, is not a model of a cognitive activity.³ Rather, it is a template noting general states of knowledge and data processing activities arranged in sequence abstracted from studying decision-making in multiple work domains.⁵ As seen in Figure 2, there are three main sections of the decision ladder, Situation Analysis (the path upwards), Solutions Evaluation (the activities and states at the top), and Solutions Execution (the path downwards).^{18, 19} Entry and exit points (grey arrows) can occur at multiple steps along the ladder. While the solid arrows represent a full traversal of the decision ladder, dotted lines represents common short cuts across the ladder from states of knowledge to other states of knowledge.

Figure 2.

Decision Ladder showing examples of different knowledge processing activities and knowledge states.

For example, the short-cut labeled “B” might occur for some very simple alerts (e.g., patient reports pain) that lead directly to a standard order entry (e.g., for pain medication). In our study, an example of the short cut labeled “A” can be seen in patient hand-offs (e.g., to the night shift). In this case, a physician might provide a set of rules that permit decision making to go directly from some data to a set of pre-specified tasks. (E.g., if patient data exceeds some threshold.) It is still up to the on-duty physician to re-formulate those tasks as a plan of specific orders that can be entered in the CPOE system.

As Figure 2 emphasizes (lower R), it is only the very last, mechanistic data activity that corresponds to the task of “order entry”, that is typically the target of CPOE systems. This ladder dramatically illustrates that the decision making that leads to a particular order can be very long with many intervening knowledge states and data processing activities. In our view, this leads directly to an opportunity for more effective design of information systems.

Each node in the decision ladder could be better supported via information systems. Some nodes are well-supported currently – e.g., the “Set of data” is, at most institutions, supported by an electronic medical record, and the final “execute” action is supported by CPOE systems. Additionally, many information systems provide some level of alerting, and some decision support that may go from diagnosis (a patient state) to a suggested order set (a set of tasks). However, there is clearly room for additional support. Few systems consider the patient state (beyond diagnosis codes), and few systems allow decision makers to evaluate competing tasks to achieve a goal.

7. System Design Implications

To support physician work, it is important to acknowledge the actual work they perform. While this may seem tautological, many designers have focused on designing based on task descriptions, paper forms, or data types rather than work. Our results suggest that improved designs for information systems that support the work of medical teams at Seattle Children’s would incorporate and use the two general themes we identified in our analysis: project management and education.

To begin, the notion of project management immediately suggests a time-line visualization of past and future events for patient care. With such a visualization, users could rapidly review past events, and then visualize different future alternatives for treatments, as well as upcoming events. Visualizing the future might allow users to better avoid potential errors, but more importantly, it will simply better prepare both caregivers and patients for upcoming changes in health care management, whether those be a change in treatment regimen, a scheduled procedure, or simply a hand-off to weekend care staff. Furthermore, beyond a single patient, our results show that physicians also need to manage their entire list of patients, as well as teams of health-care providers and their own personal time. Thus, information tools should also provide timeline visualizations of entire sets of patients, as well as work schedules of individual physicians or teams of care givers.

The work domain analysis shown in Figure 1 also has implications for improved design of systems. For example, many of the highest level goals and missions are institutionally defined, rather than arising directly from individual care givers. Thus, it would be important both to make these more visible to information system users, as well as to link these goals to the more concrete “what” and “how” levels that work toward achieving these goals. Thus, we would recommend that quality metrics be made constantly visible as users work with the system. Thus, as they consider different alternative or make ordering decision, they can more easily see the impact of choices on different metrics. Such an approach is almost the opposite of many decision support “alert” systems that only notify users when an error or problem is detected in a particular order, without considering more global context.

The decision ladder of Figure 2 breaks up the decision making process into a large variety of knowledge states and data processing activities. Again, this is in contrast to many CPOE system designs which typically only support “order entry”, merely the final step in decision making: transmitting the step-by-step sequence of orders into the computer. Instead, we envision systems that support the decision making process in multiple ways, with decision support available both on the situation analysis (up) side of the ladder, as well as the solution execution (down) side. For example, currently technology does not usually do a good job of saving or record knowledge states, such as a short-term goal to better diagnosis a patient. This lack can be particularly problematic for hand-off situations. If the system stored these sorts of goals and made them readily available, then care-givers could work to achieve these goal through a variety of methods leading to potentially different order sets.

Finally, the inpatient medicine teams at Seattle Children’s hospital are comprised of physicians and students with varying backgrounds, training, specialties and experience with the work environment. Information systems should be designed to support the diversity of these teams and their roles take into consideration the level of experience of the physician. In addition to specific medical experience or training, another sub-theme of education is work environment education. This included learning the physical locations of resources, institution policies and human resources. Our second theme of education should incorporate both medical and work environment learning and inform the design of systems so that they support multiple learning situations from group teaching to self-directed learning, as well as fulfilling immediate information needs.

8. Limitations

Our study has a number of limitations. We studied one healthcare organization, Seattle Children’s, that has a specific profile as a pediatric, urban, academic, teaching hospital. Within Seattle Children’s, we studied one physician service, focusing on physicians. However, the methods used in this research are readily applicable to other settings and other types of workers. With a common CWA framework, the researcher can define a work domain (whether it be another service, hospital or clinical role) and study work constraints.

Due to constraints in our IRB approval, we could not consider the patient perspective, nor include them in our study. This limitation makes our study unable to comment on the important role that patients and their proxies (e.g., a pediatric patient’s parents) have in the decision making process. Indeed, an important policy at Seattle Children’s is patient-centered care. Thus, our finding that physicians manage their work as a set of “projects” is limited in that it does not include or list the patient as an important team member.

9. Conclusion

We have studied the in-situ work of general inpatient medicine teams in an academic pediatric teaching hospital. As a result of our analysis, we have identified two emerging themes: The first is project management of time, resources, tasks and people, whether focused on the care of patients, the management of the team, or the physicians’ own time and resources. The second theme is education, including both medical education and work environment education. These are the prototypical general activities of the work domain of the inpatient medicine team. Building from our observational study, we applied Cognitive Work Analysis, a conceptual framework developed from studies of complex socio-technical environments, and carried out several analyses of the data. Here, we presented an example work domain analysis illustrating the relationships among high-level goals and the means and resources needed to achieve them. In addition, we developed a decision ladder that describes the entire decision making process as a complex sequence of knowledge states and data processing activities. This ladder provides a very different view of the work than that of traditional CPOE which only focuses on the final clerical order entry task.

Finally, we used our results and these analyses to consider implications on the design of systems. These designs are significantly different from current technology information systems. In the future we look forward to developing and evaluating prototypes that demonstrate these value of these design ideas. Our hypothesis is that such designs will better match the work of clinical decision making, and therefore be more accepted by physicians. Although we studied a single pediatric teaching hospital, these CWA methods and analyses could be applied to other health care environments. Our long-term goals are to improve CPOE and health information systems by supporting the two broad themes we identified.