Task Analysis of Writing Hospital Admission Orders: Evidence of a Problem-Based Approach

Abstract

As part of a broader project to improve the usability of computerized physician order entry (CPOE) systems, we set out to study the cognitive tasks physicians undertake to write “admission orders” when admitting a patient to the hospital. In particular, we evaluate the hypothesis that physicians’ mental model of diagnostic and therapeutic planning is problem based, whereas both paper-based ordering and CPOE are typically organized around functional categories of orders such as those reflected in the mnemonic ADCVAANDIML. A task analysis was performed which included think-aloud observations of physicians writing orders in clinical care settings and for fictional case-scenarios, as well as a semi-structured questionnaire. Our work finds core tasks of admitting a patient to hospital and conflicts between physicians’ mental model and traditional ordering systems. Based on our study, we suggest improvements to traditional CPOE systems.

Introduction

We set out to study how CPOE could be made more useful for admitting patients to the hospital, i.e., for writing “admission orders”. Our principal hypothesis is that there is a mismatch between a physician’s mental model of planning care and the conceptual model of order writing underlying CPOE. Specifically, we believed that the physician’s mental model for diagnostic and therapeutic planning is problem based. For example, planning is organized around each of a patient’s medical problems, such as pneumonia, diabetes, and hypertension.

Prior work has examined a problem-based framework for writing orders. Over forty years ago, Larry Weed proposed the problem-oriented medical record (POMR) which has become ubiquitous in clinical documentation [1]. However, his related information system (PROMIS) was not widely used, likely because it was too rigid for busy clinicians.

However, physicians are generally trained to write orders not organized by problem but using a mnemonic that reflects functional types of orders, such as ADCVAANDIML (for Admit, Diagnosis, Condition, Vital signs, Allergies, Activity, Nursing, Diet, IV fluids, Medications, Labs/procedures) [2]. This mnemonic-based model for writing orders on paper has been increasingly adopted by many CPOE systems. It remains unclear whether this model maps poorly to the problem-based approach physicians use to plan admission orders.

Our training and clinical experiences suggests that there is rarely discussion of how to map from a problem-based plan to mnemonic-based orders, as shown in Figure 1, and we were not successful in finding discussion of this process in the literature. Previous work has examined the cognitive complexity of CPOE, and has found that poorly-designed artifacts add unnecessary load on the physician’s working memory. We hypothesize that even CPOE systems designed to mimic the traditional mnemonic-based approach may also pose an unnecessary cognitive burden.

Figure 1.

With this motivation, our research seeks to develop a better understanding of whether and how traditional mnemonic-based ordering systems may conflict with physicians’ mental model used in planning patient care. Our initial efforts have focused on describing how physicians approach the problem of admitting a patient to the hospital. We shall use such an understanding to propose improvements in CPOE, in particular for admitting multi-problem patients.

Methods

We undertook an initial task analysis to better define the mental model that physicians use in writing admission orders. Task analysis can be described as breaking down a task in terms of required actions, cognitive processes, and their relationships [3]. To accomplish this, we used a “think-aloud” method while physicians wrote admission orders both for real patients and fictional cases. Think-aloud protocols appear not to distort physician decision-making, and they avoid the distortion of retrospective descriptions [4].

For this exploratory work, we chose to study internal medicine physicians since their patients tend to have multiple medical problems. Because residents usually write the admission orders at the study sites, we recruited internal medicine residents in their third and final year of training. The protocol was approved by the Panel for Human Subjects Research at Stanford. We approached physicians by email and obtained informed consent from all participating physicians.

THINK-ALOUD METHOD

We first observed physicians writing orders for real patients at Stanford University Medical Center (SUMC) and the Palo Alto Veterans Affairs (VA) hospital, each of which uses a different CPOE system. This purpose of this phase was to create an initial description of admission order writing process using a cognitive perspective and to inform our subsequent design of fictional case scenarios and semi-structured questionnaire.

We then performed another series of think-aloud sessions as physicians wrote admission orders for fictional cases in a more controlled setting. The three fictional cases were developed in collaboration with clinical experts and included: (1) a straightforward case of heart failure which in part served to make the subject comfortable with the think-aloud method; (2) a patient with acute coronary syndrome and gastrointestinal bleeding which presents the conflict of whether or not to treat with anticoagulants (blood thinners); and (3) a patient with multiple medical problems and potential drug interactions.

For the fictional patient think-aloud sessions, we chose to have physicians write orders on blank paper, as we believe that this context imposes fewer artifacts than either CPOE system currently used by the residents, and all residents in the study had experience writing orders on blank paper at an affiliated hospital.

SEMI-STRUCTURED QUESTIONNAIRE

After each subject wrote admission orders for the fictional cases, we administered a semi-structured questionnaire. The purpose of the semi-structured questionnaire was to explore issues which arose during the first set of think-aloud sessions.

DATA ANALYSIS

Data were recorded as audio and handwritten transcripts, supplemented with observational notes and semi-structured questionnaires. Audio tapes, typed transcripts, and observational notes were reviewed in an iterative process by two investigators (CJ, RZ). We coded both (1) the sequence of steps that physicians took to write orders and (2) related conceptual themes. We identified conceptual themes by consensus and they were revised with each iteration. Each final theme is thus supported by specific observations [5].

Results

For the real patient think-aloud observations, we studied four physicians admitting a total of nine real patients across two hospitals. For the fictional patient think-aloud observations, we observed seven physicians (one of whom participated in the real patient think-aloud) each admitting three fictional cases and administered the semi-structured questionnaire to these seven.

SEQUENCE OF TASKS

Based on the real and fictional patient think-aloud observations and the semi-structured questionnaires, subjects demonstrated a consistent workflow for the care of complex patients involving four major tasks:

1. Gather data and develop a “problem list”

2. Create plan for each problem

3. Write orders

4. Review

1. Gather data and develop a “problem list”

This involved integration of historical, physical exam, laboratory, and diagnostic data, resulting in a “problem list” to guide further planning. While this overlaps with the diagnostic thought process, creation of the problem list was a distinct step. This step involved judgments about which diagnoses could be discarded as irrelevant to the inpatient stay (e.g., osteoporosis) , while identifying other issues unique to the inpatient setting and adding them to the problem list (e.g., deep vein thrombosis (DVT) prophylaxis). We will refer to these added problems as “default problems”.

2. Create Plan for Each Problem

Six of seven subjects first created a high-level plan on a separate blank sheet of paper. Planning at this stage focused on key medications, diagnostic tests, and nursing orders specific to treating a given problem. The result was a brief outline of the treatment plan.

3. Write Orders

The plan outline above was then mapped to the order framework of the CPOE system or to blank paper orders. The first step was generally to decide how sick the patient was and thus what level of care was required (e.g., “Admit to ICU”). The remainder of ordering consisted of expanding the high-level problem-specific plans into more detail and adding “housekeeping” issues necessary for managing a hospital stay (e.g., diet, general nursing orders).

4. Review

Subjects reviewed their orders and compared against any written outline plan as described above, checking for omissions with respect to this outline.

CONCEPTUAL THEMES

Within this general workflow, we identified 5 conceptual themes as follows.

1. Order planning for complex patients is organized primarily by problem

As hypothesized, the two common frameworks used to organize orders were mnemonic- and problem-based. Physicians admitting single-problem patients used the mnemonic-based approach for both planning and writing orders. When more than one active problem appeared, all subjects constructed a problem list and nine of the ten subjects included with the problem list some form of high-level plan as described above. One subject stated, “I need to break down the parts of the patient first, and deal with those parts”.

2. Order writing alternates between problem-based and mnemonic-based frameworks

Although we theorized that there may be a single (and difficult) translation step when transferring from a problem-based high-level plan to mnemonic-based orders, we observed that users alternated back and forth between problem- and mnemonic-based frameworks while writing orders. For example, after using the problem-based framework to create the high-level plan, a common pattern was to use the first nine of the eleven categories in the ADCVAANDIML mnemonic to ensure completeness, then switch back to a problem-based approach for medications and laboratory tests. That is, within the medications and laboratory sections, orders were grouped by problem.

3. Inclusion of “default problems” on the problem list

The problem list created for planning orders was a specialized problem list, distinct from the list of problems typically listed in the “assessment” section of the formal “History and Physical” document written to summarize the patient’s clinical presentation and admission. In creating the problem list for order planning, subjects typically excluded inactive or chronic problems that did not require attention during hospitalization (e.g., osteoporosis). As explained in the sequence of tasks above, physicians then added to the list “default problems” that reflect risks or issues unique to the inpatient setting such as DVT prophylaxis. In explaining her default problems, one subject stated, “I’ll have whatever problems I have, [but] the last three are always: Prophylaxis, Code Status and Disposition.” The semi-structured questionnaires revealed that all subjects either did include them or believed they should consistently include some of them.

4. Missed interactions across order set sections

Based on both our observations and the questionnaire, none of our subjects had an explicit method for checking for drug-drug or other types of interactions. For one of the case scenarios, seven of seven subjects did recognize the critical interaction between anticoagulant medications being considered for treatment for acute coronary syndrome and the concurrent problem of gastrointestinal bleeding. However, four of the seven subjects appeared to notice this only as they considered ordering the anticoagulants and not when they created their problem list and high-level plan. Comments about having only ad-hoc methods for interaction checking included: “[I’m] aware that I need to check, but don’t do so explicitly” and “I hopefully notice if there’s a problem while writing meds”

5. Jumping between order set sections

During order-writing, we observed considerable jumping between different sections of the in-progress order set to rework earlier sections. Often this appeared to result from one order triggering another associated order. For example, one subject stated, “We’ll type and cross her and transfuse her 2 units… we’ll get a repeat hematocrit after transfusion and then q 8 hrs.” Here ordering a blood transfusion (often included in the Medication section) triggered the order to recheck the patient’s blood count (Laboratory section).

DISCUSSION

The results of our task analysis confirmed the hypothesis that when dealing with complex patients, physicians typically organize their plans by problem, not by functional categories such as ADCVAANDIML. This is consistent with Weed’s Problem-Oriented Medical Record and SOAP note approach and was summarized by one subject: “The problem is that ADCVAANDIML is great to help protect you, but it’s not how you think.” However, problem-based planning was not sufficient to generate a complete set of admission orders. Our subjects also used mnemonic-based frameworks during order writing to ensure completeness and reduce errors of omission.

More specifically, physicians blended problem-based planning with selective use of mnemonics to leverage the strengths of each approach. First, MDs created a high-level plan by problem, including essential medications, laboratory tests, and problem-specific nursing orders. This outline was then mapped to the appropriate sections within ADCVAANDIML, while the rest of the mnemonic was traversed to ensure completeness. Subjects alternated fluidly between the tasks of addressing specific problems and providing orders for general hospital care.

Physicians appeared to depend on these frameworks – the problem-based framework in particular – to divide the planning problem into smaller sub-problems that could be solved sequentially. This is consistent with cognitive research on complex problem-solving in general [6]. As Moskowitz described nearly 20 years ago: “The cognitive method for clinical decision making under uncertainty appears to be incremental, subdividing the overall decision into subproblems” [7]

This observation about cognitive processing also suggests why interactions between subproblems may be difficult for subjects to identify. While focusing on a given subproblem (e.g., plan to give anticoagulants for acute coronary syndrome), it is challenging to simultaneously consider other subproblems (e.g., to avoid anticoagulants with gastrointestinal bleeding).

The five conceptual themes that emerged from our observations have potential implications for CPOE design, as summarized in the Table and discussed further below.

Theme	Implication for CPOE
Order planning for multi-problem patients primarily done with problem-based framework	Integrate problem- based planning into order writing process
Order execution done by alternating between problem-based and mnemonic-based frameworks	Offer a “hybrid” framework for ordering
Inclusion of “default problems” on the problem list	Include default problems
Missed interactions across order set sections	Simultaneous views and cross-category interactions
Jumping between order set sections	Corollary orders and one-touch navigation

First, problem-based planning should be better integrated into the ordering process. This will seem obvious to many readers, but few CPOE systems currently do so effectively [8]. Furthermore, our findings suggest that simply incorporating problem lists into a CPOE interface will not fully address the underlying issue. The key question is how to capture the interface between planning-by-problem and order-writing? CPOE could mimic the current process, with its division between “high-level” planning and order-writing. However, this resulting mapping problem may represent an unnecessary cognitive load. The sharp distinction we observed between planning and order-writing may simply be an artifact of how mnemonic-based ordering frameworks do not map well to the internal planning process. Perhaps CPOE could be designed to eliminate this extra step by combining planning and order-writing in a single process organized by problems.

This fusion might be accomplished by offering a hybrid framework for admission orders which leverages the respective strengths of the problem-based and mnemonic-based approaches. Considering the sequence of tasks we observed, one such hybrid could involve the following steps:

• Admit to – the physician first triages the patient to the appropriate level of care, e.g., “Admit to ICU” or “Admit to floor”. This (the “A”) is the only part of the ADCVAANDIML framework that we propose belongs before the problem-based phase.

• Problem-Based – the physician then provides a set of orders for each problem on the list, focusing on problem-specific medication, laboratory and nursing orders. This covers the “M”, “L” and most “N” orders for ADCVAANDIML.

• General Hospital Care – orders are completed by traversing the remainder of the mnemonic (“DCVAANDI”) to ensure completeness. This comprises the common orders that are rarely part of the high-level plan but are needed for general hospital care such as IV fluids, vitals signs, diet, etc.

The fact that subjects included “default problems” on their problem list suggests that CPOE might offer default problems. These could be customized by the user and serve as memory aids for addressing important safety and comfort issues. Categories for default problems included by our subjects include Prophylaxis and Precautions (DVT and gastrointestinal ulcer prophylaxis), Patient Comfort (analgesics, anxiolytics, laxatives) and Health Care Maintenance (smoking cessation, vaccinations).

The challenge of interaction checking (drug-drug, drug-disease, etc.) appears to be especially challenging for interactions between items that fall under different problems of the problem-based framework (acute coronary syndrome vs. gastrointestinal bleeding), or under different sections of the mnemonic-based framework (medications vs. laboratory tests). CPOE might address this by presenting views of both frameworks simultaneously to facilitate recognition of interactions. For example, a problem-based display would facilitate recognition of drug-lab interactions within a given problem, while a mnemonic-based view of all ordered medications would help catch drug-drug interactions across problems. If the CPOE interface allows the user to choose whether to view orders in a problem-based or mnemonic-based framework, CPOE decision support should pay special attention to cross-category interactions, such as drug-drug interactions in problem view, and drug-laboratory interactions in mnemonic view. Thus, rather than highlighting all possible interactions, the user can focus on interactions that are most difficult to recognize.

Finally, the nonlinear thought process of our subjects was poorly supported by the CPOE systems they were using. This can be improved by more extensive use of “corollary orders” [9] that offer the user the option to include common associated orders without forcing them to break workflow by navigating to other sections of the order set. In addition, since such corollary orders cannot account for all nonlinear thinking, CPOE should offer one-touch navigation such that the user can rapidly browse to any previous or future section of the order set. Current CPOE often forces the user down a linear path and makes backtracking difficult. As one subject exclaimed, “The user interface isn’t that friendly – you can only do one thing at a time, you have to select one thing, then do all of that thing before going to next”.

CONCLUSION

In summary, our initial work on studying how physicians undertake order writing for hospital admission found five conceptual themes:

1. Order planning for complex patients organized primarily by problem

2. Order writing alternaings between problem-based and mnemonic-based frameworks

3. Including “default problems” on the problem list

4. Missing interactions across order set sections

5. Jumping between order set sections

Addressing these themes can improve future CPOE systems and we hope that our suggested approaches are a step in this direction.