Validating free-text order entry for a note-centric EHR

Abstract

Electronic Health Records (EHRs) have increased the utility and portability of health information by storing it in structured formats. However, EHRs separate this structured data from the rich, free-text descriptions of clinical notes. The ultimate objective of our research is to develop an interactive progress note that unifies entry, access, and retrieval of structured and unstructured health information. In this study we present the design and subsequent testing with eight clinicians of a core element of this envisioned note: free-text order entry. Clinicians saw this new order-entry paradigm as a way to save time and preserve data quality by reducing double-documentation. However, they wanted the prototype to recognize more diverse types of shorthand and apply default values to fields that remain fairly constant across orders, such as number of refills and pickup location. Future work will test more complex orders, such as cascading orders, with a broader range of clinicians.

Introduction

Electronic Health Records (EHRs) have greatly increased the utility and portability of health information by storing it in structured formats. There are numerous benefits to having patient records in these computer-readable formats including being able to perform targeted searches, exchange data between institutions, and provide real-time clinical decision support. However, current EHRs rely on an aging paradigm of windows, tables, and menus to collect and display this information. Since these elements take significant time and attention to use, providers spend precious moments navigating EHRs when they could be engaging with patients or documenting visits.

Moreover, as health records have moved from paper to computers, the progress note has lost centrality as an instrument of care. Patient information is now scattered across numerous parts of the EHR, strongly divided into structured and unstructured sections. This fragmentation imposes a range of new constraints on physicians’ documentation and is a key cause of errors¹. It can also lead to duplicate work, for instance when an order placed via a drop-down menu must later be manually documented in a note.

The ultimate goal of our work is to develop a novel EHR paradigm around interactive notes that unify entry, access, and retrieval of structured and unstructured health information. This study tests one feature of this note-centric vision: free-text order entry. The particular research questions we sought to answer with this study were: (i) what types of shorthand do clinicians use when placing free-text medication orders? (ii) what types of functionality do they expect from computerized free-text order entry? and (iii) do they see free-text order entry as a useful addition to their note?

Background

Moving medical records into a computational medium has conferred numerous benefits on the medical community including remote access to patient records and real-time clinical decision support². EHR use has also been associated with higher quality of care 3. However, there has long been recognition that by separating information into discrete categories displayed on separate pages, EHRs hinder clinicians from making inferences across categories 4, such as when referencing lab results to determine why an order was placed.

Despite this fragmentation, clinical notes remain a central component of EHRs. They unify the record by describing a patient’s history, interpreting test results, and justifying care plans. They are also where clinicians spend much of their time. Our prior research on outpatient visits across several Veterans Affairs hospitals found that clinicians spent the majority (58%) of their EHR time in notes⁵. Despite being central to clinical care, notes are increasingly difficult to write as they need to include large amounts of structured data, already documented elsewhere in the record, to meet regulatory and billing requirements and defend against lawsuits⁶.

Researchers in biomedical informatics and human-computer interaction have sought to reduce this duplicate work by making it easier to carry information across notes⁷ or copy structured information into a note⁸. Moreover, some EHR vendors have started to provide means for quickly importing information such as vital signs using in-note commands, such as Epic’s Smart Phrases⁹. This prior work has focused on pulling structured information into the progress note, but little attention has been given to the reverse conversion of creating structured data in real-time from note text. This concept is intimately connected to previous work on information retrieval, so much so that after testing the in-note information retrieval system in⁸, one participant remarked that they would “like to place orders for medications and tests” from within the note. The Regenstrief Institute has begun to explore this direction with Medical Gopher, an EHR that recognizes medication keywords written in a note and suggests placing related orders¹⁰. However the Medical Gopher does not recognize order details such as dosage or schedule and clicking on the suggested order takes users to a separate screen with menus to specify these details.

One of our objectives in creating an interactive progress note is to let clinicians take actions, such as placing an order, as they are creating their note. Towards that objective, we developed and tested a prototype note editor to observe how clinicians would expect to place medication orders using free-text. This research follows iterative prototyping methods common in human-computer interaction in which early prototypes are tested with potential end users to identify usability issues and validate that the design addresses the identified need¹¹.

ActiveNotes Prototype

To test clinicians’ methods of free-text order entry, we developed ActiveNotes (Figure 1). At its core, ActiveNotes is a note editor that lets clinicians create and edit notes. Like many EHR note editors, there are no facilities for rich-text formatting such as bolding or changing font size.

Figure 1.

The ActiveNotes Prototype in use. The center white area is the note editor. When a user types “#med”, a medication ordering dialog appears, allowing free-text orders to be placed. Text typed in this dialog is automatically mapped to structured information as indicated by the highlighted components above the medication order (i.e.. ‘Med’, ‘Strength’, ‘Schedule’, ‘Days’, and ‘Refills’ in the example). The patient information to the left of the note is static and does not reflect note content.

ActiveNotes’ core strength is in its ability to parse semi-structured text into full medication orders. To start an order, users type the tag “#med” which opens a small order specification dialog on top of the note (Figure 2). This window provides a search bar where clinicians can type their order. ActiveNotes expects the medication and dose information to be entered first and provides auto-completion for these fields based on an underlying medication database built around RxNorm and NDF¹². As text is entered in the search field, it also appears in the note wherever the #med tag was invoked. After entering the medication and dose fields, the order is complete enough for checkout and clinicians can hit their “Enter” key to finish the order. Alternatively, they can specify other aspects of the order such as form, route, schedule, refills, etc. As each part of the order is recognized, the dialog highlights the components’ corresponding label in blue in the popover window. Users can also access a standard order specification drop-down menu by clicking the triangle to the right of the search box, though they are not encouraged to use this feature.

Figure 2.

Inline order-entry. (a) Autocompletion of a medication name (b) When ActiveNotes recognizes a field, it highlights a corresponding label above the search box in blue (c) drop-down ordering

After a medication is ordered through the dialog, it can be edited or deleted through an action menu that appears when users right-click or hover over the order text (Figure 3). Anticipating future uses of ActiveNotes, we have developed order specific tasks including refill, reorder, and discontinue that can all be accomplished in a single click. These features were not tested in this study.

Figure 3.

Orders can be modified by hovering over or right-clicking on the order text

Once finished with their note, users can go to the “Order Checkout” page to review and sign any orders they placed (Figure 4). Here, ActiveNotes asks for any additional information needed to fully specify the order, but it also fills in default values for some fields that were not specified in the note, such as pickup location or number of refills. ActiveNotes currently supports only basic medication orders but we plan to expand it in the future to support more complex medication orders, such as weight-based orders, as well as orders for lab tests, radiology, and consultations.

Figure 4.

ActiveNotes’ Checkout screen where orders can be reviewed and edited before being signed. The red box highlights missing information. Users can either enter it here, or go back to the note to complete the order.

ActiveNotes is build on top of standard web technology stack and medical ontologies (Figure 5). The user interface was developed in HTML5/CSS/JavaScript and exploits open source libraries to enable more complex interactions and UI elements. The middle layer exploits standard servers and frameworks such as SQL, REST and PHP to recognize the order fields and provide autocomplete suggestions in real-time.

Figure 5.

ActiveNotes is build on standard web technologies and medical ontologies.

Evaluation

We tested ActiveNotes through a formative usability test with eight clinicians from San Diego’s Veterans Affairs hospital and UC San Diego Medical Center. The clinicians’ specializations included internal medicine (3), infectious diseases (4), and surgery (1). Our participant pool included both inpatient and outpatient practitioners, with two fellows and six senior clinicians. Each test lasted 30 minutes or less. Clinicians were first shown a short video explaining ActiveNotes’ features. Following the task scenario method of usability testing¹³, they were then given two short descriptions of canonical outpatient visits and asked to create a progress note for each visit in ActiveNotes. Each scenario gave basic demographic and assessment information and asked the clinicians to place four new medication orders from within their note. The orders were largely phrased without standard shorthand (e.g. PO, BID) so we could observe the shorthand clinicians naturally used. After completing both progress notes, the clinicians were asked to comment on their experience with ActiveNotes. We tracked how long it took clinicians to successfully place each order, the shorthand they used while placing each order, any unsupported uses of the system they attempted, and their comments on desired features and overall usefulness of ActiveNotes.

Results

Time to Order

Clinicians took, on average, 59 seconds to place each order, though this time varied greatly between clinicians and orders. For example, P3 averaged just 30 seconds to place each order whereas P8 averaged 1 minute 56 seconds. Also, it took clinicians an average of 39 seconds to place an order for “Simvastatin 20mg PO at bedtime daily. 30 tablets to be dispensed with 3 refills” whereas it took 1 minute 17 seconds to place one for “Aspirin 81mg once daily. 100 tablets to be dispensed with 3 refills”. The Asprin order took longer because ActiveNotes did not recognize the “asa” as shorthand for “Asprin”, which delayed several clinicians.

Broadly, time to order depended on whether the clinician initially used shorthand that ActiveNotes did not recognize as pertaining to a particular field. For example, whereas ActiveNotes could parse “#30” as quantity information, it could not parse “Dispense: 30”. The orders that took the longest to complete were those in which clinicians tried to use several different shorthands that ActiveNotes did not recognize and then resorted to using its fall-back dropdown menu (accessible by clicking the triangle to the right of the search box). Since one of our main objectives was to identify the shorthand clinicians naturally use, we did not instruct them on what type of shorthand ActiveNotes was programmed to recognize and let them decide when they wanted to fall-back on the dropdown menu.

Shorthand

Clinicians used a variety of shorthand when placing free-text orders. Representative variants are shown in Table 1. Whereas the medication, dose, form, route and days fields saw little variation, schedule, quantity, refills, and pickup information was entered in a number of different ways.

Table 1:

Shorthand used when placing orders

Information	Shorthand
Medication	Aspirin, asa (abbreviation)
Dose	100mg, 75mcg
Form	tab, enteric coated
Route	po, oral
Schedule	bid, at bedtime, once a day, 1x/day, prn for dizziness
Days	30 days
Quantity	#30, qty 30, Dispense: 30
Refills	rf, no refills, refills 3, one refill
Pickup	pick, mail, to be mailed, pickup at clinic, pickup window

Functionality

As is common with prototypes, our participants wanted ActiveNotes to do more. We particularly value feedback in this context since the expected functionality can guide and inform future development. We uncovered six recurring function requests; the first three regarded time saving assistance, the last three address broader integration with the EHR. The clinicians volunteered these comments and were not asked about them specifically.

Default Values

Five clinicians wanted ActiveNotes to have default values for fields like schedule, refills, and pickup. While ActiveNotes’ refill value defaulted to “3 refills” and pickup value to “Window”, it did not have a default value for schedule (such as AM). Furthermore, there is no way to see when placing an order in the note that ActiveNotes will fill in these default values on the Checkout screen unless otherwise specified.

Auto-populated Values

Four clinicians wanted ActiveNotes to automatically populate form and quantity information whenever possible. For example, ActiveNotes should automatically mark ‘Tab’ if the specified medication and dose can only come in a tablet form. Also, after entering a schedule and duration of “BID for 30 days”, ActiveNotes should assign a quantity of 60.

Order Entry Invocation

Four clinicians wanted a simpler way to invoke the order entry window than the “#med” syntax. Clinicians either wanted to type out the full order and then hit a hotkey to tell ActiveNotes to parse the preceding text, or only wanted to invoke the “#med” once per note and fill out multiple orders in a row.

Information Retrieval

Two clinicians wanted ActiveNotes to support rich information retrieval such as “retrieve most recent colonoscopy”.

Additional Orders

Three clinicians expected ActiveNotes to handle additional order types including radiology, labs, and consults.

Additional Parsing

One clinician wanted ActiveNotes to be able to parse other note text, such as active problems, and save it to the appropriate part of the EHR.

Usefulness

Clinicians saw the current implementation of ActiveNotes as useful in two distinct ways. First, they thought it was useful to not have to switch between sections of the EHR to document and place orders. As one clinician remarked:

“I love not having to go out of the screen [to place orders]” - P1

Secondly, clinicians saw directly linking orders to documentation as a way to avoid overlooking orders:

“The thing that I really like about this is… having the order directly tied to the documentation of that order in the note. The issue I run into sometimes is that I write my note and I’m waiting to do my orders and then I have to… make sure… all the things I said I was going to do in the note I actually order.” - P3

More broadly clinicians saw value in using interactive notes to populate the EHR with structured information:

“If this had the ability to take everything in the note and just automatically download it… if you just did like #activeproblems from the note it would just put it all into that section… that would dramatically improve efficiency because there’s a lot of that duplication that we’re currently doing. - P7

This is opposed to the current model of importing structured information into the note:

“As it currently stands, it’s the opposite. You have to first put in everything in the active problem list or medication list, and then you can populate it into the note by using smart keystrokes, but it would be nice when you’re initially seeing someone to not have to write the whole note and then repeat everything” - P7

Discussion

Our first research question asked what types of shorthand clinicians use while placing free-text medication orders. Whereas the shorthand for some fields was standardized (e.g. ‘asa’ for aspirin, ‘mg’ for milligrams) fields such as schedule and number of refills were specified in a number of different ways. From this observation, we realize that it will be important for interactive notes to embrace a rich set of terminology that goes beyond current ontologies, such as RxNorm, to include terms for schedule, refills, and pickup. As a first step, we propose iteratively testing ActiveNotes through large-scale online deployments to capture a broad range of shorthands.

Our second research question asked what types of functionality clinicians expect from a computerized free-text order entry system. The six recurring functionality requests fell into two categories: time saving assistance and broader integration with the EHR. Along the lines of time saving assistance, clinicians wanted to specify their order with as little typing as possible. This principle can be seen in their desire to invoke the ‘#med’ dialog only once and place multiple orders in a row. It can also be seen in their desire for ActiveNotes to recognize when a particular dose of a drug only comes in one form, or to automatically calculate quantity given a prescription’s schedule and duration. It is worth noting that those fields that clinicians wanted the most assistance with were also those that exhibited the most varied shorthand. Clinicians also wanted ActiveNotes to be more broadly integrated with the EHR. This included both extending the types of orders it recognized to include labs, imaging, and consults and extending ActiveNotes’ parsing to cover other types of structured information such as active problems and family history.

Our final question asked if clinicians saw free-text order entry as a useful addition to their note editor. They saw free-text order entry as being useful in a number of ways including being a potential time saver, less distracting than form-based input, and requiring less navigation compared to current EHRs. Furthermore, they thought it could save them from needing to enter information twice, once in a structured format, and then in an unstructured format. Textual entry may also require less attention than form-based or drop-down based entry, letting providers focus more on their patients if they choose to document during the patient encounter.

There are also potential technical benefits to letting clinicians tag note content as structured data compared with post-hoc Natural Language Processing (NLP). First, it enables the parser, whether it is looking for orders or a problem list, to use a more targeted ontology for recognizing terms. Second, it adds structure to the note itself by marking which parts of the note refer to medication, conditions, and so on. Such tagging could assist with later NLP or the development of richer note interactions related to targeted search, filtering, and highlighting. Thirdly, order entry dialogs could be expanded to include real-time clinical-decision support, potentially catching errors and drug-drug interactions while notes are being written on rounds, not while orders are being placed later.

Looking towards future designs, we observed that free-text entry of structured information is not a familiar interaction paradigm as compared to menu and drop-down driven user interfaces. While our subjects were not constrained in what they typed, some had difficulty understanding the full functionality of the system. One participant did not grasp that ActiveNotes would let him document and place orders at the same time. Instead, he wrote each order in the note twice, first with fairly standard shorthand (e.g. po, #30) while “documenting” and then with less standard shorthand when “ordering” (e.g. Dispense: 30). Beyond developing a robust interactive note, it will take time and experience before some clinicians are comfortable with this new note-creation paradigm.

Pilot tests before this study also revealed that for free-text entry to work, careful thought has to be given to its exact mechanics. For example, which keys users press to complete an order, how they select an auto-completed phrase, and how the system shows that it recognizes part of the order all need to fit clinicians’ expectations.

Conclusion

This study takes a step towards our ultimate objective of developing an interactive progress note. We tested one critical feature of such a note, free-text order entry, and found that (i) clinicians use a variety of shorthand when placing free-text orders, particularly when specifying schedule, pickup location, and number of refills, (ii) clinicians want to specify a minimum number of fields and rely on default or auto-populated values to fill in routine information, and (iii) clinicians see free-text order entry as a useful addition to their progress notes with the potential to save time and ensure orders are not overlooked.

In the future, we plan to test ActiveNotes with a broader range of clinicians across locations and disciplines, letting us observe a wide range of shorthands so we can develop a more robust order parser. Since ActiveNotes is built on web technology, we can easily deploy it on a larger scale and track visitors’ interactions with the site. Secondly, we plan to extend ActiveNotes’ ordering capabilities to include more complex medication orders, such as weight-based orders and cascading orders, as well as consults, labs, and imaging. Finally, we plan to test ActiveNotes on mobile platforms, exploring the usage of dictation for text input.

We posited that introducing a new interaction technique based on within-note free-text data entry could help solve a variety of problems that EHR users face. To investigate this claim, we designed and tested a novel note-centric approach to medication order entry and found that while work remains before being deployed in a live EHR, our in-line free-text approach was validated by representative end-users. By considering the note as the central element of the EHR and incorporating interactions and operations that typically span multiple parts of EHRs, we have an opportunity to transform documentation from being a complicated and time-consuming clerical task to a more natural interaction with the EHR that is closer to both the clinician and the patient, supporting increased efficiency, fewer errors, and ultimately, better health outcomes.