Software Prototyping

Summary

Background

Health information exchange (HIE) between Poison Control Centers (PCCs) and Emergency Departments (EDs) could improve care of poisoned patients. However, PCC information systems are not designed to facilitate HIE with EDs; therefore, we are developing specialized software to support HIE within the normal workflow of the PCC using user-centered design and rapid prototyping.

Objective

To describe the design of an HIE dashboard and the refinement of user requirements through rapid prototyping.

Methods

Using previously elicited user requirements, we designed low-fidelity sketches of designs on paper with iterative refinement. Next, we designed an interactive high-fidelity prototype and conducted scenario-based usability tests with end users. Users were asked to think aloud while accomplishing tasks related to a case vignette. After testing, the users provided feedback and evaluated the prototype using the System Usability Scale (SUS).

Results

Survey results from three users provided useful feedback that was then incorporated into the design. After achieving a stable design, we used the prototype itself as the specification for development of the actual software. Benefits of prototyping included having 1) subject-matter experts heavily involved with the design; 2) flexibility to make rapid changes, 3) the ability to minimize software development efforts early in the design stage; 4) rapid finalization of requirements; 5) early visualization of designs; 6) and a powerful vehicle for communication of the design to the programmers. Challenges included 1) time and effort to develop the prototypes and case scenarios; 2) no simulation of system performance; 3) not having all proposed functionality available in the final product; and 4) missing needed data elements in the PCC information system.

1. Background

Poisonings are a leading cause of unintentional injury deaths in the United States.[1] Poison control centers (PCCs) are 24-hour resources for poison information, clinical toxicology consultation, and poison prevention education. PCCs have been shown to reduce costs, unnecessary emergency department (ED) visits, and patient length of hospitalization [2, 3]. PCC specialists in poison information (SPIs) interface with healthcare professionals over the telephone and provide recommendations in the treatment of poisoned patients [4]. However, this reliance on telephonic communication can result in miscommunication, data loss, and error, especially in these settings of high communication loads, frequent interruptions, disaster scenarios [5], and/or routine surges in calls [6]. Health information exchange (HIE) has been proposed to improve information sharing between PCCs and EDs, thus improving decision-making, continuity of care, and reducing errors [7, 8]. Beyond just sending and receiving messages, this type of HIE requires careful consideration of the workflow and information needs of ED care providers and SPIs, as well as optimal display of the information to end-users, especially since system usability is a key factor in HIE use and success [9–12], and poor design can result in medical errors [13–16]. Currently, PCC information systems are not designed to facilitate HIE with EDs; therefore, we are developing specialized software to support the integration of HIE within the normal workflow of the PCC. While many development projects tend to design the technology at the programing code level, thereby removing the end user from the design process [17], we wanted to put the emphasis on the end user, in order to develop an attractive, effective, and efficient user interface combined with complex back-end functionalities [17]. This dashboard would then allow SPIs in the PCC immediate access to patient information from the ED, (e.g., general status, vital signs, progress notes, laboratory tests) through asynchronous communication. It would also allow SPIs to send consultation notes with patient assessment and recommendations to providers in the ED. The goal is to supplement telephonic communication and reduce the burden of communicating information that could be exchanged electronically.

2. Objective

The purpose of this report is to describe how we refined user requirements for a HIE dashboard to be used by SPIs in the PCCs through rapid prototype development, using a user-centered design approach.

3. Methods

In order to facilitate a user-centered design, we used rapid prototyping in iterative cycles that included feedback from target users on an interactive prototype of the proposed HIE software. A prototype is an example or a model of the intended end product, and ranges from low-fidelity quickly sketched drawings on paper, to high-fidelity interactive designs that look like the finished product. Low-fidelity prototypes are intended to show high-level design concepts, and do not represent details or what the final design of the product will look like. Conversely, high-fidelity prototypes are intended to closely resemble the user interface and functionality of the final product. There is a spectrum between low- and high-fidelity prototypes, which depends on how closely the prototype resembles the final product. In this project, we chose rapid prototyping with low-fidelity prototypes for the initial design process, then moved to usability testing with high-fidelity interactive prototypes. This rapid prototyping process helps to identify and solve major usability or interface problems quickly and cheaply, without having to rewrite the programing code [18], and allows us focus on the end-user design before spending efforts on the technology and programing code [17].

The participants were researchers and SPIs at an accredited regional PCC. Previously, we elicited user requirements for HIE via rigorous user interviews and analysis of PCC call recordings [4]. Using the list of requirements, we designed low-fidelity sketches by drawing ideas and designs out on paper with iterative refinement by the authors to define dashboard functionality. After a basic layout and functionality was established, we continued with iterative designs of an interactive high-fidelity prototype using the web-based interactive prototyping software HotGloo (HotGloo GbR, Hamburg, Germany). This interactive design consisted of a set of screen design images, hyperlinked together in order to imitate the look, feel, and navigation of actual software (i.e. clicking on icons led to different screen images). While users could navigate through the screens with the appearance of using functional software, the design prototype was not functional software and users would not read or write any actual data. Next, we used expert-based methods of heuristics and usability guidelines to uncover potential usability problems with the design, such as simplifying and reordering the information displayed [19].

At the end of several more iterations, we conducted a formal scenario-based usability assessment with additional end users (experienced SPIs at the Utah PCC) that were not involved with the development process. A patient case vignette was developed based on a modified PCC case of snake envenomation with a subsequent ED encounter and inpatient stay [20, 21]. The case was then checked by SPIs to make sure that the case presented was representative of a typical poison control case and included the different tasks that are performed by SPIs (e.g., consultation, follow-up). SPIs were asked to think aloud while attempting to accomplish specific tasks related to the case vignette [22]. Sessions were recorded via screen capture with the usability testing software TURF [23]. In addition to the think-aloud techniques, a researcher (pharmacist and medical informatics fellow) observed the session, took field notes, and solicited additional open-ended comments and recommendations. After the session, the SPI filled out an online post-session interview using the System Usability Scale (SUS) [24]. SUS is a 10-question, 5-point Likert scale that provides subjective measurement of system usability. We anticipated that the input from approximately four users would be able to identify most usability problems during the iterative development [25]. A general overview of the methodology is presented in ► Figure 1.

Fig. 1.

Iterative development

4. Results

Overall, feedback from the users was very positive, and they were eager to contribute their ideas for dashboard improvement and additional functionality. Because prototype development was relatively simple, we were able to easily incorporate recommendations and feedback, and then continue to test with additional users. This type of iterative development helped users become engaged in the project during early stage design, and promoted willingness to assist with future testing and design.

We received three SPI survey results, showing better than average usability [26]. Overall, users strongly agreed that they could learn the system quickly, and had high satisfaction with accessing lab values, vital signs, medication lists, and progress notes in one location. The early rounds of testing showed the need to simplify the dashboard display by removing extraneous icons, excessive information, and unnecessary tabs. The users identified inconsistencies and missing data in the prototypes, along with preferences in how information was arranged on the screen, and defined additional information needs such as patient date of birth and medical record number (MRN) for patient matching. User feedback was then incorporated into the design, and helped make the dashboard more intuitive and user friendly [26]. ► Figure 2 and ► Figure 3 show an evolution of major revisions to the dashboard display and patient summary prototypes. Additionally, not shown, when users click on the outgoing message box, they are provided a prepopulated template to structure an outgoing message to the ED, using the HL7 clinical document architecture (C-CDA) consult note template [27]. After we came to a stable prototype through testing, we gave the interactive prototype to our developers to program the actual software and finish user-interface design details.

Fig. 2.

Dashboard display prototypes A though D. Figure 1D represents the final dashboard prototype. Note: there were many other versions, but these represent the major revisions after user feedback.

Fig. 3.

Patient summary prototypes A though D. Figure D represents the final Patient summary prototype. Note: there were many other versions, but these represent the major revisions after user feedback.

5. Discussion

In this paper, we describe how we used software prototyping to refine user requirements and design an HIE dashboard. We found this approach to be useful because it provided a method to engage end users early on in the project, and we could watch ideas quickly come to life, then test and refine the display with end users. It was also beneficial to have subject-matter experts heavily involved with the design and development of the prototype [11], as well as providing flexibility to make rapid changes in iterative testing. Using the interactive prototype helped us to quickly identify usability shortcomings, and improve the dashboard design [28]. Then, the interactive prototype was used as an example of the system specifications to provide to the developers to program and work on the complex backend functionality. Typically, when programmers code, they may encounter problems, or be forced to make personal ad-hoc decisions on-demand about detail in the design and program code. An interactive prototype helps to communicate the final intended design and functionality, in a way that is quicker, easier, and more effective than supporting text descriptions. Even experienced programmers tend to rely more so on pictures in design guidelines than on the supporting text when making decisions on coding details [29]. Clear communication can help to save time and money by not having to rework or fix misunderstandings in the code later in development or usability testing [30]. If having a picture is worth a thousand words, then having an interactive picture is worth more. The prototype we developed facilitated rapid communication of ideas and established a common understanding of design and intended functionality, without needing to do software programing.

There are many options for developing prototypes or wire-frame designs, even for mobile devices [31, 32], and there exists many different methods for usability testing [22, 33, 34]. There does not appear to be one best answer for which prototyping software to use or usability testing method, and the selection of each depends on individual use cases, project requirements, and researcher experience. Successful use of prototypes really depends on properly eliciting user requirements early on, using appropriate tools, and being prepared to throw away designs as you work through several iterations [18, 35]. Additionally, rapid prototyping is intended to be short-term, so no substantial effort should be spent on each design cycle. Additional guidance for using high-fidelity prototypes are also described elsewhere [30].

5.1 Benefits

Benefits we observed by rapid prototyping include:

1. early construction in the development cycle with the ability to spread the development costs across the project;

2. no upfront large investment in time or money helps to finalize the system and user requirements [36];

3. early visualization of the possible designs promotes innovation [37];

4. and the prototype provided a power vehicle for communication of the interface design to the programmers [29].

5.2 Challenges

Some of the challenges and tradeoffs that we discovered relate to the development time required by high-fidelity prototypes and case scenarios. Low-fidelity prototypes are faster, cheaper, and could start out early with paper and pencil. There are many pros and cons related to using low- or high-fidelity prototypes [30], but we found it to be very useful to start with a low-fidelity prototype, and iteratively evolve it into a higher-fidelity prototype. Perhaps the greatest challenge is that the prototype appears to be a final product, which does not show system performance and some of the complex functions may not be feasible in the final product. Required changes from the prototype to the final product could affect usability or user satisfaction, as the absence of some functionality in the final product may disappoint users. We attempted to mitigate this by emphasizing with the users that the prototype represents the ideal software, and may not include all of the features and functionality represented. Additionally, we analyzed the current systems and message contents, and included domain experts to better understand the technical capabilities and limitations of the systems. Additionally, we found in this project that not all of the data elements needed by users are captured in the PCC information system (such as date of birth or MRN), and there is not currently a standard terminology for exchanging PCC data [38].

5.3 Study limitations

There are several limitations to this study that are worth mentioning. First, we had a limited sample size of 3 participants in the case-based usability testing. Yet, rapid and iterative design is not feasible with larger sample sizes. A full usability assessment is planned for once the software is developed. The second limitation has to do with dual process theory [39–41]. Automated thought process (“System 1”) uses a network of associations for rapid pattern-matching, while deliberate thought process (“System 2”) is a slower, rule-based, system for active reasoning. Experts tend to function with System 1 level thought, which is very difficult to articulate and verbalize. Additionally, most people stop talking when they are thinking or processing complex information, also making it difficult for experts to verbalize their goals and tasks [39, 42]. To address this limitation, we also used direct observation, screen capture, recorded PCC calls, interviews, and post-session questionnaires in obtaining system requirements and in the usability testing. Third, the usability testing sessions were carried out in the PCC; however, they were not done while the SPI was multitasking or taking other calls. Ideally, usability testing should be done in the intended healthcare environment, while the user is doing a challenging task [34]. Finally, usability testing included only one case vignette, a non-drug poison exposure. This served our goal of eliciting rapid feedback to support initial design. Use of multiple cases and different scenarios will help to identify additional usability problems and improve the system design [43]. Functional software is currently under development and formal usability testing using actual PCC cases is planned. Future usability testing will entail live documentation of PCC cases in the context of real-world operations. This will be accomplished by pairing a specialist taking live calls with a second, inactive specialist. The second specialist will participate only in usability testing of the software, using tandem headphones to follow and document live cases.

5.4 Context and future work

The rapid prototyping described in this case study was conducted as part of a larger effort to develop and evaluate a workflow-integrated HIE process for PCCs and Eds [27, 38, 44]. U.S. PCCs currently lack software and informatics tools to engage in standards-based health information exchange. We are developing this open source software to enable U.S. PCC HIE. The software is designed to facilitate both unidirectional and bidirectional HIE. In other words, PCCs can use the software to send a single document, an HL7 CCDA consultation note to a recipient ED or regional HIE organization. With additional configuration and sufficient technical capability of ED information systems, it can be used to facilitate ongoing communication between the ED and PCC during active management of a poison exposure case, partially replacing and supplementing telephone communication. We are currently studying outcomes of the latter approach, which involves a high degree of workflow integration in both the ED and PCC setting, in collaboration with Intermountain Healthcare in Salt Lake City, UT.

6. Conclusions

We found the use of a rapid prototyping method for eliciting user requirements and for designing an information dashboard engaged users, and it helped design a better user-centered display. We started with low-fidelity prototypes and iteratively designed an interactive high-fidelity prototype that served as a software development specification for our programming team. The software is currently under development and more detailed usability testing is planned in order to both refine and evaluate the software.