Patient information organization in the intensive care setting: expert knowledge elicitation with card sorting methods

Abstract

Introduction

Many electronic health records fail to support information uptake because they impose low-level information organization tasks on users. Clinical concept-oriented views have shown information processing improvements, but the specifics of this organization for critical care are unclear.

Objective

To determine high-level cognitive processes and patient information organization schema in critical care.

Methods

We conducted an open card sort of 29 patient data elements and a modified Delphi card sort of 65 patient data elements. Study participants were 39 clinicians with varied critical care training and experience. We analyzed the open sort with a hierarchical cluster analysis (HCA) and factor analysis (FA). The Delphi sort was split into three initiating groups that resulted in three unique solutions. We compared results between open sort analyses (HCA and FA), between card sorting exercises (open and Delphi), and across the Delphi solutions.

Results

Between the HCA and FA, we observed common constructs including cardiovascular and hemodynamics, infectious disease, medications, neurology, patient overview, respiratory, and vital signs. The more comprehensive Delphi sort solutions also included gastrointestinal, renal, and imaging constructs.

Conclusions

We identified primarily system-based groupings (e.g., cardiovascular, respiratory). Source-based (e.g., medications, laboratory) groups became apparent when participants were asked to sort a longer list of concepts. These results suggest a hybrid approach to information organization, which may combine systems, source, or problem-based groupings, best supports clinicians’ mental models. These results can contribute to the design of information displays to better support clinicians’ access and interpretation of information for critical care decisions.

Introduction

Technology in healthcare is pushing the boundaries of what was once thought possible. This level of sophistication comes at a cost of producing overwhelming amounts of data. Often, relevant data are dispersed throughout the electronic health record (EHR) in a way that requires clinicians to engage in low-level tasks of aggregating and organizing data before synthesizing and decision-making. Increasing clinicians’ cognitive workload with low-level tasks can contribute to patient harm and provider burnout.^1–3 Critical care, in particular, is data driven and generates more information in less time than other care settings. Information display research in critical care has identified a need for improved information prioritization and organization.⁴ Displays that present information aligned with provider mental models may reduce the need to perform low-level cognitive tasks, improve high-level cognitive support, and ultimately improve patient outcomes.⁵

A clinical concept-oriented view is an organization schema that groups data in ways that are meaningful to patient care decisions, from the clinicians’ perspective.⁶ This type of view allows clinicians to see related information together, which supports interpretation and understanding. Studies indicate that clinical concept organizations may reduce information overload⁷ and are preferred by clinicians.⁸ Creating meaningful clinical concept groups, however, is inherently difficult due to the non-explicit relationship of individual concepts to various clinical constructs; moreover, relationship links between clinical concepts may be context or patient-specific and vary by provider training and preferences.

Patient information organization has been explored by diverse methods and rigor, which may contribute to the disparity of conclusions made. Current EHRs primarily organize information in a source-oriented view,⁶ grouping data based on the source from which the information is gathered such as monitoring devices, laboratory results, medication orders, and clinician notes. Others, however, have delved into clinical-concept organizations such as disease,⁷ context,⁹ and physiological systems.^10,11 These organizations group together have information from different sources and allow clinicians to simultaneously see clinically relevant relationships such as effects of treatment on clinical problems. While some clinical-concept views have shown favorable outcomes,^7,11 methods used to identify optimal concept groupings appear to be fairly informal.

Several studies in critical care have focused on improving the organization of patient information.^12–18 Miller and colleagues^19,20 applied work domain analysis principles to develop a framework for organizing critical care information. This framework included four functions: (1) neurologic, (2) circulation and communication, (3) fluid and electrolyte (renal), and (4) gas exchange (respiratory). They developed an electronic display prototype that included these groups. Moreover, the researchers highlight that treatment variables (e.g., medications) should be grouped with associated goals and patient parameters (e.g., blood pressure) to reveal cause-effect relationships. The AWARE dashboard was developed to provide rapid access to key critical care data; these data were organized in groups to support information interpretation.^9,10,21 The parsimonious dataset used for the display was obtained from clinician information use surveys upon patient admission to the critical care unit.²¹ These data were grouped into ‘key patient centered concepts’ by subject matter experts on the design and research team.⁹ The organizational frameworks from these separate studies overlap with respect to several key functions (neurology, cardiovascular, renal, and respiratory), but there also are differences regarding both the overall organizational structure and the assignment of data within clinical concept groups. There is no clear consensus regarding the best organizational schema to align with clinicians’ mental models and information needs.

There remains a need for robust analyses to uncover critical care providers’ mental models related to organization of patient data. The primary aim of this study was to understand the high-level processes and organization schema in critical care. Given a set of high-frequency and high-priority data for typical critical care patients, how would clinicians organize that information to evaluate and make decisions about patient care? This type of concept map would provide a useful organization structure for applications that range from single page critical care dashboard displays^11,22,23 to conventional multi-page patient information displays,²⁰ to multi-patient summary displays.^3,4 We employed card sorting, concept mapping, and mixed-methods analyses²⁴ to formally elicit critical care clinicians’ mental models with respect to patient information organization.

Methods

Study Design and Materials

We used a mixed-methods approach that included two card-sorting exercises: a Delphi (Modified Delphi) sort with 65 concepts and a conventional open sort with 29 concepts. Card-sorting is a user-centered design method frequently used to understand concept relationships and translate information into concept maps.^25–27 Card-sorting results can be used to support the design of information displays that adhere to users’ mental models. Participants (representative users) are given a list of concepts (cards) to sort into meaningful groups (constructs).

In a Modified Delphi Sort (Delphi sort), the first participant sorts individual concepts into groups and subsequent participants review and edit the results.^28,29 We selected the Delphi sort method because the problem space of organizing patient data is large and Delphi sorts are expected to minimize participant effort and achieve solutions quickly. The Delphi sort has produced equivalent to superior results compared to an open sort;²⁸ however, the Delphi sort has not received the same level of experiential evaluation as conventional card sorting approaches. Due to relatively limited literature on the Delphi methodology, we were concerned the initial sort may bias the final solution. Thus, to provide additional evidence of validity and reliability, we chose to conduct three parallel unique Delphi sorts which we labeled as 3 ‘streams’. Each stream included the same concepts but involved different participants. The results of three unique solutions could then be directly compared. Two of the three streams began as open sorts, with no pre-named category headings. One of the streams was started as a hybrid sort in which we provided 14 pre-defined category names, generated from pilot test sorts. Participants could use those categories, delete them, and/or add new category names. Participants were also asked to prioritize both categories and information within categorize. Trello Boards (available at https://trello.com) were used for this activity.

A conventional open sort was then conducted on a narrower set of 29 concepts. Participants were asked to sort concepts into groups, name the groups, order groups by priority, and order concepts within group by priority. The open card sort was conducted using OptimalSort.³⁰

Concept Selection

We targeted 30 to 60 key concepts to represent the critical care environment.³¹ Critically ill patients generate far more than 30 to 60 unique data elements; therefore, we set criteria to include:

• concepts that are frequently used in critical care decisions,

• not only single time point univariate data elements but also novel concepts such as trend³² or clinical decision support³³ data elements, and

• the breadth of data sources (e.g., medications, radiology, physiological data, and notes) used in critical care.

Initially, 200 concepts were obtained from published articles^{10–12,14,21} that identified information relevant to critical care decisions. A subject matter expert (BM) categorized these concepts into approximately 10 source-based groups (e.g., laboratory, radiology, medications, and demographics). By selecting frequently used data and representative data elements from each source-based category, the dataset was reduced from 200 to 78 concepts. To confirm the findings and refine the dataset, we pilot-tested an open sort with 5 critical care physicians. The pilot-tests were conducted as a semi-structured interview. Participants were encouraged to think-aloud during the sort and to order the concepts by priority, comment on specific concepts, and evaluate the representativeness of the concept set. From the pilot test findings, we generated a broad set of 65 concepts for the Delphi sort. Then, informed by grouping and prioritization information from preliminary Delphi sort findings, we selected a narrow set of 29 concepts for the open sort activity. The narrow set focused on high-priority current-stay patient data that may be relevant for a parsimonious overview or current status display. The broader set was to provide insight into a larger hierarchical structure of the concepts.

Setting and Participants

The study protocol and research methods were approved by Institutional Review Boards (IRBs) at all 3 collaborating organizations (Duke University, University of Utah, and Trinity Health). Due to minimal participant risk and anonymous participation, a Waiver of Documentation of Consent was approved that allowed oral consent from participants. We recruited pilot sort and initial Delphi sort participants by convenience and snowball sampling from study sites. Subsequent Delphi sort and open sort participants were recruited from a booth at the Society of Critical Care Medicine (SCCM) 46th Critical Care Congress conference. Pilot sorts and initial Delphi sorts were conducted via an online conference room, and the remaining sorts were conducted in person, at the SCCM conference, with a laptop computer. The Delphi sorts were completed on days 1 and 2 of the conference. The findings (categories and prioritized order of data elements) from these were then used to select 29 high priority and representative concepts for the Open Sort on conference day 3. Prior to the exercise, participants responded to a brief survey describing their professional role, care setting, and years of experience (Table 1). Participants were required to have advanced training or experience in critical care. Although we primarily targeted providers who make patient management and treatment decisions, we included a small sample of nurses and other clinicians to provide an opportunity to identify important differences in their responses. Each participant was directed to place cards into groups at their discretion. Researchers were available for questions about use of the software. While Tullis and Wood recommend a sample size of 20 to 30 participants for conventional card-sorts, they attained correlation coefficients greater than 0.8 with as few as 8 participants.³⁴ Because we were using multiple methods to achieve our aims, we targeted 15 participants for the open sort activity. Sample sizes for Delphi sorts are less well-established, although Paul describes sample sizes of 8 to 10 as ‘typical’.²⁸ Thus, we targeted 8 participants for each stream of the Delphi sort.

Table 1.

Characteristics of Participants in Card Sorting Activities. All Participants Completed Only One Sorting Activity. Role and Experience is Provided for Participants Included in Analyses

	Delphi sort			Open sort
	Stream 1	Stream 2	Stream 3
Total participants	8	9	9	16
Excluded participants			1	5
Total included in analyses	8	9	8	11
Clinician role
Attending Physician	4	6	8	8
Resident Physician	1			2
Physician Assistant or Nurse Practitioner	1	1		1
Nurse	1	1
Not reported or other	1	1
Critical care experience in years
0-2	2	2	2	3
3-6	1	2	4	3
7-10		1		1
10+	4	4	2	4
Not reported	1
Participants with informatics experience	2	3	2	3

Analysis

The Delphi sort analysis was a descriptive comparison of similarities and differences across the three unique solutions. For the open sort, OptimalSort generated a concept similarity matrix that indicates how frequently every pair of concepts is sorted together as a percentage. We analyzed the results of the open sort using hierarchical cluster analysis (HCA) and factor analysis (FA). Both analyses were completed using MiniTab 18. HCA is frequently used to analyze card sort results.^35,36 HCA uses the similarity (or distance) of concept pairs to generate a dendrogram depicting the frequency of grouped concepts by different participants (Figure 1). For the HCA, we used Mini Tab Cluster Observations with Complete Linkage method and Euclidean Distance measure. FA supports identification of overlapping concepts across categories (factors) that are highly associated with more than one factor.²⁷ The FA was completed using the similarity matrix as the variance matrix. We used Varimax criterion³⁷ with a 12 factor parameter to calculate concept loadings. Finally, we calculated the percent agreement comparing the HCA results to the FA results, and the FA results to the aggregated Delphi solution. The percent agreement between the HCA and FA was calculated by dividing the number of HCA concepts in groups with similar labels by the total FA concepts. For this calculation, the Cardiovascular and Hemodynamics label of the HCA was split into two separate categories. The percent agreement between the FA and Delphi concepts was calculated by dividing the possible number of FA concepts in groups with similar labels by the total possible Delphi concepts. Imaging was not included in this calculation.

Figure 1.

Flow diagram of concept selection and card sort tasks. HCA, Hierarchal Cluster Analysis; FA, Factor Analysis.

Results

Delphi Sort of 65 Critical Care Concepts

The three initial sorts required between 30-40 minutes and subsequent review and refine sorts took 10 to 15 minutes to complete. Twenty-six critical care clinicians participated in the three streams of the Delphi sort. One participant was excluded from stream 3 because of failure to follow instructions. All 3 streams were initiated by a provider with some experience in informatics research. Stream 1 was initiated as a hybrid sort by a critical care nurse practitioner, stream 2 was initiated as an open sort by a cardiovascular critical care attending, and stream 3 was initiated as an open sort by a surgical critical care attending. The initial sort and final sort solutions for each of the 3 streams are provided in the attached Supplementary Appendix.

The final category labels across the 3 streams are shown in Table 2. Although there was agreement with respect to group labels, the concepts in each group varied. Figure 2 is an agreement visualization of concepts for the 3 solutions. One difference between the final streams was whether participants grouped medications, laboratory data, and imaging data into relevant physiologic systems categories. Stream 2, for example, generated a solution with no imaging or laboratory categories; imaging and laboratory concepts were grouped using a systems-based organization (e.g., arterial blood gas pH test under Respiratory and brain MRI under Neurologic). This stream also grouped many medications by system, in addition to including a Pharmacy category. The 3 solutions also varied with respect to grouping patient problems in system-based organization. Stream 2 consistently placed problems in system-based categories, while Streams 1 and 3 placed some problems under clinical notes or problem list and other problems in system-based categories.

Table 2.

Similar Category Names Across Three Streams of the Delphi Sort

Similar category names in 3 streams
1. Clinical notes	Daily Progress/Problem list	Progress note
2. Demographics and history	Demographics	Admission data/demographics
3. Hemodynamics	Hemodynamic monitoring	Flowsheet
4. Respiratory	Respiratory	Pulmonary
5. Neurologic	Neurologic	Neurology
6. Gastrointestinal	Gastrointestinal	Nutrition and GI
7. Infectious disease	Infectious disease	Immunology and oncology
8. Medications	Medications	Pharmacy
Similar category names in 2 streams
1. Cardiovascular	Cardiovascular
2. Renal	Renal
3. Imaging	Imaging
4. Laboratory	Laboratory
Category name unique to 1 stream
1. Hematologic

Figure 2.

Common concept groupings across 3 streams. Elements combined in dark gray boxes were grouped together in all 3 streams. Elements combined in lighter gray boxes were grouped together in 2 of 3 streams. A single item in a dark gray box means that concept was not grouped consistently with any other item in at least 2 of the 3 streams (placement of those items in the figure corresponds to only 1 of 3 possible positions and is, therefore, only loosely meaningful). Labels to the left and right roughly correspond to labels provided by participants.

Each stream had different solutions for managing the large quantity of data relevant to cardiovascular or hemodynamic monitoring. Stream 1 included a flowsheet category that combined common cardiovascular and respiratory data. A separate cardiovascular category grouped cardiovascular problems, interpretations, and rhythm or cardiovascular function concepts. Stream 2 included both hemodynamics and cardiovascular categories; with the cardiovascular category having similar content as the Stream 1 cardiovascular category. Finally, Stream 3 created one hemodynamic monitoring category with 12 items; however, medications and laboratory results were sorted into other categories.

Open Sort of 29 Critical Care Concepts

A total of 16 critical care clinicians participated in the open sort exercise. Five participants were excluded due to partially complete sorts where greater than 25% of cards were left unsorted. Participants had varying critical care experience ranging from 0-2 to more than 10 years (Table 1). Each participant sorted 29 concepts into a median of 7 groups.

Hierarchical cluster analysis

The results of the HCA are shown in Figure 3. Concepts can only appear once in the 9 clusters of the HCA dendrogram. We set the HCA criteria to derive 9 clusters to ensure the number of derived clusters was similar to the median of 7 groups created by participants. We labeled clusters based on participant consensus, which required at least 3 participants to have the same or synonymous group label. A subject matter expert confirmed the synonyms and cluster labels. Of the 9 derived clusters, 7 had a labeling consensus: Medications/Infusions, Cardiovascular and Hemodynamics, Vitals, Respiratory, Neurology, Overview, and Infectious Disease. Overall, the highest similarity clusters were Medications/Infusions and Overview.

Figure 3.

Dendrogram of HCA results. The HCA derived clusters are color-coded. Similarity level (x-axis) is calculated from the similarity of linked concepts and the minimum similarity of all concepts in the card-sorting similarity matrix. Concept Abbreviations: IV fluid, intravenous fluid; SVR, systemic vascular resistance; CVP, central venous pressure; ST change, ST segment change; MAP, mean arterial pressure; IE ratio, ventilator inspiratory-to-expiratory ratio; PF ratio, arterial oxygen partial pressure/fractional inspired oxygen; GCS, Glasgow coma scale; WBC count, white blood cell count.

Factor analysis

Nine factors accounted for 80% of the variance seen in the analysis. The concept loading within each factor indicates the strength of association between the factor and the concept. Loadings range from -1 to 1, which is a correlation coefficient of the concept with a factor. A loading of 1 represents a perfect positive association of the concept to a factor. We selected association thresholds of greater than 0.3 and less than -0.3, based on observed concept groupings. Figure 4 depicts the loadings for each concept across nine factors. Eight of the 9 factors had similar concept groups to the HCA;³⁸ therefore, we assigned the same HCA labels to these factors. The transferred labels from the HCA clusters to the FA factors include: Cardiovascular, Respiratory, Overview, Medications and Infusions, Neurology, Vitals, Hemodynamics, and Infectious Disease. We labeled the additional factor as Laboratory. Concepts associated with more than one factor included: IV fluid, SVR, Heart rate, MAP, Oxygen saturation, and WBC count. Potassium and bowel regimen did not load strongly on any factor.

Figure 4.

Concept loadings across factor columns. Factors with negatively loaded concept groupings are denoted by brackets.

Agreement across different sort methods and analyses

The FA groupings were similar to those identified in the HCA, with a few exceptions. The FA results include the possibility of a Laboratory category. Heart rate, MAP, and oxygen saturation were highly associated with multiple groups in the FA and grouped differently between the FA and HCA. Heart rate loaded slightly higher for Cardiovascular than Vitals. MAP and oxygen saturation loaded more strongly on Vitals, than on Cardiovascular and Respiratory, respectively. Between the HCA and FA, there was an 89% agreement in concept categories (8/9) and a 90% agreement of concepts in these categories (26/29).

The findings of the Delphi sort corroborate and extend the results of the open sort. Similar to the open sort, 2 streams of the Delphi sort generated separate Cardiovascular and Hemodynamic constructs. In one stream, participants created a construct labeled Flowsheet that encompassed many of the same concepts found in Cardiovascular, Respiratory, and Vitals of the open sort. One additional construct that arose in both sorts was overview or notes. Groups seen in the Delphi sort, but not the HCA or FA, include Renal and Imaging. For Imaging, this likely reflects differences in the content participants were asked to sort since the Delphi sort included 3 image concepts while the open sort had none. The Delphi sort Renal category had significant overlap with an unnamed HCA cluster and the FA Hemodynamics factor. Among the common concepts from the open sort FA and the aggregated Delphi solutions, there was a 72% agreement in concept categories (8/11) and an 89% agreement of concepts in these categories (24/27).

Discussion

The primary aim of this study was to better understand critical care providers’ mental models related to organizing patient information in critical care. This study used two separate concept-mapping exercises (i.e., open sort and Delphi sort) to provide robust and reliable results. The Delphi sort and open sort results converged on system-based constructs including cardiovascular, respiratory, and neurology. Additionally, the problem-based construct infectious disease was seen in both sorts. Source-based constructs, medication and laboratory, were observed overlapping both system-based and problem-based constructs.

Motivation for this work stems from a prior study suggesting current EHR organizational structures do not support critical care clinicians’ information needs⁴ and from promising evaluations of clinical concept-oriented views (e.g., problems, disease, or organ systems). Additionally, parsimonious information presentations, displaying high-priority data in an integrated display, have been associated with improved efficiency and accuracy of information interpretation.^11,16,39 Our findings align with this body of evidence. We identified primarily system-based constructs including Cardiovascular, Respiratory, Neurology, Gastrointestinal, and Renal. While a few participants created other problem-based constructs (e.g., Cardiogenic Shock and Respiratory Failure), Infectious Disease was the single problem-based construct consistently identified across analyses and exercises.

Through these analyses, there were several interesting findings. The open sort FA demonstrated that certain concepts do not clearly belong to one specific construct. Vitals may be thought of as a source-based grouping or it may be linked to the rate or frequency of data capture (i.e., continuous or frequent data from multiple devices) or some other historical data grouping. As such, it may be appropriate to group these concepts in one place or split them into clinical concept-oriented groups (e.g., Cardiovascular and Respiratory). Other concepts were associated with multiple factors in the FA. In addition to Hemodynamics, the IV fluid concept was associated with Cardiovascular. SVR was associated with Hemodynamics in addition to Cardiovascular, and WBC count was associated with Laboratory in addition to Infectious Disease. The Delphi sort supports the assertion that clinicians’ mental models align with concept-oriented groups, but conventional groupings (i.e., source-based) are resorted to when extra concepts are provided. There was disagreement within the Delphi sorts regarding the placement of individual patient problems. Participants either sorted patient problems into system-based categories or into an overview group. In the open sort, the Acute Problems concept, which did not support separate assignment of problems to systems, was placed in an overview category. Depending on details of the design, it may be appropriate to group patient problems with history and notes, and/or to group system-based problems within system-based categories.

The health care domain has yet to fully leverage card sorting techniques to elicit clinicians’ mental models to design complex interfaces. Despite building evidence that supports concept-oriented information organization, current EHRs primarily organize information in source-based groups. This study demonstrated that across different knowledge elicitation techniques, clinicians provide a fairly consistent schema for information organization. This may support transitioning patient information away from traditional source-based groupings to a primarily system-based organization. The findings and methodology used in this study could guide future user interface design in health care.

Implications for Design

Information concepts can be organized in primarily system-based groupings and, to a lesser extent, problem-based groupings. We propose that timing and data frequency may be important organizational constructs, and that certain concepts in source-based groups can be separated and placed within system or problem-based constructs. One could envision an electronic display that supports the ability to simultaneously view cardiac and respiratory dynamic patient data on a common time scale with related treatments and responses.²³ For instance, cardiovascular medications (e.g., dobutamine) can be placed in the cardiovascular construct with related cardiovascular response information. Figure 5 is our interpretation of the hierarchy and constructs identified in this study.

Figure 5.

A tree graph depicting the proposed patient information organization schema. Concepts denoted with an asterisk were not in the originally sorted concepts and were added by participants or researchers.

Variability across solutions from our design activities may suggest that multiple approaches to information organization are valid. Next steps could include generating competing solutions and comparing them using methods such as tree tests (or reverse sorts)⁴⁰ or simulation-based comparisons of performance in the context of relevant clinical tasks. Participants commented that medication and image concepts may depend on patient context. For instance, chest x-ray may be relevant to a patient’s ventilator status or trauma; these contexts would support placing the image in Respiratory, Cardiovascular and Hemodynamics, and/or Infectious Disease. Further research is needed to understand information use in different patient contexts.⁴ Moreover, research is needed to identify which specific interventions (e.g., medications, fluids, nutrition), laboratory data, or imaging should be placed and prioritized in system and problem-based constructs.²¹ A complete set of critical care data could increase the granularity of primary and secondary construct associations.

Limitations

While efforts (e.g., concepts from the literature, critical care clinician consultations, pilot testing, and integration of Delphi findings into open sort concept selection) were taken to ensure comprehensive coverage of high-priority concepts, complete coverage was not feasible due to the number of frequently used concepts in critical care. Depending on patient or practice context, other concepts may be higher priority or used more frequently than those used in this analysis. Our sample was primarily physicians but not exclusively so. The outcomes do not solely represent physician perspectives and our small sample of participants from other clinical roles does not support rigorous evaluation of generalizability across roles. From this study, it is unknown how patient context changes information groupings. While the targeted sample size for the open sort was 15,³⁴ we were limited to 11 sorts for the analyses. Finally, in the 3 Delphi sort streams, it appears the initial sorts stayed largely intact for each of the streams (see Supplementary Appendix). A stated benefit of the Delphi method is reducing the cognitive workload of sorting a complete dataset and that results are comparable to other types of sorts.²⁸ In our Delphi sort, it appears breaking down groups and reassigning concepts may have been a barrier to making substantive changes to the initial groupings.

Conclusion

These results suggest a hybrid approach to information organization, which may combine systems, source, or problem-based groupings, best supports clinicians’ mental models in a non-specific patient context. Constructs were similar between the open sort and the Delphi sort. Concept agreement within categories was comparable. In both sorts, the strongest agreement was seen in the cardiovascular and respiratory constructs. Medications had the strongest agreement when sorting the broad dataset. Participants exhibited variability in sorting the concepts temperature, nutrition management, and fluid input/output. The resulting concept maps can support better information organization for critical care decisions. Novel display approaches including dynamic multi-patient and individual-patient status views may use these findings to increase situation awareness⁴ by presenting the most important information organized to support care decisions in a single glance.^11,22,23

FUNDING

This work was supported by the National Library of Medicine of the National Institutes of Health grant numbers: R56LM011925 and T15LM007124.

Competing interest

None.

CONTRIBUTORS

Study concept and design: Melanie C Wright, Thomas Reese, Noa Segall

Acquisition, analysis and interpretation of data: Melanie C Wright, Thomas Reese, Noa Segall, Rosalie Waller, Paige Nesbitt, Joseph Tonna, Brekk C Macpherson

Drafting of the manuscript: Thomas Reese, Melanie C Wright

Critical Revision of the manuscript: Thomas Reese, Melanie C Wright, Noa Segall, Guilherme Del Fiol, Rosalie Waller

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.