Identifying Logical Clinical Context Clusters in Nursing Orders for the Purpose of Information Retrieval

Abstract

Nurses’ information needs relate to nursing orders and nursing orders have many contexts including body systems, safety practices and other clinical categories. When searching for information related to orders one search term might retrieve documents related to multiple orders. We clustered nursing orders into sets that are related by the same ‘logical clinical context.’ We then generated clusters and their search terms from a data set of 636 orders obtained from a CIS/CPOE system at an academic medical center. We refined those cluster search terms by searching an electronic nursing procedure manual to retrieve resources that could answer one of six generic nursing questions. Sixty-three cluster search terms were identified. The search terms for 100 (16%) of the orders were validated in a second hospital’s electronic nursing procedure manual; precision was 32.5%.. Our process of identifying cluster search terms may be a useful method to obtain clinically relevant information resources.

Introduction

Nursing activities differ from medical activities during patient care; these differences impact the design of processes that provide information resources at the point of care. For instance, physicians may perform surgeries and other procedures while nurses perform a different set of procedures and manage pre-operative care, manage post-operative surgical sites and tubes, monitor patients for complications and apply preventative measures on a minute-to-minute basis. Nurse care-coordination, as part of multidisciplinary health care provision, facilitates the delivery of patient-centered care. Towards the goal of improving information retrieval for nurses, we have developed a method for meeting context specific information needs based on the semantic types of nursing orders that were contained in nursing ‘logical clinical context clusters’ from here forward referred to as clusters.¹ We defined the clustering of nursing orders as a process that establishes logical relationships of the domain elements within nursing orders that exist across multiple contexts determined from the nursing perspective.¹ The representation of nursing actions, nursing interventions and nursing orders using standardized terminologies has been successful,^5,6 however, most terminologies are not optimized for the display of clinical contexts for the purposes of automated information retrieval.

As compared to general medical/surgical units, critical care units typically have an increased urgency and volume to orders.² In all clinical areas, nursing activities elicit frequent information needs that may be resolved through informatics applications such as Infobuttons.^1,3,4 Infobuttons display generic clinical questions that are anticipated according to the type of work a user is doing in the clinical information system (CIS). Infobuttons are linked to information resources with the aim of answering the clinician’s anticipated questions.⁷ Identifying clinical clusters and their cluster search terms may enable an Infobutton to provide information resources that answer anticipated questions about the order being viewed as well as other orders within that clinical cluster. Therefore, the aim of this study was to demonstrate a method for generating, refining and validating reliable and clinically logical cluster search terms from nursing orders for the purpose of information retrieval.

Background

In our previous work, we demonstrated the potential for the cluster Intra Aortic Balloon Pump (IABP) Related Orders’ cluster search term IABP to successfully retrieve resources to answer nurses’ anticipated questions for nine nursing orders contained in that cluster.¹ Part of the earlier method included the identification of six nurse generic questions that may be applicable to any nursing cluster. Semantic types of nursing orders were identified to enable the automatic recognition of nurse generic questions that could logically be asked based on that nursing order’s semantic type.¹

Nursing orders are written at varying levels of abstraction. At times an order may be too abstract (e.g. “For Severe Persistent Itching,” a term used to modify another order) or too specific (e.g. “Unna Boot,” a special gauze used to treat venous stasis ulcers) to correlate to a cluster search term or a meaningful electronic resource.

Yet, the process of retrieving a resource to answer each of the applicable generic questions for the nursing orders contained in a cluster appears efficient and replicable once clusters and their cluster search terms are identified.¹ However, issues of subjectivity, granularity and non-mutual exclusivity complicate the identification of standard clusters. For example, IABP Related Orders are related to a highly technological process of care for a balloon that sits in a patient’s aorta, which inflates and deflates to assist a patient’s weakened heart pump oxygenated blood to the rest of the body. Due to the complex technology and unique indication for use, a cluster of nursing orders that are related to this concept is largely self-contained. However, many nursing orders vary significantly in their granularity and applicability. Another cluster, such as fluid related orders, may exist within different order sets such as the patient’s cardiac assessment (central venous pressure (CVP)), respiratory assessment (shortness of breath, abnormal lung sounds) nutritional planning (fluid and salt restrictions), medication interventions (diuretics), and evaluation (decreased CVP, decreased shortness of breath, decreased abnormal lung sounds and increased urine output).

Additionally, nursing orders are often intermingled, which reflects the complexity of nursing care. For example, an IABP, is one type of central arterial catheter inserted through a patient’s femoral artery. Other types of femoral and arterial catheters also require the nurse to perform the same sterile central line dressing changes to prevent infection, sheath site checks to assess for bleeding, and extremity pulse checks that ensure the catheter has not obstructed blood flow to the patient’s leg. Due to these interconnected relationships that are both physiological and procedural, clusters are not mutually exclusive.¹ These complexities, inherent among nursing orders, call for a standardized methodology of identifying clusters.

Methods

The methodology for this study consisted of three parts: 1. Generating cluster search terms from nursing orders for the purpose of information retrieval; 2. Refining the cluster search terms using an electronic nursing procedure manual to answer at least one nurse generic question about that order; and 3. Validating the cluster search terms using a second electronic nursing procedure manual from a different institution.

To meet the aim of searching a nursing procedure manual, we excluded orders about specific laboratory values and medication allergies because these elements are typically not contained in a nursing procedure manual.⁴ All of the remaining nursing orders available from the CIS/Computerized Provider Order Entry (CPOE) system at NewYork-Presbyterian Hospital (NYP), Columbia University Medical Center (CUMC) in November 2006 were used to generate the cluster search terms.

Cluster Search Term Generation

In past work, we identified 21 semantic types (e.g. measurement) and 26 critical care clinical clusters from the nursing order data set used in this study.¹ The current study extends the clinical clustering to capture multiple granularities present in the nursing orders and multiple applicable nursing domains, not limited to critical care, for each nursing order.

Nursing orders were initially classified according to the 21 care components in Saba’s Clinical Care Classification (CCC) system (e.g. cardiac).⁸ We used content analysis to map intensive care nursing documentation to CCC terms.⁹ For this study, we did not use the nursing interventions, diagnoses, actions, or outcomes coding from Saba’s CCC because our aim was to cluster clinical concepts of care for the purpose of information retrieval, not to document the nursing process.

When the CCC care component was not sufficient to capture the essence of the nursing order we coded according to body system. For example, cardiac is a CCC component code, however, neurological and gastrointestinal are not. In some cases we modified the body system name to match the indexed concept for information retrieval. For example, GI and gastrointestinal have the same clinical meaning, but GI was the indexed nursing term. We called the combination of these classifications Clinical and Body Systems Codes (CBSCs).

Semantic types and CBSCs were used as the search terms for each order in NYP’s electronic nursing procedure manual accessible online through the hospital’s local intranet at http://infonet.nyp.org, referred to as the Infonet. For each order, if a further level of abstraction was needed to retrieve a useful resource, then a more specific search term for that CBSC was identified. For example, the CBSC neurological was sufficient as a search term for the order neurological assessment. However, it was necessary to add the specific search term dialysis, not a CBSC, to retrieve a useful resource for the order Peritoneal Dialysis Catheter Insertion, which had semantic type procedure and CBSC Physical Regulation. This method of further defining CBSC categories is consistent with Moss and colleagues who recognized that the high level of abstraction of the CCC results in a loss of granularity.⁹ When any of these search terms (e.g., semantic type, CBSC, or a more specific search term such as dialysis) or a combination of these search terms occurred three or more times in the data set, the term or the combination of terms became a cluster search term.

Initially, two researchers (SC and LC), each with at least five years of critical care nursing experience and at least three years of experience with the CIS/CPOE system from which the orders were derived, identified the cluster search terms for 45 (7%) of the nursing orders. Discrepancies in coding were discussed until consensus was reached. Generation and refinement of the remaining cluster search terms was then completed by one researcher (SC).

Cluster Search Term Refinement

The cluster search term for each nursing order was refined by searching the NYP Infonet. Refinement consisted of searching using the semantic type of the nursing order, the CBSC, and the more specific search term, if necessary. The semantic type was included because certain nursing orders consisted only of a semantic type, such as restraint. Searches were performed using all iterations to identify the search that required the fewest search terms to retrieve the most applicable resource(s). Only resources that were accessible on the first or second page of the Infonet (i.e., within two mouse clicks) and that answered at least one of the six nurse generic questions, as judged by an experienced critical care nurse (SC), were considered successes. (See Table 1 for nurse generic questions).

Table 1.

Nurse Generic Questions

How do I use X? What is the Institution protocol for X? What are the desired outcomes versus the undesired outcomes related to X? What is the appropriate patient education related to X? What is the institution contact information to obtain X? What are the possible adverse events?

Each nursing order may be represented by multiple clusters; for example, elevate head of bed may apply to a neurological or a respiratory condition. Due to these multiple possible contexts, and due to some nursing orders not having an applicable or meaningful resource in the Infonet, a fourth way of searching was performed using the original order as the search term to ensure that all possible resources in the nursing procedure manual were exhausted. This provided the opportunity to identify multiple clusters, if appropriate, for one nursing order.

Cluster Search Term Validation

In order to validate the cluster search terms, we randomly selected 100 orders. We replicated the searches using the cluster search term (either the semantic type, CBSC, specific search term or a combination of 2 or all 3 of them) for the sub-sample of orders from a publicly available electronic nursing procedure manual of a large teaching hospital (http://www.unchealthcare.org/site/Nursing). We calculated the total precision for these orders. We were unable to calculate the recall because we were unable to determine the total number of records in the database. All searches were considered successful if they were able to answer at least one of the six generic nursing questions within two mouse clicks.

Results

Seven hundred and forty eight nursing orders were identified from the CIS/CPOE at NYP/CUMC. We excluded 112 orders about specific laboratory values and medication allergies. Eight of the remaining 636 nursing orders (1.2%) had no applicable resource available in the Infonet. These failures were confirmed by extensive and direct searches for each order. The eight orders were: strain all urine, cantor tube, ice lavage, unna boot, for severe persistent itching, normal saline retics, normal saline rinses, and vigilen. Some of the nursing orders that had no resource appear to be fragments of nursing orders; however, for consistency and completeness we included all retrieved orders in our analysis.

Six CCC component codes were useable as Infonet search terms for the 636 applicable nursing orders: Cardiac, Respiratory, Health Behavioral, Nutritional, Safety, and Medication. Three body system search terms were created to represent the missing body systems: GI (Gastrointestinal), Neurological and Skin. The CCC code Urinary Elimination and the term genitourinary were both replaced with urine because nurses and nursing resources typically use the term urine to reflect management of urinary elimination and the genitourinary system as a whole.

When searching by only the semantic type, 65 (9.1%) orders retrieved a resource without an additional search term (Restraint, Vital Signs, Position, Precaution, Referral, Measurement, Notification, Activity). Only 71 (11.1%, inclusive of failed searches) orders did not benefit from including the semantic type as a search term. There were 63 (10%) orders where the combination of the semantic type and the CBSC adequately retrieved an Infonet resource that could answer one of the six nurse generic questions about that order. Of note, Health Behavioral was the only CBSC sufficient for retrieving resources without a semantic type or specific search term (3.6%) (See Figure 1).

Figure 1.

Venn Diagram of minimum cluster search terms needed for successful resource retrieval

Four hundred and eighty four orders (76%) required a specific search term. Seventy nine (12.4%) had a unique specific search term that did not recur for any other order. Sixteen orders (2.5%) had a specific search term that recurred only one other time in the data set. There were 51 specific search terms that occurred three or more times for 402 orders (63%). (See Table 2 for examples). Twelve of these specific search terms occurred 10 or more times. The three most frequent specific terms were wound, surgery and fluid occurring 44, 27 and 24 times, respectively (See Figure 2). Nine orders were categorized into multiple body systems and 55 orders (8.6%) were categorized into multiple clusters.

Table 2.

Specific Search Terms & Sample Nursing Orders

Specific Search Term	Example Order
Wound	Wet to Dry/Damp Dressing
Surgery	Ventricular Shunt
Fluid	Paracentesis
Intravenous	Filter All Intravenous lines Notify Interventional
Clinical	Radiologist Fellow

Figure 2.

Bar chart of cluster search terms most commonly assigned to the 636 nursing orders

We validated the cluster search terms by performing precision for 100 of the 636 orders via a publicly available University of North Carolina Hospital’s electronic nursing procedure manual. Again, success was defined as the ability to retrieve a resource that could answer at least one of the six generic nursing questions within two mouse clicks. We retrieved an average of 2.5 resources for 70 of these orders on the first click and an average of 1.5 resources for 21 additional orders on the second click. The precision for the 91 orders for which at least one resource was retrieved within a maximum of two mouse clicks was 32.5%. Eight of the nursing orders did not have an applicable resource in the second nursing procedure manual and one nursing order, Intake and Output, had a cluster search term, fluid, that worked in the first, but not the second procedure manual.

Discussion

Our data set consisted of non-standardized nursing orders, yet we found only a 1.2% failure rate from all combined searches in retrieving an applicable information resource from a nursing procedure manual within two clicks. The semantic type of a nursing order appears to be an informative categorization for information retrieval. Eight semantic types were sufficient for successfully retrieving a resource without an additional search term. Additionally, the semantic type of the nursing order was useful when searching for a resource for 88.9% of the orders.

At times, a CBSC that did not exist as an indexed term in the searched resources hindered the search. Yet, the coding of CBSCs was beneficial for establishing the highest level of abstraction that was useful for retrieving applicable resources. This methodology allows the formation of large clusters that may not otherwise be apparent. However, further granularity was needed for the majority of orders. The cluster search terms at this level of granularity demonstrated the large amount of orders that relate to each other. For instance, the cluster search term wound retrieved a useful resource for 7% (44/636) of the orders. When taking into account that the nursing orders used for this study were for adult, pediatric, medical, surgical, psychiatric, acute and critical care populations this may be a clinically significant finding. The other cluster search terms worth noting, fluid and surgery, each retrieved useful resources for 3.6 – 4.2% of the nursing orders.

Refinement is a needed step in the process of defining cluster search terms. The accuracy of hits is also dependent on the nursing procedure manual’s search engine. For example, “vital sign” did not retrieve the desired resources, however, “vital signs” did. The search term “intravenous” retrieved results related to intravenous care as opposed to “infusion” which retrieved results related to intravenous pumps, even though these terms were used interchangeably in the nursing orders. This trial and error process of refinement may be streamlined with future work.

Although the precision calculation was relatively low, we were pleased that the cluster search terms for 70 of the orders retrieved relevant resources within one mouse click. This validation strengthens the rationale for adopting clusters as a standardized characterization of nursing orders with the aim of facilitating Infobutton information retrieval. Additionally, clusters may inform new methodologies for the development of nursing documentation templates.

This study is limited by the single CIS/CPOE system from which the nursing orders were derived and the two nursing procedure manuals used to refine and verify the cluster search terms for those nursing orders. However, it is the authors' experience that the CIS/CPOE system is representative of nursing order systems elsewhere. Additionally, the coding was performed by critical care nurses who may have different clinical context perspectives than nurses with other areas of expertise such as psychiatric, obstetric, or neonatal. Perhaps the application of this methodology requires clinical expertise pertinent to the area of nursing for which it is intended.

Conclusion

Nursing orders relate to multiple clinical contexts and are written at varying levels of abstraction. Clustering of nursing orders appears to be a valid methodology to utilize these contexts at the level of granularity that may be useful as search terms to retrieve applicable resources from electronic nursing procedure manuals. Future work should incorporate medication and laboratory orders and other types of databases such as UpToDate and Micromedex.