The Language of Delirium: Key Words for Identifying Delirium from Medical Records

Abstract

Background

Electronic medical records (EMRs) offer the potential opportunity to streamline the search for patients with possible delirium. However, the identification of key words is a necessary first step in the effective and systematic use of EMRs for both clinical and research purposes.

Objective

To identify words and phrases commonly noted in charts of patients with delirium.

Design, Setting, and Participants

Participants were 67 patients aged 70+ undergoing major elective surgery with evidence of confusion in their medical charts nested within a cohort study of 300 patients. We report the rate of common words and phrases associated with symptoms of delirium and report positive predictive value compared with a reference standard delirium diagnosis.

Results

Eight key words or phrases (altered mental status, delirium, disoriented, hallucination, confusion, reorient, disorient and encephalopathy) had positive predictive values (PPVs) of 60–100% for diagnosis of delirium. Key words were charted more often in nursing notes compared with physician notes. For example, in patients with delirium, nursing notes had an average of 6.4 notes containing a one of the 8 key words for delirium compared with an average of 2.8 in physician charts.

Conclusions

A brief list of key words or phrases may serve as building blocks for a methodology to screen for possible delirium from charts and large databases for research and real-time clinical decision making.

INTRODUCTION

Delirium, an acute change in attention and cognition, is common among hospitalized older patients, occurring in 29–64% of older inpatients and up to 70% of long-term care residents.^1–3 Development of delirium is associated with poor outcomes both during the course of the hospitalization (e.g., higher complication risk), in post-acute care settings, and long-term follow up (e.g., prolonged cognitive and functional impairment).^{1, 4–6} Despite the frequency of delirium and its negative consequences, there is a high rate of under-recognition and lack of documentation of delirium with fewer than 3% of cases of delirium noted by ICD-9 codes in patients charts.⁷ One reason for the under-recognition of delirium is that the most widely used assessment tool for delirium, the Confusion Assessment Method (CAM), requires an in-person, bedside interview with the patient which can be time-consuming and expensive. In addition, fluctuation is a defining characteristic of delirium and interview-based methods may miss delirium that was present outside the limited duration of bedside interviews. A chart-review method that does not require direct patient interaction and that covers 24 hours exists and has been validated, but requires hand-searches of all records (e.g., nursing and physician notes, discharge summaries, etc) and relies on clinical judgment.

With the passing of the Health Information Technology for Economic and Clinical Health (HITECH) Act in 2009, there has been increased emphasis on use of Electronic Medical Records (EMRs).⁸ Under the HITECH Act, there is an emphasis on using EMRs in ways that are meaningful to clinicians in real time (e.g., during a hospital stay) and to researchers (e.g., to facilitate use of large administrative datasets).⁸ An advantage of EMRs is that they are ‘searchable’ for key words associated with a diagnosis or event. Through automated searches, such as Natural Language Processing (NLP), it is possible to search the large amount of the EMR considered ‘unstructured clinical documentation’ for key words and identify patients with a high probability of having a condition. This approach has been successfully applied in identification of post-operative complications and chronic conditions. ^{9, 10} However, the identification of key words is a necessary first step in the effective and systematic use of EMRs for clinical and research purposes. Automated approaches to identification of delirium would enhance in-hospital care of older patients and may also help to facilitate transitions to post-acute and long-term care settings.

Although algorithms for chart-identification of delirium exist,^{7, 11} there is not a list of key words on which to search EMRs for identification of delirium. The foundational work of key word identification will allow for development of algorithms that combine words that are associated with delirium for use in searching large databases. This approach would be useful for both research and clinical applications, such as searching EMRs in real-time to enhance early identification of potentially delirious patients. The purpose of this paper is to identify words and phrases commonly recorded in the charts of patients with possible delirium and to report their predictive value for the diagnosis of delirium.

METHODS

Setting and Patients

This study was conducted in the first 300 patients enrolled in a prospective observational study of older patients scheduled for major surgery, the Successful Aging after Elective Surgery (SAGES) study, described in detail previously.¹² (Table 1) Potential participants were aged 70 or older, scheduled for major orthopedic, vascular or general surgery without delirium or dementia, and with a Modified Mini-Mental State Examination (3MS) score of >68 or its education-adjusted equivalent at their baseline pre-surgery interview.¹³ Written informed consent was obtained from the patient. The present sample is a nested cohort of patients who had evidence of confusion in their medical record (N=67) drawn from the larger sample of 300 patients. A consort diagram is presented in Appendix Figure 1.

Table 1.

Sample Characteristics

	Total Sample N=300	Nested Cohort† n=63
Age at Index Surgery, Mean (SD)	76.9 (5)	77.4 (6)
Age 80+	81 (27)	16 (25)
Female Sex	166 (55)	36 (57)
Non-White or Hispanic	21 (7)	5 (8)
English as a Second Language (ESL)	20 (7)	2 (3)
Currently Married/Living with Partner	185 (62)	34 (54)
Education
0–12 Years	86 (29)	16 (25)
13–16 Years	129 (43)	28 (44)
17+ Years	85 (28)	19 (30)
Surgery Type
Orthopedic	253 (84)	52 (83)
Vascular	16 (5)	7 (11)
Gastrointestinal	31 (11)	4 (6)

Note. Values are N (%) unless otherwise specified.

^†The nested cohort consisted of patients with evidence of confusion in the chart.

Appendix Figure 1.

Consort Diagram

Chart-Based Delirium Instrument

The chart-based delirium instrument was developed to maximize sensitivity for identification of delirium and requires a chart abstraction.^{7, 11} This method is not based on ICD-9 codes or discharge diagnoses, rather it requires abstraction of clinician notes indicating confusion or related behaviors. It includes information on acute changes in mental status, time and duration of episodes, whether there was evidence of agitation associated with the episode and whether or not there was reversibility or improvement of the acute confusion during hospitalization. Trained clinical chart abstractors obtained a baseline mental status rating (from which an acute change could be judged) from pre-operative visits, previous discharge summaries and outpatient visit notes.

Indicators of confusion or change in mental status during hospitalization were obtained through review of the entire chart, including admission and daily nursing notes, admission and daily physician progress notes, notes from specialist consultations (e.g., neurology, psychiatric, geriatrics, pain service) and the discharge summary. Abstractors were provided with ‘trigger words’ or phrases that may be used to indicate delirium and prompted the rater to look into that section of the chart for details of episodes that might indicate delirium (see below for full details on trigger words). Once a potential case was identified, all notes on that patient with any indication of confusion were fully abstracted verbatim. The chart abstractors were nurses or physicians who underwent extensive training and standardization.¹²

Reference Standard for Delirium

Delirium was diagnosed based on a previously validated chart-review method, and was adjudicated independently by a geriatrician (SKI) and a neuropsychologist, both with extensive training and experience in delirium assessment, using a process that was reported previously.¹¹ Any discrepancies were resolved during consensus conferences.

Selection of Words Associated with Delirium

Abstractors were provided with trigger words that may be associated with confusion and delirium. These trigger words have been used in a previously validated chart-based approach to identifying delirium⁷ and were similar to those used in previous studies that extracted key words for confusion from electronic records.^{14, 15} Examples of trigger words were: ‘mental status change’, ‘disoriented/re-oriented’, ‘unresponsive’, and ‘agitated’. Table 3 includes a full listing of trigger words. Once the trigger words were identified, then the full notes were transcribed verbatim. Other supporting words that may be associated with confusion were also abstracted; Appendix Table 1 provides a list of the most frequently occurring supporting words. The difference between trigger words and supporting words is that appearance of trigger words prompted a full review of the record while appearance of supporting words did not. Together, the trigger and supporting words represent the key words for identification of confusion from the chart. These transcripts were entered and analyzed, and the most frequent relevant words and word combinations were selected.

Table 3.

Trigger Words for Delirium Abstraction

Word†  Source	Total Charts with Word (N=63)	Delirious Charts with Word (n=35)	Non-Delirious Charts with Word (n=28)	PPV
‘AMS’ or ‘Mental Status’
Nurse	8 (13)	8 (23)	0 (0)	100
Physician	11 (18)	11 (31)	0 (0)	100
Other	1 (2)	0 (0)	1 (4)	0
‘Deliri*’
Nurse	9 (14)	9 (26)	0 (0)	100
Physician	10 (16)	9 (26)	1 (4)	90
Other	2 (3)	2 (6)	0 (0)	100
‘Alert and Oriented’ (< 3)
Nurse	17 (27)	14 (40)	3 (11)	82
Physician	2 (3)	2 (6)	0 (0)	100
Other	0 (0)	0 (0)	0 (0)	0
‘Hallucin*’
Nurse	5 (8)	4 (11)	1 (4)	80
Physician	1 (2)	1 (3)	0 (0)	100
Other	0 (0)	0 (0)	0 (0)	0
‘Confus*’
Nurse	39 (62)	27 (77)	12 (43)	69
Physician	18 (29)	17 (49)	1 (4)	94
Other	3 (5)	0 (0)	3 (11)	0
‘Reorient*’
Nurse	31 (49)	20 (57)	11 (39)	65
Physician	1 (2)	1 (3)	0 (0)	100
Other	0 (0)	0 (0)	0 (0)	0
‘Disorient*’
Nurse	15 (24)	9 (26)	6 (21)	60
Physician	3 (5)	3 (9)	0 (0)	100
Other	1 (2)	1 (3)	0 (0)	100
‘Encephalopathy‡’
Nurse	0 (0)	0 (0)	0 (0)	0
Physician	3 (5)	3 (9)	0 (0)	100
Other	1 (2)	1 (3)	0 (0)	100
Any Word
Nurse	51 (81)	32 (91)	19 (68)	63
Physician	24 (38)	22 (63)	2 (7)	92
Other	7 (11)	3 (9)	4 (14)	43
No Word
Nurse	12 (19)	3 (9)	9 (32)	25
Physician	39 (62)	13 (37)	26 (93)	33
Other	56 (89)	32 (91)	24 (86)	57

Note. All values are n (%)

^†These words triggered an abstraction, and thus, they never appeared in the non-abstracted (n=237) charts;

^*indicates multiple different endings, such as ‘um’, ‘ous’, etc.

^‡Metabolic encephalopathy; toxic encephalopathy; drug-induced encephalopathy; toxic-metabolic (and all with “encephalopathy”) PPV= Positive Predictive Value

Appendix 1.

Other Supporting Words in Delirium Chart Abstraction

Word	Total Charts with Word (N=63)	Delirious Charts with Word (n=35)	Non-Delirious Charts with Word (n=28)
‘Forget*’	23 (37)	14 (40)	9 (32)
‘Alarm’	18 (29)	11 (31)	7 (25)
‘Commands’	17 (27)	14 (40)	3 (11)
‘Out of Bed’ or ‘OOB’	16 (25)	11 (31)	5 (18)
‘Impuls*’	12 (19)	8 (23)	4 (14)
‘Narcotic*’	12 (19)	10 (29)	2 (7)
‘Sleepy’	12 (19)	8 (23)	4 (14)
‘Agitat*’	11 (18)	10 (29)	1 (4)
‘Anxious’ or ‘Anxiety’	10 (16)	9 (26)	1 (4)
‘Redirected’	9 (14)	5 (14)	4 (14)
‘Restless’	8 (13)	7 (20)	1 (4)
‘Letharg*’	7 (11)	5 (14)	2 (7)
‘Refus*’	7 (11)	4 (11)	3 (11)
‘Multifactorial’	6 (10)	6 (17)	0 (0)
‘Sedat*’	6 (10)	6 (17)	0 (0)
‘Attent*’	5 (8)	5 (14)	0 (0)
‘Somnolent’	5 (8)	5 (14)	0 (0)
‘Hypoactive’	4 (6)	4 (11)	0 (0)
‘Fall’	3 (5)	2 (6)	1 (4)
‘Unrespons*’	3 (5)	3 (9)	0 (0)
‘Combative’	2 (3)	2 (6)	0 (0)
‘Fluctu*’	2 (3)	2 (6)	0 (0)
‘Distract*’	1 (2)	1 (3)	0 (0)
‘Not Cooperat*’ or ‘Uncooperative’	1 (2)	1 (3)	0 (0)
‘Resist’	1 (2)	1 (3)	0 (0)

Note. All values are n (%)

^†These words may have appeared in the unabstracted (n=237) charts;

^*indicates multiple different endings, such as ‘um’, ‘ous’, etc.; OOB = out of bed

Definition of Study Variables

Descriptive characteristics of study patients were examined. Race, ethnicity, education, language of origin and marital status were self-reported by the patient at the pre-operative baseline interview. Type of surgery, age and sex were abstracted from the medical record.

Statistical Analysis

We describe each trigger word, the number of charts (delirious and not) in which the word appears and also who wrote the note in which the word appeared (e.g., nurse, physician). The Positive Predictive Value (PPV) of each trigger word was calculated as the number of charts of patients diagnosed with delirium in which the word appeared divided by the total number of charts (total sample) in which the word appeared.

RESULTS

The total sample from which the current nested cohort study was drawn consisted of 300 hospitalized patients with a mean age of 77 years. The nested cohort consisted of 63 hospitalized patients with any evidence of confusion in their chart. The sample had a mean age of 77 years; one quarter was over 80 years (Table 1). Approximately half (57%) were female and 8% were non-white or Hispanic. Participants were well educated, with three quarters having higher than a high school education. The majority (82%) of participants were scheduled for orthopedic surgery. Of the 63 charts with evidence of confusion, 35 (56%) were adjudicated as delirious.

Language Representative of Delirium in Charts

We found that among patients who developed delirium, there were an average of 6.4 nursing notes containing key words for delirium compared with an average of 2.8 notes from physicians and less than 1 note, on average, from other sources (e.g., consults, discharge summaries) (data not shown).

Table 2 presents exemplar quotations from selected charts that were positive for delirium. The selected quotes represent both hyperactive and hypoactive forms of delirium. In general, quotes from cases more representative of hyperactive delirium are easier to detect as ‘abnormal’ or as a cause for concern. For example, notes from Patients 1, 8 and 9 present hallucinations, paranoid and inappropriate behaviors. These notes can be interpreted and symptoms of delirium identified without much contextualization. That is, it is clear from the brief notes that the patient is experiencing an acute change in mental status and is confused.

Table 2.

Selected Quotations from Delirium Positive Charts

Patient #	Quote†	Note Source
1	Lethargic, disoriented at times to place/time. Patient removed IV access while confused.	Nurse
	A+Ox3 @ start of this shift, though with frequent period of confusion. Patient bed alarmed for sedative and fall precaution.	Nurse
	A+Ox3. When asked questions, answered appropriately. Confused at times. Thought there was a shopping mall in the hospital.	Nurse
2	Around 3pm, pt was found to be unresponsive, disoriented. Unable to answer questions or follow commands.	Nurse
	Per reports, patient was at his baseline mental status this morning, but became more confused over the course of the day…At that time, patient was unable to keep his eyes open for more than 10 seconds at a time. Altered mental status.	ICU physician admission note
3	Mental status has been waxing and waning today. She is confused again, speaking nonsense, states she does not remember anything. Neuro: A+Ox3, answering questions appropriately, follows some commands.	Attending Physician
4	No major changes, pt continues to be somnolent, easily arousable, but then falls back asleep. Pt able to state name, date, hospital, and reason for her admission.	Nurse
	A&Ox3 in AM, by PM, confused and disoriented, team aware. 2pm, pt in bed and resting, very somnolent and tachy in the 140s. Pt not A&Ox3, garbled speech.	Nurse
5	Very somnolent and confused this morning. Unable to assess due to mental status. Patient very somnolent and confused. Unable to maintain conversation, drifts to sleep.	Acute pain service physician note
	Narcotic medication held to see if this helps confusion. Confusion clearing up, but not 100%. Unable to tell the date/year	Nurse
6	Patient initially impulsive, but mostly clear mentation early. Rapidly became profoundly delirious despite seroquel, haldol, Ativan with limited effect. Required vigilant observation. Bed alarm and frequent intervention.	Nurse
	This morning patient oriented to self/time plus hallucinations. Speech garbled. Patient confused but cooperative. Increasing clarity, increasing wakefulness, but appropriate. A+Ox3 this afternoon and after 4pm.	Nurse
7	Patient medicated with Ambien 2.5 mg po for sleep – awake at 5AM--confused, A+Ox1, urinated on floor, assisted to bed, alarm on for safety. Reoriented to surroundings.	Nurse
8	Event: confusion. Patient with increasing confusion this shift. At 9:30 pm she became more confused, yelling out, paranoid. Attempted to dial 911, telling nurse she is “scared, this isn’t a safe place”. Unable to reorient.	Nurse
9	Patient went to sleep 12:15am. ~ 3 am. Found sitting up on the bedside chair without any clothes on her. States: “I don’t know how I got here.” Assisted back to bed with maximal assist of 2.	Nurse
10	Acute change in conscious state – HCT 26.6 (decreasing 32.9) Into pt’s room @ 0630 to give PO Tyl. Pt confused, unaware of time/place. Acute change in mental status – overnight when woken a+ox3. Doesn’t remember surgeon, name of surgery, etc. Able to name DOB & spell last name. – no BUE weakness, speech clear, but kept saying “I don’t know, I’m confused”.	Nurse

^†A+O=alert and oriented; PACU=Postoperative acute care unit; ICU = Intensive Care Unit; A+O = Alert and Oriented; IV = intravenous; HCT = hematocrit; DOB = date of birth; BUE = bilateral upper extremity; PO = by mouth. Quotes are excerpts and do not necessarily represent all evidence of delirium in a patient’s chart. Quotes were edited to correct typographical errors; and 1 quote was paraphrased to protect identity.

In contrast, patients with behaviors more consistent with hypoactive delirium may be more difficult to identify from a single note. Notes from patients number 2, 4 and 5 are examples of symptoms and behaviors that would be associated with the hypoactive delirium such as extreme drowsiness. From the individual notes taken alone, however, it is difficult to identify delirium. That is, the notes and behaviors need to be put into context and a series of notes are needed to define the clinical course (fluctuation, reversibility) and to establish the presence of delirium. These notes also highlight the inherent difficulty in identifying hypoactive, as compared to hyperactive, delirium from the medical record.

Key Words for Identification of Delirium

Trigger Words

Trigger words (those prompting a full record review) found to be most useful in the identification of delirium are presented in Table 3 according to the source of the note (nurse, physician, other). The trigger words presented in Table 3 never appeared in charts that were not abstracted. Thus we were able to calculate a positive predictive value (PPV) for these words based on the full sample of 300 patients.

In general, trigger words appeared in nursing notes more often than in physician notes, likely reflecting the higher frequency of nurses charting and also their longer duration of contact with the patients. Several trigger words, although rare in abstracted charts, had high PPVs and served as clear indicators of the presence of delirium. That is, when the word appeared in the chart, the patient was highly likely to be delirious. For example, ‘mental status’ appeared in 8 (13%) of nursing notes and 11 (18%) of physician notes and had a PPV of 100%; ‘Deliri*’(* indicates multiple different endings, such as ‘um’, ‘ous’, etc.) appeared in only 9 charts and had a PPV of 90–100%. These are examples of words that could be used to identify high-probability delirium cases from clinical records on an ongoing basis or in real-time with little need for clinical interpretation.

Other trigger words required contextualization to determine whether symptoms of delirium were present. That is, the appearance of the word alone would not necessarily mean that the patient was delirious and the context of the note would need to be carefully considered. For example, ‘Confus*’ appeared in 39 (62%) nursing notes and 18 (29%) physician notes with PPVs of 69% and 94%, respectively. Therefore, in nursing notes in particular, the context of a mention of ‘confusion’ would need to be carefully considered before establishing the presence of delirium, since 31% may represent false positive results.

Supporting Words

There were also other supporting words that appeared more frequently in both delirious and non-delirious charts that may serve as additional words on which to search to identify potential cases of delirium, such as in an EMR. (Appendix Table 1) Due to their relatively high frequency in the charts and broad potential for use in nurse and physician notes, the context in which these words are recorded in the chart requires careful examination for them to be useful in identification of delirium. For example, ‘Forget*’ was included in the notes of 23 (37%) of all abstracted charts and in 14 (40%) of the notes of charts adjudicated as delirious. These words were not included in our list of trigger words and may have appeared in non-abstracted charts; thus, we were not able to calculate PPVs for these supporting words. However, these additional key words may still be useful in flagging patients during a hospitalization whom may represent possible delirium cases.

DISCUSSION

We identified key words associated with symptoms of delirium and report their prevalence and predictive value in nursing and physician notes. We provide a list of 8 words associated with confusion that can be used for searching electronic medical records to identify symptoms of delirium. Predictive value of single key words for a diagnosis of delirium ranged from 60%–100%. We found that nurses used key words associated with confusion in their notes more often than did physicians, irrespective of delirium status.

Several previous studies have focused on nurse documentation of delirium^16–19 and on the under-recognition of delirium in the nursing charts. However, few studies have compared physician and nurse charting of specific words associated with symptoms of delirium.^{15, 16} In addition, the predictive value associated with specific words has not been previously calculated. Thus, we extend the previous literature on chart-based identification of symptoms of delirium by providing key words associated with delirium and the prevalence and predictive value of each specific term. The development of a list of key words is a crucial first step in automating the identification of patients at risk for delirium who may require further screening and evaluation.

As expected, we found that, nurses charted more often than physicians. Nurses also used key words associated with symptoms of delirium in their notes more often than did physicians. These findings are similar to a study comparing nurse and physician charting of key words for delirium in patients in post acute care settings where nurses recorded words associated with delirium 30% more often than physicians,¹⁵ suggesting that it is important to review both physician and nursing notes for symptoms of delirium and that using only physician notes would miss many cases.

Using key words from a validated method for chart-identification for delirium,^{7, 11} we identified 8 words or phrases with high predictive validity for delirium. Although chart-based delirium identification methods are available, they can be time consuming and require the entire chart; and thus, they can only be done retrospectively. The increased use of the EHR offers the unprecedented opportunity for identifying patients at high risk for delirium in real-time to inform clinical decision making, such as to guide further testing and treatment. A recent pilot study in the post-acute care setting is an example of how using a key word search could inform real-time clinical decision making.¹⁵ Charts were retrospectively searched for key words associated with delirium and the discipline (nurse vs. physician) charting each key word was noted. Similar to our findings, nurses were more likely to chart key words for delirium and, importantly, when presence of the symptom of delirium was communicated to physicians, further evaluation or treatment (pharmacologic and nonpharmacologic) occurred in 70% of patients. In contrast, no action (evaluation or treatment) was noted in the charts of patients in whom presence of symptoms of delirium was not communicated to physicians, suggesting that making physicians aware of key words in the charts can facilitate early identification and treatment of delirium.

We also identified several high-yield, low frequency words or phrases that could be used to identify delirium in large or administrative datasets. For example, ‘altered mental status’ and ‘delirium’ were noted in less than one quarter of the charts but had a positive predictive value of 100%. More frequently cited words may require additional clinical judgment and thus may be more useful to flag high risk patients in real-time but would be less useful for large and administrative dataset use. For example, ‘confusion’ and ‘reorient*’ were mentioned in 77% and 57% of nursing notes in the study, respectively, and had PPVs between 65% and 69%. Although these key words would be ‘red flags’ in a patient chart and may highlight the need for closer monitoring or further testing, they would require additional contextualization and clinical evaluation in order to confidently identify delirium.

Our findings are strengthened by the rigorous chart-review method applied and adjudication of delirium diagnosis by 2 independent experts using consensus conferences. We collected detailed data on confusion verbatim from notes in the medical record allowing for identification of specific words and phrases for analysis. Several limitations of our study must also be considered. We only fully abstracted charts with evidence of confusion, therefore, the supporting words and phrases (listed in Appendix Table 1) may have appeared in the charts without evidence of confusion (e.g., those charts not fully abstracted) and therefore, we were not able to calculate PPV for these words. In addition, natural language processing approaches were not used for the present study because the charts were not fully electronic. Finally, our study included patients from 2 hospitals in a single geographical area who were cognitively intact prior to surgery, primarily Caucasian and who were healthy enough to undergo elective surgery, limiting the potential generalizability of our findings.

Conclusions

The brief listing of key words we identified might serve as a starting point for building a methodology or program for detecting possible delirium from the EHR and administrative databases for research and real-time clinical decision making. With this list of words, future work to expand and refine these listings and to develop algorithms that combine key words and phrases associated with possible delirium can help to increase the sensitivity and specificity of the search. Such work would carry important implications to improve identification of delirium in clinical practice and facilitate the transition of older patients to long-term care settings.