Abstract
Delirium, a common and serious disorder in older hospitalized patients, remains underrecognized. While several delirium predictive models have been developed, only a handful have focused on electronic health record (EHR) data. This prospective cohort study of older inpatients (≥65 years old) aimed to determine if variables within our health system’s EHR could be used to identify delirium among hospitalized patients at the bedside. Trained researchers screened daily for delirium using the 3-minute diagnostic Confusion Assessment Method (3D-CAM). Patient demographic and clinical variables were extracted from the EHR. Among 408 participants, mean age was 75 years, 60.8% were female, and 82.6% were Black. Overall rate of delirium was 16.7%. Patients with delirium were older and more likely to have an infection diagnosis, prior dementia, higher Charlson comorbidity severity of illness score, lower Braden Scale score, and higher Morse Fall Scale score in the EHR (P<0.01 for all). On multivariable analysis, a prior diagnosis of dementia (odds ratio: 5.0, 95% CI: 2.5–10.3) and a Braden score of <18 (odds ratio: 2.8, 95% CI: 1.5–5.1) remained significantly associated with delirium among hospitalized patients. Further research in the development of an automated delirium prediction model is needed.
Keywords: delirium, electronic health record, ethnoracial, diagnosis, disease prediction model
Whereas dementia is not reversible, delirium is a common and serious disorder characterized by reversible changes in cognition and thinking. It is associated with higher mortality, morbidity, institutionalization, and functional cognitive decline than dementia as well as high health care costs.1–3 In addition, there is an increased risk of developing dementia after an episode of delirium.4 This serious diagnosis has a multifactorial etiology, and its development among patients involves a complex interrelationship between the patient’s underlying predisposing factors and exposure to noxious precipitating factors.5
While there is no definitive laboratory test that can reliably diagnose delirium, there are several established screening tools.6 Use of bedside tools alone are insufficient in facilitating timely recognition and management of delirium among hospitalized patients, resulting in delirium being underdiagnosed by health care professionals.7 In fact, the presence of delirium may only be correctly identified at the bedside by nurses in 27% of delirious patients.8 Likewise, cardiologists may only recognize half of their patients with delirium, commonly leading to a misdiagnosis of mental health disorder and inappropriate treatment practices.9
Mitigation of the serious consequences of delirium may be accomplished by earlier recognition and treatment. Several delirium predictive rules have been developed for various patient populations (post-cardiac surgery,10 persistent delirium11), and a handful of studies have focused on the development of automated delirium prediction rules unique to older hospitalized patients.12–14 Overall, there is variation in the ability of the available models in predicting delirium; however, an optimized tool may be critical in enhancing the ability of the clinicians to recognize delirium at the bedside.15
The coupling of prediction rules with features of the electronic health record (EHR) is in its infancy but holds promise in aiding earlier identification of delirium. There are successful EHR-driven delirium predictive models in surgical patients16 and those without cognitive impairment,17 thus using existing EHR data to develop an automated delirium prediction model may be cost-effective and timely without requiring additional data entry by clinicians. The purpose of the study presented herein was to determine if there are variables already present within our EHR that could be used to identify delirium among older hospitalized medical patients at the bedside.
METHODS
Design and Setting
This prospective, observational cohort study was conducted at a single urban-based community teaching hospital and approved by the local institutional review board.
Participants
Inclusion criteria were age 65 years or older and admission to a medical unit in a single hospital during the study period (February 2016 to November 2017). Patients were excluded if they were non-English speaking, were comatose, had severe aphasia, had severe dementia, had an activated power of attorney, were on a ventilator, were combative, were critically ill, or were intensive care/surgical patients. Those with severe dementia, as defined by those who had an activated power of attorney, were excluded because reliably diagnosing delirium in such patients is difficult and many would not be able to participate in study screening. In our clinical experience, it is rare for hospital notes to describe the severity of dementia and those with activated power of attorney are more likely to have severe dementia. Informed verbal consent was obtained from the patient.
Cognitive Assessment
The diagnosis of delirium was established using the 3-minute diagnostic interview for Confusion Assessment Method-defined delirium (3D-CAM).18 3D-CAM was administered daily in the morning while the patient was in the hospital. Delirium was defined as any positive 3D-CAM during the hospital stay. Delirium was defined as prevalent delirium if the first 3D-CAM was positive. Incident delirium was defined as a negative first 3D-CAM and a positive subsequent 3D-CAM.
All patient participants were screened for presence of dementia using the Short-Blessed Test (SBT) within 48 hours of admission.19 Despite the known limitations in using cognitive assessment for diagnosis, we felt the routine underdiagnosis of dementia in the EHR20,21 — along with SBT’s short administration time and moderate efficacy in hospital settings (87% sensitivity and 70% specificity)22 — made its use for this study appropriate. Researchers completing the cognitive screening tests were trained by a geriatrician and PhD-prepared nurse.
Prior to beginning the study, the researchers and geriatrician interviewed 10 participants for the presence or absence of delirium using the 3D-CAM for training purposes. Online Supplemental Figure S1 describes the study process flow chart.
Potential EHR Variables
EHR variables were chosen based on previous research23 and ease of availability in the EHR. Variables that were available from EHR included patient demographics such as age, race, ethnicity, and gender. International Classification of Diseases, Tenth Revision (ICD-10) diagnostic codes were obtained from the EHR to generate a patient’s Charlson comorbidity score, which predicts the severity of illness (with higher score predicting worse outcomes).24 Additional variables included infectious diagnosis, vision impairment, a prior history of dementia per ICD-10 diagnosis, and fracture.25
Physical restraint use, sitter use, vision impairment, medical comorbidities, labs on admission, Katz activities of daily living (ADL) score, Morse Fall Scale score, and Braden Scale score were obtained from the EHR. These assessments were recorded in the EHR by staff nurses while performing their daily bedside assessments. Katz ADL is a measure of function that ranges in score from 0 to 12 (0 being dependent in all ADL and 12 being independent).26 The Morse Fall Scale predicts the risk of falls, with a score of 45 or more associated with increased risk,27 and the overlap between falls and delirium is repeatedly reported in the literature.28 The Braden Scale evaluates risk for development of pressure ulcers, with a lower score associated with higher risk; geriatric syndromes are known to have shared risk factors, hence Braden was included in this study as a marker of risk for pressure ulcers.29
Health care utilization variables collected from the EHR included length of stay, 30-day readmission rate, and 30-day mortality rate.
Data Analyses
Patient characteristics were summarized using basic descriptive statistics for both continuous (mean and standard deviation) and categorical (frequency and percentage) variables. Differences in characteristics between patients with and without delirium were compared using chi-squared tests for categorical variables and t-tests for continuous variables. Variables that were significant on univariate analysis were included in a multivariable regression model. Two-sided P-values of <0.05 were considered statistically significant. All statistical analyses were performed using SAS 9.4 software (SAS Institute Inc.).
RESULTS
A total of 2259 patients were screened. Of those, 489 were verbally consented; 24 patients were excluded due to missing information, same-visit duplication, names not corresponding, and a surgical patient. The cleaned cohort had 465 patients but included patients that were part of the study multiple times. Only looking at first visits for patients, the final cohort was 408 patients. Figure 1 describes the flow of participants in the study.
Figure 1.
Flow diagram of hospitalized participants in study of electronic health record variables to identify delirium. POA, power of attorney; pts, patients.
Baseline patient characteristics are described in Table 1. The mean age of the study cohort was 75 years (range: 65–102 years); 60.8% were female, and 82.6% self-reported race as Black. The average Charlson comorbidity score was 2.6. Using 3D-CAM criteria, delirium was present in 68 participants (16.7%). Prevalent delirium was noted to be positive in 43 participants (10.5%), while incident delirium was present in 25 participants (6.1%). A diagnosis of prior history of dementia (by ICD-10 diagnostic codes) was present in 11.5% (n=47) of participants, while researcher identification of dementia by SBT screening was 23.8% (n=97).
Table 1.
Baseline Characteristics of Hospitalized Older Patients Meeting Study Inclusion Criteria
| Characteristic | Overall N=408 |
No delirium n=340 |
Delirium n=68 |
P |
|---|---|---|---|---|
| Female sex, n (%) | 248 (60.8) | 203 (59.7) | 45 (66.2) | 0.318 |
| Non-Hispanic Black race/ethnicity, n (%) | 337 (82.6) | 280 (82.4) | 57 (83.8) | 0.7914 |
| Age in years, mean (SD) | 75 (8.2) | 75 (7.9) | 77 (9.5) | 0.028 |
| Age of ≥80 years, n (%) | 103 (25.3) | 78 (22.9) | 25 (36.8) | 0.017 |
| Diagnosis of vision impairment, n (%) | 14 (3.4) | 13 (3.8) | 1 (1.5) | 0.482 |
| Infectious diagnosis, n (%) | 68 (16.7) | 50 (14.7) | 18 (26.5) | 0.018 |
| History of dementia diagnosis by ICD-10 code, n (%) | 47 (11.5) | 25 (7.4) | 22 (32.4) | <0.001 |
| Prior dementia by ICD-10 and by SBT screening, n (%) | 21 (5.1) | 9 (2.6) | 12 (17.6) | <0.001 |
| No prior dementia by ICD-10 but presence of dementia by SBT screening, n (%) | 76 (18.6) | 59 (17.4) | 17 (25.0) | 0.1508 |
| Fracture diagnosis, n (%) | 7 (1.7) | 4 (1.2) | 3 (4.4) | 0.094 |
| Maximum pain score, mean (SD) | 5.2 (4.0) | 5.2 (4.0) | 5.6 (4.2) | 0.378 |
| Maximum Morse fall score, mean (SD) | 62.4 (22.6) | 60.0 (22.2) | 74.5 (20.7) | <0.001 |
| Morse fall score of >45, n (%) | 277 (67.9) | 219 (64.4) | 58 (85.3) | 0.001 |
| Minimum Braden score, mean (SD) | 18.3 (3.0) | 18.7 (2.8) | 16.3 (3.1) | <0.001 |
| Braden score of <18, n (%) | 134 (32.8) | 94 (27.7) | 40 (58.8) | <0.001 |
| Charlson comorbidity index, mean (SD) | 2.6 (1.8) | 2.5 (1.8) | 3.1 (1.7) | 0.019 |
| Charlson comorbidity score of >2, n (%) | 276 (67.7) | 222 (65.3) | 54 (79.4) | 0.023 |
| Prevalent delirium, n (%) | 43 (10.5) | 0 (0) | 43 (63.2) | – |
| Incident delirium, n (%) | 25 (6.1) | 0 (0) | 25 (36.8) | – |
| SBT given by researchersa | <0.001 | |||
| Normal cognition, n (%) | 193 (47.3) | 175 (51.5) | 18 (26.5) | |
| Questionable Impairment, n (%) | 107 (26.2) | 91 (26.8) | 16 (23.5) | |
| Consistent with dementia, n (%) | 97 (23.8) | 68 (20.0) | 29 (42.7) | |
| Hospital length of stay in days, mean (SD) | 4.0 (4.9) | 3.6 (4.3) | 6.1 (6.8) | 0.005 |
| 30-day mortality, n (%) | 11 (2.7) | 9 (2.7) | 2 (2.9) | >0.999 |
| 30-day readmission, n (%) | 59 (15.5) | 50 (14.7) | 9 (13.2) | 0.753 |
There were 11 patients who did not have SBT performed.
ICD-10, International Classification of Diseases, Tenth Revision; SBT, Short-Blessed Test; SD, standard deviation.
Patients with delirium were older than those without delirium (77 vs 75 years; P=0.028). They were also more likely to have a diagnosis of infection (26.5% vs 14.7%; P=0.018), prior dementia by ICD-10 diagnostic code (32.4% vs 7.4%; P<0.001), and a higher mean Charlson comorbidity score (3.1 vs 2.5; P=0.019). Furthermore, patients with delirium had a lower minimum mean Braden Scale score (16.3 vs 18.7; P<0.001) and higher mean Morse Fall Scale score (74.5 vs 60.0; P<0.001). No significant associations with delirium diagnosis were seen for the following lab values obtained from the EHR: mean blood urea nitrogen, serum creatinine, mean blood urea nitrogen and serum creatinine ratio of ≥18, hemoglobin and hematocrit, and albumin.
On multivariable analysis (Table 2), prior dementia (odds ratio: 5.0, 95% CI: 2.5–10.3) and Braden score of <18 (odds ratio: 2.8; 95% CI: 1.5–5.1) remained significantly associated with delirium among hospitalized patients.
Table 2.
Odds Ratios for Variables Included in Multivariable Regression Analysis
| Effect | Odds ratio | 95% Wald CI |
|---|---|---|
|
| ||
| Age at admission (over vs under 80 years) | 1.1 | 0.6, 2.2 |
| Infection | 1.3 | 0.6, 2.6 |
| Prior dementia by ICD-10 code | 5.0 | 2.5, 10.3 |
| Severity of illness | 1.9 | 1.0, 3.7 |
| Maximum Morse Fall Scale score (over vs under 45) | 1.7 | 0.8, 3.8 |
| Minimum Braden Scale score (under vs over 18) | 2.8 | 1.5, 5.1 |
ICD-10, International Classification of Diseases, Tenth Revision.
DISCUSSION
The EHR is an integral aspect of care and a component that may be used to improve earlier identification of delirium in older hospitalized patients. In this study, we explored EHR variables that we hypothesized could be associated with delirium among older inpatients at a single hospital. Multiple variables associated with delirium in previous studies, namely prior dementia and functional status,23,25 were associated with delirium within our study population. The fact that Braden score (ie, pressure ulcer risk) also was associated with delirium in our study further demonstrates the similarity between the risk factors for developing pressure ulcers and the risk factors for developing delirium.29
Conversely, several variables expected to be associated with delirium were not significantly associated with delirium in our study per multivariate analysis. These included age, infection, and Morse score. Moreover, pain, vision impairment, and fracture diagnosis also were not associated with delirium on univariate analysis. This may have been because data entered into the EHR is for clinical purposes and not documented specifically to meet research needs. Additionally, clinical text in the EHR may be effectively collected to further boost the ability to identify delirium,30 but we did not have this option. While fall risk (ie, Morse score) did not remain significantly associated with delirium on multivariate analysis, the univariate association provides evidence for recognizing that fall risk may be an appropriate clinical variable to initiate delirium prevention efforts, a takeaway consistent with reported evidence that preventing delirium decreases falls.31 Using Braden and Morse assessments as a means to flag/identify a patient’s risk for delirium is worthy of further exploration.
Of possible importance, 82.6% of our study participants were Black. While we conducted the study in an urban hospital where, on average, 60% of the inpatient population is Black, we did not specifically target enrollment of Black patients. Recent research demonstrates that the prevalence of delirium among older adults in the intensive care unit setting is not significantly different based on race.32 However, there has been minimal research examining the predictors of delirium based on race and/or other demographic variables not previously included in studies (eg, socioeconomic, hospital payor mix). Further research on how variables might differ among various ethnic groups is needed. Babulal et al verified the gaps in the inclusion of ethnoracial representation and provided several recommendations for future research, specifying that “precision medicine” and “precision public health” approaches are needed to target and continually refine identification of Alzheimer’s and related dementias.33
A relatively small proportion of our study participants were readmitted to the hospital within 30 days, resulting in a nonsignificant difference in the rates of readmission between those with and without delirium.
Limitations
This study has several limitations. First, the data obtained from the EHR were entered by clinicians for patient care at the bedside and may have quality issues, such as misclassification, inaccuracy, or missing.34 Secondly, ICD-10 codes may underrepresent the presence of certain diagnoses, such as low vision and prior dementia. Thus, any future predictive model using EHR data will be limited in performance due to missing these important predictors of delirium. Third, we were not able to obtain some variables, including medications, nutritional status, and presence of urinary catheter. Fourth, as with any delirium study, we screened for delirium once a day and may have missed an acute delirium diagnosis due to the fluctuating nature of the disease. Lastly, “other” was noted as a reason for exclusion in a large number of screened patients, causing a potential sampling error.
A strength to this prospective study was its inclusion of ADL, Morse, and Braden scores, which can be difficult-to-obtain EHR variables in purely retrospective studies. These variables are an integral part of patient assessment by the nurses in our health system and thus were easily available to us. Another strength was that both prevalent and incident delirium was identified.
Clinical Implications
A daily delirium marker that is automatically generated from the EHR may be useful for multiple reasons. First, earlier identification would lead to further assessment, confirmation of diagnosis, and early treatment. It also could save resources by enabling clinicians to focus their efforts on patients who are at the highest risk. Second, a systematic method for alerting interprofessional members of the health care team about patients’ risk for delirium would aid in identifying patients earlier in their hospitalization who may require additional post-acute care health services. Last, integrated health systems may be able to use the delirium marker as a quality measure to compare delirium rates between units and across hospitals. These outcomes are critical for the elderly population. Although current knowledge gaps prevent the clinical implementation of EHR-predictive models at this time, their future potential appears promising.35,36
In summary, this study found that participants with a prior diagnosis of dementia or lower Braden Scale score (ie, <18) were more likely to have delirium in a hospital setting. Further research is needed to develop an automated and dynamic (eg, updated daily) delirium prediction model inclusive of these clinical variables that can be tested in a larger, more clinically diverse patient population.
Patient-Friendly Recap
Hospital delirium is underrecognized by clinicians at the bedside. Timelier identification could prevent adverse events and lead to a better quality of life for older adults.
Authors analyzed electronic health record (EHR) data to learn which variables were associated with delirium in hospitalized patients.
Findings highlight the importance of having EHR-documented nursing assessments (eg, Morse Fall Scale, Braden Scale) for identifation of patients more likely to develop delirium.
Further research in pursuit of automated, EHR-based delirium prediction models that account for potential ethnoracial variations is warranted.
Supplementary Information
Acknowledgments
We would like to thank Jessica Kram for her assistance and feedback in writing this manuscript. We would also like to thank the research coordinators who developed the database and collected the data: Seth Heithaus, Robert Bertheaume, Sundeep Kalimisetty, Wajih Askar, Mehrnaz Abedian, Matthew Klein, Stacie (Snap) Bishop, and Mary Briggs-Sedlachek.
Footnotes
Author Contributions
Study design: Khan, Simpson, Malone. Data acquisition or analysis: Khan, Heslin, Simpson. Manuscript drafting: all authors. Critical revision: all authors.
Conflicts of Interest
None.
References
- 1.Oh ES, Fong TG, Hshieh TT, Inouye SK. Delirium in older patients: advances in diagnosis and treatment. JAMA. 2017;318:1161–74. doi: 10.1001/jama.2017.12067. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Marcantonio ER. Delirium in hospitalized older adults. N Engl J Med. 2017;377:1456–66. doi: 10.1056/nejmcp1605501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Leslie DL, Marcantonio ER, Zhang Y, Leo-Summers L, Inouye SK. One-year health care costs associated with delirium in the elderly population. Arch Intern Med. 2008;168:27–32. doi: 10.1001/archinternmed.2007.4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Caplan GA, Teodorczuk A, Streatfeild J, Agar MR. The financial and social costs of delirium. Eur Geriatr Med. 2020;11:105–12. doi: 10.1007/s41999-019-00257-2. [DOI] [PubMed] [Google Scholar]
- 5.Inouye SK, Charpentier PA. Precipitating factors for delirium in hospitalized elderly persons. Predictive model and interrelationship with baseline vulnerability. JAMA. 1996;275:852–7. doi: 10.1001/jama.1996.03530350034031. [DOI] [PubMed] [Google Scholar]
- 6.Helfand BK, D’Aquila ML, Tabloski P, et al. Detecting delirium: a systematic review of identification instruments for non-ICU settings. J Am Geriatr Soc. 2021;69:547–55. doi: 10.1111/jgs.16879. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Inouye SK, Foreman MD, Mion LC, Katz KH, Cooney M., Jr Nurses’ recognition of delirium and its symptoms: comparison of nurse and researcher ratings. Arch Intern Med. 2001;161:2467–73. doi: 10.1001/archinte.161.20.2467. [DOI] [PubMed] [Google Scholar]
- 8.Mistarz R, Eliott S, Whitfield A, Ernest D. Bedside nurse-patient interactions do not reliably detect delirium: an observational study. Aust Crit Care. 2011;24:126–32. doi: 10.1016/j.aucc.2011.01.002. [DOI] [PubMed] [Google Scholar]
- 9.Hayashi A, Kobayashi S, Matsui K, Akaho R, Nishimura K. The accuracy of delirium assessment by cardiologists treating heart failure inpatients: a single center retrospective survey. Biopsychosoc Med. 2020;14:15. doi: 10.1186/s13030-020-00188-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Koster S, Hensens AG, Schuurmans MJ, van der Palen J. Prediction of delirium after cardiac surgery and the use of a risk checklist. Eur J Cardiovasc Nurs. 2013;12:284–92. doi: 10.1177/1474515112450244. [DOI] [PubMed] [Google Scholar]
- 11.Inouye SK, Zhang Y, Jones RN, Kiely DK, Yang F, Marcantonio ER. Risk factors for delirium at discharge: development and validation of a predictive model. Arch Intern Med. 2007;167:1406–13. doi: 10.1001/archinte.167.13.1406. [DOI] [PubMed] [Google Scholar]
- 12.Oh SH, Park EJ, Jin Y, Piao J, Lee SM. Automatic delirium prediction system in a Korean surgical intensive care unit. Nurs Crit Care. 2014;19:281–91. doi: 10.1111/nicc.12048. [DOI] [PubMed] [Google Scholar]
- 13.Gavinski K, Carnahan R, Weckman M. Validation of the delirium observation screen scale in a hospitalized older population. J Hosp Med. 2016;11:494–7. doi: 10.1002/jhm.2580. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Racine AM, Tommet D, D’Aquila ML, et al. Machine learning to develop and internally validate a predictive model for post-operative delirium in a prospective, observational clinical cohort study of older surgical patients. J Gen Intern Med. 36:265–73. doi: 10.1007/s11606-020-06238-7. 20201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Lindroth H, Bratzke L, Purvis S, et al. Systemic review of prediction models for delirium in the older adult inpatient. BMJ Open. 2018;8(4):e019223. doi: 10.1136/bmjopen-2017-019223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Whitlock EL, Braehler MR, Kaplan JA, et al. Derivation, validation, sustained performance, and clinical impact of an electronic medical record-based perioperative delirium risk stratification tool. Anesth Analg. 2020;131:1901–10. doi: 10.1213/ane.0000000000005085. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Wong A, Young AT, Liang AS, Gonzales R, Douglas VC, Hadley D. Development and validation of an electronic health record-based machine learning model to estimate delirium risk in newly hospitalized patients without known cognitive impairment. JAMA Netw Open. 2018;1(4):e181018. doi: 10.1001/jamanetworkopen.2018.1018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Marcantonio ER, Ngo LH, O’Connor M, Jones RN, et al. 3D-CAM: derivation and validation of a 3-minute diagnostic interview for CAM-defined delirium: a cross-sectional diagnostic test study. Ann Intern Med. 2014;161:554–61. doi: 10.7326/m14-0865. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Katzman R, Brown T, Fuld P, Peck A, Schechter R, Schimmel H. Validation of a short Orientation-Memory-Concentration Test of cognitive impairment. Am J Psychiatry. 1983;140:734–9. doi: 10.1176/ajp.140.6.734. [DOI] [PubMed] [Google Scholar]
- 20.Quan H, Li B, Duncan Saunders L, et al. Assessing validity of ICD-9-CM and ICD-10 administrative data in recording clinical conditions in a unique dually coded database. Health Serv Res. 2008;43:1424–41. doi: 10.1111/j.1475-6773.2007.00822.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Mukadam N, Sampson EL. A systematic review of the prevalence, associations and outcomes of dementia in older general hospital inpatients. Int Psychogeriatr. 2011;23:344–55. doi: 10.1017/s1041610210001717. [DOI] [PubMed] [Google Scholar]
- 22.Hwang AB, Boes S, Nyffeler T, Schuepfer G. Validity of screening instruments for the detection of dementia and mild cognitive impairment in hospital inpatients: a systematic review of diagnostic accuracy studies. PloS One. 2019;14(7):e0219569. doi: 10.1371/journal.pone.0219569. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Kalimisetty S, Askar W, Fay B, Khan A. Models for predicting incident delirium in hospitalized older adults: a systematic review. J Patient Cent Res Rev. 2017;4:69–77. doi: 10.17294/2330-0698.1414. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Sundararajan V, Henderson T, Perry C, Muggivan A, Quan H, Ghali WA. New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality. J Clin Epidemiol. 2004;57:1288–94. doi: 10.1016/j.jclinepi.2004.03.012. [DOI] [PubMed] [Google Scholar]
- 25.Halladay CW, Sillner AY, Rudolph JL. Performance of electronic prediction rules for prevalent delirium at hospital admission. JAMA Netw Open. 2018;1(4):e181405. doi: 10.1001/jamanetworkopen.2018.1405. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW. Studies of illness in the aged. The index of ADL: a standardized measure of biological and psychosocial function. JAMA. 1963;185:914–9. doi: 10.1001/jama.1963.03060120024016. [DOI] [PubMed] [Google Scholar]
- 27.Morse JM. Preventing Patient Falls. SAGE Publications; 1997. [Google Scholar]
- 28.Sillner AY, Holle CL, Rudolph JL. The overlap between falls and delirium in hospitalized older adults: a systematic review. Clin Geriatr Med. 2019;35:221–36. doi: 10.1016/j.cger.2019.01.004. [DOI] [PubMed] [Google Scholar]
- 29.Inouye SK, Studenski S, Tinetti ME, Kuchel GA. Geriatric syndromes: clinical, research, and policy implications of a core geriatric concept. J Am Geriatr Soc. 2007;55:780–91. doi: 10.1111/j.1532-5415.2007.01156.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.McCoy TH, Jr, Chaukos DC, Snapper LA, Hart KL, Stern TA, Perlis RH. Enhancing delirium case definitions in electronic health records using clinical free text. Psychosomatics. 2017;58:113–20. doi: 10.1016/j.psym.2016.10.007. [DOI] [PubMed] [Google Scholar]
- 31.Hshieh TT, Yang T, Gartaganis SL, Yue J, Inouye SK. Hospital Elder Life Program: systematic review and meta-analysis of effectiveness. Am J Geriatr Psychiatry. 2018;26:1015–33. doi: 10.1016/j.jagp.2018.06.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Khan BA, Perkins A, Hui SL, et al. Relationship between African-American race and delirium in the ICU. Crit Care Med. 2016;44:1727–34. doi: 10.1097/ccm.0000000000001813. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Babulal GM, Quiroz YT, Albensi BC, et al. Perspectives on ethnic and racial disparities in Alzheimer’s disease and related dementias: update and areas of immediate need. Alzheimers Dement. 2019;15:292–312. doi: 10.1016/j.jalz.2018.09.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Campbell NL, Cantor BB, Hui SL, et al. Race and documentation of cognitive impairment in hospitalized older aduls. J Am Geriatr Soc. 2014;62:506–11. doi: 10.1111/jgs.12691. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Rose S. Machine learning for prediction in electronic health data. JAMA Netw Open. 2018;1(4):e181404. doi: 10.1001/jamanetworkopen.2018.1404. [DOI] [PubMed] [Google Scholar]
- 36.Lee TC, Shah NU, Haack A, Bazter SL. Clinical implementation of predictive models embedded within electronic health record systems: a systematic review. Informatics (MDPI) 2020;7(3):25. doi: 10.3390/informatics7030025. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.

