Abstract
Background
The goal of this study was to develop and validate a dynamic risk model to predict daily changes in acute brain dysfunction (ie, delirium and coma), discharge, and mortality in ICU patients.
Methods
Using data from a multicenter prospective ICU cohort, a daily acute brain dysfunction-prediction model (ABD-pm) was developed by using multinomial logistic regression that estimated 15 transition probabilities (from one of three brain function states [normal, delirious, or comatose] to one of five possible outcomes [normal, delirious, comatose, ICU discharge, or died]) using baseline and daily risk factors. Model discrimination was assessed by using predictive characteristics such as negative predictive value (NPV). Calibration was assessed by plotting empirical vs model-estimated probabilities. Internal validation was performed by using a bootstrap procedure.
Results
Data were analyzed from 810 patients (6,711 daily transitions). The ABD-pm included individual risk factors: mental status, age, preexisting cognitive impairment, baseline and daily severity of illness, and daily administration of sedatives. The model yielded very high NPVs for “next day” delirium (NPV: 0.823), coma (NPV: 0.892), normal cognitive state (NPV: 0.875), ICU discharge (NPV: 0.905), and mortality (NPV: 0.981). The model demonstrated outstanding calibration when predicting the total number of patients expected to be in any given state across predicted risk.
Conclusions
We developed and internally validated a dynamic risk model that predicts the daily risk for one of three cognitive states, ICU discharge, or mortality. The ABD-pm may be useful for predicting the proportion of patients for each outcome state across entire ICU populations to guide quality, safety, and care delivery activities.
Key Words: coma, delirium, ICU, mortality, prediction
Abbreviations: ABD-pm, acute brain dysfunction-prediction model; APACHE II, Acute Physiology and Chronic Health Evaluation II; E-PRE-DELIRIC, Early Prediction of Delirium in ICU Patients; NPV, negative predictive value; PPV, positive predictive value; PRE-DELIRIC, Prediction of Delirium in ICU Patients; SOFA, Sequential Organ Failure Assessment
Acute brain dysfunction (ie, delirium and coma) during critical illness is a form of organ dysfunction that has a substantial impact on patients’ clinical management. The majority of ICU patients experience acute brain dysfunction at some point during their ICU stay.1, 2, 3, 4 ICU delirium is an independent risk factor for death,5, 6, 7, 8 cognitive dysfunction,4, 9, 10 longer lengths of stay,11, 12 and higher costs.13, 14
Despite the clinical significance of acute brain dysfunction and its recognition as an indicator of quality and safety,15 there are few risk prediction tools that adequately predict the number of patients likely to experience acute brain dysfunction. To date, only two ICU delirium prediction models have been published: the Early Prediction of Delirium in ICU Patients (E-PRE-DELIRIC)16 model and the Prediction of Delirium in ICU Patients (PRE-DELIRIC) model.17, 18 These models consist of up to 10 clinical factors present up to the first 24 ICU hours. They are excellent models that predict the first-time incidence of ICU delirium. Although important, these models are limited to patients without delirium at admission, do not factor in dynamic daily risk factors, and cannot predict recurrent or ongoing delirium.
One may complement the PRE-DELIRIC models with a tool to assess the risk for delirium each day while in the ICU. For example, when a patient arrives at the ICU, his or her brain function is measured to be in one of three mutually exclusive states: normal, delirious, or comatose. Each subsequent day, patients may continue in their current state, transition to a new brain function state, be discharged from the ICU, or die. Each of these outcomes might be predicted by using a transitional modeling approach, which we call the acute brain dysfunction-prediction model (ABD-pm). Table 1 summarizes comparisons between the PRE-DELIRIC and ABD-pm modeling approaches.
Table 1.
Model Characteristic | PRE-DELIRIC and E-PRE-DELIRIC (Logistic Model) | Acute Brain Dysfunction-Prediction Model (Multinomial Logistic Regression) |
---|---|---|
Time-varying predictors (daily) | No | Yes |
Outcome | Delirium | Next day status: normal, delirium, coma, ICU discharge, ICU death |
Outcome frequency | One per patient | One per patient-day |
Competing risk | Does not account | Accounts for competing risks |
Excludes patients with delirium at admission | Yes | No |
E-PRE-DELIRIC = Early Prediction of Delirium in ICU Patients; PRE-DELIRIC = Prediction of Delirium in ICU Patients.
The goal of the present study was to develop a tool to accurately predict the number of patients in any given brain function state the next day, discharged, or died.
Patients and Methods
Study Design and Population
We included patients enrolled in the Bringing to Light the Risk Factors and Incidence of Neuropsychological Dysfunction in ICU Survivors (BRAIN-ICU).4, 19 This study enrolled a cohort of critically ill patients at two tertiary care medical centers (Vanderbilt University Medical Center and St. Thomas Hospital, Nashville, Tennessee).4, 20
The study protocol and the eligibility criteria have been published4 and are presented in e-Appendix 1. Adults (≥ 18 years old) admitted to medical or surgical ICUs were included who received treatment for respiratory failure or shock.21, 22 We excluded those with recent ICU exposure and who could not perform in-person follow-up. The institutional review board at Vanderbilt University Medical Center approved the study.
Candidate Predictors
Candidate factors at ICU admission included the following: age, ICU type, use of medications to treat Alzheimer’s disease, Acute Physiology and Chronic Health Evaluation II (APACHE II) acute physiology score,23 and mechanical ventilation. Daily factors included: current brain function status,17, 24 mechanical ventilation, sepsis,17, 25 modified Sequential Organ Failure Assessment (SOFA) score,26 and ICU day number (daily updated length of stay, modeled by using a second-order polynomial).27, 28 Additional daily factors included administration of benzodiazepines,24, 29 opiates,29 propofol,17, 24 antipsychotic agents,29 and statins.30 Detailed factor definitions, rationale, and missing data approaches are available in e-Appendix 1.
Model Outcomes
The outcome of the dynamic transition model included one of five mutually exclusive transition states for each day: 1) normal, 2) delirious, 3) comatose, 4) ICU discharge, or 5) ICU death. If the patient had two different states during the same day, the worst outcome on that day (coma/normal = coma; coma/delirium = coma; delirium/normal = delirium; death/any state = death; and discharged/any state = discharged) was considered. The first three states of acute brain dysfunction were defined by using the Confusion Assessment Method for the Intensive Care Unit2, 31, 32 and the Richmond Agitation Sedation Scale33, 34 (e-Table 1). Arousal level and delirium were assessed in the ICU at least twice per day by trained research nurse coordinators in all patients for up to 30 days. ICU discharge and death were final states from which no further transitions were possible.35 With five possible outcomes, there were 15 possible transitions (Fig 1).
Statistical Analysis
A transition model of in-hospital outcomes is important when considering outcomes that change from day to day.36 The ABD-pm dynamic state transition model was implemented by using multinomial logistic regression to estimate the next-day probability of acute brain dysfunction, ICU discharge, and ICU death.37 The model estimated the probability of transition to one of five function states (normal, delirium, comatose, discharged from ICU, or deceased), adjusting for current brain function state (normal, delirious, or comatose), baseline risk factors, and time-varying clinical risk factors acquired during the ICU stay.38, 39 Each transition day was treated as conditionally independent (ie, conditional on the previous mental state and other daily and baseline patient factors).
The final model was selected sequentially. The base model included the current brain function status only. This step was followed by addition of enrollment demographic/comorbidity predictors, severity of illness factors, daily medications, and ICU length of stay. At each stage, likelihood ratio testing was conducted to assess the independent prognostic significance for each factor group. Also, we included interactions of study day with baseline APACHE II score and baseline mechanical ventilation to model the differential impact of these baseline covariates on the first day of ICU stay vs later days. Internal validation and interval estimation of model fit statistics were performed by using a bootstrap method with 5,000 resamples.40
Discrimination was assessed by calculating sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each transition. Accuracy (the proportion of patients with correct classifications) was additionally assessed. Unlike traditional prediction models of binary outcomes (eg, mortality) that assess discrimination by using receiver-operating characteristic curves, there is no agreed-upon method for multiclass outcomes.41 Because clinicians often want to understand how a test predicts a specific outcome, the discussion was focused on PPV and NPV. To provide greater clinical context, we additionally calculated the predicted outcome states for the second ICU day in a randomly selected group of 100 patients admitted to the ICU, 26%, 33%, and 41% of whom were normal, delirious, and comatose, respectively, upon admission. These 100 patients had the following characteristics: mean age, 60.6 years; mean APACHE II score, 17.3; severe sepsis, 62%; mechanically ventilated, 87%; and had received benzodiazepines, 34%.
To assess calibration, we used the model to estimate transitional probabilities for each patient transition. Model-based estimates were compared with empirical estimates by using a scatter plot smoother (ie, a calibration plot). A well-calibrated model has good agreement between the model-based and empirical estimates of transition probabilities. Calibration plots were constructed for each of the five outcome states, regardless of the starting brain function state. We refer to this approach as “population-level” calibration, as the predicted outcomes represent sums of the individual transition-level probabilities. For example, the predicted number of delirious patients would be the total number of patients predicted to transition from coma, delirium, or normal to delirium over the next day. Population-level calibration was assessed across all study days as well as for the first ICU day, as the first ICU day is a critical day for ICU management. Individual transition-level calibration plots were also constructed for each of the 15 possible transitions.
The statistical software package R, version 2.15.2, and the R package “nnet” were used for all analyses (R Core Team [2012]; R Foundation for Statistical Computing, Vienna, Austria; ISBN 3-900051-07-0; http://www.R-project.org/).
Results
A total of 826 patients were enrolled in the BRAIN ICU study. Five patients withdrew consent and their collected data; 11 patients died or withdrew within 24 h of ICU admission. Thus, 810 patients were included (e-Fig 1), with a median (interquartile range) age of 61 years (51-71 years) and high severity of illness (APACHE II score: 25 [19-31]) (Table 2),23, 26, 42, 43 with 6,711 day-to-day transitions. Delirium affected 606 (75%) patients at some time during their ICU stay. Patients were normal, delirious, comatose, discharged, or died for 29%, 30%, 29%, 10%, and 2% of ICU days, respectively (Table 3). Additional daily covariates are given in Table 3.26
Table 2.
Patient Characteristic | In-Hospital Cohorta (N = 810) |
---|---|
Age at enrollment, y | 61 (51-71) |
White | 729 (90%) |
Male | 415 (51%) |
Education, y | 12 (12-14) |
Short IQCODE score42,b | 3 (3-3) |
Charlson comorbidity index43 | 2 (1-4) |
Severity of illness, APACHE II score23 at enrollmentc | 25 (19-31) |
Organ failure, SOFA score26 at enrollmentd | 9 (7-12) |
Admission diagnosis, sepsis | 241 (30%) |
ICU type | |
Medical ICU | 549 (68%) |
Surgical ICU | 261 (32%) |
Days of MV among those receiving MV | 3 (1-8) |
Ever observed delirium | 606 (75%) |
Days of delirium among those observed with delirium | 4 (2-10) |
Ever observed coma | 513 (63%) |
Days of coma among those observed with coma | 3 (2-6) |
ICU mortality | 130 (16%) |
ICU length of stay, d | 5 (3-11) |
Hospital length of stay, d | 10 (6-17) |
APACHE = Acute Physiology and Chronic Health Evaluation; IQCODE = Informant Questionnaire On Cognitive Decline in the Elderly; MV = mechanical ventilation; SOFA = Sequential Organ Failure Score.
Median (interquartile range) unless otherwise indicated.
Scores on the Short IQCODE range from 1 to 5, with a score of 3 indicating no change in cognition over the past 10 years, a score < 3 indicating improvement, and a score > 3 indicating decline in cognition, compared with 10 years before. A score ≥ 3.3 indicates an increased probability of cognitive impairment, and a score ≥ 3.6 indicates preexisting cognitive impairment.
Scores on the APACHE II range from 0 to 71, with higher scores indicating worse outcomes.
Scores on the SOFA range from 0 to 24 (from 0 to 4 for each of six organ systems), with higher scores indicating more severe organ dysfunction. A modified SOFA score was used in the regression models, which excluded the Glasgow Coma Scale components (hence ranging from 0-20) because coma was included separately in our models.
Table 3.
Daily Exposure | No. (%) |
---|---|
Total no. of ICU transitions | 6,711 (100) |
Daily ventilator use | 5,006 (75) |
Daily modified SOFA score26 | |
SOFA = 1 | 48 (1) |
SOFA = 5 | 780 (12) |
SOFA = 10 | 295 (4) |
SOFA = 15 | 28 (<1) |
Daily sepsis | 4,288 (64) |
Received benzodiazepines | 2,232 (33) |
Received opiates | 3,580 (53) |
Received propofol | 1,496 (22) |
Received antipsychotic agents | 1,183 (18) |
Transitional outcomes | |
Normal | 1,974 (29) |
Delirious | 1,995 (30) |
Comatose | 1,918 (29) |
Discharged | 697 (10) |
Died | 127 (2) |
See Table 2 for expansion of abbreviation.
Final Model Predictors
Of the 15 potential risk factors considered for predicting the various transition states, only daily statin administration was excluded, given the low likelihood ratio statistic (18.3; P = .25) found in reduced models without statins. All other variables remained in the final model, consisting of 14 individual risk factors and two prespecified interaction terms.
Model Discrimination
Overall, specificity and NPV were greater than sensitivity and PPV for all transitional outcome states. The bootstrap-validated sensitivity and specificity for delirious predictions were 0.597 (95% CI, 0.576-0.618) and 0.792 (95% CI, 0.778-0.803), with a PPV and NPV of 0.548 (95% CI, 0.539-0.556) and 0.823 (95% CI, 0.817-0.828), respectively. Compared with the overall daily prevalence of each brain function outcome (ie, normal, 0.29 of ICU days; delirious, 0.30 of ICU days; and comatose, 0.29 of ICU days), the PPVs (ie, posttest probability) of each outcome measure were substantially greater. Overall accuracy was highest for the deceased state (0.981 [95% CI, 0.979-0.981]), which is largely due to the rarity of mortality in this population (Table 4).23, 26
Table 4.
Variable | Normal | Delirious | Comatose | Discharged | Deceased |
---|---|---|---|---|---|
Sensitivity | 0.707 (0.687-0.723) | 0.597 (0.576-0.618) | 0.747 (0.719-0.766) | 0.112 (0.07-0.159) | 0.015 (0-0.039) |
Specificity | 0.844 (0.835-0.854) | 0.792 (0.778-0.803) | 0.835 (0.826-0.846) | 0.984 (0.975-0.991) | 0.999 (0.997-1) |
PPV | 0.654 (0.644-0.663) | 0.548 (0.539-0.556) | 0.644 (0.637-0.652) | 0.455 (0.393-0.522) | 0.291 (0-1) |
NPV | 0.874 (0.868-0.879) | 0.823 (0.817-0.828) | 0.892 (0.883-0.898) | 0.905 (0.902-0.909) | 0.981 (0.981-0.982) |
Accuracya | 0.804 (0.8-0.807) | 0.734 (0.729-0.738) | 0.81 (0.807-0.812) | 0.894 (0.889-0.897) | 0.981 (0.979-0.981) |
Data are presented as bootstrap-validated model fit statistics (95% CI). The final acute brain dysfunction model contains five variables at ICU admission and nine variables measured each day. Baseline variable included the following: age; medical vs surgical ICU type; use of medications to treat Alzheimer’s disease at hospital admission (as a marker of a history of dementia); APACHE II score23; and mechanical ventilation. Daily predictors included the following: current day’s brain function status (normal, delirium, or coma); MV status; sepsis; modified SOFA score26; receipt of benzodiazepines, opiates, propofol, and antipsychotic agents; and ICU length of stay. NPV = negative predictive value; PPV = positive predictive value. See Table 2 legend for expansion of other abbreviations.
Accuracy is calculated as the total number of true positive transition predictions and true negative transition predictions divided by the total number of transitions.
Population-level Calibration
Across the population and over the entire study period, model predictions of the population’s clinical conditions (remain in the ICU, discharge from the ICU, or death) closely approximated actual observations (Fig 2) with slight deviations beyond 7 days. Population-level calibration for each outcome state was outstanding for normal, delirium, coma, and ICU discharge but not for mortality (Fig 3). Population-level predictions of death, however, were greater than observed at higher predicted probabilities. When assessing the first day only, population-level prediction continued to be excellent for brain function outcomes (e-Fig 2), whereas predictions for discharge on the first day were greater than observed.
Transition-level Calibration or Individual-level Calibration
Calibration was additionally examined at the “transition-level” or “individual-level” (e-Figs 3-5). For specific transitions, the model was well calibrated for transitions from normal to delirium, delirium to delirium, and coma to delirium. The model, however, underpredicted transitions of delirium to deceased and comatose to discharge, as indicated by the lower confidence bound crossing the slope of 1 that would indicate perfect calibration. Overpredictions were most apparent for the transitions of normal to coma, normal to deceased, and delirious to normal, as indicated by the upper confidence bound crossing the slope of 1. Deviations were most notable at the high range of predictions in which the 95% confidence bands were the widest and were related to the rare outcome frequency. One can see the daily transition probabilities based upon the previous day brain function state in e-Table 2.
Discussion
We have developed an ABD-pm that predicts daily changes in brain function status while in the ICU and accounts for competing outcomes of discharge and death. The ABD-pm may be particularly useful for predicting a distribution of cognitive states in the ICU, ICU discharge, and mortality across the entire ICU population. Furthermore, the ABD-pm model may aid inpatient counseling regarding the next-day probability of being alive (NPV for mortality: 0.981) or of being in the impaired cognitive states of either delirium or coma (NPV for normal cognitive state: 0.874) or of remaining in the ICU (NPV for discharge: 0.905).
Although complementary to the PRE-DELIRIC17, 18 ICU delirium prediction models, the ABD-pm differs in important ways. First, the ABD-pm not only predicts day-to-day changes in delirium but additionally predicts transitions to competing brain function states (coma and normal), discharge, and death. Coma, for example, not only is predictive of delirium but entails its own prognostic significance.6, 44 If observed delirium rates are increased, it is potentially important to understand if the increases were predicted with increasing transitions from coma (which signals improving brain function), or rather from increasing predicted transitions from normal (which signals worsening brain function). A second difference of the daily ABD-pm is its dynamic nature to predict next-day acute brain dysfunction, discharge, or death on any given ICU day. Importantly, acute brain dysfunction and its risk factors fluctuate from day to day, and the ABD-pm can incorporate these time covariates to provide updated information each ICU day.
The unique model characteristics of the daily ABD-pm allows predictions for patient groups. For example, patients already admitted with delirium (prevalent delirium) are excluded in an ICU delirium incidence model (eg, PRE-DELIRIC) but are included in the daily ABD-pm. This population is important, especially given the known association of delirium duration with long-term cognitive dysfunction4, 9 and mortality.5, 6 Given a population’s characteristics, one can predict the number of patients likely to be alive in a given cognitive state, discharged, or deceased, on any given day. For example, using our prediction model, we calculated the predicted outcome states for the second ICU day in a randomly selected group of 100 patients admitted to the ICU, 26%, 33%, and 41% of whom were normal, delirious, and comatose, respectively, upon admission and had the following characteristics: mean age, 60.6 years; mean APACHE II score, 17.3; severe sepsis, 62%; mechanically ventilated, 87%; and had received benzodiazepines, 34%. This group would be predicted to have the following outcomes: 19 normal, 25 delirious, 41 comatose, 13 ICU discharges, and 2 ICU deaths on the next day. This information can have system-wide implications in resource planning. For example, consider this population in an ICU or across a hospital’s critical care population. One would know that the population of comatose patients will likely require mechanical ventilation and not be able to participate in ambulatory activities, whereas patients expected to be discharged will require additional ward staff and beds. Because of the high costs of ICUs, precise quality assurance and utilization management strategies are essential.
Additional system-wide implications for the ABD-pm model include the ability to apply predictions for performance measurement. For example, predicted probabilities could be used for benchmarking, by comparing the number of observed to expected outcomes states, for a specific ICU day, or across all ICU days. A population with more delirium and coma than predicted may prompt an investigation into population-level treatment practices that are having a negative impact on acute brain dysfunction outcomes. Alternatively, if there is less delirium and coma than predicted, one may learn of new processes leading to reductions in acute brain dysfunction. Each may be important tools in evaluating ICU quality and safety.
Clinicians often wish to know how well a test predicts an outcome. PPV is the probability that a positive test result predicts a specified outcome, whereas NPV is the probability that a negative test result predicts the absence of that same outcome. At the transition or individual level, the daily ABD-pm exhibited moderate PPV and outstanding NPV. It is not surprising that both PPV would be low and that calibration would be poor for rare transitions (eg, normal to coma, normal to deceased, comatose to discharge) that occur in < 2% of patient days. This does not mean that the model does not add information. For example, whereas the pretest probability of mortality (the overall prevalence of outcomes) is 0.02, the PPV is 0.29. Therefore, the model substantially increases the predicted probability of mortality, although not to the level that an individual clinician would make a specific clinical decision based on that information. Future studies that include a larger sample size may provide a greater number of such transition states and reduce the uncertainty in these predictions; however, due to lower prevalence, the PPV may still be below an actionable level. In contrast, the high NPV for outcome states may be more clinically useful for the ABD-pm model. For example, a high NPV (0.875) for a normal state may guide a clinician’s counseling of a caregiver regarding expectations for the patient’s ongoing neurologic dysfunction. Likewise, a high NPV for ICU discharge (0.905) may be helpful for a clinician or ICU manager to understand the likely need for continued ICU care for a patient.
In addition to informing prognosis and care needs for an individual patient, transition-level predictions may help risk-stratify patients to prioritize care. For example, a provider (eg, a physical therapist) may be able to identify those patients most likely to experience delirium the following day (ie, those having the highest predicted probability for delirium). Although the PPV may not allow for precise predictions of delirium, the physical therapist (a limited resource) will still be able to prioritize early mobility interventions (a potent delirium prevention intervention)45 to those patients with the highest predicted probabilities. Another example would be for targeting enrollment into a clinical trial to include the patients at the highest risk of transitioning into delirium from a specific state, thus enriching a treatment population for those at the highest risk.
Future research is needed to externally validate our model. Validation could allow development of an ICU report card that summarizes adherence to evidence-based preventive measures in the ICU and could be used to identify areas needing improvement and to drive quality improvement efforts. Efforts to improve quality need to be measured to show whether these practices lead to effective improvement in ICU care. Validation of this model could help to identify and compare best practices within an organization, to compare current practice over time and also between organizations to judge performance and identify improvements that have proven to be successful in other organizations.
There are important limitations to consider in the daily ABD-pm. First, this model was developed and internally validated; thus, the model will require external validation. Another limitation is that the outcomes were measured only twice daily. Delirium and coma may change more frequently, although the present data would be unlikely to be biased in one direction. The study model was also limited by only relying on data prospectively available in our electronic medical record and in a coded format. Although some other factors are predictive, we believed it was critical to use predictors likely to be found in electronic medical records to simplify future implementation. Some predictors were less prevalent during the time of study (eg, dexmedetomidine) that may be predictive and warrant investigation in future models.
Conclusions
We showed that the daily ABD-pm can predict the dynamic course of acute brain dysfunction for an ICU population. In addition, the model predicts the risk of death and discharge, both of which are competing risks for acute brain dysfunction. The daily ABD-pm can be applied regardless of whether patients are admitted to the ICU with delirium, which will serve as an advantage when testing whether this instrument is valid as a surveillance tool for acute brain dysfunction outcomes across ICU populations.
Acknowledgments
Author contributions: A. M. and E. E. V. had full access to all study data and take responsibility for the integrity of the data and the accuracy of the data analysis. A. M., E. E. V., P. P. P., T. D. G., and E. W. E. were responsible for the design and conduct of the study; M. S. S. and R. C. performed the statistical analysis; A. M., E. W. E., P. P. P., and E. E. V. drafted the manuscript; and E. W. E., P. P. P., T. D. G., and E. E. V. obtained funding. All authors were responsible for data acquisition, analysis, and interpretation of the data; critical revision of the article for important intellectual content; and final approval of the article.
Financial/nonfinancial disclosures: The authors have reported to CHEST the following: P. P. P. has received honoraria from Hospira. E. W. E. has received honoraria from Pfizer and Orion. None declared (A. M., M. S. S., R. C., T. D. G., A. Y. K., L. M. P., K. G. M. M., R. S. D., E. E. V.).
Role of the sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. The authors were independent from study sponsors.
Additional information: The e-Appendix, e-Tables, and e-Figures can be found in the Supplemental Materials section of the online article.
Footnotes
FUNDING/SUPPORT: Dr Pandharipande has received funding from the National Institutes of Health (NIH) [Grants AG027472; HL111111, AG035117], VA Clinical Science Research and Development Science [VA Career Development Award]. Dr Girard has received funding from the NIH [Grants AG034257] and VA Tennessee Valley Geriatric Research Education and Clinical Center. Dr Ely has received funding from the NIH [Grants AG027472; HL111111, AG035117], CTSA award No. UL1 TR002243 from the National Center for Advancing Translational Sciences, and VA Tennessee Valley Geriatric Research Education and Clinical Center. Dr Vasilevskis has received funding from the NIH [Grant AG040157] and the VA Tennessee Valley Geriatric Research Education and Clinical Center.
Supplementary Data
References
- 1.Dubois M.J., Bergeron N., Dumont M., Dial S., Skrobik Y. Delirium in an intensive care unit: a study of risk factors. Intensive Care Med. 2001;27(8):1297–1304. doi: 10.1007/s001340101017. [DOI] [PubMed] [Google Scholar]
- 2.Ely E.W., Inouye S.K., Bernard G.R. Delirium in mechanically ventilated patients: validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU) JAMA. 2001;286(21):2703–2710. doi: 10.1001/jama.286.21.2703. [DOI] [PubMed] [Google Scholar]
- 3.Pisani M.A., Murphy T.E., Van Ness P.H., Araujo K.L., Inouye S.K. Characteristics associated with delirium in older patients in a medical intensive care unit. Arch Intern Med. 2007;167(15):1629–1634. doi: 10.1001/archinte.167.15.1629. [DOI] [PubMed] [Google Scholar]
- 4.Pandharipande P.P., Girard T.D., Jackson J.C. Long-term cognitive impairment after critical illness. N Engl J Med. 2013;369(14):1306–1316. doi: 10.1056/NEJMoa1301372. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Pisani M.A., Kong S.Y., Kasl S.V., Murphy T.E., Araujo K.L., Van Ness P.H. Days of delirium are associated with 1-year mortality in an older intensive care unit population. Am J Respir Crit Care Med. 2009;180(11):1092–1097. doi: 10.1164/rccm.200904-0537OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Ely E.W., Shintani A., Truman B. Delirium as a predictor of mortality in mechanically ventilated patients in the intensive care unit. JAMA. 2004;291(14):1753–1762. doi: 10.1001/jama.291.14.1753. [DOI] [PubMed] [Google Scholar]
- 7.Salluh J.I., Wang H., Schneider E.B. Outcome of delirium in critically ill patients: systematic review and meta-analysis. BMJ. 2015;350:h2538. doi: 10.1136/bmj.h2538. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Lin S.M., Liu C.Y., Wang C.H. The impact of delirium on the survival of mechanically ventilated patients. Crit Care Med. 2004;32(11):2254–2259. doi: 10.1097/01.ccm.0000145587.16421.bb. [DOI] [PubMed] [Google Scholar]
- 9.Girard T.D., Jackson J.C., Pandharipande P.P. Delirium as a predictor of long-term cognitive impairment in survivors of critical illness. Crit Care Med. 2010;38(7):1513–1520. doi: 10.1097/CCM.0b013e3181e47be1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wolters A.E., van Dijk D., Pasma W. Long-term outcome of delirium during intensive care unit stay in survivors of critical illness: a prospective cohort study. Crit Care. 2014;18(3):R125. doi: 10.1186/cc13929. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Ely E.W., Gautam S., Margolin R. The impact of delirium in the intensive care unit on hospital length of stay. Intensive Care Med. 2001;27(12):1892–1900. doi: 10.1007/s00134-001-1132-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Thomason J.W., Shintani A., Peterson J.F., Pun B.T., Jackson J.C., Ely E.W. Intensive care unit delirium is an independent predictor of longer hospital stay: a prospective analysis of 261 non-ventilated patients. Crit Care. 2005;9(4):R375–R381. doi: 10.1186/cc3729. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Milbrandt E.B., Deppen S., Harrison P.L. Costs associated with delirium in mechanically ventilated patients. Crit Care Med. 2004;32(4):955–962. doi: 10.1097/01.ccm.0000119429.16055.92. [DOI] [PubMed] [Google Scholar]
- 14.Leslie D.L., Marcantonio E.R., Zhang Y., Leo-Summers L., Inouye S.K. One-year health care costs associated with delirium in the elderly population. Arch Intern Med. 2008;168(1):27–32. doi: 10.1001/archinternmed.2007.4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Federal Register: April 30, 2008 (Volume 73, Number 84). Proposed Rules, Page 23527-23576. https://www.gpo.gov/fdsys/pkg/FR-2008-04-30. Accessed January 12, 2018.
- 16.Wassenaar A., van den Boogaard M., van Achterberg T. Multinational development and validation of an early prediction model for delirium in ICU patients. Intensive Care Med. 2015;41(6):1048–1056. doi: 10.1007/s00134-015-3777-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.van den Boogaard M., Pickkers P., Slooter A.J. Development and validation of PRE-DELIRIC (PREdiction of DELIRium in ICu patients) delirium prediction model for intensive care patients: observational multicentre study. BMJ. 2012;344:e420. doi: 10.1136/bmj.e420. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.van den Boogaard M., Schoonhoven L., Maseda E. Recalibration of the delirium prediction model for ICU patients (PRE-DELIRIC): a multinational observational study. Intensive Care Med. 2014;40(3):361–369. doi: 10.1007/s00134-013-3202-7. [DOI] [PubMed] [Google Scholar]
- 19.National Institutes of Health Clinical Center. The BRAIN Intensive Care Unit (ICU) Study: Bringing to Light the Risk Factors (BRAIN-ICU). NCT00392795. ClinicalTrials.gov. Bethesda, MD: National Institutes of Health; 2006. http://clinicaltrials.gov/ct2/show/NCT00392795. Updated March 23, 2017.
- 20.Jackson J.C., Pandharipande P.P., Girard T.D. Depression, post-traumatic stress disorder, and functional disability in survivors of critical illness in the BRAIN-ICU study: a longitudinal cohort study. Lancet Respir Med. 2014;2(5):369–379. doi: 10.1016/S2213-2600(14)70051-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Dellinger R.P., Carlet J.M., Masur H. Surviving Sepsis Campaign guidelines for management of severe sepsis and septic shock. Crit Care Med. 2004;32(3):858–873. doi: 10.1097/01.ccm.0000117317.18092.e4. [DOI] [PubMed] [Google Scholar]
- 22.Dellinger R.P., Levy M.M., Carlet J.M. Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock: 2008. Crit Care Med. 2008;36(1):296–327. doi: 10.1097/01.CCM.0000298158.12101.41. [DOI] [PubMed] [Google Scholar]
- 23.Knaus W.A., Draper E.A., Wagner D.P., Zimmerman J.E. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–829. [PubMed] [Google Scholar]
- 24.Pandharipande P., Shintani A., Peterson J. Lorazepam is an independent risk factor for transitioning to delirium in intensive care unit patients. Anesthesiology. 2006;104(1):21–26. doi: 10.1097/00000542-200601000-00005. [DOI] [PubMed] [Google Scholar]
- 25.Pandharipande P.P., Girard T., Sanders R.D., Thompson J.L., Maze M., Ely E.W. Comparison of sedation with dexmedetomidine versus lorazepam in septic ICU patients. Crit Care. 2008;12(suppl 2) [Google Scholar]
- 26.Ferreira F.L., Bota D.P., Bross A., Melot C., Vincent J.L. Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA. 2001;286(14):1754–1758. doi: 10.1001/jama.286.14.1754. [DOI] [PubMed] [Google Scholar]
- 27.Roberts B., Rickard C.M., Rajbhandari D. Multicentre study of delirium in ICU patients using a simple screening tool. Aust Crit Care. 2005;18(1) doi: 10.1016/s1036-7314(05)80019-0. 6, 8-9, 11-14 passim. [DOI] [PubMed] [Google Scholar]
- 28.Pandharipande P., Cotton B.A., Shintani A. Prevalence and risk factors for development of delirium in surgical and trauma intensive care unit patients. J Trauma. 2008;65(1):34–41. doi: 10.1097/TA.0b013e31814b2c4d. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Pisani M.A., Murphy T.E., Araujo K.L., Slattum P., Van Ness P.H., Inouye S.K. Benzodiazepine and opioid use and the duration of intensive care unit delirium in an older population. Crit Care Med. 2009;37(1):177–183. doi: 10.1097/CCM.0b013e318192fcf9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Morandi A., Hughes C.G., Girard T.D., McAuley D.F., Ely E.W., Pandharipande P.P. Statins and brain dysfunction: a hypothesis to reduce the burden of cognitive impairment in patients who are critically ill. Chest. 2011;140(3):580–585. doi: 10.1378/chest.10-3065. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Ely E.W., Margolin R., Francis J. Evaluation of delirium in critically ill patients: validation of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) Crit Care Med. 2001;29(7):1370–1379. doi: 10.1097/00003246-200107000-00012. [DOI] [PubMed] [Google Scholar]
- 32.Gusmao-Flores D., Salluh J.I., Chalhub R.A., Quarantini L.C. The Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) and Intensive Care Delirium Screening Checklist (ICDSC) for the diagnosis of delirium: a systematic review and meta-analysis of clinical studies. Crit Care. 2012;16(4):R115. doi: 10.1186/cc11407. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Ely E.W., Truman B., Shintani A. Monitoring sedation status over time in ICU patients: reliability and validity of the Richmond Agitation-Sedation Scale (RASS) JAMA. 2003;289(22):2983–2991. doi: 10.1001/jama.289.22.2983. [DOI] [PubMed] [Google Scholar]
- 34.Sessler C.N., Gosnell M.S., Grap M.J. The Richmond Agitation-Sedation Scale: validity and reliability in adult intensive care unit patients. Am J Respir Crit Care Med. 2002;166(10):1338–1344. doi: 10.1164/rccm.2107138. [DOI] [PubMed] [Google Scholar]
- 35.Shahar Y. Dimensions of time in illness: an objective view. Ann Intern Med. 2000;132(1):45–53. doi: 10.7326/0003-4819-132-1-200001040-00008. [DOI] [PubMed] [Google Scholar]
- 36.Peelen L., de Keizer N.F., Jonge E., Bosman R.J., Abu-Hanna A., Peek N. Using hierarchical dynamic Bayesian networks to investigate dynamics of organ failure in patients in the intensive care unit. J Biomed Inform. 2010;43(2):273–286. doi: 10.1016/j.jbi.2009.10.002. [DOI] [PubMed] [Google Scholar]
- 37.Agresti A. 2nd ed. Wiley-Interscience; New York, NY: 2002. Categorical Data Analysis. [Google Scholar]
- 38.Beck J.R., Pauker S.G. The Markov process in medical prognosis. Med Decis Making. 1983;3(4):419–458. doi: 10.1177/0272989X8300300403. [DOI] [PubMed] [Google Scholar]
- 39.Sonnenberg F.A., Beck J.R. Markov models in medical decision making: a practical guide. Med Decis Making. 1993;13(4):322–338. doi: 10.1177/0272989X9301300409. [DOI] [PubMed] [Google Scholar]
- 40.Hastie T., Tibshirani R., Friedman J.H. Springer; New York, NY: 2001. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. [Google Scholar]
- 41.Fawcett T. An introduction to ROC analysis. Pattern Recogn Lett. 2006;27(8):861–874. [Google Scholar]
- 42.Jorm A.F., Jacomb P.A. The Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE): socio-demographic correlates, reliability, validity and some norms. Psychol Med. 1989;19(4):1015–1022. doi: 10.1017/s0033291700005742. [DOI] [PubMed] [Google Scholar]
- 43.Charlson M.E., Pompei P., Ales K.L., MacKenzie C.R. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–383. doi: 10.1016/0021-9681(87)90171-8. [DOI] [PubMed] [Google Scholar]
- 44.Teres D., Brown R.B., Lemeshow S. Predicting mortality of intensive care unit patients. The importance of coma. Crit Care Med. 1982;10(2):86–95. doi: 10.1097/00003246-198202000-00004. [DOI] [PubMed] [Google Scholar]
- 45.Schweickert W.D., Pohlman M.C., Pohlman A.S. Early physical and occupational therapy in mechanically ventilated, critically ill patients: a randomised controlled trial. Lancet. 2009;373(9678):1874–1882. doi: 10.1016/S0140-6736(09)60658-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.