Abstract
Objectives
Delirium in the ICU is associated with poor outcomes, but is under-detected. Here we evaluated performance of a novel, graded test for objectively detecting inattention in delirium, implemented on a custom-built computerized device (Edinburgh Delirium Test Box (EDTB-ICU)).
Design
A pilot study was conducted, followed by a prospective case-control study.
Setting
Royal Infirmary of Edinburgh General ICU.
Patients
A pilot study was conducted in an opportunistic sample of 20 patients. This was followed by a validation study in 30 selected patients with and without delirium (median age=63 years, range 23–84) who were assessed with the EDTB-ICU on up to 5 separate days. Presence of delirium was assessed using the Confusion Assessment Method for the ICU (CAM-ICU).
Measurements and Main Results
The EDTB-ICU involves a behavioral assessment and a computerised test of attention, requiring patients to count slowly-presented lights.
Thirty patients were assessed a total of 79 times (N's=31, 23, 15, 8 and 2 for subsequent assessments; 38% delirious). EDTB-ICU scores (range=0-11) were lower for patients with delirium than those without at the first (median=0 vs. 9.5), second (median=3.5 vs. 9), and third (median=0 vs. 10.5) assessments (all p<0.001). An EDTB-ICU score ≤5 was 100% sensitive and 92% specific to delirium across assessments. Longitudinally, participants' EDTB-ICU performance was associated with delirium status.
Conclusions
These findings suggest that the EDTB-ICU has diagnostic utility in detecting ICU delirium in patients with Richmond Agitation and Sedation Scale score>-3. The EDTB-ICU has potential additional value in longitudinally tracking attentional deficits, because it provides a range of scores and is sensitive to change.
Keywords: Delirium, Cognitive Assessment, Critical Care Medicine, Intensive Care Unit, Attentional Deficits
Introduction
Delirium occurs in up to 80% of ventilated patients in the ICU (1). ICU delirium is associated with adverse outcomes including prolonged hospitalisation and mortality (2–6). Despite this, the majority of ICU delirium is undetected (2). Diagnosis is critical to allow effective management of precipitants and of the distressing symptoms common in delirium (7–9).
Specific delirium assessment instruments have been developed for use in the ICU. The two most commonly used tools are the Confusion Assessment Method for the ICU (CAM-ICU) (1, 10) and the Intensive Care Delirium Screening Checklist (ICDSC) (11). Some studies show high sensitivity and specificity of these instruments for the detection of delirium (1, 10, 11); however, others have reported that performance is partly dependent on tester expertise (12, 13).
Inattention, a core feature of delirium, (14) can be objectively assessed with formal cognitive tests (15, 16). There is a lack of objective and robust measures for the assessment of inattention in delirium that can be completed by patients of varying levels of arousal, and which provide a graded measure of impairment.
We developed a novel, objective test for measuring inattention in delirium implemented on a custom-built device entitled the Edinburgh Delirium Test Box (EDTB). The EDTB successfully quantifies inattention in delirium in general ward settings, and discriminates patients with delirium from patients without delirium (17, 18).
The aims of the present study were: (1) to adapt the validated EDTB computerized test of attention for use in intubated patients in the ICU; and (2) to evaluate feasibility and performance of the EDTB as an instrument for detecting and monitoring inattention in ICU delirium.
Methods
Study design
Pilot study phase
A survey of 21 ICU nurses was first performed to identify effective communication methods for intubated (i.e. non-verbal) patients, and to gain insight into delirium assessment in routine clinical work. Next, tests of attention were adapted from previous EDTB tests (17, 18) to accommodate non-verbal responses. Finally, a pilot study in 20 selected ICU patients (7 intubated and 13 non-intubated) was performed to assess feasibility of the different response methods and to determine suitability of the attention test. Data from the pilot study informed the final assessment procedure, called the EDTB-ICU.
Main study phase
The aim of the main study was to validate the EDTB-ICU in detecting and quantifying inattention in patients with delirium. This was an exploratory, single-rater, unblinded study of 30 selected ICU patients with and without delirium as determined by the CAM-ICU (N=15 per group) who were assessed on up to five consecutive days during their ICU stay (unless patients met exclusion criteria or could not be assessed on a given day).
Ethics
The study was approved by the Scotland A Research Ethics Committee. Informed consent was obtained from participants or a legal proxy.
Participants
Patients admitted to the general ICU of the Royal Infirmary of Edinburgh were screened by two investigators for eligibility. Exclusion criteria were: age under 18 years, known pre-existing cognitive impairment, brain injury, visual or hearing impairments severe enough to preclude cognitive testing, or photosensitive epilepsy. Patients with a Richmond Agitation and Sedation Scale Score (RASS) (19) of ≤ -3 were excluded because they were considered too sedated or drowsy to be assessed using the EDTB-ICU (see Figure 1). The sample size was guided by prior studies using the EDTB in general hospital patients with delirium, where large effect sizes were found (17, 18).
Fig 1.
CONSORT diagram. EDTB-ICU = Edinburgh Delirium Test Box for the ICU; CAM-ICU = Confusion Assessment Method for the ICU.
Measurements and procedures
Participants were assessed routinely (twice daily) for delirium with the CAM-ICU (10) and RASS by clinical staff. All clinical nursing staff had completed a training program on delirium which included assessment using the CAM-ICU. Acute Physiology Age and Chronic Health Evaluation II (APACHE-II) score (20) was calculated.
Sustained visual attention was measured using the EDTB-ICU assessment which comprised 1.) a behavioral assessment to assess level of arousal and basic ability to respond to a tester, and 2.) an attention task. The attention task was implemented on the EDTB Mark 2, a purpose-built, computerized neuropsychological testing device designed for use at the patient's bedside (Supplementary Digital Content, Figure 1) (18). All EDTB-ICU assessments were performed by trained researchers (CG and KH), who were aware of patients’ CAM-ICU status as part of the study protocol (to enable selection of patients with and without delirium). CAM-ICU assessments were performed during morning ward rounds, which occurred between 11am and 2pm or immediately prior to the EDTB-ICU. Sedation holidays were not performed for the purposes of CAM-ICU or EDTB-ICU assessments.
Development of the EDTB-ICU test
In the pilot study phase, semi-structured interviews with 21 ICU nursing staff were conducted focusing on four main areas of interest: opinions on the CAM-ICU; communication methods for non-verbal patients; prevalence of ICU delirium; and delirium management. Through these interviews, several potential non-verbal response methods were selected for use in this pilot study: pointing, thumbs up indication, squeezing hand, nodding and sticking tongue out.
Next, the feasibility of these modes of non-verbal responding as a way of indicating responses to EDTB tasks was assessed in intubated patients. Participants first underwent a behavioral assessment (21) to determine if participants could respond to their name with eye contact, and then whether they were able to track the assessor’s moving finger for 5 seconds. Participants’ capacity to follow instructions was then assessed by instructing them to identify a given number on the response card held in front of them by the assessor, either verbally or non-verbally by one of the five response methods. Once a response method was identified that the patient was capable of doing, no further response methods were trialled. This served to establish a suitable method of response for the attention task (see Figure 2). If any of these methods was unsuitable for the patient due to physical limitations then these were skipped.
Fig. 2.
Flow diagram of EDTB-ICU testing procedure. All participants undertook a behavioral assessment to examine their level of arousal and ability to engage with cognitive testing, as well as establishing a feasible method of response. Participants who successfully completed this assessment then undertook a brief sustained attention task implemented on the Edinburgh Delirium Test Box Mark 2 (EDTB2).
If participants were unable to complete any element of this behavioral assessment, testing was discontinued and they were given a score relating to their performance on the behavioral assessment alone (Figure 2). Participants who were able to successfully complete the behavioral assessment (maximum score of 2) proceeded with an attention task on the EDTB.
The EDTB-ICU attention task involved counting the number of times that target lights illuminated over the course of each trial. Participants were informed that additional, distracting lights might appear, which should be ignored. The test comprised a practice trial and nine test trials, with target light illuminating between three and six times per trial. The nine trials had progressively increasing attentional demands (Supplementary Digital Content).
At the end of each trial participants were asked to indicate how many lights were shown, either verbally or by selecting an answer from four possible answers presented on the response card using non-verbal methods. Scores for the attention task were generated by assigning a score of 1 to each correct answer (excluding the practice trial), amounting to a summed total score ranging from 0 to 9. Scores on the behavioral assessment and attention task were summed, yielding a total EDTB-ICU total score between 0 (severe attentional impairment) and 11 (no attentional impairment).
Based on the results of the pilot phase, the same EDTB-ICU protocol was used without much alteration in the main study (Supplementary Digital Content).
Statistical analysis
Comparisons of EDTB-ICU test scores and all other data were made between groups with and without delirium using Mann–Whitney U tests.
Receiver operating characteristic (ROC) analyses were conducted on EDTB-ICU scores from the main study for participants’ first assessment and for data pooled across assessments.
Longitudinal changes in EDTB-ICU score were estimated using a linear mixed effects model (R function lmer; (22)) to evaluate responsiveness of the EDTB-ICU scores to change in delirium status. Predictors were time as a linear variable (assessments 1-5) and CAM-ICU result (positive or negative). The dependent variable was total EDTB-ICU score. The predictors were time represented by the number (0-4) of the five assessment occasions and the CAM-ICU result. The CAM-ICU was centered on its average within each person so that its model coefficient would represent the average person's change in response to a change in diagnosis.
Statistical analyses were carried out using PASW Statistics 18.0 software (SPSS, Inc., Somers, NY, US) and R version 3.0.1 (23). Threshold for statistical significance was set at a two-tailed p-value of ≤0.05.
Results
Pilot study
Twenty participants with ages ranged from 18 to 79 years (median=57.5 years, Inter-Quartile Range (IQR)=47-68) were recruited. Of these, five were delirious and 15 were non-delirious according to the CAM-ICU, and seven were intubated at the time of assessment. RASS scores ranged from -2 to 1.
Participants displayed a full range of scores on EDTB-ICU tasks (median=6.5, IQR=3-10, range=0-11). EDTB-ICU tasks were well tolerated by participants, with only one participant failing to complete them. In this instance the assessors discontinued testing because the participant became distressed.
Twelve participants (60%) were able to respond to tasks verbally, while 8 (40%) responded non-verbally. All non-verbal participants were capable of responding to the EDTB-ICU task via pointing (N=4), nodding (N=3) and blinking (N=1). Blinking was considered an ineffective means of communication because it was difficult to distinguish purposeful from reflexive blinking, and was therefore excluded from the main study protocol.
There were no differences in total EDTB-ICU score between verbal (median=8.5, IQR=6.75-10, range=0-11) and non-verbal participants (median=7, IQR=5-9, range=3-11; U =35.5, p=0.60).
Main study
Thirty patients were assessed during the main study phase. Fifteen patients were delirious according to CAM-ICU on at least one assessment, and 15 patients were never delirious during the study (Table 1). Two assessments were removed from analysis due to uncertain CAM-ICU status arising from conflicting nurse reports.
Table 1.
Demographic characteristics of participants in the main study; and characteristics of participants in the main study at each assessment.
| All Patients | CAM-ICU positive during at least one assessment | CAM-ICU negative during all assessments | |
|---|---|---|---|
| N | 30 | 15 | 15 |
|
| |||
| Age, years (Med., IQR) | 62.5 (52 – 70) | 65 (44 – 82) | 62 (55 – 67) |
|
| |||
| APACHE-II Score (Median, IQR) | 18 (14 – 24) | 22 (18 – 27) | 14 (11 – 18) |
|
| |||
| ICU lengths of stay, days (Med., IQR) | 6.4 (4.1 – 11) | 8 (5.2 – 13.6) | 4.8 (1.9 – 10.3) |
|
| |||
| ICU discharge destination (%, N) | |||
| Ward | 60.0% (18) | 46.7% (7) | 73.3% (11) |
| Other critical care area | 33.3% (10) | 40.0% (6) | 26.7% (4) |
| Deceased | 6.67% (2) | 13.3% (2) | 0.0% (0) |
|
| |||
| All EDTB-ICU assessments | CAM-ICU positive assessments | CAM-ICU negative assessments | |
|
| |||
| N | 79 | 30 | 49 |
|
| |||
| Mechanically ventilated (%, N) | 32.9% (26) | 60.0% (18) | 16.3% (8) |
|
| |||
| RASS score (%, N) | |||
| +3 | 2.5% (2) | 6.7% (2) | 0.0% (0) |
| +2 | 2.5% (2) | 6.7% (2) | 0.0% (0) |
| +1 | 7.6% (6) | 16.7% (5) | 20.4% (1) |
| 0 | 54.4% (43) | 10.0% (3) | 81.6% (40) |
| -1 | 25.3% (20) | 40.0% (12) | 16.3% (8) |
| -2 | 7.6% (6) | 20.0% (6) | 0.0% (0) |
|
| |||
| Total EDTB-ICU score (Med., IQR) | 7 (3 – 10) | 0 (0 – 4) | 10 (7 – 11) |
The EDTB-ICU paradigm took 3-7 minutes to administer. Verbal responses were used in 49% of assessments, while pointing was the most common non-verbal method of response (24%), followed by nodding (3%), and sticking out tongue (1%). Other responses (3%) included writing responses and holding up fingers to indicate a response. The remaining 20% of patients scored 0 or 1 on the EDTB assessment (Figure 2) indicating they were unable to track visual stimuli or follow simple commands.
EDTB-ICU performance
On the first assessment, patients with delirium (N=14) scored significantly lower on EDTB-ICU tasks (median=0, IQR=0-3, range=0-4) than patients without delirium (N=16; median=9.5, IQR=7-11, range=3-11) [U=3, p<0.001] (Figure 3). This difference was also apparent at the second assessment (Delirium: N=8, median=3.5, IQR=0.5-5, range=0-5; No delirium: N=15, median=9, IQR=6-10, range=4-11; U=3.5, p<0.001) and third assessment (Delirium: N=5, median=0, IQR=0-2.5, range=0-4; No delirium: N=10, median=10.5, IQR=8.75-11, range=7-11; U=0.0, p<0.001) (Figure 3). The number of patients who were available for a fourth (Delirium: N=1, score=5; No delirium: N=7, median=9, IQR=7-10) or fifth assessment (Delirium: N=1, score=5; No delirium: N=1, score=10) was too small to allow meaningful statistical comparisons.
Fig 3.
Boxplot illustrating the results of the Edinburgh Delirium Test Box Mark 2 (EDTB2) test for patients with and without delirium according to CAM-ICU score at 5 subsequent assessment days. The median is represented by the thick horizontal bars and median values are displayed next to these bars. The interquartile range is represented by the height of the inner boxes. The upper and lower whiskers represent scores outside the middle 50%. Values which are between one and a half and three box lengths from either end of the box are represented by open circles. *p<0.001.
ROC analyses were conducted to identify the optimal EDTB-ICU cut-point for detection of delirium. Using data from the first assessment of each participant, the area under the ROC curve (AUC) was 0.99 (95% CI=0.955-1.00). An EDTB-ICU score of ≤4 out of a maximum score of 11 had 100% sensitivity and 93.7% specificity for detection of delirium, while a cut-off of ≤5 had 100% sensitivity and 82% specificity. Using pooled results from all assessments, the AUC was 0.98 (95% CI=0.95-1.0). An EDTB-ICU score of ≤5 was found to have 100% sensitivity and 92% specificity for detection of delirium (Supplementary Digital Content, Figure 2).
The linear mixed-effects model suggested that a diagnosis of delirium was associated with a decrease in the patient's EDTB-ICU score of 6.44 at the time of diagnosis (ß=-6.44, S.E.=2.89, p=0.031). EDTB-ICU scores fluctuated over assessments in line with CAM-ICU status (Figure 4).
Fig 4.
Edinburgh Delirium Test Box for the ICU (EDTB-ICU) scores across assessments for all 11 patients who were assessed when both CAM-ICU positive (black circles) and CAM-ICU negative (white circles). A horizontal red line indicates the optimal EDTB-ICU cut-off score of ≤5 for the detection of delirium, as determined by receiver operating characteristic (ROC) analysis.
Patient outcomes
ICU length of stay ranged from 1 to 35 days (median=6.4, IQR=4.1-11.0). Participants who were CAM-ICU positive on at least one assessment had significantly longer length of stay (N=15; median=8 days) than CAM-ICU negative participants (N=15; median=4.8 days; U=63.5, p=0.043). Participants who were CAM-ICU positive during their study participation had a higher rate of ICU mortality (13.3%, N = 2) than CAM-ICU negative participants (0%).
The lowest EDTB-ICU score for each participant (representing their most severe attentional impairment across assessments) predicted length of stay (p=0.027; R2=0.163; β=-0.64, 95% CI=-1.19- -0.09). This relationship remained statistically significant (p=0.037; β=-0.41, 95% CI=-1.19- -0.04) after accounting for APACHE score (p=0.661; β=0.078; 95% CI=-0.282-0.438) and age (p=0.213; β=0.093; 95% CI=-0.57-0.244). Multicollinearity was not present between these variables (all variance inflation factor values < 1.3).
Discussion
This study provides preliminary support that the EDTB-ICU, a computerized graded test of attention for use in the ICU, has promising utility as an objective tool for detecting and monitoring delirium. The EDTB-ICU was highly sensitive (100%) and specific (92%) to delirium. Patients' EDTB-ICU scores fluctuated over time in line with CAM-ICU delirium diagnosis. These results are consistent with previous studies which have shown that attention is a defining feature of delirium.
The EDTB-ICU assessment was suitable for use in verbal and non-verbal patients with varying levels of arousal. Further, the EDTB-ICU assessment was reasonably brief, lasting 3-7 minutes; however the testing time involved would likely be reduced when administered by nursing staff familiar with the patient and able to quickly identify a suitable method of response. The EDTB-ICU therefore has promising utility as an objective tool for assessing delirium in ICU patients.
Patients with delirium ranged from being unable to track visual stimuli for only 5 seconds (despite being awake), to being able to follow simple commands and actively respond to EDTB-ICU tasks. These findings suggest that the EDTB-ICU is able to measure a continuum of attentional impairment. There is increasing acceptance of the conceptualization that delirium is a spectrum disorder incorporating sub-syndromal presentations which have also been linked to poor outcomes (24, 25); however there is a paucity of objective measures of delirium severity in ICU. The scoring system of the EDTB-ICU permits graded measurement of attentional deficits in patients. Further studies are now needed to confirm performance of the EDTB-ICU in larger unselected samples using independent raters, including whether the severity of inattention as measured with this assessment method may predict poor outcomes. Severity grading may also be valuable in assessing response to treatments.
Limitations
A case-control design using a relatively small sample was used. Further patient demographic information was not recorded. The diagnostic accuracy findings of the EDTB-ICU assessment only apply to patients with a RASS score of -2 or above, limiting generalisability. CAM-ICU was used as a reference standard for delirium diagnosis; although this instrument has good accuracy delirium detection in ICU patients (1, 10), milder cases might have been missed. Future studies should include a more comprehensive reference standard evaluation. Further, we did not compare our findings to a validated measure of delirium severity. The ICDSC measures the frequency of delirium symptoms over a 12-hour period, but does not quantify symptom severity at the time of the assessment. Despite this, it would be of value to assess the EDTB-ICU scores in relation to the ICDSC in future studies. More broadly,
The EDTB-ICU, like other ICU delirium assessments, is limited by the fact that some of the testing procedure requires participants to respond and engage with cognitive tasks. Despite this, the EDTB-ICU has valuable and distinct features compared with existing tools. The EDTB-ICU allows participants to respond using any of a number of non-verbal methods tailored to the patient. It also provides an objective means of assessing a continuum of attentional impairment over time. In addition, the EDTB-ICU has a ‘step-wise’ design incorporating a behavioral assessment to determine the patient’s ability to participate and respond as part of the overall assessment. The integrated step-wise scoring allows for finer-grained quantitative measurements across this spectrum. The EDTB-ICU assessment does not assume pre-existing knowledge on the part of the patient, and the option for non-verbal responses means than it could be easily translated into multiple languages.
The EDTB-ICU hardware is not widely available; however, we have developed software to administer EDTB tasks via smartphone (26). After appropriate validation, the EDTB-ICU assessment may have applications for both clinical practice and research, where it could be adopted as an alternative to existing instruments or as an adjunct to them, particularly in cases where an objective measure of delirium severity is desirable. Importantly, adoption of specific delirium detection processes remains problematic in many ICUs (27). The clinical utility of an objective, graded, and quantitative validated measure of attention indicative of delirium offered as provided by the EDTB-ICU may facilitate greater adoption of delirium detection and monitoring, particularly through its ability not just to detect delirium but to assist in the accurate monitoring of deterioration or resolution of delirium.
Conclusions
Effective diagnostic instruments are essential for the early and accurate diagnosis of delirium. The present study provides proof-of-principle regarding a novel method of assessing delirium in ICU patients, which is a serious and yet under-researched problem affecting large numbers of ICU patients. The EDTB-ICU provides an easy to learn, objective, rapid, sensitive and specific method of detecting delirium in the ICU, which could have additional value and complements current methods in use. Future versions of the test (particularly in shorter form) could be used as quantitative measures of inattention as an aid to diagnosis and also potentially as a means of tracking change and gauging severity, both in research studies and clinical practice. Future studies should assess the value of this novel method of delirium assessment in improving rates of detection and tracking change in larger unselected samples.
Supplementary Material
Acknowledgements
The research team would like to thank Dr Kirsty Everingham for her help with study design and data acquisition, and Mr Jonathan Adler of Eagle Designs for building and developing the Edinburgh Delirium Test Box. We also acknowledge the support of the University of Edinburgh school of Philosophy, Psychology, and Language Sciences (PPLS) and Prof. Sharon Abrahams, as well as support of the National Institute for Health Research, through the Comprehensive Clinical Research Network. This work was undertaken within the University of Edinburgh Centre for Cognitive Ageing and Cognitive Epidemiology (www.ccace.ed.ac.uk), part of the cross council Lifelong Health and Wellbeing Initiative (MR/K026992/1), for which funding from the BBSRC and MRC is gratefully acknowledged.
Footnotes
Conflict of Interest:
AMJM holds patents on computerized devices and tests for measuring attention in delirium. For the remaining authors none were declared.
Copyright form disclosure: Mr. Green disclosed off-label product use concerning the validation of a novel test for delivering a cognitive test for the detection of delirium, and it is delivered on a non-commercial device that is not approved by the FDA. Prof. Walsh’s institution received funding from Medical Research Council, and he received support for article research from Research Councils UK (RCUK). Prof. MacLullich disclosed other support from holding a U.S. patent on the testing procedure concerned in the manuscript, and he received support for article research from RCUK. Dr. Tieges received support for article research from Medical Research Council. The remaining authors have disclosed they do not have any potential conflicts of interest.
References
- 1.Ely EW, Inouye SK, Bernard GR, et al. Delirium in mechanically ventilated patients: validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU) JAMA. 2001;286(21):2703–2710. doi: 10.1001/jama.286.21.2703. [DOI] [PubMed] [Google Scholar]
- 2.Ely EW, Shintani A, Truman B, et al. Delirium as a predictor of mortality in mechanically ventilated patients in the intensive care unit. JAMA. 2004;291(14):1753–1762. doi: 10.1001/jama.291.14.1753. [DOI] [PubMed] [Google Scholar]
- 3.Jackson JC, Gordon SM, Hart RP, et al. The association between delirium and cognitive decline: a review of the empirical literature. Neuropsychol Review. 2004;14(2):87–98. doi: 10.1023/b:nerv.0000028080.39602.17. [DOI] [PubMed] [Google Scholar]
- 4.McCusker J, Cole M, Bellavance F, et al. Reliability and validity of a new measure of severity of delirium. Int Psychogeriatr. 1998;10(4):421–433. doi: 10.1017/s1041610298005493. [DOI] [PubMed] [Google Scholar]
- 5.McCusker J, Cole MG, Dendukuri N, et al. The delirium index, a measure of the severity of delirium: new findings on reliability, validity, and responsiveness. J Am Geriatr Soc. 2004;52(10):1744–1749. doi: 10.1111/j.1532-5415.2004.52471.x. [DOI] [PubMed] [Google Scholar]
- 6.Page V, Gough K. Management of delirium in the intensive care unit. Br J Hosp Med. 2010;71(7):372–376. doi: 10.12968/hmed.2010.71.7.48994. [DOI] [PubMed] [Google Scholar]
- 7.Bledowski J, Trutia A. A review of pharmacologic management and prevention strategies for delirium in the intensive care unit. Psychosomatics. 2012;53(3):203–211. doi: 10.1016/j.psym.2011.12.005. [DOI] [PubMed] [Google Scholar]
- 8.Flaherty JH, Gonzales JP, Dong B. Antipsychotics in the treatment of delirium in older hospitalized adults: a systematic review. J Am Geriatr Soc. 2011;59(Suppl 2):S269–276. doi: 10.1111/j.1532-5415.2011.03675.x. [DOI] [PubMed] [Google Scholar]
- 9.Hoyle G, Sharma V, MacLullich A, et al. Clinical aspects of delirium. J R Coll Physicians Edinb. 2008;38(2):154–157. [Google Scholar]
- 10.Ely EW, Margolin R, Francis J, et al. Evaluation of delirium in critically ill patients: validation of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) Crit Care Med. 2001;29(7):1370–1379. doi: 10.1097/00003246-200107000-00012. [DOI] [PubMed] [Google Scholar]
- 11.Bergeron N, Dubois MJ, Dumont M, et al. Intensive Care Delirium Screening Checklist: evaluation of a new screening tool. Intensive Care Med. 2001;27(5):859–864. doi: 10.1007/s001340100909. [DOI] [PubMed] [Google Scholar]
- 12.Luetz A, Heymann A, Radtke FM, et al. Different assessment tools for intensive care unit delirium: which score to use? Crit Care Med. 2010;38(2):409–418. doi: 10.1097/CCM.0b013e3181cabb42. [DOI] [PubMed] [Google Scholar]
- 13.McNicoll L, Pisani MA, Ely EW, et al. Detection of delirium in the intensive care unit: comparison of confusion assessment method for the intensive care unit with confusion assessment method ratings. J Am Geriatr Soc. 2005;53(3):495–500. doi: 10.1111/j.1532-5415.2005.53171.x. [DOI] [PubMed] [Google Scholar]
- 14.American Psychiatric Association. Diagnostic and statistical manual of mental disorders. (5th ed.). Arlington, VA: American Psychiatric Publishing; 2013. [Google Scholar]
- 15.Bhat R, Rockwood K. Delirium as a disorder of consciousness. J Neurol Neurosurg Psychiatry. 2007;78(11):1167–1170. doi: 10.1136/jnnp.2007.115998. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Meagher DJ, Leonard M, Donnelly S, et al. A comparison of neuropsychiatric and cognitive profiles in delirium, dementia, comorbid delirium-dementia and cognitively intact controls. J Neurol Neurosurg Psychiatry. 2010;81(8):876–881. doi: 10.1136/jnnp.2009.200956. [DOI] [PubMed] [Google Scholar]
- 17.Brown LJ, Fordyce C, Zaghdani H, et al. Detecting deficits of sustained visual attention in delirium. J Neurol Neurosurg Psychiatry. 2011;82(12):1334–1340. doi: 10.1136/jnnp.2010.208827. [DOI] [PubMed] [Google Scholar]
- 18.Tieges Z, McGrath A, Hall RJ, et al. Abnormal level of arousal as a predictor of delirium and inattention: an exploratory study. Am J Geriatr Psychiatry. 2013;21(12):1244–1253. doi: 10.1016/j.jagp.2013.05.003. [DOI] [PubMed] [Google Scholar]
- 19.Sessler CN, Gosnell MS, Grap MJ, et al. The Richmond Agitation-Sedation Scale: validity and reliability in adult intensive care unit patients. Am J Respir Crit Care Med. 2002;166(10):1338–1344. doi: 10.1164/rccm.2107138. [DOI] [PubMed] [Google Scholar]
- 20.Knaus WA, Draper EA, Wagner DP, et al. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–829. [PubMed] [Google Scholar]
- 21.Frontera JA. Delirium and sedation in the ICU. Neurocrit Care. 2011;14(3):463–474. doi: 10.1007/s12028-011-9520-0. [DOI] [PubMed] [Google Scholar]
- 22.Field AP, Miles JNV, Field ZC. Discovering statistics using R. London, UK: Sage Publications; 2012. [Google Scholar]
- 23.R Core Team. A language and environment for statistical computing. R Foundation for Statistical Computing; Vienna, Austria: 2014. [Google Scholar]
- 24.MacLullich AM, Hall RJ. Who understands delirium? Age Ageing. 2011;40(4):412–414. doi: 10.1093/ageing/afr062. [DOI] [PubMed] [Google Scholar]
- 25.Meagher D, Adamis D, Trzepacz P, et al. Features of subsyndromal and persistent delirium. Br J Psychiatry. 2012;200(1):37–44. doi: 10.1192/bjp.bp.111.095273. [DOI] [PubMed] [Google Scholar]
- 26.Tieges Z, Stiobhairt A, Scott K, et al. Development of a smartphone application for the objective detection of attentional deficits in delirium. Int Psychogeriatr. 2015;27(8):1251–1262. doi: 10.1017/S1041610215000186. [DOI] [PubMed] [Google Scholar]
- 27.Elliott D, Aitken LM, Bucknall TK, Seppelt IM, Webb SA, Weisbrodt L, McKinley S, Australian, New Zealand Intensive Care Society Clinical Trials G, George Institute for Global H Patient comfort in the intensive care unit: a multicentre, binational point prevalence study of analgesia, sedation and delirium management. Crit Care Resusc. 2013;15(3):213–219. [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.