Cornell Assessment of Pediatric Delirium: A Valid, Rapid, Observational Tool for Screening Delirium in the PICU

Abstract

Objective

To determine validity and reliability of the Cornell Assessment of Pediatric Delirium, a rapid observational screening tool.

Design

Double-blinded assessments were performed with the Cornell Assessment of Pediatric Delirium completed by nursing staff in the PICU. These ratings were compared with an assessment by consultation liaison child psychiatrist using the Diagnostic and Statistical Manual IV criteria as the “gold standard” for diagnosis of delirium. An initial series of duplicate Cornell Assessment of Pediatric Delirium assessments were performed in blinded fashion to assess interrater reliability. Nurses recorded the time required to complete the Cornell Assessment of Pediatric Delirium screen.

Setting

Twenty-bed general PICU in a major urban academic medical center over a 10-week period, March–May 2012.

Patients

One hundred eleven patients stratified over ages ranging from 0 to 21 years and across developmental levels.

Intervention

Two hundred forty-eight paired assessments completed.

Measurements and Main Results

The Cornell Assessment of Pediatric Delirium had an overall sensitivity of 94.1% (95% CI, 83.8–98.8%) and specificity of 79.2% (95% CI, 73.5–84.9%). Overall Cronbach’s α of 0.90 was observed, with a range of 0.87–0.90 for each of the eight items, indicating good internal consistency. A scoring cut point of 9 demonstrated good interrater reliability of the Cornell Assessment of Pediatric Delirium when comparing results of the screen between nurses (overall κ = 0.94; item range κ = 0.68–0.78). In patients without significant developmental delay, sensitivity was 92.0% (95% CI, 85.7–98.3%) and specificity was 86.5% (95% CI, 75.4–97.6%). In developmentally delayed children, the Cornell Assessment of Pediatric Delirium showed decreased specificity of 51.2% (95% CI, 24.7–77.8%) but sensitivity remained high at 96.2% (95% CI, 86.5–100%). The Cornell Assessment of Pediatric Delirium takes less than 2 minutes to complete.

Conclusions

With an overall prevalence rate of 20.6% in our study population, delirium is a common problem in pediatric critical care. The Cornell Assessment of Pediatric Delirium is a valid, rapid, observational nursing screen that is urgently needed for the detection of delirium in PICU settings.

Delirium is acute cerebral dysfunction caused by systemic illness or the effects of treatment (1). There is an urgent need for pediatric-specific research into delirium (2–5). Recognition of delirium in children in the PICU has been suboptimal; therefore, the impact of delirium and therapeutic interventions have been understudied (6–9). Pediatric delirium is associated with increased length of PICU stay (10), posttraumatic symptoms (11), and possible neurocognitive dysfunction in children after discharge (12, 13). A growing body of literature in adult critical care describes delirium as exacerbated by the use of various sedative medications and has identified risk factors that predispose to delirium (5, 14–17). An impediment to the progress of pediatric delirium research has been the absence of an easily administered and widely applicable screening tool.

The clinical diagnosis of delirium in children more than 12 months old, based on Diagnostic and Statistical Manual IV (DSM-IV) criteria, is considered valid with a presentation that is similar to adults (14, 18–24). Delirium in infants less than 12 months old has not been systematically studied, but clinical reports suggest that with developmental considerations in diagnosis, infants present with delirium with detectable deficits in awareness, cognition, and arousal (21, 25, 26). Subtypes of delirium, including hyperactive, hypoactive, and mixed type, are considered valid in children as well as adults (27).

The limitations of existing tools in the PICU population, including the Delirium Rating Scale (DRS), Pediatric Confusion Assessment Method for the ICU (pCAM-ICU), and the Pediatric Anesthesia Emergence Delirium (PAED) screen, have been discussed (6, 28–32). In brief, the DRS (33) was designed for psychiatrists’ use and is labor intensive. The pCAM-ICU (34) is an elegant cognitive tool but requires patient cooperation, is restricted to children more than 5 years old, limited in patients with developmental delay, and requires extensive nurse training. The PAED (35) designed, for immediate postoperative use by anesthesiologists, selects for the hyperactive subtype of delirium. An ideal screening tool would detect all types of delirium (hyperactive, hypoactive, and mixed), in patients of all ages and developmental levels.

Our primary objective was to describe the development of the Cornell Assessment of Pediatric Delirium (CAPD) and test its validity and reliability as a screening tool. In addition, we explored the instrument’s performance in subgroups defined by developmental delay, gender, respiratory support, prematurity, and severity of illness.

MATERIALS AND METHODS

Phase I: Development of the CAPD

The CAPD is an adaptation of the PAED. As the original PAED was designed to detect transient emergence delirium following anesthesia, it selects for patients with a hyperactive, agitated delirium subtype and would be incomplete for assessing the PICU population. Therefore, we added two elements (questions 7 and 8, Fig. 1) to improve the detection of hypoactive and mixed-type delirium. We changed the scale items from statements to questions and renamed the tool to reflect the comprehensive nature of the assessment. An initial pilot study showed feasibility for use as a rapid nursing screen (28).

Figure 1.

Cornell Assessment of Pediatric Delirium revised. RASS = Richmond Agitation and Sedation Scale.

Based on the pilot study, we made additional changes. To better capture a fluctuating course of delirium over a nurse’s shift, response options were changed from the original (not at all/just a little/quite a bit/very much/extremely) to the format “never/rarely/sometimes/often/always”. To better reflect the DSM-IV criteria for delirium, and detect alteration in cognitive functioning, we added a third novel item (question 4, Fig. 1), to assess the ability to communicate needs and wants. Content validity of the revised CAPD (Fig. 1) was evaluated by experts in the fields of pediatric critical care, development, delirium, and psychometrics.

Anchor Points

Orientation, arousal, and appropriate cognition (which are all affected in delirium) are difficult to assess in young children and even harder to measure in infants. Because of concerns about accurate screening in children under 2 years old, developmental anchor points were delineated. Based on classic texts and established scales of child development, each anchor point characterizes the normal developing child for each item on the CAPD (Table 1). Anchor points describe the associated observable behaviors in a PICU setting (rather than in the child’s natural environment) (36, 37). After piloting with nurses for clarity of language and concepts, a short training session was done and anchor point charts were provided for reference to the approximately 100 critical care nurses who participated.

TABLE 1.

Selected Cornell Assessment of Pediatric Delirium Developmental Anchor Points and Diagnostic and Statistical Manual IV Delirium Domain Correlates

Cornell Assessment of Pediatric Delirium Item	Diagnostic and Statistical Manual Delirium Domains	Selected Normal Developmental Anchor Pointsa
		Age (8 wk)	Age (1 yr)
1. Does the child make eye contact with the caregiver?	Consciousness	Follows moving object past midline, regards hand holding object, focused attention	Holds gaze. Prefers primary parent. Looks at speaker
2. Are the child’s actions purposeful?	Cognition	Symmetric movements, will passively grasp handed object	Reaches and manipulates objects, tries to change position, if mobile may try to get up
3. Is the child aware of his/her surroundings?	Consciousness Orientation	Facial brightening or smile in response to nodding head, frown to bell, coos	Prefers primary parent, upset when separated from preferred caregivers. Comforted by familiar objects (i.e., blanket or stuffed animal)
4. Does the child communicate needs and wants?	Consciousness Psychomotor activity	Cries when hungry or uncomfortable	Uses single words or signs
5. Is the child restless?	Cognition Psychomotor activity Affect/distress	No sustained awake alert state	No sustained calm state
6. Is the child inconsolable?	Orientation Cognition Affect/distress	Not soothed by usual comforting actions, for example, rocking and singing	Not soothed by usual comforting actions, for example, singing, holding, talking, and reading
7. Is the child underactive—very little movement while awake?	Orientation Affect/distress	Little if any purposive grasping, control of head and arm movements, such as pushing things that are noxious away	Little if any play, efforts to sit up, pull up, and if mobile crawl or walk around
8. Does it take the child a long time to respond to interactions?	Consciousness Psychomotor activity	Not cooing, smiling, or focusing gaze in response to interactions	Not following simple directions. If verbal, not engaging in simple dialogue with words or jargon

^aAnchor points were developed for newborn and 4 wk, 6 wk, 8 wk, 28 wk, 1 yr, and 2 yr olds.

Criterion Standard

The “gold standard” diagnosis for pediatric delirium is an assessment by a child psychiatrist using the DSM-IV criteria that require acute onset, fluctuating course, and disturbance of awareness and cognition (1). A short training session for the six psychiatric evaluators was completed.

Phase II: Assessing Psychometric Properties

Study Design

The study took place in a 20-bed general PICU in a major urban academic medical center over a 10-week period from March to May 2012.

All patients in the PICU on a given study day were eligible if there was a parent or guardian available to provide informed consent. The only exclusion criterion was a sedation score of less than −3 (deeply sedated or unarousable), using the Richmond Agitation and Sedation Scale (RASS) (38, 39). Demographic and clinical data were collected on each subject.

Reliability Testing

After informed consent was obtained, a set of paired, double-blinded assessments was performed. The bedside nurse completed the CAPD as a paper checklist. Subsequently, the psychiatrist conducted a diagnostic interview and examination. If a child was diagnosed with delirium by the psychiatrist, this was reported to the medical team caring for the child so that appropriate interventions could be taken. If the subject was still present in the PICU on the next study day, the paired assessments were repeated, up to a predetermined maximum of 5 per subject. When the assessments were completed, CAPD screening results were compared with the psychiatric diagnosis and the interrater agreement was computed.

The first 70 CAPD screens were each performed by two blinded nurses. Interrater reliability was quantified using Cohen’s κ coefficient, whereas internal consistency of the eight items was evaluated by Cronbach’s α.

Validity Testing

The enrollment goal was a minimum of 100 subjects overall and 250 encounters. The sample size calculation was based on an assumed prevalence of pediatric delirium of 15%, sensitivity of 0.90 and α level of 0.05, and inclusion of subjects from all age groups and children with and without developmental delay. The definition of “significant clinical developmental delay” was based on clinical assessment and/or parental report of developmental problems that affected the child’s behavior or ability to communicate. Children with mild or transient history of developmental problems (i.e., needing occupational therapy or motor or speech delays) but who did not have current abnormalities in communication or behavior were classified as normal for the purpose of the study.

The receiver operating characteristic (ROC) analysis was performed to find the optimal CAPD cutoff score; subsequently, sensitivity and specificity were calculated for the overall sample. In addition, in order to explore CAPD performance in subgroups, validity measures were described by age groups, developmental delay status, gender, respiratory support, prematurity, and illness severity. All CIs have been adjusted for the possible correlation between observations within subjects using a ratio estimator method (40, 41).

The study was approved by the Institutional Review Board of Weill Cornell Medical College.

RESULTS

Average PICU census on study days was 16. Approximately 68% of patients were eligible. Seventeen percent of patients had a RASS of less than −3. Fifteen percent of patients did not have a parent available to provide consent or were off the unit at the time of the study. Consent rate was 88.5% of eligible patients. In total, 111 subjects were enrolled (Fig. 2).

Figure 2.

Subject recruitment flow. RASS = Richmond Agitation and Sedation Scale.

Subject Characteristics

Admitting diagnoses are shown in Table 2. Sixty-seven subjects (60%) were male. Twenty-two subjects (20%) had significant developmental delay. Fifty-three subjects (48%) were receiving supplemental oxygen, 30 subjects (27%) were on noninvasive positive pressure ventilation, and 19 subjects (17%) were on invasive mechanical ventilation. Sixty assessments (24% of encounters) were completed with children who were intubated.

TABLE 2.

Demographic Details and Admission Diagnoses of Subjects (n = Total 111)

Characteristic	n (%)
Gender
Male	67 (60)
Female	44 (40)
Age
0–24 mo	37 (33)
2–5 yr	24 (22)
6–12 yr	25 (22.5)
13–21 yr	25 (22.5)
Developmental delaya
No	89 (80)
Yes	22 (20)
Respiratory support
Oxygen	53 (48)
Noninvasive mechanical ventilation	30 (27)
Ventilator	19 (17)
None	9 (8)
Prematurity
Yes	22 (20)
No	89 (80)
Diagnosesb
Cardiac	12
Genetic disorder	13
Hematologic/oncologic	19
Infectious/inflammatory	38
Metabolic	11
Neurologic	16
Neurosurgical	30
Respiratory insufficiency	50
Postoperative/other	56
Pediatric Index of Mortality II, %
Overall	Median = 3.00 (range, 0–57)
Pediatric delirium	Median = 4.05 (range, 0–57)
No pediatric delirium	Median = 2.00 (range, 0–57)

^aSee text for description of categories.

^bIncluding all primary and secondary diagnoses.

Criterion Standard

Interrater reliability of the initial 38 psychiatric evaluations performed by two blinded psychiatrists was excellent (Cohen’s κ = 0.95; 95% CI, 0.79–1.00), consistent with our expectation for the criterion standard.

Prevalence of Delirium

Prevalence of delirium by psychiatric assessment was 20.6% (n = 51). Among children with multiple encounters who received a diagnosis of delirium at least once (n = 21), 89.5% showed a fluctuating course. Developmental delay was a significant risk factor for delirium as children with developmental delay were diagnosed with delirium almost three times as often as children without delay (38.8% vs 13.9% of assessments, respectively). Prevalence of delirium in the “sicker” patients, as measured by Pediatric Index of Mortality II (PIM2) score above the median, was notably higher than in those children with PIM2 score below the median (29.7% vs 12.3%). This is consistent with prior pediatric delirium research (42). The lowest delirium prevalence was observed in children more than 13 years old (3.6%) and in children not on respiratory support (5.2%).

CAPD Performance

Cut point analysis showed the best sensitivity and specificity for the screening instrument (prioritizing high sensitivity) at a total CAPD score of 9 or greater. Sensitivity was 94.1% (95% CI, 83.8–98.8%) and specificity 79.2% (95% CI, 73.5–84.9%). At a cut point of greater than or equal to 9, there were three false-negative CAPD screens (Table 3) and 41 false-positive screens. Concordance between CAPD and psychiatric diagnosis was 82.3% (r = 0.62). Nurses’ CAPD interrater reliability was also highest at a cut point of 9, with κ = 0.94. κ ranged from 0.68 to 0.78 for each of the eight CAPD items.

TABLE 3.

Incidents of False-Negative Cornell Assessment of Pediatric Delirium (n = 3, 1.2%) Defined as Score Less Than 9 but Psychiatrist Rated “Delirious”

Age Group (yr)	Developmental Delay	Clinical	Cornell Assessment of Pediatric Delirium Score	Psychiatrist Observations	Other
2–5	Yes	Patient had Trisomy 21 and respiratory failure, on sedatives, opiates	6	Restless, less aware, and less communicative than baseline per caregiver	Patient well known to psychiatrist and nursing Next shift CAPD scored 12, possible fluctuating MSE
13–21	No	Patient had DiGeorge Syndrome, anxiety and mood disorders, and Asperger Syndrome at baseline; history of Epstein-Barr virus-lymphoma post stem cell transplant, in PICU for management of cerebral hemorrhage, on sedatives, opiates	7	Restless, more withdrawn than baseline per caregiver	Fluctuating MSE assessed over 5 d and was delirious by CAPD 2/5 and by psychiatrist 2/5. Had one false-positive CAPD and one false-negative CAPD. Patient was difficult to assess
6–12	No	Patient in PICU for postoperative management after thoracoabdominal resection of neuroblastoma, on opiates	5	Withdrawn affect, decreased speech, anger/mood changes, decreased attention, psychomotor retardation	Possible fluctuating MSE, previous day had concordant negative examinations. Possible improper use of CAPD

CAPD = Cornell Assessment of Pediatric Delirium, MSE = mental status examination.

CAPD performance compared with the “gold standard” psychiatric diagnosis by subgroups is reported in Table 4. In patients without significant developmental delay (73% of our population), the CAPD had both high sensitivity and specificity (92%; CI, 85.7–98.3% and 86.5%; CI, 75.4–97.6%, respectively). In children with developmental delay, the screen remained quite sensitive (96.2%; CI, 86.5–100%) but demonstrated a loss of specificity (51.2%; CI, 24.7–77.8%). Despite this, ROC analysis of the CAPD in children with developmental delay had an area under the curve of 0.86 (Fig. 3), demonstrating its applicability in this hard-to-assess population. The negative predictive value remained quite high at 98.5% (95% CI, 94.8–99.8%).

TABLE 4.

Performance of the Cornell Assessment of Pediatric Delirium Reported by Receiver Operating Characteristic Analysis, Sensitivity, and Specificity

Group	Number of Assessments	Prevalence (%)	Area Under Curve by Receiver Operating Characteristic Analysis	Sensitivity (95% CI)	Specificity (95% CI)
All PICU patients	248	20.6	94	94.1 (83.8–98.8)	79.2 (73.5–84.9)
Age (yr)
< 2	76	19.5	92	100 (100–100)	67.7 (45.9–89.6)
2–5	49	43.5	94	100 (100–100)	69.0 (36.7–100)
6–12	67	20.3	85	86.7 (65.6–100)	76.8 (55.3–98.3)
13–21	56	3.6	99	50 (0.2–100)	98.1 (94.3–100)
Developmental delay
No	181	13.9	93	92.0 (85.7–98.3)	86.5 (75.4–97.6)
Yes	67	38.8	86	96.2 (86.5–100)	51.2 (24.7–77.8)
Gender
Male	152	21.7	94	93.9 (88.8–99.1)	76.5 (59.0–93.9)
Female	96	18.8	94	94.4 (81.4–100)	83.5 (67.8–99.3)
Respiratory support
No	115	5.2	98	100 (100–100)	86.9 (73.8–100)
Yes	133	33.8	89	93.6 (87.2–100)	71.6 (54.9–88.2)
Prematurity
No	186	20.3	95	94.6 (90.5–98.7)	83.9 (71.7–96.1)
Yes	62	22.6	87	92.9 (78.9–100)	64.6 (37.6–91.6)
Pediatric Index of Mortality II
Below median	124	12.3	93	90.0 (66.9–100)	79.8 (61.7–97.9)
Above median	124	29.7	93	95.1 (91.7–98.5)	78.6 (62.6–94.7)

Figure 3.

Cornell Assessment of Pediatric Delirium performance by receiver operating curves. Thick line represents area under the cure (AUC) = 0.9364; dashed line represents AUC = 0.9582; and dashed and dotted line represents AUC = 0.8602. All = all subjects, DT = developmentally typical subjects, DD = developmentally delayed subjects.

The CAPD screen performed similarly in all age groups of children from 0 to 13 years old. The exceptional group was adolescents (> 13–21 years old) where sensitivity was lower (50%; 95% CI, 1.3–99%) and specificity was high (98.1%; 95% CI, 94.3–100%), but this is based on only two confirmed diagnoses of delirium (out of 56 total encounters) in this age group. The performance of the CAPD by gender, respiratory support, prematurity, and illness severity as determined by PIM-2 score is presented in Table 4.

CAPD Psychometric Properties

Item fit/overlap analysis showed that each of the eight items was highly correlated with the overall CAPD scale (Table 5). Cronbach’s α overall was 0.90 and for each separate item ranged from 0.87 to 0.90, indicating good internal consistency. Items 5, 6, and 7 were the least well correlated (0.65, 0.62, and 0.68, respectively) but still well above the generally accepted threshold of 0.20.

TABLE 5.

Cornell Assessment of Pediatric Delirium Internal Consistency and Item-Test Correlations

Item	Item-Test Correlation	α if Item Deleted
1	0.83	0.88
2	0.85	0.87
3	0.86	0.87
4	0.88	0.87
5	0.65	0.90
6	0.62	0.90
7	0.68	0.90
8	0.77	0.88
Test scale		0.90

DISCUSSION

With an overall prevalence rate of 20.6% in our study population, delirium is a common problem in pediatric critical care. The CAPD was designed to fill a critical gap in the ability of PICU staff to identify patients who may be suffering from delirium.

Elements of the Screen

The CAPD items are intended to correlate directly to the DSM-IV definition of delirium, which requires alteration in consciousness (including attention and awareness), and cognition (including memory, orientation, perception, and language) (Table 1). Each item was determined to fit well in the overall scale. The CAPD screen is designed to allow for behavioral, developmentally informed observations to be scaled and summarized in a total score, which indicates whether a child is likely to be delirious.

A Sensitive Screening Tool

With a sensitivity of 94.1%, the CAPD produced three false negatives out of 248 assessments. Of these (Table 3), two of the three children screened positive on a prior or subsequent CAPD. By performing the screen twice daily, these subjects would have been detected. Because one of these children had significant developmental delay, and the other a preexisting psychiatric illness, it is possible that these factors complicated the nursing assessment. The third screen may not have been performed accurately as it conflicts with the psychiatric assessment in many item responses. Larger studies are needed to further assess factors that confound detection of delirium.

Specificity in Diagnosing Delirium

Although each individual item in the tool may describe behaviors or symptoms that can be associated with other causes of cerebral dysfunction (such as sedation, agitation, pain, and anxiety), the combination of these items with a total cutoff score of 9 successfully selects for delirium.

The screen produced 41 false positives, 20 in patients with significant developmental delay. This speaks to the difficulty of diagnosing delirium in this population as these children may have other reasons for behavioral and emotional dysregulation at baseline. Psychiatric assessment for these children is more specific and would be appropriate for children with developmental delay who score greater than 9 on the CAPD. However, the screen still has high negative predictive value in this population.

Nearly half (48%) of the subjects who received a false-positive CAPD score were diagnosed with delirium at a later point in their PICU stay. We theorize that the CAPD score may trend with the patient’s waxing and waning clinical status and may be useful in identifying evolving delirium.

Applicability

It is significant that 31% of our assessments were in children less than 2 years old, and 27% of our assessments were in children who are developmentally delayed. Our study indicates that the CAPD is a valid and reliable delirium screen in these vulnerable populations. The developmental anchor points for each item were a valuable point-of-use reference for assessing the youngest of patients. With the addition of these anchor points and minimal training, the critical care nursing staff became adept at using the CAPD in all but the most developmentally delayed patients. The nurses completed the assessment midshift, after several hours of observing the child’s behavior. In every assessment, the nurses required less than 2 minutes to complete the CAPD screen.

Study Limitations

The CAPD was developed and validated in a single institution and needs to be replicated in a multi-institutional study. Preparations for such a study are ongoing.

This study found a very low prevalence of delirium in adolescents (children > 13 years old), limiting adequate determination of sensitivity and specificity of the tool in this subgroup. A larger sample size will be required for this age group.

In patients with significant developmental delay, the false-positive rate was higher, reflecting the difficulty of assessing these patients. In our study cohort, children with delay were more often diagnosed with delirium, suggesting that these patients may be at greater risk. More research is needed to reproduce this finding and address the best diagnostic approaches in this vulnerable population, who likely have baseline brain alterations or abnormalities. The possibility of a higher CAPD cut point, or a modification of scoring adjusting for baseline functioning, needs to be assessed in larger studies.

For study purposes, the CAPD and psychiatric evaluations happened during the daylight hours, at approximately noon each day. To more accurately capture delirious patients, who may be more symptomatic at night, the CAPD will need to be performed a minimum of twice daily, once by each shift nurse.

SUMMARY

The CAPD is a promising new clinical screening tool designed and validated for use in the PICU setting to detect delirium in most children. Future work will address further clinical applications of the CAPD, such as diagnostic algorithms for special populations in which delirium diagnoses are challenging. The CAPD may facilitate the development of much needed research investigating the causes, pathophysiology, treatment, and long-term implications of pediatric delirium.