Pediatric delirium: Evaluating the gold standard

Abstract

Objective

Our aim was to evaluate interrater reliability for the diagnosis of pediatric delirium by child psychiatrists.

Method

Critically ill patients (N = 17), 0–21 years old, including 7 infants, 5 children with developmental delay, and 7 intubated children, were assessed for delirium using the Diagnostic and Statistical Manual–IV (DSM–IV) (comparable to DSM–V) criteria. Delirium assessments were completed by two psychiatrists, each blinded to the other’s diagnosis, and interrater reliability was measured using Cohen’s κ coefficient along with its 95% confidence interval.

Results

Interrater reliability for the psychiatric assessment was high (Cohen’s κ = 0.94, CI [0.83, 1.00]). Delirium diagnosis showed excellent interrater reliability regardless of age, developmental delay, or intubation status (Cohen’s κ range 0.81–1.00).

Significance of results

In our study cohort, the psychiatric interview and exam, long considered the “gold standard” in the diagnosis of delirium, was highly reliable, even in extremely young, critically ill, and developmentally delayed children. A developmental approach to diagnosing delirium in this challenging population is recommended.

BACKGROUND

Delirium, acute brain dysfunction, is recognized as a serious medical problem in the adult critical care population (Barr et al., 2013). Evaluation and treatment of delirium has only recently garnered attention in the world of pediatrics (Janssen et al., 2011; Schieveld et al., 2007; Silver et al., 2012; Smith et al., 2009; 2011).

The “gold standard” diagnosis for pediatric delirium is an assessment by child psychiatrists using the Diagnostic and Statistical Manual–IV (DSM–IV) criteria. Expert consensus supports presentation of delirium in children over the age of two years as comparable to adult delirium, and clinical diagnosis, based on the DSM–IV criteria, has been found to be valid (Karnik et al., 2007; Leentjens et al., 2008; Turkel et al., 2003; 2006). Preverbal children under two years of age and developmentally delayed children may be very difficult to evaluate for alterations in awareness, consciousness, and cognition, leading some to question the validity of diagnosing delirium in this population. At the same time, some report infant presentation of delirium with recognizable deficits in awareness, cognition, and arousal when evaluated within a developmental framework by experienced practitioners (Madden et al., 2011; Schieveld et al., 2010; Silver et al., 2010; Turkel et al., 2013). Due to the lack of objective research addressing the consistency of the “gold standard,” especially in infants and children with developmental delays, we conducted a study to test the interrater reliability of child psychiatrists’ assessments.

METHODS

Design

Thirty-eight delirium assessments were completed by two psychiatrists, each blinded to the other’s diagnosis. They assessed all consented pediatric intensive care unit (PICU) patients present that day as the initial part of a validation study of the Cornell Assessment of Pediatric Delirium (CAPD) (Traube et al., 2013). Psychiatrists were also blinded to the nursing CAPD scores. The study was conducted over three weeks during March of 2012 and took place in a 20-bed general PICU in a tertiary-care academic medical center in New York City.

Subjects

All patients were eligible unless they had a sedation score equal to or lower than −3 (deeply sedated or un-arousable) on the Richmond Agitation and Sedation Scale (RASS) (Sessler et al., 2002).

Children of varying developmental abilities were included. They were described as having “significant clinical developmental delay” if developmental problems were the cause of an impairment in a child’s age-appropriate ability to communicate (a symptom that could affect psychiatrists’ assessment of the child’s baseline in relevant symptom domains) just prior to their critical illness.

After informed consent was obtained, two child psychiatrists conducted diagnostic interviews and exams on each subject to evaluate for delirium. Each psychiatrist was blinded as to the other’s conclusion. After both assessments were completed, if a child was diagnosed with delirium by either psychiatrist, this was reported to the medical team caring for the child so that appropriate interventions could be made. Individual subjects were assessed up to four times. The study was approved by the institutional review board at Weill Cornell Medical College.

Assessment Measures

Interrater reliability was quantified using Cohen’s κ coefficient along with its 95% confidence interval.

Evaluator Training

A two-hour initial training session for the six child psychiatrist evaluators, led by the first author, was completed to establish consistency in concepts and vocabulary among the group. In three subsequent training sessions, “thinking developmentally” about delirium, including keeping normal milestones in mind and a broad range of expectations regarding behavior and cognition for children at different developmental stages during critical illness, was emphasized. Manifestations of alterations in attention, consciousness, and cognition were discussed in order to understand clinical experiences with critically ill children. This framework was discussed in relation to each item on our psychiatric assessment worksheet, which was based on the Delirium Rating Scale (DRS-98) (Trzepacz et al., 2001), with the addition of descriptors, expansion of some categories, and addition of an item to denote a change in cognition from baseline.

RESULTS

When a total of 38 assessments were completed, including 17 patients, the psychiatric diagnoses were compared. Our sample included 7 infants under 2 years of age (13/38 assessments, 34%), 4 children aged 2–5 years (11/38 assessments, 29%), 4 children aged 6–12 years (11/38 assessments, 29%), and 2 adolescents aged 13–21 years (3/38 assessments, 8%). Of these, 5 had moderate to severe developmental delay (11/38 assessments, 29%) and 7 were intubated (15/38 assessments, 39%).

Overall (see Table 1), delirium was identified in seven patients, including three infants below two years of age and three patients with moderate to severe developmental delay. Among the three children who underwent multiple assessments and were diagnosed with delirium at least once, two showed a fluctuating delirium course. The child psychiatrists diagnosed delirium in 34% (13/38) of evaluations. The interrater reliability of the 38 separate psychiatric assessments was high, with κ = 0.94 (95% CI = 0.83–1.00).

Table 1.

Psychiatrist interrater reliability (Cohen’s kappa κ) for pediatric delirium by assessment (N = 38)

	Paired Assessments n (%)	Delirium Dx by Two Raters (n)	Disputed Delirium Dx by One Rater (n)**	κ (95% CI)
Overall*	38 (100%)	12	1	0.94 (0.83–1.00)
Age
0–24 m	13 (34%)	3	1	0.81 (0.45–1.00)
2–5 yr	11 (29%)	5	0	1.00
6–21 yr	14 (37%)	4	0	1.00
Devel. delay
No	27 (71%)	6	1	0.90 (0.71–1.00)
Yes	11 (29%)	6	0	1.00
Intubated
No	23 (61%)	6	1	0.89 (0.70–1.00)
Yes	15 (39%)	6	0	1.00

Abbreviations: m = months; yr = years; Dx = diagnosis; CI = confidence interval; devel. delay = developmental delay.

^*38 paired assessments were administered across 17 patients, with 65% (11/17) undergoing multiple assessments.

^**The discordant assessment occurred in a one-month-old non-intubated patient with age-appropriate development.

When analyzed by age, in children under 2 (13 assessments), Cohen’s κ was 0.81 (95% CI = 0.45–1.00). For children over 2 years of age (25 assessments), Cohen’s κ was 1.00. Likewise, interrater reliability was high for children with and without developmental delay (κ = 1.00 and 0.90 [95% CI, 0.71–1.00], respectively), and regardless of whether or not they were intubated (κ = 1.00 and 0.89 [95% CI = 0.70–1.00], respectively).

DISCUSSION

Diagnosing delirium in children, particularly infants and children with developmental delay, can be challenging. The ability to diagnose delirium requires recognition of a disturbance in consciousness and cognition (DSM–IV, criteria A and B) with a rapid onset and fluctuating course (criterion C) with a linkage to a physiologic cause (criterion D) (APA, 2000). The list of differential diagnoses includes many presentations in pediatrics and child psychiatry (Smith et al., 2011) and usually requires input from many observers over a period of time to differentiate delirium from other clinical issues (Esseveld et al., 2013; Schieveld et al., 2007; Smith et al., 2009). Delirium has many comorbidities (e.g., pain, premorbid or situational anxiety, behavioral regression) that complicate the clinical picture. However, “thinking developmentally” about infants and children and obtaining specific information about each child’s baseline cognition, communication, and behavior make this a valid and useful diagnosis.

Discordant Assessment

The one discordant assessment came very early on in the study and was the case of a one-month-old infant with acute respiratory failure due to respiratory syncytial viral pneumonia. Sleeping excessively, with no differentiation of day and night, she was generally quiet, minimally reactive, and not responding much to her parents. One psychiatrist diagnosed her decreased activity and fluctuating mental status as hypoactive delirium. The second psychiatrist felt that this activity was consistent with a “sick baby” and did not meet the criteria for delirium. By the following day, both psychiatrists independently agreed that she did not meet the criteria for a delirium diagnosis, though each noted in their assessment that the patient may have had “subclinical delirium” and needed to be watched closely. Her deliriogenic medications has been reduced, and within 24 hours her awareness, activity, and reactivity improved. On third assessment, both psychiatrists again agreed that she was not delirious. This case highlights the difficulty that may exist in making a definitive diagnosis of delirium in young infants, hypoactive delirium at any age, and subclinical cases. However, the consideration of a diagnosis of delirium led to a clinical pathway that reduced potentially offending medications, ruled out medical causes of delirium, and implemented positive environmental interventions, all of which may have benefited this child (Schieveld et al., 2009; Smith et al., 2013).

SUMMARY

Although it has been generally accepted that the psychiatric interview is the “gold standard” for diagnosing delirium, reliability for this clinical diagnosis in children has not been previously reported. Given that the diagnosis of delirium in preverbal and developmentally delayed children is challenging, it is reassuring that we were able to show a high interrater reliability for psychiatric diagnosis. With specific attention to normal development in each symptom domain, consulting child psychiatrists and other clinicians can be equipped to diagnose delirium in medically ill children of nearly all developmental ages and trajectories.