Abstract
Aim
Pediatric delirium has been well investigated and its prevalence is reported to be from 20% to 44%. For pediatric intensive care settings, several validated assessment tools for diagnosing delirium, including the Preschool Confusion Assessment Method for the Intensive Care Unit (psCAM‐ICU), are available in English. However, validated assessment tools for identifying pediatric delirium are unavailable in Japanese. Therefore, the aim of this study is to verify the Japanese translation of the psCAM‐ICU.
Methods
We enrolled patients at the Pediatric ICU at University of Tsukuba Hospital (Tsukuba, Japan) from May 2017 to February 2019. Enrollment criteria included patients aged 6 months to 5 years, and we excluded coma patients scoring under −4 on the Richmond Agitation–Sedation Scale or suffering from stroke. Pediatric patient delirium was simultaneously evaluated by three medical workers (pediatric intensivist and researchers). Psychiatrists then verified these findings against criteria of the Diagnostic and Statistical Manual of Mental Disorders – 5th Edition. We evaluated criterion validity (sensitivity and specificity) and reliability using Cohen's κ coefficient.
Results
We made a total of 56 independent assessments of 19 patients (42% female) with an average age of 18 (±15) weeks. Mechanical ventilation was used at least once in 73% of patients and the positive rate of delirium was 54% in total observation. Overall, the psCAM‐ICU showed high sensitivity, specificity (sensitivity, 0.90 [95% confidence interval [CI], 0.80–0.94]; specificity, 0.93 [95% CI, 0.83–0.97]), and high reliability within the researcher assessments (κ = 0.92; 95% CI, 0.82–1.0).
Conclusion
We verified the psCAM‐ICU and it shows high validity and reliability.
Keywords: Delirium, Japanese version, pediatric delirium, psCAM‐ICU, validation
Introduction
Delirium frequently occurs as a treatment complication in intensive care patients and pediatric delirium has a reported prevalence of 20–44%.1, 2 Delirium rapidly arises due to a physiological transformation characterized by fluctuating course, attention deficits, disorganized thinking, and an altered level of consciousness.3 Within published works, delirium has also become recognized as a biomarker of acute brain dysfunction.4 This devastating condition is associated with higher mortality,5 longer length of hospital stay, and prolonged mechanical ventilation time, making it a serious health‐care issue.6 Moreover, delirium is usually underrecognized in daily routine care;7 using assessment tools will improve the diagnostic accuracy of delirium.8, 9 Also complicating diagnosis is the further issue that, within pediatric intensive care settings, there is a shortage of reports on the effect of mechanical ventilation with respect to sedation condition and the development of delirium. There are several validated assessment tools available in English for pediatric delirium to address this issue, including the Preschool Confusion Assessment Method for the Intensive Care Unit (psCAM‐ICU).2 The psCAM‐ICU is in accordance with the Diagnostic and Statistical Manual of Mental Disorders – 5th Edition (DSM‐5) and has been reported as a reliable and valid tool for infant and preschool‐aged children.2 We have previously published a back‐translated version of the psCAM‐ICU in Japanese.10 However, as there are no currently validated pediatric delirium assessment tools in Japanese, the aim of this study is to validate our back‐translated psCAM‐ICU for clinical use.
Method
Patient selection
We registered patients in our mixed‐use (eight‐bed post‐surgical and internal medicine) pediatric ICU at the University of Tsukuba Hospital (Tsukuba, Japan) from May 2017 to February 2019 on Wednesdays during morning and afternoon shifts (9.00 am and 2.00 pm). Enrollment criteria included patients aged 6 months to 5 years and we excluded coma patients scoring under −4 on the Richmond Agitation–Sedation Scale (RASS), or suffering from stroke. We extracted characteristics, including age, sex, mechanical ventilation status, and diagnosis for pediatric ICU admission, from medical records. This study was carried out under laws equivalent to or derived from the principles of the Declaration of Helsinki and was approved by the University of Tsukuba Institutional Review Board (approval # H28‐085).
Characteristics, prevalence, and distribution of delirium
We recorded age, sex, diagnosis, mechanical ventilation status, usage of dexmedetomidine, usage of midazolam, usage of opioids, length of ICU stay (in days), Pediatric Index of Mortality 2 score (PIM2),11 Pediatric Sequential Organ Failure Assessment Score (pSOFA),12 and sedation status on the RASS. Both PIM2 and pSOFA are generally accepted methods of scoring severity and already validated for prediction of in‐hospital mortality.12, 13 The RASS was originally developed for adult patients but has recently been validated in critically ill pediatric patients.14 As Smith and colleagues examined the validity of psCAM‐ICU by using RASS,2 we deemed it suitable for our purposes. Relationships between cohort characteristics and the distribution of delirium with and without mechanical ventilation with respect to sedation condition were then analyzed.
Pediatric delirium assessment
Pediatric patient delirium was simultaneously evaluated by three medical workers (researchers and a pediatric intensivist). Researchers evaluated pediatric delirium by the Japanese version of the psCAM‐ICU while the pediatric intensivist assessed delirium in an assistive capacity by the Vanderbilt Assessment for Delirium in Infants and Children (VADIC),15 which was then verified by a psychiatrist in accordance with DSM‐5 criteria.3 As psychiatrists in Japan have little training in assessment of the varying degrees of pediatric delirium, we therefore chose a pediatric intensivist for the assessment but relied on objective diagnosis by the psychiatrist (Fig. 1). Prior to the study, the first author learned the measurement procedures of psCAM‐ICU at the original author's hospital (The Monroe Carell Jr. Children's Hospital in Vanderbilt, Nashville, TN, USA).
Figure 1.
Overview of the research.
Statistical analysis
To assess reliability between an observer's psCAM‐ICU assessment and the psychiatrist's diagnosis for pediatric delirium, we used Cohen's κ coefficient and bootstrapping method for evaluating a 95% confidence interval (CI). We also assessed reliability between two psychiatrists’ diagnoses. To assess criterion validity, we calculated sensitivity, specificity, negative predictive values (NPV), and positive predictive values (PPV) for psCAM‐ICU in our cohort.
Sample size calculations
Variability and required sample size fluctuate accordingly with the state of the cohort. Therefore, we evaluated sample size based on reliability as previously reported.16, 17 We undertook a pilot study for estimating sample size and we observed 24 paired, independent assessments out of a total of 74 with a good coefficient (κ = 0.90; 95% CI, 0.72–1.0). From this result, we estimated the κ coefficient between observers as 0.9 and the lower κ coefficient was also within the definition of Guilford's Rule of Thumb18 (lower κ = 0.7; upper κ coefficient = 0.99). The significance level (α) was 0.05 and we estimated the positive rate of delirium as 0.33 from the pilot study. Our calculations predicted that 56 observations were needed to reach significance for this study.19, 20
Results
Characteristics
From May 2017 to February 2019, we undertook a total of 162 observations (Fig. 2). Of these, 106 were excluded for the following reasons: 38 observations were in patients younger than 6 months of age, 15 observations were in patients over 5 years of age, and 53 observations were in comatose patients. With regard to the baseline characteristics of study patients, cardiac surgery patients accounted for 63% of enrollees. A total of 19 patients (42% female) were of an average age of 18 (±15) weeks. Approximately three‐quarters (73%) of patients had received mechanical ventilation at least once and the average severity score, as evaluated by PIM2, was 2.0 (±1.6). (Table 1) Of patients who participated in research evaluations, 55% were cardiac surgery patients. The average age of the 56 patients evaluated (42% female) was 12 (±11) months. Approximately half (46%) of patients had received mechanical ventilation at the time of observation and the average severity score, as evaluated by pSOFA, was 4.9 (±3.9). Nearly half of those evaluated received dexmedetomidine (48%), midazolam (60%), or opioids (53%) (Table 2).
Figure 2.
Participant flow chart, including exclusion criteria, and final registered study observations.
Table 1.
Baseline characteristics of study patients
| Variable | n = 19 |
|---|---|
| Age (months) ± SD | 18 ± 15 |
| Female, n (%) | 8 (42) |
| Diagnosis | |
| Cardiac surgical, n (%) | 12 (63) |
| Abdominal surgical, n (%) | 2 (10) |
| Thoracic surgical, n (%) | 1 (5) |
| Medical, n (%) | 4 (21) |
| PIM2 ± SD | 2.0 ± 1.6 |
| Mechanical ventilation† | 14 (73) |
Using mechanical ventilation during intensive care unit stay.
PIM2, Pediatric Index of Mortality 2; SD, standard deviation.
Table 2.
Baseline characteristics of research evaluation units among pediatric patients with delirium (n = 56)
| Variable | n = 56 |
|---|---|
| Age (months) ± SD | 12 ± 11 |
| Female, n (%) | 24 (42) |
| Diagnosis | |
| Cardiac surgical, n (%) | 31 (55) |
| Abdominal surgical, n (%) | 3 (5) |
| Thoracic surgical, n (%) | 2 (3) |
| Medical, n (%) | 20 (35) |
| Mechanical ventilation† | 26 (46) |
| pSOFA ± SD | 4.9 ± 3.9 |
| Use of dexmedetomidine, n (%) | 27 (48) |
| Use of midazolam, n (%) | 34 (60) |
| Use of opioids, n (%) | 30 (53) |
| ICU days at observation | 45 ± 53 |
Using mechanical ventilation at the observation.
ICU, intensive care unit; pSOFA, Pediatric Sequential Organ Failure Assessment; SD, standard deviation.
Distribution of delirium in all measurements
Evaluated pediatric delirium in total was 54% among 56 independent psychiatric assessments, mechanical ventilation was present in 79% of 24 independent assessments and absent among 34% of 32 independent assessments, all by DSM‐5 criteria. Sedated (RASS –1 to −3) patients tended towards high frequency of delirium compared with agitated (RASS +1 to +3) patients (RASS: −3, 100%; −1, 92%; 0, 32%; +1, 56%). There were no observations for patients with RASS scores of +2 to +4 (agitated to combative) in our cohort (Fig. 3).
Figure 3.
Distribution of pediatric delirium within the study population with and without mechanical ventilation.
Criterion validity
A total of 56 independent assessments were undertaken on 19 patients. Table 3 shows the criterion validity of the Japanese version of the psCAM‐ICU with regards to psychiatric diagnosis using DSM‐5 criteria. The psCAM‐ICU showed a sensitivity of 0.90 (95% CI, 0.80–0.94), specificity of 0.93 (95% CI, 0.83–0.97), PPV of 0.93 (95% CI, 0.83–0.97), NPV of 0.90 (95% CI, 0.80–0.94), likelihood ratio for positive results (LR+) of 13.0 (95% CI, 4.78–41.0), and a likelihood ratio for negative results (LR−) of 0.1 (95% CI, 0.05–0.22). When mechanical ventilation was present, the psCAM‐ICU showed a sensitivity of 0.89 (95% CI, 0.79–0.93), specificity of 0.80 (95% CI, 0.43–0.96), PPV of 0.94 (95% CI, 0.84–0.98), NPV of 0.66 (95% CI, 0.36–0.80), LR+ of 4.47 (95% CI, 1.40–23.14), and LR− of 0.13 (95% CI, 0.06–0.46) within 24 independent assessments. In patients without mechanical ventilation, the psCAM‐ICU showed a sensitivity of 0.91 (95% CI, 0.73–0.97), specificity of 0.95 (95% CI, 0.84–0.98), PPV of 0.91 (95% CI, 0.73–0.97), NPV of 0.95 (95% CI, 0.84–0.98), LR+ of 18.3 (95% CI, 4.59–70.49), and LR− of 0.08 (95% CI, 0.02–0.31) within 32 independent assessments.
Table 3.
Criterion validity of the Japanese Preschool Confusion Assessment Method for the Intensive Care Unit
| Variable | Without mechanical ventilation, n = 32 | With mechanical ventilation, n = 24 | Total number of observations, n = 56 |
|---|---|---|---|
| Sensitivity | 0.91 (0.73–0.97) | 0.89 (0.79–0.93) | 0.90 (0.80–0.94) |
| Specificity | 0.95 (0.84–0.98) | 0.80 (0.43–0.96) | 0.93 (0.83‐0.97) |
| PPV | 0.91 (0.73–0.97) | 0.94 (0.84–0.98) | 0.93 (0.83–0.97) |
| NPV | 0.95 (0.84–0.98) | 0.66 (0.36–0.80) | 0.90 (0.80–0.94) |
| LR+ | 18.3 (4.59–70.49) | 4.47 (1.40–23.14) | 13.0 (4.78–41.0) |
| LR− | 0.08 (0.02–0.31) | 0.13 (0.06–0.46) | 0.1 (0.05–0.22) |
Data are shown as value (95% confidence interval).
LR+, likelihood ratio for positive results; LR−, likelihood ratio for negative results; NPV, negative predictive values; PPV, positive predictive value.
Reliability
The Japanese version of the psCAM‐ICU showed high reliability for researchers with respect to total assessments (κ = 0.92; 95% CI, 0.82–1.0) (Table 4). This high reliability is unchanged between patients with mechanical ventilation (κ = 0.89; 95% CI, 0.68–1.0) and those without (κ = 0.93; 95% CI, 0.79–1.0). Reliability for the psychiatrist evaluation was high in the total population (κ = 1.0; 95% CI, 1.0–1.0), with mechanical ventilation (κ = 1.0; 95% CI, 1.0–1.0), and without mechanical ventilation (κ = 1.0; 95% CI, 1.0–1.0).
Table 4.
Reliability of the Japanese Preschool Confusion Assessment Method for the Intensive Care Unit (psCAM‐ICU)
| Variable | Without mechanical ventilation, n = 32 | With mechanical ventilation, n = 24 | Total number of observations, n = 56 |
|---|---|---|---|
| Researcher to researcher‡ | 0.93 (0.79–1.0) | 0.89 (0.68–1.0) | 0.92 (0.82–1.0) |
| Psychiatrist to psychiatrist‡ , ‡ | 1.0 (1.0–1.0) | 1.0 (1.0–1.0) | 1.0 (1.0–1.0) |
Data are shown as κ coefficient (95% confidence interval).
Researchers using psCAM‐ICU and comparison between researchers’ assessments.
Psychiatrist using the Diagnostic and Statistical Manual of Mental Disorders – 5th Edition and comparison with psychiatrist diagnosis.
Discussion
The present study is the first to report the high criterion validity and reliability of the Japanese version of the PsCAM‐ICU (sensitivity of 0.90, specificity of 0.93, reliability κ = 0.92) in our cohort. As these results are in line with the psCAM‐ICU original study, we considered the back‐translated Japanese version to have valid accuracy. Methodologically, in this study, we modified the original method and used a pediatric intensivist to assess pediatric delirium using VADIC to support a psychiatric diagnosis under DSM‐5 criteria. This change was needed to effectively bridge gaps between two cultures; merely repeating the original method might not have been accurate for Japanese clinical usage.21 As the capability of Japanese psychiatry to assess pediatric delirium in Japan is limited, we posited that a pediatric intensivist, as an experienced specialist in pediatric observation, would be more suitable in our case. Additionally, the VADIC was suitable, in our judgement, to bridge the gap between Japanese and American cultural differences. Our observed delirium rate of 54% was variable based on the presence or absence of mechanical ventilation. Under light sedation (RASS −1), non‐mechanically ventilated patients trended towards delirium compared to those under mechanical ventilation. In contrast, in more agitated states (RASS +1 to +4), mechanically ventilated patients tended to experience a higher prevalence of delirium. This result indicates that patient agitation under mechanical ventilation, as well as drowsy patients without mechanical ventilation, need careful observation and assessment for signs of delirium. In our cohort, we did not observe RASS scores of +2 to +4 (agitated to combative) as a majority of our patients were admitted for cardiac surgery and sedation is strictly controlled in these patients to prevent negative hemodynamic effects. Nor did we observe RASS scores of −2 (light sedation) in any patient. As this state is defined by briefly (less than 10 s) awakens with eye contact to voice, our average age of 18 months might have enrolled patients who were not developmentally able to sustain light sedation. In spite of the above observations, RASS is still useful for estimating pediatric patient sedation levels and plays a role in deciding when to evaluate delirium assessment by psCAM‐ICU. However, even though children over the age of 5 years can be evaluated by the validated pCAM‐ICU,22 there has never been a validation study using a Japanese version until now. With this study, pediatricians now have validated tools available for evaluating delirium in all age ranges above 6 months.
Pediatric delirium in the Japanese pediatric ICU is still a relatively undiagnosed phenomenon,23 resulting in lapses of both diagnosis and management.24 For this reason, our study is the first to provide validation for the psCAM‐ICU tool, which could promote proper diagnosis of pediatric delirium as well as encourage the creation of screening and treatment guidelines to diagnose and manage this condition.
Limitation
We acknowledge several limitations of this study. This is a single center study consisting mostly of cardiac patients (due to the nature of our institute) selected by a non‐randomized participant pool that was chosen primarily by availability during the study period. This fact might reduce the generalizability of the results. Also, we applied repeated measurements in this study and this could generate bias in predicting delirium prevalence. However, as the definition of delirium indicates fluctuations over the course of the day, we believe this inherent variability prevented such bias. Additionally, some measurements could not be taken during emergency situations. Finally, we ran an additional experiment due to changes in the sample size estimation, dividing the study into two periods, which could affect the validity of the scale.
Conclusion
We VERIFIED THE Japanese version of psCAM‐ICU as having high validity and reliability.
Disclosure
Approval of the research protocol: The Institutional Review Board of the University of Tsukuba Affiliated Hospital approved the present study (approval # H28‐085).
Informed consent: Informed consent was obtained from the guardians by opt‐out in publicity documents.
Registry and the registration no. of the study/trial: N/A.
Animal studies: N/A.
Conflict of interest: None declared.
Acknowledgments
The authors would like to thank the patients and their families for their participation in this study.
Funding information
This work was supported by Tsukuba University education/research funds.
References
- 1. Traube C, Silver G, Kearney J et al Cornell assessment of pediatric delirium. Crit. Care Med. 2014; 42: 656–63. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Smith HAB, Gangopadhyay M, Goben CM et al The preschool confusion assessment method for the ICU: valid and reliable delirium monitoring for critically ill infants and children. Crit. Care Med. 2016;44:592–600. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. American Psychiatric Association . Diagnostic and Statistical Manual of Mental Disorders : DSM‐5. Washington; London: American Psychiatric Publishing, 2014. [Google Scholar]
- 4. Almeida ICT, Soares M, Bozza FA et al The impact of acute brain dysfunction in the outcomes of mechanically ventilated cancer patients. PLoS ONE 2014; 9: 1–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Traube C, Silver G, Gerber LM et al Delirium and mortality in critically ill children: epidemiology and outcomes of pediatric delirium. Crit. Care Med. 2017; 45: 891–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Alvarez RV, Palmer C, Czaja AS et al Delirium is a common and early finding in patients in the pediatric cardiac intensive care unit. J. Pediatr. 2018; 195: 206–12. [DOI] [PubMed] [Google Scholar]
- 7. Spronk PE, Riekerk B, Hofhuis J, Rommes JH. Occurrence of delirium is severely underestimated in the ICU during daily care. Intensive Care Med. 2009; 35: 1276–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Devlin JW, Fong JJ, Schumaker G, O'Connor H, Ruthazer R, Garpestad E. Use of a validated delirium assessment tool improves the ability of physicians to identify delirium in medical intensive care unit patients. Crit. Care Med. 2007; 35: 2721–4. [DOI] [PubMed] [Google Scholar]
- 9. Devlin JW, Marquis F, Riker RR et al Combined didactic and scenario‐based education improves the ability of intensive care unit staff to recognize delirium at the bedside. Crit. Care 2008; 12: 1–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Matsuishi Y, Hoshino H, Shimojo N et al Development of the Japanese version of the Preschool Confusion Assessment Method for the ICU. Acute Med. Surg. 2018; 5: 102–105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Slater A, Shann F, Pearson G. PIM2: a revised version of the Paediatric Index of Mortality. Intensive Care Med. 2003; 29: 278–85. [DOI] [PubMed] [Google Scholar]
- 12. Matics TJ, Sanchez‐Pinto LN. Adaptation and validation of a pediatric sequential organ failure assessment score and evaluation of the Sepsis‐3 definitions in critically ill children. JAMA Pediatr. 2017; 171: e172352. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Wolfler A, Silvani P, Musicco M et al Pediatric index of Mortality 2 score in Italy: a multicenter, prospective, observational study. Intensive Care Med. 2007; 33: 1407–13. [DOI] [PubMed] [Google Scholar]
- 14. Kerson AG, DeMaria R, Mauer E et al Validity of the richmond agitation‐sedation scale (RASS) in critically ill children. J. Intensive Care. 2016; 4: 1–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Gangopadhyay M, Smith H, Pao M et al Development of the vanderbilt assessment for delirium in infants and children to standardize pediatric delirium assessment by psychiatrists. Psychosomatics. 2015; 58: 355–63. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Voepel‐Lewis T, Zanotti J, Dammeyer JA, Merkel S. Reliability and validity of the face, legs, activity, cry, consolability behavioral tool in assessing acute pain in critically ill patients. Am. J. Crit. Care 2010; 19: 55–61. [DOI] [PubMed] [Google Scholar]
- 17. Matsuishi Y, Hoshino H, Shimojo N et al Verifying the validity and reliability of the Japanese version of the Face, Legs, Activity, Cry, Consolability (FLACC) Behavioral Scale. PLoS ONE 2018; 13: e0194094. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Guilford JP. Fundamental Statistics in Psychology and Education. New York: McGraw Hill, 1956. [Google Scholar]
- 19. Rotondi MA, Donner A. A confidence interval approach to sample size estimation for interobserver agreement studies with multiple raters and outcomes. J. Clin. Epidemiol. 2012; 65: 778–84. [DOI] [PubMed] [Google Scholar]
- 20. Donner A, Rotondi MA. Sample size requirements for interval estimation of the kappa statistic for interobserver agreement studies with a binary outcome and multiple raters. Int. J. Biostat. 2010; 6: Article 31. [DOI] [PubMed] [Google Scholar]
- 21. Guillemin F, Bombardier C, Beaton D. Cross‐cultural adaptation of health‐related quality of life measures: literature review and proposed guidelines. J. Clio. itpidemiol. 1993; 46: 1417–32. [DOI] [PubMed] [Google Scholar]
- 22. Smith HAB, Boyd J, Fuchs DC et al Diagnosing delirium in critically ill children: validity and reliability of the pediatric confusion assessment method for the intensive care unit. Crit. Care Med. 2011; 39: 150–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Creten C, Van Der Zwaan S, Blankespoor RJ et al Pediatric delirium in the pediatric intensive care unit: a systematic review and an update on key issues and research questions. Minerva Anestesiol. 2011;77:1099–107. [PubMed] [Google Scholar]
- 24. Staveski SL, Pickler RH, Lin L et al Management of pediatric delirium in pediatric cardiac intensive care patients: an international survey of current practices. Pediatr Crit Care Med. 2018; 19: 538–43. [DOI] [PubMed] [Google Scholar]