Development and validation of risk-stratification delirium prediction model for critically ill patients

Abstract

The objective is to develop a model based on risk stratification to predict delirium among adult critically ill patients and whether early intervention could be provided for high-risk patients, which could reduce the incidence of delirium.

We designed a prospective, observational, single-center study. We examined 11 factors, including age, APACHE-II score, coma, emergency operation, mechanical ventilation (MV), multiple trauma, metabolic acidosis, history of hypertension, delirium and dementia, and application of Dexmedetomidine Hydrochloride. Confusion assessment method for the intensive care unit (CAM-ICU) was performed to screen patients during their ICU stay. Multivariate logistic regression analysis was used to develop the model, and we assessed the predictive ability of the model by using the area under the receiver operating characteristics curve (AUROC).

From May 17, 2016 to September 25, 2016, 681 consecutive patients were screened, 61 of whom were excluded. The most frequent reason for exclusion was sustained coma 30 (4.4%), followed by a length of stay in the ICU < 24 hours 18 (2.6%) and delirium before ICU admission 13 (1.9%). Among the remaining 620 patients (including 162 nervous system disease patients), 160 patients (25.8%) developed delirium, and 64 (39.5%) had nervous system disease. The mean age was 55 ± 18 years old, the mean APACHE-II score was 16 ± 4, and 49.2% of them were male. Spearman analysis of nervous system disease and incidence of delirium showed that the correlation coefficient was 0.186 (P < .01). We constructed a prediction model that included 11 risk factors. The AUROC was 0.78 (95% CI 0.72–0.83).

We developed the model using 11 related factors to predict delirium in critically ill patients and further determined that prophylaxis with Dexmedetomidine Hydrochloride in delirious ICU patients was beneficial. Patients who suffer from nervous system disease are at a higher incidence of delirium, and corresponding measures should be used for prevention.

Trial registration: ChiCTR-OOC-16008535.

1. Introduction

Delirium, which is known as an acute onset of fluctuating changes in mental status, changes levels of consciousness and inattentiveness. Its pathogenic factors can be divided into predisposing and precipitating factors. Because the pathophysiological mechanism is still unknown, in the early stages of studying delirium, it was also called intensive care unit (ICU) mental disturbance, ICU syndrome, pathological encephalopathy, or even acute brain failure.^[1] As research continues, there is currently a consensus that we should describe and diagnose delirium by using the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (DSM-5)^[2] in the ICU. According to multiple recent studies, the incidence of delirium is 45% to 87%;^[3–6] in 2004, the costs of delirium to the US health care system were estimated to be $4 to $16 billion annually.^[7] However, there are no epidemic data about delirium in China, especially in critically ill patients. Delirium is being increasingly recognized as a significant contributor to the morbidity and mortality of critically ill patients and can increase the number of mechanical ventilation (MV) days, ICU stays, amount of sedative drugs, and long-time impaired cognitive function.^[8–10] Correlational research ^[11] has suggested that screening can increase the rate of its diagnosis to 64%, and early essential intervention could reduce both the incidence of delirium and its duration and complications. Identifying patients who are at a high risk of delirium for early essential intervention requires providing prevention measures as soon as possible, so patients at such a high risk can obtain the greatest benefit. Further, because of the risk stratification, the low-risk patients would be safe from side effects.

According to the previous researches that the risk of delirium relies on the intricacies of predisposing and precipitating risk factors.^[12] It is important that if we could find out which factors are contributed for delirium in ICU.

It is imperative to explore and develop a delirium prediction model for critically ill patients. Our study aimed to develop a delirium prediction model based on relative factors of delirium in ICU patients during their ICU stay and validate its discrimination for critically ill patients with a high risk of delirium in this study.

1.1. Design

Prospective, observational, single-center study was conducted in 3 ICUs in 1 university hospital. Ethics approval was provided by the Medical Ethical Committee of Lanzhou University Second Hospital (study number 2016A-046). Because confusion assessment method for the intensive care unit (CAM-ICU) determinations are part of clinical practice in all ICUs, no additional interventions were performed. Consequently, data collection was not burdensome to patients, and data were captured and analyzed anonymously. Consent was signed by patients or their families or relatives when they were in a coma.

China clinical trial register number: ChiCTR-OOC-16008535.

1.2. Enrollment

All participating ICUs enrolled patients who met the inclusion criteria between May 17, 2016 and September 25, 2016. The inclusion criteria were consecutive adult patients, age >18 years old, and the exclusion criteria were delirium at ICU admission; sustained coma throughout the ICU stay; admission to the ICU for <24 hours; suffering from serious auditory or visual disorders; severely mentally disabled; or suffering from serious, receptive aphasia.

1.3. Delirium assessment

All patients were assessed by well-trained ICU nurses and doctors using the validated delirium assessment tool, the CAM-ICU, at both 9 am and 5 pm. Delirium was defined as at least 1 CAM-ICU positive criterion during their ICU stay.

1.4. Data collection

We collected data on age, APACHE-II score, coma, emergency operation, mechanical ventilation, multiple trauma, metabolic acidosis, history of hypertension, history of delirium, history of dementia, and the application of Dexmedetomidine Hydrochloride within 24 hours after admission to the ICU. We recorded each patient's data in forms.

1.5. Main outcome measure

Development of delirium during a patient's stay in the ICU.

1.6. Statistics

We determined the relationship between the chosen factors and the incidence of ICU delirium using multivariate logistic regression, and assessed the predictive ability of the model by using the area under the receiver operating characteristics curve (AUROC).

1.7. Sample size

The prediction model consists of 11 predictors, which indicated that we would need at least 5 to 10 patients with delirium. With an anticipated delirium incidence of 20% to 40% and an attrition rate of 10%, we aimed to enroll at least (11 × 10/0.2)/0.9 = 611 patients. The first aim was to develop the model, and the second was to validate the model.

2. Result

A total of 681 ICU patients were screened, including 162 with nervous system disease. Among the remaining 620 patients, 160 (25.8%) developed delirium. The most frequent reason for exclusion was sustained coma 30 (4.4%), followed by a length of stay in the ICU < 24 hours 18 (2.6%) and development of delirium at the time of ICU admission 13 (1.9%) (Fig. 1). The mean ± SD age of patients was 55.20+0.71 years, the mean APACHE-II score was 15.87+0.17, and 49.2% of the included patients were male (Table 1). Spearman analysis of nervous system disease and delirium showed that the correlation coefficient was 0.186 (P < .01). The AUROC was 0.78 (95% CI 0.72–0.83).

Figure 1.

Flow chart of development and validation study.

Table 1.

Patients’ characteristics in study.

2.1. Development of prediction model

We recorded each patient's data in forms, and we used data from first 310 critically ill patients to construct the predictive model, which consisted of 11 risk factors (Table 2). Then, the equation is Risk of delirium = 1/(1+exp−(−1.78+0.001 × Age+0.015 × APACHE- II score+0.801×Mechanical ventilation+0.358× Emergency operation+0.004×Coma+0.148×Multiple trauma+0.353×Metabolic acidosis+0.117×History of hypertension+1.377×History of delirium+0.318×History of dementia−0.57×Dexmedetomidine Hydrochloride.

Table 2.

Variables of delirium prediction model and regression coefficients.

2.2. Validation of prediction model

We used data from the last 310 critically ill patients. The AUROC was 0.78 (95% CI 0.72–0.83). We divided the complete group into the following 3 different risk groups; low, moderate, and high risk, with prediction model scores of 0% to 20%, >20% to 40%, and >40%. We also determined the sensitivity, specificity, and likelihood ratios for each risk group (Fig. 2).

Figure 2.

Area under receiver operating characteristics curve (AUROC) for validation (AUROC=0.78), and in the high-risk group, the PLR is 16.9. PLR = positive likelihood ratio.

3. Discussion

Delirium is a common clinical syndrome with an incidence of 20% to 80%, as reported in different papers; ^[13,14] however, this value is always underestimated. The critical factor for shortening the duration, mitigating the severity, and improving the poor prognosis of delirium is to diagnose it early. The incidence of delirium in our study is 25.8%, which is concordant with other research. However, the pathogenesis mechanism is still unknown. Once the critically ill patients developed delirium, regardless of the duration of ventilation, the length of ICU stays, and total hospitalization expenses all increased. In the ICU, these are many tools for assessing delirium, among which the CAM-ICU has the highest sensitivity and specificity. It is very important to screen delirium in critically ill patients, which could increase the recognition rate of delirium to 64%.^[11] Thus, we can perform the necessary intervention measures in high-risk patients earlier. However, time and energy are wasted on performing regular measures for the critically ill patients, and some of the low-risk patients may experience side effects from prophylactic measures. Similarly, the stratification of delirium in critically ill patients is also important. There are pre-existing delirium prediction models of noncritically ill patients and geriatric internal medicine, and evidence for a delirium prediction model for general critically ill patients is being collected. The PRE-DELIRIC model developed in 2012 by van den Boogaard et al^[15] showed perfect prediction, and the AUROC was 0.87 (95% CI 0.85–0.89). However, they failed in setting up the subgroup of acute cerebrovascular disease, which may be a confounding factor of the incidence of delirium. In future clinical work, the first step is to predict the incidence of delirium and the risk stratification using the prediction model, if the patient also has acute cerebrovascular disease, it is suggested that the patient be classified a high-risk patient. Immediate measures were taken to prevent delirium. This may further reduce the incidence of delirium and improve the prognosis of patients. This avoids the simply use of existing prediction models and leaves them out. Certainly, subsequent study is needed to further examine whether this approach actually benefits patients. Early identification and prevention are the key to unknown etiologies at present. Our study aims to develop a delirium prediction model by using the 11 related factors of delirium, which are now considered to be strongly related to delirium. Achieving the purpose of early identification in critically ill patients with delirium, especially in high-risk patients, will provide a reliable clinical theoretical basis for early prevention and therapy. However, our research still has some limitations: the sensitivity and specificity of CAM-ICU have been demonstrated in preexisting research;^[16–18] however, they may be decreased for multiple reasons, such as the bedside nurses’ use and understanding of CAM-ICU or if the patients are sedative or intubated.^[19] We also chose the time points of 9 am and 5 pm because these 2 times were used to artificially assess the patients by CAM-ICU, which may have decreased its sensitivity and specificity. As a result, the real incidence of ill patients with delirium would be undervalued. We did not consider any laboratory values in our study because the laboratory values that diagnose delirium are still in the exploratory stage.^[20–22] In the future, molecular markers to diagnose delirium will likely be developed. Despite the aforementioned limitations, we are still sure that our study provides innovative and original evidence. First, the related factors in our predictive model are based on a meta-analysis ^[12] of the newest and highest-level evidence, so it can more accurately represent the current situation with delirium. Second, to our knowledge, this is the first prospective observational study to develop a delirium prediction model in critically ill patients in China. Further, our study pre-established the subgroup of acute cerebrovascular disease (refers to TIA/stroke only) in that the incidence of delirium in these patients was higher than others. Consequently, the results of our study may reflect the real incidence of delirium in critically ill patients and offer more reliable and credible evidence for precisely predicting delirium. However, because of limitations in the delirium-related factors, the limitation of our prediction model will become very obvious with the discovery of more delirium-related factors. Consequently, it is time to explore pathogenesis mechanisms in molecular biology, cytology, and even susceptibility genes. The breakthrough in these areas will definitely uncover the real pathogenetic mechanism of delirium.

4. Conclusion

We successfully developed a multivariate logistic regression equation by using 11 related factors to predict delirium in critically ill patients and to determine the prophylaxis of Dexmedetomidine Hydrochloride in delirious ICU patients. Finally, patients who suffer from nervous system disease had a higher incidence of delirium and should thus receive corresponding measures to reduce the incidence, duration, and complications of delirium.