Clinical prediction models for intensive care unit admission in patients with acute poisoning: is it time for a comprehensive evaluation of their utility?

Dear editor

Evaluation of the need for intensive care unit (ICU) admission in patients with acute poisoning is a topic of interest. While traditional scoring systems for assessment have been previously documented in the literature, it’s important to note that these scoring systems are not specifically tailored for acute poisoning. ^1–3 It is exciting that two novel models for predicting ICU admissions of patients with acute poisoning were recently published in Toxicology Research.^4,5 El-Gharbawy DM et al. developed a nomogram for acute antipsychotic poisoning to predict the risk of ICU admission.⁴ Furthermore, a nomogram was created by Sharif AF et al. to assess the risk of ICU admission in cases of acute poisoning with central nervous system xenobiotics⁵ (Table 1).

Table 1.

Novel clinical prediction models for ICU admission in patients with acute poisoning.

Type of exposure	Modeling sample size	Modeling method	Variable	Verification	Evaluation
Antipsychotic4	140	logistic regression	Seizures,corrected QT interval,mean arterial blood pressure,oxygen saturation,GCS	Internal	Discrimination, Calibration
CNS xenobiotics5	143	logistic regression	PSS, GCS, HCO3, pH, RBG level	Internal	Discrimination
Multiple types of poisons14	9,679	generalized linear mixed model	Age, SBP, HR, GCS, Intoxication type,Co-morbidities	Internal+ External16–18	Discrimination
Multiple types of poisons15	1,260	logistic regression	GCS,Oxygen saturation,DBP, RR,Bicarbonate.	Internal+ External18	Discrimination

SBP, systolic blood pressure; DBP, Diastolic blood pressure; RR, Respiratory rate;HR, heart rate; GCS, Glasgow Coma Scale; CNS, central nervous system; Bicarbonate (common name) = HCO3 (molecular formula).

The nomogram and scoring systems are widely recognized as forms of presenting clinical prediction models.⁶ We have previously concentrated on clinical predictive models for acute poisoning, with limited development and even fewer validations.⁷

In practice, both the development and validation of clinical predictive models are critical.⁸ Notably, the British Medical Journal (BMJ) recently published a series of guideline reports regulating the validation of predictive models.^8–10 We appreciate the authors for their dedicated efforts and would like to offer a few suggestions to advance this work further.

First, all authors should adhere to the TRIPOD statement, as only prediction models developed and validated through a rigorous methodological process should guide clinical decision-making.¹¹ This report describes the types of prediction model studies, including development and validation. Moreover, it highlights common deficiencies of predictive models, such as issues with statistical methods, the use of small datasets, mishandling of missing data, and lack of validation.¹¹ Internal validation is a minimal expectation for research on predictive model development. Common approaches to internal validation include data splitting or resampling (eg, k-fold cross validation or bootstrapping).⁸ Nonetheless, data splitting is generally advised against when the sample size is small to moderate.⁸

Second, the sample sizes of both new models were conservatively estimated (140 and 143, respectively),^4,5 in accordance with the rule of thumb of at least 10 events per predictor variable (EPV).¹² The 10 EPV rule means that there should be at least 10 events for each predictor parameter considered for inclusion in the prediction model equation. However, this rule solely considers the number of variables in the model, overlooking factors like multicollinearity, interactions, and nonlinear relationships. Additionally, the guideline report also suggests that a larger sample size enhances the robustness of the model.¹² Consequently, further external validation is required to assess the performance of these two new models.⁸

Third, the selection of variables for modeling includes both data-driven and knowledge-driven approaches.¹³ Knowledge-driven variables are derived from prior literature, clinical expertise, and consultations with experts.¹³ Combining these data-driven and knowledge-driven approaches often contributes to developing more stable and useful models.¹³ Furthermore, there may be overlap between variables from these two approaches. The Intensive Care Unit Requirement Score (IRS)¹⁴ and the Tanta University Risk Model¹⁵ for intensive care requirement are models developed based on large sample sizes and have undergone external validation multiple times^16–18 (Table 1). The variables included in the IRS and the Tanta University Risk Model should be adequately considered in the new model. We recommend explaining the variable selection strategy as needed during model development studies to prevent overlooking important variables.

Fourth, The model for central nervous system xenobiotics reported only discrimination,⁵ while the model for antipsychotics reported both discrimination and calibration.⁴ In fact the current assessment model should be more comprehensive, including discrimination, calibration and decision curve analysis.⁹

In conclusion, several models have been developed concerning ICU admission for acute poisoning (Table 1)^4,5,14,15 Nevertheless, more comprehensive external validation of these models is required to ensure their applicability in clinical practice.^8–10

Author contributions

Qifang Shi: Writing Original draft.

Jinsong Zhang: Reviewing and Editing.

All authors read and approved the manuscript prior to submission.

Funding

Funding from the National Natural Science Foundation of China (82172184).

Conflict of interest statement: None.