Large language models for predicting postoperative complications: benefits, potential, and limitations of clinical use

Postoperative complications are common and clinically significant, occurring in up to 31.5% of patients in a prospective study and ranging from minor wound infections to death [1]. These complications can lead to prolonged hospital stays and treatment, resulting in social and economic burdens owing to the increased use of medical resources. Evaluating and predicting various postoperative complications require an approach that includes preoperative risk-based optimization, intraoperative protective strategies, and active postoperative management. Preoperative risk prediction models offer several practical benefits, such as the ability to guide modifiable risk reduction strategies that can be applied immediately before a surgical decision and the utilization of widely available and standardized clinical variables.

Several methods have been introduced to evaluate and predict postoperative complications. The American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator develops regression models that predict eight outcomes based on preoperative risk factors, utilizing reliable multicenter clinical data [2]. As this general-purpose calculator predicts most surgical risks, it can assist in clinical decision-making. The Surgical Risk Preoperative Assessment System also predicts and evaluates the risk of postoperative complications using concise common risk and outcome variables applicable to a broad patient population [3]. In contrast, the Surgical Outcome Risk Tool is simple but limited to mortality [4]. Additionally, the New Zealand Population Risk Model predicts mortality using a population-specific model [5]. However, most of these traditional models rely on linear relationships, making it difficult to reflect complex physiological interactions and impossible to account for temporal changes.

Recently, with an emphasis on personalized and precision medicine, the use of AI-based models for evaluating and predicting postoperative complications has increased. AI models overcome the shortcomings of traditional models, enabling higher discrimination in outcome prediction and offering scalability for outcomes and variable modalities. Nevertheless, existing tree- or neural network-based AI models have limitations in data representation and scalability when dealing with large amounts of unstructured data, high heterogeneity, the need for multimodal integration, or difficult feature definitions [6]. In these cases, large language models (LLMs) can be effective.

LLMs inherently possess universal linguistic and knowledge structures based on extensive prior learning. Thus, LLMs can serve as a bridge to align and integrate heterogeneous variable names and data formats generated across different domains into a common standard based on ‘clinical significance’ [7]. Furthermore, because LLMs enable basic data classification and feature identification in new environments without the need for separate retraining, the generalization barrier for models may be lower. LLMs are suitable for automatically extracting clinically important keywords from text-based clinical records and test results and converting them into structured data and can generate high-dimensional features usable in existing predictive models by integrating unstructured information. Moreover, through contextual comparisons with other patient records, LLMs can infer the implications of missing values at a medically valid level and by correcting typos or inaccurate terminology in unstructured data to fit the context, can provide support for existing models through more refined data input.

A study conducted by Lee et al. [8] explored methods for predicting postoperative outcomes, including 30-day mortality, intensive care unit admissions, and acute renal injury, using an LLM. Through external and internal validation, the model demonstrated comparability with existing tree-based models, suggesting that LLMs can serve as a general-purpose alternative to existing task-specific machine learning models and highlighting their potential for future clinical applications. However, LLMs currently face several challenges that prevent their use as independent predictive tools for preoperative risk assessment. First, they are subject to hallucinations, which can lead to clinical risks due to erroneous inferences. Second, they exhibit interpretability issues such as extreme “black boxes” owing to astronomical parameters. Finally, the probability predictions are unstable due to overconfidence or the absence of probability outputs. Resolving these issues must be prioritized for clinicians to use LLMs as decision-making aids.