Development of a machine learning model using structured and unstructured features for predicting surgery among patients with carpal tunnel syndrome: development and validation

Abstract

Carpal tunnel syndrome (CTS) is the most common entrapment neuropathy and often requires surgery when symptoms persist. However, identifying patients requiring surgery remains challenging. This study aimed to develop a predictive model for surgery in CTS patients by integrating standardized common data model (CDM)-based structured data with unstructured data from electromyography (EMG) reports and radiology reports. We retrospectively analyzed 3602 adults diagnosed with CTS at Seoul Metropolitan Boramae Medical Center, of whom 696 (19.3%) underwent surgery within 365 days. Structured CDM data were combined with unstructured text processed using topic modeling, and EMG severity grades were extracted. Predictive models were developed using LASSO logistic regression, gradient boosting machine (GBM), and random forest. Model performance was assessed using the area under the receiver operating characteristic curve (AUROC) and other metrics, and feature importance was evaluated using SHapley Additive exPlanations (SHAP) values. Models incorporating unstructured text and EMG grades outperformed those using structured data alone (AUROC 0.792 vs. 0.759, P < 0.001), with GBM achieving the best performance. SHAP analysis highlighted EMG grade and key text-derived topics as major predictors. These findings demonstrate that combining structured clinical data with unstructured text analysis can effectively predict surgical outcomes in CTS and support clinical decision-making through an integrated data-driven approach.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-026-46000-3.

Introduction

Carpal tunnel syndrome (CTS) is one of the most common compressive neuropathies caused by compression of the median nerve as it passes through the carpal tunnel in the wrist, and is reported to have a prevalence of 0.003–0.743% of the population worldwide^1–3. CTS occurs most frequently in middle-aged women aged 40–50 years⁴, and the risk increases in individuals engaged in repetitive wrist movements or those with systemic conditions such as diabetes mellitus, thyroid disorders, and rheumatoid arthritis⁵. Clinically, patients often experience pain, numbness, and tingling in the hand and fingers, predominantly affecting the thumb, index, middle, and part of the ring finger⁶. Weakness of grip strength and difficulties in fine motor skills are also frequently observed, leading to functional impairments in daily life⁷. Because these symptoms are common and often recurrent, many individuals encounter difficulties in daily activities and seek medical care. Over time, symptoms may persist or worsen, prompting consideration of surgical intervention. However, determining surgical candidacy is challenging, as it requires integrating diverse clinical information from multiple sources. Thus, there is a growing need for rapid and objective decision-support tools that can be applied in real-world clinical practice⁸. Electromyography (EMG) test reports and radiology examination reports provide key clinical information for diagnosing CTS and assessing its severity⁹. EMG quantitatively evaluates nerve compression by measuring nerve conduction velocity and response, while radiology reports assist in identifying anatomical abnormalities of the carpal tunnel and differentiating other lesions¹⁰. These reports often contain essential information about individual clinical presentations and test result interpretations that cannot be fully captured in structured data, making them crucial for diagnostic and therapeutic decision-making in CTS patients. In routine clinical practice, EMG and radiology examination reports are documented as narrative clinical text, often in semi-structured formats that combine standardized sections with free-text interpretations. Several studies have applied machine learning to CTS, including models predicting diagnosis, electrophysiologic severity, or postoperative prognosis^11,12. However, these studies almost exclusively rely on structured variables (e.g., demographic, electrophysiologic severity scores, anthropometry), with limited attempts to incorporate unstructured clinical narratives such as EMG or radiology reports for predicting surgical intervention. To address this gap, we integrate topic-modeled representations of EMG and radiology reports with structured OMOP-CDM variables to predict the need for surgical intervention in CTS patients.

Although such text data was unstructured and thus limited in its analysis, recent advances in natural language processing (NLP) technology have expanded the possibilities for systematically analyzing and quantifying large-scale clinical text¹³. In particular, topic modeling, an unsupervised learning technique, is well suited for capturing the complex characteristics of clinical narratives by extracting latent topics without predefined labels and allowing documents to belong to multiple topics simultaneously^14,15. By applying topic modeling techniques such as latent Dirichlet allocation (LDA), clinically meaningful content, such as recurring expressions and diagnostic findings from radiology and EMG reports, can be extracted and transformed into quantitative features suitable for predictive modeling^16,17. Several studies have applied NLP algorithms to text-based clinical data, such as nursing records¹⁸, to predict outcomes including hospital readmission¹⁹, post-discharge mortality²⁰, and intensive care unit readmission²¹. While some research has explored severity prediction²² and postoperative prognosis in CTS patients^23,24, studies focusing specifically on predicting surgical intervention remain relatively scarce. Recent data-driven and deep learning-based approaches have demonstrated the ability to model complex, heterogeneous biomedical data and provide clinically actionable predictions^25,26. These studies highlight the growing role of machine learning in biomedical decision support and motivate the application of similar data-driven frameworks to CTS surgical risk prediction.

Given the multifactorial and nonlinear pathways leading to CTS surgery, machine learning models provide a valuable framework for integrating diverse clinical features. In this study, algorithms the least absolute shrinkage and selection operator logistic regression (LASSO LR) was used for its interpretability; the tree-based random forest (RF) model was included to capture nonlinear interactions; and gradient boosting model (GBM) was applied for its robustness to high-dimensional, sparse topic features. These models have been widely validated in clinical prediction tasks and allow for detailed interpretability analyses using SHapley Additive exPlanations (SHAP), which is essential for clinical adoption²⁷.

The main research questions were: (1) whether incorporating unstructured text-derived features with structured clinical variables can enhance predictive performance for identifying CTS patients who require surgery, and (2) which these unstructured text-derived features with structured clinical variables most strongly influence surgical decision-making. To address these questions, this study proposes a machine learning based integrated predictive model that combines structured data with unstructured clinical narratives to enable more accurate and clinically interpretable predictions of surgical intervention in CTS patients.

Materials and methods

Data source and cohort description

This study utilized data from the Observational Medical Outcomes Partnership (OMOP) CDM established at Seoul Metropolitan Government–Seoul National University Boramae Medical Center (SMG-SNU BMC). The OMOP-CDM standardizes various Electronic Health Record (EHR) data into a unified structure and terminology, thereby providing a common analytic framework that facilitates multicenter analyses^28,29. The clinical data include demographic information, diagnoses, drugs, and procedures.

This study retrospectively evaluated patients diagnosed with CTS at Seoul Metropolitan Boramae Medical Center between January 1, 2010, and December 31, 2022. The inclusion criteria were as follows: (1) age ≥ 18 years; (2) a confirmed diagnosis of CTS (ICD-10 code: G56.0); and (3) completion of both an EMG examination and at least one imaging study prior to the index date. Patients without an EMG report or an imaging report were excluded. If a patient had multiple visits or multiple CTS-related encounters, all records were consolidated at the patient level. The index date was defined as the most recent date of either the EMG test or imaging examination. (Fig. 1). Imaging modalities covered computed tomography (CT), magnetic resonance imaging (MRI), magnetic resonance angiography (MRA), contrast-enhanced MRI, and musculoskeletal ultrasound, spanning anatomical regions such as the cervical spine, wrist, elbow, soft tissues, and peripheral arteries. The outcome was defined as the presence of CTS-related surgical procedures within 365 days after the index date. Relevant procedures in the OMOP-CDM were identified using standardized SNOMED concepts: 4066890 (Endoscopic release of transverse carpal ligament), 4166080 (Decompression of median nerve). These codes comprehensively represent surgical decompression for CTS and were used to define the outcome cohort.

Fig. 1.

Visualization of the prognostic prediction problem.

Data processing

In this study, both structured and unstructured data were utilized to develop the predictive model. The structured data were extracted from standardized OMOP-CDM tables and included patient demographics (sex, visit month, and age at visit), condition (clinical diagnoses), drug (medication), procedure (specific medical interventions performed), measurement (clinical observation records or lab test results obtained), hospital visit types, and visit frequency. Categorical variables were encoded using one-hot encoding according to the default specifications of the FeatureExtraction R package³⁰. All features, except for demographic information and the clinical index score (Charlson Comorbidity Index), were extracted based on this index date. Diagnoses, medications, procedures, measurements, and clinical notes recorded on or after the index date were excluded from feature construction, and only data documented prior to the index date were used as predictors. In this study, missing values were implicitly imputed as 0, following the default framework approach that assumes unrecorded conditions or drugs indicate absence rather than missingness³¹.

The unstructured data consisted of free-text extracted from radiology reports and EMG reports (Fig. 2). Radiology reports, written by board-certified radiologists, contained findings and impressions, which were concatenated into a single document per patient for NLP analysis. The EMG reports provide physiological diagnostic information relevant to CTS, and this study focused on the results section of these reports for analysis. In particular, the EMG grade variable extracted from EMG reports was used as a key input representing the neurophysiological severity. This variable was based on the Bland grading scale, which classifies severity into seven levels ranging from 0 (normal) to 6 (extremely severe)³². The Bland grading scale is a standardized assessment tool used to quantify the diagnostic certainty and severity of CTS based on the results of nerve conduction studies (NCS), and detailed definitions for each level are provided in Supplementary Table S1. When EMG grade values were recorded for both sides (left and right), the higher value was selected to more accurately reflect the patient’s overall severity. Depending on the experimental condition, the EMG grade variable was either extracted from the text and used as an independent categorical feature or incorporated into the topic modeling process after normalizing severity-related expressions found in the EMG report. In the former case, to avoid redundancy, severity-related expressions were removed from the input text for topic modeling, and the EMG grade variable was entered as a standalone feature. In the latter case, the EMG grade variable was not used separately; instead, the entire text including the normalized severity expressions was used for topic modeling. This approach ensured that the model captured both latent textual patterns and standardized severity information without redundancy.

Fig. 2.

Examples of text reports used in the study: (Left) example of CT imaging report, (Right) example of electromyography (EMG) examination report.

The unstructured data consisted of mixed Korean and English text. All Korean text was translated into English using an automated translation API–based method. A clinician reviewed a subset of the translated reports to ensure that key clinical terminology and EMG-related descriptors were preserved. Prior to analysis, standard preprocessing steps were applied to the unstructured text, including the removal of punctuation, stop words, personal names, and meaningless repetitive words, to ensure that the model focused on clinically relevant content. The preprocessed radiology reports and EMG reports were then analyzed using LDA, a topic modeling technique. LDA is a generative probabilistic model proposed by Blei et al.³³, which assumes that each document is composed of multiple latent topics, with each topic represented as a distribution over words^34,35. For each patient, the presence or absence of each extracted topic was encoded as a fixed-length binary vector, which was then concatenated with the structured clinical variables to construct the final feature set used for model training.

Experimental design

In this study, we designed and compared three experimental conditions to predict whether patients with CTS would undergo surgery (Fig. 3). For each experimental condition, the LDA topic model was trained independently using its corresponding preprocessed text corpus. All three models aimed to predict the same outcome surgical treatment but differed in the composition of input variables, as follows:

1. CDM-based clinical covariate: Only structured clinical variables from the OMOP-CDM were used.

2. CDM-based clinical covariate + radiology/EMG report text: Incorporating both structured clinical variables and unstructured text data, including preprocessed radiology reports and EMG reports. Topic modeling was performed using LDA on the combined text data. Expressions related to severity were normalized and retained in the text, but the corresponding EMG grade values were not used as separate variables.

3. CDM-based clinical covariate + radiology/EMG report text + EMG grade: LDA-based topic modeling was applied to the full radiology and EMG reports; however, severity-related expressions (Bland grading scale) were removed from the text prior to topic modeling. Instead, the EMG grade value based on the Bland grading scale was extracted from the EMG report and included as a separate variable.

Fig. 3.

Overview of the experimental design and model development.

Model development

Three machine learning algorithms LASSO LR, RF, and GBM were applied to develop the prediction models. LASSO LR is a probabilistic model that incorporates a regularization technique for variable selection³⁶, shrinking some regression coefficients to zero and thereby performing feature selection. This not only facilitates the identification of important features but also reduces model complexity, helping to prevent overfitting³⁷. Random Forest constructs multiple decision trees by randomly sampling the training data and aggregates the outputs of individual trees through majority voting³⁸. The randomness introduced during feature selection and the bagging process contributes to reducing overfitting³⁹. GBM is an ensemble model based on the boosting technique, where weak learners are trained sequentially. It improves performance by adjusting the weights of misclassified data points during the learning process⁴⁰. Details of the final model hyperparameters are listed in Supplementary Table S2. The entire dataset was split into a training set (75%) and a test set (25%), and threefold cross-validation was performed within the training set to evaluate model performance.

Evaluation measure

The classifier’s performance was evaluated on multiple fronts. First, the overall performance of the classification model was measured using the area under the receiver operating characteristic (AUROC) curve, and differences between models were assessed using DeLong’s test⁴¹. The resulting P values were adjusted for multiple comparisons using the Benjamini–Hochberg procedure, with a significance threshold of P < 0.05. In addition, Accuracy, F1 Score, and area under the precision-recall curve (AUPRC) were used to further assess and compare the predictive capabilities of each algorithm. The AUPRC was included specifically to account for the imbalance in outcome prevalence. Moreover, the operating probability threshold used to compute accuracy and F1-score was determined on the validation set by maximizing the Youden index⁴², and this fixed threshold was subsequently applied to the test set. To interpret the key variables contributing to the model’s predictions, SHAP analysis was conducted. SHAP analysis was based on the GBM model, which demonstrated the highest predictive performance. The relative importance of each variable was calculated and ranked based on the mean absolute SHAP value. This approach aimed to identify the major determinants of the model’s predictions and enhance clinical interpretability. Although SHAP can in principle be applied to all three models, we focused the SHAP analysis on the GBM model because it achieved the highest predictive performance and was therefore selected as the primary model for clinical interpretation. This approach aimed to identify the major determinants of the model’s predictions and enhance clinical interpretability. The methods in the present study were implemented in Python version 3.9.7 (Python Software Foundation, Wilmington, Del, USA), with scikit-learn (version 1.1.2).

Result

General characteristics of the study population

A total of 3602 patients diagnosed with CTS were included in the target population, of whom 696 (19.3%) underwent surgery within 365 days of the later date between their EMG reports and radiology reports. Table 1 presents the demographic characteristics of the patients by surgical status. Among those who underwent surgery, 596 (85.6%) were female, and the mean age of the overall cohort was 58.6 years. The mean age of patients in the surgery group was 58.0 years, compared to 60.8 years in the non-surgery group, showing a statistically significant difference between groups (P < 0.001).

Table 1.

Baseline patient characteristics.

Characteristics	All (3602)	No outcome (N = 2906)	Outcome (N = 696)	P value
Sex (Female), N (%)	2838 (78.8)	2242 (77.2)	596 (85.6)	0.22
Age group, N (%)
18–29	68 (1.9)	61 (2.1)	7 (1.0)	0.4598
30–39	199 (5.5)	182 (6.3)	17 (2.4)	0.6932
40–49	419 (11.6)	358 (12.3)	61 (8.8)	0.1107
50–59	1209 (33.5)	953 (32.8)	256 (36.8)	0.1001
60–69	928 (25.8)	752 (25.8)	176 (25.3)	0.9857
70–79	602 (16.7)	470 (16.2)	132 (18.9)	0.0831
Over 80	177 (4.9)	130 (4.5)	47 (6.8)	0.8071
General comorbidities
Diabetes mellitus	164 (4.6)	130 (4.5)	34 (5.0)	0.03
Obesity	7 (0.2)	6 (0.2)	1 (0.1)	0.42
Osteoarthritis	441 (12.2)	339 (11.7)	102 (14.7)	0.09
Rheumatoid arthritis	21 (0.6)	16 (0.5)	5 (0.7)	0.23
Depressive disorder	84 (2.3)	66 (2.3)	18 (2.6)	0.12
Medication use
Antidepressants	1001 (27.8)	797 (27.4)	204 (29.3)	0.04
Opioids	968 (26.9)	782 (26.9)	186 (26.7)	< 0.001
Antiinflammatory and antirheumatic	1555 (43.2)	1,252 (43.1)	303 (43.5)	< 0.001

Model performance

Table 2 summarizes the detailed classification performance according to each model and input variable combination. Across all algorithms, improvements in predictive performance were observed when unstructured text information and the EMG grade variable were incorporated, compared to the baseline model that used only structured CDM-based clinical covariates. For the LASSO LR model, the AUROC increased from 0.746 (95% CI 0.685–0.801) in the baseline model to 0.772 (95% CI 0.723–0.821) in the final model that included radiology report text, EMG report, and the EMG grade variable. Similarly, the AUROC for the RF model increased from 0.750 (95% CI 0.690–0.809) to 0.785 (95% CI 0.727–0.829), and for the GBM, from 0.759 (95% CI 0.711–0.815) to 0.792 (95% CI 0.732–0.830). The calibration plot is shown in Supplementary Fig. S1.

Table 2.

Predictive performance comparison according to input features.

	CDM-based clinical covariate	CDM-based clinical covariate + radiology/EMG report text model	CDM-based clinical covariate + radiology/EMG report text + EMG grade model
LASSO LR
AUROC	0.746 [0.685–0.801]	0.768 [0.715–0.814]	0.772 [0.723–0.821]
AUPRC	0.432 [0.341–0.532]	0.465 [0.357–0.498]	0.471 [0.366–0.499]
Accuracy	0.782	0.801	0.808
F1-score	0.495	0.523	0.531
RF
AUROC	0.750 [0.690–0.809]	0.779 [0.732–0.821]	0.785 [0.727–0.829]
AUPRC	0.441 [0.346–0.535]	0.478 [0.364–0.491]	0.485 [0.387–0.520]
Accuracy	0.786	0.809	0.815
F1-score	0.501	0.536	0.545
GBM
AUROC	0.759 [0.711–0.815]	0.789 [0.738–0.826]	0.792 [0.732–0.830]
AUPRC	0.459 [0.363–0.541]	0.492 [0.384–0.503]	0.499 [0.382–0.531]
Accuracy	0.794	0.818	0.823
F1-score	0.519	0.552	0.561

LASSO LR least absolute shrinkage and selection operator logistic regression, RF random forest, GBM gradient boosting machine, CDM common data model, EMG electromyography.

Topic modeling was performed using LDA on the EMG report and radiology report. The results presented are derived from the GBM model under the CDM-based clinical covariate + radiology/EMG report text + EMG grade experimental setting, which demonstrated the highest predictive performance among the evaluated models. In this setting, clinically important grade-related expressions were first removed from the text and included separately as a categorical variable in the prediction model. After comparing topic numbers set to 3, 5, and 10, the configuration with five topics yielded the highest performance and was therefore used in the final model. The number of topics was set to 3, 5, and 10 for comparison, following approaches commonly used in previous LDA-based text analyses^43,44. To determine the optimal number of topics, we compared these configurations using the C_v coherence metric and cross-validated AUROC on the training set (Supplementary Table S3). The 5-topic model achieved the highest coherence (0.56) compared with 0.48 for 3 and 0.54 for 10. In addition, the 5-topic representation yielded the highest cross-validated AUROC. Therefore, 5 was selected for all subsequent analyses. The unstructured data were thus represented by five topics, each labeled by a clinician based on the clinical relevance of the representative keywords. Table 3 presents the topics extracted from the CDM-based clinical covariate + radiology/EMG report text + EMG grade. Topics 1 and 5 captured electrophysiological phenotypes, including sensory dysfunction and severe conduction abnormalities, whereas Topic 2 reflected imaging-based CTS features such as median nerve swelling on ultrasound, and Topic 3 represented alternative neuropathic conditions such as ulnar neuropathy. The detailed results of the CDM-based clinical covariate + radiology/EMG report text model are presented in Supplementary Table S4, and the SHAP analysis of feature importance and directionality is shown in Supplementary Fig. S2.

Table 3.

Topic variables included in CDM-based clinical covariate + radiology/EMG report text + EMG grade.

No.	Topic	Keywords
Topic 1	Electrophysiological evidence of sensory nerve dysfunction	Sensory, CMAP, wave, terminal, latency, decreased, prolonged
Topic 2	Descriptions related to CTS Diagnosis	CTS, wrist, nerve, swelling, median, level, axonotmesis, soft
Topic 3	Non-median neuropathy and alternative diagnoses	Ulnar, neuropathy, elbow, axonal, lesion, active, subchondral, styloid, lunate
Topic 4	Degenerative or bony pathology	Bony, significant, abnormality, diffuse, facet, arthrosis, severe
Topic 5	Conduction delays and pathological electrophysiological findings	Stenosis, latency, amplitude, degenerative, demyelination, abnormalities, findings

CDM common data model, EMG electromyography, CMAP compound motor action potential, CTS carpal tunnel syndrome.

Feature importance

SHAP analysis was conducted to interpret the contribution of variables in the surgery prediction model (Fig. 4). Through SHAP analysis, the relative importance of each predictor and its directional effect on the likelihood of surgery were evaluated. The EMG grade variable, considered a key clinical indicator, emerged as the most important predictor, exhibiting positive SHAP values that increased with higher EMG grade scores, demonstrating consistent explanatory power within the model. Additionally, among the LDA topic variables, Topic 2, Topic 5, and Topic 1, as well as medication-related variables including tramadol and morphine prescriptions, patient age (ageyear), inpatient visit status, and prescriptions of ephedrine and dexamethasone, were identified as major contributors to the prediction.

Fig. 4.

SHAP value analysis result derived by integrated GBM model using CDM-based clinical covariate + radiology/EMG report text + EMG grade model.

For interpretable model implementation, individual patient data were also examined. For example, Fig. 5 (A) presents the SHAP force plot for a high-risk patient who actually underwent surgery. In this case, the patient had a high EMG grade of 3, and the LDA topics derived from radiology reports and EMG texts (Topic 2, Topic 5, and Topic 1) contributed significantly to the surgery prediction. All these variables increased the predicted probability of surgery. In contrast, Fig. 5B presents the SHAP results for a low-risk patient who did not undergo surgery. In this case, although the value of Topic 1 was 1, both the EMG grade and the values of Topic 5 and Topic 2 were 0, indicating the absence of key variables that increase the likelihood of surgery. Taken together, these characteristics led the model to predict a low probability of surgical intervention.

Fig. 5.

Local SHAP interpretations of the GBM model. (A) shows the force plot for a high-risk case, and (B) shows the force plot for a low-risk case. Red indicates a positive contribution of the feature to the predicted outcome, while blue indicates a negative contribution.

Discussion

In this study, we developed a predictive model for surgical intervention within 365 days in patients with CTS by integratively utilizing structured data based on the OMOP-CDM and unstructured text data extracted from radiology reports and EMG reports. The timing of surgery in CTS varies considerably depending on symptom severity, patient preference, and the duration of conservative management, and many patients transition to surgery only after prolonged non-surgical treatment. Prior studies have similarly shown that a substantial proportion of patients undergo surgical intervention after symptoms persist for more than 6–12 months, indicating that delayed surgical conversion is common⁴⁵. Based on this evidence, we selected a 365-day prediction window to capture both early and delayed surgeries.

Comparative performance analysis across various feature combinations demonstrated consistent improvements in predictive performance in the integrated model, which included radiology and EMG reports as well as the EMG grade variable, compared to the baseline model using only structured clinical variables.

The GBM model demonstrated the best predictive performance, achieving an AUROC of 0.792 when incorporating unstructured text and EMG grade, compared to 0.759 using only structured CDM variables. This improvement highlights that the inclusion of radiology and EMG report information, along with the EMG severity grade, enhances the model’s ability to capture clinically relevant factors influencing surgical decisions. In addition, improvements in accuracy, AUPRC, and F1-score indicate that the integrated model more effectively balances sensitivity and specificity, supporting its potential utility as an explainable decision-support tool in real-world clinical settings.

To extract meaningful information from unstructured text data, LDA-based topic modeling was applied to radiology reports and EMG reports. This process segregated latent clinical expressions associated with surgery into distinct topics, and the topic distribution of each document was utilized as predictive features. Clinician reviewed the keyword distributions to assign clinically relevant interpretations to each topic. Topic 2 (Descriptions related to CTS diagnosis), Topic 1 (Electrophysiological findings), and Topic 5 (Conduction delays and pathological electrophysiological findings) were found to reflect core information pertinent to actual surgical decision-making. In addition, among the structured variables, prescriptions of tramadol and morphine, patient age, inpatient visit status, and prescriptions of ephedrine and dexamethasone were identified as major contributors. The prominence of tramadol and morphine prescriptions may reflect greater pain severity or inadequate symptom control with conservative treatment. Our findings therefore suggest that prescriptions of strong analgesics may serve as proxy variables that indirectly capture patients with persistent or severe pain who are more likely to progress to surgical treatment. Therefore, tramadol and morphine prescriptions should be interpreted as indicators of symptom burden rather than independent drivers of surgical decision-making.

However, mere inclusion of unstructured text may lead to limited improvements, as topic modeling-based features can remain somewhat generic. Therefore, this study separated the EMG severity information, which is a critical clinical indicator for surgical decisions, from the text and included it as an independent EMG grade variable in the predictive model. This refined EMG grade variable enhanced the model’s sensitivity and interpretability, illustrating the effectiveness of an integrated predictive approach combining clinical knowledge derived from unstructured data with structured clinical indicators. Through this approach, the study validates the effectiveness of a complementary integration of structured clinical indicators and unstructured text. Future extensions of the OMOP-CDM-based data model should consider incorporating such unstructured text and severity indicators.

Previous studies have developed various machine learning models to predict clinical outcomes and severity in CTS patients. Park et al. developed a multiclass classification model using electrophysiologically confirmed CTS data from 1037 cases, classifying severity into mild, moderate, and severe, with Extreme Gradient Boosting (XGB) and RF models achieving accuracies of 76.6% and 75.6%, respectively⁴⁶. Yetiş et al. constructed models predicting CTS diagnosis based on anthropometric hand measurements, grip strength, and pinch strength, reporting strong performances for RF and XGBoost models at 89.5% and 86.8%, respectively⁴⁷. Recently, models have been developed not only for CTS diagnosis but also for predicting post-surgical prognosis, symptom changes, and treatment response, expanding the potential utility of clinical decision support tools^48–50. However, these previous studies primarily rely on structured clinical data (e.g., age, grip strength, anthropometry, questionnaire scores), with limited reports integrating unstructured text data for predicting surgical intervention.

This study has several limitations. First, since the model was developed and validated using data from a single institution, its generalizability is inherently limited. In particular, EMG test reports are known to exhibit institution-specific reporting styles, terminology, and formatting, which may lead to distribution shifts in the derived topic features when applied to other sites. Although the OMOP-CDM standardizes structured variables, it does not eliminate heterogeneity in unstructured EMG narratives. Therefore, the lack of external, multi-center validation highlights the need for further validation across institutions with different EMG protocols, reporting conventions, and surgical thresholds. Second, due to the inherent nature of unstructured data, radiology reports and EMG reports exhibit heterogeneous sentence structures and varying quality, potentially causing information loss during preprocessing and interpretation. In addition, unrecorded findings in EMG reports may not necessarily indicate true absence but may reflect reporting practices or clinical judgment. However, in CTS, electrophysiologic severity is commonly summarized using the Bland grading scale, which integrates both sensory and motor abnormalities and is widely used to guide surgical decision-making. Findings such as spontaneous muscle activity typically appear only in advanced disease (Bland grades 5–6) and are more relevant for postoperative prognosis than for the decision to proceed with surgery. Therefore, while individual EMG descriptors may be variably documented, the key severity information relevant to surgical decision-making is largely captured by the Bland grade used in this study. Third, the model was evaluated using a random train–test split rather than a temporal validation. Because the cohort spans more than a decade (2010–2022), changes in clinical practice, EMG reporting styles, and surgical thresholds over time may have introduced temporal drift that was not explicitly assessed in this study. As a result, the reported performance may not fully reflect real-world deployment in later calendar periods. Future studies will therefore evaluate model performance across temporally separated cohorts to assess robustness to such drift, because the original text was written in Korean, automated translation into English was required before topic modeling. Although standard preprocessing procedures were applied, automated translation may still introduce subtle distortions in terminology or context that could influence topic distributions and should be considered when interpreting the results. Although a subset of translated reports was manually reviewed and no systematic loss of key clinical terminology was observed, subtle translation-related biases cannot be completely excluded. Fifth, this study did not utilize modern embedding-based methods such as ClinicalBERT or BERTopic, which may have limited the predictive performance of the model. Additionally, the normalization of EMG grade information and topic modeling were performed using rule-based approaches, which require further refinement for automation and real-time application in clinical practice. Future studies will further extend this framework by evaluating shorter prediction horizons (e.g., 30, 90, and 180 days), which would enable time-specific and clinically actionable surgical decision support. Despite these limitations, this study is significant in demonstrating the effective integration of unstructured text data with structured CDM variables to build an explainable and quantitative predictive model for surgical decision-making.

The EMG grade was identified as the most important variable, consistent with existing medical evidence emphasizing the critical role of severity in surgical decision-making. Additionally, unstructured text-based variables including topics 2, 5, and 1, as well as structured CDM variables such as prescriptions of tramadol and morphine, patient age, and inpatient status, were also key contributors. These findings illustrate that structured and unstructured data complement each other, enabling more precise and explainable predictions.

Conclusion

In this study, we developed predictive models to predict surgery in CTS patients by integrating standardized CDM-based structured data with unstructured information extracted from radiology and EMG reports. By applying LDA-based topic modeling and incorporating EMG severity grades, the model achieved improved predictive performance and highlighted the clinical value of unstructured data. These findings highlight the potential of integrated approaches for advancing prediction-based clinical decision support systems that quantitatively utilize unstructured data. Moreover, this study serves as a meaningful example demonstrating the potential for expanding CDM-based research through the incorporation of text data.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

Ji Hyun Lee had full access to all the data in the study and took responsibility for the integrity of the data and the accuracy of the analysis. Ji Hyun Lee contributed to data analysis, literature review and manuscript writing. Borim Ryu and Yohan Lee contributed to the study design, data collection, writing of the manuscript, and supervision of the study.

Data availability

Because this study utilized a distributed research network, the data are kept within each institution and are not publicly available. However, the source code for cohort and covariates generation is available upon request to corresponding author.

Declarations

Competing interests

The authors declare no competing interests.

Ethics approval

The study protocol was approved by the Institutional Review Board of the study site (IRB No. 40-2024-17), and the study adhered to the principles of the Declaration of Helsinki. The requirements for informed consent from the study participants were waived by the IRB Committee of the SMG-SNU BMC owing to the retrospective study design.