Machine learning-assisted cancer diagnosis in patients with paraneoplastic autoantibodies

Abstract

Purpose

Paraneoplastic syndromes (PNS) are a group of rare disorders triggered by an immune response to malignancy, characterized by diverse neurological, muscular, and systemic symptoms. This study aims to leverage machine learning to develop a predictive model for cancer diagnosis in patients with paraneoplastic autoantibodies.

Methods

Demographic data included age and sex, and presenting symptoms were recorded. Laboratory data comprised serum or cerebrospinal fluid (CSF) paraneoplastic autoantibody panels. The study included participants who tested positive for at least one autoantibody. Naive Bayes model was used to predict cancer presence. Model performance was evaluated using sensitivity, specificity, likelihood ratios, predictive values, AUC-ROC, Brier score, and overall accuracy. Feature importance was assessed using SHapley Additive exPlanations (SHAP) values. A graphical user interface (GUI)-based application was developed to facilitate model use.

Results

The study included 116 participants, with an average age of 57.1 years and a higher proportion of females (53.4%). The most common presenting symptom was ‘‘Motor’’ (40.5%), followed by ‘‘Cognitive’’ (17.2%) and ‘‘Bulbar’’ (15.5%) symptoms. Cancer was present in 23 participants (19.8%). The Naive Bayes model demonstrated high performance with a sensitivity of 85.71% and specificity of 100.00%. The AUC-ROC was 0.9795, indicating excellent diagnostic capability. Age and the presence or absence of specific autoantibodies were significant predictors of cancer.

Conclusion

Machine learning models, such as the Naive Bayes classifier developed in this study, can accurately stratify cancer risk in patients with positive paraneoplastic autoantibodies.

Introduction

Paraneoplastic syndromes (PNS) are a group of disorders triggered by an immune response to a malignancy. These syndromes manifest in various ways, affecting multiple organ systems, and are often mediated by autoantibodies that target intracellular and extracellular antigens in different tissues [1, 2]. Paraneoplastic antibodies are frequently detected before the underlying cancer is identified. Therefore, their early detection in patients can facilitate earlier diagnosis of the associated malignancy [3]. The complexity of PNS necessitates a multidisciplinary approach to diagnosis and management, integrating clinical assessment with advanced laboratory techniques.

PNS are often characterized by a wide array of neurological, muscular, and systemic symptoms. Neurological manifestations include motor dysfunction, cognitive impairments, sensory disturbances, autonomic instability, and seizures. Common symptoms also extend to bulbar dysfunction, visual disturbances, and sleep abnormalities [4, 5]. These symptoms can significantly impact patients' quality of life and are indicative of the underlying autoimmune process.

The most common malignancies associated with PNS are lung cancer, breast cancer, hematological malignancies, medullary thyroid cancer, gynecological malignancies, and prostate cancer [6, 7].

The presence of specific autoantibodies is a hallmark of PNS, aiding in diagnosis and understanding disease mechanisms. These autoantibodies are detected through serum or cerebrospinal fluid (CSF) analyses and play a critical role in confirming a PNS diagnosis [8]. Their presence not only signifies an immune response to cancer but also guides therapeutic decisions.

Machine learning (ML) techniques enable the integration of diverse data types and the identification of subtle associations that might be overlooked in traditional statistical analyses. For instance, ML can help in stratifying patients based on their risk of developing cancer or other severe complications, thereby facilitating personalized treatment strategies [9].

The Naive Bayesian model is a probabilistic classifier based on Bayes' Theorem, widely used in medical research for its simplicity and efficiency in handling large datasets. It assumes that features are conditionally independent given the class, which, although a strong assumption, often yields robust performance in medical diagnoses where complex interactions may not be explicitly modeled. This model is particularly useful in predicting disease outcomes, classifying patients based on symptoms, or identifying high-risk groups by analyzing various clinical parameters. Its ability to handle noisy data and provide probabilistic predictions makes it a valuable tool in evidence-based medicine, offering clinicians a clear probabilistic understanding of the likelihood of different diagnoses. Despite its simplicity, the Naive Bayesian model often performs comparably to more complex models, especially in well-defined, structured datasets [10, 11].

This study aims to explore the potential of integrating machine learning with clinical and laboratory data to enhance early cancer detection in patients with paraneoplastic autoantibodies. Additionally, it seeks to identify the most influential clinical and biomarker features contributing to cancer diagnosis, providing insights into the interplay between autoantibodies and malignancy while advancing the application of machine learning in rare disease diagnostics. By integrating demographic data, clinical symptoms, and laboratory findings, the study seeks to improve diagnostic precision and tailor management plans for individuals affected by PNS.

Methods and materials

Data

This retrospective study used data from an article ‘Data on the utilization of paraneoplastic syndrome autoantibody testing at an academic medical center’ [12]. The overall retrospective analysis period was December 1, 2008, through November 30, 2018. All data was obtained from patient data in the electronic medical record from the University of Iowa Hospitals and Clinics (Iowa City, Iowa, United States).

The demographic data included the age and sex of the participants. Age was recorded in years, and any age values designated as ‘‘ > 89’’ were mapped to 89 for analysis consistency. Participants were assessed for a variety of presenting symptoms. The symptoms recorded included ‘‘Motor,’’ ‘‘Cognitive,’’ ‘‘Bulbar,’’ ‘‘Sensory,’’ ‘‘Autonomic,’’ ‘‘Visual,’’ ‘‘Seizures,’’ and ‘‘Sleep’’ symptoms.

The laboratory testing data included panels of serum paraneoplastic autoantibody panels offered by two reference laboratories in the United States (Mayo Clinic Laboratories, Rochester, MN; ARUP Laboratories, Salt Lake City, UT) and a cerebrospinal fluid (CSF) paraneoplastic autoantibody panel by Mayo Clinic Laboratories. ARUP Laboratories additionally offers stand-alone assays for N-methyl-D-aspartate (NMDA) receptor autoantibodies in either CSF or serum as well as separate assays for the P/Q-type voltage-gated calcium channel (VGCC) autoantibodies or voltage-gated potassium channel (VGKC) autoantibodies in serum.

The study investigated the presence of various antibodies and markers associated with neurological conditions. Samples were collected from participants and tested for the presence of the following: N-type calcium channel antibodies, Anti-neuronal nuclear antibodies (ANNA), Collapsin response mediator protein (CRMP) antibodies, Purkinje cell cytoplasmic antibodies, Striated muscle antibodies, Acetylcholine receptor (AChR) antibodies, P/Q-type calcium channel antibodies, VGKC antibodies, NMDA receptor antibodies, Glutamic acid decarboxylase 65 (GAD65) antibodies, and ANNA-1 antibodies. The patient was either positive or negative for each autoantibody.

Only patients who tested positive for at least one autoantibody were included in this study. The antibodies were either present in serum or CSF. All patients were symptomatic, but the presence of an autoantibody was the required inclusion criterion not a specific paraneoplastic syndrome.

Statistics

The average age, standard deviation (SD), median age, interquartile range (IQR), and age range were calculated to summarize the age distribution of the participants. The frequency and percentage of each gender were also computed. The distribution of presenting symptoms, cancer, and the prevalence of various antibodies were summarized using frequency counts and percentages.

Data analysis was done using SciPy [13] library and machine learning models were developed with utilization of Scikit-learn [14] library. All codes were written and implemented in python programming language. The large language model GPT-4o was used for grammar check and language refinements.

Machine learning

After one-hot encoding [15] of categorical data and scaling of numerical data, the dataset was split into training and testing sets with a ratio of 70:30. The data is split to objectively evaluate the performance of the developed model. Seventy percent of the data is used for training, allowing the model to learn hidden patterns. The remaining 30 percent, which the model has never seen before, is reserved for testing its performance. This approach reduces the likelihood of the model merely memorizing patterns, ensuring it generalizes effectively to new data. A random state of 42 was used to ensure the reproducibility of the results.

A Naive Bayes model was used to predict the presence of cancer among the participants. Naive Bayes is a probabilistic classifier based on Bayes' theorem with the assumption of feature independence. In a medical context, the Naive Bayes model can be used to predict an outcome based on patient data. For example, when diagnosing the flu, probabilities are assigned to different symptoms, such as fever and cough, based on how common they are in flu cases. The model assumes that each symptom occurs independently of the others, even though they may be related. Bayes’ Theorem is then applied to combine the probabilities of each symptom occurring given the disease, and the likelihood of the patient having the flu is calculated. If the calculated probability is higher than for other diseases, a flu diagnosis is made. This method relies on past data about the frequency of symptoms in flu cases to make a prediction. This simplicity helps in preventing overfitting, as there are fewer parameters to estimate compared to more complex models [11, 16].

A grid search with fivefold cross-validation was used to optimize the model's hyperparameters. This approach systematically evaluates a range of parameter values to identify the best combination that minimizes overfitting and maximizes performance. In this study, cross-validation involves splitting the data into five subsets, training the model on four subsets, and validating it on the remaining subset, rotating this process five times to ensure robust performance metrics.

To evaluate the diagnostic model's performance, several metrics were calculated, including sensitivity, specificity, positive and negative likelihood ratios (LR + and LR-), positive predictive value (PPV), negative predictive value (NPV), the area under the receiver operating characteristic curve (AUC-ROC), Brier score, and overall accuracy. These metrics were reported with their corresponding 95% confidence intervals (CI).

Feature importance was assessed using SHapley Additive exPlanations (SHAP) values to identify the most influential variables in predicting cancer. Shapley values measure how much each feature contributes to a model's prediction. They come from game theory, where features are seen as players in a game. To find a feature’s importance, the model’s prediction is calculated with and without that feature, across all possible combinations of features. The Shapley value for a feature is the average contribution it makes to the prediction in these combinations. Features with higher Shapley values have a bigger impact on the model's prediction, while lower values indicate less importance. This method provides a fair way to understand feature importance [17].

A graphical user interface (GUI)-based application was designed to facilitate model utilization. This application is available via contact with the corresponding author.

Due to the small size of the dataset used in this study, overfitting was a concern. Overfitting occurs when a model memorizes patterns in the training data rather than learning generalizable rules for future application [18]. To assess overfitting and for sensitivity analysis, a jitter test was conducted. In this test, random noise from a normal distribution was added to the test data to artificially degrade model performance. A significant drop in performance would indicate overfitting. Conversely, resilience against noise suggests a robust model [19]. 1 to 20 percent noise was utilized for the test.

Results

Demographics

The study included a total of 116 participants. The average age of the participants was 57.1 years with a standard deviation of 20.87 years. The median age was 63 years with an interquartile range (IQR) of 31.5 years, indicating a wide spread of ages. The age range of the participants was from 1.1 to 89 years. The gender distribution among the participants showed that a higher proportion were females, with 62 out of 116 participants (53.4%) being female, compared to 54 males (46.6%).

In terms of presenting symptoms, the most common symptom reported was ‘‘Motor,’’ which was observed in 47 participants, accounting for 40.5% of the total. Other presenting symptoms included ‘‘Cognitive,’’ which was reported by 20 participants (17.2%), and ‘‘Bulbar’’ symptoms, observed in 18 participants (15.5%). Additionally, ‘‘Sensory’’ symptoms were reported by 10 participants (8.6%), while ‘‘Autonomic’’ symptoms were noted in 9 participants (7.8%). ‘‘Visual’’ symptoms were observed in 6 participants (5.2%), and "Seizures" were reported by 4 participants (3.4%). Finally, the “Sleep" symptoms were noted in 2 participants (1.7%).

Autoantibodies and cancer prevalence

The study also investigated the presence of various autoantibodies and markers associated with neurological conditions. Cancer was reported in 23 participants, representing 19.8% of the study population. The most common cancer detected was small cell lung cancer (55%). The other cancers detected were prostate (15%), lung adenocarcinoma (5%), adrenal (5%), ovarian (5%), and others (15%). The presence of N-type calcium channel antibodies was found in 4 participants (3.4%), while anti-neuronal nuclear antibodies were detected in 5 participants (4.3%). Collapsin response mediator protein (CRMP) antibodies were also present in 2 participants (1.7%). Purkinje cell cytoplasmic antibodies were found in 3 participants (2.6%). A significant proportion of participants tested positive for striated muscle antibodies, with 38 individuals (32.8%) showing positivity. Acetylcholine receptor (AChR) antibodies were present in 16 participants, accounting for 13.8% of the total. P/Q-type calcium channel antibodies were detected in 21 participants (18.1%), while voltage-gated potassium channel antibodies were found in 23 participants (19.8%). Additionally, NMDA receptor antibodies were present in 17 participants, making up 14.7% of the study group. GAD65 antibodies were relatively rare, found in only 1 participant (0.9%).

Model performance

The sensitivity of the model was found to be 85.71% (95% CI: 42.13% to 99.64%). Specificity was calculated at 100.00% (95% CI: 87.66% to 100.00%). The negative likelihood ratio was determined to be 0.14 (95% CI: 0.02 to 0.88). The positive predictive value (PPV) was 100.00% (95% CI: 54.07% to 100.00%), and the negative predictive value (NPV) was 96.55% (95% CI: 82.02% to 99.42%). The Brier score was 0.04. The AUC-ROC was 0.9795 (CI 95% 0.9183 to 1). Lastly, the overall accuracy of the model was 97.14% (95% CI: 85.08% to 99.93%).

Features’ Importance

The most influential feature was age, with higher age values generally increasing the likelihood of a cancer diagnosis, as shown by the positive impact on the model's output. The second feature was the presence of striated muscle antibodies, which showed a negative impact on the model's output, indicating a lower likelihood of cancer diagnosis. The third significant feature was the presence of acetylcholine receptor (AChR) antibodies, which had varied impacts on the model's output, but higher values were generally associated with a lower probability of cancer. The fourth most important feature was the presence of P/Q-type calcium channel antibodies, which mostly had a positive impact on the model's prediction, suggesting a higher likelihood of cancer when these antibodies were present. Lastly, the presence of NMDA receptor antibodies showed mixed impacts, but higher values were generally associated with a lower probability of cancer, as indicated by the negative SHAP values (Fig. 1).

Fig. 1.

Feature Importance Plot. The top plot illustrates the individual contribution of each feature to the model's output. It shows how variations in each feature directly impact predictions

Sensitivity analysis

The Jitter test results demonstrated the model's robustness to noise, with no significant decline in performance. The AUC-ROC scores exhibited only a modest decrease across various noise levels after the introduction of noise (Fig. 2).

Fig. 2.

Jitter Test Results. The top plot shows the model's performance (Y-axis) as the amount of jitter (X-axis) increases. The bottom plot represents the stability of a reference model, where the performance remains nearly constant regardless of the introduced jitter, indicating high robustness to input noise or perturbations

Discussion

The model performed relatively well on the test set, demonstrating high reliability for both positive and negative results, ensuring appropriate patient risk stratification. This helps tailor individualized management for each patient. Notably, the model’s PPV is 100%, but the confidence interval is relatively large, with the lower limit close to chance level (0.50). On the other hand, the NPV is much more reliable, with a narrow confidence interval. This makes the model a good tool for screening patients.

In this study, age emerged as the most significant predictor of cancer presence in patients. Increased age is associated with a higher risk of cancer. As the body ages, somatic mutations accumulate. This, combined with chronic microinflammation and reduced immune system surveillance, can lead to a higher incidence of cancer [20].

Another predictor of presence of cancer were P/Q-type calcium-channel autoantibodies. P/Q-type calcium channel antibodies are classically associated with LEMS (Lambert-Eaton myasthenia syndrome) and small cell lung cancer. In one study, anti–P/Q-type calcium-channel antibodies were detected in the serum of all 32 patients diagnosed with both LEMS and cancer [21]. Another study found elevated levels of P/Q-type voltage-gated calcium-channel (VGCC) antibodies in 16 out of 39 patients (41%) with paraneoplastic cerebellar degeneration (PCD) [22].

There is heterogeneity in cancer risk associated with various paraneoplastic autoantibodies reported in the literature. Certain autoantibodies, such as ANNA-1 (Hu) antibodies, are considered high-risk and are associated with cancer in approximately 85% of cases. Others, like P/Q VGCC and anti-NMDA receptor (NMDAR) antibodies, are categorized as intermediate risk. Specifically, NMDAR autoantibodies are linked to cancer in 38% of cases, while P/Q VGCC antibodies are associated with cancer in 50% of cases. Notably, in patients presenting with rapidly progressive cerebellar syndrome, the presence of P/Q VGCC autoantibodies is associated with cancer in 90% of cases [23]. These findings underscore the importance of interpreting auto-antibody presence in conjunction with the patient’s clinical presentation (Table 1) and other diagnostic information. This integrative approach may explain the variable association of individual autoantibodies with cancer risk.

Table 1.

Frequency of autoantibodies by presenting symptoms

Presenting Symptom	N-type calcium channel	Anti-neuronal nuclear antibody	CRMP	Purkinje cell cytoplasmic antibody	Striated muscle	Acetyl-choline receptor	P/Q-type calcium channel	Voltage-gated potassium channel	NMDA	GAD65
Autonomic	1	1	0	0	1	0	0	0	0	0
Bulbar	2	0	0	0	1	3	2	1	0	0
Cognitive	0	0	0	0	5	2	0	4	5	0
Constitutional	0	4	0	0	3	1	0	3	5	0
Coordination	0	0	0	2	1	2	1	2	0	0
Motor	0	0	0	0	22	5	13	5	6	0
Sensory	1	0	1	1	3	1	5	5	1	0
Vision	0	0	1	0	2	2	0	3	0	1

The main limitation of this study is its small sample size. Paraneoplastic syndromes are rare, making it challenging to gather large datasets suitable for machine learning. Additionally, only a small subset of patients had cancer, resulting in an imbalanced dataset that complicates the machine learning algorithms' ability to accurately identify instances. This imbalance can hinder the model's performance and reliability.

Moreover, external validation is required to ensure that the model is not overfitting and can generalize well to real-world data. Without validation on independent datasets, there is a risk that the model's predictions may not be robust when applied outside the study population. This step is crucial for confirming the model's utility and reliability in diverse clinical settings.

To make the application of the model easier, we developed a GUI-based app with a user-friendly interface that outputs the probability of cancer based on the developed model. This has several advantages. First, it reduces the level of expertise required to use the model, as many models typically require users to be familiar with Python at some level. Second, it simplifies the process of external validation. Lastly, it allows users to experiment with different combinations of symptoms, demographics, and antibody profiles for experimental purposes.

Conclusion

Machine learning models can accurately stratify the risk of cancer in patients with positive paraneoplastic autoantibodies. This approach can reduce unnecessary investigations in low-risk patients and prioritize diagnostic resources for high-risk individuals.

Author contributions

A.M: Supervision, Project administration, Writing—review and editing M.AM: Data curation, Formal analysis, Software, Writing—review and editing S.G: Conceptualization, Project administration, Methodology, Writing—review and editing B.D: Resources, Investigation, Writing—review and editing M.R: Conceptualization, Methodology, Writing—review and editing, Software, Formal analysis A.T: Data curation, Formal analysis, Software, Writing—review and editing Visualization, Methodology, Supervision, Project administration, Writing —original draft. S.G: Conceptualization, Project administration, Methodology, Writing—review and editing. B.D: Resources, Investigation, Writing—review and editing. M.R: Conceptualization, Methodology, Writing—review and editing, Software, Formal analysis. A.T: Data curation, Formal analysis, Software, Writing—review and editing. Visualization, Methodology, Supervision, Project administration, Writing—original draft.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Data availability

The dataset supporting the conclusions of this article is publicly available online.

Declarations

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.