Skip to main content
NCBI home page
Cancers logoLink to Cancers
. 2021 Oct 30;13(21):5465. doi: 10.3390/cancers13215465

Factors Associated with Thyroid-Related Adverse Events in Patients Receiving PD-1 or PD-L1 Inhibitors Using Machine Learning Models

Woorim Kim 1,†, Young-Ah Cho 2,3,†, Dong-Chul Kim 4, A-Ra Jo 5, Kyung-Hyun Min 1, Kyung-Eun Lee 1,*
PMCID: PMC8582564  PMID: 34771631

Abstract

Simple Summary

Although immune checkpoint inhibitors have a potential role in thyroid-related complications, no study has investigated factors associated with such adverse events. This study aims to explore the factors associated with thyroid-related adverse events in patients with anti-PD-1/PD-L1 agents by training predictive models utilizing various machine learning approaches. The results of this study could be used to develop individually tailored intervention strategies to prevent immune checkpoint inhibitor-induced thyroid-related outcomes.

Abstract

Targets of immune checkpoint inhibitors (ICIs) regulate immune homeostasis and prevent autoimmunity by downregulating immune responses and by inhibiting T cell activation. Although ICIs are widely used in immunotherapy because of their good clinical efficacy, they can also induce autoimmune-related adverse events. Thyroid-related adverse events are frequently associated with anti-programmed cell death 1 (PD-1) or anti-programmed cell death-ligand 1 (PD-L1) agents. The present study aims to investigate the factors associated with thyroid dysfunction in patients receiving PD-1 or PD-L1 inhibitors and to develop various machine learning approaches to predict complications. A total of 187 patients were enrolled in this study. Logistic regression analysis was conducted to investigate the association between such factors and adverse events. Various machine learning methods were used to predict thyroid-related complications. After adjusting for covariates, we found that smoking history and hypertension increase the risk of thyroid dysfunction by approximately 3.7 and 4.1 times, respectively (95% confidence intervals (CIs) 1.338–10.496 and 1.478–11.332, p = 0.012 and 0.007). In contrast, patients taking opioids showed an approximately 4.0-fold lower risk of thyroid-related complications than those not taking them (95% CI 1.464–11.111, p = 0.007). Among the machine learning models, random forest showed the best prediction, with an area under the receiver operating characteristic of 0.770 (95% CI 0.648–0.883) and an area under the precision-recall of 0.510 (95%CI 0.357–0.666). Hence, this study utilized various machine learning models for prediction and showed that factors such as smoking history, hypertension, and opioids are associated with thyroid-related adverse events in cancer patients receiving PD-1/PD-L1 inhibitors.

Keywords: immune checkpoint inhibitors, risk factors, hyperthyroidism, hypothyroidism, machine learning

1. Introduction

Cancer has become a global health problem and a leading cause of death worldwide. In 2020, there were approximately 19.3 million new cancer cases and 10 million cancer deaths globally. The top three cancer types in terms of the estimated number of patients are breast, lung, and prostate cancers. Lung cancer is the leading cause of cancer death [1]. Therefore, the identification of molecular mechanisms through which cancer develops and metastasizes is actively pursued; in particular, T lymphocytes, especially for antigen-directed cytotoxicity, have attracted increasing interest in developing immunotherapy for cancer treatment [2]. Various negative regulators of T cell activation act as checkpoint molecules, such as cytotoxic T lymphocyte-associated protein 4 (CTLA-4) inhibitors, anti-programmed cell death 1 (PD-1) agents, and anti-programmed cell death-ligand 1 (PD-L1) agents.

Immune checkpoint inhibitors (ICIs) have been widely used as they are highly effective against various tumors. Nivolumab, the first human IgG4 monoclonal antibody against PD-1, was approved by the FDA for various indications, including metastatic melanoma, non-small-cell lung cancer, and Hodgkin’s lymphoma [3,4,5,6]. Pembrolizumab is a human IgG4k monoclonal antibody against PD-1 that received first approval through an accelerated process as it showed a response rate of 24% in metastatic melanoma patients [7]. This agent is also approved for many other indications and has been shown to be superior to chemotherapy regimens [8]. Atezolizumab is the first PD-L1-targeted humanized IgG4 monoclonal antibody that was initially approved for the treatment of urothelial carcinoma [9]. Its usage is currently indicated for various cancers such as non-small-cell lung cancer and breast cancer [10,11].

Despite the clinical advantages of ICIs, they can induce autoimmune-related adverse events. As ICIs can activate T cells, they can give rise to various immune-related adverse events affecting various organs. Among them, thyroid-related complications have become one of the most common adverse events associated with ICIs. A meta-analysis showed that, among the patients receiving PD-1 inhibitors, 3.2% developed hyperthyroidism and 7.0% developed hypothyroidism [12]. Another case report showed that a patient with metastatic mucosal melanoma treated with ipilimumab and nivolumab developed several immune-related complications, including hypothyroidism [13]. Among the three types of ICIs (PD-1, PD-L1, and CTLA4), PD-1/PD-L1 are known to be associated with thyroid dysfunction [12,14], more frequently than CTLA4. Moreover, the incidence of thyroid-related adverse events was not affected by tumor types and ICIs used [12]. This result was also confirmed by a systematic review that showed that, regardless of the type of cancer and ICI drug used, the overall survival was similar as there is no association between the tumor type and the incidence of thyroid-related complications in patients receiving ICIs [15]. In addition, Maughan et al. showed that the frequency of most immune-related adverse events with ICIs appears to be similar across tumor types [16]. Although ICIs possibly have a role in thyroid-related complications, the factors associated with such adverse events have not been investigated yet.

As of late, machine learning methods have been increasingly used for making clinical predictions. Machine learning approaches are more suitable for developing novel prediction models than traditional statistical models that utilize variables for calculation. This study explores the factors associated with the development of thyroid-related adverse events in patients administered anti-PD-1/PD-L1 agents using training predictive models through various machine learning approaches.

2. Methods

2.1. Study Patients and Data Collection

This retrospective follow-up study included 209 patients who were prescribed ICIs between July 2015 and February 2021. Patients who had been diagnosed with hypo- or hyperthyroidism and were already prescribed thyroid-related medications or had incomplete data were excluded. Baseline values of the patient characteristics were obtained on the initial prescription date of ICIs. Data were collected using electronic medical records. Data on sex, age, height, weight, smoking history, alcohol history, comorbidities, concurrent medication, cancer type, cancer stage, and Eastern Cooperative Oncology Group performance scale (ECOGPS) were collected. Thyroid-related adverse events were defined as grade 2 or higher according to the Common Terminology Criteria for Adverse Events (CTCAE), version 5.0 [17]. The CTCAE defines grade 2 hyperthyroidism as symptomatic, thyroid suppression therapy indicated, and limiting instrumental activities of daily living. It defines hypothyroidism as symptomatic, thyroid replacement indicated, and limiting instrumental activities of daily living.

This study was approved by the Institutional Review Board of the Gyeongsang National University Hospital (approval number: GNUH 2019-11-041). All procedures performed in studies involving human participants were in accordance with the Declaration of Helsinki.

2.2. Statistical Analysis and Machine Learning Methods

Chi-square test or Fisher’s exact test was used to compare categorical variables between patients with thyroid-related complications and those without complications. Multivariable logistic regression analysis was used to examine independent risk factors for thyroid-related complications. Factors having a p-value less than 0.05 in univariate analysis along with clinically relevant confounders (age, sex, and body mass index (BMI)) were included in multivariable analysis. Odds ratios and adjusted odds ratios were calculated through univariate and multivariable analyses, respectively. To test the model’s goodness of fit, we performed a Hosmer–Lemeshow test.

This study used a random forest-based classification approach to analyze the importance of different variables for factors that are associated with thyroid-related adverse events. We focused on clinically relevant predictors and included 52 variables in the machine learning model. Seven features that are most important and clinically relevant were selected to prevent over-fitting. Machine learning methods including multivariate logistic regression, elastic net, random forest, and support vector machine (SVM) were employed for the prediction of factors affecting thyroid-related complications. All of the methods were implemented with the caret R package. To assess the ability of the associated factor to predict complication, the area under the receiver-operating curve (AUROC), the area under the precision-recall curve (AUPRC), and its 95% confidence interval (CI) of each machine learning prediction model were stated in this study. A p-value of less than 0.05 was considered statistically significant. A univariate statistical analysis was conducted using IBM SPSS statistics, version 20 software (International Business Machines Corp., New York, NY, USA). All other analyses were performed using R software version 3.6.0 (R Foundation for Statistical Computing, Vienna, Austria).

Internal validation was performed to measure the performance of each machine learning model. The whole dataset was randomly divided for model development and evaluation in the prediction process. After randomly partitioning one data sample into five subsets, one subset was selected for model validation while the remaining subsets were used to establish machine learning models. This five-fold cross-validation iteration was repeated 100 times to evaluate the prediction power of the machine learning models.

3. Results

Among the patients enrolled in this study (n = 209), 22 patients were excluded due to comorbidities of hyper- or hypothyroidism, prescription of thyroid-related medications, or incomplete data. Consequently, data on 187 patients who received ICIs were used for the analysis. The median age of the included patients was 67 years (range, 37–88 years), and there were 40 (19.1%) females. Among the ICIs, pembrolizumab was utilized the most (38.0%), followed by nivolumab (31.6%) and atezolizumab (30.5%). Twenty-three patients (12.3%) experienced thyroid-related adverse events after taking ICIs. Among them, 13, 1, and 9 patients experienced hypothyroidism, hyperthyroidism, and both, respectively.

As shown in Table 1, patients with a smoking history had more thyroid-related complications than those who did not have the history (p = 0.025). Additionally, patients with hypertension and heart disease had more adverse effects than those who did not have these comorbidities (p = 0.013 and p = 0.044, respectively). Patients taking P2Y12 inhibitors revealed more associations with thyroid-related complications compared with those without medications (p = 0.032) while opioids showed less complications than those without medications (p = 0.038).

Table 1.

Factors associated with thyroid-related adverse events in patients receiving immune checkpoint inhibitors.

Characteristics Complication
(n = 23)		 No Complication
(n = 164)		 p-Value
Sex 1.000
Male 18 (78.3) 129 (78.7)
Female 5 (21.7) 35 (21.3)
Age 0.420
<65 11 (47.8) 64 (39)
≥65 12 (52.2) 100 (61)
BMI 0.921
<23 13 (61.9) 93 (60.8)
≥23 8 (38.1) 60 (39.2)
Smoking history 0.025
Yes 9 (39.1) 29 (17.7)
No 14 (60.9) 135 (82.3)
Alcohol history 0.115
Yes 4 (17.4) 12 (7.3)
No 19 (82.6) 152 (92.7)
Comorbidities
Hypertension 0.013
Yes 14 (60.9) 56 (34.1)
No 9 (39.1) 108 (65.9)
Hyperlipidemia 0.052
Yes 5 (21.7) 13 (7.9)
No 18 (78.3) 151 (92.1)
COPD 0.477
Yes 1 (4.3) 18 (11)
No 22 (95.7) 146 (89)
Diabetes mellitus 0.689
Yes 5 (21.7) 42 (25.6)
No 18 (78.3) 122 (74.4)
Gout 1.000
Yes 0 (0) 4 (2.4)
No 23 (100) 160 (97.6)
BPH 0.136
Yes 0 (0) 19 (11.6)
No 23 (100) 145 (88.4)
Parkinson’s disease 1.000
Yes 0 (0) 1 (0.6)
No 23 (100) 163 (99.4)
Osteoporosis 1.000
Yes 0 (0) 2 (1.2)
No 23 (100) 162 (98.8)
MI 0.075
Yes 2 (8.7) 2 (1.2)
No 21 (91.3) 162 (98.8)
Heart disease 0.044
Yes 4 (17.4) 8 (4.9)
No 19 (82.6) 156 (95.1)
Asthma 1.000
Yes 0 (0) 3 (1.8)
No 23 (100) 161 (98.2)
Buger’s disease 1.000
Yes 0 (0) 1 (0.6)
No 23 (100) 163 (99.4)
Angina 0.123
Yes 1 (4.3) 0 (0)
No 22 (95.7) 164 (100)
Crohn’s disease 1.000
Yes 0 (0) 1 (0.6)
No 23 (100) 163 (99.4)
HIV 1.000
Yes 0 (0) 2 (1.2)
No 23 (100) 162 (98.8)
Hepatitis B 1.000
Yes 0 (0) 3 (1.8)
No 23 (100) 161 (98.2)
Concomitant drug
Statins 0.771
Yes 3 (13) 29 (17.7)
No 20 (87) 135 (82.3)
PPIs 0.952
Yes 8 (34.8) 56 (34.1)
No 15 (65.2) 108 (65.9)
5-HT₃ Antagonists 0.625
Yes 3 (13) 16 (9.8)
No 20 (87) 148 (90.2)
D2 antagonists 0.231
Yes 1 (4.3) 1 (0.6)
No 22 (95.7) 163 (99.4)
Corticosteroids 1.000
Yes 1 (4.3) 7 (4.3)
No 22 (95.7) 157 (95.7)
Antihistamines 0.204
Yes 3 (13) 10 (6.1)
No 20 (87) 154 (93.9)
Diuretics 1.000
Yes 1 (4.3) 12 (7.3)
No 22 (95.7) 152 (92.7)
β-blockers 0.061
Yes 3 (13) 5 (3)
No 20 (87) 159 (97)
P2Y12 inhibitors 0.032
Yes 4 (17.4) 7 (4.3)
No 19 (82.6) 157 (95.7)
5HT₄ agonists 0.327
No 22 (95.7) 162 (98.8)
Yes 1 (4.3) 2 (1.2)
Antiepileptics 1.000
Yes 0 (0) 1 (0.6)
No 23 (100) 163 (99.4)
Antibiotics 0.744
Yes 2 (8.7) 21 (12.8)
No 21 (91.3) 143 (87.2)
Alpha-blockers 0.476
Yes 1 (4.3) 19 (11.6)
No 22 (95.7) 145 (88.4)
5α-Reductase inhibitors 1.000
Yes 1 (4.3) 12 (7.3)
No 22 (95.7) 152 (92.7)
NSAIDs 0.261
Yes 2 (8.7) 32 (19.5)
No 21 (91.3) 132 (80.5)
Metformin 0.185
Yes 5 (21.7) 19 (11.6)
No 18 (78.3) 145 (88.4)
Antipsychotics 0.327
Yes 1 (4.3) 2 (1.2)
No 22 (95.7) 162 (98.8)
Anticoagulants 0.350
Yes 5 (21.7) 23 (14)
No 18 (78.3) 141 (86)
ACE inhibitors/ARBs 0.684
Yes 2 (8.7) 12 (7.3)
No 21 (91.3) 152 (92.7)
Zolpidem 0.327
Yes 1 (4.3) 2 (1.2)
No 22 (95.7) 162 (98.8)
TCAs 1.000
Yes 0 (0) 1 (0.6)
No 23 (100) 163 (99.4)
Opioids 0.038
Yes 12 (52.2) 120 (73.2)
No 11 (47.8) 44 (26.8)
Aspirin 1.000
Yes 0 (0) 5 (3)
No 23 (100) 159 (97)
Dopamine 0.327
Yes 1 (4.3) 2 (1.2)
No 22 (95.7) 162 (98.8)
Benzodiazepines 0.738
Yes 3 (13) 19 (11.6)
No 20 (87) 145 (88.4)
Antivirals 1.000
Yes 0 (0) 3 (1.8)
No 23 (100) 161 (98.2)
SSRIs, SNRIs 0.600
Yes 0 (0) 7 (4.3)
No 23 (100) 157 (95.7)
Cancer stage 0.428
1 0 (0.0) 1 (0.6)
2 0 (0.0) 3 (1.8)
3 3 (13.0) 9 (5.5)
4 20 (87.0) 150 (92.0)
Diagnosis 0.223
Bladder cancer 0 (0) 15 (9.1)
Colon cancer 0 (0) 3 (1.8)
Gastric cancer 0 (0) 8 (4.9)
Hepatocellular cancer 1 (4.3) 12 (7.3)
Lung cancer 19 (82.6) 75 (45.7)
Pancreatic cancer 0 (0) 2 (1.2)
Rectal cancer 0 (0) 3 (1.8)
Renal cancer 0 (0) 3 (1.8)
Stomach cancer 0 (0) 3 (1.8)
Other 2 (8.7) 38 (23.2)
ECOGPS 0.464
0 0 (0) 1 (0.6)
1 21 (91.3) 129 (79.6)
2 2 (8.7) 18 (11.1)
3 0 (0) 14 (8.6)
Open in a new tab

BMI: body mass index: COPD: chronic obstructive pulmonary disease; BPH: benign prostatic hyperplasia; MI: myocardial infarction; HIV: human immunodeficiency viruses; PPIs: proton pump inhibitors; NSAIDs: non-steroidal anti-inflammatory drugs; ACE: angiotensin-converting enzyme; ARBs: angiotensin receptor blockers; TCAs: tricyclic antidepressants; SSRIs: selective serotonin reuptake inhibitors; SNRIs: serotonin and norepinephrine reuptake inhibitors; ECOGPS: Eastern Cooperative Oncology Group performance status.

The multivariable analysis (Table 2) included sex, age, BMI, and factors with p < 0.05 in univariate analysis (heart disease, P2Y12 inhibitors, smoking history, hypertension, and opioids). After adjusting for related covariates, patients with smoking history and hypertension showed approximately 3.7- and 4.1-fold higher incidence of thyroid-related adverse events than patients without smoking history and hypertension, respectively. Patients taking opioids revealed about 4.0-fold fewer thyroid-related complications compared with those not taking opioids. The Hosmer–Lemeshow test showed that the fitness of the multivariable analysis model was satisfactory (χ2 = 0.764, 4 degrees of freedom, p = 0.943).

Table 2.

Multivariable analysis to identify predictors for thyroid-related adverse events in patients receiving immune checkpoint inhibitors.

Characteristics Crude OR (95% CI) p-Value Adjusted OR (95% CI) p-Value
Sex 1.024 (0.355–2.952) 0.965
Age < 65 0.698 (0.291–1.677) 0.422
BMI 0.954 (0.373–2.438) 0.921
Heart disease 4.105 (1.129–14.932) 0.032
P2Y12 inhibitors 4.722 (1.265–17.631) 0.021
Smoking history 2.993 (1.183–7.574) 0.021 3.748 (1.338–10.496) 0.012
Hypertension 3.000 (1.223–7.360) 0.016 4.093 (1.478–11.332) 0.007
Opioids 0.400 (0.165–0.972) 0.043 0.248 (0.090–0.683) 0.007
Open in a new tab

Crude OR: the result from fitting the univariate logistic regression model. Adjusted OR: adjusted for sex, age, BMI, heart disease, P2Y12 inhibitors, smoking history, hypertension, and opioids. BMI: body mass index; OR: odds ratio; CI: confidence interval.

As shown in Figure 1, after feature selection by performing a five-fold cross-validated random forest approach, seven important variables (heart disease, smoking history, opioids, hypertension, sex, age, and BMI) were included in machine learning models. The average AUROC values and AUPRC values after performing five-fold cross-validated multivariate logistic regression, elastic net, random forest, and SVM models across 100 random iterations are shown in Table 3. The AUROC values for multivariate logistic regression, elastic net, and random forest indicated acceptable performances of the models: 0.71, 0.71, and 0.77, respectively (95% CI 0.587–0.827, 0.588–0.829, and 0.648–0.883, respectively). Radial kernel SVM revealed sub-optimal performances of the models and an AUROC value of 0.69 (95% CI 0.539–0.838). The AUPRC values of multivariate logistic regression, elastic net, random forest, linear kernel SVM, and radial kernel SVM were 0.47, 0.47, 0.51, 0.36, and 0.45, respectively (95% CI 0.312–0.622, 0.314–0.625, 0.357–0.666, 0.216–0.497, and 0.310–0.600, respectively). For the random forest model, which showed the best prediction, the cut-off point that maximizes the accuracy was 0.05. While the prevalence of thyroid-related adverse events was 0.12, the prediction model showed an approximately 3.5-fold higher positive predictive value. Figure 2 showed the AUROC curves of the four models that exhibited acceptable or sub-optimal interpretability and prediction rates. The details for the parameters used for training models are provided in Table 4.

Figure 1.

Figure 1

Open in a new tab

Top 10 variables by importance estimated using random forest to predict thyroid-related adverse events in patients with cancer receiving ICIs.

Table 3.

Comparisons of AUC for the logistic regression, elastic net, random forest, and SVM models.

Machine Learning Model AUROC (95% CI) AUPRC (95% CI)
Logistic regression 0.71 (0.587–0.827) 0.47 (0.312–0.622)
Elastic net 0.71 (0.588–0.829) 0.47 (0.314–0.625)
Random Forest 0.77 (0.648–0.883) 0.51 (0.357–0.666)
SVM (Linear) 0.57 (0.394–0.752) 0.36 (0.216–0.497)
SVM (Radial) 0.69 (0.539–0.838) 0.45 (0.310–0.600)
Open in a new tab

AUROC: area under the receiver-operating curve; AUPR: area under the precision-recall curve; CI: confidence interval; and SVM: support vector machine.

Figure 2.

Figure 2

Open in a new tab

Receiver operating characteristic curves for predictive performance of the elastic net (ENET), logistic regression (LR), random forest (RF), and support vector machine radial (SVM_R) models.

Table 4.

Machine learning model specifics.

Method Hyperparameter
Model Specification and Search Grids Selected Values
Elastic net λ: 100 equally spaced values in logarithmic scale between 10−4 and 0 λ: 0.01261857
α: 0, 0.2, 0.4, 0.6, 0.8, 1 α: 0.6
Random forests mtry: 1, 2, 3, 4, 5, 6, 7 mtry: 1
SVM with linear kernel C: 0, 0.001, 0.005, 0.01, 0.05, 0.1, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2, 5 C: 1
SVM with radial kernel Sigma: 2−15, 2−13, 2−11, 2−9, 2−7, 2−5, 2−3, 2−1, 2, 23 Sigma: 0.125
C: 2−5, 2−3, 2−1, 2, 23, 25, 27, 29, 211, 213, 215 C: 128
Open in a new tab

SVM: support vector machine.

4. Discussion

The main finding of this study is that smoking history, hypertension, and opioids were associated with thyroid-related adverse events in patients taking anti-PD-1 or PD-L1. Patients with a smoking history and hypertension had an approximately 4.0-fold increased risk of thyroid-related complications compared with those without these conditions. Patients taking opioids showed an approximately 4.0-fold decreased risk of thyroid-related adverse events compared with those not taking them. Random forest was proven to be the most favorable method for predicting thyroid-related complications, with an AUROC value of 0.77 (95% CI 0.648–0.883) and an AUPRC value of 0.510 (95% CI 0.357–0.666).

Tumor antigens are known to be presented to T cells by antigen-presenting-cells, which trigger the interaction between T cell receptors and the major histocompatibility complex. Several receptors act as negative regulators of the immune response at different molecular checkpoints. For instance, the PD-1/PD-L1 pathway regulates inflammatory responses by effector T cells. Once T cells are activated, they upregulate PD-1 and inflammatory signals in the tissue. This action further induces the expression of PD-L1, resulting in the downregulation of T cell activity and protecting tissues from destruction [2]. ICIs block the PD-1/PD-L1 pathway and increase T cell activation and proliferation, which causes both anti-tumor activity and immune-related complications.

Thyroid-related adverse events are one of the most common immune-related complications in patients taking ICIs. These adverse events can present as hyperthyroidism or hypothyroidism. A randomized controlled phase 3 study showed the occurrence of both hypothyroidism (10.1% in the 2-week group and 8.7% in the 3-week group) and hyperthyroidism (6.5% and 3.2%, respectively) in patients receiving pembrolizumab [18]. Another randomized controlled trial also showed hypothyroidism and hyperthyroidism to be the most common adverse events of pembrolizumab [19]. In the group administered 2 mg/kg of pembrolizumab, 8% of the patients developed hypothyroidism while 4% developed hyperthyroidism [19]. In addition, a randomized, open-label, phase 3 trial reported several endocrine complications, including thyroid dysfunction, caused by nivolumab [20]. They showed that approximately more than 11% of patients receiving nivolumab had endocrine adverse events, most of which were observed during the initial seven months of the treatment [20]. As patients with ICI-induced thyroid dysfunction did not show clinical symptoms, it becomes crucial to carefully detect any adverse event during hormone monitoring. Therefore, immune-mediated adverse events, especially thyroid-related complications, play important roles in safety when facing the management of ICIs.

Our study results revealed that current or ex-smokers receiving anti-PD-1/PD-L1 therapy are at a higher risk of thyroid dysfunction. Cigarette smoking is a known risk factor for thyroid-related complications. A previous study has shown an association between smoking and the development of Graves’ hyperthyroidism [21]. Fukata et al. revealed that smoking increased the risk of subsequent hypothyroidism, possibly because of the antithyroid effect of thiocyanate [22]. Meanwhile, smoking is known to increase the efficacy of ICIs. A meta-analysis showed that both monotherapy and combination therapy are superior to chemotherapy in smokers; however, they were less effective than chemotherapy in never-smokers [23]. A possible explanation of this phenomenon is PD-L1 upregulation caused by smoking [24,25]. It has been shown that elevated levels of PD-L1 expression increased the efficiency of anti-PD-1/PD-L1 treatment [26,27]. As a result, increased activities of ICIs may trigger thyroid abnormalities because of the high T cell activity.

This study showed that hypertension is a risk factor for thyroid-related adverse events. The renin–angiotensin system plays a vital role in the regulation of hypertension. Increased renin–angiotensin system activity is known to increase blood pressure and to induce immunosuppression in the tumor environment [28]. Coelho et al. showed that oncogenic renin–angiotensin system signaling can increase PD-L1 expression [28]. As shown in the smoking case, it can be speculated that the anti-PD-1/PD-L1 treatment may have a higher efficacy because of upregulation.

This study showed that opioid use was negatively associated with thyroid-related complications in patients treated with PD-1/PD-L1 inhibitors. Opioids play a crucial role in increasing the resistance to immunotherapy [29]. Morphine and buprenorphine reduce the levels of interleukin-4 mRNA and protein in T cells [30]. A retrospective study on 102 cancer patients administered opioids and ICIs showed poor outcomes [31], possibly because of the presence of opioid receptors on immune cells [29]. As opioid receptors are expressed in immune cells, opioids could alter immune responses [32]. Given that the use of opioids can dysregulate the immune response, opioid usage during ICI treatment could affect its efficacy and can cause autoimmune complications, including thyroid dysfunction.

The utilization of machine learning approaches to predict thyroid-related adverse events in patients receiving PD-1/PD-L1 inhibitors is a novel concept in clinical research. Machine learning algorithms are integrated into the clinical decision-making process to guide clinicians to diagnose, screen, prevent, and treat cancer patients. Machine learning methods use a training dataset to train computational models and to generate the most optimal prediction models, which can further be validated in the test dataset to ensure accuracy. In clinical settings, these models can help predict and manage thyroid-related complications in patients receiving ICIs. In a binary model such as this study, the outcome prediction performance of a model is evaluated by the ROC curve. In this study, we performed feature selection using random forest, an ensemble method of bootstrap aggregated binary classification trees [33], to overcome overfitting. We also trained various machine learning models and concluded that the random forest model outperformed the other models with the highest AUROC and AUPRC values. Hence, this model can be used for predicting thyroid-related complications in patients on ICIs.

The limitations of our study are its small sample size and the lack of a detailed mechanism. Although it has been reported that cancer types did not affect thyroid-related adverse events, 10 types of cancer and 3 ICIs could influence other clinical outcomes, thereby complicating the study outcome. Therefore, cautious interpretation is needed when applying the results of this study to real clinical settings. Moreover, because of the lack of independent data, we did not perform the external validation that is needed to examine the trained model’s performance. Further research is needed to externally validate current results to ensure accuracy for application in clinical settings. Nevertheless, to the best of our knowledge, this is the first study to investigate factors responsible for thyroid dysfunction in patients taking anti-PD-1/PD-L1 agents. In addition, this study provides important features and prediction models based on machine learning algorithms, which included logistic regression, elastic net, random forest, and SVM. Given that our study developed prediction models using the factors associated with thyroid-related adverse events in patients receiving ICIs, our findings provide additional insight to manage thyroid-related complications. Moreover, the results of this study could be utilized to design and develop individually tailored PD-1/PD-L1 inhibitor treatments for various cancer types.

Author Contributions

Conceptualization, K.-E.L. methodology, K.-E.L.; formal analysis, W.K., Y.-A.C. and D.-C.K.; data curation, Y.-A.C., A.-R.J. and K.-H.M.; writing—original draft preparation, W.K. and Y.-A.C.; writing—review and editing, W.K. and K.-E.L.; supervision, K.-E.L. All authors have read and agreed to the published version of the manuscript.

Funding

This study has been supported by the Medical Research Center Program (2017R1A5A2015541) of the NRF funded by the Korean government (MSIP).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board of Gyeongsang National University Hospital (approval number: GNUH 2019-11-041).

Informed Consent Statement

Patient consent was waived because of the retrospective nature of the study (The Institutional Review Board of the Gyeongsang National University Hospital (approval number: GNUH 2019-11-041)).

Data Availability Statement

The data presented in this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Sung H., Ferlay J., Siegel R.L., Laversanne M., Soerjomataram I., Jemal A., Bray F. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J. Clin. 2021;71:209–249. doi: 10.3322/caac.21660. [DOI] [PubMed] [Google Scholar]
  • 2.Waldman A.D., Fritz J.M., Lenardo M.J. A guide to cancer immunotherapy: From T cell basic science to clinical practice. Nat. Rev. Immunol. 2020;20:651–668. doi: 10.1038/s41577-020-0306-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Vaddepally R.K., Kharel P., Pandey R., Garje R., Chandra A.B. Review of Indications of FDA-Approved Immune Checkpoint Inhibitors per NCCN Guidelines with the Level of Evidence. Cancers. 2020;12:738. doi: 10.3390/cancers12030738. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Hodi F.S., O’Day S.J., McDermott D.F., Weber R.W., Sosman J.A., Haanen J.B., Gonzalez R., Robert C., Schadendorf D., Hassel J.C., et al. Improved survival with ipilimumab in patients with metastatic melanoma. N. Engl. J. Med. 2010;363:711–723. doi: 10.1056/NEJMoa1003466. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Antonia S.J., López-Martin J.A., Bendell J., Ott P.A., Taylor M., Eder J.P., Jäger D., Pietanza M.C., Le D.T., de Braud F., et al. Nivolumab alone and nivolumab plus ipilimumab in recurrent small-cell lung cancer (CheckMate 032): A multicentre, open-label, phase 1/2 trial. Lancet Oncol. 2016;17:883–895. doi: 10.1016/S1470-2045(16)30098-5. [DOI] [PubMed] [Google Scholar]
  • 6.Ansell S.M., Lesokhin A.M., Borrello I., Halwani A., Scott E.C., Gutierrez M., Schuster S.J., Millenson M.M., Cattry D., Freeman G.J., et al. PD-1 blockade with nivolumab in relapsed or refractory Hodgkin’s lymphoma. N. Engl. J. Med. 2015;372:311–319. doi: 10.1056/NEJMoa1411087. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Robert C., Ribas A., Wolchok J.D., Hodi F.S., Hamid O., Kefford R., Weber J.S., Joshua A.M., Hwu W.J., Gangadhar T.C., et al. Anti-programmed-death-receptor-1 treatment with pembrolizumab in ipilimumab-refractory advanced melanoma: A randomised dose-comparison cohort of a phase 1 trial. Lancet. 2014;384:1109–1117. doi: 10.1016/S0140-6736(14)60958-2. [DOI] [PubMed] [Google Scholar]
  • 8.Ribas A., Puzanov I., Dummer R., Schadendorf D., Hamid O., Robert C., Hodi F.S., Schachter J., Pavlick A.C., Lewis K.D., et al. Pembrolizumab versus investigator-choice chemotherapy for ipilimumab-refractory melanoma (KEYNOTE-002): A randomised, controlled, phase 2 trial. Lancet Oncol. 2015;16:908–918. doi: 10.1016/S1470-2045(15)00083-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Rosenberg J.E., Hoffman-Censits J., Powles T., van der Heijden M.S., Balar A.V., Necchi A., Dawson N., O’Donnell P.H., Balmanoukian A., Loriot Y., et al. Atezolizumab in patients with locally advanced and metastatic urothelial carcinoma who have progressed following treatment with platinum-based chemotherapy: A single-arm, multicentre, phase 2 trial. Lancet. 2016;387:1909–1920. doi: 10.1016/S0140-6736(16)00561-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Horn L., Mansfield A.S., Szczęsna A., Havel L., Krzakowski M., Hochmair M.J., Huemer F., Losonczy G., Johnson M.L., Nishio M., et al. First-Line Atezolizumab plus Chemotherapy in Extensive-Stage Small-Cell Lung Cancer. N. Engl. J. Med. 2018;379:2220–2229. doi: 10.1056/NEJMoa1809064. [DOI] [PubMed] [Google Scholar]
  • 11.Schmid P., Adams S., Rugo H.S., Schneeweiss A., Barrios C.H., Iwata H., Diéras V., Hegg R., Im S.A., Shaw Wright G., et al. Atezolizumab and Nab-Paclitaxel in Advanced Triple-Negative Breast Cancer. N. Engl. J. Med. 2018;379:2108–2121. doi: 10.1056/NEJMoa1809615. [DOI] [PubMed] [Google Scholar]
  • 12.Barroso-Sousa R., Barry W.T., Garrido-Castro A.C., Hodi F.S., Min L., Krop I.E., Tolaney S.M. Incidence of Endocrine Dysfunction Following the Use of Different Immune Checkpoint Inhibitor Regimens: A Systematic Review and Meta-analysis. JAMA Oncol. 2018;4:173–182. doi: 10.1001/jamaoncol.2017.3064. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Cao Y., Afzal M.Z., Shirai K. Ipilimumab and nivolumab induced immune-related adverse events in metastatic mucosal melanoma. BMJ Case Rep. 2021;14:e243713. doi: 10.1136/bcr-2021-243713. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.González-Rodríguez E., Rodríguez-Abreu D. Spanish Group for Cancer Immuno-Biotherapy (GETICA). Immune Checkpoint Inhibitors: Review and Management of Endocrine Adverse Events. Oncologist. 2016;21:804–816. doi: 10.1634/theoncologist.2015-0509. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Huang X.Z., Gao P., Song Y.X., Sun J.X., Chen X.W., Zhao J.H., Wang Z.N. Efficacy of immune checkpoint inhibitors and age in cancer patients. Immunotherapy. 2020;12:587–603. doi: 10.2217/imt-2019-0124. [DOI] [PubMed] [Google Scholar]
  • 16.Maughan B.L., Bailey E., Gill D.M., Agarwal N. Incidence of Immune-Related Adverse Events with Program Death Receptor-1- and Program Death Receptor-1 Ligand-Directed Therapies in Genitourinary Cancers. Front. Oncol. 2017;7:56. doi: 10.3389/fonc.2017.00056. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.National Institutes of Health, National Cancer Institute Common Terminology Criteria for Adverse Events (CTCAE), Version 5.0. [(accessed on 13 August 2021)];2017 November; Available online: https://ctep.cancer.gov/protocoldevelopment/electronic_applications/docs/CTCAE_v5_Quick_Reference_8.5x11.pdf.
  • 18.Robert C., Schachter J., Long G.V., Arance A., Grob J.J., Mortier L., Daud A., Carlino M.S., McNeil C., Lotem M., et al. Pembrolizumab versus Ipilimumab in Advanced Melanoma. N. Engl. J. Med. 2015;372:2521–2532. doi: 10.1056/NEJMoa1503093. [DOI] [PubMed] [Google Scholar]
  • 19.Herbst R.S., Baas P., Kim D.W., Felip E., Pérez-Gracia J.L., Han J.Y., Molina J., Kim J.H., Arvis C.D., Ahn M.J., et al. Pembrolizumab versus docetaxel for previously treated, PD-L1-positive, advanced non-small-cell lung cancer (KEYNOTE-010): A randomised controlled trial. Lancet. 2016;387:1540–1550. doi: 10.1016/S0140-6736(15)01281-7. [DOI] [PubMed] [Google Scholar]
  • 20.Motzer R.J., Escudier B., George S., Hammers H.J., Srinivas S., Tykodi S.S., Sosman J.A., Plimack E.R., Procopio G., McDermott D.F., et al. Nivolumab versus everolimus in patients with advanced renal cell carcinoma: Updated results with long-term follow-up of the randomized, open-label, phase 3 CheckMate 025 trial. Cancer. 2020;126:4156–4167. doi: 10.1002/cncr.33033. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Bertelsen J.B., Hegedüs L. Cigarette smoking and the thyroid. Thyroid. 1994;4:327–331. doi: 10.1089/thy.1994.4.327. [DOI] [PubMed] [Google Scholar]
  • 22.Fukata S., Kuma K., Sugawara M. Relationship between cigarette smoking and hypothyroidism in patients with Hashimoto’s thyroiditis. J. Endocrinol. Investig. 1996;19:607–612. doi: 10.1007/BF03349026. [DOI] [PubMed] [Google Scholar]
  • 23.Dai L., Jin B., Liu T., Chen J., Li G., Dang J. The effect of smoking status on efficacy of immune checkpoint inhibitors in metastatic non-small cell lung cancer: A systematic review and meta-analysis. EClinicalMedicine. 2021;38:100990. doi: 10.1016/j.eclinm.2021.100990. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Calles A., Liao X., Sholl L.M., Rodig S.J., Freeman G.J., Butaney M., Lydon C., Dahlberg S.E., Hodi F.S., Oxnard G.R., et al. Expression of PD-1 and Its Ligands, PD-L1 and PD-L2, in Smokers and Never Smokers with KRAS-Mutant Lung Cancer. J. Thorac. Oncol. 2015;10:1726–1735. doi: 10.1097/JTO.0000000000000687. [DOI] [PubMed] [Google Scholar]
  • 25.Garon E., Gandhi L., Rizvi N., Hui R., Balmanoukian A.S., Patnaik A., Eder J.P., Blumenshein G.R., Aggarwal C., Soria J.C., et al. Antitumor Activity of Pembrolizumab (Pembro; Mk-3475) and Correlation with Programmed Death Ligand 1 (Pd-L1) Expression in a Pooled Analysis of Patients (Pts) with Advanced Non–Small Cell Lung Carcinoma (Nsclc) Ann. Oncol. 2014;25:v1. doi: 10.1093/annonc/mdu438.51. [DOI] [Google Scholar]
  • 26.Garon E.B., Hellmann M.D., Rizvi N.A., Carcereny E., Leighl N.B., Ahn M.J., Eder J.P., Balmanoukian A.S., Aggarwal C., Horn L., et al. Five-Year Overall Survival for Patients With Advanced Non‒Small-Cell Lung Cancer Treated With Pembrolizumab: Results From the Phase I KEYNOTE-001 Study. J. Clin. Oncol. 2019;37:2518–2527. doi: 10.1200/JCO.19.00934. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Middleton G., Brock K., Savage J., Mant R., Summers Y., Connibear J., Shah R., Ottensmeier C., Shaw P., Lee S.M., et al. Pembrolizumab in patients with non-small-cell lung cancer of performance status 2 (PePS2): A single arm, phase 2 trial. Lancet Respir. Med. 2020;8:895–904. doi: 10.1016/S2213-2600(20)30033-3. [DOI] [PubMed] [Google Scholar]
  • 28.Coelho M.A., de Carné Trécesson S., Rana S., Zecchin D., Moore C., Molina-Arcas M., East P., Spencer-Dene B., Nye E., Barnouin K., et al. Oncogenic RAS Signaling Promotes Tumor Immunoresistance by Stabilizing PD-L1 mRNA. Immunity. 2017;47:1083–1099. doi: 10.1016/j.immuni.2017.11.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Botticelli A., Cirillo A., Pomati G., Cerbelli B., Scagnoli S., Roberto M., Gelibter A., Mammone G., Calandrella M.L., Cerbelli E., et al. The role of opioids in cancer response to immunotherapy. J. Transl. Med. 2021;19:119. doi: 10.1186/s12967-021-02784-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Okuyama K., Ide S., Sakurada S., Sasaki K., Sora I., Tamura G., Ohkawara Y., Takayanagi M., Ohno I. μ-opioid receptor-mediated alterations of allergen-induced immune responses of bronchial lymph node cells in a murine model of stress asthma. Allergol. Int. 2012;61:245–258. doi: 10.2332/allergolint.11-OA-0304. [DOI] [PubMed] [Google Scholar]
  • 31.Iglesias-Santamaría A. Impact of antibiotic use and other concomitant medications on the efficacy of immune checkpoint inhibitors in patients with advanced cancer. Clin. Transl. Oncol. 2020;22:1481–1490. doi: 10.1007/s12094-019-02282-w. [DOI] [PubMed] [Google Scholar]
  • 32.Meng J., Yu H., Ma J., Wang J., Banerjee S., Charboneau R., Barke R.A., Roy S. Morphine induces bacterial translocation in mice by compromising intestinal barrier function in a TLR-dependent manner. PLoS ONE. 2013;8:e54040. doi: 10.1371/journal.pone.0054040. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Breiman L. Random forests. Mach. Learn. 2001;45:5–32. doi: 10.1023/A:1010933404324. [DOI] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The data presented in this study are available from the corresponding author upon request.

Articles from Cancers are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

RESOURCES