Skip to main content
Scientific Reports logoLink to Scientific Reports
. 2020 Jul 31;10:12982. doi: 10.1038/s41598-020-70026-w

Synergistic drug combinations and machine learning for drug repurposing in chordoma

Edward Anderson 1, Tammy M Havener 1, Kimberley M Zorn 2, Daniel H Foil 2, Thomas R Lane 2, Stephen J Capuzzi 1, Dave Morris 1, Anthony J Hickey 1,3, David H Drewry 4, Sean Ekins 1,2,
PMCID: PMC7395084  PMID: 32737414

Abstract

Chordoma is a devastating rare cancer that affects one in a million people. With a mean-survival of just 6 years and no approved medicines, the primary treatments are surgery and radiation. In order to speed new medicines to chordoma patients, a drug repurposing strategy represents an attractive approach. Drugs that have already advanced through human clinical safety trials have the potential to be approved more quickly than de novo discovered medicines on new targets. We have taken two strategies to enable this: (1) generated and validated machine learning models of chordoma inhibition and screened compounds of interest in vitro. (2) Tested combinations of approved kinase inhibitors already being individually evaluated for chordoma. Several published studies of compounds screened against chordoma cell lines were used to generate Bayesian Machine learning models which were then used to score compounds selected from the NIH NCATS industry-provided assets. Out of these compounds, the mTOR inhibitor AZD2014, was the most potent against chordoma cell lines (IC50 0.35 µM U-CH1 and 0.61 µM U-CH2). Several studies have shown the importance of the mTOR signaling pathway in chordoma and suggest it as a promising avenue for targeted therapy. Additionally, two currently FDA approved drugs, afatinib and palbociclib (EGFR and CDK4/6 inhibitors, respectively) demonstrated synergy in vitro (CI50 = 0.43) while AZD2014 and afatanib also showed synergy (CI50 = 0.41) against a chordoma cell in vitro. These findings may be of interest clinically, and this in vitro- and in silico approach could also be applied to other rare cancers.

Subject terms: Computational biology and bioinformatics, Drug discovery, Diseases, Oncology

Introduction

Chordoma is a rare cancer that occurs in the bones of the skull base and spine which is part of a larger class of tumors known as sarcomas. Chordoma tumors develop from cells of the notochord, an embryonic structure that facilitates development of the spine1. The notochord disappears when the fetus is about 8 weeks old, but some notochord cells remain in the bones of the spine and skull base2. This is a rare occurrence, but when they do, these cells can turn into chordoma. A chordoma tumor usually grows slowly without symptoms for years before diagnosis, which is often in the 5th and 6th decades of life (although it can occur at any age). Studies have demonstrated that skull base chordomas are observed more often in children, whilst spinal chordomas are more frequently observed later in life2,3. It has also been described that when chordomas metastasize they frequently distribute to the lungs, liver, bones, or lymph nodes. This occurs in 30 to 40 percent of people where the tumor metastasizes to other parts of the body2. At this point in time there are no known environmental, dietary or lifestyle risk factors for this rare type of cancer. Chordomas often occur at random with no direct inherited genetic trait, however familial cases can be caused by duplications of the brachyury gene4. A SNP in the brachyury gene occurs in 95 percent of people with this tumor5,6, and furthermore, chordomas have been reported at a higher incidence in children diagnosed with the genetic disease Tuberous Sclerosis Complex (TSC)7. With a mean-survival rate of just 6 years and poor response to current medications, surgical resection is the main course of treatment2. Patients therefore need new and effective drugs to expand their treatment options and improve survival rates.

Chordoma tumors, which occur in both pediatric and adult populations, are known to overexpress multiple kinases4. Kinases are a family of ~ 500 proteins, collectively known as the kinome, integral for a multitude of cellular functions relevant to cancer pathogenesis. In a 2013 study8, a tissue microarray containing 58 chordomas was used to examine the expression of the kinases PDGFR-α, PDGFR-β, EGFR, c-Met, c-Kit, pAKT, mTOR, and HER2. Most tumors were positive by immunohistochemistry for PDGFR-α (92%), PDGFR-β (85%), c-Kit (77%), c-Met (96%), pAKT (82%), mTOR (56%), HER2 (24%), and EGFR (26%), yet imatinib, an FDA-approved drug that inhibits PDGFR-α, PDGFR-β, and c-Kit, has shown little to no efficacy in chordoma in vivo models9. A body of such molecular, preclinical, and clinical evidence of interest to chordoma oncogenesis has begun to emerge for several kinases: Epidermal Growth Factor Receptor (EGFR), Cyclin-dependent kinase 4 (CDK4), Cyclin-dependent kinase 6 (CDK6) and the mammalian target of rapamycin (mTOR). These kinases are well-studied in the field of oncology, with several FDA-approved drugs on the market targeting each kinase and they may serve as drug repurposing candidates for the treatment of chordoma.

Drug repurposing or repositioning is an approach whereby new therapeutic uses for existing drugs or clinical candidates are identified10-14. High throughput screens, virtual screening or serendipitous observations are employed to enable drug repurposing13. For example we have previously identified approved drugs active against the Ebola virus15 and Chagas Disease16 using Bayesian and other machine learning models. In addition, there are several ongoing efforts to demonstrate new uses for molecules that have been through clinical trials for other uses but were subsequently shelved. One such example is the NIH NCATS industry-provided assets that could be potentially repurposed (https://ncats.nih.gov/ntu/assets/current). We have now developed a strategy for virtual screening such compounds then testing in vitro and will describe this approach applied to chordoma.

Further, two FDA-approved kinase inhibitor drugs—palbociclib, a breast cancer drug, and afatinib (Fig. 1A,B), a non-small cell lung carcinoma drug—have shown equally robust efficacy in patient derived xenograft (PDX) and cell-line derived xenograft (CDX) models of chordoma. Palbociclib17 and afatinib18 were designed to specifically target different kinases, CDK4/6 and EGFR, respectively. The utility of each in preclinical chordoma models implies that multiple oncogenic biological pathways may drive chordoma as is true in many other cancers.

Figure 1.

Figure 1

Molecule structures evaluated in this study. (A) Palbociclib (CDK4/6) and (B) Afatinib (EGFR), (C) AZD2014 (targets mTOR1 and mTOR2), (D) RDEA119 (targets MEK1/2), (E) AZD4054 (targets endothelin A receptor). (F) AZD0530 (Src inhibitor). Images created with Mobile Molecular DataSheet (Molecular Materials Informatics, Inc., Montreal Canada).

We hypothesized that combinations of clinical kinase inhibitors in chordoma models and patients may act additively or synergistically by dampening oncogenic signaling in multiple pathways. Both primary targets and secondary targets can play a role in this. A combination therapy of afatinib and palbociclib is of particular interest but there may also be many other potential combinations of kinase inhibitors. Chordoma PDX and CDX mouse models respond equally well to afatinib and palbociclib, though these drugs target divergent and minimally overlapping regions of the kinome19. A combination of EGFR/CDK inhibitors, i.e., afatinib/palbociclib, may target multiple oncogenic signaling pathways simultaneously. Following the same rationale, we now evaluate the in vitro efficacy of EGFR/CDK inhibitor combinations prior to future in vivo PDX and CDX mouse model studies. We envision that these proposed studies will enable and support future drug combination chordoma clinical trials.

Results

Machine learning models for chordoma drug discovery

Several recently published studies of compounds screened against chordoma cell lines20,21 were used to generate Bayesian machine learning models with our Assay Central software10,12,22-29. In one published chordoma study 1097 compounds were screened against 3 chordoma cell lines (U-CH1, U-CH2, MUG-Chor1) and 27 had chordoma selective cytotoxicity20 and many of these were EGFR inhibitors. A more recent study from the Broad Institute and collaborators profiled 459 compounds in 4 chordoma cell lines (JHC7, MUG-Chor1, U-CH1, UM-Chor1) and identified 28 potent antiproliferative compounds with several kinase inhibitors including CDK 7/12/13 inhibitors21. These data sets (which for ease we have called EGFR or Broad) were used either separately or combined to generate machine learning models that were evaluated with fivefold cross validation (Table 1, Figure S1). These models were also used to score the 3 molecules not in these training sets (Table 2 and Fig. 1C,D) and their scores would suggest the value of testing them in vitro. The atom coloring feature helps to suggest which molecule features correspond to activity with a particular model [e.g. with the EGFR model AZD2014 has a large area of the molecule colored green (favorable for activity) while the other molecules have fewer green colored atoms (Fig. 2A–C)].

Table 1.

Chordoma Bayesian model statistics.

Model name Actives Size ROC F1 Kappa MCC Domain
Broad 150 454 0.67 0.54 0.23 0.25 0.36
Broad + EGFR 289 1486 0.81 0.52 0.36 0.39 0.33
EGFR 147 1064 0.84 0.44 0.30 0.37 0.28

Datasets were named as “Broad”21 and “EGFR”20, and underwent curation to remove problematic molecules before model building. Data represent fivefold cross validation. The “Broad” dataset is named as such because the data came from a chordoma screen at the Broad Institute. The “EGFR” data set is so named because it came from a paper that highlighted the activity of EGFR compounds in chordoma and is not meant to construe a dataset made up entirely of EGFR compounds. Both datasets contain a wide variety of compounds that inhibit a broad range of targets.

Table 2.

Bayesian Machine Learning predictions for chordoma activity for compounds not in the training sets.

Compound Broad Bayesian score Broad model applicability Broad/EGFR Bayesian score Broad/EGFR model applicability EGFR Bayesian score EGFR model applicability
AZD2014 0.46 0.91 0.56 0.96 0.67 0.82
RDEA11 0.62 0.66 0.66 0.67 0.60 0.57
AZD4054 0.53 0.53 0.50 0.60 0.55 0.55

Datasets were named as Broad21 and EGFR20 and models are described in Table 1. Model applicability assesses the portion of fragments overlapping with the training set molecules, higher values indicate more fragments overlapping with the training set. The “score” is the prediction score, a measure of probability of activity with higher values being desirable.

Figure 2.

Figure 2

Examples of atom highlighting derived from Bayesian models. The molecules not in each model were tested with (A) Broad, (B) Broad + EGFR and (C) EGFR models. Molecules shown are: AZD2014 (top), RDEA119 (middle), AZD4054 (bottom) on each panel. Images were generated in Assay Central (Collaborations Pharmaceuticals, Inc.).

In vitro studies

We tested AZD2014 (Fig. 1C), RDEA119 (Fig. 1D), AZD4054 (Fig. 1E) AZD0530, (Fig. 1F) in UCH1 and UCH2 cell lines, along with known chordoma inhibitors afatinib and palbociclib, using reduction of resazurin to resarufin as an assay for metabolic activity of chordoma cell lines. As previously reported18, afatinib is a potent inhibitor of U-CH1, but not U-CH2. The Bayesian model we have termed ‘EGFR’ correctly predicted the rank order of these 3 compounds not in the training sets (Table 2). Of the drugs predicted by our model, AZD2014 was the most potent against both cell lines (IC50 0.35 µM U-CH1 and 0.61 µM U-CH2, Table 3 and Fig. 3A,B), though maximal inhibition of U-CH2 cells was limited. AZD0530 and RDEA119 also showed moderate potency against U-CH1 (IC50 1.5 µM and 2.4 µM, respectively) but not U-CH2. AZD4054 had low potency, but even at 11 µM this was unexpected given that its target, the endothelin A receptor, has no known functions in chordoma.

Table 3.

Summary of resazurin reduction assays for chordoma cell lines.

Drug U-CH1 U-CH2
IC50 (µM, absolute) IC50 (µM, relative) Maximal inhibition (%) n IC50 (µM, absolute) IC50 (µM, relative) Maximal inhibition (%) n
Afatinib 0.54 0.38 93 6 9.4 8.8 98 7
Palbociclib 11.4 10.8 96 6 20.8 19.5 94 6
RDEA119 2.4 59 2 24.5 24.4 94 4
AZD0530 2.7 1.5 85 5 29.4 26.6 100 3
AZD2014 0.94 0.35 72 7 0.61 53 8
AZD4054 11.9 11.0 84 2 18.8 19.2 100 2

Data generated includes geometric mean IC50, and arithmetic mean of maximal inhibition. Absolute IC50 is the dose that reduces resazurin fluorescence by 50%, relative to positive and negative controls. Relative IC50 is the dose that has 50% of the maximal inhibition for a given drug. Where values are absent this represents the drug never reaching 50% inhibition in several replicate experiments.

Figure 3.

Figure 3

Representative dose response curves for compounds tested against chordoma cell lines in vitro. Molecules were tested in (A) U-CH1 cells and (B) U-CH2 cells. Graphs produced with R66 version 3.6.2 (https://cran.r-project.org/), with the following packages: drc 3.0–6 for general dose–response curve fitting and analysis, medrc 1.1–0 for mixed-effects modeling of dose–response data, multcomp 1.4–11 for multiple comparison testing, and ggplot2 3.2.1 for plotting.

In vitro studies—combination studies of approved drugs

Next we tested the combinations of drugs used in this study to determine whether inhibition of multiple biological pathways would result in synergistic effects on chordoma cell lines. In order to determine this we calculated the combination index (CI) for each of the pairs of drugs mixed at different ratios, where CI < 1 indicates a synergistic effect. For each of the pairs of drugs used, CI was then calculated using five eight-point dose curves, each with a constant ratio of the two drugs. In U-CH1 cells, we observed substantial synergy between afatinib and palbociclib (CI = 0.43, 95% CL 0.28–0.66, Fig. 4A, Table 4). We also observed substantial synergy between afatinib and AZD2014 (CI = 0.41, 95% CL 0.34–0.78, Fig. 4B). The combination of AZD0530 and AZD2014 showed more modest synergy (CI = 0.77, 95% CI 0.67–0.88, Fig. 4C), while the combination of palbociclib and AZD0530 had weak but detectable synergy (CI50 = 0.61, 95% CI 0.49–0.77, Fig. 4D). However, in U-CH2 cells, we did not observe synergistic effects in any tested combinations (Table 5).

Figure 4.

Figure 4

Isobolograms for combinations of compounds demonstrating significant synergy. (A) palbociclib and afatanib, (B) AZD2014 and afatanib, (C) AZD2014 and AZD0530, (D) AZD0530 and palbociclib. The Loewe Additivity isobole is shown as the diagonal grey line, representing the expected IC50 of non-interacting combinations of drugs. Measured absolute IC50 of drug combinations (black line and solid dots) represent the isobole of dose combinations with equivalent activity, with dashed lines and open dots indicating the 95% confidence band. Points below and to the left of the isobole indicate a synergistic drug interaction. Graphs produced with R66 version 3.6.2 (https://cran.r-project.org/), with the following packages: drc 3.0–6 for general dose–response curve fitting and analysis, medrc 1.1–0 for mixed-effects modeling of dose–response data, multcomp 1.4–11 for multiple comparison testing, and ggplot2 3.2.1 for plotting.

Table 4.

Summary of drug combinations in U-CH1 cells.

Effect level Drug 1 Drug 2 Ratio CI UCL LCL
50% Afatinib Palbociclib 0.0094:1 0.58 0.79 0.43
0.038:1 0.43 0.66 0.28
0.15:1 0.48 0.96 0.25
50% Afatinib AZD0530 0.025:1 0.98 2.92 0.34
0.1:1 1.10 2.63 0.48
0.4:1 1.05 2.36 0.53
50% Afatinib AZD2014 0.075:1 0.51 0.78 0.34
0.3:1 0.57 0.84 0.39
1.2:1 0.75 1.14 0.50
50% Palbociclib AZD0530 0.67:1 1.08 1.62 0.72
2.7:1 0.95 1.14 0.79
11:1 0.77 0.88 0.67
75% Palbociclib AZD2014 2:1 1.432 2.289 0.895
8:1 1.398 1.911 1.024
32:1 1.248 1.478 1.053
50% AZD0530 AZD2014 0.75:1 0.61 0.77 0.49
3:1 0.81 0.99 0.66
12:1 1.19 1.50 0.94

Each CI was calculated at the specified effect level, along with upper and lower confidence limits (UCL and LCL). Combinations with significant synergy are indicated in bold.

Table 5.

Summary of drug combinations in U-CH2 cells.

Effect level Drug 1 Drug 2 Ratio CI UCL LCL
50% Afatinib Palbociclib 0.096:1 0.68 1.45 0.33
0.38:1 0.52 1.76 0.25
1.5:1 0.59 5.47 0.40
50% Afatinib RDEA119 0.074:1 1.04 2.25 0.49
0.29:1 0.98 2.78 0.48
1.2:1 0.53  > 10 0.36
75% Afatinib AZD2014 0.62:1 0.71 2.37 0.22
2.5:1 0.83 1.41 0.49
10:1 0.75 1.14 0.50
75% Palbociclib RDEA119 0.19:1 1.00  > 10 0.33
0.76:1 0.57 5.60 0.06
3.1:1 0.38  > 10 0.03
75% RDEA119 AZD2014 2.1:1 1.77  > 10 0.08
8.5:1 3.02  > 10 1.16
34:1 1.63 4.97 0.54

Each CI was calculated at the specified effect level, along with upper and lower confidence limits (UCL and LCL). No combinations had significant synergy.

Discussion

Epidermal Growth Factor Receptor (EGFR) is a receptor tyrosine kinase (RTK)30. Activation of EGFR leads to the phosphorylation of proteins in downstream signaling pathways, including the PI3k-Akt-mTOR and RAS-RAF-MEK-MAPK pathways30. Both of these pathways are critical in regulating cellular apoptosis, proliferation, migration, and survival. Broadly speaking, over-expression of EGFR, which is located on chromosome 7p12, can increase cellular proliferation and contribute to aggressive tumor behavior31. Within the molecular context of chordoma, copy number variations at chromosome 732, i.e., gains and partial polysomies, are commonly observed in chordoma, and EGFR gene copy number variations are likewise observed8. Several FDA-approved and clinical kinase inhibitors whose main therapeutic target is EGFR have been tested in preclinical in vitro and in vivo models of chordoma33. Gefitinib and erlotinib, two FDA-approved EGFR inhibitors used in the treatment of non-small-cell lung carcinoma (NSCLC), individually inhibited U-CH1, U-CH7, UM-Ch-SCor1, and MUG-Chor1 cellular proliferation in dose-dependent manners20,34. Afatinib, another FDA-approved EGFR inhibitor used against NSCLC, had anti-proliferative activity against all chordoma cell lines tested (IC50 < 0.7 µM) except for JHC720. Afatinib was also found to promote degradation of EGFR and brachyury, both of which are crucial to chordoma cell growth20. In vivo studies for erlotinib and afatinib have been reported. Erlotinib treatment significantly lowered tumor volume relative to controls in a PDX mouse model and reduced p-EGFR9. Afatinib treatment resulted in tumor growth inhibition in three PDXs and one CDX with no signs of toxicity in any of the mouse models9. In addition to this preclinical data, some clinical evidence exists to support EGFR and its therapeutic agents as potential chordoma treatments. A retrospective study found the median PFS was ~ 15.0 months for five patients treated with erlotinib35. For afatinib, there is currently an open clinical trial in the Netherlands (NCT03083678) to evaluate its efficacy against locally advanced and metastatic chordoma.

Cyclin-dependent kinase 4 (CDK4) and cyclin-dependent kinase 6 (CDK6) are serine-threonine kinases36. CDK4/6 are known oncoproteins, as upregulation of these kinases or inactivation of CDKN2A lead to cell cycle deregulation, increased cell proliferation, and tumorigenesis37. CDK4/6 functions downstream of a number of oncogenic pathways, implying that CDK4/6 inhibition may be effective when combined with inhibitors of the upstream pathways38,39. Combination treatment of an EGFR sensitive non-small cell lung cancer PDX model with a CDK4/6 inhibitor and an EGFR inhibitor showed combinatorial benefit, providing precedent for a similar application in chordoma tumors which are also sensitive to EGFR inhibition40. In another study, palbociclib sensitized lung cancer cells to treatment with EGFR inhibitors41. Importantly, in chordoma the CDK4/6 regulatory gene CDKN2A (also known as p16) is frequently lost. CDK4/6 are thus found in an over-activated state in chordoma42. From a gene expression perspective, CDK4 and CDK6 mRNAs have been detected in all eight chordoma cell lines that were interrogated: U-CH1, U-CH2, U-CH3, U-CH6, U-CH7, U-CH10, U-CH11, and U-CH1243. Likewise, CDK4 and CDK6 protein were observed in the same eight chordoma cell line as above42. An immunohistochemistry study of 25 patient samples showed CDK4 was overexpressed in 20% of cases44. Tissue microarray analysis of 85 samples from 72 chordoma patients found that ~ 98% of tissue samples expressed CDK442. Additionally, the mean expression level of CDK4 was significantly higher for non-survivors than survivors at the time of publication42. There are three FDA-approved kinase inhibitors whose main therapeutic targets are CDK4/6. Palbociclib, one of these dual CDK4/6 inhibitors, has been tested extensively in preclinical in vitro and in vivo models of chordoma43. Palbociclib treatment resulted in decreased cell growth in a dose-responsive manner for all eight chordoma cell lines tested43. In PDX and CDX models, treatment with palbociclib resulted in significant inhibition of tumor growth in 5/6 individual models (Table S1). There is currently a clinical trial enrolling in Germany (NCT03110744) to evaluate the efficacy of palbociclib against locally advanced and metastatic chordoma.

mTOR is a serine threonine kinase which is a member of two protein complexes, mTOR complex 1 (mTORC1) and complex 2 (mTORC2). By virtue of these interactions it plays key roles in a variety of cellular processes including metabolism and proliferation. It lies in the PI3K–AKT–mTOR signaling pathway, and components of this pathway are implicated in a wide range of cancers. Chordoma is no exception. Analysis of 50 chordomas showed that a high percentage of tumors were positive for p-AKT, piTSC2, p-mTOR, total mTOR, p-P70S6K, p-RPS6, p-4EBPI and eIF-4E45. These are all linked to the complicated mTOR signaling pathway. The authors suggested that a majority of chordomas may respond to mTOR inhibitors or mTOR inhibitors in combination with other drugs. A loss of PTEN (a tumor suppressor which negatively regulates this PI3K-AKT-MTOR pathway) was also observed in 16% of these chordoma cases. A study of chordoma tumors from 111 patients demonstrated that three key proteins in the mTOR pathway (p4EBP1, pS6-Ser240/244, pS6-Ser235/236) were activated in 46% of the tumors46. In a different study of just skull base chordoma, expression of PI3 and AKT pathway genes was significantly upregulated in brachyury high expression tumors. Importantly, the transcription factor brachyury (gene name TBXT), a key driver of chordoma, seems linked to this pathway47. The U-CH2 cell line is inhibited by the dual PI3k/mTOR inhibitor BEZ-235, but not the mTORC1-specific inhibitor rapamycin47. Given our observations that U-CH2 is inhibited by AZD2014, a highly specific inhibitor of mTOR which inhibits both mTORC1 and mTORC2 activity, we hypothesize that mTORC2 has a critical role in promoting chordoma proliferation or survival. An analysis of sacral chordomas demonstrated that mTOR expression levels were significantly higher than in adjacent normal tissue48. The PI3K-mTOR pathway is also upregulated in the UCH-1 chordoma cell line. A PI3K/mTOR inhibitor inhibited both AKT and MTOR activation in this cell line. The compound inhibited proliferation and induced apoptosis49. Finally, the MTOR inhibitor MLN0128 (sapanisertib) decreased activity of the PI3K-AKT-MTOR pathway in vivo in a clival chordoma PDX model50. Taken together, clearly this pathway is consistently dysregulated in chordomas, and a promising avenue for targeted therapy. Additionally, it has been observed clinically that in one patient, rapamycin, an FDA-approved mTOR inhibitor, slowed the progression of a recurrent chordoma tumor51. The utility of these various kinase inhibitors in preclinical and clinical chordoma settings implies that multiple oncogenic biological pathways may drive chordoma.

Based on these observations, we hypothesized that combinations of clinical EGFR, CDK, and mTOR inhibitors may act synergistically by dampening oncogenic signaling in multiple pathways. Using a combination of computational and in vitro approaches we have identified AZD2014 (targets mTORC1 and mTORC2), RDEA119 (targets MEK1/2), and AZD4054 (targets endothelin A receptor) as compounds which inhibit chordoma cell lines to differing extents. We also assessed several kinase drug combinations, of these, afatinib (EGFR) and palbociclib (CDK4/6) as well as afatinib and AZD2014 showed substantial synergy against U-CH1 cells in vitro. Since each of these pairs of compounds likely have kinome inhibition profiles that are distinct from one another19, co-dosing will allow us to target two key chordoma vulnerabilities simultaneously. Another advantage of working with these compounds is that pharmacokinetics data for in vivo studies is available52-55. There is a growing body of data on the effects of small molecules on the growth of chordoma cell lines20,21. The results of our studies indicate that machine learning approaches utilizing these data20,21 can help identify compounds suitable for testing in chordoma cell line models, and that we have identified combinations of inhibitors that act on different pathways important for chordoma cell growth which can behave synergistically. This work paves the way for future in vivo evaluation of CDK / mTOR inhibitor combinations in animal models of chordoma and then in the clinic. A treatment for chordoma would fill a dire unmet clinical need, and repurposing of available medicines is likely the quickest approach to approved treatments.

Methods

Compounds

Palbociclib was obtained from Apexbio (Houston TX). Afatinib, AZD0530, RDEA11, AZD4054 and AZD2014 were obtained from Selleckchem (Houston, TX). Compounds were dissolved in DMSO, before dilution in cell culture media for cell-based assays.

In vitro testing

All chordoma cell lines were cultured in Iscove's Modified Dulbecco's Medium (IMDM): Roswell Park Memorial Institute (RPMI) in a 4:1 ratio (Gibco, Life Technologies, NY, USA) and 10% fetal bovine serum (Seradigm, VWR, USA) at 37 °C, 5% CO2 in 0.1% gelatin coated flasks. There were no media antibiotics used. Cell lines were then seeded with approximately 800 cells/well in 384 well gelatin coated plates and allowed to adhere overnight before the addition of drugs. Drugs were added in quadruplicate wells using a Tecan EVO150 with 96 MCA head (Tecan Group Ltd, Switzerland). Each plate included controls for drug solvation effects from DMSO. Plates were incubated with drugs for 48hrs prior to the addition of resazurin substrate (Alamar Blue, Biosource International, Camarillo California). Plates were then incubated for a further 18hrs before reading using an Infinite F200 microplate reader with a Connect Stacker (Tecan Group Ltd). iControl software (Version 1.11) was then used to measure the fluorescence intensity of resarufin at EX535nm and EM595nm. The resulting relative fluorescence units are proportional to cellular redox activity, which is a common proxy for the quantity of living cells56.

After adjusting for the effects of DMSO in vehicle-only controls, raw fluorescence data was fitted to a four-parameter log-logistic dose response model. This model was constrained to fit a common upper asymptote for each individual experiment, and a positive lower asymptote. Each experiment was repeated for a total of 2–7 biological replicates, and estimated IC50′s and maximal effects were combined using geometric and arithmetic means, respectively.

Combination testing

For synergy experiments, drugs were tested in combination using a fixed-ratio “ray” design57. Drugs were diluted to 10 × previously estimated IC50 and combined in specific ratios (80%:20%, 50%:50% and 20%:80%). These combined drugs were serially diluted to maintain a constant ratio. Dose–response curves were fit for each mixture of drugs. Data from multiple biological replicates was combined in a mixed-effects model, using the function metadrc() in the R package medrc.58 Absolute IC50 estimates from these models were then used to calculate the combination index CI = a/A + b/B, where A and B are doses of individual drugs that produce a specified effect, and (a, b) is the pair of doses in a combination that produces the same effect59. Confidence intervals of CI were estimated using confidence limits of each IC50 estimate in the same equation. CI < 1 indicates synergy, CI = 1 no effect, and CI > 1 antagonism.

Software for machine learning

Chordoma datasets were named as Broad21 and EGFR20. These datasets consist of molecules screened against multiple cell lines and the original authors provided potency or activity classifications which were used as a binary score. These datasets underwent curation to remove problematic molecules before model building as described elsewhere10,12,22-29. We utilized Assay Central which has been previously described in detail10,12,22-29 to prepare and merge datasets collated in Molecular Notebook60, as well as generate Bayesian models using ECFP6 descriptors61,62. Briefly, the Assay Central project includes automated workflows for curating well-defined structure–activity datasets that employ a set of rules for the detection of problematic data (i.e. abnormal valences, multiple components) that can be corrected by multiple means. Data that is compatible with machine learning is then used to generate a Bayesian model for prospective bioactivity predictions. Each Bayesian model in Assay Central includes the following metrics generated from fivefold cross validation: Recall, Precision, Specificity, F1-Score, Receiver Operating Characteristic (ROC) curve, Cohen’s Kappa (CK)63,64, and the Matthews Correlation Coefficient (MCC)65. Assay Central prediction workflows assign a probability-like score61,62, with values above 0.5 considered an active prediction, and an applicability score which assesses the portion of fragments overlapping with the training set molecules to the input compounds. Predictions were applied to the specific compounds of interest from NCATS (https://web.archive.org/web/20170716163452/https:/ncats.nih.gov/ntu/assets/2017#Adult_Indications ) that were not in the training sets (AZD0530 was in the Broad dataset and was therefore not scored by the models) prior to in vitro testing as we have done previously25.

Supplementary information

Supplementary Information. (182.9KB, docx)

Acknowledgements

We kindly acknowledge the support of Dr. Joan Levy with the Chordoma Foundation. SE kindly acknowledges NIH funding R44GM122196-02A1 from NIH NIGMS and FY2018 UNC Research Opportunities Initiative (ROI) Award. The SGC (DHD) is a registered charity (number 1097737) that receives funds from AbbVie, Bayer Pharma AG, Boehringer Ingelheim, Canada Foundation for Innovation, Eshelman Institute for Innovation, Genome Canada, Innovative Medicines Initiative (EU/EFPIA) [ULTRA-DD grant no. 115766], Janssen, Merck KGaA Darmstadt Germany, MSD, Novartis Pharma AG, Ontario Ministry of Economic Development and Innovation, Pfizer, Takeda, and Wellcome [106169/ZZ14/Z]. The authors acknowledge the generous support provided by the Eshelman Institute of Innovation and the Bryson Foundation at the UNC Eshelman School of Pharmacy.” Dr. Alex Clark is acknowledged for assistance with Assay Central.

Author contributions

E.A and T.H. performed analyses of compounds in cell lines. K.M.Z. and D.H. F performed machine learning analyses. T.L. assisted with analyses. A.J.H. and S.E. directed the research, E.A., S.J.C., D.M., A.J.H. D.H.D. and S.E. wrote the manuscript.

Competing interests

T.L., K.M.Z., D.H.F. and S.E. are employees of Collaborations Pharmaceuticals, Inc. DHD is on the Board of Directors and Scientific Advisory Board of the Chordoma Foundation. All other authors do not have competing interests.

Footnotes

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

is available for this paper at 10.1038/s41598-020-70026-w.

References

  • 1.Walcott BP, Nahed BV, Mohyeldin A, Coumans JV, Kahle KT, Ferreira MJ. Chordoma: current concepts, management, and future directions. Lancet Oncol. 2012;13:e69–76. doi: 10.1016/S1470-2045(11)70337-0. [DOI] [PubMed] [Google Scholar]
  • 2.Stacchiotti S, Sommer J, Chordoma Global Consensus G Building a global consensus approach to chordoma: a position paper from the medical and patient community. Lancet Oncol. 2015;16:71–83. doi: 10.1016/S1470-2045(14)71190-8. [DOI] [PubMed] [Google Scholar]
  • 3.Sciubba DM, Cheng JJ, Petteys RJ, Weber KL, Frassica DA, Gokaslan ZL. Chordoma of the sacrum and vertebral bodies. J. Am. Acad. Orthop. Surg. 2009;17:708–717. doi: 10.5435/00124635-200911000-00005. [DOI] [PubMed] [Google Scholar]
  • 4.Yang XR, et al. T (brachyury) gene duplication confers major susceptibility to familial chordoma. Nat. Genet. 2009;41:1176–1178. doi: 10.1038/ng.454. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Wu Z, et al. The brachyury Gly177Asp SNP is not associated with a risk of skull base chordoma in the Chinese population. Int. J. Mol. Sci. 2013;14:21258–21265. doi: 10.3390/ijms141121258. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Pillay N, et al. A common single-nucleotide variant in T is strongly associated with chordoma. Nat. Genet. 2012;44:1185–1187. doi: 10.1038/ng.2419. [DOI] [PubMed] [Google Scholar]
  • 7.Dahl NA, et al. Chordoma occurs in young children with tuberous sclerosis. J. Neuropathol. Exp. Neurol. 2017;76:418–423. doi: 10.1093/jnen/nlx032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.de Castro CV, et al. Tyrosine kinase receptor expression in chordomas: phosphorylated AKT correlates inversely with outcome. Hum. Pathol. 2013;44:1747–1755. doi: 10.1016/j.humpath.2012.11.024. [DOI] [PubMed] [Google Scholar]
  • 9.Magnaghi P, et al. Afatinib Is a new therapeutic approach in chordoma with a unique ability to target EGFR and brachyury. Mol. Cancer Ther. 2018;17:603–613. doi: 10.1158/1535-7163.MCT-17-0324. [DOI] [PubMed] [Google Scholar]
  • 10.Ekins S, Gerlach J, Zorn KM, Antonio BM, Lin Z, Gerlach A. Repurposing approved drugs as inhibitors of Kv7.1 and Nav1.8 to treat Pitt Hopkins syndrome. Pharm Res. 2019;36:137. doi: 10.1007/s11095-019-2671-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Lane TR, Comer JE, Freiberg AN, Madrid PB, Ekins S. Repurposing quinacrine against ebola virus infection in vivo. Antimicrob. Agents Chemother. 2019;63:e01142-19. doi: 10.1128/AAC.01142-19. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Hernandez HW, et al. High throughput and computational repurposing for neglected diseases. Pharm Res. 2018;36:27. doi: 10.1007/s11095-018-2558-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Baker NC, Ekins S, Williams AJ, Tropsha A. A bibliometric review of drug repurposing. Drug Discov. Today. 2018;23:661–672. doi: 10.1016/j.drudis.2018.01.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Southan C, Williams AJ, Ekins S. Challenges and recommendations for obtaining chemical structures of industry-provided repurposing candidates. Drug Discov. Today. 2013;18:58–70. doi: 10.1016/j.drudis.2012.11.005. [DOI] [PubMed] [Google Scholar]
  • 15.Ekins S, Freundlich JS, Clark AM, Anantpadma M, Davey RA, Madrid P. Machine learning models identify molecules active against the Ebola virus in vitro. F1000Res. 2015;4:1091. doi: 10.12688/f1000research.7217.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Ekins S, et al. Machine learning models and pathway genome data base for trypanosoma cruzi drug discovery. PLoS Negl. Trop. Dis. 2015;9:e0003878. doi: 10.1371/journal.pntd.0003878. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Liu M, Liu H, Chen J. Mechanisms of the CDK4/6 inhibitor palbociclib (PD 0332991) and its future application in cancer treatment (review) Oncol. Rep. 2018;39:901–911. doi: 10.3892/or.2018.6221. [DOI] [PubMed] [Google Scholar]
  • 18.Keating GM. Afatinib: a review in advanced non-small cell lung cancer. Target Oncol. 2016;11:825–835. doi: 10.1007/s11523-016-0465-2. [DOI] [PubMed] [Google Scholar]
  • 19.Klaeger S, et al. The target landscape of clinical kinase drugs. Science. 2017;358:eaan4368. doi: 10.1126/science.aan4368. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Scheipl S, et al. EGFR inhibitors identified as a potential treatment for chordoma in a focused compound screen. J. Pathol. 2016;239:320–334. doi: 10.1002/path.4729. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Sharifnia T, et al. Small-molecule targeting of brachyury transcription factor addiction in chordoma. Nat. Med. 2019;25:292–300. doi: 10.1038/s41591-018-0312-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Sandoval PJ, Zorn KM, Clark AM, Ekins S, Wright SH. Assessment of substrate-dependent ligand interactions at the organic cation transporter OCT2 using six model substrates. Mol. Pharmacol. 2018;94:1057–1068. doi: 10.1124/mol.117.111443. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Lane T, et al. Comparing and validating machine learning models for mycobacterium tuberculosis drug discovery. Mol. Pharm. 2018;15:4346–4360. doi: 10.1021/acs.molpharmaceut.8b00083. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Russo DP, Zorn KM, Clark AM, Zhu H, Ekins S. Comparing multiple machine learning algorithms and metrics for estrogen receptor binding prediction. Mol. Pharm. 2018;15:4361–4370. doi: 10.1021/acs.molpharmaceut.8b00546. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Anantpadma M, et al. Ebola virus bayesian machine learning models enable new in vitro leads. ACS Omega. 2019;4:2353–2361. doi: 10.1021/acsomega.8b02948. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Ekins S, et al. Exploiting machine learning for end-to-end drug discovery and development. Nat. Mater. 2019;18:435–441. doi: 10.1038/s41563-019-0338-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Wang PF, Neiner A, Lane TR, Zorn KM, Ekins S, Kharasch ED. Halogen substitution influences ketamine metabolism by cytochrome P450 2B6: in vitro and computational approaches. Mol. Pharm. 2019;16:898–906. doi: 10.1021/acs.molpharmaceut.8b01214. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Dalecki AG, et al. High-throughput screening and Bayesian machine learning for copper-dependent inhibitors of Staphylococcus aureus. Metallomics. 2019;11:696–706. doi: 10.1039/c8mt00342d. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Zorn KM, Lane TR, Russo DP, Clark AM, Makarov V, Ekins S. Multiple machine learning comparisons of HIV cell-based and reverse transcriptase data sets. Mol. Pharm. 2019;16:1620–1632. doi: 10.1021/acs.molpharmaceut.8b01297. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Wieduwilt MJ, Moasser MM. The epidermal growth factor receptor family: biology driving targeted therapeutics. Cell Mol. Life Sci. 2008;65:1566–1584. doi: 10.1007/s00018-008-7440-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Wee P, Wang Z. Epidermal growth factor receptor cell proliferation signaling pathways. Cancers (Basel) 2017;9:52. doi: 10.3390/cancers9050052. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Scheil S, et al. Genome-wide analysis of sixteen chordomas by comparative genomic hybridization and cytogenetics of the first human chordoma cell line, U-CH1. Genes Chromosomes Cancer. 2001;32:203–211. doi: 10.1002/gcc.1184. [DOI] [PubMed] [Google Scholar]
  • 33.Anon. Chordoma Foundation - Drug Screening Program. (2018).
  • 34.Siu IM, et al. Erlotinib inhibits growth of a patient-derived chordoma xenograft. PLoS ONE. 2013;8:e78895. doi: 10.1371/journal.pone.0078895. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Lebellec L, et al. Advanced chordoma treated by first-line molecular targeted therapies: Outcomes and prognostic factors. A retrospective study of the French Sarcoma Group (GSF/GETO) and the Association des Neuro-Oncologues d'Expression Francaise (ANOCEF) Eur. J. Cancer. 2017;79:119–128. doi: 10.1016/j.ejca.2017.03.037. [DOI] [PubMed] [Google Scholar]
  • 36.Sherr CJ, Beach D, Shapiro GI. Targeting CDK4 and CDK6: from discovery to therapy. Cancer Discov. 2016;6:353–367. doi: 10.1158/2159-8290.CD-15-0894. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Horbinski C, et al. The prognostic value of Ki-67, p53, epidermal growth factor receptor, 1p36, 9p21, 10q23, and 17p13 in skull base chordomas. Arch. Pathol. Lab Med. 2010;134:1170–1176. doi: 10.1043/2009-0380-OA.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.O'Leary B, Finn RS, Turner NC. Treating cancer with selective CDK4/6 inhibitors. Nat. Rev. Clin. Oncol. 2016;13:417–430. doi: 10.1038/nrclinonc.2016.26. [DOI] [PubMed] [Google Scholar]
  • 39.Asghar U, Witkiewicz AK, Turner NC, Knudsen ES. The history and future of targeting cyclin-dependent kinases in cancer therapy. Nat. Rev. Drug Discov. 2015;14:130–146. doi: 10.1038/nrd4504. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Kim S, et al. The potent and selective cyclin-dependent kinases 4 and 6 inhibitor ribociclib (LEE011) is a versatile combination partner in preclinical cancer models. Oncotarget. 2018;9:35226–35240. doi: 10.18632/oncotarget.26215. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Liu M, et al. PD 0332991, a selective cyclin D kinase 4/6 inhibitor, sensitizes lung cancer cells to treatment with epidermal growth factor receptor tyrosine kinase inhibitors. Oncotarget. 2016;7:84951–84964. doi: 10.18632/oncotarget.13069. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Liu T, et al. CDK4 expression in chordoma: A potential therapeutic target. J. Orthop. Res. 2018;36:1581–1589. doi: 10.1002/jor.23819. [DOI] [PubMed] [Google Scholar]
  • 43.von Witzleben A, et al. Preclinical characterization of novel chordoma cell systems and their targeting by pharmocological inhibitors of the CDK4/6 cell-cycle pathway. Cancer Res. 2015;75:3823–3831. doi: 10.1158/0008-5472.CAN-14-3270. [DOI] [PubMed] [Google Scholar]
  • 44.Yakkioui Y, et al. A comparison of cell-cycle markers in skull base and sacral chordomas. World Neurosurg. 2014;82:e311–318. doi: 10.1016/j.wneu.2013.01.131. [DOI] [PubMed] [Google Scholar]
  • 45.Presneau N, et al. Potential therapeutic targets for chordoma: PI3K/AKT/TSC1/TSC2/mTOR pathway. Br. J. Cancer. 2009;100:1406–1414. doi: 10.1038/sj.bjc.6605019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Tauziede-Espariat A, et al. Prognostic and therapeutic markers in chordomas: a study of 287 tumors. J. Neuropathol. Exp. Neurol. 2016;75:111–120. doi: 10.1093/jnen/nlv010. [DOI] [PubMed] [Google Scholar]
  • 47.Otani R, et al. Brachyury gene copy number gain and activation of the PI3K/Akt pathway: association with upregulation of oncogenic Brachyury expression in skull base chordoma. J. Neurosurg. 2018;128:1428–1437. doi: 10.3171/2016.12.JNS161444. [DOI] [PubMed] [Google Scholar]
  • 48.Chen K, et al. Expression of PTEN and mTOR in sacral chordoma and association with poor prognosis. Med. Oncol. 2014;31:886. doi: 10.1007/s12032-014-0886-7. [DOI] [PubMed] [Google Scholar]
  • 49.Schwab J, et al. Combination of PI3K/mTOR inhibition demonstrates efficacy in human chordoma. Anticancer Res. 2009;29:1867–1871. [PubMed] [Google Scholar]
  • 50.Davies JM, et al. Generation of a patient-derived chordoma xenograft and characterization of the phosphoproteome in a recurrent chordoma. J. Neurosurg. 2014;120:331–336. doi: 10.3171/2013.10.JNS13598. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Ricci-Vitiani L, et al. Chemotherapy of skull base chordoma tailored on responsiveness of patient-derived tumor cells to rapamycin. Neoplasia. 2013;15:773–782. doi: 10.1593/neo.13150. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Wind S, Schnell D, Ebner T, Freiwald M, Stopfer P. Clinical pharmacokinetics and pharmacodynamics of afatinib. Clin. Pharmacokinet. 2017;56:235–250. doi: 10.1007/s40262-016-0440-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.De Luca A, et al. Pharmacokinetic drug evaluation of palbociclib for the treatment of breast cancer. Expert Opin. Drug Metab. Toxicol. 2018;14:891–900. doi: 10.1080/17425255.2018.1514720. [DOI] [PubMed] [Google Scholar]
  • 54.Basu B, et al. First-in-human pharmacokinetic and pharmacodynamic study of the dual m-TORC 1/2 Inhibitor AZD2014. Clin. Cancer Res. 2015;21:3412–3419. doi: 10.1158/1078-0432.CCR-14-2422. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.MacDonald A, et al. Phase I study of orally administered (14)Carbon-isotope labelled-vistusertib (AZD2014), a dual TORC1/2 kinase inhibitor, to assess the absorption, metabolism, excretion, and pharmacokinetics in patients with advanced solid malignancies. Cancer Chemother. Pharmacol. 2019;83:787–795. doi: 10.1007/s00280-019-03781-x. [DOI] [PubMed] [Google Scholar]
  • 56.Jack J, Havener TM, McLeod HL, Motsinger-Reif AA, Foster M. Evaluating the role of admixture in cancer therapy via in vitro drug response and multivariate genome-wide associations. Pharmacogenomics. 2015;16:1451–1463. doi: 10.2217/pgs.15.85. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Gennings C, Carter WH, Jr, Carney EW, Charles GD, Gollapudi BB, Carchman RA. A novel flexible approach for evaluating fixed ratio mixtures of full and partial agonists. Toxicol. Sci. 2004;80:134–150. doi: 10.1093/toxsci/kfh134. [DOI] [PubMed] [Google Scholar]
  • 58.Ritz C, Jensen SM, Gerhard D, Streibig JC. Dose-Response Analysis Using R. 1. Cambridge: Chapman and Hall/CRC; 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Foucquier J, Guedj M. Analysis of drug combinations: current methodological landscape. Pharmacol Res Perspect. 2015;3:e00149. doi: 10.1002/prp2.149. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Clark AM. Molecular Notebook. http://molecularnotebook.com (2018).
  • 61.Clark AM, et al. Open source bayesian models: 1. Application to ADME/Tox and drug discovery datasets. J. Chem. Inf. Model. 2015;55:1231–1245. doi: 10.1021/acs.jcim.5b00143. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Clark AM, Ekins S. Open Source Bayesian Models: 2. Mining A "big dataset" to create and validate models with ChEMBL. J. Chem. Inf. Model. 2015;55:1246–1260. doi: 10.1021/acs.jcim.5b00144. [DOI] [PubMed] [Google Scholar]
  • 63.Carletta J. Assessing agreement on classification tasks: The kappa statistic. Comput. Linguist. 1996;22:249–254. [Google Scholar]
  • 64.Cohen J. A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 1960;20:37–46. [Google Scholar]
  • 65.Matthews BW. Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochim. Biophys. Acta. 1975;405:442–451. doi: 10.1016/0005-2795(75)90109-9. [DOI] [PubMed] [Google Scholar]
  • 66.Team RC . R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing; 2019. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Information. (182.9KB, docx)

Articles from Scientific Reports are provided here courtesy of Nature Publishing Group

RESOURCES