Abstract
Background
Following the formation of a primary carcinoma, neoplastic cells metastasize by undergoing the epithelial mesenchymal transition (EMT), which is triggered by cues from inflammatory and stromal cells in the microenvironment. EMT allows epithelial cells to lose their highly adhesive nature and instead adopt the spindle-like appearance, as well as the invasive and migratory behavior, of mesenchymal cells. We hypothesize that a bistable switch between the epithelial and mesenchymal phenotypes governs EMT, allowing the cell to maintain its mesenchymal phenotype even after it leaves the primary tumor microenvironment and EMT-inducing extracellular signal.
Results
This work presents a simple mathematical model of EMT, specifically the roles played by four key proteins in the Wnt signaling pathway: Dishevelled (Dvl), E-cadherin, β-catenin, and Slug. The model predicts that following activation of the Wnt pathway, an epithelial cell in the primary carcinoma must attain a threshold level of membrane-bound Dvl to convert to the mesenchymal-like phenotype and maintain that phenotype once it has migrated away from the primary tumor. Furthermore, sensitivity analysis of the model suggests that in both the epithelial and the mesenchymal states, the steady state behavior of E-cadherin and the transcription factor Slug are sensitive to changes in the degradation rate of Slug, while E-cadherin is also sensitive to the IC50 (half-maximal) concentration of Slug necessary to inhibit E-cadherin production. The steady state behavior of Slug exhibits sensitivity to changes in the rate at which it is induced by β-catenin upon activation of the Wnt pathway. In the presence of sufficient amount of Wnt ligand, E-cadherin levels are sensitive to the ratio of the rate of Slug activation via β-catenin to the IC50 concentration of Slug necessary to inhibit E-cadherin production.
Conclusions
The sensitivity of E-cadherin to the degradation rate of Slug, as well as the IC50 concentration of Slug necessary to inhibit E-cadherin production, shows how the adhesive nature of the cell depends on finely-tuned regulation of Slug. By highlighting the role of β-catenin in the activation of EMT and the relationship between E-cadherin and Slug, this model identifies critical parameters of therapeutic concern, such as the threshold level of Dvl necessary to inactivate the GSK-3β complex mediating β-catenin degradation, the rate at which β-catenin translocates to the nucleus, and the IC50 concentration of Slug needed to inhibit E-cadherin production.
Electronic supplementary material
The online version of this article (10.1186/s12976-017-0064-7) contains supplementary material, which is available to authorized users.
Keywords: Epithelial Mesenchymal Transition (EMT), Wnt, Bistable
Background
Epithelial tumors, or carcinomas, are the most common type of neoplasia found in humans, accounting for roughly 80% of all cancer related deaths in the Western world [1]. Characteristically, epithelial cells form polarized sheets. Within these sheets, cells are tightly adhered to each other to inhibit individual cellular movement [2]. This highly adhesive nature is attributable to molecules of E-cadherin, a transmembrane protein, on the surface of one epithelial cell binding with the E-cadherin moelcules of another epithelial cell across the extracellular space. Simultaneously, E-cadherin is bound to intracellular catenin members on its cytoplasmic tail, which stabilize the E-cadherin by linking it to the actin cytoskeleton [3, 4]. This E-cadherin-catenin complex is what gives the epithelial cells in both normal tissue and carcinomas their strong adhesive bonds and inhibits the movement of individual cells [2, 4].
In carcinomas, it is possible for neoplastic cells to undergo epithelial mesenchymal transition (EMT) [3]. Triggered by cues from the local stroma [1], EMT influences the expression of the cellular adhesion complex [5] and allows epithelial cells to acquire the spindle-like appearance of mesenchymal cells, as well as their invasive and migratory properties [5, 6]. While this process plays a key role in embryogenesis [7] and wound healing [5], the occurrence of EMT in tumor cells is thought to help drive the onset of metastasis, although the level to which EMT is involved in this process is currently debated due to the heterogeneous nature of cellular responses that can occur in the same tumor [5]. However, for some cells within a tumor, with the loss of adhesion and the acquisition of an invasive phenotype, they are able to invade the extracellular matrix around the tumor, penetrate the basement membrane of the blood vessel, and travel via the blood stream to other locations in the body where they ultimately form a metastasis [8].
There are a multitude of different pathways that can activate EMT, with significant crosstalk between these signaling systems. In mature multicellular organisms, as well as during development, the Wnt pathway helps regulate proliferation, differentiation, migration, and polarity [9, 10]. However, mutations and dysregulation of the pathway can ultimately lead to the occurrence of tumors and the activation of EMT leading to metastasis [9, 11].
The Wnt pathway has long been the focus of studies attempting to understand cellular behavior. Lee et al. [12] proposed a 15-equation mathematical model centered upon the formation of the protein complexes involved in β-catenin phosphorylation and degradation, a process that, when disrupted, leads to the accumulation of the β-catenin/ TCF complex in the nucleus. In particular, the authors attempted to explain the distinct roles that different members of the degradation complex can carry out. In addition, these authors carried out experiments in Xenopus egg extracts to support their hypothesized interactions [12]. Ramis-Conde et al. [13] built on the work of Lee and others with a multiscale mathematical model that examined the involvement of E-cadherin and β-catenin in the adhesion of epithelial cells to one another. Like Lee et al., Ramis-Conde et al. included the involvement of β-catenin in the degradation complex in the cytosol. However, Ramis-Conde et al. also focused on the behavior of E-cadherin. The authors considered the concentration of E-cadherin to be a constant that is then divided by sub-cellular localization: free E-cadherin at the membrane, free E-cadherin in the cytosol, and the E-cadherin-β-catenin membrane complex [13].
Basan et al. [14] also explored the adhesion complex relationship. However, where Ramis-Conde et al. considered the key component of cellular adhesion to be the binding of β-catenin and E-cadherin, which they modeled using ordinary differential equations (ODEs), Basan et al. examined the relationship between β-catenin and E-cadherin and the involvement of α-catenin in the adhesion complex by using reaction-diffusion equations [14]. The dynamics that exist between the adhesion complex and the Wnt pathway were explored further by Chen et al. [11]. In particular, the authors sought to understand the role spatial dynamics play in intracellular signaling. By employing a network model, Chen et al. found that the membrane adhesion complex not only competes with the degradation complex for free β-catenin, but also that the clustering of adhesion complexes can slow down the degradation complex and thus allow for an increase in nuclear signaling.
Other authors have continued to explore the influence noncanonical proteins can have on the dynamics of Wnt signaling, as well as its crosstalk with other pathways. One protein of interest has been Dickkopf1 (Dkk), which inhibits the Wnt pathway and cell proliferation, as well as promotes maturation in cells. By creating a multi-scale model to describe the relationship between the Wnt and Notch pathways, Agur et al. suggested that high levels of Dkk can push stem cells towards differentiation [15]. Kogan et al. also used mathematical modeling to examine the influence of Dkk on the Wnt pathway, but these authors examined its influence in combination with a secreted Frizzled-related protein (sFRP), another known Wnt inhibitor [9]. By proposing an ODE model with 13 variables and employing mass action kinetics, Kogan et al. suggested that the combination of sFRP1 and Dkk1 can work together to inhibit the accumulation of β-catenin in the cell, a direct outcome of the canonical Wnt signaling pathway.
Kim et al. highlighted the positive feedback loop that exists between the Wnt and ERK signaling pathways [16]. By extending the original model proposed by Lee et al. [12] and incorporating the influence of the ERK pathway, Kim et al. suggest that the crosstalk and positive feedback loop that exists between Wnt and ERK can lead to a bistable switch in cellular behavior. Shin et al. [17] found similar results to Kim et al. during their exploration of Wnt and ERK pathway crosstalk. Based on a system of six ODEs, the model proposed by the authors included the involvement of the adhesion complex, the complex responsible for β-catenin degradation, as well as the shuttling of β-catenin to the nucleus and its involvement with the transcription factor Slug. They also examined the changes in E-cadherin in response to different oncogenic stimuli: EGF and Wnt, and proposed a switch-like behavior in E-cadherin, allowing the cell to transition from the epithelial to the mesenchymal state [17]. The existence of this bistable switch was also discussed by Maclean et al. [18] who put forth a 19-equation model of the bistable switch between the epithelial and mesenchymal steady states due to cytoplasmic-nuclear shuttling of β-catenin.
It is worthwhile to note that the concept of a bistable switch is not unique to systems involving the activation of or the overlap with the Wnt signaling pathway. Recent work has examined the roles that the miR-200/ZEB, the LIN28/let-7, and the Ovol2-Zeb1 circuits may play in the activation of EMT, as well as possible overlap that may exist [19–21]. However, rather than an all or nothing switch, each of these proposed models accounts for at least one intermediate state between the epithelial and mesenchymal transitions, a state that Jolly et al. suggest is responsible for stemness and has the highest potential for tumor initiation [19, 20].
Due to the complex nature of the Wnt signaling pathway and intracellular signals as a whole, models are often quite intricate, featuring large sets of equations, and most only focus on a few of the molecular interactions β-catenin is involved in. Even models that sought to use minimal equations and variables to describe the Wnt signaling pathway, such as the work done by Benary et al., still resulted in the use of 5–6 differential equations and two conservation equations [22]. In this work, we, like Kim et al., Shin et al., and MacLean et al., hypothesize that the mechanism underlying EMT in a primary solid tumor is a bistable switch between the epithelial and the mesenchymal phenotypes: once transitioned, the cell will maintain its mesenchymal phenotype, even in the absence of sustained extracellular signaling. With regards to the Wnt pathway, we hypothesize that the bistable switch is centered around the behavior and interactions of β-catenin. Unlike many of the previous works mentioned, the model put forth here incorporates all three primary interactions β-catenin is involved in: the adhesion complex with E-cadherin, its degradation via the complex formed by GSK-3β, and its translocation to the nucleus to activate the transcription factor Slug. Additionally, even though we are examining all three relationships involved in the switch-like behavior of the cell, we propose a simplified theoretical model of three ODEs to describe the ultrasensitive feedback loop in the Wnt-β-catenin signaling pathway. By creating a simple model of the primary interactions of β-catenin in the Wnt pathway, we are able to show how activation of the Wnt pathway alone can drive EMT in a carcinoma cell, as well as explain the intracellular β-catenin interactions that may be causing the bistable switch. Our model could provide a scaffold upon which other, more complex models of EMT and the multistable switch could be built, as well as guide experimental exploration into which interactions could be targeted for prevention of EMT and metastasis.
Methods
In a Wnt-absent environment, E-cadherin binds with intracellular catenin members, such as α, β, and γ- catenin [4]. Any free β-catenin that is not bound in this complex is degraded in the cytosol by a protein complex [23]. Axin, a scaffold protein that is present in epithelial cells [24], mediates the formation of this cytosolic degradation complex along with the proteins adenomatous polyposis coli (APC), glycogen synthase kinase 3β (GSK-3β), casein kinase 1 (CK1), and protein phosphatase 2A (PP2A) [25]. When β-catenin comes in contact with Axin and APC, the casein kinase 1α (CK1α) component of Axin and GSK-3β work together to phosphorylate β-catenin and mark it for degradation [23]. However, there currently exists some controversy around the degradation of β-catenin, as the process by which it occurs is poorly understood. Experimental work suggests that it may not be necessary for the complex to release β-catenin for degradation to occur. Rather, it may be possible for the degradation complex to undergo a structural change that exposes the β-catenin and allows it to be degraded [11]. Regardless of the exact mechanisms of the process, it is through this constant degradation that the cytosolic β-catenin is maintained at a low concentration in the absence of Wnt [23].
Carcinomas are associated with an oncogenic promoting stroma that is considered ‘reactive’, and is characterized, in part, by an increase in the number of fibroblasts that now associate with the ECM [26, 27]. These fibroblasts secrete the Wnt protein, a morphogen that relies on short-range signaling in order to activate the Wnt pathway in carcinoma cells [1, 28]. Once the Wnt ligand binds to its receptor, Frizzled (Frz) [12], on epithelial cells, LRP5-LRP6 receptors are phosphorylated by CK1γ and GSK-3β [23]. The messenger protein Disheveled (Dvl) is brought to the plasma membrane where it associates with Frz. Dvl then binds with Axin at the membrane to deactivate and dismantle the Axin/APC/GSK-3β complex, allowing β-catenin to accumulate in the cytosol and subsequently translocate to the nucleus [25]. In the absence of Wnt signaling, the T-cell factor and lymphoid enhancer factor (TCF/LEF) works in the nucleus with Groucho to repress Wnt target genes [29]. However, once Wnt signaling is activated, β-catenin replaces the Groucho factors and is able to form a complex with TCF and LEF1 [23], upregulating the expression of transcription factors such as Slug [17]. Slug, a zinc-finger transcription factor, then binds to E-boxes in the promoter regions of the E-cadherin gene and prevents transcription of E-cadherin [30]. This suppression of E-cadherin production limits the concentration of E-cadherin available to bind with β-catenin, further encouraging the translocation of β-catenin to the nucleus as it accumulates in the cytosol [31], thus creating a Wnt-driven feedback loop. The continued suppression of E-cadherin production by Slug and other transcription factors ultimately causes the epithelial cell to lose its adhesion with the other surrounding epithelial cells and undergo EMT.
Within these complex dynamics, we identify 3 key relationships centered around β-catenin, which are shown in Fig. 1. In a primary carcinoma cell, β-catenin is sequestered into a complex with E-cadherin at the membrane. Any β-catenin that is not in this complex is marked for degradation. With the activation of the Wnt pathway, the degradation complex is inactivated due to the movement of Dvl to the cellular membrane, allowing β-catenin to accumulate, translocate to the nucleus, and upregulate Slug. In order to model these β-catenin relationships at the core of the canonical Wnt pathway and the manner in which they affect the key proteins involved, we employed a system of three ODEs. The changes over time in the concentration of membrane-bound E-cadherin (E), free cytosolic β-catenin (B), and Slug (S) are described in eqs. (1)–(3) below. A complete list of the parameters and their definitions is provided in Table 1.
Table 1.
Parameter | Definition | Units | Assumed Value | Source |
---|---|---|---|---|
α1 | Basal production of E-cadherin | 0.01 | estimated | |
α2 | Basal production of β-catenin | 0.01 | [13] | |
α3 | Basal production of Slug | 0.001 | estimated | |
β1 | Basal degradation of E-cadherin | 0.03 | estimated | |
β2 | Rate at which β-catenin binds to the GSK-3β / Axin/APC complex | 0.03 | estimated | |
β3 | Basal degradation of Slug | 0.03 | estimated | |
k 0 | Rate at which Dvl inactivates the GSK-3β/Axin/ APC complex | 3.7 ∙ 10−3 | estimated | |
k 1 | Rate at which E-cadherin sequesters β-catenin at the cell membrane | 0.01 | estimated | |
k 2 | Rate at which β-catenin upregulates Slug in the cell nucleus | 1 | estimated | |
IC S | Half maximal concentration of Slug required to inhibit E-cadherin transcription | nM | 3.3 | estimated |
IC B | Half maximal concentration of β-catenin needed up upregulate Slug | nM | 0.33 | estimated |
IC E | Half maximal concentration of E-cadherin needed for sequestration of β-catenin at membrane | nM | 0.033 | estimated |
IC D | Half maximal concentration of Dvl needed to inhibit the degradation of β-catenin by GSK-3β | nM | 0.67 | estimated |
n 1 | Hill Coefficient | – | 3 | estimated |
n 2 | Hill Coefficient | – | 2 | estimated |
n 3 | Hill Coefficient | – | 2 | estimated |
n 4 | Hill Coefficient | – | 5 | estimated |
1 |
2 |
3 |
In the epithelial steady state, the concentration of cytosolic β-catenin is very low: most β-catenin exists in complex form with other members of the catenin family and E-cadherin at the membrane. This relationship is represented in (2) with the term , where k 1 is the rate at which membrane-bound E-cadherin sequesters β-catenin. In the model presented by Ramis-Conde et al. [13], the authors discuss two rates to describe this interaction: the rates at which β-catenin and E-cadherin bind and unbind to form this complex. However, the authors also estimate that the rate at which the two proteins bind to form the complex is 50× faster than the rate at which they unbind; hence we have used a single term to model the reduction in free cytosolic β-catenin due to the presence of E-cadherin.
Any β-catenin that is not bound to E-cadherin is subjected to phosphorylation by the GSK-3β complex and thus marked for degradation. In the model put forth by Kogan et al. [9], the authors condense the complex process of phosphorylation degradation by assuming that once β-catenin is phosphorylated, it is quickly degraded, a process they represent with a single term. Coupled with this term is another term to represent the disassociation of β-catenin from the degradation complex prior to phosphorylation. However, in the model created by Kogan et al., the rate at which β-catenin binds with the degradation complex is 108x faster than the rate at which it escapes. Therefore, for simplicity, in this work, we assume that once β-catenin is bound to the GSK-3β complex in the epithelial steady state, it will be quickly phosphorylated and marked for degradation before it has the chance to unbind and exist freely in the cytosol. Thus, the basal degradation of β-catenin is represented by the term β 2 ∙ B in (2), where β 2 is the rate at which β-catenin binds to the GSK-3β complex. Once Wnt signaling is activated, Dvl works to break apart the GSK-3β complex by binding with Axin at the membrane, sparing β-catenin from phosphorylation and degradation. With Dvl (D) > 0, the term in (2) where k 0 is the rate at which Dvl binds to Axin and deactivates the degradation complex.
In this model, it is assumed that the cell has the ability to shuttle free β-catenin to the nucleus, as is represented by the term in (3), where k 2 is the rate at which β-catenin translocates to the nucleus and activates Slug. If free β-catenin in the cytosol is kept at a low value by the degradation complex, there is very little β-catenin available for transport. Once Wnt signaling is turned on and Dvl is upregulated, β-catenin can accumulate and translocate to the nucleus, enhancing the expression of Slug in (3). Slug then binds to E-box elements in the promoter of the E-cadherin gene, which inhibits E-cadherin production, as modeled in (1) with the term . These interactions constitute the core of the double-negative feedback loop driven by changes in the behavior of β-catenin. This double-negative feedback loop, combined with ultra-sensitivity in the interactions (represented by Hill coefficients n 1 − n 4 in (1)–(3)), creates the potential for a bistable switch.
Due to the number of parameters we were required to estimate for this model, as well as to continue our theoretical exploration into how these rates contribute to the underlying bistable switch, we nondimensionalized the system in (1)–(3) using the relationships
where e, b, s, d, and τ are dimensionless variables. Substituting these relationships into (1)–(3) the constants ε, λ, σ, and δ were defined as ε = IC E, λ = IC B, σ = IC S, and δ = IC D. For the dimensional constant T, there were many parameter combinations that would result in the appropriate units of minutes. Ultimately, T was chosen to be the ratio of half the maximal concentration of β-catenin necessary to activate Slug to the rate at which E-cadherin sequesters β-catenin at the membrane, or . This ratio was chosen to reflect the time the cell would need to build up half the amount of β-catenin necessary to activate Slug, and begin the transition from the epithelial to the mesenchymal phenotype, while the pool of free β-catenin is depleted due to sequestration by E-cadherin. By using these substitutions, nondimensionalization allows for simplification and parameter grouping for further analysis. Our new nondimensional system is:
4 |
5 |
6 |
While a range of values for each dimensionless parameter was explored (see Fig. 5), the values that were ultimately used in this model are defined in Table 2. Analysis was carried out using MATLAB software and XPPAUT [32] using the initial conditions of e = 10, b = 0, and s = 0 when the cell begins in the epithelial steady state.
Table 2.
Parameter | Definition | Value |
---|---|---|
A 1 | 10 | |
A 2 | 1 | |
A 3 | 0.01 | |
C 1 | 1 | |
C 2 | 1 | |
C 3 | 1 | |
F 1 | 0.37 | |
F 2 | 10 | |
n 1 | -- | 3 |
n 2 | -- | 2 |
n 3 | -- | 2 |
n 4 | -- | 5 |
Results and Discussion
A Bistable Switch between the Epithelial and Mesenchymal Phenotypes
The qualitative response of the epithelial cell to different levels of Wnt signaling is described in Fig. 2, which shows the changes in protein concentrations with respect to time. The values for each variable were included for reproducibility and the results are shown with the equivalent dimensional concentration, followed by the nondimensional parameter value. In both Fig. 2a and b, the cell exists in the epithelial steady state over the time range τ = [0, 20): E-cadherin level is high (0.33 nM, e = 10) whereas both β-catenin (b) and Slug (s) are low (< 0.05 nM, b, s < 0.015). At time τ = 20, Dvl is turned on, which implicitly models the release of Wnt protein from the surrounding environment and consequent movement of Dvl from the cytosol to the membrane. If Wnt signaling is very low, membrane Dvl will not be sufficiently high to induce EMT. This is illustrated in Fig. 2a (d = 1.2). The intracellular protein levels of β-catenin and Slug both increase (~0.1 nM, b ≈ 0.3; ~2.64 nM, s ≈ 0.8), but this results in only a dip in the membrane bound E-cadherin (~0.22 nM, e = 6.7), which allows the cell to maintain its adhesive nature and epithelial phenotype. Additionally, if the microenvironment were to then stop releasing the Wnt signal at time τ = 50, Dvl would detach from the membrane and return to the cytosol, meaning that the (nondimensional) concentration of membrane Dvl would return to its initial value of 0 (this is modeled by setting d = 0 at time τ = 50). Because the epithelial steady state was maintained (even with ~0.8 nM, d = 1.2), the proteins all go back to their original, pre-signaling epithelial levels.
If, however, the tumor microenvironment releases sufficient Wnt ligand, the amount of Dvl that accumulates at the membrane may be enough to induce significant changes in the proteins, as shown in Fig. 2b (3.62 nM, d = 5.4 at time τ = 20). With the activation of the Wnt pathway, free β-catenin accumulates, meaning that there is now a larger pool of β-catenin in the cytosol capable of translocating to the nucleus and upregulating Slug. Due to the rate limiting processes of translocation and the availability TCF/LEF, we witness the activation of Slug, as well the stabilization of β-catenin in the cytosol. Ultimately, both β-catenin and Slug reach much higher steady state values (~0.45 nM, b = 1.37; ~21.45 nM, s ≈ 6.5). The activation of the Wnt pathway and the subsequent changes in Slug and β-catenin are now enough such that there is a significant decrease in the steady state value of E-cadherin (~1.32 × 10−3 nM, e ≈ 0.04), meaning that β-catenin is not being as quickly sequestered into the adhesion complex, nor is it being as quickly degraded by the GSK-3β complex. Thus, β-catenin is able to continue to translocate to the nucleus. Even if membrane Dvl returns to 0 (as shown in Fig. 2b at τ = 50), the cell does not revert to the protein concentrations pertaining to the initial epithelial steady state, but has instead stabilized and maintains the protein levels of the converted mesenchymal state. In Fig. 2b, with d = 0 at τ = 50, E-cadherin is still maintained at a low value while Slug and β-catenin dip only slightly with the removal of the extracellular signal.
The bifurcation diagrams in Fig. 3a-c give further insight into the behavior of E-cadherin, β-catenin, and Slug with respect to alterations in membrane-bound Dvl (d). These figures can be thought of as concentration-response curves of protein levels with respect to membrane Dvl. If the cell starts in the epithelial steady state, such as our system does in Fig. 3, and the level of membrane Dvl is steadily increased, the cell remains in the epithelial state, with high E-cadherin and low β-catenin and Slug levels, until Dvl reaches a threshold level of about 1.33 (blue vertical dashed line, Fig. 3a-c). At this point the cell undergoes EMT and transitions into a mesenchymal-like state with low E-cadherin and high β-catenin and Slug levels. For further increases of Dvl beyond 1.33, the cell remains in the switched mesenchymal state. However, when we start from the newly attained mesenchymal steady state and move leftward in Fig. 3 A-C, reducing the level of membrane Dvl, the cell remains in the mesenchymal steady state and does not switch back to the epithelial state. In fact, in this instance, with d = 0, the values that the proteins maintain are the steady state mesenchymal concentrations of the proteins at τ = 80 in Fig. 2b. There is thus a range of levels of Dvl (d = 0 to 1.33) for which the cell can exist in one of the two distinct steady states – epithelial or mesenchymal – depending on its history. This “cellular memory” mechanism generated by the bitable switch could explain how the EMT-derived mesenchymal cell is able to retain its invasive phenotype and not revert to an epithelial state even in the absence of sustained pro-EMT signaling from the microenvironment, for example in blood or lymphatic vessels. The switch mechanism also suggests that it is not the lack of an extracellular signal per se that forces the cells to revert to the epithelial phenotype in the metastatic environment (the mesenchymal to epithelial transition, MET), but rather a different extracellular signal in the new microenvironment is required to render the switch reversible.
Sensitivity Analysis Shows E-cadherin is Influenced by Parameters Associated with Slug
To assess the sensitivity of the model to its various parameters, we carried out sensitivity analysis using a Latin Hypercube Sampling (LHS) [33] and Partial Ranked Correlation Coefficient (PRCC). LHS is a methodology that allows us to divide the range for each of our K parameters into N intervals and randomly sample a value from each interval [34]. Under the constraint that where K = 13 parameters in Eqs. (1)–(3) and K = 8 in Eqs. (4)–(6), it would be necessary to create at least 18 intervals to sample per parameter. However, in order to effectively sample the parameter space, 1 × 105 intervals per parameter were created. Sampled values for each parameter are then randomly assembled together into N parameter sets and the model is run for each different set of parameters [35]. Once the LHS has been carried out, PRCC analysis permits us to transform our parameter input values and our outcomes into ranked values and measure the correlation between the rank-transformed input parameters and the rank-transformed outcomes [35]. As the Hill coefficients are all greater than 1 and are thus ultrasensitive terms in the model, by using LHS and PRCC, we sought to understand how the other parameters in the dimensional system influenced the steady state behaviors of E-cadherin (E), β-catenin (B), and Slug (S), as well as any differences that may occur when the system was nondimensionalized. Thus, the sensitivity analysis was carried out on both systems presented. By examining the sensitivity of the steady state behavior of our system to the nondimensional parameters, we were able to reduce the parameter space and highlight any hidden relationships that may exist between the parameters and the steady state behavior of the variables.
For all of the parameters in both systems, uniform distributions were used and sensitivity analysis was carried out at different values of Dvl. The ranges of the dimensional parameter values are provided in Table 3, while the ranges for the nondimensional parameters are listed in Table 4. After examining the partial correlation coefficients between each of the individual ranked parameter values at different values of Dvl, the parameters were found to be uncorrelated with each other. Scatterplots of the nondimensional steady state values for E-cadherin (e), β-catenin (b), and Slug (s) at different values of Dvl (d) in response to the nondimensional parameters are given in the Additional file 1: Fig. S1. These plots show the monotonic behavior of the variable steady states in response to changes in the different parameters, which is required to apply PRCC. Additionally, this monotonic behavior was found to hold in the dimensional system. The range of values considered for parameter A 2 begins at 1 in Table 4. As shown in the Additional file 1, E-cadherin (e) and Slug (s) lack monotonic behavior in response to changes in A 2. At d = 0, with A 2 ϵ [0,1), b was found to be less than 0, which is not biologically realistic. While the addition of d to the system does reduce the range of A 2 for which we see b < 0, in order to be able to compare changes in sensitivity across different levels of d, the entire range of A 2 ϵ [0 1) was excluded from the analysis. Similar behavior was found with α 2, and thus the parameter range α 2 ϵ [0 0.01) was excluded from the analysis as well.
Table 3.
Parameter | Minimum Value | Maximum Value | Units |
---|---|---|---|
D | 0 | 4 | nM |
α 1 | 0 | 1.2 × 10−1 | |
α 2 | 1.0 × 10−2 | 2.0 × 10−2 | |
α 3 | 0 | 2.0 × 10−1 | |
β 1 | 0 | 1.515 × 10−1 | |
β 2 | 0 | 7.576 × 10−2 | |
β 3 | 0 | 7.576 × 10−2 | |
k 0 | 0 | 2.5 × 10−2 | |
k 1 | 4.0 × 10−3 | 1.0 × 10−2 | |
k 2 | 0 | 2.5 | |
IC S | 1.65 × 10−2 | 10.0 | nM |
IC B | 0 | 8.25 × 10−1 | nM |
IC E | 2.75 × 10−3 | 2.0 × 10−1 | nM |
IC D | 0 | 5.0 | nM |
Table 4.
Parameter | Minimum Value | Maximum Value |
---|---|---|
d | 0 | 6 |
A 1 | 0 | 120 |
A 2 | 1 | 2 |
A 3 | 0 | 2 |
C 1 | 0 | 5 |
C 2 | 0 | 2.5 |
C 3 | 0 | 2.5 |
F 1 | 0 | 2.5 |
F 2 | 0 | 25 |
Figure 4 shows the PRCC analysis for the parameters with the steady state values for E-cadherin, β-catenin, and Slug both prior to and during Wnt signaling. In Fig. 4a and b, the relationship between the dimensional parameters in Eqs. (1) - (3) and the steady state values of E, B, and S at Dvl (D) = 0 (4A) and Dvl (D) = 4 (4B) are examined. For both values of Dvl (D), the β-catenin-GSK-3β complex binding rate (β 2) influences the steady state concentration of free cytosolic β-catenin (B) while the degradation rate of Slug, β 3, significantly influences the steady state concentrations of E-cadherin (E) and Slug (S). These sensitivity relationships are preserved after the nondimensionalization is carried out and we examine the relationship between the nondimensional parameters of Eqs. (4) – (6) to the steady state behavior of e, b, and s. As shown in Figs. 4c and d, the steady state behavior of β-catenin (b) is dependent upon the nondimensional rate at which it binds to the GSK-3β complex (C 2), while the steady state behaviors of E-cadherin (e) and Slug (s) are both sensitive to the nondimensional degradation rate of Slug (C 3).
One interesting difference between the parameter analysis of the dimensional model and the nondimensional model is observed when examining sensitivity changes that occur after Wnt signaling is activated. In Fig. 4a, in the absence of Wnt signaling, the parameter k 2 is not significantly correlated with any of the dimensional variables. But, with an increase in the active membrane-bound Dvl in Fig. 4b, k 2 significantly influences the steady state concentration of Slug. Similarly, in Fig. 4c, the concentrations of the nondimensional steady state concentrations are not significantly correlated with changes in the F 2 parameter, which is defined as in Table 2 and thus includes the k 2 parameter. With the activation of the Wnt pathway in Fig. 4d (d = 6), changes in F 2 become significantly correlated with the steady state concentrations of Slug (s) and E-cadherin (e). This additional sensitivity of E-cadherin (e) to changes in F 2 could be due to the underlying sensitivity to the parameter IC S, which exists in the denominator of F 2 and was shown to influence the concentration of E in the dimensional model. However, due to the sensitivity of E-cadherin (E) to changes in IC S in Fig. 4a, it would be expected that the nondimensional concentration of E-cadherin (e) would be sensitive to changes in F 2 prior to Wnt signaling (d = 0), which is not the case. This sensitivity discrepancy suggests that, with the activation of the Wnt pathway and the increase in cytosolic β-catenin, the ratio of the rate at which β-catenin is able to activate Slug to the IC50 value of Slug necessary to inhibit E-cadherin production is influential in the steady state concentration of E-cadherin.
The sensitivity relationships in Fig. 4 suggest that the protein that is both responsible for cell-to-cell adhesion and is a characteristic marker of the epithelial phenotype, E-cadherin, is heavily influenced by changes in the concentration of Slug that is present. Even before Wnt signaling is activated, the concentration of E-cadherin present at the membrane for cell-to-cell contact is susceptible to changes in the rate at which Slug is being destroyed and the IC50 value of Slug necessary to inhibit production of E-cadherin. With the activation of the Wnt signaling pathway, the sensitivity of the level of Slug to the translocation of β-catenin to the nucleus changes. Now, with an external stimulus and activation of the Wnt pathway, the change in the compartmentalization of β-catenin from the cytosol to the nucleus significantly influences changes in the steady state behavior of Slug, which is then capable of overcoming the IC50 value necessary to inhibit E-cadherin production. This change in significance for Slug, as well as the sensitivity of E-cadherin to IC S and the ratio of k 2 to IC S, highlights the possibility of therapeutically targeting nuclear events for the prevention of the change from the epithelial to the mesenchymal phenotype, as well as the possibility that increasing the IC S value may inhibit the switch once the pathway has been activated.
Two-Parameter Bifurcations Highlight Cellular Rates important to EMT and the Bistable Switch
To further understand how changes in the parameters influence the system, particularly the bistable switch behavior, nondimensional two-parameter bifurcation diagrams were generated using XPPAUT. Shown in Fig. 5, these plots depict the steady state phenotype of the simulated cell over the parameter space. Along the x-axis of each plot, Dvl (d) is varied. Along the y-axis, the non-dimensional parameters A 1 , A 2 , A 3 , C 1 , C 2 , C 3 , F 1 , F 2, n 1, n 2, n 3, and n 4 are varied one at a time. In each of the figures, the cell can only be in the epithelial steady state for parameter values in Region I and can only be in the mesenchymal steady state in Region III. In Region II, either steady state is possible depending on the history of the cell. If the system begins in the epithelial steady state (Region I) and crosses the L1 boundary into Region II, the cell will remain in the epithelial steady state. Similarly, if the cell begins in the mesenchymal steady state (Region III) and crosses the L2 boundary into Region II, it will remain in the mesenchymal steady state. If, based on the parameter values, the system begins in Region II, it will depend upon the initial conditions whether or not the cell assumes an epithelial or mesenchymal phenotype. The nondimensional parameter value used in this model is indicated by a dashed line in each subplot of Fig. 5. For these parameter values, the cell begins as an epithelial cell in Region II. Once the concentration of Dvl (d) is increased such that the L2 boundary is crossed into Region III, the cell will make the transition from epithelial to mesenchymal. If, however, the concentration of d is then reduced at this parameter value, the newly formed mesenchymal cell would move back into Region II and conserve its mesenchymal phenotype due to the fact that the cell was unable to cross L1. Hypothetically, if a cell begins as a mesenchymal cell in Region II and the concentration of Dvl (d) is decreased so that the L1 boundary is crossed, the cell will transition from a mesenchymal cell into an epithelial cell as the system transitions from Region II to Region I. The region of bistability (i.e. Region II) is seen to be fairly robust to parameter variation.
Note that, in Fig. 5g, once F1 overcomes a certain threshold value, the system must cross over L2 and pass from Region II to Region III with the addition of enough Dvl (d) to the system. This behavior indicates that an epithelial cell beginning in Region II must switch to the mesenchymal phenotype so long as enough Wnt signaling is applied. As F1 represents the nondimensional rate in which Dvl inactivates the GSK-3β degradation complex, this result suggests that, at if the rate at which Dvl inhibits the β-catenin degradation is high enough, so long as enough Dvl is active at the membrane, β-catenin will accumulate in the cytosol and the cell will be forced from the epithelial to the mesenchymal phenotype.
Similar behavior occurs in Fig. 5h with F2. For a high value of F2, in the presence of sufficient Wnt signaling, an epithelial cell must undergo the bistable switch and enter the mesenchymal steady state. This behavior is particularly interesting in light of what we now know about the sensitivity of the Slug and E-cadherin steady state concentrations to changes in F2, as well as the relationship between F2 and k 2 and IC S. With the activation of the Wnt signal and the movement of Dvl to the membrane, if the rate at which β-catenin moves to the nucleus and binds with TCF is fast enough or the IC50 concentration necessary to inhibit E-cadherin production is low enough, Slug will be able to accumulate to a concentration sufficient to force the cell to switch from the epithelial to the mesenchymal phenotype.
Conclusions
The ability of a carcinoma cell to adopt the mesenchymal phenotype due to external signals from the microenvironment and conserve the newly acquired invasive properties in the absence of an extracellular cue suggests that the switch from the epithelial to the mesenchymal state is bistable in situ. While multiple intracellular signaling pathways can stimulate EMT, we consider Wnt signaling as a case study. The Wnt signaling pathway has been under intense scrutiny from both mathematicians and biologists in order to understand how it contributes to changes in cellular behavior. Mathematical models attempting to describe the intracellular pathway are often complex and limited to one group of protein interactions. Instead of examining one subset of the Wnt pathway, the model presented in this work examines the three key relationships centered around β-catenin that comprise the Wnt signaling pathway and drive the change in cellular phenotype. Studying the system as a whole provides us a better understanding of which interactions are likely to be responsible for the bistable switch.
This model opens up many avenues for possible theoretical and practical exploration. The bistable switch proposed in this model is an all or nothing change in behavior: the cell can either occupy the epithelial or the mesenchymal steady state. However, recent work has suggested that the transition may, instead, be a gradient, whereby the cell can occupy at least one (if not more) intermediate states as it progresses towards the mesenchymal phenotype [6]. Using a scaffold model to expand upon the complicated mechanisms involved in each step of Wnt pathway activation, as well as pathway crosstalk, may indeed reveal intermediate steady states for the cell to occupy during the transition.
With the theory of a switch underlying EMT, it would also be prudent to examine how the transition is affected by the presence of neighboring cells. The stabilization of these bonds could ultimately work against the loss of adhesion in the epithelial cell and the resulting transformation. Additionally, sensitivity analysis and exploration of the response of the system to changes in parameter values as Wnt signaling is activated suggests that excellent candidates for possible therapeutic intervention are the β-catenin degradation complex, the shuttling of β-catenin to the nucleus, and the IC 50 value of Slug necessary to inhibit E-cadherin production. While the steady state behavior of free β-catenin was shown to be sensitive to its own rate of degradation, the steady state behavior of E-cadherin is actually sensitive to changes in Slug. In particular, E-cadherin is sensitive to the degradation of Slug and the half maximal concentration of Slug necessary to inhibit E-cadherin production, as well as the ratio of rate of Slug activation via β-catenin to the half-maximal concentration of Slug necessary to inhibit E-cadherin production once the pathway has been activated. These insights, coupled with the behavior of the system in response to changes in both the F2 parameter and Dvl (d) activation in Fig. 5h, suggests that continued exploration of how the switch is affected by the presence of neighboring cells, as well as the formation of the E-cadherin- β-catenin complex, is necessary. The stabilization of intercellular bonds could ultimately work against the loss of adhesion in the epithelial cell and the resulting transformation. Possible avenues for practical exploration would be therapeutically raising the half-maximal concentration of Dvl necessary to inhibit β-catenin degradation or lowering the rate at which β-catenin translocates to the nucleus. Therapies targeted at the IC50 concentration of Slug necessary to inhibit E-cadherin production, as well as inhibiting the movement of β-catenin to the nucleus could prevent the activation of Slug and ultimately work to maintain the E-cadherin- β-catenin complex at the membrane. Pharmacological exploration of these components of the Wnt pathway could help prevent EMT prior to intravasation and metastasis, the primary cause of cancer-related mortality.
Acknowledgments
We would like to thank Dr. Sharon Lubkin, as well as the anonymous reviewers of this journal, for reading this manuscript and offering constructive feedback. Their contributions, we believe, made this a substantially stronger article.
Funding
KG received support from the Research Training Group in Mathematical Biology, funded by a National Science Foundation grant RTG/DMS – 1246991. SB received support from the US EPA STAR Program (EPA Grant Number: R835000)
Availability of Data and Materials
Not applicable.
Additional file
Authors’ Contributions
KG contributed to model development, analysis, and was a major contributor to writing the manuscript. MH contributed expert feedback in biological relevance for model development. AW contributed expert feedback on sensitivity analysis. SB contributed to model development and contributed to writing the manuscript.
Ethics Approval and Consent to Participate
Not applicable.
Consent for Publication
Not applicable.
Competing Interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Footnotes
Electronic supplementary material
The online version of this article (10.1186/s12976-017-0064-7) contains supplementary material, which is available to authorized users.
Contributor Information
Kelsey Gasior, kgasior@email.unc.edu.
Marlene Hauck, Email: marlene_hauck@ncsu.edu.
Alyson Wilson, Email: alyson_wilson@ncsu.edu.
Sudin Bhattacharya, Email: sbhattac@msu.edu.
References
- 1.Weinberg R. The biology of cancer: Garland science; 2013.
- 2.Lee JM, Dedhar S, Kalluri R, Thompson EW. The epithelial-mesenchymal transition: new insights in signaling, development, and disease. J Cell Biol. 2006;172(7):973–981. doi: 10.1083/jcb.200601018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Onder TT, Gupta PB, Mani SA, Yang J, Lander ES, Weinberg RA. Loss of E-cadherin promotes metastasis via multiple downstream transcriptional pathways. Cancer Res. 2008;68(10):3645–3654. doi: 10.1158/0008-5472.CAN-07-2938. [DOI] [PubMed] [Google Scholar]
- 4.Christofori G, Semb H. The role of the cell-adhesion molecule E-cadherin as a tumour-suppressor gene. Trends Biochem Sci. 1999;24(2):73–76. doi: 10.1016/S0968-0004(98)01343-7. [DOI] [PubMed] [Google Scholar]
- 5.Nieto MA, Huang RYJ, Jackson RA, Thiery JP. EMT: 2016. Cell. 2016;166(1):21–45. doi: 10.1016/j.cell.2016.06.028. [DOI] [PubMed] [Google Scholar]
- 6.Kalluri R, Weinberg RA. The basics of epithelial-mesenchymal transition. J Clin Investig. 2009;119(6):1420–1428. doi: 10.1172/JCI39104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Mani SA, Guo W, Liao MJ, Eaton EN, Ayyanan A, Zhou AY, Brooks M, Reinhard F, Zhang CC, Shipitsin M, et al. The epithelial-mesenchymal transition generates cells with properties of stem cells. Cell. 2008;133(4):704–715. doi: 10.1016/j.cell.2008.03.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Thiery JP. Epithelial-mesenchymal transitions in tumour progression. Nat Rev Cancer. 2002;2(6):442–454. doi: 10.1038/nrc822. [DOI] [PubMed] [Google Scholar]
- 9.Kogan Y, Halevi-Tobias KE, Hochman G, Baczmanska AK, Leyns L, Agur Z. A new validated mathematical model of the Wnt signalling pathway predicts effective combinational therapy by sFRP and Dkk. Biochem J. 2012;444:115–125. doi: 10.1042/BJ20111887. [DOI] [PubMed] [Google Scholar]
- 10.Lloyd-Lewis B, Fletcher AG, Dale TC, Byrne HM. Toward a quantitative understanding of the Wnt/−catenin pathway through simulation and experiment. Wiley Interdiscip Rev-Syst Biol. 2013;5(4):391–407. doi: 10.1002/wsbm.1221. [DOI] [PubMed] [Google Scholar]
- 11.Chen JW, Xie ZR, Wu YH. Computational Modeling of the Interplay between Cadherin-Mediated Cell Adhesion and Wnt Signaling Pathway. PLoS One. 2014;9(6):20. doi: 10.1371/journal.pone.0100702. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Lee E, Salic A, Kruger R, Heinrich R, Kirschner MW. The roles of APC and axin derived from experimental and theoretical analysis of the Wnt pathway. PLoS Biol. 2003;1(1):116–132. doi: 10.1371/journal.pbio.0000010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Ramis-Conde I, Drasdo D, Anderson ARA, Chaplain MAJ. Modeling the influence of the E-cadherin-beta-catenin pathway in cancer cell invasion: A multiscale approach. Biophys J. 2008;95(1):155–165. doi: 10.1529/biophysj.107.114678. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Basan M, Idema T, Lenz M, Joanny JF, Risler T. A Reaction-Diffusion Model of the Cadherin-Catenin System: A Possible Mechanism for Contact Inhibition and Implications for Tumorigenesis. Biophys J. 2010;98(12):2770–2779. doi: 10.1016/j.bpj.2010.03.051. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Agur Z, Kirnasovsky OU, Vasserman G, Tencer-Hershkowicz L, Kogan Y, Harrison H, Lamb R, Clarke RB. Dickkopf1 Regulates Fate Decision and Drives Breast Cancer Stem Cells to Differentiation: An Experimentally Supported Mathematical Model. PLoS One. 2011;6(9):10. doi: 10.1371/journal.pone.0024225. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Kim D, Rath O, Kolch W, Cho KH. A hidden oncogenic positive feedback loop caused by crosstalk between Wnt and ERK Pathways. Oncogene. 2007;26(31):4571–4579. doi: 10.1038/sj.onc.1210230. [DOI] [PubMed] [Google Scholar]
- 17.Shin SY, Rath O, Zebisch A, Choo SM, Kolch W, Cho KH. Functional Roles of Multiple Feedback Loops in Extracellular Signal-Regulated Kinase and Wnt Signaling Pathways That Regulate Epithelial-Mesenchymal Transition. Cancer Res. 2010;70(17):6715–6724. doi: 10.1158/0008-5472.CAN-10-1377. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.MacLean AL, Rosen Z, Byrne HM, Harrington HA. Parameter-free methods distinguish Wnt pathway models and guide design of experiments. Proc Natl Acad Sci U S A. 2015;112(9):2652–2657. doi: 10.1073/pnas.1416655112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Jolly MK, Tripathi SC, Jia D, Mooney SM, Celiktas M, Hanash SM, Mani SA, Pienta KJ, Ben-Jacob E, Levine H. Stability of the hybrid epithelial/mesenchymal phenotype. Oncotarget. 2016;7(19):27067–27084. doi: 10.18632/oncotarget.8166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Jolly MK, Huang B, Lu MY, Mani SA, Levine H, Ben-Jacob E. Towards elucidating the connection between epithelial - mesenchymal transitions and stemness. J R Soc Interface. 2014;11(101):10. doi: 10.1098/rsif.2014.0962. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Hong T, Watanabe K, Ta CH, Villarreal-Ponce A, Nie Q, Dai X. An Ovol2-Zeb1 Mutual Inhibitory Circuit Governs Bidirectional and Multi-step Transition between Epithelial and Mesenchymal States. PLoS Comput Biol. 2015;11(11):20. doi: 10.1371/journal.pcbi.1004569. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Benary U, Kofahl B, Hecht A, Wolf J. Modeling Wnt/beta-catenin target gene expression in APC and Wnt gradients under wild type and mutant conditions. Front Physiol. 2013;4:18. doi: 10.3389/fphys.2013.00021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Heuberger J, Birchmeier W. Interplay of Cadherin-Mediated Cell Adhesion and Canonical Wnt Signaling. Cold Spring Harb Perspect Biol. 2010;2(2):24. doi: 10.1101/cshperspect.a002915. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Schmitz Y, Rateitschak K, Wolkenhauer O. Analysing the impact of nucleo-cytoplasmic shuttling of beta-catenin and its antagonists APC, Axin and GSK3 on Wnt/beta-catenin signalling. Cell Signal. 2013;25(11):2210–2221. doi: 10.1016/j.cellsig.2013.07.005. [DOI] [PubMed] [Google Scholar]
- 25.Mosimann C, Hausmann G. Basler K: beta-Catenin hits chromatin: regulation of Wnt target gene activation. Nat Rev Mol Cell Biol. 2009;10(4):276–U257. doi: 10.1038/nrm2654. [DOI] [PubMed] [Google Scholar]
- 26.Chaffer CL, Weinberg RA. A Perspective on Cancer Cell Metastasis. Science. 2011;331(6024):1559–1564. doi: 10.1126/science.1203543. [DOI] [PubMed] [Google Scholar]
- 27.Kalluri R, Zeisberg M. Fibroblasts in cancer. Nat Rev Cancer. 2006;6(5):392–401. doi: 10.1038/nrc1877. [DOI] [PubMed] [Google Scholar]
- 28.Clevers H, Nusse R. Wnt/beta-Catenin Signaling and Disease. Cell. 2012;149(6):1192–1205. doi: 10.1016/j.cell.2012.05.012. [DOI] [PubMed] [Google Scholar]
- 29.Daniels DL, Weis WI. beta-catenin directly displaces Groucho/TLE repressors from Tcf/Lef in Wnt-mediated transcription activation. Nat Struct Mol Biol. 2005;12(4):364–371. doi: 10.1038/nsmb912. [DOI] [PubMed] [Google Scholar]
- 30.Shirley SH, Hudson LG, He J, Kusewitt DF. The Skinny on Slug. Mol Carcinog. 2010;49(10):851–861. doi: 10.1002/mc.20674. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.George SJ, Dwivedi A. MMPs, cadhenins, and cell proliferation. Trends Cardiovasc Med. 2004;14(3):100–105. doi: 10.1016/j.tcm.2003.12.008. [DOI] [PubMed] [Google Scholar]
- 32.XPPAUT – the differential equations tool, version 6.10 [http://www.math.pitt.edu/~bard/xpp/xpp.html].
- 33.McKay MD, Beckman RJ, Conover WJ. A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics. 1979;21(2):239–245. [Google Scholar]
- 34.Blower SM. Dowlatabadi H: sensitivity and uncertainty analysis of complex-models of disease transmission - an hiv model, as an example. Int Stat Rev. 1994;62(2):229–243. doi: 10.2307/1403510. [DOI] [Google Scholar]
- 35.Gomero B: Latin Hypercube Sampling and Partial Rank Correlation Coefficient Analysis Applied to an Optimal Control Problem. University of Tennessee; 2012.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Not applicable.