I love it when a plan comes together: Insight gained through convergence of competing mathematical models

Mathematical models of cardiac electrophysiology have long been used as research tools, and simulations with these models have allowed for the development of new hypotheses, the synthesis of experimental results, and insight into arrhythmia mechanisms [1–3]. This modeling, however, has largely been the domain of a relatively small group of experts with specialized knowledge, and the process of developing new models often involves as much art (i.e. expert judgment) as science. It has remained an open question whether these models could move from the category of specialized tools that are useful for particular questions into the category of trusted assays that are nearly always reliable.

1. The CiPA initiative: Using models for regulatory decision-making

Given this uncertainty, the modeling community was excited by the announcement of the Comprehensive in vitro Proarrhythmia Assay, or CiPA [4,5]. This initiative, a partnership between academic laboratories, regulatory bodies, and pharmaceutical companies, aims to replace current testing for drug-induced proarrhythmia (aka drug-induced Long QT Syndrome, which usually manifests pathologically as Torsades de Pointes) with a fully in vitro assay. As the CiPA has been envisioned, the assay will involve measurements of how drugs block important cardiac ion channels, recordings of how drugs influence action potentials in cultured heart cells, and numerical simulations of how drugs affect human ventricular myocytes. The announcement of the CiPA seemed to signify the graduation of mathematical modeling as a discipline. Perhaps modeling was no longer an art form that could only be practiced by a small community of experts but was instead a more mature discipline that could be trusted for decision-making.

Some aspects of CiPA’s proposed modeling component, however, gave specialists pause. These caveats came about because of limitations, familiar to the community but perhaps not well-known more broadly, of cardiac electrophysiology mathematical modeling. The first is that the exact range of a model’s utility is generally not clear. All models are developed carefully, in part by validating simulation output against experimental data. However, there is no “industry standard” for the data that should be considered, and most models are only validated under a limited range of conditions (e.g. selected pacing rates, drug treatments, etc.). The second is that there is no single model of the human ventricular myocyte, in the same way that there is no single antibody for any particular protein. Instead, in the spirit of scientific competition, multiple models of each cell type have generally been developed by several groups working independently. This leads to innovation but can also create confusion.

Although these two issues are important, they are not indictments of modeling as an approach. With respect to the first issue, it’s generally recognized that achieving consistency with experimental results often occurs iteratively rather than in one step. In other words, models can be perturbed to generate predictions, the testing of these predictions can demonstrate both when the model succeeds and when it fails, and these results can then be used to improve the model [6,7]. Indeed, it’s been observed that models are more useful when they fail than when they succeed [8]. With regards to the second issue, competing models from different groups often represent opportunities for clarifying and narrowing hypotheses. A benefit of models in general is that they allow for hypotheses to be stated in precise terms (e.g. 50% inhibition of protein X leads to a 30% increase in output Y). When predictions differ significantly between two models [9–11], the experimental tests of these predictions can allow for rigorous discrimination of the competing underlying mechanistic explanations.

Thus, those who are closely involved in the research recognize that incomplete validation and competing models are simply part of the process. But for those who wish to use models to make decisions in real time, these issues create difficulties [12]. For instance: if three competing models of the same cell type exist, which one should be trusted?

2. Current work is successfully addressing limitations of modeling studies

Although the discussion of limitations may seem to paint a pessimistic picture, the good news is that novel tools developed in recent years help to address both concerns. With respect to the first limitation, validation against incomplete data sets, recent work has shown how more predictive models may be developed. Even though it’s intuitive that more thorough validation should lead to a more robust model, recent studies have shown rigorously that tuning models based on rich sets of experimental measurements may allow for cell-specific models that can capture experimental variability [13–15]. Advances have also been made to address the second limitation, the problem of competing models. For instance, while studies published several years ago documented differences between competing models of the same cell type [9,10,16], more recent studies have, by examining a wide variety of models, been able to derive more general rules about why some models behave one way and other models behave differently [17].

In this spirit, a paper recently published in the JMCC takes an important step forward in addressing both of these concerns simultaneously [18]. A discussion of this paper may help us to understand the strengths of mathematical modeling of cardiac electrophysiology and arrhythmias, the current limitations in this field, and the innovative studies being performed to move the field forward and improve the chances that these models will be used for decision-making. The “take home” message from this paper [18] is that these issues can be overcome, if careful modeling studies satisfy two conditions: (1) simulation results from multiple models are compared with a broad range of experimental data, obtained under multiple conditions; and (2) simulations are performed not only with models meant to represent typical samples or individuals, but instead performed with a range of models that recapitulate experimental variability.

3. Models may converge when validated against the same datasets

In their study, Mann et al. [18] combined clinical data from patients with simulation results to obtain important novel insights. Electrocardiographic QT intervals were obtained, before and after β-adrenergic stimulation, from healthy volunteers (with no known ion channel mutations) and from patients with the three most common forms of Long QT Syndrome (LQT1, 2, and 3). These three diseases cause, respectively, a decrease in slow delayed rectifier K⁺ urrent (I_Ks), a decrease in rapid delayed rectifier K⁺ current (I_Kr), and an increase in the late component of Na⁺ current (I_NaL). In principle, these alterations are straightforward to implement in mathematical models, so the authors simulated these mutations in the three leading models of the human ventricular myocyte [19–21]. When they did so, they uncovered results that were disconcerting, but perhaps not shocking to practitioners in the field. Namely, each model failed in a specific way. That is, a given model might do well at simulating a particular form of LQTS but perform poorly when attempting to simulate a different form of LQTS. From the standpoint of decision-making, it would be much easier if a single model failed under all conditions — in that case, simply discard that particular model. But if each model exhibits at least one strength in addition to one or more weaknesses, how should one then proceed?

At this stage, Mann et al. performed a clever analysis that led to the manuscript’s critical insights. Namely, they recalibrated densities of ionic currents in all three models [19–21] to improve each model’s consistency with the full range of clinical data they had acquired. In other words, the underlying differential equations were kept intact, but parameters describing ion channel densities were adjusted to minimize the differences between simulation results and experimental data. When they performed this procedure, in effect forcing all three models to reproduce the clinical results, a remarkable and profound result emerged — the models became more similar than they were originally. To state this more precisely, two delayed rectifier K⁺ currents, I_Kr and I_Ks, primarily control repolarization in ventricular myocytes, but the relative contributions of I_Kr and I_Ks are species-dependent, sometimes disputed, and often dramatically different between mathematical models. Mann et al. [18] found that after the three models [19–21] were simultaneously required to reproduce the available data, the contributions of these two currents were substantially more similar between models than they were originally.

The study [18] is relevant not only because of the specific insight the results provide, but also because the work illustrates general principles that will be important for future studies in a variety of fields. One is that, as previously noted, simulations should be performed not only with models of typical cells, but with heterogeneous populations that recapitulate biological variability. This methodological choice, which allowed for the study’s key insights, follows similar recent advances in this area [22–29]. The second is that competing mathematical models of the same biological process should be thought of as an opportunity to more fully explore alternative hypotheses rather than simply an annoyance that must be dealt with.

4. Additional future work will likely lead to further insight

Although the study by Mann et al. [18] is novel and quite important, it’s useful to consider this study’s limitations, because these can illustrate some of the unresolved issues that researchers in this field are likely to tackle in the next few years.

One limitation is the fact that, when the authors forced the three models to reproduce all of the clinical results, the only parameters varied were those that determine maximal levels of the different ionic currents. Presumably, differences between competing models could result not only from ionic current densities, but also from the currents’ kinetic formulations. Although advances have recently been made in methods to constrain ion channel kinetic parameters [30], these have largely been applied to voltage clamp recordings. At the current time, a systematic examination of kinetic parameters based solely on cellular physiological data (e.g. action potentials and/or intracellular calcium) remains an issue to be addressed in future studies.

Second, although simulations were performed under both baseline conditions and after β-adrenergic stimulation, the downstream effects of β-adrenergic stimulation were simulated in a phenomenological way, by assuming that β-adrenergic stimulation simply caused increases in specific ionic currents. A few models [31,32] have been developed that couple the signaling initiated by β-receptor activation with the downstream electrophysiological and calcium handling mechanisms. Although these models have provided important novel insight, their parameters have generally not been perturbed and examined systematically as in the current paper and other important recent studies [22–29]. It’s anticipated that comprehensive analyses on combined electrophysiological and signaling models are likely to generate novel insight into issues such as which specific proteins represent the best targets for therapeutic interventions.

Third, it is worth noting that simulations of heterogeneous populations, such as those performed by Mann et al. [18] and in closely-related studies [22–29], can generate large amounts of “pseudo-data” (e.g. simulated physiology from hundreds or thousands of individuals). Drawing conclusions from these large datasets requires that the techniques of mechanistic mathematical modeling be combined with approaches from bioinformatics and statistical analysis. For instance, mechanistic insights have previously been gained using methods such as multivariable linear regression [10], logistic regression [33], partial correlation coefficients [25], and, more recently, machine learning techniques such as Support Vector Machine [34]. Despite these advances, the perception nonetheless persists that quantitative physiologists have only begun to scratch the surface of the sophisticated statistical approaches that are more commonly employed in fields such as genomics. If some of these techniques (e.g. LASSO regression [35], elastic net regression [35], and “deep learning” with neural networks [36]) are appropriately employed in mechanistic studies, it seems likely that further quantitative advances will be made.

5. Implications for the CiPA and the use of models in decision making

The interpretation of this study’s [18] implications depends, to some extent, on whether one chooses to take a “glass half full” or a “glass half empty” perspective. To emphasize the negatives, the study illustrates that if a particular mathematical model is asked to simulate multiple disease states, the model is likely to fail in some circumstances. This general phenomenon has been known for a long time; after all, George Box’s famous quip that “All models are wrong, but some are useful” [37] dates from 1987. The truth of this comment, however, provides scant consolation to someone who is forced to decide whether or not to approve an individual drug, or to continue supporting a particular drug development program. In such cases, including the CiPA [4,5], definitive answers seem much more useful than pithy nuance.

We prefer, however, to take a more optimistic viewpoint. Although the results show how individual mathematical models can sometimes fail to properly recapitulate pathological conditions, the paper also demonstrates a useful strategy for overcoming these limitations and generating simulation results that will be more reliable and robust. Approaches such as these are likely to be of tremendous use, not only in the CiPA, but also in future efforts to use mathematical models for drug evaluation and quantitative predictions.

The title of this editorial comes from the popular 1980s television series “The A Team,” a show in which a misfit group of soldiers of fortune concocted elaborate, sometimes ridiculous, schemes to help those who had nowhere else to turn. In nearly every episode, the group’s leader, Colonel Hannibal Smith, would utter his famous catchphrase, “I love it when a plan comes together,” often stated when their plot appeared at its most preposterous. Science can sometimes follow a similar path, with important new insights arising out of results that might initially seem problematic. The study by Mann et al. [18] offers an excellent example of how this can happen; i.e. seemingly inconsistent results can eventually produce convergence, or a plan that comes together beautifully.

Disclosures

The editorial’s first author, Ms. Jingqi Gong, is partially supported by a research agreement with Pfizer. These collaborators had no input in the opinions expressed here.