Skip to main content
PLOS ONE logoLink to PLOS ONE
. 2011 Mar 31;6(3):e18295. doi: 10.1371/journal.pone.0018295

Generalized Theorems for Nonlinear State Space Reconstruction

Ethan R Deyle 1, George Sugihara 1,*
Editor: Matej Oresic2
PMCID: PMC3069082  PMID: 21483839

Abstract

Takens' theorem (1981) shows how lagged variables of a single time series can be used as proxy variables to reconstruct an attractor for an underlying dynamic process. State space reconstruction (SSR) from single time series has been a powerful approach for the analysis of the complex, non-linear systems that appear ubiquitous in the natural and human world. The main shortcoming of these methods is the phenomenological nature of attractor reconstructions. Moreover, applied studies show that these single time series reconstructions can often be improved ad hoc by including multiple dynamically coupled time series in the reconstructions, to provide a more mechanistic model. Here we provide three analytical proofs that add to the growing literature to generalize Takens' work and that demonstrate how multiple time series can be used in attractor reconstructions. These expanded results (Takens' theorem is a special case) apply to a wide variety of natural systems having parallel time series observations for variables believed to be related to the same dynamic manifold. The potential information leverage provided by multiple embeddings created from different combinations of variables (and their lags) can pave the way for new applied techniques to exploit the time-limited, but parallel observations of natural systems, such as coupled ecological systems, geophysical systems, and financial systems. This paper aims to justify and help open this potential growth area for SSR applications in the natural sciences.

Introduction

A growing realization in many natural sciences is that simple idealized notions of linearly decomposable, fixed equilibrium systems often do not accord with reality. Rather, empirical measurements on ecosystems, metabolic systems, financial networks, and the like suggest a more complex, but potentially more information-rich paradigm at work [1][14]. Despite a long history of linear methods development in the engineering sciences, natural systems are generally not well described as sums of independent frequencies that can be sensibly decomposed, analyzed as non-interacting, and reassembled (e.g. Fourier or spectral analysis) in the style of traditional reductionism [15], [16]. Rather, quantitative measurements show many systems to be fundamentally non-equilibrium and unstable, in a manner more consistent with nonlinear (state dependent) dynamics occurring on a strange attractor manifold Inline graphic, where relationships between state variables cannot be studied independently of the overall system state [17][27]. This emergent comprehensive view may help explain why many natural systems, such a those mentioned above, appear so difficult to understand and predict. Mirage correlations are commonplace in nonlinear systems where the manifold may contain trajectories that can temporarily exhibit positive correlations between variables for surprisingly long time periods (and in some regions of the state space) and can subsequently and rapidly exhibit negative correlations or no relationship in other time periods (and other regions of Inline graphic). This transient property of apparent non-stationarity in correlations is one of the confounding phenomena faced by traditional linear models that require continual refitting and exhibit little or no predictive power.

In this paper, we present two general theorems that addresses the problem of characterizing the coupled dynamics of nonlinear systems using time series observations on a manifold Inline graphic. A special case of this theorem, attributed originally to Takens [12], provided the first sketch of a mathematical proof for reconstructing a diffeomorphic shadow manifold Inline graphic using lags of a single time series as coordinate axes. The basic idea, that was earlier demonstrated by Packard, Crutchfield, Farmer, and Shaw [28] and Crutchfield [2], is that under generic conditions, a shadow manifold Inline graphic can be created using time-lagged observations of Inline graphic based on a single observation function (Cartesian coordinate variable) that is a smooth and smoothly invertible Inline graphic mapping with Inline graphic. Subsequently, Sauer, Yorke, and Casdagli [29] provided a definitive proof and an explicit extension of Takens' theorem to fractal sets; their theorems are also more powerful than the original theorem, as they show embeddings are not just generic in the sense of being open and dense in the set of all mappings, but in fact almost every mapping in the sense of prevalence [30] is an embedding (see [30] for an in-depth explanation of the advantages of “prevalence” over “generic”). The theorem was also extended by Stark, Broomhead, Davies, and Huke [31], [32] and Stark [33] to include certain classes of stochastic systems. Practical methods for reconstruction have also been explored, particularly to address the presence of noise in real data (e.g. [29], [34]). Casdagli et al. [35] give a thorough treatment of such techniques based on transformations of univariate maps, showing how optimal noise reduction can be achieved. These very important prior results all focused on reconstruction from a single time series; however, as proven below, they can be extended to the more practically significant case where multiple observation functions are used to generate Inline graphic.

Here we prove the more general case of multivariate embeddings (embeddings using multiple time series and lags thereof), and show how time series information can be leveraged if multiple time series and their lags are used to construct embeddings of Inline graphic. These theorems pave the way for more extensive use of state space reconstruction methods in practical applications where long time series may not be available, so that multiple diffeomorphic embeddings may be created in factorial fashion to more fully exploit the coupled non-redundant information that can be extracted from multiple time series (multiple observation functions of dynamics on a manifold) to create predictive shadow manifolds [36]. The use of multiple time series allows the possibility of noise reduction that exceeds the limitations of univariate reconstructions in the presence of noise [35].

The possibility of extending Takens' theorem to allow lags of multiple observation functions was mentioned in Remark 2.9 from [29], but was not explicitly proven. The remark was also restricted to mappings strictly formed from consecutive lags, which is not the only possibility that needs to be considered in the multivariate case. Given the potential importance of multivariate reconstructions, we believe a full proof is required—in particular, one that extends the generalization to non-consecutive lags. We show how Takens' theorem is a special case of our more general Theorem 2 (below) and by following the structure of Takens' original proof we clarify the logic and highlight the restrictions and special cases (non-generic cases) that can arise in its application to real world systems. We then give explicit proof of a stronger version of Remark 2.9 from Sauer et al. [29] that allows non-consecutive lags. This third theorem is stronger than the first two in the sense that it shows embeddings are prevalent and not just generic. For those less familiar, we begin with a brief overview of some basic terms and concepts used in our proofs.

Some Basic Concepts of Embedding Theory

Consider the classic Lorenz attractor [37] shown in Figure 1(a), consisting of trajectories in three-dimensional space that together define a butterfly shaped surface or manifold. For simplicity, a manifold can be thought of as a generalized, n-dimensional surface embedded in some higher dimensional space, where the dimension of the manifold may be fractal (as is the case for the Lorenz attractor). More generally, an embedding is a multivariate transformation of a manifold that resolves all trajectories on the original manifold without crossings. That is, an embedding is globally Inline graphic in that it resolves all singularities in trajectories that define the manifold (singularities are points on the manifold where trajectories cross so that future paths are not uniquely determined).

Figure 1. Lorenz attractor with three shadow manifolds.

Figure 1

The Lorenz attractor [37] is shown with three shadow manifolds created from lag-coordinate transformations. The typical parameters were used: Inline graphic, Inline graphic, and Inline graphic, giving the three coupled equations as Inline graphic, Inline graphic, and Inline graphic. The solution was computed using a fourth order Runge-Kutta method with a time step of Inline graphic, and the time lag used to create the shadow manifolds was Inline graphic. (A) The trajectory shown in the Inline graphic, Inline graphic, and Inline graphic coordinates of the original system reveals a two-lobed manifold. (B) A univariate transformation using time lags of the Inline graphic-coordinate, Inline graphic, preserves this two-lobed structure (and other topological properties), verifying Takens' theorem. (C) A univariate transformation using time lags of the Inline graphic-coordinate, Inline graphic, does not preserve the two-lobed structure. Local neighborhoods of the original attractor are, however, preserved. Thus, though this mapping violates a genericity assumption of the original theorem and is not an embedding, it is an immersion of the original manifold. (D) A multivariate transformation using both the Inline graphic- and Inline graphic-coordinates, Inline graphic, fulfills the assumptions of Theorems 2 and 7. As predicted, it also preserves the two-lobed structure of the Lorenz and is a valid embedding.

An immersion is a local embedding that may not preserve the global topology of a manifold. Rather an immersion preserves the topology of every local neighborhood of the original manifold, so that each point of the tangent space of the immersed manifold has the same dimensionality as the true manifold. Thus, an immersion is a mapping that is Inline graphic between any given “piece” of the true manifold and the immersed manifold. However, this condition does not guarantee that the global topology is preserved. This is illustrated in Figure 1(c), where two different pieces of the original manifold are mapped to the same piece of the immersed manifold, producing an immersion that is not an embedding. Immersions are nonetheless a useful conceptual stepping stone for constructing proofs about embeddings, since all embeddings are necessarily immersions.

The Lorenz attractor, Figure 1(a), provides an excellent example to illustrate both of these concepts. Consider two different multivariate functions that transform the original manifold, Inline graphic and Inline graphic where Inline graphic is a small time lag as in Takens' theorem. Both of these functions map points on the true manifold to points on a shadow manifold, shown in Figures 1(b) and 1(c). Examining these shadow manifolds, it is evident that both are immersions of the Lorenz attractor, because zooming in on a particular piece of either will reveal that the tangent spaces have the same dimensionality as the original. However, only Figure 1(b) is an embedding that successfully reproduces the two lobes of the butterfly. The reconstruction in Figure 1(c), based on lags of the Inline graphic-coordinate, fails to do so, because the two fixed points of the original attractor have the same Inline graphic-coordinate; they are mapped to the same point on the shadow manifold, so the two lobes are stacked on top of each other. This singularity is a consequence of a special, non-generic symmetry in the Lorenz system that violates an assumption of Takens' theorem. Figure 1(d) shows an embedding based on lags of both Inline graphic- and Inline graphic-coordinates and is an example of the generalized mappings addressed in this paper.

Results

Two Theorems in the Style of Takens: The Generic Case

Let Inline graphic be a compact manifold of dimension Inline graphic. A dynamical system is a diffeophorism Inline graphic defining the trajectories or “flow” on Inline graphic for discrete time or a vector field Inline graphic on Inline graphic for continuous time. Takens [12] proved generically that given Inline graphic and Inline graphic, a smooth observation function Inline graphic can be used to construct an embedding of Inline graphic in Inline graphic dimensions under the transformation Inline graphic where Inline graphic. Here the components Inline graphic correspond to time-lagged observations of the dynamics on Inline graphic defined by Inline graphic. Notice that such mappings involve a single distinct observation function (i.e. a single time series), and represent a small subset in the larger set Inline graphicof all possible mappings Inline graphic that could, for example, involve multiple time series and their lags.

Takens explicitly refers only to the unlagged Inline graphic as an observation function, but in its most general sense an observation function is any Inline graphic. Thus, the functions Inline graphic, corresponding to the lags of the time series are technically observation functions as well. This bears mention, because in the more general case of mappings Inline graphic, the observation functions making up the components of Inline graphic are not all derived from a single time series, but can be various lags of multiple time-series. To treat these cases, it is necessary to acknowledge that these are all observation functions, and we will refer to distinct time series as “unlagged” observation functions.

For a mapping Inline graphic in the larger set Inline graphic of all mappings Inline graphic, consider the case with Inline graphic component functions Inline graphic which are multiple unlagged observation functions of Inline graphic (i.e. multiple time series). Again, an observation function is any function Inline graphic that assigns a real number to each point on the manifold Inline graphic. For a mapping Inline graphic, we can think of Inline graphic in terms of its Inline graphic component functions, which correspond to the coordinates in Inline graphic. These component functions may all be lags of a single distinct observation function tracking a dynamical system, as in the case of Takens, or they may be multiple observation functions, as in the case of Whitney, or they may be lags of multiple observation functions, as in Theorems 2 and 7 below.

The question arises whether general multivariate mappings Inline graphic form legitimate embeddings. Here we present two theorems: one that demonstrates that maps created from Inline graphic distinct observation functions are generically embeddings and another that shows that maps created from lags of multiple observation functions are also generically embeddings. Both of these theorems generalize Takens' theorem for which the component functions only involve a single observation function.

It follows from Whitney [38] that generically Inline graphic is an embedding. Note, however, that Whitney's work does not apply to the specific subsets of Inline graphic involving fixed lagged relationships as discussed by Takens for reconstructing attractor manifolds Inline graphic for dynamic systems. That is, Whitney's theorem is generic and does not address these specific subsets of Inline graphic which have “measure zero” (e.g. in the sense of “shy” defined in [30]). To tackle this problem, we look to the proof of Takens and see that it can be readily generalized to the other subsets of Inline graphic, including the case of generic Inline graphic.

Recall that, for a compact manifold, a mapping that is an immersion and injective is also necessarily an embedding. Thus, Takens' general approach was to first show that (i) immersions are dense in the set of mappings Inline graphic, then that (ii) there is a dense set of Inline graphic mappings within this set of immersions. Since the set of embeddings is open in the set of all possible mappings, Takens concludes that mappings in Inline graphic are generically embeddings. The critical word here is “generically,” meaning there can be exceptions (and as explained in [30], the set of such exceptions doesn't necessarily have zero measure).

To demonstrate both (i) and (ii), Takens argues that even when the property of interest (e.g. the Inline graphic property) does not hold for some particular mapping, by making an arbitrarily small perturbation, it is possible to find a nearby mapping for which that property holds. The key to the theorem and also to adapting it to other sets of mappings is finding how to make these perturbations. The proof is most straightforward for the general case involving Inline graphic distinct observation functions (each a distinct time series) because it is possible to perturb the component functions of Inline graphic independently. Thus we begin with this proof to add clarity to the more powerful main theorem 2 involving lags of multiple observation functions.

Theorem 1

Consider a compact, Inline graphic -dimensional manifold Inline graphic and a set of Inline graphic observation functions Inline graphic , where Inline graphic smoothly; by “smooth” we mean at least Inline graphic . Then it is a generic property of all possible Inline graphic that the mapping Inline graphic defined as

graphic file with name pone.0018295.e095.jpg

is an embedding.

Proof

Consider an arbitrary set of Inline graphic observation functions Inline graphic on Inline graphic. We define a corresponding mapping Inline graphic by letting each of these Inline graphic observation functions be one of the component functions of Inline graphic. Now, recall that an immersion is a map with a derivative that is globally injective, i.e. Inline graphic. We denote the total derivative of a function Inline graphic as Inline graphic. If the derivative is evaluated at a particular point Inline graphic in the domain of Inline graphic, we will write Inline graphic, and if Inline graphic is a matrix, then we denote the derivative at a particular point and along a particular tangent vector Inline graphic as Inline graphic.

For any point Inline graphic, we can perturb the co-vectors Inline graphic independently by perturbing individual Inline graphic. By making infinitesimal perturbations at points Inline graphic for which Inline graphic, we can get a set of observables Inline graphic arbitrarily close to Inline graphic such that Inline graphic for all Inline graphic—i.e., Inline graphic is an immersion. Since the set of immersions is open in the set of all mappings, there is a neighborhood Inline graphic around this Inline graphic such that every Inline graphic is an immersion.

Since immersions are local embeddings, we can find a Inline graphic such that on the manifold, Inline graphic implies Inline graphic. Here we depart from Takens' notation and let Inline graphic denote infinitesimal separations between two points on the manifold Inline graphic to avoid confusion with the later defined Inline graphic which is used to perturb the observable; Inline graphic is any fixed metric on Inline graphic. In fact for this fixed Inline graphic, there is a subset Inline graphic such that for any Inline graphic in Inline graphic, the associated map Inline graphic is an immersion, and Inline graphic implies that Inline graphic.

Next, we show that we can find a globally Inline graphic Inline graphic arbitrarily close to Inline graphic . To do this, we construct a finite collection of subsets Inline graphic such that the Inline graphic are open subsets of Inline graphic, the collection covers Inline graphic, and Inline graphic for every Inline graphic. Then, we take a partition of unity Inline graphic corresponding to these Inline graphic, so that we can vary the value of any Inline graphic by an infinitesimal amount Inline graphic without altering the value of Inline graphic for Inline graphic.

We now consider the mapping Inline graphic defined as Inline graphic. We define the set Inline graphic as Inline graphic, so that (by our choice of Inline graphic), the mapping Inline graphic is necessarily injective on the complement of Inline graphic in Inline graphic. Furthermore, note that the intersection of Inline graphic with the diagonal of Inline graphic gives the set of points Inline graphic, and therefore Inline graphic is equivalent to Inline graphic injective. Our task, then, is to perturb the manifold Inline graphic using the Inline graphic and Inline graphic so that it does not intersect the diagonal manifold Inline graphic.

At each Inline graphic we know that Inline graphic, so Inline graphic and Inline graphic cannot belong to the same Inline graphic. Consequently, varying an Inline graphic or Inline graphic only alters the value of Inline graphic at either Inline graphic or Inline graphic (respectively). In the tangent space Inline graphic, then, the direction of the Inline graphic infinitesimal changes given by the Inline graphic and Inline graphic are all linearly independent (indeed orthogonal) and as such span Inline graphic. Since the tangent spaces of Inline graphic and Inline graphic are at most Inline graphic and Inline graphic dimensional, respectively, we can construct a vector from a linear combination of Inline graphic and Inline graphic that lies outside of both Inline graphic and Inline graphic. Therefore, an infinitesimal perturbation corresponding to this linear combination will move the sub-manifolds Inline graphic and Inline graphic away from each other at the point Inline graphic without creating a new intersection at another point. By keeping the size of these perturbations sufficiently small, we ensure that we stay confined to Inline graphic, so that Inline graphic is still an immersion. This is a more transparent statement of the transversality argument used in the Takens proof (1981).

Thus, we have shown that for any arbitrary set of Inline graphic observables Inline graphic, we can find a set of observables Inline graphic arbitrarily close to Inline graphic such that Inline graphic is an embedding—i.e., there is a dense set of observables Inline graphic such that Inline graphic is an embedding. The set of embeddings is open in the set of all mappings, so this set is dense and open, meaning that the embedding property is generic over all mappings.

When mappings are confined to fixed lag relationships, Takens showed it is valid to independently perturb each component of Inline graphic at a given point of the domain by perturbing the unlagged observation function, Inline graphic, in the other parts of the domain corresponding to neighborhoods of the lagged states Inline graphic, Inline graphic, etc. This ensures that the perturbations to Inline graphic maintain the structure of the lag relationships and that we have not inadvertently left the subset of interest. As we now show, this allows the above result to be easily extended to families of maps having component functions that are the lags of multiple observation functions. This is the relevant case for many practical examples where lags of multiple time series (multiple variables or observation functions) are required to achieve a mechanistic reconstruction of Inline graphic (e.g. [20]). It also allows information on Inline graphic to be leveraged when the time series are short, as is the case in many physical and biological problems [22], [36].

Before starting the proof, however, we must clarify exactly what the “subsets of interest” are. We define these sets as follows. First, we say Inline graphic is a lag of the observable Inline graphic if we can write Inline graphic for positive Inline graphic. We consider the lags in the positive time direction only to simplify notation in the proof, noting that the results apply equally to negative lags. Let Inline graphic be the subset of Inline graphic for which Inline graphic, Inline graphic is an unlagged observable, i.e. Inline graphic is not a lag of another Inline graphic. We begin with the “unlagged” observation functions, Inline graphic, or observation functions that are not a lag of another observable in Inline graphic. Now define a set Inline graphic for each Inline graphic that contains Inline graphic and any other observation function in Inline graphic which is a lag of it. That is, Inline graphic is the set of Inline graphic that are lags of Inline graphic given as Inline graphic, where the lags Inline graphic are distinct for fixed Inline graphic. This choice of Inline graphic and Inline graphic determine a subset Inline graphic containing all choices of Inline graphic observables Inline graphic which obey the correct lag relationships under a dynamical system Inline graphic. Note that each element of Inline graphic can be identified by the dynamical system and the Inline graphic. We denote such an element, then, as Inline graphic.

Theorem 2

Consider a diffeomorphism Inline graphic on some compact manifold Inline graphic of dimension Inline graphic , along with Inline graphic observation functions Inline graphic , smoothly; by “smooth” we mean at least Inline graphic . Restrict the Inline graphic to have the lag relationships corresponding to a collection of sets Inline graphic and lags Inline graphic under the dynamical system Inline graphic , and impose the following generic [12], [39] properties on Inline graphic:

  1. The set Inline graphic of periodic points with period Inline graphic has finitely many points,

  2. The eigenvalues of Inline graphic at each Inline graphic in a compact neighborhood Inline graphic are distinct and not equal to 1.

Then, for generic Inline graphic , the mapping described by

graphic file with name pone.0018295.e261.jpg

is an embedding.

Proof

The proof of this theorem closely follows the logic of the previous proof and the original argument of Takens [12]. As noted above, any perturbations to Inline graphic via its component functions Inline graphic must remain within Inline graphic (the set of observables having the desired lag relationships under Inline graphic prescribed by the Inline graphic and the Inline graphic). Here we must also deal with points of Inline graphic that are fixed points or periodic under the dynamical system Inline graphic, i.e. the points for which there exists a Inline graphic such that Inline graphic (including the fixed point case, Inline graphic). The above proof shows that the mapping Inline graphic is generically an immersion because the co-vectors Inline graphic can be independently perturbed. This is also true for non-periodic points where there are fixed lag relationships between some observables, as we can perturb Inline graphic in the neighborhood of Inline graphic and thus perturb Inline graphic without affecting Inline graphic in the neighborhood of Inline graphic.

Note that periodic points Inline graphic can exist such that the period Inline graphic or some integer multiple of it, Inline graphic, is the fixed time lag between two observables Inline graphic belonging to the same Inline graphic. Let Inline graphic be a compact neighborhood of all such points. For Inline graphic, the vectors Inline graphic and Inline graphic cannot necessarily be perturbed independently. Nonetheless, while Inline graphic for such a point, it is not generally true that Inline graphic. By assumption, for each Inline graphic, the eigenvalues of the Inline graphic are distinct and not equal to 1. Thus, by the chain rule, it is clear that Inline graphic and Inline graphic are linearly independent. As noted above, all the other Inline graphic can be perturbed independently, so we can find a set of observables Inline graphic arbitrarily near Inline graphic in Inline graphic for which Inline graphic is an immersion on Inline graphic. Note that because the set of immersions is open, there is an open neighborhood in Inline graphic around this Inline graphic for which every set of observables in that neighborhood gives an immersion.

We must also satisfy Inline graphic injective. The proof above relied on the ability to independently perturb the manifold Inline graphic at any point Inline graphic by an infinitesimal amount in any coordinate direction. For a periodic point on Inline graphic with perioid Inline graphic and two observables related as Inline graphic and Inline graphic, it is impossible to independently perturb Inline graphic locally in the coordinate Inline graphic or Inline graphic, as you also perturb Inline graphic or Inline graphic. By assumption, the set Inline graphic has a finite number of elements. For such a generic Inline graphic and any set Inline graphic, any neighborhood of the Inline graphic will contain a set of observables Inline graphic for which the unlagged observation functions Inline graphic take distinct values at each point in Inline graphic.

We first perturb the Inline graphic to find an open neighborhood of observables which give immersions when restricted to the set Inline graphic. We then further perturb the observables to find within this neighborhood a set of observables Inline graphic for which Inline graphic is also injective and therefore an embedding (on Inline graphic). Since embeddings are dense in the space of all mappings, there is a neighborhood Inline graphic such that for all Inline graphic, the map Inline graphic is an embedding.

We now show that we can find a Inline graphic such that Inline graphic is an embedding on all of Inline graphic. We first note that at points Inline graphic, the vectors Inline graphic can be perturbed independently, so we can find Inline graphic for which Inline graphic is an immersion. Because an immersion is a local embedding, there is a Inline graphic such that for Inline graphic, Inline graphic implies that Inline graphic. Since the set of immersions is open in the set of possible mappings, there is a neighborhood Inline graphic such that for any Inline graphic, the corresponding mapping Inline graphic is an immersion. Thus, for the same Inline graphic as above, Inline graphic implies Inline graphic.

Now we need to show that there is a Inline graphic such that Inline graphic is also injective on Inline graphic. As noted in the first proof, this is equivalent to Inline graphic for the mapping Inline graphic defined as Inline graphic. If Inline graphic and Inline graphic are both in Inline graphic or Inline graphic, we already know that Inline graphic. Thus we restrict ourselves to the set Inline graphic.

To perturb the manifold Inline graphic away from Inline graphic at points of intersection, Inline graphic, we must be able to find variations for which the tangent vector Inline graphic is linearly independent from the Inline graphic tangent vectors Inline graphic and Inline graphic and lies outside of Inline graphic. In the first proof, it was obvious that each component of Inline graphic could be perturbed independently. Now we must be more careful. We do this by first creating a collection of Inline graphic open subsets of Inline graphic, Inline graphic, with the following properties:

  1. The Inline graphic cover the closure of Inline graphic.

  2. For each Inline graphic and Inline graphic, the diameter of Inline graphic is less than Inline graphic.

  3. For all choices of Inline graphic, the set Inline graphic intersects with Inline graphic for at most one Inline graphic.

  4. For Inline graphic and Inline graphic such that Inline graphic for some Inline graphic, Inline graphic, and Inline graphic, no two of Inline graphic belong to the same Inline graphic.

Take a partition of unity Inline graphic corresponding to this Inline graphic. Because of the way we constructed the Inline graphic, we can vary the value of each Inline graphic at any point Inline graphic by an infinitesimal amount without altering the value of the other Inline graphic in the neighborhood of Inline graphic. We make this explicit as follows. To perturb the Inline graphic, we take Inline graphic for Inline graphic corresponding to Inline graphic. To perturb the other Inline graphic (Inline graphic for some Inline graphic), we perturb Inline graphic for Inline graphic corresponding to Inline graphic . Consider the Inline graphic perturbations, Inline graphic, which are independent shifts at Inline graphic in distinct Inline graphic. In Inline graphic, we note that each corresponding tangent vector Inline graphic lies outside of Inline graphic. Note the Inline graphic together with any basis of Inline graphic form a linearly independent set of vectors. Since the dimension of Inline graphic is at most Inline graphic, there must be a linear combination of the Inline graphic that lies outside of both Inline graphic and Inline graphic, which can be used to perturb Inline graphic away from Inline graphic. By keeping variations in the Inline graphic sufficiently small, we can find a set of Inline graphic such that Inline graphic and Inline graphic (where Inline graphic now corresponds to the Inline graphic map). This pair gives a mapping Inline graphic that is both an immersion and injective, and thus is an embedding. Because Inline graphic was an arbitrarily small neighborhood of any point in Inline graphic, this means embeddings are dense in Inline graphic, and the set of embeddings is open in the set of mappings. Thus, the map Inline graphic given by Inline graphic is generically an embedding.

Just as Takens extends the original result for discrete time to dynamical systems in continuous time, we can extend our result as follows:

Corollary 3

Consider a smooth vector field Inline graphic on some compact manifold Inline graphic along with Inline graphic observables Inline graphic , smoothly; by “smooth” we mean at least Inline graphic . Define Inline graphic as the flow on Inline graphic . Suppose we restrict the Inline graphic to have the lag relationships corresponding to a collection of sets Inline graphic and lags Inline graphic under the discrete dynamical system Inline graphic , where Inline graphic is a constant. We impose the following generic properties on Inline graphic :

  1. For points Inline graphic such that Inline graphic , the eigenvalues of Inline graphic are distinct and not equal to 1.

  2. No periodic integral curve of Inline graphic has integer period Inline graphic .

Then, for generic Inline graphic , the mapping described by

graphic file with name pone.0018295.e453.jpg

is an embedding.

Proof

In this case, Inline graphic is a discrete time dynamical system on Inline graphic satisfying the conditions imposed in the theorem above, and this corollary follows directly.

A Theorem in the Style of Sauer et al.: The Prevalent Case

We now give an explicit proof of Remark 2.9 from [29] using the framework constructed in their original paper, but we extend the language to cover reconstructions using non-consecutive lags (from multiple time series). The proof uses Lemma 4.1, 4.6, and 4.11 from [29] to show that Inline graphic mappings and immersions are prevalent in the space Inline graphic, just as Sauer et al. use Lemma 4.6 to prove Theorem 3.3, and Lemmas 4.1 and 4.11 to prove Theorem 3.5. These lemmas are now stated (for the proofs, see their original paper).

Lemma 4

(Originally part 2 of 4.1) Let Inline graphic and Inline graphic be positive integers, Inline graphic distinct points in Inline graphic , Inline graphic in Inline graphic , and Inline graphic in Inline graphic . Then there exists a polynomial Inline graphic in Inline graphic variables of degree at most Inline graphic such that for Inline graphic , Inline graphic .

Lemma 5

(Originally 4.6) Let Inline graphic be a compact subset of Inline graphic . Let Inline graphic be Lipschitz maps. For each integer Inline graphic , let Inline graphic be the set of pairs Inline graphic in Inline graphic for which the Inline graphic matrix

graphic file with name pone.0018295.e479.jpg

has rank Inline graphic , and let Inline graphic lower boxdim Inline graphic . Define Inline graphic . If Inline graphic for all integers Inline graphic , then for Inline graphic outside a measure zero subset of Inline graphic , the map Inline graphic is Inline graphic .

Lemma 6

(Originally 4.11) Let Inline graphic be a compact subset of a smooth manifold embedding in Inline graphic . Let Inline graphic be a set of smooth maps from an open neighborhood Inline graphic of Inline graphic to Inline graphic . For each positive integer Inline graphic , let Inline graphic be the subset of the unit tangent bundle Inline graphic such that the Inline graphic matrix

graphic file with name pone.0018295.e500.jpg

has rank Inline graphic , and let Inline graphic lower boxdim Inline graphic . Define Inline graphic . If Inline graphic for all integers Inline graphic , then for almost every Inline graphic , the map Inline graphic is an immersion on Inline graphic .

To apply these lemmas, it is necessary to restrict the dimension of the sets of periodic orbits—that is, the sets Inline graphic for Inline graphic. For the case of consecutive lags, Sauer et al. state sufficient conditions to be boxdim Inline graphic. A sufficient condition for non-consecutive lags is a bit more complicated. Define the constants Inline graphic such that Inline graphic for at least one Inline graphic and Inline graphic. Also, define Inline graphic. A sufficient condition on the Inline graphic is Inline graphic.

Theorem 7

Let Inline graphic be a diffeomorphism on an open subset Inline graphic of Inline graphic , and let Inline graphic be a compact subset of Inline graphic , Inline graphic . Let Inline graphic be a collection of sets and Inline graphic a set of lag relationships as above, such that Inline graphic . Assume that for every positive integer Inline graphic , the set Inline graphic of periodic points of period Inline graphic satisfies Inline graphic , and that for each point of Inline graphic , the Jacobian Inline graphic has distinct eigenvalues. Then, for almost every set of Inline graphic observation functions Inline graphic satisfying the given lag relationships, the map

graphic file with name pone.0018295.e537.jpg

is an embedding on Inline graphic .

Proof

Without loss of generality, assume we have ordered the components of Inline graphic with Inline graphic and all its lags first, then Inline graphic and its lags, etc. That is,

graphic file with name pone.0018295.e542.jpg

To show prevalence, we find a suitable probe space (see [29]). The infinite dimensional space for the univariate theorem is the observation functions Inline graphic, smoothly. For maps constructed from multiple lags, this becomes the sets of Inline graphic unlagged observation functions. Sauer et al. take the probe space for the univariate theorem to be any set Inline graphic of polynomials in Inline graphic variables which include all such polynomials up to degree Inline graphic. It is now necessary to have a set of polynomials for each of the Inline graphic. Thus, we take the probe space for this theorem to be the Cartesian product of Inline graphic copies of Inline graphic.

Let Inline graphic be a basis for Inline graphic. We want to show that for almost all choices of Inline graphic coefficients Inline graphic, the map Inline graphic defined by the observation functions Inline graphic is an embedding. We first demonstrate that almost every Inline graphic is Inline graphic, proceeding as in the proof of Theorem 4.3 in [29].

To sensibly apply Lemma 5, we adopt the following convention: think of Inline graphic as a perturbation of Inline graphic, which is the summed effect of perturbations on each Inline graphic separately. For each pair Inline graphic, Inline graphic and Inline graphic, there is a map Inline graphic which is Inline graphic for Inline graphic if Inline graphic and Inline graphic otherwise. The components of Inline graphic are either Inline graphic or of the form Inline graphic. Consequently, Inline graphic, which matches the structure Lemma 5.

We now check that the rank of the matrix Inline graphic satisfies the conditions of Lemma 5 for each pair of distinct Inline graphic. Note that to avoid confusion with the previous section of this paper and Takens' original work, we continue to use row vectors to describe the transformations Inline graphic. However, Sauer et al. [29] prefer column vectors, so it is necessary to use of transposes in several instances. Thus, we have

graphic file with name pone.0018295.e577.jpg

Note that Inline graphic is a block diagonal matrix, and so it has rank equal to the sum of the rank of the blocks. Each of the Inline graphic blocks can be rewritten as the product of two matrices, Inline graphic and Inline graphic, where the entries of Inline graphic are values of a single polynomial Inline graphic and the entries of Inline graphic are each one of Inline graphic. Note, there are multiple possible choices for Inline graphic and Inline graphic that give the same Inline graphic.

Case 1: First consider Inline graphic and Inline graphic that do not both lie in a periodic orbit of integer period less than Inline graphic. We specify Inline graphic so that the first Inline graphic rows, where Inline graphic is the size of the set Inline graphic, correspond to the Inline graphic, and the next Inline graphic correspond to the Inline graphic. Inline graphic is onto, so the rank of Inline graphic is just the sum of the ranks of the Inline graphic. For this case, Inline graphic contains a copy of Inline graphic, and thus will have rank Inline graphic. The entire matrix Inline graphic will thus have rank Inline graphic, which satisfies the conditions of Lemma 5.

Case 2: Now consider Inline graphic and Inline graphic in separate periodic orbits with periods Inline graphic and Inline graphic such that Inline graphic and Inline graphic. Inline graphic will have Inline graphic fewer rows corresponding to the Inline graphic for some Inline graphic (there will also be a reduction in the number of rows associated with Inline graphic). In this case, Inline graphic will still contain the column space of Inline graphic and thus Inline graphic. Again the Inline graphic are onto, and so the rank of Inline graphic is the rank of Inline graphic.

The dimension of the set Inline graphic of all pairs Inline graphic and Inline graphic is Inline graphic. By the conditions placed on the size of the Inline graphic, we can conclude that Inline graphic, and thus that Lemma 5 applies to this case as well.

Case 3: Finally we consider Inline graphic and Inline graphic in the same Inline graphic-periodic orbit, Inline graphic. Now the matrix Inline graphic becomes more complicated, since some of the Inline graphic pertaining to Inline graphic may be equal to Inline graphic pertaining to Inline graphic. Consequently, the Inline graphic are no longer guaranteed to contain the column space of the identity. Each Inline graphic does contain the column space of an Inline graphic dimensional matrix with Inline graphic along the upper diagonal and a single Inline graphic off the diagonal in each column. Using elementary operations, it is possible to make the first Inline graphic columns of Inline graphic upper diagonal for some integer Inline graphic. Thus, the rank of each Inline graphic is at least Inline graphic and the entire matrix has Inline graphic.

The dimension of the set Inline graphic of all such Inline graphic and Inline graphic is just Inline graphic. By the imposed conditions, Inline graphic, and Lemma 5 applies.

Now we want show that almost every Inline graphic is an immersion. We check that the matrix

graphic file with name pone.0018295.e656.jpg

has full rank and thus satisfies the conditions of Lemma 6 for each Inline graphic in the tangent bundle Inline graphic. Note that this is a block diagonal matrix with Inline graphic blocks, so it is sufficient to show that the columns of the Inline graphicth block span the subspace Inline graphic for Inline graphic. We consider two cases.

Case 1: Consider first the subset Inline graphic of Inline graphic that are not periodic with period Inline graphic. The entries of each block are of the form Inline graphic. Since Inline graphic is a diffeomorphism and Inline graphic, we know that Inline graphic. Furthermore, the Inline graphic are distinct points. Examining Lemma 4, it is clear that the columns span Inline graphic. The dimension of Inline graphic is at most Inline graphic, so we may apply Lemma 6.

Case 2: Now consider the subset Inline graphic of Inline graphic that are periodic with period Inline graphic. By the conditions of the theorem, Inline graphic has distinct eigenvalues from Inline graphic. Therefore, Inline graphic. Furthermore, the relationship depends on Inline graphic, and again referencing Lemma 4, it is clear that the columns span Inline graphic. The dimension of Inline graphic is certainly less than Inline graphic, so we can safely apply Lemma 6.

Theorem 7 can be extended to continuous dynamical systems (smooth vector fields on a manifold) by letting the flow Inline graphic of Inline graphic be Inline graphic in the statement of the theorem.

Discussion

Theorem 1 and the more general result presented in Theorem 2 (and its corollary) were given proofs intended to follow those presented by Takens. The original “transversality” argument, however, has been replaced with what we reckon is a simpler and more direct argument. These clarify how perturbations to the observation functions can be constructed and highlight why Inline graphic dimensions are necessary to have mappings that are generically embeddings. Theorem 7 is similar to Theorem 2, but takes advantage of the more powerful framework, built around the notion of prevalence, established by Sauer et al. [29]. It also provides more specific conditions on the periodic orbits than Theorem 2 and thus can be applied to certain non-generic situations that Takens' original framework would exclude. Namely, the set of periodic points need not be finite (as required in Takens' original theorem and Theorem 2), so long as the dimensionality does not exceed the bounds stated in Theorem 7. Theorem 7 is an extension of Remark 2.9 in [29], which we explicitly proved by determining a sufficient restriction for the periodic orbits when the lags composing Inline graphic aren't necessarily consecutive.

This work also develops a language to describe a wider family of cases for reconstructing state space manifolds from multiple observational time series to encourage wider applicability of SSR in the natural sciences. For example, these results can be extended to another special case of interest for reconstructions using time derivatives [40], when multiple observation functions are available. The argument for this case is analogous to that used by Takens [12] for the case when all the derivatives are from a single observation function. Furthermore, these theorems validate heuristic work using spatial lag reconstructions and mixed spatial and temporal lag reconstructions to study spatially coupled dynamics [41].

More importantly, in terms of future applications, Theorems 2 and 7 set the stage for practical reconstruction of state space manifolds from multiple observation functions. This is significant in answering objections to single variable state space reconstruction (SSR) concerning the excessive phenomenology of lagged-coordinate embeddings [26]. These two theorems provide proof of principle for modeling attempts of nonlinear dynamics in the natural sciences involving multiple time series (e.g. [20]), and lays bare the rather non-restrictive assumptions required in such applications for building mechanistic models from multiple time series variables. Moreover, it gives support to the notion of using multiple embeddings as a potentially efficient way of extracting information from time series data of limited length, but where there are potentially many simultaneous observations of dynamics on the same attractor manifold. By reducing correlations in noise between the reconstructed coordinates, these techniques should allow reconstructions to exceed the limitations placed on univariate methods [35], as heuristic examples have already suggested [20]. The potential information leverage provided by multiple embeddings possible from novel combinations of variables (and their lags) can pave the way for a plethora of new applied techniques to exploit the time-limited, but parallel observations of nature [36]. This paper is intended to complement the existing literature on SSR and help promote this potential growth area in the natural sciences.

Acknowledgments

We wish to thank Hao Ye, James Crutchfield, John Melack, Donald DeAngelis, Simon Levin, J. Doyne Farmer, Martin Casdagli, Tim Sauer, Sarah Glaser, Chih-hao Hsieh, Stephen Munch and Charles Peretti, Michael Fogarty, Alec MacCall, Andrew Rosenberg, Les Kaufman, and Irit Altman for helpful comments and editorial advice.

Footnotes

Competing Interests: The authors have declared that no competing interests exist.

Funding: This work was supported by National Science Foundation (NSF) DEB1020372, NSF-NOAA CAMEO program (a partnership between the NSF and NOAA) NA08OAR4320894/CAMEO, EPA-STAR Fellowship, Sugihara Family Trust, Deutsche Bank-Jameson Complexity Studies Fund, and the McQuown Chair in Natural Sciences, University of California San Diego. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Casdagli M. Nonlinear prediction of chaotic time series. Physica D. 1989;35:335–356. [Google Scholar]
  • 2.Crutchfield JP. Prediction and stability in classical mechanics. 1979. Senior thesis in physics and mathematics, University of California, Santa Cruz.
  • 3.Farmer JD, Sidorowich JJ. Predicting chaotic time series. Physical Review Letters. 1987;59:845–848. doi: 10.1103/PhysRevLett.59.845. [DOI] [PubMed] [Google Scholar]
  • 4.Jenouvrier S, Weimerskirch H, Barbraud C, Park YH, Cazelles B. Evidence of a shift in the cyclicity of antarctic seabird dynamics linked to climate. Proceedings of the Royal Society of London, Series B. 2005;272:887–895. doi: 10.1098/rspb.2004.2978. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Lo TT, Hsu HH. Change in the dominant decadal patterns and the late 1980s abrupt warming in the extratropical northern hemisphere. Atmospheric Science Letters. 2010;11:210–215. [Google Scholar]
  • 6.Planque B, Batten SD. Calanus finmarchicus in the north atlantic: the year of Calanus in the context of interdecadal change. ICES Journal of Marine Science. 2000;57:1528–1535. [Google Scholar]
  • 7.Ramanathan A, Wang C, Schreiber SL. Perturbational profiling of a cell-line model of tumorigenesis by using metabolic measurements. Proceedings of the National Academy of Sciences of the United States of America. 2002;102:5992–5997. doi: 10.1073/pnas.0502267102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Ruelle D. Chaotic evolution and strange attractors: the statistical analysis of time series for deterministic nonlinear systems. 1989. Cambridge University Press, Cambridge.
  • 9.Scheffer M, Carpenter SR. Catastrophic regime shifts in ecosystems: linking theory to observation. Trends in Ecology & Evolution. 2003;18:648–656. [Google Scholar]
  • 10.Seth AK, Izhikevich E, Reeke GN, Edelman GM. Theories and measures of consciousness: an extended framework. Proceedings of the National Academy of Sciences of the United States of America. 2006;103:10799–10804. doi: 10.1073/pnas.0604347103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Soramaki K, Bech ML, Arnold J, Glass RJ, Beyeler WE. The topology of interbank payment flows. Physica A. 2007;379:317–333. [Google Scholar]
  • 12.Takens F. Detecting strange attractors in turbulence. In: Rand DA, Young LS, editors. Symposium on Dynamical Systems and Turbulence. Berlin: Springer-Verlag; 1981. pp. 366–381. , volume 898 of Lecture Notes in Mathematics. [Google Scholar]
  • 13.Wagner BK, Kitami T, Gilbert TJ, Peck D, Ramanathan A, et al. Large-scale chemical dissection of mitochondrial function. Nature Biotechnology. 2008;26:343–351. doi: 10.1038/nbt1387. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.May RM, Levin SA, Sugihara G. Complex systems: Ecology for bankers. Nature. 2008;451:893–895. doi: 10.1038/451893a. [DOI] [PubMed] [Google Scholar]
  • 15.Sugihara G. On early warning signs. 2010. Seed Magazine Global Reset 2010.
  • 16.Sugihara G. Nature is nonlinear. Kyoto Journal. 2010;75:56–57. [Google Scholar]
  • 17.Brock WA, Sayers CL. Is the business cycle characterized by deterministic chaos? Journal of Monetary Economics. 1988;22:71–90. [Google Scholar]
  • 18.Brock WA, Malliaris AG. Differential Equations, Stability and Chaos in Dynamic Economics, volume 27 of Advanced Textbooks in Economics. North-Holland 1989 [Google Scholar]
  • 19.Brock WA, Hsieh DA, LeBaron BD. Nonlinear Dynamics, Chaos, and Instability: Statistical Theory and Economic Evidence. 1991. MIT Press, Cambridge, Massachusettes.
  • 20.Dixon PA, Milicich MJ, Sugihara G. Episodic fluctuations in larval supply. Science. 1999;283:1528–1530. doi: 10.1126/science.283.5407.1528. [DOI] [PubMed] [Google Scholar]
  • 21.Hsieh CH, Glaser SM, Lucas AJ, Sugihara G. Distinguishing random environmental fluctuations from ecological catastrophes for the North Pacific Ocean. Nature. 2005;435:336–340. doi: 10.1038/nature03553. [DOI] [PubMed] [Google Scholar]
  • 22.Hsieh CH, Anderson C, Sugihara G. Extending nonlinear analysis to short ecological time series. American Naturalist. 2008;171:71–80. doi: 10.1086/524202. [DOI] [PubMed] [Google Scholar]
  • 23.Rodo X, Pascual M, Fuchs G, Faruque ASG. ENSO and cholera: a nonstationary link related to climate change? Proceedings of the National Academy of Sciences of the United States of America. 2002;99:12901–12906. doi: 10.1073/pnas.182203999. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Schaffer WM. Stretching and folding in lynx fur returns: evidence for a strange attractor in nature? American Naturalist. 1984;124:789–820. [Google Scholar]
  • 25.Sugihara G, Grenfell B, May RM. Distinguishing error from chaos in ecological time series. Philosophical Transactions of the Royal Society of London, Series B: Biological Science. 1990;330:235–250. doi: 10.1098/rstb.1990.0195. [DOI] [PubMed] [Google Scholar]
  • 26.Sugihara G, May RM. Nonlinear forecasting as a way of distinguishing chaos from measurement error in a data series. Nature. 1990;344:734–741. doi: 10.1038/344734a0. [DOI] [PubMed] [Google Scholar]
  • 27.Sugihara G. Nonlinear forecasting for the classification of natural time series. Philosophical Transactions of the Royal Society of London, Series A. 1994;348:477–495. [Google Scholar]
  • 28.Packard NH, Crutchfield JP, Farmer JD, Shaw RS. Geomtery from a time series. Physical Review Letters. 1980;45:712–716. [Google Scholar]
  • 29.Sauer T, Yorke JA, Casdagli M. Embedology. Journal of Statistical Physics. 1991;65:579–616. [Google Scholar]
  • 30.Hunt BR, Sauer T, Yorke JA. Prevalence: a translation-invariant “almost every” on infinite-dimensional spaces. Bulletin of the American Mathematical Society. 1992;27:217–238. [Google Scholar]
  • 31.Stark J, Broomhead DS, Davies ME, Huke JP. Takens embedding theorems for forced and stochastic systems. Nonlinear Analysis: Theory, Methods, and Applications. 1997;30:5303–5314. [Google Scholar]
  • 32.Stark J, Broomhead DS, Davies ME, Huke JP. Delay embeddings for forced systems: II. Stochastic forcing. Journal of Nonlinear Science. 2003;13:519–577. [Google Scholar]
  • 33.Stark J. Delay embeddings for forced systems: I. Deterministic forcing. Journal of Nonlinear Science. 1999;9:255–332. [Google Scholar]
  • 34.Gibson J, Farmer JD, Casdagli M, Eubank S. An analytic approach to practical state space reconstruction. Physica D. 1992;57:1–30. [Google Scholar]
  • 35.Casdagli M, Eubank S, Farmer JD, Gibson J. State space reconstruction in the presence of noise. Physica D. 1991;41:52–98. [Google Scholar]
  • 36.Ye H, Sugihara G, Deyle ER. Leverging information from multiple time series with ensemble state space reconstructions. 2011. In preparation.
  • 37.Lorenz EN. Deterministic nonperiodic flow. Journal of Atmospheric Science. 1963;20:130–141. [Google Scholar]
  • 38.Whitney H. Differentiable manifolds. Annals of Mathematics. 1936;37:645–680. [Google Scholar]
  • 39.Huke JP. Embedding nonlinear dynamical systems, a guide to Takens' theorem. 1993. Internal report, DRA, Malvern, UK.
  • 40.Crutchfield JP, McNamara BS. Equations of motion from a data series. Complex Systems. 1987;1:417. [Google Scholar]
  • 41.Ørstavik S, Stark J. Reconstruction and cross-prediction in coupled map lattices using spatio-temporal embedding techniques. Physics Letters A. 1998;247:145–160. [Google Scholar]

Articles from PLoS ONE are provided here courtesy of PLOS

RESOURCES