Modeling the Nonlinear Time Dynamics of Multidimensional Hormonal Systems

Abstract

In most hormonal systems (as well as many physiological systems more generally), the chemical signals from the brain, which drive much of the dynamics, can not be observed in humans. By the time the molecules reach peripheral blood, they have been so diluted so as to not be assayable. It is not possible to invasively (surgically) measure these agents in the brain. This creates a difficult situation in terms of assessing whether or not the dynamics may have changed due to disease or aging. Moreover, most biological feedforward and feedback interactions occur after time delays, and the time delays need to be properly estimated. We address the following two questions: (1) Is it possible to devise a combination of clinical experiments by which, via exogenous inputs, the hormonal system can be perturbed to new steady-states in such a way that information about the unobserved components can be ascertained; and, (2) Can one devise methods to estimate (possibly, time-varying) time delays between components of a multidimensional nonlinear time series, which are more robust than traditional methods? We present methods for both questions, using the Stress (ACTH-cortisol) hormonal system as a prototype, but the approach is more broadly applicable.

1 Introduction

Hormones play a variety of roles in both human physiology and pathology. In Figure 1 (Left) is displayed a schematic for one of the fundamental hormonal systems (or axes): the stress (ACTH-cortisol) axis. It is responsible, via the regulation of cortisol, for controlling the body’s responses to physiological stress (and to a lesser degree, psychological stress). The major obstacle to the modeling of the Stress axis (as well many physiological systems more generally) is the fact that the chemical signals from the brain that drive much of the overall dynamics, cannot be measured (at least in humans). By the time they reach peripheral blood, they have undergone such massive dilution that they cannot be assayed. This presents enormous difficulties, and makes it very hard to determine and assess relationships. Enclosed in the dashed box of Figure 1 (Left) is what is unobserved, including the associated feedback and feedforward interactions denoted by arrows.

Figure 1.

Schematic of the Stress (ACTH-cortisol) axis (Left); the arrows denote feedforward (stimulatory, (+)) and feedback (inhibitory, (−)) interactions. The hormones are corticotropin-releasing hormone (CRH), argininine vasopressin(AVP), adrenocoticotropin hormone (ACTH) and cortisol. Also, in the Middle and Right: ACTH and cortisol concentrations for a normal subject, sampled every 10 min for a day, and the deconvolved secretion rates.

Invasive surgical procedures to sample those which are ordinarily unobserved can not be done, for obvious ethical reasons. Hence, the fundamental question becomes whether or not non-invasive, statistical methods could be devised by which to acquire the needed information. For those components that are observed, blood samples can be repeatedly taken and the time profile of concentrations assayed. These concentrations must then be further deconvolved (blind deconvolution) into the underlying kinetic and secretion rates, in that it is there that the information about their function and disease are contained. In Figure 1 are the concentrations (Middle) and deconvolved secretion rates (Right) for the two observed components: ACTH and cortisol, from a normal subject sampled every 10 min for 24 hrs. CRH and AVP were unobserved.

Ordinarily, estimating the feedback and feedforward interactions which involve the unobserved would be ill-posed and not solvable. As part of this work, we present an approach, using the Stress axis as a prototype, by which a stable, partially-observed system is perturbed from its usual steady-state to multiple new ones, which in their combination, allows for the extraction of some information about the unobserved. Hence, instead of observing a subject under normal, endogenous steady-state conditions as in Figure 1 Middle, the subject would be observed under a host of new steady-states induced by exogenous infusions. Currently, such infusion studies are only in the process of being implemented, and hence no data is presently available. As a consequence we are limited to simulations of our complete system, for which individual interactions of the model have been validated (e.g., a particular feedback or feedforward arrow in Figure 1), but for which no data concerning the system as a whole is available. One aspect of the infusion experiments, within each perturbed state, involves the adminstration of physiological level and supra-physiological level boluses of AVP and/or CRH. The only study remotely similar to the present, is a study of the male reproductive system for which antagonists are available by which one can ”decouple” feedback loops. That system is composed of gonadotropin releasing hormone (GnRH) in the hypothalamus, luteinizing hormone (LH) in the pituitary and testosterone (T) in the testes. There, the feedforward drive of GnRH on LH was able to be chemically altered in a graded manner via the use of a GnRH receptor anatagonist (ganirelix) [7]. That approach is quite limited in that for most physiological systems, antagonists are not available. Hence, the present approach.

In Figure 1, the ACTH and Cortisol concentrations were deconvolved, delineating kinetics from secretion, resulting in the displayed secretion rates. We briefly describe the deconvolution methodology in Section 2.4, but the emphasis of this paper is not on those methods, however. Until the past 10 years, most endocrine modeling was directed to that of a single hormone time-profile, and not the joint dynamics of the system. The reason for this being, restriction on the amount of blood which could be withdrawn (at most .5L over a week), the amount of blood required for assays and the assay cost. The identification of abnormalities was often limited to verifying whether or not a hormone fell inside or outside its normal range. However, physiological systems are highly adaptable and will compensate for weaknesses in the early stages of pathology. Hence, it is often much later in disease that concentrations levels become abnormal. What though does change in the early stages are the dynamical interactions (e.g., the time to return to equilibrium). Accurate measures to assess the viability of a physiological system, will need to acquire information about the dynamics of the system as a whole, which brings us back to the unobserved.

The present paper, applied in nature, will focus on the following two issues: (1) (Section 2) Is it possible to devise a combination of clinical experiments by which, via exogenous inputs, the system can be perturbed to new steady-states in such a way that information about the unobserved components can be ascertained. Clinical experiments of this general paradigm are currently being designed. In animal (horse, sheep, and possibly rodent) experiments, all components can be observed, and this will serve as a test of validation of the methods; and (2) (Section 3) Can one devise methods to estimate (possibly, time-varying) time delays between components of a multidimensional nonlinear time series, which are more robust than traditional methods? The ACTH and cortisol data from Figure 1 (Middle, Right) will serve as one example in the time delay estimation section.

Endocrine systems, because they are responsible for much of the homeostatic nature of the body, are quite stable and fit the framework of multidimensional time series. Hence, time series has an important role to play in the future of endocrine methodology, but it will need to be nontraditional approaches. The reasons for this are the following, in addition to the above unobservability: (a) feedback control in biology is virtually never by a component acting on itself but via action on others (e.g., enzymes) which in turn, ultimately, alter the former; (b) the interactions between components is typically nonlinear, and there are time-delays (possibly time-varying) involved. In the 1st part of the paper (Section 2), there is very little standard time series methodology being used, mainly due to both (a)–(b) above, as well as the preliminary nature of the work. However, many of the fundamental questions in endocrinology (including Section 2) are in great need of time series modeling. Lastly, the following issue needs to be mentioned as an outstanding problem. Desensitization (or tachyphylaxis; habituation) occurs in the responsiveness of one component to another in virtually all interactions within physiological systems. It is a loss of responsiveness, sometime rapidly occurring, and rarely modeled, but it has to be accounted for in order to unmask the actual relationships. This is one of the most perplexing modeling questions confronting physiology today.

2 A Model of Hormone Dynamics

In the present paper we will use the Stress axis as a prototype. Physiological stress is a general term used to describe changes that disrupt the body’s homeostasis, and elicit defensive system responses. At the appropriate time, these defensive responses must be reversed, with homeostasis restored. Cortisol, an adrenal gland steroid, is the body’s primary mechanism for such restoration. Cortisol is controlled by ACTH, which in turn is stimulated by hypothalamic pulses of corticotropin-release hormone (CRH) and arginine vasopressin (AVP). Mathematical models of the dynamics of several hormonal systems, have been described in [9], [4], [10] and [5]. The above models were concerned with the male reproductive hormone axis (that which regulates testosterone). An earlier model of the Stress axis, which was based only upon the relationships between ACTH and cortisol, was presented in [6]. None of the preceding works included the consideration of the hypothalamic component not being observed, in that animal data for which all components were observed was often the basis of the modeling. The model in the present paper is new having not appeared elsewhere.

In general terms, the dynamics of the Stress axis are as follows. CRH and AVP are released from the hypothalamus in pulses, there being roughly 10–20 pulses per day of each. The degree to which there pulse patterns are synchronized (correlated) has not been fully resolved. CRH and/or AVP pulses then stimulate the pituitary to release an ACTH pulse, and to start synthesizing more. This hypothalamic-pituitary response is highly variable, reflecting the above mentioned issue of desensitization or habituation. CRH has a greater stimulatory effect on ACTH than does AVP. However, there is a strong synergistic effect of CRH and AVP on ACTH. Joint pulses of the two will result in an ACTH pulse several orders of magnitude larger than that due to either individually. The basic conundrum is that, without observing CRH and AVP, one doesn’t know whether the resulting ACTH pulse was due to CRH or AVP or both. Thus, one cannot just inject a bolus of CRH and/or AVP, and measure the resulting ACTH effect. There may have also been a large endogenous (i.e., the body’s own) production of CRH and/or AVP, and the injected amount does not necessarily reflect the actual relationship. This is one of the issues that our experiments need to overcome.

The released ACTH pulse then travels via the blood to the adrenal glands and stimulates the release and production of cortisol. Cortisol has affects all throughout the body, but its regulation occurs by it feeding back on the hypothalamus, inhibiting CRH and AVP (hence indirectly inhibiting ACTH), as well as directly inhibiting ACTH secretion at the pituitary. The above are the feedback and feedforward arrows for the Stress axis in Figure 1 Left. Hence, a low cortisol level would result in an increase in CRH and AVP, which causes an increase in ACTH, which in turn increases cortisol, returning cortisol to steady-state. A high cortisol level would have the opposite dynamics. There are time delays in the feedforward action of ACTH on cortisol and in the feedback of cortisol on ACTH, CRH and AVP. If one doesn’t properly estimate these time delays, then one would not be able to accurately estimate the strengths (or loss) of the feedback and feedforward interactions. Under ordinary conditions, one only observes ACTH and cortisol, not CRH and AVP. One would like to be able to estimate the feedforward time delay for ACTH on cortisol and the feedback time delay of cortisol on ACTH. Moreover, in practical terms, one may need to allow for time-varying time delays. All of this is the focus of the second half of the paper.

In the first half of the present paper, the time delays will be removed by the ”clamping” of cortisol at a fixed level. That is, the body’s production of cortisol is blocked and exogenous cortisol (dexamethasone) is infused to maintain the desired level. Hence there is no feedforward of ACTH on cortisol, and with cortisol fixed, there is no feedback by cortisol. What is then done is to infuse high constant levels of CRH or AVP or both or neither. As we will describe, this will allow us to separate out ACTH pulses due to CRH from those due to AVP. The administration of CRH or AVP boluses at the end of each experiment, will then allow for the calibration of the preceding unobserved, endogenous CRH or AVP pituitary concentrations. This is the objective of Section 2: with a sequence of four experiments, one is able to recover information about the unobserved endogenous CRH and AVP pituitary concentrations and their dose-response drive on ACTH.

Figure 2, the details of which wil be further explained in Section 2, graphically depicts the essential idea of the approach. The subpanels of Figure 2 display simulations of several of the processes described in the following subsections. The top two rows of Figure 2 display simulated hypothalamic concentrations for CRH and AVP (Hyp CRH and Hyp AVP). The third and fourth rows, ACTH secretion rate and (true, physiological-level) ACTH concentration. Row five is the observed concentrations, including measurement and assay error. The left column corresponds to no exogenous infusions and the right column to that in which AVP is exogenously infused. Generally speaking, it is these hypothalamic signals which are driving the system, although the end product of the system, cortisol, feeds back in turn and negatively inhibits CRH, AVP and ACTH, in order to maintain homeostasis (steady-state). The CRH and AVP hypothalamic signals go through several nonlinear processes as they proceed to feedforward on ACTH, which in turn feedsforward on cortisol. Hence, these hypothalamic pulses can result in weakened downstream ACTH and cortisol pulses. Rows 3–5 display ACTH secretion, (actual physiological) ACTH concentration, and observed ACTH concentration. The intuition of the experiments is described in the legend of figure 1. The above concepts are defined below in Eqns (7, 13–14, 18).

Figure 2.

The figure is designed to convey the basic idea of the present approach; the details of the depicted processes are devloped in sections 2.1–2.2. The processes are the result of simulations. The top two rows display the AVP and CRH signals from the hypothalamus (Hyp AVP; Hyp CRH) to the pituitary. The signals are pulsatile (the asterisks are the pulse times), and these endogenous signals are unobservable because of ultimate dilution. Moreover, these hypothalamic signals, in a nonlinear dose-response manner, drive ACTH pulsatile secretion. Row three displays ACTH secretion with the combined pulsatile responses to the AVP and CRH pulses (asterisks of row three). One does not actually observe ACTH secretion either, nor the actual ACTH concentration time profile (row four), but rather ACTH concentrations with measurement and assay error (row five). Hence, one wishes to inference the responsiveness of ACTH secretion to CRH or/and AVP, based upon the observed ACTH concentration. The left column corresponds to no exogenous infusions and the right column to that in which AVP is exogenously infused. The high level of continuous AVP infusion (top right subpanel) translates into a large increase in basal ACTH secretion, with the AVP pulses having virtually no effect on ACTH pulses. Hence the ACTH pulses, observed only through the observed ACTH concentrations, reflect only the dose-response due to CRH pulses. Finally, cortisol which is stimulated by ACTH, and which in turn feeds back and inhibits CRH, AVP and ACTH, is not shown here, in that it is clamped at a fixed level. Experiments such as these allow one to piece together the effects (possibly changing with age or disease) of CRH and AVP on ACTH secretion.

2.1 CRH and AVP: Pulse Times, Secretion and Concentration

CRH and AVP are each secreted by distinct networks of about 20,000 specialized neurons. Most of the time, these firings are not synchronized across the network, and the total overall amount of neurotransmitter released is quite small. However, for reasons yet to be explained, on the order of every 45–150 minutes, there is a synchronization of the firing frequencies, at which time significant amounts of the given peptide are secreted. The length of any individual CRH and AVP burst or pulse is quite short, on the order of 1–5 minutes. The CRH and AVP peptides then travel bloodborne to the pituitary gland and the pulses stimulate pulsatile secretion of ACTH. Each of CRH and AVP stimulate the release and synthesis of ACTH, with that due to CRH being greater; however, there is a synergistic effect, with the amounts of ACTH released increasing several fold. CRH and AVP are initially released from the hypothalamus into the hypophyseal vessel, leading to the pituitary, with a blood volume of about .5mL. Those molecules which are not taken up by the pituitary, enter the general circulation which is approximately 5L (for a 70 kg individual). They are not assayable because of the (roughly) 10000-fold dilution.

For CRH and AVP, there are a large number of other neurotransmitters (which are also unobservable) that stimulate and/or inhibit the two: glutamate, glycine, GABA, acetylcholine, serotonin, dopamine, etc. In the case where little is known about how such factors govern the resulting amount of mass to be released, a reasonable model is to assume the mass to be the sum of a finite amount of minimally available stores (η₀), a linear function (η₁) of hormone accumulation over the preceding interpulse interval, and an allowable exibility (a random element A_j) in individual burst mass. The A_j ’s are I.I.D. N(0, ${\sigma }_A^2$), and describe the variation in pulse mass due to nonuniform cellular release. The A_j also capture to some extent the inherent desensitization, described in the Introduction. Cortisol, unlike the above neurotransmitters, is measurable in the blood and is the strongest inhibitor of CRH and AVP, and hence this inhibition is included. The IC_50,Cor for cortisol, the concentration at which it would have half-maximal inhibition, has been well-studied and population estimates are available.

$${FB}_{\mathit{\text{Cor}}}(t)=\frac{1}{1+\mathit{\text{exp}}(IC{50}_{\mathit{\text{Cor}}}-{\beta }_{\mathit{\text{Cor}}}{X}_{\mathit{\text{Cor}}}(t))},\text{Cor FdBK}$$ (1)

$$M_j=[{\eta }_0+{\eta }_1\times (T_j-T_{j-1})+A_j]\times {FB}_{\mathit{\text{Cor}}}(T_j)$$ (2)

We can assume that there is a normalized rate of release per unit mass per unit volume, and that the mass of that pulse just scales that waveform. The mass contained in any given burst, M_j, is released according to an adaptable (hormone-, subject-) waveform. The waveform (evolution of the instantaneous secretion rate over time) is represented via the three-parameter generalized Gamma (probability) density, which encapsulates the normalized rate of secretion (mass units) over time (min) per unit distribution volume (L):

$$\psi (r)=\frac{{\beta }_3}{\mathrm{\Gamma }({\beta }_1){\beta }_2^{({\beta }_1{\beta }_3)}}{r}^{({\beta }_1{\beta }_3)-1}{e}^{-{(r/{\beta }_2)}^{{\beta }_3}}$$ (3)

This pulsatile release is assumed to be superimposed on a continuous basal release. If nothing else is known, the basal rate is assumed to be constant (β₀). This is the case for CRH and AVP. Whereas for ACTH, its basal rate is modulated by both CRH and AVP, and to a lesser degree by cortisol. Of course, if only ACTH were measured, and none of the other three, than a constant basal is appropriate for ACTH, assuming no other knowledge.

For CRH and AVP, their instantaneous endogenous secretion rates Z_En(r) will be assumed to be of the following form (with potentially different parameter values). In the present context, infusions of CRH and/or AVP at constant continuous rates is also allowed Z_Ex(r). At a given time t, the full secretion Z(r) must take account the dilution (DiluteF = 10⁻⁴) of the endogenous secretion as it goes from a volume of .5mL to that of 5L. The resulting concentrations can be written as X(t), utilizing known literature-based estimates of the two half-lives and the biexponential fraction (a):

$$Z_{En}^{(k)}(r)={\beta }_0^{(k)}+{\displaystyle \sum _{T_j^{(k)}\le r}{M}_j^{(k)}{\psi }^{(k)}(r-T_J^{(k)})\mathrm{k}=\text{CRH}(\mathrm{C}),\text{AVP}(\mathrm{V})}$$ (4)

$$Z^{(k)}(r)=Z_{Ex}^{(k)}(r)+\mathit{\text{DiluteF}}*Z_{En}^{(k)}(r)$$ (5)

$$X^{(k)}(t)=(a^{(k)}{e}^{-{\alpha }_1^{(k)}t}+(1-a^{(k)})e^{-{\alpha }_2^{(k)}t})X^{(k)}(0)+{\displaystyle {\int }_0^t(a^{(k)}{e}^{-{\alpha }_1^{(k)}(t-r)}+(1-a^{(k)})e^{-{\alpha }_2^{(k)}(t-r)})Z^{(k)}(r)dr}$$ (6)

2.2 ACTH Secretion and Concentration

The pituitary cells which secrete ACTH are the corticotropes. ACTH basal secretion is stimulated (modulated) by each of CRH and AVP, and even further stimulated by their synergism. The permeation of CRH and AVP molecules, from the blood, into the (interstitial) fluid of pituitary tissue is driven by concentration gradients. CRH and AVP pulses result in large concentrations at the pituitary due to the small blood volume (.5 mL). The concentration of CRH and AVP which acts at the pituitary, consists of the general circulation concentration as well as the endogenous secretion, at the hypophyseal vessel volume (.5mL). It is these concentrations of CRH and AVP which drive ACTH secretion. These are described by:

$${\mathit{\text{Hyp}}}^{(k))}(t)=X^{(k)}(t)+(1-\mathit{\text{DiluteF}})*Z_{En}^{(k)}(t)\mathrm{k}=\text{CRH}(\mathrm{C}),\text{AVP}(\mathrm{V})$$ (7)

We model each of ACTH basal and pulsatile secretions as the sum of three components: that due to CRH alone, that due to AVP alone, and that due to the synergism of the two. The CRH and AVP feedforward signals are defined as time-averages (over an interval l₁, l₂) of CRH and AVP pituitary concentrations (Hyp^(k)(·), and the synergism signal as the maximum of these two::

$$H^{(k)}(t)=\frac{1}{l_2-l_1}{\displaystyle {\int }_{t-l_2}^{t-l_1}{\mathit{\text{Hyp}}}^{(k)}(r)dr\mathrm{k}=\text{CRH}(\mathrm{C}),\text{AVP}(\mathrm{V})}$$ (8)

$$H^{(C/V)}(t)=\text{max}\{H^C(t),H^{(V)}(t)\}$$ (9)

Again, in the top two rows of Figure 2 are simulated CRH and AVP hypothalamic endogenous secretion rates. The physiology requires that one model the changing concentrations due to such secretions and due to their eliminations, but in essence, the basic feedforward signaling by CRH and AVP on ACTH, will look very much like scaled versions of their secretion rates.

In a given corticotrope cell, ACTH molecules are synthesized, packaged in granules, which then diffuse out towards the cell membrane where they accumulate in storage pools. The CRH and AVP pulses initiate cell signaling mechanisms, which cause the granules to merge with the membrane (exocytosis), and release its contents into the interstitium and on into blood. This is the ACTH pulse resulting from a CRH, AVP or combined pulse. The cell signaling by CRH and AVP on the corticoptrope are via different G-protein coupled receptor (GPCR) mechanisms. Cortisol inhibits this release through multiple ways. Basal (or constituitive) secretion, however, is thought to be by way of sparingly regulated exocytosis. It is believed, at least not over the short-term, that basal secretion is not strongly affected by cortisol, unlike pulsatile secretion. Below, the $T_j^i$ are the pulse times of i=CRH, AVP, which become the pulse times of ACTH; similarly, ACTH releases its mass via a waveform ψ^(A)(·). In addition, just as for CRH and AVP, allowable flexibility (a random element $A_j^i$) is introduced into individual burst masses, capturing the variation in pulse mass due to nonuniform cellular release and desensitization.

$${\mathit{\text{Bas}}}^i(t)=\frac{C_i}{1+\mathit{\text{exp}}-(EC{50}_i+B_i{H}_i(t)},\mathrm{i}=\mathrm{C},\mathrm{V},\mathrm{C}/\mathrm{V}$$ (10)

$$M_j^i=\frac{{FB}_{\mathit{\text{Cor}}}(T_j^i)*(C_i-{\mathit{\text{Bas}}}_i(T_j^i))}{1+\mathit{\text{exp}}-(EC{50}_i+B_i{H}_i(T_j^i)}$$ (11)

$${\mathit{\text{Pul}}}^i(t)={\displaystyle \sum _{T_j^i\le t}(M_j^i+A_j^i){\psi }^{(A)}(t-T_j^i),\mathrm{i}=\mathrm{C},\mathrm{V},\mathrm{C}/\mathrm{V}}$$

$$Z_i(t)={\mathit{\text{Bas}}}^i(t)+{\mathit{\text{Pul}}}^i(t)\text{ Bas and Pul Secr due to i},\mathrm{i}=\mathrm{C},\mathrm{V},\mathrm{C}/\mathrm{V}$$ (12)

$$Z^A(t)=Z^C(t)+Z^V(t)+Z^{C/V}(t)$$ (13)

$$X^A(t)=(a^{(A)}{e}^{-{\alpha }_1^{(A)}t}+(1-a^{(A)})e^{-{\alpha }_2^{(A)}t})X^A(0)+{\displaystyle {\int }_0^t(a^{(A)}{e}^{-{\alpha }_1^{(A)}(t-r)}+(1-a^{(A)})e^{-{\alpha }_2^{(A)}(t-r)})Z^A(r)dr}$$ (14)

In terms of Figure 2, the pulse times of CRH and of AVP are identified by asterisks (*) in the top two rows. These pulse times are then inherited by ACTH. In row three is the resulting ACTH secretion rate (Z^A(t)) and the pulse times (*) and in row four is the (true, physiological) ACTH concentration process. Because we don’t observed CRH or AVP, it is not possible to identify which ACTH pulses were due to CRH, which to AVP, and which to their joint synergism. Moreover, one does not observe ACTH secretion nor the physiological concentrations, but rather a time-sampling (e.g., every 10 min) of the concentrations plus measurement error. In row five of Figure 2 is the resulting subsampled (10 min rate), observed ACTH concentration profile from which information as to CRH and AVP feedforward drive is to be ascertained. As a first step, one must estimate the ACTH pulse times and secretion rate without reference to CRH and AVP, and then use those pulse times and secretions in the next step. Such a deconvolution method is briey described in Section 2.4.

2.3 Cortisol Secretion and Concentration

In the present experiments, the body’s own production of cortisol is blocked by either ketoconazole (or metyrapone), and an amount of cortisol is continuously infused in order to maintain a constant blood cortisol concentration. If it were not held constant, its dynamics would be described as follows. The ACTH feedforward signal on the adrenal gland is defined as a time-average (over an interval l₁, l₂) of CRH and ACTH concentrations (X^(A)(·):

$$H^{(k)}(t)=\frac{1}{l_2-l_1}{\displaystyle {\int }_{t-l_2}^{t-l_1}{\mathit{\text{Hyp}}}^{(k)}(r)dr}$$ (15)

$$Z^{\mathit{\text{Co}}}(t)=\frac{C_1}{1+\mathit{\text{exp}}-(Ec{50}_{\mathit{\text{AonCo}}}+B_1{H}_A(t)}$$ (16)

$$X^{\mathit{\text{Co}}}(t)=a^{(\mathit{\text{Co}})}{e}^{-{\alpha }_1^{(\mathit{\text{Co}})}t}+(1+{\alpha }^{(\mathit{\text{Co}})}{e}^{-{\alpha }_2^{(\mathit{\text{Co}})}t})X^{\mathit{\text{Co}}}(0)+{\displaystyle {\int }_0^t{a}^{(\mathit{\text{Co}})}{e}^{-{\alpha }_1^{(\mathit{\text{Co}})}(t-r)}+(1+a^{(\mathit{\text{Co}})})e^{-{\alpha }_2^{(\mathit{\text{Co}})}(t-r)})Z^{(\mathit{\text{Co}})}(r)dr}$$ (17)

Because cortisol will be kept at a constant level, we have not described its on- and off-binding to carrier proteins. cortisol is highly hydrophobic and is transported in the blood by carrier proteins: cortisol binding protein (CBG), albumin and to a lesser degree by others. If the dynamics of cortisol were also to be considered, the on- and off-rates must be included as well as the fast and slow elimination rates. Such models have been developed in [8].

2.4 Estimation of ACTH kinetics and Secretion

Estimation of the kinetic and secretory structure of hormones has primarily focused on that of a single pulsatile-secreting hormone (such as ACTH). When one only observes the single series, one model for the structure is that presented in Section 2.1, where the pulsatile mass is assumed to be a linear function of the previous interpulse interval, plus a random effect. Hence, for the asymptotics, the number of pulse times along a given realization is increasing to infinity, and hence also the number of random effects. The asymptotics (consistency, normality) of MLE estimation were established in [25], [12] and [13]. These methods were conditioned on the pulse times having been estimated in a previous step.

In [11] a pulse detection algorithm, utilizing selective smoothing, was introduced. It was based upon a parabolic PDE for which the diffusion coefficient is inversely related to the degree of positivity (magnitude) of the gradient. Points of rapid increase, such as pulse times, will be smoothed very little. In [1], the justification for the algorithm is presented. As an example of the methods, consider the ACTH concentration data displayed in Figure 1, now displayed in the top row of Figure 3. In order to remove the effects that a trend can have on the detection of pulse times from noisy concentration data, a trend is removed by a heat-equation based algorithm; this is shown in row two, as well as the collection of all local minima (the collection of all potential pulse times). The concentration data is then progressively smoothed by selective smoothing algorithm. Potential pulse times (points of increase) from the starting collection are progressively removed by the smoothing, creating a sequence of putative pulse time set (Figure 3, row 3). The estimation algorithm then estimates the fixed number of kinetic and secretory parameters, for the different pulse time sets, finding the maximum penalized likelihood estimates, under an AIC penalty on the number of pulse times. In rows 4–5 are displayed the results of the estimation: the fit to the ACTH concentrations, the estimated secretion rate and the estimated pulse times. In the above paper ([1]), a Bayesian formulation of the estimation was presented; it was based upon an alternating Metropolis-diffusion scheme which sampled from the posterior distribution for the kinetic and secretory parameters (the diffusion was in the kinetic/secretory parameters, the Metropolis step was between putative pulse time sets). The methods of this section will be applied to the resulting ACTH concentrations from the four experiments, to estimate the unobserved pulse times and to estimate the half-lives of elimination and the secretion rates. Once the secretion rates are obtained, pulse masses can be estimated. The results of the estimation to the data from the simulations of the infusion experiments are displayed below in Figure 6.

Figure 3.

As an example of the methods, consider the ACTH concentration data displayed in Figure 1, now displayed in the top row. In order to remove the effects that a trend can have on the detection of pulse times from noisy concentration data, a trend is removed by a heat-equation based algorithm; this is shown in row two, as well as the collection of all local minima (the collection of all potential pulse times). The concentration data is then progressively smoothed by selective smoothing algorithm, a PDE nonlinear diffusion for which the diffuion term is inversely related to the size of a positive gradient. Points, from the starting collection of all potential pulse times, are progressively removed by the smoothing, creating a sequence of putative pulse time set. These are displayed in row three. The estimation algorithm then estimates the fixed number of kinetic and secretory parameters, for different pulse time sets, finding the maximum penalized likelihood, under an AIC penalty on the number of pulse times. In row four is the final estimation of the model, displaying the fit to the concnetrations, and the estimated pulse times. In row five is the resulting estimated secretion rate and pulse times. In the experiments of the present paper, one will need to extract the estimated ACTH secretion rates and pulse times from observed ACTH concentrations under various infusion schemes.

Figure 6.

The top two rows display the true ACTH secretion rates (solid) and their estimates (dashed), for Exp 1–4. One can thus calculate estimated ACTH pulse masses. From the estimated basal rates for Exp 2–4, one obtain estimates of the upper asymptotes for the dose-response functions. One can then estimate the ACTH dose-response on CRH and AVP, using the observed bolus data ( the ”diamonds”). These are displayed in row 3. Those ACTH masses due to endogenous CRH and AVP are the ’asterisks’ on the y axes. Estimates of the unobserved endogenous CRH (Left) and AVP (Right) levels are depicted as ’asterisks’ on the x axes. The circle is the CRH (AVP) EC50. The solid curve is the true dose-response; the dashed curve is its MLE estimate.

2.5 Core Equations of the Model

1. CRH and AVP Pulse times, Secretion and Concentration: The CRH and AVP pulse times are assumed to be given by two independent Weibull renewal processes, in that the pulsing is more regular than a Poisson process. The parameters are λ_C, γ_C and λ_V, γ_V, where λ is a rate parameter (number of pulses/day) and γ controls the regularity of interpulse interval lengths. The resulting pulse times are represented as:

   $$\{T_1^C,T_2^C,\dots ,T_{m_C}^C\}\text{and}\{T_1^V,T_2^V,\dots ,T_{m_V}^C\}$$

   For each of CRH and AVP, there are both endogenous and exogenous (possibly, zero) secretion rates (Eqns 1–5), with the sum being their secretion rate. The resulting concentrations are described by Eqn (6).
2. ACTH Pulse times, (Basal and Pulsatile) Secretion and Concentration: The pulse times are those inherited from CRH and AVP, with very little time delay:

   $$\{T_1^C,T_2^C,\dots ,T_m^C\}\text{and}\{T_1^V,T_2^V,\dots ,T_G^m\}$$

   The basal, pulsatile and total secretion rates are described by Eqns (7–13). The CRH and AVP concentrations (from A) above, feedforward and stimulate both individually and jointly, ACTH secretion. Cortisol, on the other hand, inhibits ACTH secretion. In the present model, Cortisol is held fixed, so that there is a constant inhibition. The ACTH secretion rates that will be estimated, will be those under this degree of cortisol inhibition. One will need to repeat the experiments under different amounts of fixed cortisol in order to ascertain the effects of cortisol on CRH and AVP, and to delineate that from inhibition of cortisol directly on ACTH secretion. ACTH concentration is described by Eqn (14).

3. Cortisol Secretion and Concentration: In the present formulation, cortisol concentration is clamped at a fixed level.

4. Observed Concentrations, with error: What one then observes is a discrete-time sampling of these processes, plus joint uncertainty due to blood withdrawal, sample processing and hormone measurement errors, ε_i(k):

   $$Y^i(t_k)\stackrel{\text{def}}{=}{X}^i(t_k)+{\epsilon }^i(k),k=1,\dots ,n,i=C,V,A,\mathit{\text{Cor}}.$$ (18)

2.6 Non-Invasive CRH and AVP Information Recovery

The strategy of the proposed method is to observe the stationary system described by the equations of the previous sections, but under four different levels of steady state, where these levels are induced by infusing exogenous amounts of cortisol, CRH and/or AVP. As described below, this will allow us to resolve the following: (1) an ACTH secretory pulse could be the result of either CRH or AVP. Moreover, even if one of these were observed, e.g., CRH, because of the synergism between CRH and AVP, one still could not discern the dose-responsiveness of ACTH to CRH; (2) there are simultaneously occurring CRH and AVP feedforward and cortisol feedback on ACTH secretion. A computer simulation of the four experiments is presented in Figures 4–6. In Figure 4, the top two rows display ACTH concentrations resulting from the four experiments. The four corresponding unobserved ACTH secretions are displayed in the bottom two rows.

Figure 4.

The top two rows display the ACTH concentrations for the four computer experiments, described in the text. The concentrations correspond to a time sampling of every 10 min for roughly 1 1/2 days (2070 min). In the bottom two rows are the corresponding four unobserved ACTH secretion rates, one for each experiment. The first objective of the current methodology is to estimate the secretion rates from the concentrations.

The following is the experimental strategy. Feedback from cortisol will be clamped. That is, the stimulation by ACTH on Cortisol secretion can be halted (with the drug, ketoconazole), and a fixed constant level of cortisol will be infused into the blood. For this fixed cortisol level, there will be four experiments: Exp I–IV. In Exp I, neither CRH nor AVP are infused, but at the end, simultaneous boluses of the two are given; this experiment is designed for obtaining information on the synergism. In Exp II, AVP will be infused at a high constant level (above the known EC50). The result of this, based upon equations (1–13), is that the the ACTH basal due to AVP and to the synergism of CRH and AVP, will be at the upper asymptote of those two dose-response functions. As a result, the upper asymptote for their pulsatile ACTH dose-response functions will be (approximately) zero. The ACTH basal due to CRH will be negligible, and the responses of ACTH to CRH will appear as pulsatile. That is, the ACTH secretion rate will have pulses sitting above a high constant basal. The basal rate will be the sum of the two unknown upper asymptotes for ACTH on AVP and on CRH and AVP synergism. The ACTH pulses sitting on the high basal are all pulses due solely to CRH. That is, deconvolving the observed ACTH concentrations and removing the basal will produce ACTH dose-response pulses due solely to CRH. One can not separate the basal value into the two unobserved upper asymptotes, but in conjunction with Exp III-IV, this can be done. The last part of Exp II is a sequence of seven intravenous CRH boluses of known amounts. These values will serve as calibration for the unobserved endogenous CRH amounts which stimulated the ACTH pulses prior to the boluses.

Exp III is analogous, except CRH and AVP roles are reversed; CRH is infused at a high constant level. The ACTH basal will be the sum of the two unknown upper asymptotes for ACTH on CRH and on CRH and AVP synergism. In Exp IV, both CRH and AVP are infused at high levels, and there are no boluses. The ACTH basal will be the sum of all three upper asymptotes: ACTH on CRH, on AVP and on the CRH and AVP synergism. Once the ACTH basal rates are estimated in the three, Exp II-IV, the three dose-response upper asymptotes can be obtained by (taking differences). Figure 5 displays the various components in Exp II which result in the Exp II ACTH secretion rate (Figure 4, row 3, column 2) and ACTH concentrations (Figure 4, row 1, column 2). In Figure 5, the three columns correspond to processes involved in the creation of the ACTH secretion due, respectively, to CRH alone, AVP alone and CRH/AVP synergism. The rows of Figure 5 display the following, from the bottom up: in row 6 are the levels of CRH and AVP (and their synergistic strength) in peripheral blood; row 5 displays the CRH, AVP and CRH/AVP synergism signals which drive ACTH basal; in row 4 are the signals from row 5 passing through the ACTH dose-response. A higher basal rate limits what remains to be released as pulsatile. Because AVP was exogenously infused, its peripheral level is very high (row 6) and the resulting basal rates due to AVP and CRH/AVP will be at their upper asymptotes. This then reduces the dose-response curves for their pulsatile-driven ACTH secretion to be small, as reflected in row 2, columns 2–3. Row 1 of Figure 5 then depicts the three components of ACTH secretion (Eqn 13): that due to CRH, AVP and CRH/AVP synergism. The sum of these three is then the ACTH secretion rate displayed in Figure 4, row 3, column 2. This is unobserved; what is observed is the resulting ACTH concentration profile, Figure 4, row 1, column 2.

Figure 5.

The three columns correspond to processes involved in the creation of the ACTH secretion for Exp II, due respectively, to CRH alone, AVP alone and CRH/AVP synergism. Row 6: CRH and AVP (and their synergism) in peripheral blood; Row 5: the signals which drive ACTH basal; Row 4 are the signals from row 5 passing through the ACTH dose-response. Row 1 depicts the three components of ACTH secretion (Eq. 13), that are due to CRH, AVP and CRH/AVP synergism. The sum of these three is then the ACTH secretion rate.

The first step in acquiring information about CRH and AVP is to deconvolve the four ACTH concentration series, one for each of Exp I–IV, and to obtain estimates of the four ACTH secretion rates. This was performed using the methods described in Section 2.4. In Figure 6, the top two rows display the true secretion rates (solid curves), used in the simulation, and the estimates (dashed curves). From the estimated basal rates, as described above, the estimates of the upper asymptotes are obtained, i.e., the three numerators in Eqn (13). To improve the accuracy of the dose-response reconstruction, the ACTH pulse masses were scaled up so that the maximum of the ACTH bolus masses is at (approximately) the preceding estimated upper asymptote. From the ACTH secretion rates for Exp II and Exp III, the pulse masses during their, respective, seven CRH-driven and seven AVP-driven boluses, can be calculated. Hence, one will have seven ACTH pulse masses due to their observed CRH bolus concentration, and similarly seven due to AVP bolus concentrations. These two sets of seven pairs, are displayed in Figure 6, row 3. In addition to these seven pairs, there are the estimated ACTH pulse masses in Exp II (prior to the boluses) for which the endogenous CRH concentrations was not observed, and similarly for Exp III, there are the estimated ACTH pulse masses for which the endogenous AVP concentrations were unobserved. In Figure 6, row 3, these are the values plotted on the y axes. The goal is, using the data due to the boluses, where both CRH and ACTH values (and similarly AVP and ACTH values) were observed, one then estimates the best fitting dose-response curve, and ”pulls-back” the values on the y axes to the x-axes as estimates of their unobserved CRH and AVP stimuli, respectively. This is displayed in Figure 6, row 3. That is, for each of Exp II-III, we have 7 points that contain both x and y values, the rest only contain y (and the number of these corresponds to the number of pre-bolus estimated pulse times). One can write down the likelihood function based upon a mean-function given by Eqn (13), as two parts: one based using both the x and y values, and the other only y’s, where the x values are estimated as parameters, with the restriction of monotonicity (larger y corresponds to larger x). Such monotonicity requirements are, in and of themselves, highly regularizing. Such a situation was first established in Grenander (1956–57) [2], [3], in which it was shown that a monotone, differentiable density function could be estimated (aysmptotically consistent) without requiring any explicit regularization (i.e., the allowance of finite-dimensional parameterizations slowly increasing in dimension). Finally, the likelihood can be written as a sum of two parts, one involving the pre-bolus period (1440 min) and the other involving the bolus period (630 min). For the bolus pairs, the full system, including CRH and AVP, are observed during the bolus period (10 min data over 630 min), but only ACTH and cortisol are observed prior to the boluses (10 min data over 1440 min). In the present case, we introduce a parameter λ which weights the likelihood part due to the 7 bolus pairs, differnetly than the other likelihood part, and does penalized MLE estimation. Currently we are using λ = 1, although we intend to vary and minimize over all λ. Intuitively, one might expect a weight of λ = 1440/630 to be a possible weighting to account for the fact that bolus period consists of full observations. The ratio of observation numbers will also approximately equal to the (random) number of pulse times in the 1st 1440 min divided by 7, since 90 min is roughly the mean interpulse interval length.

On the x axes for each column of Figure 6, row 3, are the estimated CRH and AVP endogenous concentrations. Their obtainment was our objective. One can now calculate summary statistics of their distributions. Finally, one could perform the four experiments, clinically, under different fixed values of cortisol and see how the estimated CRH and AVP endogenous distributions change as cortisol changes. Being able to make such assessments, non-invasively, in situations of aging and disease would be a major breakthrough in endocrinology.

3 Time Delay Estimation in Multidimensional Nonlinear Time Series

In biology and medicine, most systems evolve under nonlinear dynamics, and maintain their stability due to inherent feedback and feedforward control mechanisms. The dynamics of biological systems are such that processes do not directly act on (regulate) their own evolution, but rather act on other processes which ultimately feedback and control the former processes. Moreover, in the case of hormonal systems, the control occurs at the level of regulation of hormonal secretion (which is often a pulsatile secretion), and operates nonlinearly. The secreted hormone then travels via the bloodsystem to initiate its action. The time from the initial hormonal release until its action elsewhere can be minutes to hours. The action can be quite rapid, initiating the release of another hormone, or it could be longer, such as the up- or down-regulation of gene expression which can require hours. Hence there are time delays between the occurrence of a pulse of one hormone, and the time that it takes for its action to occur. In this section, we present a novel method to detect and estimate feedforward (stimulatory) or feedback (inhibitory) time delays. The method involves the construction of new processes whose regular or irregular patterns will reveal the nature of the time delays.

3.1 Approximate Entropy

Approximate Entropy (ApEn), was introduced as a quantification of regularity in sequences and time-series data, initially motivated by applications to relatively short, noisy data sets ([14]). Conceptually, ApEn grades a continuum that ranges from totally ordered to maximally irregular (completely random). Mathematically, ApEn is part of a general development of approximating Markov Chains to a process; it is furthermore employed to refine the formulations of i.i.d. (independent, identically distributed) random variables, and normal numbers in number theory, via rates of convergence (of a deficit from maximal irregularity) ([20],[19]). Analytical properties for ApEn can be found in references ([14], [17], [16]); as well, it provides a finite sequence formulation of randomness, via proximity to maximal irregularity ([20],[19]). Further technical discussion of mathematical and statistical properties of ApEn, including asymptotic normality under general assumptions, error estimation for general processes, and continuous state mesh interplay can be found elsewhere ([14] – [16]).

ApEn assigns a nonnegative number to a sequence or time-series, with larger values corresponding to greater apparent process randomness (serial irregularity), smaller values to more instances of recognizable patterns or features in data. Two input parameters, a block or run length m and a tolerance window r, must be specified to compute ApEn. Briey, ApEn measures the logarithmic probability that blocks of length m that are close (within r), for m contiguous observations, remain close (within the same tolerance width r) on the next incremental comparisons; the precise mathematical definition is given next.

The definition of ApEn is as follows. Given a positive integer N and non-negative integer m, with m ≤ N, a positive real number r and a sequence of real numbers u̱ := (u(1), u(2),…u(N)), let the distance between two blocks x̱(i) and x̱(j), where x̱(i) = (u(i), u(i+1),…u(i + m − 1)), be defined by d(x̱(i), x̱(j)) = max_k=1,2,…,m(|u(i + k − 1) − u(j + k − 1)|). Then let $C_i^m(r)$ = (number of j ≤ N − m + 1 such that d(x̱(i), x̱(j)) ≤ r)=(N − m + 1).

Now define

$${\mathrm{\Phi }}^m(r)=\frac{{\displaystyle {\sum }_{t=1}^{N-m+1}\text{log}}{C}_i^m(r)}{N-m+1},$$

and the ApEn statistic is:

$$\text{ApEn}(N,m,r)(\mathrm{u̱})={\mathrm{\Phi }}^m(r)-{\mathrm{\Phi }}^{m+1}(r),m\le 1,\text{with }ApEn(0,r,N)(\mathrm{u̱})=-{\mathrm{\Phi }}^1(r).$$

The parameter ApEn(m, r) is defined as $\underset{N\to \infty }{\text{lim}}$|ApEn(m, r, N), assuming this limit exists.

ApEn(m,r) is a family of parameters; comparisons are intended with fixed m and r. For the studies below, we calculate ApEn values for all data sets, applying widely utilized and validated parameter values m=1 and r=20% of the standard deviation (SD) of the specified time-series (or r=0.2 applied to normalized y-values). Importantly, normalizing r to each time-series SD gives ApEn a translation- and scale-invariance ([16]), insuring a complementarity of variability (amplitude) and irregularity (ApEn), in that ApEn remains unchanged under uniform process magnification.

Previous evaluations including both theoretical model-based analysis ([14], [17]) and numerous diverse applications demonstrate that the input parameters indicated above produce good reproducibility for ApEn for time series of the lengths considered herein. Specifically, the SD of ApEn(m=1, r=20% SD) is generally ≤ 0.055 for all members of several very distinct classes of (representative) low order and ”weak”-dependence processes. This ensures both stable estimates for the ApEn calculations described below, and more broadly, a model-independence for ApEn, i.e., robust qualitative inference across diverse model configurations.

In applications, ApEn has been used extensively, to differentiate hormonal secretory patterns ([18],[22],[23]), as we study below, as well as heart rate dynamics, neuromuscular control and balance disorders, providing predictive markers of e.g., atrial fibrillation and Parkinsons disease ([16]). Additionally, ApEn is robust to noise and artifacts. Moreover, changes in ApEn have been shown mathematically to correspond to mechanistic inferences concerning subsystem autonomy, feedback, and coupling, in very distinct model settings ([23]).

The technical observation motivating ApEn is that if joint probability measures for processes or reconstructed dynamics that describe each of two systems are different, then their marginal probability distributions on a fixed partition, specified by well-defined conditional probabilities, are likely different. The question of full reconstruction is subsumed by a broader question, is a process nth order Markov? The general answer is not necessarily; there may be no fixed n for which the process is characterized entirely by conditioning on n previous observations. However for fixed m, these marginal conditional probabilities contain a wealth of probabilistic detail about the underlying process, often allowing discrimination. We typically need orders of magnitude fewer points to accurately estimate these marginal probabilities than to accurately reconstruct the joint measure defining the process. ApEn is nearly unaffected by noise of magnitude below r, the filter level, gives meaningful information with a reasonable number of data points, and is finite for stochastic, deterministic and composite processes.

In reference [18], we provide a theoretical basis for understanding why ApEn provides a more general measure of feature persistence than do linear correlation and spectral measures. Descriptively, correlation and spectral measures assess the degree of matching or recurring features (characteristic subblocks) at fixed spectral frequencies, whereas the ApEn formulation implicitly relaxes the fixed frequency mandate in evaluating recurrent feature matching. Thus ApEn both provides a sharper measure of equidistribution ([16]), and can identify subtle yet persistent pattern recurrences in both data and models that the aforementioned alternatives fail to do ([18]). Importantly, the autocorrelation function and power spectrum are most illuminating in linear systems, e.g., ARMA, ARIMA, and SARIMA models, for which a rich theoretical development exists. For many other classes of processes, these parameters often are much less effective at highlighting certain model characteristics, even apart from statistical considerations ([18]).

An important analytic property of ApEn is that ApEn(m, r) is the continuous state space extension of the rate of entropy. Specifically, asymptotically ApEn converges in the limit, for r < 1 or as r → 0, respectively, to either the classically defined rate of entropy in discrete state or the K-S entropy (with m increasing as a function of N) in continuous state for those models for which the latter parameters are defined. Finally, we can analytically evaluate the limiting parameter ApEn(m, r) for mth order processes by either multiple integral (continuous state) or by multiple sum (discrete-state) expressions ([14]), manifesting the representation of the m+1 fold joint probability distributions, in conjunction with the limiting invariant (stationary) measure.

3.2 Time Delay Lag

We can think of a biological signal or process as a combination of two parts: a systematic (or structural) component and an independent (or biological noise) component. The systematic component describes the dependency of the current state of the process to previous states of itself or to other processes in the biological system; the time independent component consists of the random fluctuation (noise). Feedback or feedforward can be viewed as part of the systematic structure.

The regularity of a process depends upon its dependent structure as well as the relative magnitude (compared to the systematic component) of the noise. Note that a process appears to be more regular when the relative magnitude of the independent component is small, and that it fluctuates in a less predictable way as the weight of the noise increases.

To understand how two processes (or signals) depend, in a feedforward/feedback sense, upon one other, we construct a new process by shifting the second signal and taking the difference (or some other function) of the two signals. We call the new process a lag process, with the lag being the number of steps by which the second signal is shifted. If the two signals are not independent, the systematic component of the lag process changes as the lag changes, since different amounts of the two signals are synchronized at different lags.

Given two processes {u¹} and {u²}, define the process of lag k, {v^k}, as $v_t^k=f(u_t^1,B_k{u}_t^2)$. Here B_k is a back shift operator of order k acting on the second process, and f is a general function of two processes. For each lag process {v^k} with k in a pre-specified range, a proper measure will be defined by which to compare different lags. The most commonly used lag function is the lag difference. Details are given below:

• 1.Given two processes, first standardize each process by its sample mean and sample standard deviation. Denote the two standardized processes as {u¹} and {u²}.
• 2.Choose a proper range [a, b] of lags, and for each lag k with k ∈ [a, b], define the lag-difference process {v^k}, as $v_t^k=u_t^1-u_{t+k}^2$.
• 3.Calculate the approximate entropy (ApEn) measure of each lag-difference process {v^k}. The typical ApEn parameter choices are m = 1 and .1SD ≤ r ≤ .25SD. Here SD is the sample standard deviation of {v^k}, which changes with k; alternatively, SD can be set to a fixed value across different lags.
• 4.Order the ApEn estimates according to lag k and plot ApEn vs. Lag.

The next step is to quantify lag-shifted synchronization of the two processes in terms of regularity. ApEn is estimated for each lag process, which produces a lag-indexed series of ApEn values. The crux of the method is hence based upon an understanding of the changes in regularity throughout the fluctuations as well as the overall trend of the ApEn values across lags.

Large fluctuations are expected when comparing the ApEn estimate at the true time delay lag or its immediate left and right lag neighbors. Since the state of a process at a given time point often partially depends upon its state at previous time points, the synchronization affects most significantly these three lag processes: the true lag or its immediate neighbors. Depending upon the exact lag function that is used and the true system structure, generally, one would expect to see a local ”V” shape or a ”Λ” shape at the lag of most acute feedforward or feedback, respectively. To test whether a local fluctuation of the ApEn curve is significant, a permutation technique is used to construct a confidence interval:

• 5.For standardized processes {u¹} and {u²}, hold {u¹} fixed and randomly permute {u²}, to get a new process {u^2*}.
• 6.Define lag processes using processes {u¹} and {u^2*}, and calculate ApEn (as described in steps 2–4)
• 7.Calculate the 2nd degree difference of the resulting ApEn series: let {a} denote the ApEn estimate process indexed by the time lag; then the corresponding 2nd degree difference at lag k is a_k+1 + a_k−1 − 2a_k.
• 8.To construct the confidence band, repeat steps 5–7 a large number of times, and get the upper and lower percentiles of the 2nd degree difference at each lag;
• 9.Finally, a 2nd degree difference at some time lag of the non-permuted curve outside the band is considered significant at that time lag.

3.3 Simulation Example

The method described above is appropriate when the true time delay lag is fixed (but unknown) or varies within a small range. If the true time delay between two processes varies greatly without one lag being dominant with respect to others, the method will have little power to detect any time lag. In this case, where the time delay lag is itself varying over time (i.e., is dynamic), a moving window extension (w, window width) can be helpful, given enough samples. For each lag process, ApEn can be estimated using every w consecutive samples to construct a series of time-dependent ApEn estimates. An ApEn surface can be constructed combining the ApEn series of multiple lag processes.

As an example of this extension, we simulated data from a multivariate nonlinear model. The model, which has three components, is that of Sections 2.1–2.3, except that there are not distinct CRH and AVP signals, but rather for simplicity one common hypothalamic signal. Thus, the three components correspond to CRH/AVP, ACTH and cortisol. The nonlinearity of the dynamics occurs through Equations 1, 10, 11, 16 of Section 2. The processes are simulated over a day, with the observed processes being a sampling every 10 minutes, resulting in 144 samples. The resulting time series Y₁, Y₂ and Y₃ are the concentrations for, respectively, CRH/AVP, ACTH and cortisol. There corresponding secretion rates are denoted as Z₁, Z₂ and Z₃. A feedforward time delay lag of Y₂ on Z₃ (corresponding to the feedforward of ACTH concentration on cortisol secretion) was specified at each time point and was allowed to vary between 0 to 4 (0–40 min) according to a Weibull distribution. The simulated series, the actual time delay lags and lag estimates are plotted in Figure 7.

Figure 7.

A simulated example of time-varying time delays. Top left: time series plot of Y₂ and Z₃. Middle left: the most frequent lag within each 6 hrs time window. Bottom left: contour plot of the moving window ApEn with darker curve corresponding to lower level. Upper right: the distribution of time delay lag of 144 samples. Middle right: ApEn(1, .2SD) series, SD is the corresponding standard deviation of lag-difference process at each lag; the feedforward time delay lag is identified by a “V” shape of the curve. Bottom right: 2nd degree difference of the ApEn series with 95 % confidence band based on 100 permutations.

As shown in the top right panel of Figure 7, there were four time delay lags for the current realization. The lag of 0 (0 min), 1 (10 min) and 4 (40 min) had lower frequencies than the lag of 3 (30 min). The most frequent time lag of 3 as well as the lag of 0 were captured by the drops in the ApEn plot in the middle right panel, while the other two lags were not detected. (The ApEn(1, .2SD) series was with respect to the lag-difference processes and the feedforward lag corresponded to a drop.) In the lower right panel is a plot of the second degree difference of the ApEn series and the fluctuations at lags 0 and 3 were significant. For this example, the actual time delay lag varied with time. The dominant lag within each 6 hour (36 points) time window was plotted in the middle left panel. By allowing a moving-window calculation of ApEn, the changing pattern of the time delay lag was captured by the lower contour levels (dark curves) in the bottom left panel.

The cross-correlation plot in the upper left panel of Figure 8 reached its maximum at lag 0 and was not sensitive to other values of lag. The cross-correlation method, which captures the linear relationship, failed to identify the most dominant time delay lag, a lag of 3, in this realization of a nonlinear system. ApEn based methods, which do no rely on linear assumptions, often show sharper distinctions than traditional linear methods, for general processes. (Refer to [18] for further example and theory.) In Figure 8, we also calculate a spectral-based estimate of the time delay of the feedforward of Y₂ on Z₃. The method is the usual group-delay procedure, which is the negative (−) of the derivative of the phase spectrum (Priestley, [21]) (1981)). In Figure 8, we show the spectra of the 2 series, along with the coherence (not the squared coherence), as well as the quadrature, co- and phase spectra. In the bottom right subplot is the group delay function which is a time lag estimate at each frequency. Because of the time-varying time delay, one can see that the group delay method does not really produce an estimate; it is basically zero over the first 20–40 Fourier frequencies, suggesting a zero time delay, the same as that suggested by cross-correlation. The high coherence over the first 15–20 frequencies reflects the strong feedforward relationship of Y2 stimulating Z₃.

Figure 8.

Traditional analysis of the simulated example. Top left: cross-correlation plot (for Y₂ on Z₃) estimates the lag at which the two processes are most correlated. Also displayed are the various spectral-based estimates which are used to assess relationships: the spectra of the two processes; their coherence; and, the negative of the derivative of the phase spectrum, which gives an estimate of a time delay of Y₂’s action on Z₃, at each frequency (the group delay function). The two spectra which go into the formation of the phase spectrum: the quadrature and co- spectra are also displayed. The method, one could argue, seems to suggest that there is zero time delay, the same as that given by cross-correlation.

3.4 ACTH-Cortisol Example

In order to show the potential of the present methods, we consider human data from 32 normal subjects and 21 subjects who have Cushing’s disease, a disease due to a tumor in the pituitary, which results in excessive ACTH secretion, and in turn, excessive cortisol secretion. It can have enormous negative consequences throughout the body.

The data was provided by Ferdinand Roelfsema [22],[8], University of Leiden, The Netherlands. Blood samples were taken every 10 min from 9 am to 9 am, with thus N = 144 samples in total. For each subject, ACTH and cortisol concentrations were measured from their blood samples. The units are (ng/L) and (nmol/L) for ACTH and cortisol, respectively. As described in Sections 1–2, the corresponding CRH and AVP measurements are unobserved, as are virtually all hypothalamic signals in human subjects. The ACTH and cortisol secretions and elimination kinetics were then calculated by the deconvolution program (with units (ng/L/min) and (nmol/L/min)) described in Section 2.4. The ACTH and cortisol data displayed in Figure 1 were for a subject in this study (normal subject 1); the deconvolution of this particular ACTH profile was displayed in Figure 3 and discussed in Section 2.4. Time delay estimation for this subject (normal subject 1) are displayed in Figure 9.

Figure 9.

Time delay lag estimation of normal subject 1: Upper left: ApEn(1, .2, N) by time delay lag. Lower left: second degree difference of the ApEn(1, .2, N) series with 95% and 99% confidence band based on 1000 permutations. Upper right: ApEn(1, .2SD, N) by time delay lag. Lower right: second degree difference of the ApEn(1, .2SD, N) series with 95% and 99% confidence bands based on 1000 permutations.

We applied the ApEn-based lag detection method (non-window version) to each of the subjects in the study (32 normal, 21 Cushing’s disease) to assess the difference in dominant time delays (lags) between disease and normal groups. Both ACTH and cortisol concentration measurements were first normalized and lag-difference processes were constructed with the lag varying from −10 to 10. ApEn was calculated with fixed value r = .2 as well as with r = .2SD. Here SD is the sample standard deviation of the lag process, which changes with the lag. In the case in which r is fixed, the shape of the ApEn curve reflects the changes of regularity and variation across the lag (in combination).

The result of normal subject 1 is shown in Figure 9. The ApEn(1, .2, N) and ApEn(1, .2 SD, N) series were calculated. The upper panels plot ApEn vs. Lag and the lower panels show the corresponding local fluctuation significance of each lag with 95% and 99% confidence bands. Both calculations indicate a significant fluctuation at the lag 1.

We then compared the normal and diseased groups (Figure 10). The mean ApEn over all subjects in the group is plotted against the lag. The confidence bands are also constructed based on the mean of each permuted sample in the group. The results are shown in Figure 10. The left panels are for the normal group and right for the Cushing’s disease group. The solid curves plot the case when ApEn parameter r is proportional to the sample standard deviation of the corresponding lag process; the dashed curves are for the fixed r. The normal group has an ACTH-cortisol feedforward lag of 1–2 (around 10–20 min). The Cushing disease group does not show a dominant feedforward lag. One argument for the loss of dominant feedback lag is as follows. In Cushing’s disease, ACTH is secreted in a much less pulsatile manner, resulting in very little dynamic range in its concentration values. This results in very similar behavior for cortisol secretion. The loss of variation in both, reduces the evidence of a time delay in the relationship of ACTH to cortisol.

Figure 10.

Group comparison of time delay: The left three panels refer to the normal group. Upper left: group mean ApEn(1, .2,N) and group mean ApEn(1, .2SD, N) by time delay lag. Middle left: second degree difference of the ApEn(1, .2SD, N) series with 95% and 99% confidence bands. Lower left: second degree difference of the ApEn(1, .2,N) series with 95% and 99% confidence bands. The corresponding right three panels refer to the Cushing disease group.

3.5 Extension of the method

The overall trend of the ApEn series gives some idea of the range of the time delay. If we take a lag difference of two processes, in the case where r is set as a multiple of the standard deviation of the corresponding process, ApEn then reflects the regularity changes specifically. Thus, the largest ApEn value is expected around the lag of greatest synchronization (feedforward), and the smallest ApEn around the lag of least synchronization (feedback).

As shown in the upper two panels in Figure 10, the solid curve of the normal group has a peak around lag 1–2 indicating a large amount of synchronization, while the curve of the disease group is relatively at showing a lack of dominant time delay lag. In order to distinguish between two groups, a confidence band of the ApEn curve is needed in order to statistically compare the overall shape of the curves. An analytical way to calculate the standard error of ApEn (which requires suitable mixing assumptions [24], manuscript in preparation) can be used to efficiently construct such a test.

4 Summary

In the present paper we have focused on two issues that are fundamental to endocrinology. The first is that hormonal systems are multidimensional, with one or more of the components being unobserved. The Stress axis was used as a prototype, but the partial observability of the system occurs broadly. For example, the same situation occurs in the regulation of the sex steroids (i.e., reproduction), growth hormone, thyroid hormone, as well as in physiology more broadly. The major obstacle to the modeling of these systems was described to be the fact that the brain signals, which drive much of the overall dynamics, were not observable. As part of this work, we present an approach by which a stable, partially-observed system is perturbed from its usual steady-state to multiple new ones, which in their combination, allows for the extraction of some information about the unobserved. Moreover, there are time delays in the feedback and feedforward interactions which made assessing the strengths of the interactions difficult. The present paper focused on these two issues: (1) Is it possible to devise a combination of clinical experiments by which, via exogenous inputs, the system can be perturbed to new steady-states in such a way that information about the unobserved components can be ascertained. Clinical experiments of this general paradigm are currently being designed; and (2) Can one devise methods for estimating (possibly, time-varying) time delays between components of a multidimensional nonlinear time series, which are more robust than traditional methods?

These are very dificult problems, for which the methods presented are only preliminary. Many of the fundamental questions in endocrinology are inherently time series question and the discipline as a whole is in need of time series modelers.