Dynamic Structural Equation Models for Directed Cyclic Graphs: the Structural Identifiability Problem

Abstract

Network systems are commonly encountered and investigated in various disciplines, and network dynamics that refer to collective node state changes over time are one area of particular interests of many researchers. Recently, dynamic structural equation model (DSEM) has been introduced into the field of network dynamics as a powerful statistical inference tool. In this study, in recognition that parameter identifiability is the prerequisite of reliable parameter inference, a general and efficient approach is proposed for the first time to address the structural parameter identifiability problem of linear DSEMs for cyclic networks. The key idea is to transform a DSEM to an equivalent frequency domain representation, then Masons gain is employed to deal with feedback loops in cyclic networks when generating identifiability equations. The identifiability result of every unknown parameter is obtained with the identifiability matrix method. The proposed approach is computationally efficient because no symbolic or expensive numerical computations are involved, and can be applicable to a broad range of linear DSEMs. Finally, selected benchmark examples of brain networks, social networks and molecular interaction networks are given to illustrate the potential application of the proposed method, and we compare the results from DSEMs, state-transition models and ordinary differential equation models.

1. INTRODUCTION

Network systems are common in a variety of research fields, including chemistry, physics, economics, computer science, sociology and biology [1, 2]. One particularly interesting research question on network systems is to quantitatively understand network dynamics, where network status evolves over time [3, 4]. The evolution of network status may refer to the temporal changes in network structures (e.g., edge connection), node states, or both. In this study, network structures are assumed remaining the same for simplicity so the dynamics here only refer to the collective changes of node statuses [1, 5]. Considering that accurate and reliable estimation of key parameter values is critical to quantitatively characterizing network dynamics [1, 5], it has been repeatedly stressed in many previous studies [6] that parameter identifiability analyses should be performed before any statistical inference techniques are applied to obtain parameter estimates. However, to the best knowledge of our authors, structural identifiability analysis (SIA) of complex dynamic systems like cyclic networks is a largely underexplored problem, and this study makes an attempt to fill the methodological gap.

Graphical models have long been used to mathematically describe networks and associated characteristics. Since graphical models refer to a broad range of mathematical formulations [6, 7, 8], limited by resources, here we focus on the structural equation model (SEM) representation of networks. SEM is a powerful statistical tool employed in many research fields such as economics [9, 10], environmental science [11], multivariate statistics [12, 13], social science [14] and biomedical engineering [15, 16]. While many previous studies considered static SEMs, lots of real systems are dynamic in nature such that dynamic SEM (DSEM) has been necessarily proposed to, e.g., accommodate time course observations [11, 14, 17, 18]. Different from ordinary differential equation (ODE) models [1, 5, 6], DSEMs are discrete; also, DSEMs are more general than state-transition models [19, 20] since dynamic SEMs can accommodate both con-current effects and memory effects [9, 12, 14]. Therefore, DSEMs are deemed as a powerful and flexible mathematic language for describing complex dynamic network systems.

The purpose of SIA is to verify whether unknown model parameters can be unambiguously determined for a given model structure and observation strategy. The importance of SIA has been increasingly recognized because it is the prerequisite of obtaining reliable parameter estimates from complex dynamic models and likely to occur due to model misspecification and/or lack of information (e.g., latent variables or unobserved local structures) [6, 21]. Particularly for DSEMs, while the parameter estimation techniques have drawn significant attention of certain researchers [10, 22], to the best knowledge of our authors, this is the first study that makes an attempt to address the SIA problem of DSEMs. With that said, there exist a number of previous studies on the SIA problem for static SEMs, but many of them are for recursive static SEMs that contain no feedback loops. For example, several theoretical criteria or computational methods have been established for examining model structural identifiability, such as Brito and Pearl’s conditions for bow-free models [23], Pearls back door and front door criteria [24], Tian’s accessory set approach [25], Brito and Pearl’s auxiliary sets condition [26], Tian’s partial regression analysis technique [27], and Wang’s identifiability matrix method [28, 29, 30]. For static SEMs with feedback loops (i.e., nonrecursive SEMs), only a few previous studies are publically available, including the computer algebra approach that is computationally inhibitive [31] and the cycle simplification method that treats a cycle as two connected paths in a directed acyclic graph [28, 32]. Since none of the existing works above is particularly for DSEMs, it necessarily calls for the development of novel SIA techniques applicable to such models.

DSEMs refer to a broad range of models that differ in assumptions and structures, and it is impossible to explore all such models in one study. Therefore, here we only focus on DSEMs that are: 1) linear; 2) with time-invariant parameters; 3) with equal time intervals between observations; and 4) with short-term memory (i.e., a finite-order Markov chain). In this study, a general and efficient approach is proposed to determine the structural identifiability of DSEMs with (or without) latent variables, feedback loops, and an arbitrary-order Markov property. The basic idea is to transform a DSEM to its frequency domain representation via z-transform, then employ Mason’s gain [33] to handle cyclic networks and generate identifiability equations. Then the identifiability of each unknown parameter is determined with Wang’s identifiability matrix method [28]. The proposed method involves no symbolical or expensive numerical computation, and is thus efficient. A theoretical result is obtained for a special class of DSEMs that contain only self-loops but no latent variables. Also, we have selected three benchmark models from brain networks, social networks and biological networks to illustrate the application of the proposed method in practice, and discussed the differences in the identifiability analysis results between DSEMs and ordinary differential equation (ODE) models.

This article is organized as follows. The SIA problem of DSEMs is defined in Section 2. We then introduce the identifiability equation generation method that incorporates z-transform and Mason’s gain in Section 3. How to determine the identifiability of every single model parameter based on the generated identifiability equations is described in Section 4. We also compare DSEMs and ODE models and present some benchmark models and applications in Section 5. Conclusions and discussions are presented in Section 6.

2. PROBLEM DEFINITION

Let $Y_i^t$ denote the observed value of variable Y_i at time t. Without distinguishing endogenous from exogenous variables, a linear DSEM can be given as follows

$$Y_i^t=\sum _{j\ne i}{c}_{ij}{Y}_j^t+\sum _j\left(\sum _{k=1}^o{b}_{\mathit{ij}}^k{Y}_j^{t-k}\right)+{\epsilon }_i^t,i,j=1,\cdots ,n,$$ (1)

where c_ij denotes the concurrent effect of variable Y_j on Y_i at time t, $b_{ij}^k$ denotes the delayed effect of variable Y_j on Y_i after k time intervals, and ${\epsilon }_i^t$ is a homoscedastic Gaussian white noise process with zero-mean. In this model specification, only historical self-dependence is allowed (i.e., j ≠ i at time t); also, the constant ο specifies the order of the Markov property. Eq. (1) has a corresponding directed cyclic graph (DCG) representation G = (V, E) = ({V_i}, {E_ij}), where node V_i represents variable Y_i and a directed edge E_ij from node V_j to node V_i represents the effect c_ij (or $b_{ij}^k$) of $Y_j^t$ (or $Y_j^{t-k}$) on $Y_i^t$. For illustration, a DSEM example with the first-order Markov property is given in Eq. (2), and its corresponding DCG representation is shown in Fig. 1.

Figure 1.

A simple DSEM example. An orange edge represents the interaction between two nodes at the same time point, and a light blue edge represents the node variable dependence at two consecutive time points.

$$\{\begin{array}{c}{y}_1^t={\epsilon }_1^t\hfill \\ y_2^t=c_{21}{y}_1^t+c_{23}{y}_3^t+b_{21}^1{y}_1^{t-1}+b_{22}^1{y}_2^{t-1}+b_{23}^1{y}_3^{t-1}+{\epsilon }_2^t\hfill \\ y_3^t=c_{32}{y}_2^t+b_{32}^1{y}_2^{t-1}+b_{33}^1{y}_3^{t-1}+{\epsilon }_3^t\hfill \end{array}\phantom{\}}.$$ (2)

Introduce the matrix notations C = [c_ij] and $B^k=[b_{ij}^k]$ (k = 1, …, ο), then it is straightforward to tell that there are ο + 1 coefficient matrices in total for a DSEM with a οth-order Markov property. In SIA, the model structure and observation strategy are pre-specified; therefore, it is known which parameters are zero or non-zero in C and B^k, and which node variables are observed or unobserved (i.e., latent variables). For a DSEM, the goal of SIA is to determine whether the non-zero parameters in C and B^k can be unambiguously determined given the observed variables. For this purpose, a set of identifiability equations will be generated to symbolically solve for the unknown non-zero parameters, as described in the next section. If there exists only one solution to an unknown parameter, this parameter is called globally identifiable; if there exist a finite number of solutions, an unknown parameter is called locally identifiable; and if there exist an infinite number of solutions, an unknown parameter is called unidentifiable. For instance, in Fig. 1, the coefficient matrices of the DSEM are

$$\mathbf{C}=\left[\begin{array}{ccc}\hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill c_{21}\hfill & \hfill 0\hfill & \hfill c_{23}\hfill \\ \hfill 0\hfill & \hfill c_{32}\hfill & \hfill 0\hfill \end{array}\right]\mathit{and}{\mathbf{B}}^1=\left[\begin{array}{ccc}\hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill b_{21}^1\hfill & \hfill b_{22}^1\hfill & \hfill b_{23}^1\hfill \\ \hfill 0\hfill & \hfill b_{32}^1\hfill & \hfill b_{33}^1\hfill \end{array}\right],$$ (3)

and we assume all the variables are observed. The corresponding SIA problem is to determine the number of solutions to each non-zero parameter in matrices C and B¹ (i.e., c₂₁, c₂₃, c₃₂, $b_{21}^1$, $b_{22}^1$, $b_{23}^1$, $b_{32}^1$ and $b_{33}^1$).

3. IDENTIFIABILITY EQUATION GENERATION

For static SEMs, identifiability equations are usually generated using the covariance between two observed variables [28, 32]. However, this idea is not directly applicable to DSEMs since the measurements of a variable in DESM are now time series and one has to consider the time dependence between the measurements of the same variable and between the measurements of different variables. In addition, DSEMs for cyclic networks inevitably involve feedback loops such that it is necessary to consider the feedback loop effects on parameter identifiability. To handle the aforementioned technical difficulties, a new method based on z-transform [34, 35] is described in this section to generate identifiability equations for linear DSEMs. The method consists of three key steps: 1) employ z-transform to get the frequency domain representation of a DSEM; 2) obtain the z-transfer function between each pair of observed variables based on Mason’s gain; 3) derive identifiability equations from the z-transfer functions.

Specifically, consider the model in Eq. (1), we can get the following equation after z-transform

$${\mathcal{Y}}_i(z)=\sum _{j\ne i}{c}_{ij}{\mathcal{Y}}_j(z)+\sum _j\left[\sum _{k=1}^o{b}_{ij}^k\cdot z^{-k}\cdot {\mathcal{Y}}_j(z)\right]+{\sigma }^2,$$ (4)

where z is a complex number [34, 35] and σ² is the known constant variance of ε_i. The equation above can be rearranged as follows

$$(1-\sum _{k=1}^o{b}_{ii}^k\cdot z^{-k}){\mathcal{Y}}_i(z)=\sum _{j\ne i}(c_{ij}+\sum _{k=1}^o{b}_{ij}^k\cdot z^{-k}{\mathcal{Y}}_j(z)+{\sigma }^2,$$ (5)

$$\Rightarrow {\mathcal{Y}}_i(z)=\sum _{j\ne i}\frac{c_{ij}+\sum _{k=1}^o{b}_{ij}^k\cdot z^{-k}}{1-\sum _{k=1}^o{b}_{ii}^k\cdot z^{-k}}{\mathcal{Y}}_i(z)+\frac{{\sigma }^2}{1-\sum _{k=1}^o{b}_{ii}^k\cdot z^{-k}}.$$ (6)

Let $G_{ij}(z)=\frac{c_{ij}+{\sum }_{k=1}^o{b}_{ij}^k\cdot z^{-k}}{1-{\sum }_{k=1}^o{b}_{ii}^k\cdot z^{-k}}$ and ${\xi }_i(z)=\frac{{\sigma }^2}{1-{\sum }_{k=1}^o{b}_{ii}^k\cdot z^{-k}}$, and substitute them into Eq. (6) to obtain

$${\mathcal{Y}}_i(z)=\sum _{j\ne i}[G_{ij}(z)\cdot {\mathcal{Y}}_j(z)]+{\xi }_i(z),$$ (7)

where G_ij denotes the effect of variable ${\mathcal{Y}}_j$ on variable ${\mathcal{Y}}_i$ in the frequency domain. Eq. (7) is thus the frequency domain representation of Eq. (1), which also has a corresponding graphical representation. For example, the frequency domain representation of the DSEM in Fig. 1 is visualized in Fig. 2 as a DCG.

Figure 2.

The equivalent frequency domain representation of the dynamic SEM in Fig. 1, where ξ₁ (z), ξ₂ (z) and ξ₃ (z) in Eq. (7) are not shown since they are not involved in node interactions.

Based on the frequency domain representation of a DSEM, we can get the z-transfer function between any two observed nodes. For this purpose, Mason’s gain is considered here, which is applicable to directed cyclic networks for finding the input-output relationship between two nodes. Actually, it turns out that Mason’s gain in the frequency domain representation of a DSEM is equivalent to the z-transfer function of the DSEM. Specifically, consider a sub-graph of the frequency domain representation, consisting of an input node V_j, an output node V_i, and all the paths and loops between V_i and V_j. Let G_LLt denote the gain of loop L_t in the frequency domain (i.e., $G{L}_{L_t}=\prod _l{G}_l$ and G_l is the l-th edge coefficient on loop L_t in the frequency domain) and let r denote the maximum number of non-overlapping loops in the sub-graph, then the system determinant SD of the sub-graph is defined as follows

$$\mathit{SD}=1+\sum _{s=1}^r\left({(-1)}^s\cdot \sum _m{H}_m\right),$$ (8)

where $H_m=\prod _{t=1}^{s}G{L}_{L_t}$ is the m-th gain product of s non-overlapping loops. Similarly, let GP_Pk denote the gain associated with the forward path P_k in the frequency domain (i.e., $G{P}_{P_k}=\prod _l{G}_l$ and G_l is the l-th edge coefficient on path P_k in the frequency domain), then the path determinant of P_k is calculated in the same way as SD after excluding all feedback loops that intersect with P_k. The z-transfer function (i.e., the Mason’s gain) between two observed nodes V_i and V_j is thus given as follows [33]

$${\mathrm{MG}}_{ij}=\frac{{\mathcal{Y}}_{\mathit{out}}}{{\mathcal{Y}}_{in}}=\frac{{\mathcal{Y}}_i}{{\mathcal{Y}}_j}=\frac{\sum _{k=1}^{q}G{P}_{P_k}\cdot S{D}_{P_k}}{SD},$$ (9)

where q is the total number of forward paths from V_j to V_i. When two observed nodes V_i and V_j are in the same strongly connected component (SCC), MG_ij and MG_ji are dependent because $M{G}_{ij}\cdot M{G}_{ji}=\frac{{\mathcal{Y}}_j}{{\mathcal{Y}}_i}\cdot \frac{{\mathcal{Y}}_i}{{\mathcal{Y}}_j}=1$. For this reason, we only need to keep one of the two gains. Note that the use of Mason’s gain assumes that feedback cycling is infinite; for the finite feedback cycling case, the interested reader is referred to Hayduk [36]. For illustration, three z-transfer functions can be obtained from Fig. 2 as below

$$\{\begin{array}{c}M{G}_{12}=\frac{G_{21}}{1-G_{23}\cdot G_{32}}\hfill \\ =\frac{c_{21}+(b_{21}^1-c_{21}{b}_{33}^1)z^{-1}-b_{21}^1{b}_{33}^1{z}^{-2}}{1-c_{23}{c}_{32}-(b_{22}^1+b_{33}^1+c_{23}{b}_{32}^1+c_{32}{b}_{23}^1)z^{-1}+(b_{22}^1{b}_{33}^1-b_{23}^1{b}_{32}^1)z^{-2}}\hfill \\ M{G}_{23}=\frac{G_{32}}{1-G_{23}\cdot G_{32}}\hfill \\ =\frac{c_{32}+(b_{32}^1-c_{32}{b}_{22}^1)z^{-1}-b_{22}^1{b}_{32}^1{z}^{-2}}{1-c_{23}{c}_{32}-(b_{22}^1+b_{33}^1+c_{23}{b}_{32}^1+c_{32}{b}_{23}^1)z^{-1}+(b_{22}^1{b}_{33}^1-b_{23}^1{b}_{32}^1)z^{-2}}\hfill \\ M{G}_{13}=\frac{G_{21}\cdot G_{32}}{1-G_{23}\cdot G_{32}}\hfill \\ =\frac{c_{21}{c}_{32}+(c_{21}{b}_{32}^1+c_{32}{b}_{21}^1)z^{-1}+b_{21}^1{b}_{32}^1{z}^{-2}}{1-c_{23}{c}_{32}-(b_{22}^1+b_{33}^1+c_{23}{b}_{32}^1+c_{32}{b}_{23}^1)z^{-1}+(b_{22}^1{b}_{33}^1-b_{23}^1{b}_{32}^1)z^{-2}}\hfill \end{array}\phantom{\}}$$ (10)

Now we can derive the identifiability equations from the z-transfer functions. In a z-transfer function, the coefficients in front of z^−k (k = 0,1, ⋯, p) form a polynomial $f(c_{ij},b_{ij}^k)$ of unknown model parameters. If two variables Y_i and Y_j are observed in a DSEM, then the corresponding z-transfer function MG_ij can be uniquely determined through their observation values [34, 35], i.e., MG_ij is known, because their observation values are time series rather than a single value. Thus all the coefficients $f(c_{ij},b_{ij}^k)$ in front of z^−k of MG_ij are also known, i.e., $f(c_{ij},b_{ij}^k)=\mathcal{C}$, where $\mathcal{C}$ is a known constant, although its exact value may not be given. This fact allows us to obtain one identifiability equation from each z^−k term. Obviously, from one z-transfer function, we can get multiple identifiability equations as suggested in Eq. (10) and shown in Fig. 3(a). In the case that there exist some duplicate identifiability equations, we keep only one identifiability equation. For example, in Eq. (10), three z-transfer functions have the same denominator, then we only keep one denominator and ignore the other two when generating identifiability equations.

Figure 3.

Illustration of identifiability equations, identifiability matrices and the reduction results for the DSEM in Fig. 1.

4. STRUCTURAL IDENTIFIABILITY DETERMINATION

Since each identifiability equation is a symbolic polynomial equation and the order of each unknown parameter is at most one in all the identifiability equations, we can determine the structural identifiability of each parameter using the previously proposed identifiability-matrix method [28]. Briefly, this method consists of two steps: conversion of identifiability equations to identifiability matrices and determination of parameter identifiability by matrix reduction and grouping. Here we illustrate this method using the example in Fig. 3(a). First, we convert each identifiability equation in Fig. 3(a) to an identifiability matrix, as showed in Fig. 3(b). Then we reduce all the identifiability matrices by simplifying and removing dependent rows. The reduction results of the identifiability matrices in Fig. 3(b) are shown in Fig. 3(c), and we can tell from the identifiability matrices IE₁, IE₄, IE₇ and IE₁₀ that the unknown parameters c₂₁, c₂₃ and c₃₂ are globally identifiable because each of these four matrices has only one row and there is only one 1 element in this row. We then find locally identifiable and unidentifiable parameters by grouping the remaining matrices. In Fig. 3(c), after excluding IE₁, IE₄, IE₇ and IE₁₀, the remaining identifiability matrices are found to be in the same group because each matrix has more than one 1 elements and these matrices directly or indirectly couple with each other in the sense that the j-th columns of the two matrices both have at least one 1 element. There are 5 unknown parameters and 8 identifiability matrices in the group, then all the remaining parameters, i.e., $b_{21}^1$, $b_{22}^1$, $b_{23}^1$, $b_{32}^1$ and $b_{33}^1$, are locally identifiable. It is worth noting that structural identifiability is determined with the number of symbolic solutions in the identifiability-matrix method [37], since structural identifiability analysis does not use real data.

As illustrated in Fig. 3, the proposed method is applicable to general linear DSEMs; however, for specific DSEMs, we can directly determine structural identifiability of every unknown parameter according to the following theorem (see Appendix for proof).

Theorem 1. In a DSEM with only observed variables and self-feedback loops (i.e., $b_{ij}^k\ne 0$), an edge parameter is globally identifiable if the corresponding edge is not on any self-loops or if the corresponding edge is on a self-loop of a node and this node has at least one first-order neighbor node that has no self-loops; otherwise, the parameter is locally identifiable.

5. RESULTS

5.1. Comparisons

While the focus of this study is DSEM identifiability, it is worth mentioning other mainstream dynamic models like ODEs and state-transition models. DSEMs, ODEs and state-transition models all can describe dynamic systems; however, ODEs are typically continuous and deterministic while DSEMs and state-transition models are discrete and stochastic. More importantly, DSEMs consider both concurrent effects and memory effects but ODEs only consider concurrent interactions and state-transition models only consider memory effects. Therefore, state-transition models may be deemed as a special case of DSEMs. For illustration, consider a dynamic system that contains 3 observed nodes and 1 unobserved node, with node V₁ having an input u. The three different models of this system are given in Eq. (11), Eq. (12) and Eq. (13), and their graphic representations are shown in Fig. 4(a), (b) and (c), respectively.

Figure 4.

Three different models of a dynamic system. Green nodes are observed and gray nodes are unobserved, orange edges indicate concurrent effects and light blue edges indicate memory effects.

$$\{\begin{array}{c}{\stackrel{.}{y}}_1=u\hfill \\ {\stackrel{.}{y}}_2=c_{21}{y}_1+c_{23}{y}_3\hfill \\ {\stackrel{.}{y}}_3=c_{32}{y}_2\hfill \\ {\stackrel{.}{y}}_4=c_{41}{y}_1+c_{43}{y}_3\hfill \end{array}\phantom{\}},$$ (11)

$$\{\begin{array}{c}{y}_1^t=b_{11}^1{y}_1^{t-1}+{\epsilon }_1^t\hfill \\ y_2^t=b_{22}^1{y}_2^{t-1}+b_{23}^1{y}_3^{t-1}+{\epsilon }_2^t\hfill \\ y_3^t=b_{31}^1{y}_1^{t-1}+{\epsilon }_3^t\hfill \\ y_4^t=b_{43}^1{y}_3^{t-1}+b_{44}^1{y}_4^{t-1}+{\epsilon }_4^t\hfill \end{array}\phantom{\}},$$ (12)

$$\{\begin{array}{c}{y}_1^t=b_{11}^1{y}_1^{t-1}+{\epsilon }_1^t\hfill \\ y_2^t=c_{21}{y}_1^t+c_{23}{y}_3^t+b_{22}^1{y}_2^{t-1}+b_{23}^1{y}_3^{t-1}+{\epsilon }_2^t\hfill \\ y_3^t=c_{32}{y}_2^t+b_{31}^1{y}_1^{t-1}+{\epsilon }_3^t\hfill \\ y_4^t=c_{41}{y}_1^t+c_{43}{y}_3^t+b_{43}^1{y}_3^{t-1}+b_{44}^1{y}_4^{t-1}+{\epsilon }_4^t\hfill \end{array}\phantom{\}}.$$ (13)

DAISY [38] is a widely-used software package for structural identifiability analysis of ODE systems. It is based on differential algebra methods and can handle both linear and non-linear systems. For the ODE model in Fig. 4(a), we can analyze this example with DAISY and the identifiability result is shown in Fig. 5(a). To the best knowledge of our authors, there are not existing methods or tools that can determine parameter identifiability of state-transition models or DSEMs; therefore, we seek to the proposed method in this study. We obtain the identifiability results of the two models in Fig. 4(b) and (c), as shown in Fig. 5(b) and (c), respectively. Although the DSEM in Fig. 5(c) has more unknown model parameters than the other two models, all its edges are at least locally identifiable while in Fig. 5(a) and (b) all the edges connected with the unobserved node V₃ are unidentifiable. The particular reason for this case is that here the DSEM model corresponds to a more complex network structure (i.e., there are more paths between two observed nodes), and we can generate a larger number of identifiability equations to verify the identifiability of unknown parameters. More specifically, for three parameters c₂₃, c₃₂ and c₄₃ of Fig. 5(a), we can generate only two identifiability equations: ${\mathcal{C}}_1=c_{23}{c}_{32}$ and ${\mathcal{C}}_2=c_{32}{c}_{43}$. Also, for three parameters $b_{23}^1$, $b_{31}^1$ and $b_{43}^1$ of Fig. 5(b), we can get only two identifiability equations: ${\mathcal{C}}_1=b_{23}^1{b}_{31}^1$ and ${\mathcal{C}}_2=b_{31}^1{b}_{43}^1$. For these six parameters of Fig. 5(c), however, we can obtain seven reduced identifiability equations ${\mathcal{C}}_1=c_{23}{c}_{32}$, ${\mathcal{C}}_2=c_{23}{b}_{31}^1$, ${\mathcal{C}}_3=b_{23}^1{b}_{31}^1$, ${\mathcal{C}}_4=c_{32}{c}_{43}$, ${\mathcal{C}}_5=b_{43}^1{c}_{32}$, ${\mathcal{C}}_6=b_{31}^1{c}_{43}$, ${\mathcal{C}}_6=b_{31}^1{c}_{43}$ and ${\mathcal{C}}_7=b_{31}^1{b}_{43}^1$.

Figure 5.

The identifiability analysis results of the three models in Fig. 4. Green edges are globally identifiable, purple edges are locally identifiable and red edges are unidentifiable.

5.2. Application

To illustrate the application of the proposed method, we select three benchmark models in different disciplines from public literature. The first benchmark model is for investigating the neural correlation of speech [39]. The original graph contains 8 observed nodes and 8 directed edges, in which each node represents a region of interest (ROI) and each directed edge represents a theoretically plausible neural activation path. We assume that the corresponding DSEM has a first-order Markov property, and randomly add 3 new edges between two successive time points to get a DSEM as shown in Fig. 6(a). With the proposed method in this study, we can determine that all the parameters including the original 8 edge parameters and 3 new added parameters are globally identifiable. This is consistent with the conclusion drawn based on Theorem 1, that is, all the edge parameters are globally identifiable for a DSEM without feedback loops and latent variables.

Figure 6.

Two benchmark models without latent variables. Orange edges indicate concurrent effects and light blue edges indicate memory effects.

The second bench model is about social networks, selected from the well-known karate club study of Zachary [40]. Zachary tracked the club members for over two years, and a total of 34 members were divided into two groups: the club administrator’s group and the instructor’s group. Flere we choose the sub-network among the first six members. Different from Zachary’s study, we consider the sub-network as a directed graph by randomly setting edge directions. To introduce a 2nd-order Markov property to the model, we add three new edges between two successive time points, and two new edges between time point t and t+2. Then we derive a DSEM as shown in Fig. 6(b) from Zachary’s karate club study. We can determine that every parameter is also globally identifiable with the proposed method, although this model has a higher order Markov property. Since the model contains feedback loops at the same time point (i.e., $V_1^t\to V_3^t\to V_4^t\to V_1^t$) and between different time points (i.e., $V_1^t\to V_6^t\to V_1^{t+1}$, $V_1^{t-2}\to V_6^t\to V_1^{t+1}$ and $V_1^{t-1}\to V_5^{t-1}\to V_6^t\to V_1^{t+1}$) in addition to self-feedback loops (i.e., $V_2^{t-1}\to V_2^t$), Theorem 1 is not applicable to this model.

The third benchmark model is about biological networks, and is derived from the replication subnetwork of the within-host influenza virus life cycle [41]. Influenza A virus replication is a complex process, involving the interactions among many different biomolecules. It is therefore usually infeasible for one single experimental study to observe all the components and their interactions simultaneously, leading to the existence of latent variables. This subnetwork has 7 nodes, and we select one node as unobserved; also, we assume that this system has a first-order Markov property. With three new edges added between two successive time points, we get a DSEM as showed in Fig. 7(a). Different from the previous two benchmark models, this model contains one latent variable V_UAP56 and one loop that consists of the edges between different time points, i.e., $V_{\mathit{vRNPs}}^{t-1}\to V_{\mathit{UAP}56}^t\to V_{\mathit{vRNPs}}^{t+1}$. Because of the existence of feedback loops, Theorem 1 is also not applicable to this model. Then the identifiability results can be obtained using the proposed method, as in Fig. 7(b), which shows that all the edges associated with the latent variables are unidentifiable and the other edges are globally identifiable. More specifically, for the four unknown edge parameters associated with node V_UAP56, we can only get three identifiability equations after reduction: ${\mathcal{C}}_1=b_{\mathit{vRNPs}\to UAP56}^1{b}_{UAP56\to vRNPs}^1$, ${\mathcal{C}}_2=c_{\mathit{RNPcomplex}\to UAP56}{b}_{UAP56\to vRNPs}^1$ and ${\mathcal{C}}_3=b_{UAP56\to vRNPs}^1{b}_{viralRNA\to UAP56}^1$.

Figure 7.

One benchmark model with latent variables. Observed nodes are colored green and unobserved nodes are colored gray. Orange edges and light blue edges of the original model represent interactions at the same time point and between two successive time points, respectively. Green edges are globally identifiable and red edges are unidentifiable in the identifiability results.

6. CONCLUSIONS AND DISCUSSIONS

The SIA of DSEMs is a previously untouched topic in existing investigations. In this study, we systematically investigated DSEMs and their structural identifiability problem. A general and computerizable solution is proposed for the first time. More specifically, the proposed approach can handle a broad range of DSEMs (e.g., with latent variables, a finite-order Markov property and feedback loops). Similar to the identifiability matrix method, the proposed method can determine the structural identifiability of every single parameter. Because of no symbolic or expensive numerical computations, the proposed method is computationally efficient. A theoretical result is also obtained for a special class of DSEMs that contain only self-loops but no latent variables. Finally, we compare DSEMs and other dynamic models to illustrate the difference of these model structures and their identifiability results, and three selected benchmark DSEMs from different disciplines are given to illustrate the application of the proposed method.

It is worth mentioning that in addition to explicit latent variables, the proposed method can also handle implicit latent variables, i.e., the case of disturbance correlation. Similar to the work of Wang et al. [28], disturbance correlation can be handled by adding new dummy variables. That is, if two disturbance terms ε_i and ε_j are dependent, there exists an implicit latent variable or structure that affects both V_i and V_j. A dummy unobserved variable V_k can be added, and it will introduce additional edges such as V_k → V_i and V_k → V_j at the current time point as well as edges between different time points.

It should be stressed that the SIA of dynamic SEMs is quite different from the SIA of static SEMs. Firstly, a dynamic SEM contains both concurrent effects and memory effects between variables while a static SEM cannot account for memory effects. The difference of study focus leads to different generation methods of identifiability equations. Wright’s path coefficient method or its variations are adopted for static SEMs to generate identifiability equations [28]. In comparison, z-transfer functions are used for dynamic SEMs to generate identifiability equations. Wright’s path coefficient method considers the common ancestor node effects in addition to path effects. Nevertheless, z-transfer function deals with only path effects. For example, given a graph with 3 edges, V_i → V_j, V_k → V_i and V_k → V_j, the z-transfer function MG_ij contains only the effect of the path V_i → V_j, but the Wright’s path coefficient Cov_ij includes the effect of V_k on both V_i and V_j (i.e., two paths V_k → V_i and V_k → V_j) in addition to the effect of the path V_i → V_j. Since the generation methods of identifiability equations are different, unsurprisingly, the identifiability results are also different. Given a static SEM G_s, we can generate a dynamic SEM G_d by some new edges between different time points (i.e., adding some memory effects between variables). Then generally speaking, the identifiability results of G_d is better than the results of G_s due to the introduction of temporal observations. That is, some unidentifiable edge parameters of G_s may become identifiable in graph G_d. One can verify this conclusion with the example in Fig. 6(b).

This work also has some limitations. For instance, to use z-transform, all the observations are assumed to be equally spaced, which may not be the case in practice. And real dynamic systems may contain both discrete and continuous variables, or involve nonlinear interactions, to which the proposed method is not applicable. In addition, the computation complexity of the proposed method is Ο($N_o^2$ · N³), where N_ο is the number of observed variables and N is the total number of variables. The proposed method cannot directly be applicable to high-dimensional systems until it has been specially optimized for high-dimensional systems. However, we expect to tackle the more complex SIA problems for DSEMs in the future, and this study provides a foundation for such future investigations.

APPENDIX A. PROOF OF THEOREM 1

Theorem 1. In a DSEM with only observed variables and self-feedback loops (i.e., $b_{ij}^k\ne 0$), an edge parameter is globally identifiable if the corresponding edge is not on any self-loops or if the corresponding edge is on a self-loop of a node and this node has at least one first-order neighbor node that has no self-loops; otherwise, the parameter is locally identifiable.

Proof: Consider a DSEM G without any feedback loops and latent variables first. After z-transform, let G′ denote the frequency domain representation of G. In this case, the coefficient associated with each edge in G′ is the Mason’s gain between two end nodes of the corresponding edge, and the denominator of the Mason’s gain is 1 because there are not feedback loops. Since every node of G′ is observed and G′ is a DAG, all edge coefficients of G′ are globally identifiable according to Kline’s work [42]. More specifically, for any two observed nodes V_i and V_j, the Mason’s gain $M{G}_{ij}=c_{ij}+{\sum }_{k=1}^o{b}_{ij}^k\cdot z^{-k}$ is known, and thus we can get ο + 1 identifiability equations

$$\begin{gathered} I{E}_1:{\mathcal{C}}_1=c_{ij} \\ I{E}_2:{\mathcal{C}}_2=b_{ij}^1 \\ I{E}_3:{\mathcal{C}}_3=b_{ij}^2 \\ I{E}_3:{\mathcal{C}}_3=b_{ij}^2 \\ ⋮ \\ I{E}_{o+1}:{\mathcal{C}}_{o+1}=b_{ij}^o \end{gathered}$$ (A-1)

Since each identifiability equation in Eq. (A-1) contains only one monomial that has only one unknown parameter of order 1, then every unknown parameter has a unique solution and is therefore globally identifiable.

Now consider a DSEM G with only self-loops but no latent variables. Without loss of generality, assume node V_j has a self-loop (multiple self-loops are not allowed). There are two different cases to consider: 1) at least one of the first-order neighbor nodes of V_j has no self-loops, and 2) every one of the first-order neighbor nodes of V_j has a self-loop.

For the first case, assume that node V_i is the first-order in-neighbor node of V_j and V_i has no self-loops. Because the path from V_i to V_j does not pass any other loops except for the self-loop of V_j, the Mason’s gain between V_i and V_j is $M{G}_{ij}=\frac{c_{ij}+{\sum }_{k=1}^o{b}_{ij}^k\cdot z^{-k}}{1-{\sum }_{k=1}^o{b}_{ij}^k\cdot z^{-k}}$. Since both V_i and V_j are observed, MG_ij is a known constant. Then we can generate 2ο + 1 identifiability equations from MG_ij. Besides the same ο + 1 equations in Eq. (A-1), ο more identifiability equations are now generated on parameters $b_{ii}^k$ (k = 1, ⋯, ο). Same as Eq. (A-1), each of these ο identifiability equation contains only one monomial that has only one unknown parameter of order 1. Therefore, $b_{ij}^k$ and $b_{ii}^k$ are all globally identifiable.

For the second case, every one of the first-order neighbor nodes of V_j has a self-loop. Without loss of generality, let node V_i be one in-neighbor node of V_j and it has a self-loop. Different from the previous case, the Mason’s gain between V_i and V_j is

$$\begin{gathered} M{G}_{ij}=\frac{c_{ij}+\sum _{k=1}^o{b}_{ij}^k\cdot z^{-k}}{\left(1-\sum _{k=1}^o{b}_{ii}^k\cdot z^{-k}\right)\cdot \left(1-\sum _{k=1}^o{b}_{jj}^k\cdot z^{-k}\right)}, \\ =\frac{c_{ij}+\sum _{k=1}^o{b}_{ii}^k\cdot z^{-k}}{1+\sum _{l=1}^{2o}S{P}_l\cdot z^{-l}} \end{gathered}$$ (A-2)

with SP_l being a symbolic polynomial as follows

$$S{P}_l=\{\begin{array}{c}-(b_{ii}^1+b_{jj}^1)l=1\hfill \\ \sum _{m=1}^{l-1}{b}_{ii}^m\cdot b_{jj}^{l-m}-(b_{ii}^l+b_{jj}^l)1<l\le o\hfill \\ \sum _{m=\lfloor l∕2\rfloor }^o{b}_{ii}^m\cdot b_{jj}^{l-m}o<l\le 2o\hfill \end{array}\phantom{\}},$$ (A-3)

where ⌊l/2⌋ is the floor integer of l/2. We can generate 2ο new identifiability equations from Eq. (A-3) besides the o+1 equations as in Eq. (A-1). Same as the case without any feedback loops and latent variables, we can conclude from these ο + 1 equations that parameters c_ij and $b_{ij}^k$ are globally identifiable. It is known from Eq. (A-3) that these 2ο new equations contain only 2ο unknown parameters, i.e., $b_{ii}^k$ and $b_{jj}^k$ (k = 1, ⋯, ο), but these equations are not always linear. We can determine the structural identifiability of parameters $b_{ii}^k$ and $b_{jj}^k$ (k = 1, ⋯, ο) using the Jacobian matrix. It can be found that none of the equations can be represented as a linear combination of other equations; that is, the rank of the corresponding Jacobian matrix is 2ο. However, since the highest order of many equations is greater than 1, there exist a finite number of solutions to these equations. Therefore, the parameters associated with self-loops (i.e., $b_{ii}^k$) are locally identifiable in this case.

In summary, the theorem holds. □