Evaluating the Gilbert–Varshamov Bound for Constrained Systems

Abstract

We revisit the well-known Gilbert–Varshamov (GV) bound for constrained systems. In 1991, Kolesnik and Krachkovsky showed that the GV bound can be determined via the solution of an optimization problem. Later, in 1992, Marcus and Roth modified the optimization problem and improved the GV bound in many instances. In this work, we provide explicit numerical procedures to solve these two optimization problems and, hence, compute the bounds. We then show that the procedures can be further simplified when we plot the respective curves. In the case where the graph presentation comprises a single state, we provide explicit formulas for both bounds.

1. Introduction

From early applications in magnetic recording systems to recent applications in DNA-based data storage [1,2,3,4] and energy-harvesting [5,6,7,8,9,10], constrained codes have played a central role in enhancing reliability in many data storage and communications systems (see also [11] for an overview). Specifically, for most data storage systems, certain substrings are more prone to errors than others. Thus, by forbidding the appearance of such strings, that is, by imposing constraints on the codewords, the user is able to reduce the likelihood of error. We refer to the collection of words that satisfy the constraints as the constrained space $\mathcal{S}$.

To further reduce the error probability, one can impose certain distance constraints on the codebook. In this work, we focus on the Hamming metric and consider the maximum size of a codebook whose words belong to the constrained space $\mathcal{S}$ and whose pairwise distance is at least of a certain value d. Specifically, we study one of the most well-known and fundamental lower bounds of this quantity—the Gilbert–Varshamov (GV) bound.

To determine the GV bound, one requires two quantities: the size of the constrained space, $\left|\mathcal{S}\right|$, and, also, the ball volume, that is, the number of words with a distance of at most $d-1$ from a “center” word. In the case where the space is unconstrained, i.e., $\mathcal{S}={\{0,1\}}^n$, the ball volume does not depend on the center. Then, the GV bound is simply $\left|\mathcal{S}\right|/V$, where V is the ball volume of a center. However, for most constrained systems, the ball volume varies with the center. Nevertheless, Kolesnik and Krachkovsky showed that the GV lower bound can be generalized to $\left|\mathcal{S}\right|/4\overline{V}$, where $\overline{V}$ is the average ball volume [12]. This was further improved by Gu and Fuja to $\left|\mathcal{S}\right|/\overline{V}$ in [13] (see pp. 242–243 in [11] for additional details). In the same paper [12], they showed the asymptotic rate of average ball volume can be computed via an optimization problem. Later, Marcus and Roth modified the optimization problem by including an additional constraint and variable [14], and the resulting bound, which we refer to as GV-MR bound, improves the usual GV bound. Furthermore, in most cases, the improvement is strictly positive.

However, about three decades later, very few works have evaluated these bounds for specific constrained systems. To the best of our knowledge, in all works that numerically computed the GV bound and/or GV-MR bound, the constrained systems of interest have, at most, eight states [15]. In [15], the authors wrote that “evaluation of the bound required considerable computation”, referring to the GV-MR bound.

In this paper, we revisit the optimization problems defined by Kolesnik and Krachkovsky [12] and Marcus and Roth [14] and develop a suite of explicit numerical procedures that solve these problems. In particular, to demonstrate the feasibility of our methods, we evaluated and plotted the GV and GV-MR bounds for a constrained system involving 120 states in Figure 1b.

Figure 1.

Lower bounds for optimal asymptotic code rates $R(\delta ;\mathcal{S})$ for the class of sliding-window constrained codes.

We provide a high-level description of our approach. For both optimization problems, we first characterized the optimal solutions as roots of certain equations. Then, using the celebrated Newton–Raphson iterative procedure, we proceeded to find the roots of these equations. However, as the latter equations involved the largest eigenvalues of certain matrices, each Newton–Raphson iteration required the (partial) derivatives of these eigenvalues (in some variables). To resolve this, we made modifications to another celebrated iterative procedure—the power iteration method—and the resulting procedures computed the GV and GV-MR bounds efficiently for a specific relative distance $\delta$. Interestingly, if we plot the bounds for $0\le \delta \le 1$, the numerical procedure can be further simplified. Specifically, by exploiting certain properties of the optimal solutions, we provided procedures that use less Newton–Raphson iterations.

Parts of this paper were presented in the IEEE International Symposium on Information Theory (ISIT 2022) [16]. In the next section, we provide the formal definitions and state the optimization problems that compute the GV bound.

2. Preliminaries

Let $\Sigma =\{0,1\}$ be the binary alphabet and let ${\Sigma }^n$ denote the set of all words of length n over $\Sigma$. A labeled graph $\mathcal{G}=(\mathcal{V},\mathcal{E},\mathcal{L})$ is a finite directed graph with states $\mathcal{V}$, edges$\mathcal{E}\subseteq \mathcal{V}\times \mathcal{V}$, and an edge labeling $\mathcal{L}:\mathcal{E}\to {\Sigma }^s$ for some $s\ge 1$. Here, we use $v_i\stackrel{\sigma }{\to }{v}_j$ to mean that there is an edge from $v_i$ to $v_j$ with label $\sigma$. The labeled graph $\mathcal{G}$ is deterministic if, for each state, the outgoing edges have distinct labels.

A constrained system $\mathcal{S}$ is, then, the set of all words obtained by reading the labels of paths in a labeled graph $\mathcal{G}$. We say that $\mathcal{G}$ is a graph presentation of $\mathcal{S}$. We further denote the set of all n-length words $\mathcal{S}$ by ${\mathcal{S}}_n$. Alternatively, ${\mathcal{S}}_n$ is the set of all words obtained by reading the labels of $(n/s)$-length paths in $\mathcal{G}$. Then, the capacity of $\mathcal{S}$, denoted by $\mathrm{Cap}\left(\mathcal{S}\right)$, is given by $\mathrm{Cap}\left(\mathcal{S}\right)\triangleq {lim\; sup}_{n\to \infty }log|{\mathcal{S}}_n|/n$. It is well-known that $\mathrm{Cap}\left(\mathcal{S}\right)$ corresponds to the largest eigenvalue of the adjacency matrix ${\mathit{A}}_{\mathcal{G}}$ (see, for example, [11]). Here, ${\mathit{A}}_{\mathcal{G}}$ is a $\left(\right|\mathcal{V}|\times |\mathcal{V}\left|\right)$-matrix whose rows and columns are indexed by $\mathcal{V}$. For each entry $(u,v)\in \mathcal{V}\times \mathcal{V}$, we set the corresponding entry to be one if $(u,v)$ is an edge, and zero otherwise.

Every constrained system can be presented by a deterministic graph $\mathcal{G}$. Furthermore, any deterministic graph can be transformed into a primitive deterministic graph $\mathcal{H}$ such that the capacity of $\mathcal{G}$ is same as the capacity of the constrained system presented by some irreducible component (maximal irreducible subgraph) of $\mathcal{H}$ (see, for example, Marcus et al. [11]). It should be noted that a graph $\mathcal{G}$ is primitive if there exists a positive integer ℓ such that ${\left({\mathit{A}}_{\mathcal{G}}\right)}^{\ell }$ is strictly positive. Therefore, we henceforth assume that our graphs are deterministic and primitive. When $\left|\mathcal{V}\right|=1$, we call this a single-state graph presentation and study these graphs in Section 5.

For $\mathit{x},\mathit{y}\in \mathcal{S}$, $d_H(\mathit{x},\mathit{y})$ is the Hamming distance between x and y. We fix $1\le d\le n$, and a fundamental problem in coding theory is finding the largest subset $\mathcal{C}$ of ${\mathcal{S}}_n$ such that $d_H(\mathit{x},\mathit{y})\ge d$ for all distinct $\mathit{x},\mathit{y}\in \mathcal{C}$. Let $A(n,d;\mathcal{S})$ denote the size of largest subset $\mathcal{C}$.

In terms of asymptotic rates, we fix $0\le \delta \le 1$, and our task is to find the highest attainable rate, denoted by $R\left(\delta \right)$, which is given by $R(\delta ;\mathcal{S})\triangleq {lim\; sup}_{n\to \infty }logA(n,⌊\delta n⌋;\mathcal{S})/n$.

2.1. Review of Gilbert–Varshamov Bound

To define the GV bound, we need to determine the total ball size. Specifically, for $\mathit{x}\in {\mathcal{S}}_n$ and $0\le r\le n$, we define $V(\mathit{x},r;\mathcal{S})\triangleq \left|\right\{\mathit{y}\in {\mathcal{S}}_n:d_H(\mathit{x},\mathit{y})\le r\left\}\right|$. We further define $T(n,d;\mathcal{S})={\displaystyle {\sum }_{\mathit{x}\in {\mathcal{S}}_n}}V(\mathit{x},d-1;\mathcal{S})$ . Then, the GV bound, as given by Gu and Fuja [13,17], states that there exists an $(n,d;\mathcal{S})$ code of size at least $|{\mathcal{S}}_n{|}^2/T(n,d;\mathcal{S})$.

In terms of asymptotic rates, there exists a family of $(n,⌊\delta n⌋;\mathcal{S})$ codes such that their rates approach

$$R_{\mathrm{GV}}\left(\delta \right)=2\mathrm{Cap}\left(\mathcal{S}\right)-\stackrel{\sim }{T}\left(\delta \right),$$ (1)

where $\stackrel{\sim }{T}\left(\delta \right)\triangleq {lim\; sup}_{n\to \infty }logT(n,⌊\delta n⌋;\mathcal{S})/n$ .

In this paper, our main task is to determine $R_{\mathrm{GV}}\left(\delta \right)$efficiently. We observe that since $\mathrm{Cap}\left(\mathcal{S}\right)=\stackrel{\sim }{T}\left(0\right)$, it suffices to find efficient ways of determining $\stackrel{\sim }{T}\left(\delta \right)$. It turns out that $\stackrel{\sim }{T}\left(\delta \right)$ can be found via the solution of a convex optimization problem. Specifically, given a labeled graph $\mathcal{G}=(\mathcal{V},\mathcal{E},\mathcal{L})$, we define its product graph $\mathcal{G}\prime =(\mathcal{V}\prime ,\mathcal{E}\prime ,\mathcal{L}\prime )$ as follows:

• $\mathcal{V}\prime \triangleq \mathcal{V}\times \mathcal{V}$.

• For $(v_i,v_j),(v_k,v_{\ell })\in \mathcal{V}\prime$, and $({\sigma }_1,{\sigma }_2)\in {\Sigma }^s\times {\Sigma }^s$, we draw an edge $(v_i,v_j)\stackrel{({\sigma }_1,{\sigma }_2)}{\to }(v_k,v_{\ell })$ if and only if both $v_i\stackrel{{\sigma }_1}{\to }{v}_k$ and $v_j\stackrel{{\sigma }_2}{\to }{v}_{\ell }$ belong to $\mathcal{E}$.

• Then, we label the edges in $\mathcal{E}\prime$ with the function $\mathcal{L}\prime :\mathcal{E}\prime \to {\mathbb{Z}}_{\ge 0}$, where $\mathcal{L}\prime \left((v_i,v_j)\stackrel{({\sigma }_1,{\sigma }_2)}{\to }(v_k,v_{\ell })\right)=d_H({\sigma }_1,{\sigma }_2)/s$.

A stationary Markov chain P on a graph $\mathcal{G}=(\mathcal{V},\mathcal{E},\mathcal{L})$ is a probability distribution function $P:\mathcal{E}\to [0,1]$ such that ${\displaystyle {\sum }_{e\in E}}P\left(e\right)=1$ and, for any state $u\in \mathcal{G}$, the sum of the probabilities of the outgoing edges equals the sum of the probabilities of the incoming edges. We denote by $\mathcal{M}\left(\mathcal{G}\right)$ the set of all stationary Markov chains on $\mathcal{G}$. For a state $u\in \mathcal{V}$, let ${\mathcal{E}}_u$ denote the set of outgoing edges from u in $\mathcal{G}$. The state vector ${\pi }^T={\left({\pi }_u\right)}_{u\in V}$ of a stationary Markov chain P on $\mathcal{G}$ is defined by ${\pi }_u={\displaystyle {\sum }_{e\in E_u}}P\left(e\right)$. The entropy rate of a stationary Markov chain is defined by

$$\begin{array}{c}\hfill H\left(P\right)=-{\displaystyle \sum _{u\in \mathcal{V}}}{\displaystyle \sum _{e\in {\mathcal{E}}_u}}{\pi }_{u}P\left(e\right)log\left(P\left(e\right)\right)\end{array}$$

Furthermore, $\stackrel{\sim }{T}\left(\delta \right)$ can be obtained by solving the following optimization problem [12,14]:

$$\stackrel{\sim }{T}\left(\delta \right)=sup\left\{H\left(P\right):P\in \mathcal{M}(\mathcal{G}\times \mathcal{G}),{\displaystyle \sum _{e\in \mathcal{E}\prime }}P\left(e\right)D\left(e\right)\le \delta \right\}.$$ (2)

To this end, we consider the dual problem of (2). Specifically, we define a $\left(\right|\mathcal{V}{|}^2\times \left|\mathcal{V}{|}^2\right)$-distance matrix ${\mathit{T}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)$ whose rows and columns are indexed by $\mathcal{V}\prime$. For each entry indexed by $e\in {\mathcal{V}}^{\prime }\times {\mathcal{V}}^{\prime }$, we set the entry to be zero if $e\notin {\mathcal{E}}^{\prime }$ and we set it to be $y^{D\left(e\right)}$ if $e\in {\mathcal{E}}^{\prime }$. Then, the dual problem can be stated in terms of the dominant eigenvalue of the matrix ${\mathit{T}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)$.

By applying the reduction techniques from [14], we can reduce the problem size by a factor of two. Formally, in the case of $s=1$, we define a $\left(\genfrac{}{}{0pt}{}{\left|\mathcal{V}\right|+1}{2}\right)\times \left(\genfrac{}{}{0pt}{}{\left|\mathcal{V}\right|+1}{2}\right)$-reduced distance matrix ${\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)$ whose rows and columns are indexed by ${\mathcal{V}}^{\left(2\right)}\triangleq \{(v_i,v_j):1\le i\le j\le \left|\mathcal{V}\right|\}$ using the following procedure.

Two states $s_1=(v_i,v_j)$ and $s_2=(v_k,v_{\ell })$ in $\mathcal{G}\times \mathcal{G}$ are said to be equivalent if $v_i=v_{\ell }$ and $v_j=v_k$. The matrix ${\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)$ is then obtained by merging all pairs of equivalent states $s_1$ and $s_2$. That is, we add the column indexed by $v_2$ to the column indexed by $v_1$ and then remove the row and column which are indexed by $v_2$. It should be noted that it may be possible to reduce the size of this matrix ${\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)$ further. However, for the ease of exposition, we did not consider this case in this work.

Following this procedure, we observe that the entries in the matrix ${\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)$ can be described by the rules in Table 1. Moreover, the dominant eigenvalue of ${\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)$ is the same as that of ${\mathit{T}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)$. Then, by strong duality, computing (2) is equivalent to solving the following dual problem [18,19] (see also, [20]):

$$\stackrel{\sim }{T}\left(\delta \right)=inf\left\{-\delta logy+log\Lambda \left({\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)\right):0\le y\le 1\right\}.$$ (3)

Here, we use $\Lambda \left(\mathit{M}\right)$ to denote the dominant eigenvalue of matrix M. To simplify further, we write $\Lambda (y;\mathit{B})\triangleq \Lambda \left({\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)\right)$.

Table 1.

We set the $\left((v_i,v_j),(v_k,v_{\ell })\right)$ entry of the matrix ${\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)$ according to subgraph induced by the states $v_i$,$v_j$,$v_k$ Gilbert–Varshamov $v_{\ell }$. Here, $\overline{\sigma }$ denotes the complement of $\sigma$.

${\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(\mathit{y}\right)$ at Entry $\left(({\mathit{v}}_{\mathit{i}},{\mathit{v}}_{\mathit{j}}),({\mathit{v}}_{\mathit{k}},{\mathit{v}}_{\mathit{\ell }})\right)$	Subgraph Induced by the States $\{{\mathit{v}}_{\mathit{i}},{\mathit{v}}_{\mathit{j}},{\mathit{v}}_{\mathit{k}},{\mathit{v}}_{\mathit{\ell }}\}$
0
1
y
$2y$

Since the objective function in (3) is convex, it follows from standard calculus that any local minimum solution $y^{*}$ in the interval $[0,1]$ is also a global minimum solution. Furthermore, $y^{*}$ is a zero of the first derivative of the objective function. If we consider the numerator of this derivative, then $y^{*}$ is a root of the function

$$F\left(y\right)\triangleq y{\Lambda }^{\prime }(y;\mathit{B})-\delta \Lambda (y;\mathit{B}).$$ (4)

In Corollary 1, we showed that there is only one $y^{*}$ such that $F\left(y^{*}\right)=0$ and $F^{\prime }\left(y\right)$ is strictly positive for all values of y. Therefore, to evaluate the GV bound for a fixed $\delta$, it suffices to determine $y^{*}$.

Later, Marcus and Roth [14] improved the GV bound (1) by considering certain subsets of the constrained space $\mathcal{S}$. This entails the inclusion of an additional constraint defined in the optimization problem (2), and, correspondingly, an additional variable in the dual problem (3). Specifically, they considered certain subsets $\mathcal{S}\left(p\right)\subseteq \mathcal{S}$ where each symbol in the words of $\mathcal{S}\left(p\right)$ appears with a certain frequency dependent on the parameter p. We describe this in more detail in Section 4.

2.2. Our Contributions

• (A)In Section 3, we develop the numerical procedures to compute $\stackrel{\sim }{T}\left(\delta \right)$ for a fixed $\delta$ and, hence, determine the GV bound (1). Our procedure modifies the well-known power iteration method to compute the derivatives of $\Lambda (y;\mathit{B})$. After that, using these derivatives, we apply the classical Newton–Raphson method to determine the root of (4). In the same section, we also study procedures to plot the GV curve, that is, the set $\{(\delta ,R_{\mathrm{GV}}\left(\delta \right)):0\le \delta \le 1\}$. Here, we demonstrate that the GV curve can be plotted without any Newton–Raphson iterations.
• (B)In Section 4, we then develop similar power iteration methods and numerical procedures to compute the GV-MR bound. Similar to the GV curve, we also provide a plotting procedure that uses significantly less Newton–Raphson iterations.
• (C)In Section 5, we provide explicit formulas for the computation of the GV bound and GV-MR bound for graph presentations that have exactly one state but multiple parallel edges.
• (D)In Section 6, we validate our methods by computing the GV and the GV-MR bounds for some specific constrained systems. For comparison purposes, we also plot a simple lower bound that is obtained by using an upper estimate of the ball size. From the plots in Figure 1, Figure 2 and Figure 3, it is also clear that the GV and GV-MR bounds are significantly better. We also observe that the GV bound and GV-MR bound for subblock energy-constrained codes (SECCs) obtained through our procedures improve the GV-type bound given by Tandon et al. (Proposition 12 in [21]).

Figure 2.

Lower bounds for optimal asymptotic code rates $R(\delta ;\mathcal{S})$ for the class of runlength limited codes.

Figure 3.

Lower bounds for optimal asymptotic code rates $R(\delta ;\mathcal{S})$ where $\mathcal{S}$ is the class of $(3,2)$-SECCs (subblock energy-constrained codes).

3. Evaluating the Gilbert–Varshamov Bound

In this section, we first describe a numerical procedure that solves (3) and, hence, determine $R_{\mathrm{GV}}\left(\delta \right)$ for fixed values of $\delta$. Then, we show that the procedure can be simplified when we compute the GV curve, that is, the set of points $\{(\delta ,R_{\mathrm{GV}}\left(\delta \right)):\delta \in \left[0,1\right]\}$. Here, we eschew notation and use $\left[a,b\right]$ to denote the interval $\{x:a\le x\le b\}$, if $a<b$, and the interval $\{x:b\le x\le a\}$ otherwise.

Below, we provide formal description of our procedure to obtain the GV bound for a fixed relative distance $\delta$.

Procedure 1 (GV bound for fixed relative distance).

Input: Adjacency matrix ${\mathit{A}}_{\mathcal{G}}$, reduced distance matrix ${\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)$, and relative minimum distance $\delta$

Output: GV bound, that is, $R_{\mathrm{GV}}\left(\delta \right)$ as defined in (1)

• (1)Apply the Newton–Raphson method to obtain $y^{*}$ such that $F\left(y^{*}\right)$ is approximately zero.

  • Fix the tolerance value $ϵ$.

  • Set $t=0$ and pick an initial guess $0\le y_t\le 1$.

  • While $|y_t-y_{t-1}|>ϵ$,

    • –
      Compute the next guess $y_{t+1}$ as follows:

      $$y_{t+1}=y_t-{\displaystyle \frac{F\left(y_t\right)}{F^{\prime }\left(y_t\right)}}=y_t-{\displaystyle \frac{y_t{\Lambda }^{\prime }(y_t;\mathit{B})-\delta \Lambda (y_t;\mathit{B}))}{(1-\delta ){\Lambda }^{\prime }(y_t;\mathit{B})+y_t{\Lambda }^{\prime \prime }(y_t;\mathit{B})}}.$$
    • –In this step, apply the power iteration method to compute $\Lambda (y_t;\mathit{B})$, $\Lambda \prime (y_t;\mathit{B})$, and $\Lambda \prime \prime (y_t;\mathit{B})$.
    • –Increment t by one.

  • Set $y^{*}\leftarrow y_t$.

• (2)Determine $R_{\mathrm{GV}}\left(\delta \right)$ using $y^{*}$. Specifically, compute $\stackrel{\sim }{T}\left(\delta \right)\triangleq -\delta log{y}^{*}+log\Lambda (y^{*};\mathit{B})$, $\mathrm{Cap}\left(\mathcal{S}\right)\triangleq log\Lambda \left({\mathit{A}}_{\mathcal{G}}\right)$, and $R_{\mathrm{GV}}\left(\delta \right)\triangleq 2\mathrm{Cap}\left(\mathcal{S}\right)-\stackrel{\sim }{T}\left(\delta \right)$.

Throughout Section 3 and Section 4, we illustrate our numerical procedures via a running example using the class of sliding window-constrained codes (SWCCs). Formally, we fix a window length L and window weightw, and say that a binary word satisfies the $(L,w)$-sliding window weight constraint if the number of ones in every consecutive L bits is at least w. We refer to the collection of words that meet this constraint as an $(L,w)$-SWCC constrained system. The class of SWCCs was introduced by Tandon et al. for the application of simultaneous energy and information transfer [7,10]. Later, Immink and Cai [8,9] studied encoders for this constrained system and provided a simple graph presentation that uses only $\left(\genfrac{}{}{0pt}{}{L}{w}\right)$ states.

In the next example, we illustrate how the numerical procedure can be used to compute the GV bound for the value when $\delta =0.1$.

Example 1.

Let $L=3$ and $w=2$, and we consider a $(3,2)$-SWCC constrained system. From [8], we have the following graph presentation with states x11, 101, and  110

Then, the corresponding adjacency and reduced distance matrices are as follows:

$${\mathit{A}}_{\mathcal{G}}=\left[\begin{array}{ccc}1& 1& 0\\ 0& 0& 1\\ 1& 0& 0\end{array}\right],{\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)=\left[\begin{array}{cccccc}1& 2y& 0& 1& 0& 0\\ 0& 0& 1& 0& y& 0\\ 1& y& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 1\\ 0& 0& 1& 0& 0& 0\\ 1& 0& 0& 0& 0& 0\end{array}\right].$$

To determine the GV bound at $\delta =0.1$, we first approximate the optimal point $y^{*}$ for which $-\delta logy+log\Lambda (y;\mathit{B})$ is minimized.

We apply the Newton–Raphson method to find a zero of the function $F\left(y\right)$. Now, with the initial guess $y_0=0.3$, we apply the power iteration method to determine

$$\Lambda (0.3;\mathit{B})=1.659,\Lambda \prime (0.3;\mathit{B})=0.694,\Lambda \prime \prime (0.3;\mathit{B})=0.183.$$

Then, we compute that $y_1\approx 0.238$. Repeating the computations, we have that $y_2\approx 0238$. Since $|y_2-y_1|$ is less than the tolerance value ${10}^{-5}$, we set $y^{*}=0.238$. Hence, we have that $\stackrel{\sim }{T}\left(0.1\right)=0.9$. Applying the power iteration method to either ${\mathit{A}}_{\mathcal{G}}$ or ${\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(0\right)$, we compute the capacity of the $(3,2)$-SWCC constrained system to be $\mathrm{Cap}\left(\mathcal{S}\right)=0.551$. Then, the GV bound is given by $R_{\mathrm{GV}}\left(0.1\right)=2\left(0.551\right)-0.9=0.202$.

We discuss the convergence issues arising from Procedure 1. We observe that there are two different iterative processes in Step 1, namely, (a) the power iteration method to compute the values $\Lambda (y_t;\mathit{B})$, $\Lambda \prime (y_t;\mathit{B})$, and $\Lambda \prime \prime (y_t;\mathit{B})$, and (b) the Newton–Raphson method that determines the zero of $F\left(y\right)$.

• (a)We recall that $\Lambda (y;\mathit{B})$ is the largest eigenvalue of the reduced distance matrix ${\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)$. If we apply naive methods to compute this dominant eigenvalue, the computational complexity increases very rapidly with the matrix size. Specifically, if $\mathcal{G}$ has M states, then the reduced distance matrix has dimensions $\Theta \left(M^2\right)\times \Theta \left(M^2\right)$ and finding its characteristic equation takes $O\left(M^6\right)$ time. Even then, determining the exact roots of characteristic equations with at least five degrees is generally impossible. Therefore, we turn to the numerical procedures like the ubiquitous power iteration method [22]. However, the standard power iteration method is only able to compute the dominant eigenvalue $\Lambda (y;\mathit{B})$. Nevertheless, we can modify the power iteration method to compute $\Lambda (y;\mathit{B})$ and its higher order derivatives. In Appendix A, we demonstrate that under certain mild assumptions, the modified power iteration method always converges. Moreover, using the sparsity of the reduced distance matrix, we have that each iteration can be completed in $O\left(M^2\right)$ time.
• (b)Next, we discuss whether we can guarantee that $y_t$ converges to $y^{*}$ as t approaches infinity. Even though the Newton–Raphson method converges in all our numerical experiments, we are unable to demonstrate that it always converges for $F\left(y\right)$. Nevertheless, we can circumvent this issue if we are interested in plotting the GV curve. Specifically, if our objective is to determine the curve $\{(\delta ,R_{\mathrm{GV}}\left(\delta \right)):\delta \in \left[0,1\right]\}$, it turns out that we do not need to implement the Newton–Raphson iterations and we discuss this next.

We fix some constrained system $\mathcal{S}$. Let us define its corresponding GV curve to be the set of points $\mathcal{GV}\left(\mathcal{S}\right)\triangleq \{(\delta ,R_{\mathrm{GV}}\left(\delta \right)):\delta \in \left[0,1\right]\}$. Here, we demonstrate that the GV curve can be plotted without any Newton–Raphson iterations.

To this end, we observe that when $F\left(y^{*}\right)=0$, we have that $\delta =y^{*}\Lambda \prime (y^{*};\mathit{B})/\Lambda (y^{*};\mathit{B})$. Hence, we eschew notation and define the function

$$\delta \left(y\right)\triangleq y\Lambda \prime (y;\mathit{B})/\Lambda (y;\mathit{B}).$$ (5)

We further define ${\delta }_{\max }=\delta \left(1\right)=\Lambda \prime (1;\mathit{B})/\Lambda (1;\mathit{B})$. In this section, we prove the following theorem.

Theorem 1.

Let $\mathcal{G}$ be the graph presentation for the constrained system $\mathcal{S}$. If we define the function

$${\rho }_{\mathrm{GV}}\left(y\right)\triangleq 2\mathrm{Cap}\left(\mathcal{S}\right)+\delta \left(y\right)logy-log\Lambda (y;\mathit{B}),$$ (6)

then the corresponding GV curve is given by

$$\mathcal{GV}\left(\mathcal{S}\right)=\left\{(\delta \left(y\right),{\rho }_{\mathrm{GV}}\left(y\right)):y\in [0,1]\right\}\cup \left\{(\delta ,0):\delta \ge {\delta }_{\max }\right\}.$$ (7)

Before we prove Theorem 1, we discuss its implications. It should be noted that to compute $\delta \left(y\right)$ and $\rho \left(y\right)$, it suffices to determine $\Lambda (y;\mathit{B})$ and $\Lambda \prime (y;\mathit{B})$ using the modified power iteration methods described in Appendix A. In other words, no Newton–Raphson iterations are required. We also have additional computational savings, as we do not need to apply the power iteration method to compute the second derivative ${\Lambda }^{\prime \prime }(y;\mathit{B})$.

Example 2.

We continue our example and plot the GV curve for the $(3,2)$-SWCC constrained system in Figure 1a. Before plotting, we observe that when $y=0$, we have $\left(\delta \right(0),\rho (0\left)\right)=(0,0.551)=(0,\mathrm{Cap}(\mathcal{S}\left)\right)$, as expected. When $y=1$, we have $\delta \left(1\right)={\delta }_{\max }=0.313$. Indeed, both $\rho \left(1\right)$ and $R_{\mathrm{GV}}\left({\delta }_{\max }\right)$ are equal to zero and we have that $R_{\mathrm{GV}}\left(\delta \right)=0$ for $\delta \ge {\delta }_{\max }$.

Next, we compute a set of 100 points on the GV curve. If we apply Procedure 1 to compute $R_{\mathrm{GV}}\left(\delta \right)$ for 100 values of δ in the interval $[0,{\delta }_{\max }]$, we require 275 Newton–Raphson iterations and 6900 power iterations to find these points. In contrast, applying Theorem 1, we compute $\left(\delta \right(y),\rho (y\left)\right)$ for 100 values of y in the interval $\left[0,1\right]$. This does not require any Newton–Raphson iterations and involves only 2530 power iterations.

To prove Theorem 1, we demonstrate the following lemmas. Our first lemma is immediate from the definitions of $R_{\mathrm{GV}}$, $\delta$, and $\rho$ in (1), (5), and (6), respectively.

Lemma 1.

$R_{\mathrm{GV}}\left(\delta \left(y\right)\right)=\rho \left(y\right)$ for all $y\in [0,1]$.

The next lemma studies the behaviour of both $\delta$ and $\rho$ as functions in y.

Lemma 2.

In terms of y, the functions $\delta \left(y\right)$ and $\rho \left(y\right)$ are monotone increasing and decreasing, respectively. Furthermore, we have that $\left(\delta \right(0),\rho (0\left)\right)=(0,\mathrm{Cap}(\mathcal{S}\left)\right)$, $(\delta \left(1\right),\rho \left(1\right))=({\delta }_{\max },0)$ and $R_{\mathrm{GV}}\left(\delta \right)=0$ for $\delta \ge {\delta }_{\max }$.

Proof. 

To simplify notation, we write $\Lambda (y;\mathit{B})$, ${\Lambda }^{\prime }(y;\mathit{B})$, and ${\Lambda }^{\prime \prime }(y;\mathit{B})$ as $\Lambda$, ${\Lambda }^{\prime }$, and ${\Lambda }^{\prime \prime }$, respectively.

First, we show that ${\delta }^{\prime }\left(y\right)$ is positive for $0\le y<1$. Differentiating the expression in (5), we have that ${\delta }^{\prime }\left(y\right)>0$ is equivalent to

$$\Lambda ({\Lambda }^{\prime }+y{\Lambda }^{\prime \prime })-y{\left({\Lambda }^{\prime }\right)}^2>0.$$ (8)

We recall that (3) is a convex minimization problem. Hence, the second order derivative of the objective function is always positive. In other words,

$$\frac{\delta }{y^2}+\frac{\Lambda \prime \prime \Lambda -{\left({\Lambda }^{\prime }\right)}^2}{{\Lambda }^2}>0.$$

Substituting $\delta$ with $y{\Lambda }^{\prime }/\Lambda$ and multiplying by $y{\Lambda }^2$, we obtain (8), as desired.

Next, we show that $\rho$ is monotone decreasing. We recall that $\rho \left(y\right)=R_{\mathrm{GV}}\left(\delta \left(y\right)\right)=\mathrm{Cap}\left(\mathcal{S}\right)-\stackrel{\sim }{T}\left(\delta \right)$. Since $\stackrel{\sim }{T}\left(\delta \right)$ yields the asymptotic rate of the total ball size, we have that as y increases, $\delta \left(y\right)$ increases and so, $\stackrel{\sim }{T}\left(\delta \right)$ increases. Therefore, $\rho \left(y\right)$ decreases, as desired.

Next, we show that $\rho \left(1\right)=0$. When $y=1$, we have from (6) that $\rho \left(1\right)=2\mathrm{Cap}\left(\mathcal{S}\right)-log\Lambda (1;\mathit{B})$. Now, we recall that ${\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)$ shares the same dominant eigenvalue as the matrix ${\mathit{T}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)$ [12]. Furthermore, it can be verified that when $y=1$, ${\mathit{T}}_{\mathcal{G}\times \mathcal{G}}\left(1\right)$ is tensor product of ${\mathit{A}}_{\mathcal{G}}$ and ${\mathit{A}}_{\mathcal{G}}$. That is, ${\mathit{T}}_{\mathcal{G}\times \mathcal{G}}\left(1\right)={\mathit{A}}_{\mathcal{G}}\otimes {\mathit{A}}_{\mathcal{G}}$. It then follows from standard linear algebra that $\Lambda (1;\mathit{B})=\Lambda (1;\mathit{T})=\Lambda {\left({\mathit{A}}_{\mathcal{G}}\right)}^2$. Thus, $log\Lambda (1;\mathit{B})=2\mathrm{Cap}\left(\mathcal{S}\right)$ and $\rho \left(1\right)=0$. In this instance, we also have that $\stackrel{\sim }{T}\left({\delta }_{\max }\right)=2\mathrm{Cap}\left(\mathcal{S}\right)$.

Finally, for $\delta \ge {\delta }_{\max }$, we have that $\stackrel{\sim }{T}\left({\delta }_{\max }\right)=2\mathrm{Cap}\left(\mathcal{S}\right)$ and thus, $R_{\mathrm{GV}}\left(\delta \right)=0$, as required.    □

Theorem 1 is then immediate from Lemmas 1 and 2.

We have the following corollary that immediately follows from Lemma 2. This corollary then implies that $y^{*}$ yields the global minimum for the optimization problem.

Corollary 1.

When $0\le \delta \le {\delta }_{max}=\frac{{\Lambda }^{\prime }(1,\mathit{B})}{\Lambda (1,\mathit{B})}$, $F\left(y\right)\triangleq y{\Lambda }^{\prime }(y;\mathit{B})-\delta \Lambda (y;\mathit{B})$ has a unique zero in $\left[0,1\right]$. Furthermore, $F^{\prime }\left(y\right)$ is strictly positive for all $y\in \left[0,1\right]$.

4. Evaluating Marcus and Roth’s Improvement of the Gilbert–Varshamov Bound

In [14], Marcus and Roth improved the GV lower bound for most constrained systems by considering subsets $\mathcal{S}\left(p\right)$ of $\mathcal{S}$ where p is some parameter. Here, we focus on the case $s=1$ and set p to be the normalized frequency of edges whose labels correspond to one. Specifically, we set $\mathcal{S}\left(p\right)\triangleq \{\mathit{x}\in \mathcal{S}:\mathrm{wt}\left(\mathit{x}\right)=⌊p\left|\mathit{x}\right|⌋\}$.

Next, let ${\mathcal{S}}_n\left(p\right)$ be the set of all words/paths of length n in $\mathcal{S}\left(p\right)$ and we define $S\left(p\right)\triangleq {lim\; sup}_{n\to \infty }\frac{1}{n}log\left|{\mathcal{S}}_n\left(p\right)\right|$.

Similar to before, we define $\stackrel{\sim }{T}(p,\delta )={lim\; sup}_{n\to \infty }\frac{1}{n}logT(⌊\delta n⌋,n;{\mathcal{S}}_n\left(p\right))$. Since ${\mathcal{S}}_n\left(p\right)$ is a subset of ${\mathcal{S}}_n$, it follows from the usual GV argument that there exists a family of $(n,\lfloor \delta n\rfloor ;\mathcal{S})$ codes whose rates approach $2S\left(p\right)-\stackrel{\sim }{T}(p,\delta )$ for all $0\le p\le 1$. Therefore, we have the following lower bound on asymptotic achievable code rates:

$$R_{\mathrm{MR}}\left(\delta \right)=sup\{2S\left(p\right)-\stackrel{\sim }{T}(p,\delta ):0\le p\le 1\}.$$ (9)

Now, a key result from [14] is that both $S\left(p\right)$ and $\stackrel{\sim }{T}(p,\delta )$ can be obtained via two different convex optimization problems. For succinctness, we state the dual formulations of these optimization problems.

First, $S\left(p\right)$ can be obtained from the following problem:

$$S\left(p\right)=inf\left\{-plogz+log\Lambda \left({\mathit{C}}_{\mathcal{G}}\left(z\right)\right):z\ge 0\right\}.$$ (10)

Here, ${\mathit{C}}_{\mathcal{G}}\left(z\right)$ is the following $\left(\right|\mathcal{V}|\times |\mathcal{V}\left|\right)$ matrix ${\mathit{C}}_{\mathcal{G}}\left(z\right)$ whose rows and columns are indexed by $\mathcal{V}$. For each entry indexed by e, we set ${\left({\mathit{C}}_{\mathcal{G}}\left(z\right)\right)}_e$ to be zero if $e\notin \mathcal{E}$, and $z^{\mathcal{L}\left(e\right)}$ if $e\in \mathcal{E}$.

As before, we simplify notation by writing $\Lambda (z;\mathit{C})\triangleq \Lambda \left({\mathit{C}}_{\mathcal{G}}\left(z\right)\right)$. Again, following the convexity of (10), we are interested in finding the zero of the following function:

$$G_1\left(z\right)\triangleq z{\Lambda }^{\prime }(z;\mathit{C})-p\Lambda (z;\mathit{C}).$$ (11)

Next, $\stackrel{\sim }{T}(p,\delta )$ can be obtained via the following optimization:

$$\stackrel{\sim }{T}(p,\delta )=inf\left\{-2plogx-\delta logy+log\Lambda \left({\mathit{D}}_{\mathcal{G}\times \mathcal{G}}(x,y)\right):x\ge 0,0\le y\le 1\right\}.$$ (12)

Here, ${\mathit{D}}_{\mathcal{G}\times \mathcal{G}}(x,y)$ is a $\left(\genfrac{}{}{0pt}{}{\left|\mathcal{V}\right|+1}{2}\right)\times \left(\genfrac{}{}{0pt}{}{\left|\mathcal{V}\right|+1}{2}\right)$-reduced distance matrix indexed by ${\mathcal{V}}^{\left(2\right)}$. To define the entry of matrix ${\mathit{D}}_{\mathcal{G}\times \mathcal{G}}(x,y)$ indexed by $((v_i,v_j),(v_k,v_{\ell }))$, we look at the vertices $v_i$, $v_j$, $v_k$, and $v_{\ell }$ and follow the rules given in Table 2.

Table 2.

We set the $\left((v_i,v_j),(v_k,v_{\ell })\right)$ entry of the matrix ${\mathit{D}}_{\mathcal{G}\times \mathcal{G}}(x,y)$ according to the subgraph induced by the states $v_i$,$v_j$,$v_k$, and $v_{\ell }$.

${\mathit{D}}_{\mathcal{G}\times \mathcal{G}}(\mathit{x},\mathit{y})$ at Entry $\left(({\mathit{v}}_{\mathit{i}},{\mathit{v}}_{\mathit{j}}),({\mathit{v}}_{\mathit{k}},{\mathit{v}}_{\mathit{\ell }})\right)$	Subgraph Induced by the States $\{{\mathit{v}}_{\mathit{i}},{\mathit{v}}_{\mathit{j}},{\mathit{v}}_{\mathit{k}},{\mathit{v}}_{\mathit{\ell }}\}$
0
1
$x^2$
$xy$
$2xy$

Again, we write $\Lambda (x,y;\mathit{D})\triangleq \Lambda \left({\mathit{D}}_{\mathcal{G}\times \mathcal{G}}(x,y)\right)$. Furthermore, following the convexity of (12), we have that if the optimal solution is obtained at x and y, then

$$\begin{array}{cc}\hfill G_2(x,y)& \triangleq x{\Lambda }_x(x,y;\mathit{D})-2p\Lambda (x,y;\mathit{D})=0.\hfill \end{array}$$ (13)

$$\begin{array}{cc}\hfill G_3(x,y)& \triangleq y{\Lambda }_y(x,y;\mathit{D})-\delta \Lambda (x,y;\mathit{D})=0.\hfill \end{array}$$ (14)

To this end, we consider the function $\Delta \left(x\right)={\Lambda }_y(x,1;\mathit{D})/\Lambda (x,1;\mathit{D})$ for $x>0$ and set ${\delta }_{max}=sup\{\Delta \left(x\right):x>0\}$. As with the previous section, we develop a numerical procedure to solve the optimization problem (9). To this end, we have the following critical observation.

Theorem 2.

For a given $\delta <{\delta }_{\max }$, consider the optimization problem

$$\begin{array}{cc}\hfill & sup\{-2plogz+2log\Lambda (z;\mathit{C})+2plogx+\delta logy-log\Lambda (x,y;\mathit{D}):\hfill \\ \hfill & G_1\left(z\right)=G_2(x,y)=G_3(x,y)=0\}.\hfill \end{array}$$

If $(p^{*},x^{*},y^{*},z^{*})$ is an optimal solution, then $x^{*}=z^{*}$. Furthermore, if $0\le p^{*}\le 1$, then $x^{*},z^{*}\ge 0$ and $0\le y^{*}\le 1$.

Proof. 

Let ${\lambda }_1,{\lambda }_2,$ and ${\lambda }_3$ be real-valued variables and we define $L(p,x,y,z,{\lambda }_1,{\lambda }_2,{\lambda }_3)\triangleq G(p,x,y,z)+{\lambda }_1{G}_1\left(z\right)+{\lambda }_2{G}_2(x,y)+{\lambda }_3{G}_3(x,y)$. Using the Lagrangian multiplier theorem, we have that $\partial L/\partial p=\partial L/\partial x=\partial L/\partial y=\partial L/\partial z=0$ for any optimal solution. Solving these equations with the constraints $G_1\left(z\right)=G_2(x,y)=G_3(x,y)=0$, we have that ${\lambda }_1={\lambda }_2={\lambda }_3=0$ and $x=z$ for any optimal solution.

Now, when $p^{*}\in [0,1]$, using $G_1\left(z\right)=0$, let us define $z\left(p\right)\triangleq z{\Lambda }^{\prime }(z;\mathit{C})/\Lambda (z;\mathit{C})$. Then, proceeding as with the proof of Lemma 2, we see that $z\left(p\right)$ is monotone increasing with $z\left(0\right)=0$. Therefore, $z^{*}=z\left(p^{*}\right)$ is zero.

Similarly, given $p^{*}$ and $x^{*}$, we use $G_3(x^{*},y)=0$ to define $\delta \left(y\right)=y{\Lambda }_y(x^{*},y;\mathit{D})/\Lambda (x^{*},y;\mathit{D})$. Again, we can proceed as with the proof of Lemma 2 to show that $\delta \left(y\right)$ is monotone increasing. Furthermore, since $\delta \left(y^{*}\right)<{\delta }_{\max }=\delta \left(1\right)$, we have that $y^{*}\in [0,1]$.    □

Therefore, to determine $R_{\mathrm{MR}}\left(\delta \right)$ for any fixed $\delta$, it suffices to find x, y, z, and p such that $G_1\left(z\right)=G_2(x,y)=G_3(x,y)=0$ and $x=z$.

Now, the optimization in Theorem 2 does not constrain the values of p. Furthermore, for certain constrained systems, there are instances where p falls outside the interval $[0,1]$. In this case, instead of solving the optimization problem (9), we set p to be either zero or one, and we solve the corresponding optimization problems (10) and (12). Specifically, if we have $p^{*}<0$, then we set $p^{*}=0$ and $x^{*}=0$, or if $p^{*}>1$, then we set $p^{*}=1$ and $x^{*}=\infty$. Hence, the resulting rates that we obtain are a lower bound for the GV-MR bound.  

Procedure 2 ($R_{\mathrm{MR}}\left(\delta \right)$ for fixed $\delta \le {\delta }_{\max }$).

Input: Matrices ${\mathit{C}}_{\mathcal{G}}\left(x\right)$, ${\mathit{D}}_{\mathcal{G}}(x,y)$

Output: $R_{\mathrm{MR}}\left(\delta \right)$ or $R_{\mathrm{LB}}\left(\delta \right)$, where $R_{\mathrm{MR}}\left(\delta \right)\ge R_{\mathrm{LB}}\left(\delta \right)$.

• (1)
  Apply the Newton–Raphson method to obtain $p^{*},x^{*},$ and $y^{*}$ such that $G_1\left(x^{*}\right)$, $G_2(x^{*},y^{*})$, and $G_3(x^{*},y^{*})$ are approximately zero. Specifically, do the following:

  • Fix a tolerance value $ϵ$
  • Set $t=0$ and pick an initial guess $p_t\ge 0$, $x_t\ge 0$, $0\le y_t\le 1$.
  • While $|p_t-p_{t-1}|+|x_t-x_{t-1}|+|y_t-y_{t-1}|>ϵ$ ,
    • –
      Compute the next guess $p_{t+1},x_{t+1},y_{t+1}$:

      $$\left[\begin{array}{c}{p}_{t+1}\\ x_{t+1}\\ y_{t+1}\end{array}\right]=\left[\begin{array}{c}{p}_t\\ x_t\\ y_t\end{array}\right]-{\left[\begin{array}{ccc}\frac{\partial G_1}{\partial p}& \frac{\partial G_1}{\partial x}& \frac{\partial G_1}{\partial y}\\ \frac{\partial G_2}{\partial p}& \frac{\partial G_2}{\partial x}& \frac{\partial G_2}{\partial y}\\ \frac{\partial G_3}{\partial p}& \frac{\partial G_3}{\partial x}& \frac{\partial G_3}{\partial y}\end{array}\right]}^{-1}\left[\begin{array}{c}{G}_1\left(x_t\right)\\ G_2(x_t,y_t)\\ G_3(x_t,y_t)\end{array}\right].$$
    • –
      Here, apply the power iteration method to compute $\Lambda (x_t;\mathit{C})$, ${\Lambda }^{\prime }(x_t;\mathit{C})$, ${\Lambda }^{\prime \prime }(x_t;\mathit{C})$, $\Lambda (x_t,y_t;\mathit{D})$, ${\Lambda }_x(x_t,y_t;\mathit{D})$, ${\Lambda }_y(x_t,y_t;\mathit{D})$, ${\Lambda }_{xx}(x_t,y_t;\mathit{D})$, ${\Lambda }_{yy}(x_t,y_t;\mathit{D})$, and ${\Lambda }_{xy}(x_t,y_t;\mathit{D})$.
    • –
      Increment t by one.
  • Set $p^{*}\leftarrow p_t$, $x^{*}\leftarrow x_t$, $y^{*}\leftarrow y_t$.
• (2A)If $0\le p^{*}\le 1$, set $R_{\mathrm{MR}}\left(\delta \right)\leftarrow 2log\Lambda (x^{*};\mathit{C})+\delta log{y}^{*}-log\Lambda (x^{*},y^{*};\mathit{D})$.
• (2B)Otherwise,

  • If $p^{*}<0$, set $p^{*}\leftarrow 0$, $x^{*}\leftarrow 0$, and $y^{*}\leftarrow \mathrm{solution}\mathrm{of}{G}_3(0,y)=0$.

  • If $p^{*}>1$, set $p^{*}\leftarrow 1$, $x^{*}\leftarrow \infty$, and $y^{*}\leftarrow \mathrm{solution}\mathrm{of}{G}_3(\infty ,y)=0$.

  Finally, set $R_{\mathrm{LB}}\left(\delta \right)\leftarrow 2log\Lambda (x^{*};\mathit{C})+\delta log{y}^{*}-log\Lambda (x^{*},y^{*};\mathit{D})$.

Remark 1.

Let $p^{*}$ be the value computed at Step 1. When $p^{*}$ falls outside the interval $[0,1]$, we set $p^{*}\in \{0,1\}$, and we argued earlier that the value returned $R_{\mathrm{LB}}\left(\delta \right)$ (at Step 2B) is, at most, $R_{\mathrm{MR}}\left(\delta \right)$. Nevertheless, we conjecture that $R_{\mathrm{LB}}\left(\delta \right)=R_{\mathrm{MR}}\left(\delta \right)$.

As before, we develop a plotting procedure that minimizes the use of Newton–Raphson iterations.

We note that we have three scenarios for $\Delta \left(x\right)$. If $\Delta \left(x\right)$ is monotone decreasing, then ${\delta }_{max}={lim}_{x\to 0}\Delta \left(x\right)$ and we set $x^{\#}=0$. If $\Delta \left(x\right)$ is monotone increasing, then ${\delta }_{max}={lim}_{x\to \infty }\Delta \left(x\right)$ and we set $x^{\#}=\infty$. Otherwise, $\Delta \left(x\right)$ is maximized for some positive value and we set $x^{\#}$ to be this value. Next, to obtain the GV-MR curve (see Remark 2); we iterate over $x\in \left[1,x^{\#}\right]$. It should be noted that if $y\left(x^{\#}\right)<1$ or, equivalently, $\delta \left(x^{\#}\right)<{\delta }_{\max }$, we obtain a lower bound on the GV-MR curve by iterating over $y\in \left[y\left(x^{\#}\right),1\right]$. Similar to Theorem 1, we define

$${\rho }_{\mathrm{MR}}\left(x\right)\triangleq 2log\Lambda (x;\mathit{C})+\delta \left(x\right)logy\left(x\right)-log\Lambda (x,y\left(x\right);\mathit{D}),$$ (15)

and

$${\rho }_{\mathrm{LB}}\left(y\right)\triangleq 2log\Lambda (x^{\#};\mathit{C})+\delta \left(y\right)logy-log\Lambda (x^{\#},y;\mathit{D}).$$ (16)

Finally, we state the following analogue of Theorem 1.

Theorem 3.

We define ${\delta }_{max}$, $x^{\#}$ as before. For $x\in \left[1,x^{\#}\right]$, we set

$$\begin{array}{cc}\hfill p\left(x\right)& \leftarrow x{\Lambda }^{\prime }(x;\mathit{C})/\Lambda (x;\mathit{C}),\hfill \\ \hfill y\left(x\right)& \leftarrow solutiono\mathrm{of}{G}_2(x,y)=0,\hfill \\ \hfill \delta \left(x\right)& \leftarrow y\left(x\right){\Lambda }_y(x,y\left(x\right);\mathit{D})/\Lambda (x,y\left(x\right);\mathit{D}),\hfill \end{array}$$

If $y\left(x^{\#}\right)<1$, then for $y\in \left[y\left(x^{\#}\right),1\right]$, we set

$$\begin{array}{cc}\hfill \delta \left(y\right)& \leftarrow y{\Lambda }_y(x^{\#},y;\mathit{D})/\Lambda (x^{\#},y;\mathit{D}),\hfill \end{array}$$

then, the corresponding GV-MR curve is given by

$$\left\{(\delta \left(x\right),{\rho }_{\mathrm{MR}}\left(x\right)):x\in \left[1,x^{\#}\right]\right\}\cup \{(\delta \left(y\right),{\rho }_{\mathrm{LB}}\left(y\right)):y\in \left[y\left(x^{\#}\right),1\right]\}\cup \left\{(\delta ,0):\delta \ge {\delta }_{\max }\right\}.$$ (17)

where ${\rho }_{\mathrm{MR}}$ and ${\rho }_{\mathrm{LB}}$ are defined in (15) and (16), respectively.

Example 3.

We continue our example and evaluate the GV-MR bound for the $(3,2)$-SWCC constrained system. In this case, the matrices of interest are

$${\mathit{C}}_{\mathcal{G}}\left(z\right)=\left[\begin{array}{ccc}z& 1& 0\\ 0& 0& z\\ z& 0& 0\end{array}\right]and{\mathit{D}}_{\mathcal{G}\times \mathcal{G}}(x,y)=\left[\begin{array}{cccccc}{x}^2& 2xy& 0& 1& 0& 0\\ 0& 0& x^2& 0& xy& 0\\ x^2& xy& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& x^2\\ 0& 0& x^2& 0& 0& 0\\ x^2& 0& 0& 0& 0& 0\end{array}\right].$$

Here, we observe that $\Delta \left(x\right)$ is a monotone decreasing function and so, we set $x^{\#}=0.01$ and ${\delta }_{max}={lim}_{x\to 0}\Delta \left(x\right)\approx 0.426$. If we apply Procedure 2 to compute $R_{\mathrm{MR}}\left(\delta \right)$ for 100 points in $\left[0,{\delta }_{max}\right]$, we require 437 Newton–Raphson iterations and 85,500 power iterations. In contrast, we use Theorem 3 to compute $(\delta \left(x\right),{\rho }_{\mathrm{MR}}\left(x\right))$ for 100 values of x in the interval $\left[1,x^{\#}\right]$. This requires 323 Newton–Raphson iterations and involves 22,296 power iterations. The resulting GV-MR curve is given in Figure 1a.

Remark 2.

Strictly speaking, the GV-MR curve described by (17) may not be equal to the curve defined by the optimization problem (15). Nevertheless, the curve provides a lower bound for the optimal asymptotic code rates and we conjecture that the GV-MR curve described by (17) is a lower bound for the curve defined by the optimization problem (15).

5. Single-State Graph Presentation

In this section, we focus on graph presentations that have exactly one state. Here, we allow these single-state graph presentations to contain the parallel edges and their labels to be binary strings of length possibly greater than one. Now, for these constrained systems, the procedures to evaluate the GV bound and its MR improvements can be greatly simplified. This is because the matrices ${\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)$, ${\mathit{C}}_{\mathcal{G}}\left(z\right)$, and ${\mathit{D}}_{\mathcal{G}\times \mathcal{G}}(x,y)$ are all of dimensions one by one. Therefore, determining their respective dominant eigenvalues is straightforward and does not require the power iteration method. The results in this section follow directly from previous sections and our objective is to provide explicit formulas whenever possible.

Formally, let $\mathcal{S}$ be the constrained system with graph presentation $\mathcal{G}=(\mathcal{V},\mathcal{E},\mathcal{L})$ such that $\left|V\right|=1$ and $\mathcal{L}:\mathcal{E}\to {\Sigma }^s$ with $s\ge 1$ (existing methods that determine the GV bound for constrained systems with $\left|V\right|\ge 1$ assume that the edge-labels have single letters, i.e., $s=1$. In other words, previous methods developed in [12,14] do not apply).

We further define ${\alpha }_t\triangleq \#\{(\mathit{x},\mathit{y})\in \mathcal{L}{\left(\mathcal{E}\right)}^2:d_{\mathit{H}}(\mathit{x},\mathit{y})=t\}$ for $0\le t\le s$. Then. the corresponding adjacency and reduced distance matrices are as follows:

$${\mathit{A}}_{\mathcal{G}}=\left[\begin{array}{c}\left|\mathcal{E}\right|\end{array}\right]\mathrm{and}{\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)=\left[\begin{array}{c}{\displaystyle \sum _{t\ge 0}}{\alpha }_t{y}^t\end{array}\right].$$

Then, we compute the capacity using its definition as $\mathrm{Cap}\left(\mathcal{S}\right)=(log|\mathcal{E}\left|\right)/s$.

To compute $\stackrel{\sim }{T}\left(\delta \right)$, we consider the following extension of the optimization problem (3) for the case $s\ge 1$:

$$\begin{array}{cc}\hfill \stackrel{\sim }{T}\left(\delta \right)& =\frac{1}{s}inf\left\{-\delta slogy+log\lambda (y;\mathit{B}):0\le y\le 1\right\}\hfill \\ \hfill & =\frac{1}{s}inf\left\{-\delta slogy+log\left({\displaystyle \sum _{t\ge 0}}{\alpha }_t{y}^t\right):0\le y\le 1\right\}.\hfill \end{array}$$ (18)

As before, following the convexity of the objective function in (18), we have that the optimal y is the zero (in the interval $[0,1]$) of the function

$$F\left(y\right)\triangleq {\displaystyle \sum _{t\ge 0}}(t-\delta s){\alpha }_t{y}^t.$$ (19)

So, for fixed values of $\delta$, we can use the Newton–Raphson procedure to compute the root y of (19), and, hence, evaluate $R_{\mathrm{GV}}\left(\delta \right)$. It should be noted that the power iteration method is not required in this case.

On the other hand, to plot the GV curve, we have the following corollary of Theorem 1.

Corollary 2.

Let $\mathcal{G}$ be the single-state graph presentation for a constrained system $\mathcal{S}$. Then, the corresponding GV curve is given by

$$\mathcal{GV}\left(\mathcal{S}\right)\triangleq \left\{(\delta ,R_{\mathrm{GV}}\left(\delta \right)):\delta \in [0,1]\right\}=\left\{\left(\delta \right(y),\rho (y\left)\right):y\in [0,1]\right\}\cup \left\{(\delta ,0):\delta \ge {\delta }_{\max }\right\},$$ (20)

where

$$\begin{array}{cc}\hfill {\delta }_{\max }& =\frac{{\displaystyle {\sum }_{t\ge 0}}t{\alpha }_t}{{s\left|\mathcal{E}\right|}^2},\hfill \\ \hfill \delta \left(y\right)& =\frac{{\displaystyle {\sum }_{t\ge 0}}t{\alpha }_t{y}^t}{s\left({\displaystyle {\sum }_{t\ge 0}}{\alpha }_t{y}^t\right)},\hfill \\ \hfill \rho \left(y\right)& =\frac{1}{s}\left(log\frac{{\left|\mathcal{E}\right|}^2}{{\displaystyle {\sum }_{t\ge 0}}{\alpha }_t{y}^t}-\frac{{\displaystyle {\sum }_{t\ge 0}}t{\alpha }_t{y}^t}{{\displaystyle {\sum }_{t\ge 0}}{\alpha }_t{y}^t}logy\right).\hfill \end{array}$$

We illustrate this evaluation procedure via an example of the class of subblock energy-constrained codes (SECCs). Formally, we fix a subblock length L and energy constraintw. A binary word x of length $mL$ is said to satisfy the $(L,w)$-subblock energy constraint if we partition x into m subblocks of length L, then the number of ones in every subblock is at least w. We refer to the collection of words that meet this constraint as an $(L,w)$-SECC constrained system. The class of SECCs was introduced by Tandon et al. for the application of simultaneous energy and information transfer [7]. Later, in [21], a GV-type bound was introduced (see Proposition 12 in [21] and also, (28)) and we make comparisons with the GV bound (20) in the following example.

Example 4.

Let $L=3$ and $w=2$ and we consider a $(3,2)$-SECC constrained system. It is straightforward to observe that the graph presentation is as follows with the single state x. Here, $s=L=3$.

Then, the corresponding adjacency and reduced distance matrices are as follows:

$${\mathit{A}}_{\mathcal{G}}=\left[\begin{array}{c}4\end{array}\right],{\mathit{B}}_{\mathcal{G}\times \mathcal{G}}\left(y\right)=\left[\begin{array}{c}4+6y+6y^2\end{array}\right].$$

First, we determine the GV bound at $\delta =1/3$. We observe that $F\left(y\right)=-4+6y^2$ and, so, the optimal point y for (18) is $\sqrt{2/3}$ (the unique solution to $F\left(y\right)$ in the interval $[0,1]$). Hence, we have that $\stackrel{\sim }{T}(1/3)\approx 1.327$. On the other hand, the capacity of a $(3,2)$-SECC constrained system is $\mathrm{Cap}\left(\mathcal{S}\right)=2/3$. Therefore, the GV bound is given by $R_{\mathrm{GV}}(1/3)=0.006$.

In contrast, the GV-type lower bound given by Proposition 12 in [21] is zero for $\delta >0.174$. Hence, the evaluation of the GV bound yields a significantly better lower bound. In fact, we can show that $R_{\mathrm{GV}}\left(\delta \right)>0$ for all $\delta \le {\delta }_{\max }=3/8$.

To plot the GV curve, using the fact that ${\delta }_{\max }=3/8$, we have that

$$\mathcal{GV}\left(\mathcal{S}\right)=\left\{\left(\frac{y+2y^2}{2+3y+3y^2},\frac{1}{3}log\frac{8}{2+3y+3y^2}+\frac{3y+6y^2}{2+3y+3y^2}logy\right):y\in [0,1]\right\}\cup \left\{(\delta ,0):\delta \ge \frac{3}{8}\right\}.$$

We plot the curve in Section 6.

From this example, we see that our methods yield better lower bounds in terms of asymptotic coding rates for a specific pair of $(L,w)$. It is open to determine how much improvement can be achieved for general pairs of L and w.

Next, we evaluate the GV-MR bound. To this end, we consider some proper subset $\mathcal{P}\subset \mathcal{E}$ and define

$$\begin{array}{cc}\hfill {\alpha }_t& \triangleq \#\{(\mathit{x},\mathit{y})\in \mathcal{L}{\left(\mathcal{E}\right)}^2:d_{\mathit{H}}(\mathit{x},\mathit{y})=t,\mathit{x},\mathit{y}\in \mathcal{P}\},\hfill \\ \hfill {\beta }_t& \triangleq \#\{(\mathit{x},\mathit{y})\in \mathcal{L}\left(\mathcal{E}\right):d_{\mathit{H}}(\mathit{x},\mathit{y})=t,(\mathit{x}\in \mathcal{P},\mathit{y}\notin \mathcal{P})\mathrm{or}(\mathit{x}\notin \mathcal{P},\mathit{y}\in \mathcal{P})\},\hfill \\ \hfill {\gamma }_t& \triangleq \#\{(\mathit{x},\mathit{y})\in \mathcal{L}\left(\mathcal{E}\right):d_{\mathit{H}}(\mathit{x},\mathit{y})=t,\mathit{x},\mathit{y}\notin \mathcal{P}\}.\hfill \end{array}$$

Then, we consider the following matrices:

$${\mathit{C}}_{\mathcal{G}}\left(z\right)=\left[\begin{array}{c}\left|\mathcal{E}\right|-\left|\mathcal{P}\right|+\left|\mathcal{P}\right|z\end{array}\right]\mathrm{and}{\mathit{D}}_{\mathcal{G}\times \mathcal{G}}(x,y)=\left[\begin{array}{c}{\displaystyle \sum _{t\ge 0}}({\alpha }_t{x}^2+{\beta }_{t}x+{\gamma }_t)y^t\end{array}\right].$$

Setting p to be the normalized frequency of edges in $\mathcal{P}$, we obtain $S\left(p\right)$ by solving the optimization problem (10).

Specifically, we have that

$$S\left(p\right)=\frac{1}{s}\left(H\left(p\right)+p+log\left|\mathcal{P}\right|+(1-p)log\left(\right|\mathcal{E}|-|\mathcal{P}\left|\right)\right),$$ (21)

and this value is achieved when

$$z=\frac{p\left(\right|\mathcal{E}|-|\mathcal{P}\left|\right)}{(1-p)\left|\mathcal{P}\right|}.$$ (22)

To compute $\stackrel{\sim }{T}(p,\delta )$, we consider the following extension of the optimization problem (12) for the case $s\ge 1$.

$$\begin{array}{cc}\hfill \stackrel{\sim }{T}(p,\delta )& =\frac{1}{s}inf\left\{-2plogx-\delta slogy+log\lambda (y;\mathit{D}):0\le y\le 1\right\}\hfill \\ \hfill & =\frac{1}{s}inf\left\{-2plogx-\delta slogy+log\left({\displaystyle \sum _{t\ge 0}}({\alpha }_t{x}^2+{\beta }_{t}x+{\gamma }_t)y^t\right):0\le y\le 1\right\}.\hfill \end{array}$$ (23)

As before, following the convexity of the objective function in (23), we have that the optimal x and y are the zeroes (in the interval $[0,1]$) of the functions

$$\begin{array}{cc}\hfill G_2(x,y)\triangleq & 2(1-p)\left({\displaystyle \sum _{t\ge 0}}{\alpha }_t{y}^t\right)x^2+(1-2p)\left({\displaystyle \sum _{t\ge 0}}{\beta }_t{y}^t\right)x-2p\left({\displaystyle \sum _{t\ge 0}}{\gamma }_t{y}^t\right)\hfill \\ \hfill G_3(x,y)\triangleq & {\displaystyle \sum _{t\ge 0}}(t-\delta s)({\alpha }_t{x}^2+{\beta }_{t}x+{\gamma }_t)y^t\hfill \end{array}$$ (24)

So, for fixed values of p and $\delta$, we can use the Newton–Raphson procedure to compute the roots x and y of (24), and, hence, evaluate $R_{\mathrm{GV}}(p,\delta )$. It should be noted that the power iteration method is not required in this case. We find $x^{\#}$ as defined in Section 4 and set

$${\rho }_{\mathrm{MR}}\left(x\right)\triangleq 2log\left(\right|\mathcal{E}|-\left|\mathcal{P}\right|+\left|\mathcal{P}\right|x)+\delta \left(x\right)logy\left(x\right)-log{\displaystyle \sum _{t\ge 0}}({\alpha }_t{x}^2+{\beta }_{t}x+{\gamma }_t)y{\left(x\right)}^t.$$ (25)

Furthermore, if $y\left(x^{\#}\right)<1$, we set

$${\rho }_{\mathrm{LB}}\left(y\right)\triangleq 2log\left(\right|\mathcal{E}|-\left|\mathcal{P}\right|+\left|\mathcal{P}\right|x^{\#})+\delta \left(y\right)logy-log{\displaystyle \sum _{t\ge 0}}({\alpha }_t{\left(x^{\#}\right)}^2+{\beta }_t{x}^{\#}+{\gamma }_t)y^t.$$ (26)

Next, to plot the GV-MR curve, we have the following corollary of Theorem 3.

Corollary 3.

Let $\mathcal{G}$ be the single-state graph presentation for a constrained system $\mathcal{S}$. For $x\in \left[1,x^{\#}\right]$, we set

$$\begin{array}{cc}\hfill p\left(x\right)& =\frac{\left|\mathcal{P}\right|x}{\left(\right|\mathcal{E}|-|\mathcal{P}\left|\right)+\left|\mathcal{P}\right|x)},\hfill \\ \hfill \delta \left(x\right)& =\frac{{\displaystyle {\sum }_{t\ge 1}}t({\alpha }_t{x}^2+{\beta }_{t}x+{\gamma }_t)y{\left(x\right)}^t}{s{\displaystyle {\sum }_{t\ge 0}}({\alpha }_t{x}^2+{\beta }_{t}x+{\gamma }_t)y{\left(x\right)}^t},\hfill \end{array}$$

where $y\left(x\right)$ is the smallest root of the equation

$$\begin{array}{cc}\hfill 2\left(\right|\mathcal{E}|-\left|\mathcal{P}\right|)\left({\displaystyle \sum _{t\ge 0}}{\alpha }_t{y}^t\right)x+\left(\right|\mathcal{E}|-\left|\mathcal{P}\right|-\left|\mathcal{P}\right|x)\left({\displaystyle \sum _{t\ge 0}}{\beta }_t{y}^t\right)-2\left|\mathcal{P}\right|\left({\displaystyle \sum _{t\ge 0}}{\gamma }_t{y}^t\right)& =0.\hfill \end{array}$$

If $y\left(x^{\#}\right)<1$, then for $y\in \left[y\left(x^{\#}\right),1\right]$, we set

$$\begin{array}{cc}\hfill \delta \left(y\right)& =\frac{{\displaystyle {\sum }_{t\ge 1}}t({\alpha }_t{\left(x^{\#}\right)}^2+{\beta }_t{x}^{\#}+{\gamma }_t)y^t}{s{\displaystyle {\sum }_{t\ge 0}}({\alpha }_t{\left(x^{\#}\right)}^2+{\beta }_t{x}^{\#}+{\gamma }_t)y^t},\hfill \end{array}$$

Then, the corresponding GV-MR curve is given by

$$\left\{(\delta \left(x\right),{\rho }_{\mathrm{MR}}\left(x\right)):x\in \left[1,x^{\#}\right]\right\}\cup \{(\delta \left(y\right),{\rho }_{\mathrm{LB}}\left(y\right)):y\in \left[y\left(x^{\#}\right),1\right]\}\cup \left\{(\delta ,0):\delta \ge {\delta }_{\max }\right\}.$$ (27)

where ${\rho }_{\mathrm{MR}}$ and ${\rho }_{\mathrm{LB}}$ are defined in (25) and (26), respectively.

Example 5.

We continue our example and evaluate the GV-MR bound for the $(3,2)$-SECC constrained system. We have the following single-state graph presentation:

Then, the matrices of interest are:

$${\mathit{C}}_{\mathcal{G}}=\left[\begin{array}{c}1+3z\end{array}\right],{\mathit{D}}_{\mathcal{G}\times \mathcal{G}}(x,y)=\left[\begin{array}{c}(3+6y^2)x^2+6xy+1\end{array}\right].$$

Since ${\mathit{C}}_{\mathcal{G}}$ and ${\mathit{D}}_{\mathcal{G}\times \mathcal{G}}(x,y)$ are both singleton matrices, we have $\Lambda (z;\mathit{C})=1+3z$ and $\Lambda (x,y;\mathit{D})=(3+6y^2)x^2+6xy+1$. Then, $G_1\left(z\right)=-p(1+3z)+3z$, $G_2(x,y)=3(1+2y^2)x^2(1-p)+3xy(1-2p)-p$ and $G_3(x,y)=4x^2{y}^2-3\delta (1+2y^2)x^2+2xy(1-3\delta )-\delta$. Now, we apply Theorem 2 and express $p,y,$ and δ in terms of x where $x\in [1,x^{\#}]$ where $x^{\#}\to \infty$.

$$\begin{array}{cc}\hfill p& =\frac{3x}{(1+3x)}\hfill \\ \hfill y& =\frac{x-1}{2x}\hfill \\ \hfill \delta & =\frac{2x(x-1)}{(9x^2-1)}\hfill \end{array}$$

Now, we observe that we have $y\left(x^{\#}\right)=1/2$. Since we can still increase y to 1, we apply the GV bound with $p=1$ and $x=z=x^{\#}$ once we reach the boundary that is $p=1$. Hence, at the boundary, we solve the following problem:

$$\begin{array}{cc}\hfill S\left(1\right)& =2log3\hfill \\ \hfill \stackrel{\sim }{T}(1,\delta )& =inf\left\{-2logx-3\delta logy+log(3(1+2y^2)x^2+6xy+1):1/2\le y\le 1;x=x^{\#}\to \infty \right\}\hfill \\ & =inf\left\{-3\delta logy+log3+log(1+2y^2):1/2\le y\le 1\right\}\hfill \\ \hfill R_{\mathrm{MR}}\left(\delta \right)& =S\left(1\right)-\stackrel{\sim }{T}(1,\delta ).\hfill \end{array}$$

By setting $F\left(y\right)=-3\delta (1+2y^2)+4y^2=0$, we get $\delta =4y^2/3(1+2y^2)$ where $y\in [1/2,1]$ and we plot the respective curve.

6. Numerical Plots

In this section, we apply our numerical procedures to compute the GV and the GV-MR bounds for some specific constrained systems. In particular, we consider the $(L,w)$-SWCC constrained systems defined in Section 3, the ubiquitous $(d,k)$-runlength limited systems (see, for example, p. 3 in [11]) and the $(L,w)$-subblock energy constrained codes recently introduced in [7]. In addition to the GV and GV-MR curves, we also plot a simple lower bound. For each $\delta \in \left[0,1/2\right]$, any ball size is at most $2\mathbb{H}\left(\delta n\right)$. So, for any constrained system $\mathcal{S}$, we have that $\tilde{T}\left(\delta \right)\le \mathrm{Cap}\left(\mathcal{S}\right)+\mathbb{H}\left(\delta \right)$. Therefore, we have that

$$R(\delta ;\mathcal{S})\le \mathrm{Cap}\left(\mathcal{S}\right)-\mathbb{H}\left(\delta \right).$$ (28)

From the plots in Figure 1, Figure 2 and Figure 3, it is also clear that the computations of (7) and (17) yield a significantly better lower bound.

6.1. $(L,w)$-Sliding Window Constrained Codes

We fix L and w. We recall from Section 3 that a binary word satisfies the $(L,w)$-sliding window weight constraint if the number of ones in every consecutive L bits is at least w and the $(L,w)$-SWCC constrained system refers to the collection of words that meet this constraint. From [8,9], we have a simple graph presentation that uses only $\left(\genfrac{}{}{0pt}{}{L}{w}\right)$ states. To validate our methods, we choose $(L,w)\in \left\{\right(3,2),(10,7\left)\right\}$ and the corresponding graph presentations have 3 and 120 states, respectively. Applying the plotting procedures described in Theorems 1 and 3, we obtain Figure 1.

6.2. $(d,k)$-Runlength Limited Codes

Next, we revisit the ubiquitous runlength constraint. We fix d and k. We say that a binary word satisfies the $(d,k)$-RLL constraint if each run of zeroes in the word has a length of at least d and at most k. Here, we allow the first and last runs of zeroes to have a length of less than d. We refer to the collection of words that meet this constraint as a $(d,k)$-RLL constrained system. It is well known that a $(d,k)$-RLL constrained system has the graph presentation with $k+1$ states (see, for example, [11]). Here, we choose $(d,k)\in \left\{\right(1,3),(3,7\left)\right\}$ to validate our methods and apply Theorems 1 and 3 to obtain Figure 2. For $(d,k)=(3,7)$, we corroborate our results with those derived in [15]. Specifically, Winick and Yang determined the GV bound (1) for the $(3,7)$-RLL constraint and remarked that the “evaluation of the (GV-MR) bound required considerable computation” for “a small improvement”. In Table 3, we verify this statement.

Table 3.

Comparison of the GV-MR bound with lower bound [15] for $(3,7)$-RLL constrained systems.

$\mathit{\delta }$	GV-MR Bound (15)	GV Bound [15] (see Equation (1))
0	0.406	0.406
0.05	0.255	0.225
0.1	0.163	0.163
0.15	0.095	0.094
0.2	0.048	0.044
0.25	0.018	0.012

6.3. $(L,w)$-Subblock Energy-Constrained Codes

We fix L and w. We recall from Section 5 that a binary word satisfies the $(L,w)$-subblock energy constraint if each subblock of length L has a weight of at least w and the $(L,w)$-SECC constrained system refers to the collection of words that meet this constraint. Then, the corresponding graph presentation has a single state $\mathtt{x}$ with ${\displaystyle {\sum }_{i=0}^w}\left(\genfrac{}{}{0pt}{}{L}{i}\right)$ edges, where each edge is labeled by a word of length L and weight at least w. We apply the methods in Section 5 to determine the GV and GV-MR bounds.

For the GV bound, we provide the explicit formula for ${\alpha }_t$ and proceed as in Example 4.

$${\alpha }_t=\left(\genfrac{}{}{0pt}{}{L}{t}\right)(\left|\mathcal{E}\right|-{\displaystyle \sum _{j=1}^t}{\displaystyle \sum _{k=0}^{\lceil \frac{j}{2}\rceil -1}}\left(\genfrac{}{}{0pt}{}{L-t}{w-j+k}\right)\left(\genfrac{}{}{0pt}{}{t}{k}\right))$$ (29)

Similarly, for GV-MR bound, we provide the explicit formula for ${\alpha }_t$, ${\beta }_t$, and ${\gamma }_t$ and proceed as in Example 5.

$$\begin{array}{cc}\hfill {\alpha }_t& =\left(\genfrac{}{}{0pt}{}{L}{w}\right)\left(\genfrac{}{}{0pt}{}{L-w}{i/2}\right)\left(\genfrac{}{}{0pt}{}{w}{i/2}\right)\mathrm{if}t\mathrm{is}\mathrm{even},\mathrm{otherwise},{\alpha }_t=0.\hfill \end{array}$$ (30)

$$\begin{array}{cc}\hfill {\beta }_t& =2\left(\genfrac{}{}{0pt}{}{L}{w}\right){\displaystyle \sum _{j=1}^{\lfloor \frac{t}{2}\rfloor }}\left(\genfrac{}{}{0pt}{}{L-w}{t-j}\right)\left(\genfrac{}{}{0pt}{}{w}{j}\right)-2{\alpha }_t\hfill \end{array}$$ (31)

$$\begin{array}{cc}\hfill {\gamma }_t& =\left(\genfrac{}{}{0pt}{}{L}{t}\right)\left(\right|\mathcal{E}|-{\displaystyle \sum _{j=1}^t}{\displaystyle \sum _{k=0}^{\lceil \frac{j}{2}\rceil -1}}\left(\genfrac{}{}{0pt}{}{L-t}{w-j+k}\right)\left(\genfrac{}{}{0pt}{}{t}{k}\right))-{\alpha }_t-{\beta }_t\hfill \end{array}$$ (32)

In Figure 3, we plot the GV bound and GV-MR bounds. We remark that the simple lower bound (28) corresponds to Proposition 12 in [21].

Appendix A. Power Iteration Method for Derivatives of Dominant Eigenvalues

Throughout this appendix, we assume that A is a diagonalizable matrix with dominant eigenvalue ${\lambda }_1$ and whose corresponding eigenspace has dimension one. Let ${\mathit{e}}_1$ be the unit eigenvector whose entries are positive in this space. Then, the power iteration method is a well-known numerical procedure that finds the dominant eigenvalue ${\lambda }_1$ and the corresponding eigenvector ${\mathit{e}}_1$ efficiently.

Now, in the preceding sections, the entries in the matrix A are given functions in either one or two variables and, thus, the dominant eigenvalue ${\lambda }_1$ is a function in the same variables. Moreover, the numerical procedures in these sections require us to compute the higher order (partial) derivatives of this dominant eigenvalue function ${\lambda }_1$. To the best of our knowledge, we are unaware of any algorithms or numerical procedures that estimate the values of these derivatives. Hence, in this appendix, we modify the power iteration method to compute these estimates.

Formally, let A be an irreducible nonnegative diagonalizable square matrix with dominant eigenvalue ${\lambda }_1$ and corresponding unit eigenvector ${\mathit{e}}_1$. Since A is diagonalizable, A has n eigenvectors ${\mathit{e}}_1,{\mathit{e}}_2,\dots ,{\mathit{e}}_n$ that form an orthonormal basis for ${\mathbb{R}}^n$. Let ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_n$ be the corresponding eigenvalues and, so, we have that

$$\mathit{A}{\mathit{e}}_i={\lambda }_i{\mathit{e}}_i\mathrm{for}\mathrm{all}i=1,2,...,n.$$ (A1)

Since A is irreducible, the dominant eigenspace has dimension one and, also, the dominant eigenvalue is real and positive. Therefore, we can assume that ${\lambda }_1>|{\lambda }_2|\ge \cdots \ge |{\lambda }_n|$.

We first assume that the entries of A are functions in the variable z. Hence, ${\lambda }_i$ and the entries of ${\mathit{e}}_i$ are functions in z too. Then Power Iteration I then evaluates both ${\lambda }_1$ and ${\lambda }_1^{\prime }$ for some fixed value of z, while Power Iteration II additionally evaluates the second order derivative ${\lambda }_1^{\prime \prime }$.

The case where the entries of A are functions in two variables x and y is discussed at the end of the appendix. Here, Power Iteration III evaluates higher order partial derivatives of ${\lambda }_1$ for certain fixed values of x and y. For ease of exposition, we provide detailed proofs for the correctness of Power Iteration I and the proofs can be extended for Power Iteration II and Power Iteration III.

We continue our discussion where the entries of A are univariate functions in z. We differentiate each entry of A with respect to z to obtain the matrix ${\mathit{A}}^{\prime }$. Furthermore, for all $1\le i\le n$, we differentiate each entry of eigenvectors ${\mathit{e}}_i$ and the eigenvalue ${\lambda }_i$ to obtain ${\mathit{e}}_i^{\prime }$ and ${\lambda }_i^{\prime }$, respectively. Specifically, it follows from (A1) that

$${\mathit{A}}^{\prime }{\mathit{e}}_i+\mathit{A}{\mathit{e}}_i^{\prime }={\lambda }_i^{\prime }{\mathit{e}}_i+{\lambda }_i{\mathit{e}}_i^{\prime }\mathrm{for}\mathrm{all}i=1,2,\cdots ,n.$$ (A2)

Then, the following procedure computes both ${\lambda }_1$ and ${\lambda }_1^{\prime }$.

Power Iteration I.

Input: Irreducible nonnegative diagonalizable matrix A

Output: Estimates of ${\lambda }_1$ and ${\lambda }_1^{\prime }$

• (1)
  Initialize ${\mathit{q}}^{\left(0\right)}$ such that all its entries are strictly positive.

  • Fix a tolerance value $ϵ$.
  • While $|{\mathit{q}}^{\left(k\right)}-{\mathit{q}}^{(k-1)}|>ϵ$,
    • –
      Set

      $$\begin{array}{cc}\hfill {\lambda }^{\left(k\right)}& =\parallel \mathit{A}{\mathit{q}}^{(k-1)}\parallel ,\hfill \\ \hfill {\mathit{q}}^{\left(k\right)}& =\frac{\mathit{A}{\mathit{q}}^{(k-1)}}{{\lambda }^{\left(k\right)}},\hfill \\ \hfill {\mu }^{\left(k\right)}& =\parallel {\mathit{A}}^{\prime }{\mathit{q}}^{(k-1)}+\mathit{A}{\mathit{r}}^{(k-1)}-{\lambda }^{\left(k\right)}{\mathit{r}}^{(k-1)}\parallel ,\hfill \\ \hfill {\mathit{r}}^{\left(k\right)}& =\frac{\mathit{A}{\mathit{r}}^{(k-1)}+{\mathit{A}}^{\prime }{\mathit{q}}^{(k-1)}-{\mu }^{\left(k\right)}{\mathit{q}}^{(k-1)}}{{\lambda }^{\left(k\right)}}.\hfill \end{array}$$
    • –
      Increment k by one.
  • (2)
    Set ${\lambda }_1\leftarrow {\lambda }^{\left(k\right)}$ and ${\lambda }_1^{\prime }\leftarrow {\mu }^{\left(k\right)}$.

Theorem A1.

If A is an irreducible nonnegative diagonalizable matrix and ${\mathit{q}}^{\left(0\right)}$ has positive components with unit norm, then, as $k\to \infty$, we have

$${\lambda }^{\left(k\right)}\to {\lambda }_1,{\mathit{q}}^{\left(k\right)}\to {\mathit{e}}_1,{\mu }^{\left(k\right)}\to {\lambda }_1^{\prime }.$$

Here, ${\mathit{q}}^{\left(k\right)}\to {\mathit{e}}_1$ means that $∥{\mathit{q}}^{\left(k\right)}-{\mathit{e}}_1∥\to 0$ as $k\to \infty$.

Before we present the proof of Theorem A1, we remark that the usual power iteration method computes only ${\lambda }^{\left(k\right)}$ and ${\mathit{q}}^{\left(k\right)}$. Then, it is well-known (see, for example, [22]) that ${\lambda }^{\left(k\right)}$ and ${\mathit{q}}^{\left(k\right)}$ tend to ${\lambda }_1$ and ${\mathit{e}}_1$, respectively.

Now, since ${\mathit{e}}_i$ spans ${\mathbb{R}}^n$, we can write ${\mathit{q}}^{\left(0\right)}={\displaystyle {\sum }_{i=1}^n}{\alpha }_i{\mathit{e}}_i$ for any initial vector ${\mathit{q}}^{\left(0\right)}$. The next technical lemma provides closed formulas for ${\lambda }^{\left(k\right)}$, ${\mathit{q}}^{\left(k\right)}$, ${\mu }^{\left(k\right)}$, and ${\mathit{r}}^{\left(k\right)}$ in terms of ${\lambda }_i$, ${\mathit{e}}_i$ and ${\alpha }_i$.

Lemma A1.

Let ${\mathit{q}}^{\left(0\right)}={\displaystyle {\sum }_{i=1}^n}{\alpha }_i{\mathit{e}}_i$. Then,

$$\begin{array}{cc}\hfill {\mathit{q}}^{\left(k\right)}& =\frac{{\displaystyle {\sum }_{i=1}^n}{\alpha }_i{\lambda }_i^k{\mathit{e}}_i}{\parallel {\displaystyle {\sum }_{i=1}^n}{\alpha }_i{\lambda }_i^k{\mathit{e}}_i\parallel },\hfill \end{array}$$ (A3)

$$\begin{array}{cc}\hfill {\lambda }^{\left(k\right)}& =\frac{\parallel {\displaystyle {\sum }_{i=1}^n}{\alpha }_i{\lambda }_i^k{\mathit{e}}_i\parallel }{\parallel {\displaystyle {\sum }_{i=1}^n}{\alpha }_i{\lambda }_i^{k-1}{\mathit{e}}_i\parallel },\hfill \end{array}$$ (A4)

$$\begin{array}{cc}\hfill {\mathit{r}}^{\left(k\right)}& =\frac{{\displaystyle {\sum }_{i=1}^n}({\alpha }_i{\mathit{e}}_i^{\prime }+{\alpha }_i^{\prime }{\mathit{e}}_i){\lambda }_i^k+(k{\lambda }_i^{\prime }-{\displaystyle {\sum }_{j=1}^k}{\mu }^{\left(j\right)}){\alpha }_i{\lambda }_i^{k-1}{\mathit{e}}_i}{\parallel {\displaystyle {\sum }_{i=1}^n}{\alpha }_i{\lambda }_i^k{\mathit{e}}_i\parallel },\hfill \end{array}$$ (A5)

$$\begin{array}{cc}\hfill {\mu }^{\left(k\right)}& =\frac{\parallel {\displaystyle {\sum }_{i=1}^n}({\alpha }_i{\mathit{e}}_i^{\prime }+{\alpha }_i^{\prime }{\mathit{e}}_i){\lambda }_i^{k-1}({\lambda }_i-{\lambda }^{\left(k\right)})+{\alpha }_i{\lambda }_i^{k-1}{\lambda }_i^{\prime }{\mathit{e}}_i+((k-1){\lambda }_i^{\prime }-{\displaystyle {\sum }_{j=1}^{k-1}}{\mu }^{\left(j\right)}){\alpha }_i{\lambda }_i^{k-2}({\lambda }_i-{\lambda }^{\left(k\right)}){\mathit{e}}_i\parallel }{\parallel {\displaystyle {\sum }_{i=1}^n}{\alpha }_i{\lambda }_i^{k-1}{\mathit{e}}_i\parallel }.\hfill \end{array}$$ (A6)

Proof. 

Since ${\mathit{q}}^{\left(k\right)}$ is defined recursively as ${\mathit{q}}^{\left(k\right)}=\frac{\mathit{A}{\mathit{q}}^{(k-1)}}{{\lambda }^{\left(k\right)}}=\frac{\mathit{A}{\mathit{q}}^{(k-1)}}{\parallel \mathit{A}{\mathit{q}}^{(k-1)}\parallel }$, we have that

$${\mathit{q}}^{\left(k\right)}=\frac{{\mathit{A}}^k{\mathit{q}}^{\left(0\right)}}{\parallel {\mathit{A}}^k{\mathit{q}}^{\left(0\right)}\parallel }.$$

Then, it follows from Equation (A1) that

$${\mathit{A}}^k{\mathit{q}}^{\left(0\right)}={\mathit{A}}^k{\displaystyle \sum _{i=1}^n}{\alpha }_i{\mathit{e}}_i={\displaystyle \sum _{i=1}^n}{\alpha }_i\left({\mathit{A}}^k{\mathit{e}}_i\right)={\displaystyle \sum _{i=1}^n}{\alpha }_i{\lambda }_i^k{\mathit{e}}_i,$$ (A7)

and, so, we obtain (A3). Similarly, from (A1), we have that

$${\lambda }^{\left(k\right)}=\parallel \mathit{A}{\mathit{q}}^{(k-1)}\parallel =\frac{\parallel {\mathit{A}}^k{\mathit{q}}^{\left(0\right)}\parallel }{\parallel {\mathit{A}}^{k-1}{\mathit{q}}^{\left(0\right)}\parallel }=\frac{\parallel {\displaystyle {\sum }_{i=1}^n}{\alpha }_i{\lambda }_i^k{\mathit{e}}_i\parallel }{\parallel {\displaystyle {\sum }_{i=1}^n}{\alpha }_i{\lambda }_i^{k-1}{\mathit{e}}_i\parallel },$$

as required for (A4).

Next, we note that ${\mathit{r}}^{\left(0\right)}={\displaystyle {\sum }_{i=1}^n}{\alpha }_i{\mathit{e}}_i^{\prime }+{\displaystyle {\sum }_{i=1}^n}{\alpha }_i^{\prime }{\mathit{e}}_i$. Then, using the recursive definition of ${\mathit{r}}^{\left(k\right)}$, we have

$${\mathit{r}}^{\left(k\right)}=\frac{{\mathit{A}}^k{\mathit{r}}^{\left(0\right)}+{\displaystyle {\sum }_{j=0}^{k-1}}{\mathit{A}}^j{\mathit{A}}^{\prime }{\mathit{A}}^{k-j-1}{\mathit{q}}^{\left(0\right)}-\left({\displaystyle {\sum }_{j=1}^k}{\mu }^{\left(j\right)}\right){\mathit{A}}^{k-1}{\mathit{q}}^{\left(0\right)}}{\parallel {\mathit{A}}^k{\mathit{q}}^{\left(0\right)}\parallel }.$$ (A8)

Then, from (A1), we have

$${\mathit{A}}^k{\mathit{r}}^{\left(0\right)}={\mathit{A}}^k\left({\displaystyle \sum _{i=1}^n}{\alpha }_i{\mathit{e}}_i^{\prime }+{\displaystyle \sum _{i=1}^n}{\alpha }_i^{\prime }{\mathit{e}}_i\right)={\displaystyle \sum _{i=1}^n}{\alpha }_i\left({\mathit{A}}^k{\mathit{e}}_i^{\prime }\right)+{\displaystyle \sum _{i=1}^n}{\alpha }_i^{\prime }{\lambda }_i^k{\mathit{e}}_i.$$ (A9)

and, from (A2),

$${\mathit{A}}^{\prime }{\displaystyle \sum _{i=1}^n}{\alpha }_i{\lambda }_i^{k-j-1}{\mathit{e}}_i={\displaystyle \sum _{i=1}^n}{\alpha }_i{\lambda }_i^{k-j-1}\left({\mathit{A}}^{\prime }{\mathit{e}}_i\right)={\displaystyle \sum _{i=1}^n}{\alpha }_i{\lambda }_i^{k-j-1}({\lambda }_i^{\prime }{\mathit{e}}_i+{\lambda }_i{\mathit{e}}_i^{\prime }-\mathit{A}{\mathit{e}}_i^{\prime }).$$

Therefore, using (A1) again,

$$\begin{array}{cc}\hfill {\displaystyle \sum _{j=0}^{k-1}}{\mathit{A}}^j{\mathit{A}}^{\prime }{\displaystyle \sum _{i=1}^n}{\alpha }_i{\lambda }_i^{k-j-1}{e}_i& ={\displaystyle \sum _{j=0}^{k-1}}{\mathit{A}}^j{\displaystyle \sum _{i=1}^n}{\alpha }_i{\lambda }_i^{k-j-1}({\lambda }_i^{\prime }{e}_i+{\lambda }_i{e}_i^{\prime }-\mathit{A}{e}_i^{\prime })\hfill \\ \hfill & =k{\displaystyle \sum _{i=1}^n}{\alpha }_i{\lambda }_i^{k-1}{\lambda }_i^{\prime }{e}_i+{\displaystyle \sum _{i=1}^n}{\alpha }_i{\lambda }_i^k{e}_i^{\prime }-{\displaystyle \sum _{i=1}^n}{\alpha }_i\left({\mathit{A}}^k{e}_i^{\prime }\right).\hfill \end{array}$$

Therefore, we obtain (A5).

Finally, we recall that ${\mu }^{\left(k\right)}$ is defined as

$${\mu }^{\left(k\right)}=\parallel {\mathit{A}}^{\prime }{\mathit{q}}^{(k-1)}+\mathit{A}{\mathit{r}}^{(k-1)}-{\lambda }^{\left(k\right)}{\mathit{r}}^{(k-1)}\parallel .$$

Then, by replacing ${\mathit{r}}^{(k-1)}$ and ${\mathit{q}}^{(k-1)}$ from (A5) and (A3), respectively, and then using Equation (A2), we obtain (A6). □

Finally, we are ready to demonstrate the correctness of Power Iteration I.

Proof of Theorem A1.

Since A is an irreducible nonnegative diagonalizable matrix, ${\lambda }_1$ is real positive and there exists $0<ϵ<1$ such that $\frac{|{\lambda }_i|}{{\lambda }_1}<ϵ\mathrm{for}\mathrm{all}i=2,3,\cdots ,n$ (see, for example, [11]). For purposes of brevity, we write

$${\Phi }_k={\displaystyle \sum _{i=1}^n}{\alpha }_i{\lambda }_i^k{\mathit{e}}_i$$ (A10)

and, so, we can rewrite (A3) as

$${\mathit{q}}^{\left(k\right)}=\frac{{\Phi }_k}{\parallel {\Phi }_k\parallel }=\frac{{\lambda }_1^k}{\parallel {\Phi }_k\parallel }\frac{{\Phi }_k}{{\lambda }_1^k}=\frac{{\lambda }_1^k}{\parallel {\Phi }_k\parallel }\left({\alpha }_1{\mathit{e}}_1+{\displaystyle \sum _{i=2}^n}{\alpha }_i\frac{{\lambda }_i^k}{{\lambda }_1^k}{\mathit{e}}_i\right).$$

Now, since ${\lambda }_i^k/{\lambda }_1^k\le {ϵ}^k$ for all $i=2,\dots ,n$, we have that

$$∥\frac{{\Phi }_k}{{\lambda }_1^k}-{\alpha }_1{\mathit{e}}_1∥\le C_1{ϵ}^k\mathrm{for}\mathrm{some}\mathrm{constant}{C}_1.$$ (A11)

Then, using the triangle inequality, we have that as $k\to \infty$, $\left|\frac{\parallel {\Phi }_k\parallel }{{\lambda }_1^k}-{\alpha }_1\right|\to 0$ and, thus, $\frac{{\lambda }_1^k}{\parallel {\Phi }_k\parallel }\to \frac{1}{{\alpha }_1}$. Therefore, $\parallel {\mathit{q}}^{\left(k\right)}-{\mathit{e}}_1\parallel \to 0$ as required.

It should be noted that since $\frac{{\lambda }_1^k}{\parallel {\Phi }_k\parallel }$ tends to a finite limit, we have that $\frac{{\lambda }_1^k}{\parallel {\Phi }_k\parallel }$ is bounded above by some constant. In other words, we have that

$$\frac{{\lambda }_1^k}{\parallel {\Phi }_k\parallel }\le C_2\mathrm{for}\mathrm{some}\mathrm{constant}{C}_2.$$ (A12)

Next, we show the following inequality:

$$|{\lambda }^{\left(k\right)}-{\lambda }_1|\le C_3{ϵ}^{k-1}\mathrm{for}\mathrm{some}\mathrm{constant}{C}_3.$$ (A13)

Using (A4), we have that

$$\frac{\parallel {\Phi }_k-{\lambda }_1{\Phi }_{k-1}\parallel }{\parallel {\Phi }_{k-1}\parallel }=\frac{{\lambda }_1^{k-1}}{\parallel {\Phi }_{k-1}\parallel }\frac{{\displaystyle {\sum }_{i=1}^n}{\alpha }_i{\lambda }_i^k{\mathit{e}}_i-{\alpha }_i{\lambda }_1{\lambda }_i^{k-1}{\mathit{e}}_i}{{\lambda }_1^{k-1}}=\left(\frac{{\lambda }_1^{k-1}}{\parallel {\Phi }_{k-1}\parallel }\right)·{\lambda }_1·{\displaystyle \sum _{i=2}^n}{\alpha }_i\left(\frac{{\lambda }_i^k}{{\lambda }_1^k}-\frac{{\lambda }_i^{k-1}}{{\lambda }_1^{k-1}}\right){\mathit{e}}_i.$$

Now, observe that $\left(\frac{{\lambda }_i^k}{{\lambda }_1^k}-\frac{{\lambda }_i^{k-1}}{{\lambda }_1^{k-1}}\right)\le 2{ϵ}^{k-1}$ for $i=2,\dots ,n$. Since $\frac{{\lambda }_1^{k-1}}{\parallel {\Phi }_{k-1}\parallel }\le C_2$, we have (A13) after applying the triangle inequality.

Again, to reduce clutter, we introduce the following abbreviations:

$$\begin{array}{cc}\hfill D_k& ={\displaystyle \sum _{i=1}^n}({\alpha }_i{\mathit{e}}_i^{\prime }+{\alpha }_i^{\prime }{\mathit{e}}_i){\lambda }_i^{k-1}({\lambda }_i-{\lambda }^{\left(k\right)}),\hfill \\ \hfill E_k& ={\displaystyle \sum _{i=1}^n}{\alpha }_i{\lambda }_i^{k-1}{\lambda }_i^{\prime }{\mathit{e}}_i,\hfill \\ \hfill F_k& ={\displaystyle \sum _{i=1}^n}\left((k-1){\lambda }_i^{\prime }-{\displaystyle \sum _{j=1}^{k-1}}{\mu }^{\left(j\right)}\right){\alpha }_i{\lambda }_i^{k-2}({\lambda }_i-{\lambda }^{\left(k\right)}){\mathit{e}}_i.\hfill \end{array}$$

Thus, we can rewrite (A6) as

$${\mu }^{\left(k\right)}=\frac{\parallel D_k+E_k+F_k\parallel }{\parallel {\Phi }_{k-1}\parallel }\le {\lambda }_1^{\prime }+\frac{\parallel D_k\parallel }{\parallel {\Phi }_{k-1}\parallel }+\frac{\parallel E_k-{\lambda }_1^{\prime }{\Phi }_{k-1}\parallel }{\parallel {\Phi }_{k-1}\parallel }+\frac{\parallel F_k\parallel }{\parallel {\Phi }_{k-1}\parallel }.$$

Next, we bound each of the summands on the right-hand side. Specifically, we show the following inequalities:

$$\begin{array}{cc}\hfill \frac{\parallel D_k\parallel }{\parallel {\Phi }_{k-1}\parallel }+\frac{\parallel E_k-{\lambda }_1^{\prime }{\Phi }_{k-1}\parallel }{\parallel {\Phi }_{k-1}\parallel }& \le C_4{ϵ}^{k-1}\mathrm{for}\mathrm{some}\mathrm{constant}{C}_4,\hfill \end{array}$$ (A14)

$$\begin{array}{cc}\hfill \frac{\parallel F_k\parallel }{\parallel {\Phi }_{k-1}\parallel }& \le C_5(k-1){ϵ}^{k-1}+C_5\left({\displaystyle \sum _{j=1}^{k-1}}{\mu }^{\left(k\right)}\right){ϵ}^{k-1}\mathrm{for}\mathrm{some}\mathrm{constant}{C}_5.\hfill \end{array}$$ (A15)

To demonstrate (A14), we consider

$$\frac{\parallel D_k\parallel }{{\lambda }_1^{k-1}}=∥{\displaystyle \sum _{i=1}^n}({\alpha }_i{\mathit{e}}_i^{\prime }+{\alpha }_i^{\prime }{\mathit{e}}_i)\frac{{\lambda }_i^{k-1}}{{\lambda }_1^{k-1}}({\lambda }_i-{\lambda }^{\left(k\right)})∥\le \parallel {\alpha }_1{\mathit{e}}_1^{\prime }+{\alpha }_1^{\prime }{\mathit{e}}_1\parallel |{\lambda }_1-{\lambda }^{\left(k\right)}|+{ϵ}^{k-1}{\displaystyle \sum _{i=2}^n}\parallel {\alpha }_i{\mathit{e}}_i^{\prime }+{\alpha }_i^{\prime }{\mathit{e}}_i\parallel |{\lambda }_i-{\lambda }^{\left(k\right)}|.$$

We use (A13) to bound the first summand by some constant multiple of ${ϵ}^{k-1}$. On the other hand, we have $|{\lambda }_i-{\lambda }^{\left(k\right)}|\le |{\lambda }_i-{\lambda }_1|+|{\lambda }_1-{\lambda }^{\left(k\right)}|\le max\{|{\lambda }_i-{\lambda }_1|:2\le i\le n\}+C_3{ϵ}^{k-1}$ for $2\le i\le n$. In other words, the second summand is also bounded by some constant multiple of ${ϵ}^{k-1}$. Next, we consider

$$\frac{\parallel E_k-{\lambda }_1^{\prime }{\Phi }_{k-1}\parallel }{{\lambda }_1^{k-1}}=∥{\displaystyle \sum _{i=1}^n}{\alpha }_i\frac{{\lambda }_i^{k-1}}{{\lambda }_1^{k-1}}({\lambda }_i^{\prime }-{\lambda }_1^{\prime }){\mathit{e}}_i∥\le {ϵ}^{k-1}{\displaystyle \sum _{i=2}^n}\left|{\alpha }_i({\lambda }_i^{\prime }-{\lambda }_1^{\prime })\right|.$$

and, so, $\frac{\parallel E_k-{\lambda }_1^{\prime }{\Phi }_{k-1}\parallel }{{\lambda }_1^{k-1}}$ is also bounded by a multiple of ${ϵ}^{k-1}$. Therefore, since $\frac{{\lambda }_1^{k-1}}{\parallel {\Phi }_{k-1}\parallel }\le C_2$, we have (A14). Using similar methods, we can establish (A15).

Next, we apply (A14) and then recursively apply (A15) until the right-hand side is free of ${\mu }^{\left(i\right)}$s. Then, it follows that

$${\mu }^{\left(k\right)}\le {\lambda }_1^{\prime }+C_4{ϵ}^{k-1}+C_5(k-1){ϵ}^{k-1}+\prod _{j=2}^{k-1}(1+C_5{ϵ}^{k-j})+C_5{ϵ}^{k-1}{\displaystyle \sum _{i=1}^{k-1}}({\lambda }_1^{\prime }+C_4{ϵ}^{k-i-1}{C}_5(k-i-1){ϵ}^{k-i-1})\prod _{j=2}^i(1+C_5{ϵ}^{k-j})).$$ (A16)

Furthermore, since $i\le k-1$, ${\prod }_{j=2}^i(1+C_5{ϵ}^{k-j})\le {\prod }_{j=2}^{k-1}(1+C_5{ϵ}^{k-j})$, we can rewrite (A16) as

$${\mu }^{\left(k\right)}\le {\lambda }_1^{\prime }+C_4{ϵ}^{k-1}+C_5(k-1){ϵ}^{k-1}+\prod _{j=2}^{k-1}(1+C_5{ϵ}^{k-j})\left(1+C_5{ϵ}^{k-1}{\displaystyle \sum _{i=1}^{k-1}}({\lambda }_1^{\prime }+C_4{ϵ}^{k-i-1}{C}_5(k-i-1){ϵ}^{k-i-1})\right).$$ (A17)

Next, it follows from standard calculus that ${\prod }_{j=2}^{k-1}(1+C_5{ϵ}^{k-j})<e^{\frac{C_5}{1-ϵ}}$. Furthermore, since $ϵ<1$, we have ${\displaystyle {\sum }_{i=0}^{k-2}}{ϵ}^j<\frac{1}{1-ϵ}$ and ${\displaystyle {\sum }_{i=0}^{k-2}}j{ϵ}^j<\frac{1}{{(1-ϵ)}^2}$. Putting everything together, we have

$${\mu }^{\left(k\right)}\le {\lambda }_1^{\prime }+C_4{ϵ}^{k-1}+C_5(k-1){ϵ}^{k-1}+C_5{ϵ}^{k-1}{e}^{\frac{C_5}{1-ϵ}}\left(1+(k-1){\lambda }_1^{\prime }+\frac{C_4}{1-ϵ}+\frac{C_5}{{(1-ϵ)}^2}\right).$$ (A18)

As $k\to \infty$, since $ϵ<1$, we have ${ϵ}^k\to 0$ and $k{ϵ}^k\to 0$. Therefore, ${lim}_{k\to \infty }{\mu }^{\left(k\right)}\le {\lambda }_1^{\prime }$. Using similar methods, we have that ${lim}_{k\to \infty }{\mu }^{\left(k\right)}\ge {\lambda }_1^{\prime }$ and, so, ${lim}_{k\to \infty }{\mu }^{\left(k\right)}={\lambda }_1^{\prime }$, as required. □

Next, we modify Power Iteration I so as to compute the higher order derivatives. We omit a detailed proof as it is similar to the proof of Theorem A1.

Power Iteration II.

Input: Irreducible nonnegative diagonalizable matrix A

Output: Estimates of ${\lambda }_1$, ${\lambda }_1^{\prime }$, and ${\lambda }_1^{\prime \prime }$

• (1)Initialize ${\mathit{q}}^{\left(0\right)}$ such that all its entries are strictly positive.

  • Fix a tolerance value $ϵ$.

  • While $|{\mathit{q}}_{\left(k\right)}-{\mathit{q}}_{(k-1)}|>ϵ$,

    • –
      Set

      $$\begin{array}{cc}\hfill {\lambda }^{\left(k\right)}& =\parallel \mathit{A}{\mathit{q}}^{(k-1)}\parallel ,\hfill \\ \hfill {\mathit{q}}^{\left(k\right)}& =\frac{\mathit{A}{\mathit{q}}^{(k-1)}}{{\lambda }^{\left(k\right)}},\hfill \\ \hfill {\mu }^{\left(k\right)}& =\parallel {\mathit{A}}^{\prime }{\mathit{q}}^{(k-1)}+\mathit{A}{\mathit{r}}^{(k-1)}-{\lambda }^{\left(k\right)}{\mathit{r}}^{(k-1)}\parallel ,\hfill \\ \hfill {\mathit{r}}^{\left(k\right)}& =\frac{\mathit{A}{\mathit{r}}^{(k-1)}+{\mathit{A}}^{\prime }{\mathit{q}}^{(k-1)}-{\mu }^{\left(k\right)}{\mathit{q}}^{(k-1)}}{{\lambda }^{\left(k\right)}},\hfill \\ \hfill {\nu }^{\left(k\right)}& =\parallel {\mathit{A}}^{\prime \prime }{q}^{(k-1)}+2{\mathit{A}}^{\prime }{\mathit{r}}^{(k-1)}+\mathit{A}{s}^{(k-1)}-{\lambda }^{\left(k\right)}{\mathit{s}}^{(k-1)}-2{\mu }^{\left(k\right)}{\mathit{r}}^{(k-1)}\parallel ,\hfill \\ \hfill s^{\left(k\right)}& =\frac{{\mathit{A}}^{\prime \prime }{q}^{(k-1)}+2{\mathit{A}}^{\prime }{\mathit{r}}^{(k-1)}+\mathit{A}{\mathit{s}}^{(k-1)}-2{\mu }^{\left(k\right)}{\mathit{r}}^{(k-1)}-{\nu }^{\left(k\right)}{\mathit{q}}^{(k-1)}}{{\lambda }^{\left(k\right)}}.\hfill \end{array}$$
    • –Increment k by one.
  • (2)Set ${\lambda }_1\leftarrow {\lambda }^{\left(k\right)}$, ${\lambda }_1^{\prime }\leftarrow {\mu }^{\left(k\right)}$ and ${\lambda }_1^{\prime \prime }\leftarrow {\nu }^{\left(k\right)}$.

Theorem A2.

If A is an irreducible nonnegative diagonalizable matrix and ${\mathit{q}}^{\left(0\right)}$ has positive components with unit norm, then, as $k\to \infty$, we have

$${\lambda }^{\left(k\right)}\to {\lambda }_1,{\mathit{q}}^{\left(k\right)}\to {\mathit{e}}_1,{\mu }^{\left(k\right)}\to {\lambda }_1^{\prime },{\nu }^{\left(k\right)}\to {\lambda }_1^{\prime \prime }.$$

Finally, we end this appendix with a power iteration method that computes the partial derivatives when the elements of the given matrix are bivariate functions.

Power Iteration III.

Input: Irreducible nonnegative diagonalizable matrix A

Output: Estimates of ${\lambda }_1$, ${\left({\lambda }_1\right)}_x$, ${\left({\lambda }_1\right)}_y$, ${\left({\lambda }_1\right)}_{xx}$, ${\left({\lambda }_1\right)}_{yy}$, and ${\left({\lambda }_1\right)}_{xy}$

• (1)Initialize ${\mathit{q}}^{\left(0\right)}$ such that all its entries are strictly positive.

  • Fix a tolerance value $ϵ$.

  • While $|{\mathit{q}}^{\left(k\right)}-{\mathit{q}}^{(k-1)}|>ϵ$,

    • –
      Set

      $$\begin{array}{cc}\hfill {\lambda }^{\left(k\right)}& =\parallel \mathit{A}{q}^{(k-1)}\parallel ,\hfill \\ \hfill q^{\left(k\right)}& =\frac{\mathit{A}{q}^{(k-1)}}{{\lambda }^{\left(k\right)}},\hfill \\ \hfill {\lambda }_x^{\left(k\right)}& =\parallel {\mathit{A}}_x{\mathit{q}}^{(k-1)}+\mathit{A}{\mathit{q}}_x^{(k-1)}-\lambda {\mathit{q}}_x^{(k-1)}\parallel ,\hfill \\ \hfill {\mathit{q}}_x^{\left(k\right)}& =\frac{{\mathit{A}}_x{\mathit{q}}^{(k-1)}+\mathit{A}{\mathit{q}}_x^{(k-1)}-{\lambda }_x^{(k-1)}{\mathit{q}}^{(k-1)}}{{\lambda }^{\left(k\right)}},\hfill \\ \hfill {\lambda }_y^{\left(k\right)}& =\parallel {\mathit{A}}_y{\mathit{q}}^{(k-1)}+\mathit{A}{\mathit{q}}_y^{(k-1)}-\lambda {\mathit{q}}_y^{(k-1)}\parallel ,\hfill \\ \hfill {\mathit{q}}_y^{\left(k\right)}& =\frac{{\mathit{A}}_y{\mathit{q}}^{(k-1)}+\mathit{A}{\mathit{q}}_y^{(k-1)}-{\lambda }_y^{(k-1)}{\mathit{q}}^{(k-1)}}{{\lambda }^{\left(k\right)}},\hfill \\ \hfill {\lambda }_{xx}^{\left(k\right)}& =\parallel {\mathit{A}}_{xx}{\mathit{q}}^{(k-1)}+2{\mathit{A}}_x{\mathit{q}}_x^{(k-1)}+\mathit{A}{\mathit{q}}_{xx}^{(k-1)}-{\lambda }^{(k-1)}{\mathit{q}}_{xx}^{(k-1)}-2{\lambda }_x^{(k-1)}{\mathit{q}}_x^{(k-1)}\parallel ,\hfill \\ \hfill {\mathit{q}}_{xx}^{\left(k\right)}& =\frac{{\mathit{A}}_{xx}{\mathit{q}}^{(k-1)}+2{\mathit{A}}_x{\mathit{q}}_x^{(k-1)}+\mathit{A}{\mathit{q}}_{xx}^{(k-1)}-2{\lambda }_x^{(k-1)}{\mathit{q}}_x^{(k-1)}-{\lambda }_{xx}^{(k-1)}{\mathit{q}}^{(k-1)}}{{\lambda }^{\left(k\right)}}\hfill \\ \hfill {\lambda }_{yy}^{\left(k\right)}& =\parallel {\mathit{A}}_{yy}{\mathit{q}}^{(k-1)}+2{\mathit{A}}_y{\mathit{q}}_y^{(k-1)}+\mathit{A}{\mathit{q}}_{yy}^{(k-1)}-{\lambda }^{(k-1)}{\mathit{q}}_{yy}^{(k-1)}-2{\lambda }_y^{(k-1)}{\mathit{q}}_y^{(k-1)}\parallel ,\hfill \\ \hfill {\mathit{q}}_{yy}^{\left(k\right)}& =\frac{{\mathit{A}}_{yy}{\mathit{q}}^{(k-1)}+2{\mathit{A}}_y{\mathit{q}}_y^{(k-1)}+\mathit{A}{\mathit{q}}_{yy}^{(k-1)}-2{\lambda }_y^{(k-1)}{\mathit{q}}_y^{(k-1)}-{\lambda }_{yy}^{(k-1)}{\mathit{q}}^{(k-1)}}{{\lambda }^{\left(k\right)}}\hfill \\ \hfill {\lambda }_{xy}^{\left(k\right)}& =\parallel {\mathit{A}}_{xy}{\mathit{q}}^{(k-1)}+{\mathit{A}}_x{\mathit{q}}_y^{(k-1)}+{\mathit{A}}_y{\mathit{q}}_x^{(k-1)}+\mathit{A}{\mathit{q}}_{xy}^{(k-1)}-{\lambda }^{(k-1)}{\mathit{q}}_{xy}^{(k-1)}-{\lambda }_x^{(k-1)}{\mathit{q}}_y^{(k-1)}-{\lambda }_y^{(k-1)}{\mathit{q}}_x^{(k-1)}\parallel ,\hfill \\ \hfill {\mathit{q}}_{xy}^{\left(k\right)}& =\frac{{\mathit{A}}_{xy}{\mathit{q}}^{(k-1)}+{\mathit{A}}_x{\mathit{q}}_y^{(k-1)}+{\mathit{A}}_y{\mathit{q}}_x^{(k-1)}+\mathit{A}{\mathit{q}}_{xy}^{(k-1)}-{\lambda }_{xy}^{(k-1)}{\mathit{q}}^{(k-1)}-{\lambda }_x^{(k-1)}{\mathit{q}}_y^{(k-1)}-{\lambda }_y^{(k-1)}{\mathit{q}}_x^{(k-1)}}{{\lambda }^{\left(k\right)}}.\hfill \end{array}$$
    • –Increment k by one.

  • Set ${\lambda }^{\left(k\right)}\leftarrow {\lambda }_1$, ${\lambda }_x^{\left(k\right)}\leftarrow {\left({\lambda }_1\right)}_x$, ${\lambda }_y^{\left(k\right)}\leftarrow {\left({\lambda }_1\right)}_y$, ${\lambda }_{xx}^{\left(k\right)}\leftarrow {\left({\lambda }_1\right)}_{xx}$, ${\lambda }_{yy}^{\left(k\right)}\leftarrow {\left({\lambda }_1\right)}_{yy}$, ${\lambda }_{xy}^{\left(k\right)}\leftarrow {\left({\lambda }_1\right)}_{xy}$.

Theorem A3.

If A is an irreducible nonnegative diagonalizable matrix and ${\mathit{q}}^{\left(0\right)}$ has positive components with unit norm, then, as $k\to \infty$, we have ${\lambda }_{xx}^{\left(k\right)}\to {\left({\lambda }_1\right)}_{xx}$, ${\lambda }_{yy}^{\left(k\right)}\to {\left({\lambda }_1\right)}_{yy}$, ${\lambda }_{xy}^{\left(k\right)}\to {\left({\lambda }_1\right)}_{xy}$.

Author Contributions

Conceptualization, K.G. and H.M.K.; software, K.G.; writing—original draft preparation, K.G.; writing—review and editing, K.G. and H.M.K. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

No new data were created or analyzed in this study. Data sharing is not applicable to this article.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Funding Statement

The work of Han Mao Kiah was supported by the Ministry of Education, Singapore, under its MOE AcRF Tier 2 Award under Grant MOE-T2EP20121-0007 and MOE AcRF Tier 1 Award under Grant RG19/23.