A zoo of computable binary normal sequences

Abstract

Historically there has been a virtual absence of constructive methods to produce broad classes of “certifiably random” infinite sequences, despite considerable interest in this endeavor. Previously, we proved a theorem that yielded explicit algorithms to produce diverse sets of normal numbers, reasonable candidates for random sequences, given their limiting equidistribution of subblocks of all lengths. Herein, we develop this algorithmic approach much further, systematizing the normal number generation process in several ways. We construct delineated, distinct sets of normal numbers (classified by the extent to which initial segments deviate from maximal irregularity), with virtually any allowable specified rate of convergence to 0 of this deviation, encompassing arbitrarily fast and slow rates, and accommodating asymmetric behavior above or below a centered median. As a corollary, we provide an explicit construction of a normal number that satisfies the Law of the Iterated Logarithm. We also produce distinct families of “biased” normal numbers, with virtually any specified rate of convergence of the bias (to 0). This latter theory is in part motivated by the remarkable observation that the binary version of Champernowne’s number, which is also normal, is biased—any initial segment has more 1s than 0s. Finally, we construct an interesting normal sequence with arbitrarily fast convergence to equidistribution of singleton blocks, yet arbitrarily slow convergence of pairs, which has profound implications both for probability theory, and for metrics to evaluate the “near-randomness” of sequences.

The need for broad collections of “truly random” infinite sequences and numbers arises in myriad contexts across mathematics, and the natural and social sciences. As discussed previously, although probability theory, algorithmic complexity, and cryptoanalysis have intuitive appeal as potential sources of such collections, there is no extant methodology to actually construct, designate, or display certifiably random individual sequences from these theories (1, 2). In contrast, in 1909 Borel formulated the concept of a normal number to a base b (3), for which all possible k-tuples (blocks) of consecutive digits from the set {0,1,2,…,b - 1} occur with limiting frequency 1/b^k. This equidistribution characterization provides a verifiable specification for individual sequences, as described more than 60 years ago by Paul Erdos—“a normal number being one whose digits exhibit a complete randomness” (4). However, Borel’s theorem that such numbers form a set of Lebesgue measure 1 in the unit interval (i.e., almost all numbers are normal) (3) neither indicates, nor is naturally convertible to attendant algorithms for explicit constructions. Indeed, there are remarkably few sets of known explicitly computable normal numbers.

Probably best known among them is Champernowne’s number .1234567891011… (5), although an earlier example that seems to have been overlooked in much of the literature on normal numbers was put forth by Norbert Wiener in 1930 (ref. 6, pp. 201–203). Champernowne’s result was thematically generalized by Besicovitch (7), Copeland and Erdos (4) and Davenport and Erdos (8) to include, e.g., sequences of consecutive squares and primes such as 0.1491625… and 0.235711…. Subsequently, Stoneham (9) and Korobov (10) established normality for rapidly converging sums of reciprocals of powers of highly specified (e.g., ergodic prime) sequences; Bailey and Crandall (11) generalized this work, and also show normality for certain specially synchronized “perturbed” random number generators. These examples and the accompanying proofs, while interesting in their own right, reinforce the considerable difficulty involved in verifying that a particular number is normal, and thus underscore the need for broadly applicable methods to produce general classes of normal numbers.

Previously we specified constructive methods to produce large classes of certifiably binary normal numbers (1, 2), both by appropriate concatenations of finite maximally irregular sequences (Theorem Pr-2), and by an innocuous-looking yet essential perturbation strategy applied to established normal numbers (Theorem Pr-4). These emerged from consideration of C-random sequences, an equivalent characterization of normality based on a measure of sequential irregularity, approximate entropy (ApEn) (1). The purposes of this paper are twofold. First, we provide a much more systematic and extended development of this approach, which facilitates construction of normal numbers satisfying a diversity of distributional constraints. Second, we introduce two previously unexplored aspects of the behavior of normal sequences: (i) bias; and (ii) convergence rates of k-tuples to equidistribution for k≥2, which in particular accommodates (potentially dramatic) variation with k of these rates within the same normal number.

There frequently appears to be an implicit presumption that normality is an ultimate endpoint in the characterization of the randomness of infinite-sequence expansions of numbers (e.g., ref. 12). Yet we can probe more sharply than asymptotic equidistribution, or, following the phrasing of Erdos, “refine randomness.” In virtually any application, one utilizes the initial segment of a putatively random sequence. Thus we should want to assess the degree to which these initial segments are not maximally irregular [or satisfy other standard criteria for finite random sequences—e.g., those given by Knuth (13) or Golomb (14)]. As part of the development of ApEn, we also quantify the extent to which any sequence deviates from maximal irregularity, via a set of deficit (def_m) functions. It is relatively straightforward to show that there exists no infinite sequence with the property that all finite initial segments are maximally irregular (Theorem 1, ref. 1), and accordingly, all def_m functions have “nontrivial asymptotic dynamics” (i.e., are positive infinitely often). Previously, we demonstrated sharp distinctions among normal numbers on the basis of asymptotic rates of convergence of def₀ to 0 (1, 2). In the present paper, we go much further—not only do we utilize def₀ to classify and distinguish normal numbers, but we now effectively engineer normal numbers to virtually any allowable (def₀) specification.

With this background at hand, below we provide four primary new results. As prologue, Theorem 1 establishes that the binary version of Champernowne’s number, rescaled to a sequence with values ± 1, has a “surprising” attribute—namely, partial sums of any initial segment are always positive. Thus we can regard this as a biased normal number. This recognition motivated further study of the identification of bias in normal numbers. In Theorem 2, we provide an algorithmic technique to produce normals with virtually any specified extent of bias, relative to the constraint of normality of the infinite sequence. In turn, Theorem 2 became an essential input for the more general, two-sided Theorem 3, which provides the algorithmic basis for construction of classes of (unbiased) normal numbers with virtually any allowable rate of convergence of def₀ to 0, encompassing arbitrarily fast and conversely slow rates. Accordingly, Theorems 2 and 3 provide an extensive collection of delineated normal sequences. Lastly, we construct a normal number denoted Seq(0_vs1), with sharp contrast between asymptotic characteristics of blocks of differing lengths, here between singletons and pairs. Subject to the constraint of normality, Seq(0_vs1) has rapid convergence of the frequency of occurrence of 1-blocks to ½, yet slow convergence of the frequency of 2-blocks to ¼. This is the prototype for constructing normal numbers with rates of convergence of the frequency of k-tuples to equidistribution that vary substantially with k.

Herein, we provide the central ideas for construction of broad classes of normal numbers, and statements of our main new results, Theorems 1–3 and the construction of Seq(0_vs1). Complete proofs are given in the SI Appendix. We distinguish between theorems that we previously proved and new results by denoting the former by Theorems Pr-1 through Pr-4, and new findings by Theorems 1–3. Throughout, we restrict attention to binary sequences of 0s and 1s.

Basic Ideas and Previous Results

Approximate Entropy (ApEn).

We recall several definitions and results from prior work (1, 2, 15). First, we quantify irregularity utilizing approximate entropy, ApEn (precise definition in SI Appendix). For a length N sequence of real numbers measures the logarithmic frequency with which blocks (subsequences of contiguous sequence points) of length m that are close together—i.e., within a tolerance range r—remain close together for blocks augmented by one position. Larger values of ApEn imply greater irregularity in , while smaller values correspond to more instances of recognizable patterns in the sequence.

In our binary setting, we set r < 1 as our measure of resolution, and observe that we monitor exact matches in blocks. With this restriction at hand we suppress the dependence of ApEn on r below. In this setting, an alternative formulation of ApEn (Theorem S1, SI Appendix) provides an appealing interpretation of ApEn as a difference between m-block and m + 1-block Shannon entropies, defined along single sequences. Specifically, for a sequence and a k-block {v₁,v₂,…,v_k}∈{0,1}^k, define in , with denoting the frequency of such occurrences. Let . Then we deduce that

A length N sequence is defined as {m,N}-irregular if it achieves the maximal ApEn(m,N) value among all sequences of length N; and is defined as N-irregular (N-random) if it is {m,N}-irregular for m = 0,1,2,…,m_crit(N), where we specify m_crit(N) = max(m∶2^m < N) (2). This choice of m_crit(N) is consistent with the intuitive idea of finite random sequence, as shown previously (16):

Theorem Pr-1.

Let X(m) be as defined above. A sequence is N-random if and only if for 1 ≤ m ≤ m_crit(N) + 1, the expression is a minimum (among length N sequences).

Thus Theorem Pr-1 gives a useful equivalence of maximally irregular ApEn sequences and maximally equidistributed sequences, while grading the remaining sequences in terms of proximity to maximality.

Note that while the existence and identification of {m,N} random sequences is straightforward, it is not a priori apparent that maximally irregular (N-random) sequences exist for all lengths N, i.e., that sequences exist that satisfy maximality of ApEn values for all values of m ≤ m_crit(N) simultaneously. It has been known for many years that such maximal sequences exist for full cycle sequences of length N = 2^L for any L (17, 18). Indeed, a subset of full-cycle (deBruijn) sequences can be generated by linear shift registers associated with primitive polynomials of degree k, which exists for all degrees k (17); an outstanding source for algorithms to construct full cycles is Fredricksen (18). However, the non–“power of 2” case had been an open question until recently, when both Carpi and DeLuca (19) and Spindler (20) proved the existence of maximally irregular sequences for all lengths N, by independent methods. Carpi and DeLuca do so via the study of uniform words (seen as equivalent to our definition of maximally irregular sequences), and moreover, show this existence over any finite length alphabet. Spindler proves the result for the binary case alone. Notably, both proofs lead to relatively efficient and fast algorithms for the production of maximal sequences. Furthermore, both proofs elucidate Theorem Pr-1. Specifically, given a maximally irregular sequence of any length N, for all 1 ≤ m ≤ m_crit(N) + 1, the numbers of occurrences of any two length m blocks in differ by at most 1, and thus approximate a frequency of 1/2^m as closely as possible.

For infinite sequences , and , assuming this limit exists. Asymptotic ApEn(m) values converge to log 2 along maximally random binary sequences (1). This fact motivates the following formulation of an infinite “random sequence.”

Definition 1:

An infinite binary sequence is called computationally random, denoted as C-random, iff for all m≥0.

Importantly, observe that the normality of a binary number reduces to the condition that for all m≥0. Similarly, for an infinite sequence of binary random variables with probability p = 1/2 each of 0 and 1, joint independence as defined by probability theory implies C-randomness of realizations with probability 1. Thus both these fundamental notions are seen as limit statements, without rates of convergence, which we refine below.

Next, recall the technology to quantify proximity of a finite sequence to maximal irregularity.

Definition 2:

For a length N sequence , let .

For infinite sequences these deficit functions measure how close an initial segment of length N is to being {m,N}-irregular. Normality reduces to the condition that (1).

For a length N sequence , define by . This mapping of {0,1}^N to {-1,1}^N allows direct comparisons of our constructions to the almost sure (a.s.) laws of probability theory.

We previously defined excess functions for singleton blocks (1).

Definition 3:

For a length N sequence , define in in ; in in ; and .

Note that ; ; and , where .

Also, we observed (1) that for small values of def₀,

[1]

We illustrate several of the above definitions by representative calculations, applied to a Prefer-one deBruijn sequence (18) of length 64, left-shifted so that the 6-block of 1s leads. This sequence .

For the initial segment of length 24, ; ; . Accordingly, ; ; . Also, ; ; . For singletons, these calculations manifest many more 1s than 0s (17 vs. 7); the ApEn(1,24) and calculations calibrate a non-equidistribution of pairs, with 12 occurrences of {1,1}, 2 of {0,0}, 5 of {1,0}, and 4 of {0,1}. Furthermore, , and , with corresponding values , and ; and and . Note that a comparison of and affirms Eq. 1. Also, observe that although at the cycle endpoint, there is maximal equidistribution (with , initial segments for N < 64 are typically nonmaximal, confirmed by the def_n calculations above.

Further intuition about ApEn and def_n can be obtained by reviewing binary sequences of lengths 5 and 6, a comparison of two binary sequences of length N = 20, and the first N digits in the binary and decimal expansions of e, π , √2 , and √3 (1, 16).

Lastly:

1. For a sequence , , i.e., for each entry j;

2. for a finite sequence , —i.e., the sequence of digits of written in reversed order;

3. denotes the length of sequence ;

4. denotes the concatenation of sequences and .

Essential Previous Results.

The central result of our paper (2) was a theorem that provides explicit rules for concatenating maximally irregular sequences of nondecreasing length such that the limiting infinite sequences are normal numbers, indicated next. This also utilizes a wrap-around version of ApEn, denoted ApEn_w, in which we consider sequences of length N in a cyclical arrangement, and (analogous to ApEn) we define N-wr-irregular sequences. One essential aspect of the proof was the demonstration that for any ε > 0 and positive integer k, there exists (explicitly calculated) N_k,ε such that for all N > N_k,ε, the N-irregular and N-wr-irregular sequences are all Besicovitch (k, ε)-normal (7).

Theorem Pr-2.

(ref. 2). Define the base-2 sequence , with Lt(v_i) a nondecreasing integer-valued function of i. Let . If for all i, (i) v_i is either a Lt(v_i)-irregular or a Lt(v_i)-wr-irregular sequence, (ii) lim _i→∞Lt(v_i) → ∞, and (iii) nLt(v_n) = O(L_n), then is normal in base 2.

The length restrictions on concatenates imposed by Theorem Pr-2 allow for substantial variation in sequence structure, and accordingly, a means to produce large collections of normal numbers, since diverse classes of functions f(i)≔Lt(v_i) satisfy the conditions of the theorem, including both polynomial and slowly varying functions.

Below we require a collection of highly irregular normal numbers. We produce a family of such numbers by controlling the length function in the concatenations of finite maximally irregular sequences to be the nearest even integer at or below the n-fold iterate of log(i) for large i, a very slowly growing mapping.

Definition 4:

Fix n. Define Define for N≥Exp_Iter-n(1); f_n(N)≔0 otherwise. Then define g_n(N) as the greatest even integer ≤ f_n(N), and F_IterLog-n(N)≔ max (6, g_n(N)).

Now for all i, select a maximally irregular sequence v_i of length F_IterLog-n(i). Let , where w_m = v₁ ∨ v₂ ∨ …v_m. By Theorem Pr-2, Seq(F_IterLog-n) is normal in base 2 (2). Furthermore:

Theorem Pr-3.

(ref. 2). Define . Then for , for all sufficiently large N, .

Remark 1:

The Seq(F_IterLog-n) provide a nearly “best possible” class of normal numbers, in terms of rapidity of convergence of lim sup _N→∞def₀ to 0. It is easy to show that for any normal , must infinitely often be at least as large as the order of 1/N² (ref. 2, p. 10372). By comparison, for Seq(F_IterLog-n) is bounded above by 1/N² times a function that can be chosen to increase arbitrarily slowly (2).

We also produce large classes of normal numbers via a perturbation strategy:

Theorem Pr-4.

(ref. 1). Assume we are given a binary normal number , a binary sequence , and f: Z⁺ → R⁺ such that f(N)/N → 0 as N → ∞. If {the number of i ≤ N such that u(i) ≠ v(i)} ≤ f(N) for all N, then is a binary normal number.

Fig. 1 of ref. 2 illustrates highly pronounced differences among several normal numbers in the asymptotic rates of convergence for , based on constructions derived from Theorems Pr-2–Pr-4.

The Peculiarity of BinChamp

Denote the binary analogue of Champernowne’s number as BinChamp≔.11011100101… . This sequence of binary digits written in ascending order is known to be a binary normal number, by virtually the same proof as Champernowne produced to show normality for the base-10 sequence (5). Indeed, it is generally viewed as the canonical constructive example of such a number. Yet, as noted previously (2), there is a pronounced local bias of excess 1s in BinChamp; e.g., . We now report the following curious feature:

Theorem 1.

There are always more 1s than 0s in any initial segment of BinChamp—i.e., in the first N digits for any N.

Proof Sketch.

It is straightforward to establish a local excess of 1s at natural breakpoints, by observing that the subsequence comprising {2ⁱ,2ⁱ + 1,…,2ⁱ⁺¹ - 1}_base2 is composed of the concatenation of the 2ⁱ blocks of all possible i-tuples of 1s and 0s, each headed by 1, thus contributing a local excess of 2ⁱ. We then apply a relatively straightforward induction argument to bound local intermediate fluctuations, showing that any counterdirectional fluctuations vary more slowly.

If we map u≔BinChamp to a sequence of 1s and -1s via , and consider the partial sums , the above theorem tells us that S_N is positive for all N. This implies that BinChamp, despite being a normal number (with asymptotic frequency for both 1s and 0s of ½), should not be used as a model for a realization of fair (“random”) coin tossing. Symmetric binary one-dimensional random walks are recurrent (21), with the normative properties of an infinite number of sign changes (zero crossings) and returns to {0} for S_N, while S_N specified from BinChamp lacks these attributes. This peculiarity of BinChamp is vividly illustrated in Fig. 1, in which BinChamp is contrasted with output (“RNG”) derived from Knuth’s (updated) recommended portable random-number generator (13). This latter algorithm, based on lagged Fibonacci sequences with subtraction, produces “random” integers between 0 and 2³⁰, from which we define a binary projection by W_i = 1 if X_i/2³⁰ > 1/2, W_i = 0 otherwise.*

In Fig. 1, two differences between BinChamp and RNG are glaringly apparent. First, in Fig. 1A we see that the deviation of BinChamp from 0 is orders of magnitude larger than that for RNG. We can quantify this—in ref. 2, p. 10371, we showed that for , . This convergence rate for def₀(N) thus measures the very slow rate at which the bias of excess 1s in BinChamp decreases towards asymptotic equidistribution. In contrast, RNG appears to be a “typical” random sequence that, to first order, satisfies the Law of the Iterated Logarithm (LIL). For sequences that satisfy the LIL, we showed (2) that , which converges to 0 orders of magnitude more quickly than does the BinChamp rate, consistent with Fig. 1A. Second, we note the stark contrast between the absence of zero crossings for BinChamp and the recurrent nature of RNG, manifested here in 601 zero crossings of RNG for N ≤ 400,000 (most apparent in Fig. 1B and in a close-up view in SI Appendix, Fig. S1).

Furthermore, the theory of symmetric random walk not only establishes an infinite number of zero crossings (for S_N), but actually provides an explicit normal approximation for the distribution of the number of such zero crossings prior to N (which qualitatively agrees with our count for the RNG for N ≤ 400,000). More generally, the theory produces an important distributional requirement, an explicit arc sine law for sojourn times and zero crossings.

Fig. 1.

Number of 1s − number of 0s in first N digits of (i) BinChamp, .11011100101… (binary numbers written in ascending order) and (ii) a “typical” binary pseudorandom sequence denoted RNG (see text for specification): (A) full range; (B) y-axis absolute values ≤ 750. Note that this difference is > 0 for all N for BinChamp (proved in the text), in contrast to the large number of zero crossings for RNG. Also observe the magnitude difference in variation from 0 between these two series. Both the absence of sign changes and the relatively huge deviations from centrality vividly illustrate the atypicality of BinChamp, the canonical constructive example of a binary normal number, yet also highlight the need for quantitative technology to delineate normal numbers into subclasses of different types.

Biased Normal Numbers

We next construct biased (more colloquially, ‘tilted’) binary normal numbers, and then partition them into a continuum of distinct subclasses by the rate of convergence of lim to 0.

Theorem 2.

Pick any function g: R⁺ → R⁺ such that (i) lim _x→∞g(x) = 0; (ii) is nondecreasing, and diverges to ∞ as x → ∞; and (iii) there exists n such that for , when g is restricted to Z⁺, . Then

(A) we can construct a binary normal sequence such that .

Moreover, if g is a regularly varying function, then

(B) we can construct a binary normal sequence such that , i.e.,

and (C), we can do (B) such that is ultimately biased, i.e., there exists some N_max so that for all N > N_max, .

Corollary 2.1.

Assuming the conditions in Theorem 2, if g is regularly varying, we can construct a biased binary normal sequence (i.e., with for all N) such that .

Proof:

In the construction above, upon initial specification of , replace all 1s with 0s for all N ≤ N_max. This change to a finite initial segment neither affects the normality nor the asymptotic convergence rates of .

Thus we can construct a vast range of distinct classes of biased normal numbers, delineated by the rate of convergence of def₀ to 0, for which any initial segment always has more 0s than 1s (or conversely, more 1s than 0s). Observe that all functions in the parametrized families g_β(x) = x^-β and h_β(x) = (log x)^-β, for 0 < β < 2, are regularly varying and satisfy conditions (i) through (iii) in Theorem 2. Thus we can produce biased normals with arbitrarily fast (possible) convergence rates (Remark 1), and conversely, arbitrarily slow rates, by respectively selecting β for g_β(x) to be as close to 2, or to 0, as required.

Two-Sided Rates of Convergence

Theorem 2 implies that we can construct collections of sequences with a range of rates of convergence of def₀ to 0. However, the sequences satisfying Theorem 2 are all biased. We wish to construct more symmetric normal sequences that asymptotically behave like specified functions both above and below the centered midline, which we do next. Although in the context of the general theory the def_n functions are most natural in quantifying the difference from maximal irregularity, for the next theorem we distinguish between above and below the midline, and accordingly, cast this theorem in an EXC_0,1 formulation, which provides the requisite detail.

Theorem 3.

Let Ψ be the set of all regularly varying functions k: R⁺ → R⁺ such that (i) lim _N→∞k(N)/N → 0; (ii) k(N) is nondecreasing, and lim _N→∞k(N) → ∞; and (iii) there exists some n such that lim _N→∞k(N)/F_IterLog-n(N) → ∞. Let functions g, h be members of Ψ. Then we can construct a binary normal sequence such that

Proof Sketch.

We apply a “zig-zag” procedure, in which we alternate locally positively and negatively biased excursions of exponentially increasing lengths, with the local unidirectional excursions specified by Theorem 2 to achieve the stated convergence rates, and with intermediate recentering to 0 (via negation, in reversed order) to facilitate estimates.

One subset of C-random sequences merits particular mention. The LIL, an a.s. property of sums of independent, identically distributed (i.i.d.) random variables, is a cornerstone of modern probability theory. It refines both the Strong Law of Large Numbers (SLLN) and the Central Limit Theorem (CLT), providing a sharp (normalizing) factor for maximal and minimal values of partial sums. The LIL, interpreted for individual sequences on {0,1}, is equivalent to which identifies a narrowly specified subclass among all normal numbers.

Define , and observe that lil(x) is a member of Ψ defined in the statement of Theorem 3. Then we deduce

Corollary 3.1.

Let g(x) = h(x) = lil(x). Then any sequence constructed according to the procedure specified in Theorem 3 is a normal number that satisfies the LIL.

Hence we now have algorithmically constructed examples of normal numbers that satisfy the LIL. Of note, we are unaware of the existence of any explicitly constructed normal numbers that were previously proved to satisfy this law. Conversely, by applying Theorem 3 to any functions g(N) or h(N) that are not asymptotic to lil(N), we can produce many distinct families of normal numbers that do not satisfy the LIL.

Finally, note that Theorems 2 and 3 are remarkably general. First, assuming regularly varying functions, Theorem 2 actually yields convergence, i.e., a lim (not only a lim sup) result. Also, when the conditions of the theorems are viewed in the language of EXC, as in Theorem 3, they are seen to be very modest–(normative) regular variation, k(N) monotonically divergent to infinity at rate o(N), plus a technical condition (iii) that is almost always satisfied. Indeed, given regular variation and conditions (i) and (ii), condition (iii) is violated only by relatively pathologic counterexamples (SI Appendix).

A Normal Number with Contrasting Convergence Rates for Singletons and Pairs

Our delineation of normal numbers above focuses on the def₀ rate, which already provides a considerable means to stratify normal numbers. We next present a normal sequence denoted by Seq(0_vs1) that, subject to the constraint of normality, has arbitrarily fast convergence to equidistribution of singletons (given by def₀). Yet Seq(0_vs1) also has arbitrarily slow convergence of pairs (given by def₁), manifesting regularities in pairs, in particular systematic bias against (1,1). Recast, the distribution of singletons in initial segments of Seq(0_vs1) will hover extremely closely to ½ 1s and ½ 0s, yet pairs {0,0}, {0,1}, {1,0}, and {1,1} will be much less equidistributed than that for a typical binary infinite sequence.

There are two keys to this construction. First, we concatenate maximally irregular sequences of very slowly increasing lengths to ensure that single digit distributions hew closely to maximal equidistribution. The second, decisive theme is an intermediate negation step that prohibits occurrences of {1,1} and {0,0} at liaison (interface) blocks.

I. Specification. Recalling Definition 4, define G_IterLog-n(N)≔greatest power of 2 ≤ F_IterLog-n(N). For each i, define v_i as a periodic portion of a full-cycle deBruijn sequence of length G_Iterlog-n(i) such that v_i starts and ends with 1. Then let Seq(G_Iterlog-n)≔ lim _m→∞w_m, where w_m = v₁ ∨ v_1-neg ∨ v₂ ∨ v_2-neg… ∨ v_m ∨ v_m-neg. This is our “interesting” sequence, also denoted by Seq(0_vs1).

Note that Seq(0_vs1) commences as 1001011010010110…; typical 8-block concatenation regions are 11000101001110101100010100111010… Observe a relative shortage of {1,1} and {0,0} blocks. For the first 16 terms, there are 8 1s and 8 0s (Seq(0_vs1) is perfectly equidistributed in singletons), yet 2 occurrences each of {1,1} and {0,0}, with 6 occurrences of {1,0} and 5 of {0,1}.

II. Normality. Since deBruijn sequences are maximally irregular, negations of maximally irregular sequences are also maximal, and the lengths of successive terms are nondecreasing, it follows from Theorem Pr-2 that Seq(G_Iterlog-n) is normal in base 2.

III. def₀ estimate. Define . Then for , for all sufficiently large N, . Thus these Seq(G_Iterlog-n) [like their Seq(F_Iterlog-n) counterparts) provide a nearly best possible class of normal numbers, insofar as rapidity of convergence of lim sup _N→∞def₀ to 0.

IV. def₁ estimate. We observe a wild contrast between singleton and pairwise dynamics, relative to the normality constraint.

A. Bias. Choose any m, with Lt(v_m) = 2^k. Then

[2]

Thus if k = 2, the fraction of occurrences is ≤ 1/8; if k = 3, the fraction is ≤ 3/16.

The essential observations are that for all i ≤ m, (i) v_i ∨ v_i-neg has (Lt(v_i)/4) - 1 occurrences of {1,1} strictly within v_i, and Lt(v_i)/4 occurrences of {1,1} strictly within v_i-neg; and (ii) crucially, there is never a liaison {1,1} from the last digit of v_i to the first digit of v_i-neg (this is always {1,0}), nor from the last digit of v_i-neg to the first digit of v_i+1 (this is always {0,1}). Observation (i) follows from the property that the deBruijn sequence v_i has Lt(v_i)/4 occurrences of {1,1}, when viewed in a cyclical setting, but “loses” one (the wrap-around) occurrence in a direct count, since each v_i starts and ends with 1. Observation (ii) follows from the fact that all v_is start and end with 1, and all v_i-negs start and end with 0.

It then follows readily from Eq. 2 that the frequency

[3]

Therefore the bias against {1,1} converges to 0 extraordinarily slowly as a function of m, namely, O(1/ log(log(… log(m)))).

B. We then apply a frequency-based equivalent formulation of ApEn (Theorem S1) to Eq. 3 to produce the following upper bound: For all m suitably large, , which in turn readily allows us to deduce that for all large N, at concatenate endpoints along {w_m}.

Finally, by the specification of F_IterLog-n(N) as approximately , we can select “n” in IterLog-n so that def₁(Seq(0_vs1)^(N)) can be made to converge to 0 arbitrarily slowly.

Thus we have shown that there can be remarkable contrast between asymptotic characteristics of blocks of differing lengths for normal numbers. Moreover, we have produced another example of a biased normal number, here in pairwise blocks.

Additionally, note that we critically require the intermediate negations to achieve the desired contrast. If we study v₁ ∨ v₂ ∨ v₃ … with the v_i’s defined as in this construction, the limit is normal, with both def₀ and def₁ converging extremely rapidly to 0.

Perspectives

1. Our primary objective was to produce constructive methods to generate diverse sets of normal numbers. Of course, many of these sequences are very predictable, reinforcing that (approximate) equidistribution, irregularity, and normality are orthogonal concepts to algorithmic complexity.

The pronounced differences among normal numbers reinforce the perspective that grouping all normal numbers into a single asymptotically equidistributed category is often inadequately nonspecific. In particular, this highlights a critical limitation of classical probability theory—each of the normal numbers constructed above would be candidate members of a single “RANDOM SEQUENCE” category. However, rates of convergence (to equidistribution) of the def_n functions, or the presence and extent of a bias are now seen as well-defined characteristics of a specified sequence. This is especially important in virtually any application, in which we typically utilize the initial segment of a putatively random sequence, and thus should want to assess the degree to which these segments are not maximally irregular.

A further, yet arbitrary, question concerns the choice of constraints beyond normality that one might impose to designate a sequence as “random.” For pseudorandom finite sequences, there are usually a relatively short list of standard properties, e.g., (i) well-distribution relative to arithmetic progressions and (ii) small multiple correlations—see Golomb (14) and Knuth (13) for detailed discussions. However, for infinite sequences, normality implies that most if not all of these properties are satisfied, including (i) and (ii) (22). For infinite sequences, beyond normality, researchers often mandate that “random” sequences satisfy the a.s. laws of probability theory, in particular the LIL. Such mandates, however, lead to conundrums; we now see that sequences satisfying the LIL are less asymptotically equidistributed than a continuum of explicitly constructed normal numbers. Additionally, distributional requirements including, e.g., the arc sine law for sojourn times for classical random walks noted above have not, to our knowledge, been tendered as a necessary attribute for “randomness,” and such would likely further constrain sets of candidate sequences as, for example, in a Kollektiv in the sense of von Mises (23).

Nonetheless, probability and statistical theory retain vast utility, but now properly viewed as a tool to discern distributional characteristics of generic ensemble populations, as opposed to specific attributes of individual sequences, and critically, with the aforementioned caveats on the term generic.

2. In ref. 1, we define the deficit from maximal equidistribution as the maximum of the functions for m ≤ m_crit(N). Thematically similar is the normality measure of pseudorandomness for finite sequences (22), in which a maximum is further taken along all initial subsegments of . In either case, a highly irregular or pseudorandom sequence is one in which the maximum is small. However, as illustrated in the juxtaposition of def₀ and def₁ for Seq(0_vs1), we now see that by considering each block length (def_n) distinctly, rather than aggregating them into a single composite measure, we provide a more complete understanding of asymptotic sequence variation. To reinforce this point, for Seq(0_vs1), on finite initial segments, both De and would in effect monitor and identify the lack of pairwise equidistribution, with the nearly perfect equidistribution of singletons “under the radar” or unquantified.

Furthermore, note that the verification of many classical limit theorems in probability theory, including the SLLN, CLT, and LIL, reduces to evaluation of the distributional characteristics of normalized summands S_N (of the first N sequence elements). In particular (since partial summands S_N are directly and easily related to def₀ via Definition 3 and Eq. 1), this can be recast in terms of single-digit variation from centrality. Thus these limit theorems have essentially nothing to say about higher-order block dynamics, i.e., def_n for n≥1.

These points highlight the considerable utility that the collection of higher-order def_n functions can provide. More broadly, we anticipate that the study of distributional characteristics of blocks of length > 1 by def_n for n≥1 will result in new findings in numerous contexts.

Finally, these considerations raise serious questions about exactly what is (or should be) meant by a normal number-based model of fair coin tossing as realizations of i.i.d. binary random variables. Higher-order “atypical” block dependencies or biases among normal numbers can emerge, even if classical 1-block distributions are nicely behaved. We will probe this much further in a forthcoming paper.

3. Historically, there has been a significant dichotomy between finite and infinite length approaches to the development of highly irregular or nearly equidistributed sequences. Finite-sequence developments have centered on the study of linear congruential random-number generators (13), and also on linear shift registers, with primitive (irreducible) polynomials and k-ergodic primes essential objects of interest, as are algorithms to produce full cycles (deBruijn sequences) of length 2ⁿ. More recently, long, highly irregular sequences have been demonstrated from constructions based on the Liouville function and the Legendre symbol (22), and by Alon et al. (24), via the concatenation of multiple copies of specified realizations of primitive polynomials over a finite field. However, these methods have not been directly extendable to an infinite sequence setting. In contrast, infinite length constructions towards producing normal numbers have focused on theorems that either assure Besicovitch (k, ε)—normality for almost all concatenates, for any k and ε, or carefully estimate and control exponential sums as indicated above.

A central perspective above is that we provide a natural liaison between finite- and infinite-sequence construction and evaluation, both via Theorem Pr-2 (construction), and via ApEn definitions in both settings (evaluation), the infinite case being a limit of the finite one. This link allows us to view the finite-sequence results in a larger framework, and also enhances the utility of identifying efficient algorithms for (nearly) maximally irregular finite sequences.

Finally, one can extend Theorem Pr-2 to variants in which we relax the maximality of irregularity condition to a near-maximality of irregularity of concatenates, so long as the proximity to maximal ApEn varies suitably with concatenate length. These allow for broader collections and numbers of the concatenates (some of which may be considerably easier to produce than true maximal elements), leading to further classes of normal numbers.