Irregular arrays and randomization

Abstract

Although they lie at the conceptual core of a wide range of scientific questions, the notions of irregular or “random” arrangement and the process of randomization itself have never been unambiguously defined. Algorithmic implementation of these concepts requires a combinatorial, rather than a probability-theoretic, formulation. We introduce vector versions of approximate entropy to quantify the degrees of irregularity of planar (and higher dimensional) arrangements. Selection rules, applied to the elements of irregular permutations, define randomization in strictly combinatorial terms. These concepts are developed in the context of Latin square arrangements and valid randomization of them. Conflicts and tradeoffs between the objectives of irregular arrangements and valid randomization are highlighted. Extensions to broad classes of designs, and a diverse range of scientific applications are indicated, including lattice-based models in physics and signal detection in seismology and physiology.

The idea of random arrangements of points in planar arrays arises in a vast diversity of scientific problems. Lattice-based models in physics (1), the statistical design of experiments (2–4), image and pattern recognition (5, 6), and forest ecology (7) are illustrative of the myriad contexts in which this concept is prominent. Yet despite numerous application-specific definitions of “too regular” (8–10) or “sufficiently irregular” (11–14), the scientific literature across multiple fields remains silent on producing an algorithmic formulation of the extent or degrees of irregularity of planar arrangements, for either finite or for infinite arrays. This omission is closely linked to the ambiguity of the expressions “picking at random” and “randomization.” A precise formulation of these notions is combinatorial, rather than probability-theoretic, in nature (15). Indeed, a framework for clarifying what is meant by both “regular arrangement” and “randomization” can be based on a family of approximate entropy measures, whose one-dimensional versions already have been effectively used to grade the irregularity of numerical sequences (15, 16).

The primary purpose of this paper is to define and apply vector versions of approximate entropy (ApEn) that can be used to grade the irregularity of planar or higher dimensional arrangements. After providing an initial core framework, we proceed via an in-depth discussion of Latin square arrangements—i.e., an n-row × n-column arrangement of n distinct symbols where each symbol occurs once in each row and once in each column—and Fisher’s notion (14) of valid randomization set. This context is one of the simplest in which to illustrate the primary issues that are generic to an explicit algorithmic formulation of the concept of random arrangement in two or more dimensions. Furthermore, reevaluation of a formulation of randomization, with its genesis in Fisher’s early specification of design of experiments (2, 14, 17), is of critical importance, because applications of this methodology have proliferated throughout many disciplines, including the physical and social sciences, and medicine. The ambiguity of the current formulation, as it has developed over more than 75 years, can be removed by recasting it in strictly combinatorial terms. Such a reformulation is the secondary purpose of this paper. Finally, we introduce a select set among the vast range of potential applications of vector-ApEn, as indicated in Notes and Applications, for which use of vector-ApEn is likely to yield a substantial payoff.

For Latin squares, nearest-neighbor designs, and general block designs, we associate sets of allowable arrangements with finite maximally irregular sequences and specified selection rules, thereby making precise the diverse notions of “valid randomization” (11, 17–19). We also identify conflicts and tradeoffs between the objectives of valid randomization and high degree of irregularity of experimental arrangements. Furthermore, it is key to note that we highlight N × N Latin squares, N ≤ 9, not only for conceptual clarity and computational feasibility, but also because these sized arrays are widely used in myriad practical settings.

In much of the core discussion, we focus on a binary or yes/no determination to decide whether Latin squares are sufficiently irregular to be considered as potential candidates for a valid randomization set. Already, this focus provides considerable evaluatory capability, as it adds a dimension to previously considered perspectives. Yet further utility of vector-ApEn will be demonstrated for medium to large arrays, where we would typically use this technology not only to determine (nearly) maximally irregular elements, but especially, to grade and thereby distinguish among the large class of remaining arrays. Thus one will be able to quantify oftentimes subtle or “insidious” structural differences among arrays, even where clear features or symmetries are far from evident, of potential consequence in several applications indicated below.

Irregularity of Planar Arrangements.

There are manifold options for making precise the concept of degrees of irregularity in the plane, as a consequence of both diverse geometrical configurations and projected applications. A readily adaptable basic building block that we use is a vector adaptation of approximate entropy (15, 20).

Definition 1.

Given a finite two-dimensional integer lattice A, a finite set S, a function u:A → S, a non-negative integer m, a positive real number r, and a specified vector direction v, for all points i in A, define the block x_v(i) = (u(i), u(i + v) … , u(i + (m − 1)v)). Blocks that exceed the boundaries of A are either, depending on the application, excluded from consideration or are defined via a wraparound condition.§

Define d(x_v(i), x_v(j)) = max_k=1,2,...,m(|u(i + (k − 1) v) − u(j + (k − 1)v)|). Let A_m,v := {all points i in A for which a length m block in direction v starting at i is included for consideration}.

Then let C_i^m(r) = (number of j such that d(x_v(i), x_v(j)) ≤ r)/|A_m,v|), where |A_m,v| is the cardinality of A_m,v. Define

Then let vector-ApEn_v(m, r)(u) = Φ_v^m(r) − Φ_v^m+1(r), m ≥ 1; and vector-ApEn_v (0, r)(u) = −Φ_v^l(r).

In this paper we restrict attention to integer alphabets for S, and set r < 1 as a measure of resolution. With this choice of r, we are monitoring precise matches in the blocks x_v(i) and x_v(j). Thus, with this restriction at hand we suppress the dependence of ApEn on r, denoting vector-ApEn_v(m, r)(u) as vector-ApEn_v (m)(u). Vector-ApEn_v (m)(u) compares the logarithmic frequency of matches of blocks of length m (for m ≥ 1) with the same quantity for blocks of length m + 1. Small values of vector-ApEn imply strong regularity, or persistence, of patterns in u in the vector direction v, with the converse interpretation for large values.

The vector direction v designates arrangements of points on which the irregularity of u is specified a priori to be of particular importance. For example, if v = (0, 1), then vector-ApEn measures irregularity along the rows of A and disregards possible patterns, or the lack thereof, in other directions.¶ v = (1, 0) focuses on column irregularity; and v = (1, 2) or (2, 1) or (−1, 2) emphasizes knight’s move (as in chess) patterns. In many applications, it is necessary to guarantee irregularity in two or more directions simultaneously. This requires evaluation of vector-ApEn for a set V of designated vectors. For example, simultaneous row, column, and diagonal irregularity assessment entails calculation of vector-ApEn for all elements v in V = {(1, 0); (0, 1); (1, −1); (1, 1)}. Let 𝒰 be a restricted class of functions u: A → S, including the possibility that we might choose 𝒰 = {all functions with domain A and range S}. For a given 𝒰 and V we formalize the idea of a maximally irregular function, U⁺. To this end, write V = {v₁, v₂, … , v_k}, and let U_i = {u: max_u∈𝒰 vector-ApEn v_i(m)(u) is attained for m = 0, 1, 2, … , m_crit}. Then form the intersection

Definition 2.

The elements u⁺ of U⁺ are denoted maximally irregular functions with respect to 𝒰 and V.

The determination of m_crit will depend on the size and shape of A, the choice of vector, v, and constraints defining the elements of 𝒰. For the small-sized square arrays illustrated below, we restrict to m_crit = 1 in the primary text, which assesses replicability of pairs of contiguous points along the specified vector direction. In the endnotes, we discuss the enhanced refinement offered by choosing m_crit ≥ 2.

We now consider two sets of Latin square examples to develop intuition for vector-ApEn. As well, the importance of studying Latin squares in the broader context of reevaluating Fisher’s randomization and experimental design approaches is underscored by the following quotation:

“ ‘What would you do,’ I had asked, ‘if drawing a Latin square at random for an experiment, you happened to draw a Knut-Vik (Knight’s move) square?’ Sir Ronald said he thought he would draw again and that, ideally, a theory explicitly excluding regular squares should be developed.” (L. J. Savage to R. A. Fisher, 1952, p. 88, ref. 21).

Quite evidently, Fisher himself was well aware of considerable differences among Latin squares insofar as the (extent of) persistence of internal features, although he lacked a technology to quantify this recognition.

Formally, we provide the following specification: Let A be a square array of side N, i.e., a lattice with elements i = (i₁, i₂), where 1 ≤ i_j ≤ N for j = 1, 2. Let S = {1, 2, … , N}, and 𝒰 := {u such that u(i) defines an N × N Latin square}.

Example 1.

Consider the following four 4 × 4 Latin squares.

For A, vector-ApEn_(1,0)(1) = vector-ApEn_(0,1)(1) = 0; for B, vector-ApEn_(1,0)(1) = vector-ApEn_(0,1)(1) = 0.637; for C, vector-ApEn_(1,0)(1) = 0.637, and vector-ApEn_(0,1)(1) = 1.099; and for D, vector-ApEn_(1,0)(1) = vector-ApEn_(0,1)(1) = 1.099. These calculations manifest differing extent of feature replicability in the (1, 0) and (0, 1) directions, with A quite regular in both directions, B intermediately irregular in both directions, C maximally irregular in rows, yet intermediate in columns, and D maximally irregular in both rows and columns. Alternatively, in A, e.g., in rows, there are three occurrences each of four pairs [(1, 2), (2, 3), (3, 4), and (4, 1)], and no occurrences of the other eight possible pairs. In B, in rows, four pairs occur twice [(1, 2), (2, 1), (3, 4), and (4, 3)], whereas four pairs occur once [(1, 4), (2, 3), (3, 2), and (4, 1)]. In D, in rows, each of the 12 pairs (i, j), 1 ≤ i, j ≤ 4, i ≠ j, occur precisely once. (Similar interpretation follows readily for columns.)

Example 2.

A Knut-Vik (KV) square, referred to above, is defined as a Latin square such that all cells with the same symbol can be traversed by a series of knight’s moves (as on a chessboard) without visiting cells with other symbols. The following KV square, denoted KV1, is generated via a very simple procedure that results in considerable, visually apparent regularities: each row, cyclically, is moved down, then to the right (forward) two places, in a wraparound fashion:

Wraparound vector-ApEn, where we wraparound both for rows and columns (thus forming a “lattice point torus”) vividly discloses the consistency of the regularities, with wrap vector-ApEn_(1,0)(1) = wrap vector-ApEn_(0,1)(1) = wrap vector-ApEn_(1,2)(1) = 0. Thus we quantify the striking persistence of subblocks within this square in three distinct vector directions—rows, columns, and knight’s move (1, 2).

Next, we introduce the restricted classes of Latin squares according to the following.

Definition 3.

An n × n Latin square u(i) is called row complete if the n(n − 1) ordered pairs (u(i₁, i₂), u(i₁, i₂ + 1)) are all distinct. It is called column complete if the n(n − 1) ordered pairs (u(i₁, i₂), u(i₁ + 1, i₂)) are all distinct. A Latin square is complete if it is both row complete and column complete. A diagonal square (22) has u(1, 1), u(2, 2), … , u(n, n) distinct and u(1, n), u(2, n − 1), … , u(n, 1) distinct.

These Latin squares arise as the maximally irregular arrangements (via vector-ApEn, with m = 1) for the designated vector families indicated below.

There are clear limits to the restrictions in kinds of irregularity that can be simultaneously imposed. For example, for 4 × 4 squares, because C ∩ D = ∅, we cannot simultaneously maximize irregularity in row, column, and both diagonal directions throughout the square. However, further restricting either C or D by less stringent constraints can lead to useful arrangements that are also interpretable as irregular. For example, consider C ∩ {u: u is diagonal and pairs in adjacent cells in each diagonal direction occur with equal frequency on opposite sides of the longest diagonals}. This restriction defines the class of completely balanced polycross designs (23, 24), a subset of C ∩ {u: vector-ApEn_(1,1)(m)(u) = vector-ApEn_(1,−1)(m)(u), m = 0, 1}.

It might seem, at this point, that the complete squares or the polycross designs—depending on the application—should qualify for the interpretation of random 4 × 4 Latin squares. Although this appears entirely appropriate based on an irregularity criterion, the essential point, observed by Bailey (25), is that neither of these families forms a valid randomization set, the definition of which we now recall.

In the setting of experimental design Fisher (14) defined a Latin square subject to its classical (26) combinatorial formulation and a process of randomization. In the randomization step a square is selected from a set of Latin squares having the property that every pair of cells, not in the same row or column belongs equally frequently, across the set, to the same symbol. Such a collection of Latin squares is referred to as a valid randomization set.

For this property to hold, a set S must have the property that every pair of cells lying in different rows and columns has the same symbol in exactly 1/(n − 1) of the squares in S. (For a superb discussion of valid randomization sets for quite general designs, see Bailey and Rowley; ref. 18.) It is then straightforward to observe that neither the set of 48 complete 4 × 4 Latin squares nor the set of 24 polycross designs satisfy this property, for n = 4.

Latin Squares.

Latin squares are among the simplest row-column experimental designs. They have a history of explicit utilization (23, 27) dating to 1788 and appear implicitly in card puzzles as early as 1624. Simplicity, range of applicability (23), and the fact that these designs are at the core of the earliest attempts to define random arrangements make them ideally suitable for illustrating diverse aspects of irregularity of planar arrangements.

Quasi-Complete Squares.

For small orders (e.g., order ≤ 9) completeness is an intuitively reasonable minimal criterion for implementation of the idea of maximally irregular Latin squares. However, there are no complete squares of order 3, 5, and 7 [Owens (28) identified all complete squares of order 4, 6, and 8]). To satisfy the need for irregular squares for orders where complete squares do not exist, Freeman (29) proposed the use of quasi-complete squares. By this we mean a square of order n in which the n(n − 1) adjacent pairs of elements that occur in the rows include each unordered pair of distinct elements exactly twice; similarly for the columns. Unlike complete Latin squares, quasi-complete squares exist for all finite orders.

Within the class of quasi-complete squares, the most irregular, in the sense of minimal repetition of specific ordered pairs in adjacent cells in rows and columns, are those squares that best approximate complete squares. These squares are members of U⁺ = U₁ ∩ U₂, where U_i = {u: max_u∈𝒰 vector-ApEn v_i(m)(u) is attained for m = 0 and m = 1}, for i = 1, 2. V = {v₁, v₂} = {(1, 0); (0, 1)}, and 𝒰 = quasi-complete squares.

There is considerable variation in degrees of irregularity among quasi-complete squares of given order. For example, let

𝒰_A and 𝒰_B both are quasi-complete squares of order 5; however, 𝒰_A is more irregular than 𝒰_B, with vector-ApEn_(1,0)(1) (𝒰_A) = 1.040 > vector-ApEn_(1,0)(1) (𝒰_B) = 0.693, and vector-ApEn_(0,1)(1)(𝒰_A) = 1.040 > vector-ApEn_(0,1)(1)(𝒰_B) = 0.693, manifesting the observation that 10 ordered pairs occur exactly once in each of rows and columns of 𝒰_A whereas no ordered pairs occur only once in either rows or columns of 𝒰_B. From an experimental design perspective, maximally irregular quasi-complete squares allow the criterion of directedness (29)—i.e., the ordered pair (p, q) is interpreted as different from the ordered pair (q, p)—to be well-approximated, while respecting a size restriction of odd order for which there are no complete squares.

A natural criticism of the interpretation of quasi-complete, and particularly of complete, squares as being very irregular is the fact that many of them have striking symmetry or other internal similarity properties. For example, a Latin square of order 2m is said to have σ-symmetry (23) by rows if u(i, j) + u(i, 2m + 1 − j) = 2m + 1 for all coordinates (i, j) (similarly for columns). All 24 of the 4 × 4 completely balanced polycross designs are σ-symmetric by rows and columns. They are also magic squares in the sense that row, column, and diagonal entries all sum to 10. Indeed, diagonal Latin squares of any order are also magic squares.

Many of the remaining 4 × 4 complete squares, as well as numerous 5 × 5 quasi-complete squares are symmetric about the main diagonal. The central point is that minimizing repetition of pairs of adjacent entries in rows and columns, one natural notion of irregular, is nearly incompatible with a simultaneous objective of eliminating symmetries in Latin squares of orders 4 and 6.

Many complete Latin squares of orders 8 and 10 have no σ-symmetry (24); e.g., of the 88 standardized complete squares of order 8, 24 have no σ-symmetry; of the 5,488 standardized complete squares of order 10, 3,660 have no σ-symmetry. We propose that candidates for a valid randomization set of order 2m should, minimally, be complete, lack σ-symmetry, and have no diagonal or total symmetry. Among quasi-complete higher odd-order squares, elimination of symmetries also should be a minimal prerequisite for such selection.

Another internally persistent feature prominent among complete squares is the presence of Latin subsquares. For squares of order 4 and 6, this is unavoidable, because every Latin square of these sizes has proper Latin subsquares. However, for squares of order 8, there are exactly three Latin squares that have no Latin subsquares of any size. Given the large number of complete squares of order 8, we must compromise in meeting the multiple objectives of minimal number of Latin subsquares, completeness, and a large set of irregular squares as candidates to comprise a valid randomization set. How this should be, if indeed it can be done to ensure valid randomization sets for squares of order 8 and higher, is an open question.

Mutually Orthogonal Squares.

Definition 4(i). Latin squares L₁ = ||a_ij|| and L₂ = ||b_ij|| on n symbols are said to be orthogonal if every ordered pair of symbols occurs exactly once among the n² pairs (a_ij, b_ij), 1 ≤ i, j ≤ n.

(ii) A set of n-1 pairwise orthogonal Latin squares of order n is called a complete set (CMOS = complete mutually orthogonal system).

Fisher (30) pointed out that a complete set of mutually orthogonal Latin squares (CMOS) is a valid randomization set. To facilitate their use in the design of experiments, Fisher and Yates (FY, ref. 31, 1938 and 1963 editions) published CMOS for squares of orders 3, 4, 5, 7, 8, and 9.¶ As noted at the outset, these are sizes of Latin squares that find very wide practical utilization. Examination of the CMOS in FY (ref. 31, 1963 edition, pp. 88–89) reveals:

(i) For squares of order 4, vector-ApEn_(1,0)(1)(u) = vector-ApEn_(0,1)(1)(u) ≈ 0.637 where u is any member of the CMOS. This is only intermediate irregularity, because vector-ApEn_(1,0)(1)(u) = vector-ApEn_(0,1)(1)(u) ≈ 1.099 for all complete squares of order 4. Furthermore, no 4 × 4 complete square has an orthogonal mate, because they all have no transversals.

(ii) There is one strict KV and one weak KV square** in the CMOS of orders 5 and 7. None of the members of these CMOS are quasi-complete.

(iii) No member of CMOS of order 8 is quasi-complete. Six of the seven squares in the CMOS are magic squares, and the remaining square has identical elements for all entries on the major and minor diagonals. All squares in the CMOS are σ-symmetric by rows.

(iv) For order 9, no squares in the CMOS are quasi-complete.

Hence selection of an element from any CMOS in ref. 31 is guaranteed to yield a square with a more regular arrangement than would be the case if one put a higher priority on irregularity relative to validity. The case of squares of orders 5 and 7 is particularly vexing, because two of the four members (respectively, two of six members) of the respective CMOS are KV. This is in direct conflict with Fisher’s view, expressed in the quotation above (21), that these arrangements are too regular.

For squares of orders 5 and 7, it is possible to meet the twin demands of high irregularity, as defined by quasi-completeness, and validity by CMOS, as demonstrated by Campbell and Geller (32). For example, the following four squares are each quasi-complete, and they form a CMOS.

These squares are equally row and column irregular, with vector-ApEn_(1,0)(1) = vector-ApEn_(0,1)(1) = 1.040 for each square. Again, 10 ordered pairs occur exactly once in each of rows and columns, in each square.

The requirement of irregularity that maximizes the number of distinct ordered pairs in adjacent positions by rows and columns within a CMOS of quasi-complete squares restricts the size of valid randomization sets, i.e., those made up of multiple CMOS. This kind of tradeoff is unavoidable once irregularity constraints are imposed. However, our ability via vector-ApEn to grade the degree of irregularity of squares facilitates the clarification, subject to end-use requirements, of what can be viewed as suitable candidates for a sufficiently large valid randomization set.

“Random” Arrangements by Row and Column Permutations

Fisher and Yates (ref. 31, 1963, 6th ed., p. 24) advise: “To select a square at random from all possible squares of a given size up to 6 × 6 proceed as follows:

(Step 1) Select one of the given squares (ref. 31, pp. 86–87) by using the key numbers printed below each square, selecting a number at random from all possible key numbers. If the key number falls in the second of the two groups, use the conjugate number.

(Step 2) In the case of 3 × 3, 4 × 4, and 5 × 5 squares, permute all rows except the first of the selected square, and all columns. Alternatively, permute all rows except the first and assign the letters to the treatments at random. For 6 × 6 squares permute all rows and all columns at random and then assign the letters to the treatments at random.”

The ambiguity in step 1 that arises from the instruction to “select at random” will be clarified in the next section. However, step 2 remains a problem present in the literature on design of experiments since the first edition of ref. 31 in 1938. The central observation is that without sharp restrictions on the class of allowable permutations, there is no control over the degree of irregularity of the resulting arrangement.

For example, suppose step 1 yields the square

This is square number 2 in FY’s first transformation set of 4 × 4 squares. As noted in Example 1, vector-ApEn_(1,0)(1)(𝒰) = 0 = vector-ApEn_(0,1)(1)(𝒰), quantifying the pronounced regularities that can be seen visually. Applying the permutation (_{1 4 2 3}^{1 2 3 4}) to the rows and (_{4 2 3 1}^{1 2 3 4}) to the columns of 𝒰 yields the intermediately irregular square

which is column complete, with vector-ApEn_(1,0)(1)(𝒰′) = 1.099, yet not row-complete, with vector-ApEn_(0,1)(1)(𝒰′) = 0.637.

Conversely, if the same row permutation is applied, followed by the column permutation (_{1 2 4 3}^{1 2 3 4}), we obtain

which belongs to U⁺ = class of complete squares of order 4.

In Bailey’s unified approach (33) to design of experiments, an example is presented where 5 × 5 Latin squares arise at stage 6 of the design steps, and these are the allowable designs. The very regular arrangement (with vector-ApEn_(1,0)(1) = vector-ApEn_(0,1)(1) = 0)

is displayed as the design to be “randomized” at stage 7. Bailey, appropriately, restricts the allowable permutations to be the transitive ones, and specifically applies the transpositions (2 3) (4 5) to the rows and the cycle (1 4 2) to the columns. The resulting arrangement is

This square is not even quasi-complete, and vis-a-vis our earlier discussion, should not qualify as an adequately irregular arrangement. The open problem implied by these examples is determination of sets of allowable permutations linked to vector-ApEn irregularity constraints.

Randomization.

Although the idea of randomization has been one of the core principles of design and analysis of experiments, it remains a “source of obscurity that has plagued statistics for the past 70 or more years” (ref. 4, p. 141). The source of the obscurity that remains to be resolved is the ambiguity in the words “chance” and “picking at random.” Fisher and Yates (31) provided a laboratory technique for obtaining a Latin square from a valid randomization set. At the critical juncture of actually making a selection, we find the disconcerting sentence, “All random selections that were required in the above procedure were made by the use of two packs of playing cards” (ref. 31, p. 18). The disconcerting element of this procedure is that one is assuming that the playing cards are shuffled into a “random arrangement,” without an elucidation of what a well-shuffled deck is, nor an algorithm for generating one.

Adapting the formulation of maximally irregular sequence of finite length (15) to include maximally irregular permutations‡‡ relative to the lexicographic ordering (1, 2, 3, … , n) allows us to give an unambiguous formulation of the process of randomization. To this end let S be a valid randomization set whose elements satisfy irregularity conditions given by vector-ApEn. Label the elements of S in some specified order 1, 2, 3, … , |S|. Then let 𝔓₀ be the maximally irregular permutations of (1, 2, 3, … , |S|). We denote the elements of 𝔓₀ by π₁, π₂, … π_|𝔓0| where π_j = (π_j (1), π_j (2), … , π_j (|S|)) = permuted elements of {1, 2, 3, … , |S|}. Finally, define 𝔓 to be either 𝔓₀ by itself, or 𝔓₀ augmented by additional nearly maximally irregular permutations so that elements (π₁, π₂, … π_|𝔓|) of 𝔓 satisfy

1

This condition is motivated by the objective of having all Latin squares in S present in 𝔓 with equal frequency, namely, . We now provide

Definition 5.

A valid randomization scheme consists of (i) the delineation of a valid randomization set, S, containing sufficiently irregular squares; (ii) the selection of a permutation π_j ∈ 𝔓 by a selector who is not the investigator and who does not inform the investigator of the choice; (iii) the selection by the investigator of an integer, k, where 1 ≤ k ≤ |S|, denoting the k^th element in the irregular permutation π_j; namely π_j(k); and (iv) presentation to the investigator, by the selector, of the Latin square π_j(k).

This process implements what we interpret to mean “picking at random” from the set S. The selector may be a computer program that chooses the j^th element from an irregular permutation of {1, 2, 3, … , |𝔓|}; however, the investigator should not have control over the selection of π_j. This restriction is designed to eliminate any potential biases by the investigator in a choice of a particular Latin square in S.

It is imperative to contrast the sets S and 𝔓 in the above definition with Fisher’s treatment of CMOS as viable candidates for valid randomization sets. Fisher successfully codified the notion of pairwise independence among a finite set of Latin squares, suggested by the analogous mandate from axiomatic probability theory. However, he failed to act on the observation that the Latin square elements themselves are hardly interchangeable as candidates for selection, with pronounced variation in the extent of internal regularities among the candidate elements.

Selection rules other than the above algorithm can be invoked to expand the set of algorithms that could be interpreted as mechanisms of randomization. However, the basic thrust of our discussion is clarification that meaningful operational formulations of the idea of randomization are combinatorial in character. It is this insight that removes the ambiguity in Fisher’s original formulation.

Notes and Applications.

1) The vector-ApEn definition generalizes without modification to n ≥ 3-dimensional and to nonrectangular lattices A, thus providing a much wider range of applicability than for the two-dimensional, square lattices featured above.

2) For the 4 × 4 and 5 × 5 Latin squares above, ApEn with m_crit = 1 already provides considerable utility, allowing one to recover, e.g., row and column completeness. For medium and larger sized arrays, where we will use m_crit ≥ 2, there will be enhanced refinement into irregularity classes in the given vector direction(s). For example, for the vector v = (0, 1) with m = 2, we can refine row-complete squares into distinct subclasses, based on an analysis of the extent of equidistribution of 3-blocks (triples) along a row direction.

3) A significant development, distinct and complementary to that above, would be irregularity specifications for very large and infinite lattices A. A clarification of what is meant by “Poisson in the plane” seems especially important, in this setting. For example, on page 2 of the superb volume of Aldous (34), we find the directive “Set down points y according to a Poisson point process of rate λ per unit area.” The disconcerting fact is that constructive algorithms for, and combinatorial characterizations of, Poisson arrays still lie in the future.

4) One also could develop a spatial form of ApEn to assess array irregularity along parametrized curves, rather than along vector (linear) directions. The choice(s) of curves typically would be tailored to the requirements of specific applications. For example, in agricultural experiments where the influence of plant density on yield is considered (35), many density levels and row spacings need to be evaluated. With the objective of minimizing land area used and maximizing the number of density levels and row spacing levels, Nelder (36) introduced a family of fanshape designs. A requirement of irregular arrangements of treatments at given density and spacing levels would necessitate grading of fanshape designs via vector-ApEn along curved arcs.

5) Even within the class of KV squares, as defined in Example 2, there is room for delineation via vector-ApEn. For instance, compare KV1 in that example, for which wrap vector-ApEn_(1,0)(1) = wrap vector-ApEn_(0,1)(1) = wrap vector-ApEn_(1,2)(1) = 0, to

KV2 is also KV, in that it satisfies the knight’s move criteria stated in Example 2, yet intuitively, it appears less regular than KV1. Vector-ApEn quantifies this intuitive difference: for KV2, wrap vector-ApEn_(1,0)(1) = wrap vector-ApEn_(0,1)(1) = 1.068, and wrap vector-ApEn_(1,2)(1) = 1.124.

6) Bailey (37), appropriately criticizing many of FY’s published Latin squares (31) for lack of stratifiability, recommends the use of experimental arrangements where the Latin square is the Cayley table of an Abelian group. This restriction can introduce a high degree of regularity, as exemplified by the Cayley tables of elementary Abelian 2-groups of orders 4 and 8 exhibited on p. 53 of ref. 37. This raises the question of the range of achievable degrees of irregularity for Cayley tables of Abelian groups when, for example, V = {(1, 0); (0, 1)}.

7) One of the crucial observations from the core text is the relative conflict between simultaneously realizing objectives of high degrees of vector irregularity, while minimizing other symmetries. Several points are in order:

(i) The severity of this conflict is most pronounced for very small squares, e.g., the 4 × 4 and 5 × 5 cases, and it already lessens considerably (insofar as extremal instances) for mid-sized and larger squares, e.g., N × N, N ≥ 20.

(ii) Our point is not to mandate which of symmetry vs. irregularity should be rated as more important by the user, but rather, to elucidate the conflict, and especially, to provide quantitative means from which the user can deduce explicit preferences as to how to balance these tradeoffs, based on automated procedures applied to a suitable loss function.

(iii) If visual symmetries are apparent, this generally manifests some form of regularity, in the sense of feature replication, and vector-ApEn, applied to an appropriately redefined set of base atoms and choice of metric, typically then can quantify this symmetry directly. We illustrate the crux of the idea in the 4 × 4 Latin square setting, where we now view the Latin square as consisting of four elemental subpieces, namely, the four subblocks Sub(i, j), 1 ≤ i, j ≤ 2, where

in conjunction with the distance metric δ, with δ = 0 if two subblocks identically match (for corresponding elements), otherwise δ = 1. Consider the “trivial” application of vector-ApEn, with m = 0 (thus vector choice v is irrelevant), which simply assesses one-dimensional equidistribution of the four width-2 subblocks in this partition. Consider

Then vector-ApEn(0) (A) = 0.693, vector-ApEn(0) (B) = 1.040, and vector-ApEn(0) (C) = 1.386, manifesting for A, two occurrences each of (₂¹ ₁²) and (₄³ ₃⁴), for B, two occurrences of (₄³ ₃⁴) and one each for the two subblocks (₂¹ ₁²) and (₁² ₂¹), and for C, one occurrences of each of the four 2 × 2 subblocks.

Furthermore, we can reintroduce r back into consideration in vector-ApEn, as generally is used in continuous state applications, e.g., refs. 38 and 39. This provides considerably more power insofar as grading “near matches.” For instance, if we wished to consider bilateral symmetry (of subblocks) a notable feature, we could redefine our metric δ on subblocks, with δ = 0 if two subblocks identically match, δ = 1/2 if two subblocks are bilaterally symmetric to one another, otherwise δ = 1. Then while vector-ApEn(0, r = 0) (B) = 1.040, observe that vector-ApEn(0, r = 0.5) (B) = 0.693, calibrating the near-match at this level of (δ) resolution.

Finally, this result only hints at the power of such a viewpoint, as vector-ApEn applications with m ≥ 1 go yet again much further, and the range of sensible near match metrics δ and partition choices is vast.

(iv) The methodology of iii may prove very useful in applications of vector-ApEn to image and pattern recognition programs and to characterizations of limiting cellular automata, to assess the degree of repeatability of prescribed features. Redefined sets of base atoms would be shapes or features of critical interest, either on the same scale as the original atoms, or much larger, thus providing a more macroscopic assessment of spatial irregularity.

8) Many models within physics are lattice-based systems. Probably the best known and simplest of these is the nearest-neighbor Ising model (1). This binary state model has been interpreted respectively as modeling a magnet (via spin), a lattice gas, and as an alloy, with one of two allowed species at each site. More general lattice systems are N-vector models (including the classical Heisenberg model), often considered better models of a magnet; spin-glasses (40); two-dimensional 6- and 8-vertex models (of ferroelectrics), in which the random variables are indexed by bonds in the lattice, rather than sites; and lattice gauge models, thought fundamental to understanding elementary particle interactions (1). Determining relationships between changes in vector-ApEn in the above models and physical correlates would seem highly worthwhile, either theoretically or experimentally.

Also, within solid-state physics, we speculate that grading the extent of array disorder may possibly prove useful in assessing or predicting (i) crystal and alloy strength and/or stability under stresses; (ii) phase transitions, either liquid-to-gas, solid-to-liquid, or frigid-to-super-conductive; and (iii) performance characteristics of semiconductors.

9) The analysis of travelling waves oftentimes requires a quantification of subtle changes, particularly as to the extent of formation, and conversely, the extent of dissolution or dissipation of wave fronts, above and beyond an identification of primary wave “pulses” and resultant derived frequency and amplitude statistical values. Although considerable signal-to-noise analysis methodology has been developed for and applied to this setting, to clarify wave fronts, in the ubiquitous instances where the extent of insidious or subordinate activity is the primary feature of interest, a critical and further assessment of the wave patterns is required, to which vector-ApEn should readily apply, both in two- and three-dimensional settings. This recognition may be particularly critical near the genesis of an upcoming event of presumed consequence. Two quite different, important applications in this context are (i) seismologic event formation and (ii) (atrial) fibrillation/arrhythmia mechanisms within physiology. The need for and utility of monitoring irregularity changes, above pulse characteristics, in a one-dimensional setting, has been explained and well-documented in a number of endocrinologic studies (38, 39), where, as well, both correlation and spectral methods have failed to illuminate the essential systemic changes.

ABBREVIATIONS

ApEn

approximate entropy

KV

Knut-Vik

CMOS

complete mutually orthogonal system

FY

Fisher and Yates