Skip to main content
Journal of Applied Statistics logoLink to Journal of Applied Statistics
. 2020 May 14;47(13-15):2421–2430. doi: 10.1080/02664763.2020.1765322

Models with commutative orthogonal block structure: a general condition for commutativity

C Santos a,e,CONTACT, C Nunes b, C Dias c,e, JT Mexia d,e
PMCID: PMC9041981  PMID: 35707425

Abstract

A linear mixed model whose variance-covariance matrix is a linear combination of known pairwise orthogonal projection matrices that add to the identity matrix, is a model with orthogonal block structure (OBS). OBS have estimators with good behavior for estimable vectors and variance components, moreover it may be interesting that the least squares estimators give the best linear unbiased estimators, for estimable vectors. We can achieve it, requiring commutativity between the orthogonal projection matrix, on the space spanned by the mean vector, and the orthogonal projection matrices involved in the expression of the variance-covariance matrix. This commutativity condition defines a more restrict class of OBS, named COBS (model with commutative orthogonal block structure). With this work we aim to present a commutativity condition, resorting to a special class of matrices, named U-matrices.

KEYWORDS: U-matrices, best linear unbiased estimators, mixed models, models with commutative orthogonal block structure

1. Introduction

Linear mixed models play an important role in the design and analysis of experiments and have a wide use in several fields.

In the framework of the design of experiments in agricultural trials, Nelder [13,14] introduced models with orthogonal block structure (OBS), which are linear mixed models whose variance-covariance matrix is a linear combination of known pairwise orthogonal projection matrices (POPM) that add up to the identity matrix. OBS continue to play a central part in the theory of randomized block designs, (see [2,3]), which highlights the interest on the adequacy of the estimators, see e.g. [1,6].

OBS allow optimal estimation for variance components of blocks and contrasts of treatments [8] moreover we may be interested in that least squares estimators (LSE), giving best linear unbiased estimators (BLUE), for estimable vectors. For this purpose, we must impose a commutativity condition on OBS, as it was done in Fonseca et al. [10] when introducing models with commutative orthogonal block structure (COBS). COBS has been the subject of extensive research, addressing, e.g. estimation, inference and operations with models, see e.g. [4–6,8,12,15].

This paper is structured as follows. A framework for models with COBS and some of their interesting results is provided in Section 2. Section 3 is dedicated to our main goal, which is to present a commutativity condition and other results enabling the obtention of BLUE. A real data application, considering an experiment with grapevines, is presented in Section 4 to illustrate the usefulness of the methodology. We conclude this work in Section 5, with some comments.

2. Models with commutative orthogonal block structure

To study COBS we resort to an approach based on their algebraic structure, since this leads to interesting results on the estimation of variance components and on the building up of models, see [10].

Let us consider a linear mixed model

Y=i=0wXiβi, (1)

where β0 is fixed and β1,,βw are random vectors with null mean vectors, variance-covariance matrices σ12Ic1σw2Icw, where ci=rank(Xi), i=1,,w, and null cross-covariance matrices.

The mean vector of Y is

μ=X0β0 (2)

and the variance-covariance matrix is given by

V(σ2)=i=1wσi2Mi, (3)

where Mi=XiXiT, i=1,,w.

The space spanned by the mean vector μ is Ω=R(X0), so the orthogonal projection matrix (OPM), on Ω, is

T=X0(X0TX0)+X0T=X0X0+,

see e.g. [5], where+denotes the Moore–Penrose inverse.

When the matrices M1,,Mw commute, they generate a commutative Jordan algebra of symmetric matrices, CJAS, Inline graphic, this is, a linear space constituted by symmetric matrices that commute and containing the squares of its matrices [11]. The CJAS, Inline graphic has a unique basis, its principal basis, Q, constituted by known pairwise orthogonal orthogonal projection matrices, POPM, Q1,,Qm, see [17]. Thus the matrices Mi, i=1,,w, are linear combinations of the matrices of the principal basis of the CJAS, which means that

Mi=j=1mbi,jQj. (4)

Considering γj=i=1wbi,jσi2, j=1,,m, the canonical variance components, the variance-covariance matrix of Y will take the form

V=j=1mγjQj. (5)

When i=1wMi, belonging to Inline graphic, is invertible, Inline graphic is a complete CJAS and the matrices of its principal basis add up to the identity matrix, i.e.

j=1mQj=In, (6)

and model (1) is a model with OBS.

When dealing with OBS, inference usually involves orthogonal projections on the range spaces of the matrices Qj, j=1,,m, which is somewhat complex due to the combination of estimators obtained from different projections, see e.g. [4]. Imposing a commutativity condition on the OPM on the space spanned by the mean vector, T, and the POPM Qj, j=1,,m, leads to a special class of OBS, those of models with COBS, see [10]. For this class of models we do not have the difficulty associated with orthogonal projections mentioned above, allowing, additionally, the least square estimators, for estimable vectors, to be UBLUE. According to the version of the Gauss-Markov theorem in [18], UBLUE are BLUE whatever the variance components.

3. Generalizing the commutativity condition

Assuming the rows of matrix X0 to correspond to the sets of levels of the fixed effects factors, the mean values of the observations will be determined by those sets. Let us consider that there are n˙ sets of levels associated to r1,,rn˙, contiguous rows of X0. If the components of β0, β0,1,,β0,n˙, are the corresponding mean values, we can reorder the observations to have the block diagonal matrix

X0=D(1r1,,1rn˙), (7)

where 1rl, corresponds to the vector with all rl components equal to 1, l=1,,n˙. So, the orthogonal projection matrix on the space spanned by the mean vector, is given by

T=D(1r1Jr1,,1rn˙Jrn˙) (8)

where Jrl=1rl1rlT, l=1,,n˙.

The fundamental partition of Y will be constituted by the sub-vectors Y1,,Yn˙, corresponding to the n˙ sets of the levels of the fixed effects factors, see [16]. Then the variance- covariance matrix can be defined by

V=[V1,1V1,n˙Vn˙,1Vn˙,n˙], (9)

with Vl,l the variance-covariance matrix of Yl, l=1,,n˙, and Vl,h the cross-covariance matrix of Yl and Yh, lh.

When T, the OPM on the space spanned by the mean vector μ, commutes with the POPM Qj, j=1,,m, the OPM also commutes with the variance-covariance matrix of Y,V.

From (8) and (9) we have

TV=[1r1Jr1V1,11r1Jr1V1,n˙1rn˙Jrn˙Vn˙,11rn˙Jrn˙Vn˙,n˙] (10)

and

VT=[V1,11r1Jr1V1,n˙1rn˙Jrn˙Vn˙,11r1Jr1Vn˙,n˙1rn˙Jrn˙] (11)

So, the matrices T and V commute if and only if

{1r1Jr1V1,1=V1,11r1Jr11r1Jr1V1,n˙=V1,n˙1rn˙Jrn˙1rn˙Jrn˙Vn˙,1=Vn˙,11r1Jr11rn˙Jrn˙Vn˙,n˙=Vn˙,n˙1rn˙Jrn˙. (12)

These equalities imply that we must have

r1==rn˙=r

and equalities (12) may be condensed into

JrVl,h=Vl,hJr,l,h=1,,n˙. (13)

Now, given a matrix

U=[u1,1u1,rur,1ur,r] (14)

we have

JrU=[l=1rul,1l=1rul,rl=1rul,1l=1rul,r] (15)

and

UJr=[h=1ru1,hh=1ru1,hh=1rur,hh=1rur,h]. (16)

So, to have the equality

JrU=UJr (17)

we must have

l=1rul,h=h=1rul,h=u¯r,l,h=1,,r,

with u¯=l=1rh=1rul,h, which means that the sums of the elements in any row or column of matrix U are equal. Thus, matrix U is called a U-matrix, see [16].

Going back to the product of matrices V and T, we see that these matrices commute if and only if the sub-matrices Vl,h,l,h=1,,m, are U-matrices. We thus have the following result.

Proposition 1

For the LSE of β0 be UBLUE it is necessary and sufficient that r1==rn˙=r and the sub-matrices Vl,h,l,h=1,,n˙ be U-matrices.

Since we have

X0=D(1r,,1r)=Im1r, (18)

where denotes the Kronecker matrices product, and taking n˙=m we also have

(X0TX0)1=1rIm, (19)

and so (X0TX0)1X0T=(1/r)D(1r,,1r). Thus, the components of

β0~=(X0TX0)1X0T(Y1TYmT)T

will be the mean values y0,1,,y0,m of the components of the sub-vectors Y1,,Ym.

We are thus led to replace β0 and β0~ by μ0=(μ0,1,,μ0,m) and μ~0=(μ~0,1,,μ~0,m), respectively, which enables us to consider other parametrizations, taking

μ0=Gβ0, (20)

where G will have linearly independent column vectors. Then μ~0 will be the matrix of sub-vectors means and, since

β0=G+μ0 (21)

we have the estimator

β0~=G+μ~0. (22)

We also have the following proposition.

Proposition 2

The estimator β0~ is UBLUE.

Proof:

Let β0 be another unbiased estimator of β0. Then, for cTβ0 we have the unbiased estimator cTβ0~=aTμ~0 with aT=cTG+ and cTβ0=aTμ0, with μ0=Gβ0. Since μ0 is an unbiased estimator of μ0, and μ~0 is UBLUE for μ0, we have Var(cTβ0~)Var(cTβ0) whatever the variance components. Given that c is arbitrary, β0~ is BLUE. Since this holds for all variance components β0~ is UBLUE.

Similarly, we may consider

λ=Uμ0, (23)

with the column vectors of U linearly independent. We now have the result.

Proposition 3

λ~=Uμ~0 will be UBLUE for λ.

Proof:

Since the column vectors of U are linearly independent we have μ0=U+λ and μ~0=U+λ~ . Given λ an unbiased estimator of λ, μ0=U+λ will be an unbiased estimator of λ since its mean vector will be U+Uμ0=μ0. We also have λ=Uμ0. Moreover, for cTλ we have the unbiased estimators cTλ~=cTUμ~0=(UTc)Tμ~0 and cTλ=cTUμ0=(UTc)Tμ0. Since μ~0 is BLUE for μ0, we have Var(cTλ~)Var(cTλ), whatever c, which shows that λ~ is BLUE. Since this holds for all variance components λ~ is UBLUE.

4. An application

Let’s consider an experiment with ‘Touriga Nacional’ grapevine and two fixed effects factors:

  • Location (in the experiment), with three levels;

  • Origin, with two levels.

These two factors cross. Given the great number of clones, some ones were randomly chosen and considered as the levels of a random effects factor nested in the factor Origin. For each origin, three clones were randomly chosen. Lastly five grapevines were considered for each clone in each location. This experiment was analyzed, see [7,9], using its algebraic structure, namely using CJA. For completeness sake we now apply our approach.

We have n˙=3×2=6 sub-vectors each with r=3×5=15 observations. These vectors are presented in Table 1.

Table 1. Production in Kg.

  ORIGIN 1 ORIGIN 2
LOCATION Clone 1 Clone 2 Clone 3 Clone 1 Clone 2 Clone 3
1 3,00 1,00 1,10 1,75 1,10 1,05
1,85 1,10 1,50 3,50 1,05 1,25
0,75 1,00 1,80 2,50 0,50 2,00
1,35 1,60 1,45 2,00 1,05 1,50
1,45 1,50 1,25 0,65 1,25 2,10
2 1,80 1,60 0,85 2,00 1,20 1,00
0,70 1,75 0,65 3,00 1,35 2,70
2,50 0,50 0,55 2,55 1,20 2,15
1,70 1,35 0,90 3,00 0,30 2,10
0,40 1,10 0,09 2,65 2,50 2,70
3 1,05 0,75 0,90 1,60 1,05 1,60
1,50 0,65 0,90 3,05 1,95 1,10
1,15 0,90 0,55 0,25 2,00 2,05
0,85 0,85 0,70 1,66 2,20 1,50
1,15 1,05 0,35 2,65 2,35 3,00

With μ: the general mean; αi: the effect of the i-th location, i=1,2,3; βj: the effect of the j-th origin, j=1,2; γi,j: the interaction between the i-th location and the j-th origin, i=1,2,3, j=1,2; al,j: the random effect of the l-th clone of the j-th origin, l=1,2,3;j=1,2; we have, for the sub-vectors, the model equation

Yi,j=(μ+αi+βj+γi,j)115+[a1,ja2,ja3,j]15+ei,j;i=1,2,3;j=1,2, (24)

where the ei,j, i=1,2,3,j=1,2, will be normal with null mean vector and variance-covariance matrix σl2I15 independent from the vector aj, with components (a1,j,a2,j,a3,j), j=1,2, which will be normal with null mean vector and covariance matrix σa2I3. We can order these sub-vectors using the index l=2(i1)+j;i=1,2,3;j=1,2.

It is straightforward to obtain the covariance and cross-covariance matrices of the sub-vectors. We thus get

V1,1=V2,2=V3,3=V4,4=V5,5=V6,6=σa2I3J5+σl2I15
V1,2=V2,1=V3,4=V4,3=V5,6=V6,5=015,15
V1,3=V3,1=V1,5=V5,1=V2,4=V4,2=σa2I3I5
V1,4=V4,1=V2,3=V3,2=V1,6=V6,1=015,15
V1,6=V6,1=V2,5=V5,2=V3,6=V6,3=015,15

It is easy to see that all these matrices are U-matrices. Thus, the LSE estimator of the vector μ0 with components

μi,j=μ+αi+βj+γi,j;i=1,2,3;j=1,2 (25)

will be UBLUE.

In this application we will focus in the LSE for β. The ANOVA analysis is standard. It is interesting to point out that, since

X0=D(1r,,1r)=Im1r, (26)

we have

(X0TX0)1X0T=1rD(1r,,1r)=1rIm1r. (27)

Thus, the components of β0~ will be the means of the components of sub-vectors. We will represent those means by μ~i,j, i=1,2,3;j=1,2. Taking

{μ~i,.=12(μ~i,1+μ~i,2),i=1,2,3μ~.,j=13(μ~1,j+μ~2,j+μ~3,j),j=1,2μ~.,.=16i=13j=12μ~i,j

we get the estimators

{α~i=μ~i,.μ~.,.,i=1,2,3β~j=μ~.,jμ~.,.,j=1,2γ~i,j=μ~i,jμ~i,.μ~.,j+μ~.,.,i=1,2,3;j=1,2.

According to Proposition 1 these estimators will be UBLUE.

Besides this, using software R, we carried out a standard ANOVA whose main results are presented in Table 2.

Table 2. Model summary and ANOVA table.

Linear mixed model fit by REML. t-tests use Satterthwaite's method [
lmerModLmerTest]
Formula: Grapevines ∼ Location * Origin + (1 | Clone/Origin)
REML criterion at convergence: 179.5
Scaled residuals:
Min 1Q Median 3Q Max
−3.2873 −0.5357 −0.0656 0.5693 2.5275
Random effects:      
Groups Name Variance Std.Dev.
Origin:Clone (Intercept) 0.02634 0.1623
Clone (Intercept) 0.05287 0.2299
Residual   0.37638 0.6135
Number of obs: 90, groups: Origin:Clone, 6; Clone, 3
Fixed effects:          
  Estimate Std. Error df t value Pr(>|t|)
(Intercept) 1.9449 0.5952 30.7182 3.268 0.00267 **
Location −0.7187 0.2505 81.9992 −2.869 0.00523 **
Origin −0.2238 0.3670 25.5218 −0.610 0.54737
Location:Origin 0.4387 0.1584 81.9992 2.769 0.00695 **
         
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ‘ 1
Correlation of Fixed Effects:
  (Intr) Locatn Origin
Location  −0.842    
Origin  −0.925 0.819  
Locatn:Orgn 0.798 −0.949 −0.863
  Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Location 3.0988907 3.0988907 1 81.99925 8.2333400 0.005229186
Origin 0.1399722 0.1399722 1 25.52179 0.3718876 0.547368822
Location:Origin 2.8864267 2.8864267 1 81.99925 7.6688514 0.006945885

From the results presented in Table 2, we conclude that interaction between the fixed effects factors (Location and Origin) and Location are significant.

5. Final comments

The use of linear mixed models is suitable for correlated data due to, for example, repeated measurements. From Nelder’s work emerged a particular class of linear mixed models, named OBS, that took a central role in the theory of randomized block designs, giving rise to several lines of research. As a relevant step towards the adequacy of the estimators came a special class of OBS, called COBS, which allows the estimation of relevant parameters to be optimized. COBS are based on commutativity between T, the OPM on the space spanned by the mean vector, and the POPM Qj, j=1,,m. The commutativity condition we presented is easy to verify and guaranties UBLUE estimators, obtained through least squares, for the coefficients vector and estimable vectors. Thus, we consider that our aims were attained.

Acknowledgements

We thank the referees for their valuable comments on our manuscript which led to significant improvement of this work.

Funding Statement

This work was partially supported by national founds of FCT-Foundation for Science and Technology under UID/MAT/00297/2019 and UID/MAT/00212/2019.

Disclosure statement

No potential conflict of interest was reported by the author(s).

References

  • 1.Bailey R.A., Ferreira S.S., Ferreira D., and Nunes C., Estimability of variance components when all model matrices commute. Linear. Algebra. Appl. 492 (2016), pp. 144–160. doi: 10.1016/j.laa.2015.11.002 [DOI] [Google Scholar]
  • 2.Calinski T. and Kageyama S., Block Designs: A Randomization Approach, Vol. I: Analysis, Lecture Note in Statistics, 150, Springer-Verlag, New York, 2000.
  • 3.Calinski T. and Kageyama S., Block Designs: A Randomization Approach, Vol. II: Design, Lecture Note in Statistics, 170, Springer-Verlag, New York, 2003.
  • 4.Carvalho F., Mexia J.T., and Oliveira M., Estimation in models with commutative orthogonal block structure. J. Stat. Theory. Pract. 3 (2009), pp. 523–533. doi: 10.1080/15598608.2009.10411942 [DOI] [Google Scholar]
  • 5.Carvalho F., Mexia J.T., and Santos C., Commutative orthogonal block structure and error orthogonal models. Electron. J. Linear Algebra 25 (2013), pp. 119–128. [Google Scholar]
  • 6.Carvalho F., Mexia J.T., Santos C., and Nunes C., Inference for types and structured families of commutative orthogonal block structures. Metrika 78 (2015), pp. 337–372. doi: 10.1007/s00184-014-0506-8 [DOI] [Google Scholar]
  • 7.Ferreira S. S., Inferência para Modelos Ortogonais com Segregação, Unpublished PhD thesis, University of Beira Interior, 2006. [Google Scholar]
  • 8.Ferreira S., Ferreira D., Nunes C., and Mexia J.T., Estimation of variance components in linear mixed models with commutative orthogonal block structure. Rev. Colomb. Estad. 36 (2013), pp. 261–271. [Google Scholar]
  • 9.Ferreira S.S. and Mexia J.T., Alternative analysis of an experiment on grapevines. Colloq. Biometryczne 34 (2004), pp. 89–95. [Google Scholar]
  • 10.Fonseca M., Mexia J.T., and Zmyslony R., Inference in normal models with commutative orthogonal block structure. Acta Comment. Univ. Tartu. Math. 12 (2008), pp. 3–16. [Google Scholar]
  • 11.Jordan P., Von Neumann J., and Wigner E., On the algebraic generalization of the quantum mechanical formalism. Ann. Math. 36 (1934), pp. 26–64. [Google Scholar]
  • 12.Mexia J.T., Vaquinhas R., Fonseca M., and Zmyslony R., COBS: Segregation, matching, crossing and nesting. Latest Trends and Applied Mathematics, simulation, Modelling, 4-th International Conference on Applied Mathematics, Simulation, Modelling (ASM’10), 2010, pp. 249–255. [Google Scholar]
  • 13.Nelder J.A., The analysis of randomized experiments with orthogonal block structure I. block structure and the null analysis of variance. Proc. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 283 (1965), pp. 147–162. [Google Scholar]
  • 14.Nelder J.A., The analysis of randomized experiments with orthogonal block structure II. Treatment structure and the general analysis of variance. Proc. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 283 (1965), pp. 163–178. [Google Scholar]
  • 15.Santos C., Error orthogonal models: structure, operations and inference, PhD thesis, University of Beira Interior, 2012. [Google Scholar]
  • 16.Santos C., Nunes C., and Mexia J.T., OBS, COBS and mixed models associated to commutative Jordan Algebra. Bulletin of the ISI, LXII, Proceedings of 56th session of the International Statistical Institute. Lisbon, 2008, pp. 3271–3274. [Google Scholar]
  • 17.Seely J., Quadratic subspaces and completeness. Ann. Math. Statist. 42 (1971), pp. 710–721. doi: 10.1214/aoms/1177693420 [DOI] [Google Scholar]
  • 18.Zmyślony R., A characterization of best linear unbiased estimators in the general linear model. Math. Stat. Probab. Theory 2 (1978), pp. 365–373. doi: 10.1007/978-1-4615-7397-5_27 [DOI] [Google Scholar]

Articles from Journal of Applied Statistics are provided here courtesy of Taylor & Francis

RESOURCES