On the role of parameterization in models with a misspecified nuisance component

Significance

Statistical models are often chosen based on a combination of scientific understanding and flexibility or mathematical convenience. While the aspects of core scientific relevance may be relatively securely specified in terms of interpretable interest parameters, the rest of the formulation is often chosen somewhat arbitrarily. In many statistical models this formulation includes so-called nuisance parameters, which are of no direct subject-matter concern, but are needed to complete the model or reflect the complexity of the data. This paper contributes to the foundations of statistics by studying the interplay between model structure and likelihood inference, under misspecification of the nuisance component.

Abstract

The paper is concerned with inference for a parameter of interest in models that share a common interpretation for that parameter but that may differ appreciably in other respects. We study the general structure of models under which the maximum likelihood estimator of the parameter of interest is consistent under arbitrary misspecification of the nuisance part of the model. A specialization of the general results to matched-comparison and two-groups problems gives a more explicit and easily checkable condition in terms of a notion of symmetric parameterization, leading to a broadening and unification of existing results in those problems. The role of a generalized definition of parameter orthogonality is highlighted, as well as connections to Neyman orthogonality. The issues involved in obtaining inferential guarantees beyond consistency are briefly discussed.

Scientific practice in nearly every field relies on the use of mathematical models: a provisional base for describing how observable quantities materialize. Some models, such as Einstein’s theory of gravitation, provide predictions that can be verified with such accuracy that they are often considered to be the truth. This is, however, an atypical situation: The processes underlying most observable data are much too complicated to be described by exact laws and often require a probabilistic element. A useful statistical model captures the essence of the data-generating mechanism in a way that can accommodate new data collected under slightly different conditions. When the primary purpose of a model is to provide insight and explanation, as is typical in many scientific contexts, stable aspects should ideally be capable of interpretation. For this, statistical models will usually have a relatively small number of scientifically interpretable parameters of interest. So-called nuisance parameters are used to complete the specification of the model.

In contexts where prediction is the main focus, such as arise in many applications of machine learning, model-free approaches are sometimes advocated. This is justified on the grounds that faithful representation of a data-generating process is both unnecessary for such a task and also unrealistically optimistic with very large or complex sets of data. For instance in commercial recommender systems, the criteria of speed and accuracy of predictions are key, and the identification of interpretable parameters unimportant. In sharp contrast are contexts in which the goal is to understand how a treatment or exposure influences the distribution of outcomes, having accounted, to the extent feasible, for the diversity of individuals. Breiman (1) described this separation succinctly as the “two cultures” of statistical thought. The development in this paper is closer to the view expressed by Cox (2) in the discussion of that work.

We consider sets of statistical models that share a common interpretation for the parameter of subject-matter interest but differ in their specification of the nuisance component. A fundamental question is whether reliable inference for the interest parameter is still achievable using standard approaches when the nuisance aspect is misspecified. An exemplar setting is in understanding the effect of treatment or exposure on the health outcomes of individuals. These individuals may exhibit considerable heterogeneity and complex interdependencies stemming, for instance, from certain shared genetic traits. Modeling such interdependencies is a formidable challenge and often tackled by introducing a parameter for each individual. If these are, postulated to have been drawn from a parametric distribution, the resulting doubly stochastic models are called random-effects models. Since the choice of parametric distribution for the individual-specific nuisance parameters is typically based on mathematical convenience, the prospect of misspecification seems likely.

The extensive literature on inference under model misspecification appears to have started with Cox (3), who considered two nonnested models for a vector random variable $Y$, specifying joint density or mass functions $m$ and $\check{m}$, say, where the latter is completely known up to a finite-dimensional parameter $\theta$. With $\widehat{\theta }$ the maximum likelihood estimator under model $\check{m}$, Cox (3) showed that when the true density function is $m$, $\widehat{\theta }$ is asymptotically normal with expected value ${\theta }_m^0$ and covariance matrix given by what is now known as the sandwich formula. The quantity ${\theta }_m^0$ solves

$${\mathbb{E}}_m{[{\mathrm{\nabla }}_{\theta }log\check{m}(Y; \theta )]}_{\theta ={\theta }_m^0}=0,$$

where ${\mathbb{E}}_m$ is expectation under $m$. Equivalently ${\theta }_m^0$ minimizes with respect to $\theta$ the Kullback–Leibler divergence

$$\int m(y)log(\frac{m(y)}{\check{m}(y; \theta )})dy.$$

More rigorous discussions of the distribution theory and regularity conditions were provided by Huber (4), Kent (5), and White (6, 7).

Inferential guarantees for the quantity ${\theta }_m^0$ follow directly under classical regularity conditions on the family of models. Our interest, by contrast, is in the estimation of the true value of an interest parameter, which is assumed to have a common interpretation in both the true distribution and in the fitted model. Our focus is on uncovering the structure for which standard likelihood theory retains its first-order theoretical guarantees, providing foundational insight into the limits of likelihood inference. Reference to the unsolved nature of this question is widespread; see, for instance, Evans and Didelez (8, section 5).

A different line of exploration, with connections to the double-robustness literature [e.g., Robins et al. (9) and Chernozhukov et al. (10)] is the so-called assumption-lean approach to modeling of Vansteelandt and Dukes (11), who center their analyses on model-free estimands rather than parameters of a given model. The ideal estimand coincides with a particular model parameter when the chosen model includes $m$ and retains a degree of interpretability when the model is misspecified. The tension between estimands and model parameters has a long history, going back to Fisher and Neyman; see Cox (12).

In the discussion of ref. 11, Battey (13) conjectured a condition under which the maximum likelihood estimator for an interest parameter is consistent in spite of arbitrary misspecification of the nuisance part of the model. The intuition for that incomplete claim came from a highly involved calculation in a particular paired-comparisons model studied in Battey and Cox (14). Any role played by the assumed model for the random effects was unclear from that calculation and is clarified by the more general insight provided here.

In Section 1, we study the general structure for such a consistency result, and specialize the analysis to settings of common relevance, providing more explicit conditions on the parameterization for matched-pair and two-group problems. An important conclusion from this analysis is that the conditions for consistency can sometimes be checked without knowledge of the true model. Section 2 shows through this route that some key insights have been overlooked in a rather large literature on misspecified random effects distributions, particularly the role of parameterization. Section 2.C recovers two well-known results for generalized linear models as an illustration of the more general statements, and Section 2.D provides a recent example in the context of a marginal structural model, for which the more general results of Section 1 provide elucidation. While some of the material in Section 1 is rather technical, we have aimed to provide intuition for the main results and have provided several examples in Section 2. The examples are deliberately detached from specific subject-matter details, in order that they be broadly useful for a range of scientific applications.

1. Consistency in General Misspecified Models

1.A. General Conditions for Consistency.

Let $m$ represent the density function for the outcomes, parameterized in terms of an interest parameter $\psi$ with true value ${\psi }^{\ast }$. The assumed model, while sharing the same interpretable interest parameter, is misspecified in other ways. The joint density function for the observations under the assumed model has parameters $(\psi ,\lambda )\in \mathrm{\Psi }\times \mathrm{\Lambda }$, and $\ell (\psi ,\lambda )=log \check{m}(y; \psi ,\lambda )$ denotes the observed log-likelihood function for that model, viewed as a function of $(\psi ,\lambda )$ for observed data $y=(y_1,\dots ,y_n)$. As above, maximization of $\ell (\psi ,\lambda )$ gives estimates $(\widehat{\psi },\widehat{\lambda })$; their dependence on $y$ is suppressed in the notation. As functions of the random variable $Y=(Y_1,\cdots ,Y_n)$ the probability limit of the maximum likelihood estimator, as $n\to \infty$, is $({\psi }_m^0,{\lambda }_m^0)$, the solution to

$$\begin{array}{c}\hfill {\mathbb{E}}_m[{\mathrm{\nabla }}_{(\psi ,\lambda )}\ell ({\psi }_m^0,{\lambda }_m^0)]=0.\end{array}$$ [1.1]

We consider the model to be misspecified if there are no values of $\lambda$ in $\mathrm{\Lambda }$ for which the true density $m$ is recovered. This precludes the situation in which the true distribution belongs to a submodel of an assumed encompassing family, briefly discussed in Section 3.C.

The following example will be used repeatedly to illustrate key ideas and notation.

Example 1.1

Consider a matched comparison problem in which, for each of $n$ twin pairs, one individual from each pair is chosen at random to receive a treatment, the other being the untreated control. Let $Y_{i1}$ and $Y_{i0}$ denote the time until some medical event of interest $($e.g., recovery from illness$)$ for the treated and untreated individual in the $i$th pair. A simple parametric model specifies the outcomes $Y_{i1}$ and $Y_{i0}$ as exponentially distributed of rates ${\gamma }_i\psi >0$ and ${\gamma }_i/\psi >0$, respectively. The pair-specific nuisance parameter ${\gamma }_i$ captures, for instance, genetic differences between the pairs of twins. The parameter of interest $\psi$ with true value ${\psi }^{\ast }$ quantifies the effect of the treatment: ${\psi }^2$ being the multiplicative effect of the treatment on the instantaneous probability of recovery relative to baseline. Other parameterizations are possible, but the above symmetric parameterization will be shown to play an important role. Suppose the pair effects are modeled as gamma distributed of shape $\kappa$ and rate $\rho$. Then, under the assumed model, the joint density function for the outcomes in any given pair at $(y_1,y_0)$ is

$$\frac{\mathrm{\Gamma }(\kappa +2){\rho }^{\kappa }}{\mathrm{\Gamma }(\kappa ){(y_1\psi +y_0/\psi +\rho )}^{\kappa +2}}.$$ [1.2]

The true random effects distribution could have been quite different, so that the model Eq. 1.2 is misspecified. Nevertheless, the interpretation of the interest parameter $\psi$ is stable over the different specifications. The notional nuisance parameter is $\lambda =(\kappa ,\rho )$.

Remark 1.1

A different analysis is possible in this example, treating the pair effects ${\gamma }_1,\dots ,{\gamma }_n$ as fixed arbitrary constants, which can be eliminated from the analysis by basing likelihood inference on the distribution of the ratios $Z_i=Y_{i1}/Y_{i0}$; this is discussed briefly in Section 3.B.

Definition 1.1 (parameter m-orthogonality):

Let ${\mathrm{\nabla }}_{\psi \lambda }^2\ell (\psi ,\lambda )$ denote the cross-partial derivative of the assumed log-likelihood function. The parameter $\psi$ is said to be $m$-orthogonal to the notional parameter $\lambda$ if ${\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi \lambda }^2\ell (\psi ,\lambda )]=0$. This can hold globally for all $(\psi ,\lambda )$ or at particular values, the notation $\mathrm{\Psi }{\perp }_m\mathrm{\Lambda }$ indicating global $m$-orthogonality and $\mathrm{\Psi }{\perp }_m\lambda$ and $\psi {\perp }_m\mathrm{\Lambda }$ indicating, respectively, local $m$-orthogonality at $\lambda$ for any $\psi$, and local $m$-orthogonality at $\psi$ for any $\lambda$.

To gain some intuition for parameter $m$-orthogonality, consider first the much stronger condition, ${\mathrm{\nabla }}_{\psi \lambda }^2\ell (\psi ,\lambda )=0$ for all $\psi$ and $\lambda$. This corresponds to what is called a cut in the parameter space and implies ${\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi \lambda }^2\ell (\psi ,\lambda )]=0$ for any $m$ so that the true density $m$ trivially plays no role. In this strongest setting, parameter orthogonality is a purely geometric property of the log-likelihood function that holds only in certain parameterizations. It is effectively an absence of torsion, implying that, for any fixed $\lambda$, the corresponding log-likelihood function over $\mathrm{\Psi }$ is maximized at the same point. This parameter separation in the likelihood function is, however, limited to a relatively small number of models. The weaker condition ${\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi \lambda }^2\ell (\psi ,\lambda )]=0$ only requires the absence of torsion to hold on average over hypothetical repeated draws from the true distribution. If the model is correctly specified, the usual definition of parameter orthogonality with respect to Fisher information is recovered.

Propositions 1.1 and 1.2 give two alternative general conditions for consistency of $\widehat{\psi }$ in spite of arbitrary misspecification of the nuisance part of the model. Their proofs, and those of subsequent results, are in SI Appendix.

Proposition 1.1

Let the observed log-likelihood function for the assumed model be strictly concave as a function of $(\psi ,\lambda )$. Then, ${\psi }_m^0={\psi }^{\ast }$ if and only if ${\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi }\ell ({\psi }^{\ast },{\lambda }_m^0)]=0$. The latter condition is equivalent to ${\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi }\ell ({\psi }^{\ast },\lambda )]=0$ for all $\lambda$ if and only if ${\psi }^{\ast }{\perp }_m\mathrm{\Lambda }$.

Remark 1.2

Local orthogonality at a particular value, ${\lambda }^{\prime }$ say, is not sufficient to ensure that ${\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi }\ell ({\psi }^{\ast },{\lambda }^{\prime })]=0$ implies ${\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi }\ell ({\psi }^{\ast },{\lambda }_m^0)]=0$, as is clear from the proof of Proposition 1.1.

Remark 1.3

In Example 1.1, it was shown in ref. 14 that $\psi$ is globally $m$-orthogonal to $\lambda =(\kappa ,\rho )$ for any distribution over the random effects, and the consistency of the maximum likelihood estimator of $\psi$ was established through an explicit calculation. An initial motivation for the present work was to understand how sensitive that conclusion was to various aspects of the model formulation.

The first part of Proposition 1.1, i.e., the requirement of an unbiased estimating equation ${\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi }\ell ({\psi }^{\ast },{\lambda }_m^0)]=0$, is almost immediate. Since ${\lambda }_m^0$ depends on the unknown $m$ and is typically unavailable, verification of the orthogonality condition ${\psi }^{\ast }{\perp }_m\mathrm{\Lambda }$ uniformly over $m$, in the event of its validity, appears to be the simplest way of establishing consistency. The simplest special cases in which orthogonality can be checked are those in which there is a parameter cut. Such examples include misspecification of the dispersion component of a generalized linear model, discussed in Section 2.C, and Gaussian linear mixed models with a misspecified distribution over the random effects. Robustness of inference in the latter case has often been observed empirically, e.g., Schielzeth et al. (15) without reference to any underlying structure. A more elaborate example with a parameter cut is the causal model of ref. 8 outlined in Section 2.D. Example 1.1 obeys the weaker form ${\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi \lambda }^2\ell (\psi ,\lambda )]=0$ from Proposition 1.1, as does the unbalanced two-group problem of Example 2.4 in Section 2.A, in which the pair-specific parameters are replaced by stratum-specific parameters. The relevant structure underpinning these paired and two-group examples is elucidated in Section 1.C, where some intuition is provided.

The unbalanced two-group problem of Example 2.3, on the other hand, violates the $m$-orthogonality assumption of Proposition 1.1 with respect to one of its two nuisance parameters. Nevertheless, Cox and Wong (16) showed that the maximum likelihood estimator of the interest parameter in Example 2.3 is consistent, suggesting that Proposition 1.1, while easier to verify, is too strong for some situations. Proposition 1.2 presents weaker conditions for consistency.

Suppose that the orthogonality condition ${\psi }^{\ast }{\perp }_m\mathrm{\Lambda }$ from Proposition 1.1 fails, so that there exists at least one $\lambda \ne {\lambda }_m^0$ such that ${\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi }\ell ({\psi }^{\ast },\lambda )]\ne 0$. Let

$$\begin{array}{c}\hfill g_{\psi }({\psi }^{\ast },\lambda ):={\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi }\ell ({\psi }^{\ast },\lambda )],g_{\lambda }({\psi }^{\ast },\lambda ):={\mathbb{E}}_m[{\mathrm{\nabla }}_{\lambda }\ell ({\psi }^{\ast },\lambda )],\end{array}$$ [1.3]

and partition the inverse of the Fisher information matrix $i:=i({\psi }^{\ast },\lambda ):={\mathbb{E}}_m[-{\mathrm{\nabla }}_{(\psi ,\lambda )}^2\ell ({\psi }^{\ast },\lambda )]$ as

$$\begin{array}{c}\hfill \left(\begin{array}{cc}{i}^{\psi \psi }& i^{\psi \lambda }\\ i^{\lambda \psi }& i^{\lambda \lambda }\end{array}\right):={\left(\begin{array}{cc}{i}_{\psi \psi }& i_{\psi \lambda }\\ i_{\lambda \psi }& i_{\lambda \lambda }\end{array}\right)}^{-1}.\end{array}$$ [1.4]

In Proposition 1.2 and its proof, we refer to $g_{\psi }({\psi }^{\ast },\lambda )$ and $g_{\lambda }({\psi }^{\ast },\lambda )$ in the shorthand $g_{\psi }$ and $g_{\lambda }$ respectively.

Proposition 1.2

If $i^{\psi \psi }{g}_{\psi }+i^{\psi \lambda }{g}_{\lambda }=0$ for all $\lambda \in \mathrm{\Lambda }$, then ${\psi }_m^0={\psi }^{\ast }$. If $\psi$ and $\lambda$ are both scalar parameters, the condition reduces to $i_{\lambda \lambda }{g}_{\psi }=i_{\psi \lambda }{g}_{\lambda }$.

That Proposition 1.2 is more general than Proposition 1.1 is seen on noting that the orthogonality condition of Proposition 1.1 implies $i^{\psi \lambda }=0$ and $g_{\psi }=0$, so that the condition of Proposition 1.2 also holds. The conditions of Propositions 1.1 and 1.2 are clearly not necessary. In particular, if ${\lambda }_m^0$ can be calculated, the route to establishing consistency or inconsistency of $\widehat{\psi }$ is more direct. Any practical relevance, beyond general theoretical understanding, is to situations in which ${\lambda }_m^0$ is not calculable, typically because $m$ is unknown. The conditions in Propositions 1.1 and 1.2, while dependent on $m$, are nevertheless checkable in some situations, as illustrated in the examples of subsequent sections.

A generalization involves a second nuisance parameter vector, $\nu \in \mathcal{N}$ say, such that ${\psi }^{\ast }{\perp }_m\mathcal{N}$ and $\lambda {\perp }_m\mathcal{N}$ in Proposition 1.1 or $\mathrm{\Lambda }{\perp }_m\mathcal{N}$ in Proposition 1.2. It can be shown via a straightforward extension of the arguments leading to Propositions 1.1 and 1.2 that the conclusion of the latter is unchanged by this modification. Examples involving a second nuisance parameter are provided in Section 2.

The consistency demonstrated by Battey and Cox (14) for Example 1.1 arises as a special case of Proposition 1.1, while the argument of Cox and Wong (16) for an unbalanced doubly stochastic two-group problem turns out to be an application of Proposition 1.2 and Corollary 1.1 below. We return to this example in Section 2.B.

1.B. Parameter Orthogonality and Orthogonalization in Misspecified Models.

In view of Propositions 1.1 and 1.2, a key question is when the assumed Fisher information matrix, with expectation computed under an erroneous model, coincides with the true Fisher information matrix in some or all regions of the parameter space. Such coincidence implies in particular that parameter orthogonality established under a notional model is true more generally. The practical import is that otherwise $\mathrm{\Psi }{\perp }_m\mathrm{\Lambda }$ or its local analog would typically not be verifiable without knowledge of $m$. A partial answer is presented as Proposition 1.3, whose proof is immediate by the definition of sufficiency.

Proposition 1.3

Suppose that the cross-partial derivative of the assumed log-likelihood function ${\mathrm{\nabla }}_{\psi \lambda }^2\ell (\psi ,\lambda )$ depends on the data only through a sufficient statistic $S=(S_1,\dots ,S_k)$, for some $1\le k\le n$. Write ${\check{ı}}_{\psi \lambda }(\psi ,\lambda )={\mathbb{E}}_{(\psi ,\lambda )}[-{\mathrm{\nabla }}_{\psi \lambda }^2\ell (\psi ,\lambda )]$, where ${\mathbb{E}}_{(\psi ,\lambda )}$ means expectation under the assumed model. Provided that ${\mathrm{\nabla }}_{\psi \lambda }^2\ell (\psi ,\lambda )$ is additive in $S_1,\dots ,S_k$ and ${\mathbb{E}}_m(S_j)={\mathbb{E}}_{(\psi ,\lambda )}(S_j)$ for all $j$, then $i_{\psi \lambda }(\psi ,\lambda )={\check{ı}}_{\psi \lambda }(\psi ,\lambda )$.

Corollary 1.1

Under the conditions of Proposition 1.3, a sufficient condition for parameter $m$-orthogonality $\mathrm{\Psi }{\perp }_m\mathrm{\Lambda }$ is ${\check{ı}}_{\psi \lambda }(\psi ,\lambda )=0$ for all $(\psi ,\lambda )\in \mathrm{\Psi }\times \mathrm{\Lambda }$, and similarly for local $m$-orthogonality.

Remark 1.4

For Example 1.1, $m$-orthogonality can be established more directly via Proposition 2.1.

In general, a reparameterization of a model from parameters $\varphi$ to $\theta =\theta (\varphi )$ is called interest-respecting if the parameter of interest is common to both: if $\varphi =(\psi ,\xi )$ then $\theta =(\psi ,\lambda (\psi ,\xi ))$. The relevance of interest-respecting reparameterizations in scientific work is that the more important interpretable aspects of the model are retained. In ref. 17, interest-respecting orthogonal reparameterizations were explored for improving inference in correctly specified models. The role in misspecified settings is amplified in view of Proposition 1.1, which concerns the first-order properties of the estimator.

In a slightly more explicit notation, suppose that $\varphi$ is a parameterization for which the $(r,s)$th component satisfies $i_{\mathit{rs}}^{(\varphi )}(\varphi )={\check{ı}}_{\mathit{rs}}^{(\varphi )}(\varphi )=0$. Consider an interest-respecting reparameterization from an initial parameter $\varphi$ to a new parameter $\theta$, with coordinates

$${\varphi }^r({\theta }^1,\dots ,{\theta }^p),{\theta }^a({\varphi }^1,\dots ,{\varphi }^p),(a,r=1,\dots ,p),$$

where ${\theta }^1={\varphi }^1=\psi$. An implication of Corollary 1.1 is that the approach of Cox and Reid (17) can still be used in misspecified models, provided that the additional structure of Proposition 1.3 is present in the parameterization that is orthogonal under the assumed model. Specifically, from a starting parameterization $\varphi$, an orthogonal parameterization $\theta$ is any solution in ${\theta }^2,\dots ,{\theta }^p$ to the system of $p-1$ differential equations

$${\check{ı}}_{\mathit{ab}}^{(\varphi )}\frac{\partial {\varphi }^a(\theta )}{\partial {\theta }^1}+{\check{ı}}_{1b}^{(\varphi )}=0,b=2,\dots ,p,$$

or in matrix notation

$$\frac{\partial \varphi (\theta )}{\partial {\theta }^1}=-{({\check{ı}}_{··}^{(\varphi )})}^{-1}({\check{ı}}_{·1}^{(\varphi )}),$$

where ${\check{ı}}_{··}^{(\varphi )}$ is the Fisher information matrix under the assumed model without the row and column corresponding to ${\varphi }^1$, and ${\check{ı}}_{·1}^{(\varphi )}$ is the excluded column.

Even under the assumptions of Proposition 1.3, with model misspecification, the asymptotic variance of $\widehat{\psi }$ is not $i^{\psi \psi }$ but is given by the sandwich formula of ref. 3. This means that inferential guarantees beyond consistency are not, in general, available using standard likelihood-based approaches; see Section 3.A for a brief discussion of some exceptions.

1.C. Symmetric Parameterizations and Induced Antisymmetry.

For the matched-pair and two-group examples of Sections 2.A and 2.B, a fundamental role is played by symmetric parameterization in transformation models, which induces an antisymmetry on the log-likelihood derivative and thereby the $m$-orthogonality of Proposition 1.1 under arbitrary misspecification of the model. The class of problems covered in this subsection exemplifies settings in which $m$-orthogonality can be straightforwardly verified without knowledge of $m$. A simplified explanation is that in transformation models, there is a convenient duality between the parameter space and the sample space which results in the cancelation of the relevant terms in ${\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi \lambda }^2\ell (\psi ,\lambda )]$ provided that the appropriate parameterization is used.

Let $\mathcal{P}$ be a set of continuous probability measures on $\mathcal{Y}$. A transformation model under the action of $G$ [e.g., Barndorff–Nielsen and Cox (18, p. 53]] is a subset of $\mathcal{P}$ parameterized by $\gamma \in \mathrm{\Gamma }$ and possibly other parameters suppressed in the notation, such that

$${\mathcal{P}}_G:=\{p\in \mathcal{P}:p(gE; g\gamma )=p(E; \gamma ),g\in G,E\in \mathcal{E},\gamma \in \mathrm{\Gamma }\},$$

where $\mathcal{E}$ is the set of measurable events on the sample space $\mathcal{Y}$. The definition implies that $G$ acts on both the sample space and the parameter space. In particular, the action of $G$ on $\mathbb{F}$, say, is a continuous map $G\times \mathbb{F}\to \mathbb{F}$, $(g,x)\to gx$ for $x\in \mathbb{F}$, where in the transformation model, $x$ represents either the data point $y\in \mathcal{Y}$ or the parameter value $\gamma \in \mathrm{\Gamma }$. In general, $\mathcal{Y}$ and $\mathrm{\Gamma }$ need not be equal, although this will often be the case. The identity element on $\mathcal{Y}$ or on $\mathrm{\Gamma }$ is written $e=g^{-1}g=g{g}^{-1}$, the context leaving no ambiguity.

The relevant group actions for present purposes depend only on the interest parameter $\psi$, so that we may write, in a more explicit notation, $g=g_{\psi }\in G$. Two examples serve to illustrate the properties of the transformation models. In location models, $\mathcal{Y}=\mathrm{\Gamma }=\mathbb{R}$, the group action is addition and $g_{\psi }\gamma =\gamma +\psi$, giving $g{g}^{-1}=g_{\psi }{g}_{-\psi }=e=g_0$. In scale models, $\mathcal{Y}=\mathrm{\Gamma }={\mathbb{R}}^{+}$, the group action is multiplication and $g_{\psi }\gamma =\gamma \psi$, giving $g{g}^{-1}=g_{\psi }{g}_{(1/\psi )}=e=g_1$.

Let $U$ be a random variable with a distribution depending only on $\gamma$. This is best thought of as the random variable at baseline with respect to $\psi$, i.e., $\psi =0$ in location models and $\psi =1$ in scale models. Write the probability density function of $U$ at $u$ as $f_U(u; \gamma )du$. The following definition plays an important role in establishing consistency for a treatment parameter in matched-pair and two-groups problems.

Definition 1.2 (symmetric parameterization):

Let $Y_1$ and $Y_0$ be independent random variables with probability measures in ${\mathcal{P}}_G$ and density functions $f_1$ and $f_0$, respectively. Their joint distribution is said to be parameterized $\psi$-symmetrically with respect to $(\psi ,\gamma )$ if $g=g_{\psi }\in G$ depends only on $\psi$ and if the density functions $f_1$ and $f_0$ relate to $f_U$ by

$$f_U(u; \gamma )du=f_1(gu; g\gamma )d(gu)=f_0(g^{-1}u; g^{-1}\gamma )d(g^{-1}u).$$

In other words, the joint density function, when expressed in terms of $u_1=g^{-1}{y}_1$ and $u_0=g{y}_0$, is symmetric in $u_1$ and $u_0$:

$$\begin{array}{c}\hfill f_1(y_1; g\gamma )f_0(y_0; g^{-1}\gamma )d{y}_{1}d{y}_0=f_U(u_1; \gamma )f_U(u_0; \gamma )d{u}_{1}d{u}_0.\end{array}$$ [1.5]

Definition 1.2 says that the random variables $U_1=g^{-1}{Y}_1$ and $U_0=g{Y}_0$ are equal in distribution to $U$, which has a “standardized form.” The inverted commas express that $f_U$ might not be a standardized form in the conventional sense, as it depends on $\gamma$ and possibly other parameters suppressed in the notation. The symmetric parameterization can equivalently be written

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill f_1(y_1; g\gamma )d{y}_1=& f_U(g^{-1}{y}_1; \gamma )J_1^{+}d{y}_1\hfill \\ \hfill f_0(y_0; g^{-1}\gamma )d{y}_0=& f_U(g{y}_0; \gamma )J_0^{+}d{y}_0,\hfill \end{array}\end{array}$$ [1.6]

where

$$J_1^{+}=|d(g^{-1}{y}_1)/d{y}_1|,J_0^{+}=|d(g{y}_0)/d{y}_0|,$$

satisfy $J_1^{+}{J}_0^{+}=1$. In a location model, Eq. 1.6 becomes

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill f_1(y_1; \gamma +\psi )d{y}_1=& f_U(y_1-\psi ; \gamma )d{y}_1\hfill \\ \hfill f_0(y_0; \gamma -\psi )d{y}_0=& f_U(y_0+\psi ; \gamma )d{y}_0,\hfill \end{array}\end{array}$$ [1.7]

while in a scale model Eq. 1.6 becomes

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill f_1(y_1; \gamma \psi )d{y}_1=& f_U(y_1/\psi ; \gamma )(1/\psi )d{y}_1\hfill \\ \hfill f_0(y_0; \gamma /\psi )d{y}_0=& f_U(y_0\psi ; \gamma )\psi d{y}_0.\hfill \end{array}\end{array}$$ [1.8]

Distributions within the scale family are often parameterized in terms of rate, or inverse scale, which reverses the roles of $g$ and $g^{-1}$ relative to their actions in the scale parameterization. Example 1.1 is of this form.

In a location-scale model, the action of $G$ is $g=g_{{\psi }_2}○ g_{{\psi }_1}$, where $g_{{\psi }_1}$ is multiplication by a positive scalar ${\psi }_1$ and $g_{{\psi }_2}$ is addition of a scalar ${\psi }_2$, its inverse being $g^{-1}=g_{{\psi }_1}^{-1}○ g_{{\psi }_2}^{-1}$. Thus, a two-parameter symmetric parameterization in a location-scale model is in terms of $g\gamma ={\psi }_1\gamma +{\psi }_2$ and $g^{-1}\gamma =(\gamma -{\psi }_2)/{\psi }_1$. For the applications we have in mind, the interest parameter $\psi$ represents a treatment effect. Thus, if the treatment is assumed to affect either the location or the scale but not both, a location-scale distribution can effectively be treated as either location or scale. Examples include the extreme-value distributions, the most common parametric families arising in renewal theory and used in survival modeling, and members of the elliptically symmetric family in one or more dimensions, which encompasses the Gaussian, Student-t, Cauchy, and logistic distributions.

Consider, as a function of $\psi$ alone, the log-likelihood contribution of $y_1$ and $y_0$, realizations of $Y_1$ and $Y_0$. This is of the form

$$\begin{array}{cc}\hfill \ell (\psi ; \gamma ,y_1,y_0)=& logL(\psi ; \gamma ,y_1,y_0)\hfill \\ \hfill =& log{f}_1(y_1; g\gamma )+log{f}_0(y_0; g^{-1}\gamma ).\hfill \end{array}$$

Variables to the right of the semicolon in the log-likelihood function $\ell$ are treated as fixed (although arbitrary), together with any other parameters suppressed in the notation.

Definition 1.3 (antisymmetry):

The symmetric parameterization of $f_1$ and $f_0$ is said to induce antisymmetry on the associated log-likelihood derivative with respect to $\psi$ if ${\mathrm{\nabla }}_{\psi }\ell (\psi ; \gamma ,y_1,y_0)$, when expressed in terms of $u_1=g^{-1}{y}_1$ and $u_0=g{y}_0$, satisfies ${\mathrm{\nabla }}_{\psi }\ell (\psi ; \gamma ,u_1,u_0)=-{\mathrm{\nabla }}_{\psi }\ell (\psi ; \gamma ,u_0,u_1)$.

Example 1.2 (Continuation of Example 1.1):

In the exponential matched pair problem, the gamma distribution over the random effects is irrelevant for illustrating the symmetric parameterization and induced antisymmetry, as these definitions are conditional on $\gamma$ within a single pair. Since multiplication of rates corresponds to division of scale, it is natural, for consistency with Eq. 1.8, to define the group operation $g$ as division by $\psi$ rather than multiplication by $\psi$. Thus let $u_1=g^{-1}{y}_1=\psi y_1$ and $u_0=g{y}_0=y_0/\psi$. Conditionally on $\gamma$, the joint density function of $(Y_1,Y_0)$ is

$$f_1(y_1; {\psi }^{\ast },\gamma )f_0(y_0; {\psi }^{\ast },\gamma )d{y}_{1}d{y}_0={\gamma }^{2}exp\{-\gamma (u_1+u_0)\}d{u}_{1}d{u}_0,$$

which is symmetric in $u_1$ and $u_0$. The log-likelihood derivative with respect to $\psi$ is

$${\mathrm{\nabla }}_{\psi }\ell (\psi ; u_0,u_1)=-\gamma (y_1-y_0/\psi )=-\gamma (u_1/\psi -u_0/\psi ),$$

where the right hand side is $-(-\gamma (u_0/\psi -u_1/\psi ))=-{\mathrm{\nabla }}_{\psi }\ell (\psi ; u_1,u_0)$. This shows that the log-likelihood derivative is antisymmetric in the sense of Definition 1.3.

The main transformation models arising in statistics are the location models and the scale models. For these, we show in Example 1.3 that the symmetric parameterization automatically induces antisymmetry on the associated log-likelihood derivative. We also show this for a rotation family on the circle in Example 1.4. We conjecture that antisymmetry according to Definition 1.3 is a necessary consequence of the $\psi$-symmetric parameterization for any transformation model, but in the absence of general group-theoretic proof, we present the following necessary and sufficient conditions for asymmetry, that may be checked on a case-by-case basis for more exotic groups. Examples 1.3 and 1.4 illustrate the application of Proposition 1.4.

Proposition 1.4

Suppose that the joint distribution of $Y_1$ and $Y_0$ is parameterized $\psi$-symmetrically with respect to $(\psi ,\gamma )$ in the sense of Definition 1.2. The parameterization induces antisymmetry on the log-likelihood derivative in the sense of Definition 1.3 if and only if

$$a(u_1,u_0):=\frac{\partial u_1}{\partial \psi }+\frac{\partial u_0}{\partial \psi }=-a(u_0,u_1),$$

and

$$c(u_1,u_0):=(\frac{\partial }{\partial \psi }|\frac{\partial u_1}{\partial y_1}|)|\frac{\partial u_0}{\partial y_0}|+(\frac{\partial }{\partial \psi }|\frac{\partial u_0}{\partial y_0}|)|\frac{\partial u_1}{\partial y_1}|=-c(u_0,u_1),$$

where $u_1=g^{-1}{y}_1$ and $u_0=g{y}_0$.

A clearer but more cumbersome expression of the conditions $a(u_1,u_0)=-a(u_0,u_1)$ and $c(u_1,u_0)=-c(u_0,u_1)$ makes the dependence of the partial derivatives on $u_1$ and $u_0$ explicit. In this more explicit notation, $a(u_1,u_0)=-a(u_0,u_1)$ amounts to $(\partial u_1/\partial \psi )(u_1)=-(\partial u_0/\partial \psi )(u_1)$ and vice versa.

Example 1.3

In a symmetric parameterization of a location model $\partial u_1/\partial \psi =-1=-\partial u_0/\partial \psi$ and

$$(\frac{\partial }{\partial \psi }|\frac{\partial u_1}{\partial y_1}|)|\frac{\partial u_0}{\partial y_0}|=0=-(\frac{\partial }{\partial \psi }|\frac{\partial u_0}{\partial y_0}|)|\frac{\partial u_1}{\partial y_1}|.$$

Thus, $a(u_1,u_0)=0=-a(u_0,u_1)$ and $c(u_1,u_0)=0=-c(u_0,u_1)$. In a symmetric parameterization of a scale model $\partial u_1/\partial \psi =-u_1/\psi$, $\partial u_0/\partial \psi =u_0/\psi$, and

$$(\frac{\partial }{\partial \psi }|\frac{\partial u_1}{\partial y_1}|)|\frac{\partial u_0}{\partial y_0}|=-\frac{1}{\psi }=-(\frac{\partial }{\partial \psi }|\frac{\partial u_0}{\partial y_0}|)|\frac{\partial u_1}{\partial y_1}|,$$

so that $a(u_1,u_0)=-a(u_0,u_1)$ and $c(u_1,u_0)=0=-c(u_0,u_1)$. Thus both cases satisfy the condition of Proposition 1.4.

Example 1.4

For a rotation model on ${\mathbb{R}}^p$, $\mathcal{Y}\subset {\mathbb{R}}^p$, $\mathrm{\Gamma }=[0,2\pi )$ and the group action is matrix multiplication by a rotation matrix $g\in G\subseteq \text{SO}(p)$, where $\text{SO}(p)$ is the special orthogonal group of dimension $p$. The identity element is matrix multiplication by the $p$-dimensional identity matrix and $f_U(u; \gamma )$ is a distribution on the hypersphere, where $\gamma$ is a rotation parameter. The von Mises–Fisher distribution is one such example. Consider the case of $p=2$.

Let $v\in {\mathbb{R}}^2$ be a point on a circle of fixed arbitrary radius, either a location parameter of a probabilistic model or a data point on the circle. In a symmetric parameterization of a rotation model, $v_{\gamma }=g_{\gamma }v$ defines a (counterclockwise) rotation of angle $v_{\gamma }^{\text{T}}v=\gamma$ and $v_1=g_{\psi }{v}_{\gamma }$, $v_0=g_{\psi }^{-1}{v}_{\gamma }$ define (counterclockwise and clockwise) rotations of angles $v_1^{\text{T}}{v}_{\gamma }=\psi$ and $v_0^{\text{T}}{v}_{\gamma }=-\psi$, respectively, where

$$\begin{array}{cc}\hfill g_{\psi }^{-1}& ={\left(\begin{array}{cc}cos\psi & -sin\psi \\ sin\psi & cos\psi \end{array}\right)}^{-1}=\left(\begin{array}{cc}cos\psi & sin\psi \\ -sin\psi & cos\psi \end{array}\right)\hfill \\ \hfill & =\left(\begin{array}{cc}cos(-\psi +2\pi )& -sin(-\psi +2\pi )\\ sin(-\psi +2\pi )& cos(-\psi +2\pi )\end{array}\right)\in G.\hfill \end{array}$$

and $g_{\psi }{g}_{\gamma }=g_{\gamma +\psi }\in G$, $g_{\psi }^{-1}{g}_{\gamma }=g_{\gamma -\psi }\in G$. Thus, in Eq. 1.6 $g\gamma$ should be understood either as addition of angles $g\gamma =\gamma +\psi$ or multiplication of rotation matrices $g\gamma =g_{\psi }{g}_{\gamma }$.

Let $y_1={(y_{11},y_{12})}^{\text{T}}$, $y_0={(y_{01},y_{02})}^{\text{T}}$. Then

$$\begin{array}{cc}\hfill u_1=g^{-1}{y}_1& =\left(\begin{array}{c}{y}_{11}cos\psi +y_{12}sin\psi \\ -y_{11}sin\psi +y_{12}cos\psi \end{array}\right),\hfill \\ \hfill u_0=g{y}_0& =\left(\begin{array}{c}{y}_{01}cos\psi -y_{02}sin\psi \\ y_{01}sin\psi +y_{02}cos\psi \end{array}\right).\hfill \end{array}$$

The Jacobian determinants $J_1^{+}$ and $J_0^{+}$ are both equal to $|{cos}^2\psi +{sin}^2\psi |=1$. Thus, $c(u_1,u_0)=0=-c(u_0,u_1)$ in Proposition 1.4. Since

$$\begin{array}{cc}\hfill \frac{\partial u_1}{\partial \psi }& =\left(\begin{array}{c}-y_{11}sin\psi +y_{12}cos\psi \\ -y_{11}cos\psi -y_{12}sin\psi \end{array}\right)=g_{\theta }^{-1}{u}_1,\hfill \\ \hfill \frac{\partial u_0}{\partial \psi }& =\left(\begin{array}{c}-y_{01}sin\psi -y_{02}cos\psi \\ y_{01}cos\psi -y_{02}sin\psi \end{array}\right)=g_{\theta }{u}_0,\hfill \end{array}$$

with $\theta =\pi /2$, the quantity $a(u_1,u_0)$ from Proposition 1.4 is

$$\begin{array}{cc}\hfill a(u_1,u_0)=& \left(\begin{array}{cc}cos(-\pi /2)& -sin(-\pi /2)\\ sin(-\pi /2)& cos(-\pi /2)\end{array}\right)u_1\hfill \\ \hfill & +\left(\begin{array}{cc}cos(\pi /2)& -sin(\pi /2)\\ sin(\pi /2)& cos(\pi /2)\end{array}\right)u_0=-a(u_0,u_1),\hfill \end{array}$$

verifying the conditions for the $\psi$-symmetric parameterization of rotation families on the circle.

2. Examples

2.A. Matched Pairs.

Let $Y_{i1}$ and $Y_{i0}$ be random variables corresponding to observations on treated and untreated individuals in a matched pair design, where $i=1,\dots ,n$ is the pair index. The treatment effect is represented by a parameter $\psi$ while the pair effects are encapsulated in the pair-specific nuisance parameter ${\gamma }_i$, avoiding explicit modeling assumptions in terms of covariates. As noted in Example 1.1, two approaches to analysis treat the pair effects as fixed arbitrary constants, or as independent and identically distributed random variables. In the latter case, it is common to model the distribution parametrically, typically producing efficiency gains in the treatment effect estimator if the parametric assumptions hold to an adequate order of approximation.

The relevant calculations are the same for every pair, and we therefore omit the pair index $i$ from the notation. Let the true joint density function of $(Y_1,Y_0)$ be given by

$$\begin{array}{c}\hfill m(y_1,y_0)=\int f_1(y_1; {\psi }^{\ast },\gamma )f_0(y_0; {\psi }^{\ast },\gamma )f(\gamma )d\gamma ,\end{array}$$ [2.1]

with $f(\gamma )$ an unknown density function for the nuisance parameters, which are treated as independent and identically distributed across pairs.

Conditionally on $\gamma$, let $\ell (\psi ; \gamma ,y_1,y_0)$ denote the log-likelihood contribution for $\psi$ from a single pair.

Proposition 2.1

Suppose that the assumed model over $\gamma$ is parameterized by $\lambda$, producing a log-likelihood contribution $\ell (\psi ,\lambda ; y_1,y_0)$, assumed strictly concave. Suppose further that, conditionally on $\gamma$, $Y_1$ and $Y_0$ have a joint distribution that is parameterized $\psi$-symmetrically (Definition 1.2). Then, provided that the group induces antisymmetry on the log-likelihood derivative (Definition 1.3), it follows that ${\psi }^{\ast }{\perp }_m\mathrm{\Lambda }$ and

$$\begin{array}{cc}\hfill 0& ={\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi }\ell ({\psi }^{\ast },\lambda )]\hfill \\ \hfill & ={\int }_{\mathcal{Y}}{\int }_{\mathcal{Y}}({\mathrm{\nabla }}_{\psi }\ell ({\psi }^{\ast },\lambda ; y_1,y_0))m(y_1,y_0)d{y}_{1}d{y}_0,\hfill \end{array}$$ [2.2]

for all $\lambda \in \mathrm{\Lambda }$. Thus, ${\psi }_m^0={\psi }^{\ast }$ by Proposition 1.1.

Example 2.1 (Continuation of Examples 1.1 and 1.2):

Example 1.2 establishes antisymmetry of the log-likelihood derivative for the symmetric parameterization of the exponential matched pairs problem. Consistency of $\widehat{\psi }$ thus follows by Proposition 2.1 regardless of the true distribution and assumed model for the random effects $\gamma$. This considerably extends the result of ref. 14 and provides deeper insight into the structure of inference under model misspecification.

Remark 2.1

It is not too difficult to check following calculations similar to that in Appendix A.4 of ref. 14 that if we parameterize this model nonsymmetrically, for example, with rates ${\gamma }_i\theta$ and ${\gamma }_i$, the resulting estimator is not consistent if the gamma random effects assumption is erroneous.

Proposition 1.4 gives easily verifiable conditions for antisymmetry of the log-likelihood derivative, which holds for all location and scale families in Example 1.3. The following explicit calculation for the normal distribution mirrors Example 1.2.

Example 2.2 (normal matched pairs with arbitrary mixing):

Let $Y_1$ and $Y_0$ be normally distributed with unit variance and means $\gamma +{\psi }^{\ast }$ and $\gamma -{\psi }^{\ast }$, respectively, $\gamma$ being treated as random. The conclusion is unchanged if an unknown variance parameter is present. Conditionally on $\gamma$, with $u_1=y_1-{\psi }^{\ast }$, $u_0=y_0+{\psi }^{\ast }$, the joint density function of $(Y_1,Y_0)$ is

$$\begin{array}{cc}& f_1(y_1; {\psi }^{\ast },\gamma )f_0(y_0; {\psi }^{\ast },\gamma )d{y}_{1}d{y}_0\hfill \\ \hfill & =exp[-\{{(u_1-\gamma )}^2+{(u_0-\gamma )}^2\}/2]d{u}_{1}d{u}_0,\hfill \end{array}$$

which is symmetric in $u_1$ and $u_0$ by construction as a result of using a $\psi$-symmetric parameterization with respect to addition, the relevant group action for location models. Conditionally on $\gamma$, the likelihood as a function of $\psi$ is

$$\begin{array}{cc}\hfill L(\psi ; y_0,y_1,\gamma )& \propto exp\{-\frac{1}{2}{(y_1-\gamma -\psi )}^2-\frac{1}{2}{(y_0-\gamma +\psi )}^2\}\hfill \\ \hfill & \propto exp[-\frac{1}{2}\{{(y_1-\psi )}^2+{(y_0+\psi )}^2\}],\hfill \end{array}$$ [2.3]

where the proportionality symbol on the second line absorbs the multiplicative term $exp\{-{\gamma }^2+\gamma (y_1+y_0)\}$, which does not depend on $\psi$. On taking logarithms and differentiating,

$${\mathrm{\nabla }}_{\psi }\ell (\psi ; u_0,u_1)=y_1-y_0-2\psi =u_1-u_0=-{\mathrm{\nabla }}_{\psi }\ell (\psi ; u_1,u_0),$$

verifying the antisymmetry condition of Definition 1.3.

For any assumed random effects distribution for $\gamma$ with density function parameterized by $\lambda$, an application of Proposition 2.1 guarantees that the resulting maximum likelihood estimator $\widehat{\psi }$ is consistent for ${\psi }^{\ast }$ under arbitrary misspecification of the random effects distribution.

Remark 2.2

A separate point in connection with Example 2.2 is that, due to the separation in the log-likelihood function specific to this case (a parameter cut in the terminology of ref. 19), $\widehat{\psi }=(Y_1-Y_0)/2$ regardless of the assumed random effects distribution, and for an analysis from $n$ pairs $\widehat{\psi }=(S_1-S_0)/2n$, where $S_j=\sum _{i=1}^n{Y}_{\mathit{ij}}$, where $i$ is the pair index. For any finite-variance random effects distribution $f$, $\text{var}(Y_1)=\text{var}(Y_0)=1+{\text{var}}_f(\gamma )$ and $\text{cov}(Y_1,Y_0)={\text{var}}_f(\gamma )$. Thus $\text{var}(\widehat{\psi })=2/n$ irrespective of $f$ and of the assumed (erroneous) model over the random effects.

Remark 2.3

In the context of stratum-specific random effects, Lindsay (20, Thm 4.2] established the limiting distribution of his proposed estimator of the interest parameter, valid whether or not the random effects distribution is misspecified, under the assumption that his estimator is consistent. The latter aspect, central to the present paper, was not discussed in that work.

2.B. Two-Group Problems.

Section 2.A dealt with an experimental setting in which the matching ensures balance. A more realistic situation in an observational context is that observations on treated and untreated individuals are stratified into groups that are as similar as possible. This in general produces unbalanced strata, having a potentially different number of individuals in the treated and untreated groups for each stratum.

Inference is based on the sufficient statistics $S_{j1}$ and $S_{j0}$ within treatment groups and strata. Let ${(Y_{ij1})}_{i=1}^{r_{j1}}$ and ${(Y_{ij0})}_{i=1}^{r_{j0}}$ be observations within the $j$th stratum for treated and untreated individuals, respectively. Some examples fix ideas. If the observations ${(Y_{ij1})}_{i=1}^{r_{j1}}$ and ${(Y_{ij0})}_{i=1}^{r_{j0}}$ are normally distributed with means ${\gamma }_j+{\psi }^{\ast }$ and ${\gamma }_j-{\psi }^{\ast }$ and variance $\tau$, the likelihood contribution to the $j$th stratum depends on the data only through $\sum _{i=1}^{r_{j1}}{Y}_{ij1}/r_{j1}$ and $\sum _{i=1}^{r_{j0}}{Y}_{ij0}/r_{j0}$, whose distributions are normal with means unchanged, and variances $\tau /r_{j1}$ and $\tau /r_{j0}$, respectively. If, conditionally on ${\gamma }_j$, the individual observations are Poisson distributed counts of rates ${\gamma }_j{\psi }^{\ast }$ and ${\gamma }_j/{\psi }^{\ast }$, the sufficient statistics are sums of these counts, Poisson distributed of rates $r_{j1}{\gamma }_j{\psi }^{\ast }$ and $r_{j0}{\gamma }_j/{\psi }^{\ast }$. As a third example, if the originating variables are exponentially distributed of rates ${\gamma }_j{\psi }^{\ast }$ and ${\gamma }_j/{\psi }^{\ast }$, the sufficient statistics are gamma-distributed sums of shape and rate parameters $(r_{j1},{\gamma }_j{\psi }^{\ast })$ and $(r_{j0},{\gamma }_j/{\psi }^{\ast })$, respectively. The matched comparison setting of Section 2.A is a special case of this formulation with $r_{j1}=r_{j0}=1$ for all $j$. The more general balanced case with $r_{j1}=r_{j0}=r$ is also implicitly covered by Propositions 1.4 and 2.1.

The situation when $r_{j1}\ne r_{j0}$ is more complicated as it implies that even if the distributions of $S_{j1}$ and $S_{j0}$ belong to a group family, the transformations required to express their distributions in the standardized form $f_U$ depend on the different values of $r_{j1}$ and $r_{j0}$, so that Definition 1.2 is violated. The following extension of Proposition 2.1, which assumes the same random effects formulation for the nuisance parameters ${\gamma }_j$, omits the subscripts for the strata as before.

Proposition 2.2

Suppose that, conditionally on $\gamma$, $S_1$ and $S_0$ are independent random variables with probability measures in ${\mathcal{P}}_G$. Let $h_1\in G$ and $h_0\in G$, not depending on $\psi$ or $\gamma$, be such that $T_1=h_1{S}_1$ and $T_0=h_0{S}_0$ have a joint distribution that is parameterized $\psi$-symmetrically in the sense of Definition 1.2 and such that the log-likelihood derivative, when expressed in terms of $u_1=g^{-1}{t}_1$ and $u_0=g{t}_0$, is antisymmetric in the sense of Definition 1.3. Then, provided that $\ell (\psi ,\lambda )$ is strictly concave, ${\psi }^{\ast }\perp \mathrm{\Lambda }$ and

$$\begin{array}{cc}\hfill 0& ={\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi }\ell ({\psi }^{\ast },\lambda )]\hfill \\ \hfill & =\int \int ({\mathrm{\nabla }}_{\psi }\ell ({\psi }^{\ast },\lambda ; s_1,s_0))m(s_1,s_0)d{s}_{1}d{s}_0,\hfill \end{array}$$ [2.4]

for all $\lambda \in \mathrm{\Lambda }$. Thus, ${\psi }_m^0={\psi }^{\ast }$ by Proposition 1.1.

For discrete problems such as the two-group Poisson situation mentioned above, there is no natural group and Propositions 2.1 and 2.2 are therefore inapplicable. Direct verification of Proposition 1.2 is sometimes more fruitful, as illustrated in the following examples.

Example 2.3 (stratified two-group Poisson problem with unbalanced strata):

Cox and Wong (16) considered misspecification of a random effects distribution in a stratified two-group problem. The distributions of counts $S_{j1}$ and $S_{j0}$ in the treated and untreated groups for stratum $j$ are, conditionally on ${\gamma }_j$, Poisson with means $r_{j1}{\gamma }_{j}exp({\theta }^{\ast })$ and $r_{j0}{\gamma }_{j}exp(-{\theta }^{\ast })$, respectively, where $r_{j1}$ and $r_{j0}$ reflect the number of patients at risk in each group and ${\gamma }_j$ are stratum-specific nuisance parameters.

The problem can equivalently be parameterized in terms of $\psi =e^{\theta }$ but the above formulation was used by Cox and Wong (16). The result of Proposition 2.3 was noted there; we complete their proof in SI Appendix.

Proposition 2.3 [Cox and Wong (16)].

Suppose that in Example 2.3, the nuisance parameters ${({\gamma }_j)}_{j=1}^n$ are modeled as independent and identically distributed random variables with gamma density $\xi {(\xi \gamma )}^{\omega -1}exp(-\xi \gamma )/\mathrm{\Gamma }(\omega )$, parameterized in terms of the notionally orthogonal parameters $\nu =\xi /\omega$ and $\omega >0$. Provided that there is a member of the assumed model that gives the same expectation ${\mathbb{E}}_{\nu ,\omega }({\gamma }_j)=1/\nu$ as under the true random effects distribution for ${({\gamma }_j)}_{j=1}^n$, $\widehat{\theta }$ is consistent for ${\theta }^{\ast }$.

Example 2.4 (time to the first event in a two-group problem with unbalanced strata):

We describe here the generating mechanism for an exponential analog of Example 2.3. Suppose the available data in Example 2.3 consist also of the failure times. If these are, for each individual, independent exponentially distributed of rates ${\gamma }_{j}exp({\theta }^{\ast })$ and ${\gamma }_{j}exp(-{\theta }^{\ast })$ in the $j$th stratum, with $r_{j1}$ and $r_{j0}$ individuals at risk in the treated and untreated groups, then the times $S_{j1}$ and $S_{j0}$ to the first failure in each group are the group minima, exponentially distributed with rates $r_{j1}{\gamma }_{j}exp({\theta }^{\ast })$ and $r_{j0}{\gamma }_{j}exp(-{\theta }^{\ast })$, respectively. By using only the minima across strata, information is sacrificed, which would normally not be advised unless to check on modeling assumptions. After reparameterization, $\psi =e^{\theta }$, the problem satisfies the conditions of Proposition 2.2 with $T_{j1}=r_{j1}{S}_{j1}$ and $T_{j0}=r_{j0}{S}_{j0}$. Thus, $\widehat{\psi }$ is consistent for ${\psi }^{\ast }= exp({\theta }^{\ast })$.

2.C. Generalized Linear Models.

A simple example in which Proposition 1.1 is directly applicable is misspecified dispersion in exponential-family regression problems. Suppose that the true density function belongs to the class of canonical exponential family regression models with unknown regression coefficient $\psi$ on a set of known covariates $x_1,\dots ,x_n$ (each column vectors of the same dimension as $\psi$), and dispersion parameters ${\varphi }_1,\dots ,{\varphi }_n$. The form of the exponential-family log-likelihood function with respect to ${\varphi }_i$ is such that ${\psi }^{\ast }{\perp }_m{\varphi }_i$ for all $i$. In particular this implies that if ${\varphi }_i$ is modeled erroneously as ${\varphi }_i(\lambda )$, perhaps depending on covariates other than or including $x_i$, ${\psi }^{\ast }{\perp }_m\mathrm{\Lambda }$. Write the log-likelihood function of the assumed model as

$$\begin{array}{c}\hfill \ell (\psi ,\lambda )={\displaystyle \sum _{i=1}^n}{\varphi }_i^{-1}(\lambda )\{y_i{x}_i^{\text{T}}\psi -K(x_i^{\text{T}}\psi )\}+{\displaystyle \sum _{i=1}^n}h\{y_i; {\varphi }_i(\lambda )\}.\end{array}$$ [2.5]

We do not distinguish notationally between realizations of the random variables $Y_1,\dots ,Y_n$ and arbitrary evaluation points for their joint density function conditional on covariates. With outcomes treated as random, the estimating equation for $\psi$ is unbiased in the sense that ${\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi }\ell ({\psi }^{\ast },\lambda )]=0$ for all $\lambda \in \mathrm{\Lambda }$. Thus, $\widehat{\psi }$ is consistent in the presence of arbitrary misspecification of the dispersion parameters.

In a continuation of this generalized linear models example, we assume this time a simplified specification for dispersion but allow for relevant covariates to be omitted. Let $Z=(X,W)$ denote the included and omitted covariates. The conditional probability density or mass function of $Y=(Y_1,\dots ,Y_n)$ at $y=(y_1,\dots ,y_n)$ is, with ${\theta }^{\ast }={({\psi }^{\ast \text{T}},{\lambda }^{\ast \text{T}})}^{\text{T}}$.

$$\begin{array}{cc}& m(y; z_1^{\text{T}}{\theta }^{\ast },\dots ,z_n^{\text{T}}{\theta }^{\ast })\hfill \\ \hfill & = exp[{\varphi }^{-1}\{{\theta }^{\ast \text{T}}{\displaystyle \sum _{i=1}^n}{z}_i{y}_i-{\displaystyle \sum _{i=1}^n}K(z_i^T{\theta }^{\ast })\}]{\displaystyle \prod _{i=1}^n}h(y_i,\varphi ),\hfill \end{array}$$

Let $\check{m}(y; x_1^{\text{T}}\psi ,\dots ,x_n^{\text{T}}\psi )$ be the assumed model, in which the outcome is modeled only in terms of $X$. The orthogonality condition ${\psi }^{\ast }{\perp }_m\mathrm{\Lambda }$ is

$$\begin{array}{c}\hfill {\displaystyle \sum _{i=1}^n}{K}^{″}(x_i^{\text{T}}{\psi }^{\ast }+w_i^{\text{T}}\lambda )x_i{w}_i^{\text{T}}=0,\end{array}$$ [2.6]

for all $\lambda$, a highly restrictive condition. By Proposition 1.1, the additional conditions under which ${\psi }_m^0={\psi }^{\ast }$ are those that set ${\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi }\ell ({\psi }^{\ast },\lambda )]=0$ for all $\lambda$, where $\ell$ is the log-likelihood function under the assumed model. Since $\lambda$ is not present under the assumed model,

$$\begin{array}{cc}\hfill {\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi }\ell ({\psi }^{\ast },\lambda )]& ={\displaystyle \sum _{i=1}^n}\{K^{\prime }(z_i^{\text{T}}{\theta }^{\ast })-K^{\prime }(x_i^{\text{T}}{\psi }^{\ast })\}{x}_i\hfill \\ \hfill & ={\displaystyle \sum _{i=1}^n}{K}^{″}(x_i^{\text{T}}{\psi }^{\ast }+{\overline{\xi }}_i)x_i{w}_i^{\text{T}}{\lambda }^{\ast },\hfill \end{array}$$

where ${\overline{\xi }}_i$ is on a line between $0$ and $w_i^{\text{T}}{\lambda }^{\ast }$ and the first equality is because $K^{\prime }(z_i^{\text{T}}{\theta }^{\ast })={\mathbb{E}}_m(Y_i\mid Z_i=z_i)$. Since $K^{″}(x_i^{\text{T}}{\psi }^{\ast }+{\overline{\xi }}_i)\ne K^{″}(x_i^{\text{T}}{\psi }^{\ast }+w_i^{\text{T}}\lambda )$ in general, the second condition ${\mathbb{E}}_m[{\mathrm{\nabla }}_{\psi }\ell ({\psi }^{\ast },\lambda )]=0$ typically does not hold exactly even when the columns of $X$ and $W$ are orthogonal. An exception is the normal-theory linear model, for which $K(\zeta )={\zeta }^2/2$ so that $K^{″}(\zeta )=1$. For an insightful discussion of noncollapsibility in logistic and other regression models, see ref. 21, and for some relevant approximations, see ref. 22.

Although both conclusions of this subsection are well established in the literature, their purpose is to illustrate application of Proposition 1.1 to some well-known settings.

2.D. Marginal Structural Models.

Evans and Didelez (8) presented a further example in which consistency arises as a special case of Proposition 1.1. Their paper introduced the idea of a frugal parameterization of a marginal structural model, constructed to complete the specification of a model respecting the interventional components of interest, but without loss or redundancy. In a causal system of the form, e.g., $Z\to X$, $Z\to Y$, $X\to Y$, the authors suggest a reparameterization of the joint distribution $p_{\mathit{ZXY}}$ from $(p_Z,p_{X|Z},p_{Y|ZX})$ to $(p_Z,p_{X|Z},p_{Y|X}^{\ast },{\varphi }_{YZ|X}^{\ast })$, where $p_{Y|X}^{\ast }$, with finite-dimensional parameter ${\theta }_{Y|X}^{\ast }$, is typically the unobservable interventional distribution of interest for $Y\mid \text{do}(X)$, and ${\varphi }_{YZ|X}^{\ast }$ is whatever is necessary to complete the joint model. The quantity $p_{X|Z}$ is known as the propensity score. We refer to the original paper for a detailed discussion of these quantities, which in turn are expressed in terms of finite-dimensional parameters, say $\psi$ and $\lambda$. The vector parameter $\psi$ with true value ${\psi }^{\ast }$ comprises ${\varphi }_{YZ|X}^{\ast }$ and the main component of interest ${\theta }_{Y|X}^{\ast }$ which together specify $p_Z{p}_{Y|ZX}$, while $\lambda$ characterizes the propensity score $p_{X|Z}$. A consequence of the above parameterization within the class of marginal structural models is Theorem 5.1 of ref. 8, which establishes consistency and asymptotic normality of the maximum likelihood estimator $\widehat{\psi }$ of ${\psi }^{\ast }$, even if $p_{X|Z}=p_{X|Z}(\lambda )$ is misspecified. As noted in ref. 8, the conclusion arises as a consequence of the parameter cut between $p_Z{p}_{Y|ZX}$ and $p_{X|Z}$, and therefore between $\psi$ and $\lambda$. The parameter cut implies the parameter orthogonality condition ${\psi }^{\ast }{\perp }_m\mathrm{\Lambda }$, and Evans and Didelez (8) implicitly establish in their proof the remaining condition of Proposition 1.1.

3. Discussion

3.A. Stronger Inferential Guarantees.

Even if consistency holds, inferential guarantees entail estimation of variance, a difficult problem due to failure of Bartlett’s second identity. Such difficulties persist under the stronger condition of Proposition 1.3.

In the notation of Section 1, let

$$\begin{array}{cc}\hfill q& ={\mathbb{E}}_m[({\mathrm{\nabla }}_{(\psi ,\lambda )}\ell ){({\mathrm{\nabla }}_{(\psi ,\lambda )}\ell )}^{\text{T}}],\hfill \\ \hfill \check{q}& ={\mathbb{E}}_{(\psi ,\lambda )}[({\mathrm{\nabla }}_{(\psi ,\lambda )}\ell ){({\mathrm{\nabla }}_{(\psi ,\lambda )}\ell )}^{\text{T}}],\hfill \end{array}$$

both functions of $\psi$ and $\lambda$. Proposition 1.3, Eq. 1.1 and consistency of $\widehat{\psi }$ imply

$${\mathbb{E}}_{(\psi ,\lambda )}[{\mathrm{\nabla }}_{(\psi ,\lambda )}\ell ({\psi }^{\ast },{\lambda }_m^0)]=0,$$

and differentiation gives Bartlett’s second identity under the assumed model:

$$-\check{ı}({\psi }^{\ast },{\lambda }_m^0)+\check{q}({\psi }^{\ast },{\lambda }_m^0)=0.$$

The relevant quantities for establishing the limiting distributions of standard test statistics are, however, $i$ and $q$, where $i$ was defined in Section 1. These do not in general satisfy $-i({\psi }^{\ast },{\lambda }_m^0)+q({\psi }^{\ast },{\lambda }_m^0)=0$, even with the additional structure of Proposition 1.3. An exception is Example 2.2 because of the particular simplicity of this case.

It can be shown using now standard arguments that, with the log-likelihood function constructed from $n$ independent observations, provided that $\widehat{\psi }$ is consistent for ${\psi }^{\ast }$,

$$\begin{array}{c}\hfill \sqrt{n}(\widehat{\psi }-{\psi }^{\ast }){\to }_{d}N(0,{(i^{-1}q{i}^{-1})}_{\psi \psi }),\end{array}$$ [3.1]

the variance being the so-called sandwich formula, in which the constituent terms are evaluated at $({\psi }^{\ast },{\lambda }_m^0)$. This reduces to $i^{\psi \psi }({\psi }^{\ast },{\lambda }_m^0)=i^{\psi \psi }({\psi }^{\ast },{\lambda }^{\ast })$ when the model is correctly specified, so that the usual asymptotic result is recovered. The standard refs. 3, 5–7, and 23 have ${\psi }_m^0$ in place of ${\psi }^{\ast }$ in Eq. 3.1. Qualitatively similar results hold for the profile score and profile likelihood ratio statistics, the main point being that confidence set estimation is infeasible without knowledge of $m$ unless $\check{q}({\psi }^{\ast },{\lambda }_m^0)=q({\psi }^{\ast },{\lambda }_m^0)$ to some adequate order of approximation. In general, this requires, not only that the expectations of the sufficient statistics are robust to misspecification but also that the expectations of any relevant squares and cross terms are stable.

Suppose, however, that the parameterization is orthogonal in the sense ${\psi }^{\ast }{\perp }_m{\lambda }_m^0$. Then,

$${(i^{-1}q{i}^{-1})}_{\psi \psi }=i^{\psi \psi }{q}_{\psi \psi }{i}^{\psi \psi },$$

and the familiar result is also recovered if $q_{\psi \psi }{i}^{\psi \psi }={\text{I}}_{\text{dim}(\psi )}$, a weaker requirement than $q=i$ especially if $\psi$ is a scalar. Under the additional structure of Proposition 1.3, inference based on the assumed model is unaffected by the misspecification.

3.B. Treating Incidental Parameters As Fixed or Random.

In the context of Section 2.A, the lack of general inferential guarantees beyond consistency may well be an argument for treating the nuisance parameters as fixed. Although the conceptual distinction is consequential for the analysis, a formulation in which incidental parameters are treated as fixed and arbitrary, and one in which they are treated as independent and identically distributed from a totally unspecified distribution, are numerically indistinguishable. In this sense, the fixed-parameter formulation is essentially nonparametric for the nuisance component, with the distinguishing feature that it evades estimation of the infinite-dimensional nuisance parameter.

The group structure used to define the symmetric parameterization in Proposition 2.1 also allows elimination of nuisance parameters by suitable preliminary maneuvers when the nuisance parameters are treated as fixed. In the exponential matched pairs setting of Example 2.1, (14) compared inference based on the distribution of $Y_{i1}/Y_{i0}$ to that based on modeling the pair effects by a gamma distribution. The random pair effects model is more efficient when the gamma distribution is correct, but efficiency degrades substantially under misspecification, compared to maximum likelihood estimation based on the distribution of ratios.

Proposition 2.1 hinges on a symmetric parameterization having been chosen, which is possible only under the group structure of Section 1.C. Outside of this setting, there are no guarantees of consistency of the maximum likelihood estimator with misspecified random effects distribution, while it may still be possible to eliminate the pair effects with exact or approximate conditioning arguments, as illustrated in the second example of ref. 14.

3.C. Overstratification and Other Encompassing Models.

Two essentially equivalent ways to adjust for potential confounders are to stratify by observed explanatory features, leading to strata effects ${({\gamma }_j)}_{j=1}^m$ as in Example 2.3, or to adjust for the explanatory features in a regression analysis. Both approaches are a form of conditioning by model formulation. It is arguably more relevant in this context to treat the strata effects as fixed rather than random. In De Stavola and Cox (24), this modification of Example 2.3 was considered, showing the extent to which overstratification decreases efficiency of the estimator. As is intuitively clear, no bias is incurred through overstratification, yet the conditions of Propositions 1.1 and 1.2 do not always hold, as illustrated by the following example. The explanation is that the true density is nested within the encompassing model, so that ${\lambda }_m^0={\lambda }^{\ast }=0$. The overstratified model is therefore not misspecified according to the definition below Eq. 1.1.

To isolate the point at issue, we assume that any dispersion parameters are equal to 1, the conclusion of Proposition 3.1 being unchanged for arbitrary dispersion parameter by the discussion of Section 2.C.

Proposition 3.1

Suppose that, conditional on observed explanatory features, the probability density or mass function of $Y=(Y_1,\dots ,Y_n)$ at $y=(y_1,\dots ,y_n)$ is of canonical exponential family regression form, that is,

$$\begin{array}{cc}& m(y; x_1^{\text{T}}{\psi }^{\ast },\dots ,x_n^{\text{T}}{\psi }^{\ast })\hfill \\ \hfill & =exp\{{\displaystyle \sum _{i=1}^n}{y}_i{x}_i^{\text{T}}{\psi }^{\ast }-{\displaystyle \sum _{i=1}^n}K(x_i^T{\psi }^{\ast })\}{\displaystyle \prod _{i=1}^n}h(y_i).\hfill \end{array}$$

Suppose further that the analysis is overstratified so that under the assumed model the log-likelihood function is $\ell (\psi ,\lambda )=s^{\text{T}}\psi +t^{\text{T}}\lambda -K(\psi ,\lambda )$, where $s=\sum _{i=1}^n{x}_i{y}_i$, $t=\sum _{i=1}^n{w}_i{y}_i$ and $K(\psi ,\lambda )=\sum _{i=1}^{n}K(x_i^{\text{T}}\psi +w_i^{\text{T}}\lambda )$. The conditions of Propositions 1.1 and 1.2 are in general violated, yet the maximum likelihood estimator $\widehat{\psi }$ is consistent for ${\psi }^{\ast }$.

Another type of encompassing model arises in random effects models when the assumed family of distributions for the random effects is rich enough to include the true distribution, for instance if the postulated mixture class is nonparametric [Kiefer and Wolfowitz (25)], as discussed in Section 3.B. This type of situation can be understood in terms of the convex geometry of mixture distributions [Lindsay (26)] and was specialized to the case of binary matched pairs with logistic probabilities by Neuhaus et al. (27). Their conditions on the mixing distributions with regard to the resulting cell probabilities effectively imply that the model is correctly specified according to the definition below Eq. 1.1.

3.D. Estimating Equations and Neyman Orthogonality.

A generalization of the score Eq. 1.1 is to other estimating equations, often but not always obtained as the vector of partial derivatives of a convex loss function. The model need only be partially specified in terms of a known function $h$ of the data, an interest parameter $\psi$ and a nuisance parameter $\lambda$, such that the expectation when the assumed model is true satisfies ${\mathbb{E}}_m[h({\psi }^{\ast },{\lambda }^{\ast }; Y)]={\mathbb{E}}_{({\psi }^{\ast },{\lambda }^{\ast })}[h({\psi }^{\ast },{\lambda }^{\ast }; Y)]=0$. If the model is misspecified the limiting solutions satisfy, in analogy with Eq. 1.1,

$${\mathbb{E}}_m[h({\psi }_m^0,{\lambda }_m^0; Y)]=0.$$

A special choice of $h$, establishing a connection to the double robustness literature, is the Neyman-orthogonal score for $\psi$, defined as

$$s_{\text{N}}^{\ast }(\psi ; \lambda ,Y)={\ell }_{\psi }(\psi ,\lambda )-w^{\ast \text{T}}{\ell }_{\lambda }(\psi ,\lambda ),$$

where ${\ell }_{\psi }$ and ${\ell }_{\lambda }$ are the partial derivatives of $\ell$ with respect to $\psi$ and $\lambda$ and $w^{\ast \text{T}}$ is a matrix of dimension $\text{dim}(\psi )\times \text{dim}(\lambda )$ given by $w^{\ast \text{T}}=i_{\psi \lambda }^{\ast }{i}_{\lambda \lambda }^{\ast -1}$ when the model is correctly specified. Here, $i_{\psi \lambda }^{\ast }$ and $i_{\lambda \lambda }^{\ast }$ are the components of the Fisher information matrix at the true parameter values $({\psi }^{\ast },{\lambda }^{\ast })$. Consider instead

$$\begin{array}{c}\hfill s_{\text{N}}(\psi ; \lambda ,Y)={\ell }_{\psi }(\psi ,\lambda )-w^{\text{T}}{\ell }_{\lambda }(\psi ,\lambda ),w^{\text{T}}=i_{\psi \lambda }{i}_{\lambda \lambda }^{-1},\end{array}$$ [3.2]

where $i_{\psi \lambda }$ and $i_{\lambda \lambda }$ are as defined in Eq. 1.4. Write the condition $i^{\psi \psi }{g}_{\psi }+i^{\psi \lambda }{g}_{\lambda }=0$ of Proposition 1.2 as

$$\begin{array}{cc}\hfill g_{\psi }+i_{\psi \psi .\lambda }{i}^{\psi \lambda }{g}_{\lambda }& =0,\hfill \\ \hfill i_{\psi \psi .\lambda }& ={(i^{\psi \psi })}^{-1}=i_{\psi \psi }-i_{\psi \lambda }{i}_{\lambda \lambda }^{-1}{i}_{\lambda \psi },\hfill \end{array}$$ [3.3]

where $i_{\psi \psi .\lambda }$ is the Fisher information for $\psi$ at $({\psi }^{\ast },\lambda )$, computed under the true model, having adjusted for estimation of $\lambda$. On noting that $i^{\psi \lambda }=-i^{\psi \psi }{i}_{\psi \lambda }{i}_{\lambda \lambda }^{-1}$, we see that the information identity $i^{\psi \psi }{g}_{\psi }+i^{\psi \lambda }{g}_{\lambda }=0$ of Proposition 1.2 is equivalent to requiring that the Neyman orthogonal score Eq. 3.2, with score adjustment $w$ computed under the true distribution, has zero expectation under the same distribution for all $\lambda \in \mathrm{\Lambda }$. Specification of $w$ in Eq. 3.2 when the model is misspecified thus hinges on the conditions of Proposition 1.3 being satisfied.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

There are no data underlying this work.