Inference Using Sample Means of Parametric Nonlinear Data Transformations

In empirical HSR, statistics of key analytic interest are often of the following general form

$$\widehat{\mathit{\gamma }}=\sum _{i=1}^N\frac{\mathrm{g}(\widehat{\mathit{\beta }},{\mathrm{X}}_{\mathrm{i}})}{N}$$ (1)

where $\mathit{\gamma }=\mathrm{E}[\mathrm{g}(\mathit{\beta },\mathrm{X})]$ is the parameter of ultimate interest to be estimated by equation (1), g( ) is a known (possibly nonlinear) transformation, $\widehat{\mathit{\beta }}$ is a pre‐estimate of β—a vector of “deeper” model parameters, and X _i denotes a vector of observed data on X for the ith member of a sample of size n (i = 1,…, N). The three most commonly encountered formulations of equation (1)—average treatment effect (ATE), average marginal effect (AME), and average incremental effect (AIE)—correspond to the following formulations of g( ), respectively

$$\mathrm{m}(\mathit{\beta },1,X_o)-\mathrm{m}(\mathit{\beta },0,X_o)$$ (2)

$$\mathrm{g}(\mathit{\beta },\mathrm{X})=\frac{\mathit{\partial }\mathrm{m}(\mathit{\beta },X_p,X_o)}{\mathit{\partial }{X}_p}$$ (3)

$$\mathrm{m}(\mathit{\beta },X_p+\mathrm{\Delta },X_o)-\mathrm{m}(\mathit{\beta },X_p,X_o)$$ (4)

where $\mathrm{m}(\mathit{\beta },X_p,X_o)=E[Y|X_p,X_o]$ is a regression function written so as to highlight the distinction between a policy‐relevant regressor of interest, X _p, and a vector of regression controls, X _o; $\mathrm{X}=[X_p{X}_o]$; β is a vector of regression parameters, and Δ is a known exogenous (usually policy‐driven) increment to X _p. After the regression parameter estimates are obtained [e.g., $\widehat{\mathit{\beta }}$—estimated via the nonlinear least (NLS) method], under fairly general conditions, in conjunction with equation (1), the formulations in equation (2), (3), and (4), respectively yield consistent estimators of the ATE when X _p is binary; the AME when X _p is continuous and interest is in the effect attributable to an infinitesimal policy change; and the AIE when X _p is discrete or continuous and the relevant policy increment is Δ. In this note, we focus on the specification and computation of the correct “t‐statistic” for equation (1) as derived from standard asymptotic theory. This t‐statistic has the following general form

$$\frac{\sqrt{N}(\widehat{\mathit{\gamma }}-{\mathit{\gamma }}^{†})}{se(\widehat{\mathit{\gamma }})}$$ (5)

where ${\mathit{\gamma }}^{†}$ is the relevant “null” value of γ (as in a test of the null hypothesis ${\mathrm{H}}_0:\mathit{\gamma }={\mathit{\gamma }}^{†}$), and $se(\widehat{\mathit{\gamma }})$ is the asymptotic standard error of equation (1) defined as $se(\widehat{\mathit{\gamma }})\equiv \sqrt{\widehat{\mathrm{avar}}(\widehat{\mathit{\gamma }})}$, with $\widehat{a\mathrm{var}}(\widehat{\mathit{\gamma }})$ being a consistent estimator of the asymptotic variance of $\widehat{\mathit{\gamma }}$. Under slightly stronger conditions than those required for the consistency of equation (1), it can be shown that equation (5) is asymptotically standard normal distributed. In the remainder of this note, we take the consistency and asymptotic normality of $\widehat{\mathit{\gamma }}$ as given, and concentrate on the correct formulation of $se(\widehat{\mathit{\gamma }})$ as derived from standard asymptotic theory. In the Appendix, we show that for most (if not all) of the useful forms of equation (1)

$$\widehat{a\mathrm{var}}(\widehat{\mathit{\gamma }})=A+B$$ (6)

where$A=\left(\frac{\sum _{i=1}^N{\mathrm{\nabla }}_{\mathit{\beta }}\mathrm{g}(\widehat{\mathit{\beta }},{\mathrm{X}}_i)}{N}\right)\widehat{\mathrm{AVAR}}(\widehat{\mathit{\beta }}){\left(\frac{\sum _{i=1}^N{\mathrm{\nabla }}_{\mathit{\beta }}\mathrm{g}(\widehat{\mathit{\beta }},{\mathrm{X}}_i)}{N}\right)}^{\prime }$

$$B=\frac{\sum _{i=1}^N{\left(\mathrm{g}(\widehat{\mathit{\beta }},{\mathrm{X}}_i)-\widehat{\mathit{\gamma }}\right)}^2}{N}$$

${\mathrm{\nabla }}_{\mathit{\beta }}\mathrm{g}(\widehat{\mathit{\beta }},{\mathrm{X}}_i)$ (a row vector) denotes the gradient of $\mathrm{g}(\mathit{\beta },\mathrm{X})$evaluated at X_i and $\widehat{\mathit{\beta }}$, and $\widehat{\mathrm{AVAR}}(\widehat{\mathit{\beta }})$ is an estimator of the asymptotic covariance matrix of $\widehat{\mathit{\beta }}$. Dowd, Greene, and Norton (2014) opine that inclusion of B in equation (6) “seems incorrect to us” and exclude it from the suggested formulation of the asymptotic standard error of $\widehat{\mathit{\gamma }}$ given in their equation (18). That the inclusion of B is not “incorrect” is proven by the derivation in an appendix that is included among the supplementary materials for this paper.1, 2

So how does our derivation of equation (6) differ from the approach taken by Dowd, Greene, and Norton (2014) in deriving their equation (18)? To answer we must take a closer look at the sampling assumptions underlying the respective derivations. Unlike Dowd, Greene, and Norton (2014), we assume that the sample observations for all the relevant variables, including X, are drawn randomly from the relevant joint distribution for the population of interest. We impose no sampling restriction on X and we allow it to be random—the same assumption that we make for the other elements of the data vector. For example, for the case in which $\widehat{\mathit{\beta }}$ is obtained by regressing Y on X (based on a correctly specified nonlinear model), we treat both Y and X as random in sampling. What we have described here is simple random sampling (SRS), which is clearly the most commonly encountered type of sampling in empirical HSR. Moreover, we adopt the conventional approach to deriving the asymptotic properties of $\widehat{\mathit{\gamma }}$ (in particular, its asymptotic standard error). Conventional asymptotic theory assumes SRS and focuses on the limiting properties of estimators as the sample size (N in our case) approaches ∞. (The analysis assumes that the same sample is used to estimate β and the mean effect on m( ), and that the objective of the analysis involves generalization to a population that is potentially large compared to the sample; equation (6) can be modified for alternative assumptions).

Dowd, Greene, and Norton (2014), on the other hand, supplant SRS with an unrealistic fixed‐in‐repeated‐sampling assumption (FIRS) in which the matrix of observations on X (say χ with N rows and K columns; K being the number of regressors in X) is fixed (nonrandom) so that, in sampling, only the observations on Y are randomly drawn. Moreover, they assume that increases in the sample size are not effected by increasing N but instead by holding it and χ fixed and drawing repeated observations on Y for each of the fixed rows of χ. Denote the number of such FIRS observations on Y as N*. Their formulation of the asymptotic standard error of $\widehat{\mathit{\gamma }}$ is obtained while fixing χ and N and allowing N* to approach ∞. Both the FIRS and its attendant asymptotics are unrealistic and irrelevant in the present context because there are no empirical contexts in HSR for which this assumption could be reasonably maintained. The FIRS assumption is characteristic of an experiment where no generalization is intended beyond the specific designed distribution of X, rather than of an analysis of a random sample drawn from and intended to be representative of a larger population.

Given that (a) the formulation of the asymptotic standard error of $\widehat{\mathit{\gamma }}$ in equation (6) is realistic, relevant, and indeed correct; (b) the practical significance of including B can only be conclusively evaluated in the context of each particular empirical application after it has been estimated; and (c) the calculation of B imposes only minimal marginal computational burden, there remains no reasonable justification for excluding it from equation (6) as Dowd, Greene, and Norton (2014) recommend in their equation (18).

Supporting information

Appendix S1: Derivation of Equation (6).