Nonparametric estimation of multivariate scale mixtures of uniform densities

Abstract

Suppose that U = (U₁, … , U_d) has a Uniform ([0, 1]^d) distribution, that Y = (Y₁, … , Y_d) has the distribution G on ${\mathbb{R}}_{+}^d$, and let X = (X₁, … , X_d) = (U₁Y₁, … , U_dY_d). The resulting class of distributions of X (as G varies over all distributions on ${\mathbb{R}}_{+}^d$) is called the Scale Mixture of Uniforms class of distributions, and the corresponding class of densities on ${\mathbb{R}}_{+}^d$ is denoted by ${\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$. We study maximum likelihood estimation in the family ${\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$. We prove existence of the MLE, establish Fenchel characterizations, and prove strong consistency of the almost surely unique maximum likelihood estimator (MLE) in ${\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$. We also provide an asymptotic minimax lower bound for estimating the functional f ↦ f(x) under reasonable differentiability assumptions on f ∈ ${\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$ in a neighborhood of x. We conclude the paper with discussion, conjectures and open problems pertaining to global and local rates of convergence of the MLE.

1. Introduction and summary

Fix a non-negative integer k, and suppose that X₁, … , X_n are i.i.d. random variables distributed according to a density in the convex family of k-monotone densities (with respect to Lebesgue measure) on (0, ∞):

$${\mathcal{F}}_k≔\left\{f_{k,G}(\cdot )\equiv {\int }_0^{\infty }\phantom{\mid }k\frac{{(y-\cdot )}_{+}^{k-1}}{y^k}dG\left(y\right)\mid G\in {\mathcal{g}}_1\right\},$$ (1.1)

where ${\mathcal{g}}_1$ will denote the set of all distribution functions on (0, ∞) grounded at 0. Here, we use the notation x₊ ≡ x · 1_[x≥0] for any $x\in \mathbb{R}$. It has been shown by Williamson [59] that the family ${\mathcal{F}}_k$ is identifiably indexed by ${\mathcal{g}}_1$. In other words, if G₁, G₂ are distinct elements in ${\mathcal{g}}_1$, then f_k,G1 (·) and f_k,G2 (·) differ on a Lebesgue non-null set. Note that ${\mathcal{F}}_k$ is exactly the collection of all scale mixtures of Beta (1, k) densities.

The Beta (1, 1) distribution is the standard uniform distribution, U(0, 1). Therefore, the class ${\mathcal{F}}_1$ coincides with the class of all scale mixtures of uniform densities on (0, ∞). A well-known theorem by Khintchine (see, e.g., [16, p.158]) asserts that the class of densities on (0, ∞) with concave distribution functions is one and the same with our class ${\mathcal{F}}_1$. It can be seen that ${\mathcal{F}}_1$ is also the class of all upper semi-continuous, non-increasing densities on (0, ∞). This class is induced by order restrictions, a term we use to explicitly mean that there exists a partial ordering (⪡) on the common support $\mathcal{X}$ of the densities in ${\mathcal{F}}_1$ such that f ∈ ${\mathcal{F}}_1$ if and only if f is isotone with respect to this ordering: i.e., f ∈ ${\mathcal{F}}_1$ if and only if f (x) ≤ f (y) whenever x , y ∈ $\mathcal{X}$ such that x ⪡ y. In this case, (⪡) is the natural partial ordering, ≥, on (0, ∞).

Non-increasing, upper semi-continuous densities (in short, monotone densities) arise naturally via connections with renewal theory and uniform mixing (see, e.g., [60]). Maximum likelihood estimation of monotone densities on (0, ∞) was initiated by Grenander [18,19], with related work by Ayer et al. [3], Brunk [11], van Eeden [51-55]. Asymptotic theory of the MLE in ${\mathcal{F}}_1$ (the Grenander estimator) was developed by Prakasa Rao [44] with later contributions by [20,21,8,9,30]. See [4] for descriptions of the behavior of the Grenander estimator at zero.

Nonparametric estimation in families of densities described by order restrictions goes back at least to the work of [18,19,11,12,45], with further development by Wegman [56-58], Sager [48,49]. Also see the books by Barlow et al. [5] and Robertson et al. [46]. [40-43] addressed estimation in various order restricted classes of multivariate densities from the perspective of the excess mass approach studied previously by e.g., [48,49,36]. Polonik shows that (under reasonable assumptions) the MLE in such classes exists and coincides with an estimator he constructs and calls the silhouette. Forcing the elements of the class to be upper semi-continuous, the MLE is seen to be unique. Brunk [11] also gives a graphical construction of the maximum likelihood estimator, and establishes L₁-consistency of the MLE.

In this paper, our goal is to extend the notion of “monotone densities” to higher dimensions; i.e., to densities on (0, ∞)^d with d > 1. Such an extension is not unique: for example, we may consider the family, ${\mathcal{F}}_{\mathrm{BDD}}\left(d\right)$, of “block-decreasing densities” (a term coined by Biau and Devroye [6]) that contains all upper-semicontinuous densities on (0, ∞)^d that are non-increasing in each coordinate, while keeping all other coordinates fixed. This class was perhaps first introduced by Robertson [45]. The particular proper subclass of ${\mathcal{F}}_{\mathrm{BDD}}\left(d\right)$ studied here is the family ${\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$ of all multivariate scale mixtures of uniform densities; i.e., the family of upper semi-continuous densities on (0, ∞)^d of the form

$$f_G\left(\mathit{x}\right)={\int }_{{(0,\infty )}^d}\left(\frac{1}{\mid \mathit{y}\mid }{1}_{(0,y]}\left(\mathit{x}\right)\right)dG\left(\mathit{y}\right),\mathit{x}\in {(0,\infty )}^d$$ (1.2)

for some $G\in {\mathcal{g}}_d$, the set of all distribution functions on (0, ∞)^d that are grounded (zero) at 0; here we use the notation $\mid y\mid \equiv {\prod }_{i=1}^d{y}_i$ for y = ( y₁, … , y_d)’ ∈ (0, ∞)^d. For any fixed $G\in {\mathcal{g}}_d$, it is clear that if Y = (Y₁, … , Y_d)’ is distributed according to G on (0, ∞)^d and if U₁, … , U_d are i.i.d. U(0, 1) (and independent of Y), then the vector X := (U₁Y₁, … , U_dY_d) is distributed according to f_G(·) on (0, ∞)^d.

Whereas the family ${\mathcal{F}}_{\mathrm{BDD}}\left(d\right)$ is characterized by order restrictions (and thus the results by Polonik apply), its subclass ${\mathcal{F}}_{\mathrm{SMU}}$ is not; as will be made more explicit in Section 2, densities in the class ${\mathcal{F}}_{\mathrm{SMU}}$ also satisfy non-negativity restrictions on their d-dimensional differences around all rectangles. Because of this additional shape restriction, estimation in this family requires separate treatment

A univariate parallelism to the latter point would be to consider the family ${\mathcal{F}}_2$ in (1.1), induced by mixtures of triangular densities; this class can easily be seen to be exactly the class of all non-increasing, convex (and hence continuous) densities on (0, ∞). Thus ${\mathcal{F}}_2\subset {\mathcal{F}}_1$ is not an order-constrained class of densities, in contrast to its superclass ${\mathcal{F}}_1$. Convex densities arise in connection with Poisson process models for bird migration and scale mixtures of triangular densities (see, e.g., [26,2,32]). Estimation of non-increasing, convex densities on (0, ∞) was apparently initiated by Anevski [1] and was further pursued by Anevski [2] and Jongbloed [28]. The asymptotic distribution theory and further characterizations of the nonparametric MLE of such a density and its first derivative at a fixed point (both under reasonable assumptions) was obtained by Groeneboom et al. [24,25]. These authors show that the local rate of convergence of the MLE of the functional f ↦ f (x) is of the order n^⅖, whereas the Grenander estimator (the MLE in ${\mathcal{F}}_1$) converges locally at the rate of only n^⅓.

The developments here have several motivations. One of these is to provide a multivariate family of shape-constrained densities with convergence rates for reasonable estimators which are (nearly) independent of the dimension d of the underlying space. As will be seen from the lower bound calculations in Section 4, it seems that the SMU class studied here may provide such a class. Another motivation comes from problems concerning multivariate analogues of interval censored data; see e.g. [27,61,62]. These apparently quite different models involve very similar mathematical considerations, and it might be helpful to develop methods for multivariate interval censored data problems by first studying the somewhat simpler SMU model.

Here is an outline of the remainder of the present paper. In Section 2, we provide characterizations of the family ${\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$ that will prove useful in the sequel. Section 3 addresses existence, strong, pointwise consistency as well as L₁ and Hellinger consistency of a sequence of maximum likelihood estimators in ${\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$. In Section 4, we derive a local asymptotic minimax lower bound for estimation of f (x) at a fixed point x under for which f satisfies ∂^df (x)/(∂x₁ … ∂x_d) ≠ 0. The lower bound entails a rate of convergence of n^⅓ for all dimensions d and yields a constant depending on f which reduces to the known lower bound constant for d = 1. The paper concludes in Section 5 with a discussion of conjectures and open problems related with both the local (pointwise) and the global (L₁ and Hellinger) rates of convergence of the MLE in ${\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$.

2. Properties of the Scale Mixtures of Uniform family of densities

2.1. Properties of ${\mathcal{F}}_{SMU}\left(d\right)$

A density function, f , on (0, ∞)^d will be called a (multivariate) Scale Mixture of Uniform densities if there exists a distribution function, G, on (0, ∞)^d such that

$$f\left(\mathit{x}\right)=f_G\left(x\right)={\int }_{{(0,\infty )}^d}\frac{1}{\mid \mathit{v}\mid }{1}_{\left(0,v\right]}\left(\mathit{x}\right)dG\left(\mathit{v}\right)$$ (2.1)

$$={\int }_{\mathit{v}\ge \mathit{x}}\frac{1}{\mid \mathit{v}\mid }dG\left(\mathit{v}\right)\text{for all}\mathit{x}\in {(0,\infty )}^d.$$ (2.2)

It is clear from (2.2) that a SMU density is also a block-decreasing density: f_G(·) is non-increasing in each coordinate, while keeping all other coordinates fixed. Also, the map G ↦ f_G is identifiable in the following sense: if G₁ ≠ G₂, then f_G1 ≠ f_G2 on a set of positive Lebesgue measure; also see Theorem 2.3 below. The following lemma gives a formal statement of a slightly more general result. The proof is standard.

Lemma 2.1

Two upper semi-continuous and block-decreasing functions f and g on ${\mathbb{R}}^d$ differ nowhere in the interior of their support or else on a Lebesgue non-negligible set.

The distribution function F_G corresponding to X ~ f_G is given by

$$F_G\left(\mathit{x}\right)={\int }_{{(0,\infty )}^d}\frac{\mid \mathit{x}\wedge \mathit{v}\mid }{\mid \mathit{v}\mid }dG\left(\mathit{v}\right),$$ (2.3)

where ≤ denotes the natural partial ordering on ${\mathbb{R}}^d$, while

$$\mathit{x}\wedge \mathit{v}\equiv (x_1,\dots ,x_d)\wedge (v_1,\dots ,v_d)=(\min \{x_1,v_1\},\dots ,\min \{x_d,v_d\}),$$

and x∨ν ≡ (x1, … , x_d)∨(ν₁, … , ν_d) = (max{x₁, ν₁}, … , max{x_d, ν_d}). The distribution function F_G of X is generally not concave when d > 1, unlike the case when d = 1. An SMU density (and a block-decreasing density, in general) can possibly diverge at the origin, whereas the pointwise bound f (x) ≤ 1/|x| holds since, for x ∈ (0, ∞)^d we have

$$1={\int }_{{(0,\infty )}^d}f\left(\mathit{y}\right)\mathit{dy}\ge {\int }_{\left(0,\mathit{x}\right]}f\left(\mathit{y}\right)d\mathit{y}\ge \mid x\mid f\left(\mathit{x}\right).$$

Further, a d-dimensional analogue of the proof of [13, Theorem 6.2, p. 173] can be used to show that

$$\underset{\mid \mathit{x}\mid \to \infty }{\lim }\left\{\mid x\mid f\left(\mathit{x}\right)\right\}=\underset{\mathit{x}\downarrow 0}{\lim }\left\{\mid \mathit{x}\mid f\left(\mathit{x}\right)\right\}=0,$$ (2.4)

whenever f is a block-decreasing density on (0, ∞)^d.

For any two points x, y ∈ [0, ∞)^d, such that x ≤ y, we write [x, y] ≡ [x₁, y₁] × … × [x_d, y_d], [x, y) ≡ [x₁, y₁) × … × [x_d, y_d), (x, y] ≡ (x₁, y₁] × … × (x_d, y_d], (x, y) ≡ (x₁, y₁) × … × (x_d, y_d) for the natural closed, lower-closed upper open, lower open upper closed, and open rectangles respectively. Note that the closed rectangle [x, y] has (at most) 2^d vertices, the points u = (u₁, … , u_d) where each u_i is either x_i or y_i. Following [7], we write sgn_[x,y](u) ∈ {−1, 1}, the signum of the vertex u, according as the number of i, 1 ≤ i ≤ d, satisfying u_i = x_i is odd or even respectively.

Thus any two vertices defining an edge of the rectangle have alternating signs. Then, if u = (u₁, … , u_d) is some vertex of [x, y] and δ ∈ {−1,+1} is its signum, then (δ, u) is an element of the set

$${\Delta }_d\left[x,y\right]=\left\{\left({(-1)}^{\sum _{i=1}^d\left\{1_{[u_i=x_i]}\right\}},\mathit{u}\right)\mid \mathit{u}\in \{x_1,y_1\}\times \dots \times \{x_d,y_d\}\phantom{\mid }\right\}.$$

Definition 2.1

For an upper semicontinuous and coordinatewise decreasing function g : (0, ∞)^d → [0, ∞) define the g-volume of a (possibly degenerate) rectangle [x, y) by:

$$V_g\left[x,y\right)=\sum _{(\delta ,\mathit{u})\in {\Delta }_d\left[x,y\right]}\left\{\delta g\left(\mathit{u}\right)\right\},$$ (2.5)

provided that g is defined and is finite for all u in the summand. Correspondingly, for an upper semicontinuous and coordinatewise increasing function g : (0, ∞)^d → [0, ∞), we define the g-volume of a rectangle (x, y] by the sum on the right side of (2.5).

It is easily seen that for an SMU density, f_G, the f_G-volume of any rectangle [x, y) is always of the sign (−1)^d: indeed, consider (2.2) and observe that

$${(-1)}^d{V}_{fG}\left[x,y\right)={\int }_{[x,y)}\frac{1}{\mid \mathit{v}\mid }dG\left(\mathit{v}\right)\ge 0.$$ (2.6)

From (2.6), or, alternatively, from the fact that the class of sets [x, y) is a π-system which generates the Borel σ-field of subsets of [0, ∞)^d and then extending as in [7], it is clear that (−1)^dV_f extends uniquely to a (non-negative) measure on the Borel σ-field ${\mathcal{B}}_{+}^d={\mathcal{B}}^d\cap {(0,\infty )}^d$ given by

$${(-1)}^d{V}_f\left(A\right)={\int }_A\frac{1}{\mid \mathit{v}\mid }dG\left(\mathit{v}\right)\text{for}A\in {\mathcal{B}}_{+}^d;$$

in particular,

$${(-1)}^d{V}_f\left(x,y\right]={\int }_{(x,y]}\frac{1}{\mid \mathit{v}\mid }dG\left(\mathit{v}\right).$$

This argument extends easily to an arbitrary upper semicontinuous function g with the (−1)^dg-volumes of all rectangles [x, y) non-negative.

Lemma 2.2

Suppose that g is a non-negative, upper semi-continuous function satisfying (−1)^dV_g [x, y) ≥ 0 for all lower-closed upper open rectangles [x, y), and vanishing if any coordinate tends to ∞. Then (−1)^dV_g can be extended to a countably additive measure on ${\mathcal{B}}_{+}^d$.

Of course it is easy to exhibit a block-decreasing density that is not an SMU density: consider the uniform density on the closed triangle in ${\mathbb{R}}_{+}^2$ with vertices (0, 0), (0, 1) and (1, 0). Then,

$${(-1)}^2{V}_f[(1∕8,1∕8),(1∕2,3∕4))=-2<0,$$

showing that this density is not an SMU density, even though it is block-decreasing.

The following theorem establishes identifiability of the mixing distribution G as well as providing a useful characterization of SMU densities.

Theorem 2.3

1. For the class of SMU densities ${\mathcal{F}}_{SMU}\left(d\right)=\{f_G:G\in {\mathcal{g}}_d\}$ with f_G as given in (2.1), f ∈ ${\mathcal{F}}_{SMU}\left(d\right)$ if and only if f ≡ f_G, where G ∈ G_d is given by

   $$G\left(\mathit{x}\right)={\int }_{{(0,\infty )}^d}{(-1)}^d{V}_f\left(u,x\right]\cdot 1_{[u\le x]}d\mathit{u}.$$ (2.7)

   Thus there is a one-to-one correspondence between $G\in {\mathcal{g}}_d$ and $f_G\in {\mathcal{F}}_{SMU}\left(d\right)$.

2. Suppose that the Lebesgue density f on (0, ∞)^d is such that it converges to zero in each coordinate, while keeping all other coordinates fixed. Then, f is an SMU density if and only if (−1)^dV_f [x, y) ≥ 0 for all 0 ≤ x ≤ y.

Proof. (a) Suppose that f ≡ f_G, for $G\in {\mathcal{g}}_d$ (recall that this implies that G(0) = 0), is an SMU density evaluated at an arbitrary x ∈ (0, ∞)^d as:

$$f\left(\mathit{x}\right)={\int }_{{(0,\infty )}^d}\frac{1}{\mid \mathbf{y}\mid }{1}_{(0,\mathit{x}]}dG\left(\mathit{y}\right)={\int }_{y_1\ge x_1}\cdots {\int }_{y_d\ge x_d}\frac{1}{\mid \mathbf{y}\mid }dG\left(\mathit{y}\right),$$ (2.8)

so that df (x) = (−1)^d|x|⁻¹ dG(x) and thus,

$$\begin{array}{cc}\hfill G\left(\mathit{x}\right)& ={\int }_{{(0,\infty )}^d}{1}_{\left(0,\mathit{x}\right]}\left(\mathit{y}\right)\mid y\mid d\left\{{(-1)}^{d}f\left(\mathit{y}\right)\right\}\hfill \\ \hfill & ={\int }_{(0,\mathit{x}]}{\int }_{(0,\mathit{x}]}{1}_{(0,\mathit{y}]}\left(\mathit{u}\right)d\mathit{u}d\left\{{(-1)}^{d}f\left(\mathit{y}\right)\right\}\hfill \\ \hfill & ={\int }_{(0,\mathit{x}]}\left\{{\int }_{\mathit{y}\in \left(u,x\right]}d\left\{{(-1)}^{d}f\left(\mathit{y}\right)\right\}\right\}d\mathit{u}\hfill \\ \hfill & ={\int }_{(0,\mathit{x}]}{(-1)}^d{V}_f\left(u,x\right]d\mathit{u},\hfill \end{array}$$

where the second to last equality follows by Fubini–Tonelli.

We will now show that G is unique: suppose that (2.8) above holds for $G=G_i\in {\mathcal{g}}_d$ and i = 1, 2. Recall that this implies that G₁(0) = G₂(0) = 0 and, thus, G₀(·) ≡ G₁(·) – G₂(·) is such that G₀(0) = 0, ∫_{(0, ∞)^d} G₀(x) dx = 0 and

$$0={\int }_{{(0,\infty )}^d}\frac{1}{\mid \mathit{y}\mid }{1}_{(0,\mathit{x}]}d{G}_0\left(\mathit{y}\right)={\int }_{(0,\mathit{x}]}\frac{1}{\mid \mathit{y}\mid }d{G}_0\left(\mathit{y}\right)$$ (2.9)

holds for all x ∈ (0, ∞)^d and, thus, necessarily G₀(x) has to be independent of x and therefore everywhere equal to its value at 0: G₀(0) = 0. This completes the assertion of uniqueness, since G₁ ≡ G₂.

(b) If f is in ${\mathcal{F}}_{\mathrm{SMU}}$, there exists $G\in {\mathcal{g}}_d$ such that

$$f\left(\mathit{x}\right)={\int }_{{(0,\infty )}^d}\frac{1}{\mid \mathit{y}\mid }{1}_{(0,\mathit{y}]}\left(\mathit{x}\right)dG\left(\mathit{y}\right)={\int }_{\mathit{y}\ge \mathit{x}}\frac{1}{\mid \mathit{y}\mid }dG\left(\mathit{y}\right),$$

so that it is easily seen that (−1)^dV_f [x, y) = ∫_[x,y) |y|⁻¹ dG(y) ≥ 0 holds true for all 0 ≤ x ≤ y.

On the other hand, assume that the Lebesgue density f is such that it converges to zero in each coordinate, while keeping all other coordinates fixed, and satisfies (−1)^dV_f [x, y] ≥ 0 for all 0 ≤ x ≤ y. By Lemma 2.2, this implies that for x₁ ≤ x₂ ≤ x, elements of (0, ∞)^d, we have (−1)^dV_f [x₁, x) ≥ (−1)^dV_f [x₂, x) and, letting x → ∞, this yields f (x₁) ≥ f (x₂) because we assumed that f vanishes as any one of its coordinates diverges to infinity, so that V_f [x_i, x) → (−1)^df (x_i) for i ∈ {1, 2}. Thus, f is block-decreasing.

Hence, by appealing to part (a), it thus suffices to show that G, as defined on (0, ∞)^d by (2.7) is a valid distribution function. Indeed, this is easily shown along the lines of the following sketch. In particular, (i) G is grounded at 0 trivially by inspection: G(0) = 0. (ii) By virtue of the fact that f is block-decreasing, 0 ≤ lim_|x|→∞ f (x) ≤ lim_|x|→∞{1/|x|} = 0 is is true and this can be used to show straightforwardly that lim_{x1∧…∧xd→∞} G(x₁, … , x_d) = 1. (iii) Similarly, it is an easy task to show that V_G(x, y] ≥ 0 for all 0 ≤ x ≤ y. Conditions (i)–(iii) are necessary and sufficient for G to be a bona-fide distribution function. This completes the proof.

2.2. Lebesgue measurability of block-decreasing functions

Now we note a technical fact concerning the (Lebesgue) measurability of block-decreasing functions which will be needed in our proofs in Section 3.2.

Proposition 2.4

Let f be a real-valued, non-negative function on (0, ∞)^d that is non-increasing and convergent to zero in each coordinate x_j, keeping all other coordinates fixed, as x_j coordinate tends to ∞. Then:

1. f is Lebesgue-measurable.

2. There exists such a function f that is not Borel-measurable. Such an f exists with f also satisfying sup{f (x) | x ∈ (0, ∞)^d} < ∞.

Proof. Proposition 2.4 (a) follows from Theorem 3 of [31]. Proposition 2.4 (b) is standard and follows from Proposition 1.2.2 in [50].

3. Existence and consistency of the MLE

Let X₁, … , X_n be i.i.d. random vectors distributed according to some density $f_0=f_{G_0}\in {\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$ where f₀ is unknown. Our goal is to estimate the unknown SMU density, f₀, based on X₁, … , X_n. We will be interested in maximizing the likelihood function $f\mapsto {\prod }_{i=1}^{n}f\left({\mathit{X}}_i\right)$ or, equivalently, the log-likelihood function $f\mapsto n{\mathbb{P}}_n\log \left\{f\left(\mathit{X}\right)\right\}$ over $f\in {\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$ where ${\mathbb{P}}_n=n^{-1}{\sum }_{i=1}^n{\delta }_{{\mathit{X}}_i}$ is the empirical measure of the data. Any such maximizer,${\widehat{f}}_n\in {\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$, should one exist, will be called a (nonparametric) maximum likelihood estimator of f₀, based on X₁, … , X_n. Since f₀ = f_G0 is given by (2.1) it follows from Theorem 2.3 that estimation of $f_0\in {\mathcal{F}}_{\mathrm{SMU}}$ is equivalent to estimation of G₀.

3.1. On existence and uniqueness of an MLE

We begin with a definition followed by the main theorem of this subsection.

Definition 3.1 (Rectangular Grid Generated by Data)

Suppose that x₁, … , x_n are ( fixed or random) elements in (0, ∞)^d and suppose that xi = (x_i1, … , x_id)’ where i = 1, 2, … , n. Define the matrix A = [x_ij] ∈ M_n×d((0, ∞)) whose ith row is exactly ${\mathit{x}}_i^{\prime }$, for i ∈ {1, 2, … , n}. Also let A^# = { (x(i₁),1, x(i₂),2, … , x(i_d),d) | i₁, … , i_d ∈ {1, 2, … , n}} denote the rectangular grid generated by A, where x_(i),j denotes the ith smallest element among x_1j, … , x_nj where i ∈ {1, 2, … , n} and j ∈ {1, 2, … , d}. In particular, x_* = (x_(1),1, x_(1),2, … , x_(1),d) and x_* = (x_(n),1, x_(n),2, … , x_(n),d) denote the element-wise minimum and maximum of x₁, … , x_n, respectively. For each fixed j ∈ {1, 2, … , d}, let n_j(A) := card({x_i,j | i = 1, 2, … , n}), and notice that we have: card$\left(A^{\#}\right)={\prod }_{j=1}^d{n}_j\left(A\right)\equiv N\le n^d$.

Theorem 3.1 (Existence and Characterization of an MLE in ${\mathcal{F}}_{SMU}\left(d\right)$)

1. A maximum likelihood estimator (MLE),${\widehat{f}}_n\equiv f_{{\widehat{G}}_n}\in {\mathcal{F}}_{SMU}\left(d\right)$ of $f_0\equiv f_{G_0}\in {\mathcal{F}}_{SMU}\left(d\right)$ almost surely exists, where ${\widehat{G}}_n\in {\mathcal{g}}_d$ is a purely-atomic probability measure, with at most n atoms, all of which are concentrated on A^#–the rectangular grid generated by the data X₁, … , X_n.

2. For almost all ω, the unique MLE,${\widehat{f}}_n\equiv f_{{\widehat{G}}_n}\in {\mathcal{F}}_{SMU}\left(d\right)$, is completely characterized by the following Fenchel conditions:

   $${\mathbb{P}}_n\left\{\frac{1_{[\mathit{X}\le \mathit{x}]}}{\widehat{f_n}\left(\mathit{X}\right)}\right\}\le \mid \mathit{x}\mid ;forall\mathit{x}\in {(0,\infty )}^d,$$ (3.1)

   $$and{\mathbb{P}}_n\left\{\frac{1_{[\mathit{X}\le \mathit{y}]}}{\widehat{f_n}\left(\mathit{X}\right)}\right\}=\mid \mathit{y}\mid ;ifandonlyif$$ (3.2)

   y ∈ (0, ∞)^d satisfies ${\widehat{G}}_n\left(\left\{\mathit{y}\right\}\right)>0$; or, equivalently,

   $${(-1)}^d\underset{\in \downarrow 0}{\lim }\left\{V_{\widehat{f_n}}[y,y+\in 1)\right\}>0.$$

Maximum likelihood estimation in mixture models has been studied in general by Lindsay [34], and this material is nicely summarized in [35, Chapter 5]. To prove the present theorem, we will therefore appeal to the results in [35, Chapter 5] and [47]. We begin with three lemmas.

Lemma 3.2

The support set $\mathcal{y}\equiv \mathrm{supp}\left({\widehat{G}}_n\right)$ of the mixing measure ${\widehat{G}}_n$ of any MLE ${\widehat{f}}_n$ is contained in the grid A^# ⊂ (0, ∞)^d generated by the observed data X₁, … , X_n; i.e., $\mathcal{y}\subset A^{\#}$.

Proof. First we show that $\mathcal{y}\subset (0,{\mathit{X}}^{\ast }]$ where X^* ≡ X₁ ∨ … ∨ X_n and the maximums are taken coordinatewise. If ${\widehat{f}}_n$ maximizes $L_n\left(f\right)=n{\mathbb{P}}_n\log f\left(x\right)$ over $f\in {\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$ and there is some y ∈ (0, ∞)^d \(0, X^*] with $y\in \mathcal{y}$, then${\widehat{f}}_n\left(y\right)>0$. Since ${\widehat{f}}_n$ is block decreasing, this implies that $0<{\int }_{(0,{\mathit{X}}^{\ast }]}{\widehat{f}}_n\left(\mathit{x}\right)d\mathit{x}\equiv \beta <1$. Then consider $\tilde{f}\left(\mathit{x}\right)\equiv ({\widehat{f}}_n\left(\mathit{x}\right)∕\beta )1_{(0,{\mathit{X}}^{\ast }]}\left(\mathit{x}\right)$ it is easily seen that $\tilde{f}\in {\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$ and has greater likelihood than${\widehat{f}}_n$, contradicting the assumption that ${\widehat{f}}_n$ maximizes the likelihood. Thus $\mathcal{y}\subset (0,{\mathit{X}}^{\ast }]$, and we may restrict attention to the class of estimators with support contained in (0, X^*], say ${\mathcal{K}}^{\ast }\left(d\right)$. Suppose that ${\widehat{f}}_n\in {\mathcal{K}}^{\ast }\left(d\right)$. Consider the mixing measure G̃_n defined by

$${\tilde{G}}_n\equiv \sum _{j:{\mathit{W}}_j\in A^{\#}}{\pi }_j{\delta }_{{\mathit{W}}_j}∕\sum _{j:{\mathit{W}}_j\in A^{\#}}{\pi }_j\equiv C\sum _{j:{\mathit{W}}_j\in A^{\#}}{\pi }_j{\delta }_{{\mathit{W}}_j}$$

where

$${\pi }_j\equiv {(-1)}^d{V}_{\widehat{f_n}}[{\mathit{W}}_j,{{\mathit{W}}_j}^{+})\cdot \mid {\mathit{W}}_j\mid ,\text{for}{\mathit{W}}_j\in A^{\#}$$

where ${\mathit{W}}_j^{+}\in A^{\#}$ defines the smallest rectangle above and right of W_j in the partition of [0, X^*] defined by the data. Then it is easy to see that

$$\tilde{f}\left(\mathit{x}\right)={\int }_{{(0,\infty )}^d}\frac{1}{\mid \mathit{u}\mid }{1}_{(0,\mathit{u}]}\left(\mathit{x}\right)d{\tilde{G}}_n\left(\mathit{u}\right)$$

satisfies

$$\begin{array}{cc}\hfill \tilde{f}\left({\mathit{W}}_j\right)& =C\sum _{k:{\mathit{W}}_k\ge {\mathit{W}}_j}\frac{{\pi }_j}{\mid {\mathit{W}}_j\mid }\hfill \\ \hfill & =C\sum _{k:{\mathit{W}}_k\ge {\mathit{W}}_j}{(-1)}^d{V}_{\widehat{f_n}}[{\mathit{W}}_j,{\mathit{W}}_k)\hfill \\ \hfill & =C{(-1)}^d{V}_{\widehat{f_n}}[{\mathit{W}}_j,2{\mathit{X}}^{\ast })=C\widehat{f_n}\left(X_j\right),\hfill \end{array}$$

and this implies that

$$\tilde{f}\left(\mathit{x}\right)=C\sum _{j:{\mathit{W}}_j\in A^{\#}}{1}_{({\mathit{W}}_j^{-},{\mathit{W}}_j]}\left(\mathit{x}\right)$$

where ${\mathit{W}}_i^{-}$ defines the smallest rectangle below and to the left of W_j in the partition of [0, X^*] defined by the data. If${\widehat{f}}_n\ne \tilde{f}$, then there exists $\mathit{y}\in ({\mathit{W}}_j^{-},W_j]$ for some W_j ∈ A^# such that ${\widehat{f}}_n\left(\mathit{y}\right)\ne \tilde{f}\left(\mathbf{y}\right)$, and then necessarily ${\widehat{f}}_n\left(\mathit{y}\right)>\tilde{f}\left(\mathit{y}\right)=\tilde{f}\left({\mathit{W}}_j\right)$.

This yields, since ${\tilde{f}}_n\in {\mathcal{K}}^{\ast }\left(d\right)$,

$$\begin{array}{cc}\hfill 1=& {\int }_{(0,{\mathit{X}}^{\ast })}\tilde{f}\left(\mathit{x}\right)d\mathit{x}=C\sum _{j:W_j\in A^{\#}}\left\{{\widehat{f}}_n\left({\mathit{W}}_j\right){\int }_{({\mathit{W}}_j^{-},W_j]}d\mathit{x}\right\}\hfill \\ \hfill <& C\sum _{j:{\mathit{W}}_j\in A^{\#}}{\widehat{f}}_n\left({\mathit{W}}_j\right){\int }_{({\mathit{W}}_j^{-},W_j]}{\widehat{f}}_n\left(\mathit{x}\right)d\mathit{x}=C{\int }_{(0,{\mathit{X}}^{\ast })}{\widehat{f}}_n\left(\mathit{x}\right)d\mathit{x}=C\hfill \end{array}$$

since $f\in {\mathcal{K}}^{\ast }\left(d\right)$. Thus f̃ has a greater log-likelihood than ${\widehat{f}}_n$, and it follows that $\mathrm{supp}\left({\widehat{G}}_n\right)\subset A^{\#}$.

Now we can prove uniqueness of the MLEs ${\widehat{f}}_n$ and ${\widehat{G}}_n$.

Lemma 3.3

There exists a set of points $\mathcal{y}=\{{\mathit{y}}_1,\dots ,{\mathit{y}}_m\}\subset {(0,\infty )}^d$ with m ≤ n such that a ${\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$ density ${\widehat{f}}_n$ with corresponding mixing measure ${\widehat{G}}_n$ is the MLE only if $\mathrm{supp}\left({\widehat{G}}_n\right)\subset \mathcal{y}$. Thus any MLE has the form

$${\widehat{f}}_n\left(\mathit{x}\right)=\sum _{j=1}^m{\pi }_j\frac{1}{\mid {\mathit{y}}_j\mid }1(0,{\mathit{y}}_j]\left(\mathit{x}\right)$$ (3.3)

where π_j ≥ 0, ${\sum }_{j=1}^m{\pi }_j=1$. Moreover, the vector ${\left({\widehat{f}}_n\left(X_i\right)\right)}_{i=1}^n$ is unique.

Proof. As in [34,35], define Γ (u) ∈ (0, ∞)ⁿ by

$$\Gamma \left(\mathit{u}\right)≔\left(\frac{1}{\mid \mathit{u}\mid }{1}_{(0,\mathit{u}]}\left({\mathit{X}}_1\right),\dots ,\frac{1}{\mid \mathit{u}\mid }{1}_{(0,\mathit{u})}\left({\mathit{X}}_n\right)\right),$$

and define the set Γ ≡ {Γ (u) | u ∈ (0, ∞)^d}. Then Γ is a closed and bounded, hence compact, subset of [0, ∞)ⁿ. Thus by Rockafellar [47, Theorem 17.2] $\overline{\mathrm{conv}\left(\Gamma \right)}=\mathrm{conv}\left(\overline{\Gamma }\right)=\mathrm{conv}\left(\Gamma \right)$ is also a compact subset of [0, ∞)ⁿ. Thus the continuous function ${\prod }_{i=1}^n{z}_i$ attains its supremum on conv(Γ ). Let $S={\mathrm{argmax}}_{z\in \mathrm{conv}\left(\Gamma \right)}{\sum }_{i=1}^n\log z_i$. Since the intersection of Γ and the interior (0, ∞)ⁿ of [0, ∞)ⁿ is not empty, we have S ∈ (0, ∞)ⁿ. Since ${\sum }_{i=1}^n\log z_i$ is strictly concave, S consists of a single point, $\widehat{f}={\left({\widehat{f}}_i\right)}_{i=1}^n>0$. Therefore for any $\mathrm{MLE}{\widehat{f}}_n$ it follows that the vector ${\left({\widehat{f}}_n\left(X_i\right)\right)}_{i=1}^n$ is unique. Note that the gradient of ${\sum }_{i=1}^n$ log z_i at f̂ is proportional to $1∕\widehat{f}\equiv {(1∕{\widehat{f}}_i)}_{i=1}^n$

Now dim(conv(Γ )) = n; if we consider the n points u_i = X_i, then the n vectors Γ (u_i) = (1_(0,Xi](X₁), … , 1_(0,Xi](X_n))/|X_i|, i = 1, … , n, are almost surely linearly independent. (In fact, the matrix M with rows |X_i|Γ (X_i), i = 1, … , n has det(M) = 1 a.s. if the X_i’s are i.i.d. with any density f .) By Rockafellar [47, Theorem 27.4] the vector $1∕\widehat{f}$ belongs to the normal cone of conv(Γ) at f̂ . Since 1/f̂ > 0 we have f̂ ∈ ∂(conv(Γ )) and the plane τ defined by ${\sum }_{i=1}^n{z}_i∕{\widehat{f}}_i=n$ is a support plane of conv(Γ ) at f̂ . Thus for ν_i = 1/(nf̂_i), i = 1, … , n, it follows that

$$q\left(\mathit{u}\right)\equiv \mid \mathit{u}\mid -\sum _{i=1}^n{v}_i{1}_{(0,\mathit{u}]}\left({\mathit{X}}_i\right)\ge 0$$

for all u ∈ [0, ∞)^d and q(u) = 0 if u = 0 or Γ (u) ∈ τ . We let $\mathcal{y}$ denote the set of vectors u such that Γ (u) ∈ τ ; i.e., $\Gamma \left(\mathcal{y}\right)=\tau \cap \Gamma$.

The intersection τ ∩ conv(Γ ) is an exposed face of conv(Γ ); see e.g. [47, p. 162]. By Rockafellar [47, Theorem 18.3], $\tau \cap \mathrm{conv}\left(\Gamma \right)=\mathrm{conv}\left(\Gamma \left(\mathcal{y}\right)\right)$, and by Theorem 18.1, $\mathrm{supp}\left({\widehat{G}}_n\right)\subset \mathcal{y}$. This implies that for any MLE${\widehat{f}}_n$, the support of the corresponding mixing measure ${\widehat{G}}_n$ is a subset of $\mathcal{y}$, and thus any MLE has form (3.3) with ${\mathit{y}}_j\in \mathcal{y}$ for j = 1, … ,m. To see that m ≤ n, note that ${\mathit{y}}_j\in \mathcal{y}\subset A^{\#}$ satisfy

$$\mid {\mathit{y}}_j\mid =\sum _{i=1}^n{v}_i{1}_{(0,{\mathit{y}}_j]}\left({\mathit{X}}_i\right)=\langle \mathit{v},\mid {\mathit{y}}_j\mid \Gamma \left({\mathit{y}}_j\right)\rangle ,j=1,\dots ,m.$$ (3.4)

Suppose that the vectors ${\left\{\mid {\mathit{y}}_j\mid \Gamma \left({\mathit{y}}_j\right)\right\}}_{j=1}^m$ are linearly dependent; i.e.,

$$\sum _{j=1}^m{b}_j\mid {\mathit{y}}_j\mid \Gamma \left({\mathit{y}}_j\right)=0$$

in ${\mathbb{R}}^n$ for some b_j, j = 1, … ,m. Since all the coordinates of the |y_j|Γ (y_j) vectors take values in {0, 1}, this system of equations is algebraically equivalent to the same system in which all the b_j’s take only integer values, i.e., $b_j\in \mathbb{Z}$ for j = 1, … ,m.

Then it follows on the one hand that

$$\begin{array}{cc}\hfill \sum _{j=1}^m{b}_j\langle \mathit{v},\mid {\mathit{y}}_j\mid \Gamma \left({\mathit{y}}_j\right)\rangle =& \sum _{j=1}^m{b}_j\sum _{i=1}^n{v}_i{1}_{(0,{\mathit{y}}_j]}\left({\mathit{X}}_i\right)\hfill \\ \hfill =& \langle \mathit{v},\sum _{j=1}^m{b}_j\mid {\mathit{y}}_j\mid \Gamma \left({\mathit{y}}_j\right)\rangle =\langle \mathit{v},0\rangle =0,\hfill \end{array}$$

and hence, by (3.4), ${\sum }_{j=1}^m{b}_j\mid {\mathit{y}}_j\mid =0$ , or, since y_j = W_ij ∈ A^# for some i_j,

$$\sum _{j=1}^m{b}_j\mid {\mathit{W}}_{i_j}\mid =0$$

with all $b_j\in \mathbb{Z}$. But this equation has at most countably many solutions {|W_ij |, j = 1, … ,m}, and hence occurs with $P_0^n$ probability 0. That is, for any fixed vector $\mathit{b}={\left(b_j\right)}_{j=1}^k$ with all $b_j\in \mathbb{Z}$, the function $f_{\mathit{b}}({\mathit{X}}_1,\dots ,{\mathit{X}}_n)={\sum }_{j=1}^k{b}_j\mid {\mathbf{W}}_{i_j}\mid$ has at most a finite number of zeros, so $P_0^n(f_b(X_1,\dots ,X_n)=0)=0$, and since $\mathbb{Z}$ is countable $P_0^n\left({\cup }_{\mathit{b}\in {\mathbb{Z}}^k}\{f_{\mathit{b}}({\mathit{X}}_1,\dots ,{\mathit{X}}_n)=0\}\right)=0$. Thus $P_0^n\left({{\cap }_{\mathit{b}\in \mathbb{Z}}}^k\{f_{\mathit{b}}({\mathit{X}}_1,\dots ,{\mathit{X}}_n)\ne 0\}\right)=1$. Hence it follows that the linear dependence condition only holds on an event with probability 0.

Thus the vectors |y_j|Γ (y_j), j = 1, … ,m are linearly independent almost surely $P_0^n$, and hence m ≤ n ($P_0^n$-almost surely).

Lemma 3.4

The discrete mixing measure ${\widehat{G}}_n$ which defines an MLE is $P_0^n$-almost surely unique.

Proof. Suppose that there exist two different MLE’s ${\widehat{f}}_n^1$and ${\widehat{f}}_n^2$. then

$${\widehat{f}}_n^l\left(x\right)=\sum _{j=1}^m{\pi }_j^l\frac{1}{\mid {\mathit{y}}_j\mid }{1}_{(0,{\mathit{y}}_j]}\left(\mathit{x}\right),l=1,2,$$

where ${\pi }_j^l\ge 0$ and ${\sum }_{j=1}^m{\pi }_j^l=1$ for l = 1, 2. Therefore

$${\delta }_n\left(\mathit{x}\right)\equiv {\widehat{f}}_n^l\left(\mathit{x}\right)-{\widehat{f}}_n^2\left(\mathit{x}\right)=\sum _{j=1}^m{r}_j\frac{1}{\mid {\mathit{y}}_j\mid }{1}_{(0,{\mathit{y}}_j]}\left(\mathit{x}\right)$$

where $r_j\equiv {\pi }_j^1-{\pi }_j^2$ has at least n zeros (since we know that

$${\left({\widehat{f}}_n^1\left({\mathit{X}}_i\right)\right)}_{i=1}^n={\left({\widehat{f}}_n^2\left({\mathit{X}}_i\right)\right)}_{i=1}^n={\left({\widehat{f}}_n\left({\mathit{X}}_i\right)\right)}_{i=1}^n$$

is unique). So, uniqueness holds if the vectors

$${\left(1_{(0,{\mathit{y}}_j]}\left({\mathit{X}}_{\mathit{i}}\right)\right)}_{i=1}^n\in {\{0,1\}}^n,\text{for}j=1,\dots ,m\le n$$

are (almost surely) linearly independent. But this follows from the proof of Lemma 3.3.

Theorem 3.1 does not assert that the MLE is always unique. An MLE is $P_0^n$ almost surely unique, but we now present an example in which there exist an infinite number of MLE’s.

Example 3.1 (A MLE in ${\mathcal{F}}_{\mathit{SMU}}$ is Not Always Unique)

To be able to graphically illustrate the set Γ , in the proof of Theorem 3.1, we need to restrict consideration to n = 2 and in order that we be able to graphically illustrate the MLE(s) we need to restrict consideration to d = 2. Suppose that X₁ = (1, 3) and X₂ = (3, 2) are the observation points. The set

$$\Gamma \equiv \left\{\phantom{\mid }\frac{1}{u_1{u}_2}(1_{(0,\mathit{u}]}\left({\mathit{X}}_1\right),1_{(0,\mathit{u}]}\left({\mathit{X}}_2\right))\mid \mathit{u}=(u_1,u_2)\in {(0,\infty )}^2\right\}$$

and its convex hull, Conv(Γ ), are illustrated in Fig. 1.

Fig. 1.

The sets Γ and Conv(Γ ) based on two observations: X₁ = (1, 3) and X₂ = (3, 2).

Using [35, Theorem 22, p. 118], it follows that any MLE,${\widehat{f}}_2$, will have a unique value for $\widehat{\mathit{f}}\equiv ({\widehat{f}}_2\left({\mathit{X}}_1\right),{\widehat{f}}_2\left({\mathit{X}}_2\right))$ that is given by $\widehat{\mathit{f}}=({\tilde{w}}_1^{-1},{\tilde{w}}_2^{-1})$ where w̃ = (w̃₁,w̃₂) maximizes the function (w₁,w₂) ↦ log(w₁w₂) on the set

$$\left\{\phantom{\mid }(w_1,w_2)\in {(0,\infty )}^2\mid \frac{w_1}{3}\le 2\text{and}\frac{w_2}{6}\le 2\right\}.$$

It is immediate that w̃ = (6, 12) from which we conclude that f̃ = (1/6, 1/12) has exactly two representations as convex combinations in terms of pairs of the points {A₁, A₂, A₃} (see Fig. 1(a) again):

$$\left(\frac{1}{6},\frac{1}{12}\right)=\frac{1}{2}\left(0,\frac{1}{6}\right)+\frac{1}{2}\left(\frac{1}{3},0\right),\text{and}\left(\frac{1}{6},\frac{1}{12}\right)=\frac{1}{4}\left(\frac{1}{3},0\right)+\frac{3}{4}\left(\frac{1}{9},\frac{1}{9}\right).$$

These two convex combinations yield two different maximum likelihood estimators, as shown in Fig. 2(a) and (b).

Fig. 2.

Two maximum likelihood estimators in ${\mathcal{F}}_{\mathrm{SMU}}\left(2\right)$, supported on the grid generated by the data: X₁ = (1, 3) and X₂ = (3, 2). The two figures show the contour/level plots of the respective maximum likelihood densities.

It should be noted, however, that infinitely many maximum likelihood estimators exist in this case since each convex combination of these two MLEs is again an MLE, by virtue of linearity of f_G (recall (2.1)) as a function of the mixing distribution, G.

3.2. Strong pointwise consistency of the MLE

Let X₁, X₂, … , X_n, … be the coordinate random elements on the (completed) infinite product space $({\Omega }^{\infty },{\mathcal{A}}^{\infty },P^{\infty })$ such that these coordinates are i.i.d. according to f₀ ≡ f_G0 on (0, ∞)^d. Let $A\in {\mathcal{A}}^{\infty }$ be the event (with P^∞-probability one) that for each $n\in \mathbb{Z}$ there exists a unique SMU density, ${\widehat{f}}_n\equiv f_{{\widehat{G}}_n}$, maximizing the log-likelihood.

From Theorem 2.3 we have that for each $n\in \mathbb{N}$ and a fixed ω ∈ A, there exists a unique Borel probability measure,${\widehat{G}}_n$ on ((0, ∞)^d, || · ||₂), such that

$${\widehat{f}}_n\left(\mathit{x}\right)={\int }_{{(0,\infty )}^d}\frac{1}{\mid \mathit{u}\mid }{1}_{(0,\mathit{u})}\left(\mathit{x}\right)d{\widehat{G}}_n\left(\mathit{u}\right)={\int }_{\mathit{u}\ge \mathit{x}}\frac{1}{\mid \mathit{u}\mid }d{\widehat{G}}_n\left(\mathit{u}\right)$$ (3.5)

holds true for all x ∈ (0, ∞)^d. We are ready to formulate and prove the following proposition.

Proposition 3.5 (Strong Consistency of the MLE in ${\mathcal{F}}_{\mathit{SMU}}$)

1. 1. The sequence of maximum likelihood mixing distributions ${\left\{{\widehat{G}}_n\right\}}_{n=1}^{\infty }$ converges weakly to G₀ as n → ∞, P^∞-almost surely.
   2. In addition, for Lebesgue almost all x ∈ (0, ∞)^d,${\widehat{f}}_n\left(\mathit{x}\right)\to {}_{a.s.}f_0\left(\mathit{x}\right)$ as n → ∞. In particular, if f₀ is continuous at x ∈ (0, ∞)^d, then

      $$\mid {\widehat{f}}_n\left(\mathit{x}\right)-f_0\left(\mathit{x}\right)\mid {\to }_{a.s.}0asn\to \infty .$$
2. The sequence of maximum likelihood estimators, ${\left\{{\widehat{f}}_n\right\}}_{n=1}^{\infty }$, is strongly consistent in the total variation (or L₁) and in the Hellinger metrics. That is,

   $${\int }_{{(0,\infty )}^d}\mid {\widehat{f}}_n\left(\mathit{x}\right)-f_0\left(\mathit{x}\right)\mid d\mathit{x}{\to }_{a.s.}0asn\to \infty ,$$

   and, with $h^2(p,q)=\int {\{\sqrt{p\left(\mathit{x}\right)}-\sqrt{q\left(\mathit{x}\right)}\}}^{2}d\mathit{x}$,

   $$h({\widehat{f}}_n,f_0){\to }_{a.s.}0asn\to \infty .$$

Proof. (a) (i) To be able to apply Theorems 3.4, 3.5 and 3.7 of [39], with the refinement on page 143 of the same article, we need to provide the relevant setup as well as establish the assumptions of Pfanzagl’s theorems. We do this below.

Let ${\mathcal{C}}_0({(0,\infty )}^d,{\parallel \cdot \parallel }_2)$ denote the set of all real-valued, continuous functions on (0, ∞)^d that vanish at ∞. Let Θ_* denote the set of all Borel sub-probability measures on (0, ∞)^d, equipped with the vague topology, τ , which makes the space a compact, metrizable, topological space, and thus with a countable base. It is also a convex subset of the linear space of all finite, signed, Borel measures on ((0, ∞)^d, || · ||₂). For clarity, the vague topology is the smallest topology that makes the functions

$$\mu \mapsto {\int }_{{(0,\infty )}^d}g\left(\mathit{x}\right)d\mu \left(\mathit{x}\right)$$

continuous, for each $g\in {\mathcal{C}}_0({(0,\infty )}^d,{\parallel \cdot \parallel }_2)$. By metrizability, the topology τ is completely characterized by convergent sequences, ${\theta }_n\stackrel{v}{\Rightarrow }\theta$ as n → ∞, on (Θ_*, τ ).

Let also Θ ⊆ Θ_* be the set of all Borel probability measures on (0, ∞)^d, and notice that μ ∈ Θ. Also, for each θ_* ∈ Θ_* there exists a unique c ∈ [0, 1] and a unique θ ∈ Θ, such that θ_* = cθ. Further, notice that letting m(ν, ·) ≡ fν (·), for each ν ∈ Θ_*, and $M_n(\cdot )\equiv {\mathbb{P}}_n\log \left\{m(\cdot ,\mathit{X})\right\}$, we have

$$M_n\left({\theta }_{\ast }\right)=\log \left\{c\right\}+M_n\left(\theta \right)\le M_n\left(\theta \right),\text{since}c\in [0,1],$$

whence, sup_θ∈Θ* (M_n(θ)) = sup_θ∈Θ (M_n(θ)).

With reference measure the Lebesgue measure λ ≡ Q and for each ν ∈ Θ_*, let P_ν ∈ Θ_* be the sub-probability, Borel measure on ((0, ∞)^d, || · ||₂) with Radon–Nikodym derivative with respect to λ being f_ν , Lebesgue almost surely. Then by virtue of Fubini–Tonelli, P_ν ∈ Θ when and only when ν ∈ Θ. Also, notice that for each fixed x ∈ (0, ∞)^d, the functional ν ↦ fν (x) is not vaguely continuous at any ν ∈ Θ_* with a discontinuity point on the boundary of [x,∞). However, since for a fixed x ∈ (0, ∞)^d, the function $\mathit{y}\mapsto 1_{[\mathit{x},\infty )}\left(\mathit{y}\right)∕\mid \mathit{y}\mid$ is easily seen to be an upper semi-continuous function on (0, ∞)^d–vanishing at ∞, Doob [15, Theorem 10, p. 138], applies and asserts that the function ν ↦ f_ν (x) on (Θ_*, τ ) is itself (vaguely) upper semi-continuous. Since this holds for all x ∈ (0, ∞)^d, it holds almost-surely. Also, the mapping ν ↦ f_ν (x) is affine on Θ_* (and hence concave also).

It remains to establish that for each fixed τ -open subset U of Θ_*, the real-valued function T_U (·) on (0, ∞)^d defined by

$$T_U\left(\mathit{x}\right)=\underset{\nu \in U}{\sup }\left\{{\int }_{{(0,\infty )}^d}\frac{1}{\mid \mathit{u}\mid }{1}_{(0,\mathit{u}]}\left(\mathit{x}\right)d\nu \left(\mathit{u}\right)\right\}$$

is a $\mathcal{A}$-measurable function. We can choose to take $\mathcal{A}$ to be the Lebesgue σ-field, in which case measurability follows by observing that T_U (·) is a block-decreasing function and appeal to Proposition 2.4.

We now apply Theorem 3.4 of [39] to our setting and further appeal to the fact that a vaguely convergent sequence of probability measures with limit a probability measure, is, in fact, weakly convergent. This gives the desired conclusion: the random sequence of maximum likelihood mixing probability measures ${\left\{{\widehat{G}}_n\right\}}_{n=1}^{\infty }$ converges weakly to G₀ as n → ∞, P^∞-almost surely.

(ii) Combining the fact that, for each fixed x ∈ (0, ∞)^d, ν ↦ f_ν(x) is vaguely upper semi-continuous on Θ_* with the conclusion of part (a)(i), we get

$$\underset{n\to \infty }{\overline{\lim }}\left\{f_{{\widehat{G}}_n}\left(\mathit{x}\right)\right\}\le f_0\left(\mathit{x}\right);P^{\infty }-a.s.\text{for}\text{all}\mathit{x}\in {(0,\infty )}^d.$$ (3.6)

Let

$$F_{G_0}(\cdot )={\int }_{{(0,\infty )}^d}\frac{\mid \cdot \wedge \mathit{u}\mid }{\mid \mathit{u}\mid }d{G}_0\left(\mathit{u}\right)$$

and

$$F_{{\widehat{G}}_n}(\cdot )={\int }_{{(0,\infty )}^d}\frac{\mid \cdot \wedge \mathit{u}\mid }{\mid \mathit{u}\mid }d{\widehat{G}}_n\left(\mathit{u}\right)$$

be the distribution functions corresponding to the densities f₀(·) and ${\widehat{f}}_n(\cdot )$, respectively, $n\in \mathbb{N}$. These distribution functions are everywhere continuous on the Euclidean set (0, ∞)^d. In fact, since for each fixed x ∈ (0,∞)^d, the function u ↦ |x ∧ u| / |u| is bounded (by 1) and continuous on (0, ∞)^d, we then have that

$$F_{{\widehat{G}}_n}\left(\mathit{x}\right){\to }_{a.s.}{F}_{G_0}\left(\mathit{x}\right)\text{for}\text{all}\mathit{x}\in {(0,\infty )}^d$$ (3.7)

follows directly by the definition of almost sure weak convergence of the mixing random measures ${\left\{{\widehat{G}}_n\right\}}_{n=1}^{\infty }$ to G₀, established in part (a)(i).

Let B be the set of points on (0, ∞)^d at which f₀ is continuous. Then B^c has Lebesgue measure zero, λ(B_c ) = 0, exactly because f₀ is discontinuous on the boundary ∂[x₀, ∞) for a (possibly non-existent) x₀ ∈ (0, ∞)^d where P₀ is discontinuous (i.e., such that P₀({x₀}) > 0). Since P₀ can have at most countably many discontinuity points x₀ ∈ (0, ∞)^d and since λ(∂[x₀, ∞)) = 0, we get by countable subadditivity of λ that indeed λ(B_c ) = 0.

Fix arbitrary x ∈ B and ε > 0. Then, since f₀ is lower semi-continuous at x, there exists an open neighborhood U_x,ε of x such that for every y ∈ U_x,ε we have that f₀(y) > f₀(x) – ε. In particular, there exists an U_x,ε ∍ x_ε > x satisfying f₀(xε) > f₀(x) – ε. Since f₀ is block-decreasing, we have:

$$\frac{V_{F_{G_0}}(\mathit{x},{\mathit{x}}_{\in })}{\lambda \left((\mathit{x},{\mathit{x}}_{\in }]\right)}=\frac{{\int }_{(\mathit{x},{\mathit{x}}_{\in }]}\left\{f_0\left(\mathit{y}\right)\right\}d\mathit{y}}{\lambda \left((\mathit{x},{\mathit{x}}_{\in }]\right)}\ge f_0\left({\mathit{x}}_{\in }\right)>f_0\left(\mathit{x}\right)-∊.$$ (3.8)

Further, for each fixed $n\in \mathbb{N}$, since ${\widehat{f}}_n(\cdot )$ is block-decreasing (as a SMU density), we have

$$f_{{\widehat{G}}_n}\left(\mathit{x}\right)\ge \frac{{\int }_{(\mathit{x},{\mathit{x}}_{\in }]}\left\{f_{{\widehat{G}}_n}\left(\mathit{y}\right)\right\}d\mathit{y}}{\lambda \left((\mathit{x},{\mathit{x}}_{\in }]\right)}$$ (3.9)

$$=\frac{V_{F_{{\widehat{G}}_n}}(\mathit{x},{\mathit{x}}_{\in }]}{\lambda \left((\mathit{x},{\mathit{x}}_{\in }]\right)}.$$ (3.10)

Eq. (3.7) further implies that

$$V_{F_{{\widehat{G}}_n}}(\mathit{x},{\mathit{x}}_{\in }]\to V_{F_{G_0}}(\mathit{x},{\mathit{x}}_{\in }],\mathrm{as}n\to \infty .$$ (3.11)

Combining Eqs. (3.8)–(3.11) and the fact that ε > 0 was arbitrary, we get

$$\underset{n\to \infty }{\underset{¯}{\lim }}\left\{f_{{\widehat{G}}_n}\left(\mathit{x}\right)\right\}\ge f_0\left(\mathit{x}\right);P^{\infty }-a.s.\text{for}\mathit{x}\in B.$$ (3.12)

Eqs. (3.6) and (3.12) yield the assertion: for Lebesgue almost all x ∈ (0, ∞)^d (and, in particular, at the points of continuity of f ), $f_{{\widehat{G}}_n}\left(\mathit{x}\right)\to {}_{a.s.}f_0\left(\mathit{x}\right)$ as n → ∞holds.

(b) Showing consistency in the L₁ (total-variation) norm is a direct consequence of part (a) (ii) and Glick’s Theorem, [17]; see also [14, p. 25].

Convergence in the Hellinger metric follows from the following well-known inequalities of [33, p.46]:

$$h^2(P,Q)\le \frac{1}{2}{\parallel P-Q\parallel }_{L_1}\le h(P,Q){\{2-h^2(P,Q)\}}^{\frac{1}{2}},$$

where $h^2(P,Q)=2^{-1}\int {(\sqrt{dP}-\sqrt{dQ})}^2$ is the squared Hellinger metric and || · ||_L1 is the L₁-norm.

4. A local asymptotic minimax lower bound

Let X_i := (X_i,1, … , X_i,d)’ for i = 1, 2, … , n be i.i.d. random vectors from density $f\in {\mathcal{F}}_{\mathit{SMU}}\left(d\right)$. For a fixed x₀ ≡ (x_0,1, … , x_0,d)’ ∈ (0, ∞)^d, we want to estimate the functional T(f) := f(x₀) on the basis of X₁, … , X_n. We shall make the following assumption:

Assumption 4.1

Suppose that $f\in {\mathcal{F}}_{\mathit{SMU}}$ is continuously differentiable at x₀, f (x₀) > 0, and, in particular, there exists an open ball A(x₀) around x₀ such that f is everywhere strictly positive on A(x₀) and where (∂/∂x_j)f(x₀) < 0 exist for all j ∈ {1, 2, … , d} and are continuous on A(x₀) ⊆ (0, ∞)^d. Further, we assume that the full mixed derivative of f exists, is continuous on A(x₀), and satisfies

$${\phantom{\mid }{(-1)}^d\frac{{\partial }^{d}f}{\partial x_1\cdots \partial x_d}\left(\mathit{x}\right)\mid }_{\mathit{x}=\mathit{y}}>0\text{for}\text{all}\mathit{y}\in A\left({\mathit{x}}_0\right).$$

Proposition 4.1

Suppose that $f\in {\mathcal{F}}_{\mathit{SMU}}$ satisfies Assumption 4.1 at the fixed point x0 ∈ (0, ∞)^d. Then there is a sequence $\left\{f_n\right\}\subset {\mathcal{F}}_{\mathit{SMU}}$ such that any estimator sequence {T_n} of f (x₀) satisfies

$$\begin{array}{cc}\hfill & \underset{n\to \infty }{\underset{¯}{\lim }}\max \left\{{\mathit{E}}_{f_n}\left\{n^{\frac{1}{3}}\mid T_n-f_n\left({\mathit{x}}_0\right)\mid \right\},{\mathit{E}}_f\left\{n^{\frac{1}{3}}\mid T_n-f\left({\mathit{x}}_0\right)\mid \right\}\right\}\hfill \\ \hfill & \ge \frac{e^{-\frac{1}{3}}}{2^d}{\left\{3^{d-1}\right\}}^{\frac{1}{3}}{\left\{{\phantom{\mid }{(-1)}^d\frac{{\partial }^{d}f\left(\mathit{x}\right)}{\partial x_1\cdots \partial x_d}\mid }_{\mathit{x}={\mathit{x}}_0}\cdot f\left({\mathit{x}}_0\right)\right\}}^{\frac{1}{3}}.\hfill \end{array}$$ (4.1)

Remark

The lower bound in Proposition 4.1 should be contrasted to a similar lower bound for estimation of f(x₀) for $f\in {\mathcal{F}}_{\mathrm{BDD}}$ which is derived by Pavlides [38]. In that case the natural hypothesis is ∂f (x₀)/∂x_i< 0 for i = 1, … , d, and the resulting rate of convergence is n^1/(d+2).

To prove Proposition 4.1 we will make use of the following lemma. It was established in the form presented here by [23]; see also Groeneboom and Jongbloed [22,29].

Lemma 4.2.

Let $\mathcal{F}$ be a class of densities on a measurable space $(\mathcal{X},\mathcal{F})$ and f a fixed element of $\mathcal{F}$. Let ${\mathcal{F}}_f$ denote any open Hellinger ball with center $f\in \mathcal{F}$. Assume that there exists a sequence ${\left\{f_n\right\}}_{n=1}^{\infty }\subseteq \mathcal{F}$ such that and

$$\underset{n\to \infty }{\lim }\left\{\sqrt{n}h(f_n,f)\right\}=\alpha$$ (4.2)

and

$$\underset{n\to \infty }{\lim }\mid T\left(f_n\right)-T\left(f\right)\mid =\beta$$ (4.3)

both hold for some constants 0 < α, β < ∞, and where T is a functional on $\mathcal{F}$. Here, $h^2(f_n,f)\equiv 2^{-1}\int {\{\sqrt{f_n\left(x\right)}-\sqrt{f\left(x\right)}\}}^{2}d\mu \left(x\right)$, is the Hellinger distance between the μ-densities f_n and f · Let l(·) be a convex function, symmetric about zero, which is non-decreasing on [0, ∞).

Then, it holds that

$$\underset{n\to \infty }{\underset{¯}{\lim }}\left\{R_{n,1}\left({\mathcal{F}}_f\right)\right\}\ge l\left(\frac{1}{4}\beta e^{-2{\alpha }^2}\right)$$ (4.4)

where $R_{n,1}\left(\mathcal{F}\right)\equiv {\inf }_{T_n}{\sup }_{g\in \mathcal{F}}{\mathit{E}}_{g^{\otimes n}}\left\{l(T_n-T\left(g\right))\right\}$ is the minimax risk for estimating the functional T(f) based on n i.i.d observations from $\mathcal{F}$.

In particular, for the loss l(x) = |x| on we have

$$\underset{n\to \infty }{\underset{¯}{\lim }}\left\{R_{n,\mid \cdot \mid }\left({\mathcal{F}}_f\right)\right\}\ge l\left(\frac{1}{4}\beta e^{-2{\alpha }^2}\right)$$ (4.5)

Hereafter, fix an otherwise arbitrary vector h := (h₁, …, h_d) ∈ (0, ∞)^d, and define H := diag(h) ∈ M_d×d ((0, ∞)) . For each $k\in \mathbb{N}$, consider the perturbation rectangle

$$I_n\left(k\right)≔\underset{i=1}{\overset{d}{\otimes }}\left[x_{0,i}-n^{-\frac{1}{k}}{h}_i,x_{0,i}+n^{-\frac{1}{k}}{h}_i\right],$$

only for those positive integers n ≥ n₀(k, x₀, h) for which I_n(k) ⊆ A(x₀) for all n ≥ n₀. The two-dimensional case, d = 2, is illustrated in Fig. 3.

Fig. 3.

Perturbation rectangle I_n(k), for the case d = 2, with center x₀ = (x₀₁, x₀₂) and h = (h₁, h₂).

Recall Assumption 4.1. Let b := (∂^d/∂x₁ … ∂x_d)f(x)|_x=x0 and observe that (−1)^db > 0. Finally, define the functions h_n on I_n(3d) as follows:

$$h_n(y_1,\dots ,y_d)≔{(-1)}^d\prod _{i=1}^d\left\{1_{\left(x_{0,i},x_{0,i}+n^{-\frac{1}{3d}}{h}_i\right]}\left(y_i\right)-1_{\left[{x_{0,i-n}}^{-\frac{1}{3d}}{h}_i,x_{0,i}\right]}\left(y_i\right)\right\},$$

and

$$g_n\left(\mathit{y}\right)≔b{\int }_{\mathit{u}\underset{¯}{\succ }\mathit{y}}\{1_{I_n\left(3d\right)}\left(\mathit{u}\right)\cdot h_n\left(\mathit{u}\right)\}d\mathit{u},$$

where we observe that g_n(y) ≥ 0 for all y ∈ I_n(3d), since x₀ is the center of the rectangle I_n(3d). In fact, consideration of the geometry of the definition of g_n(·) reveals that, for y ∈ I_n, g_n(y) is equal to (−1)^db > 0 times the volume of the rectangle [ν_n(y) ∧ y, ν_n(y) ∨ y], where ν_n(y) is defined as that vertex of I_n that is closest in L₂-distance from y ∈ I_n. Since I_n is a decreasing sequence of compact sets, it is then immediately clear that g_n(y) is (pointwise) non-increasing in $n\in \mathbb{N}$, for each fixed y ∈ (0, ∞)^d.

Assume that $f\in {\mathcal{F}}_{\mathrm{SMU}}$, and for fixed vectors x₀, h ∈ (0, ∞)^d we further assume that f satisfies Assumption 4.1. For n ≥ n₀(3d, x₀, h), define the perturbed density, f_n of f at x₀, by

$$f_n\left(\mathit{x}\right)=\{\begin{array}{cc}\frac{f\left(\mathit{x}\right)+\theta g_n\left(\mathit{x}\right)}{d_n}:\hfill & \mathrm{if}\mathit{x}\in I_n\left(3d\right)\hfill \\ \frac{f\left(\mathit{x}\right)}{d_n}:\hfill & \mathrm{if}\mathit{x}\in I_n^c\left(3d\right)\hfill \end{array}\phantom{\}}$$ (4.6)

for some arbitrary but fixed θ ∈ (0, 1) and where d_n is the normalizing constant for f_n, uniquely determined by ∫_{(0, ∞)^d} f_n(x) dx = 1. We will see the importance of the value of b and the fact that 0 < θ < 1 in the following proposition that establishes that ${\left\{f_n\right\}}_{n\ge n_1}\subseteq {\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$ for a sufficiently large $n_1\in \mathbb{N}$.

Proposition 4.3

There exists a positive integer n₁ := n₁(d, x₀, h) ≥ n₀(3d, x₀, h) such that $f_n\in {\mathcal{F}}_{\mathrm{SMU}}$ for all n ≥ n₁.

Proof. Since $f\in {\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$, we get from Theorem 2.3 that

$$V_f[\mathit{x},\mathit{y}]\ge 0,\text{for}\text{all}d-\text{boxes}[\mathit{x},\mathit{y}].$$ (4.7)

From the definition of g_n(·), we see that its full, mixed partial derivative exists in a neighborhood of x₀. Hence, by definition and the fact that (−1)^db > 0 and θ ∈ (0, 1), we have that

$$\begin{array}{cc}\hfill {\phantom{\mid }{(-1)}^d\frac{{\partial }^d{f}_n}{\partial x_1\cdots \partial x_d}\left(\mathit{x}\right)\mid }_{\mathit{x}=\mathit{y}}\ge & {\phantom{\mid }{(-1)}^d\frac{{\partial }^{d}f}{\partial x_1\cdots \partial x_d}\left(\mathit{x}\right)\mid }_{\mathit{x}=\mathit{y}}-{(-1)}^{d}b\theta \hfill \\ \hfill & =\left[{\phantom{\mid }{(-1)}^d\frac{{\partial }^{d}f}{\partial x_1\cdots \partial x_d}\left(\mathit{x}\right)\mid }_{\mathit{x}=\mathit{y}}-{(-1)}^{d}b\right]+(1-\theta ){(-1)}^{d}b\hfill \\ \hfill & \ge 2^{-1}(1-\theta ){(-1)}^{d}b>0,\hfill \end{array}$$ (4.8)

where the second to last inequality follows from Assumption 4.1 that the full mixed partial derivative of f exists and is continuous at x₀ from which we get, by definition of continuity, that there exists a large enough positive integer n₁ := n₁(d, x₀, h) ≥ n₀(3d, x₀, h) such that

$${\phantom{\mid }{(-1)}^d\frac{{\partial }^{d}f}{\partial x_1\cdots \partial x_d}\left(\mathit{x}\right)\mid }_{\mathit{x}=\mathit{y}}-{(-1)}^{d}b\ge -2^{-1}(1-\theta ){(-1)}^{d}b$$

holds true for all y ∈ I_n(3d) and n ≥ n₁. The result in (4.8) suggests that

$${(-1)}^d{V}_{fn}[\mathit{x},\mathit{y}]\equiv {(-1)}^d{\int }_{(\mathit{x},\mathit{y}]}\left\{{\phantom{\mid }\frac{{\partial }^d{f}_n}{\partial w_1\cdots \partial w_n}\left(\mathbf{w}\right)\mid }_{\mathit{w}=\mathit{u}}\right\}d\mathit{u}\ge 0$$

holds true for all d-boxes (x, y] with x, y ∈ I_n(3d) and n ≥ n₁.

The last case not considered is the one that is exactly one between x and y, in the d-box [x, y], is an element of I_n(3d). See also Fig. 4. For this case, we can appeal to Lemma 2.2 by setting [x₀, y₀] := [x, y] ∩ I_n(3d)–the latter being well-defined as the intersection of two rectangles is itself an rectangle. Then, from Lemma 2.2 and (4.7), we have,

$${(-1)}^d{V}_{fn}[\mathit{x},\mathit{y}]={(-1)}^d{V}_{fn}[{\mathit{x}}_0,{\mathit{y}}_0]+{(-1)}^d\sum _{i=1}^m\left\{V_{fn}[{\mathit{x}}_i,{\mathit{y}}_i]\right\}\ge 0+0=0,$$

exactly since $[x_i,\mathit{y}]\subseteq I_n^c\left(3d\right)$ for all i ∈ {1, 2, … ,m} (where m is as defined in Lemma 2.2). For completeness, notice that we were not concerned above with end-point discontinuities of f (or f_n) on the entailed rectangle, subsets of I_n(3d), as, in fact, f (and f_n) is (are) continuous there for n ≥ n₁, by Assumption 4.1.

Fig. 4.

Perturbation rectangle I_n(k), for the case d = 2, with two rectangles intersecting I_n(k) but otherwise not subsets of it.

All these observations finally yield that (−1)^dV_fn [x, y] ≥ 0 holds true for all d-boxes [x, y] and thus Theorem 2.3 asserts that $f_n\in {\mathcal{F}}_{\mathrm{SMU}}$ for all n ≥ n₁.

We are ready to prove the main proposition of this section.

Proof. Recall Proposition 4.3. First, we establish that

$${\int }_{l_n}{g}_n\left(\mathit{x}\right)d\mathit{x}={(-1)}^{d}b\prod _{i=1}^d\left\{h_i^2\right\}\cdot n^{-\frac{2}{3}},$$ (4.9)

where, hereafter, I_n will be the short-hand form for I_n(3d). By definition, notice that,

$$\begin{array}{cc}\hfill \frac{1}{b}{\int }_{I_n}{g}_n\left(\mathit{x}\right)d\mathit{x}=& {\int }_{I_n}{\int }_{I_n}\prod _{i=1}^d\left\{1_{[x_i\le u_i]}\right\}{h}_n\left(\mathit{u}\right)d\mathit{u}d\mathit{x}\hfill \\ \hfill =& {\int }_{I_n}{h}_n\left(\mathit{u}\right)\left\{{\int }_{I_n}{1}_{(0,\mathit{u}]}\left(\mathit{x}\right)d\mathit{x}\right\}d\mathit{u}\hfill \\ \hfill =& {\int }_{I_n}\prod _{i=1}^d\left\{u_i-\left(x_{0i}-h_i{n}^{-\frac{1}{3d}}\right)\right\}{h}_n\left(\mathit{u}\right)d\mathit{u}\hfill \\ \hfill =& \prod _{i=i}^d\{{\int }_{x_{0i}-h_i{n}^{-\frac{1}{3d}}}^{x_{0i}+h_i{n}^{-\frac{1}{3d}}}([u_i-(x_{0i}-h_i{n}^{-\frac{1}{3d}})]\phantom{)}\phantom{\}}\hfill \\ \hfill & \times \phantom{\{}\phantom{(}\left[1_{[x_{0i}-h_i{n}^{-\frac{1}{3d}},x_{0i}]}\left(u_i\right)-1_{\left(x_{0i},x_{0i}+h_i{n}^{-\frac{1}{3d}}\right]}\left(u_i\right)\right])d{u}_i\}\hfill \\ \hfill & =\prod _{i=1}^d\{{\int }_{x_{0i}-h_i{n}^{-\frac{1}{3d}}}^{x_{0i}}[u_i-(x_{0i}-h_i{n}^{-\frac{1}{3d}})]d{u}_i\phantom{\}}\hfill \\ \hfill & -\phantom{\{}{\int }_{x_{0i}}^{x_{0i}+h_i{n}^{-\frac{1}{3d}}}[u_i-(x_{0i}-h_i{n}^{-\frac{1}{3d}})]d{u}_i\}\hfill \\ \hfill & =\prod _{i=1}^d\left\{{\int }_0^{h_i{n}^{-\frac{1}{3d}}}[-y+h_i{n}^{-\frac{1}{3d}}]dy-{\int }_0^{h_i{n}^{-\frac{1}{3d}}}[w+h_i{n}^{-\frac{1}{3d}}]dw\right\}\hfill \\ \hfill =& \prod _{i=1}^d\left\{{\int }_0^{h_i{n}^{-\frac{1}{3d}}}(-2y)dy\right\}={(-1)}^d\prod _{i=1}^d\left\{h_i^2{n}^{-\frac{2}{3d}}\right\}={(-1)}^d\prod _{i=1}^d\left\{h_i^2\right\}\cdot n^{-\frac{2}{3}},\hfill \end{array}$$

thus yielding (4.9).

We next derive another equality, the most important fact about it being the factor n⁻¹ on the right hand side:

$${\int }_{l_n}{g}_n^2\left(\mathit{x}\right)d\mathit{x}={\left(\frac{8}{3}\right)}^d{b}^2\prod _{i=1}^d\left\{h_i^3\right\}\cdot n^{-1}.$$ (4.10)

Before we start deriving (4.10), let us first define four rectangles $R_j^i$ with j = 1, 2, 3, 4 for each i ∈ {1, 2, …, d}:

1. $$R_1^i=\left[x_{0i}-h_i{n}^{-\frac{1}{3d}},x_{0i}\right]\times \left[x_{0i}-h_i{n}^{-\frac{1}{3d}},x_{0i}\right],$$
2. $$R_2^i=\left[x_{0i}-h_i{n}^{-\frac{1}{3d}},x_{0i}\right]\times \left(x_{0i},x_{0i}+h_i{n}^{-\frac{1}{3d}}\right],$$
3. $$R_3^i=\left(x_{0i},x_{0i}+h_i{n}^{-\frac{1}{3d}}\right]\times \left[x_{0i}-h_i{n}^{-\frac{1}{3d}},x_{0i}\right],$$
4. $$R_4^i=\left(x_{0i},x_{0i}+h_i{n}^{-\frac{1}{3d}}\right]\times \left(x_{0i},x_{0i}+h_i{n}^{-\frac{1}{3d}}\right].$$

Then, by definition:

$$\begin{array}{cc}\hfill \frac{1}{b^2}{\int }_{l_n}{g}_n^2\left(\mathit{x}\right)d\mathit{x}=& {\int }_{l_n}{\left\{{\int }_{l_n}{h}_n\left(\mathit{u}\right)1_{[\mathit{x}\le \mathit{u}]}d\mathit{u}\right\}}^{2}d\mathit{x}\hfill \\ \hfill =& {\int }_{l_n}{\int }_{l_n}{\int }_{l_n}{h}_n\left(\mathit{u}\right)h_n\left(\mathit{v}\right)1_{[\mathit{x}\le \mathit{u}\wedge \mathit{v}]}d\mathit{v}d\mathit{u}d\mathit{x}\hfill \\ \hfill =& {\int }_{l_n}{\int }_{l_n}\left\{\prod _{i=1}^d[(u_i\wedge v_i)-(x_{0i}-h_i{n}^{-\frac{1}{3d}})]\times h_n\left(\mathit{u}\right)h_n\left(\mathit{v}\right)\right\}d\mathit{v}d\mathit{u}\hfill \\ \hfill =& \prod _{i=1}^d\{{\int }_{R_1^i+R_3^i}[(u\wedge v)-(x_{0i}-h_i{n}^{-\frac{1}{3d}})]dvdu\phantom{\}}\hfill \\ \hfill & -2{\int }_{R_2^i}\phantom{\{}((u\wedge v)-(x_{0i}-h_i{n}^{-\frac{1}{3d}})]dvdu\}\hfill \\ \hfill =& 2^d\prod _{i=1}^d\{S_{1i}+S_{2i}-S_{3i}\},\hfill \end{array}$$ (4.11)

where the last equality follows by symmetry and Fubini–Tonelli and the integrals in the braces are to be evaluated below:

$$\begin{array}{cc}\hfill S_{1i}\equiv & {\int }_{x_{0i}-h_i{n}^{-\frac{1}{3d}}}^{x_{0i}}{\int }_v^{x_{0i}}\left\{v-\left(x_{0i}-h_i{n}^{-\frac{1}{3d}}\right)\right\}dudv\hfill \\ \hfill =& {\int }_{x_{0i}-h_i{n}^{-\frac{1}{3d}}}^{x_{0i}}\left\{(x_{0i}-v)\left(v-x_{0i}+h_i{n}^{-\frac{1}{3d}}\right)\right\}dv\hfill \\ \hfill =& {\int }_{x_{0i}+h_i{n}^{-\frac{1}{3d}}}^{h_i{n}^{-\frac{1}{3d}}}\left\{y\left(-y+h_i{n}^{-\frac{1}{3d}}\right)\right\}dy\left[\text{change of variable}\right]\hfill \end{array}$$

while, again, by a change of variable argument:

$$\begin{array}{cc}\hfill S_{2i}\equiv & {\int }_{x_{0i}}^{x_{0i}+h_i{n}^{-\frac{1}{3d}}}{\int }_v^{x_{0i}+h_i{n}^{-\frac{1}{3d}}}\left\{v-\left(x_{0i}-h_i{n}^{-\frac{1}{3d}}\right)\right\}dudv\hfill \\ \hfill =& {\int }_{x_{0i}}^{x_{0i}+h_i{n}^{-\frac{1}{3d}}}\left\{\left[(x_{0i}-v)+h_i{n}^{-\frac{1}{3d}}\right]\left[(v-x_{0i})+h_i{n}^{-\frac{1}{3d}}\right]\right\}dv\hfill \\ \hfill =& {\int }_0^{h_i{n}^{-\frac{1}{3d}}}\left\{\left(-y+h_i{n}^{-\frac{1}{3d}}\right)\left(y+h_i{n}^{-\frac{1}{3d}}\right)\right\}dy,\hfill \end{array}$$

and similarly:

$$\begin{array}{cc}\hfill S_{3i}\equiv & {\int }_{x_{0i}-h_i{n}^{-\frac{1}{3d}}}^{x_{0i}}\left\{h_i{n}^{-\frac{1}{3d}}\left(v-x_{0i}+h_i{n}^{-\frac{1}{3d}}\right)\right\}dv\hfill \\ \hfill =& h_i{n}^{-\frac{1}{3d}}{\int }_0^{h_i{n}^{-\frac{1}{3d}}}\left\{h_i{n}^{-\frac{1}{3d}}-y\right\}dy.\hfill \end{array}$$

Let now q_i := h_in^−1/3d, for i ∈ {1, 2, …, d}, and observe that

$$S_{1i}+S_{2i}-S_{3i}={\int }_0^{q_i}\{y(q_i-y)+q_i^2-y^2+q_i^2-q_{i}y\}dy=\cdots =\frac{4}{3}{h}_i^3{n}^{-\frac{1}{d}},$$

so that plugging al these in (4.11) yields the desired (4, 10).

Now, recall from the definition of f_n that θ ∈ (0, 1) was arbitrary but fixed. Also, from ∫_{(0, ∞)^d}f_n(x) dx = 1 we can get an explicit expression for the normalizing constant d_n:

$$\begin{array}{cc}\hfill d_n=& {\int }_{l_n}f\left(\mathit{x}\right)d\mathit{x}+{\int }_{l_n^c}f\left(\mathit{x}\right)d\mathit{x}+\theta {\int }_{l_n}{g}_n\left(\mathit{x}\right)d\mathit{x}\hfill \\ \hfill =& 1+\theta {\int }_{l_n}{g}_n\left(\mathit{x}\right)d\mathit{x}=1+{(-1)}^d\theta b\prod _{i=1}^d\left\{h_i^2\right\}\cdot n^{-\frac{2}{3}},\hfill \end{array}$$ (4.12)

where the second to last equality follows from ∫_{(0, ∞)^d}f(x) dx = 1, while the last equality follows from (4.9). Notice from (4.12) that d_n ↓ 1 as n ↑ ∞. Also, from the easily verifiable identity $g_n\left({\mathit{x}}_0\right)={(-1)}^{d}b{\Pi }_{i=1}^d\left\{h_i\right\}{n}^{-1∕3}$, we have

$$\begin{array}{cc}\hfill n^{\frac{1}{3}}\mid f_n\left({\mathit{x}}_0\right)-f\left({\mathit{x}}_0\right)\mid =& n^{\frac{1}{3}}\mid \frac{f\left({\mathit{x}}_0\right)+{(-1)}^{d}b\prod _{i=1}^d\left\{h_i\right\}{n}^{-\frac{1}{3}}}{d_n}-f\left({\mathit{x}}_0\right)\mid \hfill \\ \hfill =& \mid n^{\frac{1}{3}}\left\{\frac{1}{d_n}-1\right\}f\left({\mathit{x}}_0\right)+\frac{{(-1)}^{d}b\theta \prod _{i=1}^d\left\{h_i\right\}}{d_n}\mid \hfill \\ \hfill \to & {(-1)}^{d}b\theta \prod _{i=1}^d\left\{h_i\right\}(>0),\text{as}n\to \infty .\hfill \end{array}$$ (4.13)

Also,

$$\begin{array}{cc}\hfill 2n{h}^2(f_n,f)=& n{\int }_{l_n}{\left\{\sqrt{f_n\left(\mathit{x}\right)}-\sqrt{f\left(\mathit{x}\right)}\right\}}^{2}d\mathit{x}+n{\int }_{l_n^c}{\left\{\sqrt{f_n\left(\mathit{x}\right)}-\sqrt{f\left(\mathit{x}\right)}\right\}}^{2}d\mathit{x}\hfill \\ \hfill =& n{\int }_{l_n}{\left\{\frac{f_n\left(\mathit{x}\right)-f\left(\mathit{x}\right)}{\sqrt{f_n\left(\mathit{x}\right)}+\sqrt{f\left(\mathit{x}\right)}}\right\}}^{2}d\mathit{x}+{\delta }_n^2{\int }_{l_n^c}f\left(\mathit{x}\right)d\mathit{x},\hfill \end{array}$$ (4.14)

where,

$$\begin{array}{cc}\hfill {\delta }_n\equiv & \sqrt{n}\left\{1-\frac{1}{\sqrt{d_n}}\right\}=\sqrt{n}\left\{\frac{\sqrt{d_n}-1}{\sqrt{d_n}}\right\}\hfill \\ \hfill =& \frac{\sqrt{n}\left\{\sqrt{1+\mathcal{O}\left(n^{-\frac{2}{3}}\right)}-1\right\}}{\sqrt{d_n}}\to 0,\text{as}n\to \infty ,\hfill \end{array}$$

with the convergence on the last display following from (4.12). Applying this to (4.14), we have:

$$2n{h}^2(f_n,f)=n{\int }_{l_n}{\left\{\frac{f_n\left(\mathit{x}\right)-f\left(\mathit{x}\right)}{\sqrt{f_n\left(x\right)}+\sqrt{f\left(\mathit{x}\right)}}\right\}}^{2}d\mathit{x}+o\left(1\right)$$ (4.15)

as n → ∞, because $0\le {\int }_{I_n^c}f\left(\mathit{x}\right)d\mathit{x}\le 1$

For fixed $n\in \mathbb{N}$, such that f and g_n be continuous and strictly positive on I_n, let x_(n) and x⁽ⁿ⁾ denote, respectively, a minimizer and a maximizer of f on the compact set I_n. Let also y_(n) and y_(n) denote, respectively, a minimizer and a maximizer of g_n on the compact set I_n. Observe that, since I_n is a decreasing sequence of compact sets converging to {x₀}, all of x_(n), x⁽ⁿ⁾, y_(n) and y⁽ⁿ⁾ converage to x₀ as n → ∞. Also,

$$\begin{array}{cc}\hfill \underset{x\in l_n}{\sup }\mid \frac{f_n\left(\mathit{x}\right)-f\left(\mathit{x}\right)}{f\left(\mathit{x}\right)}\mid & =\underset{x\in l_n}{\sup }\mid \left(\frac{1}{d_n}-1\right)+\frac{\theta g_n\left(\mathit{x}\right)}{d_{n}f\left(x\right)}\mid \hfill \\ \hfill & \le \left(1-\frac{1}{d_n}\right)+\frac{\theta \underset{x\in l_n}{\sup }\left\{g_n\left(\mathit{x}\right)\right\}}{d_n\underset{x\in l_n}{\inf }\left\{f\left(\mathit{x}\right)\right\}}\hfill \\ \hfill & \to 0,\text{as}n\to \infty ,\hfill \end{array}$$ (4.16)

because g_n is pointwise non-increasing in $n\in \mathbb{N},g_n\left(x_0\right)=\mathcal{O}\left(n^{-1∕3}\right)$ and f(x₀) > 0.

Also,

$$\begin{array}{cc}\hfill D_1\left(n\right)& \equiv {\int }_{l_n}{\{f_n\left(\mathit{x}\right)-f\left(\mathit{x}\right)\}}^{2}d\mathit{x}\hfill \\ \hfill & =\frac{1}{d_n^2}{\int }_{l_n}\left\{{\theta }^2{g}_n^2\left(\mathit{x}\right)-\mathcal{O}\left(n^{-\frac{2}{3}}\right)f\left(\mathit{x}\right)g_n\left(\mathit{x}\right)+\mathcal{O}\left(n^{-\frac{4}{3}}\right)f^2\left(\mathit{x}\right)\right\}d\mathit{x}\hfill \end{array}$$

and noticing that

$$0\le {\int }_{l_n}\left\{g_n\left(\mathit{x}\right)f\left(\mathit{x}\right)\right\}d\mathit{x}\le f\left({\mathit{x}}^{\left(n\right)}\right){\int }_{l_n}\left(\right\{g_n\left(\mathit{x}\right)\}d\mathit{x}=\mathcal{O}\left(n^{-\frac{2}{3}}\right),$$

so that,

$$\begin{array}{cc}\hfill n{D}_1\left(n\right)=& \frac{n}{d_n^2}\left\{{\left(\frac{8}{3}\right)}^d{\theta }^2{b}^2\prod _{i=1}^d\left\{h_i^3\right\}\cdot n^{-1}+o\left(n^{-\frac{4}{3}}\right)\right\}\hfill \\ \hfill & \to {\left(\frac{8}{3}\right)}^d{\theta }^2{b}^2\prod _{i=1}^d\left\{h_i^3\right\},\text{as}n\to \infty .\hfill \end{array}$$ (4.17)

Now, since f is block-decreasing, we have,

$$0<f\left({\mathit{x}}_0+n^{-\frac{1}{3d}}{\mathit{I}}_d\mathit{h}\right)\le f\left(\mathit{x}\right)\le f({\mathit{x}}_0-n^{-\frac{1}{3d}}{\mathit{I}}_d\mathit{h})$$

for all x ∈ I_n and n ≥ n₁. Hence,

$$\frac{n{D}_1\left(n\right)}{f({\mathit{x}}_0-n^{-\frac{1}{3d}}{\mathit{I}}_d\mathit{h})}\le n{\int }_{l_n}\frac{{\{f_n\left(\mathit{x}\right)-f\left(\mathit{x}\right)\}}^2}{f\left(\mathit{x}\right)}d\mathit{x}\le \frac{n{D}_1\left(n\right)}{f({\mathit{x}}_0+n^{-\frac{1}{3d}}{\mathit{I}}_d\mathit{h})}$$

which, ahead with Eq. (4.17) and sandwich, yields

$$n{\int }_{l_n}\frac{{\{f_n\left(\mathit{x}\right)-f\left(\mathit{x}\right)\}}^2}{f\left(\mathit{x}\right)}d\mathit{x}\to {\left(\frac{8}{3}\right)}^d{\theta }^2{b}^2\cdot \frac{\prod _{i=1}^d\left\{h_i^3\right\}}{f\left(x_0\right)},\text{as}n\to \infty .$$

Applying all of the above to (4.15), and appealing to Lemma 2 of [29], we get

$$n{h}^2(f_n,f)=\frac{1}{8}{\int }_{l_n}\frac{{\{f_n\left(\mathit{x}\right)-f\left(\mathit{x}\right)\}}^2}{f\left(\mathit{x}\right)}d\mathit{x}+o\left(1\right)$$ (4.18)

$$\to \frac{8^{d-1}}{3^{d}f\left({\mathit{x}}_0\right)}{\theta }^2{b}^2\prod _{i=1}^d\left\{h_i^3\right\}$$ (4.19)

as n → ∞, so that by applying (4.13) and (4.19) to Lemma 4.2, we get

$$\begin{array}{cc}\hfill \underset{n\to \infty }{\underset{¯}{\lim }}& \underset{T_n}{\inf }\max \left\{{\mathit{E}}_{f_n}\left\{n^{\frac{1}{3}}\mid T_n-f_n\left({\mathit{x}}_0\right)\mid \right\},{\mathit{E}}_f\left\{n^{\frac{1}{3}}\mid T_n-f\left({\mathit{x}}_0\right)\mid \right\}\right\}\hfill \\ \hfill & \ge \frac{1}{4}\left\{{(-1)}^{d}b\right\}\theta c\exp \left\{-\frac{2^{3d-2}}{3^{d}f\left({\mathit{x}}_0\right)}{\theta }^2{b}^2{c}^3\right\}≕G_{f,x_0}(c,\theta )\hfill \end{array}$$

where $c\equiv {\Pi }_{i=1}^d\left\{h_i\right\}$. For a fixed θ ∈ (0, 1) the maximum of G_f,x0(c, θ) is attained at

$$c\left(\theta \right)={\left\{\frac{3^{d-1}f\left({\mathit{x}}_0\right)}{2^{3d-2}{\theta }^2{b}^2}\right\}}^{\frac{1}{3}}$$

and is equal to

$$G_f(c\left(\theta \right),\theta )=\frac{e^{-\frac{1}{3}}}{2^d}{\left\{3^{d-1}\theta \right\}}^{\frac{1}{3}}{\left\{{(-1)}^d{\phantom{\mid }\frac{{\partial }^{d}f\left(\mathit{x}\right)}{\partial x_1\cdots \partial x_d}\mid }_{x=x_0}f\left({\mathit{x}}_0\right)\right\}}^{\frac{1}{3}},$$

the latter being an increasing function of θ ∈ (0, 1).

This implies that

$$\begin{array}{cc}\hfill \underset{n\to \infty }{\underset{¯}{\lim }}& \underset{T_n}{\inf }\max \left\{{\mathit{E}}_{f_n}\left\{n^{\frac{1}{3}}\mid T_n-f_n\left({\mathit{x}}_0\right)\mid \right\},{\mathit{E}}_f\left\{n^{\frac{1}{3}}\mid T_n-f\left({\mathit{x}}_0\right)\mid \right\}\right\}\hfill \\ \hfill & \ge \frac{e^{-\frac{1}{3}}}{2^d}{\{\theta \cdot 3^{d-1}\}}^{\frac{1}{3}}{\left\{{(-1)}^d{\phantom{\mid }\frac{{\partial }^{d}f\left(\mathit{x}\right)}{\partial {\mathit{x}}_1\cdots \partial {\mathit{x}}_d}\mid }_{\mathit{x}={\mathit{x}}_0}\cdot f\left({\mathit{x}}_0\right)\right\}}^{\frac{1}{3}}.\hfill \end{array}$$

Overall, we are allowed to take θ ↑ 1 in the above display, even if θ = 1 is not a valid configuration, yielding the lower bound in the wording of the proposition. The proof is thus complete.

5. Discussion and open problems

Once consistency has been established, interest focuses on rates of convergence of the MLE and other properties, including the behavior of ${\widehat{f}}_n$ at zero and pointwise limiting distributions. We have the following conjectures concerning the MLE ${\widehat{f}}_n$ for the class ${\mathcal{F}}_{\mathrm{SMU}}\left(d\right)$. Work is currently underway on all of these further problems.

Conjecture 1

If f₀(0) < ∞, then we conjecture that $P_0({\widehat{f}}_n\left(0\right)\le M{\left(\log n\right)}^{d-1})\to 1forsomeM>0$.

Conjecture 2

If f₀(0) < ∞ and f₀ is concentrated on [0, M1] for some 0 < M < ∞, then $h({\widehat{f}}_n,f_0)=O_p\left(n^{-1∕3}{\left(\log n\right)}^Y\right)$ for some γ depending only on d.

Concerning rates of convergence of the estimators at a fixed point, we do not yet have any upper bound results to accompany the lower bound results of Proposition 4.1. Thus there remain the following two possibilities: (a) the pointwise rate of convergence under Assumption 4.1 is n^1/3, and we expect convergence in distribution with the rate n^1/3, or, (b) the lower bound given in Proposition 4.1 is not yet sharp, and we should expect log terms in the rate (as might be expected from the covering number results of [10]). Our corresponding conjectures for these two possible scenarios are given below as Conjectures 3a and 3b respectively.

Conjecture 3a

Suppose that f₀ has ∂^df₀(x)/∂x₁…∂x_d continuous in a neighborhood of x₀ with

$${\partial }^d{f}_0\left({\mathit{x}}_0\right)\equiv {\phantom{\mid }\frac{{\partial }^d{f}_0\left(\mathit{x}\right)}{\partial x_1\cdots \partial x_d}\mid }_{\mathit{x}={\mathit{x}}_0}\ne 0.$$

Let $\{W\left(\mathit{t}\right):\mathit{t}\in {\mathbb{R}}^d\}$ be a 2^d-sided Brownian sheet process on ${\mathbb{R}}^d$ and let

$$\mathbb{Y}\left(\mathit{t}\right)\equiv \sqrt{f_0\left({\mathit{x}}_0\right)}W\left(\mathit{t}\right)+\frac{{(-1)}^d}{2^d}{(-1)}^d{\partial }^d{f}_0\left({\mathit{x}}_0\right){\mid \mathit{t}\mid }^2.$$

Then, in keeping with our lower bound results of Section 4, we conjecture that

$$n^{1∕3}({\widehat{f}}_n\left({\mathit{x}}_0\right)-f_0\left({\mathit{x}}_0\right))\to {}_d\partial ^d\mathbb{H}\left(\mathit{t}\right){\mid }_{\mathit{t}=0}$$

where the process $\mathbb{H}$ is determined by

1. $$\mathbb{H}\left(\mathit{t}\right)\ge \mathbb{Y}\left(t\right)\text{for all}\mathit{t}\in {\mathbb{R}}^d,$$
2. $${\int }_{{\mathbb{R}}^d}(\mathbb{H}\left(\mathit{t}\right)-\mathbb{Y}\left(\mathit{t}\right))d\left({\partial }^d\mathbb{H}\left(\mathit{t}\right)\right)=0,\text{and}$$
3. $$V_{{\partial }^d\mathbb{H}}[\mathit{u},\mathit{v})\ge 0\text{for all}\mathit{u}\le \mathit{v}\in {\mathbb{R}}^d.$$

Partial results concerning Conjecture 3a were obtained in [37].

Conjecture 3b

As suggested in part by the covering number results of [10], the pointwise rate of convergence is (n/(log n)^d–1/2)^1/3. This would entail an improved version of Proposition 4.1. In this case, we do not yet have conjectures concerning the limiting distribution.