Using the sample maximum to estimate the parameters of the underlying distribution

Abstract

We propose novel estimators for the parameters of an exponential distribution and a normal distribution when the only known information is a sample of sample maxima; i.e., the known information consists of a sample of m values, each of which is the maximum of a sample of n independent random variables drawn from the underlying exponential or normal distribution. We analyze the accuracy and precision of the estimators using extreme value theory, as well as through simulations of the sampling distributions. For the exponential distribution, the estimator of the mean is unbiased and its variance decreases as either m or n increases. Likewise, for the normal distribution, we show that the estimator of the mean has negligible bias and the estimator of the variance is unbiased. While the variance of the estimators for the normal distribution decreases as m, the number of sample maxima, increases, the variance increases as n, the sample size over which the maximum is computed, increases. We apply our method to estimate the mean length of pollen tubes in the flowering plant Arabidopsis thaliana, where the known biological information fits our context of a sample of sample maxima.

1 Introduction

Consider the scenario where one has obtained data where each observation is the maximum value of n independent, identically distributed random variables drawn from either an exponential distribution or a normal distribution with unknown parameters. That is, $X_{ij}\stackrel{\text{iid}}{\sim }Exp\left(\beta \right)$ or $X_{ij}\stackrel{\text{iid}}{\sim }N(\mu ,{\sigma }^2)$ for i = 1, …, n and known data is drawn from $Y_j=\text{max}{\left\{X_{ij}\right\}}_{i=1}^n$ for j = 1, …, m. Here we present a process to estimate the mean β of the underlying exponential distribution or the mean μ and variance σ² of the underlying normal distribution from only the set of Y_j’s.

Much previous research has been conducted in the field of extreme value theory on the distribution of the maximum from a sample of n independent, identically distributed random variables. In particular, the Fisher-Tippett-Gnedenko theorem states that the distribution of the sample maximum, after proper rescaling, can only converge to one of three types of distributions: the Gumbel distribution, the Fréchet distribution, or the Weibull distribution [1]. When the original underlying distribution is exponential or normal, the limiting distribution of the rescaled sample maximum is the Gumbel distribution. Extreme value theory has been applied in many applications, such as estimating the probability of an extreme flood, severe adverse side effect of a drug, maximum environmental load on a structure, or large insurance loss [2]. In these applications, the underlying distribution and its parameters are typically known and the focus is on estimating the probability distribution of the sample maximum. The focus of our scenario is novel in that we are using a known sample of sample maxima to estimate unknown parameters of the underlying distribution.

Standard techniques for estimating unknown parameters, such as method of moments or maximum likelihood estimation, typically assume that the known information consists of a sample of observations drawn directly from the underlying population distribution. However, in our scenario under consideration, the direct observations are unknown. Rather, we only know the maximum value of each sample of direct observations. The estimators we propose in the following sections are the first, to our knowledge, for estimating unknown population parameters when the only known information is a sample of sample maxima.

2 Estimator for exponential distribution

We begin by considering the case where the underlying distribution is exponential with unknown mean β. In Theorem 1 below, we propose an estimator for β and compute its expected value and variance.

Theorem 1. Let $X_{ij}\stackrel{iid}{\sim }Exp\left(\beta \right)$ for i = 1, …, n and $Y_j=\text{max}{\left\{X_{ij}\right\}}_{i=1}^n$ for j = 1, …, m. Set

$$\widehat{\beta }=\frac{\overline{Y}}{H_n}=\frac{1}{m{H}_n}\sum _{j=1}^m{Y}_j.$$ (1)

Then

$$E\left(\widehat{\beta }\right)=\beta ,var\left(\widehat{\beta }\right)=\frac{{\beta }^2{G}_n}{m{H}_n^2}$$ (2)

where

$$H_n=\sum _{i=1}^n\frac{1}{i},G_n=\sum _{i=1}^n\frac{1}{i^2}.$$

Proof. From the formula for $\widehat{\beta }$ given in Eq (1), it directly follows that

$$E\left(\widehat{\beta }\right)=\frac{E\left(Y_j\right)}{H_n},\text{var}\left(\widehat{\beta }\right)=\frac{\text{var}\left(Y_j\right)}{m{H}_n^2}.$$

Hence, it only remains to compute the expected value and variance of the maximum of a single sample of n independent Exp(β) random variables. Let X_(i) denote the ith smallest observation from such a sample. Then Y_j = X_(n) can be decomposed as the following telescoping sum:

$$Y_j=X_{\left(n\right)}=X_{\left(1\right)}+(X_{\left(2\right)}-X_{\left(1\right)})+\cdots +(X_{\left(n\right)}-X_{(n-1)}).$$

Due to the memoryless property of the exponential distribution, X₍₂₎ − X₍₁₎ is independent of X₍₁₎. Moreover, while X₍₁₎ is the minimum of n independent Exp(β) random variables, X₍₂₎ − X₍₁₎ can be viewed as the minimum of a sample of n − 1 independent Exp(β) random variables. Likewise, all of the terms in the telescoping sum for Y_j = X_(n) are independent with X_(i+1) − X_(i) equal in distribution to the minimum of a sample of n − i independent Exp(β) random variables, which in turn is equal in distribution to Exp(β/(n − i)). Thus,

$$E\left(Y_j\right)=E\left(X_{\left(n\right)}\right)=\frac{\beta }{n}+\frac{\beta }{n-1}+\cdots +\frac{\beta }{2}+\beta =\beta \sum _{i=1}^n\frac{1}{i}=\beta H_n$$

and

$$\text{var}\left(Y_j\right)=\text{var}\left(X_{\left(n\right)}\right)=\frac{{\beta }^2}{n^2}+\frac{{\beta }^2}{{(n-1)}^2}+\cdots +\frac{{\beta }^2}{2^2}+{\beta }^2={\beta }^2\sum _{i=1}^n\frac{1}{i^2}={\beta }^2{G}_n.$$

Substituting these expressions for E(Y_j) and var(Y_j) into the equations for the expected value and variance of $\widehat{\beta }$ produces the formulas given in Eq (2).

As a consequence of Theorem 1, we have shown that $\widehat{\beta }$ is an unbiased estimator for β and that its variance decreases at a rate proportional to $\frac{1}{m}$. Since $G_n\to \frac{{\pi }^2}{6}$ and H_n → ∞ at rate log n as n → ∞, the variance of $\widehat{\beta }$ decreases at a rate proportional to $\frac{1}{{\left(\text{log}n\right)}^2}$. Thus, the precision of the estimator $\widehat{\beta }$ can be improved more rapidly by increasing m, the number of sample maxima, compared to increasing n, the sample size over which the maximum is computed.

3 Estimators for normal distribution

We now consider the case where the underlying distribution is normal with unknown mean μ and unknown variance σ². In Theorem 2 below, we propose estimators for μ and σ² and analyze their expected value, while in Theorem 3 we analyze the variance of the estimators.

Theorem 2. Let $X_{ij}\stackrel{iid}{\sim }N(\mu ,{\sigma }^2)$ for i = 1, …, n and $Y_j=\text{max}{\left\{X_{ij}\right\}}_{i=1}^n$ for j = 1, …, m. Let $\overline{Y}$ and $S_Y^2$ denote the sample mean and sample variance, respectively, of the Y_j’s. Set

$$\widehat{\mu }=\overline{Y}-\frac{k_n}{\sqrt{c_n}}{S}_Y,{\widehat{\sigma }}^2=\frac{S_Y^2}{c_n}$$ (3)

where k_n denotes the mean and c_n denotes the variance of the maximum of n independent, identically distributed N(0, 1) random variables. Then $E\left({\widehat{\sigma }}^2\right)={\sigma }^2$, while $E\left(\widehat{\mu }\right)>\mu$ with $E\left(\widehat{\mu }\right)\to \mu$ as m → ∞.

Proof. The cumulative distribution function of Y_j is given by

$$\begin{array}{cc}\hfill F_{Y_j}\left(y\right)& =P(Y_j\le y)={\left[P(X_{ij}\le y)\right]}^n\hfill \\ & ={\left[P\right(\frac{X_{ij}-\mu }{\sigma }\le \frac{y-\mu }{\sigma }\left)\right]}^n={[\Phi (\frac{y-\mu }{\sigma }\left)\right]}^n.\hfill \end{array}$$

Differentiating, the probability density function of Y_j is

$$f_{Y_j}\left(y\right)=n{[\Phi (\frac{y-\mu }{\sigma }\left)\right]}^{n-1}\varphi \left(\frac{y-\mu }{\sigma }\right)\frac{1}{\sigma },$$

where Φ(z) denotes the cumulative distribution function and ϕ(z) denotes the probability density function of a N(0, 1) random variable. The expected value of Y_j can then be calculated as

$$\begin{array}{cc}\hfill E\left(Y_j\right)& ={\int }_{-\infty }^{\infty }y{f}_{Y_j}\left(y\right)dy={\int }_{-\infty }^{\infty }yn{[\Phi (\frac{y-\mu }{\sigma }\left)\right]}^{n-1}\varphi \left(\frac{y-\mu }{\sigma }\right)\frac{1}{\sigma }dy\hfill \\ & =\mu +\sigma {\int }_{-\infty }^{\infty }zn{[\Phi (z\left)\right]}^{n-1}\varphi \left(z\right)dz=\mu +\sigma k_n.\hfill \end{array}$$

We also obtain that the variance of Y_j is

$$\begin{array}{cc}\hfill \text{var}\left(Y_j\right)& ={\int }_{-\infty }^{\infty }{(y-E\left(Y_j\right))}^2{f}_{Y_j}\left(y\right)dy\hfill \\ & ={\int }_{-\infty }^{\infty }{(y-\mu -\sigma k_n)}^{2}n{[\Phi (\frac{y-\mu }{\sigma }\left)\right]}^{n-1}\varphi \left(\frac{y-\mu }{\sigma }\right)\frac{1}{\sigma }dy\hfill \\ & ={\sigma }^2{\int }_{-\infty }^{\infty }{(z-k_n)}^{2}n{[\Phi (z\left)\right]}^{n-1}\varphi \left(z\right)dz={\sigma }^2{c}_n.\hfill \end{array}$$

We can now use the equations for E(Y_j) and var(Y_j) to compute the expected value of the estimators. In particular, the expected value of the estimator of the variance is

$$E\left({\widehat{\sigma }}^2\right)=\frac{E\left(S_Y^2\right)}{c_n}=\frac{\text{var}\left(Y_j\right)}{c_n}=\frac{{\sigma }^2{c}_n}{c_n}={\sigma }^2,$$

while the estimator of the mean is

$$E\left(\widehat{\mu }\right)=E\left(Y_j\right)-\frac{k_n}{\sqrt{c_n}}E\left(S_Y\right)>(\mu +\sigma k_n)-\frac{k_n}{\sqrt{c_n}}\left(\sigma \sqrt{c_n}\right)=\mu .$$

Although Jensen’s inequality implies that $\widehat{\mu }$ has positive bias since $E\left(S_Y\right)<\sqrt{E\left(S_Y^2\right)}=\sqrt{\text{var}\left(Y_j\right)}$, the bias of the sample standard deviation goes to zero as the sample size increases. Hence, $E\left(\widehat{\mu }\right)\to \mu$ as m → ∞.

Note that the constants k_n and c_n that appear in the formulas for the estimators depend upon only the sample size n. Exact integral expressions exist for k_n and c_n and are given in the proof of Theorem 2, but the integrals cannot be evaluated in closed form. However, the constants can be approximated either analytically or numerically.

In [3], Cramér showed that b_n(Z_(n) − b_n) converges in distribution to the standard Gumbel distribution, where Z_(n) is the maximum of a sample of n independent, identically distributed N(0, 1) random variables and

$$b_n={\left(2\text{log}n\right)}^{\frac{1}{2}}-\frac{\frac{1}{2}\text{log}\left(4\pi \text{log}n\right)}{{\left(2\text{log}n\right)}^{\frac{1}{2}}}.$$

Since the standard Gumbel distribution has mean equal to γ ≈ 0.5772, the Euler-Mascheroni constant, and variance equal to $\frac{{\pi }^2}{6}$, we can use these values to approximate

$$k_n\approx \frac{\gamma }{b_n}+b_n,c_n\approx \frac{{\pi }^2}{6b_n^2}.$$ (4)

Fig 1 displays the values of the analytic approximations for k_n and c_n given in Eq (4) for n ranging from 10 to 100,000, along with corresponding numerical approximations, plotted on a semi-log scale. The numerical approximations for k_n and c_n were computed from 10,000 realizations of a simulation of the maximum.

Fig 1. From the top curve to the bottom, the plot displays the values of the analytic approximation for k_n (solid) and the numerical approximation for k_n (dashed), along with the analytic approximation for c_n (dotted) and the numerical approximation for c_n (dot-dashed).

While Theorem 2 showed that $\widehat{\mu }$ is positively biased with the bias approaching zero as m → ∞, the estimation bias is fairly minimal even for relatively small values of m. Fig 2 displays the sampling distributions of the estimators for μ and σ² for varying values of m and n. In all the simulations, the sampling distribution of $\widehat{\mu }$ is fairly centered around the true value of μ = 0. Setting the true value of μ to a nonzero value simply shifts the sampling distribution of $\widehat{\mu }$ and has no effect on ${\widehat{\sigma }}^2$. We also observe from Fig 2 that the variability of the sampling distributions of the estimators decreases as m, the number of sample maxima, increases, but increases as n, the sample size over which the maximum is computed, increases. We derive an analytical justification for this behavior in Theorem 3 below.

Fig 2. Estimates of $\widehat{\mu }$ (top) and ${\widehat{\sigma }}^2$ (bottom) from 100 realizations with n, m = 10, 100, and 1000.

The horizontal lines indicate the true values of μ and σ².

Theorem 3. Let $X_{ij}\stackrel{iid}{\sim }N(\mu ,{\sigma }^2)$ for i = 1, …, n and $Y_j=\text{max}{\left\{X_{ij}\right\}}_{i=1}^n$ for j = 1, …, m. Let $\overline{Y}$ and $S_Y^2$ denote the sample mean and sample variance, respectively, of the Y_j’s. Set

$$\widehat{\mu }=\overline{Y}-\frac{k_n}{\sqrt{c_n}}{S}_Y,{\widehat{\sigma }}^2=\frac{S_Y^2}{c_n}$$

where k_n denotes the mean and c_n denotes the variance of the maximum of n independent, identically distributed N(0, 1) random variables. Then

$$var\left({\widehat{\sigma }}^2\right)={\sigma }^4(\frac{2}{m-1}+\frac{\kappa }{m})$$

and

$$var\left(\widehat{\mu }\right)\approx \frac{{\sigma }^2{c}_n}{m}+\frac{{\sigma }^2{k}_n^2}{4}(\frac{2}{m-1}+\frac{\kappa }{m})-\frac{{\sigma }^2{k}_n\sqrt{c_n}{\gamma }_1}{m}$$

where γ₁ is the skewness and κ is the excess kurtosis of the distribution of Y_j.

Proof. The variance of $S_Y^2$ can be computed as

$$\text{var}\left(S_Y^2\right)={\left(\text{var}\left(Y_j\right)\right)}^2(\frac{2}{m-1}+\frac{\kappa }{m}),$$

where κ is the excess kurtosis of the distribution of Y_j [4]. Since we computed that var(Y_j) = σ²c_n in Theorem 2, we obtain the desired result

$$\text{var}\left({\widehat{\sigma }}^2\right)=\text{var}\left(\frac{S_Y^2}{c_n}\right)={\sigma }^4(\frac{2}{m-1}+\frac{\kappa }{m})$$

for the variance of the variance estimator. Now for the variance of the mean estimator, we compute that

$$\text{var}\left(\widehat{\mu }\right)=\text{var}(\overline{Y}-\frac{k_n}{\sqrt{c_n}}{S}_Y)=\text{var}\left(\overline{Y}\right)+\frac{k_n^2}{c_n}\text{var}\left(S_Y\right)-2\frac{k_n}{\sqrt{c_n}}\text{cov}(\overline{Y},S_Y).$$

To simplify the above expression, we use the fact that $\text{var}\left(\overline{Y}\right)=\frac{\text{var}\left(Y_j\right)}{m}$, along with the approximations

$$\text{var}\left(S_Y\right)\approx \frac{\text{var}\left(S_Y^2\right)}{4\text{var}\left(Y_j\right)},\text{cov}(\overline{Y},S_Y)\approx \frac{\text{cov}(\overline{Y},S_Y^2)}{2\sqrt{\text{var}\left(Y_j\right)}}=\frac{\text{var}\left(Y_j\right){\gamma }_1}{2m},$$

where γ₁ is the skewness of the distribution of Y_j [5]. Using these approximations, we obtain the desired result

$$\text{var}\left(\widehat{\mu }\right)\approx \frac{{\sigma }^2{c}_n}{m}+\frac{{\sigma }^2{k}_n^2}{4}(\frac{2}{m-1}+\frac{\kappa }{m})-\frac{{\sigma }^2{k}_n\sqrt{c_n}{\gamma }_1}{m}$$

for the variance of the mean estimator.

As n increases, the value of κ increases monotonically from 0, the excess kurtosis of the normal distribution, to 12/5, the excess kurtosis of the Gumbel distribution [1]. Thus, from Theorem 3, we observe that the variance of the sampling distribution of ${\widehat{\sigma }}^2$ increases proportionally to σ⁴ as σ increases and decreases proportionally to 1/m as m increases, but only slightly increases as n increases.

As with the excess kurtosis, the skewness of the distribution of Y_j increases monotonically from the value for the normal distribution, i.e., γ₁ = 0, to the value for the standard Gumbel distribution, i.e., γ₁ ≈ 1.13955, as n increases [6]. Since the constant c_n decreases towards 0 while the constant k_n increases towards infinity, the dominant term in the variance of $\widehat{\mu }$ increases proportionally to $k_n^2$ as n increases. We also observe from Theorem 3 that the variance of the sampling distribution of $\widehat{\mu }$ increases proportionally to σ² as σ increases and decreases proportionally to 1/m as m increases. These relationships explain the behavior of the sampling distributions displayed in Fig 2.

4 Biological application

During fertilization in flowering plants, once pollen land on the stigma, the pollen will grow tubes that travel down through a transmitting tract from the stigma toward an ovule. Pollen compete against each other in a race towards the limited number of ovules to determine which pollen will father the seeds. The mean length of the population of pollen tubes at various time points is of interest to plant biologists, yet, to date, there are only measures of the lengths of the longest pollen tubes in such competitions [7]. Since the pollen tube lengths must have a positive value, it is reasonable to assume that the lengths follow an exponential distribution. Hence, our method described in Section 2 will allow the mean pollen tube length to be estimated given the structure of the experimental data.

In [7], Swanson et al. measured the longest pollen tube lengths at four time points for two accessions (i.e., specific geographical populations) of Arabidopsis thaliana in a laboratory setting. For both the Columbia and Landsberg accessions, either m = 8 or m = 9 individual plants were used for each time point. The average number of pollen tubes within each plant was n = 933 for the Columbia accession and n = 727 for the Landsberg accession. Table 1 reports the sample mean of the longest pollen tube from the m plants for each accession after 3, 6, 9, and 24 hours. Using these sample means of the longest lengths and Eq (2) from Theorem 1, we then estimated the overall mean length for each accession at each time point. The resulting estimates and their standard errors are listed in Table 1.

Table 1. The sample mean, $\overline{y}$, of the longest pollen tube from m plants, where each plant contained an average of n pollen tubes, along with the corresponding estimated mean pollen length, $\widehat{\beta }$, and its standard error, for two accessions of Arabidopsis thaliana at varying time points.

Accession	time (hrs)	m	$\overline{y}$ (mm)	$\widehat{\beta }$ (mm)	${\text{SE}}_{\widehat{\beta }}$ (mm)
Columbia (n = 933)	3	9	0.690	0.093	0.005
	6	8	1.069	0.144	0.009
	9	9	2.538	0.342	0.020
	24	9	2.778	0.375	0.022
Landsberg (n = 727)	3	9	0.474	0.066	0.004
	6	8	0.676	0.094	0.006
	9	8	1.795	0.251	0.016
	24	9	2.325	0.324	0.019

To further evaluate the validity of the assumption that the population of pollen tube lengths is exponentially distributed, we produced Q-Q plots of the distribution of the maximum pollen tube length from the laboratory experiments performed by Swanson et al. versus the distribution of the maximum from an exponential distribution. The theoretical distribution of the maximum from an exponential distribution was simulated using 10,000 realizations of $Y_j=\text{max}{\left\{X_{ij}\right\}}_{i=1}^n$ where $X_{ij}\stackrel{\text{iid}}{\sim }Exp\left(\widehat{\beta }\right)$ for i = 1, …, n, using the values of n and $\widehat{\beta }$ that are listed in Table 1. The Q-Q plots, displayed in Fig 3, show a roughly linear relationship, supporting the assumption that the underlying distribution of pollen tube lengths is exponential. Moreover, we performed a Kolmogorov-Smirnov test of the equality of the empirical and theoretic distributions for each accession of Arabidopsis thaliana at each time point. The smallest resulting p-value was 0.52 (corresponding to the Landsberg accession at 9 hours), further indicating that there is no evidence that the distribution of pollen tube lengths differs significantly from an exponential distribution.

Fig 3. Q-Q plots of the distribution of the maximum pollen tube length from laboratory experiments versus the distribution of the maximum from an exponential distribution.

Data Availability

All relevant data are within the manuscript, Supporting Information files, and on the OSF data repository: https://osf.io/zg4ka/.

Funding Statement

A.C. was supported by the National Science Foundation under grant IOS-1645508 (https://www.nsf.gov/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.