On conjugate families and Jeffreys priors for von Mises–Fisher distributions

Abstract

This paper discusses characteristics of standard conjugate priors and their induced posteriors in Bayesian inference for von Mises–Fisher distributions, using either the canonical natural exponential family or the more commonly employed polar coordinate parameterizations. We analyze when standard conjugate priors as well as posteriors are proper, and investigate the Jeffreys prior for the von Mises–Fisher family. Finally, we characterize the proper distributions in the standard conjugate family of the (matrix-valued) von Mises–Fisher distributions on Stiefel manifolds.

1. Introduction

A random unit length vector in ${\mathbb{R}}^d$ has a von Mises–Fisher (or Langevin, short: vMF) distribution with parameter $\theta \in {\mathbb{R}}^d$ if its density with respect to the uniform distribution on the unit hypersphere ${\mathbb{S}}^{d-1}=\{x\in {\mathbb{R}}^d:\parallel x\parallel =1\}$ is given by

$$f(x|\theta )=e^{\theta \prime x}/{}_{0}F_1(;d/2;\parallel \theta {\parallel }^2/4),$$

where, using the rising factorial ${(\nu )}_n=\Gamma (\nu +n)/\Gamma (\nu )$,

$${}_{0}F_1(;\nu ;z)=\sum _{n=0}^{\infty }\frac{1}{{(\nu )}_n}\frac{z^n}{n!}=\sum _{n=0}^{\infty }\frac{\Gamma (\nu )}{\Gamma (\nu +n)}\frac{z^n}{n!}$$

is a generalized hypergeometric series and related to the modified Bessel function of the first kind $I_{\nu }$ via

$${}_{0}F_1(;\nu +1;{\kappa }^2/4)=\frac{I_{\nu }(\kappa )\Gamma (\nu +1)}{{(\kappa /2)}^{\nu }}$$

(e.g., Mardia and Jupp, 1999, p. 168).

We note that the vMF distribution is commonly parameterized using polar coordinates, i.e., $\theta =\kappa \mu$, where $\kappa =\parallel \theta \parallel$ and $\mu \in {\mathbb{S}}^{d-1}$ are the concentration and mean direction parameters, respectively (if $\theta \ne 0$, $\mu$ is uniquely determined as $\theta /\parallel \theta \parallel )$. Using $\theta$ as the parameter, the family $\mathcal{F}$ of vMF distributions on ${\mathbb{S}}^{d-1}$ becomes a natural exponential family through the uniform distribution U on ${\mathbb{S}}^{d-1}$, commonly written as

$$f(x|\theta )=e^{\theta \prime x-M(\theta )},$$

where in the vMF case, the cumulant transform $M(\theta )$ of U is given by

$$e^{M(\theta )}={\int }_{{\mathbb{S}}^{d-1}}{e}^{\theta \prime x}\mathit{dU}(x)={}_{0}F_1(;d/2;\parallel \theta {\parallel }^2/4).$$

Bayesian inference for the vMF distribution is first discussed in Mardia and El-Atoum (1976), who give conjugate priors for $\mu$ when $\kappa$ is known, and derive the Jeffreys prior for the polar coordinates ($\mu$, $\kappa$) parameterization. Guttorp and Lockhart (1988) introduce a Bayesian approach for finding the direction of a signal based on developing standard (e.g., Gutiérrez-Peña and Smith, 1997, Definition 3.1) conjugate priors for the von Mises (vM) distribution (i.e., for d=2) using the canonical ($\theta$) parameterization. Damien and Walker (1999) present a full Bayesian analysis of circular data using the vM distribution by employing standard conjugate priors for the polar coordinates ($\mu$, $\kappa$) parameterization, and developing a Gibbs sampler for this family of distributions. Nuñez-Antonio and Gutiérrez-Peña (2005) provide a full Bayesian analysis of directional (i.e., $d\ge 2$) data using the vMF distribution, again using standard ($\mu$, $\kappa$) conjugate priors and obtaining samples from the posterior using a sampling-importance-resampling method found to outperform Gibbs sampling. Bangert et al. (2010) construct (possibly infinite) mixtures of vMF distributions using standard conjugate priors for the ($\mu$, $\kappa$) parameterization and Dirichlet (process) priors for the mixing probabilities.

Interestingly, none of these references explicitly discuss when the employed priors (and respective posteriors) are actually proper, or whether the conjugate families obtained using the $\theta$ or ($\mu$, $\kappa$) parameterizations are the same. In this paper, we settle these open issues, and also discuss Jeffreys priors for the general ($d\ge 2$) vMF family (Section 2). We also provide results for (matrix-valued) vMF distributions on Stiefel manifolds (Section 3).

2. Results

2.1. Propriety of priors from the standard conjugate family

In what follows, it will be convenient to write

$$C_d(\kappa )=1/{}_{0}F_1(;d/2;{\kappa }^2/4),$$

so that $e^{-M(\theta )}=C_d(\parallel \theta \parallel )$. Let $\theta =\theta (\lambda )$ be a suitable parameterization of $\theta$. For a sample $x_1,\dots ,x_n$ of independent, identically distributed (i.i.d.) observations from the vMF family $\mathcal{F}$, the likelihood function for $\lambda$ is given by

$$L(\lambda |s,n)=e^{s\prime \theta (\lambda )-\mathit{nM}(\theta (\lambda ))}=C_d{(\parallel \theta (\lambda )\parallel )}^n{e}^{s\prime \theta (\lambda )},$$

where $s=x_1+\cdots +x_n$ is the resultant of the sample. Following Gutiérrez-Peña and Smith (1997, Definition 3.1), the standard conjugate family for $\mathcal{F}$ relative to $\lambda$, denoted by ${\mathcal{C}}_{\lambda }(\mathcal{F})$, has densities

$$\pi (\lambda |s,\nu )\propto L(\lambda |s,\nu ).$$

Using such a prior with parameters s₀ and ${\nu }_0$ will result in a posterior with parameters $s_n=s_0+x_1+\cdots +x_n$ and ${\nu }_n={\nu }_0+n$. As clearly

$$\frac{s_n}{{\nu }_n}=\frac{{\nu }_0}{{\nu }_0+n}\frac{s_0}{{\nu }_0}+\frac{n}{{\nu }_0+n}\overline{x},$$

${\nu }_0$ can be interpreted as the prior sample size, and $s_n/{\nu }_n$ as a sample size weighted average of the “prior mean” $s_0/{\nu }_0$ and the sample mean $\overline{x}$.

The standard conjugate family ${\mathcal{C}}_{\theta }(\mathcal{F})$ relative to the canonical parameter $\theta$ has several important properties, in particular the linear relationship between the posterior mean and the sample mean (Diaconis and Ylvisaker, 1979).

We note that the densities $\pi (\lambda |s,\nu )$ are usually taken relative to the Lebesgue measure, which does not quite fit the needs of the commonly used polar coordinates ($\mu$, $\kappa$) parameterization of the vMF family $\mathcal{F}$. Let us generally write $\eta$ for the reference measure employed. Previous work using the ($\mu$, $\kappa$) parameterization seem to take $\eta$ as the product of the Lebesgue measure on $[0,\infty )$ (for $\kappa$) and the uniform distribution U on ${\mathbb{S}}^{d-1}$ (for $\mu$), i.e., $d\eta \propto d\kappa \mathit{dU}(\mu )$. As for $\theta =\kappa \mu$ we have $d\theta =a_d{\kappa }^{d-1}d\kappa \mathit{dU}(\mu )$ (where a_d is the area of the unit hypersphere). The latter may be more natural as reference measure, turning the standard conjugate family relative to the polar coordinates ($\mu$, $\kappa$) parameterization into the (obvious generalization) of what Gutiérrez-Peña and Smith (1997) call the DY-conjugate family for $\mathcal{F}$ relative to the parameterization.

Let ${\mathcal{H}}_{\lambda ;\eta }$ denote the set of all hyperparameters s and $\nu$ for which $\pi (\lambda |s,\nu )$ is a proper distribution on the employed parameter space $\Lambda$ (using $\eta$ as reference measure), i.e.,

$${\mathcal{H}}_{\lambda ;\eta }=\left\{(s,\nu ):{\int }_{\Lambda }{C}_d{(\parallel \theta (\lambda )\parallel )}^{\nu }{e}^{s\prime \theta (\lambda )}d\eta (\lambda )<\infty \right\},$$

and let

$$J(\alpha ,\beta ,\nu )={\int }_0^{\infty }\frac{{\kappa }^{\alpha }{C}_d{(\kappa )}^{\nu }}{C_d(\beta \kappa )}d\kappa .$$

We have the following results.

Theorem 1

For the canonical parameterization $\theta$ of the vMF family and the Lebesgue measure as reference measure $\eta$,

$${\mathcal{H}}_{\theta ;\eta }=\{(s,\nu ):\parallel s\parallel <\nu \},$$

and the normalizing constant is the inverse of $a_{d}J(d-1,\parallel s\parallel ,\nu )$.

In the following a parameter $\alpha$ is introduced which allows to cover both cases of reference measures when using the ($\mu$, $\kappa$) parameterization: the Lebesgue measure (leading to $\alpha =d-1$) and the product of the Lebesgue measure on $[0,\infty )$ and the uniform distribution U on ${\mathbb{S}}^{d-1}$ which is employed in previous work (leading to $\alpha =0)$. Other choices for $\alpha$ lead to additional possible reference measures.

Theorem 2

For the polar coordinates ($\mu$, $\kappa$) parameterization of the vMF family and the reference measure $d\eta ={\kappa }^{\alpha }d\kappa \mathit{dU}(\mu )$ with $\alpha \ge 0$,

$${\mathcal{H}}_{\kappa ,\mu ;\eta }=\{(s,\nu ):\parallel s\parallel <\nu \mathit{or}\parallel s\parallel =\nu <1-2(\alpha +1)/(d-1)\},$$

and the normalizing constant is the inverse of $J(\alpha ,\parallel s\parallel ,\nu )$.

Note that if $d\ge 2$, $1-2(\alpha +1)/(d-1)>0$ is equivalent to $(d-1)/2>\alpha +1$ or $(d-3)/2>\alpha$, which if $\alpha \ge 0$ is only possible if $d\ge 4$. Thus, the set $\{(s,\nu ):\parallel s\parallel =\nu <1-2(\alpha +1)/(d-1)\}$ is non-empty only if $d\ge 4$ and $\alpha <(d-3)/2$, and clearly can only contain points for which $\parallel s\parallel =\nu <1$.

For the proof, we use the following result.

Lemma 1

If $\alpha$ and $\beta$ are nonnegative, $J(\alpha ,\beta ,\nu )<\infty$ if and only if $\beta <\nu$ or $0\le \beta =\nu <1-2(\alpha +1)/(d-1)$.

Proof of Lemma 1

Using the asymptotic approximation $I_{\nu }(\kappa )\approx e^{\kappa }/\sqrt{2\pi \kappa }$ for $\kappa \to \infty$ and $\nu$ fixed (e.g., Abramowitz and Stegun, 1972, http://dlmf.nist.gov/10.40), we have

$$C_d(\kappa )\propto {\kappa }^{d/2-1}/I_{d/2-1}(\kappa )\approx \sqrt{2\pi }{\kappa }^{(d-1)/2}{e}^{-\kappa }.$$

Hence, for large $\kappa$, the integrand in J is “approximately proportional” to

$$\frac{{\kappa }^{\alpha }{\kappa }^{\nu (d-1)/2}{e}^{-\nu \kappa }}{{(\beta \kappa )}^{(d-1)/2}{e}^{-\beta \kappa }}\propto {\kappa }^{\alpha +(\nu -1)(d-1)/2}{e}^{-(\nu -\beta )\kappa }.$$

Thus, the integral diverges if $\nu <\beta$, and converges if $\nu >\beta$. If $\nu =\beta$, convergence requires $-1>\alpha +(\nu -1)(d-1)/2$, or equivalently, $\nu -1<-2(\alpha +1)/(d-1)$ as asserted. □

Proof of Theorem 1

Transforming to polar coordinates $\theta =\kappa \mu$, we obtain

$${\int }_{{\mathbb{R}}^d}{C}_d{(\parallel \theta \parallel )}^{\nu }{e}^{s\prime \theta }d\theta =a_d{\int }_0^{\infty }{C}_d{(\kappa )}^{\nu }\left({\int }_{{\mathbb{S}}^{d-1}}{e}^{\kappa s\prime \mu }\mathit{dU}(\mu )\right){\kappa }^{d-1}d\kappa =a_d{\int }_0^{\infty }{\kappa }^{d-1}{C}_d{(\kappa )}^{\nu }\frac{1}{C_d(\kappa \parallel s\parallel )}d\kappa =a_{d}J(d-1,\parallel s\parallel ,\nu ),$$

interchanging the order of integration being justified by nonnegativity of the integrand. The assertion now follows from Lemma 1. □

Proof of Theorem 2

If $d\eta ={\kappa }^{\alpha }d\kappa \mathit{dU}(\mu )$, we have

$${\int }_0^{\infty }{\int }_{{\mathbb{S}}^{d-1}}{C}_d{(\kappa )}^{\nu }{e}^{\kappa s\prime \mu }{\kappa }^{\alpha }d\kappa \mathit{dU}(\mu )={\int }_0^{\infty }{\kappa }^{\alpha }\frac{C_d{(\kappa )}^{\nu }}{C_d(\kappa \parallel s\parallel )}d\kappa =J(\alpha ,\parallel s\parallel ,\nu ),$$

whence the theorem follows by again using Lemma 1. □

We see that for the canonical parameterization, the hyperparameters giving proper distributions are the ones for which $\nu >0$ and the “prior mean” $s/\nu$ lies in the interior of (the convex hull of) the unit hypersphere ${\mathbb{S}}^{d-1}$. This is not a coincidence: in fact, one can alternatively establish Theorem 1 (and equivalently, Theorem 2 for $\alpha =d-1$) without explicit convergence computations using the general results of Diaconis and Ylvisaker (1979), see also Gutiérrez-Peña and Smith (1997, Theorem 3.1). Let $\mu$ be a probability measure on (the Borel sets of) ${\mathbb{R}}^d$ with bounded support $\mathcal{S}$ and consider the natural exponential family through $\mu$ with density $f(x|\theta )=e^{\theta \prime x-M(\theta )}$, where $e^{M(\theta )}=\int e^{\theta \prime x}d\mu (x)$, and the standard conjugate family with densities $\pi (\theta |s,\nu )=e^{s\prime \theta -\nu M(\theta )}$ (with respect to the Lebesgue measure). As $\mathcal{S}$ is bounded and $\mu$ is finite, $\Theta =\{\theta :M(\theta )<\infty \}={\mathbb{R}}^d$. Let $\mathcal{X}$ be the interior of the convex hull of $\mathcal{S}$. Then by Theorem 1 of Diaconis and Ylvisaker (1979), if $\mathcal{X}$ is nonempty (and hence “the observation set is genuinely d-dimensional” Diaconis and Ylvisaker, 1979, p. 271) $\pi (\theta |s,\nu )$ is proper if and only if $\nu >0$ and $s/\nu \in \mathcal{X}$. (The reference actually uses $\nu s$ where we use s.) In the vMF case, $\mathcal{S}={\mathbb{S}}^{d-1}$, with convex hull the closed unit ball, and interior the open unit ball $\mathcal{X}=\{x\in {\mathbb{R}}^d:\parallel x\parallel <1\}$. Hence, $\pi (\theta |s,\nu )$ is proper if and only if $\nu >0$ and $\parallel s/\nu \parallel <1$, or equivalently, $\parallel s\parallel <\nu$, again establishing Theorem 1.

We also note that for $\alpha \le d-1$ and $\parallel s\parallel <\nu$, Theorem 1 implies that

$$a_d{\int }_1^{\infty }{\int }_{{\mathbb{S}}^{d-1}}{C}_d{(\kappa )}^{\nu }{e}^{\kappa s\prime \mu }{\kappa }^{\alpha }d\kappa \mathit{dU}(\mu )\le a_d{\int }_1^{\infty }{\int }_{{\mathbb{S}}^{d-1}}{C}_d{(\kappa )}^{\nu }{e}^{\kappa s\prime \mu }{\kappa }^{d-1}d\kappa \mathit{dU}(\mu )={\int }_{\parallel \theta \parallel \ge 1}{C}_d{(\parallel \theta \parallel )}^{\nu }{e}^{s\prime \theta }d\theta <\infty ,$$

so that s and $\nu$ give a proper conjugate distribution for the polar coordinates ($\mu$, $\kappa$) parameterization with reference measure $d\eta ={\kappa }^{\alpha }d\kappa \mathit{dU}(\mu )$. However, neither results for the case $\alpha >d-1$ nor necessity of the condition $\parallel s\parallel <\nu$ can be established using the general framework (and in fact, Theorem 2 shows that the condition is not necessary if $d\ge 4$ and $0\le \alpha <(d-3)/2)$.

2.2. Propriety of posteriors from improper standard conjugate priors

Quite interestingly, canonical priors employed in the literature are improper if a vague prior is intended (see for example Nuñez-Antonio and Gutiérrez-Peña, 2005, who use $\nu =0$ and s=0). However, we note that if $\parallel s\parallel =\nu >0$ and $x_1,\dots ,x_n\in {\mathbb{S}}^{d-1}$, then $\parallel s+x_1+\cdots +x_n\parallel \le \parallel s\parallel +\parallel x_1\parallel +\cdots +\parallel x_n\parallel \le n+\nu$ with equality if and only if $x_1=\cdots =x_n=s/\nu$ which is a zero set for samples obtained from the vMF with fixed parameter $\theta$. Hence intuitively, we expect that improper standard conjugate priors with $\parallel s\parallel =\nu >0$ “almost always” yield proper posteriors. For the case $\parallel s\parallel =\nu =0$ (as in the examples) we have $\parallel x_1+\cdots +x_n\parallel \le \parallel x_1\parallel +\cdots +\parallel x_n\parallel \le n$ with equality if and only if $x_1=\cdots =x_n$ which is a zero set for samples obtained from the vMF with fixed parameter $\theta$ and $n\ge 2$.

This can be formalized as follows. Let $\pi$ be the density (with respect to $\eta$) of a $\sigma \mathrm{-}\mathrm{finite}$ measure on $\Lambda$ and define ${\mu }_{\pi }^{(n)}$ on ${\Xi }^{(n)}={\mathbb{S}}^{d-1}\times \cdots \times {\mathbb{S}}^{d-1}$ (n times), the space of all ${\mathbb{S}}^{d-1}$ valued samples of size n, via

$${\mu }_{\pi }^{(n)}(A)={\int }_{\Lambda }{\int }_{A}f(x_1|\theta (\lambda ))\cdots f(x_n|\theta (\lambda ))\mathit{dU}(x_1)\cdots \mathit{dU}(x_n)\pi (\lambda )d\eta (\lambda )={\int }_A{\int }_{\Lambda }f(x_1|\theta (\lambda ))\cdots f(x_n|\theta (\lambda ))\pi (\lambda )d\eta (\lambda )\mathit{dU}(x_1)\cdots \mathit{dU}(x_n).$$

Writing $g_{\pi }(x_1,\dots ,x_n)={\int }_{\Lambda }f(x_1|\theta (\lambda ))\cdots f(x_n|\theta (\lambda ))\pi (\lambda )d\eta (\lambda )$,

$${\mu }_{\pi }^{(n)}(A)={\int }_A{g}_{\pi }(x_1,\dots ,x_n)\mathit{dU}(x_1)\cdots \mathit{dU}(x_n),$$

i.e., ${\mu }_{\pi }^{(n)}$ is a generalized “mixture” of the distribution of i.i.d. samples of size n from the vMF family. Let

$$A_n(s,\nu )=\{(x_1,\dots ,x_n):\parallel s+x_1+\cdots +x_n\parallel \ge (n+\nu )\}.$$

Theorem 3

If $\parallel s\parallel =\nu >0$ ($s=\nu =0$), then ${\mu }_{\pi }^{(n)}(A_n(s,\nu ))=0$ for all $n\ge 1$ (respectively, $\ge 2$) and arbitrary $\pi$.

Proof of Theorem 3

Let $\nu >0$. From the above, $(x_1,\dots ,x_n)\in A_n(s,\nu )$ if and only if $x_1=\cdots =x_n=s/\nu$. Clearly, for i.i.d. random variables $(X_1,\dots ,X_n)$ from the vMF distribution with parameter $\theta$, $\mathbb{P}((X_1,\dots ,X_n)\in A_n(s,\nu )|\theta )={\int }_{A_n(s,\nu )}f(x_1|\theta )\cdots f(x_n|\theta )\mathit{dU}(x_1)\cdots \mathit{dU}(x_n)=0$ and hence, ${\mu }_{\pi }^{(n)}(A_n(s,\nu ))={\int }_{\Lambda }0·\pi (\lambda )d\eta (\lambda )=0$. As $A_n(0,0)$ consists of all $(x_1,\dots ,x_n)$ for which $x_1=\cdots =x_n$, the assertion for the second case follows along the lines of the first case. □

If we use the canonical parameterization and $\pi (\theta )=e^{\theta \prime s-\nu M(\theta )}$, then by the above,

$$g_{\pi }(x_1,\dots ,x_n)={\int }_{\Theta }{e}^{\theta \prime (x_1+\cdots +x_n)-\mathit{nM}(\theta )}{e}^{\theta \prime s-\nu M(\theta )}d\theta =a_{d}J(d-1,\parallel s+x_1+\cdots +x_n\parallel ,n+\nu )$$

is infinite if and only if $(x_1,\dots ,x_n)\in A_n(s,\nu )$. If $\parallel s\parallel =\nu >0$, this is a zero set under the product of uniforms on ${\Xi }^{(n)}$, and hence (again) ${\mu }_{\pi }^{(n)}(A_n(s,\nu ))={\int }_A{g}_{\pi }(x_1,\dots ,x_n)\mathit{dU}(x_1)\cdots \mathit{dU}(x_n)=0$. On the other hand, if $\parallel s\parallel >\nu$, then clearly ${\mu }_{\pi }^{(n)}(A_n(s,\nu ))=\infty$: in this sense, it is always possible to obtain improper posteriors when employing an improper standard conjugate prior with $\parallel s\parallel >\nu$ (we notice however that such priors are admittedly “strange”, as the corresponding prior sample means $s/\nu$ are outside the unit ball and hence “impossible”).

If X has a vMF distribution with parameter $\theta$, the theory of regular exponential models (cf., e.g., Mardia and Jupp, 1999, pp. 32–33) implies that $E_{\theta }(X)=\partial M(\theta )/\partial \theta \prime$ so that

$$E_{\theta }(X)=-\frac{d\log (C_d(\parallel \theta \parallel ))}{d\parallel \theta \parallel }\frac{\partial \parallel \theta \parallel }{\partial \theta \prime }=-\frac{C_d\prime (\parallel \theta \parallel )}{C_d(\parallel \theta \parallel )}\frac{\theta }{\parallel \theta \parallel }=\frac{A_d(\parallel \theta \parallel )}{\parallel \theta \parallel }\theta ,$$

where one can show that the logarithmic derivative A_d of $1/C_d$ satisfies (Schou, 1978)

$$A_d(\kappa )=\frac{I_{d/2}(\kappa )}{I_{d/2-1}(\kappa )},$$

that

$$A_d(\kappa )=\frac{1}{d}\kappa -\frac{1}{d^2(d+2)}{\kappa }^3+\mathrm{O}({\kappa }^5),$$

as $\kappa \to 0$ (so that $A_d(\kappa )/\kappa$ is in fact $C^{\infty }$ provided we take its value at zero to be $1/d$), and that

$$A_d(\kappa )=1-\frac{(d-1)}{2}\frac{1}{\kappa }+\frac{(d-1)(d-3)}{8}\frac{1}{{\kappa }^2}+\mathrm{O}({\kappa }^{-3})$$

as $\kappa \to \infty$. Hence, for i.i.d. random variables $X_1,\dots ,X_n$ from the vMF distribution with parameter $\theta$, ${\overline{X}}_n\to E_{\theta }(X)=A_d(\parallel \theta \parallel )\theta /\parallel \theta \parallel$ which is less than one in length, and hence

$$\frac{\parallel S_n\parallel }{{\nu }_n}=\frac{\parallel s+X_1+\cdots +X_n\parallel }{n+\nu }\to \parallel E_{\theta }(X)\parallel <1,$$

with probability one as $n\to \infty$. Thus, if we write

$$p_n(\theta |s,\nu )=P_{\theta }((X_1,\dots ,X_n)\in A_n(s,\nu )),$$

$p_n(\theta |s,\nu )\to 0$ as $n\to \infty$ for all $\theta$, and using continuity arguments one can easily see that this convergence is uniform on compact subsets of $\Theta ={\mathbb{R}}^d$. On the other hand, $\parallel E_{\theta }(X)\parallel \to 1$ for $\parallel \theta \parallel \to \infty$, so the convergence cannot be uniform over $\Theta$. It would be very interesting to find the rate at which ${\sup }_{\parallel \theta \parallel \le \kappa }{p}_n(\theta |s,\nu )$ tends to zero, which would then allow one to characterize the improper prior densities $\pi$ for which ${\mu }_{\pi }^{(n)}(A_n(s,\nu ))={\int }_{\Theta }{p}_n(\theta |s,\nu )\pi (\theta )d\theta \to 0$ as $n\to \infty$.

2.3. Jeffreys prior

A commonly suggested non-informative prior is the Jeffreys prior (Jeffreys, 1961), defined as the square root of the determinant of the Fisher information matrix relative to the parameterization employed. When using the canonical parameter, again by the theory of regular exponential models, $I(\theta )={\mathrm{var}}_{\theta }(X)={\partial }^{2}M(\theta )/\partial \theta \partial \theta \prime$ so that

$$I(\theta )=A{\prime }_d(\parallel \theta \parallel )\frac{\theta }{\parallel \theta \parallel }\frac{\theta \prime }{\parallel \theta \parallel }+A_d(\parallel \theta \parallel )\left(\frac{1}{\parallel \theta \parallel }{I}_d-\theta \frac{1}{\parallel \theta {\parallel }^2}\frac{\theta \prime }{\parallel \theta \parallel }\right)=A{\prime }_d(\parallel \theta \parallel )\frac{\theta }{\parallel \theta \parallel }\frac{\theta \prime }{\parallel \theta \parallel }+\frac{A_d(\parallel \theta \parallel )}{\parallel \theta \parallel }\left(I_d-\frac{\theta }{\parallel \theta \parallel }\frac{\theta \prime }{\parallel \theta \parallel }\right)=\frac{A_d(\parallel \theta \parallel )}{\parallel \theta \parallel }{I}_d+\left(A{\prime }_d(\parallel \theta \parallel )-\frac{A_d(\parallel \theta \parallel )}{\parallel \theta \parallel }\right)\frac{\theta }{\parallel \theta \parallel }\frac{\theta \prime }{\parallel \theta \parallel },$$

where I_d denotes the d-dimensional identity matrix. By a well known result from linear algebra, $\det (I_d+\beta \mathit{vv}\prime )=1+\beta v\prime v$ and thus $\det (\gamma I_d+\beta \mathit{vv}\prime )={\gamma }^d(1+\beta v\prime v/\gamma )$ and in particular if $\parallel v\parallel =1$, $\det (\gamma I_d+\beta \mathit{vv}\prime )={\gamma }^d(1+\beta /\gamma )={\gamma }^{d-1}(\beta +\gamma )$, so that

$$\det (I(\theta ))={\left(\frac{A_d(\parallel \theta \parallel )}{\parallel \theta \parallel }\right)}^{d-1}A{\prime }_d(\parallel \theta \parallel ),$$

generalizing the result obtained in Guttorp and Lockhart (1988) for the case d=2 (the sign in the reference is not correct).

Theorem 4

The Jeffreys prior $\pi (\theta )\propto \sqrt{\det (I(\theta ))}$ for the canonical parameter $\theta$ only depends on $\parallel \theta \parallel$ and behaves like $1/\parallel \theta {\parallel }^{(d+1)/2}$ for $\theta \to \infty$.

Proof of Theorem 4

The first assertion is immediate by observing that $\det (I(\theta ))$ depends on $\theta$ only via its length. To obtain the asymptotic behavior, we can use the asymptotics of $A_d(\kappa )$ and the fact that $A{\prime }_d(\kappa )=1-A_d(\kappa )(A_d(\kappa )+(d-1)/\kappa )$ (Schou, 1978). Thus, as $\kappa \to \infty$,

$$A{\prime }_d(\kappa )\approx 1-\left(1-\frac{d-1}{2\kappa }\right)\left(1-\frac{d-1}{2\kappa }+\frac{d-1}{\kappa }\right)=1-\left(1-\frac{d-1}{2\kappa }\right)\left(1+\frac{d-1}{2\kappa }\right)=\frac{{(d-1)}^2}{4{\kappa }^2},$$

such that for $\parallel \theta \parallel \to \infty$,

$$\det (I(\theta ))\approx {\left(\frac{1}{\parallel \theta \parallel }\right)}^{d-1}\frac{{(d-1)}^2}{4\parallel \theta {\parallel }^2}=\frac{\mathrm{const}}{\parallel \theta {\parallel }^{d+1}}.\square$$

We note that the Jeffreys prior “looks different” from the densities employed in the standard conjugate family ${\mathcal{C}}_{\theta }(\mathcal{F})$ relative to the canonical parameter. Following Gutiérrez-Peña and Smith (1997), one can rigorously establish that it is not contained in this family by verifying that the skewness vector $\partial \log (\det (I(\theta )))/\partial \theta \prime$ of the vMF family is not linear in its mean parameter, which is straightforward from the above expression for $\det (I(\theta ))$.

The Jeffreys prior with respect to the canonical parameterization is given by

$$\pi (\theta )\propto \sqrt{{\left(\frac{A_d(\parallel \theta \parallel )}{\parallel \theta \parallel }\right)}^{d-1}A{\prime }_d(\parallel \theta \parallel )}={\left(\frac{A_d(\parallel \theta \parallel )}{\parallel \theta \parallel }\right)}^{(d-1)/2}{\left(1-A_d(\parallel \theta \parallel )\left(A_d(\parallel \theta \parallel )+\frac{(d-1)}{\parallel \theta \parallel }\right)\right)}^{1/2}.$$

If the polar coordinates ($\mu$, $\kappa$) parameterization is employed two alternative parameterizations are possible for the mean direction parameter $\mu$ in order to obtain an unrestricted set of parameters. The $(\mu {\prime }_0,\kappa )$ parameterization with ${\mu }_0=({\mu }_1,\dots ,{\mu }_{d-1})\prime$ consists of the first $d-1$ dimensions of the mean direction parameter $\mu$ and the $(m\prime ,\kappa )$ parameterization uses the spherical polar coordinates for $\mu$ with $m=({\varphi }_1,\dots ,{\varphi }_{d-1})\prime$. If these parameterizations are used the Jeffreys prior derived for the canonical parameterization needs to be multiplied with the Jacobians which are given by

$${\kappa }^{d-1}/{\mu }_d,$$

for the $(\mu {\prime }_0,\kappa )$ parameterization and

$${\kappa }^{d-1}\prod _{j=1}^{d-2}\sin {({\varphi }_j)}^{d-1-j},$$

for the $(m\prime ,\kappa )$ parameterization. Note that the latter is also given in Mardia and El-Atoum (1976), which should have $\kappa -(d-1)A_d(\kappa )-\kappa A_d^2(\kappa )$ instead of $\kappa -A_d(\kappa )-\kappa A_d^2(\kappa )$ and needs $d-1-j$ instead of $d-2$ in the exponents of the sinuses.

Clearly, the Jeffreys prior is not proper. The following shows that “almost all” posteriors obtained from it (and in fact, from arbitrary possibly improper priors which increase at most polynomially in $\parallel \theta \parallel$) are proper for samples of size $n\ge 2$.

Theorem 5

Consider the canonical parameterization of the vMF family with the Lebesgue reference measure. Let $\pi (\theta )$ be $\mathrm{O}(\parallel \theta {\parallel }^{\gamma })$ for some finite $\gamma \ge 0$ as $\parallel \theta \parallel \to \infty$ and $B_n(\pi )=\{(x_1,\dots ,x_n):{\int }_{\Theta }f(x_1|\theta )\cdots f(x_n|\theta )\pi (\theta )d\theta <\infty \}$. Then for all $n\ge 2$, ${\mu }_{\pi }^{(n)}(B_n{(\pi )}^c)=0$.

Proof of Theorem 5

Writing $s=x_1+\cdots +x_n$, we have

$${\int }_{\parallel \theta \parallel \ge 1}f(x_1|\theta )\cdots f(x_n|\theta )\pi (\theta )d\theta ={\int }_{\parallel \theta \parallel \ge 1}{e}^{\theta \prime (x_1+\cdots +x_n)-\mathit{nM}(\theta )}\pi (\theta )d\theta \le \mathrm{const}{\int }_1^{\infty }{\int }_{{\mathbb{S}}^{d-1}}{e}^{\kappa s\prime \mu }{C}_d{(\kappa )}^n{\kappa }^{\gamma }{\kappa }^{d-1}\mathit{dU}(\mu )d\kappa =\mathrm{const}{\int }_1^{\infty }{\kappa }^{\gamma +d-1}\frac{C_d{(\kappa )}^n}{C_d(\kappa \parallel s\parallel )}d\kappa .$$

By Lemma 1, this is finite provided that $\parallel s\parallel <n$. Hence, $B_n{(\pi )}^c\subseteq A_n(0,0)$ which is a zero set under the product of uniforms on ${\Xi }^{(n)}$ provided that $n\ge 2$. □

2.4. Propriety of prior and posterior distributions in applications

Guttorp and Lockhart (1988) perform a full Bayesian analysis employing the canonical parameterization for 2-dimensional data. Rather than using the standard conjugate prior for $\theta$, they use a flat prior on $\mu$ and the conjugate prior they derived for $\kappa$ with $\mu$ known. Note that for the conjugate prior for $\kappa$ to be proper, the same conditions on $\parallel s\parallel$ and $\nu$ need to be satisfied as for the conjugate prior for $\theta$. In order to parameterize the $\kappa$ prior only the length of s and $\nu$ need to be specified. For their application, Guttorp and Lockhart (1988) use three different prior distributions for $\kappa$: a data-based, a high precision and a low precision prior. In all three cases the priors for $\kappa$ are proper because $\parallel s\parallel <\nu$. This is also clear by construction: the parameters of the priors are determined by specifying constraints for the moments of the prior distribution or quantities derived from the moments.

Damien and Walker (1999) employ the polar coordinates ($\mu$, $\kappa$) parameterization with the Lebesque measure as reference measure, i.e., $\alpha =0$. They use two different prior distributions in their two numerical examples. In the first example, they set all prior parameters to zero. This is the same prior Nuñez-Antonio and Gutiérrez-Peña (2005) use in their examples and refer to as vague prior. Given the updates for the posterior parameters as well as the interpretation of $\nu$ as the prior sample size, this seems to be an obvious choice. This prior is improper, but as shown in Section 2.2, posteriors will be proper almost surely for samples of size $n\ge 2$. In their second example, Damien and Walker (1999) use a flat prior for $\mu$ and a conjugate prior for $\kappa$ with $\mu$ known. Referring to the low precision prior in Guttorp and Lockhart (1988), $\parallel s\parallel =\nu =5$ are employed as parameters. The low precision prior in Guttorp and Lockhart (1988) actually is equal to $\parallel s\parallel =9.8824$ and $\nu =10$, while the values for the high precision prior are $\parallel s\parallel =4.99893$ and $\nu =5$. Interpreting $\nu$ as the prior sample size, larger values of $\nu$ imply more informative priors. However, the precision induced by the prior will depend on the average length given by $\parallel s\parallel /\nu$. The closer this value is to 1 the higher is the precision induced by the prior. Using an approximation for large $\kappa$, Guttorp and Lockhart (1988) derived that the prior mean and variance of the precision parameter $\kappa$ depend on $\nu$ as well as the difference $\nu -\parallel s\parallel$. By rounding $\parallel s\parallel$ to the same value as $\nu$, Damien and Walker (1999) use an improper prior and the interpretation as a low precision prior, as induced by the prior mean and standard deviation, obviously is lost. As shown in Section 2.2, the posterior from this prior is almost surely proper for samples of size $n\ge 1$.

Bangert et al. (2010) use conditionally conjugate priors for $\mu$ and $\kappa$. The prior for $\mu$ is a von Mises–Fisher distribution with precision parameter equal to 0.1 and mean parameter equal to the mean direction of the data. For $\kappa$ they use the conjugate prior for known $\mu$. Setting the prior parameters equal to $\nu =5$ and $\parallel s\parallel =4.7$, they employ a proper prior.

To sum up, these previous applications indicate that the non-informative but improper prior with $\parallel s\parallel =\nu =0$ seems to be an obvious choice if no prior information is available. This seems to be unproblematic because the posteriors will be almost surely proper for sample sizes $n\ge 2$ by Theorem 3. If prior information on the precision parameter $\kappa$ is to be included, a flat prior for $\mu$ is employed and the conjugate prior with known $\mu$ for $\kappa$. Another possibility, when using Gibbs sampling for estimation, is to employ conditionally conjugate priors. In general the parameters of the prior of $\kappa$ are chosen to reflect the prior information available for the moments of $\kappa$, which leads to proper priors.

3. Extensions

The vMF family on ${\mathbb{S}}^{d-1}$ can straightforwardly be generalized to the vMF family on the Stiefel manifold $V_k({\mathbb{R}}^d)$, the set of orthogonal k-frames in ${\mathbb{R}}^d$, or equivalently,

$$V_k({\mathbb{R}}^d)=\{X\in {\mathbb{R}}^{d\times k}:X\prime X=I_k\},$$

so that $V_1({\mathbb{R}}^d)$ corresponds to ${\mathbb{S}}^{d-1}$. The vMF family on $V_k({\mathbb{R}}^d)$ has densities

$$f(X|A)=e^{\mathrm{tr}(A\prime X)-M(A)},$$

with respect to the uniform distribution U on $V_k({\mathbb{R}}^d)$, where

$$e^{M(A)}={\int }_{V_k({\mathbb{R}}^d)}{e}^{\mathrm{tr}(A\prime X)}\mathit{dU}(X)={}_{0}F_1(;d/2;A\prime A/4)$$

is a generalized hypergeometric function with matrix argument (e.g., Mardia and Jupp, 1999, p. 289). This family of distributions is useful as a probability distribution over orthonormal matrices and for example Hoff (2009) indicates that it arises as a posterior distribution for the orthonormal matrices in factor analysis when uniform priors are used. For a further discussion of this family of distributions in relation to orientation statistics see Downs (1972) and Khatri and Mardia (1977).

Clearly, $X\mapsto \mathrm{vec}(X)$ defines a one-to-one correspondence between ${\mathbb{R}}^{d\times k}$ and ${\mathbb{R}}^{\mathit{dk}}$ with $\mathrm{tr}(X\prime A)=\mathrm{vec}(X)\prime \mathrm{vec}(A)$. The standard conjugate family for the vMF family on $V_k({\mathbb{R}}^d)$ (relative to the canonical parameter) is thus given by the family of densities

$$p(A|S,\nu )\propto e^{\mathrm{tr}(S\prime A)-\nu M(A)}.$$

Let $\parallel S{\parallel }_2$ denote the spectral norm (matrix 2-norm, the largest singular value) of S.

Theorem 6

The distributions in the standard conjugate family of the vMF family on the Stiefel manifold $V_k({\mathbb{R}}^d)$ are proper if and only if $\parallel S{\parallel }_2<\nu$.

Proof of Theorem 6

The support of U is $\mathcal{S}=V_k({\mathbb{R}}^d)$, the convex hull of which is the closed unit ball in the spectral norm (e.g., Journée et al., 2010 or Gallivan and Absil, 2010), and hence has non-empty interior

$$\mathcal{X}=\{X\in {\mathbb{R}}^{d\times k}:\parallel X{\parallel }_2<1\}=\{X\in {\mathbb{R}}^{d\times k}:X\prime X\prec I_k\}.$$

Using Theorem 1 of Diaconis and Ylvisaker (1979), the standard conjugate distributions are proper if and only if $\nu >0$ and $S/\nu \in \mathcal{X}$, or equivalently, if and only if $\parallel S{\parallel }_2<\nu$. □

The matrix vMF distributions are typically parameterized using the canonical parameter A. Alternatively, the analogue to the polar coordinates ($\mu$, $\kappa$) parameterization in the vector case is using the (right) polar decomposition of $A=\mathit{MK}$, where the polar part (or orientation) M is in the Stiefel manifold $V_k({\mathbb{R}}^d)$ and the elliptical part (or concentration) K is a symmetric, non-negative definite matrix (e.g., Mardia and Jupp, 1999, p. 286).

If A has full rank, K is the unique symmetric matrix root of $A\prime A$, and (e.g., Cadet, 1996, adjusting for the different normalizations employed) $\mathit{dA}=c(K)\mathit{dKdU}(M)$, where $c(K)=a_{d,k}\det {(K)}^{d-k}{\prod }_{i<j}({\lambda }_i+{\lambda }_j)$ with $a_{d,k}$ the (generalized) volume of $V_k({\mathbb{R}}^d)$ and ${\lambda }_1,\dots ,{\lambda }_k>0$ the eigenvalues of K. Hence,

$${\int }_{{\mathbb{R}}^{d\times k}}{e}^{\mathrm{tr}(S\prime A)-\nu M(A)}\mathit{dA}={\int }_{K\succcurlyeq 0}{\int }_{V_k({\mathbb{R}}^d)}{e}^{\mathrm{tr}(S\prime \mathit{MK})}{({}_{0}F_1(;d/2;K^2/4))}^{-\nu }c(K)\mathit{dK}\mathit{dU}(M)={\int }_{K\succcurlyeq 0}c(K){({}_{0}F_1(;d/2;K^2/4))}^{-\nu }{\int }_{V_k({\mathbb{R}}^d)}{e}^{\mathrm{tr}(\mathit{KS}\prime M)}\mathit{dU}(M)\mathit{dK}={\int }_{K\succcurlyeq 0}c(K)\frac{{}_{0}F_1(;d/2;\mathit{KS}\prime \mathit{SK}/4)}{{}_{0}F_1{(;d/2;K^2/4)}^{\nu }}\mathit{dK}.$$

Thus, if we consider the standard conjugate family of the matrix vMF family on the Stiefel manifold $V_k({\mathbb{R}}^d)$ relative to the polar coordinates parameterization $A=\mathit{MK}$ with elements $\pi (M,K|S,\nu )\propto e^{\mathrm{tr}(S\prime \mathit{MK})}/{}_{0}F_1{(;d/2;K^2/4)}^{\nu }$, and reference measures of the form $c{(K)}^{\alpha }\mathit{dK}\mathit{dU}(M)$, then as discussed in Section 2 for the polar parameterization of the vector vMF distribution, if $0\le \alpha \le 1$ Theorem 6 implies that distributions in this conjugate family are proper provided that $\parallel S{\parallel }_2<\nu$. Again, necessity of this condition for such values of $\alpha$, or the characterization of the hyperparameters giving proper distributions if $\alpha >1$ cannot be established.

For this, one needs to be able to characterize S and $\nu$ (and $\alpha$) for which

$$J_k(\alpha ,S,\nu )={\int }_{K\succcurlyeq 0}c{(K)}^{\alpha }\frac{{}_{0}F_1(;d/2;\mathit{KS}\prime \mathit{SK}/4)}{{}_{0}F_1{(;d/2;K^2/4)}^{\nu }}\mathit{dK}<\infty ,$$

which seems quite challenging, requiring suitable “large K” asymptotics for ${}_{0}F_1(;d/2;K^2/4)$ and ${}_{0}F_1(;d/2;\mathit{KS}\prime \mathit{SK}/4)$. We note that Butler and Wood (2003) give Laplace approximations for ${}_{0}F_1$ (and corresponding Bessel functions) of matrix arguments (but do not formally establish validity as an asymptotic approximation). Muirhead (1978, p. 22) gives an asymptotic approximation for ${}_{0}F_1(;d/2;A\prime A/4)$ for the case where all singular values of A are large. For the above, a generalization to the case where some singular values are large is needed. We leave this for future research.