Bayesian influence analysis: a geometric approach

Summary

In this paper we develop a general framework of Bayesian influence analysis for assessing various perturbation schemes to the data, the prior and the sampling distribution for a class of statistical models. We introduce a perturbation model to characterize these various perturbation schemes. We develop a geometric framework, called the Bayesian perturbation manifold, and use its associated geometric quantities including the metric tensor and geodesic to characterize the intrinsic structure of the perturbation model. We develop intrinsic influence measures and local influence measures based on the Bayesian perturbation manifold to quantify the effect of various perturbations to statistical models. Theoretical and numerical examples are examined to highlight the broad spectrum of applications of this local influence method in a formal Bayesian analysis.

1. Introduction

A formal Bayesian analysis of data z = (z₁, . . . , z_n) involves the specification of a sampling distribution p(z | θ) and a prior distribution p(θ), where θ = (θ₁, . . . ,θ_k)^T represents the parameters of inferential interest and varies in an open set Θ of R^k. To carry out Bayesian inference, we usually use Markov chain Monte Carlo methods to simulate samples from the posterior distribution p(θ | z), which is proportional to p(z | θ) p(θ). Subsequently, we can calculate posterior quantities of θ in R^k, such as the posterior mean M(h) = ∫ h(θ) p(θ | z)dθ of a function h(θ). For notational simplicity, we do not emphasize the dominating measure explicitly throughout the paper. There is a great deal of interest in the degree to which posterior inferences are sensitive to p(θ), p(z | θ) and (z₁, . . . , z_n) (Kass et al., 1989; McCulloch, 1989; Berger, 1990, 1994; Dey et al., 1996; Gustafson, 2000; Sivaganesan, 2000; Oakley & O’Hagan, 2004).

There are three major formal influence techniques, including case influence measures and global and local robustness approaches, for quantifying the degree of dependence of the posterior distribution on these three key elements of Bayesian analysis including the prior, the sampling distribution and the data (Berger, 1990, 1994). In Bayesian analysis, case influence measures primarily calculate the influence of a set of observations in order to identify outliers and influential observations. Most case influence measures are based on the posterior and/or predictive distribution through either case deletion or perturbation (Guttman & Peña, 1993; Peña & Guttman, 1993; Carlin & Polson, 1991; Peng & Dey, 1995). For instance, several case influence diagnostics have been developed to quantify the possible outlyingness of a set of observations based on mean-shift or variance-shift models (Guttman & Peña, 1993; Peña & Guttman, 1993).

The key idea of the global robustness approach is to compute a range of posterior quantities as the perturbation to each of the three key elements changes in a certain set of distributions, and then determine the extremal ones. There are drawbacks with this approach, including the scale chosen for the posterior quantities, the size of the perturbation and its limitation to linear functionals as well as simplicity of models. To address the scale issue, several scaled versions of the range have been proposed for the prior perturbation class (Ruggeri & Sivaganesan, 2000).

The local robustness approach primarily computes the derivatives of posterior quantities with respect to a minor perturbation to p(θ) and p(z | θ). In the frequentist literature, Cook’s (1986) influence approach is particularly useful for perturbing p(z | θ) in order to detect influential observations and assess model misspecification in parametric and semiparametric models (Zhu & Lee, 2001; Zhu et al., 2007). McCulloch (1989) further extends the local influence approach of Cook (1986) to assess the effects of perturbing the prior in a Bayesian analysis. In the Bayesian literature, several analogues of local influence have been developed using either the curvature of influence measures (Lavine, 1992; Dey & Birmiwal, 1994; Millar & Stewart, 2007; Van der Linde, 2007) or the Fréchet derivative of the posterior with respect to the prior (Berger, 1994; Gustafson & Wasserman, 1995; Dey et al., 1996; Gustafson, 1996; Berger et al., 2000). Very little has been done on developing general Bayesian influence analysis methods for simultaneously perturbing z, p(θ) and p(z | θ), assessing their effects and examining their applications in statistical models (Berger et al., 2000). To our knowledge, Clarke & Gustafson (1998) is one of the few papers on simultaneously perturbing {z, p(θ), p(z | θ)} in the context of independent and identically distributed data.

A key motivation for the proposed methodology is to unify influence concepts for many complex Bayesian models, for which very few or no methods exist, so that the effects of different perturbations can be identified. These models include many Bayesian parametric and semiparametric models, perhaps with missing data; see the Supplementary Material. Our development includes formal assessment of outliers and influential points as well as sensitivity analyses regarding the three major components of the Bayesian model: the prior, sampling distribution, and the data. For instance, sensitivity to the data can be evaluated by perturbing all the data points by random noise, redoing the analysis, and getting a spectrum of different inferences defined by noise (Wang et al., 2009; Clarke, 2010).

2. The Bayesian perturbation model and manifold

2.1. The Bayesian perturbation model

We develop a Bayesian model to characterize various perturbation schemes to z, p(z | θ) and p(θ). We introduce perturbations into the model p(z, θ) = p(z | θ) p(θ) through a vector ω = ω(z, θ), which varies in a set Ω. That is, ω is a mapping from the product space of the sample space 𝒵 and the parameter space Θ to Ω. Generally, ω includes many perturbation schemes including the additive ∊-contamination class to the prior as detailed below. Moreover, ω must be chosen carefully so that the perturbation is meaningful and sensible.

Let p(z, θ | ω) be the probability density of (z, θ) for the perturbed model. We assume that the probability measures of p(z, θ | ω) for all ω ∈ Ω have a common dominating measure and that there is an ω⁰ ∈ Ω such that p(z, θ | ω⁰) = p(z, θ) for all (z, θ). We refer to p(z, θ | ω⁰) = p(z, θ) as the baseline joint distribution, where ω⁰ can be regarded as the central point of Ω representing no perturbation. We define the Bayesian perturbation model ℳ as a family of probability densities p(z, θ | ω) as ω varies in Ω. The Bayesian perturbation model includes individual perturbation schemes to z, p(θ) and p(z | θ), and their combinations. We focus on each individual scheme as follows.

Example 1. The Bayesian perturbation model for the prior includes many existing schemes, such as the additive ∊-contamination class and the linear and nonlinear perturbation classes. For instance, the additive ∊-contamination scheme is given by p(θ | ω) = p(θ) + λ{g(θ) − p(θ)}, where λ ∈ [0, 1] and g(θ) belongs to a class of contaminating distributions, denoted by 𝒢 (Berger, 1994; Dey & Birmiwal, 1994). In this case, Ω = {ω = λ{g(θ) − p(θ)} : (λ, g(·)) ∈ [0, 1] × 𝒢} and ω (z, θ ) are independent of the data. Thus, ω⁰ = 0 and p(z, θ | ω) = p(z | θ) p(θ | ω).

Example 2. The Bayesian perturbation model for the data includes many perturbation schemes to individual data observations of z (Cook, 1986; Guttman & Peña, 1993; Peña & Guttman, 1993; Zhu et al., 2007). The perturbation scheme to data points is proposed for identifying outliers and influential observations. As an illustration, we consider the standard linear regression model $y_i=x_i^{\text{T}}\beta +{∊}_i$, where x_i is a p × 1 covariate vector, β is a p × 1 vector of regression coefficients and the ∊_i are independently and identically distributed N(0, σ²) random variables. Let c_l be an l × 1 vector with all elements equal to c for a fixed scalar c and an integer l, written as 1_n, 1_p and 0_m. A perturbation scheme to perturb the covariate x_i is given by x_i(ω_i) = x_i + ω_i1_p. In this case, $z_i={(y_i,x_i^{\text{T}})}^{\text{T}}$, θ = (β^T, σ²)^T, ω = (ω₁, …, ω_n)^T, ω⁰ = 0_n and Ω is a subset of Rⁿ. An alternative perturbation scheme to the linear regression model is the well-known mean shift model (Guttman & Peña, 1993; Peña & Guttman, 1993). It is assumed that $y_i=x_i^{\text{T}}\beta +{\omega }_i+{∊}_i$ for i in a set of k distinct integers chosen from the set {1, . . . , n}, denoted by I = {i₁, . . . , i_k}, and $y_i=x_i^{\text{T}}\beta +{∊}_i$ for all other i s. In this case, the perturbation scheme is ω = (ω_i1, . . . , ω_ik)^T and ω⁰ = 0_k. Another important scheme is a geometric mixture model for case deletion or case weights (Millar & Stewart, 2007; Van der Linde, 2007). Specifically, let q(z_i) be an arbitrary density of z_i independent of θ, then the geometric mixture model for perturbing the ith observation is given by p(z | θ, ω) = {Π_{j ≠ i} p(z_j | θ)} p(z_i | θ)^λ q(z_i)^1−λ/{∫ p(z_i | θ)^λ q(z_i)^1−λ dz_i}, where ω = λ varies in [0, 1] and p(z_i | θ) is the density of z_i under the linear model assumption. In this case, ω⁰ = 1 represents no perturbation. When λ = 0, p(z_i | θ) disappears in p(z | θ, 0), which is equivalent to deleting z_i.

Example 3. The Bayesian perturbation model for the sampling distribution includes many perturbation schemes to p(z | θ) such as the additive ∊-contamination class. We may also consider a class of perturbed sampling distributions p(z | θ, ω) defined by

$$p(z|\theta ,\omega )=p(z|\theta )\text{exp}\{\sum _{j=1}^m{\omega }_j{u}_j(z;\theta )-0.5\sum _{j=1}^m{\omega }_j^2{u}_j{\left(z;\theta \right)}^2-C(\theta ,\omega )\},$$ (1)

where C(θ, ω) is the normalizing constant, ω = (ω₁, . . . , ω_m)^T is an m × 1 vector and u_j (z; θ) is a fixed scalar function having zero mean under p(z | θ). In this case, ω⁰ = 0_m represents no perturbation. The number m in the perturbation (1) can either be as small as 1 or can increase with n (Copas & Eguchi, 2005; Zhu et al., 2007).

2.2. The Bayesian perturbation manifold

We develop a new geometric framework, called a Bayesian perturbation manifold, to measure each perturbation ω in the Bayesian perturbation model. Based on this manifold, we are able to measure the amount of perturbation, the extent to which each component of a perturbation model contributes to p(z, θ) and the degree of orthogonality for the components of the perturbation model. Such a quantification is useful for rigorously assessing the relative influence of each component in the Bayesian analysis, and can reveal any discrepancies among the data, the prior or the sampling model.

For an infinite dimensional set Ω, we assume throughout the paper that ℳ forms a Riemannian Hilbert manifold (Friedrich, 1991; Lang, 1995) under some regularity conditions. For a given p(z, θ | ω) ∈ ℳ, we consider a smooth curve C(t) = p{z, θ | ω(t)} through the space of perturbation models ℳ with open interval domains containing 0 and p{z, θ | ω (0)} = p(z, θ | ω). Note that ω may be different from ω⁰. We require C(t) to be smooth enough such that ℓ̇{z, θ | ω (t)} = d log p{z, θ | ω(t)}/dt, called the tangent or derivative vector, exists with ∫ ℓ̇{z, θ | ω (t)}² p{z, θ | ω(t)}dzdθ < ∞ for all t in the open interval domain. Since p{z, θ | ω (t)} is the joint density of (z, θ) given ω(t), that is ∫ p{z, θ | ω(t)}dzdθ = 1, the tangent space of ℳ at ω, denoted by T_ωℳ, is formed by the tangent vectors ℓ̇{z, θ | ω(0)} for all possible smooth curves C(t) such that ∫ ℓ̇{z, θ | ω(0)} p{z, θ | ω(0)}dzdθ = 0. We can introduce the inner product of any two tangent vectors ν₁(ω) and ν₂(ω) in T_ωℳ as

$$<v_1,v_2>(\omega )=\int \{v_1(\omega )v_2(\omega )\}p(z,\theta |\omega )dzd\theta .$$ (2)

When ω varies in a Euclidean space and is independent of z and θ, the inner product < ν₁, ν₂ > (ω) in (2) is closely associated with the Fisher information. See Example 6 for details. Thus, the squared length ||ν(ω)||² of a tangent vector ν(ω) ∈ T_ωℳ is < ν, ν > (ω) = ∫ ν(ω)² p(z, θ | ω)dzdθ. The length of the curve C(t) from t₁ to t₂ is

$$S_C\{\omega (t_1),\omega (t_2)\}={\int }_{t_1}^{t_2}{\left[<\dot{\ell }\{z,\theta |\omega (t)\},\dot{\ell }\{z,\theta |\omega (t)\}>\{\omega (t)\}\right]}^{1/2}dt.$$ (3)

Next, we need to introduce the concept of a geodesic, which is a direct extension of the straight line in Euclidean space, on ℳ. Consider a real function f (ω) defined on ℳ and a smooth curve p{z, θ | ω(t)} in ℳ with p{z, θ | ω(0)} = p(z, θ | ω) and ℓ̇{z, θ | ω(0)} = ν(ω). We define df[ν](ω) = lim_t→0 t⁻¹(f[p{z, θ | ω(t)}] − f[p{z, θ | ω(0)}]) as the directional derivative of f at the perturbation distribution p(z, θ | ω) in the direction of ν(ω) ∈ T_ωℳ. We consider two smooth vector fields u(ω) and v(ω), which are not only the tangent vectors in T_ωℳ, but also smooth functions of ω in Ω. We define the directional derivative of a vector field u(ω) in the direction of v(ω), called the connection, which is given by du[v](ω) = lim_t→0 t⁻¹[u{ω(t)} − u{ω(0)}]. Intuitively, if ω varies in a Euclidean space, then du[v](ω) is closely associated with the second derivative of ℓ(z, θ | ω) with respect to ω. We consider the Levi–Civita connection, which has several nice geometric properties (Amari, 1990; Lang, 1995) and is given by

$${\nabla }_{v}u(\omega )=du[v](\omega )-0.5\{u(\omega )v(\omega )p(z,\theta |\omega )-\int u(\omega )v(\omega )p(z,\theta |\omega )dzd\theta \}.$$

A geodesic with respect to the Levi–Civita connection on ℳ is a smooth curve γ(t) = p{z, θ | ω(t)} on ℳ with open interval domain (a, b) and ℓ̇{z, θ | ω(t)} = v{ω(t)} such that the Levi–Civita connection ▿_vv{ω(t)} = 0. Intuitively speaking, as one moves tangent vectors of a geodesic along the same geodesic, one can keep them pointing in the same direction. Moreover, geodesics can be interpreted as the shortest local path between points on ℳ. For a fixed perturbation distribution p(z, θ | ω) and a given direction of v(ω) ∈ T_ωℳ, there is a unique geodesic γ(t) = p{z, θ | ω(t)} with open interval domains covering 0 such that γ(0) = p(z, θ | ω) and γ̇(0) = v(ω). Finally, based on these geometric quantities of ℳ, we introduce the definition of a Bayesian perturbation manifold.

Definition 1. A Bayesian perturbation manifold (ℳ, < u, v >, ▿_v u) is the manifold ℳ with an inner product < u, v > and the Levi–Civita connection ▿_v u.

When Ω is an open set of R^m, under some regularity conditions, the Bayesian perturbation manifold is an m-dimensional manifold (Amari, 1990, p. 16; Kass & Vos, 1997; Zhu et al., 2007). Now, we examine some examples of Bayesian perturbation manifolds based on several perturbations to the data, the prior, and the sampling distribution.

Example 1, continued. We consider the Bayesian perturbation model for the ∊-contamination class to the prior given by ℳ= {{(1 − λ) p(θ) + λg(θ)} p(z | θ) : λ ∈ [0, 1], g(·) ∈ 𝒢}. In this case, ω(t) = t{g(θ) − p(θ)} for a given g(·) ∈ 𝒢, and therefore we consider the smooth curve C_g(t) = p{z, θ | ω(t)} = [p(θ) + t{g(θ) − p(θ)}] p(z | θ). It can be shown that v_g{ω(t)} = ℓ̇{z, θ | ω(t)} = {g(θ) − p(θ)}/[p(θ) + t{g(θ) − p(θ)}]. For any two densities g₁(·) and g₂(·) in 𝒢, we can calculate the tangent vectors v_gi {ω(0)} = {g_i(θ) − p(θ)}{p(θ)}⁻¹ for i = 1, 2 and their inner product as

$$<v_{g_1},v_{g_2}>({\omega }^0)=\int \left[g_1(\theta ){\{p(\theta )\}}^{-1}-1\right]\left[g_2(\theta ){\{p(\theta )\}}^{-1}-1\right]p(\theta )d\theta ,$$

which is also independent of p(z | θ). In particular, < v_g, v_g > (ω⁰) = ∫{g(θ)/p(θ) − 1}² p(θ)dθ reduces to the L² norm considered in Gustafson (1996).

We further consider a Bayesian perturbation model for the sole perturbation scheme to hyper-parameters of the prior given by ℳ= {p(z, θ | ω) = p(θ | ω) p(z | θ) : ω = (ω₁, . . . ,ω_m)^T}, in which ω is independent of both z and θ. Let ω(t) = (ω₁, . . . , ω_j−1, ω_j + t, ω_j+1, . . . , ω_m)^T, ℓ(θ | ω) = log p(θ | ω) and ω_k(t) be the kth component of ω(t). Since ℓ(z, θ | ω) = log p(θ | ω) + log p(z | θ), we have

$$\dot{\ell }\{z,\theta |\omega (0)\}=d\ell \{z,\theta |\omega (t)\}/dt{|}_{t=0}=\sum _{k=1}^m\left[{\dot{\omega }}_k(t){\partial }_{{\omega }_k}\ell \{\theta |\omega (t)\}\right]{|}_{t=0}={\partial }_{{\omega }_j}\ell (\theta |\omega ),$$

where ω̇_k(t) = dω_k(t)/dt and ∂_ωj = ∂ / ∂ω_j. Therefore, T_ωℳ is spanned by the m functions ∂_ωjℓ(θ | ω) pointwise in ω. Since ∫ p(z | θ)dz = 1, the inner product between ∂_ωjℓ(θ | ω) and ∂_ωkℓ(θ | ω), denoted by G_jk(ω), is given by

$$\begin{array}{ll}{G}_{jk}(\omega )\hfill & =\int {\partial }_{{\omega }_j}\ell (\theta |\omega ){\partial }_{{\omega }_k}\ell (\theta |\omega )p(\theta |\omega )p(z|\theta )dzd\theta \hfill \\ \hfill & =\int {\partial }_{{\omega }_j}\ell (\theta |\omega ){\partial }_{{\omega }_k}\ell (\theta |\omega )p(\theta |\omega )d\theta ,\hfill \end{array}$$ (4)

which is independent of p(z | θ).

Furthermore, suppose that p(θ) = p(θ₁) p(θ₂ | θ_[1]) . . . p(θ_m | θ_[m−1]) has a hierarchical structure, where θ_[j] = (θ₁, . . . ,θ_j) and p(θ_j | θ_[j−1]) denote the density of the conditional distribution of θ_j given θ_[j−1]. Then, we perturb each level of p(θ) such that p(θ | ω) = p(θ₁ | ω₁) p(θ₂ | θ_[1], ω₂) . . . p(θ_m | θ_[m−1], ω_m), ∫ p(θ₁ | ω₁)dθ₁ = 1 and ∫ p(θ_j | θ_[j−1], ω_j)dθ_j = 1 for j = 2, . . . , m. In this case, T_ωℳ is spanned by the m functions ∂_ω1 log p(θ₁ | ω₁) and ∂_ωj log p(θ_j | θ_[j−1],ω_j) for j = 2, . . . , m. Moreover, G_jk(ω) = 0 for all j ≠ k. For instance, it can be shown that G₁₂(ω) = ∫ ∂_ω1 log p(θ₁ | ω₁) ∂_ω2 log p(θ₂ | θ_[1],ω₂) p(θ | ω)dθ = ∂_ω1 ∂_ω2 ∫ p(θ₁ | ω₁) p(θ₂ | θ₁, ω₂)dθ₂dθ₁ = ∂_ω1 ∂_ω21 = 0. Thus, different components of ω are orthogonal to each other (Zhu et al., 2007). Furthermore, it follows from (4) that G₁₁(ω) = ∫ {∂_ω1 log p(θ₁ | ω₁)}² p(θ₁ | ω)dθ₁ and G_jj(ω) = ∫ {∂_ωj log p(θ_j | θ_[j−1], ω_j)}² p(θ_j | θ_[j−1] ,ω)dθ_j for j ⩾ 2.

Combining the above results, we are led to the following proposition, whose proof can be found in the Supplementary Material.

Proposition 1. Consider any Bayesian perturbation model to the prior given by ℳ= {p(θ | ω) p(z | θ) : ω ∈ Ω}. If ω is independent of z, then the metric tensor of its Bayesian perturbation manifold ℳ is independent of the specification of the sampling distribution p(z | θ).

Proposition 1 has important implications. The independence property ensures that existing results on local robustness to the prior can be considered as a special case of the new method developed here (McCulloch, 1989; Gustafson, 1996).

Example 4. Consider a Bayesian perturbation model given by

$$\begin{array}{l}𝒨=\{p(z,\theta |\omega )=p(\theta |{\omega }_p)p(z|\theta ,{\omega }_s):\omega ={({\omega }_p^{\text{T}},{\omega }_s^{\text{T}})}^{\text{T}},\int p(\theta |{\omega }_p)d\theta \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\int p(z;\theta ,{\omega }_s)dz=1\},\end{array}$$

in which ω_p = (ω₁, . . . ,ω_m)^T and ω_s = (ω_m+1, . . . ,ω_m+n)^T are assumed to be independent of both z and θ. We consider ω(t) = (ω₁, . . . , ω_j−1, ω_j + t, ω_j+1, . . . ,ω_m+n)^T with ω(0) = ω for each j ∈ {1, . . . , m + n}. Thus, ω̇_k(0) = dω_k(0)/dt = 1 for k = j and 0 otherwise. Letting ℓ(θ | ω_p) = log p(θ | ω_p) and ℓ(z | θ, ω_s) = log p(z | θ, ω_s), we have

$$\dot{\ell }\{z,\theta |\omega (0)\}=\sum _{k=1}^{m+n}{\dot{\omega }}_k(0){\partial }_{{\omega }_k}\text{log}p(z,\theta |\omega )={\partial }_{{\omega }_j}\ell (\theta |{\omega }_p)+{\partial }_{{\omega }_j}\ell (z|\theta ,{\omega }_s).$$ (5)

Since ω_s and ω_p have no components in common, T_ωℳ is spanned by m + n functions including ∂_ωjℓ(θ | ω_p) for j = 1, . . . , m and ∂_ωjℓ(z | θ, ω_s) for j = m + 1, . . . , m + n. Note that ∫ ∂_ωkℓ(θ | ω_p) ∂_ωjℓ(z | θ, ω_s) p(z, θ | ω)dzdθ = ∫ ∂_ωkp(θ | ω_p) ∂_ωjp(z | θ, ω_s) dzdθ = ∂_ωk1∂_ωj1= 0 holds for any j, k. Therefore, it follows from (5) that the inner product of ∂_ωjℓ(z, θ | ω) and ∂_ωkℓ(z, θ | ω), denoted by G_jk(ω), is

$$\int {\partial }_{{\omega }_j}\ell (\theta |{\omega }_p){\partial }_{{\omega }_k}\ell (\theta |{\omega }_p)p(z,\theta |\omega \}dzd\theta +\int {\partial }_{{\omega }_j}\ell (z|\theta ,{\omega }_s){\partial }_{{\omega }_k}\ell (z|\theta ,{\omega }_s)p(z,\theta |\omega \}dzd\theta .$$ (6)

Moreover, the first term of (6) can be simplified to ∫ ∂_ωjℓ(θ | ω_p) ∂_ωkℓ(θ | ω_p)p(θ | ω_p)dθ since ∫ p(z | θ, ω_s)dz = 1. For j = 1, . . . , m and k = m + 1, . . . , m + n, it follows from (6) that <∂_ωjℓ(z, θ | ω) ∂_ωkℓ(z, θ | ω) >= 0 since ∂_ωkℓ(θ | ω_p) = 0 and ∂_ωjℓ(z | θ, ω_s) = 0. Thus, ω_s and ω_p are orthogonal to each other with respect to < ∂_ωjℓ(z, θ | ω), ∂_ωkℓ(z, θ | ω) >.

Combining the above results, we obtain the following proposition.

Proposition 2. Consider ℳ= {p(z, θ | ω) = p(θ | ω_p)p(z | θ, ω_s): $\omega ={({\omega }_p^{\text{T}},{\omega }_s^{\text{T}})}^{\text{T}}$}. Assume that ω_p is independent of z and ∫ p(θ | ω_p)dθ = ∫ p(z | θ, ω_s)dz = 1. Consider two smooth curves p{z, θ | ω_(k)(t)} with ω_(k)(t) = {ω_(k),p(t), ω_(k),s (t)}^T such that ω₍₁₎ (0) = ω₍₂₎ (0) = ω and ω_(1),p(t) and ω_(2),s(t) are independent of t. For any two tangent vectors v_k(ω) = ℓ̇{z, θ | ω_(k)(0)} ∈ T_ωℳ for k = 1, 2, we have < v₁, v₂ > (ω) = 0.

Proposition 2 has important implications. For simultaneous perturbations to the prior and the sampling distribution, it ensures that ω_p and ω_s are geometrically orthogonal to each other. Thus, we can separate out the influence of the prior from that of the data and the sampling distribution.

Finally, we consider a simultaneous perturbation model, denoted by p(z, θ | ω_p, ω_d, ω_s), in which ω_p, ω_d and ω_s represent individual perturbations to the prior, the data and the sampling distribution, respectively. In addition to Propositions 1 and 2, we can obtain the following theorem.

Theorem 1. Let ℳ= {p(z, θ | ω) = p(θ | ω_p) p(z | θ, ω_d, ω_s) : ω = (ω_p, ω_d, ω_s)} with ∫ p(θ | ω_p)dθ = ∫ p(z | θ, ω_d, ω_s)dz = 1 and that ω_p is independent of z. Consider two smooth curves p{z, θ | ω_(k)(t)} with ω_(k)(t) = {ω_(k),p(t), ω_(k),d (t),ω_(k),s (t)}^T passing through ω₍₁₎(0) = ω₍₂₎(0) = ω and having two tangent vectors v_k(ω) = ℓ̇{z, θ | ω_(k)(0)} ∈ T_ωℳ, k = 1, 2. Then:

1. if ω_(1),p(t) and {ω_(2),d(t), ω_(2),s(t)} are independent of t, then < v₁, v₂ > (ω) = 0;

2. if {ω_(1),p(t), ω_(1),d(t)} and {ω_(2),p(t), ω_(2),s(t)} are independent of t and p(z | θ, ω_d, ω_s) = p₁(z | θ, ω_d) p₂(z | θ, ω_s) for any (ω_d, ω_s), then < v₁, v₂ > (ω) = 0.

For simultaneous perturbations to the prior, the data, and the sampling distribution, Theorem 1 (i) ensures that ω_p and (ω_d, ω_s) are geometrically orthogonal to each other. If p(z | θ, ω_d, ω_s) = p₁(z | θ, ω_d) p₂(z | θ, ω_s), then ω_p, ω_d, and ω_s are geometrically orthogonal to each other.

3. Influence measures and their properties

3.1. Intrinsic influence measures

We consider some objective functions, such as the ϕ-divergence function, the posterior mean, and the Bayes factor, and develop associated intrinsic influence measures for quantifying the effects of perturbing the three key elements of a Bayesian analysis. An objective function of interest for sensitivity analysis is often chosen to be a functional of the perturbed posterior distribution of θ given z, given by p(θ | z, ω) = p(z, θ | ω)/ ∫ p(z, θ | ω)dθ and p(θ | z, ω⁰), which is the unperturbed posterior distribution of θ given z. Such an objective function, denoted by f (ω, ω⁰) = f {p(θ | z, ω), p(θ | z, ω⁰)}, can be also regarded as a mapping from ℳ × ℳ to R. Throughout the paper, we assume that f (ω, ω⁰) is a smooth function of ω and is a path-independent function of p(θ | z, ω) and p(θ | z, ω⁰) such that f (ω, ω) = 0 for any ω ∈ Ω. For instance, f (ω, ω⁰) can be set as the total variation distance of p(θ | z, ω⁰) and p(θ | z, ω) (Dey et al., 1996). Most standard influence measures such as the range (Berger, 1990, 1994) can be regarded as special cases of f (ω, ω⁰).

A large value of these influence measures can be caused by both the perturbation ω to the baseline distribution regardless of the observed data and the discrepancies between the observed data and the fitted model p(z, θ ). Since the purpose of any influence analysis is to detect the discrepancies between the observed data and p(z, θ), we suggest rescaling f (ω, ω⁰) by using the shortest distance between p(z, θ | ω) and p(z, θ | ω⁰). We explicitly quantify the distance between p(z, θ | ω) and p(z, θ | ω⁰) by using their minimal geodesic distance, denoted by d(ω, ω⁰). If ℳ is a complete and finite-dimensional Riemannian manifold, then the Hopf–Rinow theorem states that any two points on ℳ can be joined by a minimal geodesic (Ekeland, 1978). Furthermore, if ℳ is a complete infinite-dimensional Riemannian manifold, any two points on ℳ can be joined by a path which is almost a minimal geodesic (Ekeland, 1978). We introduce an intrinsic influence measure for comparing ω and ω⁰ ∈ Ω as follows. Geometrically, an intrinsic measure is invariant to certain reparameterizations.

Definition 2. The intrinsic influence measure for comparing p(θ | z, ω) to p(θ | z, ω⁰) is defined as IGI_f (ω, ω⁰) = f(ω, ω⁰)²/d(ω, ω⁰)².

The proposed IGI_f (ω, ω⁰) can be interpreted as the ratio of the change of the objective function relative to the minimal distance between p(z, θ | ω) and p(z, θ | ω⁰) on ℳ. Since f(ω, ω⁰) is path-independent and d(ω, ω⁰) is invariant to smooth reparametrization of ω, IGI_f (ω, ω⁰) is also invariant. Moreover, we suggest identifying the most influential ω in Ω, denoted by ω̂_I, which maximizes IGI_f (ω, ω⁰) for all ω ∈ Ω.

Example 5. We consider the logarithm BF(ω, ω⁰) = log ∫ p(z, θ | ω)dθ − log ∫ p(z, θ | ω⁰)dθ of the Bayes factor for comparing p(z | θ, ω) and p(z | θ, ω⁰), which can be regarded as a statistic for testing hypotheses of ω against ω⁰ (Kass & Raftery, 1995). Under mild conditions, BF(ω, ω⁰) is a smooth mapping from ℳ to R. We can set f (ω, ω⁰) = BF(ω, ω⁰) and obtain the intrinsic influence measure

$${\text{IGI}}_{BF}(\omega ,{\omega }^0)=\frac{\text{BF}{(\omega ,{\omega }^0)}^2}{d{(\omega ,{\omega }^0)}^2}.$$

3.2. First-order local influence measures

We consider the local behaviour of f {ω(t), ω⁰} as t approaches 0 along all possible smooth curves p{z, θ | ω(t)} passing through ω⁰, that is ω(0) = ω⁰. Since f {ω(t), ω⁰} is a function from R to R, it follows by Taylor’s series expansion that f {ω(t), ω⁰} = f {ω(0), ω⁰} + ḟ{ω(0)}t + 0.5 f̈{ω(0)}t² + o(t²), where ḟ{ω(0)} and f̈{ω(0)} denote the first- and second order derivatives of f{ω(t), ω⁰} with respect to t evaluated at t = 0. We need to distinguish between ḟ{ω(0)} ≠ 0 for some smooth curves ω(t) and ḟ{ω(0)} = 0 for all smooth curves ω(t). We first consider the case ḟ{ω(0)} ≠ 0 for some smooth curves ω(t). Let ℓ̇{z, θ | ω(0)} = v ∈ T_ω(0)ℳ. Then, ḟ{ω(0)} = df [v]{ω(0)} is the directional derivative of f in the direction of v ∈ T_ω(0)ℳ (Lang, 1995). We are led to the following definition.

Definition 3. The first-order local influence measure is defined as FI_f[v]{ω(0)} = lim_t→0 IGI_f {ω(0), ω(t)} = [df [v]{ω(0)}]²/[< v, v > {ω(0)}].

To carry out a sensitivity analysis, we use the tangent vector v_F,max in T_ω(0) ℳ, which maximizes FI_f [v]{ω(0)} and is invariant to reparameterization of ω(t). We now have the following result.

Theorem 2. The quantity FI_f [v]{ω(0)} is invariant to smooth reparameterization of ω(t).

In addition to the invariance property in Theorem 2, FI_f [v]{ω(0)} is a direct generalization of the first-order measure for a finite-dimensional perturbation manifold (Zhu et al., 2007; Wu & Luo, 1993).

Example 5 (continued). We set f {ω(t), ω⁰} = BF{ω(t), ω⁰}. Since d[BF{ω(t), ω⁰}]/dt = ∫ ℓ̇{z, θ | ω(0)}[p{z, θ | ω(0)}/ ∫ p{z, θ | ω(0)}dθ]dθ = ∫ ℓ̇{z, θ | ω(0)} p{θ | z, ω(0)}dθ, we have

$${\text{FI}}_f[v]\{\omega (0)\}=\frac{{\left[\int \dot{\ell }\{z,\theta |\omega (0)\}p\{\theta |z,\omega (0)\}d\theta \right]}^2}{\int \dot{\ell }{\{z,\theta |\omega (0)\}}^{2}p\{z,\theta |\omega (0)\}dzd\theta }.$$

It is relatively easy to compute FI_f [v]{ω(0)} for a specific perturbation. For instance, for the contamination to the prior given by p{θ | ω(t)} = p(θ) + t{g(θ) − p(θ)}, it can be shown that

$${\text{FI}}_f[v]\{\omega (0)\}=\frac{{\left(\int \left[g(\theta ){\{p(\theta )\}}^{-1}-1\right]p\{\theta |z,\omega (0)\}d\theta \right)}^2}{\int {\left[g(\theta ){\{p(\theta )\}}^{-1}-1\right]}^{2}p(\theta )d\theta }=\frac{{\left[p_g(z){\{p(z)\}}^{-1}-1\right]}^2}{\int {\left[g(\theta ){\{p(\theta )\}}^{-1}-1\right]}^{2}p(\theta )d\theta },$$

where p(z) = ∫ p(z | θ) p(θ)dθ and p_g(z) = ∫ g(θ) p{z | θ, ω(0)}dθ. Since the ratio of p_g(z) to p(z) is the Bayes factor in favour of g(θ) versus p(θ), FI_f[v]{ω(0)} is the square of the normalized Bayes factor of g(θ) versus p(θ).

Example 6. Consider the Bayesian perturbation manifold ℳ = {p(z, θ | ω) : ω ∈ Ω ⊂ R^m} and p{z, θ | ω(t)} as a smooth curve on ℳ, in which ω is not a function of z and θ, such as the perturbation scheme in the mean-shift model, and ω(t) = {ω₁(t), . . . , ω_m(t)}^T is a smooth vector of t. Let v_h = (v_h,1, . . . , v_h,m)^T = dω(0)/dt. By using the chain rule, we have

$$\begin{array}{l}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}v\{\omega (0)\}=d\ell \{z,\theta |\omega (t)\}/dt{|}_{t=0}=\sum _{k=1}^m{\dot{\omega }}_k(t){\partial }_{{\omega }_k}\ell \{z,\theta |\omega (0)\}=\sum _{k=1}^m{v}_{h,k}{\partial }_{{\omega }_k}\ell \{z,\theta |\omega (0)\},\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}df[v]\{\omega (0)\}=df\{\omega (t),{\omega }^0\}/dt{|}_{t=0}=\sum _{k=1}^m{v}_{h,k}{\partial }_{{\omega }_k}f\{\omega (0)\}=v_h^{\text{T}}{\partial }_{\omega }f\{\omega (0)\},\\ <v,v>\{\omega (0)\}=\sum _{j,k=1}^m{v}_{h,j}{v}_{h,k}<{\partial }_{{\omega }_j}\ell \{z,\theta |\omega (0)\}{\partial }_{{\omega }_k}\ell \{z,\theta |\omega (0)\}>\{\omega (0)\}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=v_h^{\text{T}}G\{\omega (0)\}{v}_h,\end{array}$$ (7)

where ∂_ωkf(ω) denotes the first-order partial derivative of f (ω, ω⁰) with respect to ω_k and G{ω(0)} = ∫[∂_ωℓ{z, θ | ω(0)}]^⊗2 p(z, θ | ω)dzdθ is an m × m Fisher information matrix with respect to ω. Thus, it follows from (7) and the definition of FI_f [v]{ω(0)} that ${\text{FI}}_f[v]\{\omega (0)\}={\left[df[v]\{\omega (0)\}\right]}^2/[<v,v>\{\omega (0)\}]={\left[v_h^{\text{T}}{\partial }_{\omega }f\{\omega (0)\}\right]}^2/v_h^{\text{T}}G\{\omega (0)\}{v}_h$. Finally, we obtain v_F,max{ω(0)} = argmax_vFI_f [v]{ω(0)} = [G{ω(0)}]^−1/2 ∂_ωf{ω(0)}.

3.3. Second-order local influence measures

We use f̈{ω(0)} to assess the second-order local influence of ω to a statistical model (Zhu et al., 2007). However, for a general smooth curve ω(t) on ℳ, f̈{ω(0)} is not geometrically well behaved (Lang, 1995; Zhu et al., 2007). We consider only the geodesic p{z, θ | ω(t)}, denoted by Exp_ω(0) (tv), passing through Exp_ω(0)(tv) |_t=0= ω(0) with initial direction ℓ̇{z, θ | ω(0)} = v{ω(0)} ∈ T_ω(0)ℳ. It follows from a Taylor’s series expansion (Lang, 1995; Zhu et al., 2007) that

$$f\{{\text{Exp}}_{\omega (0)}(tv),{\omega }^0\}=f\{\omega (0),{\omega }^0\}+tdf[v]\{\omega (0)\}+0.5t^2\ddot{f}\{{\text{Exp}}_{\omega (0)}(tv)\}{|}_{t=0}+o{\left(t\right)}^2,$$ (8)

where f̈{Exp_ω(0)(tv)} = d²f{Exp_ω(0)(tv), ω⁰}/dt². Geometrically, f̈{Exp_ω(0)(tv)} |_t=0 in (8) is called the Riemannian Hessian and is denoted by Hess(f)(v, v){ω(0)} (Lang, 1995). The Riemannian Hessian is symmetric. We now introduce a second-order influence measure.

Definition 4. The second-order influence measure in the direction v ∈ T_ω(0)ℳ is defined as SI_f [v]{ω(0)} = Hess(f)(v, v){ω(0)}/[<v, v> {ω(0)}].

Geometrically, SI_f [v]{ω(0)} is invariant to scalar transformations and smooth transformations. To carry out a sensitivity analysis, we use the tangent vector v_S,max ∈ T_ω(0)ℳ, which maximizes SI_f [v]{ω(0)} for all v ∈ T_ω(0)ℳ. There is a direct connection between the second-order measures in finite- and infinite-dimensional spaces. Therefore, the diagnostic method proposed here can be regarded as an extension of existing local influence approaches (Cook, 1986; Zhu et al., 2007) to an infinite dimensional setting.

Example 6, continued. We consider the Bayesian perturbation model in Example 6. If df[v]{ω(0)} = 0 for all v ∈ T_ω(0)ℳ, then Hess(f)(v, v){ω(0)} reduces to $v_h^{\text{T}}{H}_f\{\omega (0)\}{v}_h$, where $H_f\{\omega (0)\}={\partial }_{\omega }^{2}f\{\omega (0)\}$, in which ${\partial }_{\omega }^{2}f\{\omega (0)\}$ denotes the second-order partial derivative of f (ω, ω⁰) with respect to ω (Zhu et al., 2007). In this case, ${\text{SI}}_f[v]\{\omega (0)\}=v_h^{\text{T}}{H}_f(\omega ,{\omega }^0)v_h/v_h^{\text{T}}G\{\omega (0)\}{v}_h$ and v_S,max equals the eigenvector of G(ω)^−1/2 H_f {ω(0)}G(ω)^−1/2 corresponding to its largest eigenvalue. Let e_j be an m × 1 vector with jth element 1 and 0 otherwise. We also suggest an index plot of SI_f [e_j] to examine influential cases (Zhu et al., 2007, p. 2572).

3.4. Bayesian influence analysis

We now summarize the four key steps in carrying out our proposed influence analysis.

• Step 1. Construct a Bayesian perturbation model p(z, θ | ω).

• Step 2. Given the Bayesian perturbation model, we calculate the geometric quantities, such as <v, v> {ω(0)}, of the perturbation manifold.

• Step 3. Choose an objective function f (ω, ω⁰) and calculate IGI_f (ω, ω⁰) and ω̂_I = argmax_ω∈Ω IGI_f (ω, ω⁰)

In Step 3, we need to compute f (ω, ω⁰) and d(ω, ω⁰). Since f (ω, ω⁰) is a function of p(θ | z, ω) and p(θ | z, ω⁰), we use Markov chain Monte Carlo methods to draw random samples from p(θ | z, ω) and p(θ | z, ω⁰) and then evaluate f (ω, ω⁰) (Chen et al., 2000). We use the Dijkstra algorithm (Dijkstra, 1959) to approximate the geodesic distance between p(z, θ | ω) and p(z, θ | ω⁰). The main idea of this method is to discretize the model {p(z, θ | ω) : ω ∈ Ω} into a simpler space {p(z, θ | ω) : ω ∈ Ω_D}, where Ω_D contains a set of the refined grid points of Ω and then we approximate d(ω, ω⁰) (Dijkstra, 1959). Based on the set of the refined grid points Ω_D, we then calculate {IGI_f (ω, ω⁰) : ω ∈ Ω_D} and approximate ω̂_I by using argmax_{ω ∈ ΩD} IGI_f (ω, ω₀).

Step 4. If df [v]{ω(0)} ≠ 0, then we calculate v_F,max to assess local influence of minor perturbations to the model. However, if df [v]{ω(0)} is 0 for all v, then we compute SI_f [v]{ω(0)} and find v_S,max = argmax[SI_f [v]{ω(0)}].

In Step 4, we need to compute FI_f [v]{ω(0)} and SI_f [v]{ω(0)}. For many infinite-dimensional manifolds, such as the additive ∊-contamination class, v varies in a set 𝒱, which may be well approximated by a finite number of grid points {v_l: l = 1, . . . , K₀}. We can approximate argmax_v[FI_f [v]{ω(0)}] and argmax_v [SI_f [v]{ω(0)}] by argmax_vl[FI_f [v_l]{ω(0)}] and argmax_vl[SI_f [v_l]{ω(0)}, respectively.

4. A theoretical example

We consider a dataset z = (z₁, . . . , z_n)^T to illustrate the potential applications of our proposed diagnostics. Assume that z₁, . . . , z_n are independent and identically distributed from a N(θ, 1) distribution and the baseline prior distribution of θ is the density corresponding to a $N({\mu }_0,{\sigma }_0^2)$ distribution. Letting $\overline{z}={\sum }_{i=1}^n{z}_i/n$, we have $p(\theta |z)\propto \text{exp}\left[-0.5(n+1/{\sigma }_0^2){\{\theta -(n\overline{z}+{\mu }_0/{\sigma }_0^2)/(n+1/{\sigma }_0^2)\}}^2\right]$.

We first consider a simple perturbation to the location of the baseline prior, whose perturbed model is given by

$$p(z,\theta |\omega )=p(z|\theta )p(\theta |\omega )=p(z|\theta )\text{exp}\{-0.5{\left(\theta -\omega -{\mu }_0\right)}^2/{\sigma }_0^2\}/{\left(2\pi {\sigma }_0^2\right)}^{0.5}$$

for ω ∈ [ω_L, ω_U], where ω_L and ω_U are known scalars. We set $E(\theta |z,\omega )=\int \theta p(\theta |z,\omega )d\theta =\{n\overline{z}+\left(\omega +{\mu }_0\right)/{\sigma }_0^2\}/\left(n+1/{\sigma }_0^2\right)$ and f (ω, ω⁰) = E(θ | z, ω) − E(θ | z, ω⁰). Thus, following Berger (1990), we have that the range of f (ω, ω⁰) equals $f({\omega }_U,{\omega }^0)-f({\omega }_L,{\omega }^0)=({\omega }_U-{\omega }_L)/(n{\sigma }_0^2+1)$. A large range can be caused by a large ω_U − ω_L, which is associated with the size of the perturbation to the prior, as shown later.

We compute the intrinsic structure of p(z, θ | ω) and the intrinsic influence measure. We can calculate the geodesic distance between p(z, θ | ω_L) and p(z, θ | ω_U). Since ω(t) = t and $\dot{\ell }\{z,\theta |\omega (t)\}=\left(\theta -{\mu }_0-t\right)/{\sigma }_0^2$, we have $<\dot{\ell }\{z,\theta |\omega (t)\}=\dot{\ell }\{z,\theta |\omega (t)\}>\{\omega (t)\}=1/{\sigma }_0^2$ and $d({\omega }_L,{\omega }_U)={\int }_{{\omega }_L}^{{\omega }_U}dt/{\sigma }_0=({\omega }_U-{\omega }_L)/{\sigma }_0$, which is the size of the sole perturbation to the prior regardless of the data. Both small σ₀ and large ω_U − ω_L can introduce large perturbations. When f (ω, ω⁰) = E(θ | z, ω) − E(θ | z, ω⁰), we have ${\text{IGI}}_f(\omega ,{\omega }^0)={\sigma }_0^2/{(n{\sigma }_0^2+1)}^2$, which is independent of ω. This indicates that relative to the perturbation of the prior, f (ω, ω⁰) does not change too much. A large range gives a false indication of the extent of nonrobustness, which is actually caused by large perturbations to the prior (Sivaganesan, 2000).

Secondly, we consider a simultaneous perturbation to the prior and the model, given by

$$p(z,\theta |\omega )\propto \text{exp}\left\{-0.5\sum _{i=1}^n{(z_i-{\omega }_i-\theta )}^2-0.5{(\theta -{\mu }_0-{\omega }_{n+1})}^2/{\sigma }_0^2\right\},$$ (9)

where ω = (ω₁, . . . ,ω_n+1)^T ∈ Rⁿ⁺¹. In this case, ω⁰ = 0_n+1 represents no perturbation. Let δ_ij equal 1 for i = j and 0 otherwise. Following Example 6, we can show that for i, j = 1, . . . , n,

$$\begin{array}{l}{\partial }_{{\omega }_i}\ell (z,\theta |\omega )=(z_i-{\omega }_i-\theta ),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\partial }_{{\omega }_{n+1}}\ell (z,\theta |\omega )=(\theta -{\mu }_0-{\omega }_{n+1})/{\sigma }_0^2,\\ <{\partial }_{{\omega }_i}\ell (z,\theta |\omega ),{\partial }_{{\omega }_j}\ell (z,\theta |\omega )>(\omega )={\delta }_{ij},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}<{\partial }_{{\omega }_i}\ell (z,\theta |\omega ),{\partial }_{{\omega }_{n+1}}\ell (z,\theta |\omega )>(\omega )=0,\\ <{\partial }_{{\omega }_{n+1}}\ell (z,\theta |\omega ),{\partial }_{{\omega }_{n+1}}\ell (z,\theta |\omega )>(\omega )=1/{\sigma }_0^2.\end{array}$$ (10)

Thus, when σ₀ ≠ 1, ω_i for i = 1, . . . , n and ω_n+1 introduce different levels of perturbation to the fitted model p(z, θ | ω). Furthermore, since < ∂_ωiℓ(z, θ | ω), ∂_ωjℓ(z, θ | ω) > (ω) for all i, j are independent of ω, the manifold ℳ determined by (9) is a flat manifold (Lang, 1995). For any ω in Rⁿ⁺¹, the geodesic connecting p(z, θ | ω) and p(z, θ | ω⁰) is given by p(z, θ; tω) for t ∈ [0, 1]. By using (3), we can show that $d{(\omega ,{\omega }^0)}^2={\sum }_{i=1}^n{\omega }_i^2+{\omega }_{n+1}^2/{\sigma }_0^2$, which quantifies the size of the perturbation scheme (9) to the prior and the fitted model.

We calculate the logarithm of the Bayes factor BF(ω, ω⁰) as discussed in Example 5. Since the terms in the exponential function of (9) form a quadratic function of θ, we can explicitly calculate BF(ω, ω⁰) = P(ω) − P(ω⁰), where P(ω) = log ∫ p(z, θ | ω)dθ equals

$$C-0.5\left[{({\omega }_{n+1}+{\mu }_0)}^2/{\sigma }_0^2+\sum _{i=1}^n{(z_i-{\omega }_i)}^2-{\{({\omega }_{n+1}+{\mu }_0)/{\sigma }_0^2+\sum _{i=1}^n(z_i-{\omega }_i)\}}^2/(n+1/{\sigma }_0^2)\right],$$

and C is a scalar independent of ω. Now recall the results of Example 5. For a smooth curve ω(t) ∈ Rⁿ⁺¹ with ω(0) = ω⁰, FI_f [v]{ω(0)} is determined by ∂_ωBF(ω, ω⁰) and v_F,max(ω) = {G(ω⁰)}^−1/2∂_ωBF(ω, ω⁰), in which $G({\omega }^0)=\text{diag}(1,\dots ,1,{\sigma }_0^{-2})$ as calculated in (10). Taking derivatives of BF(ω, ω⁰) with respect to ω, we get

$$\begin{array}{l}{\partial }_{{\omega }_{n+1}}\text{BF}(\omega ,{\omega }^0)=-({\omega }_{n+1}+{\mu }_0)/{\sigma }_0^2+\{({\omega }_{n+1}+{\mu }_0)/{\sigma }_0^2+\sum _{i=1}^n(z_i-{\omega }_i)\}/(n{\sigma }_0^2+1),\\ \hspace{0.17em}\hspace{0.17em}{\partial }_{{\omega }_i}\text{BF}(\omega ,{\omega }^0)=z_i-{\omega }_i-\{({\omega }_{n+1}+{\mu }_0)/{\sigma }_0^2+\sum _{i=1}^n(z_i-{\omega }_i)\}/(n+1/{\sigma }_0^2)\end{array}$$

for i = 1, . . . , n, which yields

$$v_{F,\text{max}}({\omega }^0)={\left\{z_1-\frac{n\overline{z}+{\mu }_0/{\sigma }_0^2}{n+1/{\sigma }_0^2},\dots ,z_n-\frac{n\overline{z}+{\mu }_0/{\sigma }_0^2}{n+1/{\sigma }_0^2},\frac{n(\overline{z}-{\mu }_0){\sigma }_0}{n{\sigma }_0^2+1}\right\}}^{\text{T}}.$$ (11)

By inspecting the first n components of v_F,max(ω⁰), we can identify outlying points z_i which are far from the posterior mean of θ, while the last component of v_F,max(ω⁰) can pick up an influential hyperparameter μ₀.

Thirdly, we consider a simultaneous perturbation to the prior and the sampling distribution,

$$p(z,\theta |\omega )\propto \text{exp}\left\{-0.5\sum _{i=1}^n{\omega }_i{(z_i-\theta )}^2-0.5{\omega }_{n+1}{(\theta -{\mu }_0)}^2/{\sigma }_0^2+0.5\sum _{i=1}^{n+1}\text{log}({\omega }_i),\right\}$$

where ω = (ω₁, . . . ,ω_n+1)^T ∈ Rⁿ⁺¹. In this case, ω⁰ = 1_n+1 represents no perturbation. Following Example 6, we can show that for i, j = 1, . . . , n,

$$\begin{array}{l}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\partial }_{{\omega }_i}\ell (z,\theta |\omega )=-0.5{(z_i-\theta )}^2+0.5{\omega }_i^{-1},\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\partial }_{{\omega }_{n+1}}\ell (z,\theta |\omega )=-0.5{(\theta -{\mu }_0)}^2/{\sigma }_0^2+0.5{\omega }_{n+1}^{-1},\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}<{\partial }_{{\omega }_i}\ell (z,\theta |\omega )={\partial }_{{\omega }_j}\ell (z,\theta |\omega )>(\omega )=0.5{\omega }_i^{-2}{\delta }_{ij},\\ <{\partial }_{{\omega }_i}\ell (z,\theta |\omega ),={\partial }_{{\omega }_{n+1}}\ell (z,\theta |\omega )>(\omega )=0,\\ <{\partial }_{{\omega }_{n+1}}\ell (z,\theta |\omega ),{\partial }_{{\omega }_{n+1}}\ell (z,\theta |\omega )>(\omega )=0.5{\omega }_{n+1}^{-2}.\end{array}$$ (12)

Thus, G(ω⁰) is an (n + 1) × (n + 1) identity matrix.

We consider a sensitivity analysis for predictive distributions (Lavine, 1992; Millar & Stewart, 2007). Let z_n+1 denote a future observation from N(θ, 1), the predictive density of z_n+1 given z, denoted by p(z_n+1 | z, ω), is shown to be $N\left\{({\sum }_{i=1}^n{\omega }_i{z}_i+{\omega }_{n+1}{\mu }_0/{\sigma }_0^2)/({\sum }_{i=1}^n{\omega }_i+{\omega }_{n+1}/{\sigma }_0^2),1/({\sum }_{i=1}^n{\omega }_i+{\omega }_{n+1}/{\sigma }_0^2)\right\}$. We set f(ω, ω⁰) = ∫ z_n+1 p(z_n+1 | z, ω)dz_n+1 − ∫ z_n+1 p(z_n+1 | z, ω⁰)dz_n+1. Now recall the results of Example 6 and the metric tensor in (12). For a smooth curve ω(t) ∈ Rⁿ⁺¹ with ω(0) = ω⁰, FI_f[v]{ω(0)} is determined by ∂_ωf(ω) and v_F,max (ω) = ∂_ωf(ω, ω⁰), which are given by

$$\begin{array}{l}{\partial }_{{\omega }_{n+1}}f(\omega ,{\omega }^0)={\sigma }_0^{-2}{\mu }_0/\left(\sum _{i=1}^n{\omega }_i+{\omega }_{n+1}/{\sigma }_0^2\right)\\ {\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}-{\sigma }_0^{-2}\left({\omega }_{n+1}{\mu }_0/{\sigma }_0^2+\sum _{i=1}^n{z}_i{\omega }_i\right)/\left(\sum _{i=1}^n{\omega }_i+{\omega }_{n+1}/{\sigma }_0^2\right)}^2,\\ \hspace{0.17em}\hspace{0.17em}{\partial }_{{\omega }_i}f(\omega ,{\omega }^0)=z_i/\left(\sum _{i=1}^n{\omega }_i+{\omega }_{n+1}/{\sigma }_0^2\right)\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{-\left({\omega }_{n+1}{\mu }_0/{\sigma }_0^2+\sum _{i=1}^n{z}_i{\omega }_i\right)/\left(\sum _{i=1}^n{\omega }_i+{\omega }_{n+1}/{\sigma }_0^2\right)}^2\end{array}$$

for i = 1, . . . , n. This yields that v_F,max,(ω⁰) is proportional to

$$\frac{1}{n+1/{\sigma }_0^2}{\left(z_1-\frac{n\overline{z}+{\mu }_0/{\sigma }_0^2}{n+1/{\sigma }_0^2},\dots ,z_n-\frac{n\overline{z}+{\mu }_0/{\sigma }_0^2}{n+1/{\sigma }_0^2}+\frac{n({\mu }_0-\overline{z}){\sigma }_0^2}{n{\sigma }_0^2+1}\right)}^{\text{T}}.$$ (13)

We observe that v_F,max,(ω⁰) in (13) is closely associated with v_F,max(ω⁰) in (11), and thus v_F,max (ω⁰) is able to pick up outlying points z_i and an influential hyperparameter μ₀.

Finally, we examine a more general setting in which z_i (i = 1, . . . , 50) are independent N(θ_i, 1) variables, with the θ_i independently generated from a Dirichlet process prior DP(c₀F₁), where the base measure F₁ is that of a N(5, 1) distribution and the confidence parameter c₀ is set equal to 2 (Escobar, 1994). Furthermore, the z_i were changed to z_i + 5 for i = 49 and 50, which can be regarded as two outliers. We fit a model with z_i ∼ N(θ_i, 1) and θ_i ∼ DP(2F₀), where F₀ is the probability measure of a N(0, 1) distribution. The base measure F₀ is misspecified due to the difference between the means of a N(0, 1) and the true base measure N(5, 1). We consider a simultaneous perturbation to the prior and the data. We have

$$\begin{array}{l}p(z,\theta |\omega )\propto \text{exp}(-0.5\sum _{i=1}^n{(z_i-{\omega }_i-{\theta }_i)}^2\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\sum _{i=1}^n\text{log}\left[c_0{F}_0({\theta }_i)+c_0{\omega }_{n+1}\{F_1({\theta }_i)-F_0({\theta }_i)\}+\sum _{j=1}^{i-1}{\delta }_{{\theta }_j}({\theta }_i)\right]).\end{array}$$ (14)

In this case, ω⁰ = 0_n+1 represents no perturbation. By differentiating ℓ(z, θ | ω) = log p(z, θ | ω) in (14) with respect to each component of ω, we have that for i = 1, . . . , n,

$$\begin{array}{l}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\partial }_{{\omega }_i}\ell (z,\theta ;\omega )=z_i-{\omega }_i-{\theta }_i,\\ {\partial }_{{\omega }_{n+1}}\ell (z,\theta |\omega )=\sum _{i=1}^n\frac{c_0\{F_1({\theta }_i)-F_0({\theta }_i)\}}{c_0{F}_0({\theta }_i)+c_0{\omega }_{n+1}\{F_1({\theta }_i)-F_0({\theta }_i)\}+{\sum }_{j=1}^{i-1}{\delta }_{{\theta }_j}({\theta }_i)}.\end{array}$$

Since ∫ (z_i − ω_i − θ_i) p(z, θ | ω)dz = 0 and ∫ (z_i − ω_i − θ_i)(z_j − ω_j − θ_j) p(z, θ | ω)dz = δ_ij, we have

$$\begin{array}{l}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}<{\partial }_{{\omega }_i}\ell (z,\theta |\omega ),{\partial }_{{\omega }_j}\ell (z,\theta |\omega )>(\omega )={\delta }_{ij},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}<{\partial }_{{\omega }_i}\ell (z,\theta |\omega ),{\partial }_{{\omega }_{n+1}}\ell (z,\theta |\omega )>(\omega )=0,\\ <{\partial }_{{\omega }_{n+1}}\ell (z,\theta |\omega ),{\partial }_{{\omega }_{n+1}}\ell (z,\theta |\omega )>(\omega )=E\left[{\{{\partial }_{{\omega }_{n+1}}\ell (z,\theta |\omega )\}}^2\right].\end{array}$$

Similar to (11), we set f (ω, ω⁰) = BF(ω, ω⁰) and substitute the results from (7) to calculate v_F,max(ω⁰) using 50 000 Markov chain Monte Carlo samples generated from the posterior distribution p(θ₁, . . . ,θ_n | z₁, . . . , z₅₀) after a 5000 sample burn-in. Inspecting the components of v_F,max(ω⁰) reveals the outlying cases 49 and 50 and shows the sensitivity to the misspecified base measure F₀ of the Dirichlet process prior for θ_i in Fig. 1.

Fig. 1.

Simultaneous perturbation model using a Dirichlet process prior and perturbing individual observations: (a) local influence measures v_B,max(ω⁰) for the logarithm of the Bayes factor f (ω, ω⁰) = BF(ω, ω⁰), from which the outlying cases 49 and 50 and the perturbation to the Dirichlet process prior were detected; (b) index plot of metric tensor g_ii (ω⁰) for the perturbation (15).

In addition to this theoretical example, an extensive simulation and a real data analysis involving missing data are given in the Supplementary Material. In practice, we suggest an iterative process to carry out the four-step influence analysis in § 3.4. If one is concerned about sensitivity to the prior, then one may introduce some finite-dimensional perturbation as in Example 1 to all hyperparameters of the prior and identify influential hyperparameters according to their local influence measures. Then, for a few influential hyperparameters, one further perturbs their associated prior distribution using the additive ∊-contamination class and then carries out intrinsic influence analysis. If one is concerned about the sampling distribution, then one may introduce various perturbations including the additive ∊-contamination class and the perturbation model (1) to p(z | θ) and use the local influence measures to detect which part of p(z | θ) is sensitive to minor perturbations. Then, one may focus on these influential parts and carry out an intrinsic influence analysis. After refining the prior and the sampling distribution, one may then perturb individual observations and detect a set of influential observations. After examining the information from each influence analysis, we carry out a simultaneous perturbation to z, p(θ) and p(z | θ). We start with a local influence analysis to examine the sensitivity of all components and then focus on a few influential components using an intrinsic influence analysis.

Appendix

Proof of Proposition 1. Consider any two smooth curves p{z, θ | ω_(k)(t)} = p{θ | ω_(k)(t)} p(z | θ) with p{z, θ | ω_(k)(0)} = p(θ | ω) p(z | θ) for k = 1, 2. For each k, by differentiating ℓ{z, θ | ω_(k)(t)} with respect to t, we obtain a tangent vector v_k (ω) = ℓ{z, θ | ω_(k)(0)} = d log p{θ | ω_(k)(t)}/dt |_t=0 ∈ T_ωℳ, which is independent of p(z | θ). Furthermore, letting d_t = d/dt, the inner product of v₁(ω) and v₂(ω) is given by ∫ [d_t log p{θ | ω₍₁₎(t)}][d_t log p{θ | ω₍₂₎(t)}] p{z, θ | ω}dzdθ = ∫ [d_t log p{θ | ω₍₁₎(t)}][d_t log p{θ | ω₍₂₎(t)}] p{θ | ω}dθ, which is also independent of p(z | θ).

Proof of Proposition 2. Consider two smooth curves p{z, θ | ω_(k)(t)} with ω_(k)(t) = {ω_(k),p(t)^T, {ω_(k),s(t)^T}^T such that ω₍₁₎(0) = ω₂(0) = ω and (ω_(1),p(t) and ω_(1),s(t) are independent of t. Let ℓ(z | θ, ω_(1),s) = log p(z | θ, ω_(1),s). Since ω_(1),p(t) is independent of t,

$$v_1(\omega )=\dot{\ell }\{z,\theta |{\omega }_{(1)}(0)\}=\frac{d}{dt}\text{log}p\{\theta |{\omega }_{(1),p}(t)\}{|}_{t=0}+\frac{d}{dt}\text{log}p\{z|\theta ,{\omega }_{(1),s}(t)\}{|}_{t=0}\dot{\ell }\{z|\theta ,{\omega }_{(1),s}(0)\}.$$

Let ℓ(θ | ω_(2),p) = log p(θ | ω_(2),p). Similarly, we have

$$v_2(\omega )=\dot{\ell }\{z,\theta |{\omega }_{(2)}(0)\}=\frac{d}{dt}\text{log}p\{\theta |{\omega }_{(2),p}(t)\}{|}_{t=0}\dot{\ell }\{\theta |{\omega }_{(2),p}(0)\}.$$

Thus, the inner product of v₁(ω) and v₂(ω), denoted by < v₁, v₂ > (ω), is given by

$$\begin{array}{ll}\int \dot{\ell }\{\theta |{\omega }_{(2),p}(0)\}\dot{\ell }\{z|\theta ,{\omega }_{(1),s}(0)\}p(z,\theta |\omega )dzd\theta \hfill & =\hspace{0.17em}\int \frac{dp\{\theta |{\omega }_{(2),p}(0)\}}{dt}\frac{dp\{z|\theta ,{\omega }_{(1),s}(0)\}}{dt}dzd\theta \hfill \\ \hfill & =\int \left(\frac{dp\{\theta |{\omega }_{(2),p}(0)\}}{dt}\left[\int \frac{dp\{z|\theta ,{\omega }_{(1),s}(0)\}}{dt}dz\right]\right)d\theta \hfill \\ \hfill & =\int \left(\frac{dp\{\theta |{\omega }_{(2),p}(0)\}}{dt}\frac{d\left[\int dp\{z|\theta ,{\omega }_{(1),s}(0)\}dz\right]}{dt}\right)d\theta \hfill \\ \hfill & =\int \left[\frac{dp\{\theta |{\omega }_{(2),p}(0)\}}{dt}\frac{d1}{dt}\right]d\theta =0.\hfill \end{array}$$

Proof of Theorem 1. Since Theorem 1 (i) follows from Proposition 2, we focus on Theorem 1 (ii). Since {ω_(1),p(t), ω_(1),d(t)} and {ω_(2),p(t), ω_(2),s(t)} are independent of t and p(z | θ, ω_d, ω_s) = p₁(z | θ, ω_d) p₂(z | θ, ω_s), we have

$$\begin{array}{l}{v}_1(\omega )=\dot{\ell }\{z,\theta |{\omega }_{(1)}(0)\}=\frac{d}{dt}\text{log}{p}_1\{z|\theta ,{\omega }_{(1),s}(t)\}{|}_{t=0},\\ v_2(\omega )=\dot{\ell }\{z,\theta |{\omega }_{(2)}(0)\}=\frac{d}{d{t}^{\prime }}\text{log}{p}_2\{z|\theta ,{\omega }_{(2),d}(t^{\prime })\}{|}_{t^{\prime }=0}.\end{array}$$

Thus, < v₁, v₂ > (ω) is given by

$$\begin{array}{l}{\int \frac{d\text{log}{p}_1\{z|\theta ,{\omega }_{(1),s}(t)\}}{dt}|}_{t=0}{\frac{d\text{log}{p}_2\{z|\theta ,{\omega }_{(2),d}(t^{\prime })\}}{d{t}^{\prime }}|}_{t^{\prime }=0}p(z,\theta |\omega )dzd\theta \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\int \frac{d{p}_1\{z|\theta ,{\omega }_{(1),s}(0)\}}{dt}\frac{d{p}_2\{z|\theta ,{\omega }_{(2),d}(0)\}}{d{t}^{\prime }}p(\theta |{\omega }_p)dzd\theta =\frac{d^{2}1}{dtd{t}^{\prime }}=0.\end{array}$$

Proof of Theorem 2. Consider a smooth curve p{z, θ | ω(t)}. Let R(s) : [c₁, c₂] → [−∊, ∊] be the first-order differential map such that R(c₃) = 0 and Ṙ (c₃) = dR(s)/ds |_s=c3 ≠ 0 for a c₃ ∈ (c₁, c₂). Then, p[z, θ | ω{R(s)}] is a differential map from [c₁, c₂] to ℳ. It follows from the chain rule that ḟ[ω{R(s)}] = d_s f[ω{R(s)}, ω⁰] = d_r f {ω(r), ω⁰} Ṙ(s) and d_sℓ [z, θ | ω {R(s)}] = d_r ℓ{z, θ | ω(r)} Ṙ(s), where Ṙ (s) = d_s R(s), d_c = d/dc, d_r = d/dr, and d_s = d/ds. Thus, as ω(0) = ω⁰, we have

$$df\left[\dot{R}(c_3)v\right]\left[\omega \{R(c_3)\}\right]=\dot{R}(c_3)df[v](\omega ),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{and}<\dot{R}(c_3)v,\dot{R}(c_3)v>(\omega )=\dot{R}{(c_3)}^2<v,v>(\omega ).$$