Wavelet invariants for statistically robust multi-reference alignment

Abstract

We propose a nonlinear, wavelet-based signal representation that is translation invariant and robust to both additive noise and random dilations. Motivated by the multi-reference alignment problem and generalizations thereof, we analyze the statistical properties of this representation given a large number of independent corruptions of a target signal. We prove the nonlinear wavelet-based representation uniquely defines the power spectrum but allows for an unbiasing procedure that cannot be directly applied to the power spectrum. After unbiasing the representation to remove the effects of the additive noise and random dilations, we recover an approximation of the power spectrum by solving a convex optimization problem, and thus reduce to a phase retrieval problem. Extensive numerical experiments demonstrate the statistical robustness of this approximation procedure.

1. Introduction

The goal in classic multi-reference alignment (MRA) is to recover a hidden signal $f:ℝ\to ℝ$ from a collection of noisy measurements. Specifically, the following data model is assumed.

Model 1 (Classic MRA).

The classic MRA data model consists of M independent observations of a compactly supported, real-valued signal $f\in {\mathbf{L}}^2(ℝ)$:

$$y_j(x)=f\left(x-t_j\right)+{\epsilon }_j(x),1⩽j⩽\mathit{M},$$ (1.1)

where:

1. $\mathrm{supp}\left(y_j\right)\subseteq \left[-\frac{1}{2},\frac{1}{2}\right]$ for 1 ⩽ j ⩽ M.

2. ${\left\{t_j\right\}}_{j=1}^M$ are independent samples of a random variable $t\in ℝ$.

3. ${\left\{{\epsilon }_j(x)\right\}}_{j=1}^M$ are independent white noise processes on $\left[-\frac{1}{2},\frac{1}{2}\right]$, with variance σ².

The signal is thus subjected to both random translation and additive noise. The MRA problem arises in numerous applications, including structural biology [32,64,65,70,71,79], single cell genomic sequencing [51], radar [43,85], crystalline simulations [76], image registration [18,40,69] and signal processing [85]. It is a simplified model relevant for cryo-electron microscopy (cryo-EM), an imaging technique for molecules that achieves near atomic resolution [11,14,75]. In this application one seeks to recover a three-dimensional reconstruction of the molecule from many noisy two-dimensional images/projections [41]. Although MRA ignores the tomographic projection of cryo-EM, investigation of the simplified model provides important insights. For example, [5,66] investigate the optimal sample complexity for MRA and demonstrate that M = Θ(σ⁶) is required to fully recover f in the low signal-to-noise regime when the translation distribution is periodic; this optimal sample complexity is the same for cryo-EM [7,82]. Recent work has established an improved sample complexity of M = Θ(σ⁴) for MRA when the translation distribution is aperiodic [1], and this rate has been shown to also hold in the more complicated setting of cryo-EM, if the viewing angles are non-uniformly distributed [72]. Problems closely related to Model 1 include the heterogenous MRA problem, where the unknown signal f is replaced with a template of k unknown signals f₁, . . . , f_k [16,54,66,77], as well as multi-reference factor analysis, where the underlying (random) signal follows a low-rank factor model and one seeks to recover its covariance matrix [50].

Approaches for solving MRA generally fall into two categories: synchronization methods and methods that estimate the signal directly, i.e. without estimating nuisance parameters. Synchronization methods attempt to recover the signal by aligning the translations and then averaging. They include methods based on angular synchronization [8,15,24,67,73,84], where for each pair of signals the best pairwise shift is computed and then the translations are estimated from this pairwise information [6], and semi-definite programming [4,9,10,25], which approximates the quasi-maximum likelihood estimator of the shifts by relaxing a non-convex rank constraint. However, these methods fail in the low signal-to-noise regime. Methods that estimate the signal directly include both the method of moments [44,48,72] and expectation maximization, or EM-type, algorithms [1,30]; a number of EM-type algorithms have also been developed for the more complicated cryo-EM problem [33,68]. An important special case of the method of moments is the method of invariants, which seeks to recover f by computing translation invariant features, and thus avoids aligning the translations. However, the task is a difficult one, as a complete representation is needed to recover the signal, and yet the representation may be difficult to invert and corrupted by statistical bias. Generally, the signal is recovered from translation invariant moments, which are estimated in the Fourier domain [29,44]. Recent work [5,14] utilizes such Fourier invariants (mean, power spectrum and bispectrum) and recovers $\widehat{f}$ by solving a non-convex optimization problem on the manifold of phases.

Classic MRA however fails to capture many of the biological phenomena arising in molecular imaging, such as the random rotations of the molecules and the tomographic projection associated with the imaging of three-dimensional objects. Another shortcoming is that the model fails to capture the dynamics that arise from flexible regions in macromolecular structures. These flexible regions are very important in structural biology, for example in understanding molecular interactions [36,39,52,53] and molecular recognition of epigenetic regulators of histone tails [17,31,58]. The large-scale dynamics of these regions makes imaging challenging [81], and thus sample preparation in cryo-EM generally seeks to minimize these dynamics by focusing on well-folded macromolecules frozen in vitreous ice [63]. However, this ‘may severely impact... the nature of the intrinsic dynamics and interactions displayed by macromolecules’ [63]. Although modern cryo-EM is making great strides in understanding flexible systems [3,37,38,59], formulating models that are more capable of capturing the motions associated with the flexible regions of macromolecules could open the door to applying cryo-EM more broadly, i.e. to less well-folded macromolecules. Mathematically, the motion of the flexible region can be modeled as a diffeomorphism. See Fig. 1, which shows a molecule with a flexible side chain (1(a)) and a diffeomorphism resulting from movement of the flexible region (1(b)). Figure 1(a) is taken from [63], and Fig. 1(b) was obtained by deforming it.

Fig. 1.

Dynamics arising from flexible regions in macromolecular structures [63].

This article thus generalizes the classic MRA problem to include a random diffeomorphism. Specifically, we consider recovering a hidden signal $f:ℝ\to ℝ$ from

$$y_j(x)=L_{{\tau }_j}f\left(x-t_j\right)+{\epsilon }_j(x),1⩽j⩽\mathit{M},$$

where L_τ is a dilation operator that dilates by a factor of (1−τ). The dilation operator L_τ is a simplified model for more general diffeomorphisms L_ζ f(x) = f(ζ(x)), since in the simplest case when ζ(x) is affine, L_ζ simply translates and dilates f (see Section 2.1). Dilations are also relevant for the analysis of time-warped audio signals, which can arise from the Doppler effect and in speech processing and bioacoustics. For example, [60–62] consider a stationary random signal f(x), which is time-warped, i.e. $D_{\zeta }f(x)=\sqrt{{\zeta }^{\prime }(x)}f(\zeta (x))$, and use a maximum likelihood approach to estimate ζ. In [27,28], a similar stochastic time warping model is analyzed using wavelet based techniques. The noisy dilation MRA model considered here corresponds to the simplest case of time-warping, when ζ is an affine function. This special case is in fact very important in imaging applications [22,23,46,57,69,80], where it is critical to compute features which are scale invariant, as objects are naturally dilated by the ‘zoom’ of an image.

A new approach is needed to solve this more general MRA problem, as Fourier invariants will fail, being unstable to the action of diffeomorphisms, including dilations. The instability occurs in the high frequencies, where even a small diffeomorphism can significantly alter the Fourier modes. We instead propose ${\mathbf{L}}^2(ℝ)$ wavelet coefficient norms as invariants, using a continuous wavelet transform. This approach is inspired by the invariant scattering representation of [56], which is provably stable to the actions of small diffeomorphisms. However, here we replace local averages of the modulus of the wavelet coefficients with global averages (i.e. integrations) of the modulus squared, thus providing rigid invariants that can be statistically unbiased. Similar invariant coefficients have been utilized in a number of applications including predicting molecular properties [34,35] and quantum chemical energies [45], and in microcanonical ensemble models for texture synthesis [19]. Recent work [42] has also generalized such coefficients to graphs.

1.1. Notation

The Fourier transform of a signal $f\in {\mathbf{L}}^1(ℝ)$ is

$$\widehat{f}(\omega )=\int f(x)e^{-ix\omega }\text{d}x.$$

We remind the reader that compactly supported ${\mathbf{L}}^2(ℝ)$ functions are in ${\mathbf{L}}^1(ℝ)$. The power spectrum is the nonlinear transform $P:{\mathbf{L}}^2(ℝ)\to {\mathbf{L}}^1(ℝ)$ that maps f to

$$(Pf)(\omega )=|\widehat{f}(\omega ){|}^2,\omega \in ℝ.$$

We denote f(x) ⩽ Cg(x) for some absolute constant C by f(x) ≲ g(x). We also write f(x) = O(g(x)) if |f(x)| ⩽ Cg(x) for all x ⩾ x₀ for some constants x₀, C > 0; f(x) = o(g(x)) denotes f(x)/g(x) → 0 as x → ∞; f(x) = Θ(g(x)) denotes C₁g(x) ⩽ |f(x)| ⩽ C₂g(x) for all x ⩾ x₀ for some constants x₀, C₁, C₂ > 0. The minimum of a and b is denoted a ∧ b, and the maximum by a ∨ b.

2. MRA models and the method of invariants

Standard MRA models are generalized to models that include deformations of the underlying signal in Section 2.1. Section 2.2 reviews power spectrum invariants and introduces ${\mathbf{L}}^2(ℝ)$ wavelet coefficient invariants. Theorem 2.4 proves wavelet coefficient invariants computed with a continuous wavelet transform and a suitable mother wavelet are equivalent to the power spectrum, showing there is no information loss in the transition from one representation to the other.

2.1. MRA data models

A standard MRA scenario considers the problem of recovering a signal $f\in {\mathbf{L}}^2(ℝ)$ in which one observes random translations of the signal, each of which is corrupted by additive noise. The problem is particularly difficult when the signal-to-noise ratio (SNR) is low, as registration methods become intractable. In [5,13,14,16,54,74] the authors propose a method using Fourier-based invariants, which are invariant to translations and thus eliminate the need to register signals.

A more general MRA scenario incorporates random deformations of the signal f, which could be used to model underlying physical variability that is not captured by rigid transformations and additive noise models. For example [4,7] consider a discrete signal f corrupted by an arbitrary group action, [47,85] consider random deformations arising in RADAR and [2] considers a generalization of MRA where signals are rescaled by random constants. Another natural mathematical model is small, random diffeomorphisms, which leads to observations of the form

$$y_j(x)=L_{{\zeta }_j}f\left(x-t_j\right)+{\epsilon }_j(x),1⩽j⩽\mathit{M},$$ (2.1)

where ${\zeta }_j\in {\mathbf{C}}^1(ℝ)$ is a random diffeomorphism, $t_j\in ℝ$ is a random translation and the signals ε_j(x) are independent white noise random processes. The transform L_ζ is the action of the diffeomorphism ζ on f,

$$L_{\zeta }f(x)=f(\zeta (x)).$$

If ${\Vert {\left({\zeta }^{-1}\right)}^{\prime }\Vert }_{\infty }<\infty$, then one can verify $L_{\zeta }:{\mathbf{L}}^2(ℝ)\to {\mathbf{L}}^2(ℝ)$.

One of the keys to the Fourier invariant approach of [5,13,14,16,54,74] is the authors can unbias the Fourier invariants of the noisy signals, thus allowing them to devise an unbiased estimator of the Fourier invariants of the signal f (or a mixture of signals in the heterogeneous MRA case). For the diffeomorphism model (2.1) this would require developing a procedure for unbiasing the (Fourier) invariants of ${\left\{y_j\right\}}_{j=1}^M$ against both additive noise and random diffeomorphisms.

In order to get a handle on the difficulties associated with the proposed diffeomorphism model, in this paper we consider random dilations of the signal f, which corresponds to restricting the diffeomorphism to be of the form

$$\zeta (x)=\frac{x}{1-\tau },\left|\tau \right|⩽1/2.$$

Specifically, we assume the following noisy dilation MRA model.

Model 2 (Noisy dilation MRA data model).

The noisy dilation MRA data model consists of M independent observations of a compactly supported, real-valued signal $f\in {\mathbf{L}}^2(ℝ)$:

$$y_j(x)=L_{{\tau }_j}f\left(x-t_j\right)+{\epsilon }_j(x),1⩽j⩽\mathit{M},$$ (2.2)

where L_τ is an ${\mathbf{L}}^1(ℝ)$ normalized dilation operator,

$$L_{\tau }f(x)={(1-\tau )}^{-1}f\left({(1-\tau )}^{-1}x\right).$$

In addition, we assume the following:

1. $\mathrm{supp}\left(y_j\right)\subseteq \left[-\frac{1}{2},\frac{1}{2}\right]$ for 1 ⩽ j ⩽ M.

2. ${\left\{t_j\right\}}_{j=1}^M$ are independent samples of a random variable $t\in ℝ$.

3. ${\left\{{\tau }_j\right\}}_{j=1}^M$ are independent samples of a bounded, symmetric random variable τ satisfying

   $$\tau \in ℝ,\mathbb{E}(\tau )=0,\mathrm{Var}(\tau )={\eta }^2,\left|\tau \right|⩽1/2.$$

4. ${\left\{{\epsilon }_j(x)\right\}}_{j=1}^M$ are independent white noise processes on $\left[-\frac{1}{2},\frac{1}{2}\right]$ with variance σ².

Remark 2.1

The interval $\left[-\frac{1}{2},\frac{1}{2}\right]$ is arbitrary and can be replaced with any interval of length 1. In addition, the spatial box size is arbitrary, i.e. $\left[-\frac{1}{2},\frac{1}{2}\right]$, can be replaced with $\left[-\frac{N}{2},\frac{N}{2}\right]$. All results still hold with $\sigma \sqrt{N}$ replacing σ wherever it appears.

Thus, the hidden signal f is supported on an interval of length 1, and we observe M independent instances of the signal that have been randomly translated, randomly dilated and corrupted by additive white noise. We assume the hidden signal is real, but the proposed methods can also handle complex valued signals with minor modifications. Recall ε(x) is a white noise process if ε(x) = dB_x, i.e. it is the derivative of a Brownian motion with variance σ².

While the noisy dilation MRA model does not capture the full richness of the diffeomorphism model, it already presents significant mathematical difficulties. Indeed, as we show in Section 5, Fourier invariants, specifically the power spectrum, cannot be used to form accurate estimators under the action of dilations and random additive noise. The reason is that Fourier measurements are not stable to the action of small dilations (measured here by |τ|), since the displacement of $\widehat{L_{\tau }f}(\omega )$ relative to $\widehat{f}(\omega )$ depends on |ω|. Intuitively, high-frequency modes are unstable, and yet high frequencies are often critical; for example removing high frequencies increases the sample complexity needed to distinguish between signals in a heterogeneous MRA model [5]. We thus replace Fourier-based invariants with wavelet coefficient invariants, which are defined in Section 2.2. As we show the wavelet invariants of the signal f can be accurately estimated from wavelet invariants of the noisy signals ${\left\{y_j\right\}}_{j=1}^M$, with no information loss relative to the power spectrum of f.

For future reference we also define the following dilation MRA model, which includes random translations and random dilations but no additive noise. Thus, Models 1 and 3 are both special cases of Model 2.

Model 3 (Dilation MRA data model).

The dilation MRA data model consists of M independent observations of a compactly supported, real-valued signal $f\in {\mathbf{L}}^2(ℝ)$:

$$y_j(x)=L_{{\tau }_j}f\left(x-t_j\right),1⩽j⩽M,$$ (2.3)

where L_τ is an ${\mathbf{L}}^1(ℝ)$ normalized dilation operator,

$$L_{\tau }f(x)={(1-\tau )}^{-1}f\left({(1-\tau )}^{-1}x\right).$$

In addition, we assume (i)–(iii) of Model 2.

2.2. Method of invariants

We now discuss how invariant representations can be used to solve MRA data models and introduce the wavelet invariants used in this article.

2.2.1. Motivation and related work

Let T_tf(x) = f(x − t) denote the operator that translates by t acting on a signal f. Invariant measurement models seek a representation $\Phi (f)\in \mathcal{B}$ in a Banach space $\mathcal{B}$ such that

$$\Phi \left(T_{t}f\right)=\Phi (f),\forall t\in ℝ.$$ (2.4)

In MRA problems, one additionally requires that

$$\Phi (f)=\Phi (g)\iff g=T_{t}f\text{for}\text{some}t\in ℝ.$$ (2.5)

The first condition (2.4) removes the need to align random translations of the signal f, whereas the second condition (2.5) ensures that if one can estimate Φ(f) from the collection ${\left\{\Phi \left(y_j\right)\right\}}_{j=1}^M$, then one can recover an estimate of f (up to translation) by solving

$$f^{\star }=\underset{g\in {\mathbf{L}}^1\cap {\mathbf{L}}^2(ℝ)}{\arg \inf }\Vert \Phi (g)-\Phi (f)\Vert \mathcal{B},$$ (2.6)

where $\Vert \cdot {\Vert }_{\mathcal{B}}$ is the Banach space norm.

When the observed signals ${\left\{y_j\right\}}_{j=1}^M$ are corrupted by more than just a random translation, though, as in Model 2, estimating Φ(f) from ${\left\{\Phi \left(y_j\right)\right\}}_{j=1}^M$ is not always straightforward. Indeed, one would like to compute

$${\overline{\Phi }}_M(f)=\frac{1}{M}\sum _{j=1}^M\Phi \left(y_j\right),$$ (2.7)

but the quantity ${\overline{\Phi }}_M(f)$ is not always an unbiased estimator of Φ(f), meaning that ${\lim }_{M\to \infty }{\overline{\Phi }}_M(f)\ne \Phi (f)$. In order to circumvent this issue, one must select a representation Φ such that

$$\mathbb{E}\Phi \left(y_j\right)=\Phi (f)+b_{\Phi }(f,\mathcal{M}),$$ (2.8)

where $b_{\Phi }(f,\mathcal{M})$ is a bias term depending on the choice of Φ, f, and the signal corruption model $\mathcal{M}$. If (2.8) holds and if we can compute a $\tilde{b}$ such that $\mathbb{E}{\tilde{b}}_{\Phi }\left(y_j,\mathcal{M}\right)=b_{\Phi }(f,\mathcal{M})+\delta$ for $\left|b_{\Phi }(f,\mathcal{M})\right|\gg |\delta |$, then one can amend (2.7) to reduce the bias

$${\tilde{\Phi }}_M(f)=\frac{1}{M}\sum _{j=1}^M\left(\Phi \left(y_j\right)-{\tilde{b}}_{\Phi }\left(y_j,\mathcal{M}\right)\right),$$

in which case

$$\underset{M\to \infty }{\lim }{\tilde{\Phi }}_M(f)=\Phi (f)+\delta$$

almost surely by the law of large numbers. The main difficulty therefore is twofold. On the one hand, one must design a representation Φ that satisfies (2.4), (2.5) and (2.8) with a bias b that can be estimated; on the other hand, the optimization (2.6) must be tractable. For random translation plus additive noise models (i.e., Model 1), the authors of [5,14] describe a representation Φ based on Fourier invariants that satisfies the outlined requirements and for which one can solve (2.6) despite the optimization being non-convex. The Fourier invariants include $\widehat{f}(0)$ (i.e. the integral of f), the power spectrum of f and the bispectrum of f. Each invariant captures successively more information in f. While $\widehat{f}(0)$ carries limited information, the power spectrum recovers the magnitude of the Fourier transform, namely it recovers the non-negative, real-valued function ρ(ω) such that $\widehat{f}(\omega )=\rho (\omega )e^{i\theta (\omega )}$, but the phase information Θ(ω) is lost. Since $\widehat{T_{t}f}(\omega )=e^{-i\omega t}\widehat{f}(\omega )$, the power spectrum is invariant to translations as the Fourier modulus kills the phase factor induced by a translation t of f. However, it is in general not possible to recover a signal from its power spectrum, although in certain special cases the phase information can be resolved; results along these lines are in the field of phase retrieval [26,78]. The bispectrum is also translation invariant and invertible so long as $\widehat{f}(\omega )\ne 0$ [66].

In Section 5 we show that it is impossible to significantly reduce the power spectrum bias for Model 2, which includes translations, dilations and additive noise. We thus propose replacing the power spectrum with the ${\mathbf{L}}^2(ℝ)$ norms of the wavelet coefficients of the signal f. These invariants satisfy (2.4) and (2.8) for Model 2 and yield a convex formulation of (2.6). They do not satisfy (2.5) for general $f\in {\mathbf{L}}^2(ℝ)$, but Theorem 2.4 in Section 2.2.2 shows that knowing the wavelet invariants of f is equivalent to knowing the power spectrum of f, which means that any phase retrieval setting in which recovery is possible will also be possible with the specified wavelet invariants. For example if the signal lives in a spline or shift invariant space in addition to being realvalued, then it can be recovered from its phaseless measurements [26,78].

2.2.2. Wavelet invariants

We now define the wavelet invariants used in this article. A wavelet $\psi \in {\mathbf{L}}^2(ℝ)$ is a waveform that is localized in both space and frequency and has zero average,

$$\int \psi (x)\text{d}x=0.$$

Note throughout this article ψ will always denote a wavelet in ${\mathbf{L}}^1\cap {\mathbf{L}}^2(ℝ)$ with zero average, satisfying ‖ψ‖₂ = 1 as well as the classic admissability condition $\int \frac{|\widehat{\psi }(\omega ){|}^2}{\omega }\text{d}\omega <\infty$. A dilation of the wavelet by a factor λ ∈ (0,∞) is denoted,

$${\psi }_{\lambda }(x)={\lambda }^{1/2}\psi (\lambda x),$$

where the normalization guarantees that ‖ψ_λ‖₂ = ‖ψ‖₂ = 1. The continuous wavelet transform W computes

$$Wf=\left\{f\ast {\psi }_{\lambda }(x):\lambda \in (0,\infty ),x\in ℝ\right\}.$$

The parameter λ corresponds to a frequency variable. Indeed, if ξ₀ is the central frequency of ψ, the wavelet coefficients f ∗ ψ_λ recover the frequencies of f in a band of size proportional to λ centered at λξ₀. Thus, high frequencies are grouped into larger packets, which we shall use to obtain a stable, invariant representation of f.

The wavelet transform Wf is equivariant to translations but not invariant. Integrating the wavelet coefficients over x yields translation invariant coefficients, but they are trivial since $\int {\psi }_{\lambda }=0$. We therefore compute ${\mathbf{L}}^2(ℝ)$ norms in the x variable, yielding the following nonlinear wavelet invariants:

Definition 2.1 (Wavelet invariants).

The L² wavelet invariants of a real-valued signal $f\in {\mathbf{L}}^1\cap {\mathbf{L}}^2(ℝ)$ are given by

$$(Sf)(\lambda )={\Vert f*{\psi }_{\lambda }\Vert }_2^2,\lambda \in (0,\infty ),$$ (2.9)

where ψ_λ(x) = λ^1/2ψ(λx) are dilations of a mother wavelet ψ.

Throughout this article ψ can be taken as a Morlet wavelet, in which case ψ is constructed to have frequency centered at ξ by $\psi (x)=C_{\xi }{\pi }^{-1/4}{e}^{-x^2/2}\left(e^{i\xi x}-e^{-{\xi }^2/2}\right)$ for $C_{\xi }={\left(1-e^{-{\xi }^2}-2e^{-3{\xi }^2/4}\right)}^{-1/2}$, but results hold more generally for what we refer to as k-admissible wavelets, where k ⩾ 0 is an even integer. See Appendix A for a precise description of this admissibility criteria. The wavelet invariants can be expressed in the frequency domain as

$$(Sf)(\lambda )={\frac{1}{2\pi }\int |\widehat{f}(\omega )|}^2{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega ,$$

which motivates the following definition of ‘wavelet invariant derivatives’.

Definition 2.2 (Wavelet invariant derivatives).

The n-th derivative of (Sf)(λ) is defined as

$${(Sf)}^{(n)}(\lambda ):={\frac{1}{2\pi }\int |\widehat{f}(\omega )|}^2\frac{d^n}{d{\lambda }^n}{\left|{\widehat{\psi }}_{\lambda }(\omega )\right|}^2\text{d}\omega .$$

Remark 2.2

Definition 2.1 assumes $f:ℝ\to ℝ$, which allows the wavelet ψ to be either real or complex. Our results can easily be extended to complex f, but a strictly complex wavelet would be needed, with Sf(λ) computed for all λ ∈ (−∞, ∞) \ 0.

Remark 2.3

For a discrete signal of length n, computing the wavelet invariants via a continuous wavelet transform is O(n²), while computing the power spectrum is O(n log n). Thus, one pays a computational cost to achieve greater stability with no loss of information. On the other hand, if wavelet invariants are computed for a dyadic wavelet transform (i.e. only for O(log n) λ’s), the computational cost is the same and stability is maintained, but more information is lost.

Remark 2.4

When $(Pf)(\omega )={|\widehat{f}(\omega )|}^2$ is continuous, Definition 2.2 reduces to a normal derivative, i.e. one can check that ${(Sf)}^{(n)}(\lambda )=\frac{d^n}{d{\lambda }^n}(Sf)(\lambda )$. However, when Pf is not continuous, in general ${(Sf)}^{(n)}(\lambda )\ne \frac{d^n}{d{\lambda }^n}(Sf)(\lambda )$ and (Sf)⁽ⁿ⁾(λ) is more convenient for controlling the error of the estimators proposed in this article. Throughout this article, the notation (Sf)⁽ⁿ⁾(λ) will thus denote the derivative of Definition 2.2 and $\frac{d^n}{d{\lambda }^n}(Sf)(\lambda )$ will denote the standard derivative.

Under mild conditions, one can show that $S:{\mathbf{L}}^2(ℝ)\to {\mathbf{L}}^1\cap \mathbf{C}(0,\infty )$. The values λ = 2^j for $j\in \mathbb{Z}$ correspond to rigid versions of first-order ${\mathbf{L}}^2(ℝ)$ wavelet scattering invariants [56]. The continuous wavelet transform Wf is extremely redundant; indeed, for suitably chosen mother wavelets, the dyadic wavelet transform with λ = 2^j for $j\in \mathbb{Z}$ is a complete representation of f. However, the corresponding operator S restricted to λ = 2^j is not invertible. When one utilizes every frequency λ ∈ (0,∞), though, the resulting ${\mathbf{L}}^2(ℝ)$ norms $(Sf)(\lambda )={\Vert f\ast {\psi }_{\lambda }\Vert }_2^2$ uniquely determine the power spectrum of f, so long as the wavelet ψ satisfies a type of independence condition.

Condition 2.3

Define

$${|{\widehat{\psi }}_{\lambda }^{+}(\omega )|}^2=\left({|{\widehat{\psi }}_{\lambda }(\omega )|}^2+{|{\widehat{\psi }}_{\lambda }(-\omega )|}^2\right)\cdot \mathbf{1}(\omega ⩾0).$$

If for any finite sequence ${\left\{{\omega }_i\right\}}_{i=1}^n$ of distinct positive frequencies, the collection ${\{{|{\widehat{\psi }}_{\lambda }^{+}\left({\omega }_i\right)|}^2\}}_{i=1}^n$ is linearly independent functions of λ, we say the wavelet ψ satisfies the linear independence condition.

Remark 2.5

Condition 2.3 is stated in terms of ${|{\widehat{\psi }}_{\lambda }^{+}(\omega )|}^2$ to avoid assumptions on whether ψ is real or complex. When $\psi (x)\in ℝ$, ${|{\widehat{\psi }}_{\lambda }^{+}(\omega )|}^2=2{|{\widehat{\psi }}_{\lambda }(\omega )|}^2$ for ω ⩾ 0. When ψ is complex analytic, ${|{\widehat{\psi }}_{\lambda }^{+}(\omega )|}^2={|{\widehat{\psi }}_{\lambda }(\omega )|}^2$. When $\psi \in ℂ$ but not complex analytic, ${|{\widehat{\psi }}_{\lambda }^{+}(\omega )|}^2$ simply incorporates a reflection of ${|{\widehat{\psi }}_{\lambda }(\omega )|}^2$ about the origin. Since we assume $f(x)\in ℝ,{|{\widehat{\psi }}_{\lambda }^{+}(\omega )|}^2$ uniquely defines (Sf)(λ), since $(Sf)(\lambda )=\frac{1}{2\pi }\langle |\widehat{f}{|}^2,{|{\widehat{\psi }}_{\lambda }^{+}|}^2\rangle$ by the Plancherel and Fourier convolution theorems.

Theorem 2.4

Let $f\in {\mathbf{L}}^1\cap {\mathbf{L}}^2(ℝ)$ and assume ψ satisfies Condition 2.3 and $\widehat{\psi }$ has compact support. Then,

$$Sf=Sg\iff Pf=Pg.$$

Proof.

First assume Pf = Pg, which means ${|\widehat{f}(\omega )|}^2=|\widehat{g}(\omega ){|}^2$ for almost every $\omega \in ℝ$. Using the Plancheral and Fourier convolution theorems,

$$\begin{gathered} (Sf)(\lambda )=\int {|f*{\psi }_{\lambda }(x)|}^2\text{d}x={\frac{1}{2\pi }\int |\widehat{f}(\omega )|}^2{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega \\ =\frac{1}{2\pi }\int |\widehat{g}(\omega ){|}^2{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega =(Sg)(\lambda ),\forall \lambda \in (0,\infty ). \end{gathered}$$

Now suppose Sf = Sg. Since Sf and Sg are continuous in λ, we have

$$0=(Sf)(\lambda )-(Sg)(\lambda )=\frac{1}{2\pi }\int \left({|\widehat{f}(\omega )|}^2-|\widehat{g}(\omega ){|}^2\right){|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega ,\forall \lambda \in (0,\infty ).$$

Since $f\in {\mathbf{L}}^1\cap {\mathbf{L}}^2(ℝ)$ we have $\widehat{f}\in {\mathbf{L}}^2\cap {\mathbf{L}}^{\infty }(ℝ)$ and thus $Pf\in {\mathbf{L}}^1\cap {\mathbf{L}}^{\infty }(ℝ)$. By interpolation we have $Pf\in {\mathbf{L}}^2(ℝ)$, and the same for Pg. By applying Lemma 2.1 (stated below) with p(ω) = (Pf)(ω) − (Pg)(ω) (note p is continuous since $f,g\in {\mathbf{L}}^1(ℝ)$), we conclude Pf = Pg for almost every ω. □

Lemma 2.1

Let $p\in {\mathbf{L}}^2(ℝ)$ be continuous and assume p(ω) = p(−ω), $\widehat{\psi }$ has compact support and Condition 2.3. Then,

$$\int p(\omega ){|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega =0\forall \lambda >0\Rightarrow p=0\text{a}.\text{e}.$$

The proof of Lemma 2.1 is in Appendix C. We remark that many wavelets satisfy Condition 2.3 and have compactly supported Fourier transform, so Theorem 2.4 is broadly applicable. For example, Proposition 2.1 below proves that any complex analytic wavelet with compactly supported Fourier transform satisfies Condition 2.3. Morlet wavelets satisfy Condition 2.3 (see Lemma C.1 in Appendix C) but do not have compactly supported Fourier transform; however, $\widehat{\psi }$ does have fast decay for a Morlet wavelet and numerically we observe no issues. We also note, the assumption that $\widehat{\psi }$ has compact support in Theorem 2.4 can be removed if f, g are bandlimited. The following Proposition, proved in Appendix C, gives some sufficient conditions guaranteeing Condition 2.3.

Proposition 2.1

The following are sufficient to guarantee Condition 2.3:

1. $|\widehat{\psi }(\omega ){|}^2$ has a compact support contained in the interval [a, b], where a and b have the same sign, e.g. complex analytic wavelets with compactly supported Fourier transform.

2. $|\widehat{\psi }(\omega ){|}^2\in {\mathbf{C}}^{\infty }(ℝ)$ and there exists an N such that all derivatives of order at least N are non-zero at ω = 0, e.g. the Morlet wavelet.

Remark 2.6

In practice, Pf, Sf are implemented as discrete vectors, and Sf is obtained from Pf via matrix multiplication, i.e. Sf = F(Pf) for some real matrix F with F^TF strictly positive definite. Thus, $\Vert Pf-Pg{\Vert }_2⩽{\sigma }_{\min }^{-1}\Vert Sf-Sg{\Vert }_2$, where σ_min > 0 is the smallest singular value of the matrix F, and the spectral decay of F, which can be explicitly computed, thus determines the stability of the representation. The smoother the wavelet, the more rapidly the spectrum decays, since when Pψ ∈ C^p, F^TF is defined by a C^p kernel and thus has eigenvalues that decay like o(1/n^p+1) [20]. There is thus a tradeoff between smoothness and stability. In this article we choose smoothness over stability, since smoothness is required for unbiasing noisy dilation MRA, and in our experiments the Morlet wavelet yielded the best results. We therefore invert the representation by solving an optimization problem that is initialized to be close to the desired solution (see Section 6.5), and we avoid computing the pseudo-inverse of F, which is unstable for our smooth wavelet.

3. Unbiasing for classic MRA

In this section we consider the classic MRA model (Model 1). We discuss unbiasing results for both the power spectrum and wavelet invariants, as well as simulation results comparing the two methods. In the following proposition we establish unbiasing results for the power spectrum by rederiving some results from [14], extended to the continuum setting. The proposition is proved in Appendix D.

Proposition 3.1

Assume Model 1. Define the following estimator of (Pf)(ω):

$$(\widetilde{Pf})(\omega ):=\frac{1}{M}\sum _{j=1}^M\left(P{y}_j\right)(\omega )-{\sigma }^2.$$

Then with probability at least 1 − 1/t²,

$$|(Pf)(\omega )-(\widetilde{Pf})(\omega )|⩽\frac{2t\sigma }{\sqrt{M}}\left(\Vert f{\Vert }_1+\sigma \right).$$ (3.1)

We obtain an identical result for wavelet invariants (Proposition 3.2) when signals are corrupted by additive noise only. See Appendix D for the proof.

Proposition 3.2

Assume Model 1. Define the following estimator of (Sf)(λ):

$$(\widetilde{Sf})(\lambda ):=\frac{1}{M}\sum _{j=1}^M\left(S{y}_j\right)(\lambda )-{\sigma }^2.$$

Then with probability at least 1 − 1/t²,

$$|(Sf)(\lambda )-(\widetilde{Sf})(\lambda )|⩽\frac{2t\sigma }{\sqrt{M}}\left(\Vert f{\Vert }_1+\sigma \right).$$ (3.2)

As M → ∞, the error of both the power spectrum and wavelet invariant estimators decays to zero at the same rate, and one can perfectly unbias both representations. As demonstrated in Section 5, this is not possible for noisy dilation MRA (Model 2), as there is a non-vanishing bias term. However, a nonlinear unbiasing procedure on the wavelet invariants can significantly reduce the bias.

We illustrate and compare additive noise unbiasing for power spectrum estimation using $\left(\widetilde{Pf}\right)$, the power spectrum method of Proposition 3.1 and $(\widetilde{Sf})$, the wavelet invariant method of Proposition 3.2. To approximate (Pf) from the wavelet invariants $(\widetilde{Sf})$, we apply the convex optimization algorithm described in Section 6.5 to obtain $\left(\widetilde{P_{S}f}\right)$, the power spectrum approximation that best matches the wavelet invariants $(\widetilde{Sf})$. Thus, throughout this article, $\left(\widetilde{P_{S}f}\right)$ denotes a power spectrum estimator obtained by first unbiasing wavelet invariants and then running an optimization procedure, while $(\widetilde{Pf})$ denotes an estimator computed by directly unbiasing the power spectrum. Our simulations compare the L² error of both of these estimators, i.e. we compare $\Vert Pf-\widetilde{Pf}{\Vert }_2$ and ${\Vert Pf-\widetilde{P_{S}f}\Vert }_2$.

Figure 2(a) shows the uncorrupted power spectrum (red curve) of a medium frequency Gabor function $(f(x)=e^{-5x^2}\cos (16x))$, and the power spectrum after the signal is corrupted by additive noise with level σ = 2⁻³ (blue curve); the SNR of the experiment is 0.56 (see Section 6.1). Figure 2(b) shows the L² error of the power spectrum estimation for the two methods as a function of log₂(M) for a fixed SNR, and Fig. 2(c) shows the L² error as a function of log₂(σ) for a fixed M. The L² errors for the two methods are similar; however, estimation via wavelet invariants is advantageous when the sample size M is small or the additive noise level σ is large. As M becomes very large or σ very small, the power spectrum method is preferable as the smoothing procedure of the wavelet invariants may numerically erase some extremely small scale features of the original power spectrum.

Fig. 2.

Simulation results for additive noise model for medium frequency Gabor $f(x)=e^{-5x^2}\cos (16x)$.

4. Unbiasing for dilation MRA

In this section we analyze the dilation MRA model (Model 3). We thus assume the signals have been randomly translated and dilated but there is no additive noise.

In fact there is a simple algorithm to recover f under this model. Since ${\Vert f_{{\tau }_j}\Vert }_2^2=\Vert f{\Vert }_2^2/\left(1-{\tau }_j\right)$, $\frac{1}{M}{\sum }_{j=1}^{M}1/{\Vert f_{{\tau }_j}\Vert }_2^2$ is an unbiased estimator of $1/{\Vert f\Vert }_2^2$, and so ${\Vert f\Vert }_2^2$ can be accurately approximated. Once is recovered, one can take any signal y_j and dilate it so that ${\Vert y_j\Vert }_2^2=\Vert f{\Vert }_2^2$, and the result will be an accurate approximation of the hidden signal f for M large. However, this approach collapses in the presence of even a small amount of additive noise. In the presence of additive noise, an alternative is to attempt a synchronization by centering each signal. The center c_f of signal f can be defined in the classical way by

$$c_f=\frac{1}{\Vert f{\Vert }_2^2}\int x|f(x){|}^2\text{d}x.$$

Since the signals y_j(x − (c_f + t_j)) are perfectly aligned, one can thus attempt an alignment by defining ${\tilde{y}}_j(x)=y_j\left(x-c_{y_j}\right)$. However c_yj − (c_f + t_j) = O(σ ∨ σ² + η), so significant errors arise in the synchronization that cannot be resolved by averaging. As our goal is ultimately to produce a method that can be extended to the noisy dilation MRA model, we abandon both the trivial solution (which cannot be extended to noisy dilation MRA) and the synchronization approach (which produces large errors) and explore a method based on empirical averages.

We first observe that random dilations cause $\frac{1}{M}{\sum }_{j=1}^M\left(P{y}_j\right)(\omega )$ and $\frac{1}{M}{\sum }_{j=1}^M\left(S{y}_j\right)(\lambda )$ to be biased estimators of (Pf)(ω) and (Sf)(λ), and the bias for both is O(η²), where η² is the variance of the dilation distribution. However, if the moments of the dilation distribution are known and Pf, Sf are sufficiently smooth, one can apply an unbiasing procedure to the above estimators so that the resulting bias is O(η^k+2), where k ⩾ 2 is an even integer.

Throughout this section, we assume k ⩾ 2 is an even integer, and define the constants C_i from the first k/2 even moments of τ by $\mathbb{E}\left[{\tau }^i\right]=C_i{\eta }^i$ for i = 2, 4, . . . , k. Note since we assume $\mathbb{E}\left[{\tau }^2\right]={\eta }^2$, C₂ = 1. We define the constants B₂, B₄, . . . , B_k by solving

$$\frac{C_i}{i!}-\frac{B_2{C}_{i-2}}{(i-2)!}-\dots -\frac{B_{i-2}{C}_2}{2!}-B_i=0$$ (4.1)

for i = 2, 4, . . . , k; these constants are deterministic functions of the moments of τ. A non-recursive formula related to the Euler numbers can be derived, which defines B_i explicitly in terms of C₂, . . . , C_i; however, the recursive formula (4.1) is easier to implement numerically.

We introduce two additional moment-based constants that are defined by the C_i, B_i constants:

$$T:=\underset{i=0,2,\dots }{\max }{C}_i^{\frac{1}{i}}$$ (4.2)

$$E:=\underset{i=0,2,\dots ,k}{\max }\underset{j=0,\dots ,k+2-i}{\max }{\left(\frac{T^j}{j!}|B_i|\right)}^{\frac{1}{i+j}},$$ (4.3)

where C₀, |B₀| = 1, and when i = j = 0 in (4.3), ${\left(\frac{T^j}{j!}|B_i|\right)}^{\frac{1}{i+j}}$ is replaced with 1.

Remark 4.1

Since the distribution of τ is bounded, we are guaranteed that T < ∞, and in general can consider both T and E to be O(1) constants. For example for the uniform distribution, $T⩽\sqrt{3}$ and $|B_i|⩽\frac{|\mathrm{Euler}(i)|}{i!}⩽1$, which gives $E⩽\sqrt{3}$.

We utilize the following two lemmas, which are proved in Appendix E, to derive results for both the power spectrum and wavelet invariants.

Lemma 4.1

Let F_λ(τ) = L((1 − τ)λ) for some function L ∈ C^k+2(0, ∞) and a random variable τ satisfying the assumptions of Section 2.1, and let k ⩾ 2 be an even integer. Assume there exist functions ${\Lambda }_i:ℝ\to ℝ$, $R:ℝ\to ℝ$ such that

$$|{\lambda }^i{L}^{(i)}(\lambda )|⩽{\Lambda }_i(\lambda )\text{for}0⩽i⩽k+2,\frac{{\Lambda }_{k+2}((1-\tau )\lambda )}{{\Lambda }_{k+2}(\lambda )}⩽R(\lambda ),$$

and define the following estimator of L(λ):

$$G_{\lambda }(\tau ):=F_{\lambda }(\tau )-B_2{\eta }^2{F}_{\lambda }^{\prime \prime }(\tau )-B_4{\eta }^4{F}_{\lambda }^{(4)}(\tau )-\dots -B_k{\eta }^k{F}_{\lambda }^{(k)}(\tau ).$$

Then G_λ(τ) satisfies

$$|\mathbb{E}{G}_{\lambda }(\tau )-L(\lambda )|\lesssim kR(\lambda ){\Lambda }_{k+2}(\lambda ){(2E\eta )}^{k+2}$$

$$\mathrm{Var}{G}_{\lambda }(\tau )\lesssim k^{2}R{(\lambda )}^2\Lambda {(\lambda )}^2$$

where

$$\Lambda {(\lambda )}^2:=\sum _{0⩽i,j⩽k+2,i+j⩾2}{\Lambda }_i(\lambda ){\Lambda }_j(\lambda ){(2E\eta )}^{i+j}$$

and E is the absolute constant defined in (4.3).

Lemma 4.2

Let the assumptions and notation of Lemma 4.1 hold, and let τ₁, . . . , τ_M be independent. Define

$$\tilde{L}(\lambda ):=\frac{1}{M}\sum _{j=1}^M{G}_{\lambda }\left({\tau }_j\right).$$

Then with probability at least 1 − 1/t²

$$|\tilde{L}(\lambda )-L(\lambda )|\lesssim kR(\lambda )\left({\Lambda }_{k+2}(\lambda ){(2E\eta )}^{k+2}+\frac{t\Lambda (\lambda )}{\sqrt{M}}\right).$$

The deviation of the estimator $\tilde{L}(\lambda )$ from L(λ) thus depends on two things: (1) the bias of the estimator that is O(η^k+2) and (2) the standard deviation of the estimator that is $O\left(\eta M^{-\frac{1}{2}}\right)$, since Λ(λ) = O(η).

4.1. Power spectrum results for dilation MRA

We now show how this unbiasing procedure based on both the moments of τ and the even derivatives of Py can be used to obtain an estimator of Pf.

Proposition 4.1

Assume Model 3 and $Pf\in {\mathbf{C}}^{k+2}(ℝ)$. Define the following estimator of (Pf)(ω):

$$(\widetilde{Pf})(\omega ):=\frac{1}{M}\sum _{j=1}^M\left[\left(P{y}_j\right)(\omega )-B_2{\eta }^2{\omega }^2{\left(P{y}_j\right)}^{\prime \prime }(\omega )-\dots -B_k{\eta }^k{\lambda }^k{\left(P{y}_j\right)}^{(k)}(\omega )\right]$$

where the constants B_i satisfy (4.1). Let

$${\Omega }_i(\omega )=|{\omega }^i{(Pf)}^i(\omega )|\text{for}0⩽i⩽k+2,R(\omega )=\underset{\tau }{\max }\frac{{\Omega }_{k+2}((1-\tau )\omega )}{{\Omega }_{k+2}(\omega )}.$$

Then for all ω ≠ 0, with probability at least 1 − 1/t²,

$$|(\widetilde{Pf})(\omega )-(Pf)(\omega )|\lesssim kR(\omega )\left({\Omega }_{k+2}(\omega ){(2E\eta )}^{k+2}+\frac{t\Omega (\omega )}{\sqrt{M}}\right),$$ (4.4)

where

$$\Omega (\omega )=\sum _{0⩽i,j⩽k+2,i+j⩾2}{\Omega }_i(\omega ){\Omega }_j(\omega ){(2E\eta )}^{i+j}.$$

Proof.

Since Pf is a translation invariant representation, we can ignore the translation factors ${\left\{t_k\right\}}_{k=1}^M$ and consider the model $y_j=L_{{\tau }_j}f$. In addition since $y_j(x)\in ℝ$, (Py_j)(ω) = (Py_j)(−ω) and it is sufficient to consider ω ∈ (0, ∞). Proposition 4.1 then follows directly from Lemma 4.2 with λ = ω, L = Pf since (Py_j)(ω) = (Pf)((1 − τ_j)ω) = F_ω(τ_j), Λ_i = Ω_i, and Λ = Ω. □

We postpone a discussion of the shortcomings of Proposition 4.1 to Section 4.3, where we compare the power spectrum and wavelet invariant results for dilation MRA.

4.2. Wavelet invariant results for dilation MRA

We now apply the same unbiasing procedure to the wavelet invariants. Unlike for the power spectrum, where the error may depend on the frequency ω (see (4.4) and Section 4.3), the wavelet invariant error can be uniformly bounded independently of λ with high probability. The following two Lemmas establish bounds on the derivatives of (Sf)(λ) and are needed to prove Proposition 4.2; they are proved in Appendix B.

Lemma 4.3 (Low-frequency bound).

Assume $P\psi \in {\mathbf{C}}^m(ℝ)$ and $f\in {\mathbf{L}}^1(ℝ)$. Then the quantity |λ^m(Sf)^(m)(λ)| can be bounded uniformly over all λ. Specifically:

$$|{\lambda }^m{(Sf)}^{(m)}(\lambda )|⩽{\Psi }_m\Vert f{\Vert }_1^2$$

for Ψ_m defined in (A1).

Lemma 4.4 (High-frequency bound for differentiable functions).

Assume $P\psi \in {\mathbf{C}}^m(ℝ)$, and $f^{\prime }\in {\mathbf{L}}^1(ℝ)$. Then the quantity |λ^m(Sf)^(m)(λ)| can be bounded by

$$|{\lambda }^m{(Sf)}^{(m)}(\lambda )|⩽\frac{{\Theta }_m}{{\lambda }^2}{\Vert f^{\prime }\Vert }_1^2$$

for Θ_m defined in (A2).

When ψ is a Morlet wavelet or more generally when ψ is (k + 2)-admissible as described in Appendix A, these lemmas allow one to bound the error of the order k wavelet invariant estimator for dilation MRA in terms of the following quantities:

$${\Lambda }_i(\lambda )={\Psi }_i\Vert f{\Vert }_1^2\wedge \frac{{\Theta }_i}{{\lambda }^2}{\Vert f^{\prime }\Vert }_1^2,\Lambda {(\lambda )}^2=\sum _{0⩽i,j⩽k+2,i+j⩾2}{\Lambda }_i(\lambda ){\Lambda }_j(\lambda ){(2E\eta )}^{i+j},$$ (4.5)

where Ψ_i, Θ_i are defined in (A1), (A2) and E is defined in (4.3).

Proposition 4.2

Assume Model 3, the notation in (4.5), and that ψ is (k + 2)-admissible. Define the following estimator of (Sf)(λ):

$$(\widetilde{Sf})(\lambda ):=\frac{1}{M}\sum _{j=1}^M\left[\left(S{y}_j\right)(\lambda )-B_2{\eta }^2{\lambda }^2{\left(S{y}_j\right)}^{\prime \prime }(\lambda )-\dots -B_k{\eta }^k{\lambda }^k{\left(S{y}_j\right)}^{(k)}(\lambda )\right]$$

where the constants B_i satisfy (4.1). Then with probability at least 1 − 1/t²,

$$|(\widetilde{Sf})(\lambda )-(Sf)(\lambda )|\lesssim k\left({\Lambda }_{k+2}(\lambda ){(2E\eta )}^{k+2}+\frac{t\Lambda (\lambda )}{\sqrt{M}}\right).$$

Proof.

Since Sf is a translation invariant representation, we can ignore the translation factors ${\left\{t_k\right\}}_{k=1}^M$ and consider the model $y_j=L_{{\tau }_j}f$. Since ψ is k + 2-admissible, $\widehat{\psi }\in {\mathbf{C}}^{k+2}(ℝ)$, which guarantees(Sf)(λ) ∈ C^k+2(0, ∞). We note that since $f\in {\mathbf{L}}^1(ℝ)$, Pf is continuous, and the Leibniz integral rule guarantees that ${(Sf)}^{(n)}(\lambda )=\frac{d^n}{d{\lambda }^n}(Sf)(\lambda )$ for 1 ⩽ n⩽ k + 2. By applying Lemma 4.3, we have $|{\lambda }^i{(Sf)}^{(i)}(\lambda )|⩽{\Psi }_i\Vert f{\Vert }_1^2$ for all 0 ⩽ i ⩽ k + 2, so that Lemma 4.2 holds for L(λ) = (Sf)(λ), ${\Lambda }_i(\lambda )={\Psi }_i\Vert f{\Vert }_1^2$, and R(λ) = 1. Now by applying Lemma 4.4, we have $|{\lambda }^i{(Sf)}^{(i)}(\lambda )|⩽{\Theta }_i{\Vert f^{\prime }\Vert }_1^2/{\lambda }^2$ for all 0 ⩽ i ⩽ k + 2, so that Lemma 4.2 also holds for L(λ) = (Sf)(λ), ${\Lambda }_i(\lambda )={\Theta }_i{\Vert f^{\prime }\Vert }_1^2/{\lambda }^2$, and R(λ) = 4 (note since $|\tau |⩽\frac{1}{2}$, ${\Lambda }_{k+2}((1-\tau )\lambda )/{\Lambda }_{k+2}(\lambda )⩽4$). Thus, Lemma 4.2 in fact holds with ${\Lambda }_i(\lambda )=\left({\Psi }_i\Vert f{\Vert }_1^2\wedge \frac{{\Theta }_i}{{\lambda }^2}{\Vert f^{\prime }\Vert }_1^2\right)$; since (Sy_j)(λ) = (Sf)((1 − τ_j)λ) = F_λ(τ_j), we obtain Proposition 4.2 □

Since ${\Lambda }_i(\lambda )⩽{\Psi }_i\Vert f{\Vert }_1^2$, Proposition 4.2 guarantees that the error can be uniformly bounded independent of λ. In addition, if the signal is smooth, the error for high-frequency λ will have the favorable scaling λ⁻². An important question in practice is how to choose k, i.e. what order wavelet invariant estimator minimizes the bias. Consider for example when $f^{\prime }\notin {\mathbf{L}}^1(ℝ)$, and ${\Lambda }_{k+2}(\lambda )={\Psi }_{k+2}\Vert f{\Vert }_1^2$. By using a second-order estimator, we can decrease the bias from O(η²) to O(η⁴), and we can further decrease the bias to O(η⁶) by choosing k = 4. However, Ψ_k increases very rapidly in k. Indeed, as can be seen from (A1), Ψ_k increases like k!. Thus, one possible heuristic (assuming η is known) is to choose $k=\tilde{k}$ where $\tilde{k}$ minimizes the bias upper bound kΨ_k+2(2Eη)^k+2. Since Ψ_k increases factorially, Ψ_k ∼ (C_k)^k for some constant C, and $\tilde{k}+2$ will be inversely proportional to η, that is $(\tilde{k}+2)~{\eta }^{-1}$. The following corollary of Proposition 4.2 then holds for any $k⩽\tilde{k}$.

Corollary 4.1

Under the assumptions of Proposition 4.2, if Ψ_i(2Eη)ⁱ is decreasing for i ⩽ k + 2, then with probability at least 1 − 1/t²:

$$|(\widetilde{Sf})(\lambda )-(Sf)(\lambda )|\lesssim \Vert f{\Vert }_1^2\left(k{\Psi }_{k+2}{(2E\eta )}^{k+2}+\frac{t{k}^2\eta }{\sqrt{M}}\right).$$ (4.6)

Similarly, if Θ_i(2Eη)ⁱ is decreasing for i ⩽ k + 2, then with probability at least 1 − 1/t²:

$$|(\widetilde{Sf})(\lambda )-(Sf)(\lambda )|\lesssim \frac{{\Vert f^{\prime }\Vert }_1^2}{{\lambda }^2}\left(k{\Theta }_{k+2}{(2E\eta )}^{k+2}+\frac{t{k}^2\eta }{\sqrt{M}}\right).$$ (4.7)

Remark 4.2

We observe that for a discrete lattice I of λ values, we can define the discrete 1-norm by $\Vert g{\Vert }_{{\mathbf{L}}^1(I)}={\sum }_{\lambda \in I}|g(\lambda )|\Delta \lambda$. Assume the lattice has cardinality n, and that Ψ_i(2Eη)ⁱ, Θ_i(2Eη)ⁱ are decreasing for i ⩽ k + 2. Applying Proposition 4.2 with $t=\sqrt{n}s$ and a union bound over the lattice gives

$$\Vert \widetilde{Sf}-Sf{\Vert }_{{\mathbf{L}}^1(I)}\lesssim k\left(\Vert f{\Vert }_1^2{\Psi }_{k+2}+{\Vert f^{\prime }\Vert }_1^2{\Theta }_{k+2}\right){(2E\eta )}^{k+2}+\frac{s\sqrt{n}{k}^2\eta }{\sqrt{M}}\left(\Vert f{\Vert }_1^2+{\Vert f^{\prime }\Vert }_1^2\right)$$

with probability at least 1 − 1/s². When n ≪ M, which is the context for MRA, the 1-norm of the error is O(η^k+2) as M → ∞.

4.3. Comparison

Although Propositions 4.2 and 4.1 at first glance appear quite similar, the wavelet invariant method has several important advantages over the power spectrum method, which we enumerate in the following remarks.

Remark 4.3

Proposition 4.2 (wavelet invariants) applies to any signal satisfying $f\in {\mathbf{L}}^1(ℝ)$, but Proposition 4.1 requires $Pf\in {\mathbf{C}}^{k+2}(ℝ)$. Thus, as k is increased, the power spectrum results apply to an increasingly restrictive function class. Furthermore, as discussed in Section 5, if the signal contains any additive noise, Py_j is not even C¹, which means the unbiasing procedure of Proposition 4.1 cannot be applied. On the other hand, by choosing $P\psi \in {\mathbf{C}}^{\infty }(ℝ)$, Sf will inherit the smoothness of the wavelet, and the wavelet invariant results will hold for any $f\in {\mathbf{L}}^1(ℝ)$ and any k.

Remark 4.4

Since (Pf_τ)(ξ) = (Pf)((1 − τ)ξ), dilation will transport the frequency content at ξ to (1 − τ)ξ, so that the displacement is τξ. Thus, when ξ is very large, |(Pf)(ξ) − (Pf_τ)(ξ)| can be large even for τ small. Because the wavelet invariants bin the frequency content, and these bins become increasingly large in the high frequencies, this does not occur for wavelet invariants. More specifically, there is always a signal f and frequency ξ for which $|(Pf)(\xi )-(\widetilde{Pf})(\xi )|$ is large regardless of k. Consider for example when $(Pf)(\omega )=e^{-{(\omega -\xi )}^2}$. Then Ω_k(ξ) ∼ ξ^k, and $|(Pf)(\xi )-\left(\widetilde{Pf}\right)(\xi )|\gtrsim 1$. However, for M large enough, the order k wavelet invariant estimator satisfies $|(Sf)(\lambda )-(\widetilde{Sf})(\lambda )|=O\left(k{\Psi }_{k+2}{\eta }^{k+2}\right)$ for all λ. The wavelet invariants are thus stable for high-frequency signals, where the power spectrum fails.

Remark 4.5

For the wavelet invariants there will be a unique $\tilde{k}$ that minimizes kΨ_k+2(2Eη)^k+2, and $\tilde{k}$ does not depend on λ. Furthermore, $\tilde{k}$ can be explicitly computed given the wavelet ψ and moment constant E. On the other hand, the minimum of kΩ_k+2(ω)(2Eω)^k+2 with respect to k will depend on both the frequency ω and the signal f, so that $\tilde{k}=\tilde{k}(\omega ,f)$, and it becomes unclear how to choose the unbiasing order.

4.4. Simulation results for dilation MRA

We first illustrate the unbiasing procedure of Propositions 4.1 and 4.2 for the high-frequency signal $f(x)=e^{-5x^2}\cos (32x)$. Figure 3 shows the power spectrum estimator $\widetilde{Pf}$ and the wavelet invariant estimator $\widetilde{P_{S}f}$ for k = 0, 2, 4 for both small and large dilations, where $\widetilde{P_{S}f}$ denotes the combined wavelet invariant unbiasing plus optimization procedure (see Section 6.5). Higher order unbiasing is beneficial for both methods for small dilations but fails for the power spectrum for large dilations. Both methods will of course fail for η large enough, but for high-frequency signals the power spectrum fails much sooner.

Fig. 3.

Order k = 0, 2, 4 power spectrum estimators $\widetilde{Pf}$ (first two figures) and wavelet invariant estimators $\widetilde{P_{S}f}$ (last two figures) for the signal $f_3(x)=e^{-5x^2}\cos (32x)$. Figures 3(a) and 3(c) show small dilations and Figs 3(b) and 3(d) show large dilations.

Next we compare ${\Vert Pf-\widetilde{Pf}\Vert }_2$ and ${\Vert Pf-\widetilde{P_{S}f}\Vert }_2$, the L² error of estimating the power spectrum of the target signal via the power spectrum estimators of Proposition 4.1 and via the wavelet invariant estimators of Proposition 4.2, followed by a convex optimization procedure. We consider order k = 0, 2, 4 estimators for both the power spectrum and wavelet invariants on the following Gabor atoms of increasing frequency:

$$f_1(x)=e^{-5x^2}\cos (8x)$$

$$f_2(x)=e^{-5x^2}\cos (16x)$$

$$f_3(x)=e^{-5x^2}\cos (32x).$$

These functions satisfy f = Real(h) where $(Ph)(\omega )=(\pi /5)e^{-{(\omega -\xi )}^2/10}$ for ξ = 8, 16, 32, and thus exhibit the behavior described in Remark 4.4.

Simulation results are shown in Fig. 4; the horizontal axis shows log₂(M) while the vertical axis shows log₂(Error). For each value of M, the error was calculated for 10 independent simulations and then averaged. The unbiasing procedure of Propositions 4.1 and 4.2 requires knowledge of the moments of the dilation distribution, but in practice these are unknown. Thus, the first two even moments of the dilation distribution (η²,C₄η⁴) were estimated empirically with the fourth-order estimators described in Section 6.3 (see Definition 6.1). For the low-frequency signal, the fourth-order power spectrum estimator was best for both small and large dilations and is preferable due to the lower computational cost (see Remark 2.3). For the high-frequency signal, the fourth-order wavelet invariant estimator was best for large dilations and WSC k = 2 and k = 4 were best and equivalent for small dilations. For the medium-frequency signal, the higher order power spectrum estimators were best for small dilations while the higher order wavelet invariant estimators were best for large dilations. Thus, the simulation results confirm that the wavelet invariants will have an advantage over Fourier invariants when the signals are either high frequency or corrupted by large dilations. We remark that one obtains nearly identical error plots with oracle knowledge of the dilation moments, indicating that the empirical moment estimation procedure is highly accurate in the absence of additive noise, even for small M values.

Fig. 4.

L² error with standard error bars for dilation model (empirical moment estimation). Top row shows results for small dilations (η = 0.06) and bottom row shows results for large dilations (η = 0.12). First, second, third column shows results for low, medium, high frequency Gabor signals. All plots have the same axis limits.

5. Noisy dilation MRA model

Finally, we consider the noisy dilation MRA model (Model 2) where signals are randomly translated and dilated and corrupted by additive noise. Section 5.1 gives unbiasing results for wavelet invariants and Section 5.2 reports relevant simulations.

5.1. Wavelet inariant results for noisy dilation MRA

To state Proposition 5.1 as succinctly as possible, we also define the following quantity

$$\Psi :=\sum _{m=0,2,\dots ,k}{\Psi }_m{(E\eta )}^m,$$ (5.1)

where E is defined in (4.3) and Ψ_m is defined in (A1).

Proposition 5.1

Assume Model 2 and that ψ is (k + 2)-admissible. Define the following estimator of (Sf)(λ):

$$(\widetilde{Sf})(\lambda ):=\frac{1}{M}\sum _{j=1}^M\left[\left(S{y}_j\right)(\lambda )-B_2{\eta }^2{\lambda }^2{\left(S{y}_j\right)}^{\prime \prime }(\lambda )-\dots -B_k{\eta }^k{\lambda }^k{\left(S{y}_j\right)}^{(k)}(\lambda )\right]-{\sigma }^2$$

where the constants B_i satisfy (4.1). Then with probability at least 1 − 1/t²

$$|(\widetilde{Sf})(\lambda )-(Sf)(\lambda )|\lesssim k{\Lambda }_{k+2}(\lambda ){(2E\eta )}^{k+2}+\frac{t}{\sqrt{M}}\left[k\Lambda (\lambda )+\Psi {\sigma }^2+\sqrt{\Psi \left({\Lambda }_0(\lambda )+\Lambda (\lambda )\right)}\sigma \right],$$ (5.2)

where E, Λ(λ), Ψ are as defined in (4.3), (4.5), (5.1).

The following corollary is an immediate consequence of Proposition 5.1.

Corollary 5.1

Let the assumptions of Proposition 5.1 hold, and in addition assume Ψ_i(2Eη)ⁱ is decreasing for i ⩽ k + 2. Then with probability at least 1 − 1/t²

$$|(\widetilde{Sf})(\lambda )-(Sf)(\lambda )|\lesssim k{\Psi }_{k+2}{(2E\eta )}^{k+2}\Vert f{\Vert }_1^2+\frac{tk}{\sqrt{M}}\left[k\eta \Vert f{\Vert }_1^2+\sigma \Vert f{\Vert }_1+{\sigma }^2\right].$$ (5.3)

We remark that there are two components to the estimation error bounded by the right-hand side of (5.3): the first two terms are the error due to dilation, as in Corollary 4.1 of Proposition 4.2, and the last two terms are the error due to additive noise, as given in Proposition 3.2. Thus, the wavelet invariant representation allows for a decomposition of the error of the noisy dilation MRA model into the sum of the errors of the random dilation model and the additive noise model. This is possible because the representation inherits the differentiability of the wavelet and is not possible when $P\psi \notin {\mathbf{C}}^k(ℝ)$, in which case the dilation unbiasing procedure has a more complicated effect on the additive noise. A result equivalent to Proposition 5.1 cannot be made for the power spectrum, because the nonlinear unbiasing procedure of Proposition 4.1 cannot be applied to the power spectra of signals from the noisy dilation MRA corruption model, since they are not differentiable in the presence of additive noise.

Proof of Proposition 5.1.

Since Sf is a translation invariant representation, we can ignore the translation factors ${\left\{t_j\right\}}_{j=1}^M$ and consider the model $y_j=f_{{\tau }_j}+{\epsilon }_j$. For notational convenience, we define the following order k derivative ‘unbiasing’ operator:

$$A_{\lambda }g(\lambda ):=g(\lambda )-B_2{\eta }^2{\lambda }^2\frac{d}{d{\lambda }^2}g(\lambda )-\dots -B_k{\eta }^k{\lambda }^k\frac{d}{d{\lambda }^k}g(\lambda ),$$ (5.4)

which is defined on any function of λ, so that we can express our estimator by

$$\begin{gathered} (\widetilde{Sf})(\lambda )=\frac{1}{M}\sum _{j=1}^M\left[\frac{1}{2\pi }\int {|{\widehat{y}}_j(\omega )|}^2{A}_{\lambda }{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega \right]-{\sigma }^2 \\ =\frac{1}{M}\sum _{j=1}^M\left[\frac{1}{2\pi }\int (|{{\widehat{f}}_{{\tau }_j}(\omega )|}^2+{\widehat{f}}_{{\tau }_j}(\omega ){\overline{\widehat{\epsilon }}}_j(\omega )+{\overline{\widehat{f}}}_{{\tau }_j}(\omega ){\widehat{\epsilon }}_j(\omega )+{|{\widehat{\epsilon }}_j(\omega )|}^2)A_{\lambda }{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega \right]-{\sigma }^2. \end{gathered}$$

We can thus decompose the error as follows:

$$\begin{array}{l}|(\widetilde{Sf})(\lambda )-(Sf)(\lambda )|⩽\underset{\text{Cross}\text{Term}\text{Error}}{\underbrace{{\left|\frac{1}{M}\sum _{j=1}^M\frac{1}{2\pi }\int \left({\widehat{f}}_{{\tau }_j}(\omega ){\overline{\widehat{\epsilon }}}_j(\omega )+{\overline{\widehat{f}}}_{{\tau }_j}(\omega ){\widehat{\epsilon }}_j(\omega )\right)A_{\lambda }\right|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega \mid }}\mid \\ +\underset{\text{Dilation}\text{Error}}{\underbrace{{\left|\frac{1}{M}\sum _{j=1}^M\frac{1}{2\pi }\int |{\widehat{f}}_{{\tau }_j}(\omega )\right|}^2{A}_{\lambda }{\left|{\widehat{\psi }}_{\lambda }(\omega )\right|}^2\text{d}\omega -(Sf)(\lambda )\mid }}+\underset{\text{Additive}\text{Noise}\text{Error}}{\underbrace{{\left|\frac{1}{M}\sum _{j=1}^M\frac{1}{2\pi }\int \right|{\widehat{\epsilon }}_j(\omega )|}^2{A}_{\lambda }{\left|{\widehat{\psi }}_{\lambda }(\omega )\right|}^2\text{d}\omega -{\sigma }^2\mid }}.\end{array}$$

To bound the above terms we utilize the following two Lemmas, which are proved in Appendix F.

Lemma 5.1

Let the notation and assumptions of Proposition 5.1 hold, and let A_λ be the operator defined in (5.4). Then with probability at least 1 − 1/t²

$${\left|\frac{1}{M}\sum _{j=1}^M\frac{1}{2\pi }\int \right|{\widehat{\epsilon }}_j(\omega )|}^2{A}_{\lambda }{\left|{\widehat{\psi }}_{\lambda }(\omega )\right|}^2\text{d}\omega -{\sigma }^2\mid ⩽\frac{2t\sqrt{k}\Psi {\sigma }^2}{\sqrt{M}}.$$

Lemma 5.2

Let the notation and assumptions of Proposition 5.1 hold, and let A_λ be the operator defined in (5.4). Then with probability at least 1 − 1/t²

$${|\frac{1}{M}\sum _{j=1}^M\frac{1}{2\pi }\int \left({\widehat{f}}_{{\tau }_j}(\omega ){\overline{\widehat{\epsilon }}}_j(\omega )+{\overline{\widehat{f}}}_{{\tau }_j}(\omega ){\widehat{\epsilon }}_j(\omega )\right)A_{\lambda }|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega |\lesssim \frac{t}{\sqrt{M}}\sqrt{\Psi \left({\Lambda }_0(\lambda )+\Lambda (\lambda )\right)}\sigma .$$

Applying Proposition 4.2 to bound the dilation error, Lemma 5.1 to bound the additive noise error, and Lemma 5.2 to bound the cross term error gives (5.2). □

5.2. Simulation results for noisy dilation MRA

We once again consider the Gabor atoms of varying frequency introduced in Section 4.4, and compare the L² error of estimating the power spectrum by (1) averaging the power spectra of the noisy signals, and applying additive noise unbiasing; this is the zero-order power spectrum method (PS k = 0), defined in Proposition 3.1, and (2) by approximating the wavelet invariants by the estimators given in Proposition 5.1 for k = 0, 2, 4, and then applying the optimization procedure described in Section 6.5; we refer to these methods as WSC k = i for i = 0, 2, 4. We emphasize that for the noisy dilation MRA model, it is impossible to define higher order methods for the power spectrum.

We first consider the errors obtained given oracle knowledge of the noise moments, both additive and dilation. Results are shown in Fig. 5 for all parameter combinations resulting from σ = 2⁻⁴, 2⁻³ (giving SNR = 2.2, 0.56) and η = 0.06, 0.12. The horizontal axis shows log₂(M) and the vertical axis shows log₂(Error); for each value of M, the error was calculated for 10 independent simulations and then averaged. For all simulations τ was given a uniform distribution, a challenging regime for dilations, and the sample size ranged over 16 ⩽ M ⩽ 131, 072. For the medium- and high-frequency signals, for large enough M, WSC k = 2 and WSC k = 4 have significantly smaller error than the order zero estimators, indicating that the nonlinear unbiasing procedure of Proposition 5.1 contributes a definitive advantage. For the high-frequency signal and large M, the error using WSC k = 4 is decreased by a factor of about 3 from the PS k = 0 error. For small dilations (η = 0.06), there is not much of a difference in performance between WSC k = 2 and WSC k = 4, but the gap between these estimators widens for large dilations (η = 0.12), as the fourth-order correction becomes more important. For the low-frequency signal under small dilations, PS k = 0 achieves the smallest error for large M. However, when M is small or the dilations are large, the WSC estimators have the advantage for the low-frequency signal as well, and WSC k = 4 is once again the best estimator for large M.

Fig. 5.

L² error with standard error bars for noisy dilation MRA model (oracle moment estimation). First, second and third column shows results for low-, medium- and high-frequency Gabor signals. All plots have the same axis limits.

We note that although in general recovering the power spectrum is insufficient for recovering the signal, the signal can be recovered when $\widehat{f}(\omega )\in ℝ$ and $\widehat{f}(\omega )⩾0$ by taking the inverse Fourier transform of the root power spectrum. Figure 6 shows the approximate signals recovered by this procedure from PS k = 0 (Fig. 6(c)) and WSC k = 4 (Fig. 6(b)) for the high-frequency Gabor signal f₃(x) (Fig. 6(a)). The WSC-recovered signal is a much better approximation of the target signal. The recovered power spectra are shown in Fig. 6(d); PS k = 0 is much flatter than the target power spectrum, while WSC k = 4 is a good approximation of both the shape and height of the target power spectrum.

Fig. 6.

Signal recovery results for $f_3(x)=e^{-5x^2}\cos (32x)$ with M = 20, 000, η = 0.12, SNR = 2.2.

Appendix G outlines an empirical procedure for estimating the moments of τ in the special case when t = 0 in the noisy dilation MRA model (i.e. no random translations). All simulations reported in Fig. 5 are repeated (with minor modifications) with empirical additive and dilation moment estimation, and the results are reported in Fig. G7 of Appendix G.

Appendix H contains additional simulation results for a variety of high-frequency signals.

Remark 5.1

One could also solve noisy dilation MRA with an EM algorithm. Appendix I describes how the method proposed in [1] can be extended to solve Model 2. Although EM algorithms provide a flexible tool for accurate parameter estimation in a variety of MRA models, the primary disadvantage is the high computational cost of each iteration. Each iteration costs O(Mn³), while wavelet invariant estimators can be computed in O(Mn²). In addition the statistical priors chosen may bias the signal reconstruction [12], and the algorithm will generally only converge to a local maximum. In this article we thus explore whether it is possible to solve noisy dilation MRA more efficiently and accurately by nonlinear unbiasing procedures.

6. Numerical implementation

In this section we describe the numerical implementation of the proposed method used to generate the results reported in Sections 3, 4.4 and 5.2. Section 6.1 describes how signals were generated, and Sections 6.2 and 6.3 describe empirical procedures for estimating the additive noise level and the moments of the dilation distribution τ. Finally, Section 6.4 discusses how the derivatives used for unbiasing were computed, and Section 6.5 describes the convex optimization algorithm used to recover Pf from Sf. All simulations used a Morlet wavelet constructed with ξ = 3π/4.

6.1. Signal generation and SNR

All signals were defined on [−N/4, N/4] and then padded with zeros to obtain a signal defined on [−N/2, N/2]; the additive noise was also defined on [−N/2, N/2]. Signals were sampled at a rate of 1/2^ℓ, thus resolving frequencies in the interval [−2^ℓπ, 2^ℓπ] with a frequency sampling rate of 2π/N. We used N = 2⁵ and ℓ = 5 in all experiments, keeping the box size and resolution fixed. For each experiment with hidden signal f, the SNR was calculated by $\text{SNR}=\left(\frac{1}{N}{\int }_{-N/2}^{N/2}f{(x)}^2\text{d}x\right)/{\sigma }^2$.

6.2. Empirical estimation of additive noise level

The additive noise level σ² can be estimated from the mean vertical shift of the mean power spectrum $\frac{1}{M}{\sum }_{j=1}^M{|{\widehat{y}}_j(\omega )|}^2$ in the tails of the distribution. Specifically, for Σ = [−2^ℓπ, 2^ℓπ] \ [−2^ℓ−1π, 2^ℓ−1π], we define

$${\tilde{\sigma }}^2=\frac{1}{|\Sigma |}\sum _{\omega \in \Sigma }\frac{1}{M}\sum _{j=1}^M{|{\widehat{y}}_j(\omega )|}^2.$$

If we choose ℓ large enough so that the target signal frequencies are essentially contained in the interval $\left[-2^{\ell -1}\pi ,2^{\ell -1}\pi \right],{|{\widehat{y}}_j(\omega )|}^2={|{\widehat{\epsilon }}_j(\omega )|}^2$ for ω ∈ Σ, and this is a robust and unbiased estimation procedure since $\mathbb{E}{|{\widehat{\epsilon }}_j(\omega )|}^2={\sigma }^2$ by Lemma D.1.

6.3. Empirical moment estimation for dilation MRA

Given the additive noise level, the moments of the dilation distribution τ for dilation MRA (Model 3) can be empirically estimated from the mean and variance of the random variables α_m(y_j) defined by

$${\alpha }_m\left(y_j\right)={\int }_0^{2^{\ell }\pi }{\omega }^m{|{\widehat{y}}_j(\omega )|}^2\text{d}\omega$$ (6.1)

for integer m ⩾ 0. More specifically, we define the order m squared coefficient of variation by

$$C{V}_m:=\frac{\mathrm{Var}\left[{\alpha }_m\left(y_j\right)\right]}{{|\mathbb{E}\left[{\alpha }_m\left(y_j\right)\right]|}^2}.$$ (6.2)

The following proposition guarantees that for M large the second and fourth moments of the dilation distribution can be recovered from CV₀, CV₁. In fact one could continue this procedure for higher m values, i.e. ${\left\{C{V}_m\right\}}_{m=0}^{k/2-1}$ will define estimators of the first $\frac{k}{2}$ even moments of τ, accurate up to O(η^k+2), but for brevity we omit the general case.

Proposition 6.1

Assume Model 3 and CV₀, CV₁ defined by (6.1) and (6.2). Then

$$C{V}_0={\eta }^2+\left(3C_4-3\right){\eta }^4+O\left({\eta }^6\right)$$

$$C{V}_1=4{\eta }^2+\left(25C_4-33\right){\eta }^4+O\left({\eta }^6\right).$$

Proof.

Since $y_j=L_{{\tau }_j}f\left(x-t_j\right)$,

$$\begin{gathered} {\alpha }_m\left(y_j\right)={{\int }_0^{2^{\ell }\pi }{\omega }^m|\widehat{f}\left(\left(1-{\tau }_j\right)\omega \right)|}^2\text{d}\omega \\ ={{\int }_0^{2^{\ell }\pi \left(1-{\tau }_j\right)}\frac{{\xi }^m}{{\left(1-{\tau }_j\right)}^m}|\widehat{f}(\xi )|}^2\frac{\text{d}\xi }{\left(1-{\tau }_j\right)} \\ ={\left(1-{\tau }_j\right)}^{-(m+1)}{\alpha }_m(f), \end{gathered}$$

where we assume we have chosen ℓ large enough so that the target signal frequencies are essentially supported in [−2^ℓ−1π, 2^ℓ−1π]. Thus,

$$C{V}_m=\frac{\mathbb{E}\left[{\alpha }_m{\left(y_j\right)}^2\right]-{\left(\mathbb{E}\left[{\alpha }_m\left(y_j\right)\right]\right)}^2}{{\left(\mathbb{E}\left[{\alpha }_m\left(y_j\right)\right]\right)}^2}=\frac{\mathbb{E}\left[{\left(1-{\tau }_j\right)}^{-2(m+1)}\right]}{{\left(\mathbb{E}\left[{\left(1-{\tau }_j\right)}^{-(m+1)}\right]\right)}^2}-1.$$

When m = 0, we have

$$\begin{gathered} C{V}_0=\frac{\mathbb{E}\left[{\left(1-{\tau }_j\right)}^{-2}\right]}{{\left(\mathbb{E}\left[{\left(1-{\tau }_j\right)}^{-1}\right]\right)}^2}-1 \\ =\frac{\mathbb{E}\left[1+2\tau +3{\tau }^2+4{\tau }^3+5{\tau }^4+O\left({\tau }^5\right)\right]}{{\left(\mathbb{E}\left[1+\tau +{\tau }^2+{\tau }^3+{\tau }^4+O\left({\tau }^5\right)\right]\right)}^2}-1 \\ =\frac{1+3{\eta }^2+5C_4{\eta }^4+O\left({\eta }^6\right)}{{\left(1+{\eta }^2+C_4{\eta }^4+O\left({\eta }^6\right)\right)}^2}-1 \\ =\frac{1+3{\eta }^2+5C_4{\eta }^4+O\left({\eta }^6\right))}{1+2{\eta }^2+\left(2C_4+1\right){\eta }^4+O\left({\eta }^6\right)}-1 \\ =\left(1+3{\eta }^2+5C_4{\eta }^4+O\left({\eta }^6\right)\right)\left(1-2{\eta }^2+\left(3-2C_4\right){\eta }^4+O\left({\eta }^6\right)\right)-1 \\ ={\eta }^2+\left(3C_4-3\right){\eta }^4+O\left({\eta }^6\right). \end{gathered}$$

When m = 1, we have

$$\begin{gathered} C{V}_1=\frac{\mathbb{E}\left[{\left(1-{\tau }_j\right)}^{-4}\right]}{{\left(\mathbb{E}\left[{\left(1-{\tau }_j\right)}^{-2}\right]\right)}^2}-1 \\ =\frac{\mathbb{E}\left[1+4\tau +10{\tau }^2+20{\tau }^3+35{\tau }^4+O\left({\tau }^5\right)\right]}{{\left(\mathbb{E}\left[1+2\tau +3{\tau }^2+4{\tau }^3+5{\tau }^4+O\left({\tau }^5\right)\right]\right)}^2}-1 \\ =\frac{1+10{\eta }^2+35C_4{\eta }^4+O\left({\eta }^6\right)}{{\left(1+3{\eta }^2+5C_4{\eta }^4+O\left({\eta }^6\right)\right)}^2}-1 \\ =\frac{1+10{\eta }^2+35C_4{\eta }^4+O\left({\eta }^6\right)}{\left(1+6{\eta }^2+\left(9+10C_4\right){\eta }^4+O\left({\eta }^6\right)\right)}-1 \\ =\left(1+10{\eta }^2+35C_4{\eta }^4+O\left({\eta }^6\right)\right)\left(1-6{\eta }^2+\left(27-10C_4\right){\eta }^4+O\left({\eta }^6\right)\right)-1 \\ =4{\eta }^2+\left(25C_4-33\right){\eta }^4+O\left({\eta }^6\right). \end{gathered}$$

We cannot compute CV_m exactly, but by replacing Var, $\mathbb{E}$ with their finite sample estimators, we obtain an approximate ${\widetilde{CV}}_m\to C{V}_m$ as M → ∞. Motivated by Proposition G.1, we thus use ${\widetilde{CV}}_0$, ${\widetilde{CV}}_1$ to define estimators of η² and C₄η⁴.

Definition 6.1

Assume Model 3 and let ${\widetilde{CV}}_0$, ${\widetilde{CV}}_1$ be the empirical versions of (6.2). Define the second-order estimator of η² by ${\tilde{\eta }}^2={\widetilde{CV}}_0$. Define the fourth-order estimators of (η², C₄η⁴) by the unique positive solution (${\tilde{\eta }}^2$, ${\tilde{C}}_4$) of

$${\widetilde{CV}}_0={\eta }^2+\left(3C_4-3\right){\eta }^4$$

$${\widetilde{CV}}_1=4{\eta }^2+\left(25C_4-33\right){\eta }^4.$$

For noisy dilation MRA (Model 2), estimating the dilation moments is more difficult. We give a procedure for estimating the moments in the special case t = 0 in Appendix G. Empirical moment estimation procedures that are simultaneously robust to translations, dilations and additive noise are an important area of future research.

6.4. Derivatives

All derivatives were approximated numerically using finite difference calculations. A sixth-order finite difference approximation was used for second derivatives, and a fourth-order finite difference approximation was used for fourth derivatives. This procedure was done on the empirical mean for each representation, not the individual signals. In fact since the wavelet is known, $\frac{d^n}{d{\lambda }^n}{|{\widehat{\psi }}_{\lambda }(\omega )|}^2$ could be computed analytically, and (Sy_j)⁽ⁿ⁾(λ) computed using Definition 2.2. Thus error due to finite difference approximations could be avoided for wavelet invariant derivatives.

6.5. Optimization

In this section we describe the convex optimization algorithm for computing $\left(\widetilde{P_{S}f}\right)$, the power spectrum approximation that best matches the wavelet invariants $(\widetilde{Sf})$. Since the wavelet invariants are only computed for λ > 0, we also incorporate zero frequency information into the loss function via $(\widetilde{Pf})(0)$, an approximation of the power spectrum at frequency zero. For all of the examples reported in this article, the quasi-newton algorithm was used to solve an unconstrained optimization problem minimizing the following convex loss function:

$$\mathrm{loss}(\widehat{g}):=\sum _{\lambda }{\left(\langle {\widehat{g}}^2,{|{\widehat{\psi }}_{\lambda }^{+}|}^2\rangle -\widetilde{Sf}(\lambda )\right)}^2+{\left(\widehat{g}{(0)}^2-(\widetilde{Pf})(0)\right)}^2.$$

where

$${|{\widehat{\psi }}_{\lambda }^{+}(\omega )|}^2=\left({|{\widehat{\psi }}_{\lambda }(\omega )|}^2+{|{\widehat{\psi }}_{\lambda }(-\omega )|}^2\right)\cdot \mathbf{1}(\omega ⩾0).$$

Letting $\widehat{g}\ast$ denote the minimizer of the above loss function, we then define $\left(\widetilde{P_{S}f}\right):=\widehat{g}\ast {(\omega )}^2$. Theorem 2.4 ensures that when the loss function is defined with the exact wavelet invariants Sf, it has a unique minimizer corresponding to Pf. Whenever $f(x)\in ℝ$, the symmetry of (Pf)(ω) ensures that $(Sf)(\lambda )=\langle |\widehat{f}{|}^2,{|{\widehat{\psi }}_{\lambda }^{+}|}^2\rangle$, and thus it is sufficient to optimize over the non-negative frequencies and then symmetrically extend the solution. Such a procedure ensures the output of the optimization algorithm is symmetric while avoiding adding constraints to the optimization. The algorithm was initialized using the mean power spectrum with additive noise unbiasing only, i.e. PS k = 0. The optimization output does depend on various numerical tolerance parameters, which were held fixed for all examples.

Remark 6.1

Alternatively, one can invert the representation by applying a pseudo-inverse with Tikhonov regularization. Specifically, if F is the matrix defining the wavelet invariants, so that Sy = F(Py), then one can define $\left(\widetilde{P_{S}f}\right)={\left(F^{T}F+\lambda I\right)}^{-1}{F}^T(\widetilde{Sf})$. This procedure however requires careful selection of the hyper-parameter λ and did not work as well as inverting via optimization in our experiments.

7. Conclusion

This article considers a generalization of classic MRA, which incorporates random dilations in addition to random translations and additive noise and proposes solving the problem with a wavelet invariant representation. These wavelet invariants have several desirable properties over Fourier invariants, which allow for the construction of unbiasing procedures that cannot be constructed for Fourier invariants. Unbiasing the representation is critical for high-frequency signals, where even small diffeomorphisms cause a large perturbation. After unbiasing, the power spectrum of the target signal can be recovered from a convex optimization procedure.

Several directions remain for further investigation, including extending results to higher dimensions and considering rigid transformations instead of translations. Such extensions could be especially relevant to image processing, where variations in the size of an object can be modeled as dilations. Incorporating the effect of tomographic projection would also lead to results more directly relevant to problems such as cryo-EM. The tools of the present article, although significantly reducing the bias, do not allow for a completely unbiased estimator for noisy dilation MRA due to the bad scaling of certain intrinsic constants. Thus, an important open question is whether it is possible to define unbiased estimators for noisy dilation MRA using a different approach. The noisy dilation MRA model of this article corresponds to linear diffeomorphisms, and constructing unbiasing procedures that apply to more general diffeomorphisms is also an important future direction. In addition, one can construct wavelet invariants that characterize higher order auto-correlation functions such as the bispectrum, and future work will investigate full signal recovery with such invariants.

A. Wavelet admissibility conditions

This appendix describes the wavelet admissibility conditions that are needed for the main results in this article, namely Propositions 4.2 and 5.1. The wavelet ψ is k-admissible if $\widehat{\psi }\in {\mathbf{C}}^k(ℝ)$ and Ψ_k < ∞, Θ_k < ∞ where

$${\Psi }_k:=\frac{1}{2\pi }\sum _{i=0}^k\left(\begin{array}{l}k\hfill \\ i\hfill \end{array}\right)\frac{k!}{i!}{\Vert {\omega }^i{(P\psi )}^{(i)}(\omega )\Vert }_1,$$ (A.1)

$${\Theta }_k:=\frac{1}{2\pi }\sum _{i=0}^k\left(\begin{array}{l}k\hfill \\ i\hfill \end{array}\right)\frac{k!}{i!}{\Vert {\omega }^{i-2}{(P\psi )}^{(i)}(\omega )\Vert }_1.$$ (A.2)

For ψ to be k-admissible, it is sufficient for $\widehat{\psi }\in {\mathbf{C}}^k(ℝ)$, (Pψ)⁽ⁱ⁾ to decay faster than ωⁱ⁺¹, and $\int \frac{|\widehat{\psi }(\omega ){|}^2}{{\omega }^2}\text{d}\omega <\infty$ (see Lemma B.1 in Appendix B). The condition $\int \frac{|\widehat{\psi }(\omega ){|}^2}{{\omega }^2}\text{d}\omega <\infty$ is slightly stronger than the classic admissability condition $C_{\psi }:=\int \frac{|\widehat{\psi }(\omega ){|}^2}{\omega }\text{d}\omega <\infty$ [55, Theorem 4.4]. When $\widehat{\psi }$ is continuously differentiable, $\widehat{\psi }(0)=0$ is sufficient to guarantee C_ψ < ∞; but here we need $\widehat{\psi }(\omega )~{\omega }^{\frac{1}{2}+ϵ}$ for some ϵ > 0 as ω → 0. If this condition is removed, we are not guaranteed Θ_k < ∞, but all results in fact still hold, with ${\Lambda }_k(\lambda )={\Psi }_k\Vert f{\Vert }_1^2$ replacing ${\Lambda }_k(\lambda )={\Psi }_k\Vert f{\Vert }_1^2\wedge \frac{{\Theta }_k{\Vert f^{\prime }\Vert }_1^2}{{\lambda }^2}$ in Propositions 4.2 and 5.1. Any wavelet with fast decay satisfies this stronger admissibility condition, and it ensures that a smooth signal will enjoy a fast decay of wavelet invariants.

Remark A.1

The Morlet wavelet ψ(x) =→g(x)(e^iξx − C) is k-admissible for any k, since $\widehat{\psi }\in {\mathbf{C}}^{\infty }(ℝ)$, Pψ has fast decay, and $\widehat{\psi }(\omega )~\omega$ as ω → 0. One can also choose $\widehat{\psi }$ to be an order k + 1-spline of compact support.

B. Properties of wavelet invariants

This appendix establishes several important properties of wavelet invariants. Lemma B.1 gives sufficient conditions guaranteeing that a wavelet is k-admissible. Lemmas 4.3 and 4.4 bound wavelet invariant derivatives. Lemma B.2 bounds terms that arise in the dilation unbiasing procedure of Sections 4.2 and 5.

Lemma B.1 (k-admissible).

If $\widehat{\psi }\in {\mathbf{C}}^k(ℝ)$, (Pψ)⁽ⁱ⁾ decays fast than ωⁱ⁺¹, and $\int \frac{|\widehat{\psi }(\omega ){|}^2}{{\omega }^2}\text{d}\omega <\infty$, then ψ is k-admissible.

Proof.

We first note that $\widehat{\psi }\in {\mathbf{C}}^k(ℝ)$ guarantees $P\psi \in {\mathbf{C}}^k(ℝ)$. Since (Pψ)⁽ⁱ⁾ decays faster than ωⁱ⁺¹ and $P\psi \in {\mathbf{C}}^k(ℝ)$, ${\omega }^i{(P\psi )}^{(i)}(\omega )\in {\mathbf{L}}^1(ℝ)$ for 0 ⩽ i ⩽ k, so Ψ_k < ∞. Also $P\psi \in {\mathbf{C}}^k(ℝ)$ and ${\omega }^i{(P\psi )}^{(i)}\in {\mathbf{L}}^1(ℝ)$ implies ${\omega }^{i-2}{(P\psi )}^{(i)}\in {\mathbf{L}}^1(ℝ)$ for 2 ⩽ i ⩽ k. In addition, ${\omega }^{-2}(P\psi )(\omega )\in {\mathbf{L}}^1(ℝ)$ by assumption. Thus, to conclude Θ_k < ∞, it only remains to show ${\omega }^{-1}{(P\psi )}^{\prime }(\omega )\in {\mathbf{L}}^1(ℝ)$. Since (Pψ)′ is continuous and decays faster than ω², only the integrability around the origin needs to be verified. We note that $\int \frac{|\widehat{\psi }(\omega ){|}^2}{{\omega }^2}\text{d}\omega <\infty$ and Pψ continuous implies Pψ ∼ ω^1+ϵ for some ϵ > 0 as ω → 0. Thus, (Pψ)′ ∼ ω^ϵ as ϵ → 0, so that ω⁻¹(Pψ)′ ∼ ω^ϵ−1; the function is thus integrable around the origin since ϵ − 1 > −1. □

Lemma 4.3 (Low frequency bound).

Assume $P\psi \in {\mathbf{C}}^m(ℝ)$ and $f\in {\mathbf{L}}^1(ℝ)$. Then the quantity |λ^m(Sf)^(m)(λ)| can be bounded uniformly over all λ. Specifically,

$$|{\lambda }^m{(Sf)}^{(m)}(\lambda )|⩽{\Psi }_m\Vert f{\Vert }_1^2$$

for Ψ_m defined in (A.1).

Proof.

Let $g(\omega )=(P\psi )(\omega )=|\widehat{\psi }(\omega ){|}^2$, and let

$$g_{\lambda }(\omega ):=\frac{1}{\lambda }g\left(\frac{\omega }{\lambda }\right)={|{\widehat{\psi }}_{\lambda }(\omega )|}^2.$$

Utilizing Definition 2.2 we obtain

$${\lambda }^m{(Sf)}^{(m)}(\lambda )=\frac{1}{2\pi }\int |\widehat{f}(\omega ){|}^2\left[{\lambda }^m\frac{d^m}{d{\lambda }^m}{g}_{\lambda }(\omega )\right]\text{d}\omega .$$

Expanding the derivative gives

$$\begin{gathered} {\lambda }^m\frac{d^m}{d{\lambda }^m}{g}_{\lambda }(\omega )=C_{m,0}{g}_{\lambda }(\omega )+C_{m,1}\omega g_{\lambda }^{\prime }(\omega )+C_{m,2}{\omega }^2{g}_{\lambda }^{\prime \prime }(\omega )+\dots C_{m,m}{\omega }^m{g}_{\lambda }^{(m)}(\omega ), \\ {C}_{m,i}={\left(-1\right)}^m\left(\begin{array}{c}m\\ i\end{array}\right)\frac{m!}{i!}. \end{gathered}$$

Utilizing $\Vert \widehat{f}{\Vert }_{\infty }⩽\Vert f{\Vert }_1$ and $g_{\lambda }^{(i)}(\omega )=\frac{1}{{\lambda }^{i+1}}{g}^{(i)}\left(\frac{\omega }{\lambda }\right)$, one obtains

$$\begin{gathered} |{\lambda }^m{(Sf)}^{(m)}(\lambda )|⩽\sum _{i=0}^m\frac{|C_{m,i}|}{2\pi }\int |\widehat{f}(\omega ){|}^2|{\omega }^i{g}_{\lambda }^{(i)}(\omega )|\text{d}\omega \\ ⩽\Vert f{\Vert }_1^2\sum _{i=0}^m\frac{|C_{m,i}|}{2\pi }\int |{\omega }^i{g}_{\lambda }^{(i)}(\omega )|\text{d}\omega \\ =\Vert f{\Vert }_1^2\sum _{i=0}^m\frac{|C_{m,i}|}{2\pi }\int |{\omega }^i{g}^{(i)}(\omega )|\text{d}\omega \\ =\Vert f{\Vert }_1^2\sum _{i=0}^m\frac{|C_{m,i}|}{2\pi }\cdot {\Vert {\omega }^i{g}^{(i)}(\omega )\Vert }_1 \\ ={\Psi }_m\Vert f{\Vert }_1^2. \end{gathered}$$

Lemma 4.4 (High frequency bound for differentiable functions).

Assume $P\psi \in {\mathbf{C}}^m(ℝ)$, and $f^{\prime }\in {\mathbf{L}}^1(ℝ)$. Then the quantity |λ^m(Sf)^(m)(λ)| can be bounded by

$$|{\lambda }^m{(Sf)}^{(m)}(\lambda )|⩽\frac{{\Theta }_m}{{\lambda }^2}{\Vert f^{\prime }\Vert }_1^2$$

for Θ_m defined in (A.2).

Proof.

Recall from the proof of Lemma 4.3 that

$$|{\lambda }^m{(Sf)}^{(m)}(\lambda )|⩽\sum _{i=0}^m\frac{|C_{m,i}|}{2\pi }\int |\widehat{f}(\omega ){|}^2|{\omega }^i{g}_{\lambda }^{(i)}(\omega )|\text{d}\omega$$

where $g_{\lambda }(\omega )=\frac{1}{\lambda }g\left(\frac{\omega }{\lambda }\right)={|{\widehat{\psi }}_{\lambda }(\omega )|}^2$ and $C_{m,i}={(-1)}^m\left(\begin{array}{c}m\\ i\end{array}\right)\frac{m!}{i!}$. Since $\Vert \omega \widehat{f}(\omega ){\Vert }_{\infty }⩽{\Vert f^{\prime }\Vert }_1$ and $g_{\lambda }^{(i)}(\omega )=\frac{1}{{\lambda }^{i+1}}{g}^{(i)}\left(\frac{\omega }{\lambda }\right)$, we obtain

$$\begin{gathered} |{\lambda }^m{(Sf)}^{(m)}(\lambda )|⩽\sum _{i=0}^m\frac{|C_{m,i}|}{2\pi }\int |\omega \widehat{f}(\omega ){|}^2|{\omega }^{i-2}{g}_{\lambda }^{(i)}(\omega )|\text{d}\omega \\ ⩽{\Vert f^{\prime }\Vert }_1^2\sum _{i=0}^m\frac{|C_{m,i}|}{2\pi }\int |{\omega }^{i-2}{g}_{\lambda }^{(i)}(\omega )|\text{d}\omega \\ =\frac{{\Vert f^{\prime }\Vert }_1^2}{{\lambda }^2}\sum _{i=0}^m\frac{|C_{m,i}|}{2\pi }\int |{\omega }^{i-2}{g}^{(i)}(\omega )|\text{d}\omega \\ =\frac{{\Vert f^{\prime }\Vert }_1^2}{{\lambda }^2}\sum _{i=0}^m\frac{|C_{m,i}|}{2\pi }\cdot {\Vert {\omega }^{i-2}{g}^{(i)}(\omega )\Vert }_1 \\ =\frac{{\Theta }_m}{{\lambda }^2}{\Vert f^{\prime }\Vert }_1^2. \end{gathered}$$

Lemma B.2

Assume $Pf\in {\mathbf{C}}^0(ℝ)$ and ψ is m-admissible, and let B_m, E,Ψ_m,Θ_m be as defined in (4.1), (4.3), (A.1) (A.2). Then,

$${\frac{1}{2\pi }\int |\widehat{f}(\omega ){|}^2\cdot |B_m{\eta }^m{\lambda }^m\frac{d^m}{d{\lambda }^m}|{\widehat{\psi }}_{\lambda }(\omega )|}^2\mid \text{d}\omega ⩽{(E\eta )}^m{\Lambda }_m(\lambda ),$$

where

$${\Lambda }_m(\lambda )=\left(\Vert f{\Vert }_1^2{\Psi }_m\wedge \frac{{\Vert f^{\prime }\Vert }_1^2{\Theta }_m}{{\lambda }^2}\right).$$

Proof.

From the proof of Lemma 4.3:

$${\frac{1}{2\pi }\int |\widehat{f}(\omega ){|}^2\cdot |{\lambda }^m\frac{d^m}{d{\lambda }^m}|{\widehat{\psi }}_{\lambda }(\omega )|}^2\mid \text{d}\omega ⩽{\Psi }_m\Vert f{\Vert }_1^2.$$

From the proof of Lemma 4.4:

$${\frac{1}{2\pi }\int |\widehat{f}(\omega ){|}^2\cdot |{\lambda }^m\frac{d^m}{d{\lambda }^m}|{\widehat{\psi }}_{\lambda }(\omega )|}^2\mid \text{d}\omega ⩽{\Theta }_m\frac{{\Vert f^{\prime }\Vert }_1^2}{{\lambda }^2}.$$

Utilizing |B_m| ⩽ E_m gives

$${\frac{1}{2\pi }\int |\widehat{f}(\omega ){|}^2\cdot |B_m{\eta }^m{\lambda }^m\frac{d^m}{d{\lambda }^m}|{\widehat{\psi }}_{\lambda }(\omega )|}^2\mid \text{d}\omega ⩽{(E\eta )}^m\left(\Vert f{\Vert }_1^2{\Psi }_m\wedge \frac{{\Vert f^{\prime }\Vert }_1^2{\Theta }_m}{{\lambda }^2}\right).$$

The following Corollary is obtained from Lemma B.2 when f is a dirac-delta function.

Corollary B.1

Assume ψ is m-admissible, and let B_m, E,Ψ_m be as defined in (4.1), (4.3), (A1). Then,

$${\frac{1}{2\pi }\int |B_m{\eta }^m{\lambda }^m\frac{d^m}{d{\lambda }^m}|{\widehat{\psi }}_{\lambda }(\omega )|}^2\mid \text{d}\omega ⩽{(E\eta )}^m{\Psi }_m.$$

C. Power spectrum and wavelet invariant equivalence

This appendix contains supporting results for demonstrating the equivalence of the power spectrum and wavelet invariants. Lemma 2.1 establishes that wavelet invariants uniquely determine any bandlimited L² function, as long as the wavelet satisfies the linear independence Condition 2.3 and a mild integrability condition. Proposition 2.1 gives two criteria that are sufficient to guarantee Condition 2.3. Finally, Lemma C.1 establishes that the Morlet wavelet satisfies Condition 2.3.

Lemma 2.1

Let $\text{p}\in {\mathbf{L}}^2(ℝ)$ be continuous and assumep(w) = p(−w), $\widehat{\psi }$ has compact support and Condition 2.3. Then

Proof.

Since p is continuous, there exists an ϵ > 0 such that on (0, ϵ) one either has p = 0, p > 0, or p < 0. Claim: one must have p = 0. Suppose not, and without loss of generality assume p > 0 on (0, ϵ) and that the support of ${|{\widehat{\psi }}^{+}(\omega )|}^2$ is contained in the interval [1, 2]. Now choose λ₀ small enough so that ${|{\widehat{\psi }}_{{\lambda }_0}^{+}(\omega )|}^2$ is supported on [ϵ/1, ϵ) i.e. λ₀ = ϵ/2. Clearly, there must exist a subset $\mathcal{M}\subseteq [ϵ/2,ϵ]$ of positive measure such that ${|{\widehat{\psi }}_{{\lambda }_0}^{+}(\omega )|}^2>0$ on $\mathcal{M}$. Then,

$$0={\int }_0^{\infty }p(\omega ){|{\widehat{\psi }}_{{\lambda }_0}^{+}(\omega )|}^2\text{d}\omega ={\int }_{ϵ/2}^{ϵ}p(\omega ){|{\widehat{\psi }}_{{\lambda }_0}^{+}(\omega )|}^2\text{d}\omega ⩾{\int }_{\mathcal{M}}p(\omega ){|{\widehat{\psi }}_{{\lambda }_0}^{+}(\omega )|}^2\text{d}\omega ⩾0.$$

We conclude

$${\int }_{\mathcal{M}}p(\omega ){|{\widehat{\psi }}_{{\lambda }_0}^{+}(\omega )|}^2\text{d}\omega =0,$$

but this is impossible since the integrand is strictly positive on $\mathcal{M}$. We thus conclude that p = 0 on (0, ϵ). Thus, it is sufficient to only consider frequencies [ϵ, ∞].

Assume $\int p(\omega ){|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega =0$ for all λ. Since p(ω) = p(−ω),

$$\int p(\omega ){|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega ={\int }_0^{\infty }p(\omega ){|{\widehat{\psi }}_{\lambda }^{+}(\omega )|}^2\text{d}\omega ={\int }_{ϵ}^{\infty }p(\omega ){|{\widehat{\psi }}_{\lambda }^{+}(\omega )|}^2\text{d}\omega ={\langle p,{|{\widehat{\psi }}_{\lambda }^{+}|}^2\rangle }_I=0\forall \lambda ,$$

where I = [ϵ, ∞). We now define ${|{\widehat{\varphi }}_{\lambda }^{+}(\omega )|}^2:={\lambda }^{-\beta }{|{\widehat{\psi }}_{\lambda }^{+}(\omega )|}^2$ for some β > 0, and observe that

$${\int }_0^{\infty }p(\omega ){|{\widehat{\varphi }}_{\lambda }^{+}(\omega )|}^2\text{d}\omega =\forall \lambda \Rightarrow {{\int }_0^{\infty }|{\langle p,{|{\widehat{\varphi }}_{\lambda }^{+}|}^2\rangle }_{{ℝ}^{+}}|}^2\text{d}\lambda ={{\int }_0^{\infty }|{\langle p,{|{\widehat{\varphi }}_{\lambda }^{+}|}^2\rangle }_I|}^2\text{d}\lambda =0.$$

Note

$$\begin{gathered} {\int }_0^{\infty }{|{\langle p,{|{\widehat{\varphi }}_{\lambda }^{+}|}^2\rangle }_I|}^2\text{d}\lambda ={\int }_0^{\infty }{\langle p,{|{\widehat{\varphi }}_{\lambda }^{+}|}^2\rangle }_I{\langle \overline{p},{|{\widehat{\varphi }}_{\lambda }^{+}|}^2\rangle }_I\text{d}\lambda \\ ={\int }_0^{\infty }\left({\int }_{I}p\left({\omega }_1\right){|{\widehat{\varphi }}_{\lambda }^{+}\left({\omega }_1\right)|}^2\text{d}{\omega }_1\right)\left({\int }_I\overline{p\left({\omega }_2\right)}{|{\widehat{\varphi }}_{\lambda }^{+}\left({\omega }_2\right)|}^{2}d{\omega }_2\right)\text{d}\lambda \\ ={\int }_I\overline{p\left({\omega }_2\right)}\left({\int }_{I}p\left({\omega }_1\right)\left({\int }_0^{\infty }{|{\widehat{\varphi }}_{\lambda }^{+}\left({\omega }_1\right)|}^2{|{\widehat{\varphi }}_{\lambda }^{+}\left({\omega }_2\right)|}^2\text{d}\lambda \right)\text{d}{\omega }_1\right)\text{d}{\omega }_2. \end{gathered}$$

We now apply the change of variable ω_i = 1/ξ_i, and let g(ξ_i) = p(1/ξ_i). We obtain

$$0={\int }_0^{1/ϵ}\overline{g\left({\xi }_2\right)}\left({\int }_0^{1/ϵ}g\left({\xi }_1\right)\left({\int }_0^{\infty }\frac{1}{{\xi }_1^2{\xi }_2^2}|{\widehat{\varphi }}_{\lambda }^{+}\left(\frac{1}{{\xi }_1}\right){|}^2{\left|{\widehat{\varphi }}_{\lambda }^{+}\left(\frac{1}{{\xi }_2}\right)\right|}^2\text{d}\lambda \right)\text{d}{\xi }_1\right)\text{d}{\xi }_2.$$ (C.1)

Now consider the kernel

$$k\left({\xi }_1,{\xi }_2\right)={\int }_0^{\infty }\frac{1}{{\xi }_1^2{\xi }_2^2}\left|{\widehat{\varphi }}_{\lambda }^{+}\left(\frac{1}{{\xi }_1}\right)\right|2{\left|{\widehat{\varphi }}_{\lambda }^{+}\left(\frac{1}{{\xi }_2}\right)\right|}^2\text{d}\lambda .$$

Note that k is a strictly positive definite kernel function if for any finite sequence ${\left\{{\xi }_i\right\}}_{i=1}^n$ in [0, 1/ϵ], the n by n matrix A defined by

$$A_{ij}=k\left({\xi }_i,{\xi }_j\right)$$

is strictly positive definite [83]. Viewing ${\tilde{\xi }}_i(\lambda )={\xi }_i^{-2}{|{\widehat{\varphi }}_{\lambda }^{+}\left(1/{\xi }_i\right)|}^2$ as functions of λ, we see that

$$A_{ij}={\langle {\tilde{\xi }}_i(\lambda ),{\tilde{\xi }}_j(\lambda )\rangle }_{{ℝ}^{+}}$$

and A is thus a Gram matrix. Since the ${\tilde{\xi }}_i(\lambda )$ are linearly independent if and only if the ${|{\widehat{\psi }}_{\lambda }^{+}\left({\omega }_i\right)|}^2$ are linearly independent, and the ${|{\widehat{\psi }}_{\lambda }^{+}\left({\omega }_i\right)|}^2$ are linearly independent by assumption, we can conclude that A and thus k are strictly positive definite. Now consider the corresponding integral operator on [0, 1/ϵ]:

$$Kg\left({\xi }_2\right)={\int }_0^{1/ϵ}g\left({\xi }_1\right)k\left({\xi }_1,{\xi }_2\right)\text{d}{\xi }_1.$$

Since $\psi \in {\mathbf{L}}^1(ℝ)$, ${|{\widehat{\psi }}_{\lambda }^{+}|}^2$ and thus ${|{\widehat{\varphi }}_{\lambda }^{+}|}^2$ are continuous, and k will thus be continuous as long as it remains bounded. To check boundedness we observe that k(ξ₁, ξ₂)² ⩽ k(ξ₁, ξ₁)k(ξ₂, ξ₂) [21], and

$$\begin{gathered} k(\xi ,\xi )={\int }_0^{\infty }\frac{1}{{\xi }^4}{\left|{\widehat{\varphi }}_{\lambda }^{+}\left(\frac{1}{\xi }\right)\right|}^4\text{d}\lambda \\ ={\int }_0^{\infty }\frac{1}{{\xi }^4}\frac{1}{{\lambda }^{2+2\beta }}{\left|{\widehat{\psi }}^{+}\left(\frac{1}{\lambda \xi }\right)\right|}^4\text{d}\lambda \\ ={\int }_0^{\infty }\frac{1}{{\xi }^4}{\left(\omega \xi \right)}^{2+2\beta }{\left|{\widehat{\psi }}^{+}(\omega )\right|}^4\frac{\text{d}\omega }{\xi {\omega }^2} \\ ={\xi }^{2\beta -3}{\int }_0^{\infty }{\omega }^{2\beta }{\left|{\widehat{\psi }}^{+}(\omega )\right|}^4\text{d}\omega \\ ⩽3{\xi }^{2\beta -3}{\int }_0^{\infty }{\omega }^{2\beta }|\widehat{\psi }(\omega ){|}^4\text{d}\omega \\ ⩽3{\xi }^{2\beta -3}{\Vert {\omega }^{\beta }P\psi \Vert }_2^2. \end{gathered}$$

Since $\widehat{\psi }$ has a compact support, clearly ${\Vert {\omega }^{\beta }P\psi \Vert }_2^2<\infty$, and k is thus bounded on the compact interval [0, 1/ϵ] as long as β ⩾ 3/2. Since k is continuous and [0, 1/ϵ] is compact, K : L²[0, 1/ϵ] → L²[0, 1/ϵ] is a compact, self-adjoint operator and by Mercer’s Theorem K is also strictly positive definite [83]. Since ${\langle Kg,g\rangle }_{[0,1/ϵ]}=0$ by (C.1), we conclude g = 0 in L²[0, 1/ϵ]. Thus, p(1/ξ) = 0 for almost every ξ ∈ (0, 1/ϵ], which implies p(ω) = 0 for almost every ω ∈ [ϵ, ∞). Since p(ω) = p(−ω) and p = 0 on (0, ϵ), p = 0 for almost every $\omega \in ℝ$. □

Proposition 2.1

The following are sufficient to guarantee Condition 2.3:

1. $|\widehat{\psi }(\omega ){|}^2$ has a compact support contained in the interval [a, b], where a and b have the same sign, e.g. complex analytic wavelets with compactly supported Fourier transform.

2. $|\widehat{\psi }(\omega ){|}^2\in {\mathbf{C}}^{\infty }(ℝ)$ and there exists an N such that all derivatives of order at least N are non-zero ω = 0, e.g. the Morlet wavelet.

Proof.

Let ${\left\{{\omega }_i\right\}}_{i=1}^n$ be a finite sequence of distinct positive frequencies, and let ${\tilde{\omega }}_i(\lambda )=\frac{1}{|\lambda |}{|{\widehat{\psi }}^{+}\left(\frac{{\omega }_i}{\lambda }\right)|}^2$ denote the corresponding functions of λ.

First assume (i). Without loss of generality we assume that [a, b] is a positive interval and that $|\widehat{\psi }(\omega ){|}^2>0$ on (a, a + ϵ) for some ϵ > 0. Clearly, ${|{\widehat{\psi }}^{+}(\omega )|}^2=|\widehat{\psi }(\omega ){|}^2$. A simple calculation shows that the support of ${\tilde{\omega }}_i(\lambda )$ is contained in the interval $\left[\frac{{\omega }_i}{b},\frac{{\omega }_i}{a}\right]$, and ${\tilde{\omega }}_i(\lambda )>0$ in a neighborhood of $\frac{{\omega }_i}{a}$. Assume we have ordered the ω_i so that ω₁ > . . . > ω_n > 0. Now suppose

$$c_1{\tilde{\omega }}_1(\lambda )+\cdots +c_n{\tilde{\omega }}_n(\lambda )=0.$$

Note ${\tilde{\omega }}_1(\lambda )$ is the only function in the above collection with support in a neighborhood of $\frac{{\omega }_1}{a}$; thus, we must have c₁ = 0, so that

$$c_2{\tilde{\omega }}_2(\lambda )+\cdots +c_n{\tilde{\omega }}_n(\lambda )=0.$$

But now ${\tilde{\omega }}_2(\lambda )$ is the only function in the above collection with support in a neighborhood of $\frac{{\omega }_2}{a}$, so we must have c₂ = 0, and proceeding iteratively we conclude that c₁ = . . . = c_n = 0. Thus, ${\left\{{\tilde{\omega }}_i(\lambda )\right\}}_{i=1}^n$ is a linearly independent set, and Condition 2.3 holds.

Now assume (ii). Since ${\frac{d^n}{d{\omega }^n}\left({|{\widehat{\psi }}^{+}(\omega )|}^2\right)|}_{\omega =0}={2\frac{d^n}{d{\omega }^n}\left(|\widehat{\psi }(\omega ){|}^2\right)|}_{\omega =0}$, ${|{\widehat{\psi }}^{+}(\omega )|}^2$ is ${\mathbf{C}}^{\infty }(ℝ)$ and all derivatives of order at least N are non-zero at ω = 0. Note ${\{{\tilde{\omega }}_i(\lambda )\}}_{i=1}^n={\{|\lambda {|}^{-1}{|{\widehat{\psi }}^{+}\left({\omega }_i/\lambda \right)|}^2\}}_{i=1}^n$ are linearly independent if and only if ${\{{|{\widehat{\psi }}^{+}\left({\omega }_i/\lambda \right)|}^2\}}_{i=1}^n$ are linearly independent. Defining $\tilde{\lambda }=1/\lambda$, this holds if and only if ${\{{|{\widehat{\psi }}^{+}\left({\omega }_i\tilde{\lambda }\right)|}^2\}}_{i=1}^n={\{g\left({\omega }_i\tilde{\lambda }\right)\}}_{i=1}^n$ are linearly independent as functions of $\tilde{\lambda }$, where we define $g(\omega )={|{\widehat{\psi }}^{+}(\omega )|}^2$. Assume

$$c_{1}g\left({\omega }_1\tilde{\lambda }\right)+c_{2}g\left({\omega }_2\tilde{\lambda }\right)+\cdots +c_{n}g\left({\omega }_n\tilde{\lambda }\right)=0.$$

Differentiating m times for N ⩽ m ⩽ N + n − 1, we obtain

$$\begin{gathered} c_1{\omega }_1^N{g}^{(N)}\left({\omega }_1\tilde{\lambda }\right)+\cdots +c_n{\omega }_n^N{g}^{(N)}\left({\omega }_n\tilde{\lambda }\right)=0 \\ ⋮ \\ {c}_1{\omega }_1^{N+n-1}{g}^{(N+n-1)}\left({\omega }_1\tilde{\lambda }\right)+\cdots +c_n{\omega }_n^{N+n-1}{g}^{(N+n-1)}\left({\omega }_n\tilde{\lambda }\right)=0. \end{gathered}$$

The above holds for all $\tilde{\lambda }$. We now take the limit as $\tilde{\lambda }\to 0$ to obtain

$$\begin{gathered} g^{(N)}(0)\left({\omega }_1^N{c}_1+{\omega }_2^N{c}_2+\dots {\omega }_n^N{c}_n\right)=0 \\ {g}^{(N+1)}(0)\left({\omega }_1^{N+1}{c}_1+{\omega }_2^{N+1}{c}_2+\dots {\omega }_n^{N+1}{c}_n\right)=0 \\ ⋮ \\ {g}^{(N+n-1)}(0)\left({\omega }_1^{N+n-1}{c}_1+{\omega }_2^{N+n-1}{c}_2+\dots {\omega }_n^{N+n-1}{c}_n\right)=0. \end{gathered}$$

Since g(m) (0) ≠ 0, we obtain

$$\begin{array}{l}\left[\begin{array}{ccc}{\omega }_1^N& \dots & {\omega }_n^N\\ {\omega }_1^{N+1}& \dots & {\omega }_n^{N+1}\\ ⋮& & ⋮\\ {\omega }_1^{N+n-1}& \dots & {\omega }_n^{N+n-1}\end{array}\right]\left[\begin{array}{c}{c}_1\\ c_2\\ ⋮\\ c_n\end{array}\right]=\left[\begin{array}{c}0\\ 0\\ ⋮\\ 0\end{array}\right]\\ \underset{:=A}{\underbrace{\left[\begin{array}{ccc}1& \dots & 1\\ {\omega }_1& \cdots & {\omega }_n\\ ⋮& & ⋮\\ {\omega }_1^{(n-1)}& \dots & {\omega }_n^{(n-1)}\end{array}\right]}}\underset{:=B}{\underbrace{\left[\begin{array}{cccc}{\omega }_1^N& 0& \dots & 0\\ 0& {\omega }_2^N& \dots & 0\\ ⋮& & ⋮& \\ 0& 0& \dots & {\omega }_n^N\end{array}\right]}}\left[\begin{array}{c}{c}_1\\ c_2\\ ⋮\\ c_n\end{array}\right]=\left[\begin{array}{c}0\\ 0\\ ⋮\\ 0\end{array}\right].\end{array}$$

Since A is a Vandermonde matrix constructed from distinct ω_i, det(A) ≠ 0. Since the ω_i are non-zero, det(B) ≠ 0. Thus, det(AB) = det(A) det(B) ≠ 0. We conclude AB is invertible and so all c_i = 0, which gives Condition 2.3. □

Lemma C.1

Suppose we construct a Morlet wavelet with parameter ξ, that is $\psi (x)=C_{\xi }{\pi }^{-1/4}{e}^{-x^2/2}\left(e^{i\xi x}-e^{-{\xi }^2/2}\right)$ for $C_{\xi }={\left(1-e^{-{\xi }^2}-2e^{-3{\xi }^2/4}\right)}^{-1/2}$. Then, for almost all $\xi \in {ℝ}^{+}$, the wavelet satisfies Condition 2.3.

Proof.

The Fourier transform $\widehat{\psi }$ has form

$$\widehat{\psi }(\omega )={\tilde{C}}_{\xi }{e}^{-{\omega }^2/2}\left(e^{\xi \omega }-1\right)$$

for some constant ${\tilde{C}}_{\xi }$ depending on ξ, so that

$$g(\omega ):={\tilde{C}}_{\xi }^{-2}|\widehat{\psi }(\omega ){|}^2=e^{-{\omega }^2}{\left(e^{\xi \omega }-1\right)}^2.$$

From direct calculation or a computer algebra system, one obtains

$$g^{(n)}(0)=\{\begin{array}{ll}{H}_n(\xi )-2H_n(\xi /2)\hfill & n\text{odd}\hfill \\ H_n(\xi )-2H_n(\xi /2)+\frac{{(-1)}^{\frac{n}{2}}n!}{\left(\frac{n}{2}\right)!}\hfill & n\text{even}\hfill \end{array}$$

where H_n(ξ) is the n^th degree physicist’s Hermite polynomial. We have g′(0) = 0, but for n > 1, g⁽ⁿ⁾(0) = 0 only when ξ is a root of the above polynomial. Since the set of roots of the polynomials ${\left\{g^{(n)}(0)\right\}}_{n=1}^{\infty }$ is countable, if ξ is selected at random from $ℝ$, it is not a root of any of these polynomials with probability 1, and g⁽ⁿ⁾(0) ≠ 0 for all n. Thus, the wavelet satisfies criterion (ii) of Proposition 2.1, and thus the linear independence Condition 2.3. □

D. Supporting results: classic MRA

This appendix contains supporting results for Section 3. The first two lemmas (Lemmas D.1 and Lemma D.2) establish additive noise bounds for the power spectrum and are needed to prove Proposition 3.1. The next two lemmas (Lemmas D.3 and Lemma D.4) establish additive noise bounds for wavelet invariants and are needed to prove Proposition 3.2.

Lemma D.1

Let ε(x) be a white noise processes on $\left[-\frac{1}{2},\frac{1}{2}\right]$ with variance σ². Then, for all frequencies ω, ξ,

$$\mathbb{E}\left[|\widehat{\epsilon }(\omega ){|}^2\right]={\sigma }^2$$ (D.1)

$$\mathbb{E}\left[|\widehat{\epsilon }(\omega ){|}^4\right]⩽3{\sigma }^4$$ (D.2)

$$\mathbb{E}\left[|\widehat{\epsilon }(\omega ){|}^2|\widehat{\epsilon }(\xi ){|}^2\right]⩽3{\sigma }^4.$$ (D.3)

Proof.

By Proposition J.1,

$$\begin{gathered} \mathbb{E}\left[|\widehat{\epsilon }(\omega ){|}^2\right]=\mathbb{E}[\widehat{\epsilon }(\omega )\widehat{\widehat{\epsilon }(\omega )}] \\ =\mathbb{E}\left[\left({\int }_{-1/2}^{1/2}{e}^{-i\omega x}\text{d}{B}_x\right)\left({\int }_{-1/2}^{1/2}{e}^{i\omega x}\text{d}{B}_x\right)\right] \\ ={\sigma }^2{\int }_{-1/2}^{1/2}\text{d}x \\ ={\sigma }^2, \end{gathered}$$

which shows (D.1). By Proposition J.2,

$$\begin{gathered} \mathbb{E}\left[|\widehat{\epsilon }(\omega ){|}^4\right]=\mathbb{E}\left[\widehat{\epsilon }{(\omega )}^2{(\overline{\widehat{\epsilon }}(\omega ))}^2\right] \\ =\mathbb{E}\left[{\left({\int }_{-1/2}^{1/2}{e}^{-i\omega x}\text{d}{B}_x\right)}^2{\left({\int }_{-1/2}^{1/2}{e}^{i\omega x}\text{d}{B}_x\right)}^2\right] \\ =2{\sigma }^4{\left({\int }_{-1/2}^{1/2}\text{d}x\right)}^2+{\sigma }^4\left({\int }_{-1/2}^{1/2}{e}^{-2i\omega x}\text{d}x\right)\left({\int }_{-1/2}^{1/2}{e}^{2i\omega x}\text{d}x\right) \\ ⩽2{\sigma }^4+{\sigma }^4\left({\int }_{-1/2}^{1/2}|e^{-2i\omega x}|\text{d}x\right)\left({\int }_{-1/2}^{1/2}|e^{2i\omega x}|\text{d}x\right) \\ =3{\sigma }^4, \end{gathered}$$

which shows (D.2). Finally, by Proposition J.3, we have

$$\begin{gathered} \mathbb{E}\left[|\widehat{\epsilon }(\omega ){|}^2|\widehat{\epsilon }(\xi ){|}^2\right]=\mathbb{E}\left[\left({\int }_{-1/2}^{1/2}{e}^{-i\omega x}\text{d}{B}_x\right)\left({\int }_{-1/2}^{1/2}{e}^{i\omega x}\text{d}{B}_x\right)\left({\int }_{-1/2}^{1/2}{e}^{-i\xi x}\text{d}{B}_x\right)\left({\int }_{-1/2}^{1/2}{e}^{i\xi x}\text{d}{B}_x\right)\right] \\ ={\sigma }^4\left[\left({\int }_{-1/2}^{1/2}{e}^{-i(\omega +\xi )x}\text{d}x\right)\left({\int }_{-1/2}^{1/2}{e}^{i(\omega +\xi )x}\text{d}x\right)\right] \\ +{\sigma }^4\left[\left({\int }_{-1/2}^{1/2}{e}^{i(\xi -\omega )x}\text{d}x\right)\left({\int }_{-1/2}^{1/2}{e}^{i(\omega -\xi )x}\text{d}x\right)+\left({\int }_{-1/2}^{1/2}\text{d}x\right)\left({\int }_{-1/2}^{1/2}\text{d}x\right)\right] \\ ⩽{\sigma }^4\left[3\left({\int }_{-1/2}^{1/2}\text{d}x\right)\left({\int }_{-1/2}^{1/2}\text{d}x\right)\right] \\ =3{\sigma }^4, \end{gathered}$$

which gives (D.3). □

Lemma D.2

Let ε(x) be a white noise processes on $\left[-\frac{1}{2},\frac{1}{2}\right]$ with variance σ². Then, for any signal $f\in {\mathbf{L}}^1(ℝ)$,

$$\mathbb{E}[(P(f+\epsilon ))(\omega )]=(Pf)(\omega )+{\sigma }^2$$

$$\mathrm{Var}[(P(f+\epsilon ))(\omega )]⩽4{\sigma }^2(Pf)(\omega )+2{\sigma }^4.$$

Proof.

Since $\mathbb{E}[\widehat{\epsilon }(\omega )]=\mathbb{E}[\overline{\widehat{\epsilon }(\omega )}]=0$ and $\mathbb{E}\left[|\widehat{\epsilon }(\omega ){|}^2\right]={\sigma }^2$ by Lemma D.1,

$$\begin{gathered} \mathbb{E}[(P(f+\epsilon ))(\omega )]=\mathbb{E}[(\widehat{f}(\omega )+\widehat{\epsilon }(\omega ))(\overline{\widehat{f}(\omega )}+\overline{\widehat{\epsilon }(\omega )})] \\ =\mathbb{E}\left[|\widehat{f}(\omega ){|}^2+\widehat{f}(\omega )\overline{\widehat{\epsilon }(\omega )}+\widehat{\epsilon }(\omega )\overline{\widehat{f}(\omega )}+|\widehat{\epsilon }(\omega ){|}^2\right] \\ =(Pf)(\omega )+{\sigma }^2. \end{gathered}$$

We now control Var[(P(f + ε))(ω)]. Note that:

$${[(P(f+\epsilon ))(\omega )]}^2={\left(|\widehat{f}(\omega ){|}^2+\widehat{f}(\omega )\overline{\widehat{\epsilon }(\omega )}+\widehat{\epsilon }(\omega )\overline{\widehat{f}(\omega )}+|\widehat{\epsilon }(\omega ){|}^2\right)}^2$$

and that

$$\begin{gathered} \mathbb{E}\left[|\widehat{\epsilon }(\omega ){|}^2\widehat{\epsilon }(\omega )\right]=\mathbb{E}\left[\left({\int }_{-1/2}^{1/2}{e}^{-i\omega x}\text{d}{B}_x\right)\left({\int }_{-1/2}^{1/2}{e}^{i\omega s}\text{d}{B}_s\right)\left({\int }_{-1/2}^{1/2}{e}^{-i\omega p}\text{d}{B}_p\right)\right] \\ =0, \end{gathered}$$

since even when x = s = p, $\mathbb{E}\left[{\left(\Delta B_x\right)}^3\right]=0$. Ignoring the terms with zero expectation, we thus get

$$\begin{gathered} \mathbb{E}\left[(P(f+\epsilon )){(\omega )}^2\right]=\mathbb{E}\left(|\widehat{f}(\omega ){|}^4+4|\widehat{f}(\omega ){|}^2|\widehat{\epsilon }(\omega ){|}^2+|\widehat{\epsilon }(\omega ){|}^4+\widehat{f}{(\omega )}^2\widehat{\widehat{\epsilon }{(\omega )}^2}+\widehat{\epsilon }{(\omega )}^2{\widehat{\widehat{f}(\omega )}}^2\right) \\ ⩽\mathbb{E}\left(|\widehat{f}(\omega ){|}^4+6|\widehat{f}(\omega ){|}^2|\widehat{\epsilon }(\omega ){|}^2+|\widehat{\epsilon }(\omega ){|}^4\right) \\ ={\left[(Pf)(\omega )\right]}^2+6{\sigma }^2(Pf)(\omega )+3{\sigma }^4 \end{gathered}$$

where the last line follows from Lemma D.1. Thus,

$$\begin{gathered} \mathrm{Var}[(P(f+\epsilon ))(\omega )]=\mathbb{E}\left[(P(f+\epsilon )){(\omega )}^2\right]-{\left(\mathbb{E}[(P(f+\epsilon ))(\omega )]\right)}^2 \\ ⩽{\left[(Pf)(\omega )\right]}^2+6{\sigma }^2(Pf)(\omega )+3{\sigma }^4-{\left((Pf)(\omega )+{\sigma }^2\right)}^2 \\ =4{\sigma }^2(Pf)(\omega )+2{\sigma }^4. \end{gathered}$$

□

Proposition 3.1

Assume Model 1. Define the following estimator of (Pf)(ω):

$$(\widetilde{Pf})(\omega ):=\frac{1}{M}\sum _{j=1}^M\left(P{y}_j\right)(\omega )-{\sigma }^2.$$

Then, with probability at least 1 − 1/t²,

$$|(Pf)(\omega )-(\widetilde{Pf})(\omega )|⩽\frac{2t\sigma }{\sqrt{M}}\left(\Vert f{\Vert }_1+\sigma \right).$$ (3.1)

Proof.

Let $f^{t_j}(x)=f\left(x-t_j\right)$ so that $y_j=f^{t_j}+{\epsilon }_j$. We first note since $\widehat{f^{t}j}(\omega )=e^{-i\omega t_j}\widehat{f}(\omega )$, the power spectrum is translation invariant, that is $\left(P{f}^{t_j}\right)(\omega )=(Pf)(\omega )$ for all ω, t_j. Thus, by Lemma D.2,

$$\mathbb{E}\left[\left(P{y}_j\right)(\omega )\right]=\mathbb{E}\left[\left(P\left(f^{t_j}+{\epsilon }_j\right)\right)(\omega )\right]=\left(P{f}^{t_j}\right)(\omega )+{\sigma }^2=(Pf)(\omega )+{\sigma }^2$$

and

$$\mathrm{Var}\left[\left(P{y}_j\right)(\omega )\right]=\mathrm{Var}\left[\left(P\left(f^{t_j}+{\epsilon }_j\right)\right)(\omega )\right]⩽4{\sigma }^2\left(P{f}^{t_j}\right)(\omega )+2{\sigma }^4=4{\sigma }^2(Pf)(\omega )+2{\sigma }^4.$$

Since the y_j are independent,

$$\mathrm{Var}\left(\frac{1}{M}\sum _{j=1}^M\left(P{y}_j\right)(\omega )\right)⩽\frac{1}{M}\left(4{\sigma }^2(Pf)(\omega )+2{\sigma }^4\right).$$

Applying Chebyshev’s inequality to the random variable $X=\frac{1}{M}{\sum }_{j=1}^M\left(P{y}_j\right)(\omega )$, we obtain

$$\mathbb{P}\left(\left|\frac{1}{M}\sum _{j=1}^M\left(P{y}_j\right)(\omega )-\left((Pf)(\omega )+{\sigma }^2\right)\right|⩾\frac{t\left(2\sigma \sqrt{(Pf)(\omega )}+\sqrt{2}{\sigma }^2\right)}{\sqrt{M}}\right)⩽\frac{1}{t^2}.$$

Observing that $\sqrt{(Pf)(\omega )}=|\widehat{f}(\omega )|⩽\Vert f{\Vert }_1$ gives (3.1). □

Lemma D.3

Let ε(x) be a white noise processes on $\left[-\frac{1}{2},\frac{1}{2}\right]$ with variance σ². Then,

$$\mathbb{E}[(S\epsilon )(\lambda )]={\sigma }^2$$

$$\mathbb{E}\left[(S\epsilon ){(\lambda )}^2\right]⩽3{\sigma }^4.$$

Proof.

Since $\mathbb{E}\left[|\widehat{\epsilon }(\omega ){|}^2\right]={\sigma }^2$ by Lemma D.1, we have

$$\begin{gathered} \mathbb{E}[(S\epsilon )(\lambda )]=\mathbb{E}\left[{\Vert \epsilon *{\psi }_{\lambda }\Vert }_2^2\right] \\ =\mathbb{E}\left[\frac{1}{2\pi }{\Vert \widehat{\epsilon }\cdot {\widehat{\psi }}_{\lambda }\Vert }_2^2\right] \\ =\mathbb{E}\left[\frac{1}{2\pi }\int |\widehat{\epsilon }(\omega ){|}^2{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega \right] \\ =\frac{{\sigma }^2}{2\pi }\int {|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega \\ ={\sigma }^2{\Vert {\psi }_{\lambda }\Vert }_2^2 \\ ={\sigma }^2. \end{gathered}$$

Since by Lemma D.1, $\mathbb{E}\left[|\widehat{\epsilon }(\omega ){|}^2|\widehat{\epsilon }(\xi ){|}^2\right]⩽3{\sigma }^4$, we also have

$$\begin{gathered} \mathbb{E}\left[(S\epsilon ){(\lambda )}^2\right]=\mathbb{E}\left[{\Vert \epsilon *{\psi }_{\lambda }\Vert }_2^4\right] \\ =\mathbb{E}\left[\frac{1}{{(2\pi )}^2}{\Vert \widehat{\epsilon }\cdot {\widehat{\psi }}_{\lambda }\Vert }_2^2{\Vert \widehat{\epsilon }\cdot {\widehat{\psi }}_{\lambda }\Vert }_2^2\right] \\ =\mathbb{E}\left[\frac{1}{{(2\pi )}^2}\iint |\widehat{\epsilon }(\omega ){|}^2|\widehat{\epsilon }(\xi ){|}^2{|{\widehat{\psi }}_{\lambda }(\omega )|}^2{|{\widehat{\psi }}_{\lambda }(\xi )|}^2\text{d}\omega \text{d}\xi \right] \\ ⩽\frac{3{\sigma }^4}{{(2\pi )}^2}\iint {|{\widehat{\psi }}_{\lambda }(\omega )|}^2{|{\widehat{\psi }}_{\lambda }(\xi )|}^2\text{d}\omega \text{d}\xi \\ =3{\sigma }^4{\left({\Vert {\psi }_{\lambda }\Vert }_2^2\right)}^2 \\ =3{\sigma }^4. \end{gathered}$$

□

Lemma D.4

Let ε(x) be a white noise processes on $\left[-\frac{1}{2},\frac{1}{2}\right]$ with variance σ². Then, for any signal $f\in {\mathbf{L}}^1(ℝ)$

$$\mathbb{E}[(S(f+\epsilon ))(\lambda )]=(Sf)(\lambda )+{\sigma }^2$$

$$\mathrm{Var}[(S(f+\epsilon ))(\lambda )]⩽4{\sigma }^2(Sf)(\lambda )+2{\sigma }^4.$$

Proof.

Utilizing $\mathbb{E}[\epsilon ]=\mathbb{E}[\overline{\epsilon }]=0$ and Lemma D.3, we have

$$\begin{gathered} \mathbb{E}[(S(f+\epsilon ))(\lambda )]=\mathbb{E}\left[\int {|(f+\epsilon )*{\psi }_{\lambda }(u)|}^2\text{d}u\right] \\ =\int {|f*{\psi }_{\lambda }(u)|}^2+\mathbb{E}\left[\int {|\epsilon *{\psi }_{\lambda }(u)|}^2\text{d}u\right] \\ =(Sf)(\lambda )+\mathbb{E}[(S\epsilon )(\lambda )] \\ =(Sf)(\lambda )+{\sigma }^2. \end{gathered}$$

To bound $\mathbb{E}\left[(S(f+\epsilon )){(\lambda )}^2\right]$, note that

$$\begin{gathered} {\left[(S(f+\epsilon ))(\lambda )\right]}^2 \\ ={\left(\int {|f*{\psi }_{\lambda }\left(u_1\right)|}^2+\left(\epsilon *{\psi }_{\lambda }\left(u_1\right)\right)\left(\overline{f}*\overline{{\psi }_{\lambda }\left(u_1\right)}\right)+\left(f*{\psi }_{\lambda }\left(u_1\right)\right)\left(\overline{\epsilon }*\overline{{\psi }_{\lambda }\left(u_1\right)}\right)+\mid \epsilon *{\psi }_{\lambda }\left(u_1\right)\right)|}^2\text{d}{u}_1) \\ \cdot {\left(\int {|f*{\psi }_{\lambda }\left(u_2\right)|}^2+\left(\epsilon *{\psi }_{\lambda }\left(u_2\right)\right)\left(\overline{f}*\overline{{\psi }_{\lambda }\left(u_2\right)}\right)+\left(f*{\psi }_{\lambda }\left(u_2\right)\right)\left(\overline{\epsilon }*\overline{{\psi }_{\lambda }\left(u_2\right)}\right)+\mid \epsilon *{\psi }_{\lambda }\left(u_2\right)\right)|}^2\text{d}{u}_2). \end{gathered}$$

When we take expectation, any term involving one or three ε terms disappears, so that

$$\begin{gathered} \mathbb{E}\left[(S(f+\epsilon )){(\lambda )}^2\right]=\mathbb{E}[\iint {|f*{\psi }_{\lambda }\left(u_1\right)|}^2{|f*{\psi }_{\lambda }\left(u_2\right)|}^2\text{d}{u}_1\text{d}{u}_2 \\ {+\iint {|f*{\psi }_{\lambda }\left(u_1\right)|}^2\mid \epsilon *{\psi }_{\lambda }\left(u_2\right))|}^2\text{d}{u}_1\text{d}{u}_2 \\ +\iint \left(\epsilon *{\psi }_{\lambda }\left(u_1\right)\right)\left(\overline{f}*\overline{{\psi }_{\lambda }\left(u_1\right)}\right)\left(\epsilon *{\psi }_{\lambda }\left(u_2\right)\right)\left(\overline{f}*\overline{{\psi }_{\lambda }\left(u_2\right)}\right)\text{d}{u}_1\text{d}{u}_2 \\ +\iint \left(\epsilon *{\psi }_{\lambda }\left(u_1\right)\right)\left(\overline{f}*\overline{{\psi }_{\lambda }\left(u_1\right)}\right)\left(f*{\psi }_{\lambda }\left(u_2\right)\right)\left(\overline{\epsilon }*\overline{{\psi }_{\lambda }\left(u_2\right)}\right)\text{d}{u}_1\text{d}{u}_2 \\ +\iint \left(f*{\psi }_{\lambda }\left(u_1\right)\right)\left(\overline{\epsilon }*\overline{{\psi }_{\lambda }\left(u_1\right)}\right)\left(\epsilon *{\psi }_{\lambda }\left(u_2\right)\right)\left(\overline{f}*\overline{{\psi }_{\lambda }\left(u_2\right)}\right)\text{d}{u}_1\text{d}{u}_2 \\ +\iint \left(f*{\psi }_{\lambda }\left(u_1\right)\right)\left(\overline{\epsilon }*\overline{{\psi }_{\lambda }\left(u_1\right)}\right)\left(f*{\psi }_{\lambda }\left(u_2\right)\right)\left(\overline{\epsilon }*\overline{{\psi }_{\lambda }\left(u_2\right)}\right)\text{d}{u}_1\text{d}{u}_2 \\ {+\iint \mid \epsilon *{\psi }_{\lambda }\left(u_1\right))|}^2{|f*{\psi }_{\lambda }\left(u_2\right)|}^2\text{d}{u}_1\text{d}{u}_2 \\ {+\iint \mid \epsilon *{\psi }_{\lambda }\left(u_1\right))|}^2\mid \epsilon *{\psi }_{\lambda }\left(u_2\right)){|}^2\text{d}{u}_1\text{d}{u}_2] \\ ⩽\mathbb{E}[\iint {|f*{\psi }_{\lambda }\left(u_1\right)|}^2{|f*{\psi }_{\lambda }\left(u_2\right)|}^2\text{d}{u}_1\text{d}{u}_2 \\ {+6\iint {|f*{\psi }_{\lambda }\left(u_1\right)|}^2\mid \epsilon *{\psi }_{\lambda }\left(u_2\right))|}^2\text{d}{u}_1\text{d}{u}_2 \\ {+\iint \mid \epsilon *{\psi }_{\lambda }\left(u_1\right))|}^2\mid \epsilon *{\psi }_{\lambda }\left(u_2\right)){|}^2\text{d}{u}_1\text{d}{u}_2] \\ =\mathbb{E}\left[{[(Sf)(\lambda )]}^2+6(Sf)(\lambda )(S\epsilon )(\lambda )+{[(S\epsilon )(\lambda )]}^2\right] \\ =\left[(Sf){(\lambda )}^2\right]+6{\sigma }^2(Sf)(\lambda )+3{\sigma }^4, \end{gathered}$$

where the last line follows from Lemma D.3. Thus,

$$\begin{gathered} \mathrm{Var}[(S(f+\epsilon ))(\lambda )]=\mathbb{E}\left[(S(f+\epsilon )){(\lambda )}^2\right]-{\left(\mathbb{E}[(S(f+\epsilon ))(\lambda )]\right)}^2 \\ ⩽{\left[(Sf)(\lambda )\right]}^2+6{\sigma }^2(Sf)(\lambda )+3{\sigma }^4-{\left[(Sf)(\lambda )+{\sigma }^2\right]}^2 \\ =4{\sigma }^2(Sf)(\lambda )+2{\sigma }^4. \end{gathered}$$

□

Proposition 3.2

Assume Model 1. Define the following estimator of (Sf)(λ):

$$(\widetilde{Sf})(\lambda ):=\frac{1}{M}\sum _{j=1}^M\left(S{y}_j\right)(\lambda )-{\sigma }^2.$$

Then, with probability at least 1 − 1/t²,

$$|(Sf)(\lambda )-(\widetilde{Sf})(\lambda )|⩽\frac{2t\sigma }{\sqrt{M}}\left(\Vert f{\Vert }_1+\sigma \right).$$ (D.4)

Proof.

Let $f^{t_j}(x)=f\left(x-t_j\right)$ so that $y_j=f^{t_j}+{\epsilon }_j$. We first note that the wavelet invariants are translation invariant, that is $S{f}^{t_j}=Sf$ for all t_j. We now compute the mean and variance of the coefficients (Sy_j)(λ). By Lemma D.4,

$$\mathbb{E}\left[\left(S{y}_j\right)(\lambda )\right]=\mathbb{E}\left[\left(S\left(f^{t_j}+{\epsilon }_j\right)\right)(\lambda )\right]=\left(S{f}^{t_j}\right)(\lambda )+{\sigma }^2=(Sf)(\lambda )+{\sigma }^2$$

and

$$\mathrm{Var}\left[\left(S{y}_j\right)(\lambda )\right]=\mathrm{Var}\left[\left(S\left(f^{t_j}+{\epsilon }_j\right)\right)(\lambda )\right]⩽4{\sigma }^2\left(S{f}^{t_j}\right)(\lambda )+2{\sigma }^4=4{\sigma }^2(Sf)(\lambda )+2{\sigma }^4.$$

Since the y_j are independent,

$$\mathrm{Var}\left[\frac{1}{M}\sum _{j=1}^M\left(S{y}_j\right)(\lambda )\right]⩽\frac{1}{M}\left[4{\sigma }^2(Sf)(\lambda )+2{\sigma }^4\right].$$

Applying Chebyshev’s inequality to the random variable $X=\frac{1}{M}{\sum }_{j=1}^M\left(S{y}_j\right)(\lambda )$ gives

$$\mathbb{P}\left(\left|\frac{1}{M}\sum _{j=1}^M\left(S{y}_j\right)(\lambda )-\left[(Sf)(\lambda )+{\sigma }^2\right]\right|⩾\frac{t\left(2\sigma \sqrt{(Sf)(\lambda )}+\sqrt{2}{\sigma }^2\right)}{\sqrt{M}}\right)⩽\frac{1}{t^2}.$$

By Young’s convolution inequality, $(Sf)(\lambda )={\Vert f*{\psi }_{\lambda }\Vert }_2^2⩽\Vert f{\Vert }_1^2{\Vert {\psi }_{\lambda }\Vert }_2^2=\Vert f{\Vert }_1^2$, which gives (D.4). □

E. Supporting results: dilation MRA

This appendix contains the technical details of the dilation unbiasing procedure that is central to Propositions 4.1, 4.2 and 5.1. Lemma 4.1 bounds the bias and variance of the estimator, and Lemma 4.2 bounds the error of the estimator given M independent samples.

Lemma 4.1

Let Fλ(τ) = L((1 − τ)λ) for some function L ∈ C^k+2(0, ∞) and a random variable τ satisfying the assumptions of Section 2.1, and let k ⩾ 2 be an even integer. Assume there exist functions ${\Lambda }_i:ℝ\to ℝ$, $R:ℝ\to ℝ$ such that

$$|{\lambda }^i{L}^{(i)}(\lambda )|⩽{\Lambda }_i(\lambda )\text{for}0⩽i⩽k+2,\frac{{\Lambda }_{k+2}((1-\tau )\lambda )}{{\Lambda }_{k+2}(\lambda )}⩽R(\lambda ),$$

and define the following estimator of L(λ):

$$G_{\lambda }(\tau ):=F_{\lambda }(\tau )-B_2{\eta }^2{F}_{\lambda }^{\prime \prime }(\tau )-B_4{\eta }^4{F}_{\lambda }^{(4)}(\tau )-\dots -B_k{\eta }^k{F}_{\lambda }^{(k)}(\tau ).$$

Then G_λ(τ) satisfies

$$|\mathbb{E}{G}_{\lambda }(\tau )-L(\lambda )|\lesssim kR(\lambda ){\Lambda }_{k+2}(\lambda ){(2E\eta )}^{k+2}$$

$$\mathrm{Var}{G}_{\lambda }(\tau )\lesssim k^{2}R{(\lambda )}^2\Lambda {(\lambda )}^2$$

where

$$\Lambda {(\lambda )}^2:=\sum _{0⩽i,j⩽k+2,i+j⩾2}{\Lambda }_i(\lambda ){\Lambda }_j(\lambda ){(2E\eta )}^{i+j}$$

and E is the absolute constant defined in (4.3).

Proof.

We Taylor expand F_λ(τ) about τ = 0

$$F_{\lambda }(\tau )=F_{\lambda }(0)+F_{\lambda }^{\prime }(0)\tau +\frac{F_{\lambda }^{\prime \prime }(0)}{2}{\tau }^2+\dots +\frac{F_{\lambda }^{(k+1)}(0)}{(k+1)!}{\tau }^{k+1}+\underset{:=R_0(\tau ,\lambda )}{\underbrace{{\int }_0^{\tau }\frac{F_{\lambda }^{(k+2)}(t)}{(k+1)!}{(\tau -t)}^{k+1}\text{d}t}}.$$

We note

$$\mathbb{E}\left[F_{\lambda }(\tau )\right]=F_{\lambda }(0)+\frac{F_{\lambda }^{\prime \prime }(0)}{2}{\eta }^2+\dots +\frac{F_{\lambda }^k(0)}{k!}{C}_k{\eta }^k+\mathbb{E}\left[R_0(\tau ,\lambda )\right],$$

which motivates an unbiasing with the first k/2 even derivatives, and thus a Taylor expansion of these derivatives

$$F_{\lambda }(\tau )=F_{\lambda }(0)+F_{\lambda }^{\prime }(0)\tau +\dots +\frac{F_{\lambda }^{(k+1)}(0)}{(k+1)!}{\tau }^{k+1}+\underset{:=R_0(\tau ,\lambda )}{\underbrace{{\int }_0^{\tau }\frac{F_{\lambda }^{(k+2)}(t)}{(k+1)!}{(\tau -t)}^{k+1}\text{d}t}}$$

$$F_{\lambda }^{\prime \prime }(\tau )=F_{\lambda }^{\prime \prime }(0)+F_{\lambda }^{(3)}(0)\tau +\dots +\frac{F_{\lambda }^{(k+1)}(0)}{(k-1)!}{\tau }^{k-1}+\underset{:=R_2(\tau ,\lambda )}{\underbrace{{\int }_0^{\tau }\frac{F_{\lambda }^{(k+2)}(t)}{(k-1)!}{(\tau -t)}^{k-1}\text{d}t}}$$

$$\begin{gathered} F_{\lambda }^{(4)}(\tau )=F_{\lambda }^{(4)}(0)+F_{\lambda }^{(5)}(0)\tau +\dots +\frac{F_{\lambda }^{(k+1)}(0)}{(k-3)!}{\tau }^{k-3}+\underset{:=R_4(\tau ,\lambda )}{\underbrace{{\int }_0^{\tau }\frac{F_{\lambda }^{(k+2)}(t)}{(k-3)!}{(\tau -t)}^{k-3}\text{d}t}} \\ ⋮ \end{gathered}$$

$$F_{\lambda }^{(k)}(\tau )=F_{\lambda }^{(k)}(0)+F_{\lambda }^{(k+1)}(0)\tau +\underset{:=R_k(\tau ,\lambda )}{\underbrace{{\int }_0^{\tau }{F}_{\lambda }^{(k+2)}(t)(\tau -t)\text{d}t}}.$$

Multiplication of the i^th even derivative by B_iηⁱ gives

$$F_{\lambda }(\tau )=F_{\lambda }(0)+F_{\lambda }^{\prime }(0)\tau +\dots +\frac{F_{\lambda }^{(k+1)}(0)}{(k+1)!}{\tau }^{k+1}+R_0(\tau ,\lambda )$$

$$B_2{\eta }^2{F}_{\lambda }^{\prime \prime }(\tau )=B_2{\eta }^2{F}_{\lambda }^{\prime \prime }(0)+B_2{\eta }^2{F}_{\lambda }^{(3)}(0)\tau +\dots +B_2{\eta }^2\frac{F_{\lambda }^{(k+1)}(0)}{(k-1)!}{\tau }^{k-1}+B_2{\eta }^2{R}_2(\tau ,\lambda )$$

$$\begin{gathered} B_4{\eta }^4{F}_{\lambda }^{(4)}(\tau )=B_4{\eta }^4{F}_{\lambda }^{(4)}(0)+B_4{\eta }^4{F}_{\lambda }^{(5)}(0)\tau +\dots +B_4{\eta }^4\frac{F_{\lambda }^{(k+1)}(0)}{(k-3)!}{\tau }^{k-3}+B_4{\eta }^4{R}_4(\tau ,\lambda ) \\ ⋮ \end{gathered}$$

$$B_k{\eta }^k{F}_{\lambda }^{(k)}(\tau )=B_k{\eta }^k{F}_{\lambda }^{(k)}(0)+B_k{\eta }^k{F}_{\lambda }^{(k+1)}(0)\tau +B_k{\eta }^k{R}_k(\tau ,\lambda ).$$

We want an estimator that targets F_λ(0) = L(λ). We thus consider the following variable as an estimator:

$$G_{\lambda }(\tau ):=F_{\lambda }(\tau )-B_2{\eta }^2{F}_{\lambda }^{\prime \prime }(\tau )-B_4{\eta }^4{F}_{\lambda }^{(4)}(\tau )-\dots -B_k{\eta }^k{F}_{\lambda }^{(k)}(\tau )$$

and show that $\mathbb{E}\left[G_{\lambda }(\tau )\right]=F_{\lambda }(0)+O\left({\eta }^{k+2}\right)$ for constants B_i chosen according to (4.1). We have

$$\begin{gathered} \mathbb{E}\left[F_{\lambda }(\tau )\right]=F_{\lambda }(0)+F_{\lambda }^{\prime \prime }(0)\frac{C_2}{2}{\eta }^2+\dots +F_{\lambda }^{(k)}(0)\frac{C_k}{k!}{\eta }^k+\mathbb{E}\left[R_0(\tau ,\lambda )\right] \\ \mathbb{E}\left[B_2{\eta }^2{F}_{\lambda }^{\prime \prime }(\tau )\right]=F_{\lambda }^{\prime \prime }(0)B_2{\eta }^2+F_{\lambda }^{(4)}(0)\frac{B_2{C}_2}{2}{\eta }^4+\dots +F_{\lambda }^{(k)}(0)\frac{B_2{C}_{k-2}}{(k-2)!}{\eta }^k+\mathbb{E}\left[B_2{\eta }^2{R}_2(\tau ,\lambda )\right] \\ \mathbb{E}\left[B_4{\eta }^4{F}_{\lambda }^{(4)}(\tau )\right]=F_{\lambda }^{(4)}(0)B_4{\eta }^4+F_{\lambda }^{(6)}(0)\frac{B_4{C}_2}{2}{\eta }^6+\dots +F_{\lambda }^{(k)}(0)\frac{B_4{C}_{k-4}}{(k-4)!}{\eta }^k+\mathbb{E}\left[B_4{\eta }^4{R}_4(\tau ,\lambda )\right] \\ ⋮ \\ \mathbb{E}\left[B_{k-2}{\eta }^{k-2}{F}_{\lambda }^{(k-2)}(\tau )\right]=F_{\lambda }^{(k-2)}(0)B_{k-2}{\eta }^{k-2}+F_{\lambda }^{(k)}(0)\frac{B_{k-2}{C}_2}{2}{\eta }^k+\mathbb{E}\left[B_{k-2}{\eta }^{k-2}{R}_{k-2}(\tau ,\lambda )\right] \\ \mathbb{E}\left[B_k{\eta }^k{F}_{\lambda }^{(k)}(\tau )\right]=F_{\lambda }^{(k)}(0)B_k{\eta }^k+\mathbb{E}\left[B_k{\eta }^k{R}_k(\tau ,\lambda )\right]. \end{gathered}$$

That is,

$$\begin{gathered} \mathbb{E}\left[G_{\lambda }(\tau )\right]=F_{\lambda }(0)+F_{\lambda }^{\prime \prime }(0)\left(\frac{C_2}{2!}-B_2\right){\eta }^2+F_{\lambda }^{(4)}(0)\left(\frac{C_4}{4!}-\frac{B_2{C}_2}{2!}-B_4\right){\eta }^4 \\ +F_{\lambda }^{(6)}(0)\left(\frac{C_6}{6!}-\frac{B_2{C}_4}{4!}-\frac{B_4{C}_2}{2!}-B_6\right){\eta }^6\dots \\ +F_{\lambda }^{(k)}(0)\left(\frac{C_k}{k!}-\frac{B_2{C}_{k-2}}{(k-2)!}-\dots -\frac{B_{k-2}{C}_2}{2!}-B_k\right){\eta }^k+H_1(\lambda ) \end{gathered}$$

where

$$H_1(\lambda )=\mathbb{E}\left[R_0(\lambda ,\tau )-B_2{\eta }^2{R}_2(\tau ,\lambda )-\dots -B_k{\eta }^k{R}_k(\lambda ,\tau )\right].$$

Since (4.1) guarantees that

$$\begin{gathered} B_2=\frac{C_2}{2!} \\ {B}_4=\frac{C_4}{4!}-{\left(\frac{C_2}{2!}\right)}^2 \\ {B}_6=\frac{C_6}{6!}-\frac{C_2{C}_4}{2!4!}-\left(\frac{C_4}{4!}-{\left(\frac{C_2}{2!}\right)}^2\right)\frac{C_2}{2!} \\ ⋮ \\ {B}_k=\frac{C_k}{k!}-\frac{B_2{C}_{k-2}}{(k-2)!}-\dots -\frac{B_{k-2}{C}_2}{2!}, \end{gathered}$$

the coefficients of η², η⁴, . . . , η^k vanish, and we obtain

$$\mathbb{E}\left[G_{\lambda }(\tau )\right]=F_{\lambda }(0)+H_1(\lambda ).$$

First we bound the bias H₁(λ). In the remainder of the proof we let B₀ = −1 to simplify notation, so that

$$H_1(\lambda )=\sum _{i=0,2,\dots ,k}-B_i{R}_i(\lambda ,\tau ){\eta }^i.$$

We first obtain a bound for |B_iR_i(λ, τ)ηⁱ|. Note

$$\begin{gathered} (k+1-i)!{\eta }^i{R}_i(\lambda ,\tau )={\eta }^i{\int }_0^{\tau }{F}_{\lambda }^{(k+2)}(t){\left(\tau -t\right)}^{k+1-i}\text{d}t \\ ={\eta }^i{\int }_0^{\tau }{\lambda }^{k+2}{L}^{(k+2)}((1-t)\lambda ){\left(\tau -t\right)}^{k+1-i}\text{d}t. \end{gathered}$$

We observe that

$$\begin{gathered} \left|{((1-t)\lambda )}^{k+2}{L}^{(k+2)}((1-t)\lambda )\right|⩽{\Lambda }_{k+2}((1-t)\lambda ) \\ \left|{\lambda }^{k+2}{L}^{(k+2)}((1-t)\lambda )\right|⩽\frac{1}{{(1-t)}^{k+2}}\frac{{\Lambda }_{k+2}((1-t)\lambda )}{{\Lambda }_{k+2}(\lambda )}{\Lambda }_{k+2}(\lambda ) \\ \left|{\lambda }^{k+2}{L}^{(k+2)}((1-t)\lambda )\right|⩽\frac{R(\lambda ){\Lambda }_{k+2}(\lambda )}{{(1-t)}^{k+2}} \end{gathered}$$

so that

$$-\frac{R(\lambda ){\Lambda }_{k+2}(\lambda )}{{(1-t)}^{k+2}}⩽{\lambda }^{k+2}{L}^{(k+2)}((1-t)\lambda )⩽\frac{R(\lambda ){\Lambda }_{k+2}(\lambda )}{{(1-t)}^{k+2}}.$$

Now assume first of all that τ is positive. We have

$$\begin{gathered} \left|(k+1-i)!{\eta }^i{R}_i(\lambda ,\tau )\right|⩽{\eta }^{i}R(\lambda ){\Lambda }_{k+2}(\lambda ){\int }_0^{\tau }\frac{{(\tau -t)}^{k+1-i}}{{(1-t)}^{k+2}}\text{d}t \\ ⩽{\eta }^{i}R(\lambda ){\Lambda }_{k+2}(\lambda ){\int }_0^{\tau }\frac{{\tau }^{k+1-i}}{{(1-t)}^{k+2}}\text{d}t \\ ={\eta }^i{\tau }^{k+1-i}R(\lambda ){\Lambda }_{k+2}(\lambda )\frac{1}{(k+1)}\left(\frac{1}{{(1-\tau )}^{k+1}}-1\right) \\ ⩽\frac{2^{k+2}R(\lambda )}{k+1}{\eta }^i{\tau }^{k+2-i}{\Lambda }_{k+2}(\lambda ) \end{gathered}$$

where the last line follows since $\frac{1}{{(1-\tau )}^{k+1}}⩽2\cdot 2^{k+1}\tau$ for $\tau \in \left[0,\frac{1}{2}\right]$. A similar argument can be applied when τ is negative, and we can conclude

$$|B_i{\eta }^i{R}_i(\lambda ,\tau )|⩽\frac{2^{k+2}R(\lambda )}{(k+1)(k+1-i)!}{\Lambda }_{k+2}(\lambda )|B_i|{\eta }^i|\tau {|}^{k+2-i},$$ (E.1)

which gives

$$\begin{gathered} \mathbb{E}|B_i{\eta }^i{R}_i(\lambda ,\tau )|⩽\frac{2^{k+2}R(\lambda )}{(k+1)(k+1-i)!}{\Lambda }_{k+2}(\lambda )T^{k+2-i}|B_i|{\eta }^{k+2} \\ =\frac{2^{k+2}(k+2-i)R(\lambda )}{k+1}{\Lambda }_{k+2}(\lambda )\frac{T^{k+2-i}}{(k+2-i)!}|B_i|{\eta }^{k+2}. \end{gathered}$$

We thus obtain

$$\begin{gathered} |\mathbb{E}\left[G_{\lambda }(\tau )\right]-L(\lambda )|=|H_1(\lambda )|⩽\frac{R(\lambda ){\Lambda }_{k+2}(\lambda )}{k+1}{\left(2E\eta \right)}^{k+2}\sum _{i=0,2,\dots ,k}(k+2-i) \\ \lesssim R(\lambda )k{\Lambda }_{k+2}(\lambda ){\left(2E\eta \right)}^{k+2}, \end{gathered}$$

which establishes the bound on the bias. We now bound the variance. We note

$$G_{\lambda }(\tau )=\underset{:=(\mathbf{I})}{\underbrace{\sum _{i=0,2,\dots ,k}\sum _{j=0,1,\dots ,k+1-i}\frac{-B_i}{j!}{F}_{\lambda }^{(i+j)}(0){\eta }^i{\tau }^j}}+\underset{:=(\mathbf{I}I)}{\underbrace{\sum _{i=0,2,\dots ,k}-B_i{R}_i(\lambda ,\tau ){\eta }^i}}.$$

Thus,

$$\begin{gathered} \mathrm{Var}\left[G_{\lambda }(\tau )\right]=\mathbb{E}\left[G_{\lambda }{\left(\tau \right)}^2\right]-\mathbb{E}{\left[G_{\lambda }(\tau )\right]}^2 \\ =\mathbb{E}[(\mathbf{I})(\mathbf{I})]+2\mathbb{E}[(\mathbf{I})(\mathbf{II})]+\mathbb{E}[(\mathbf{II})(\mathbf{II})]-F_{\lambda }{\left(0\right)}^2-2F_{\lambda }(0)H_1(\lambda )-H_1{\left(\lambda \right)}^2 \\ ⩽\underset{:=(\mathbf{A})}{\underbrace{\left(\mathbb{E}[(\mathbf{I})(\mathbf{I})]-F_{\lambda }{\left(0\right)}^2\right)}}+\underset{:=(\mathbf{B})}{\underbrace{\left(2\mathbb{E}[(\mathbf{I})(\mathbf{II})]-2F_{\lambda }(0)H_1(\lambda )\right)}}+\underset{:=(\mathbf{C})}{\underbrace{\mathbb{E}[(\mathbf{II})(\mathbf{II})]}} \end{gathered}$$

and we proceed to bound each term.

$$(\mathbf{I})(\mathbf{I})-F_{\lambda }{(0)}^2=\sum _{i=0,2,\dots ,k}\sum _{\ell =0,2,\dots ,k}\sum _{j=0}^{k+1-i}\sum _{s=0}^{k+1-\ell }\frac{B_i{B}_{\ell }}{j!\ell !}{F}_{\lambda }^{(i+j)}(0)F_{\lambda }^{(\ell +s)}(0){\eta }^{i+\ell }{\tau }^{j+s}{\mathbf{1}}_E$$

where 1_E is an indicator function indicating that i,j,ℓ,s are not all zero. We have

$$\begin{gathered} \mathbb{E}|\frac{B_i{B}_{\ell }}{j!\ell !}{F}_{\lambda }^{(i+j)}(0)F_{\lambda }^{(\ell +s)}(0){\eta }^{i+\ell }{\tau }^{j+s}|⩽\frac{|B_i{B}_{\ell }|}{j!\ell !}{C}_{j+s}{\Lambda }_{i+j}(\lambda ){\Lambda }_{\ell +s}(\lambda ){\eta }^{i+\ell +j+s} \\ ⩽\frac{|B_i{B}_{\ell }|}{j!\ell !}{T}^j{T}^s{\Lambda }_{i+j}(\lambda ){\Lambda }_{\ell +s}(\lambda ){\eta }^{i+\ell +j+s} \\ ⩽E^{i+j}{E}^{\ell +s}{\Lambda }_{i+j}(\lambda ){\Lambda }_{\ell +s}(\lambda ){\eta }^{i+\ell +j+s} \\ =\left({\Lambda }_{i+j}(\lambda ){(E\eta )}^{i+j}\right)\left({\Lambda }_{\ell +s}(\lambda ){(E\eta )}^{\ell +s}\right). \end{gathered}$$

Noting that only terms where j + s is even survive expectation, and letting $\tilde{i}=i+j$ and $\tilde{\ell }=\ell +s$, we obtain

$$\begin{gathered} \mathbb{E}[(\mathbf{I})(\mathbf{I})]-F_{\lambda }{\left(0\right)}^2 \\ ⩽\sum _{i=0,2,\dots ,k}\sum _{\ell =0,2,\dots ,k}\sum _{j=0}^{k+1-i}\sum _{s=0}^{k+1-\ell }{\Lambda }_{i+j}(\lambda ){\left(4T\eta \right)}^{i+j}{\Lambda }_{\ell +s}(\lambda ){\left(4T\eta \right)}^{\ell +s}{\mathbf{1}}_E\mathbf{1}(j+s\text{even}) \\ =\sum _{\tilde{i}=0}^{k+1}\sum _{\tilde{\ell }=0}^{k+1}{C}_{\tilde{i},\tilde{\ell }}{\Lambda }_{\tilde{i}}(\lambda ){\left(E\eta \right)}^{\tilde{i}}{\Lambda }_{\tilde{\ell }}(\lambda ){\left(E\eta \right)}^{\tilde{\ell }} \end{gathered}$$

for coefficients $C_{\tilde{i}\tilde{\ell }}$ such that C_0,0 = 0, $C_{\tilde{i}\tilde{\ell }}=0$ if $\tilde{i}+\tilde{\ell }$ is odd, and $C_{\tilde{i}\tilde{\ell }\tilde{\ell }}⩽k^2$. Thus,

$$\mathbb{E}[(\mathbf{I})(\mathbf{I})]-F_{\lambda }{(0)}^2⩽k^2\sum _{\begin{array}{c}2⩽\tilde{i}+\tilde{\ell }⩽2k+2\\ \tilde{i}+\tilde{\ell }\text{even}\end{array}}{\Lambda }_{\tilde{i}}(\lambda ){\Lambda }_{\tilde{\ell }}(\lambda ){(E\eta )}^{\tilde{i}+\tilde{\ell }}⩽k^2\Lambda {(\lambda )}^2.$$

Next we bound $\mathbb{E}[(\mathbf{II})(\mathbf{II})]$.

$$(\mathbf{II})(\mathbf{II})=\sum _{i=0,2,\dots ,k}\sum _{\ell =0,2,\dots k}{B}_i{B}_{\ell }{R}_i(\lambda ,\tau )R_{\ell }(\lambda ,\tau ){\eta }^{i+\ell }.$$

Utilizing Equation (E.1), we have

$$|B_i{B}_{\ell }{R}_i(\lambda ,\tau )R_{\ell }(\lambda ,\tau ){\eta }^{i+\ell }|⩽\frac{2^{2k+4}R{(\lambda )}^2|B_i{B}_{\ell }|}{{(k+1)}^2(k+1-i)!(k+1-\ell )!}{\Lambda }_{k+2}{(\lambda )}^2{\eta }^{i+\ell }|\tau {|}^{2k+4-i-\ell },$$

which gives

$$\begin{gathered} \mathbb{E}\left|B_i{B}_{\ell }{R}_i(\lambda ,\tau )R_{\ell }(\lambda ,\tau ){\eta }^{i+\ell }\right| \\ ⩽\frac{2^{2k+4}R{\left(\lambda \right)}^2{T}^{2k+4-i-\ell }|B_i{B}_{\ell }|}{{(k+1)}^2(k+1-i)!(k+1-\ell )!}{\Lambda }_{k+2}{\left(\lambda \right)}^2{\eta }^{2k+4} \\ ⩽\frac{R{\left(\lambda \right)}^2(k+2-i)(k+2-\ell )}{{\left(k+1\right)}^2}\left(\frac{T^{k+2-i}|B_i|}{(k+2-i)!}\right)\left(\frac{T^{k+2-\ell }|B_{\ell }|}{(k+2-\ell )!}\right){\Lambda }_{k+2}{\left(\lambda \right)}^2{\left(2\eta \right)}^{2k+4} \\ ⩽\frac{R{\left(\lambda \right)}^2(k+2-i)(k+2-\ell )}{{\left(k+1\right)}^2}{\Lambda }_{k+2}{\left(\lambda \right)}^2{\left(2E\eta \right)}^{2k+4} \end{gathered}$$

so that

$$\begin{gathered} \mathbb{E}[(\mathbf{II})(\mathbf{II})]⩽\frac{R{\left(\lambda \right)}^2}{{\left(k+1\right)}^2}{\Lambda }_{k+2}{\left(\lambda \right)}^2{\left(2E\eta \right)}^{2k+4}\sum _{i=0,2,\dots ,k}\sum _{\ell =0,2,\dots k}(k+1-i)(k+2-\ell ) \\ \lesssim k^{2}R{\left(\lambda \right)}^2{\Lambda }_{k+2}{\left(\lambda \right)}^2{\left(2E\eta \right)}^{2k+4} \\ ⩽k^{2}R{\left(\lambda \right)}^2\Lambda {\left(\lambda \right)}^2. \end{gathered}$$

Finally we bound the cross term $2\mathbb{E}[(\mathbf{I})(\mathbf{II})]-2F_{\lambda }(0)H_1(\lambda )$.

$$(\mathbf{I})(\mathbf{II})=\sum _{i=0,2,\dots ,k}\sum _{j=0}^{k+1-i}\sum _{\ell =0,2,\dots ,k}\frac{B_i}{j!}{F}_{\lambda }^{(i+j)}(0){\eta }^i{\tau }^j{B}_{\ell }{R}_{\ell }(\lambda ,\tau ){\eta }^{\ell }$$ (E.2)

Since $|F_{\lambda }^{(i+j)}(0)|⩽{\Lambda }_{i+j}(\lambda )$ and $|B_{\ell }{R}_{\ell }(\lambda ,\tau ){\eta }^{\ell }|⩽\frac{2^{k+2}R(\lambda )|B_{\ell }|}{(k+1)(k+1-\ell )!}{\Lambda }_{k+2}(\lambda ){\eta }^{\ell }{\tau }^{k+2-\ell }$ from (E.1), we have

$$\left|\frac{B_i}{j!}{F}_{\lambda }^{(i+j)}(0){\eta }^i{\tau }^j{B}_{\ell }{R}_{\ell }(\lambda ,\tau ){\eta }^{\ell }\right|⩽\frac{2^{k+2}R(\lambda )|B_i{B}_{\ell }|}{(k+1)j!(k+1-\ell )!}{\Lambda }_{i+j}(\lambda ){\Lambda }_{k+2}(\lambda ){\eta }^{i+\ell }{\tau }^{k+2+j-\ell }$$

so that

$$\begin{gathered} \mathbb{E}\left|\frac{B_i}{j!}{F}_{\lambda }^{(i+j)}(0){\eta }^i{\tau }^j{B}_{\ell }{R}_{\ell }(\lambda ,\tau ){\eta }^{\ell }\right| \\ ⩽\frac{2^{k+2}R(\lambda )T^{k+2+j-\ell }\left|B_i{B}_{\ell }\right|}{(k+1)j!(k+1-\ell )!}{\Lambda }_{i+j}(\lambda ){\Lambda }_{k+2}(\lambda ){\eta }^{i+j+k+2} \\ =\frac{2^{k+2}R(\lambda )(k+2-\ell )}{(k+1)}\left(\frac{T^j\left|B_i\right|}{j!}\right)\left(\frac{T^{k+2-\ell }\left|B_{\ell }\right|}{(k+2-\ell )!}\right){\Lambda }_{i+j}(\lambda ){\Lambda }_{k+2}(\lambda ){\eta }^{i+j+k+2} \\ =\frac{R(\lambda )(k+2-\ell )}{(k+1)}\left[{(E\eta )}^{i+j}{\Lambda }_{i+j}(\lambda )\right]\cdot \left[{(2E\eta )}^{k+2}{\Lambda }_{k+2}(\lambda )\right]. \end{gathered}$$

The same bound holds for the terms of F_λ(0)H₁(λ), which arise from i = 0, j = 0 in (E.2), so that

$$\begin{gathered} 2\mathbb{E}[(\mathbf{I})(\mathbf{II})]-2F_{\lambda }(0)H_1(\lambda ) \\ \lesssim \left(\sum _{i=0,2,\dots ,k}\sum _{j=0}^{k+1-i}{\left(E\eta \right)}^{i+j}{\Lambda }_{i+j}(\lambda )\right)\left(\sum _{\ell =0,2,\dots ,k}\frac{R(\lambda )(k+2-\ell )}{(k+1)}{\left(2E\eta \right)}^{k+2}{\Lambda }_{k+2}(\lambda )\right) \\ \lesssim \left(k\sum _{\tilde{i}=0}^{k+1}{\Lambda }_{\tilde{i}}(\lambda ){\left(E\eta \right)}^{\tilde{i}}\right)\left(kR(\lambda ){\left(2E\eta \right)}^{k+2}{\Lambda }_{k+2}(\lambda )\right) \\ ⩽k^{2}R(\lambda )\sum _{\tilde{i}=0}^{k+1}{\Lambda }_{\tilde{i}}(\lambda ){\Lambda }_{k+2}(\lambda ){\left(2E\eta \right)}^{\tilde{i}+k+2} \\ ⩽k^{2}R(\lambda )\Lambda {\left(\lambda \right)}^2. \end{gathered}$$

Thus, $\mathrm{Var}\left[G_{\lambda }(\tau )\right]\lesssim k^{2}R{(\lambda )}^2\Lambda {(\lambda )}^2$ and the lemma is proved. □

Lemma 4.2

Let the assumptions and notation of Lemma 4.1 hold, and let τ₁, . . . , τ_M be independent. Define

$$\tilde{L}(\lambda ):=\frac{1}{M}\sum _{j=1}^M{G}_{\lambda }\left({\tau }_j\right).$$

Then, with probability at least 1 − 1/t²,

$$|\tilde{L}(\lambda )-L(\lambda )|\lesssim kR(\lambda )\left({\Lambda }_{k+2}(\lambda ){(2E\eta )}^{k+2}+\frac{t\Lambda (\lambda )}{\sqrt{M}}\right).$$

Proof.

By Lemma 4.1 and the independence of the τ_j, we have

$$|L(\lambda )-\mathbb{E}\tilde{L}(\lambda )|\lesssim kR(\lambda ){\Lambda }_{k+2}(\lambda ){(2E\eta )}^{k+2}$$

$$\mathrm{Var}\tilde{L}(\lambda )\lesssim \frac{1}{M}{k}^2\Lambda {(\lambda )}^2$$

so by Chebyshev’s inequality we can conclude that with probability at least 1 − 1/t, we have

$$|\tilde{L}(\lambda )-\mathbb{E}[\tilde{L}(\lambda )]|⩽\frac{tkR(\lambda )\Lambda (\lambda )}{\sqrt{M}},$$

which gives

$$\begin{gathered} |L(\lambda )-\tilde{L}(\lambda )|⩽|L(\lambda )-\mathbb{E}[\tilde{L}(\lambda )]|+|\mathbb{E}[\tilde{L}(\lambda )]-\tilde{L}(\lambda )| \\ \lesssim kR(\lambda ){\Lambda }_{k+2}(\lambda ){\left(2E\eta \right)}^{k+2}+\frac{tkR(\lambda )\Lambda (\lambda )}{\sqrt{M}}. \end{gathered}$$

□

F. Supporting results: noisy dilation MRA

This appendix contains supporting results needed to prove Proposition 5.1, which defines a wavelet invariant estimator for noisy dilation MRA. Lemma 5.1 controls the additive noise error and Lemma 5.2 controls the cross-term error. Lemma F.1 guarantees that the dilation unbiasing procedure applied to the additive noise still has mean σ², which is needed to prove Lemma 5.1.

Lemma 5.1

Let the notation and assumptions of Proposition 5.1 hold, and let A_λ be the operator defined in (5.4). Then, with probability at least 1 − 1/t²,

$${|\frac{1}{M}\sum _{j=1}^M\frac{1}{2\pi }\int |{\widehat{\epsilon }}_j(\omega )|}^2{A}_{\lambda }{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega -{\sigma }^2\mid ⩽\frac{2t\sqrt{k}\Psi {\sigma }^2}{\sqrt{M}}.$$

Proof.

Let

$$D\left({\epsilon }_j,\lambda \right):=\frac{1}{2\pi }\int {|\widehat{{ϵ}_j}(\omega )|}^2{A}_{\lambda }{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega .$$

By Lemma D.1, ${\mathbb{E}}_{\epsilon }\left[{|{\widehat{\epsilon }}_j(\omega )|}^2\right]={\sigma }^2$, and we thus obtain

$$\begin{gathered} {\mathbb{E}}_{\epsilon }\left[D\left({\epsilon }_j,\lambda \right)\right]={\mathbb{E}}_{\epsilon }\left[\frac{1}{2\pi }\int {|\widehat{{\epsilon }_j}(\omega )|}^2{A}_{\lambda }{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega \right] \\ ={\mathbb{E}}_{\epsilon }[\frac{1}{2\pi }\int {|{\widehat{\epsilon }}_j(\omega )|}^2{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega -\frac{1}{2\pi }\int {|{\widehat{\epsilon }}_j(\omega )|}^2{B}_2{\eta }^2{\lambda }^2\frac{d}{d{\lambda }^2}{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega -\dots \\ -\frac{1}{2\pi }\int {|\widehat{{\epsilon }_j}(\omega )|}^2{B}_k{\eta }^k{\lambda }^k\frac{d}{d{\lambda }^k}{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega ] \\ ={\sigma }^2(\frac{1}{2\pi }\int {|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega -\frac{B_2{\eta }^2}{2\pi }\int {\lambda }^2\frac{d}{d{\lambda }^2}{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega -\dots \\ -\frac{B_k{\eta }^k}{2\pi }\int {\lambda }^k\frac{d}{d{\lambda }^k}{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega ) \\ ={\sigma }^2(1-0-\dots -0) \\ ={\sigma }^2, \end{gathered}$$

where we have used Lemma F.1 to conclude $\int {\lambda }^m\left(\frac{d^m}{d{\lambda }^m}{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\right)\text{d}\omega =0$ m = 2, . . . , k. Also since ${\left(a_1+\dots +a_n\right)}^2⩽n\left(a_1^2+\dots +a_n^2\right)$ by the Cauchy–Schwarz inequality, we obtain

$${\mathbb{E}}_{\epsilon }\left[D{\left({\epsilon }_j,\lambda \right)}^2\right]⩽{\mathbb{E}}_{\epsilon }\left[k\sum _{m=0,2,\mathrm{..},k}{\left(\frac{B_m{\eta }^m}{2\pi }\int {|{\widehat{\epsilon }}_j(\omega )|}^2{\lambda }^m\frac{d^m}{d{\lambda }^m}{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega \right)}^2\right]$$

where we let $\frac{d}{d{\lambda }^0}{|{\widehat{\psi }}_{\lambda }(\omega )|}^2$ denote ${|{\widehat{\psi }}_{\lambda }(\omega )|}^2$ and B0 = 1. By Lemma D.1, we have ${\mathbb{E}}_{\epsilon }\left[{|{\epsilon }_j(\omega )|}^2{|{\epsilon }_j(\xi )|}^2\right]⩽3{\sigma }^4$ for all frequencies ω, ξ, so that

$$\begin{gathered} {\mathbb{E}}_{\epsilon }\left[{\left(\frac{B_m{\eta }^m}{2\pi }\int {|\widehat{{\epsilon }_j}(\omega )|}^2{\lambda }^m\frac{d^m}{d{\lambda }^m}{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega \right)}^2\right] \\ ⩽{\mathbb{E}}_{\epsilon }\left[{\frac{B_m^2{\eta }^{2m}}{4{\pi }^2}\iint {|{\widehat{\epsilon }}_j(\omega )|}^2{|\widehat{{\epsilon }_j}(\xi )|}^2|{\lambda }^m\frac{d^m}{d{\lambda }^m}|{\widehat{\psi }}_{\lambda }(\omega )|}^2|\cdot |{\lambda }^m\frac{d^m}{d{\lambda }^m}{\left|{\widehat{\psi }}_{\lambda }(\xi )\right|}^2\mid \text{d}\omega \text{d}\xi \right] \\ ⩽3{\sigma }^4{\left({\frac{1}{2\pi }\int |B_m{\eta }^m{\lambda }^m\frac{d^m}{d{\lambda }^m}|{\widehat{\psi }}_{\lambda }(\omega )|}^2\mid \text{d}\omega \right)}^2 \\ ⩽3{\sigma }^4{\Psi }_m^2{\left(E\eta \right)}^{2m}, \end{gathered}$$

where the last line follows from Corollary B.1 in Appendix B. We thus obtain

$$\begin{gathered} {\mathbb{E}}_{\epsilon }\left[D{\left({\epsilon }_j,\lambda \right)}^2\right]⩽k\sum _{m=0,2,\mathrm{..},k}{\mathbb{E}}_{\epsilon }\left[{\left(\frac{B_m{\eta }^m}{2\pi }\int {|{\widehat{\epsilon }}_j(\omega )|}^2{\lambda }^m\frac{d^m}{d{\lambda }^m}{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega \right)}^2\right] \\ ⩽3k{\sigma }^4\sum _{m=0,2,\mathrm{..},k}{\Psi }_m^2{\left(E\eta \right)}^{2m}:=(\mathbf{I}) \end{gathered}$$

so that

$${\mathbb{E}}_{\epsilon }\left[D\left({\epsilon }_j,\lambda \right)-{\sigma }^2\right]=0$$

$${\mathrm{Var}}_{\epsilon }\left[D\left({\epsilon }_j,\lambda \right)-{\sigma }^2\right]={\mathrm{Var}}_{\epsilon }\left[D\left({\epsilon }_j,\lambda \right)\right]⩽{\mathbb{E}}_{\epsilon }\left[{\left(D\left({\epsilon }_j,\lambda \right)\right)}^2\right]⩽(\mathbf{I}).$$

Thus,

$${\mathrm{Var}}_{\epsilon }\left(\frac{1}{M}\sum _{j=1}^{M}D\left({\epsilon }_j,\lambda \right)-{\sigma }^2\right)⩽\frac{(\mathbf{I})}{M}$$

so that by Chebyshev’s inequality with probability at least 1 − 1/t²

$$\left|\frac{1}{M}\sum _{j=1}^{M}D\left({\epsilon }_j,\lambda \right)-{\sigma }^2\right|⩽\frac{t\sqrt{(\mathbf{I})}}{\sqrt{M}}⩽t\sqrt{3k}\left(\sum _{m=0,2,\dots ,k}{\Psi }_m{(E\eta )}^m\right)\frac{{\sigma }^2}{\sqrt{M}}=2t\sqrt{k}\Psi \frac{{\sigma }^2}{\sqrt{M}}.$$

□

Lemma 5.2

Let the notation and assumptions of Proposition 5.1 hold, and let A_λ be the operator defined in (5.4). Then, with probability at least 1 − 1/t²,

$${\left|\frac{1}{M}\sum _{j=1}^M\frac{1}{2\pi }\int \left({\widehat{f}}_{{\tau }_j}(\omega ){\widehat{\epsilon }}_j(\omega )+{\widehat{f}}_{{\tau }_j}(\omega ){\widehat{\epsilon }}_j(\omega )\right)A_{\lambda }\right|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega \mid \lesssim \frac{t}{\sqrt{M}}\sqrt{\Psi \left({\Lambda }_0(\lambda )+\Lambda (\lambda )\right)}\sigma .$$

Proof.

We have

$$\frac{1}{M}\sum _{j=1}^M\frac{1}{2\pi }\int \left({\widehat{f}}_{{\tau }_j}(\omega ){\widehat{\widehat{\epsilon }}}_j(\omega )+{\widehat{f}}_{{\tau }_j}(\omega ){\widehat{\epsilon }}_j(\omega )\right)A_{\lambda }{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega =\frac{1}{M}\sum _{j=1}^M{Y}_j+{\overline{Y}}_j$$

where

$$Y_j:=\frac{1}{2\pi }\int \left(\overline{\widehat{f_{{\tau }_j}}}(\omega ){\widehat{\epsilon }}_j(\omega )\right)A_{\lambda }{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\text{d}\omega .$$

The random variable Y_j has randomness depending on both ε_j and τ_j. Note that

$${\mathbb{E}}_{\epsilon ,\tau }\left[Y_j\right]={\mathbb{E}}_{\epsilon ,\tau }\left[{\mathbb{E}}_{\epsilon ,\tau }\left[Y_j\mid {\tau }_j\right]\right]$$

since Y_j is integrable. Thus, since ${\mathbb{E}}_{\epsilon ,\tau }\left[{\widehat{\epsilon }}_j(\omega )\right]=0$, we obtain ${\mathbb{E}}_{\epsilon ,\tau }\left[Y_j\mid {\tau }_j\right]=0$, which yields ${\mathbb{E}}_{\epsilon ,\tau }\left[Y_j\right]=0$. We also have

$$\begin{gathered} {\mathrm{Var}}_{\epsilon ,\tau }\left[Y_j\right]={\mathbb{E}}_{\epsilon ,\tau }\left[Y_j^2\right] \\ ⩽{\mathbb{E}}_{\epsilon ,\tau }\left[{\left({\frac{1}{2\pi }\int |\overline{{\widehat{f}}_{{\tau }_j}}(\omega )|\cdot |{\widehat{\epsilon }}_j(\omega )|\cdot |A_{\lambda }|{\widehat{\psi }}_{\lambda }(\omega )|}^2\mid \text{d}\omega \right)}^2\right] \\ ⩽{\mathbb{E}}_{\epsilon ,\tau }\left[\left({\frac{1}{2\pi }\int {|\overline{{\widehat{f}}_{{\tau }_j}}(\omega )|}^2\cdot |A_{\lambda }|{\widehat{\psi }}_{\lambda }(\omega )|}^2\mid \text{d}\omega \right)\left({\frac{1}{2\pi }\int {|{\widehat{\epsilon }}_j(\omega )|}^2\cdot |A_{\lambda }|{\widehat{\psi }}_{\lambda }(\omega )|}^2\mid \text{d}\omega \right)\right] \\ ={\mathbb{E}}_{\tau }\left[{\frac{1}{2\pi }\int {|\overline{{\widehat{f}}_{{\tau }_j}}(\omega )|}^2\cdot |A_{\lambda }|{\widehat{\psi }}_{\lambda }(\omega )|}^2\mid \text{d}\omega \right]{\mathbb{E}}_{\epsilon }\left[{\frac{1}{2\pi }\int {|{\widehat{\epsilon }}_j(\omega )|}^2\cdot |A_{\lambda }|{\widehat{\psi }}_{\lambda }(\omega )|}^2\mid \text{d}\omega \right]. \end{gathered}$$

Letting B₀ = 1 and applying Lemma B.2, we have

$$\begin{gathered} {\mathbb{E}}_{\tau }\left[{\frac{1}{2\pi }\int {|\overline{{\widehat{f}}_{{\tau }_j}}(\omega )|}^2\cdot |A_{\lambda }|{\widehat{\psi }}_{\lambda }(\omega )|}^2\mid \text{d}\omega \right]⩽{\mathbb{E}}_{\tau }\left[{\sum _{m=0,2,\dots ,k}\frac{1}{2\pi }\int {|{\widehat{f}}_{{\tau }_j}(\omega )|}^2\cdot |B_m{\eta }^m{\lambda }^m\frac{d^m}{d{\lambda }^m}|{\widehat{\psi }}_{\lambda }(\omega )|}^2|\text{d}\omega \right] \\ ⩽{\mathbb{E}}_{\tau }\left[\sum _{m=0,2,\dots ,k}{(E\eta )}^m\left({\Vert f_{{\tau }_j}\Vert }_1^2{\Psi }_m\wedge \frac{{\Vert f_{{\tau }_j}^{\prime }\Vert }_1^2{\Theta }_m}{{\lambda }^2}\right)\right] \\ ⩽\sum _{m=0,2,\dots ,k}{(E\eta )}^m\left(\Vert f{\Vert }_1^2{\Psi }_m\wedge \frac{4{\Vert f^{\prime }\Vert }_1^2{\Theta }_m}{{\lambda }^2}\right) \\ ⩽4\sum _{m=0,2,\dots ,k}{(E\eta )}^m{\Lambda }_m(\lambda ) \\ \lesssim {\Lambda }_0(\lambda )+\Lambda (\lambda ) \end{gathered}$$

since ${\Vert {\tau }_j\Vert }_{\infty }⩽\frac{1}{2}$ guarantees ${\Vert f_{{\tau }_j}^{\prime }\Vert }_1=\frac{1}{1-{\tau }_j}{\Vert f^{\prime }\Vert }_1⩽2{\Vert f^{\prime }\Vert }_1$. Also,

$$\begin{gathered} {\mathbb{E}}_{\epsilon }\left[{\frac{1}{2\pi }\int {|{\widehat{\epsilon }}_j(\omega )|}^2\cdot |A_{\lambda }|{\widehat{\psi }}_{\lambda }(\omega )|}^2|\text{d}\omega \right]⩽{\mathbb{E}}_{\epsilon }\left[{\sum _{m=0,2,\dots ,k}\frac{1}{2\pi }\int {|{\widehat{\epsilon }}_j(\omega )|}^2\cdot |B_m{\eta }^m{\lambda }^m\frac{d^m}{d{\lambda }^m}|{\widehat{\psi }}_{\lambda }(\omega )|}^2|\text{d}\omega \right] \\ ={\sigma }^2\left({\sum _{m=0,2,\dots ,k}\frac{1}{2\pi }\int |B_m{\eta }^m{\lambda }^m\frac{d^m}{d{\lambda }^m}|{\widehat{\psi }}_{\lambda }(\omega )|}^2|\text{d}\omega \right) \\ ⩽{\sigma }^2\sum _{m=0,2,\dots ,k}{(E\eta )}^m{\Psi }_m \\ ={\sigma }^2\Psi \end{gathered}$$

where the second line follows from Lemma D.1 in Appendix D and the next to last line from Corollary B.1 in Appendix B. We thus have

$${\mathbb{E}}_{\epsilon ,\tau }\left[Y_j\right]=0$$

$${\mathrm{Var}}_{\epsilon ,\tau }\left[Y_j\right]\lesssim {\sigma }^2\Psi \left({\Lambda }_0(\lambda )+\Lambda (\lambda )\right)$$

and an identical argument can be applied to the $\overline{Y_j}$ so that by Chebyshev’s inequality with probability at least 1 − 1/t²

$$\left|\frac{1}{M}\sum _{j=1}^M{Y}_j+{\overline{Y}}_j\right|⩽\left|\frac{1}{M}\sum _{j=1}^M{Y}_j\right|+\left|\frac{1}{M}\sum _{j=1}^M\overline{Y_j}\right|\lesssim t\sqrt{\Psi }\sqrt{{\Lambda }_0(\lambda )+\Lambda (\lambda )}\frac{\sigma }{\sqrt{M}}.$$

□

Lemma F.1

Assume ψ is k-admissible. Then,

$$\int {\lambda }^m\left(\frac{d^m}{d{\lambda }^m}{|{\widehat{\psi }}_{\lambda }(\omega )|}^2\right)\text{d}\omega =0$$ (F.1)

for all 1 ⩽ m ⩽ k.

Proof.

We recall that since ψ is k-admissible, ${|{\widehat{\psi }}_{\lambda }(\omega )|}^2\in {\mathbf{C}}^k(ℝ)$, and to simplify notation we let $g=|\widehat{\psi }{|}^2$ and

$$g_{\lambda }(\omega )=\frac{1}{\lambda }g\left(\frac{\omega }{\lambda }\right)={|{\widehat{\psi }}_{\lambda }(\omega )|}^2.$$

We first establish that

$${\lambda }^k\left(\frac{d}{d{\lambda }^k}{g}_{\lambda }(\omega )\right)=\frac{d}{d\omega }\left(-\omega {\lambda }^{k-1}\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega )\right)-(k-1){\lambda }^{k-1}\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega ).$$ (F.2)

The proof is by induction. When k = 1, we obtain

$$\text{LHS}\text{of}\text{Eqn}.\text{(F}.\text{2)}=\lambda \frac{d}{d\lambda }\left(\frac{1}{\lambda }g\left(\frac{\omega }{\lambda }\right)\right)=-\frac{\omega }{{\lambda }^2}{g}^{\prime }\left(\frac{\omega }{\lambda }\right)-\frac{1}{\lambda }g\left(\frac{\omega }{\lambda }\right)=-\omega g_{\lambda }^{\prime }(\omega )-g_{\lambda }(\omega )$$

and

$$\text{RHS}\text{of}\text{Eqn}.(\text{F}.2)=\frac{d}{d\omega }\left(-\omega g_{\lambda }(\omega )\right)=-\omega g_{\lambda }^{\prime }(\omega )-g_{\lambda }(\omega ),$$

so the base case is established. We now assume that Equation (F.2) holds and show it also holds for k+1 replacing k. By the inductive hypothesis

$$\begin{gathered} \frac{d}{d{\lambda }^k}{g}_{\lambda }(\omega )=\frac{d}{d\omega }\left(-\omega {\lambda }^{-1}\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega )\right)-(k-1){\lambda }^{-1}\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega ) \\ \frac{d}{d{\lambda }^{k+1}}{g}_{\lambda }(\omega )=\frac{d}{d\omega }\left(-\omega {\lambda }^{-1}\frac{d}{d{\lambda }^k}{g}_{\lambda }(\omega )+\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega )\omega {\lambda }^{-2}\right) \\ -(k-1)\left({\lambda }^{-1}\frac{d}{d{\lambda }^k}{g}_{\lambda }(\omega )+\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega )\left(-{\lambda }^{-2}\right)\right) \\ =\frac{d}{d\omega }\left(-\omega {\lambda }^{-1}\frac{d}{d{\lambda }^k}{g}_{\lambda }(\omega )\right)-(k-1){\lambda }^{-1}\frac{d}{d{\lambda }^k}{g}_{\lambda }(\omega ) \\ +\underset{=-{\lambda }^{-1}\frac{d}{d{\lambda }^k}{g}_{\lambda }(\omega )\text{by}\text{inductive}\text{hypothesis}}{\underbrace{\frac{d}{d\omega }\left(\omega {\lambda }^{-2}\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega )\right)+(k-1){\lambda }^{-2}\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega )}} \\ =\frac{d}{d\omega }\left(-\omega {\lambda }^{-1}\frac{d}{d{\lambda }^k}{g}_{\lambda }(\omega )\right)-k{\lambda }^{-1}\frac{d}{d{\lambda }^k}{g}_{\lambda }(\omega ) \end{gathered}$$

so that

$${\lambda }^{k+1}\frac{d}{d{\lambda }^{k+1}}{g}_{\lambda }(\omega )=\frac{d}{d\omega }\left(-\omega {\lambda }^k\frac{d}{d{\lambda }^k}{g}_{\lambda }(\omega )\right)-k{\lambda }^k\frac{d}{d{\lambda }^k}{g}_{\lambda }(\omega ).$$

Thus, (F.2) is established. We now use integration by parts to show (F.2) implies (F.1) in the Lemma. The proof of (F.1) is once again by induction. When k = 1, we have already shown

$$\lambda \left(\frac{d}{d\lambda }{g}_{\lambda }(\omega )\right)=-\omega g_{\lambda }^{\prime }(\omega )-g_{\lambda }(\omega ).$$ (F.3)

Integration by parts gives

$$\int \omega g_{\lambda }^{\prime }(\omega )\text{d}\omega ={\left(\omega g_{\lambda }(\omega )\right)|}_{-\infty }^{\infty }-\int g_{\lambda }(\omega )\text{d}\omega =\int g_{\lambda }(\omega )\text{d}\omega .$$

Note ωg_λ(ω) vanishes at ±∞ since $g\in {\mathbf{L}}^1(ℝ)$ guarantees $g_{\lambda }\in {\mathbf{L}}^1(ℝ)$, and thus g_λ must decay faster that ω⁻¹. Utilizing (F.3),

$$\int \omega g_{\lambda }^{\prime }(\omega )-g_{\lambda }(\omega )\text{d}\omega =0\Rightarrow \int \lambda \left(\frac{d}{d\lambda }{g}_{\lambda }(\omega )\right)\text{d}\omega =0$$

and the base case is established. We now assume

$$\int {\lambda }^{k-1}\left(\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega )\right)\text{d}\omega =0.$$

By integrating Equation (F.2), we obtain

$$\begin{gathered} \int {\lambda }^k\left(\frac{d}{d{\lambda }^k}{g}_{\lambda }(\omega )\right)d\omega \\ =\int \frac{d}{d\omega }\left(-\omega {\lambda }^{k-1}\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega )\right)\text{d}\omega -(k-1)\underset{=0\text{by}\text{induc}.\text{hypo}.}{\underbrace{\int {\lambda }^{k-1}\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega )\text{d}\omega }} \\ =\int -\omega \frac{d}{d\omega }\left({\lambda }^{k-1}\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega )\right)\text{d}\omega -\underset{=0\text{by}\text{induc}.\text{hypo}.}{\underbrace{\int {\lambda }^{k-1}\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega )\text{d}\omega }} \\ =-{\omega {\lambda }^{k-1}\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega )|}_{-\infty }^{\infty }+\underset{=0\text{by}\text{induc}.\text{hypo}.}{\underbrace{\int {\lambda }^{k-1}\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega )\text{d}\omega }} \\ =0. \end{gathered}$$

We are guaranteed $-\omega {\lambda }^{k-1}\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega )$ vanishes at ±∞ since in the proof of Lemma 4.3 we showed ${\lambda }^{k-1}\frac{d}{d{\lambda }^{k-1}}{g}_{\lambda }(\omega )={\sum }_{j=0}^{k-1}{C}_j{\omega }^j{g}_{\lambda }^{(j)}(\omega )$, and ${\omega }^j{g}_{\lambda }^{(j)}\in {\mathbf{L}}^1(ℝ)$ implies ${\omega }^{j+1}{g}_{\lambda }^{(j)}$ vanishes at ±∞. □

G. Moment estimation for noisy dilation MRA

In this appendix we outline a moment estimation procedure for noisy dilation MRA (Model 2) in the special case t = 0, i.e. signals are randomly dilated and subjected to additive noise but are not translated. This procedure is a generalization of the method presented in Section 6.3.

Given the additive noise level, the moments of the dilation distribution τ can be empirically estimated from the mean and variance of the random variables β_m(y_j) defined by

$${\beta }_m\left(y_j\right)={\int }_0^{2^{\ell }\pi }{\omega }^m{\widehat{y}}_j(\omega )\text{d}\omega$$ (G.1)

for integer m ⩾ 0. To account for the effect of additive noise on the above random variables, we define

$$g_m(\ell ,\sigma )={\int }_0^{2^{\ell }\pi }{\int }_0^{2^{\ell }\pi }\frac{2{\sigma }^2{\xi }^m{\omega }^m\sin \left(\frac{1}{2}(\xi -\omega )\right)}{(\xi -\omega )}\text{d}\omega \text{d}\xi$$ (G.2)

and an order m additive noise adjusted squared coefficient of variation by

$$C{V}_m:=\frac{\mathrm{Var}\left[{\beta }_m\left(y_j\right)\right]-g_m(\ell ,\sigma )}{{|\mathbb{E}\left[{\beta }_m\left(y_j\right)\right]|}^2}.$$ (G.3)

Remark G.1

If the noisy signals are supported in $\left[-\frac{N}{2},\frac{N}{2}\right]$ instead of $\left[-\frac{1}{2},\frac{1}{2}\right]$, (G.2) is replaced with

$$g_m(N,\ell ,\sigma )={\int }_0^{2^{\ell }\pi }{\int }_0^{2^{\ell }\pi }\frac{2{\sigma }^2{\xi }^m{\omega }^m\sin \left(\frac{N}{2}(\xi -\omega )\right)}{(\xi -\omega )}\text{d}\omega \text{d}\xi .$$

The following proposition mirrors Proposition 6.1 for dilation MRA; its proof appears at the end of Appendix G.

Proposition G.1

Assume Model 2 with t = 0 and CV₀, CV₁ defined by (G.1), (G.2) and (G.3). Then,

$$C{V}_0={\eta }^2+\left(3C_4-3\right){\eta }^4+O\left({\eta }^6\right)$$

$$C{V}_1=4{\eta }^2+\left(25C_4-33\right){\eta }^4+O\left({\eta }^6\right).$$

Once again we cannot compute CV_m exactly, but by replacing Var, $\mathbb{E}$ with their finite sample estimators, we obtain approximations ${\widetilde{CV}}_m$ that can be used to define estimators of the dilation moments.

Definition G.1

Assume Model 3 with t = 0 and ${\widetilde{CV}}_0$, ${\widetilde{CV}}_1$ the empirical counterparts of (G.3). Define the second-order estimator of η² by ${\tilde{\eta }}^2={\widetilde{CV}}_0$. Define the fourth-order estimators of (η², C₄η⁴) by the unique positive solution (${\tilde{\eta }}^2$, ${\tilde{C}}_4$) of

$${\widetilde{CV}}_0={\eta }^2+\left(3C_4-3\right){\eta }^4$$

$${\widetilde{CV}}_1=4{\eta }^2+\left(25C_4-33\right){\eta }^4.$$

As M → ∞, the second-order moment estimator is accurate up to O(η⁴) and the fourth-order moment estimators are accurate up to O(η⁶). However, in the finite sample regime, the g_m(ℓ, σ) appearing in (G.3) will be replaced with $g_m(\ell ,\sigma )\pm O\left({\sigma }^2/\sqrt{M}\right)$, so that the estimators given in Definition G.1 are subject to an error of order $O\left({\sigma }^2/\sqrt{M}\right)$. More generally, the additive noise fluctuations imply that to estimate the first k/2 even moments of τ up to an O(η^k+1) error will require ${\sigma }^2/\sqrt{M}⩽{\eta }^{k+1}$, or M ⩾ σ⁴η^−2(k+1).

Having established an empirical moment estimation procedure for noisy dilation MRA when t = 0, we repeat the simulations of Section 5.2 on the restricted model, but estimate the additive and dilation moments empirically. Since accurately estimating the moments of τ is difficult for σ large, we make three modifications to the oracle set-up. First, we lower the additive noise level by a factor of 2 from the oracle simulations, and consider all parameter combinations resulting from σ = 2⁻⁵, 2⁻⁴ (giving SNR = 9.0, 2.2) and η = 0.06, 0.12. Secondly, we take M substantially larger than for the oracle simulations, with 16, 384 ⩽ M ⩽ 370, 727. Thirdly, we compute WSC k = 4 only for large dilations. For large dilations (η², C₄η⁴) are approximated with fourth-order estimators, while for small dilations η² is approximated with a second-order estimator (see Definition G.1).

Results are shown in Fig. G7, and the same overall behavior observed in the oracle simulations for large M holds. The additive noise level was estimated empirically as described in Section 6.2. For the medium- and high-frequency signal, WSC k = 2 has substantially smaller error than both PS k = 0 and WSC k = 0; for the large-frequency signal, the error is decreased by at least a factor of 2 for large dilations and a factor of 4 for small dilations relative to both zero order estimators. When WSC k = 4 is defined, it has a smaller error than WSC k = 2 for the high-frequency signal, while WSC k = 2 is preferable for the low- and medium-frequency signal. We observe that for the oracle simulations WSC k = 4 is preferable for all frequencies, so this is most likely due to error in the moment estimation degrading the WSC k = 4 estimator. For the low-frequency signal, PS k = 0 once again achieves the smallest error for small dilations, while for large dilations the higher order wavelet methods appear to surpass PS k = 0 for M large enough.

Fig. G7.

L² error with standard error bars for noisy dilation MRA model (t = 0, empirical moment estimation). First, second and third column shows results for low-, medium- and high-frequency signals. All plots have the same axis limits.

Proof of Proposition G.1.

Since $y_j=L_{{\tau }_j}f+{\epsilon }_j$, we have

$$\begin{gathered} \mathbb{E}\left[{\beta }_m\left(y_j\right)\right]=\mathbb{E}\left[{\int }_0^{2^{\ell }\pi }{\omega }^m\left({\widehat{f}}_{{\tau }_j}(\omega )+{\widehat{\epsilon }}_j(\omega )\right)\text{d}\omega \right] \\ =\mathbb{E}\left[{\int }_0^{2^{\ell }\pi }{\omega }^m{\widehat{f}}_{{\tau }_j}(\omega )\text{d}\omega \right] \\ =\mathbb{E}\left[{\int }_0^{2^{\ell }\pi }{\omega }^m\widehat{f}\left(\left(1-{\tau }_j\right)\omega \right)\text{d}\omega \right] \\ =\mathbb{E}\left[{\int }_0^{2^2\pi \left(1-{\tau }_j\right)}\frac{{\xi }^m}{{\left(1-{\tau }_j\right)}^m}\widehat{f}(\xi )\frac{\text{d}\xi }{\left(1-{\tau }_j\right)}\right] \\ ={\beta }_m(f)\mathbb{E}\left[{\left(1-{\tau }_j\right)}^{-(m+1)}\right]. \end{gathered}$$

We now compute the variance. We first establish that

$$g_m(\ell ,\sigma )=\mathbb{E}\left[\left({\int }_0^{2^{\ell }\pi }{\omega }^m{\widehat{\epsilon }}_j(\omega )\text{d}\omega \right)\left({\int }_0^{2^{\ell }\pi }{\omega }^m\overline{{\widehat{\epsilon }}_j(\omega )}\text{d}\omega \right)\right].$$

By Thm 4.5 of [49]

$$\begin{gathered} \mathbb{E}\left[{\widehat{\epsilon }}_j(\omega )\overline{{\widehat{\epsilon }}_j(\xi )}\right]=\mathbb{E}\left[\left({\int }_{-1/2}^{1/2}{e}^{-i\omega t}\text{d}{B}_t\right)\left({\int }_{-1/2}^{1/2}{e}^{i\xi t}\text{d}{B}_t\right)\right] \\ ={\sigma }^2{\int }_{-1/2}^{1/2}{e}^{i(\xi -\omega )t}\text{d}t \\ =\frac{2{\sigma }^2\sin \left(\frac{1}{2}(\xi -\omega )\right)}{(\xi -\omega )} \end{gathered}$$

so that

$$\begin{gathered} \mathbb{E}\left[\left({\int }_0^{2^{\ell }\pi }{\widehat{\epsilon }}_j(\omega )\text{d}\omega \right)\left({\int }_0^{2^{\ell }\pi }\overline{{\widehat{\epsilon }}_j(\omega )}\text{d}\omega \right)\right]={\int }_0^{2^{\ell }\pi }{\int }_0^{2^{\ell }\pi }{\omega }^m{\xi }^m\mathbb{E}\left[{\widehat{\epsilon }}_j(\omega )\overline{{\widehat{\epsilon }}_j(\xi )}\right]\text{d}\omega \text{d}\xi \\ ={\int }_0^{2^{\ell }\pi }{\int }_0^{2^{\ell }\pi }{\omega }^m{\xi }^m\frac{2{\sigma }^2\sin \left(\frac{1}{2}(\xi -\omega )\right)}{(\xi -\omega )}\text{d}\omega \text{d}\xi \\ =g_m(\ell ,\sigma ). \end{gathered}$$

We thus obtain

$$\begin{gathered} \left[{|{\beta }_m\left(y_j\right)|}^2\right]=\mathbb{E}\left[\left({\int }_0^{2^{\ell }\pi }{\omega }^m\left({\widehat{f}}_{{\tau }_j}(\omega )+{\widehat{\epsilon }}_j(\omega )\right)\text{d}\omega \right)\left({\int }_0^{2^{\ell }\pi }{\omega }^m\left(\overline{{\widehat{f}}_{{\tau }_j}(\omega )}+\overline{{\widehat{\epsilon }}_j(\omega )}\right)\text{d}\omega \right)\right] \\ =\mathbb{E}[\left({\int }_0^{2^{\ell }\pi }{\omega }^m\widehat{f}\left(\left(1-{\tau }_j\right)\omega \right)\text{d}\omega \right)\left({\int }_0^{2^{\ell }\pi }{\omega }^m\overline{\widehat{f}\left(\left(1-{\tau }_j\right)\omega \right)}\text{d}\omega \right) \\ +\left({\int }_0^{2^{\ell }\pi }{\omega }^m{\widehat{\epsilon }}_j(\omega )\text{d}\omega \right)\left({\int }_0^{2^{\ell }\pi }{\omega }^m\overline{{\widehat{\epsilon }}_j(\omega )}\text{d}\omega \right)] \\ =\mathbb{E}\left[{\left(1-{\tau }_j\right)}^{-2(m+1)}{\beta }_m(f)\overline{{\beta }_m(f)}\right]+g_m(\ell ,\sigma ) \\ ={|{\beta }_m(f)|}^2\mathbb{E}\left[{\left(1-{\tau }_j\right)}^{-2(m+1)}\right]+g_m(\ell ,\sigma ). \end{gathered}$$

Thus,

$$\begin{gathered} \mathrm{Var}\left[{\beta }_m\left(y_j\right)\right]-g_m(\ell ,\sigma )=\mathbb{E}\left[{|{\beta }_m\left(y_j\right)|}^2\right]-g_m(\ell ,\sigma )-{|\mathbb{E}\left[{\beta }_m\left(y_j\right)\right]|}^2 \\ ={|{\beta }_m(f)|}^2\mathbb{E}\left[{\left(1-{\tau }_j\right)}^{-2(m+1)}\right]-{|{\beta }_m(f)|}^2{\left(\mathbb{E}\left[{\left(1-{\tau }_j\right)}^{-(m+1)}\right]\right)}^2. \end{gathered}$$

Dividing by ${|\mathbb{E}\left[{\beta }_m\left(y_j\right)\right]|}^2$ gives

$$C{V}_m=\frac{\mathbb{E}\left[{\left(1-{\tau }_j\right)}^{-2(m+1)}\right]}{{\left(\mathbb{E}\left[{\left(1-{\tau }_j\right)}^{-(m+1)}\right]\right)}^2}-1,$$

and the remainder of the proof is identical to the proof of Proposition 6.1. □

H. Additional simulations for noisy dilation MRA

We investigate the L² error of estimating the power spectrum using PS (k = 0) and WSC (k = 0, 2, 4) for three additional high-frequency functions:

$$f_4(x)=1.175\cos (32x)\cdot \mathbf{1}(x\in [-0.2,0.2])$$

$$f_5(x)=0.299{\exp }^{-0.04x^2}\cos \left(30x+1.5x^2\right)$$

$$f_6(x)=(2.304/\pi )\cos (35x)\mathrm{sinc}(3x).$$

The multiplicative constants were chosen so that the L² norms of f₄, f₅, f₆ are comparable with the L² norms of the Gabor signals f₁, f₂, f₃ defined in Section 4.4. The signal f₄ is not continuous and has compact support, with a slowly decaying, oscillating Fourier transform given by ${\widehat{f}}_4(\omega )/0.47=\mathrm{sinc}(0.2(\omega -32))+\mathrm{sinc}(0.2(-\omega -32))$. The signal f₅ is a linear chirp with a constantly varying instantaneous frequency. The signal f₆ is slowly decaying in space, with a discontinuous Fourier transform of compact support given by ${\widehat{f}}_6(\omega )/0.384=\mathbf{1}(\omega \in [-38,-32])+\mathbf{1}(\omega \in [32,38])$.

Implementation details were as described in Section 6, and simulations were run with oracle moment estimation on the full model (parameter values as described in Section 5.2). Figure H8 shows the L² error. As for the high-frequency Gabor in Section 5.2, WSC (k = 2) and WSC (k = 4) significantly outperformed the zero order estimators. In addition for large dilations, the WSC (k = 4) outperformed WSC (k = 2) on f₄ and f₆.

Fig. H8.

L² error with standard error bars for noisy dilation MRA model (oracle moment estimation). First, second and third columns show results for f₄, f₅ and f₆. All plots for the same signal have the same axis limits.

I. Expectation maximization algorithm for noisy dilation MRA

In this appendix we discuss how the expectation-maximization (EM) algorithm proposed in [1] can be extended to solve noisy dilation MRA. We first summarize the EM framework, which differentiates between observed data $y={\left\{y_j\right\}}_{j=1}^M$, latent variables $s={\left\{s_j\right\}}_{j=1}^M$ and model parameters x. The goal is to produce the x that maximizes the marginalized likelihood function

$$p(y|x)=\int p(y,s|x)\text{d}s.$$

Maximizing p(y|x) directly is generally not tenable because enumerating the various values for s is too costly. However, EM algorithms can be used to find local maxima of the above function, by iterating between estimating the conditional distribution of latent variables given the current estimate of parameters (E-step) and estimating parameters given the current estimate of the conditional distribution of latent variables (M-step). Specifically, the iterative procedure updates x^k, the current estimate of x, by

$$Q\left(x|x^k\right)={\mathbb{E}}_{s|y,x^k}[\log p(y,s|x)]\text{E-step}$$ (I.1)

$$x^{k+1}=\arg \underset{x}{\max }Q\left(x|x^k\right)\text{M-step}.$$ (I.2)

Since (under certain conditions) log p(y|x) improves at least as much as Q at each iteration [30], the algorithm converges to a local maximum of p(y|x). This framework can be applied to noisy dilation MRA, and explicit formulas for both the E-step and M-step can be derived. Assume for simplicity that signals have been discretized to have length n and that the translation distribution ρ_t and dilation distribution ρ_τ are unknown and also discrete with n possible values ${\left\{t^{\ell }\right\}}_{\ell =1}^n$ and ${\left\{{\tau }^q\right\}}_{q=1}^n$, respectively. Letting x = (f, ρ_t, ρ_τ) denote the parameters, sj = (t^j, τ^j) denote the latent/nuisance variables, and px denote conditioning on x, the likelihood function has form

$$p(y,s|x)=p_x(y|s)p_x(s)=\prod _{j=1}^M\frac{1}{{\left(2\pi {\sigma }^2\right)}^{\frac{n}{2}}}\exp \left(-\frac{1}{2{\sigma }^2}{\Vert L_{{\tau }_j}{T}_{t_j}f-y_j\Vert }_2^2\right){\rho }_t\left(t_j\right){\rho }_{\tau }\left({\tau }_j\right).$$

Thus (up to a constant), the log likelihood has form

$$\log p(y,s|x)=\sum _{j=1}^M-\frac{1}{2{\sigma }^2}{\Vert L_{{\tau }_j}{T}_{t_j}f-y_j\Vert }_2^2+\sum _{j=1}^M\log {\rho }_t\left(t_j\right)+\sum _{j=1}^M\log {\rho }_{\tau }\left({\tau }_j\right).$$ (I.3)

Given the current estimate $x^k=\left(f^k,{\rho }_t^k,{\rho }_{\tau }^k\right)$ of parameters, the E-step is performed by first computing the conditional distribution of the latent variables

$$w_k^{\ell ,q,j}=\mathbb{P}\left(t_j=t^{\ell },{\tau }_j={\tau }^q|x^k\right)=C_k^j\exp \left(-\frac{1}{2{\sigma }^2}{\Vert L_{{\tau }_j}{T}_{t_j}{f}^k-y_j\Vert }_2^2\right){\rho }_t^k\left(t^{\ell }\right){\rho }_{\tau }^k\left({\tau }^q\right),$$ (I.4)

where $C_k^j$ is a normalizing constant so that ${\sum }_{\ell ,q}{w}_k^{\ell ,q,j}=1$. These weights are then used to compute Q, that is, by combining (I1), (I3) and (I4):

$$Q\left(f,{\rho }_t,{\rho }_{\tau }|f^k,{\rho }_t^k,{\rho }_{\tau }^k\right)=\sum _{j=1}^M\sum _{\ell =1}^n\sum _{q=1}^n{w}_k^{\ell ,q,j}\left(-\frac{1}{2{\sigma }^2}{\Vert L_{{\tau }_j}{T}_{t_j}f-y_j\Vert }_2^2+\log {\rho }_t\left(t^{\ell }\right)+\log {\rho }_{\tau }\left({\tau }^q\right)\right),$$ (I.5)

up to a constant. The M-step is then computed by

$$\left(f^{k+1},{\rho }_t^{k+1},{\rho }_{\tau }^{k+1}\right)=\arg \underset{f,{\rho }_t,{\rho }_{\tau }}{\max }Q\left(f,{\rho }_t,{\rho }_{\tau }|f^k,{\rho }_t^k,{\rho }_{\tau }^k\right).$$ (I.6)

Since f, ρ_t, ρ_τ all appear in distinct sums in (I5), performing the maximization in (I6) is straightforward. Since ${\Vert L_{{\tau }_j}{T}_{t_j}f-y_i\Vert }_2^2=\frac{1}{1-{\tau }_q}f-T_{\mathcal{l}}^{-1}{L}_{\tau }^{-1}{y}_j{\Vert }_2^2,$ it is easy to check that

$${\Vert L_{{\tau }_j}{T}_{t_j}f-y_j\Vert }_2^2=\frac{1}{1-{\tau }_q}f-T_{\ell }^{-1}{L}_{\tau }^{-1}{y}_j{\Vert }_2^2$$

$$f^{k+1}=\frac{1}{C}\sum _{j=1}^M\sum _{\ell =1}^n\sum _{q=1}^n\frac{w_k^{\ell ,q,j}}{\left(1-{\tau }_q\right)}{T}_{\ell }^{-1}{L}_{\tau }^{-1}{y}_j,C=\sum _{j=1}^M\sum _{\ell =1}^n\sum _{q=1}^n\frac{w_k^{\ell ,q,j}}{\left(1-{\tau }_q\right)}.$$ (I.7)

Using Lemma 15 in [1], one can also obtain closed form expressions for the updates to ${\rho }_t^k$, ${\rho }_{\tau }^k$:

$${\rho }_t^{k+1}\left(t^{\ell }\right)=\frac{{\tilde{w}}_k^{\ell }}{{\sum }_{{\ell }^{\prime }}{\tilde{w}}_k^{{\ell }^{\prime }}}\text{for}{\tilde{w}}_k^{\ell }=\sum _j\sum _q{w}_k^{\ell ,q,j},{\rho }_{\tau }^{k+1}\left({\tau }^q\right)=\frac{{\tilde{v}}_k^q}{{\sum }_{q^{\prime }}{\tilde{v}}_k^{q^q}}\text{for}{\tilde{v}}_k^q=\sum _j\sum _{\ell }{w}_k^{\ell ,q,j}.$$

Note when a discrete signal defined on some fixed grid is dilated, its dilation is defined on a different grid. Thus, computing (I.4) and (I.7) will involve off-grid interpolation, a subtlety not arising in classic MRA, and this interpolation may contribute additional error. We also note that one can always force the translation distribution to be uniform by retranslating the signals uniformly, and in this case all sums over ℓ in this section could be eliminated. This would improve the computational complexity of the algorithm but may be disadvantageous in terms of sample complexity, as in classic MRA a uniform translation distribution requires a larger sample size for accurate estimation than an aperiodic translation distribution [1].

J. Supporting results: stochastic calculus

This appendix contains several stochastic calculus results that are used to control the statistics of the additive noise. Proposition J.1 is a simple generalization of Thm 4.5 of [49]. Proposition J.2 controls the second moment of the stochastic quantity in Proposition J.1 and is in fact a special case of Proposition J.3. Both Propositions J.2 and J.3 are proved with standard techniques from stochastic calculus, and for brevity we omit the proofs.

Proposition J.1

Assume ${\int }_0^{T}f{(t)}^2\text{d}t<\infty$, ${\int }_0^T{\overline{f(t)}}^2\text{d}t<\propto$, and let B_t be a Brownian motion with variance σ². Then,

$$\mathbb{E}\left[\left({\int }_0^{T}f(t)\text{d}{B}_t\right)\left({\int }_0^T\overline{f(t)}\text{d}{B}_t\right)\right]={\sigma }^2{\int }_0^{T}f(t)\overline{f(t)}\text{d}t.$$

Proposition J.2

Let f(t) be a bounded and continuous complex deterministic function on [0, T], and let B_t be a Brownian motion with variance σ². Then, for a fixed non-random time T, we have

$$\mathbb{E}\left[{\left({\int }_0^{T}f(t)\text{d}{B}_t\right)}^2{\left({\int }_0^T\overline{f(t)}\text{d}{B}_t\right)}^2\right]=2{\sigma }^4{\left({\int }_0^T|f(t){|}^2\text{d}t\right)}^2+{\sigma }^4\left({\int }_0^{T}f{(t)}^2\text{d}t\right)\left({\int }_0^T{\overline{f(t)}}^2\text{d}t\right).$$

Corollary J.1

When f(t) is real, the above reduces to

$$\mathbb{E}\left[{\left({\int }_0^{T}f(t)\text{d}{B}_t\right)}^4\right]=3{\sigma }^4{\left({\int }_0^{T}f{(t)}^2\text{d}t\right)}^2.$$

Proposition J.3

Let f(t), g(t) be bounded and continuous complex deterministic functions on [0, T], and let B_t be a Brownian motion with variance σ². Then, for a fixed non-random time T, we have

$$\begin{gathered} \mathbb{E}\left[\left({\int }_0^{T}f(t)\text{d}{B}_t\right)\left({\int }_0^T\overline{f(t)}\text{d}{B}_t\right)\left({\int }_0^{T}g(t)\text{d}{B}_t\right)\left({\int }_0^T\overline{g(t)}\text{d}{B}_t\right)\right] \\ ={\sigma }^4[\left({\int }_0^{T}f(t)g(t)\text{d}t\right)\left({\int }_0^T\overline{f(t)g(t)}\text{d}t\right)+\left({\int }_0^{T}f(t)\overline{g(t)}\text{d}t\right)\left({\int }_0^T\overline{f(t)}g(t)\text{d}t\right) \\ +\left({\int }_0^T|f(t){|}^2\text{d}t\right)\left({\int }_0^T|g(t){|}^2\text{d}t\right)]. \end{gathered}$$