Waveform information from quantum mechanical entropy

Abstract

Although the entropy of a given signal-type waveform is technically zero, it is nonetheless desirable to use entropic measures to quantify the associated information. Several such prescriptions have been advanced in the literature but none are generally successful. Here, we report that the Fourier-conjugated ‘total entropy’ associated with quantum-mechanical probabilistic amplitude functions (PAFs) is a meaningful measure of information in non-probabilistic real waveforms, with either the waveform itself or its (normalized) analytic representation acting in the role of the PAF. Detailed numerical calculations are presented for both adaptations, showing the expected informatic behaviours in a variety of rudimentary scenarios. Particularly noteworthy are the sensitivity to the degree of randomness in a sequence of pulses and potential for detection of weak signals.

1. Introduction

Let α=α(t) be a real, square-integrable function of a real independent variable, t. The following analysis is restricted to cases where t is one-dimensional; the generalization for arbitrary dimensionality follows readily therefrom. For convenience, let the L²-norm, or energy, of α, written here as ∥α∥, be unity. Finally, suppose that α represents some informatic signal such as an audio waveform or digital transmission. It is appropriate to consider t to represent a time-like variable throughout, but the analysis is generic.

In the conventional probabilistic paradigm of information theory, it is meaningful to attribute a quantity of entropy to the process by which signals such as α are generated in a given system but not to any particular signal [1]. Entropy, however, is fundamentally a measure of both indeterminacy and information, and it is therefore appropriate to seek some measure of entropy to quantify the information intrinsic to a given α. The difficulty in such an endeavour is that standard entropy measures are defined in terms of probabilistic distributions or probabilistic density functions (PDF's). In principle, some special treatment is therefore required in order to adapt measures of entropy for use as measures of information in non-probabilistic waveforms like α. Existing approaches generally rely on some secondary distribution or other probabilistic construct to which a measure of indeterminacy may be attributed. See for instance [2–4]. Although a variety of such well-formulated methods have been introduced, all feature some non-trivial dependence on model or parametrization. Consequently, there is no single generally successful prescription for the informatic entropy of a waveform like α.

There exist additional difficulties in formulating a rigorous, general entropic measure of information that are inherent to the Shannon-type measures themselves. For instance, consider the differential Shannon entropy, which is defined here generally such that

$$H\{\lambda \}\equiv -\int \lambda \log (\lambda )\hspace{0.17em}\mathrm{d}t$$ 1.1

for any given λ=λ(t) and for any independent variable. In equation (1.1) and throughout $z\log (z)$ is defined to vanish for any z=0. Integrals with no explicit limits implicitly span the real axis. Naturally, equation (1.1) is meaningful only when λ is a meaningful PDF, which restricts us to real, non-negative, non-singular λ such that

$$\int \lambda \hspace{0.17em}\mathrm{d}t=1.$$ 1.2

In the probabilistic context H{λ} is meant to measure the indeterminacy in a continuous, real variable whose outcomes are governed by a PDF, λ. Interpreted more broadly, in abstraction from any particular variable, H{λ} is an intrinsic informatic attribute of λ. Strictly speaking, however, H{λ} alone is a somewhat unsatisfactory measure in both cases because the argument of its logarithm is not generally dimensionless [5]. Consequently, H{λ} is sensitive to the unit of t. Specifically for any real, positive b we may express λ equivalently as a function λ′=λ′(t′) of t′=t/b such that λ′(t′) dt′=ρ dt, hence λ′=bλ. The corresponding entropy, H{λ′}, reduces to $H\{\lambda \}-\log (b)$, which is to say that H{λ} is not invariant under a simple scale-transformation in t.

Moreover, H{λ} may be negative and is unbounded. While it is reasonable to expect the entropy associated with some infinitely expansive PDF to be infinitely large, the negativity and lack of a lower bound pose particularly challenging problems for H{λ} as a meaningful measure per se for an arbitrary λ.

In the quantum mechanical context, where a given PDF is stated as the squared absolute magnitude of a probabilistic amplitude function (PAF), there exists a natural resolution to the problems cited with respect to a single measure of the form H{λ}. Specifically, let ψ=ψ(t) be a generally complex, square-integrable function of t with unit energy. As such the squared absolute magnitude of ψ, defined here as

$$\rho =\rho (t)\equiv |\psi (t){|}^2$$ 1.3

is generally meaningful as a PDF and ψ is accordingly meaningful as a PAF. The Fourier counterpart to ρ is the spectral density,

$$\sigma =\sigma (\omega )\equiv |\hat{\psi }{|}^2,$$ 1.4

where

$$\hat{\psi }=\hat{\psi }(\omega )=\frac{1}{\sqrt{2\pi }}\int \psi \hspace{0.17em}{\mathrm{e}}^{-\mathrm{i}\omega t}\hspace{0.17em}\mathrm{d}t$$ 1.5

is the Fourier transform (FT) of ψ and ω∈ℜ is the independent variable conjugate to t. Throughout this work, $\hat{y}=\hat{y}(\omega )$ is the FT of any y=y(t). As the ‘hat’-notation is inconvenient in some cases, let also $\mathcal{F}\{y\}=\mathcal{F}\{y\}(\omega )$ be the FT of any y. The inverse FT of $\hat{y}$ is accordingly written here as ${\mathcal{F}}^{-1}\{\hat{y}\}=y$, having the explicit form

$${\mathcal{F}}^{-1}\{\hat{y}\}={\mathcal{F}}^{-1}\{\hat{y}\}(t)\equiv \frac{1}{\sqrt{2\pi }}\int \hat{y}\hspace{0.17em}{\mathrm{e}}^{\mathrm{i}\omega t}\hspace{0.17em}\mathrm{d}\omega .$$ 1.6

Together ρ and σ form a dimensionless two-variable PDF

$$f=f(t,\omega )\equiv \rho \sigma ,$$ 1.7

with

$$\int \int f\hspace{0.17em}\mathrm{d}t\hspace{0.17em}\hspace{0.17em}\mathrm{d}\omega =1.$$ 1.8

The total differential Shannon entropy associated with f is

$$-\int \int f\log (\hspace{0.17em}f)\hspace{0.17em}\mathrm{d}t\hspace{0.17em}\mathrm{d}\omega =H\{\rho \}+H\{\sigma \}.$$ 1.9

As f follows from a given ψ, it is appropriate to interpret the measure in equation (1.9) as an attribute thereof. For convenience, let us therefore define

$$S\{\psi \}\equiv H\{|\psi {|}^2\}+H\{|\hat{\psi }{|}^2\}$$ 1.10

for a given ψ.

In order to examine the sensitivity of S{ψ} to arbitrary scale-transformations of the form $t\to t^{\prime }$, we must introduce also the implicit conjugate transformation $\omega \to {\omega }^{\prime }$, where ω′=bω. Let σ′=σ′(ω′) be defined such that σ′ dω′=σ dω. It follows directly from the appropriate substitutions in the explicit form of H{σ′} that $H\{{\sigma }^{\prime }\}=H\{\sigma \}+\log (b)$. Consequently, S{ψ} is scale-invariant and thus insensitive to the unit with which t is measured. In addition to being unit-independent S{ψ} is subject to the lower bound

$$S\{\psi \}\ge 1+\log (\pi )$$ 1.11

in one dimension [6]. Furthermore, S{ψ} has emerged as a fundamentally significant information-theoretic measure in quantum systems. See, for instance, [7–12].

Although our nominal signal, α, has unit energy like ψ, it is not a PAF. Apart from being exclusively real, it simply does not represent any probabilistic process, nor does it necessarily represent any amplitude. In the conventional understanding, therefore, some secondary complex amplitude function must be constructed in association with α in order to adapt S{ψ} for waveforms like α. As in the case of secondary PDF's, such an endeavour generally introduces model-dependence and the need for some parametrization.

The purpose of this work, however, is to demonstrate that the total entropy in equation (1.10) is a sensitive, meaningful measure of information in non-probabilistic waveforms such as α without requiring any external models or parameters. Specifically, in addition to considering the most simple adaptation, S{α}, we consider S{ϕ}, where ϕ is the normalized analytic representation (AR) of a given α. Section 2 describes the normalized AR and the motivation for considering it as a characteristic amplitude function for a given α. In §3, are defined the relevant discrete quantities necessary for computation. Section 4 explores the informatic behaviours of S{α} and S{ϕ} with explicit numerical studies of their discrete analogues. Section 5 provides a brief summary and discussion of the results.

2. Analytic representation as an amplitude function

It follows from the realness of α, equation (1.5) and the Euler theorem that $\hat{\alpha }$ is Hermitian. In other words, half of the Fourier spectrum associated with α is informatically redundant. Eliminating the redundant contributions thereby produces a generally complex function that contains the same information as does α. The function so produced is proportional to the well-known AR of α, whose imaginary part is the Hilbert transform of α. According to the central thesis of this present work, the AR of a given signal, with the appropriate normalization, is an appropriate candidate for a PAF-like function that is characteristic to a given real waveform.

Before proceeding, it is instructive to review how the AR and the Hilbert transform (HT) are related generally. Let g=g(t) be a real, square-integrable function of t. By expressing g explicitly as the inverse FT of $\hat{g}$ and exploiting the Hermitian symmetry of $\hat{g}$, we obtain

$$g=\frac{1}{\sqrt{2\pi }}{\int }_0^{\mathrm{\infty }}({\hat{g}}^{\ast }\hspace{0.17em}{\mathrm{e}}^{-\mathrm{i}\omega t}+\hat{g}\hspace{0.17em}{\mathrm{e}}^{\mathrm{i}\omega t})\hspace{0.17em}\mathrm{d}\omega ,$$ 2.1

where ${\hat{g}}^{\ast }$ is the complex conjugate of $\hat{g}$. For a given g, it is therefore natural to define

$$\gamma =\gamma (t)\equiv \frac{1}{\sqrt{2\pi }}{\int }_0^{\mathrm{\infty }}\hat{g}\hspace{0.17em}{\mathrm{e}}^{\mathrm{i}\omega t}\hspace{0.17em}\mathrm{d}\omega$$ 2.2

such that equation (2.1) becomes

$$g={\gamma }^{\ast }+\gamma ,$$ 2.3

where $\gamma \in \mathbb{C}$.

By construction γ represents the contributions to g from non-negative Fourier frequencies, whereas γ* represents the contributions from non-positive frequencies. The AR of a given g, written here as $\mathcal{A}\{g\}=\mathcal{A}\{g\}(t)$, may be defined most simply as

$$\mathcal{A}\{g\}(t)\equiv 2\gamma ,$$ 2.4

hence

$$\mathrm{Re}[\mathcal{A}\{g\}]=g.$$ 2.5

It is also appropriate to define the AR of g in terms of its Fourier transform according to

$$\mathcal{F}\{\mathcal{A}\{g\}\}=\{\begin{array}{ll}2\hat{g}& \mathrm{for}\hspace{0.17em}\omega >0\\ \hat{g}& \mathrm{for}\hspace{0.17em}\omega =0\\ 0& \mathrm{for}\hspace{0.17em}\omega <0.\end{array}$$ 2.6

The above equation ensures that the zero-frequency Fourier component contributes equally to $\mathcal{A}\{g\}$ and its negative-frequency analogue.

Let h=h(t) be the imaginary part of a given $\mathcal{A}\{g\}$, hence

$$\mathrm{i}h=\gamma -{\gamma }^{\ast }.$$ 2.7

With γ expressed explicitly, substituting the dummy variable τ for t in $\hat{g}$, and with additional exploitation of the Hermitian symmetry of $\hat{g}$, equation (2.7) produces

$$h=\frac{1}{\pi }{\int }_0^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}g(\tau )\sin (\omega (t-\tau ))\hspace{0.17em}\mathrm{d}\tau \hspace{0.17em}\hspace{0.17em}\mathrm{d}\omega .$$ 2.8

Finally, integrating equation (2.8) with respect to ω in accordance with the Riemann–Lebesque Lemma produces

$$h=\frac{1}{\pi }\mathrm{PV}[\int \frac{g(\tau )}{t-\tau }\hspace{0.17em}\mathrm{d}\tau ],$$ 2.9

where ‘PV’ signifies the principal value of the integral. The right-hand side of equation (2.9) is identical to the Hilbert transform of g, written here as $\mathfrak{H}\{g\}=\mathfrak{H}\{g\}(t)$, where

$$\mathfrak{H}\{g\}(t)\equiv g\hspace{0.17em}\ast \hspace{0.17em}\frac{1}{\pi t}$$ 2.10

for any g.

Returning to the primary subject, suppose that a PAF is sought to represent a given α. In order to be a meaningful descriptor of α, the PAF should contain essentially the same information as does α and its absolute magnitude should envelop α. The above criteria are realized naturally through the analytic representation. In particular, due to the orthogonality between g and $\mathfrak{H}\{g\}$, the imaginary part of $\mathcal{A}\{g\}$ is intrinsically in quadrature with the real part. Consequently, $|\mathcal{A}\{g\}|$ naturally forms an envelope function for g. Furthermore because the HT preserves the norm of square-integrable functions, we have $\parallel \mathcal{A}\{g\}\parallel =2\parallel g\parallel$. We therefore may readily scale $\mathcal{A}\{\alpha \}$ to create a unity-normalized analogue. Specifically, for a given α let

$$\varphi \{\alpha \}(t)\equiv \frac{1}{\sqrt{2}}(\alpha +\mathrm{i}\mathfrak{H}\{\alpha \})=\frac{\mathcal{A}\{\alpha \}}{\sqrt{2}}$$ 2.11

be defined to represent the characteristic PAF associated with a given α. For convenience, we use ϕ=ϕ(t)=ϕ{α}=ϕ{α}(t). Note that we may express ϕ equivalently as

$$\varphi =\frac{1}{\sqrt{\pi }}{\int }_0^{\mathrm{\infty }}\hat{\alpha }\hspace{0.17em}{\mathrm{e}}^{\mathrm{i}\omega t}\hspace{0.17em}\mathrm{d}\omega ,$$ 2.12

which is analogous to $\sqrt{2}\gamma$.

Associated with ϕ is the time-domain PDF-like function, |ϕ|². The corresponding spectral PDF is likewise $|\hat{\varphi }{|}^2$, where

$$\hat{\varphi }=\{\begin{array}{ll}\sqrt{2}\hat{\alpha }& \mathrm{for}\hspace{0.17em}\omega >0\\ \frac{\hat{\alpha }}{\sqrt{2}}& \mathrm{for}\hspace{0.17em}\omega =0\\ 0& \mathrm{for}\hspace{0.17em}\omega <0.\end{array}$$ 2.13

From a given α, it is thereby possible to construct the unitless, two-dimensional PDF,

$$\epsilon =\epsilon (t,w)\equiv |\varphi {|}^2|\hat{\varphi }{|}^2,$$ 2.14

which has the form of f.

As a rudimentary example suppose that we have

$$\alpha =c\exp \left(\frac{-t^2}{d^2}\right)\cos (\theta t),$$ 2.15

where the positive constants d and θ are external parameters, and

$$c={\left(\frac{8}{\pi }\right)}^{1/4}\frac{1}{\sqrt{d}}.$$ 2.16

In principle, it is possible to obtain the associated PAF from equation (2.11) by calculating the HT of α directly, but the authors are unaware of any published analytical solution to $\mathfrak{H}\{\alpha \}$ for this case. We may, however, solve for ϕ alternatively by first finding the FT of α and then performing the integration in equation (2.12). With

$$\hat{\alpha }=\frac{cd}{2\sqrt{2}}\exp \left(\frac{-d^2({\theta }^2+{\omega }^2)}{4}\right)(1+\exp (d^2\theta \omega )),$$ 2.17

we find

$$\varphi =\frac{1}{\sqrt{2}}\{\alpha +\mathrm{i}c\exp \left(\frac{-t^2}{d^2}\right)\mathrm{Im}\left[{\mathrm{e}}^{\mathrm{i}\theta t}\mathrm{erf}(\frac{\theta d}{2}+\frac{\mathrm{i}t}{d})\right]\},$$ 2.18

which implies

$$\mathfrak{H}\{\alpha \}=c\exp \left(\frac{-t^2}{d^2}\right)\mathrm{Im}\left[{\mathrm{e}}^{\mathrm{i}\theta t}\mathrm{erf}(\frac{\theta d}{2}+\frac{\mathrm{i}t}{d})\right].$$ 2.19

Owing to the properties of the error function, we have

$$\underset{\theta d\to \mathrm{\infty }}{lim}\mathfrak{H}\{\alpha \}=c\exp \left(\frac{-t^2}{d^2}\right)\sin (\theta t)$$ 2.20

for α of the form equation (2.15).

Figure 1 is a plot of the real and imaginary parts of ϕ for the case in which d=1 and θ=10. Note that the imaginary part is approximately proportional to the right-hand side of equation (2.20) in this case, as θd=10. Consequently, the corresponding PDF, |ϕ|², very nearly recovers the asymptotic Gaussian envelope.

Figure 1.

Real and imaginary parts of ϕ, in solid and dashed lines, respectively, for α of the form equation (2.15), with d=1 and θ=10.

3. Discrete analogues

We may readily define the discrete analogue of S{ψ} in association with the discrete analogue of ψ. Specifically let the discrete amplitude,

$$\mathbf{\text{v}}=\left[\begin{array}{c}\mathbf{\text{v}}[1]\\ ⋮\\ \mathbf{\text{v}}[n]\end{array}\right]=\left[\begin{array}{c}{v}_1\\ ⋮\\ v_n\end{array}\right]$$ 3.1

be an n-dimensional complex vector such that

$$\mathrm{\Delta }t|\mathbf{\text{v}}{|}_2^2=\mathrm{\Delta }t\sum _{j=1}^n{v}_j^{\ast }{v}_j=1,$$ 3.2

where |z|₂ signifies the usual ℓ²-norm of any complex vector z and Δt=Δt{v} is some positive real number representing implicitly the uniform interval between nodes on the t-axis at which v is evaluated. Because S{ψ} is insensitive to the unit of t we are free to put Δt=1 and normalize v accordingly. For clarity, however, it is useful to preserve Δt explicitly. Analogously to ρ we then define the discrete distribution

$$\mathbf{\text{p}}=\left[\begin{array}{c}{p}_1\\ ⋮\\ p_n\end{array}\right]\equiv \left[\begin{array}{c}{v}_1^{\ast }{v}_1\\ ⋮\\ v_n^{\ast }{v}_n\end{array}\right].$$ 3.3

The discrete spectral amplitude associated with v is

$${\mathcal{F}}_{\mathrm{D}}\{\mathbf{\text{v}}\}=\left[\begin{array}{c}{\mathcal{F}}_{\mathrm{D}}\{\mathbf{\text{v}}\}[1]\\ ⋮\\ {\mathcal{F}}_{\mathrm{D}}\{\mathbf{\text{v}}\}[n]\end{array}\right],$$ 3.4

where ${\mathcal{F}}_{\mathrm{D}}\{\mathbf{\text{z}}\}$ is the discrete Fourier transform (DFT) of a given z and is defined here generally such that

$${\mathcal{F}}_{\mathrm{D}}\{\mathbf{\text{v}}\}[k]=\frac{\mathrm{\Delta }t}{\sqrt{2\pi }}\sum _{j=1}^n{v}_j\exp \left(\frac{-2\pi \mathrm{i}(k-1)(j-1)}{n}\right)$$ 3.5

for $k=1,\dots ,n$. Because the DFT does not generally preserve the norm of a vector it is necessary to introduce the normalization factor

$$\beta \equiv \frac{1}{\sqrt{\mathrm{\Delta }\omega }|{\mathcal{F}}_{\mathrm{D}}\{\mathbf{\text{v}}\}{|}_2}$$ 3.6

such that

$$\mathrm{\Delta }\omega |\beta {\mathcal{F}}_{\mathrm{D}}\{\mathbf{\text{v}}\}{|}_2^2=1,$$ 3.7

where

$$\mathrm{\Delta }\omega =\frac{2\pi }{n\mathrm{\Delta }t}.$$ 3.8

The discrete analogue of σ is accordingly

$$\mathbf{\text{q}}=\left[\begin{array}{c}{q}_1\\ ⋮\\ q_n\end{array}\right]\equiv {\beta }^2\left[\begin{array}{c}{\mathcal{F}}_{\mathrm{D}}^{\ast }\{\mathbf{\text{v}}\}[1]{\mathcal{F}}_{\mathrm{D}}\{\mathbf{\text{v}}\}[1]\\ ⋮\\ {\mathcal{F}}_{\mathrm{D}}^{\ast }\{\mathbf{\text{v}}\}[n],{\mathcal{F}}_{\mathrm{D}}\{\mathbf{\text{v}}\}[n]\end{array}\right].$$ 3.9

Finally, analogously to S{ψ}, the total entropy associated with a given v is

$$S_{\mathrm{D}}\{\mathbf{\text{v}}\}\equiv -\mathrm{\Delta }t\sum _{j=1}^n{p}_j\log (p_j)-\mathrm{\Delta }\omega \sum _{j=1}^n{q}_j\log (q_j),$$ 3.10

where n=n{v}.

The characteristic amplitude in equation (2.11) and its Fourier transform may be defined analogously for discrete, or sampled, signals [13,14]. First, in analogy to α, let

$$\mathbf{\text{a}}=\left[\begin{array}{c}{a}_1\\ ⋮\\ a_n\end{array}\right]$$ 3.11

be an n-dimensional real vector such that

$$\mathrm{\Delta }t|\mathbf{\text{a}}{|}_2^2=1$$ 3.12

for some positive, real Δt. For convenience, and because it causes no confusion, we assign n=n{a} and Δt=Δt{a} henceforth. Let the discrete Hilbert transform (DHT) be defined generally such that the kth element of the DHT of a, written here as ${\mathfrak{H}}_{\mathrm{D}}\{\mathbf{\text{a}}\}$, is

$${\mathfrak{H}}_{\mathrm{D}}\{\mathbf{\text{a}}\}[k]=\frac{1}{\pi }\sum _{j=1;j\ne k}^n\frac{a_j}{(k-j)}$$ 3.13

for k=1,…,n. As with the DFT, in general the DHT only approximately preserves the norm of a vector. Let us therefore define

$$\eta \equiv \frac{1}{\sqrt{\mathrm{\Delta }t}|{\mathfrak{H}}_{\mathrm{D}}\{\mathbf{\text{a}}\}{|}_2}$$ 3.14

such that

$$\mathrm{\Delta }t|\eta {\mathfrak{H}}_{\mathrm{D}}\{\mathbf{\text{a}}\}{|}_2^2=1.$$ 3.15

Finally, let us define

$$\mathbf{\text{u}}=\mathbf{\text{u}}\{\mathbf{\text{a}}\}=\left[\begin{array}{c}{u}_1\\ ⋮\\ u_n\end{array}\right]\equiv \frac{1}{\sqrt{2}}(\mathbf{\text{a}}+\mathrm{i}\eta {\mathfrak{H}}_{\mathrm{D}}\{\mathbf{\text{a}}\})$$ 3.16

as the discrete analogue of ϕ.

It is worth noting that the discrete terms introduced above are essentially nothing more than the most elementary numerical approximations of their respective continuous analogues. They are meant to embody only those principles already embodied in the continuous terms.

It is also important to mention that calculation of the AR requires some care in the case of discrete or sampled signals. Specifically consider some g with some non-trivial compact support [t₁,t₂]. In general, $\mathfrak{H}\{g\}$ may be non-vanishing outside of [t₁,t₂]. In order to compute properly the AR for a discrete waveform, it is therefore necessary to ‘pad’ the original waveform with a sufficient number of zeros. In all calculations presented in the following section, the number of added zeros is at least an order of magnitude greater than the size of the initial waveform, thereby ensuring precision.

4. Waveform information

Because the total entropy is an effective measure of information in PAFs, and because ϕ is a complex, square-integrable function with unit energy that contains essentially the same information as does α, we propose that S{ϕ} is an appropriate, generally meaningful measure of information in real, non-probabilistic waveforms like α. This section validates the informatic utility of S{ϕ} with detailed computations of the discrete analogue, S_D{u} in fundamental informatic scenarios.

Additionally, we present corresponding computations of the presumably naively constructed quantity, S_D{a}, which is the discrete analogue of S{α}. Given that α and ϕ contain essentially the same information, if S{ϕ} is a meaningful measure of information then so too should be S{α}. Note, however, that we should not expect S{ϕ} and S{α} to be identical. In particular, the relationship between H{|α|²} and H{|ϕ|²} is not analytically reducible. In contrast, it follows directly from the explicit form of $H\{|\hat{\alpha }{|}^2\}$ that:

$$H\{|\hat{\alpha }{|}^2\}=H\{|\hat{\varphi }{|}^2\}+\log (2).$$ 4.1

We therefore expect S{ϕ} and S{α} to differ generally. Despite the expected differences, we find that the discrete analogues of S{ϕ} and S{α} are strongly correlated and that both are effective measures of information. In fact, the observed consistencies between S_D{u} and S_D{a} constitute a separate validation of the informatic relevance of the total entropy to non-probabilistic waveforms like α.

In each of the following calculations, the discrete vectors are padded such that the total number of points is 2²⁰, and the number of computationally introduced zeros is thus 2²⁰−n in each case. The varying number of computational zeros introduces only a negligible influence.

As a first test, let us examine the behaviours of S_D{u} and S_D{a} for a waveform consisting of randomly generated noise. Specifically, suppose that a has the form

$$a_k=C\nu (0,1;k),$$ 4.2

where ν(μ,δ;k) represents the kth sample of normally distributed noise with mean μ and variance δ, and C is a constant for a given randomly generated waveform, chosen such that $\mathrm{\Delta }t|\mathbf{\text{a}}{|}_2^2=1$. For convenience, we put Δt=1 here. Figure 2 shows the normalized entropies, $S_{\mathrm{D}}\{\mathbf{\text{u}}\}/\log (n)$ and $S_{\mathrm{D}}\{\mathbf{\text{a}}\}/\log (n)$, as functions of ${\log }_{10}(n)$. Each data point represents the average among 10 independently randomized a. Note that both measures behave as if asymptotically approaching unity.

Figure 2.

Average normalized entropies, S_D{u} and S_D{a}, of n samples of normally distributed noise plotted as a function of ${\log }_{10}(n)$ in asterisks and circles, respectively.

Let us next consider the complementary case of a correlated signal, such that a consists essentially of some regularly repeating sequence. As more and more sample points are added to such a waveform, the amount of added information decreases and the information contained in a asymptotically approaches some fixed quantity. As an example, let a have the form

$$a_k=B\sin ({\omega }_0{t}_k),$$ 4.3

where B is chosen for appropriate normalization. Figure 3 shows the entropies, S_D{u} and S_D{a}, for a of the form equation (4.3) for ω₀=0.3 and t_k=1,2,…,n. Note that both measures exhibit the expected asymptotic behaviour, but with each converging to a different value.

Figure 3.

Entropies, S_D{u} and S_D{a}, of n samples of a normalized sine wave of the form equation (4.3) with ω₀=0.3 plotted as a function of ${\log }_{10}(n)$ in asterisks and circles, respectively.

It is instructive in this case to show how the two Fourier-conjugated components constituting the total entropy behave individually. Figure 4 contains plots of the Fourier components of S_D{u} for the same scenario shown in figure 3. Note how the t-domain and ω-domain components increase and decrease respectively, while their sum is asymptotically fixed, representing presumably the fixed information in the waveform. Analogous behaviour is observed in the components of S_D{a}.

Figure 4.

Individual Fourier components of S_D{u} along with S_D{u} for scenario shown in figure 3. The t-domain contribution is plotted in diamonds and the ω-domain contribution is plotted in ‘+’. Their sum, the total entropy, is shown in asterisks, as in figure 3.

Next let us examine the behaviours of S_D{u} and S_D{a} in the case of a simple, ‘bare’ signal with a varying degree of additive background noise. In general, we expect the entropy of the net waveform to increase as the energy of the noise increases in relation to the energy of the bare signal—provided that the bare signal is sufficiently simple. Let us therefore introduce a discrete waveform of the form

$$a_k=c^{\prime }\exp \left(\frac{-t_k^2}{d^2}\right)\cos (\theta t_k)+C^{\prime }\nu (0,1;k).$$ 4.4

The signal-to-noise ratio (SNR), with ‘signal’ referring here to the bare signal, is

$$\mathrm{SNR}=\frac{\parallel c^{\prime }\exp (-t_k^2/d^2)\cos (\theta t_k)\parallel }{\parallel C^{\prime }\nu (0,1;k)\parallel }.$$ 4.5

The constants c′ and C′ are to be chosen to effect a given SNR while ensuring that $\mathrm{\Delta }t|\mathbf{\text{a}}{|}_2^2=1$.

Figure 5 contains plots of S_D{u} and S_D{u} as functions of the SNR for a of the form equation (4.4) with d=1 and θ=10. In each calculation, the signal was constructed with n=9001 samples taken at equally spaced points spanning [−8,8]. The value of Δt follows. Each point on the plot represents the average of 10 separate calculations, each of which was performed with a separately generated background noise function. The standard deviation in each plotted point is smaller than the interval implied by the size of the symbol with which the point is plotted. The dashed line in figure 5 marks the average entropy, S_D{u}, of 9001 samples of Gaussian noise, which is approximately 9.4, and the dot-dashed line represents the analogous background noise for S_D{a}. The entropy of the noise is the limiting entropy as the SNR vanishes (or becomes infinitely negative as measured in dB). Conversely in the limit as the SNR diverges S_D{u} is very nearly $1+\log (\pi )$ in this case due to the nearly Gaussian envelope of the signal and is independent of the number of samples insofar as n is sufficiently large. Both measures evolve with respect to the SNR in broad accordance with expectations of measures of information. Also noteworthy is the sensitivity of the entropy measures to the presence of signals even below −20 dB.

Figure 5.

Average entropies, S_D{u} and S_D{a}, for a of the form equation (4.4), with d=1 and θ=10, plotted as a function of the SNR in asterisks and circles, respectively. The dashed and dot-dashed lines show the average entropies of the noise alone for each case.

As a fourth test of the waveform entropy, let us consider a sequence of bits represented as a sequence of binary-valued square pulses. The informatic complexity of α in such a case should depend on the arrangement of the bits. For example, the sequence ‘101010…’ conveys less information than a sequence of bits arranged in an irregular manner. The informatic entropy of the former should presumably be smaller than that of the latter. Figure 6 shows plots of S_D{u} and S_D{a} in a sequence of 1024 bits originally ordered as ‘101010…’ but with m consecutive bits somewhere in the sequence being reordered randomly. The corresponding signal, a, was constructed by assigning 8 consecutive sample points $a_k=\frac{1}{64}$ to each ‘1’ in the sequence and 8 consecutive sample points a_k=0 to each ‘0’. We accordingly have n=8192, hence the value $\frac{1}{64}$ assigned to the ‘1's ensures unit energy. Each plotted point represents the average of 10 separate calculations, with the location of the scrambled sub-sequence chosen randomly each time. Note the expected broad correlation between entropy and the degree of randomization, characterized by m. It is also interesting to note that S_D{u}<S_D{a} in this case, whereas the opposite was observed in the three preceding studies. This does not have any significance per se, but suggests that the relationship between S_D{u} and S_D{a} is complex and deserving of further investigation.

Figure 6.

Average entropy of a sequence of 1024 bits, encoded in square waves, originally ordered as ‘101010…’, but with m consecutive bits somewhere in the sequence being randomly reordered.

5. Summary

The foregoing analysis addresses a fundamental issue in information theory, namely the characterization of the informatic content of real waveforms. We have demonstrated that the total entropy associated with quantum-mechanical PAF's is a meaningful measure of information in waveforms like α. Specifically, the computations described in §4 validate that S_D{u} and S_D{a} exhibit the behaviours expected of a sensitive measure of information over a broad range of rudimentary, non-trivial scenarios.

Our approach differs notably from previous studies in that we have not sought to construct a secondary distribution or other probabilistic artifice to describe a given waveform, but rather we have simply attributed a measure of entropy to the waveform itself—either directly or through the AR. The broad correlation between S_D{u} and S_D{a} is noteworthy in that it is not guaranteed a priori, and α is neither complex nor probabilistic. More to the point, the observed consistencies between S_D{u} and S_D{a} affirm that both measures, and their continuous analogues, are meaningful measures of information. Specifically, if S{ϕ} measures information then so should S{α} because ϕ and α contain essentially the same information.

It is worth noting that the computations plotted in figure 5 indicate that the entropy prescriptions described in this work may be useful in the detection of weak signals in noisy backgrounds. Moreover, the sensitivity to the degree of randomness in the arrangement of a sequence of square pulses indicates a connection to Kolmogorov complexity that begs further investigation. Such a sensitivity may be useful in cryptological applications, for instance. Additional applications will be the subject of future work.

Data accessibility

This work did not rely on any experimental data.

Authors' contributions

S.F. conceived the basic model. W.S. and A.W. contributed to analysis and testing. A.W. contributed to mathematical analysis of the Hilbert transform.

Competing interests

We have no competing interests.

Funding

This work was funded through the Naval Innovative Science and Engineering (NISE) program as implemented at SPAWAR Systems Center Atlantic.