Bandwidth Detection of Graph Signals with a Small Sample Size

Abstract

Bandwidth is the crucial knowledge to sampling, reconstruction or estimation of the graph signal (GS). However, it is typically unknown in practice. In this paper, we focus on detecting the bandwidth of bandlimited GS with a small sample size, where the number of spectral components of GS to be tested may greatly exceed the sample size. To control the significance of the result, the detection procedure is implemented by multi-stage testing. In each stage, a Bayesian score test, which introduces a prior to the spectral components, is adopted to face the high dimensional challenge. By setting different priors in each stage, we make the test more powerful against alternatives that have similar bandwidth to the null hypothesis. We prove that the Bayesian score test is locally most powerful in expectation against the alternatives following the given prior. Finally, numerical analysis shows that our method has a good performance in bandwidth detection and is robust to the noise.

1. Introduction

Graph signal (GS) is a versatile model for describing information in irregular domains, which has been widely used in sensor networks [1,2], biological networks [3,4] and image and 3-D point cloud processing [5,6]. Graph signal processing (GSP) theory generalizes the classic discrete signal processing theory to irregular domains by introducing graph Fourier transform (GFT) [7,8]. In most real world networks, adjacent vertices usually have similar signals, which leads to bandlimitedness or approximate bandlimitedness in graph spectral domain. For example, the differences of temperature measured by sensors within short distances may not vary a lot; people tend to have similar interests or views with their friends in the social network.

Bandwidth is a widely used prior of GS in selecting the best sensor placements to monitor spatial phenomena [9,10,11], low-pass filter designed for denoising [7,8,12] and estimating the signals on all the sensors from partial observations [13,14]. However, the bandwidth of GS is typically unknown in practice. There are only a few works considering the problem of bandwidth detection. The bandwidth estimation in Reference [9] relies on noise-free samples. The scenario of noisy observations is considered in Reference [15], but it only applies to the spectral sparse case. Meanwhile, the estimation accuracy of the bandwidth in Reference [15] highly depends on the parameter controlling the sparsity of the GS, and the process of finding an optimal parameter is time-consuming.

The detection of bandwidth for a GS aims to detect the number of spectral components in a GS, which can be seen as a model selection problem in linear regression. The well-known model selection procedures like Akaike information criterion (AIC) and Bayesian information criterion (BIC) [16] pick the most likely model under some criteria. But they do not consider the significance of the chosen model. In contrast, hypothesis testing, such as likelihood ratio (LR) test and F-test can provide the significance of a model. If the bandwidth of a GS is k, the largest index of non-zero elements in frequency coefficients should be k. Let the vertex number of a GS be N, the regression coefficients of the last $N-k$ frequency components are denoted by ${\widehat{\mathit{f}}}_{-k}$. To test whether the bandwidth of the GS is k, the hypothesis can be expressed as

$$\begin{array}{c}\hfill H_0:{\widehat{\mathit{f}}}_{-k}=\mathbf{0},H_A:{\widehat{\mathit{f}}}_{-k}\ne \mathbf{0}.\end{array}$$ (1)

The null hypothesis will be rejected if the model with the assumed bandwidth k is not significant enough. Therefore, the bandwidth of GS can be detected by applying tests over all the assumed bandwidths to see whether there is a model achieving the given significance.

For large-scale GS, such as social networks, it is impossible to sample the signal on every vertex due to huge data collection cost, which means we can only have the sample size $M<N-k$ sometimes. In this high dimensional situation, the conventional test for testing regression coefficients such as LR test and F-test is infeasible, which will be explained in Section 3.1. There are plenty of literature on testing the coefficients of high dimensional linear regression. For example, Reference [17,18] propose new test statistics by modifying the F-statistic, and Reference [19] introduces Bayesian priors to the parameters being tested. When $k\ll N$, the spectrum of the GS can be seen as sparse. Several approaches reconstruct the signal in compressive sensing which estimate the bandwidth as a by-product [20,21]. The frequency components detected by compressive sensing methods can help estimate the spectrum sparse signal well. However, the bandwidth obtained by the compressive sensing methods may be inaccurate since the choose frequency components in the whole space without considering the bandlimitedness. More accurate bandwidth is need in some applications like filter design and sampling set design of which the performance highly depend on the bandwidth prior.

In this letter, we try to detect the bandwidth of bandlimited GS with small sample size, which is not much larger than the bandwidth and there is no sparsity constraint to the spectrum. The bandwidth is detected by a forward multi-stage test. The bandwidth of model being tested is increasing over stages and the bandwidth is obtained when the null hypothesis is accepted. Since the samples are not adequate, we try to use the limited samples to better distinguish the assumed bandwidth from those close to it in each stage for a small detection bias. Therefore, we customize priors for ${\widehat{\mathit{f}}}_{-k}$ in each stage to describe our attention on testing whether some elements in ${\widehat{\mathit{f}}}_{-k}$ are non-zero. By doing so, we only use the limited samples to focus on testing a few alternatives in each stage. Bayesian score test [19] is adopted in this paper, but we do not give uniform prior to the parameters being tested as Reference [19] does. Since the bandwidth being tested is increasing over stages, our attentions on coefficients of different frequencies are also updated in each stage which makes our method able to locate the true bandwidth with a small bias even though the samples are insufficient.

The widely used multi-stage test for model selection, stepwise regression ([16] Chapter 10) does poorly with a small sample size since the estimation of the regression coefficients in it is biased [22]. In contrast, our method is proved to be locally most powerful (LMP) in expectation against alternatives following the given prior, which means it performs well in distinguishing among models with neighbouring bandwidth even using a small amount of samples in each stage.

2. Problem Formulation

Consider an N-vertex undirected connected graph $\mathcal{G}=(\mathcal{V},\mathcal{E},\mathit{W})$, where $\mathcal{V}$ is the vertex set, $\mathcal{E}$ is the edge set and $\mathit{W}$ is the $N\times N$ weighted adjacency matrix with $\mathit{W}(i,j)=w_{ij}$. A GS defined on $\mathcal{V}$ can be represented as a vector $\mathit{f}\in {\mathbb{R}}^N$, and its element $f_i$ represents the signal value at the i-th vertex in $\mathcal{V}$. Laplacian matrix of the undirected graph is given by $\mathit{L}=\mathit{D}-\mathit{W}$, where $\mathit{D}$ is the diagonal degree matrix $\mathrm{diag}\{d_1,\cdots ,d_N\}$ with $d_i={\sum }_j{w}_{ij}$. Since $w_{ij}=w_{ji}$ for undirected graphs, Laplacian matrix is symmetric. Therefore, it has real non-negative eigenvalues $0={\lambda }_1\le {\lambda }_2\le \cdots \le {\lambda }_N$ and an orthogonal set of eigenvectors $\mathit{V}$. As a result, the spectral decomposition of graph Laplacian is given as $\mathit{L}=\mathit{V}\mathbf{\Lambda }{\mathit{V}}^T$, where $\mathbf{\Lambda }$ is a diagonal matrix of eigenvalues. The columns of $\mathit{V}$ denoted by ${\left\{{\mathit{v}}_i\right\}}_{1\le i\le N}$ are regarded as the graph Fourier bases and the eigenvalues are regarded as frequencies [7]. The expansion coefficients of $\mathit{f}$ corresponding to eigenvectors are defined as $\widehat{\mathit{f}}$.

A GS is called bandlimited when there exists a $K\in \{0,1,\cdots ,N-1\}$ such that its GFT $\widehat{\mathit{f}}$ satisfies ${\widehat{f}}_i=0$ for all $i\ge K$ [10]. The smallest such K is called the bandwidth of $\mathit{f}$. If all the frequency coefficients are non-zero, the GS is not bandlimited. If $\mathit{f}$ is a signal with bandwidth K, then it satisfies $\mathit{f}={\mathit{V}}_K{\widehat{\mathit{f}}}_K$, where ${\mathit{V}}_K$ denotes the first K columns of $\mathit{V}$ and ${\widehat{\mathit{f}}}_K$ denotes the first K coefficients of $\widehat{\mathit{f}}$.

Suppose that $M(M<N)$ noisy observations $\mathit{y}\in {\mathbb{R}}^M$ are sampled from $\mathit{f}\in {\mathbb{R}}^N$ to detect the bandwidth, the observation model can be summarized as

$$\mathit{y}=\mathbf{\Psi }(\mathit{f}+\mathit{w}),$$ (2)

where $\mathit{w}$ is an $N\times 1$ vector with ith element representing the observation noise on the ith vertex and $\mathbf{\Psi }:{\mathbb{R}}^N\to {\mathbb{R}}^M$ is the sampling matrix, of which the element in the ith row and jth column is defined as

$$\begin{array}{c}\hfill {\left(\mathbf{\Psi }\right)}_{i,j}=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\mathrm{the}j\mathrm{th}\mathrm{vertex}\mathrm{is}\mathrm{the}i\mathrm{th}\mathrm{sample};\\ 0,\hfill & \mathrm{otherwise}.\end{array}\right.\end{array}$$ (3)

We use a simple example to illustrate the role of the sampling matrix. For a graph with 5 vertices and signal $\mathit{f}={[f_1,\cdots ,f_5]}^{\mathrm{T}}$, if the 1st, 2nd and 4th vertices are sampled, then the sampling matrix is a matrix with 3 rows and 5 columns as follow

$$\begin{array}{c}\hfill \mathbf{\Psi }=\left[\begin{array}{ccccc}1& 0& 0& 0& 0\\ 0& 1& 0& 0& 0\\ 0& 0& 0& 1& 0\end{array}\right].\end{array}$$ (4)

The sampled graph signal is $\mathbf{\Psi }\mathit{f}={[f_1,f_2,f_4]}^{\mathrm{T}}$.

In this paper, we assume that the observation noise on each sample follows the Gaussian distribution $\mathcal{N}(0,{\sigma }^2)$ and the noise on all the samples are independently and identically distributed (i.i.d.), so the observation follows $\mathit{w}\sim \mathcal{N}(\mathbf{0},{\sigma }^2{\mathit{I}}_{N\times N})$, where ${\mathit{I}}_{N\times N}$ is an $N\times N$ identity matrix.

By dividing the columns of $\mathit{V}$ into two parts ${\mathit{V}}_k$ and ${\mathit{V}}_{-k}$ for any assumed bandwidth $k\in \{1,2,\cdots ,N-1\}$, the observation model (2) can be rewritten as

$$\begin{array}{c}\hfill \mathit{y}={\mathit{U}}_k{\widehat{\mathit{f}}}_k+{\mathit{U}}_{-k}{\widehat{\mathit{f}}}_{-k}+\mathbf{\Psi }\mathit{w},\end{array}$$ (5)

where ${\widehat{\mathit{f}}}_k$ and ${\widehat{\mathit{f}}}_{-k}$ denote the first k and the last $N-k$ elements of $\widehat{\mathit{f}}$, respectively, and ${\mathit{U}}_k=\mathbf{\Psi }{\mathit{V}}_k$, ${\mathit{U}}_{-k}=\mathbf{\Psi }{\mathit{V}}_{-k}$. If the actual bandwidth $K=k$, $H_0$ in (1) will be accepted. Otherwise, $H_0$ will be rejected.

For a GS with actual bandwidth K, the hypothesis with assumed bandwidth $k<K$ should all be rejected. Therefore, we detect the bandwidth of GS in a forward multi-stage way with assumed bandwidth increasing over stages. The bandwidth is obtained when the null hypothesis is accepted. By doing so, we reduce the number of models to be tested from N to K.

3. Bandwidth Detection

3.1. High Dimensional Challenge

F-test and LR test are common approaches to test the hypothesis (1) in linear regression model. Let the maximum likelihood estimation (MLE) of $\widehat{\mathit{f}}$ under $H_0$ and $H_1$ be ${\widehat{\mathit{f}}}_{\mathrm{R}}$ and ${\widehat{\mathit{f}}}_{\mathrm{U}}$, respectively. The corresponding sum of squared residuals (SSR) are denoted by $\mathrm{SSR}\left({\widehat{\mathit{f}}}_{\mathrm{R}}\right)$ and $\mathrm{SSR}\left({\widehat{\mathit{f}}}_{\mathrm{U}}\right)$. Then, LR test statistic ([23] Chapter 10) equals $Mlog(\mathrm{SSR}\left({\widehat{\mathit{f}}}_{\mathrm{U}}\right)/\mathrm{SSR}\left({\widehat{\mathit{f}}}_{\mathrm{R}}\right))$, which is asymptotically distributed as ${\chi }^2(N-k)$ as the sample size approaches to infinity. When the sample size is small, the test statistic of LR test has no specific distribution which makes it hard to implement. F-test statistic ([23] Chapter 10) equals

$$\begin{array}{cc}\hfill \frac{\mathrm{SSR}\left({\widehat{\mathit{f}}}_{\mathrm{U}}\right)-\mathrm{SSR}\left({\widehat{\mathit{f}}}_{\mathrm{R}}\right)/N-k}{\mathrm{SSR}\left({\widehat{\mathit{f}}}_{\mathrm{R}}\right)/M-k}& =\frac{{\mathit{y}}^T{\mathit{M}}_k{\mathit{U}}_{-k}{\left({\mathit{U}}_{-k}^T{\mathit{M}}_k{\mathit{U}}_{-k}\right)}^{-1}{\mathit{U}}_{-k}^T{\mathit{M}}_k\mathit{y}/N-k}{{\mathit{y}}^T{\mathit{M}}_k\left({\mathit{I}}_{M\times M}-{\mathit{U}}_{-k}{\left({\mathit{U}}_{-k}^T{\mathit{M}}_k{\mathit{U}}_{-k}\right)}^{-1}{\mathit{U}}_{-k}^T\right){\mathit{M}}_k\mathit{y}/M-k},\hfill \end{array}$$ (6)

where ${\mathit{M}}_k={\mathit{I}}_{M\times M}-{\mathit{U}}_k{\left({\mathit{U}}_k^T{\mathit{U}}_k\right)}^{-1}{\mathit{U}}_k^T$ is a projection matrix that projects signals to the complementary subspace of the column space of ${\mathit{U}}_k$. F-test is exact in finite samples, however, to ensure that ${\mathit{U}}_k^T{\mathit{U}}_k$ in ${\mathit{M}}_k$ is invertible, we have to let $M\ge k$. Similarly, to ensure that ${\mathit{U}}_{-k}^T{\mathit{M}}_k{\mathit{U}}_{-k}$ is invertible, there must be $M\ge N-k$. Therefore, (6) requires that $M\ge max(K,N-k)$, which means the sample size needs to be bigger than the assumed bandwidth, as well as the dimension of ${\widehat{\mathit{f}}}_{-k}$. For large-scale GS, the sampling cost for bandwidth detection using F-test or LR test is too high. Therefore, a test with small sample size is preferred. The sample size only needs to be larger than the assumed bandwidth, that is, $M>k$, in score test ([23] Chapter 10). However, when $N-k>M$, there may exist some alternatives which have ${\widehat{\mathit{f}}}_{-k}\ne \mathbf{0}$ but ${\mathit{U}}_{-k}{\widehat{\mathit{f}}}_{-k}=\mathbf{0}$. We can never hope to have any power against these alternatives. Since the GS is known to be bandlimited, we should pay more attention on testing whether the low frequency coefficients in ${\widehat{\mathit{f}}}_{-k}$ are non-zero than the high frequency coefficients. Thus, we adopt Bayesian score test [19] to allocate our attention among the alternatives by designing a prior for ${\widehat{\mathit{f}}}_{-k}$. Different from Reference [19], which pays equal attention to all the alternatives, we customize a prior for ${\widehat{\mathit{f}}}_{-k}$ to fit our bandwidth detection purpose.

3.2. Design of the Prior

In the test for each bandwidth, the alternative that ${\widehat{\mathit{f}}}_{-k}=\mathbf{\alpha }$ and ${\widehat{\mathit{f}}}_{-k}=-\mathbf{\alpha }$ for every $\mathbf{\alpha }\ne \mathbf{0}$ contributes equally in rejecting $H_0$, so we assign $\mathit{E}\left({\widehat{\mathit{f}}}_{-k}\right)=\mathbf{0}$ to make the prior unbiased, where $\mathit{E}(·)$ denotes the expectation of the given stochastic variable. The covariance matrix of ${\widehat{\mathit{f}}}_{-k}$ can be assigned as

$$\begin{array}{c}\hfill \mathit{E}\left({\widehat{\mathit{f}}}_{-k}{\widehat{\mathit{f}}}_{-k}^T\right)={\tau }^2{\mathbf{\Sigma }}_{-k},\end{array}$$ (7)

where ${\mathbf{\Sigma }}_{-k}$ is a positive semidefinite $k\times k$ matrix to be designed and ${\tau }^2$ is an unknown parameter. In this paper, we let ${\mathbf{\Sigma }}_{-k}$ be a diagonal matrix, then each of its diagonal element is the variance of the corresponding element in ${\widehat{\mathit{f}}}_{-k}$. Since $\mathit{E}\left({\widehat{\mathit{f}}}_{-k}\right)=\mathbf{0}$, a larger variance indicates the corresponding element in ${\widehat{\mathit{f}}}_{-k}$ is more likely to have a larger amplitude. To decide whether $H_0$ should be accepted, more attention should be payed to the element in ${\widehat{\mathit{f}}}_{-k}$ with larger variance. Thus, the attention among elements in ${\widehat{\mathit{f}}}_{-k}$ in bandwidth detection is linked to their variances in the prior. The attention is allocated by designing a ${\mathbf{\Sigma }}_{-k}$.

In our forward multi-stage test, the bandwidth being tested updates over stages, thus ${\mathbf{\Sigma }}_{-k}$ also needs to be updated. To avoid the the multi-stage test accepting the null hypothesis too early, the test should be more distinguishable from the model with bandwidth close to the null hypothesis in each stage. Therefore, we use the limited samples to make the test concentrate on distinguishing bandwidth close to k. The design of ${\mathbf{\Sigma }}_{-k}$ follows the guideline that as the frequency increases, element in ${\widehat{\mathit{f}}}_{-k}$ with higher frequency is paid less attention. Since we cannot determine a bandwidth larger than $M-1$ with sample size M, we set the attention on ${\widehat{\mathit{f}}}_{-k}$ with frequency larger than M to a small constant $\delta$. For example, the ith diagonal element of ${\mathbf{\Sigma }}_{-k}$ can be designed to be

$$\begin{array}{c}\hfill {\left({\mathbf{\Sigma }}_{-k}\right)}_{i,i}=\frac{1}{Z}exp\left(-\frac{i^2}{2{\sigma }^2}\right)+\delta ,\end{array}$$ (8)

where $Z=exp\left(-1/\left(2{\sigma }^2\right)\right)$ is the normalization factor. According to the three-standard-deviations property of normal distribution, $\sigma$ is set to be $(M-k)/3$ to make the attention on coefficients with frequencies larger than M equals to $\delta$ with high probability. Equation (8) is not the only choice of ${\mathbf{\Sigma }}_{-k}$, but this form is applicable. The details about how it affects the power of the test is analyzed later in Section 3.3.

As the multi-stage test moves forward, the attentions among frequency coefficients update, and in each stage, the test is distinguishable among bandwidth close to k, which makes the small-bias bandwidth detection with small sample size possible. An illustration of the attention update is shown in Figure 1.

Figure 1.

Illustration of the attentions on coefficients of different frequencies in different stages. Each line illustrates the attention distribution of one stage. Attention equals to 0 indicates that the corresponding frequency coefficient does not need to be tested.

We can complete the specification of the distribution of ${\widehat{\mathit{f}}}_{-k}$ by choosing a value for ${\tau }^2$ and a distribution shape. Let $L_k({\widehat{\mathit{f}}}_{-k};\mathit{y})$ be the likelihood of ${\widehat{\mathit{f}}}_{-k}$, for a specific prior distribution of ${\widehat{\mathit{f}}}_{-k}$, the likelihood of ${\tau }^2$ is

$$\begin{array}{c}\hfill {\overline{L}}_k({\tau }^2;\mathit{y})=\int L_k({\widehat{\mathit{f}}}_{-k};\mathit{y})\pi \left({\widehat{\mathit{f}}}_{-k}\right|{\tau }^2)\mathrm{d}{\widehat{\mathit{f}}}_{-k},\end{array}$$ (9)

where $\pi \left({\widehat{\mathit{f}}}_{-k}\right|{\tau }^2)$ is the distribution of ${\widehat{\mathit{f}}}_{-k}$ for a given ${\tau }^2$.

Given ${\overline{L}}_k({\tau }^2;\mathit{y})$, we convert hypothesis (1) to

$$\begin{array}{c}\hfill {\overline{H}}_0:{\tau }^2=0,{\overline{H}}_A:{\tau }^2\ne 0.\end{array}$$ (10)

${\tau }^2=0$ implies ${\widehat{\mathit{f}}}_{-k}=\mathbf{0}$, since they lead to the same distribution of $\mathit{y}$. Thus $H_0:{\widehat{\mathit{f}}}_{-k}=\mathbf{0}$ will be rejected if ${\overline{H}}_0$ is rejected. The score test of ${\overline{H}}_0$ is named the Bayesian score test of $H_0$ with the given prior of ${\widehat{\mathit{f}}}_{-k}$.

The score test of ${\overline{H}}_0$ in stage k is a one-sided test for one parameter, the test statistic is

$$\begin{array}{cc}\hfill S_{{\mathbf{\Sigma }}_{-k}}& =\frac{\mathrm{d}}{\mathrm{d}{\tau }^2}log{\overline{L}}_k(0;\mathit{y}),\hfill \end{array}$$ (11)

$$\begin{array}{cc}\hfill & =\frac{1}{2}{\mathbf{g}}_k^T{\mathbf{\Sigma }}_{-k}{\mathbf{g}}_k-\frac{1}{2}\mathrm{tr}\left({\mathbf{\Sigma }}_{-k}{\mathit{F}}_k\right),\hfill \end{array}$$ (12)

where $\mathrm{tr}\left({\mathbf{\Sigma }}_{-k}{\mathit{F}}_k\right)$ denotes the trace of ${\mathbf{\Sigma }}_{-k}{\mathit{F}}_k$. ${\overline{H}}_0$ will be rejected when $S_{{\mathbf{\Sigma }}_{-k}}\ge t$ for some threshold t. From (11), we can find that the test statistic of ${\tau }^2$ can be seen as the slope of log-likelihood function of ${\tau }^2$ under ${\overline{H}}_0$. The slope will equal to 0 when ${\tau }^2$ equals its MLE ${\widehat{\tau }}^2$. The more closer ${\widehat{\tau }}^2$ to 0, the more closer $S_{{\mathbf{\Sigma }}_{-k}}$ to 0. If $S_{{\mathbf{\Sigma }}_{-k}}$ is larger than a given t, which means ${\widehat{\tau }}^2$ is much larger than 0, then ${\overline{H}}_0$ will be rejected. Equation () is given by Reference [19], where ${\mathbf{g}}_k={\sigma }^{-2}{\mathit{U}}_{-k}{\mathit{M}}_k\mathit{y}$ and ${\mathit{F}}_k={\sigma }^{-2}{\mathit{U}}^T\mathit{U}$ are the gradient and Fisher information matrix of $log{L}_k(\mathbf{0};\mathit{y})$, respectively.

3.3. Method

Considering that the second part of () is not related to the observations, it is more convenient to work with the equivalent test statistic ${\sigma }^{-2}{\mathit{y}}^T{\mathit{M}}_k{\mathit{U}}_{-k}{\mathbf{\Sigma }}_{-k}{\mathit{U}}_{-k}^T{\mathit{M}}_k\mathit{y}$. Because ${\sigma }^2$ is not known, we plug in the MLE result $\widehat{{\sigma }^2}\propto {\mathit{y}}^T{\mathit{M}}_k\mathit{y}$ under the null hypothesis, the resulting test statistic is

$$\begin{array}{c}\hfill S_{{\mathbf{\Sigma }}_{-k}}=\frac{{\mathit{y}}^T{\mathit{M}}_k{\mathit{U}}_{-k}{\mathbf{\Sigma }}_{-k}{\mathit{U}}_{-k}^T{\mathit{M}}_k\mathit{y}}{{\mathit{y}}^T{\mathit{M}}_k\mathit{y}}.\end{array}$$ (13)

• Interpretation of the test statistic: ${\mathit{M}}_k\mathit{y}$ indicates the part of $\mathit{y}$ out of the range of ${\mathit{U}}_k$. The numerator of (13) can be rewritten as $Q={\sum }_{i=1}^M{\left({\mathbf{\Sigma }}_{-k}\right)}_{i,i}{Q}_i$ with $Q_i={\left({\mathit{y}}^T{\mathit{M}}_k{\mathit{U}}_{k+i}\right)}^2$. Larger $Q_i$ indicates a larger energy of ${\mathit{M}}_k\mathit{y}$ lying in the range of ${\mathit{U}}_{k+i}$, which links to a larger amplitude of the ith element in ${\widehat{\mathit{f}}}_{-k}$. $S_{{\mathbf{\Sigma }}_{-k}}$ equals Q, which is a weighted sum of $Q_i$, normalized by the energy of ${\mathit{M}}_k\mathit{y}$. When the weight ${\left({\mathbf{\Sigma }}_{-k}\right)}_{i,i}$ decreases with i, $S_{{\mathbf{\Sigma }}_{-k}}$ will be larger if $Q_i$ is larger for smaller i. Therefore, the test is more powerful against $H_A$ that has non-zero frequency coefficients in low frequency than that has non-zero frequency coefficients in high frequency. This is in accordance with the purpose of our design that the test should be more distinguishable from the model with bandwidth close to k.

Finally, we decide whether the null hypothesis is accepted by a p-value threshold. For a given significance level $\alpha$, if $p\le \alpha$, the null hypothesis is rejected. Let $S_{-k}$ be the observation value of $S_{{\mathbf{\Sigma }}_{-k}}$, then the p-value of the test is

$$\begin{array}{cc}\hfill p& =P_{H_0}(S_{{\mathbf{\Sigma }}_{-k}}>S_{-k}),\hfill \\ \hfill & =P_{H_0}({\mathit{y}}^T{\mathit{M}}_k({\mathit{U}}_{-k}{\mathbf{\Sigma }}_{-k}{\mathit{U}}_{-k}^T-S_{-k}){\mathit{M}}_k\mathit{y}>0).\hfill \end{array}$$ (14)

Since ${\mathit{M}}_k\mathit{y}={\mathit{M}}_k(\mathit{y}-{\mathit{U}}_k{\widehat{\mathit{f}}}_k)={\mathit{M}}_k\mathit{w}$ under null hypothesis, it follows a multivariate normal distribution. Thus, p is the probability that the quadratic form of normal variables is non-negative. The distribution of the quadratic forms in normal variables is approximated by a ${\chi }^2$ distribution in Reference [24] and p-value can be calculated approximately as,

$$\begin{array}{c}\hfill p=\left\{\begin{array}{cc}P({\chi }_{d_k}^2>d_k-h_k),\hfill & \hfill \mathrm{tr}\left({\mathit{X}}_k^3\right)>0;\\ P({\chi }_{d_k}^2<d_k-h_k),\hfill & \hfill \mathrm{tr}\left({\mathit{X}}_k^3\right)<0;\\ \Phi \left(\frac{\mathrm{tr}\left({\mathit{X}}_k\right)}{\sqrt{2\mathrm{tr}\left({\mathit{X}}_k^2\right)}}\right),\hfill & \hfill \mathrm{tr}\left({\mathit{X}}_k^3\right)=0,\end{array}\right.\end{array}$$ (15)

where ${\mathit{X}}_k={\mathit{M}}_k{\mathit{U}}_{-k}{\mathbf{\Sigma }}_{-k}{\mathit{U}}_{-k}^T{\mathit{M}}_k-S_{-k}{\mathit{M}}_k$, $d_k={\left[\mathrm{tr}\left({\mathit{X}}_k^2\right)\right]}^3/{\left[\mathrm{tr}\left({\mathit{X}}_k^3\right)\right]}^2$, and $h_k=\mathrm{tr}\left({\mathit{X}}_k^2\right)\mathrm{tr}\left({\mathit{X}}_k\right)/$$\mathrm{tr}\left({\mathit{X}}_k^3\right)$.

Our algorithm for the multi-stage bandwidth detection is shown in Algorithm 1. If $p\le \alpha$ for all the stages, the output of Algorithm 1 will be ‘None’, which means the bandwidth cannot be determined with the given sample size. Then we can say that the bandwidth is larger than $M-1$.

Algorithm 1: Multi-stage Bandwidth Detection.

Input Samples $\mathit{y}$ and significance level $\alpha$;

Output Detected bandwidth K or None;

1: for $k=1,2,\cdots ,M-1$ do

2:  Calculate test statistic according to (13);

3:  Calculate the p-value of the test according to (15);

4:  if $p>\alpha$ then

5:   Let bandwidth $K=k$ and stop;

6:  else

7:   $k=k+1$;

8:  end if

9: end for

3.4. Power Analysis

In the situation $N-k>M$, there may exist some alternatives which have ${\widehat{\mathit{f}}}_{-k}\ne \mathbf{0}$ but ${\mathit{U}}_{-k}{\widehat{\mathit{f}}}_{-k}=\mathbf{0}$. It is impossible to find a test which is optimal against all the alternatives, that is, uniformly most powerful. If the true ${\widehat{\mathit{f}}}_{-k}$ has large deviations from $H_0$, it is very easy to detect. However, if the deviations are small, the detection becomes harder. In the Bayesian score test of $H_0$, the deviations of the true ${\widehat{\mathit{f}}}_{-k}$ from $H_0$ is denoted by $\tau \mathbf{\xi }$, where $\mathbf{\xi }$ is assumed to follow a prior distribution with $\mathit{E}\left(\mathbf{\xi }\right)=\mathbf{0}$ and $\mathit{E}\left(\mathbf{\xi }{\mathbf{\xi }}^{\mathbf{T}}\right)\propto {\mathbf{\Sigma }}_{-k}$ and ${\tau }^2$ indicates how large the deviation is. Next, we will analyze the power of the Bayesian score test of $H_0$ when ${\tau }^2$ is small.

Definition 1 (Locally most powerful (LMP) [25]). Consider the problem of testing the simple null hypothesis $H:\theta ={\theta }_0$ against the one-sided alternative $K:\theta >{\theta }_0$. A significance level α test ${\Phi }_0$ with power function ${\beta }_{{\Phi }_0}\left(\theta \right)$ is said to be LMP if given any other level α test Φ with power function ${\beta }_{\Phi }\left(\theta \right)$, there exists $\Delta >0$ such that ${\beta }_{{\Phi }_0}\left(\theta \right)\ge {\beta }_{\Phi }\left(\theta \right)$ for all $\theta \in K$ with $0\le \theta -{\theta }_0\le \Delta$.

An LMP test is one of the best tests for detecting small deviations from null hypothesis, though it is not good at detecting all kinds of alternatives. In each stage of Algorithm 1, we aim at deciding whether the bandwidth is k or not, so small deviations in the coefficients of frequencies around k should be detected as possible as we can. Therefore, a LMP test is preferred in each stage. In Theorem 1, we will show that the Bayesian score test in each stage of Algorithm 1 is LMP in expectation.

Theorem 1.

Let $\overline{w}\left({\widehat{\mathit{f}}}_{-k}\right)$ be the power function of the level α Bayesian score test of $H_0$ and let $w\left({\widehat{\mathit{f}}}_{-k}\right)$ be the power function of any level α test of $H_0$. The Bayesian score test of $H_0$ is LMP in expectation against all ${\widehat{\mathit{f}}}_{-k}\in H_A$ with ${\widehat{\mathit{f}}}_{-k}=\tau \mathbf{\xi }$, where $0\le {\tau }^2\le \Delta$, that is ${\mathit{E}}_{\mathbf{\xi }}\left[w\left(\tau \mathbf{\xi }\right)\right]\le {\mathit{E}}_{\mathbf{\xi }}\left[\overline{w}\left(\tau \mathbf{\xi }\right)\right]$.

Proof of Theorem 1. 

Let $\overline{\beta }\left({\tau }^2\right)$ be the power function of the level $\alpha$ score test of ${\overline{H}}_0$. Since ${\widehat{\mathit{f}}}_{-k}=\mathbf{0}$ and ${\tau }^2=0$ lead to the same distribution of $\mathit{y}$, the level $\alpha$ tests of $H_0$ and ${\overline{H}}_0$ lead to the same critical region. The same conclusion holds for any other level $\alpha$ tests of $H_0$ and ${\overline{H}}_0$ with power function $w\left({\widehat{\mathit{f}}}_{-k}\right)$ and $\beta \left({\tau }^2\right)$. It has been proved in Reference [25] that the one-side score test for one dimensional parameter is LMP. Therefore, for the given level $\alpha$, there exists a $\Delta >0$ such that for all ${\tau }^2\in {\overline{H}}_A$ with $0\le {\tau }^2\le \Delta$, the power function of score test for (10) is larger than that of any other test for (10)

$$\begin{array}{c}\hfill \beta \left({\tau }^2\right)\le \overline{\beta }\left({\tau }^2\right).\end{array}$$ (16)

Let the critical region of the level $\alpha$ Bayesian score test be A, we have

$$\begin{array}{cc}\hfill \beta \left({\tau }^2\right)& ={\int }_A{\overline{L}}_k({\tau }^2;\mathit{y})\mathrm{d}\mu ,\hfill \end{array}$$ (17)

where $\mu$ is a dominating measure. According to (9), ${\overline{L}}_k({\tau }^2;\mathit{y})$ is the marginalized distribution of $L_k({\widehat{\mathit{f}}}_{-k};\mathit{y})$. According to (), every distribution shape of ${\widehat{\mathit{f}}}_{-k}=\tau \mathbf{\xi }$ with $\mathit{E}\left(\mathbf{\xi }\right)=\mathbf{0}$ and $\mathit{E}\left(\mathbf{\xi }{\mathbf{\xi }}^{\mathbf{T}}\right)\propto {\mathbf{\Sigma }}_{-k}$ leads to the same $S_{{\mathbf{\Sigma }}_{-k}}$, and therefore the same power function. So (17) can be written as

$$\begin{array}{c}\hfill \beta \left({\tau }^2\right)={\int }_A\left(\int L_k(\tau \mathbf{\xi };\mathit{y})\pi \left(\mathbf{\xi }\right)\mathrm{d}\mathbf{\xi }\right)\mathrm{d}\mu =\int \left({\int }_A{L}_k(\tau \mathbf{\xi };\mathit{y})\mathrm{d}\mu \right)\pi \left(\mathbf{\xi }\right)\mathrm{d}\mathbf{\xi }={\mathit{E}}_{\mathbf{\xi }}\left[\overline{w}\left(\tau \mathbf{\xi }\right)\right].\end{array}$$ (18)

According to (16) and (18), we have ${\mathit{E}}_{\mathbf{\xi }}\left[w\left(\tau \mathbf{\xi }\right)\right]\le {\mathit{E}}_{\mathbf{\xi }}\left[\overline{w}\left(\tau \mathbf{\xi }\right)\right]$. □

To make Theorem 1 more intuitive, we give the following example.

Example: Suppose there are two alternatives to be tested in step k, $H_A^{\mathrm{l}}={\widehat{\mathit{f}}}_{-k}=[0.1,0,\cdots ,0]$, which has a small deviation $0.1$ at a low frequency and $H_A^{\mathrm{h}}={\widehat{\mathit{f}}}_{-k}=[0,\cdots ,0.1,\cdots ,0]$, which has the same deviation at high frequency. The prior of ${\widehat{\mathit{f}}}_{-k}$ is given as (8), which means the test pays more attention to the low frequency components. Then, the power of testing $H_0$ against $H_A^{\mathrm{l}}$ will be larger than the power of testing $H_0$ against $H_A^{\mathrm{h}}$ at the same significance level. As a result, the average power of the Bayesian score test is increased since there are more alternatives having small deviations at the low frequencies in a bandlimited GS.

Furthermore, the proof of Theorem 1 does not rely on the specific form of ${\mathbf{\Sigma }}_{-k}$, which means that it will hold for different designs of ${\mathbf{\Sigma }}_{-k}$. We can design different ${\mathbf{\Sigma }}_{-k}$ to allocate attentions among frequency components to achieve various testing purposes with small sample size, for example, spectrum anomaly detection.

4. Numerical Analysis

4.1. Bandwidth Detection

An accurate bandwidth is helpful in various applications. For example, an accurate bandwidth in low-pass filter design for graph signals can help remove the noise and keep the original signal well; An accurate bandwidth can also help in choosing the minimal sample size in the sampling set design for graph signals. So in this section, we first validate the performance of bandwidth detection on an Erdos-Rènyi random graph with $N=250$ and the probability of edge presence being 0.25. The frequency coefficients of bandlimited GS are independently generated from a uniform distribution over the interval $[0,1]$. Gaussian noise is added to the GS to produce the observations, the signal to noise ratio (SNR) is calculated by $\mathrm{SNR}=10log\left({\parallel \mathit{f}\parallel }_2^2/\left(N{\sigma }^2\right)\right)$. The performance of bandwidth detection is shown in bias of 1000 simulations, the SNR is set to 20 dB and vertices are sampled randomly in each simulation. In this simulation, we design two priors for the alternatives in our method follows the guideline in Section 3.2 and the significance level of the test is set to $\alpha =0.05$. Prior 1 is given by (8) with $\delta =0.01$. Prior 2 is given by

$$\begin{array}{c}\hfill {\left({\mathbf{\Sigma }}_{-k}\right)}_{i,i}=\left\{\begin{array}{cc}1-\frac{i-1}{M-k}+\delta \hfill & \mathrm{if}i\le M-k+1;\hfill \\ \delta \hfill & \mathrm{if}i>M-k+1,\hfill \end{array}\right.\end{array}$$ (19)

with $\delta =0.01$. To show the effectiveness of our prior designed for bandwidth detection, we compare the mean bandwidth obtained by giving ${\widehat{\mathit{f}}}_{-k}$ our designed priors and uniform prior ${\mathbf{\Sigma }}_{-k}={\mathit{I}}_{(N-k)\times (N-k)}$, as shown in Figure 2. The test with uniform prior fails to detect the bandwidth when $M=80$, while the test with designed prior 1 and prior 2 performs well under the same sample size. When the sample size increases to $2N$, the test with uniform prior turns out to have an acceptable performance, but still worse than the test with designed priors with $M=80$. This implies that the prior we designed for bandwidth detection is very helpful for saving the sample size. We can also find that the bandwidth detection performance of our method with prior 1 and prior 2 are comparable, which indicates the guideline of prior design in our algorithm improves the detection performance instead of a specific form of prior.

Figure 2.

Bias of bandwidth detection under different priors.

The performance of our method is also compared with the bandwidth estimation method in Reference [15], in which a dictionary of 80 bandlimited kernels is constructed with $\beta ={10}^3$ and the regularization parameter $\mu$ in it is set to 0.01 by cross validation. We first simulate the performance of bandwidth detection at different noise levels for bandwidth $K=\{10,30,50\}$, the sample size is set to $M=80$. The results are shown in Figure 3. We can find that the bias and standard deviation (SD) of Algorithm 1 and sparsity based method [15] are similar for small bandwidth at low noise level. For large bandwidth, the bias and SD of our method are significantly smaller than those of Reference [15]. This is because the GS is no longer sparse for large bandwidth, which is out of the scope of sparsity based method [15]. At high noise level, Algorithm 1 is much more robust than Reference [15] especially when the bandwidth is close to the sample size.

Figure 3.

Bias and SD of bandwidth detection at different noise levels.

In Figure 4, we show that the increase of sample size can help decrease the bias and SD of Algorithm 1. However, when the sample size is abundant, the performance improvement caused by the increase of sample size can be ignored.

Figure 4.

Bias and SD of bandwidth detection with different sample sizes when $K=30$.

4.2. Signal Estimation

Bandwidth is an useful prior when estimating signals from partial observations. In this section, we use the bandwidth detected by Algorithm 1 as a prior in signal estimation and a least squares (LS) estimator is used to estimate the signal with the form

$${\mathit{f}}^{\prime }={\mathit{V}}_K{\left({\mathit{V}}_K^T{\mathit{V}}_K\right)}^{-1}{\mathit{V}}_K^T\mathit{y}.$$ (20)

The estimation performance is evaluated by the normalized mean square error (NMSE), which is

$$\mathrm{NMSE}=\frac{\parallel ({\mathit{f}}^{\prime }-\mathit{f}){\parallel }_2^2}{{\parallel \mathit{f}\parallel }_2^2}.$$ (21)

We first show the estimation performance on the same graph signal with that in Section 4.1. The bandwidth of signal varies from 10 to 60, the sample size is set to $M=80$ and the SNR is set to 20 dB. Our method is compared with two methods based on sparsity, namely BP [26] and BCS [21], which also select frequency components to estimate the signal. The result is show in Figure 5, we can find that as the bandwidth increases, the performance of the sparsity base methods degrade rapidly.

Figure 5.

NMSE of graph signal (GS) estimation of different algorithms under different bandwidths.

This is because the sparsity based methods select frequency components in the whole space and the sparsity constraint makes the amount of frequency components they selected as small as possible. Therefore, they are not fit for the application when the frequency spectrum is not sparse.

We also compare the performance of the signal estimation in real-data set with BP and BCS. The data set comprises 22 signals corresponding to the average temperature on January 1st in the intervals 1998–2019 measured by 119 stations in China [27]. Each station is identified with a vertex and the distance between 2 vertices is calculated by the haversine formula. A 3-NN unweighted graph is constructed. The GS is shown in Figure 6a and the corresponding spectrum is shown in Figure 6b.

Figure 6.

GS constructed from the temperature measured by 119 stations on 1st January 1998 and its spectrum. (a) GS on the 3-NN graph. (b) Spectrum of the GS in (a).

$i.i.d.$ Gaussian noise with SNR = 20 dB is added to the signal to generate observations. 200 simulations are implemented for each signal, the vertices are sampled randomly in each simulation. We can find that the temperatures distribute over the graph smoothly, so there are only a few low frequency components, which is sparse in spectrum. Since the GS is approximately bandlimited in real-data, the approximate bandwidth obtained by Algorithm 1 is used in (20). The average NMSE of all the simulations under different sample sizes is shown in Figure 7.

Figure 7.

NMSE of GS estimation of different algorithms under different sample sizes.

We can find that the estimation performance of our method is also better than the sparsity based methods in real-data. Since BP ensures the sparsity of the signal by $l_1$-norm, it results in obtaining the same signal values on each vertex in this experiment. BCS selects 14 nonadjacent frequency components in all the frequency components, including high frequencies and low frequencies. The approximate bandwidth obtained by Algorithm 1 is 12, which means we estimate the GS using the first 12 frequency components. We can find that if the GS is known to be bandlimited, the signal estimation performance can be improved by detecting the bandwidth accurately first and using the bandwidth as a prior in estimation. If the knowledge that the GS is bandlimited is not take advantage of, more samples are needed to achieve the same estimation performance.

5. Conclusions

In this letter, we proposed a multi-stage Bayesian score test approach for the bandlimited GS bandwidth detection with a small sample size. By customizing a prior for frequency coefficients being tested, we made the test more distinguishable from the models with bandwidths close to the assumed one. In practice, our method may obtain a bandwidth smaller than the true bandwidth K if there is a frequency band of which the coefficients are zeros in band, since we focus on testing the frequency coefficients close to the assumed one but ignore the others in the small sample size situation. We will try to improve the performance under this situation in the future. In addition, we would like to analyze how the different sampling sets affect the performance of bandwidth detection and design a sampling set to make the test more powerful.

Author Contributions

Conceptualization, X.X. and H.F.; methodology, X.X.; validation, X.X.; formal analysis, X.X.; investigation, X.X.; resources, X.X.; writing—original draft preparation, X.X.; writing—review and editing, X.X., H.F. and B.H.; visualization, X.X. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Shanghai Municipal Natural Science Foundation number 19ZR1404700 and 2020 Okawa Foundation Research Grant.

Conflicts of Interest

The authors declare no conflict of interest.