Distributed Hypothesis Testing with Privacy Constraints

Abstract

We revisit the distributed hypothesis testing (or hypothesis testing with communication constraints) problem from the viewpoint of privacy. Instead of observing the raw data directly, the transmitter observes a sanitized or randomized version of it. We impose an upper bound on the mutual information between the raw and randomized data. Under this scenario, the receiver, which is also provided with side information, is required to make a decision on whether the null or alternative hypothesis is in effect. We first provide a general lower bound on the type-II exponent for an arbitrary pair of hypotheses. Next, we show that if the distribution under the alternative hypothesis is the product of the marginals of the distribution under the null (i.e., testing against independence), then the exponent is known exactly. Moreover, we show that the strong converse property holds. Using ideas from Euclidean information theory, we also provide an approximate expression for the exponent when the communication rate is low and the privacy level is high. Finally, we illustrate our results with a binary and a Gaussian example.

1. Introduction

In the distributed hypothesis testing (or hypothesis testing with communication constraints) problem, some observations from the environment are collected by the sensors in a network. They describe these observations over the network which are finally received by the decision center. The goal is to guess the joint distribution governing the observations at terminals. In particular, there are two possible hypotheses $\mathcal{H}=0$ or $\mathcal{H}=1$, where the joint distribution of the observations is specified under each of them. The performance of this system is characterized by two criteria: the type-I and the type-II error probabilities. The probability of deciding on $\mathcal{H}=1$ (respectively $\mathcal{H}=0$) when the original hypothesis is $\mathcal{H}=0$ (respectively $\mathcal{H}=1$) is referred to as the type-I error (type-II error) probability. There are several approaches for defining the performance of a hypothesis test. First, we can maximize the exponent (exponential rate of decay) of the Bayesian error probability. Second, we can impose that the type-II error probability decays exponentially fast and we can then maximize the exponent of the type-I error probability; this is known as the Hoeffding regime. The approach in this work is the Chernoff-Stein regime in which we upper bound the type-II error probability by a non-vanishing constant and we maximize the exponent of the type-II error probability.

A special case of interest is testing against independence where the joint distribution under $\mathcal{H}=1$ is the product of the marginals under $\mathcal{H}=0$. The optimal exponent of type-II error probability for testing against independence is determined by Ahlswede and Csiszár in [1]. Several extensions of this basic problem are studied for a multi-observer setup [2,3,4,5,6], a multi-decision center setup [7,8] and a setup with security constraints [9]. The main idea of the achievable scheme in these works is typicality testing [10,11]. The sensor finds a jointly typical codeword with its observation and sends the corresponding bin index to the decision center. The final decision is declared based on typicality check of the received codeword with the observation at the center. We note that the coding scheme employed here is reminiscent of those used for source coding with side information [12] and for different variants of the information bottleneck problem [13,14,15,16].

1.1. Injecting Privacy Considerations into Our System

We revisit the distributed hypothesis testing problem from a privacy perspective. In many applications such as healthcare systems, there is a need to randomize the data before publishing it. For example, hospitals often have large amounts of medical records of their patients. These records are useful for performing various statistical inference tasks, such as learning about causes of a certain ailment. However, due to privacy considerations of the patients, the data cannot be published as is. The data needs to be sanitized, quantized, perturbed and then be fed to a management center before statistical inference, such as hypothesis testing, is being done.

In the proposed setup, we use a privacy mechanism to sanitize the observation at the terminal before it is compressed; see Figure 1. The compression is performed at a separate terminal called transmitter, which communicates the randomized data over a noiseless link of rate R to a receiver. The hypothesis testing is performed using the received data (the compression index and additional side information) to determine the correct hypothesis governing the original observations. The privacy criterion is defined by the mutual information [17,18,19,20] of the published and original data.

Figure 1.

Hypothesis testing with communication and privacy constraints.

There is a long history of research to provide appropriate metrics to measure privacy. To quantify the information leakage an observation $\widehat{X}$ can induce on a latent variable X, Shannon’s mutual information $I(X;\widehat{X})$ is considered in [17,18,19,20]. Smith [18] proposed to use Arimoto’s mutual information of order ∞, $I_{\infty }(X;\widehat{X})$. Barthe and Köpf [21,22,23] proposed the maximal information leakage ${max}_{P_X}{I}_{\infty }(X;\widehat{X})$. We refer the reader to [24] for a survey on the existing information leakage measures. A different line of works, in statistics, computer science, and other related fields, concerns differential privacy, initially proposed in [25]. Furthermore, a generalized notion—$(ϵ,\delta )$-differential privacy [26]—provides a unified mathematical framework for data privacy. The reader is referred to the survey by Dwork [27] and the statistical framework studied by Wasserman and Zhou [28] and the references therein.

The privacy mechanism can be either memoryless or non-memoryless. In the former, the distribution of the randomized data at each time instant depends on the original sequence at the same time and not on the previous history of the data.

1.2. Description of Our System Model

We propose a coding scheme for the proposed setup. The idea is that the sensor, upon observing the source sequence, performs a typicality test and obtains its belief of the hypothesis. If the belief is $\mathcal{H}=0$, it publishes the randomized data based on a specific memoryless mechanism. However, if its belief is $\mathcal{H}=1$, it sends an all-zero sequence to let the transmitter know about its decision. The transmitter communicates the received data, which is a sanitized version of the original data or an all-zero sequence, over the noiseless link to the receiver. In this scheme, the whole privacy mechanism is non-memoryless since the typicality check of the source sequence which uses the history of the observation, determines the published data. It is shown that the achievable error exponent recovers previous results on hypothesis testing with zero and positive communication rates in [10]. Our work is related to a recent work [29] where a general hypothesis testing setup is considered from a privacy perspective. However, in [29], the problem at hand is different from ours. The authors consider equivocation and average distortion as possible measures of privacy whereas we constrain the mutual information between the original and released (published) data.

A difference of the proposed scheme with some previous works is highlighted as follows. The privacy mechanism even if it is memoryless, cannot be viewed as a noiseless link of a rate equivalent to the privacy criterion. Particularly, the proposed model is different from cascade hypothesis testing problem of [8] or similar works [3,4] which consider consecutive noiseless links for data compression and distributed hypothesis testing. The difference comes from the fact that in these works, a codeword is chosen jointly typical with the observed sequence at the terminal and its corresponding index is sent over the noiseless link. However, in our model, the randomized sequence is not necessarily jointly typical with the original sequence. Thus, there is a need for an achievable scheme which lets the transmitter know whether the original data is typical or not.

The problem of hypothesis testing against independence with a memoryless privacy mechanism is also considered. A coding scheme is proposed where the sensor outputs the randomized data based on the memoryless privacy mechanism. The optimality of the achievable type-II error exponent is shown by providing a strong converse. Specializing the optimal error exponent to a binary example shows that an increase in the privacy criterion (a less stringent privacy mechanism) results in a larger type-II error exponent. Thus, there exists a trade-off between privacy and hypothesis testing criteria. The optimal type-II error exponent is further studied for the case of restricted privacy mechanism and zero-rate communication. The Euclidean approach of [30,31,32,33] is used to approximate the error exponent for this regime. The result confirms the trade-off between the privacy criterion and type-II error exponent. Finally, a Gaussian setup is proposed and its optimal error exponent is established.

1.3. Main Contributions

The contributions of the paper are listed in the following:

• An achievable type-II error exponent is proposed using a non-memoryless privacy mechanism (Theorem 1 in Section 3);

• The optimal error exponent of testing against independence with a memoryless privacy mechanism is determined. In addition, a strong converse is also proved (Theorem 2 in Section 4.1);

• A binary example is proposed to show the trade-off between the privacy and error exponent (Section 4.3);

• An Euclidean approximation [30] of the error exponent is provided (Section 4.4);

• A Gaussian setup is proposed and its optimal error exponent is derived (Proposition 2 in Section 4.5).

1.4. Notation

The notation mostly follows [34]. Random variables are denoted by capital letters, e.g., X, Y, and their realizations by lower case letters, e.g., x, y. The alphabet of the random variable X is denoted as $\mathcal{X}$. Sequences of random variables and their realizations are denoted by $(X_i,\dots ,X_j)$ and $(x_i,\dots ,x_j)$ and are abbreviated as $X_i^j$ and $x_i^j$. We use the alternative notation $X^j$ when $i=1$. Vectors and matrices are denoted by boldface letters, e.g., $\mathbf{k}$, $\mathbf{W}$. The ${\ell }_2$-norm of $\mathbf{k}$ is denoted as $\parallel \mathbf{k}\parallel$. The notation ${\mathbf{k}}^T$ denotes the transpose of $\mathbf{k}$.

The probability mass function (pmf) of a discrete random variable X is denoted as $P_X$, the conditional pmf of X given Y is denoted as $P_{X|Y}$. The notation $D(P_X\parallel Q_X)$ denotes the Kullback-Leibler (KL) divergence between two pmfs $P_X$ and $Q_X$. The total variation distance between two pmfs $P_X$ and $Q_X$ is denoted by $|P_X-Q_X|=\frac{1}{2}{\sum }_x|P_X\left(x\right)-Q_X\left(x\right)|$. We use $\mathrm{tp}(x^n,y^n)$ to denote the joint type of $(x^n,y^n)$.

For a given $P_{XY}$ and a positive number $\mu$, we denote by ${\mathcal{T}}_{\mu }^n\left(P_{XY}\right)$, the set of jointly $\mu$-typical sequences [34], i.e., the set of all $(x^n,y^n)$ whose joint type is within $\mu$ of $P_{XY}$ (in the sense of total-variation distance). The notation ${\mathcal{T}}^n\left(P_X\right)$ denotes for the type class of the type $P_X$.

The notation $h_{\mathrm{b}}(·)$ denotes the binary entropy function, $h_{\mathrm{b}}^{-1}(·)$ its inverse over $\left[0,\frac{1}{2}\right]$, and $a\star b\triangleq a(1-b)+(1-a)b$ for $0\le a,b\le 1$. The differential entropy of a continuous random variable X is $h\left(X\right)$. All logarithms $log(·)$ are taken with respect to base 2.

1.5. Organization

The remainder of the paper is organized as follows. Section 2 describes a mathematical setup for our proposed problem. Section 3 discusses hypothesis testing with general distributions. The results for hypothesis testing against independence with a memoryless privacy mechanism are provided in Section 4. The paper is concluded in Section 5.

2. System Model

Let $\mathcal{X}$, $\mathcal{Y}$, and $\widehat{\mathcal{X}}$ be arbitrary finite alphabets and let n be a positive integer. Consider the hypothesis testing problem with communication and privacy constraints depicted in Figure 1. The first terminal in the system, the Randomizer, receives the sequence $X^n=(X_1,\dots ,X_n)\in {\mathcal{X}}^n$ and outputs the sequence ${\widehat{X}}^n=({\widehat{X}}_1,\dots ,{\widehat{X}}_n)\in {\widehat{\mathcal{X}}}^n$, which is a noisy version of $X^n$ under a privacy mechanism determined by the conditional probability distribution $P_{{\widehat{X}}^n|X^n}$; the second terminal, the Transmitter, receives the sequence ${\widehat{X}}^n$; the third terminal, the Receiver, observes the side-information sequence $Y^n=(Y_1,\dots ,Y_n)\in {\mathcal{Y}}^n$. Under the null hypothesis

$$\begin{array}{c}\hfill \mathcal{H}=0:(X^n,Y^n)\sim \mathrm{i}.\mathrm{i}.\mathrm{d}.P_{XY},\end{array}$$ (1)

whereas under the alternative hypothesis

$$\begin{array}{c}\hfill \mathcal{H}=1:(X^n,Y^n)\sim \mathrm{i}.\mathrm{i}.\mathrm{d}.Q_{XY},\end{array}$$ (2)

for two given pmfs $P_{XY}$ and $Q_{XY}$.

The privacy mechanism is described by the conditional pmf $P_{{\widehat{X}}^n|X^n}$ which maps each sequence $X^n\in {\mathcal{X}}^n$ to a sequence ${\widehat{X}}^n\in {\widehat{\mathcal{X}}}^n$. For any $({\widehat{x}}^n,x^n,y^n)\in {\widehat{\mathcal{X}}}^n\times {\mathcal{X}}^n\times {\mathcal{Y}}^n$, the joint distributions considering the privacy mechanism are given by

$$\begin{array}{cc}\hfill P_{\widehat{X}XY}^n({\widehat{x}}^n,x^n,y^n)& \triangleq P_{{\widehat{X}}^n|X^n}\left({\widehat{x}}^n\right|x^n)·\prod _{i=1}^n{P}_{XY}(x_i,y_i),\hfill \end{array}$$ (3)

$$\begin{array}{cc}\hfill Q_{\widehat{X}XY}^n({\widehat{x}}^n,x^n,y^n)& \triangleq P_{{\widehat{X}}^n|X^n}\left({\widehat{x}}^n\right|x^n)·\prod _{i=1}^n{Q}_{XY}(x_i,y_i).\hfill \end{array}$$ (4)

A memoryless/local privacy mechanism is defined by a conditional pmf $P_{\widehat{X}|X}$ which stochastically and independently maps each entry $X_i\in \mathcal{X}$ of $X^n$ to a released ${\widehat{X}}_i\in \widehat{\mathcal{X}}$ to construct ${\widehat{X}}^n$. Consequently, for the memoryless privacy mechanism, the conditional pmf $P_{{\widehat{X}}^n|X^n}\left({\widehat{x}}^n\right|x^n)$ factorizes as follows:

$$\begin{array}{c}\hfill P_{{\widehat{X}}^n|X^n}\left({\widehat{x}}^n\right|x^n)=\prod _{i=1}^n{P}_{\widehat{X}|X}\left({\widehat{x}}_i\right|x_i)=P_{\widehat{X}|X}^n\left({\widehat{x}}^n\right|x^n),\forall ({\widehat{x}}^n,x^n)\in {\widehat{\mathcal{X}}}^n\times {\mathcal{X}}^n.\end{array}$$ (5)

There is a noise-free bit pipe of rate R from the transmitter to the receiver. Upon observing ${\widehat{X}}^n$, the transmitter computes the message $M={\varphi }^{\left(n\right)}\left({\widehat{X}}^n\right)$ using a possibly stochastic encoding function ${\varphi }^{\left(n\right)}:{\widehat{\mathcal{X}}}^n\to \{0,\dots ,\lfloor 2^{nR}\rfloor \}$ and sends it over the bit pipe to the receiver.

The goal of the receiver is to produce a guess of $\mathcal{H}$ using a decoding function $g^{\left(n\right)}:{\mathcal{Y}}^n\times \{0,...,\lfloor 2^{nR}\rfloor \}\to \{0,1\}$ based on the observation $Y^n$ and the received message M. Thus the estimate of the hypothesis is $\widehat{\mathcal{H}}=g^{\left(n\right)}(Y^n,M)$.

This induces a partition of the sample space ${\widehat{\mathcal{X}}}^n\times {\mathcal{X}}^n\times {\mathcal{Y}}^n$ into an acceptance region ${\mathcal{A}}_n$ defined as follows:

$$\begin{array}{c}{\mathcal{A}}_n\triangleq \left\{({\widehat{x}}^n,x^n,y^n):g^{\left(n\right)}(y^n,{\varphi }^{\left(n\right)}\left({\widehat{x}}^n\right))=0\right\},\hfill \end{array}$$ (6)

and a rejection region denoted by ${\mathcal{A}}_n^c$.

Definition 1.

For any $ϵ\in [0,1)$ and for a given rate-privacy pair $(R,L)\in {\mathbb{R}}_{+}^2$, we say that a type-II exponent $\theta \in {\mathbb{R}}_{+}$ is $(ϵ,R,L)$-achievable if there exists a sequence of functions and conditional pmfs $({\varphi }^{\left(n\right)},g^{\left(n\right)},P_{{\widehat{X}}^n|X^n})$, such that the corresponding sequences of type-I and type-II error probabilities at the receiver are defined as

$$\begin{array}{c}\hfill {\alpha }_n\triangleq P_{\widehat{X}XY}^n\left({\mathcal{A}}_n^c\right)and{\beta }_n\triangleq Q_{\widehat{X}XY}^n\left({\mathcal{A}}_n\right),\end{array}$$ (7)

respectively, and they satisfy

$$\begin{array}{cc}\hfill \underset{n\to \infty }{lim\; sup}{\alpha }_n& \le ϵ\mathrm{and}\underset{n\to \infty }{lim\; inf}\frac{1}{n}log\frac{1}{{\beta }_n}\ge \theta .\hfill \end{array}$$ (8)

Furthermore, the privacy measure

$$T_n\triangleq \frac{1}{n}I(X^n;{\widehat{X}}^n),$$ (9)

satisfies

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim\; sup}{T}_n\le L.\end{array}$$ (10)

The optimal exponent ${\theta }_{ϵ}^{*}(R,L)$ is the supremum of all $(ϵ,R,L)$-achievable $\theta \in {\mathbb{R}}_{+}$.

3. General Hypothesis Testing

3.1. Achievable Error Exponent

The following presents an achievable error exponent for the proposed setup.

Theorem 1.

For a given $ϵ\in [0,1)$ and a rate-privacy pair $(R,L)\in {\mathbb{R}}_{+}^2$, the optimal type-II error exponent ${\theta }_{ϵ}^{*}(R,L)$ for the multiterminal hypothesis testing setup under the privacy constraint L and the rate constraint R satisfies

$$\begin{array}{c}\hfill rCl{\theta }_{ϵ}^{*}(R,L)\ge \underset{\begin{array}{c}{P}_{U|\widehat{X}},P_{\widehat{X}|X}:\\ R\ge I(U;\widehat{X})\\ L\ge I(X;\widehat{X})\end{array}}{max}\underset{\begin{array}{c}{\tilde{P}}_{U\widehat{X}XY}\in {\mathcal{P}}_{U\widehat{X}XY}\end{array}}{min}D({\tilde{P}}_{U\widehat{X}XY}\parallel P_{U|\widehat{X}}{P}_{\widehat{X}|X}{Q}_{XY}),\end{array}$$ (11)

where the set ${\mathcal{P}}_{U\widehat{X}XY}$ is defined as

$${\mathcal{P}}_{U\widehat{X}XY}\triangleq \left\{{\tilde{P}}_{U\widehat{X}XY}\left|\begin{array}{c}{\tilde{P}}_X=P_X,\hfill \\ {\tilde{P}}_{UY}=P_{UY},\\ {\tilde{P}}_{U\widehat{X}}=P_{U\widehat{X}}\end{array}\right.\right\}.$$ (12)

Given $P_{U|\widehat{X}}$ and $P_{\widehat{X}|X}$, the mutual informations in (11) are calculated according to the following joint distribution:

$$\begin{array}{cc}\hfill P_{U\widehat{X}|X}& \triangleq P_{U|\widehat{X}}·P_{\widehat{X}|X}.\hfill \end{array}$$ (13)

Proof. 

The coding scheme is given in the following section. For the analysis, see Appendix A. □

3.2. Coding Scheme

In this section, we propose a coding scheme for Theorem 1, under fixed rate and privacy constraints $(R,L)\in {\mathbb{R}}_{+}^2$. Fix the joint distribution $P_{U\widehat{X}XY}$ as in (13). Let $P_U\left(u\right)$ be the marginal distribution of $U\in \mathcal{U}$ defined as

$$\begin{array}{c}\hfill P_U\left(u\right)\triangleq \sum _{\widehat{x}\in \widehat{\mathcal{X}}}{P}_{U|\widehat{X}}\left(u\right|\widehat{x})\sum _{x\in \mathcal{X}}{P}_{\widehat{X}X}(\widehat{x},x).\end{array}$$ (14)

Fix positive $\mu >0$ and $\zeta >0$, an arbitrary blocklength n and two conditional pmfs $P_{\widehat{X}|X}$ and $P_{U|\widehat{X}}$ over finite auxiliary alphabets $\widehat{\mathcal{X}}$ and $\mathcal{U}$. Fix also the rate and privacy leakage level as

$$\begin{array}{c}\hfill R=I(U;\widehat{X})+\mu ,\mathrm{and}L=I(\widehat{X};X)+\zeta .\end{array}$$ (15)

Codebook Generation: Randomly and independently generate a codebook

$$\begin{array}{c}\hfill {\mathcal{C}}_U\triangleq \left\{U^n\left(m\right):m\in \{0,\dots ,\lfloor 2^{nR}\rfloor \}\right\},\end{array}$$ (16)

by drawing $U^n\left(m\right)$ in an i.i.d. manner according to $P_U$. The codebook is revealed to all terminals.

Randomizer: Upon observing $x^n$, it checks whether $x^n\in {\mathcal{T}}_{\mu /4}^n\left(P_X\right).$ If successful, it outputs the sequence ${\widehat{x}}^n$ where its i-th component ${\widehat{x}}_i$ is generated based on $x_i$, according to $P_{\widehat{X}|X}\left({\widehat{x}}_i\right|x_i)$. If the typicality check is not successful, the randomizer then outputs $0^n$ which is an all-zero sequence of length n, where ${\widehat{x}}^n=0^n$.

Transmitter: Upon observing ${\widehat{x}}^n$, if ${\widehat{x}}^n\ne 0^n$, the transmitter finds an index m such that $\left(u^n\left(m\right),{\widehat{x}}^n\right)\in {\mathcal{T}}_{\mu /2}^n\left(P_{U\widehat{X}}\right).$ If successful, it sends the index m over the noiseless link to the receiver. Otherwise, if the typicality check is not successful or ${\widehat{x}}^n=0^n$, it sends $m=0$.

Receiver: Upon observing $y^n$ and receiving the index m, if $m=0$, the receiver declares $\widehat{\mathcal{H}}=1$. If $m\ne 0$, it checks whether $\left(u^n\left(m\right),y^n\right)\in {\mathcal{T}}_{\mu }^n\left(P_{UY}\right).$ If the test is successful, the receiver declares $\widehat{\mathcal{H}}=0$; otherwise, it sets $\widehat{\mathcal{H}}=1$.

Remark 1.

In the above scheme, the sequence ${\widehat{X}}^n$ is chosen to be an n-length zero-sequence when the randomizer finds that $X^n$ is not typical according to $P_X$. Thus, the privacy mechanism is not memoryless and the sequence ${\widehat{X}}^n$ is not identically and independently distributed (i.i.d.). A detailed analysis in Appendix A shows that the privacy criterion is not larger than L as the blocklength $n\to \infty$.

3.3. Discussion

In the following, we discuss some special cases. First, suppose that $R=0$. The following corollary shows that Theorem 1 recovers Han’s result [1] for distributed hypothesis testing with zero-rate communication.

Corollary 1

(Theorem 5 in [10]). Suppose that $Q_{XY}>0$. For all $ϵ\in [0,1)$, the optimal error exponent of the zero-rate communication for any privacy mechanism (including non-memoryless mechanisms) is given by the following:

$$\begin{array}{c}\hfill {\theta }_{ϵ}^{*}(0,L)=\underset{\begin{array}{c}{\tilde{P}}_{XY}:\\ {\tilde{P}}_X=P_X\\ {\tilde{P}}_Y=P_Y\end{array}}{min}D({\tilde{P}}_{XY}\parallel Q_{XY}).\end{array}$$ (17)

Proof. 

The proof of achievability follows by Theorem 1, in which $\widehat{X}$ is arbitrary and the auxiliary $U=\varnothing$ due to the zero-rate constraint. The proof of the strong converse follows along the same lines as [35]. □

Remark 2.

Consider the case of $R>0$ and $L=0$ where $\widehat{X}$ is independent of X. Using Theorem 1, the optimal error exponent is lower bounded as follows:

$$\begin{array}{c}\hfill {\theta }_{ϵ}^{*}(R,0)\ge \underset{\begin{array}{c}{\tilde{P}}_{XY}:\\ {\tilde{P}}_X=P_X\\ {\tilde{P}}_Y=P_Y\end{array}}{min}D({\tilde{P}}_{XY}\parallel Q_{XY}).\end{array}$$ (18)

However, there is no known converse result in this case where the communication rate is positive. Comparing this special case with the one in Corollary 1 shows that the proposed model does not, in general, admit symmetry between the rate and privacy constraints. However, we will see from some specific examples in the following that the roles of R and L are symmetric.

Now, suppose that L is so large such that $L>H\left(X\right)$. The following corollary shows that Theorem 1 recovers Han’s result in [10] for distributed hypothesis testing over a rate-R communication link.

Corollary 2

(Theorem 2 in [10]). Assuming $L>H\left(X\right)$, the optimal error exponent is lower bounded as the following:

$${\theta }_{ϵ}^{*}(R,L)\ge \underset{\begin{array}{c}{P}_{U|X}:\\ R\ge I(U;X)\end{array}}{max}\underset{\begin{array}{c}{\tilde{P}}_{UXY}:\\ {\tilde{P}}_{UX}=P_{UX}\\ {\tilde{P}}_{UY}=P_{UY}\end{array}}{min}D({\tilde{P}}_{UXY}\parallel P_{U|X}{Q}_{XY}).$$ (19)

Proof. 

The proof follows from Theorem 1 by specializing to $\widehat{X}=X$. □

The above two special cases reveal a trade-off between the privacy criterion and the achievable error exponent when the communication rate is positive, i.e., $R>0$. An increase in L results in a larger achievable error exponent. This observation is further illustrated by an example in Section 4.3 to follow.

4. Hypothesis Testing against Independence with a Memoryless Privacy Mechanism

In this section, we consider testing against independence where the joint pmf under $\mathcal{H}=1$ factorizes as follows:

$$\begin{array}{c}\hfill Q_{XY}=P_X·P_Y.\end{array}$$ (20)

The privacy mechanism is assumed to be memoryless here.

4.1. Optimal Error Exponent

The following theorem, which includes a strong converse, states the optimal error exponent for this special case.

Theorem 2.

For any $(R,L)\in {\mathbb{R}}_{+}^2$, define

$${\theta }_{ϵ}^{*}(R,L)=\underset{\begin{array}{c}{P}_{U|\widehat{X}},P_{\widehat{X}|X}:\\ R\ge I(U;\widehat{X})\\ L\ge I(X;\widehat{X})\end{array}}{max}I(U;Y).$$ (21)

Then, for any $ϵ\in [0,1)$ and any $(R,L)\in {\mathbb{R}}_{+}^2$, the optimal error exponent for testing against independence when using a memoryless privacy mechanism is given by (21), where it suffices to choose $\left|\mathcal{U}\right|\le |\widehat{\mathcal{X}}|+1$ and $|\widehat{\mathcal{X}}|\le |\mathcal{X}|$ according to Caratheodory’s theorem [36] (Theorem 15.3.5).

Proof. 

The coding scheme is given in the following section. For the rest of proof, see Appendix B. □

4.2. Coding Scheme

In this section, we propose a coding scheme for Theorem 2. Fix the joint distribution as in (13), and the rate and privacy constraints as in (15). Generate the codebook ${\mathcal{C}}_U$ as in (16).

Randomizer: Upon observing $x^n$, it outputs the sequence ${\widehat{x}}^n$ in which the i-th component ${\widehat{x}}_i$ is generated based on $x_i$, according to $P_{\widehat{X}|X}\left({\widehat{x}}_i\right|x_i)$.

Transmitter: It finds an index m such that $\left(u^n\left(m\right),{\widehat{x}}^n\right)\in {\mathcal{T}}_{\mu /2}^n\left(P_{U\widehat{X}}\right).$ If successful, it sends the index m over the noiseless link to the receiver. Otherwise, it sends $m=0$.

Receiver: Upon observing $y^n$ and receiving the index m, if $m=0$, the receiver declares $\widehat{\mathcal{H}}=1$. If $m\ne 0$, it checks whether $\left(u^n\left(m\right),y^n\right)\in {\mathcal{T}}_{\mu }^n\left(P_{UY}\right).$ If the test is successful, the receiver declares $\widehat{\mathcal{H}}=0$; otherwise, it sets $\widehat{\mathcal{H}}=1$.

Remark 3.

In the above scheme, the sequence ${\widehat{X}}^n$ is i.i.d. since it is generated based on the memoryless mechanism $P_{\widehat{X}|X}$.

When the communication rate is positive, there exists a trade-off between the optimal error exponent and the privacy criterion. The following example elucidates this trade-off.

4.3. Binary Example

In this section, we study hypothesis testing against independence for a binary example. Suppose that under both hypotheses, we have $X\sim \mathrm{Bern}\left(\frac{1}{2}\right)$. Under the null hypothesis,

$$\begin{array}{c}\hfill \mathcal{H}=0:Y=X\oplus N,N\sim \mathrm{Bern}\left(q\right)\end{array}$$ (22)

for some $0\le q\le 1$, where N is independent of X. Under the alternative hypothesis

$$\mathcal{H}=1:Y\sim \mathrm{Bern}\left(\frac{1}{2}\right),$$ (23)

where Y is independent of X. The cardinality constraint shows that it suffices to choose $|\widehat{\mathcal{X}}|=2$. Among all possible privacy mechanisms, the choice of $P_{\widehat{X}|X}\left(1\right|0)=P_{\widehat{X}|X}\left(1\right|1)$ and $P_{\widehat{X}|X}\left(0\right|0)=P_{\widehat{X}|X}\left(0\right|1)$ minimizes the mutual information $I(X;\widehat{X})$. Thus, we restrict to this choice which also results in $\widehat{X}\sim \mathrm{Bern}\left(\frac{1}{2}\right)$.

The cardinality bound on the auxiliary random variable U is $\left|\mathcal{U}\right|\le 3$. The following proposition states that it is also optimal to choose $P_{U|\widehat{X}}$ to be a BSC.

Proposition 1.

The optimal error exponent of the proposed binary setup is given by the following:

$$\begin{array}{cc}\hfill {\theta }_{ϵ}^{*}(R,L)& =1-h_{\mathrm{b}}\left(q\star h_{\mathrm{b}}^{-1}(1-L)\star h_{\mathrm{b}}^{-1}(1-R)\right).\hfill \end{array}$$ (24)

Proof. 

For the proof of achievability, choose the following auxiliary random variables:

$$\begin{array}{cc}\hfill \widehat{X}& =X\oplus \widehat{Z},\widehat{Z}\sim \mathrm{Bern}\left(p_1\right)\hfill \end{array}$$ (25)

$$\begin{array}{cc}\hfill U& =\widehat{X}\oplus Z,Z\sim \mathrm{Bern}\left(p_2\right),\hfill \end{array}$$ (26)

for some $0\le p_1,p_2\le 1$ where $\widehat{Z}$ and Z are independent of X and $(X,\widehat{X})$, respectively. The optimal error exponent of Theorem 2 reduces to the following:

$$\begin{array}{cc}\hfill {\theta }_{ϵ}^{*}(R,L)& =\underset{\begin{array}{c}0\le p_1,p_2\le 1:\\ R\ge 1-h_{\mathrm{b}}\left(p_2\right)\\ L\ge 1-h_{\mathrm{b}}\left(p_1\right)\end{array}}{max}1-h_{\mathrm{b}}(q\star p_1\star p_2),\hfill \end{array}$$ (27)

which can be simplified to (24). For the proof of the converse, see Appendix C. □

Figure 2 illustrates the error exponent versus the privacy parameter L for a fixed rate R. There is clearly a trade-off between ${\theta }_{ϵ}^{*}(R,L)$ and L. For a less stringent privacy requirement (large L), the error exponent ${\theta }_{ϵ}^{*}(R,L)$ increases.

Figure 2.

${\theta }_{ϵ}^{*}(R,L)$ versus L for $q=0.1$ and various values of R.

4.4. Euclidean Approximation

In this section, we propose Euclidean approximations [30,31] for the optimal error exponent of testing against independence scenario (Theorem 2) when $R\approx 0$ and $L\approx 0$. Consider the optimal error exponent as follows:

$$\begin{array}{c}\hfill {\theta }_{ϵ}^{*}(R,L)=\underset{\begin{array}{c}{P}_{U|\widehat{X}},P_{\widehat{X}|X}:\\ R\ge I(U;\widehat{X})\\ L\ge I(X;\widehat{X})\end{array}}{max}I(U;Y).\end{array}$$ (28)

Let $\mathbf{W}$ of dimension $\left|\mathcal{Y}\right|\times \left|\mathcal{X}\right|$, denote the transition matrix $P_{Y|X}$, which is itself induced by $P_X$ and the joint distribution $P_{XY}$. Now, consider the rate constraint as follows:

$$\begin{array}{cc}\hfill I(U;\widehat{X})& =\sum _{u\in \mathcal{U}}{P}_U\left(u\right)D\left(P_{\widehat{X}|U}(·|u)\parallel P_{\widehat{X}}\right)\le R.\hfill \end{array}$$ (29)

Assuming $R\approx 0$, we let $P_{\widehat{X}|U}(·|u)$ be a local perturbation from $P_{\widehat{X}}(·)$, where we have

$$P_{\widehat{X}|U}(·|u)=P_{\widehat{X}}(·)+{\psi }_u(·),$$ (30)

for a perturbation ${\psi }_u(·)$ satisfying

$$\begin{array}{c}\hfill \sum _{\widehat{x}\in \widehat{\mathcal{X}}}{\psi }_u\left(\widehat{x}\right)=0,\end{array}$$ (31)

in order to preserve the row stochasticity of $P_{\widehat{X}|U}$. Using a ${\chi }^2$-approximation [30], we can write:

$$\begin{array}{cc}\hfill D\left(P_{\widehat{X}|U}(·|u)\parallel P_{\widehat{X}}\right)& \approx \frac{1}{2}·loge·{∥{\mathbf{v}}_u∥}^2,\hfill \end{array}$$ (32)

where ${\mathbf{v}}_u$ denotes the length-$|\widehat{\mathcal{X}}|$ column vector of weighted perturbations whose $\widehat{x}$-th component is defined as:

$$\begin{array}{cc}\hfill v_u\left(\widehat{x}\right)& \triangleq \frac{1}{\sqrt{P_{\widehat{X}}\left(\widehat{x}\right)}}·{\psi }_u\left(\widehat{x}\right),\forall \widehat{x}\in \widehat{\mathcal{X}}.\hfill \end{array}$$ (33)

Using the above definition, the rate constraint in (29) can be written as:

$$\begin{array}{cc}\hfill \sum _{u\in \mathcal{U}}{P}_U\left(u\right){∥{\mathbf{v}}_u∥}^2& \le \frac{2R}{loge}.\hfill \end{array}$$ (34)

Similarly, consider the privacy constraint as the following:

$$\begin{array}{cc}\hfill I(X;\widehat{X})& =\sum _{\widehat{x}\in \widehat{\mathcal{X}}}{P}_{\widehat{X}}\left(\widehat{x}\right)D\left(P_{X|\widehat{X}}(·|\widehat{x})\parallel P_X\right)\le L.\hfill \end{array}$$ (35)

Assuming $L\approx 0$, we let $P_{X|\widehat{X}}(·|\widehat{x})$ be a local perturbation from $P_X(·)$ where

$$\begin{array}{c}\hfill P_{X|\widehat{X}}(·|\widehat{x})=P_X(·)+{\varphi }_{\widehat{x}}(·),\end{array}$$ (36)

for a perturbation ${\varphi }_{\widehat{x}}(·)$ that satifies:

$$\begin{array}{c}\hfill \sum _{x\in \mathcal{X}}{\varphi }_{\widehat{x}}\left(x\right)=0.\end{array}$$ (37)

Again, using a ${\chi }^2$-approximation, we obtain the following:

$$\begin{array}{c}\hfill D\left(P_{X|\widehat{X}}(·|\widehat{x})\parallel P_X\right)\approx \frac{1}{2}loge{∥{\mathbf{v}}_{\widehat{x}}∥}^2,\end{array}$$ (38)

where ${\mathbf{v}}_{\widehat{x}}$ is a length-$\left|\mathcal{X}\right|$ column vector and its x-th component is defined as follows:

$$\begin{array}{cc}\hfill v_{\widehat{x}}\left(x\right)& \triangleq \frac{1}{\sqrt{P_X\left(x\right)}}·{\varphi }_{\widehat{x}}\left(x\right),\forall x\in \mathcal{X}.\hfill \end{array}$$ (39)

Thus, the privacy constraint in (35) can be written as:

$$\begin{array}{cc}\hfill \sum _{\widehat{x}\in \widehat{\mathcal{X}}}{P}_{\widehat{X}}\left(\widehat{x}\right){∥{\mathbf{v}}_{\widehat{x}}∥}^2& \le \frac{2L}{loge}.\hfill \end{array}$$ (40)

For any $x\in \mathcal{X}$ and $u\in \mathcal{U}$, we define the following:

$$\begin{array}{}\mathrm{(41)}& \hfill {\mathsf{\Lambda }}_u\left(x\right)& \triangleq \sum _{\widehat{x}\in \widehat{\mathcal{X}}}{\psi }_u\left(\widehat{x}\right){\varphi }_{\widehat{x}}\left(x\right)\hfill \mathrm{(42)}& & =\sqrt{P_X\left(x\right)}\sum _{\widehat{x}\in \widehat{\mathcal{X}}}\sqrt{P_{\widehat{X}}\left(\widehat{x}\right)}{v}_u\left(\widehat{x}\right)v_{\widehat{x}}\left(x\right),\hfill \end{array}$$

and the corresponding length-$\left|\mathcal{X}\right|$ column vector ${\mathsf{\Lambda }}_u$ defined as follows:

$$\begin{array}{c}\hfill {\mathsf{\Lambda }}_u=\left[\sqrt{P_X}\right]{\mathbf{V}}_{\widehat{X}}\left[\sqrt{P_{\widehat{X}}}\right]{\mathbf{v}}_u,\end{array}$$ (43)

where $\left[\sqrt{P_X}\right]$ denotes a diagonal $\left|\mathcal{X}\right|\times \left|\mathcal{X}\right|$-matrix, so that its $(x,x)$-th element ($x\in \mathcal{X}$) is $\sqrt{P_X\left(x\right)}$, and $\left[\sqrt{P_{\widehat{X}}}\right]$ is defined similarly. Moreover, ${\mathbf{V}}_{\widehat{X}}$ refers to the $\left|\mathcal{X}\right|\times |\widehat{\mathcal{X}}|$-matrix defined as follows:

$$\begin{array}{c}\hfill {\mathbf{V}}_{\widehat{X}}\triangleq \left[\begin{array}{cccccc}{\mathbf{v}}_1& {\mathbf{v}}_2& \dots & {\mathbf{v}}_{\widehat{x}}& \dots & {\mathbf{v}}_{|\widehat{\mathcal{X}}|}\end{array}\right].\end{array}$$ (44)

Let ${\left[\sqrt{P_Y}\right]}^{-1}$ be the inverse of diagonal $\left|\mathcal{Y}\right|\times \left|\mathcal{Y}\right|$-matrix $\left[\sqrt{P_Y}\right]$. As shown in Appendix D, the optimization problem in (28) can be written as follows:

$$\begin{array}{c}\underset{\begin{array}{c}{\left\{{\mathbf{v}}_u\right\}}_{u\in \mathcal{U}},{\mathbf{V}}_{\widehat{X}}:\\ -\sqrt{P_{\widehat{X}}\left(\widehat{x}\right)}\le {\mathbf{v}}_u\left(\widehat{x}\right)\le \frac{1-P_{\widehat{X}}\left(\widehat{x}\right)}{\sqrt{P_{\widehat{X}}\left(\widehat{x}\right)}}\\ -\sqrt{P_X\left(x\right)}\le {\mathbf{v}}_{\widehat{x}}\left(x\right)\le \frac{1-P_X\left(x\right)}{\sqrt{P_X\left(x\right)}}\end{array}}{max}\frac{1}{2}loge\left[\sum _{u\in \mathcal{U}}{P}_U\left(u\right)·{∥{\left[\sqrt{P_Y}\right]}^{-1}\mathbf{W}\left[\sqrt{P_X}\right]{\mathbf{V}}_{\widehat{X}}\left[\sqrt{P_{\widehat{X}}}\right]{\mathbf{V}}_u∥}^2\right]\hfill \end{array}$$ (45)

$$\begin{array}{c}\mathrm{subject}\mathrm{to}:\sum _{u\in \mathcal{U}}{P}_U\left(u\right){∥{\mathbf{v}}_u∥}^2\le \frac{2R}{loge},\hfill \end{array}$$ (46)

$$\begin{array}{c}\sum _{\widehat{x}\in \widehat{\mathcal{X}}}{P}_{\widehat{X}}\left(\widehat{x}\right){∥{\mathbf{v}}_{\widehat{x}}∥}^2\le \frac{2L}{loge}.\hfill \end{array}$$ (47)

The following example specializes the above approximation to the binary case.

Example 1.

Consider the binary setup of Example 4.3 and the choice of auxiliary random variables in (26). Since the privacy mechanism is assumed to be a BSC, we have

$$\begin{array}{c}\hfill {\mathbf{P}}_X={\left[\frac{1}{2}\frac{1}{2}\right]}^T,{\mathbf{P}}_{\widehat{X}}={\left[\frac{1}{2}\frac{1}{2}\right]}^T,\end{array}$$ (48)

Now, we consider the vectors ${\mathbf{v}}_{u=0}$ and ${\mathbf{v}}_{u=1}$ defined as

$$\begin{array}{cc}\hfill {\mathbf{v}}_{u=0}& ={\left[\begin{array}{cc}\sqrt{2}{\xi }_1& -\sqrt{2}{\xi }_1\end{array}\right]}^T,\hfill \end{array}$$ (49)

$$\begin{array}{cc}\hfill {\mathbf{v}}_{u=1}& ={\left[\begin{array}{cc}-\sqrt{2}{\xi }_1& \sqrt{2}{\xi }_1\end{array}\right]}^T.\hfill \end{array}$$ (50)

for some positive ${\xi }_1$. This yields the following:

$$\begin{array}{cc}\hfill {\mathbf{P}}_{\widehat{X}|U=0}& ={\mathbf{P}}_{\widehat{X}}+{\left[{\xi }_1-{\xi }_1\right]}^T,\hfill \end{array}$$ (51)

$$\begin{array}{cc}\hfill {\mathbf{P}}_{\widehat{X}|U=1}& ={\mathbf{P}}_{\widehat{X}}+{\left[-{\xi }_1{\xi }_1\right]}^T\hfill \end{array}$$ (52)

We also choose the vectors ${\mathbf{v}}_{\widehat{x}=0}$ and ${\mathbf{v}}_{\widehat{x}=1}$ as follows:

$$\begin{array}{cc}\hfill {\mathbf{v}}_{\widehat{x}=0}& ={\left[\begin{array}{cc}\sqrt{2}{\xi }_2& -\sqrt{2}{\xi }_2\end{array}\right]}^T,\hfill \end{array}$$ (53)

$$\begin{array}{cc}\hfill {\mathbf{v}}_{\widehat{x}=1}& ={\left[\begin{array}{cc}-\sqrt{2}{\xi }_2& \sqrt{2}{\xi }_2\end{array}\right]}^T,\hfill \end{array}$$ (54)

which results in

$$\begin{array}{cc}\hfill {\mathbf{P}}_{X|\widehat{X}=0}& ={\mathbf{P}}_X+{\left[{\xi }_2-{\xi }_2\right]}^T,\hfill \end{array}$$ (55)

$$\begin{array}{cc}\hfill {\mathbf{P}}_{X|\widehat{X}=1}& ={\mathbf{P}}_X+{\left[-{\xi }_2{\xi }_2\right]}^T.\hfill \end{array}$$ (56)

Notice that the matrix $\mathbf{W}$ is given by

$$\begin{array}{c}\hfill \mathbf{W}=\left[\begin{array}{cc}1-q& q\\ q& 1-q\end{array}\right].\end{array}$$ (57)

Thus, the optimization problem in (45) and (47) reduces to the following:

$$\begin{array}{c}\underset{-\frac{1}{2}\le {\xi }_1,{\xi }_2\le \frac{1}{2}}{max}8loge{(1-2q)}^2|{\xi }_1{|}^2{\left|{\xi }_2\right|}^2\hfill \end{array}$$ (58)

$$\begin{array}{c}subjectto:4|{\xi }_1{|}^2\le \frac{2R}{loge}and4{\left|{\xi }_2\right|}^2\le \frac{2L}{loge}.\hfill \end{array}$$ (59)

Solving the above optimization yields

$$\begin{array}{cc}\hfill {\theta }_{ϵ}^{*}(R\approx 0,L\approx 0)& \approx \frac{2}{loge}{(1-2q)}^{2}RL.\hfill \end{array}$$ (60)

For some values of parameters, the approximation in (60) is compared to the error exponent of (24) in Figure 3. We observe that when $R=L\approx 0$, the approximation turns out to be excellent.

Figure 3.

${\theta }_{ϵ}^{*}(R\approx 0,L\approx 0)$ versus L for $q=0.1$ and $R=L$.

Remark 4.

The trade-off between the optimal error exponent and the privacy can again be verified from (60) in the case of $L\approx 0$ and $R\approx 0$. As L becomes larger (which corresponds to a less stringent privacy requirement), the error exponent also increases. For a fixed error exponent, a trade-off between R and L exists. An increase in R results in a decrease of L.

4.5. Gaussian Setup

In this section, we consider hypothesis testing against independence over a Gaussian example. Suppose that $X\sim \mathcal{N}(0,1)$ and under the null hypothesis $\mathcal{H}=0$, the sources X and Y are jointly Gaussian random variables distributed as $\mathcal{N}(0,{\mathbf{G}}_{XY})$, where ${\mathbf{G}}_{XY}$ is defined as the following:

$$\begin{array}{c}\hfill {\mathbf{G}}_{XY}\stackrel{\Delta }{=}\left[\begin{array}{cc}1\hfill & \rho \hfill \\ \rho \hfill & 1\hfill \end{array}\right],\end{array}$$ (61)

for some $0\le \rho \le 1$.

Under the alternative hypothesis $\mathcal{H}=1$, we assume that X and Y are independent Gaussian random variables, each distributed as $\mathcal{N}(0,1)$. Consider the privacy constraint as follows:

$$\begin{array}{cc}\hfill L& \ge I(X;\widehat{X})=h\left(X\right)-h\left(X\right|\widehat{X}).\hfill \end{array}$$ (62)

For a Gaussian source X, the conditional entropy $h\left(X\right|\widehat{X})$ is maximized for a jointly Gaussian $(X,\widehat{X})$. This choice minimizes the RHS of (62). Thus, without loss of optimality, we choose

$$\begin{array}{c}\hfill X=\widehat{X}+Z,Z\sim \mathcal{N}\left(0,2^{-2L}\right),\end{array}$$ (63)

where Z is independent of $\widehat{X}$. The following proposition states that it is optimal to choose U jointly Gaussian with $(X,\widehat{X},Y)$.

Proposition 2.

The optimal error exponent of the proposed Gaussian setup is given by

$$\begin{array}{c}\hfill {\theta }_{ϵ}^{*}(R,L)=\frac{1}{2}log\left(\frac{1}{1-{\rho }^2·(1-2^{-2R})·(1-2^{-2L})}\right).\end{array}$$ (64)

Proof. 

For the proof of achievability, we choose $\widehat{X}$ as in (63). Also, let

$$\begin{array}{c}\hfill \widehat{X}=U+\widehat{Z},\widehat{Z}\sim \mathcal{N}(0,{\beta }^2),\end{array}$$ (65)

for some ${\beta }^2\ge 0$, where $\widehat{Z}$ is independent of U. It can be shown that Theorem 2 remains valid when it is extended to the continuous alphabet [5]. For the details of the simplification and also the proof of converse, see Appendix E. □

Remark 5.

If $L=\infty$, the above proposition recovers the optimal error exponent of Rahman and Wagner [5] (Corollary 7) for testing against independence of Gaussian sources over a noiseless link of rate R.

5. Summary and Discussion

In this paper, distributed hypothesis testing with privacy constraints is considered. A coding scheme is proposed where the sensor decides on one of hypotheses and generates the randomized data based on its decision. The transmitter describes the randomized data over a noiseless link to the receiver. The privacy mechanism in this scheme is non-memoryless. The special case of testing against independence with a memoryless privacy mechanism is studied in detail. The optimal type-II error exponent of this case is established, together with a strong converse. A binary example is proposed where the trade-off between the privacy criterion and the error exponent is reported. Euclidean approximations are provided for the case in which the privacy level is high and and the communication rate is vanishingly small. The optimal type-II error exponent of a Gaussian setup is also established.

A future line of research is to study the second-order asymptotics of our model. The second-order analysis of distributed hypothesis testing without privacy constraints and with zero-rate communication was studied in [37]. In our proposed model, the trade-off between the privacy and type-II error exponent is observed, i.e., a less stringent privacy requirement yields a larger error exponent. The next step is to see whether the trade-off between privacy and error exponent affects the second-order term.

Another potential line for future research is to consider other metrics of privacy instead of the mutual information. A possible candidate is to use the maximal leakage [21,22,23] and to analyze the performance in tandem with distributed hypothesis testing problem.

Appendix A. Proof of Theorem 1

The analysis is based on the scheme described in Section 3.2.

Error Probability Analysis: We analyze the type-I and type-II error probabilities averaged over all random codebooks. By standard arguments [36] (p. 204), it can be shown that there exists at least one codebook that satisfies constraints on the error probabilities.

For the considered $\mu >0$ and the considered blocklength n, let ${\mathcal{P}}_{\mu }^n$ be the set of all joint types ${\pi }_{U\widehat{X}XY}$ over ${\mathcal{U}}^n\times {\widehat{\mathcal{X}}}^n\times {\mathcal{X}}^n\times {\mathcal{Y}}^n$ which satisfy the following constraints:

$$\begin{array}{cc}\hfill |{\pi }_X-P_X|& \le \mu /4,\hfill \end{array}$$ (A1)

$$\begin{array}{cc}\hfill |{\pi }_{U\widehat{X}}-P_{U\widehat{X}}|& \le \mu /2,\hfill \end{array}$$ (A2)

$$\begin{array}{cc}\hfill |{\pi }_{UY}-P_{UY}|& \le \mu .\hfill \end{array}$$ (A3)

First, we analyze the type-I error probability. For the case of $M\ne 0$, we define the following event:

$$\begin{array}{c}\hfill \mathcal{E}\triangleq \left\{(U^n\left(M\right),Y^n)\notin {\mathcal{T}}_{\mu }^n\left(P_{UY}\right)\right\}.\end{array}$$ (A4)

Thus, type-I error probability can be upper bounded as follows:

$$\begin{array}{}\mathrm{(A5)}& \hfill {\alpha }_n& \le Pr\left[{\widehat{X}}^n=0^n\mathrm{or}M=0\mathrm{or}\mathcal{E}|\mathcal{H}=0\right]\hfill \mathrm{(A6)}& & \le Pr\left[{\widehat{X}}^n=0^n|\mathcal{H}=0\right]+Pr\left[M=0|{\widehat{X}}^n\ne 0^n,\mathcal{H}=0\right]+Pr\left[\mathcal{E}|M\ne 0,{\widehat{X}}^n\ne 0^n,\mathcal{H}=0\right]\hfill \mathrm{(A7)}& & \le ϵ/3+Pr\left[M=0|{\widehat{X}}^n\ne 0^n,\mathcal{H}=0\right]+Pr\left[\mathcal{E}|M\ne 0,{\widehat{X}}^n\ne 0^n,\mathcal{H}=0\right]\hfill \mathrm{(A8)}& & \le ϵ/3+ϵ/3+Pr\left[\mathcal{E}|M\ne 0,{\widehat{X}}^n\ne 0^n,\mathcal{H}=0\right]\hfill \mathrm{(A9)}& & \le ϵ/3+ϵ/3+ϵ/3\hfill \mathrm{(A10)}& & =ϵ,\hfill \end{array}$$

where (A7) follows from AEP [36] (Theorem 3.1.1); (A8) follows from the covering lemma [34] (Lemma 3.3) and the rate constraint (15), (A10) follows from Markov lemma [34] (Lemma 12.1). In all justifications, n is taken to be sufficiently large.

Next, we analyze the type-II error probability. The acceptance region at the receiver is

$$\begin{array}{c}{\mathcal{A}}_n^{\mathrm{Rx}}=\bigcup _m\left\{\left({\widehat{x}}^n,x^n,y^n\right):{\widehat{x}}^n\ne 0^n,\left(u^n\left(m\right),{\widehat{x}}^n,x^n,y^n\right)\in {\mathcal{T}}_{\mu }^n\left(P_{U\widehat{X}XY}\right)\right\}.\hfill \end{array}$$ (A11)

The set ${\mathcal{A}}_n^{\mathrm{Rx}}$ is contained within the following acceptance region ${\overline{\mathcal{A}}}_n$:

$$\begin{array}{c}{\overline{\mathcal{A}}}_n=\bigcup _m\left\{\left({\widehat{x}}^n,x^n,y^n\right):{\widehat{x}}^n\ne 0^n,\left(u^n\left(m\right),{\widehat{x}}^n,x^n,y^n\right)\in \bigcup _{\pi \in {\mathcal{P}}_{\mu }^n}{\mathcal{T}}^n\left(\pi \right)\right\}.\hfill \end{array}$$ (A12)

Let ${\mathcal{F}}_m\triangleq \{\left(U^n\left(m\right),{\widehat{X}}^n,X^n,Y^n\right)\in {\mathcal{P}}_{\mu }^n\}$. Therefore, the average of type-II error probability over all codebooks is upper bounded as follows:

$$\begin{array}{}\mathrm{(A13)}& \hfill {\mathbb{E}}_{\mathcal{C}}\left[{\beta }_n\right]& \le Q_{\widehat{X}XY}^n\left({\overline{\mathcal{A}}}_n\right)\hfill \mathrm{(A14)}& & \le \sum _{m}Pr\left[{\widehat{X}}^n\ne 0^n,{\mathcal{F}}_m|\mathcal{H}=1\right]\hfill \mathrm{(A15)}& & \le \sum _{m}Pr\left[{\mathcal{F}}_m|{\widehat{X}}^n\ne 0^n,\mathcal{H}=1\right]\hfill \mathrm{(A16)}& & \le 2^{nR}·{(n+1)}^{\left|\mathcal{U}\right|·|\widehat{\mathcal{X}}|·|\mathcal{X}|·|\mathcal{Y}|}·\underset{{\pi }_{U\widehat{X}XY}\in {\mathcal{P}}_{\mu }^n}{max}{2}^{-nD({\pi }_{U\widehat{X}XY}\parallel P_U{P}_{\widehat{X}|X}{Q}_{XY})}\hfill \mathrm{(A17)}& & ={(n+1)}^{\left|\mathcal{U}\right|·|\widehat{\mathcal{X}}|·|\mathcal{X}|·|\mathcal{Y}|}·2^{-n{\tilde{\theta }}_{\mu }},\hfill \end{array}$$

where

$${\displaystyle {\tilde{\theta }}_{\mu }\triangleq \underset{{\pi }_{U\widehat{X}XY}\in {\mathcal{P}}_{\mu }^n}{min}D({\pi }_{U\widehat{X}XY}\parallel P_U{P}_{\widehat{X}|X}{Q}_{XY})-R,}$$ (A18)

and (A16) follows from the upper bound of Sanov’s theorem [36] (Theorem 11.4.1). Hence,

$$\begin{array}{}\mathrm{(A19)}& \hfill {\tilde{\theta }}_{\mu }& =\underset{{\pi }_{U\widehat{X}XY}\in {\mathcal{P}}_{\mu }^n}{min}D({\pi }_{U\widehat{X}XY}\parallel P_U{P}_{\widehat{X}|X}{Q}_{XY})-R\hfill \mathrm{(A20)}& & =\underset{{\pi }_{U\widehat{X}XY}\in {\mathcal{P}}_{\mu }^n}{min}D({\pi }_{U\widehat{X}XY}\parallel P_U{P}_{\widehat{X}|X}{Q}_{XY})-I(U;\widehat{X})-\mu \hfill \mathrm{(A21)}& & =\underset{{\pi }_{U\widehat{X}XY}\in {\mathcal{P}}_{\mu }^n}{min}D({\pi }_{U\widehat{X}XY}\parallel P_{U|\widehat{X}}{P}_{\widehat{X}|X}{Q}_{XY})+\delta \left(\mu \right),\hfill \end{array}$$

where $\delta \left(\mu \right)\to 0$ as $\mu \to 0$. Equality (A20) follows from the rate constraint in (15) and (A21) holds because $|{\pi }_{U\widehat{X}}-P_{U\widehat{X}}|<\mu /2$.

Privacy Analysis: We first analyze the privacy under $\mathcal{H}=0$. Notice that ${\widehat{X}}^n$ is not necessarily i.i.d. because according to the scheme in Section 3.2, ${\widehat{X}}^n$ is forced to be an all-zero sequence if the Randomizer decides that $X^n$ is not typical. However, conditioned on the event that $X^n\in {\mathcal{T}}_{\mu }^n\left(P_X\right)$, the sequence ${\widehat{X}}^n$ is i.i.d. according to the conditional pmf $P_{\widehat{X}|X}$. The privacy measure $T_n$ satisfies

$$\begin{array}{c}\hfill n{T}_n=I(X^n;{\widehat{X}}^n)=H\left({\widehat{X}}^n\right)-H\left({\widehat{X}}^n\right|X^n).\end{array}$$ (A22)

We now provide a lower bound on $H\left({\widehat{X}}^n\right|X^n)$ as follows

$$\begin{array}{cc}\hfill H\left({\widehat{X}}^n\right|X^n)& \ge \sum _{x^n\in {\mathcal{T}}_{\mu }^n\left(P_X\right)}{P}_X^n\left(x^n\right)H\left({\widehat{X}}^n\right|X^n=x^n)\hfill \end{array}$$ (A23)

For any $x^n\in {\mathcal{T}}_{\mu }^n\left(P_X\right)$ and for ${\mu }^{\prime }>\mu$, it holds that

$$\begin{array}{}\mathrm{(A24)}& \hfill H\left({\widehat{X}}^n\right|X^n=x^n)& =-\sum _{{\widehat{x}}^n\in {\widehat{\mathcal{X}}}^n}{P}_{\widehat{X}|X}^n\left({\widehat{x}}^n\right|x^n)log{P}_{\widehat{X}|X}^n\left({\widehat{x}}^n\right|x^n)\hfill \mathrm{(A25)}& & \ge -\sum _{{\widehat{x}}^n\in {\mathcal{T}}_{{\mu }^{\prime }}^n\left(P_{\widehat{X}|X}(·|x^n)\right)}{P}_{\widehat{X}|X}^n\left({\widehat{x}}^n\right|x^n)log{P}_{\widehat{X}|X}^n\left({\widehat{x}}^n\right|x^n)\hfill \mathrm{(A26)}& & \ge -\sum _{{\widehat{x}}^n\in {\mathcal{T}}_{{\mu }^{\prime }}^n\left(P_{\widehat{X}|X}(·|x^n)\right)}{P}_{\widehat{X}|X}^n\left({\widehat{x}}^n\right|x^n)log\left[2^{-n(1-{\mu }^{\prime })H\left(\widehat{X}\right|X)}\right]\hfill \mathrm{(A27)}& & \ge n{(1-{\mu }^{\prime })}^{2}H\left(\widehat{X}\right|X)\hfill \end{array}$$

where (A26) is true because for any ${\widehat{x}}^n\in {\mathcal{T}}_{{\mu }^{\prime }}^n\left(P_{\widehat{X}|X}(·|x^n)\right)$, it holds that $P_{\widehat{X}|X}^n\left({\widehat{x}}^n\right|x^n)\le 2^{-n(1-{\mu }^{\prime })H\left(\widehat{X}\right|X)}$, and (A27) follows because the conditional typicality lemma [34] (Chapter 2) implies that $P_{\widehat{X}|X}^n({\mathcal{T}}_{{\mu }^{\prime }}^n\left(P_{\widehat{X}|X}(·|x^n)\right|x^n)\ge 1-{\mu }^{\prime }$ for n sufficiently large.

Combining (A23) and (A27), we obtain

$$\begin{array}{}\mathrm{(A28)}& \hfill H\left({\widehat{X}}^n\right|X^n)& \ge n{(1-{\mu }^{\prime })}^{2}H\left(\widehat{X}\right|X)\sum _{x^n\in {\mathcal{T}}_{\mu }^n\left(P_X\right)}{P}_X^n\left(x^n\right)\hfill \mathrm{(A29)}& & \ge n{(1-{\mu }^{\prime })}^2(1-\mu )H\left(\widehat{X}\right|X),\hfill \end{array}$$

where (A29) follows because the AEP [36] (Theorem 3.1.1) implies that $P_X^n\left({\mathcal{T}}_{\mu }^n\left(P_X\right)\right)\ge 1-\mu$ for n sufficiently large.

Hence, we have

$$\begin{array}{}\mathrm{(A30)}& \hfill I(X^n;{\widehat{X}}^n)& =H\left({\widehat{X}}^n\right)-H\left({\widehat{X}}^n\right|X^n)\hfill \mathrm{(A31)}& & \le nH\left(\widehat{X}\right)-H\left({\widehat{X}}^n\right|X^n)\hfill \mathrm{(A32)}& & \le nH\left(\widehat{X}\right)-n(1-{\mu }^{\prime \prime })H\left(\widehat{X}\right|X)\hfill \mathrm{(A33)}& & =nI(X;\widehat{X})+n{\mu }^{\prime \prime }H\left(\widehat{X}\right|X)\hfill \mathrm{(A34)}& & \le nL+n{\mu }^{\prime \prime }·log\left|\widehat{\mathcal{X}}\right|\hfill \mathrm{(A35)}& & =nL+n\zeta ,\hfill \end{array}$$

where ${\mu }^{\prime \prime }\triangleq 1-{(1-{\mu }^{\prime })}^2(1-\mu )\ge 0$, and $\zeta \triangleq {\mu }^{\prime \prime }·log\left|\widehat{\mathcal{X}}\right|$.

Next, consider the privacy analysis under $\mathcal{H}=1$. Please note that when $P_X=Q_X$, the analysis is similar to that of $\mathcal{H}=0$. Thus, we assume that $P_X\ne Q_X$ in the following. From (A22), the privacy measure $T_n$ satisfies:

$$n{T}_n=I(X^n;{\widehat{X}}^n)\le H\left({\widehat{X}}^n\right).$$ (A36)

To upper bound $H\left({\widehat{X}}^n\right)$, we calculate the probability $P_{{\widehat{X}}^n}\left({\widehat{x}}^n\right)$ for ${\widehat{x}}^n=0^n$ as follows:

$$\begin{array}{}\mathrm{(A37)}& \hfill P_{{\widehat{X}}^n}\left(0^n\right)& =\sum _{x^n\in {\mathcal{T}}_{\mu }^n\left(P_X\right)}{P}_{\widehat{X}|X}^n\left(0^n\right|x^n)·Q_X^n\left(x^n\right)+\sum _{x^n\notin {\mathcal{T}}_{\mu }^n\left(P_X\right)}{P}_{\widehat{X}|X}^n\left(0^n\right|x^n)·Q_X^n\left(x^n\right)\hfill \mathrm{(A38)}& & =\sum _{x^n\notin {\mathcal{T}}_{\mu }^n\left(P_X\right)}{P}_{\widehat{X}|X}^n\left(0^n\right|x^n)·Q_X^n\left(x^n\right)\hfill \mathrm{(A39)}& & =\sum _{x^n\notin {\mathcal{T}}_{\mu }^n\left(P_X\right)}{Q}_X^n\left(x^n\right)\hfill \mathrm{(A40)}& & =1-Q_X^n\left({\mathcal{T}}_{\mu }^n\left(P_X\right)\right)\hfill \mathrm{(A41)}& & \ge 1-2^{-n(D(P_X\parallel Q_X)+\delta \left(\mu \right))}\triangleq 1-{\gamma }_n,\hfill \end{array}$$

where ${\gamma }_n\to 0$ exponentially fast as $n\to \infty$. Here, (A38) follows because if $x^n\in {\mathcal{T}}_{\mu }^n\left(P_X\right)$, then $P_{\widehat{X}|X}^n\left(0^n\right|x^n)=0$, (A39) follows because when $x^n\notin {\mathcal{T}}_{\mu }^n\left(P_X\right)$, then $P_{\widehat{X}|X}^n\left(0^n\right|x^n)=1$ and (A41) follows from Sanov’s theorem and the continuity of the relative entropy in its first argument [38] (Lemma 1.2.7).

Write $H\left({\widehat{X}}^n\right)$ as $H\left(P_{{\widehat{X}}^n}\right)$ and let $P_{0^n}$ be the distribution on ${\widehat{\mathcal{X}}}^n$ that places all its probability mass on $0^n\in {\widehat{\mathcal{X}}}^n$. Since $H\left(P_{0^n}\right)=0$, by the uniform continuity of entropy [38] (Lemma 1.2.7),

$$\begin{array}{c}\hfill H\left(P_{{\widehat{X}}^n}\right)\le 2|P_{{\widehat{X}}^n}-P_{0^n}|·log\frac{|\widehat{\mathcal{X}}{|}^n}{2|P_{{\widehat{X}}^n}-P_{0^n}|}.\end{array}$$ (A42)

Since ${\gamma }_n\to 0$ exponentially fast, the same holds true for $|P_{{\widehat{X}}^n}-P_{0^n}|$ and so by (A42), $H\left(P_{{\widehat{X}}^n}\right)=H\left({\widehat{X}}^n\right)\to 0$. Therefore, under $\mathcal{H}=1$, we have $T_n\to 0$ as $n\to \infty$.

Letting $n\to \infty$ and then letting $\mu ,{\mu }^{\prime },\gamma \to 0$, we obtain ${\tilde{\theta }}_{\mu }\to \theta$ and ${lim\; sup}_{n\to n}{T}_n\le L$, with $\theta$ given by the RHS of (11). This establishes the proof of Theorem 1.

Appendix B. Proof of Theorem 2

Achievability: The analysis is based on the scheme of Section 4.2. It follows similar steps as in [1]. Recall the definition of the event $\mathcal{E}$ in (A4). Consider the type-I error probability as follows:

$$\begin{array}{}\mathrm{(A43)}& \hfill {\alpha }_n& \le Pr\left[M=0\mathrm{or}\mathcal{E}|\mathcal{H}=0\right]\hfill \mathrm{(A44)}& & \le Pr\left[M=0|\mathcal{H}=0\right]+Pr\left[\mathcal{E}|M\ne 0,\mathcal{H}=0\right]\hfill \mathrm{(A45)}& & \le ϵ/2+ϵ/2\hfill \mathrm{(A46)}& & =ϵ,\hfill \end{array}$$

where (A46) follows from covering lemma [34] (Lemma 3.3) and the rate constraint in (15), and also the Markov lemma [34] (Lemma 12.1). Now, consider the type-II error probability as follows:

$$\begin{array}{}\mathrm{(A47)}& \hfill {\beta }_n& =Pr[\widehat{\mathcal{H}}=0|\mathcal{H}=1]\hfill \mathrm{(A48)}& & =Pr[\widehat{\mathcal{H}}=0,M\ne 0|\mathcal{H}=1]\hfill \mathrm{(A49)}& & \le Pr[\widehat{\mathcal{H}}=0|\mathcal{H}=1,M\ne 0]\hfill \mathrm{(A50)}& & =Pr[\widehat{\mathcal{H}}=0|\mathcal{H}=1,M=1],\hfill \end{array}$$

where the last equality follows from the symmetry of the code construction. Now, the average of type-II error probability over all codebooks satisfies:

$$\begin{array}{c}\hfill {\mathbb{E}}_{\mathcal{C}}\left[{\beta }_n\right]\le 2^{-n\left[I\right(U;Y)-\delta (\mu \left)\right]},\end{array}$$ (A51)

where $\delta \left(\mu \right)$ is a function that tends to zero as $\mu \to 0$. The privacy analysis is straightforward since the privacy mechanism is memoryless whence we have

$$\begin{array}{c}\hfill \frac{1}{n}I(X^n;{\widehat{X}}^n)=I(X;\widehat{X})=L+\zeta ,\end{array}$$ (A52)

where the last equality follows from the privacy constraint in (15). This concludes the proof of achievability.

Converse: Now, we prove the strong converse. It involves an extension of the $\eta$-image characterization technique [4,38]. The proof steps are given as follows. First, we find a truncated distribution $P_{{\underset{\_}{\widehat{X}}}^n}$ which is arbitrarily close to $P_{\widehat{X}}^n$ in terms of entropy. Then, we analyze the type-II error probability under a constrained type-I error probability. Finally, a single-letter characterization of the rate and privacy constraints is given.

• (1) Construction of a Truncated Distribution:

Since the privacy machanism is memoryless, we conclude that $(X^n,{\widehat{X}}^n,Y^n)$ is i.i.d. according to $P_{X\widehat{X}Y}\triangleq P_{\widehat{X}|X}{P}_{XY}$. For a given $P_{\widehat{X}Y}$, define $V^n\left(y^n\right|{\widehat{x}}^n)\triangleq P_{Y|\widehat{X}}^n\left(y^n\right|{\widehat{x}}^n)$ for all ${\widehat{x}}^n\in {\widehat{\mathcal{X}}}^n$ and $y^n\in {\mathcal{Y}}^n$. A set $B\subseteq {\mathcal{Y}}^n$ is an $\eta$-image of the set $A\subseteq {\widehat{\mathcal{X}}}^n$ over the channel $V^n$ if

$$V^n\left(B|{\widehat{x}}^n\right)\ge \eta ,\forall {\widehat{x}}^n\in A.$$ (A53)

The privacy mechanism is the same under both hypotheses, thus, we can define the acceptance region based on $({\widehat{x}}^n,y^n)$ as follows:

$$\begin{array}{c}\hfill {\mathcal{A}}_n\triangleq \left\{({\widehat{x}}^n,y^n):g^{\left(n\right)}(y^n,{\varphi }^{\left(n\right)}\left({\widehat{x}}^n\right))=0\right\}.\end{array}$$ (A54)

For any encoding function ${\varphi }^{\left(n\right)}$ and an acceptance region ${\mathcal{A}}_n\subseteq {\widehat{\mathcal{X}}}^n\times {\mathcal{Y}}^n$, let ${\tau }_n$ denote the cardinality of codebook and define the following sets:

$$\begin{array}{cc}\hfill C_i& \stackrel{\Delta }{=}\left\{{\widehat{x}}^n\in {\widehat{\mathcal{X}}}^n:{\varphi }^{\left(n\right)}\left({\widehat{x}}^n\right)=i\right\},\hfill \end{array}$$ (A55)

$$\begin{array}{cc}\hfill D_i& \stackrel{\Delta }{=}\left\{y^n\in {\mathcal{Y}}^n:g^{\left(n\right)}(y^n,i)=0\right\},1\le i\le {\tau }_n.\hfill \end{array}$$ (A56)

The acceptance region can be written as follows:

$$\begin{array}{c}\hfill {\mathcal{A}}_n=\bigcup _{i=1}^{{\tau }_n}\left(C_i\times D_i\right),\end{array}$$ (A57)

where $C_i\cap C_j=\varphi$ for all $i\ne j$. Define the set ${\mathcal{B}}_n\left(\eta \right)$ as follows:

$$\begin{array}{c}{\mathcal{B}}_n\left(\eta \right)\triangleq \left\{{\widehat{x}}^n:V^n\left(D_{{\varphi }^{\left(n\right)}\left({\widehat{x}}^n\right)}|{\widehat{x}}^n\right)\ge \eta \right\}.\hfill \end{array}$$ (A58)

Fix $ϵ\in [0,1)$ and notice that the type-I error probability is upper-bounded as

$$\begin{array}{cc}\hfill {\alpha }_n& =P_{\widehat{X}Y}^n\left({\mathcal{A}}_n^c\right)\le ϵ,\hfill \end{array}$$ (A59)

which we can write equivalently as

$$\begin{array}{}\mathrm{(A60)}& \hfill 1-ϵ& \le P_{\widehat{X}Y}^n\left({\mathcal{A}}_n\right)\hfill \mathrm{(A61)}& & =\sum _{{\widehat{x}}^n\in {\mathcal{B}}_n\left(\eta \right)}{P}_{\widehat{X}}^n\left({\widehat{x}}^n\right)V^n\left(D_{{\varphi }^{\left(n\right)}\left({\widehat{x}}^n\right)}|{\widehat{x}}^n\right)+\sum _{{\widehat{x}}^n\in {\mathcal{B}}_n^c\left(\eta \right)}{P}_{\widehat{X}}^n\left({\widehat{x}}^n\right)V^n\left(D_{{\varphi }^{\left(n\right)}\left({\widehat{x}}^n\right)}|{\widehat{x}}^n\right)\hfill \mathrm{(A62)}& & \le P_{\widehat{X}}^n\left({\mathcal{B}}_n\left(\eta \right)\right)+\eta \left(1-P_{\widehat{X}}^n\left({\mathcal{B}}_n\left(\eta \right)\right)\right),\hfill \end{array}$$

where the first term is because $V^n\left(D_{{\varphi }^{\left(n\right)}\left({\widehat{x}}^n\right)}|{\widehat{x}}^n\right)\le 1$; and the second term is because for any ${\widehat{x}}^n\in {\mathcal{B}}_n^c\left(\eta \right)$, we have $V^n\left(D_{{\varphi }^{\left(n\right)}\left({\widehat{x}}^n\right)}|{\widehat{x}}^n\right)<\eta$.

In what follows, let $\eta =\frac{1-ϵ}{2}$. Inequality (A62) implies

$$\begin{array}{c}\hfill P_{\widehat{X}}^n\left({\mathcal{B}}_n\left(\eta \right)\right)\ge \frac{1-ϵ}{1+ϵ}.\end{array}$$ (A63)

Let ${\mu }_n=n^{-1/3}$. For the typical set ${\mathcal{T}}_{{\mu }_n}^n\left(P_{\widehat{X}}\right)$, we have

$$\begin{array}{c}\hfill P_{\widehat{X}}^n\left({\mathcal{T}}_{{\mu }_n}^n\left(P_{\widehat{X}}\right)\right)\ge 1-\frac{|\widehat{\mathcal{X}}|}{2{\mu }_{n}n}.\end{array}$$ (A64)

Hence,

$$\begin{array}{}\mathrm{(A65)}& \hfill P_{\widehat{X}}^n\left({\mathcal{T}}_{{\mu }_n}^n\left(P_{\widehat{X}}\right)\cap {\mathcal{B}}_n\left(\eta \right)\right)& \ge P_{\widehat{X}}^n\left({\mathcal{T}}_{{\mu }_n}^n\left(P_{\widehat{X}}\right)\right)+P_{\widehat{X}}^n\left({\mathcal{B}}_n\left(\eta \right)\right)-1\hfill \mathrm{(A66)}& & \ge \frac{1-ϵ}{1+ϵ}-\frac{|\widehat{\mathcal{X}}|}{2{\mu }_{n}n}.\hfill \end{array}$$

For any $0<\delta <\frac{1-ϵ}{1+ϵ}$ and for sufficiently large n,

$$\begin{array}{cc}\hfill P_{\widehat{X}}^n\left({\mathcal{T}}_{{\mu }_n}^n\left(P_{\widehat{X}}\right)\cap {\mathcal{B}}_n\left(\eta \right)\right)& \ge \delta .\hfill \end{array}$$ (A67)

We can also write ${\mathcal{T}}_{{\mu }_n}^n\left(P_{\widehat{X}}\right)$ as

$$\begin{array}{c}\hfill {\mathcal{T}}_{{\mu }_n}^n\left(P_{\widehat{X}}\right)=\bigcup _{{\widehat{P}}_{\widehat{X}}:|{\widehat{P}}_{\widehat{X}}-P_{\widehat{X}}|\le {\mu }_n}{\mathcal{T}}^n\left({\widehat{P}}_{\widehat{X}}\right).\end{array}$$ (A68)

Combining the above equations, we get

$$\begin{array}{c}\hfill \sum _{{\widehat{P}}_{\widehat{X}}:|{\widehat{P}}_{\widehat{X}}-P_{\widehat{X}}|\le {\mu }_n}{P}_{\widehat{X}}^n\left({\mathcal{T}}^n\left({\widehat{P}}_{\widehat{X}}\right)\cap {\mathcal{B}}_n\left(\eta \right)\right)\ge \delta .\end{array}$$ (A69)

Let ${\tilde{P}}_{\widehat{X}}$ denote the type which maximizes the $P_{\widehat{X}}^n$-probability of the type class among all such types. As there exist at most ${(n+1)}^{|\widehat{\mathcal{X}}|}$ possible types, it holds that

$$\begin{array}{c}\hfill P_{\widehat{X}}^n\left({\mathcal{T}}^n\left({\tilde{P}}_{\widehat{X}}\right)\cap {\mathcal{B}}_n\left(\eta \right)\right)\ge \frac{\delta }{{(n+1)}^{|\widehat{\mathcal{X}}|}}.\end{array}$$ (A70)

Define the set ${\mathsf{\Psi }}_n\left(\eta \right)\triangleq {\mathcal{T}}^n\left({\tilde{P}}_{\widehat{X}}\right)\cap {\mathcal{B}}_n\left(\eta \right)$. We can write the probability in (A70) as

$$\begin{array}{}\mathrm{(A71)}& \hfill P_{\widehat{X}}^n\left({\mathcal{T}}^n\left({\tilde{P}}_{\widehat{X}}\right)\cap {\mathcal{B}}_n\left(\eta \right)\right)& =\sum _{{\widehat{x}}^n\in {\mathsf{\Psi }}_n\left(\eta \right)}{P}_{\widehat{X}}^n\left({\widehat{x}}^n\right)\hfill \mathrm{(A72)}& & =\sum _{{\widehat{x}}^n\in {\mathsf{\Psi }}_n\left(\eta \right)}{2}^{-n\left[D({\tilde{P}}_{\widehat{X}}\parallel P_{\widehat{X}})+H_{{\tilde{P}}_{\widehat{X}}}\left(\widehat{X}\right)\right]}\hfill \mathrm{(A73)}& & \le \sum _{{\widehat{x}}^n\in {\mathsf{\Psi }}_n\left(\eta \right)}{2}^{-n[H\left(\widehat{X}\right)-{\delta }_1]}\hfill \end{array}$$

where ${\delta }_1\to 0$ as $n\to \infty$ due to the fact that $D({\tilde{P}}_{\widehat{X}}\parallel P_{\widehat{X}})\ge 0$ and $|{\tilde{P}}_{\widehat{X}}-P_{\widehat{X}}|\le {\mu }_n$ so the entropies are also arbitrarily close. It then follows from (A70) and (A73) that

$$\begin{array}{c}\hfill \frac{1}{n}log\left|{\mathsf{\Psi }}_n\left(\eta \right)\right|\ge H\left(\widehat{X}\right)-{\delta }_2,\end{array}$$ (A74)

where ${\delta }_2\to 0$ as ${\mu }_n\to 0$.

The encoding function ${\varphi }^{\left(n\right)}$ partitions the set ${\mathsf{\Psi }}_n\left(\eta \right)$ into ${\tau }_n$ non-intersecting subsets ${\left\{S_i\right\}}_{i=1}^{{\tau }_n}$ such that ${\varphi }^{\left(n\right)}\left({\widehat{x}}^n\right)=i$ for any ${\widehat{x}}^n\in S_i$. Define the following distribution:

$$\begin{array}{cc}\hfill P_{{\underset{\_}{\widehat{X}}}^n}\left({\widehat{x}}^n\right)& \triangleq \frac{P_{\widehat{X}}^n\left({\widehat{x}}^n\right)·\mathbb{1}\left\{{\widehat{x}}^n\in {\mathsf{\Psi }}_n\left(\eta \right)\right\}}{P_{\widehat{X}}^n\left({\mathsf{\Psi }}_n\left(\eta \right)\right)}.\hfill \end{array}$$ (A75)

Please note that this distribution, henceforth denoted as $P^{\left(n\right)}$, corresponds to a uniform distribution over ${\mathsf{\Psi }}_n\left(\eta \right)$ because all sequences in ${\mathsf{\Psi }}_n\left(\eta \right)$ have the same type ${\tilde{P}}_{\widehat{X}}$, and the probability is uniform on a type class under any i.i.d. measure.

Finally, define the following truncated joint distribution:

$$\begin{array}{c}\hfill P_{\underset{\_}{M}{\underset{\_}{X}}^n{\underset{\_}{\widehat{X}}}^n{\underset{\_}{Y}}^n}(m,x^n,{\widehat{x}}^n,y^n)\triangleq \mathbb{1}\left\{{\varphi }^{\left(n\right)}\left(x^n\right)=m\right\}{P}_{X|\widehat{X}}^n\left(x^n\right|{\widehat{x}}^n)P_{{\underset{\_}{\widehat{X}}}^n}\left({\widehat{x}}^n\right)P_{Y|X}^n\left(y^n\right|x^n).\end{array}$$ (A76)

• (2) Analysis of Type-II Error Exponent:

The proof of the upper bound on the error exponent relies on the following Lemma A1. For a set $A\subseteq {\widehat{\mathcal{X}}}^n$, let $\mathcal{B}(A,\eta )$ denote the collection of all $\eta$-images of A, define $Q_{X\widehat{X}Y}\triangleq P_{\widehat{X}|X}{Q}_{XY}$ and

$$\begin{array}{c}\hfill {\kappa }_{V^n}(A,Q_{\widehat{X}Y},\eta )\triangleq \frac{{min}_{B\in \mathcal{B}(A,\eta )}{Q}_{\widehat{X}Y}^n(A\times B)}{P_{\widehat{X}}^n\left(A\right)}.\end{array}$$ (A77)

This quantity is a generalization of the minimum cardinality of the $\eta$-images in [38] and is closely related to the minimum type-II error probability associated with the set A.

For the testing against independence setup, $Q_{\widehat{X}Y}=P_{\widehat{X}}·P_Y$, and thus

$$\begin{array}{c}\hfill \frac{Q_{\widehat{X}Y}^n(A\times B)}{P_{\widehat{X}}^n\left(A\right)}=\frac{P_{\widehat{X}}^n\left(A\right)P_Y^n\left(B\right)}{P_{\widehat{X}}^n\left(A\right)}=P_Y^n\left(B\right),\end{array}$$ (A78)

and ${\kappa }_{V^n}(A,Q_{\widehat{X}Y},\eta )$ is simply written as ${\kappa }_{V^n}(A,\eta )$ and is given by

$$\begin{array}{c}\hfill {\kappa }_{V^n}(A,\eta )\triangleq \underset{B\in \mathcal{B}(A,\eta )}{min}{P}_Y^n\left(B\right).\end{array}$$ (A79)

Lemma A1

(Lemma 3 in [4]). For any set $A\subseteq {\widehat{\mathcal{X}}}^n$, consider a distribution $P_A^{\left(n\right)}$ over A and let $P_A^{\left(n\right)}{V}^n$ be its corresponding output distribution induced by the channel $V^n$, i.e.,

$$P_A^{\left(n\right)}{V}^n\left(y^n\right)\triangleq \sum _{{\widehat{x}}^n\in A}{P}_A^{\left(n\right)}\left({\widehat{x}}^n\right)V^n\left(y^n|{\widehat{x}}^n\right).$$ (A80)

Then, for every ${\delta }^{\prime }>0$, $0<\eta <1$, we have

$$\begin{array}{c}\hfill {\kappa }_{V^n}(A,\eta )\ge 2^{-D(P_A^{\left(n\right)}{V}^n\parallel P_Y^n)-n{\delta }^{\prime }}\end{array}$$ (A81)

for sufficiently large n.

Let $P_i^{\left(n\right)}{V}^n$ be the distribution of the random variable ${\underset{\_}{Y}}^n$ given $\underset{\_}{M}=i$. The type-II error probability can be lower-bounded as:

$$\begin{array}{}\mathrm{(A82)}& \hfill {\beta }_n& \ge \sum _{{\widehat{x}}^n\in {\mathsf{\Psi }}_n\left(\eta \right)}{P}_{\widehat{X}}^n\left({\widehat{x}}^n\right)·P_Y^n\left(D_{{\varphi }^{\left(n\right)}\left({\widehat{x}}^n\right)}\right)\hfill \mathrm{(A83)}& & =\sum _{i=1}^{{\tau }_n}{P}_{\widehat{X}}^n\left(S_i\right)·P_Y^n\left(D_i\right)\hfill \mathrm{(A84)}& & \ge \sum _{i=1}^{{\tau }_n}{P}_{\widehat{X}}^n\left(S_i\right)·{\kappa }_{V^n}(S_i,\eta )\hfill \mathrm{(A85)}& & =P_{\widehat{X}}^n\left({\mathsf{\Psi }}_n\left(\eta \right)\right)·\sum _{i=1}^{{\tau }_n}{P}^{\left(n\right)}\left(S_i\right)·{\kappa }_{V^n}(S_i,\eta )\hfill \mathrm{(A86)}& & \ge 2^{-n{\delta }^{\prime }}·P_{\widehat{X}}^n\left({\mathsf{\Psi }}_n\left(\eta \right)\right)·\sum _{i=1}^{{\tau }_n}{P}^{\left(n\right)}\left(S_i\right)·2^{-D\left(P_i^{\left(n\right)}{V}^n\parallel P_Y^n\right)}\hfill \mathrm{(A87)}& & \ge 2^{-n{\delta }^{\prime }}·P_{\widehat{X}}^n\left({\mathsf{\Psi }}_n\left(\eta \right)\right)·2^{-{\sum }_{i=1}^{{\tau }_n}{P}^{\left(n\right)}\left(S_i\right)·D\left(P_i^{\left(n\right)}{V}^n\parallel P_Y^n\right)}\hfill \mathrm{(A88)}& & \ge \frac{2^{-n{\delta }^{\prime }}\delta }{{(n+1)}^{|\widehat{\mathcal{X}}|}}·2^{-{\sum }_{i=1}^{{\tau }_n}{P}^{\left(n\right)}\left(S_i\right)·D\left(P_i^{\left(n\right)}{V}^n\parallel P_Y^n\right)},\hfill \end{array}$$

where (A84) follows from the definition of ${\kappa }_{V^n}(S_i,\eta )$, (A86) follows because Lemma A1 implies that for any distribution $P_i^{\left(n\right)}$ over the set $S_i$ it holds that ${\kappa }_{V^n}(S_i,\eta )\ge 2^{-D\left(P_i^{\left(n\right)}{V}^n\parallel P_Y^n\right)-n{\delta }^{\prime }}$, (A87) follows because of the convexity of the function $t\mapsto 2^t$, and (A88) follows by (A70) and the fact that $Pr\left(A\right)\ge Pr(A\cap B)$. Hence,

$$\begin{array}{c}\hfill -\frac{1}{n}log{\beta }_n-{\delta }^{\prime \prime }\le \frac{1}{n}\sum _{i=1}^{{\tau }_n}{P}^{\left(n\right)}\left(S_i\right)·D\left(P_i^{\left(n\right)}{V}^n\parallel P_Y^n\right),\end{array}$$ (A89)

where ${\delta }^{\prime \prime }\triangleq {\delta }^{\prime }-\frac{1}{n}log\frac{\delta }{{(n+1)}^{|\widehat{\mathcal{X}}|}}$.

• (3) Single-letterization Steps and Analyses of Rate and Privacy Constraints:

In the following, we proceed to provide a single-letter characterization of the upper bound in (A89). Considering the fact that $P^{\left(n\right)}\left(S_i\right)=P_{\underset{\_}{M}}\left(i\right)$, the right-hand-side of (A89) can be upper-bounded as follows:

$$\begin{array}{}\mathrm{(A90)}& \hfill \frac{1}{n}\sum _{i=1}^{{\tau }_n}{P}^{\left(n\right)}\left(S_i\right)·D(P_i^{\left(n\right)}{V}^n\parallel P_Y^n)& =\frac{1}{n}\sum _{i=1}^{{\tau }_n}\sum _{y^n\in {\mathcal{Y}}^n}{P}_{\underset{\_}{M}{\underset{\_}{Y}}^n}(i,y^n)log\frac{P_{{\underset{\_}{Y}}^n|\underset{\_}{M}}\left(y^n\right|i)}{P_Y^n\left(y^n\right)}\hfill \mathrm{(A91)}& & =-\frac{1}{n}H\left({\underset{\_}{Y}}^n\right|\underset{\_}{M})-\frac{1}{n}\sum _{y^n\in {\mathcal{Y}}^n}{P}_{{\underset{\_}{Y}}^n}\left(y^n\right)log{P}_Y^n\left(y^n\right)\hfill \mathrm{(A92)}& & =-\frac{1}{n}H\left({\underset{\_}{Y}}^n\right|\underset{\_}{M})-\frac{1}{n}\sum _{y^n\in {\mathcal{Y}}^n}{P}_{{\underset{\_}{Y}}^n}\left(y^n\right)\sum _{t=1}^{n}log{P}_Y\left(y_t\right)\hfill \mathrm{(A93)}& & =-\frac{1}{n}H\left({\underset{\_}{Y}}^n\right|\underset{\_}{M})-\frac{1}{n}\sum _{t=1}^n\sum _{\begin{array}{c}{y}^n\in {\mathcal{Y}}^n\end{array}}{P}_{{\underset{\_}{Y}}^n}\left(y^n\right)log{P}_Y\left(y_t\right)\hfill \mathrm{(A94)}& & =-\frac{1}{n}H\left({\underset{\_}{Y}}^n\right|\underset{\_}{M})-\frac{1}{n}\sum _{t=1}^n\sum _{y_t\in \mathcal{Y}}{P}_{{\underset{\_}{Y}}_t}\left(y_t\right)log{P}_Y\left(y_t\right)\hfill \mathrm{(A95)}& & =-\frac{1}{n}H\left({\underset{\_}{Y}}^n\right|\underset{\_}{M})+\frac{1}{n}\sum _{t=1}^n\left[H\left({\underset{\_}{Y}}_t\right)+D(P_{{\underset{\_}{Y}}_t}\parallel P_Y)\right]\hfill \mathrm{(A96)}& & =\frac{1}{n}\sum _{t=1}^n\left[H\left({\underset{\_}{Y}}_t\right)-H\left({\underset{\_}{Y}}_t\right|\underset{\_}{M},{\underset{\_}{Y}}^{t-1})+D(P_{{\underset{\_}{Y}}_t}\parallel P_Y)\right]\hfill \mathrm{(A97)}& & \le \frac{1}{n}\sum _{t=1}^{n}I(\underset{\_}{M},{\underset{\_}{X}}^{t-1},{\underset{\_}{\widehat{X}}}^{t-1};{\underset{\_}{Y}}_t)+\frac{1}{n}\sum _{t=1}^{n}D(P_{{\underset{\_}{Y}}_t}\parallel P_Y)\hfill \mathrm{(A98)}& & =\frac{1}{n}\sum _{t=1}^{n}I({\underset{\_}{U}}_t;{\underset{\_}{Y}}_t)+\frac{1}{n}\sum _{t=1}^{n}D(P_{{\underset{\_}{Y}}_t}\parallel P_Y)\hfill \mathrm{(A99)}& & =I(\underset{\_}{U};\underset{\_}{Y})+D(P_{\underset{\_}{Y}}\parallel P_Y).\hfill \end{array}$$

Here, (A96)–(A99) are justified in the following:

• (A96) follows by the chain rule;

• (A97) follows from the Markov chain ${\underset{\_}{Y}}^{t-1}\to (\underset{\_}{M},{\underset{\_}{X}}^{t-1},{\underset{\_}{\widehat{X}}}^{t-1})\to {\underset{\_}{Y}}_t$;

• (A98) follows from the definition

  $$\begin{array}{c}\hfill {\underset{\_}{U}}_t\triangleq (\underset{\_}{M},{\underset{\_}{X}}^{t-1},{\underset{\_}{\widehat{X}}}^{t-1});\end{array}$$ (A100)
• (A99) follows by defining a time-sharing random variable T over $\{1,\dots ,n\}$ and the following

  $$\begin{array}{c}\hfill \underset{\_}{U}\stackrel{\Delta }{=}({\underset{\_}{U}}_T,T),\underset{\_}{Y}\stackrel{\Delta }{=}{\underset{\_}{Y}}_T.\end{array}$$ (A101)

This leads to the following upper-bound on the type-II error exponent:

$$\begin{array}{c}\hfill -\frac{1}{n}log{\beta }_n\le I(\underset{\_}{U};\underset{\_}{Y})+D(P_{\underset{\_}{Y}}\parallel P_Y)+{\delta }^{\prime \prime }.\end{array}$$ (A102)

Next, the rate constraint satisfies the following:

$$\begin{array}{}\mathrm{(A103)}& \hfill nR& \ge H\left(\underset{\_}{M}\right)\hfill \mathrm{(A104)}& & \ge I(\underset{\_}{M};{\underset{\_}{\widehat{X}}}^n)\hfill \mathrm{(A105)}& & =H\left({\underset{\_}{\widehat{X}}}^n\right)-H\left({\underset{\_}{\widehat{X}}}^n\right|\underset{\_}{M})\hfill \mathrm{(A106)}& & =log\left|{\mathsf{\Psi }}_n\left(\eta \right)\right|-H\left({\underset{\_}{\widehat{X}}}^n\right|\underset{\_}{M})\hfill \mathrm{(A107)}& & \ge n(H\left(\widehat{X}\right)-{\delta }_2)-H\left({\underset{\_}{\widehat{X}}}^n\right|\underset{\_}{M})\hfill \mathrm{(A108)}& & =nH\left(\widehat{X}\right)-\sum _{t=1}^{n}H\left({\underset{\_}{\widehat{X}}}_t\right|{\underset{\_}{\widehat{X}}}^{t-1},\underset{\_}{M})-n{\delta }_2\hfill \mathrm{(A109)}& & =nH\left(\widehat{X}\right)-\sum _{t=1}^{n}H\left({\underset{\_}{\widehat{X}}}_t\right|{\underset{\_}{U}}_t)-n{\delta }_2\hfill \mathrm{(A110)}& & =nH\left(\widehat{X}\right)-nH\left(\underset{\_}{\widehat{X}}\right|\underset{\_}{U})-n{\delta }_2\hfill \end{array}$$

where (A106) follows because the distribution $P_{{\underset{\_}{\widehat{X}}}^n}$ is uniform over the set ${\mathsf{\Psi }}_n\left(\eta \right)$; (A107) follows from (A74); (A109) follows from the definition in (A100); (A110) follows by defining $\underset{\_}{\widehat{X}}\triangleq {\underset{\_}{\widehat{X}}}_T$.

Finally, the privacy measure satisfies the following:

$$\begin{array}{}\mathrm{(A111)}& \hfill nL& \ge I(X^n;{\widehat{X}}^n)\hfill \mathrm{(A112)}& & \ge I({\underset{\_}{X}}^n;{\underset{\_}{\widehat{X}}}^n)\hfill \mathrm{(A113)}& & =H\left({\underset{\_}{\widehat{X}}}^n\right)-H\left({\underset{\_}{\widehat{X}}}^n\right|{\underset{\_}{X}}^n)\hfill \mathrm{(A114)}& & =log\left|{\mathsf{\Psi }}_n\left(\eta \right)\right|-H\left({\underset{\_}{\widehat{X}}}^n\right|{\underset{\_}{X}}^n)\hfill \mathrm{(A115)}& & =n\left(H\left(\widehat{X}\right)-{\delta }_2\right)-H\left({\underset{\_}{\widehat{X}}}^n\right|{\underset{\_}{X}}^n)\hfill \mathrm{(A116)}& & =n\left(H\left(\widehat{X}\right)-{\delta }_2\right)-\sum _{t=1}^{n}H\left({\underset{\_}{\widehat{X}}}_t\right|{\underset{\_}{\widehat{X}}}^{t-1},{\underset{\_}{X}}^n)\hfill \mathrm{(A117)}& & \ge n\left(H\left(\widehat{X}\right)-{\delta }_2\right)-\sum _{t=1}^{n}H\left({\underset{\_}{\widehat{X}}}_t\right|{\underset{\_}{X}}_t)\hfill \mathrm{(A118)}& & =nH\left(\widehat{X}\right)-nH\left(\underset{\_}{\widehat{X}}\right|\underset{\_}{X})-n{\delta }_2,\hfill \end{array}$$

where (A112) follows because $({\underset{\_}{X}}^n,{\underset{\_}{\widehat{X}}}^n)$ are functions of $(X^n,{\widehat{X}}^n)$ and from data processing inequality, (A114) follows because $P_{{\underset{\_}{\widehat{X}}}^n}$ is uniform over the set ${\mathsf{\Psi }}_n\left(\eta \right)$ (see definition in (A75)), (A115) follows from (A74), and (A118) follows by defining $\underset{\_}{X}\triangleq ({\underset{\_}{X}}_T,T)$, $\underset{\_}{\widehat{X}}\triangleq ({\underset{\_}{\widehat{X}}}_T,T)$ and choosing T uniformly over $\{1,\dots ,n\}$.

Since ${\mathsf{\Psi }}_n\left(\eta \right)\subseteq {\mathcal{T}}^n\left({\tilde{P}}_{\widehat{X}}\right)$, for any $\widehat{x}\in \widehat{\mathcal{X}}$,

$$\begin{array}{}\mathrm{(A119)}& \hfill P_{\underset{\_}{\widehat{X}}}\left(\widehat{x}\right)& =\frac{1}{n}\sum _{t=1}^n{P}_{{\underset{\_}{\widehat{X}}}_t}\left(\widehat{x}\right)\hfill \mathrm{(A120)}& & =\sum _{{\widehat{x}}^n\in {\mathsf{\Psi }}_n\left(\eta \right)}\frac{N\left(\widehat{x}|{\widehat{x}}^n\right)}{n·|{\mathsf{\Psi }}_n\left(\eta \right)|}\hfill \mathrm{(A121)}& & ={\tilde{P}}_{\widehat{X}}\left(\widehat{x}\right).\hfill \end{array}$$

Recall that $|{\tilde{P}}_{\widehat{X}}-P_{\widehat{X}}|\le {\mu }_n$ with ${\mu }_n=n^{-1/3}$. Hence, from (A121), it holds that $|P_{\underset{\_}{\widehat{X}}}-P_{\widehat{X}}|\le {\mu }_n$. By the definitions of $\underset{\_}{\widehat{X}}$, $\underset{\_}{X}$ and $\underset{\_}{Y}$, we have $P_{\widehat{X}|X}=P_{\underset{\_}{\widehat{X}}|\underset{\_}{X}}$ and $P_{Y|X}=P_{\underset{\_}{Y}|\underset{\_}{X}}=V$. The random variable U is chosen over the same alphabet as $\underset{\_}{U}$ and such that $P_{U|\widehat{X}}=P_{\underset{\_}{U}|\underset{\_}{\widehat{X}}}$.

Since $P_Y\left(y\right)>0$ for all $y\in \mathcal{Y}$, letting $n\to \infty$ and ${\mu }_n\to 0$ and the uniform continuity of the involved information-theoretic quantities yields the following upper bound on the optimal error exponent:

$$\begin{array}{c}\hfill {\theta }_{ϵ}^{*}(R,L)\le I(U;Y),\end{array}$$ (A122)

subject to the rate constraint:

$$\begin{array}{c}\hfill R\ge I(U;\widehat{X}),\end{array}$$ (A123)

and the privacy constraint:

$$\begin{array}{ccc}\hfill L& \ge \hfill & \hfill I(X;\widehat{X}).\end{array}$$ (A124)

This concludes the proof of converse.

Appendix C. Proof of Converse of Proposition 1

We simplify Theorem 2 for the proposed binary setup. As discussed in Section 4.3, from the fact that $|\widehat{\mathcal{X}}|=2$ and the symmetry of the source X on its alphabet, without loss of optimality, we can choose $P_{\widehat{X}|X}$ to be a BSC. First, consider the rate constraint:

$$\begin{array}{}\mathrm{(A125)}& \hfill R& \ge I(U;\widehat{X})\hfill \mathrm{(A126)}& & =H\left(\widehat{X}\right)-H\left(\widehat{X}\right|U)\hfill \mathrm{(A127)}& & =1-H\left(\widehat{X}\right|U),\hfill \end{array}$$

which can be equivalently written as the following:

$$\begin{array}{cc}\hfill H\left(\widehat{X}\right|U)& \ge 1-R.\hfill \end{array}$$ (A128)

Also, the privacy criterion can be simplified as follows:

$$\begin{array}{}\mathrm{(A129)}& \hfill L& \ge I(\widehat{X};X)\hfill \mathrm{(A130)}& & =H\left(\widehat{X}\right)-H\left(\widehat{X}\right|X)\hfill \mathrm{(A131)}& & =1-H\left(\widehat{X}\right|X)\hfill \mathrm{(A132)}& & =1-H\left(\widehat{Z}\right),\hfill \end{array}$$

which can be equivalently written as

$$\begin{array}{c}\hfill H\left(\widehat{Z}\right)\ge 1-L.\end{array}$$ (A133)

Now, consider the error exponent $\theta$ as follows:

$$\begin{array}{}\mathrm{(A134)}& \hfill \theta & \le I(U;Y)\hfill \mathrm{(A135)}& & =H\left(Y\right)-H\left(Y\right|U)\hfill \mathrm{(A136)}& & =H\left(Y\right)-H(X\oplus N|U)\hfill \mathrm{(A137)}& & =H\left(Y\right)-H(\widehat{X}\oplus \widehat{Z}\oplus N|U)\hfill \mathrm{(A138)}& & \le H\left(Y\right)-h_{\mathrm{b}}\left(h_{\mathrm{b}}^{-1}\left(H\left(\widehat{X}\right|U)\right)\star h_{\mathrm{b}}^{-1}(1-L)\star q\right)\hfill \mathrm{(A139)}& & \le H\left(Y\right)-h_{\mathrm{b}}\left(h_{\mathrm{b}}^{-1}(1-R)\star h_{\mathrm{b}}^{-1}(1-L)\star q\right),\hfill \end{array}$$

where (A138) follows from Mrs. Gerber’s lemma [39] (Theorem 1) and the fact that $(\widehat{Z},N)$ is independent of U and also from (A133); (A139) follows from (A128). This concludes the proof of the proposition.

Appendix D. Euclidean Approximation of Testing agianst Independence

We analyze the Euclidean approximation with the parameters, $\mathbf{W}$, ${\psi }_u\left(\widehat{x}\right)$, ${\varphi }_{\widehat{x}}\left(x\right)$ and ${\mathsf{\Lambda }}_u\left(x\right)$ defined in Section 4.4. Notice that since $U\to \widehat{X}\to X\to Y$ forms a Markov chain, it holds that, for any $u\in \mathcal{U}$,

$$\begin{array}{c}\hfill {\mathbf{P}}_{Y|U=u}=\mathbf{W}{\mathbf{P}}_{X|U=u}.\end{array}$$ (A140)

Now, consider the following chain of equalities for any $x\in \mathcal{X}$:

$$\begin{array}{}\mathrm{(A141)}& \hfill P_{X|U}\left(x\right|u)& =\sum _{\widehat{x}\in \widehat{\mathcal{X}}}{P}_{X\widehat{X}|U}(x,\widehat{x}|u)\hfill \mathrm{(A142)}& & =\sum _{\widehat{x}\in \widehat{\mathcal{X}}}{P}_{\widehat{X}|U}\left(\widehat{x}\right|u)P_{X|\widehat{X},U}\left(x\right|\widehat{x},u)\hfill \mathrm{(A143)}& & =\sum _{\widehat{x}\in \widehat{\mathcal{X}}}{P}_{\widehat{X}|U}\left(\widehat{x}\right|u)P_{X|\widehat{X}}\left(x\right|\widehat{x})\hfill \mathrm{(A144)}& & =\sum _{\widehat{x}\in \widehat{\mathcal{X}}}\left(P_{\widehat{X}}\left(\widehat{x}\right)+{\psi }_u\left(\widehat{x}\right)\right)\left(P_X\left(x\right)+{\varphi }_{\widehat{x}}\left(x\right)\right)\hfill \mathrm{(A145)}& & =P_X\left(x\right)+\sum _{\widehat{x}\in \widehat{\mathcal{X}}}{\psi }_u\left(\widehat{x}\right){\varphi }_{\widehat{x}}\left(x\right)+\sum _{\widehat{x}\in \widehat{\mathcal{X}}}{P}_{\widehat{X}}\left(\widehat{x}\right){\varphi }_{\widehat{x}}\left(x\right)+P_X\left(x\right)\sum _{\widehat{x}\in \widehat{\mathcal{X}}}{\psi }_u\left(\widehat{x}\right)\hfill \mathrm{(A146)}& & =P_X\left(x\right)+\sum _{\widehat{x}\in \widehat{\mathcal{X}}}{\psi }_u\left(\widehat{x}\right){\varphi }_{\widehat{x}}\left(x\right),\hfill \end{array}$$

where (A143)—(A146) are justified in the following:

• (A143) follows from the Markov chain $U\to \widehat{X}\to X$ where given $\widehat{X}$, U and X are independent;

• (A144) follows from (30) and (36);

• (A146) follows from (31) and also from (36) which yields the following:

  $$\begin{array}{c}\hfill \sum _{\widehat{x}\in \widehat{\mathcal{X}}}{P}_{\widehat{X}}\left(\widehat{x}\right)·{\varphi }_{\widehat{x}}\left(x\right)=0.\end{array}$$ (A147)

With the definition of ${\mathsf{\Lambda }}_u\left(x\right)$ in (42), we can write

$$\begin{array}{cc}\hfill P_{X|U}\left(x\right|u)& =P_X\left(x\right)+{\mathsf{\Lambda }}_u\left(x\right),\forall x\in \mathcal{X},u\in \mathcal{U}.\hfill \end{array}$$ (A148)

Thus, we get

$$\begin{array}{}\mathrm{(A149)}& \hfill {\mathbf{P}}_{Y|U=u}& =\mathbf{W}{\mathbf{P}}_X+\mathbf{W}{\mathsf{\Lambda }}_u\hfill \mathrm{(A150)}& & ={\mathbf{P}}_Y+\mathbf{W}{\mathsf{\Lambda }}_u.\hfill \end{array}$$

Applying the ${\chi }^2$-approximation and using (A150), we can rewrite $I(U;Y)$ as follows:

$$\begin{array}{cc}\hfill I(U;Y)& \approx \frac{1}{2}loge\sum _{u\in \mathcal{U}}{P}_U\left(u\right){∥{\left[\sqrt{P_Y}\right]}^{-1}\mathbf{W}{\mathsf{\Lambda }}_u∥}^2\hfill \end{array}$$ (A151)

The above approximation with the definition of the vector ${\mathsf{\Lambda }}_u$ in (43) yields the optimization problem in (45).

Appendix E. Proof of Proposition 2

Achievability: We specialize the achievable scheme of Theorem 2 to the proposed Gaussian setup. We choose the auxiliary random variables as in (63) and (65). Notice that from the Markov chain $U\to \widehat{X}\to X\to Y$ and also the Gaussian choice of $\widehat{X}$ in (63) which was discussed in Section 4.5, we can write $Y=\rho \widehat{X}+F$ where $F\sim \mathcal{N}\left(0,1-{\rho }^2·(1-2^{-2L})\right)$ is independent of $\widehat{X}$. These choices of auxiliary random variables lead to the following rate constraint:

$$\begin{array}{c}\hfill R\ge \frac{1}{2}log\left(\frac{1-2^{-2L}}{{\beta }^2}\right),\end{array}$$ (A152)

which can be equivalently written as:

$$\begin{array}{c}\hfill 2^{-2R}·\left(1-2^{-2L}\right)\le {\beta }^2.\end{array}$$ (A153)

The optimal error exponent is also lower bounded as follows

$$\begin{array}{c}\hfill {\theta }_{ϵ}^{*}(R,L)\ge \frac{1}{2}log\left(\frac{1}{1-{\rho }^2·\left(1-2^{-2L}-{\beta }^2\right)}\right).\end{array}$$ (A154)

Combining (A153) and (A154) gives the lower bound on the error exponent in (64).

Converse: Consider the following upper bound on the optimal error exponent in Theorem 2:

$$\begin{array}{}\mathrm{(A155)}& \hfill {\theta }_{ϵ}^{*}(R,L)& \le I(U;Y)\hfill \mathrm{(A156)}& & =h\left(Y\right)-h\left(Y\right|U)\hfill \mathrm{(A157)}& & =\frac{1}{2}log\left(2\pi e\right)-h\left(Y\right|U)\hfill \mathrm{(A158)}& & =\frac{1}{2}log\left(2\pi e\right)-h\left(\rho \widehat{X}+F|U\right)\hfill \mathrm{(A159)}& & \le \frac{1}{2}log\left(2\pi e\right)-\frac{1}{2}log\left(2^{2h\left(\rho \widehat{X}|U\right)}+2\pi e\left(1-{\rho }^2·(1-2^{-2L})\right)\right)\hfill \mathrm{(A160)}& & \le \frac{1}{2}log\left(2\pi e\right)-\frac{1}{2}log\left({\rho }^2{2}^{2h\left(\widehat{X}|U\right)}+2\pi e\left(1-{\rho }^2·(1-2^{-2L})\right)\right),\hfill \end{array}$$

where (A159) follows from the entropy power inequality (EPI) [34] (Chapter 2). Now, consider the rate constraint as follows:

$$\begin{array}{}\mathrm{(A161)}& \hfill R& \ge I(\widehat{X};U)\hfill \mathrm{(A162)}& & =h\left(\widehat{X}\right)-h\left(\widehat{X}\right|U)\hfill \mathrm{(A163)}& & =\frac{1}{2}log\left(2\pi e\left(1-2^{-2L}\right)\right)-h\left(\widehat{X}\right|U),\hfill \end{array}$$

which is equivalent to

$$\begin{array}{c}\hfill 2^{2h\left(\widehat{X}\right|U)}\ge 2\pi e·2^{-2R}·\left(1-2^{-2L}\right).\end{array}$$ (A164)

Considering (A160) with (A164) yields the following upper bound on the error exponent:

$$\begin{array}{}\mathrm{(A165)}& \hfill {\theta }_{ϵ}^{*}(R,L)& \le \frac{1}{2}log\left(2\pi e\right)-\frac{1}{2}log\left(2\pi e{\rho }^2{2}^{-2R}\left(1-2^{-2L}\right)+2\pi e\left(1-{\rho }^2(1-2^{-2L})\right)\right)\hfill \mathrm{(A166)}& & =\frac{1}{2}log\left(\frac{1}{1-{\rho }^2(1-2^{-2R})(1-2^{-2L})}\right).\hfill \end{array}$$

This concludes the proof of the proposition.

Author Contributions

Investigation, A.G.; Supervision, S.S. and V.Y.F.T.; Writing—original draft, S.B.A.

Funding

This research was partially funded by grants R-263-000-C83-112 and R-263-000-C54-114.

Conflicts of Interest

The authors declare no conflict of interest.