Privacy-Aware Distributed Hypothesis Testing

Abstract

A distributed binary hypothesis testing (HT) problem involving two parties, a remote observer and a detector, is studied. The remote observer has access to a discrete memoryless source, and communicates its observations to the detector via a rate-limited noiseless channel. The detector observes another discrete memoryless source, and performs a binary hypothesis test on the joint distribution of its own observations with those of the observer. While the goal of the observer is to maximize the type II error exponent of the test for a given type I error probability constraint, it also wants to keep a private part of its observations as oblivious to the detector as possible. Considering both equivocation and average distortion under a causal disclosure assumption as possible measures of privacy, the trade-off between the communication rate from the observer to the detector, the type II error exponent, and privacy is studied. For the general HT problem, we establish single-letter inner bounds on both the rate-error exponent-equivocation and rate-error exponent-distortion trade-offs. Subsequently, single-letter characterizations for both trade-offs are obtained (i) for testing against conditional independence of the observer’s observations from those of the detector, given some additional side information at the detector; and (ii) when the communication rate constraint over the channel is zero. Finally, we show by providing a counter-example where the strong converse which holds for distributed HT without a privacy constraint does not hold when a privacy constraint is imposed. This implies that in general, the rate-error exponent-equivocation and rate-error exponent-distortion trade-offs are not independent of the type I error probability constraint.

1. Introduction

Data inference and privacy are often contradicting objectives. In many multi-agent system, each agent/user reveals information about its data to a remote service, application or authority, which in turn, provides certain utility to the users based on their data. Many emerging networked systems can be thought of in this context, from social networks to smart grids and communication networks. While obtaining the promised utility is the main goal of the users, privacy of data that is shared is becoming increasingly important. Thus, it is critical that the users ensure a desired level of privacy for the sensitive information revealed, while maximizing the utility subject to this constraint.

In many distributed learning or distributed decision-making applications, typically the goal is to learn the joint probability distribution of data available at different locations. In some cases, there may be prior knowledge about the joint distribution, for example, that it belongs to a certain set of known probability distributions. In such a scenario, the nodes communicate their observations to the detector, which then applies hypothesis testing (HT) on the underlying joint distribution of the data based on its own observations and those received from other nodes. However, with the efficient data mining and machine learning algorithms available today, the detector can illegitimately infer some unintended private information from the data provided to it exclusively for HT purposes. Such threats are becoming increasingly imminent as large amounts of seemingly irrelevant yet sensitive data are collected from users, such as in medical research [1], social networks [2], online shopping [3] and smart grids [4]. Therefore, there is an inherent trade-off between the utility acquired by sharing data and the associated privacy leakage.

There are several practical scenarios where the above-mentioned trade-off arises. For example, consider the issue of consumer privacy in the context of online shopping. A consumer would like to share some information about his/her shopping behavior, e.g., shopping history and preferences, with the shopping portal to get better deals and recommendations on relevant products. The shopping portal would like to determine whether the consumer belongs to its target age group (e.g., below 30 years old) before sending special offers to this customer. Assuming that the shopping patterns of the users within and outside the target age groups are independent, the shopping portal performs a hypothesis test to check if the consumer’s shared data is correlated with the data of its own customers. If the consumer is indeed within the target age group, the shopping portal would like to gather more information about this potential customer, particular interests, more accurate age estimation, etc.; while the user is reluctant to provide any further information. Yet another relevant example is the issue of user privacy in the context of wearable Internet of Things (IoT) devices such as smart watches and fitness trackers, which collect information on routine daily activities, and often have a third-party cloud interface.

In this paper, we study distributed HT (DHT) with a privacy constraint, in which an observer communicates its observations to a detector over a noiseless rate-limited channel of rate R nats per observed sample. Using the data received from the observer, the detector performs binary HT on the joint distribution of its own observations and those of the observer. The performance of the HT is measured by the asymptotic exponential rate of decay of the type II error probability, known as the type II error exponent (or error exponent henceforth), for a given constraint on the type I error probability (definitions will be given below). While the goal is to maximize the performance of the HT, the observer also wants to maintain a certain level of privacy against the detector for some latent private data that is correlated with its observations. We are interested in characterizing the trade-off between the communication rate from the observer to the detector over the channel, error exponent achieved by the HT and the amount of information leakage of private data. A special case of HT known as testing against conditional independence (TACI) will be of particular interest. In TACI, the detector tests whether its own observations are independent of those at the observer, conditioned on additional side information available at the detector.

1.1. Background

Distributed HT without any privacy constraint has been studied extensively from an information- theoretic perspective in the past, although many open problems remain. The fundamental results for this problem are first established in [5], which includes a single-letter lower bound on the optimal error exponent and a strong converse result which states that the optimal error exponent is independent of the constraint on the type I error probability. Exact single-letter characterization of the optimal error exponent for the testing against independence (TAI) problem, i.e., TACI with no side information at the detector, is also obtained. The lower bound established in [5] is further improved in [6,7]. Strong converse is studied in the context of complete data compression and zero-rate compression in [6,8], respectively, where in the former, the observer communicates to the detector using a message set of size two, while in the latter using a message set whose size grows sub-exponentially with the number of observed samples. The TAI problem with multiple observers remains open (similar to several other distributed compression problems when a non-trivial fidelity criterion is involved); however, the optimal error exponent is obtained in [9] when the sources observed at different observers follow a certain Markov relation. The scenario in which, in addition to HT, the detector is also interested in obtaining a reconstruction of the observer’s source, is studied in [10]. The authors characterize the trade-off between the achievable error exponent and the average distortion between the observer’s observations and the detector’s reconstruction. The TACI is first studied in [11], where the optimality of a random binning-based encoding scheme is shown. The optimal error exponent for TACI over a noisy communication channel is established in [12]. Extension of this work to general HT over a noisy channel is considered in [13], where lower bounds on the optimal error exponent are obtained by using a separation-based scheme and also using hybrid coding for the communication between the observer and the detector. The TACI with a single observer and multiple detectors is studied in [14], where each detector tests for the conditional independence of its own observations from those of the observer. The general HT version of this problem over a noisy broadcast channel and DHT over a multiple access channel is explored in [15]. While all the above works consider the asymmetric objective of maximizing the error exponent under a constraint on the type I error probability, the trade-off between the exponential rate of decay of both the type I and type II error probabilities are considered in [16,17,18].

Data privacy has been a hot topic of research in the past decade, spanning across multiple disciplines in computer and computational sciences. Several practical schemes have been proposed that deal with the protection or violation of data privacy in different contexts, e.g., see [19,20,21,22,23,24]. More relevant for our work, HT under mutual information and maximal leakage privacy constraints have been studied in [25,26], respectively, where the observer uses a memoryless privacy mechanism to convey a noisy version of its observed data to the detector. The detector performs HT on the probability distribution of the observer’s data, and the optimal privacy mechanism that maximizes the error exponent while satisfying the privacy constraint is analyzed. Recently, a distributed version of this problem has been studied in [27], where the observer applies a privacy mechanism to its observed data prior to further coding for compression, and the goal at the detector is to perform a HT on the joint distribution of its own observations with those of the observer. In contrast with [25,26,27], we study DHT with a privacy constraint, but without considering a separate privacy mechanism at the observer. In Section 2, we will further discuss the differences between the system model considered here and that of [27].

It is important to note here that the data privacy problem is fundamentally different from that of data security against an eavesdropper or an adversary. In data security, sensitive data is to be protected against an external malicious agent distinct from the legitimate parties in the system. The techniques for guaranteeing data security usually involve either cryptographic methods in which the legitimate parties are assumed to have additional resources unavailable to the adversary (e.g., a shared private key) or the availability of better communication channel conditions (e.g., using wiretap codes). However, in data privacy problems, the sensitive data is to be protected from the same legitimate party that receives the messages and provides the utility; and hence, the above-mentioned techniques for guaranteeing data security are not applicable. Another model frequently used in the context of information-theoretic security assumes the availability of different side information at the legitimate receiver and the eavesdropper [28,29]. A DHT problem with security constraints formulated along these lines is studied in [30], where the authors propose an inner bound on the rate-error exponent-equivocation trade-off. While our model is related to that in [30] when the side information at the detector and eavesdropper coincide, there are some important differences which will be highlighted in Section 2.3.

Many different privacy measures have been considered in the literature to quantify the amount of private information leakage, such as k-anonymity [31], differential privacy (DP) [32], mutual information leakage [33,34,35], maximal leakage [36], and total variation distance [37] to count a few; see [38] for a detailed survey. Among these, mutual information between the private and revealed information (or, equivalently, the equivocation of private information given the revealed information) is perhaps the most commonly used measure in the information-theoretic studies of privacy. It is well known that a necessary and sufficient condition to guarantee statistical independence between two random variables is to have zero mutual information between them. Furthermore, the average information leakage measured using an arbitrary privacy measure is upper bounded by a constant multiplicative factor of that measured by mutual information [34]. It is also shown in [33] that a differentially private scheme is not necessarily private when the information leakage is measured by mutual information. This is done by constructing an example that is differentially private, yet the mutual information leakage is arbitrarily high. Mutual information-based measures have also been used in cryptographic security studies. For example, the notion of semantic security defined in [39] is shown to be equivalent to a measure based on mutual information in [40].

A rate-distortion approach to privacy is first explored by Yamamoto in [41] for a rate-constrained noiseless channel, where in addition to a distortion constraint for legitimate data, a minimum distortion requirement is enforced for the private part. Recently, there have been several works that have used distortion as a security or privacy metric in several different contexts, such as side-information privacy in discriminatory lossy source coding [42] and rate-distortion theory of secrecy systems [43,44]. More specifically, in [43], the distortion-based security measure is analyzed under a causal disclosure assumption, in which the data samples to be protected are causally revealed to the eavesdropper (excluding the current sample), yet the average distortion over the entire block has to satisfy a desired lower bound. This assumption ensures that distortion as a secrecy measure is more robust (see ([43], Section I-A)), and could in practice model scenarios in which the sensitive data to be protected is eventually available to the eavesdropper with some delay, but the protection of the current data sample is important. In this paper, we will consider both equivocation and average distortion under a causal disclosure assumption as measures of privacy. In [45], error exponent of a HT adversary is considered to be a privacy measure. This can be considered to be the opposite setting to ours, in the sense that while the goal here is to increase the error exponent under a privacy leakage constraint, the goal in [45] is to reduce the error exponent under a constraint on possible transformations that can be applied on the data.

It is instructive to compare the privacy measures considered in this paper with DP. Towards this, note that average distortion and equivocation (see Definitions 1 and 2) are “average case” privacy measures, while DP is a “worst case” measure that focuses on the statistical indistinguishability of neighboring datasets that differ in just one entry. Considering this aspect, it may appear that these privacy measures are unrelated. However, as shown in [46], there is an interesting connection between them. More specifically, the maximum conditional mutual information leakage between the revealed data Y and an entry in the dataset $X_i$ given all the other $n-1$ entries $X^{-i}=X^n\setminus \left\{X_i\right\}$, i.e., $I(Y;X_i|X^{-i})$, is sandwiched between the so-called $ϵ$- DP and $(ϵ,\delta )$-DP in terms of the strength of the privacy measure, where the maximization is over all distributions $P_{X^n}$ on $X^n$ and entries $i\in [1:n]$ ([46], Theorem 1). This implies that as a privacy measure, equivocation (equivalent to mutual information leakage) is weaker than $ϵ$- DP, and stronger than $(ϵ,\delta )$-DP, at least for some probability distributions on the data. On the other hand, equivocation and average distortion are relatively well-behaved privacy measures compared to DP, and often result in clean and exact computable characterizations of the optimal trade-off for the problem at hand. Moreover, as already shown in [39,40,47,48], the trade-off resulting from “average” constraints turns out to be the same as that with more stricter constraints in many interesting cases. Hence, it is of interest to consider such average case privacy measures as a starting point for further investigation with stricter measures.

DP has been used extensively in privacy studies including those that involve learning and HT [49,50,51,52,53,54,55,56,57,58,59]. More relevant to the distributed HT problem at hand is the local differentially private model employed in [49,50,51,56], in which, depending on the privacy requirement, a certain amount of random noise is injected into the user’s data before further processing, while the utility is maximized subject to this constraint. Nevertheless, there are key differences between these models and ours. For example, in [49], the goal is to learn from differentially private “examples”, the underlying “concept” (model that maps examples to “labels”) such that the error probability in predicting the label for future examples is minimized, irrespective of the statistics of the examples. Hence, the utility in [49] is to learn an unknown model accurately, whereas our objective is to test between two known probability distributions. Furthermore, in our setting (unlike [49,50,51,56]), there is an additional requirement to satisfy in terms of the communication rate. These differences perhaps also make DP less suitable as a privacy measure in our model relative to equivocation and average distortion. On one hand, imposing a DP measure in our setting may be overly restrictive since there are only two probability distributions involved and DP is tailored for situations where the statistics of the underlying data is unknown. On the other hand, DP is also more unwieldy to analyze under a rate constraint compared to mutual information or average distortion.

The amount of private information leakage that can be tolerated depends on the specific application at hand. While it may be possible to tolerate a moderate amount of information leakage in applications like online shopping or social networks, it may no longer be the case in matters related to information sharing among government agencies or corporations. While it is obvious that maximum privacy can be attained by revealing no information, this typically comes at the cost of zero utility. On the other hand, maximum utility can be achieved by revealing all the information, but at the cost of minimum privacy. Characterizing the optimal trade-off between the utility and the minimum privacy leakage between these two extremes is a fundamental and challenging research problem.

1.2. Main Contributions

The main contributions of this work are as follows.

1. In Section 3, Theorem 1 (resp. Theorem 2), we establish a single-letter inner bound on the rate-error exponent-equivocation (resp. rate-error exponent-distortion) trade-off for DHT with a privacy constraint. The distortion and equivocation privacy constraints we consider, which is given in (6) and (7), respectively, are slightly stronger than what is usually considered in the literature (stated in (8) and (9), respectively).

2. Exact characterizations are obtained for some important special cases in Section 4. More specifically, a single-letter characterization of the optimal rate-error exponent-equivocation (resp. rate-error exponent-distortion) trade-off is established for:

   • (a)TACI with a privacy constraint (for vanishing type I error probability constraint) in Section 4.1, Proposition 1 (resp. Proposition 2),
   • (b)DHT with a privacy constraint for zero-rate compression in Section 4.2, Proposition 4 (resp. Proposition 3).

   Since the optimal trade-offs in Propositions 3 and 4 are independent of the constraint on the type I error probability, they are strong converse results in the context of HT.

3. Finally, in Section 5, we provide a counter-example showing that for a positive rate $R>0$, the strong converse result does not hold in general for TAI with a privacy constraint.

1.3. Organization

The organization of the paper is as follows. Basic notations are introduced in Section 2.1. The problem formulation and associated definitions are given in Section 2.2. Main results are presented in Section 3 to Section 5. The proofs of the results are presented either in the Appendix or immediately after the statement of the result. Finally, Section 6 concludes the paper with some open problems for future research.

2. Preliminaries

2.1. Notations

$\mathbb{N}$, $\mathbb{R}$ and ${\mathbb{R}}_{\ge 0}$ stand for the set of natural numbers, real numbers and non-negative real numbers, respectively. For $a\in {\mathbb{R}}_{\ge 0}$, $\left[a\right]:=\{i\in \mathbb{N},i\le a\}$ and for $a\in \mathbb{R}$, $a^{+}:=max\{0,a\}$ ($:=$ represents equality by definition). Calligraphic letters, e.g., $\mathcal{A}$, denotes sets, while $\left|\mathcal{A}\right|$ and ${\mathcal{A}}^c$ denotes its cardinality and complement, respectively. $\mathbb{1}(·)$ denotes the indicator function, while $O(·)$, $o(·)$ and $\Omega (·)$ stands for the standard asymptotic notations of Big-O, Little-O and Big-$\Omega$, respectively. For a real sequence ${\left\{a_n\right\}}_{n\in \mathbb{N}}$ and $b\in \mathbb{R}$, $a_n\stackrel{\left(n\right)}{\to }b$ represents ${lim}_{n\to \infty }{a}_n=b$. Similar notations apply for asymptotic inequalities, e.g., $a_n\stackrel{\left(n\right)}{\ge }b$, means that ${lim}_{n\to \infty }{a}_n\ge b$. Throughout this paper, the base of the logarithms is taken to be e, and whenever the range of the summation is not specified, it means summation over the entire support, e.g., ${\sum }_u$ denotes ${\sum }_{u\in \mathcal{U}}$.

All the random variables (r.v.’s) considered in this paper are discrete with finite support unless specified otherwise. We denote r.v.’s, their realizations and support by upper case, lower case and calligraphic letters (e.g., X, x and $\mathcal{X}$), respectively. The joint probability distribution of r.v.’s X and Y is denoted by $P_{XY}$, while their marginals are denoted by $P_X$ and $P_Y$. The set of all probability distributions with support $\mathcal{X}$ and $\mathcal{X}\times \mathcal{Y}$ are represented by $\mathcal{P}\left(\mathcal{X}\right)$ and $\mathcal{P}(\mathcal{X}\times \mathcal{Y})$, respectively. For $j,i\in \mathbb{N}$, the random vector $(X_i,\dots ,X_j)$, $j\ge i$, is denoted by $X_i^j$, while $X^j$ stands for $(X_1,\dots ,X_j)$. Similar notation holds for the vector of realizations. $X-Y-Z$ denotes a Markov chain relation between the r.v.’s X, Y and Z. ${\mathbb{P}}_P\left(\mathcal{E}\right)$ denotes the probability of event $\mathcal{E}$ with respect to the probability measure induced by distribution P, and ${\mathbb{E}}_P[·]$ denotes the corresponding expectation. The subscript P is omitted when the distribution involved is clear from the context. For two probability distributions P and Q defined on a common support, $P<<Q$ denotes that P is absolutely continuous with respect to Q.

Following the notation in [60], for $P_X\in \mathcal{P}\left(\mathcal{X}\right)$ and $\delta \ge 0$, the $P_X$-typical set is

$$\begin{array}{c}\hfill {\mathcal{T}}_{{\left[P_X\right]}_{\delta }}^n:=\left\{x^n\in {\mathcal{X}}^n:\left|P_X\left(x^{\prime }\right)-\frac{1}{n}\sum _{i=1}^n\mathbb{1}(x_i=x^{\prime })\right|\le \delta ,\forall x^{\prime }\in \mathcal{X}\right\},\end{array}$$

and the $P_X$-type class (set of sequences of type or empirical distribution $P_X$) is ${\mathcal{T}}_{P_X}^n:={\mathcal{T}}_{{\left[P_X\right]}_0}^n$. The set of all possible types of sequences of length n over an alphabet ${\mathcal{X}}^n$ and the set of types in ${\mathcal{T}}_{{\left[P_X\right]}_{\delta }}^n$ are denoted by ${\mathcal{P}}^n\left(\mathcal{X}\right)$ and ${\mathcal{P}}_n\left({\mathcal{T}}_{{\left[P_X\right]}_{\delta }}^n\right)$, respectively. Similar notations apply for pairs and larger combinations of r.v.’s, e.g., ${\mathcal{T}}_{{\left[P_{XY}\right]}_{\delta }}^n$, ${\mathcal{T}}_{P_{XY}}^n$, ${\mathcal{P}}^n(\mathcal{X}\times \mathcal{Y})$ and ${\mathcal{P}}_n\left({\mathcal{T}}_{{\left[P_{XY}\right]}_{\delta }}^n\right)$. The conditional $P_{Y|X}$ type class of a sequence $x^n\in {\mathcal{X}}^n$ is

$$\begin{array}{c}\hfill {\mathcal{T}}_{P_{Y|X}}^n\left(x^n\right):=\left\{y^n:(x^n,y^n)\in {\mathcal{T}}_{P_{XY}}^n\right\}.\end{array}$$ (1)

The standard information-theoretic quantities like Kullback–Leibler (KL) divergence between distributions $P_X$ and $Q_X$, the entropy of X with distribution $P_X$, the conditional entropy of X given Y and the mutual information between X and Y with joint distribution $P_{XY}$, are denoted by $D\left(P_X\right|\left|Q_X\right)$, $H_{P_X}\left(X\right)$, $H_{P_{XY}}\left(X\right|Y)$ and $I_{P_{XY}}(X;Y)$, respectively. When the distribution of the r.v.’s involved are clear from the context, the last three quantities are denoted simply by $H\left(X\right)$, $H\left(X\right|Y)$ and $I(X;Y)$, respectively. Given realizations $X^n=x^n$ and $Y^n=y^n$, $H_e\left(x^n\right|y^n)$ denotes the conditional empirical entropy given by

$$\begin{array}{c}\hfill H_e\left(y^n\right|x^n):=H_{P_{\tilde{X}\tilde{Y}}}\left(\tilde{Y}\right|\tilde{X}),\end{array}$$ (2)

where $P_{\tilde{X}\tilde{Y}}$ denotes the joint type of $(x^n,y^n)$. Finally, the total variation between probability distributions $P_X$ and $Q_X$ defined on the same support $\mathcal{X}$ is

$$\begin{array}{c}\hfill \left|\right|P_X-Q_X\left|\right|:=\frac{1}{2}\sum _{x\in \mathcal{X}}|P_X\left(x\right)-Q_X\left(x\right)|.\end{array}$$

2.2. Problem Formulation

Consider the HT setup illustrated in Figure 1, where $(U^n,V^n,S^n)$ denote n independent and identically distributed (i.i.d.) copies of triplet of r.v.’s $(U,V,S)$. The observer observes $U^n$ and sends the message index M to the detector over an error-free channel, where $M\sim f_n(·|U^n)$ and $f_n:{\mathcal{U}}^n\to \mathcal{P}\left(\mathcal{M}\right)$, $\mathcal{M}=\left[e^{nR}\right]$. Given its own observation $V^n$, the detector performs a HT on the joint distribution of $U^n$ and $V^n$ with null hypothesis

$$\begin{array}{c}\hfill H_0:(U^n,V^n)\sim \prod _{i=1}^n{P}_{UV},\end{array}$$

and alternate hypothesis

$$\begin{array}{c}\hfill H_1:(U^n,V^n)\sim \prod _{i=1}^n{Q}_{UV}.\end{array}$$

Figure 1.

DHT with a privacy constraint.

Let H and $\widehat{H}$ denote the r.v.’s corresponding to the true hypothesis and the output of the HT, respectively, with support $\mathcal{H}=\widehat{\mathcal{H}}=\{0,1\}$, where 0 denotes the null hypothesis and 1 the alternate hypothesis. Let $g_n:\mathcal{M}\times {\mathcal{V}}^n\to \mathcal{P}\left(\widehat{\mathcal{H}}\right)$ denote the decision rule at the detector, which outputs $\widehat{H}\sim g_n(M,V^n)$. Then, the type I and type II error probabilities achieved by a $\left(f_n,g_n\right)$ pair are given by

$$\begin{array}{c}\hfill {\alpha }_n\left(f_n,g_n\right):=\mathbb{P}(\widehat{H}=1|H=0)=P_{\widehat{H}}\left(1\right),\end{array}$$

and

$$\begin{array}{c}\hfill {\beta }_n\left(f_n,g_n\right):=\mathbb{P}(\widehat{H}=0|H=1)=Q_{\widehat{H}}\left(0\right),\end{array}$$

respectively, where

$$\begin{array}{cc}\hfill P_{\widehat{H}}\left(1\right)& =\sum _{u^n,m,v^n}\left[\prod _{i=1}^n{P}_{UV}(u_i,v_i)\right]f_n\left(m\right|u^n)g_n\left(1\right|m,v^n),\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill Q_{\widehat{H}}\left(0\right)& =\sum _{u^n,m,v^n}\left[\prod _{i=1}^n{Q}_{UV}(u_i,v_i)\right]f_n\left(m\right|u^n)g_n\left(0\right|m,v^n).\hfill \end{array}$$

Let $P_{U^n{V}^n{S}^{n}M\widehat{H}}$ and $Q_{U^n{V}^n{S}^{n}M\widehat{H}}$ denote the joint distribution of $(U^n,V^n,S^n,M,\widehat{H})$ under the null and alternate hypotheses, respectively. For a given type I error probability constraint $ϵ$, define the minimum type II error probability over all possible detectors as

$$\begin{array}{cc}\hfill & {\overline{\beta }}_n\left(f_n,ϵ\right):=\underset{g_n}{inf}{\beta }_n\left(f_n,g_n\right),\hfill \\ \hfill & \mathrm{such} \mathrm{that}{\alpha }_n\left(f_n,g_n\right)\le ϵ.\hfill \end{array}$$ (3)

The performance of HT is measured by the error exponent achieved by the test for a given constraint $ϵ$ on the type I error probability, i.e., ${lim\; inf}_{n\to \infty }-\frac{1}{n}log\left({\overline{\beta }}_n(f_n,ϵ)\right)$. Although the goal of the detector is to maximize the error exponent achieved for the HT, it is also curious about the latent r.v. $S^n$ that is correlated with $U^n$. $S^n$ is referred to as the private part of $U^n$, which is distributed i.i.d. according to the joint distribution $P_{SUV}$ and $Q_{SUV}$ under the null and alternate hypothesis, respectively. It is desired to keep the private part as concealed as possible from the detector. We consider two measures of privacy for $S^n$ at the detector. The first is the equivocation defined as $H\left(S^n\right|M,V^n)$. The second one is the average distortion between $S^n$ and its reconstruction ${\widehat{S}}^n$ at the detector, measured according to an arbitrary bounded additive distortion metric $d:\mathcal{S}\times \widehat{\mathcal{S}}\to [0,D_m]$ with multi-letter distortion defined as

$$\begin{array}{c}\hfill d(s^n,{\widehat{s}}^n):=\sum _{i=1}^{n}d(s_i,{\widehat{s}}_i).\end{array}$$ (4)

We will assume the causal disclosure assumption, i.e., ${\widehat{S}}_i$ is a function of $S^{i-1}$ in addition to $(M,V^n)$. The goal is to ensure that the error exponent for HT is maximized, while satisfying the constraints on the type I error probability $ϵ$ and the privacy of $S^n$. In the sequel, we study the trade-off between the rate, error exponent (henceforth also referred to simply as the error exponent) and privacy achieved in the above setting. Before delving into that, a few definitions are in order.

Definition 1.

For a given type I error probability constraint ϵ, a rate-error exponent-distortion tuple $(R,\kappa ,{\Delta }_0,{\Delta }_1)$ is achievable, if there exists a sequence of encoding and decoding functions $f_n:{\mathcal{U}}^n\to \mathcal{P}\left(\mathcal{M}\right)$, and $g_n:\mathcal{M}\times {\mathcal{V}}^n\to \mathcal{P}\left(\widehat{\mathcal{H}}\right)$ such that

$$\begin{array}{cc}\hfill \underset{n\to \infty }{lim\; inf}\frac{-log\left({\overline{\beta }}_n(f_n,ϵ)\right)}{n}& \ge \kappa ,\hfill \end{array}$$ (5)

and for any $\gamma >0$, there exists an $n_0\in \mathbb{N}$ such that

$$\begin{array}{cc}\hfill \underset{{\left\{g_{i,n}^{\left(r\right)}\right\}}_{i=1}^n}{inf}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=j\right]& \ge n{\Delta }_j-\gamma ,\forall n\ge n_0,j=0,1,\hfill \end{array}$$ (6)

where ${\widehat{S}}_i\sim g_{i,n}^{\left(r\right)}(·|M,V^n,S^{i-1})$, and $g_{i,n}^{\left(r\right)}:\left[e^{nR}\right]\times {\mathcal{V}}^n\times {\mathcal{S}}^{i-1}\to \mathcal{P}\left({\widehat{\mathcal{S}}}_i\right)$ denotes an arbitrary stochastic reconstruction map at the detector. The rate-error exponent-distortion region ${\mathcal{R}}_d\left(ϵ\right)$ is the closure of the set of all such achievable $(R,\kappa ,{\Delta }_0,{\Delta }_1)$ tuples for a given ϵ.

Definition 2.

For a given type I error probability constraint ϵ, a rate-error exponent-equivocation (It is well known that equivocation as a privacy measure is a special case of average distortion under the causal disclosure assumption and log-loss distortion metric [43]. However, we provide a separate definition of the rate-error exponent-equivocation region for completeness.) $(R,\kappa ,{\Lambda }_0,{\Lambda }_1)$ tuple is achievable, if there exists a sequence of encoding and decoding functions $f_n:{\mathcal{U}}^n\to \mathcal{P}\left(\mathcal{M}\right)$ and $g_n:\left[e^{nR}\right]\times {\mathcal{V}}^n\to \mathcal{P}\left(\widehat{\mathcal{H}}\right)$ such that (5) is satisfied, and for any $\gamma >0$, there exists a $n_0\in \mathbb{N}$ such that

$$\begin{array}{c}\hfill H\left(S^n\right|M,V^n,H=i)\ge n{\Lambda }_i-\gamma ,\forall n\ge n_0,i\in \{0,1\}.\end{array}$$ (7)

The rate-error exponent-equivocation region ${\mathcal{R}}_e\left(ϵ\right)$ is the closure of the set of all such achievable $(R,\kappa ,{\Lambda }_0,{\Lambda }_1)$ tuples for a given ϵ.

Please note that the privacy measures considered in (6) and (7) are stronger than

$$\begin{array}{cc}\hfill & \underset{n\to \infty }{lim\; inf}\underset{{\left\{g_{i,n}^{\left(r\right)}\right\}}_{i=1}^n}{inf}\mathbb{E}\left[\frac{1}{n}d\left(S^n,{\widehat{S}}^n\right)|H=i\right]\ge {\Delta }_i,i=0,1,\hfill \end{array}$$ (8)

$$\begin{array}{cc}\hfill \mathrm{and} & \underset{n\to \infty }{lim\; inf}\frac{1}{n}H\left(S^n\right|M,V^n,H=i)\ge {\Lambda }_i,i=0,1,\hfill \end{array}$$ (9)

respectively. To see this for the equivocation privacy measure, note that if $H\left(S^n\right|M,V^n,H=i)=n{\Lambda }_i^{*}-n^a$, $i=0,1$, for some $a\in (0,1)$, then an equivocation pair $({\Lambda }_0^{*},{\Lambda }_1^{*})$ is achievable under the constraint given in (9), while it is not achievable under the constraint given in (7).

2.3. Relation to Previous Work

Before stating our results, we briefly highlight the differences between our system model and the ones studied in [27,30]. In [27], the observer applies a privacy mechanism to the data before releasing it to the transmitter, which performs further encoding prior to transmission to the detector. More specifically, the observer checks if $U^n\in {\mathcal{T}}_{{\left[P_U\right]}_{\delta }}^n$ and if successful, sends the output of a memoryless privacy mechanism applied to $U^n$, to the transmitter. Otherwise, it outputs a n-length zero-sequence. The privacy mechanism plays the role of randomizing the data (or adding noise) to achieve the desired privacy. Such randomized privacy mechanisms are popular in privacy studies, and have been used in [25,26,61]. In our model, the tasks of coding for privacy and compression are done jointly by using all the available data samples $U^n$. Also, while we consider the equivocation (and average distortion) between the revealed information and the private part as the privacy measure, in [27], the mutual information between the observer’s observations and the output of the memoryless mechanism is the privacy measure. As a result of these differences, there exist some points in the rate-error exponent-privacy trade-off that are achievable in our model, but not in [27]. For instance, a perfect privacy condition ${\Lambda }_0=0$ for testing against independence in ([27], Theorem 2) would imply that the error exponent is also zero, since the output of the memoryless mechanism has to be independent of the observer’s observations (under both hypotheses). However, as we later show in Example 2, a positive error exponent is achievable while guaranteeing perfect privacy in our model.

On the other hand, the difference between our model and [30] arises from the difference in the privacy constraint as well as the privacy measure. Specifically, the goal in [30] is to keep $U^n$ private from an illegitimate eavesdropper, while the objective here is to keep a r.v. $S^n$ that is correlated with $U^n$ private from the detector. Also, we consider the more general average distortion (under causal disclosure) as a privacy measure, in addition to equivocation in [30]. Moreover, as already noted, the equivocation privacy constraint in (7) is more stringent than (9) that is considered in [30]. To satisfy the distortion requirement or the stronger equivocation privacy constraint in (7), we require that the a posteriori probability distribution of $S^n$ given the observations $(M,V^n)$ at the detector is close in some sense to a desired “target" memoryless distribution. To achieve this, we use a stochastic encoding scheme to induce the necessary randomness for $S^n$ at the detector, which to the best of our knowledge has not been considered previously in the context of DHT. Consequently, the analysis of the type I and type II error probabilities and privacy achieved are novel. Another subtle yet important difference is that the marginal distributions of $U^n$ and the side information at the eavesdropper are assumed to be the same under the null and alternate hypotheses in [30], which is not the case here. This necessitates separate analysis for the privacy achieved under the two hypotheses.

Next, we state some supporting results that will be useful later for proving the main results.

2.4. Supporting Results

Let

$$\begin{array}{c}\hfill g_{{\mathcal{A}}_n}^{\left(d\right)}(m,v^n)=\mathbb{1}\left((m,v^n)\in {\mathcal{A}}_n^c\right)\end{array}$$ (10)

denote a deterministic detector with acceptance region ${\mathcal{A}}_n\subseteq \left[e^{nR}\right]\times {\mathcal{V}}^n$ for $H_0$ and ${\mathcal{A}}_n^c$ for $H_1$. Then, the type I and type II error probabilities are given by

$$\begin{array}{cc}\hfill {\alpha }_n\left(f_n,g_n\right)& :=P_{M{V}^n}\left({\mathcal{A}}_n^c\right)={\mathbb{E}}_P\left[\mathbb{1}\left((M,V^n)\in {\mathcal{A}}_n^c\right)\right],\hfill \end{array}$$ (11)

$$\begin{array}{cc}\hfill {\beta }_n\left(f_n,g_n\right)& :=Q_{M{V}^n}\left({\mathcal{A}}_n\right)={\mathbb{E}}_Q\left[\mathbb{1}\left((M,V^n)\in {\mathcal{A}}_n\right)\right].\hfill \end{array}$$ (12)

Lemma 1.

Any error exponent that is achievable is also achievable by a deterministic detector of the form given in (10) for some ${\mathcal{A}}_n\subseteq \left[e^{nR}\right]\times {\mathcal{V}}^n$, where ${\mathcal{A}}_n$ and ${\mathcal{A}}_n^c$ denote the acceptance regions for $H_0$ and $H_1$, respectively.

The proof of Lemma 1 is given in Appendix A for completeness. Due to Lemma 1, henceforth we restrict our attention to a deterministic $g_n$ as given in (10).

The next result shows that without loss of generality (w.l.o.g), it is also sufficient to consider $g_{i,n}^{\left(r\right)}$ (in Definition 1) to be a deterministic function of the form

$$\begin{array}{c}\hfill g_{i,n}^{\left(r\right)}={\left\{{\overline{\varphi }}_{i,n}(·,·,·)\right\}}_{i=1}^n\end{array}$$ (13)

for the minimization in (6), where ${\overline{\varphi }}_{i,n}:\mathcal{M}\times {\mathcal{V}}^n\times {\mathcal{S}}^{i-1}\to \widehat{\mathcal{S}}$, $i\in \left[n\right]$, denotes an arbitrary deterministic function.

Lemma 2.

The infimum in (6) is achieved by a deterministic function $g_{i,n}^{\left(r\right)}$ as given in (13), and hence it is sufficient to restrict to such deterministic $g_{i,n}^{\left(r\right)}$ in (6).

The proof of Lemma 2 is given in Appendix B. Next, we state some lemmas that will be handy for upper bounding the amount of privacy leakage in the proofs of the main results stated below. The following one is a well-known result proved in [60] that upper bounds the difference in entropy of two r.v.’s (with a common support) in terms of the total variation distance between their probability distributions.

Lemma 3.

([60], Lemma 2.7) Let $P_X$ and $Q_X$ be distributions defined on a common support $\mathcal{X}$ and let $\rho :=\left|\right|P_X-Q_X\left|\right|$. Then, for $\rho \le \frac{1}{4}$

$$\begin{array}{c}\hfill |H_{P_X}-H_{Q_X}|\le -2\rho log\left(\frac{2\rho }{\left|\mathcal{X}\right|}\right).\end{array}$$

The next lemma will be handy in proving Theorems 1 and 2, Proposition 3 and the counter-example for strong converse presented in Section 5.

Lemma 4.

Let $(X^n,Y^n)$ denote n i.i.d. copies of r.v.’s $(X,Y)$, and $P_{X^n{Y}^n}={\prod }_{i=1}^n{P}_{XY}$ and $Q_{X^n{Y}^n}={\prod }_{i=1}^n{Q}_{XY}$ denote two joint probability distributions on $(X^n,Y^n)$. For $\delta >0$, define

$$\begin{array}{c}\hfill \Pi (x^n,\delta ,P_X):=\mathbb{1}\left(x^n\notin {\mathcal{T}}_{{\left[P_X\right]}_{\delta }}^n\right).\end{array}$$ (14)

If $P_X\ne Q_X$, then for $\delta >0$ sufficiently small, there exists $\overline{\delta }>0$ and $n_0(\delta ,|\mathcal{X}|,|\mathcal{Y}\left|\right)\in \mathbb{N}$ such that for all $n\ge n_0(\delta ,|\mathcal{X}|,|\mathcal{Y}\left|\right)$,

$$\begin{array}{c}\hfill \parallel Q_{Y^n}(·)-Q_{Y^n|\Pi (X^n,\delta ,P_X)}(·|1)\parallel \le e^{-n\overline{\delta }}.\end{array}$$ (15)

If $P_X=Q_X$, then for any $\delta >0$, there exists $\overline{\delta }>0$ and $n_0(\delta ,|\mathcal{X}|,|\mathcal{Y}\left|\right)\in \mathbb{N}$ such that for all $n\ge n_0(\delta ,|\mathcal{X}|,|\mathcal{Y}\left|\right)$,

$$\begin{array}{cc}\hfill \parallel Q_{Y^n}(·)-Q_{Y^n|\Pi (X^n,\delta ,P_X)}(·|0)\parallel & \le e^{-n\overline{\delta }},\hfill \end{array}$$ (16)

Also, for any $\delta >0$, there exists $\overline{\delta }>0$ and $n_0(\delta ,|\mathcal{X}|,|\mathcal{Y}\left|\right)\in \mathbb{N}$ such that for all $n\ge n_0(\delta ,|\mathcal{X}|,|\mathcal{Y}\left|\right)$,

$$\begin{array}{cc}\hfill \parallel P_{Y^n}(·)-P_{Y^n|\Pi (X^n,\delta ,P_X)}(·|0)\parallel & \le e^{-n\overline{\delta }}.\hfill \end{array}$$ (17)

Proof. 

The proof is presented in Appendix C. ☐

In the next section, we establish an inner bound on ${\mathcal{R}}_e\left(ϵ\right)$ and ${\mathcal{R}}_d\left(ϵ\right)$.

3. Main Results

The following two theorems are the main results of this paper providing inner bounds for ${\mathcal{R}}_e\left(ϵ\right)$ and ${\mathcal{R}}_d\left(ϵ\right)$, respectively.

Theorem 1.

For $ϵ\in (0,1)$, $(R,\kappa ,{\Lambda }_0,{\Lambda }_1)\in {\mathcal{R}}_e\left(ϵ\right)$ if there exists an auxiliary r.v. W, such that $(V,S)-U-W$, and

$$\begin{array}{cc}\hfill R& \ge I_P(W;U|V),\hfill \end{array}$$ (18)

$$\begin{array}{cc}\hfill \kappa & \le {\kappa }^{*}(P_{W|U},R),\hfill \end{array}$$ (19)

$$\begin{array}{cc}\hfill {\Lambda }_0& \le H_P\left(S\right|W,V),\hfill \end{array}$$ (20)

$$\begin{array}{cc}\hfill {\Lambda }_1& \le \mathbb{1}\left(P_U=Q_U\right)H_Q\left(S\right|W,V)+\mathbb{1}\left(P_U\ne Q_U\right)H_Q\left(S\right|V),\hfill \end{array}$$ (21)

where

$$\begin{array}{cc}\hfill {\kappa }^{*}(P_{W|U},R)& :=min\left(E_1\left(P_{W|U}\right),E_2(R,P_{W|U})\right),\hfill \end{array}$$

$$\begin{array}{cc}\hfill E_1\left(P_{W|U}\right)& :=\underset{P_{\tilde{U}\tilde{V}\tilde{W}}\in {\mathcal{L}}_1(P_{UW},P_{VW})}{min}D(P_{\tilde{U}\tilde{V}\tilde{W}}\left|\right|Q_{UV}{P}_{W|U}),\hfill \end{array}$$ (22)

$$\begin{array}{cc}\hfill E_2(R,P_{W|U})& :=\left\{\begin{array}{cc}\begin{array}{c}min\\ P_{\tilde{U}\tilde{V}\tilde{W}}\in {\mathcal{L}}_2(P_{UW},P_V)\end{array}D(P_{\tilde{U}\tilde{V}\tilde{W}}\left|\right|Q_{UV}{P}_{W|U})+(R-I_P(U;W|V)),\hfill & if{I}_P(U;W)>R,\hfill \\ \infty ,\hfill & otherwise,\hfill \end{array}\right.\hfill \\ \hfill {\mathcal{L}}_1(P_{UW},P_{VW})& :=\{P_{\tilde{U}\tilde{V}\tilde{W}}\in \mathcal{P}(\mathcal{U}\times \mathcal{V}\times \mathcal{W}):P_{\tilde{U}\tilde{W}}=P_{UW},P_{\tilde{V}\tilde{W}}=P_{VW}\},\hfill \\ \hfill {\mathcal{L}}_2(P_{UW},P_V)& :=\{P_{\tilde{U}\tilde{V}\tilde{W}}\in \mathcal{P}(\mathcal{U}\times \mathcal{V}\times \mathcal{W}):P_{\tilde{U}\tilde{W}}=P_{UW},P_{\tilde{V}}=P_V,H_P\left(W\right|V)\le H\left(\tilde{W}\right|\tilde{V})\},\hfill \\ \hfill P_{SUVW}& :=P_{SUV}{P}_{W|U},and{Q}_{SUVW}:=Q_{SUV}{P}_{W|U}.\hfill \end{array}$$ (23)

Theorem 2.

For a given bounded additive distortion measure $d(·,·)$ and $ϵ\in (0,1)$, $(R,\kappa ,{\Delta }_0,{\Delta }_1)\in {\mathcal{R}}_d\left(ϵ\right)$ if there exist an auxiliary r.v. W and deterministic functions $\varphi :\mathcal{W}\times \mathcal{V}\to \widehat{S}$ and ${\varphi }^{\prime }:\mathcal{V}\to \widehat{S}$, such that $(V,S)-U-W$ and (18) and (19),

$$\begin{array}{cc}\hfill {\Delta }_0& \le \underset{\varphi (·,·)}{min}{\mathbb{E}}_P\left[d\left(S,\varphi (W,V)\right)\right],\hfill \end{array}$$ (24)

$$\begin{array}{cc}\hfill \mathrm{and} {\Delta }_1& \le \mathbb{1}\left(P_U=Q_U\right)\underset{\varphi (·,·)}{min}{\mathbb{E}}_Q\left[d\left(S,\varphi (W,V)\right)\right]+\mathbb{1}\left(P_U\ne Q_U\right)\underset{{\varphi }^{\prime }(·)}{min}{\mathbb{E}}_Q\left[d\left(S,{\varphi }^{\prime }\left(V\right)\right)\right],\hfill \end{array}$$ (25)

are satisfied, where $P_{SUVW}$ and $Q_{SUVW}$ are as defined in Theorem 1.

The proof of Theorems 1 and 2 is given in Apppendix Appendix D. While the rate-error exponent trade-off in Theorems 1 and 2 is the same as that achieved by the Shimokawa-Han-Amari (SHA) scheme [7], the coding strategy achieving it is different due to the requirement of the privacy constraint. As mentioned above, in order to obtain a single-letter lower bound for the achievable distortion (and achievable equivocation) of the private part at the detector, it is required that the a posteriori probability distribution of $S^n$ given the observations $(M,V^n)$ at the detector is close in some sense to a desired “target” memoryless distribution. For this purpose, we use the so-called likelihood encoder [62,63] (at the observer) in our achievability scheme. The likelihood encoder is a stochastic encoder that induces the necessary randomness for $S^n$ at the detector, and to the best of our knowledge has not been used before in the context of DHT. The analysis of the type I and type II error probabilities and the privacy achieved by our scheme is novel and involves the application of the well-known channel resolvability or soft-covering lemma [62,64,65]. Properties of the total variation distance between probability distributions mentioned in [43] play a key role in this analysis. The analysis also reveals the interesting fact that the coding schemes in Theorems 1 and 2, although quite different from the SHA scheme, achieves the same lower bound on the error exponent.

Theorems 1 and 2 provide single-letter inner bounds on ${\mathcal{R}}_d\left(ϵ\right)$ and ${\mathcal{R}}_e\left(ϵ\right)$, respectively. A complete computable characterization of these regions would require a matching converse. This is a hard problem, since such a characterization is not available even for the DHT problem without a privacy constraint, in general (see [5]). However, it is known that a single-letter characterization of the rate-error exponent region exists for the special case of TACI [11]. In the next section, we show that TACI with a privacy constraint also admits a single-letter characterization, in addition to other optimality results.

4. Optimality Results for Special Cases

4.1. TACI with a Privacy Constraint

Assume that the detector observes two discrete memoryless sources $Y^n$ and $Z^n$, i.e., $V^n=(Y^n,Z^n)$. In TACI, the detector tests for the conditional independence of U and Y, given Z. Thus, the joint distribution of the r.v.’s under the null and alternate hypothesis are given by

$$H_0:P_{SUYZ}:=P_{S|UYZ}{P}_{U|Z}{P}_{Y|UZ}{P}_Z,$$ (26a)

and

$$H_1:Q_{SUYZ}:=Q_{S|UYZ}{P}_{U|Z}{P}_{Y|Z}{P}_Z,$$ (26b)

respectively.

Let ${\mathcal{R}}_e$ and ${\mathcal{R}}_d$ denote the rate-error exponent-equivocation and rate-error exponent-distortion regions, respectively, for the case of vanishing type I error probability constraint, i.e.,

$$\begin{array}{cc}\hfill {\mathcal{R}}_e& :=\underset{ϵ\to 0}{lim}{\mathcal{R}}_e\left(ϵ\right)and{\mathcal{R}}_d:=\underset{ϵ\to 0}{lim}{\mathcal{R}}_d\left(ϵ\right).\hfill \end{array}$$

Assume that the privacy constraint under the alternate hypothesis is inactive. Thus, we are interested in characterizing the set of all tuples $(R,\kappa ,{\Lambda }_0,{\Lambda }_1)\in {\mathcal{R}}_e$ and $(R,\kappa ,{\Delta }_0,{\Delta }_1)\in {\mathcal{R}}_d$, where

$$\begin{array}{cc}\hfill {\Lambda }_1& \le {\Lambda }_{min}:=H_Q\left(S\right|U,Y,Z),\hfill \\ \hfill \mathrm{and} {\Delta }_1& \le {\Delta }_{min}:=\underset{\varphi (u,y,z)}{min}{\mathbb{E}}_Q\left[d\left(S,\varphi (U,Y,Z)\right)\right].\hfill \end{array}$$ (27)

Please note that ${\Lambda }_{min}$ and ${\Delta }_{min}$ correspond to the equivocation and average distortion of $S^n$ at the detector, respectively, when $U^n$ is available directly at the detector under the alternate hypothesis. The above assumption is motivated by scenarios, in which the observer is more eager to protect $S^n$ when there is a correlation between its own observation and that of the detector, such as the online shopping portal example mentioned in Section 1. In that example, $U^n$, $S^n$ and $Y^n$ corresponds to shopping behavior, more information about the customer, and customers data available to the shopping portal, respectively.

For the above-mentioned case, we have the following results.

Proposition 1.

For the HT given in (26), $(R,\kappa ,{\Lambda }_0,{\Lambda }_{min})\in {\mathcal{R}}_e$ if and only if there exists an auxiliary r.v. W, such that $(Z,Y,S)-U-W$, and

$$\begin{array}{cc}\hfill \kappa & \le I_P(W;Y|Z),\hfill \end{array}$$ (28)

$$\begin{array}{cc}\hfill R& \ge I_P(W;U|Z),\hfill \end{array}$$ (29)

$$\begin{array}{cc}\hfill {\Lambda }_0& \le H_P\left(S\right|W,Z,Y),\hfill \end{array}$$ (30)

for some joint distribution of the form $P_{SUYZW}:=P_{SUYZ}{P}_{W|U}$.

Proof. 

For TACI, the inner bound in Theorem 1 yields that for $ϵ\in (0,1)$, $(R,\kappa ,{\Lambda }_0,{\Lambda }_1)\in {\mathcal{R}}_e\left(ϵ\right)$ if there exists an auxiliary r.v. W, such that $(Y,Z,S)-U-W$, and

$$\begin{array}{cc}\hfill R& \ge I_P(W;U|Y,Z),\hfill \end{array}$$ (31)

$$\begin{array}{cc}\hfill \kappa & \le {\kappa }^{*}(P_{W|U},R),\hfill \end{array}$$ (32)

$$\begin{array}{cc}\hfill {\Lambda }_0& \le H_P\left(S\right|W,Y,Z),\hfill \end{array}$$ (33)

$$\begin{array}{cc}\hfill {\Lambda }_1& \le H_Q\left(S\right|W,Y,Z),\hfill \end{array}$$ (34)

where

$$\begin{array}{cc}\hfill & {\kappa }^{*}(P_{W|U},R):=min\left(E_1\left(P_{W|U}\right),E_2(R,P_{W|U})\right),\hfill \end{array}$$

$$\begin{array}{cc}\hfill & E_1\left(P_{W|U}\right):=\underset{P_{\tilde{U}\tilde{Y}\tilde{Z}\tilde{W}}\in {\mathcal{L}}_1(P_{UW},P_{YZW})}{min}D(P_{\tilde{U}\tilde{Y}\tilde{Z}\tilde{W}}\left|\right|Q_{UYZ}{P}_{W|U}),\hfill \\ \hfill & E_2(R,P_{W|U})\hfill \end{array}$$ (35)

$$\begin{array}{cc}\hfill & :=\left\{\begin{array}{cc}\begin{array}{c}min\\ P_{\tilde{U}\tilde{Y}\tilde{Z}\tilde{W}}\in {\mathcal{L}}_2(P_{UW},P_{YZ})\end{array}D(P_{\tilde{U}\tilde{Y}\tilde{Z}\tilde{W}}\left|\right|Q_{UYZ}{P}_{W|U})+(R-I_P(U;W|Y,Z)),\hfill & if{I}_P(U;W)>R,\hfill \\ \infty ,\hfill & \mathrm{otherwise},\hfill \end{array}\right.\hfill \\ \hfill & {\mathcal{L}}_1(P_{UW},P_{YZW}):=\{P_{\tilde{U}\tilde{Y}\tilde{Z}\tilde{W}}\in \mathcal{P}(\mathcal{U}\times \mathcal{Y}\times \mathcal{Z}\times \mathcal{W}):P_{\tilde{U}\tilde{W}}=P_{UW},P_{\tilde{Y}\tilde{Z}\tilde{W}}=P_{YZW}\},\hfill \\ \hfill & {\mathcal{L}}_2(P_{UW},P_{YZ}):=\{P_{\tilde{U}\tilde{Y}\tilde{Z}\tilde{W}}\in \mathcal{P}(\mathcal{U}\times \mathcal{Y}\times \mathcal{Z}\times \mathcal{W}):P_{\tilde{U}\tilde{W}}=P_{UW},P_{\tilde{Y}\tilde{Z}}=P_{YZ},H_P\left(W\right|Y,Z)\le H\left(\tilde{W}\right|\tilde{Y}\tilde{Z})\},\hfill \\ \hfill & P_{SUYZW}:=P_{SUYZ}{P}_{W|U},Q_{SUYZW}:=Q_{S|YZ}{P}_{U|Z}{P}_{Y|Z}{P}_Z{P}_{W|U}.\hfill & \hfill \end{array}$$ (36)

Please note that since $(Y,Z,S)-U-W$, we have

$$\begin{array}{c}\hfill I_P(W;U)\ge I_P(W;U|Y,Z).\end{array}$$ (37)

Let ${\mathcal{B}}^{\prime }:=\{P_{W|U}:I_P(U;W|Z)\le R\}$. Then, for $P_{W|U}\in {\mathcal{B}}^{\prime }$, we have,

$$\begin{array}{cc}\hfill E_1(R,P_{W|U})& =\underset{P_{\tilde{U}\tilde{Y}\tilde{Z}\tilde{W}}\in {\mathcal{L}}_1(P_{UW},P_{YZW})}{min}D(P_{\tilde{U}\tilde{Y}\tilde{Z}\tilde{W}}\left|\right|Q_{UYZ}{P}_{W|U})=I_P(Y;W|Z),\hfill \\ \hfill E_2(R,P_{W|U})& \ge I_P(U;W|Z)-I_P(U;W|Y,Z)=I_P(Y;W|Z).\hfill \end{array}$$

Hence,

$$\begin{array}{c}\hfill {\kappa }^{*}(P_{W|U},R)\ge I_P(Y;W|Z).\end{array}$$ (38)

By noting that ${\Lambda }_{min}\le H_Q\left(S\right|W,Y,Z)$ (by the data processing inequality), we have shown that for ${\Lambda }_1\le {\Lambda }_{min}$, $(R,\kappa ,{\Lambda }_0,{\Lambda }_1)\in {\mathcal{R}}_e$ if (28)–(30) are satisfied. This completes the proof of achievability.

Converse: Let $(R,\kappa ,{\Lambda }_0,{\Lambda }_1)\in {\mathcal{R}}_e$. Let T be a r.v. uniformly distributed over $\left[n\right]$ and independent of all the other r.v.’s $(U^n,Y^n,Z^n,S^n,M)$. Define an auxiliary r.v. $W:=(W_T,T)$, where $W_i:=(M,Y^{i-1},S^{i-1},Z^{i-1},Z_{i+1}^n)$, $i\in \left[n\right]$. Then, we have for sufficiently large n that

$$\begin{array}{cc}\hfill nR& \ge H_P\left(M\right)\ge H_P\left(M\right|Z^n)\ge I_P(M;U^n|Z^n)\hfill \\ \hfill & ={\sum }_{i=1}^n{I}_P(M;U_i|U^{i-1},Z^n)\hfill \end{array}$$ (39)

$$\begin{array}{cc}\hfill & ={\sum }_{i=1}^n{I}_P(M,U^{i-1},Z^{i-1},Z_{i+1}^n;U_i|Z_i)\hfill \\ \hfill & ={\sum }_{i=1}^n{I}_P(M,U^{i-1},Z^{i-1},Z_{i+1}^n,Y^{i-1},S^{i-1};U_i|Z_i)\hfill \\ \hfill & \ge {\sum }_{i=1}^n{I}_P(M,Z^{i-1},Z_{i+1}^n,Y^{i-1},S^{i-1};U_i|Z_i)\hfill \\ \hfill & ={\sum }_{i=1}^n{I}_P(W_i;U_i|Z_i)=n{I}_P(W_T;U_T|Z_T,T)\hfill \end{array}$$ (40)

$$\begin{array}{cc}\hfill & =n{I}_P(W_T,T;U_T|Z_T)\hfill \end{array}$$ (41)

$$\begin{array}{ccc}\hfill & =n{I}_P(W;U|Z).\hfill & \hfill \end{array}$$ (42)

Here, (39) follows since the sequences $(U^n,Z^n)$ are memoryless; (40) follows since $(Y^{i-1},S^{i-1})-(M,U^{i-1},Z^n)-U_i$ form a Markov chain; and, (41) follows from the fact that T is independent of all the other r.v.’s.

The equivocation of $S^n$ under the null hypothesis can be bounded as follows.

$$\begin{array}{cc}\hfill H\left(S^n\right|M,Y^n,Z^n,H=0)& ={\sum }_{i=1}^{n}H\left(S_i\right|M,S^{i-1},Y^n,Z^n,H=0)\hfill \end{array}$$

$$\begin{array}{cc}\hfill & \le {\sum }_{i=1}^{n}H\left(S_i\right|M,Y^{i-1},S^{i-1},Z^{i-1},Z_{i+1}^n,Y_i,Z_i,H=0)\hfill \\ \hfill & ={\sum }_{i=1}^{n}H\left(S_i\right|W_i,Y_i,Z_i,H=0)\hfill \\ \hfill & =nH\left(S_T\right|W_T,Y_T,Z_T,T,H=0)\hfill \end{array}$$ (43)

$$\begin{array}{ccc}\hfill & =n{H}_P\left(S\right|W,Y,Z),\hfill & \hfill \end{array}$$ (44)

where $P_{SUYZW}=P_{SUYZ}{P}_{W|U}$ for some conditional distribution $P_{W|U}$. In (43), we used the fact that conditioning reduces entropy.

Finally, we prove the upper bound on $\kappa$. For any encoding function $f_n$ and decision region ${\mathcal{A}}_n\subseteq \mathcal{M}\times {\mathcal{Y}}^n\times {\mathcal{Z}}^n$ for $H_0$ such that ${ϵ}_n\to 0$, we have,

$$\begin{array}{cc}\hfill D\left(P_{M{Y}^n{Z}^n}\left|\right|Q_{M{Y}^n{Z}^n}\right)& \ge P_{M{Y}^n{Z}^n}\left({\mathcal{A}}_n\right)log\left(\frac{P_{M{Y}^n{Z}^n}\left({\mathcal{A}}_n\right)}{Q_{M{Y}^n{Z}^n}\left({\mathcal{A}}_n\right)}\right)+P_{M{Y}^n{Z}^n}\left({\mathcal{A}}_n^c\right)log\left(\frac{P_{M{Y}^n{Z}^n}\left({\mathcal{A}}_n^c\right)}{Q_{M{Y}^n{Z}^n}\left({\mathcal{A}}_n^c\right)}\right)\hfill \\ \hfill & \ge -H\left({ϵ}_n\right)-(1-{ϵ}_n)log\left({\overline{\beta }}_n\left(f_n,{ϵ}_n\right)\right).\hfill \end{array}$$ (45)

Here, (45) follows from the log-sum inequality [60]. Thus,

$$\begin{array}{cc}\hfill \underset{n\to \infty }{lim\; sup}\frac{-log\left({\overline{\beta }}_n\left(f_n,{ϵ}_n\right)\right)}{n}& \le \underset{n\to \infty }{lim\; sup}\frac{1}{n}D\left(P_{M{Y}^n{Z}^n}\left|\right|Q_{M{Y}^n{Z}^n}\right)\hfill \end{array}$$

$$\begin{array}{cc}\hfill & =\underset{n\to \infty }{lim\; sup}\frac{1}{n}{I}_P(M;Y^n|Z^n)\hfill \end{array}$$ (46)

$$\begin{array}{ccc}\hfill & =H_P\left(Y\right|Z)-\underset{n\to \infty }{lim\; inf}\frac{1}{n}{H}_P\left(Y^n\right|M,Z^n),\hfill & \hfill \end{array}$$ (47)

where (46) follows since $Q_{M{Y}^n{Z}^n}=P_{M{Z}^n}{P}_{Y^n|Z^n}$. The last term can be single-letterized as follows:

$$\begin{array}{cc}\hfill H_P\left(Y^n\right|M,Z^n)& ={\sum }_{i=1}^n{H}_P\left(Y_i\right|Y^{i-1},M,Z^n)\hfill \\ \hfill & \ge {\sum }_{i=1}^n{H}_P\left(Y_i\right|Y^{i-1},S^{i-1},M,Z^n)\hfill \\ \hfill & ={\sum }_{i=1}^n{H}_P\left(Y_i\right|Z_i,W_i)\hfill \\ \hfill & =n{H}_P\left(Y_T\right|Z_T,W_T,T)\hfill \\ \hfill & =n{H}_P\left(Y\right|Z,W).\hfill & \hfill \end{array}$$ (48)

Substituting (48) in (47), we obtain

$$\begin{array}{c}\hfill \kappa \le I_P(Y;W|Z).\end{array}$$ (49)

Also, note that $(Z,Y)-U-W$ holds. To see this, note that $(U_i,Y_i,Z_i,S_i)$ are i.i.d across $i\in \left[n\right]$. Hence, any information in $W_i$ on $(Y_i,Z_i,S_i)$ is only through M as a function of $U_i$, and so given $U_i$, $W_i$ is independent of $(Y_i,Z_i,S_i)$. The above Markov chain then follows from the fact that T is independent of $(U^n,Y^n,Z^n,S^n,M)$. This completes the proof of the converse and the theorem. ☐

Next, we state the result for TACI with a distortion privacy constraint, where the distortion is measured using an arbitrary distortion measure $d(·,·)$. Let ${\Delta }_{min}:={min}_{\varphi (u,y,z)}{\mathbb{E}}_Q\left[d\left(S,\varphi (U,Y,Z)\right)\right]$.

Proposition 2.

For the HT given in (26), $(R,\kappa ,{\Delta }_0,{\Delta }_{min})\in {\mathcal{R}}_d$ if and only if there exist an auxiliary r.v. W and a deterministic function $\varphi :\mathcal{W}\times \mathcal{Y}\times \mathcal{Z}\to \widehat{\mathcal{S}}$ such that

$$\begin{array}{cc}\hfill R& \ge I_P(W;U|Z),\hfill \end{array}$$ (50)

$$\begin{array}{cc}\hfill \kappa & \le I_P(W;Y|Z),\hfill \end{array}$$ (51)

$$\begin{array}{cc}\hfill {\Delta }_0& \le \underset{\varphi (·,·,·)}{min}{\mathbb{E}}_P\left[d\left(S,\varphi (W,Y,Z)\right)\right],\hfill \end{array}$$ (52)

for some $P_{SUYZW}$ as defined in Proposition 1.

Proof. 

The proof of achievability follows from Theorem 2, similarly to the way Proposition 1 is obtained from Theorem 1. Hence, only differences will be highlighted. Similar to the inequality ${\Lambda }_{min}\le H_Q\left(S\right|U,Y,Z)$ in the proof of Proposition 1, we need to prove the inequality ${\Delta }_{min}\le {\mathbb{E}}_Q\left[d\left(S,\varphi (W,Y,Z)\right)\right]$, where $Q_{SUYZW}:=Q_{SUYZ}{P}_{W|U}$ for some conditional distribution $P_{W|U}$. This can be shown as follows:

$$\begin{array}{cc}\hfill & \underset{\varphi (·,·,·)}{min}{\mathbb{E}}_Q\left[d\left(S,\varphi (W,Y,Z)\right)\right]\hfill \\ \hfill & =\sum _{u,y,z}{Q}_{UYZ}(u,y,z)\sum _w{P}_{W|U}\left(w\right|u)\underset{\varphi (w,y,z)}{min}\sum _s{Q}_{S|UYZ}\left(s\right|u,y,z)d(s,\varphi (w,y,z))\hfill \\ \hfill & \ge \sum _{u,y,z}{Q}_{UYZ}(u,y,z)\sum _{w,s}{P}_{W|U}\left(w\right|u)Q_{S|UYZ}\left(s\right|u,y,z)d(s,{\varphi }^{*}(u,y,z))\hfill \\ \hfill & \ge \sum _{u,y,z}{Q}_{UYZ}(u,y,z)\underset{\varphi (u,y,z)}{min}\sum _{w,s}{P}_{W|U}\left(w\right|u)Q_{S|UYZ}\left(s\right|u,y,z)d(s,\varphi (u,y,z))\hfill \\ \hfill & =\sum _{u,y,z}{Q}_{UYZ}(u,y,z)\underset{\varphi (u,y,z)}{min}\sum _s{Q}_{S|UYZ}\left(s\right|u,y,z)d(s,\varphi (u,y,z))\hfill \\ \hfill & =\underset{\varphi (·,·,·)}{min}{\mathbb{E}}_Q\left[d\left(S,\varphi (U,Y,Z)\right)\right]:={\Delta }_{min},\hfill & \hfill \end{array}$$ (53)

where in (53), ${\varphi }^{*}(u,y,z)$ is chosen such that

$$\begin{array}{c}\hfill {\varphi }^{*}(u,y,z):=\underset{\varphi (w,y,z),w\in \mathcal{W}}{arg\; min}\sum _s{Q}_{S|UYZ}\left(s\right|u,y,z)d(s,\varphi (w,y,z)).\end{array}$$

Converse: Let $W=(W_T,T)$ denote the auxiliary r.v. defined in the converse of Proposition 1. Inequalities (50) and (51) follow similarly as obtained in Proposition 1. We prove (52). Defining ${\tilde{\varphi }}_n(M,Y^n,Z^n,S^{i-1},i):={\overline{\varphi }}_{i,n}(M,Y^n,Z^n,S^{i-1})$, we have

$$\begin{array}{cc}\hfill \underset{g_{i,n}^{\left(r\right)}}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=0\right]& =\underset{{\left\{{\tilde{\varphi }}_n(m,y^n,z^n,s^{i-1},i)\right\}}_{i=1}^n}{min}\mathbb{E}\left[\sum _{i=1}^{n}d\left(S_i,{\tilde{\varphi }}_n(M,Y^n,Z^n,S^{i-1},i)\right)|H=0\right]\hfill \\ \hfill & =\underset{{\left\{{\tilde{\varphi }}_n(·,·,·,·,·)\right\}}_{i=1}^n}{min}\mathbb{E}\left[\sum _{i=1}^{n}d\left(S_i,{\tilde{\varphi }}_n(W_i,Z_i,Y_i,Y_{i+1}^n,i)\right)|H=0\right]\hfill \\ \hfill & \le \underset{\left\{\varphi (w_i,z_i,y_i,i)\right\}}{min}\mathbb{E}\left[\sum _{i=1}^{n}d\left(S_i,\varphi (W_i,Z_i,Y_i,i)\right)|H=0\right]\hfill \\ \hfill & =n\underset{\left\{\varphi \right(·,·,·,·\left)\right\}}{min}\mathbb{E}\left[\mathbb{E}\left[d\left(S_T,\varphi (W_T,Z_T,Y_T,T)\right)|T\right]|H=0\right]\hfill \\ \hfill & =n\underset{\left\{\varphi \right(·,·,·,·\left)\right\}}{min}\mathbb{E}\left[d\left(S_T,\varphi (W_T,Z_T,Y_T,T)\right)|H=0\right]\hfill \\ \hfill & =n\underset{\left\{\varphi \right(w,z,y\left)\right\}}{min}\mathbb{E}\left[d\left(S,\varphi (W,Z,Y)\right)|H=0\right],\hfill & \hfill \end{array}$$ (54)

where (54) is due to (A1) (in Appendix B). Hence, any ${\Delta }_0$ satisfying (6) satisfies

$$\begin{array}{cc}\hfill {\Delta }_0& \le \underset{\left\{\varphi \right(w,z,y\left)\right\}}{min}{\mathbb{E}}_P\left[d\left(S,\varphi (W,Z,Y)\right)\right].\hfill \end{array}$$

This completes the proof of the converse and the theorem. ☐

A more general version of Propositions 1 and 2 is claimed in [66] as Theorems 7 and 8, respectively, in which a privacy constraint under the alternate hypothesis is also imposed. However, we have identified a mistake in the converse proof; and hence, a single-letter characterization for this general problem remains open.

To complete the single-letter characterization in Propositions 1 and 2, we bound the alphabet size of the auxiliary r.v. W in the following lemma, whose proof is given in Appendix E.

Lemma 5.

In Propositions 1 and 2, it suffices to consider auxiliary r.v.’s W such that $\left|\mathcal{W}\right|\le \left|\mathcal{U}\right|+2$.

The proof of Lemma 5 uses standard arguments based on the Fenchel–Eggleston–Carathéodory’s theorem and is given in Appendix E.

Remark 1.

When $Q_{S|UYZ}=Q_{S|YZ}$, a tight single-letter characterization of ${\mathcal{R}}_e$ and ${\mathcal{R}}_d$ exists even if the privacy constraint is active under the alternate hypothesis. This is due to the fact that given $Y^n$ and $Z^n$, M is independent of $S^n$ under the alternate hypothesis. In this case, $(R,\kappa ,{\Lambda }_0,{\Lambda }_1)\in {\mathcal{R}}_e$ if and only if there exists an auxiliary r.v. W, such that $(Z,Y,S)-U-W$, and

$$\begin{array}{cc}\hfill \kappa & \le I_P(W;Y|Z),\hfill \end{array}$$ (55)

$$\begin{array}{cc}\hfill R& \ge I_P(W;U|Z),\hfill \end{array}$$ (56)

$$\begin{array}{cc}\hfill {\Lambda }_0& \le H_P\left(S\right|W,Z,Y),\hfill \end{array}$$ (57)

$$\begin{array}{cc}\hfill {\Lambda }_1& \le H_Q\left(S\right|Z,Y),\hfill \end{array}$$ (58)

for some $P_{SUYZW}$ as in Proposition 1. Similarly, we have that $(R,\kappa ,{\Delta }_0,{\Delta }_1)\in {\mathcal{R}}_d$ if and only if there exist an auxiliary r.v. W and a deterministic function $\varphi :\mathcal{W}\times \mathcal{Y}\times \mathcal{Z}\to \widehat{\mathcal{S}}$ such that (55) and (56),

$$\begin{array}{cc}\hfill {\Delta }_0& \le \underset{\varphi (·,·,·)}{min}{\mathbb{E}}_P\left[d\left(S,\varphi (W,Y,Z)\right)\right],\hfill \end{array}$$ (59)

$$\begin{array}{cc}\hfill {\Delta }_1& \le \underset{\varphi (·,·,·)}{min}{\mathbb{E}}_Q\left[d\left(S,\varphi (Y,Z)\right)\right],\hfill \end{array}$$ (60)

are satisfied for some $P_{SUYZW}$ as in Proposition 1.

The computation of the trade-off given in Proposition 1 is challenging despite the cardinality bound on the auxiliary r.v. W provided by Lemma 5, as closed form solutions do not exist in general. To see this, note that the inequality constraints defining ${\mathcal{R}}_e$ are not convex in general, and hence even computing specific points in the trade-off could be a hard problem. This is evident from the fact that in the absence of the privacy constraint in Proposition 1, i.e., (30), computing the maximum error exponent for a given rate constraint is equivalent to the information bottleneck problem [67], which is known to be a hard non-convex optimization problem. Also, the complexity of brute force search is exponential in $\left|\mathcal{U}\right|$, and hence intractable for large values of $\left|\mathcal{U}\right|$. Below we provide an example which can be solved in closed form and hence computed easily.

Example 1.

Let $\mathcal{V}=\mathcal{U}=\mathcal{S}=\{0,1\}$, $V=Y$, $Z=$ constant, $V-S-U$, $P_U\left(0\right)=Q_U\left(0\right)=0.5$, $P_{S|U}\left(0\right|0)=P_{S|U}\left(1\right|1)=Q_{S|U}\left(0\right|0)=Q_{S|U}\left(1\right|1)=1-q$, $P_{V|S}\left(0\right|0)=P_{V|S}\left(1\right|1)=1-p$ and $Q_{V|S}\left(0\right|0)=Q_{V|S}\left(1\right|1)=0.5$. Then, $(R,\kappa ,{\Lambda }_0,0)\in {\mathcal{R}}_e$ if there exists $r\in [0,0.5]$ such that

$$\begin{array}{cc}\hfill R& \ge 1-h_b\left(r\right),\hfill \end{array}$$ (61)

$$\begin{array}{cc}\hfill \kappa & \le 1-h_b((r*q)*p),\hfill \end{array}$$ (62)

$$\begin{array}{cc}\hfill {\Lambda }_0& \le h_b\left(p\right)+h_b(q*r)-h_b(p*(q*r)),\hfill \end{array}$$ (63)

where for $a,b\in \mathbb{R}$, $a*b:=(1-a)·b+(1-b)·a$, and $h_b:[0,1]\mapsto [0,1]$ is the binary entropy function given by

$$\begin{array}{c}\hfill h_b\left(t\right)=-(1-t)log(1-t)-tlog\left(t\right).\end{array}$$

The above characterization (Numerical computation shows that the characterization given in (61)–(63) is exact even when $q\in (0,1)$.) is exact for $q=0$, i.e., $(R,\kappa ,{\Lambda }_0,0)\in {\mathcal{R}}_e$ only if there exists $r\in [0,0.5]$ such that (61)–(63) are satisfied.

Proof. 

Taking $\mathcal{W}=\{0,1\}$, and $P_{W|U}\left(0\right|0)=P_{W|U}\left(1\right|1)=1-r$, the constraints defining the trade-off given in Proposition 1 simplifies to

$$\begin{array}{cc}\hfill I_P(U;W)& =1-h_b\left(r\right),\hfill \\ \hfill I_P(V;W)& =1-h_b((r*q)*p),\hfill \\ \hfill H_P\left(S\right|V,W)& =H_P\left(S\right|W)-I_P(S;V|W)\hfill \\ \hfill & =H_P\left(S\right|W)+H_P\left(V\right|S)-H_P\left(V\right|W)\hfill \\ \hfill & =h_b(r*q)+h_b\left(p\right)-h_b(p*(q*r)).\hfill & \hfill \end{array}$$

On the other hand, if $q=0$, note that $S=U$. Hence, the same constraints can be bounded as follows:

$$\begin{array}{cc}\hfill I_P(U;W)& =1-H_P\left(U\right|W),\hfill \end{array}$$

$$\begin{array}{cc}\hfill I_P(V;W)& =1-H_P\left(V\right|W)\le 1-h_b\left(h_b^{-1}\left(H\left(U\right|W)\right)*p\right),\hfill \\ \hfill H_P\left(U\right|V,W)& =H_P\left(U\right|W)+H_P\left(V\right|U)-H_P\left(V\right|W)\hfill \end{array}$$ (64)

$$\begin{array}{ccc}\hfill & \le h_b\left(p\right)+H_P\left(U\right|W)-h_b\left(h_b^{-1}\left(H_P\left(U\right|W)\right)*p\right),\hfill & \hfill \end{array}$$ (65)

where $h_b^{-1}:[0,1]\mapsto [0,0.5]$ is the inverse of the binary entropy function. Here, the inequality in (64) and (65) follows by an application of Mrs Gerber’s lemma [68], since $V=U\oplus N_p$ under the null hypothesis and $N_p\sim Ber\left(p\right)$ is independent of U and W. Also, ${\Lambda }_{min}=0$ since $S=U$. Noting that $H_P\left(U\right|W)\in [0,1]$, and defining $r:=h_b^{-1}\left(H_P\left(U\right|W)\right)\in [0,0.5]$, the result follows. ☐

Figure 2 depicts the curve $\left(1-h_b\left(r\right),1-h_b(p*(q*r)),h_b\left(p\right)+h_b(r*q)-h_b(p*(r*q))\right)$ for $q=0$ and $p\in \{0.15,0.25,0.35\}$, as r is varied in the range $[0,0.5]$. The projection of this curve on the $R-\kappa$ and $\kappa -{\Lambda }_0$ plane is shown in Figure 3a,b, respectively, for $q\in \{0,0.1\}$ and the same values of p. As expected, the error exponent $\kappa$ increases with rate R while the equivocation ${\Lambda }_0$ decreases with $\kappa$ at the boundary of ${\mathcal{R}}_e$.

Figure 2.

$(R,\kappa ,{\Lambda }_0)$ trade-off at the boundary of ${\mathcal{R}}_e$ in Example 1 (Axes units are in bits)

Figure 3.

Projections of Figure 2 in the $R-\kappa$ plane and $\kappa -{\Lambda }_0$ plane

Proposition 1 (resp. Proposition 2) provide a characterization of ${\mathcal{R}}_e$ (resp. ${\mathcal{R}}_d$) under the condition of vanishing type I error probability constraint. Consequently, the converse part of these results are known as weak converse results in the context of HT. In the next subsection, we establish the optimal error exponent-privacy trade-off for the special case of zero-rate compression. This trade-off is independent of the type I error probability constraint $ϵ\in (0,1)$, and hence known as a strong converse result.

4.2. Zero-Rate Compression

Assume the following zero-rate constraint on the communication between the observer and the detector,

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}\frac{log\left(\right|\mathcal{M}\left|\right)}{n}=0.\end{array}$$ (66)

Please note that (66) does not imply that $\left|\mathcal{M}\right|=0$, i.e., nothing can be transmitted, but that the message set cardinality can grow at most sub-exponentially in n. Such a scenario is motivated practically by low power or low bandwidth constrained applications in which communication is costly. Propositions 3 and 4 stated below provide an optimal single-letter characterization of ${\mathcal{R}}_d\left(ϵ\right)$ and ${\mathcal{R}}_e\left(ϵ\right)$ in this case. While the coding schemes in the achievability part of these results are inspired from that in [6], the analysis of privacy achieved at the detector is new. Lemma 4 serves as a crucial tool for this purpose. We next state the results. Let

$$\begin{array}{cc}\hfill {\Delta }_0^{max}& :=\underset{{\varphi }^{\prime }(·)}{min}{\mathbb{E}}_P\left[d\left(S,{\varphi }^{\prime }\left(V\right)\right)\right],\hfill \end{array}$$ (67a)

$$\begin{array}{cc}\hfill \mathrm{and} {\Delta }_1^{max}& :=\underset{{\varphi }^{\prime }(·)}{min}{\mathbb{E}}_Q\left[d\left(S,{\varphi }^{\prime }\left(V\right)\right)\right].\hfill \end{array}$$ (67b)

Proposition 3.

For $ϵ\in (0,1)$, $(0,\kappa ,{\Delta }_0,{\Delta }_1)\in {\mathcal{R}}_d\left(ϵ\right)$ if and only if it satisfies,

$$\begin{array}{cc}\hfill \kappa & \le \underset{P_{\tilde{U}\tilde{V}}\in {\mathcal{L}}^{\prime }(P_U,P_V)}{min}D(P_{\tilde{U}\tilde{V}}\left|\right|Q_{UV}),\hfill \end{array}$$ (68)

$$\begin{array}{cc}\hfill {\Delta }_0& \le {\Delta }_0^{max},\hfill \end{array}$$ (69)

$$\begin{array}{cc}\hfill {\Delta }_1& \le {\Delta }_1^{max},\hfill \end{array}$$ (70)

where ${\varphi }^{\prime }:\mathcal{V}\to \widehat{S}$ is a deterministic function and

$$\begin{array}{cc}\hfill {\mathcal{L}}^{\prime }(P_U,P_V)& =\{P_{\tilde{U}\tilde{V}}\in \mathcal{P}(\mathcal{U}\times \mathcal{V}):P_{\tilde{U}}=P_U,P_{\tilde{V}}=P_V\}.\hfill \end{array}$$

Proof. 

First, we prove that $(0,\kappa ,{\Delta }_0,{\Delta }_1)$ satisfying (68)–(70) is achievable. While the encoding and decoding scheme is the same as that in [6], we mention it for the sake of completeness.

Encoding: The observer sends the message $M=1$ if $U^n\in {\mathcal{T}}_{{\left[P_U\right]}_{\delta }}^n$, $\delta >0$, and $M=0$ otherwise.

Decoding: The detector declares $\widehat{H}=0$ if $M=1$ and $V^n\in {\mathcal{T}}_{{\left[P_V\right]}_{\delta }}^n$, $\delta >0$. Otherwise, $\widehat{H}=1$ is declared.

We analyze the type I and type II error probabilities for the above scheme. Please note that for any $\delta >0$, the weak law of large numbers implies that

$$\begin{array}{c}\hfill \mathbb{P}\left(U^n\in {\mathcal{T}}_{{\left[P_U\right]}_{\delta }}^n\cap V^n\in {\mathcal{T}}_{{\left[P_V\right]}_{\delta }}^n\left)\right|H=0\right)=\mathbb{P}\left(M=1\cap V^n\in {\mathcal{T}}_{{\left[P_V\right]}_{\delta }}^n\left)\right|H=0\right)\stackrel{\left(n\right)}{\to }1.\end{array}$$

Hence, the type I error probability tends to zero, asymptotically. The type II error probability can be written as follows:

$$\begin{array}{cc}\hfill {\beta }_n(f_n,g_n)& =\mathbb{P}(U^n\in {\mathcal{T}}_{{\left[P_U\right]}_{\delta }}^n\cap V^n\in {\mathcal{T}}_{{\left[P_V\right]}_{\delta }}^n)|H=1)\hfill \\ \hfill & =\sum _{\begin{array}{c}{u}^n\in {\mathcal{T}}_{{\left[P_U\right]}_{\delta }}^n,\\ v^n\in {\mathcal{T}}_{{\left[P_V\right]}_{\delta }}\end{array}}{Q}_{U^n{V}^n}(u^n,v^n)\le {(n+1)}^{\left|\mathcal{U}\right|\left|\mathcal{V}\right|}{e}^{-n({\kappa }^{*}-O\left(\delta \right))}=e^{-n\left({\kappa }^{*}-\frac{\left|\mathcal{U}\right|\left|\mathcal{V}\right|log(n+1)}{n}-O\left(\delta \right)\right)},\hfill & \hfill \end{array}$$

where

$$\begin{array}{c}\hfill {\kappa }^{*}=\underset{P_{\tilde{U}\tilde{V}}\in {\mathcal{L}}^{\prime }(P_U,P_V)}{min}D(P_{\tilde{U}\tilde{V}}\left|\right|Q_{UV}).\end{array}$$

Next, we lower bound the average distortion for $S^n$ achieved by this scheme at the detector. Defining

$$\begin{array}{cc}\hfill \Pi (U^n,\delta ,P_U)& :=\mathbb{1}\left(U^n\notin {\mathcal{T}}_{{\left[P_U\right]}_{\delta }}^n\right),\hfill \end{array}$$ (71)

$$\begin{array}{cc}\hfill {\rho }_n^{\left(0\right)}\left(\delta \right)& :=∥P_{S^n{V}^n}(·)-P_{S^n{V}^n|\Pi (U^n,\delta ,P_U)}(·|0)∥,,\hfill \end{array}$$ (72)

$$\begin{array}{cc}\hfill {\rho }_n^{\left(1\right)}\left(\delta \right)& :=∥Q_{S^n{V}^n}(·)-Q_{S^n{V}^n|\Pi (U^n,\delta ,P_U)}(·|1)∥,\hfill \\ \hfill {\varphi }_n^{\prime }\left(v^n\right)& :=({\varphi }^{\prime }\left(v_1\right),\cdots ,{\varphi }^{\prime }\left(v_n\right)),\hfill \end{array}$$ (73)

we can write

$$\begin{array}{cc}\hfill & |\underset{{\left\{{\overline{\varphi }}_i(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=0\right]-n\underset{{\varphi }^{\prime }\left(v\right)}{min}{\mathbb{E}}_P\left[d\left(S,{\varphi }^{\prime }\left(V\right)\right)\right]|\hfill \\ \hfill & =|\underset{{\left\{{\overline{\varphi }}_i(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=0\right]-\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|H=0\right]|\hfill \\ \hfill & \le |\underset{{\left\{{\overline{\varphi }}_i(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=0\right]-\mathbb{P}\left(M=1|H=0\right)\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|M=1,H=0\right]|\hfill \\ \hfill & +\mathbb{P}\left(M=0|H=0\right)\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|M=0,H=0\right]\hfill \\ \hfill & \le |\underset{{\left\{{\overline{\varphi }}_i(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=0\right]-\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|M=1,H=0\right]|\hfill \\ \hfill & +\mathbb{P}\left(M=0|H=0\right)[\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|M=1,H=0\right]\hfill \\ \hfill & +\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|M=0,H=0\right]]\hfill \\ \hfill & =|\underset{{\left\{{\overline{\varphi }}_i(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=0\right]-\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|\Pi (U^n,\delta ,P_U)=0,H=0\right]|\hfill \\ \hfill & +\mathbb{P}\left(\Pi (U^n,\delta ,P_U)=1|H=0\right)[\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|M=1,H=0\right]\hfill \\ \hfill & +\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|M=0,H=0\right]]\hfill \end{array}$$ (74)

$$\begin{array}{cc}\hfill & \le n{D}_m{\rho }_n^{\left(0\right)}\left(\delta \right)+2e^{-n\Omega \left(\delta \right)}n{D}_m\hfill \end{array}$$ (75)

$$\begin{array}{ccc}\hfill & \stackrel{\left(n\right)}{\to }0,\hfill & \hfill \end{array}$$ (76)

where (74) is since $\Pi (U^n,\delta ,P_U)=1-M$ with probability one by the encoding scheme; (75) follows from

$$\begin{array}{c}\hfill \mathbb{P}\left(\Pi (U^n,\delta ,P_U)=1|H=0\right)=\mathbb{P}\left(U^n\notin {\mathcal{T}}_{{\left[P_U\right]}_{\delta }}^n|H=0\right)\le e^{-n\Omega \left(\delta \right)}\end{array}$$ (77)

and ([43], Property 2(b)); and, (76) is due to (17). Similarly, it can be shown using (16) that if $Q_U=P_U$, then

$$\begin{array}{c}\hfill |\underset{{\left\{{\overline{\varphi }}_{i,n}(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=1\right]-n\underset{{\varphi }^{\prime }\left(v\right)}{min}{\mathbb{E}}_Q\left[d\left(S,{\varphi }^{\prime }\left(V\right)\right)\right]|\stackrel{\left(n\right)}{\to }0.\end{array}$$ (78)

On the other hand, if $Q_U\ne P_U$ and $\delta$ is small enough, we have

$$\begin{array}{c}\hfill \mathbb{P}\left(M=0|H=1\right)=\mathbb{P}\left(\Pi (U^n,\delta ,P_U)=1|H=1\right)\ge 1-e^{-n\left(D\right(P_U\left|\right|Q_U)-O\left(\delta \right))}\stackrel{\left(n\right)}{\to }1.\end{array}$$ (79)

Hence, we can write for $\delta$ small enough,

$$\begin{array}{cc}\hfill & |\underset{{\left\{{\overline{\varphi }}_i(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=1\right]-n\underset{{\varphi }^{\prime }\left(v\right)}{min}{\mathbb{E}}_Q\left[d\left(S,{\varphi }^{\prime }\left(V\right)\right)\right]|\hfill \\ \hfill & =|\underset{{\left\{{\overline{\varphi }}_i(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=1\right]-\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|H=1\right]|\hfill \\ \hfill & \le |\underset{{\left\{{\overline{\varphi }}_i(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=1\right]-\mathbb{P}\left(M=0|H=0\right)\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|M=0,H=1\right]|\hfill \\ \hfill & +\mathbb{P}\left(M=1|H=1\right)\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|M=1,H=1\right]\hfill \\ \hfill & \le |\underset{{\left\{{\overline{\varphi }}_i(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=1\right]-\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|M=0,H=1\right]|\hfill \\ \hfill & +\mathbb{P}\left(M=1|H=1\right)[\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|M=1,H=1\right]\hfill \\ \hfill & +\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|M=0,H=1\right]]\hfill \\ \hfill & =|\underset{{\left\{{\overline{\varphi }}_i(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=1\right]-\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|\Pi (U^n,\delta ,P_U)=1,H=1\right]|\hfill \\ \hfill & +\mathbb{P}\left(\Pi (U^n,\delta ,P_U)=0|H=1\right)[\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|M=1,H=1\right]\hfill \\ \hfill & +\underset{{\varphi }_n^{\prime }\left(v^n\right)}{min}\mathbb{E}\left[d\left(S^n,{\varphi }_n^{\prime }\left(V^n\right)\right)|M=0,H=1\right]]\hfill \end{array}$$ (80)

$$\begin{array}{cc}\hfill & \le n{D}_m{\rho }_n^{\left(1\right)}\left(\delta \right)+2e^{-n\left(D\right(P_U\left|\right|Q_U)-O\left(\delta \right))}n{D}_m\hfill \end{array}$$ (81)

$$\begin{array}{ccc}\hfill & \stackrel{\left(n\right)}{\to }0,\hfill & \hfill \end{array}$$ (82)

where (80) is since $\Pi (U^n,\delta ,P_U)=1-M$ with probability one; (81) is due to (79) and ([43], Property 2(b)); and, (82) follows from (15). This completes the proof of the achievability.

We next prove the converse. Please note that by the strong converse result in [8], the right hand side (R.H.S) of (68) is an upper bound on the achievable error exponent for all $ϵ\in (0,1)$ even without a privacy constraint (hence, also with a privacy constraint). Also,

$$\begin{array}{cc}\hfill \underset{g_{i,n}^{\left(r\right)}}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=0\right]& \le \underset{{\left\{{\varphi }^{\prime }\left(v_i\right)\right\}}_{i=1}^n}{min}\sum _{i=1}^n{\mathbb{E}}_{P_{S_i{V}_i}}\left[d\left(S_i,{\varphi }^{\prime }\left(V_i\right)\right)\right]\hfill \\ \hfill & =n\underset{\left\{{\varphi }^{\prime }\left(v\right)\right\}}{min}{\mathbb{E}}_P\left[d(S,{\varphi }^{\prime }\left(V\right))\right].\hfill & \hfill \end{array}$$ (83)

Here, (83) follows from the fact that the detector can always reconstruct ${\widehat{S}}_i$ as a function of $V_i$ for $i\in \left[n\right]$. Similarly,

$$\begin{array}{cc}\hfill & \underset{g_{i,n}^{\left(r\right)}}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=1\right]\le n\underset{\left\{{\varphi }^{\prime }\left(v\right)\right\}}{min}{\mathbb{E}}_Q\left[d(S,{\varphi }^{\prime }\left(V\right))\right].\hfill \end{array}$$

Hence, any achievable ${\Lambda }_0$ and ${\Lambda }_1$ must satisfy (69) and (70), respectively. This completes the proof. ☐

The following Proposition is the analogous result to Proposition 3 when the privacy measure is equivocation.

Proposition 4.

For $ϵ\in (0,1)$, $(0,\kappa ,{\Lambda }_0,{\Lambda }_1)\in {\mathcal{R}}_e\left(ϵ\right)$ if and only if it satisfies (68) and

$$\begin{array}{cc}\hfill {\Lambda }_0& \le H_P\left(S\right|V),\hfill \end{array}$$ (84)

$$\begin{array}{cc}\hfill {\Lambda }_1& \le H_Q\left(S\right|V).\hfill \end{array}$$ (85)

Proof. 

For proving the achievability part, the encoding and decoding scheme is the same as in Proposition 3. Hence, the analysis of the error exponent given in Proposition 3 holds. To lower bound the equivocation of $S^n$ at the detector, defining $\Pi (U^n,\delta ,P_U)$, ${\rho }_n^{\left(0\right)}\left(\delta \right)$ and ${\rho }_n^{\left(1\right)}\left(\delta \right)$ as in (71)–(73), we can write

$$\begin{array}{cc}\hfill & |n{H}_P\left(S\right|V)-H\left(S^n\right|M,V^n,H=0)|\hfill \\ \hfill & =|H\left(S^n\right|V^n,H=0)-H\left(S^n\right|M,V^n,H=0)|\hfill \\ \hfill & \le |H(S^n,V^n|H=0)-H(S^n,V^n|M,H=0)|\hfill \\ \hfill & \le |H(S^n,V^n|H=0)-\mathbb{P}\left(M=1|H=0\right)H(S^n,V^n|M=1,H=0)|\hfill \\ \hfill & +\mathbb{P}\left(M=0|H=0\right)H(S^n,V^n|M=0,H=0)\hfill \\ \hfill & \le |H(S^n,V^n|H=0)-H(S^n,V^n|M=1,H=0)|\hfill \\ \hfill & +\mathbb{P}\left(M=0|H=0\right)\left(H(S^n,V^n|M=1,H=0)+H(S^n,V^n|M=0,H=0)\right)\hfill \\ \hfill & \le |H(S^n,V^n|H=0)-H(S^n,V^n|\Pi (U^n,\delta ,P_U)=0,H=0)|\hfill \\ \hfill & +\mathbb{P}\left(\Pi (U^n,\delta ,P_U)=1|H=0\right)\left(H(S^n,V^n|M=1,H=0)+H(S^n,V^n|M=0,H=0)\right)\hfill \\ \hfill & \stackrel{\left(n\right)}{\le }-2{\rho }_n^{\left(0\right)}\left(\delta \right)log\left(\frac{{\rho }_n^{\left(0\right)}\left(\delta \right)}{{\left|\mathcal{S}\right|}^n{\left|\mathcal{V}\right|}^n}\right)+2e^{-n\Omega \left(\delta \right)}log\left({\left|\mathcal{S}\right|}^n{\left|\mathcal{V}\right|}^n\right)\hfill \end{array}$$ (86)

$$\begin{array}{ccc}\hfill & \stackrel{\left(n\right)}{\to }0,\hfill & \hfill \end{array}$$ (87)

where (86) follows due to Lemma 3, ([60], Lemma 2.12) and the fact that entropy of a r.v. is bounded by the logarithm of cardinality of its support; and, (87) follows from (17) in Lemma 4 since $\delta >0$. In a similar way, it can be shown using (16) that if $Q_U=P_U$, then

$$\begin{array}{cc}\hfill & |H\left(S^n\right|V^n,H=1)-H\left(S^n\right|M,V^n,H=1)|\stackrel{\left(n\right)}{\to }0.\hfill \end{array}$$ (88)

On the other hand, if $Q_U\ne P_U$ and $\delta$ is small enough, we can write

$$\begin{array}{cc}\hfill & |n{H}_Q\left(S\right|V)-H\left(S^n\right|M,V^n,H=1)|\hfill \\ \hfill & =|H\left(S^n\right|V^n,H=1)-H\left(S^n\right|M,V^n,H=1)|\hfill \\ \hfill & \le |H(S^n,V^n|H=1)-H(S^n,V^n|M,H=1)|\hfill \\ \hfill & \le |H(S^n,V^n|H=1)-H(S^n,V^n|M=0,H=1)|\hfill \\ \hfill & +\mathbb{P}\left(\Pi (U^n,\delta ,P_U)=0|H=1\right)\left(H(S^n,V^n|M=0,H=1)+H(S^n,V^n|M=1,H=1)\right)\hfill \\ \hfill & \le -2{\rho }_n^{\left(1\right)}\left(\delta \right)log\left(\frac{{\rho }_n^{\left(1\right)}\left(\delta \right)}{{\left|\mathcal{S}\right|}^n{\left|\mathcal{V}\right|}^n}\right)+2e^{-n\left(D\right(P_U\left|\right|Q_U)-O\left(\delta \right))}log\left({\left|\mathcal{S}\right|}^n{\left|\mathcal{V}\right|}^n\right),\hfill & \hfill \end{array}$$ (89)

where (89) follows from Lemma 3 and (79). It follows from (15) in Lemma 4 that for $\delta >0$ sufficiently small, ${\rho }_n^{\left(1\right)}\left(\delta \right)\le e^{-n\overline{\delta }}$ for some $\overline{\delta }>0$, thus implying that the R.H.S. of (89) tends to zero. This completes the proof of achievability.

The converse follows from the results in [6,8] that the R.H.S of (68) is the optimal error exponent achievable for all values of $ϵ\in (0,1)$ even when there is no privacy constraint, and the following inequality

$$\begin{array}{c}\hfill H\left(S^n\right|M,V^n,H=j)\le H\left(S^n\right|V^n,H=j),j=0,1.\end{array}$$ (90)

This concludes the proof of the Proposition. ☐

In Section 2.2, we mentioned that it is possible to achieve a positive error exponent with perfect privacy in our model. Here, we provide an example of TAI with an equivocation privacy constraint under both hypothesis, and show that perfect privacy is possible. Recall that TAI is a special case of TACI, in which $Z=$ constant, and hence, the null and alternate hypothesis are given by

$$\begin{array}{cc}\hfill & H_0:(U^n,Y^n)\sim \prod _{i=1}^n{P}_{UY},\hfill \\ \hfill \mathrm{and} & H_1:(U^n,Y^n)\sim \prod _{i=1}^n{P}_U{P}_Y.\hfill \end{array}$$

Example 2.

Let $\mathcal{S}=\mathcal{U}=\{0,1,2,3\}$, $\mathcal{Y}=\{0,1\}$,

$$P_{SU}=0.125·\left[\begin{array}{cccc}1& 1& 0& 0\\ 1& 1& 0& 0\\ 0& 0& 1& 1\\ 0& 0& 1& 1\end{array}\right],P_{Y|U}=\left[\begin{array}{cc}1& 0\\ 0& 1\\ 1& 0\\ 0& 1\end{array}\right],$$

$P_{SUY}:=P_{SU}{P}_{Y|U}$ and $Q_{SUY}:=P_{SU}{P}_Y$, where $P_Y={\sum }_{u\in \mathcal{U}}{P}_U\left(u\right)P_{Y|U}\left(y\right|u)$. Then, we have $H_Q\left(S\right|Y)=H_P\left(S\right)=H_P\left(U\right)=2$ bits. Also, noting that under the null hypothesis, $Y=Umod2$, $H_P\left(S\right|Y)=2$ bits. It follows from the inner bound given by Equations (31)–(34), and, (37) and (38) that $(R,\kappa ,{\Lambda }_0,{\Lambda }_1)\in {\mathcal{R}}_e\left(ϵ\right)$, $ϵ\in (0,1)$ if

$$\begin{array}{cc}\hfill R& \ge I_P(W;U),\hfill \\ \hfill \kappa & \le I_P(W;Y),\hfill \\ \hfill {\Lambda }_0& \le H_P\left(S\right|W,Y),\hfill \\ \hfill {\Lambda }_1& \le H_Q\left(S\right|W,Y)=H_Q\left(S\right|W),\hfill \end{array}$$

where $P_{SUYW}:=P_{SUY}{P}_{W|U}$ and $Q_{SUYW}:=Q_{SUY}{P}_{W|U}$ for some conditional distribution $P_{W|U}$. If we set $W:=Umod2$, then we have $I_P(U;W)=1$ bit, $I_P(Y;W)=H_P\left(Y\right)=1$ bit, $H_P\left(S\right|W,Y)=H_P\left(S\right|Y)=2$ bits, and $H_Q\left(S\right|W)=H_P\left(S\right|Y)=2$ bits. Thus, by revealing only W to the detector, it is possible to achieve a positive error exponent while ensuring maximum privacy under both the null and alternate hypothesis, i.e., the tuple $(1,1,2,2)\in {\mathcal{R}}_e\left(ϵ\right)$, $\forall ϵ\in (0,1)$.

5. A Counter-Example to the Strong Converse

Ahlswede and Csiszár obtained a strong converse result for the DHT problem without a privacy constraint in [5], where they showed that for any positive rate R, the optimal achievable error exponent is independent of the type I error probability constraint $ϵ$. Here, we explore whether a similar result holds in our model, in which an additional privacy constraint is imposed. We will show through a counter-example that this is not the case in general. The basic idea used in the counter-example is a “time-sharing” argument which is used to construct from a given coding scheme that achieves the optimal rate-error exponent-equivocation trade-off under a vanishing type I error probability constraint, a new coding scheme that satisfies the given type I error probability constraint ${ϵ}^{*}$ and the same error exponent as before, yet achieves a higher equivocation for $S^n$ at the detector. This concept has been used previously in other contexts, e.g., in the characterization of the first-order maximal channel coding rate of additive white gaussian noise (AWGN) channel in the finite block-length regime [69], and subsequently in the characterization of the second order maximal coding rate in the same setting [70]. However, we will provide a self-contained proof of the counter-example by using Lemma 4 for this purpose.

Assume that the joint distribution $P_{SUV}$ is such that $H_P\left(S\right|U,V)<H_P\left(S\right|V)$. Proving the strong converse amounts to showing that any $(R,\kappa ,{\Lambda }_0,{\Lambda }_1)\in {\mathcal{R}}_e\left(ϵ\right)$ for some $ϵ\in (0,1)$ also belongs to ${\mathcal{R}}_e$. Consider TAI problem with an equivocation privacy constraint, in which $R\ge H_P\left(U\right)$ and ${\Lambda }_1\le {\Lambda }_{min}$. Then, from the optimal single-letter characterization of ${\mathcal{R}}_e$ given in Proposition 1, it follows by taking $W=U$ that $(H_P\left(U\right),I_P(V;U),H_P\left(S\right|V,U),{\Lambda }_{min})\in {\mathcal{R}}_e$. Please note that $I_P(V;U)$ is the maximum error exponent achievable for any type I error probability constraint $ϵ\in (0,1)$, even when $U^n$ is observed directly at the detector. Thus, for vanishing type I error probability constraint $ϵ\to 0$ and $\kappa =I_P(V;U)$, the term $H_P\left(S\right|V,U)$ denotes the maximum achievable equivocation for $S^n$ under the null hypothesis. From the proof of Proposition 1, the coding scheme achieving this tuple is as follows:

1. Quantize $u^n$ to codewords in ${\mathcal{B}}_n=\{u^n\left(j\right)\in {\mathcal{T}}_{{\left[P_U\right]}_{\delta }}^n,j\in \left[e^{n(H_P\left(U\right)+\eta )}\right]\}$ and send the index of quantization to the detector, i.e., if $u^n\in {\mathcal{T}}_{{\left[P_U\right]}_{\delta }}^n$, send $M=j$, where j is the index of $u^n$ in ${\mathcal{B}}_n$. Else, send $M=0$.

2. At the detector, if $M=0$, declare $\widehat{H}=1$. Else, declare $\widehat{H}=0$ if $(u^n\left(M\right),v^n)\in {\mathcal{T}}_{{\left[P_{UV}\right]}_{{\delta }^{\prime }}}^n$ for some ${\delta }^{\prime }>\delta$, and $\widehat{H}=1$ otherwise.

The type I error probability of the above scheme tends to zero asymptotically with n. Now, for a fixed ${ϵ}^{*}>0$, consider a modification of this coding scheme as follows:

1. If $u^n\in {\mathcal{T}}_{{\left[P_U\right]}_{\delta }}^n$, send $M=j$ with probability $1-{ϵ}^{*}$, where j is the index of $u^n$ in ${\mathcal{B}}_n$, and with probability ${ϵ}^{*}$, send $M=0$. If $u^n\notin {\mathcal{T}}_{{\left[P_U\right]}_{\delta }}^n$, send $M=0$.

2. At the detector, if $M=0$, declare $\widehat{H}=1$. Else, declare $\widehat{H}=0$ if $u^n\left(M\right),v^n)\in {\mathcal{T}}_{{\left[P_{UV}\right]}_{{\delta }^{\prime }}}^n$ for some ${\delta }^{\prime }>\delta$, and $\widehat{H}=1$ otherwise.

It is easy to see that for this modified coding scheme, the type I error probability is asymptotically equal to ${ϵ}^{*}$, while the error exponent remains the same as $I(V;U)$ since the probability of declaring $\widehat{H}=0$ is decreased. Recalling that $\Pi (u^n,\delta ,P_U):=\mathbb{1}\left(u^n\notin {\mathcal{T}}_{{\left[P_U\right]}_{\delta }}^n\right)$, we also have

$$\begin{array}{cc}\hfill & \frac{1}{n}H\left(S^n|M,V^n,H=0\right)\hfill \\ \hfill & =(1-{\gamma }_n)(1-{ϵ}^{*})\frac{1}{n}H\left(S^n|U^n,V^n,\Pi (U^n,\delta ,P_U)=0,H=0\right)+(1-{\gamma }_n){ϵ}^{*}\hfill \\ \hfill & \frac{1}{n}H\left(S^n|M=0,V^n,\Pi (U^n,\delta ,P_U)=0,H=0\right)+{\gamma }_n\frac{1}{n}H\left(S^n|M=0,V^n,\Pi (U^n,\delta ,P_U)=1,H=0\right)\hfill \\ \hfill & \ge (1-{\gamma }_n)(1-{ϵ}^{*})\left(H_P\left(S|U,V\right)-{\gamma }_n^{\prime \prime }\right)+(1-{\gamma }_n){ϵ}^{*}\frac{1}{n}H\left(S^n|M=0,V^n,\Pi (U^n,\delta ,P_U)=0,H=0\right)\hfill \\ \hfill & +{\gamma }_n\frac{1}{n}H\left(S^n|M=0,V^n,\Pi (U^n,\delta ,P_U)=1,H=0\right)\hfill \\ \hfill & >(1-{\gamma }_n)(1-{ϵ}^{*})\left(H_P\left(S|U,V\right)-{\gamma }_n^{\prime \prime }\right)+(1-{\gamma }_n){ϵ}^{*}\left(H_P\left(S\right|U,V)-\frac{{\gamma }_n^{\prime }}{n}\right)\hfill \end{array}$$ (91)

$$\begin{array}{cc}\hfill & +{\gamma }_n\frac{1}{n}H\left(S^n|M=0,V^n,H=0,\Pi (U^n,\delta ,P_U)=1\right)\hfill \end{array}$$ (92)

$$\begin{array}{cc}\hfill & =(1-{\gamma }_n)(1-{ϵ}^{*})\left(H_P\left(S|U,V\right)-{\gamma }_n^{\prime \prime }\right)+(1-{\gamma }_n){ϵ}^{*}\left(H_P\left(S|U,V\right)-\frac{{\gamma }_n^{\prime }}{n}\right)+{\gamma }_n^{‴}\hfill \end{array}$$ (93)

$$\begin{array}{ccc}\hfill & =(1-{\gamma }_n)H_P\left(S|U,V\right)-{\overline{\gamma }}_n,\hfill & \hfill \end{array}$$ (94)

where ${\left\{{\gamma }_n^{\prime \prime }\right\}}_{n\in \mathbb{N}}$ denotes some sequence of positive numbers such that ${\gamma }_n^{\prime \prime }\stackrel{\left(n\right)}{\to }0$, and

$$\begin{array}{cc}\hfill & {\gamma }_n:=\mathbb{P}\left(U^n\notin {\mathcal{T}}_{{\left[P_U\right]}_{\delta }}^n|H=0\right)\le e^{-n\Omega \left(\delta \right)}\stackrel{\left(n\right)}{\to }0,\hfill \\ \hfill & {\gamma }_n^{\prime }:=-2{\rho }_n^{*}log\left(\frac{2{\rho }_n^{*}}{{\left|\mathcal{S}\right|}^n}\right),\hfill \end{array}$$ (95)

$$\begin{array}{cc}\hfill & {\rho }_n^{*}:=∥P_{S^n{V}^n|\Pi (U^n,\delta ,P_U),M}(·|0,0)-P_{S^n{V}^n}(·)∥=∥P_{S^n{V}^n|\Pi (U^n,\delta ,P_U)}(·|0)-P_{S^n{V}^n}(·)∥,\hfill \end{array}$$ (96)

$$\begin{array}{cc}\hfill & {\gamma }_n^{\prime \prime \prime }:=\frac{{\gamma }_n}{n}H\left(S^n\right|M=0,V^n,,H=0,\Pi (U^n,\delta ,P_U)=1)\stackrel{\left(n\right)}{\to }0,\hfill \\ \hfill & {\overline{\gamma }}_n:=(1-{\gamma }_n)(1-{ϵ}^{*}){\gamma }_n^{″}+(1-{\gamma }_n){ϵ}^{*}\frac{{\gamma }_n^{\prime }}{n}-{\gamma }_n^{‴}.\hfill \end{array}$$ (97)

Equation (91) follows similarly to the proof of Theorem 1 in [71]. Equation (92) is obtained as follows:

$$\begin{array}{cc}\hfill & \frac{1}{n}H\left(S^n|M=0,V^n,I_U(U^n,\delta )=0,H=0\right)\hfill \end{array}$$

$$\begin{array}{cc}\hfill & \ge \frac{1}{n}H\left(S^n|V^n,H=0\right)-\frac{{\gamma }_n^{\prime }}{n}\hfill \end{array}$$ (98)

$$\begin{array}{ccc}\hfill & >H_P\left(S\right|U,V)-\frac{{\gamma }_n^{\prime }}{n}.\hfill & \hfill \end{array}$$ (99)

Here, (98) is obtained by an application of Lemma 3; and (99) is due to the assumption that $H_P\left(S\right|U,V)<H_P\left(S\right|V)$.

It follows from Lemma 4 that ${\rho }_n^{*}\stackrel{\left(n\right)}{\to }0$, which in turn implies that

$$\begin{array}{c}\hfill \frac{{\gamma }_n^{\prime }}{n}\stackrel{\left(n\right)}{\to }0.\end{array}$$ (100)

From (95), (97) and (100), we have that ${\overline{\gamma }}_n\stackrel{\left(n\right)}{\to }0$. Hence, Equation (94) implies that $(H_P\left(U\right),I_P(V;U),{\Lambda }_0^{*},{\Lambda }_{min})\in {\mathcal{R}}_e\left({ϵ}^{*}\right)$ for some ${\Lambda }_0^{*}>H_P\left(S\right|U,V)$. Since $(H_P\left(U\right),I_P(V;U),{\Lambda }_0^{*},{\Lambda }_{min})\notin {\mathcal{R}}_e$, this implies that in general, the strong converse does not hold for HT with an equivocation privacy constraint. The same counter-example can be used in a similar manner to show that the strong converse does not hold for HT with an average distortion privacy constraint either.

6. Conclusions

We have studied the DHT problem with a privacy constraint, with equivocation and average distortion under a causal disclosure assumption as the measures of privacy. We have established a single-letter inner bound on the rate-error exponent-equivocation and rate-error exponent-distortion trade-offs. We have also obtained the optimal rate-error exponent-equivocation and rate-error exponent-distortion trade-offs for two special cases, when the communication rate over the channel is zero, and for TACI under a privacy constraint. It is interesting to note that the strong converse for DHT does not hold when there is an additional privacy constraint in the system. Extending these results to the case when the communication between the observer and detector takes place over a noisy communication channel is an interesting avenue for future research. Yet another important topic worth exploring is the trade-off between rate, error probability and privacy in the finite sample regime for the setting considered in this paper.

Abbreviations

The following abbreviations are used in this manuscript:

HT	Hypothesis testing
DHT	Distributed hypothesis testing
TACI	Testing against conditional independence
TAI	Testing against independence
DP	Differential privacy
KL	Kullback–Leibler
SHA	Shimokawa-Han-Amari

Appendix A. Proof of Lemma 1

Please note that for a stochastic detector, the type I and type II error probabilities are linear functions of $P_{\widehat{H}|M,V^n}$. As a result, for each fixed n and $f_n$, ${\alpha }_n\left(f_n,g_n\right)$ and ${\beta }_n\left(f_n,g_n\right)$ for a stochastic detector $g_n$ can be thought of as the type I and type II errors achieved by “time-sharing” among a finite number of deterministic detectors. To see this, consider some ordering on the elements of the set $\mathcal{M}\times {\mathcal{V}}^n$ and let ${\nu }_i:=P_{\widehat{H}|M,V^n}\left(0\right|i)$, $i\in [1:N]$, where i denotes the $i^{th}$ element of $\mathcal{M}\times {\mathcal{V}}^n$ and $N=|\mathcal{M}\times {\mathcal{V}}^n|$. Then, we can write

$$P_{\widehat{H}|M,V^n}=\left[\begin{array}{cc}{\nu }_1& 1-{\nu }_1\\ {\nu }_2& 1-{\nu }_2\\ ⋮& ⋮\\ {\nu }_N& 1-{\nu }_N\end{array}\right].$$

Then, it is easy to see that $P_{\widehat{H}|M,V^n}={\sum }_{i=1}^N{\nu }_i{I}_i$, where $I_i:=[e_{i}1-e_i]$ and $e_i$ is an N length vector with 1 at the $i^{th}$ component and 0 elsewhere. Now, suppose $({\alpha }_n^{\left(1\right)},{\beta }_n^{\left(1\right)})$ and $({\alpha }_n^{\left(2\right)},{\beta }_n^{\left(2\right)})$ denote the pair of type I and type II error probabilities achieved by deterministic detectors $g_n^{\left(1\right)}$ and $g_n^{\left(2\right)}$, respectively. Let ${\mathcal{A}}_{1,n}$ and ${\mathcal{A}}_{2,n}$ denote their corresponding acceptance regions for $H_0$. Let $g_n^{\left(\theta \right)}$ denote the stochastic detector formed by using $g_n^{\left(1\right)}$ and $g_n^{\left(2\right)}$ with probabilities $\theta$ and $1-\theta$, respectively. From the above-mentioned linearity property, it follows that $g_n^{\left(\theta \right)}$ achieves type I and type II error probabilities of ${\alpha }_n\left(f_n,g_n^{\left(\theta \right)}\right)=\theta {\alpha }_n^{\left(1\right)}+(1-\theta ){\alpha }_n^{\left(2\right)}$ and ${\beta }_n\left(f_n,g_n^{\left(\theta \right)}\right)=\theta {\beta }_n^{\left(1\right)}+(1-\theta ){\beta }_n^{\left(2\right)}$, respectively. Let $r\left(\theta \right)=min(\theta ,1-\theta )$. Then, for $\theta \in (0,1)$,

$$\begin{array}{c}\hfill -\frac{1}{n}log\left({\beta }_n\left(f_n,g_n^{\left(\theta \right)}\right)\right)\le min\left(-\frac{1}{n}log{\beta }_n^{\left(1\right)},-\frac{1}{n}log{\beta }_n^{\left(2\right)}\right)-\frac{1}{n}log\left(r\left(\theta \right)\right).\end{array}$$

Hence, either

$$\begin{array}{c}\hfill {\alpha }_n^{\left(1\right)}\le {\alpha }_n\left(f_n,g_n^{\left(\theta \right)}\right)and-\frac{1}{n}log{\beta }_n^{\left(1\right)}\ge -\frac{1}{n}log\left({\beta }_n\left(f_n,g_n^{\left(\theta \right)}\right)\right)+\frac{1}{n}log\left(r\left(\theta \right)\right),\end{array}$$

or

$$\begin{array}{c}\hfill {\alpha }_n^{\left(2\right)}\le {\alpha }_n\left(f_n,g_n^{\left(\theta \right)}\right)and-\frac{1}{n}log{\beta }_n^{\left(2\right)}\ge -\frac{1}{n}log\left({\beta }_n\left(f_n,g_n^{\left(\theta \right)}\right)\right)+\frac{1}{n}log\left(r\left(\theta \right)\right).\end{array}$$

Thus, since $\frac{1}{n}log\left(r\left(\theta \right)\right)\stackrel{\left(n\right)}{\to }0$, a stochastic detector does not offer any advantage over deterministic detectors in the trade-off between the error exponent and the type I error probability.

Appendix B. Proof of Lemma 2

Let ${\tilde{P}}_{S^n{U}^n{V}^{n}M{\widehat{S}}^n}^{({\mathcal{C}}_n,0)}=P_{S^n{U}^n{V}^{n}M}{\prod }_{i=1}^n{\tilde{P}}_{{\widehat{S}}_i|M,V^n,S^{i-1}}$ and ${\tilde{P}}_{S^n{U}^n{V}^{n}M{\widehat{S}}^n}^{({\mathcal{C}}_n,1)}=Q_{S^n{U}^n{V}^{n}M}{\prod }_{i=1}^n{\tilde{P}}_{{\widehat{S}}_i|M,V^n,S^{i-1}}$ denote the joint distribution of the r.v.’s $(S^n,U^n,V^n,M,{\widehat{S}}^n)$ under hypothesis $H_0$ and $H_1$, respectively, where ${\tilde{P}}_{{\widehat{S}}_i|M,V^n,S^{i-1}}$ denotes $g_{i,n}^{\left(r\right)}$ for $i\in \left[n\right]$. Then, we have

$$\begin{array}{cc}\hfill & \underset{g_{i,n}^{\left(r\right)}}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=j\right]\hfill \\ \hfill & =\underset{{\left\{{\tilde{P}}_{{\widehat{S}}_i|M,V^n,S^{i-1}}\right\}}_{i=1}^n}{min}{\mathbb{E}}_{{\tilde{P}}^{\left(j\right)}}\left[d\left(S^n,{\widehat{S}}^n\right)\right]\hfill \\ \hfill & =\underset{{\left\{{\tilde{P}}_{{\widehat{S}}_i|M,V^n,S^{i-1}}\right\}}_{i=1}^n}{min}\frac{1}{n}\sum _{i=1}^n{\mathbb{E}}_{{\tilde{P}}^{\left(j\right)}}\left[d\left(S_i,{\widehat{S}}_i\right)\right]\hfill \\ \hfill & =\frac{1}{n}\sum _{i=1}^n\sum _{(m,v^n,s^{i-1})}{\tilde{P}}_{M{V}^n{S}^{i-1}}^{\left(j\right)}(m,v^n,s^{i-1})\underset{{\tilde{P}}_{{\widehat{S}}_i|M,V^n,S^{i-1}}(·|m,v^n,s^{i-1})}{min}\sum _{{\widehat{s}}_i}{\tilde{P}}_{{\widehat{S}}_i|M,V^n,S^{i-1}}\left({\widehat{s}}_i\right|m,v^n,s^{i-1})\hfill \\ \hfill & {\mathbb{E}}_{{\tilde{P}}_{S_i|M,V^n,S^{i-1}}^{\left(j\right)}(·|m,v^n,s^{i-1})}\left[d\left(S_i,{\widehat{s}}_i\right)\right]\hfill \\ \hfill & =\frac{1}{n}\sum _{i=1}^n\sum _{m,v^n,s^{i-1}}{\tilde{P}}_{M{V}^n{S}^{i-1}}^{\left(j\right)}(m,v^n,s^{i-1}){\mathbb{E}}_{{\tilde{P}}_{S_i|M,V^n,S^{i-1}}^{\left(j\right)}(·|m,v^n,s^{i-1})}\left[d\left(S_i,{\varphi }_{ij}(m,v^n,s^{i-1})\right)\right],\hfill & \hfill \end{array}$$

where

$$\begin{array}{c}\hfill {\varphi }_{ij}(m,v^n,s^{i-1})=\underset{\widehat{s}\in \widehat{\mathcal{S}}}{arg\; min}{\mathbb{E}}_{{\tilde{P}}_{S_i|M,V^n,S^{i-1}}^{\left(j\right)}(·|m,v^n,s^{i-1})}\left[d(S_i,\widehat{s})\right].\end{array}$$

Continuing, we have

$$\begin{array}{cc}\hfill & \underset{g_{i,n}^{\left(r\right)}}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=j\right]\hfill \\ \hfill & =\frac{1}{n}\sum _{i=1}^n\sum _{m,v^n,s^{i-1}}{\tilde{P}}_{M{V}^n{S}^{i-1}}^{\left(j\right)}(m,v^n,s^{i-1})\underset{{\varphi }_i(m,v^n,s^{i-1})}{min}{\mathbb{E}}_{{\tilde{P}}_{S_i|M,V^n,S^{i-1}}^{\left(j\right)}(·|m,v^n,s^{i-1})}\left[d\left(S_i,{\varphi }_i(m,v^n,s^{i-1})\right)\right]\hfill \\ \hfill & =\underset{{\left\{{\varphi }_i(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}\frac{1}{n}\sum _{i=1}^n{\mathbb{E}}_{{\tilde{P}}^{\left(j\right)}}\left[d\left(S_i,{\varphi }_i(M,V^n,S^{i-1})\right)\right].\hfill & \hfill \end{array}$$ (A1)

This completes the proof.

Appendix C. Proof of Lemma 4

We will first prove (15). Fix $\delta >0$. For $\gamma >0$, define the following sets:

$$\begin{array}{cc}\hfill {\mathcal{B}}_{0,\gamma }^{\delta }& :=\left\{y^n\in {\mathcal{T}}_{{\left[P_Y\right]}_{\gamma }}^n:P_{Y^n}\left(y^n\right)\ge P_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|0)\right\},\hfill \end{array}$$ (A2)

$$\begin{array}{cc}\hfill {\mathcal{C}}_{0,\gamma }^{\delta }& :=\left\{y^n\in {\mathcal{T}}_{{\left[P_Y\right]}_{\gamma }}^n:P_{Y^n}\left(y^n\right)<P_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|0)\right\},\hfill \\ \hfill {\mathcal{B}}_{1,\gamma }^{\delta }& :=\left\{y^n\in {\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n:Q_{Y^n}\left(y^n\right)\ge Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|0)\right\},\hfill \\ \hfill {\mathcal{C}}_{1,\gamma }^{\delta }& :=\left\{y^n\in {\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n:Q_{Y^n}\left(y^n\right)<Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|0)\right\},\hfill \\ \hfill {\mathcal{B}}_{2,\gamma }^{\delta }& :=\left\{y^n\in {\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n:Q_{Y^n}\left(y^n\right)\ge Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)\right\},\hfill \\ \hfill {\mathcal{C}}_{2,\gamma }^{\delta }& :=\left\{y^n\in {\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n:Q_{Y^n}\left(y^n\right)<Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)\right\}.\hfill \end{array}$$ (A3)

Then, we can write

$$\begin{array}{cc}\hfill & \parallel Q_{Y^n}(·)-Q_{Y^n|\Pi (X^n,\delta ,P_X)}(·|1)\parallel \hfill \\ \hfill & =\frac{1}{2}\sum _{y^n}|Q_{Y^n}\left(y^n\right)-Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)|\hfill \\ \hfill & =\frac{1}{2}\sum _{y^n\notin {\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n}|Q_{Y^n}\left(y^n\right)-Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)|+\frac{1}{2}\sum _{y^n\in {\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n}|Q_{Y^n}\left(y^n\right)-Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)|\hfill \\ \hfill & \le \frac{1}{2}\sum _{y^n\notin {\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n}{Q}_{Y^n}\left(y^n\right)+Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)+\frac{1}{2}\sum _{y^n\in {\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n}|Q_{Y^n}\left(y^n\right)-Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)|.\hfill & \hfill \end{array}$$ (A4)

Next, note that

$$\begin{array}{cc}\hfill Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)& =Q_{Y^n}\left(y^n\right)\frac{Q_{\Pi (X^n,\delta ,P_X)|Y^n}\left(1\right|y^n)}{Q_{\Pi (X^n,\delta ,P_X)}\left(1\right)}\le \frac{Q_{Y^n}\left(y^n\right)}{Q_{\Pi (X^n,\delta ,P_X)}\left(1\right)}\le 2Q_{Y^n}\left(y^n\right),\hfill \end{array}$$ (A5)

for sufficiently large n (depending on $\left|\mathcal{X}\right|$), since $Q_{\Pi (X^n,\delta ,P_X)}\left(1\right)\stackrel{\left(n\right)}{\to }1$. Thus, for n large enough,

$$\begin{array}{cc}\hfill & \sum _{y^n\notin {\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n}{Q}_{Y^n}\left(y^n\right)+Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)\le 3\sum _{y^n\notin {\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n}{Q}_{Y^n}\left(y^n\right)\le e^{-n\Omega \left(\gamma \right)}.\hfill \end{array}$$ (A6)

We can bound the last term in (A4) as follows:

$$\begin{array}{cc}\hfill & \sum _{y^n\in {\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n}|Q_{Y^n}\left(y^n\right)-Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)|\hfill \\ \hfill & =\sum _{y^n\in {\mathcal{B}}_{2,\gamma }^{\delta }}{Q}_{Y^n}\left(y^n\right)-Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)+\sum _{y^n\in {\mathcal{C}}_{2,\gamma }^{\delta }}{Q}_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)-Q_{Y^n}\left(y^n\right)\hfill \\ \hfill & =\sum _{y^n\in {\mathcal{B}}_{2,\gamma }^{\delta }}{Q}_{Y^n}\left(y^n\right)-Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)+\sum _{y^n\in {\mathcal{C}}_{2,\gamma }^{\delta }}{Q}_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)-Q_{Y^n}\left(y^n\right)\hfill \\ \hfill & =\sum _{y^n\in {\mathcal{B}}_{2,\gamma }^{\delta }}{Q}_{Y^n}\left(y^n\right)\left(1-\frac{Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)}{Q_{Y^n}\left(y^n\right)}\right)+\sum _{y^n\in {\mathcal{C}}_{2,\gamma }^{\delta }}{Q}_{Y^n}\left(y^n\right)\left(\frac{Q_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|1)}{Q_{Y^n}\left(y^n\right)}-1\right)\hfill \\ \hfill & =\sum _{y^n\in {\mathcal{B}}_{2,\gamma }^{\delta }}{Q}_{Y^n}\left(y^n\right)\left(1-\frac{Q_{\Pi (X^n,\delta ,P_X)|Y^n}\left(1\right|y^n)}{Q_{\Pi (X^n,\delta ,P_X)}\left(1\right)}\right)+\sum _{y^n\in {\mathcal{C}}_{2,\gamma }^{\delta }}{Q}_{Y^n}\left(y^n\right)\left(\frac{Q_{\Pi (X^n,\delta ,P_X)|Y^n}\left(1\right|y^n)}{Q_{\Pi (X^n,\delta ,P_X)}\left(1\right)}-1\right)\hfill \end{array}$$ (A7)

$$\begin{array}{ccc}\hfill & \le \sum _{y^n\in {\mathcal{B}}_{2,\gamma }^{\delta }}{Q}_{Y^n}\left(y^n\right)\left(1-Q_{\Pi (X^n,\delta ,P_X)|Y^n}\left(1\right|y^n)\right)+\sum _{y^n\in {\mathcal{C}}_{2,\gamma }^{\delta }}{Q}_{Y^n}\left(y^n\right)\left(\frac{1}{Q_{\Pi (X^n,\delta ,P_X)}\left(1\right)}-1\right).\hfill & \hfill \end{array}$$ (A8)

Let $P_{\tilde{Y}}$ denote the type of $y^n$ and define

$$\begin{array}{c}\hfill E_n(\delta ,\gamma ):=\underset{P_{\tilde{Y}}\in {\mathcal{P}}_n\left({\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n\right)}{min}\underset{P_{\tilde{X}}\in {\mathcal{P}}_n\left({\mathcal{T}}_{{\left[P_X\right]}_{\delta }}^n\right)}{min}D(P_{\tilde{X}|\tilde{Y}}\left|\right|Q_{X|Y}\left|P_{\tilde{Y}}\right).\end{array}$$

Then, for $y^n\in {\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n$, arbitrary $\tilde{\gamma }>0$ and n sufficiently large (depending on $\left|\mathcal{X}\right|,\left|\mathcal{Y}\right|,\delta ,\gamma$), it follows from ([60], Lemma 2.6) that

$$\begin{array}{cc}\hfill & Q_{\Pi (X^n,\delta ,P_X)|Y^n}\left(1\right|y^n)\ge 1-e^{-n\left(E_n(\delta ,\gamma )-\tilde{\gamma }\right)},\hfill \end{array}$$ (A9)

$$\begin{array}{cc}\hfill \mathrm{and} & Q_{\Pi (X^n,\delta ,P_X)}\left(1\right)\ge 1-e^{-n\left(D\right(P_X\left|\right|Q_X)-\tilde{\gamma })}.\hfill \end{array}$$ (A10)

From (A4), (A6) and (A8)–(A10), it follows that

$$\begin{array}{c}\hfill \parallel Q_{Y^n}(·)-Q_{Y^n|\Pi (X^n,\delta ,P_X)}(·|1)\parallel \le e^{-n\Omega \left(\gamma \right)}+e^{-n\left(E_n(\delta ,\gamma )-\tilde{\gamma }\right)}+e^{-n\left(D\right(P_X\left|\right|Q_X)-\tilde{\gamma })}.\end{array}$$ (A11)

We next show that $E_n(\delta ,\gamma )>0$ for sufficiently small $\delta >0$ and $\gamma >0$. This would imply that the R.H.S of (A11) converges exponentially to zero (for $\tilde{\gamma }$ small enough) with exponent $\overline{\delta }:=min\left(\Omega \left(\gamma \right),E_n(\delta ,\gamma )-\tilde{\gamma },D(P_X\left|\right|Q_X)-\tilde{\gamma }\right)$, thus proving (15). We can write,

$$\begin{array}{cc}\hfill E_n(\delta ,\gamma )& \ge \underset{P_{\tilde{Y}}\in {\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n}{min}\underset{P_{\tilde{X}}\in {\mathcal{T}}_{{\left[P_X\right]}_{\delta }}^n}{min}D(P_{\tilde{X}}\left|\right|{\widehat{Q}}_X)\hfill \end{array}$$ (A12)

$$\begin{array}{cc}\hfill & \ge 2\left[\underset{P_{\tilde{Y}}\in {\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n}{min}\underset{P_{\tilde{X}}\in {\mathcal{T}}_{{\left[P_X\right]}_{\delta }}^n}{min}{\parallel P_{\tilde{X}}-{\widehat{Q}}_X\parallel }^2\right],\hfill \end{array}$$ (A13)

where

$$\begin{array}{c}\hfill {\widehat{Q}}_X\left(x\right):=\sum _y{P}_{\tilde{Y}}\left(y\right)Q_{X|Y}\left(x\right|y).\end{array}$$

Here, (A12) follows due to the convexity of KL divergence (A13) is due to Pinsker’s inequality [60]. We also have from the triangle inequality satisfied by total variation that,

$$\begin{array}{c}\hfill \parallel P_{\tilde{X}}-{\widehat{Q}}_X\parallel \ge \parallel P_X-Q_X\parallel -\parallel P_{\tilde{X}}-P_X\parallel -\parallel {\widehat{Q}}_X-Q_X\parallel .\end{array}$$

For $y^n\in {\mathcal{T}}_{{\left[Q_Y\right]}_{\gamma }}^n$,

$$\begin{array}{c}\hfill \parallel {\widehat{Q}}_X-Q_X\parallel \le \parallel Q_{X|Y}{P}_{\tilde{Y}}-Q_{XY}\parallel \le \parallel P_{\tilde{Y}}-Q_Y\parallel \le O\left(\gamma \right).\end{array}$$

Also, for $P_{\tilde{X}}\in {\mathcal{T}}_{{\left[P_X\right]}_{\delta }}^n$,

$$\begin{array}{c}\hfill \parallel P_{\tilde{X}}-P_X\parallel \le O\left(\delta \right).\end{array}$$

Hence,

$$\begin{array}{cc}\hfill E_n(\delta ,\gamma )& \ge 2{\left(\parallel P_X-Q_X\parallel -O\left(\gamma \right)-O\left(\delta \right)\right)}^2.\hfill \end{array}$$

Since $P_X\ne Q_X$, $E_n(\delta ,\gamma )>0$ for sufficiently small $\gamma >0$ and $\delta >0$. This completes the proof of (15).

We next prove (17). Similar to (A4) and (A5), we have,

$$\begin{array}{cc}\hfill & \parallel P_{Y^n}(·)-P_{Y^n|\Pi (X^n,\delta ,P_X)}(·|0)\parallel \hfill \\ \hfill & \le \frac{1}{2}\sum _{y^n\notin {\mathcal{T}}_{{\left[P_Y\right]}_{\gamma }}^n}\left[P_{Y^n}\left(y^n\right)+P_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|0)\right]+\frac{1}{2}\sum _{y^n\in {\mathcal{T}}_{{\left[P_Y\right]}_{\gamma }}^n}|P_{Y^n}\left(y^n\right)-P_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|0)|,\hfill & \hfill \end{array}$$ (A14)

and

$$\begin{array}{c}\hfill P_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|0)\le 2P_{Y^n}\left(y^n\right),\end{array}$$ (A15)

since $P_{\Pi (X^n,\delta ,P_X)}\left(0\right)\stackrel{\left(n\right)}{\to }1$.

Also, for $\gamma <\frac{\delta }{\left|\mathcal{Y}\right|}$ and sufficiently large n (depending on $\delta ,\gamma ,\left|\mathcal{X}\right|,\left|\mathcal{Y}\right|$), we have

$$\begin{array}{cc}\hfill & \sum _{y^n\in {\mathcal{T}}_{{\left[P_Y\right]}_{\gamma }}^n}|P_{Y^n}\left(y^n\right)-P_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|0)|\hfill \\ \hfill & =\sum _{y^n\in {\mathcal{B}}_{0,\gamma }^{\delta }}{P}_{Y^n}\left(y^n\right)-P_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|0)+\sum _{y^n\in {\mathcal{C}}_{0,\gamma }^{\delta }}{P}_{Y^n|\Pi (X^n,\delta ,P_X)}\left(y^n\right|0)-P_{Y^n}\left(y^n\right)\hfill \\ \hfill & \le \sum _{y^n\in {\mathcal{B}}_{0,\gamma }^{\delta }}{P}_{Y^n}\left(y^n\right)\left(1-P_{\Pi (X^n,\delta ,P_X)|Y^n}\left(0\right|y^n)\right)+\sum _{y^n\in {\mathcal{C}}_{0,\gamma }^{\delta }}{P}_{Y^n}\left(y^n\right)\left(\frac{1}{P_{\Pi (X^n,\delta ,P_X)}\left(0\right)}-1\right)\hfill \\ \hfill & \le \sum _{y^n\in {\mathcal{B}}_{0,\gamma }^{\delta }}{P}_{Y^n}\left(y^n\right)e^{-n\Omega (\delta -\gamma |\mathcal{Y}\left|\right)}+\sum _{y^n\in {\mathcal{C}}_{0,\gamma }^{\delta }}{P}_{Y^n}\left(y^n\right)e^{-n\Omega \left(\delta \right)}\hfill \end{array}$$ (A16)

$$\begin{array}{ccc}\hfill & \le e^{-n\Omega (\delta -\gamma |\mathcal{Y}\left|\right)},\hfill & \hfill \end{array}$$ (A17)

where to obtain (A16), we used

$$\begin{array}{cc}\hfill & P_{\Pi (X^n,\delta ,P_X)}\left(0\right)\ge 1-e^{-n\Omega \left(\delta \right)},\hfill \end{array}$$ (A18)

$$\begin{array}{cc}\hfill \mathrm{and} & P_{\Pi (X^n,\delta ,P_X)|Y^n}\left(0\right|y^n)\ge 1-e^{-n\Omega (\delta -\gamma |\mathcal{Y}\left|\right)},for{y}^n\in {\mathcal{B}}_{0,\gamma }^{\delta }and\gamma <\frac{\delta }{\left|\mathcal{Y}\right|}.\hfill \end{array}$$ (A19)

Here, (A18) follows from ([60], Lemma 2.12), and (A19) follows from ([60], Lemmas 2.10 and 2.12), respectively. Thus, from (A14), (A15) and (A17), we can write that,

$$\begin{array}{c}\hfill \parallel P_{Y^n}(·)-P_{Y^n|\Pi (X^n,\delta ,P_X)}(·|0)\parallel \le e^{-n\Omega \left(\gamma \right)}+e^{-n\Omega (\delta -\gamma |\mathcal{Y}\left|\right)}\stackrel{\left(n\right)}{\to }0.\end{array}$$

This completes the proof of (17). The proof of (16) is exactly the same as (17), with the only difference that the sets ${\mathcal{B}}_{1,\gamma }^{\delta }$ and ${\mathcal{C}}_{1,\gamma }^{\delta }$ are used in place of ${\mathcal{B}}_{0,\gamma }^{\delta }$ and ${\mathcal{C}}_{0,\gamma }^{\delta }$, respectively.

Appendix D. Proof of Theorems 1 and 2

We describe the encoding and decoding operations which are the same for both Theorems 1 and 2. Fix positive numbers (small) $\eta ,\delta >0$, and let ${\delta }^{\prime }:=\frac{\delta }{2},\widehat{\delta }:=\left|\mathcal{U}\right|\delta ,\tilde{\delta }:=2\delta$ and $\overline{\delta }:=\frac{{\delta }^{\prime }}{\left|\mathcal{V}\right|}$.

Codebook Generation: Fix a finite alphabet $\mathcal{W}$ and a conditional distribution $P_{W|U}$. Let ${\mathbb{B}}_n=\left\{W^n\left(j\right),j\in \left[M_n^{\prime }\right]\right\}$, $M_n^{\prime }:=e^{n(I_P(U:W)+\eta )}$, denote a random codebook such that each $W^n\left(j\right)$ is randomly and independently generated according to distribution ${\prod }_{i=1}^n{P}_W\left(w_i\right)$, where

$$\begin{array}{c}\hfill P_W\left(w\right)=\sum _{u\in \mathcal{U}}{P}_U\left(u\right)P_{W|U}\left(w\right|u).\end{array}$$

Denote a realization of ${\mathbb{B}}_n$ by ${\mathcal{B}}_n$ and the support of ${\mathbb{B}}_n$ by ${\mathfrak{B}}_n$.

Encoding: For a given codebook ${\mathcal{B}}_n$, let

$$\begin{array}{cc}\hfill & P_{{\mathsf{E}}_{\mathsf{u}}}^{\left({\mathcal{B}}_n\right)}\left(j\right|u^n):=\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i\left(j\right))}{{\sum }_{j^{\prime }}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i\left(j^{\prime }\right)))},\hfill \end{array}$$ (A20)

denote the likelihood encoding function. If $I_P(U;W)+\eta +\left|\mathcal{U}\right|\left|\mathcal{W}\right|\frac{log(n+1)}{n}>R$, the observer performs uniform random binning on the indices in $\left[M_n^{\prime }\right]$, i.e., for each $j\in \left[M_n^{\prime }\right]$, it selects an index uniformly at random from the set ${\tilde{\mathcal{M}}}_n:=\left[e^{n\left(R-\left|\mathcal{U}\right|\left|\mathcal{W}\right|\frac{log(n+1)}{n}\right)}\right]$. Denote the random binning function by $f_{\mathbb{B}}$ and a realization of it by $f_{\mathsf{B}}$. If $I_P(U;W)+\eta +\left|\mathcal{U}\right|\left|\mathcal{W}\right|\frac{log(n+1)}{n}\le R$, set $f_{\mathbb{B}}$ as the identity function with probability one, i.e., $f_{\mathbb{B}}\left(j\right)=j$. If $u^n\in {\mathcal{T}}_{{\left[P_U\right]}_{{\delta }^{\prime }}}^n$, then the observer outputs the message $m=(t,f_{\mathsf{B}}\left(j\right))$ if $I_P(U;W)+\eta +\left|\mathcal{U}\right|\left|\mathcal{W}\right|\frac{log(n+1)}{n}>R$ or $m=(t,j)$ otherwise, where $j\in \left[M_n^{\prime }\right]$ is chosen randomly with probability $P_{{\mathsf{E}}_{\mathsf{u}}}^{\left({\mathcal{B}}_n\right)}\left(j\right|u^n)$ and t denotes the index of the joint type of $(u^n,w^n\left(j\right))$ in the set of types ${\mathcal{P}}^n(\mathcal{U}\times \mathcal{W})$. If $u^n\notin {\mathcal{T}}_{{\left[P_U\right]}_{{\delta }^{\prime }}}^n$, the observer outputs the error message $M=0$. Please note that $\left|\mathcal{M}\right|\le e^{nR}$ since the total number of types in ${\mathcal{P}}^n(\mathcal{U}\times \mathcal{W})$ is upper bounded by ${(n+1)}^{\left|\mathcal{U}\right|\left|\mathcal{W}\right|}$ ([60], Lemma 2.2). Let ${\mathbb{C}}_n:=({\mathbb{B}}_n,f_{\mathbb{B}})$, and let ${\mathcal{C}}_n=({\mathcal{B}}_n,f_{\mathsf{B}})$ and ${\mu }_n(·)$ denote its realization and probability distribution, respectively. For a given ${\mathcal{C}}_n$, let $f_n^{\left({\mathcal{C}}_n\right)}$ represent the encoder induced by the above operations, where $f_n^{\left({\mathcal{C}}_n\right)}:{\mathcal{U}}^n\to \mathcal{P}\left(\mathcal{M}\right)$ and $\mathcal{M}:=\left[e^{nR}\right]$.

Decoding: If $M=0$ or $t\notin {\mathcal{T}}_{{\left[P_{UW}\right]}_{\delta }}^n$, $\widehat{H}=1$ is declared. Else, given $m=(t,f_{\mathsf{B}}\left(j\right))$ and $V^n=v^n$, the detector decodes for a codeword ${\widehat{w}}^n:=w^n\left(\widehat{j}\right)\in {\mathcal{T}}_{{\left[P_W\right]}_{\widehat{\delta }}}^n$ in the codebook ${\mathcal{B}}_n$ such that

$$\begin{array}{cc}\hfill \widehat{j}& =\underset{\begin{array}{c}l:f_{\mathsf{B}}\left(l\right)=f_{\mathsf{B}}\left(j\right),\\ w^n\left(l\right)\in {\mathcal{T}}_{{\left[P_W\right]}_{\widehat{\delta }}}^n\end{array}}{arg\; min}{H}_e\left(w^n\left(l\right)\right|v^n),if{I}_P(U;W)+\eta +\frac{1}{n}\left|\mathcal{U}\right|\left|\mathcal{W}\right|log(n+1)>R,\hfill \\ \hfill \widehat{j}& =j,\mathrm{otherwise}.\hfill \end{array}$$

Denote the above decoding rule by $P_{\mathsf{ED}}^{\left({\mathcal{C}}_n\right)}$, where $P_{\mathsf{ED}}^{\left({\mathcal{C}}_n\right)}:\mathcal{M}\times {\mathcal{V}}^n\to \mathcal{J}$. The detector declares $\widehat{H}=0$ if $({\widehat{w}}^n,v^n)\in {\mathcal{T}}_{{\left[P_{WV}\right]}_{\tilde{\delta }}}^n$ and $\widehat{H}=1$ otherwise. Let $g_n^{\left({\mathcal{C}}_n\right)}:\mathcal{M}\times {\mathcal{V}}^n\to \widehat{\mathcal{H}}$ stand for the decision rule induced by the above operations.

System induced distributions and auxiliary distributions:

The system induced probability distribution when $H=0$ is given by

$$\begin{array}{cc}\hfill & {\tilde{P}}^{({\mathcal{C}}_n,0)}(s^n,u^n,v^n,j,w^n,m,\widehat{j},{\widehat{w}}^n)\hfill \\ \hfill & =\left[\prod _{i=1}^n{P}_{SUV}(s_i,u_i,v_i,z_i)\right]P_{{\mathsf{E}}_{\mathsf{u}}}^{\left({\mathcal{B}}_n\right)}\left(j\right|u^n)\mathbb{1}(w^n\left(j\right)=w^n)\mathbb{1}(f_{\mathsf{B}}\left(j\right)=m)\mathbb{1}\left(\widehat{j}=P_{\mathsf{ED}}^{\left({\mathcal{C}}_n\right)}(m,v^n)\right)\hfill \end{array}$$

$$\begin{array}{cc}\hfill \mathbb{1}(w^n\left(\widehat{j}\right)={\widehat{w}}^n),\mathrm{if} u^n\in {\mathcal{T}}_{{\left[P_U\right]}_{{\delta }^{\prime }}}^n,& \hfill \end{array}$$ (A21)

and

$$\begin{array}{cc}\hfill {\tilde{P}}^{({\mathcal{C}}_n,0)}(s^n,u^n,v^n,m)=\left[\prod _{i=1}^n{P}_{SUV}(s_i,u_i,v_i)\right]\mathbb{1}(m=0),\mathrm{if} u^n\notin {\mathcal{T}}_{{\left[P_U\right]}_{{\delta }^{\prime }}}^n.& \hfill \end{array}$$ (A22)

Consider two auxiliary distribution $\tilde{\Psi }$ and $\Psi$ given by

$$\begin{array}{cc}\hfill & {\tilde{\Psi }}^{({\mathcal{C}}_n,0)}(s^n,u^n,v^n,j,w^n,m,\widehat{j},{\widehat{w}}^n)\hfill \\ \hfill & :=\left[\prod _{i=1}^n{P}_{SUV}(s_i,u_i,v_i)\right]P_{{\mathsf{E}}_{\mathsf{u}}}^{\left({\mathcal{B}}_n\right)}\left(j\right|u^n)\mathbb{1}(w^n\left(j\right)=w^n)\mathbb{1}(f_{\mathsf{B}}\left(j\right)=m)\mathbb{1}\left(\widehat{j}=P_{\mathsf{ED}}^{\left({\mathcal{C}}_n\right)}(m,v^n)\right)\hfill \\ \hfill & \mathbb{1}(w^n\left(\widehat{j}\right)={\widehat{w}}^n),\hfill & \hfill \end{array}$$ (A23)

and

$$\begin{array}{cc}\hfill & {\Psi }^{({\mathcal{C}}_n,0)}(s^n,u^n,v^n,j,w^n,m,\widehat{j},{\widehat{w}}^n)\hfill \\ \hfill & :=\frac{1}{M_n^{\prime }}\mathbb{1}(w^n\left(j\right)=w^n)\left[\prod _{i=1}^n{P}_{U|W}\left(u_i\right|w_i)\right]\left[\prod _{i=1}^n{P}_{VS|U}(v_i,s_i|u_i)\right]\mathbb{1}(f_{\mathsf{B}}\left(j\right)=m)\mathbb{1}\left(\widehat{j}=P_{\mathsf{ED}}^{\left({\mathcal{C}}_n\right)}(m,v^n)\right)\hfill \\ \hfill & \mathbb{1}(w^n\left(\widehat{j}\right)={\widehat{w}}^n).\hfill & \hfill \end{array}$$ (A24)

Let ${\tilde{P}}^{({\mathcal{C}}_n,1)}$ and ${\tilde{\Psi }}^{({\mathcal{C}}_n,1)}$ denote probability distributions under $H=1$ defined by the R.H.S. of (A21)–(A23) with $P_{SUV}$ replaced by $Q_{SUV}$, and let ${\Psi }^{({\mathcal{C}}_n,1)}$ denote the R.H.S. of (A24) with $P_{VS|U}$ replaced by $Q_{VS|U}$. Please note that the encoder $f_n^{\left({\mathcal{C}}_n\right)}$ is such that $P_{{\mathsf{E}}_{\mathsf{u}}}^{\left({\mathcal{B}}_n\right)}\left(j\right|u^n)={\Psi }^{({\mathcal{C}}_n,0)}\left(j\right|u^n)$ and hence, the only difference between the joint distribution ${\Psi }^{({\mathcal{C}}_n,0)}$ and ${\tilde{\Psi }}^{({\mathcal{C}}_n,0)}$ is the marginal distribution of $U^n$. By the soft-covering lemma [62,64], it follows that for some ${\gamma }_1>0$,

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mu }_n}\left[∥{\Psi }_{U^n}^{({\mathbb{C}}_n,0)}-{\tilde{\Psi }}_{U^n}^{({\mathbb{C}}_n,0)}∥\right]\le e^{-n{\gamma }_1}\stackrel{\left(n\right)}{\to }0.\end{array}$$ (A25)

Hence, from ([43], Property 2(d)), it follows that

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mu }_n}\left[∥{\Psi }^{({\mathbb{C}}_n,0)}-{\tilde{\Psi }}^{({\mathbb{C}}_n,0)}∥\right]\le e^{-n{\gamma }_1}.\end{array}$$ (A26)

Also, note that the only difference between the distributions ${\tilde{P}}^{({\mathcal{C}}_n,0)}$ and ${\tilde{\Psi }}^{({\mathcal{C}}_n,0)}$ is $P_{{\mathsf{E}}_{\mathsf{u}}}^{\left({\mathcal{B}}_n\right)}$ when $U^n\notin {\mathcal{T}}_{{\left[P_U\right]}_{{\delta }^{\prime }}}^n$. Since

$$\begin{array}{c}\hfill \mathbb{P}\left(U^n\notin {\mathcal{T}}_{{\left[P_U\right]}_{{\delta }^{\prime }}}^n|H=0\right)\le e^{-n\Omega \left({\delta }^{\prime }\right)},\end{array}$$ (A27)

it follows that

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mu }_n}\left[∥{\tilde{P}}^{({\mathbb{C}}_n,0)}-{\tilde{\Psi }}^{({\mathbb{C}}_n,0)}∥\right]\le e^{-n\Omega \left({\delta }^{\prime }\right)}.\end{array}$$ (A28)

Equations (A26) and (A28) together imply via ([43], Property 2(c)) that

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mu }_n}\left[∥{\tilde{P}}^{({\mathbb{C}}_n,0)}-{\Psi }^{({\mathbb{C}}_n,0)}∥\right]\le e^{-n\Omega \left({\delta }^{\prime }\right)}+e^{-n{\gamma }_1}\stackrel{\left(n\right)}{\to }0.\end{array}$$ (A29)

Please note that for $l\in \{0,1\}$, the joint distribution ${\Psi }^{({\mathcal{C}}_n,l)}$ satisfies

$$\begin{array}{c}\hfill S_i-(w_i\left(J\right),V_i)-(M,w^n\left(J\right),V^n,S^{i-1}),i\in \left[n\right].\end{array}$$ (A30)

Also, since $I_P(U;W)+\eta >0$, by the application of soft-covering lemma,

$$\begin{array}{cc}\hfill {\mathbb{E}}_{{\mu }_n}\left[\sum _{i=1}^n∥P_W-{\Psi }_{W_i\left(J\right)}^{({\mathbb{C}}_n,l)}∥|H=l\right]& \le e^{-{\gamma }_{3}n}\stackrel{\left(n\right)}{\to }0,l=0,1,\hfill \end{array}$$ (A31)

for some ${\gamma }_3>0$.

If $Q_U=P_U$, then it again follows from the soft-covering lemma that

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mu }_n}\left[∥{\Psi }_{U^n}^{({\mathbb{C}}_n,1)}-{\tilde{\Psi }}_{U^n}^{({\mathbb{C}}_n,1)}∥\right]\le e^{-{\gamma }_{1}n}\stackrel{\left(n\right)}{\to }0,\end{array}$$ (A32)

thereby implying that

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mu }_n}\left[∥{\Psi }^{({\mathbb{C}}_n,1)}-{\tilde{\Psi }}^{({\mathbb{C}}_n,1)}∥\right]\le e^{-{\gamma }_{1}n}.\end{array}$$ (A33)

Also, note that the only difference between the distributions ${\tilde{P}}^{({\mathcal{C}}_n,1)}$ and ${\tilde{\Psi }}^{({\mathcal{C}}_n,1)}$ is $P_{{\mathsf{E}}_{\mathsf{u}}}^{\left({\mathcal{B}}_n\right)}$ when $U^n\notin {\mathcal{T}}_{{\left[P_U\right]}_{{\delta }^{\prime }}}^n$. Since $Q_U=P_U$ implies $\mathbb{P}\left(U^n\notin {\mathcal{T}}_{{\left[P_U\right]}_{{\delta }^{\prime }}}^n|H=1\right)\le e^{-n\Omega \left({\delta }^{\prime }\right)}$, it follows that

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mu }_n}\left[∥{\tilde{P}}^{({\mathbb{C}}_n,1)}-{\tilde{\Psi }}^{({\mathbb{C}}_n,1)}∥\right]\le e^{-n\Omega \left({\delta }^{\prime }\right)}.\end{array}$$ (A34)

Equations (A33) and (A34) together imply that

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mu }_n}\left[∥{\tilde{P}}^{({\mathbb{C}}_n,1)}-{\Psi }^{({\mathbb{C}}_n,1)}∥\right]\le e^{-n\Omega \left({\delta }^{\prime }\right)}+e^{-{\gamma }_{1}n}\stackrel{\left(n\right)}{\to }0.\end{array}$$ (A35)

Let ${\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,0)}}={\mathbb{E}}_{{\mu }_n}\left[{\mathbb{P}}_{{\tilde{P}}^{({\mathbb{C}}_n,0)}}\right]$ and ${\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}={\mathbb{E}}_{{\mu }_n}\left[{\mathbb{P}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\right]$ denote the expected probability measure (random coding measure) induced by PMF’s ${\tilde{P}}^{({\mathbb{C}}_n,0)}$ and ${\tilde{P}}^{({\mathbb{C}}_n,1)}$, respectively. Then, note that from (A24), (A29), (A31) and the weak law of large numbers,

$$\begin{array}{c}\hfill {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,0)}}\left(\left(U^n,W^n\left(J\right)\right)\in {\mathcal{T}}_{{\left[P_{UW}\right]}_{\delta }}^n\right)\ge 1-e^{-n\Omega \left(\delta \right)}\stackrel{\left(n\right)}{\to }1.\end{array}$$ (A36)

Analysis of type I and type II error probabilities:

We analyze type I and type II error probabilities of the coding scheme mentioned above averaged over the random ensemble ${\mathbb{C}}_n$.

Type I error probability:

Please note that a type I error occurs only if one of the following events occur:

$$\begin{array}{cc}\hfill {\mathcal{E}}_{\mathsf{TE}}& =\left\{(U^n,V^n)\notin {\mathcal{T}}_{{\left[P_{UV}\right]}_{\overline{\delta }}}^n\right\},\hfill \\ \hfill {\mathcal{E}}_{\mathsf{SE}}& =\left\{T\notin {\mathcal{P}}_n\left({\mathcal{T}}_{{\left[P_{UW}\right]}_{\delta }}^n\right)\right\},\hfill \\ \hfill {\mathcal{E}}_{\mathsf{ME}}& =\left\{\left(V^n,W^n\left(J\right)\right)\notin {\mathcal{T}}_{{\left[P_{VW}\right]}_{\tilde{\delta }}}^n\right\},\hfill \\ \hfill {\mathcal{E}}_{\mathsf{DE}}& =\{\exists l\in \left[e^{n(I_P(U;W)+\eta )}\right],l\ne J:f_{\mathbb{B}}\left(l\right)=f_{\mathbb{B}}\left(J\right),W^n\left(l\right)\in {\mathcal{T}}_{{\left[P_W\right]}_{\widehat{\delta }}}^n,\hfill \\ \hfill & H_e\left(W^n\left(l\right)|V^n\right)\le H_e\left(W^n\left(J\right)|V^n\right)\}.\hfill & \hfill \end{array}$$

Let $\mathcal{E}:={\mathcal{E}}_{\mathsf{TE}}\cup {\mathcal{E}}_{\mathsf{SE}}\cup {\mathcal{E}}_{\mathsf{ME}}\cup {\mathcal{E}}_{\mathsf{DE}}$. Then, the expected type I error probability over ${\mathbb{C}}_n$ be upper bounded as

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mu }_n}\left[{\alpha }_n\left(f_n^{\left({\mathbb{C}}_n\right)},g_n^{\left({\mathbb{C}}_n\right)}\right)\right]\le {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,0)}}\left(\mathcal{E}\right).\end{array}$$ (A37)

Please note that ${\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,0)}}\left({\mathcal{E}}_{\mathsf{TE}}\right)$ tends to 0 asymptotically by the weak law of large numbers. From (A36), ${\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,0)}}\left({\mathcal{E}}_{\mathsf{SE}}\right)\stackrel{\left(n\right)}{\to }0$. Given ${\mathcal{E}}_{\mathsf{SE}}^c$ and ${\mathcal{E}}_{\mathsf{TE}}^c$ holds, it follows from the Markov chain relation $V-U-W$ and the Markov lemma [68] that ${\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,0)}}\left({\mathcal{E}}_{\mathsf{ME}}\right)\stackrel{\left(n\right)}{\to }0$. Also, as in the proof of Theorem 2 in [13], it follows that

$$\begin{array}{cc}\hfill & {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,0)}}\left({\mathcal{E}}_{\mathsf{DE}}\right|V^n=v^n,W^n\left(J\right)=w^n,{\mathcal{E}}_{\mathsf{ME}}^c\cap {\mathcal{E}}_{\mathsf{SE}}^c\cap {\mathcal{E}}_{\mathsf{TE}}^c)\le e^{-n\left(R-I_P(U;W|V)-{\delta }_n^{\left(1\right)}\right)},\hfill \end{array}$$ (A38)

where ${\delta }_n^{\left(1\right)}\stackrel{\left(n\right)}{\to }\eta +O\left(\delta \right)$. Thus, if $R>I_P(U;W|V)$, it follows by choosing $\eta =O\left(\delta \right)$ that for $\delta >0$ small enough, the R.H.S. of (A38) tends to zero asymptotically. By the union bound on probability, the R.H.S. of (A37) tends to zero.

Type II error probability:

Let ${\delta }^{″}=\left|\mathcal{W}\right|\tilde{\delta }$. Please note that a type II error occurs only if $V^n\in {\mathcal{T}}_{{\left[P_V\right]}_{{\delta }^{″}}}^n$ and $M\ne 0$, i.e., $U^n\in {\mathcal{T}}_{{\left[P_U\right]}_{{\delta }^{\prime }}}^n$ and $T\in {\mathcal{T}}_{{\left[P_{UW}\right]}_{\delta }}^n$. Hence, we can restrict the type II error analysis to only such $(U^n,V^n)$. Denoting the event that a type II error occurs by ${\mathcal{D}}_0$, we have

$$\begin{array}{cc}\hfill {\mathbb{E}}_{{\mu }_n}\left[{\beta }_n\left(f_n^{\left({\mathbb{C}}_n\right)},g_n^{\left({\mathbb{C}}_n\right)}\right)\right]=\sum _{u^n,v^n}{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(U^n=u^n,V^n=v^n){\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{D}}_0\right|U^n=u^n,V^n=v^n).& \hfill \end{array}$$ (A39)

Let ${\mathcal{E}}_{\mathsf{NE}}:={\mathcal{E}}_{\mathsf{SE}}^c\cap \left\{V^n\in {\mathcal{T}}_{{\left[V\right]}_{{\delta }^{″}}}^n\right\}\cap \left\{U^n\in {\mathcal{T}}_{{\left[U\right]}_{{\delta }^{\prime }}}^n\right\}$. The last term in (A39) can be upper bounded as follows:

$$\begin{array}{cc}\hfill & {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{D}}_0\right|U^n=u^n,V^n=v^n)\hfill \\ \hfill & ={\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{E}}_{\mathsf{NE}}\right|U^n=u^n,V^n=v^n){\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{D}}_0\right|U^n=u^n,V^n=v^n,{\mathcal{E}}_{\mathsf{NE}})\hfill \\ \hfill & \le {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{D}}_0\right|U^n=u^n,V^n=v^n,{\mathcal{E}}_{\mathsf{NE}})\hfill \\ \hfill & =\sum _{j,\tilde{m}}{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(J=j,f_{\mathbb{B}}\left(J\right)=\tilde{m}|U^n=u^n,V^n=v^n,{\mathcal{E}}_{\mathsf{NE}})\hfill \\ \hfill & {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{D}}_0\right|U^n=u^n,V^n=v^n,J=j,f_{\mathbb{B}}\left(J\right)=\tilde{m},{\mathcal{E}}_{\mathsf{NE}})\hfill \end{array}$$ (A40)

$$\begin{array}{cc}\hfill & ={\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{D}}_0\right|U^n=u^n,V^n=v^n,J=1,f_{\mathbb{B}}\left(J\right)=1,{\mathcal{E}}_{\mathsf{NE}})\hfill \\ \hfill & =\sum _{\begin{array}{c}{w}^n\in {\mathcal{W}}^n\end{array}}{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(W^n\left(1\right)=w^n|U^n=u^n,V^n=v^n,J=1,f_{\mathbb{B}}\left(J\right)=1,{\mathcal{E}}_{\mathsf{NE}})\hfill \end{array}$$ (A41)

$$\begin{array}{ccc}\hfill & {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{D}}_0\right|U^n=u^n,V^n=v^n,J=1,f_{\mathbb{B}}\left(J\right)=1,W^n\left(1\right)=w^n,{\mathcal{E}}_{\mathsf{NE}}).\hfill & \hfill \end{array}$$ (A42)

where (A41) follows since the term in (A40) is independent of the indices $(j,\tilde{m})$ due to the symmetry of the codebook generation, encoding and decoding procedure. The first term in (A42) can be upper bounded as

$$\begin{array}{cc}\hfill & {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(W^n\left(1\right)=w^n|U^n=u^n,V^n=v^n,J=1,f_{\mathbb{B}}\left(J\right)=1,{\mathcal{E}}_{\mathsf{NE}})\hfill \\ \hfill & \le \frac{1}{|{\mathcal{T}}_{P_{\tilde{W}|\tilde{U}}}|}\le e^{-n(H\left(\tilde{W}\right|\tilde{U})-\frac{1}{n}|\mathcal{U}\left|\right|\mathcal{W}|log(n+1))}.\hfill & \hfill \end{array}$$ (A43)

To obtain (A43), we used the fact that $P_{{\mathsf{E}}_{\mathsf{u}}}^{\left({\mathcal{B}}_n\right)}\left(1\right|u^n)$ in (A20) is invariant to the joint type $P_{\tilde{U}\tilde{W}}$ of $(U^n,W^n\left(1\right))=(u^n,w^n)$ (keeping all the other codewords fixed). This in turn implies that given ${\mathcal{E}}_{\mathsf{NE}}$, each sequence in the conditional type class ${\mathcal{T}}_{P_{\tilde{W}|\tilde{U}}}\left(u^n\right)$ is equally likely (in the randomness induced by ${\mathbb{B}}_n$ and stochastic encoding in (A20)) and its probability is upper bounded by $\frac{1}{\left|{\mathcal{T}}_{P_{\tilde{W}|\tilde{U}}}\right|}$. Defining the events

$$\begin{array}{cc}\hfill & {\mathcal{E}}_{\mathsf{BE}}:=\left\{\exists l\in \left[M_n^{\prime }\right],l\ne J,f_{\mathbb{B}}\left(l\right)=M,W^n\left(l\right))\in {\mathcal{T}}_{{\left[P_W\right]}_{\widehat{\delta }}}^n,(V^n,W^n\left(l\right))\in {\mathcal{T}}_{{\left[P_{VW}\right]}_{\tilde{\delta }}}^n\right\},\hfill \end{array}$$ (A44)

$$\begin{array}{cc}\hfill & \mathcal{F}:=\{U^n=u^n,V^n=v^n,J=1,f_{\mathbb{B}}\left(J\right)=1,W^n\left(1\right)=w^n,{\mathcal{E}}_{\mathsf{NE}}\},\hfill \end{array}$$ (A45)

$$\begin{array}{cc}\hfill & {\mathcal{F}}_1:=\{U^n=u^n,V^n=v^n,J=1,f_{\mathbb{B}}\left(J\right)=1,W^n\left(1\right)=w^n,{\mathcal{E}}_{\mathsf{NE}},{\mathcal{E}}_{\mathsf{BE}}^c\},\hfill \end{array}$$ (A46)

$$\begin{array}{ccc}\hfill \mathrm{and} & {\mathcal{F}}_2:=\{U^n=u^n,V^n=v^n,J=1,f_{\mathbb{B}}\left(J\right)=1,W^n\left(1\right)=w^n,{\mathcal{E}}_{\mathsf{NE}},{\mathcal{E}}_{\mathsf{BE}}\},\hfill & \hfill \end{array}$$ (A47)

the last term in (A42) can be written as

$$\begin{array}{cc}\hfill & {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{D}}_0\right|\mathcal{F})={\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{E}}_{\mathsf{BE}}^c\right|\mathcal{F}){\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{D}}_0\right|{\mathcal{F}}_1)+{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{E}}_{\mathsf{BE}}\right|\mathcal{F}){\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{D}}_0\right|{\mathcal{F}}_2).\hfill \end{array}$$ (A48)

The analysis of the terms in (A48) is essentially similar to that given in the proof of Theorem 2 in [13], except for a subtle difference that we mention next. To bound the binning error event ${\mathcal{E}}_{\mathsf{BE}}$, we require an upper bound similar to

$$\begin{array}{c}\hfill {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left(W^n\left(l\right)={\tilde{w}}^n|\mathcal{F}\right)\le 2{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(W^n\left(l\right)={\tilde{w}}^n),\forall {\tilde{w}}^n\in {\mathcal{W}}^n,\end{array}$$ (A49)

that is used in the proof of Theorem 2 in [13]. Please note that the stochastic encoding scheme considered here is different from the encoding scheme in [13]. In place (A49), we will show that for $l\ne 1$,

$$\begin{array}{cc}\hfill & {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(W^n\left(l\right)={\tilde{w}}^n|\mathcal{F})\le 3{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(W^n\left(l\right)={\tilde{w}}^n),\hfill \end{array}$$ (A50)

which suffices for the proof. Please note that

$$\begin{array}{cc}\hfill & {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(W^n\left(l\right)={\tilde{w}}^n|\mathcal{F})\hfill \\ \hfill & ={\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(W^n\left(l\right)={\tilde{w}}^n|U^n=u^n,V^n=v^n)\frac{{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(W^n\left(1\right)=w^n|W^n\left(l\right)={\tilde{w}}^n,U^n=u^n,V^n=v^n)}{{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(W^n\left(1\right)=w^n|U^n=u^n,V^n=v^n)}\hfill \\ \hfill & \frac{{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(J=1|W^n\left(1\right)=w^n,W^n\left(l\right)={\tilde{w}}^n,U^n=u^n,V^n=v^n)}{{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(J=1|W^n\left(1\right)=w^n,U^n=u^n,V^n=v^n)}\hfill \\ \hfill & \frac{{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(f_{\mathbb{B}}\left(J\right)=1|J=1,W^n\left(1\right)=w^n,W^n\left(l\right)={\tilde{w}}^n,U^n=u^n,V^n=v^n)}{{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(f_{\mathbb{B}}\left(J\right)=1|J=1,W^n\left(1\right)=w^n,U^n=u^n,V^n=v^n)}\hfill \end{array}$$ (A51)

$$\begin{array}{ccc}\hfill & \frac{{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{E}}_{\mathsf{NE}}\right|f_{\mathbb{B}}\left(J\right)=1,J=1,W^n\left(1\right)=w^n,W^n\left(l\right)={\tilde{w}}^n,U^n=u^n,V^n=v^n)}{{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{E}}_{\mathsf{NE}}\right|f_{\mathbb{B}}\left(J\right)=1,J=1,W^n\left(1\right)=w^n,U^n=u^n,V^n=v^n)}\hfill & \hfill \end{array}$$ (A52)

Since the codewords are generated independently of each other and the binning operation is done independent of the codebook generation, we have

$$\begin{array}{cc}\hfill & {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(W^n\left(1\right)=w^n|W^n\left(l\right)={\tilde{w}}^n,U^n=u^n,V^n=v^n)={\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(W^n\left(1\right)=w^n|U^n=u^n,V^n=v^n),\hfill \end{array}$$ (A53)

and

$$\begin{array}{cc}\hfill & {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(f_{\mathbb{B}}\left(J\right)=1|J=1,W^n\left(1\right)=w^n,W^n\left(l\right)={\tilde{w}}^n,U^n=u^n,V^n=v^n)\hfill \\ \hfill & ={\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(f_{\mathbb{B}}\left(J\right)=1|J=1,W^n\left(1\right)=w^n,U^n=u^n,V^n=v^n).\hfill \end{array}$$ (A54)

Also, note that

$$\begin{array}{cc}\hfill & {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{E}}_{\mathsf{NE}}\right|f_{\mathbb{B}}\left(J\right)=1,J=1,W^n\left(1\right)=w^n,W^n\left(l\right)={\tilde{w}}^n,U^n=u^n,V^n=v^n)\hfill \\ \hfill & ={\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}\left({\mathcal{E}}_{\mathsf{NE}}\right|f_{\mathbb{B}}\left(J\right)=1,J=1,W^n\left(1\right)=w^n,U^n=u^n,V^n=v^n).\hfill \end{array}$$ (A55)

Next, consider the term in (A51). Let

$$\begin{array}{cc}\hfill & {\mathcal{F}}^{\prime }:=\{W^n\left(1\right)=w^n,U^n=u^n,V^n=v^n\},\hfill \\ \hfill & {\mathcal{F}}^{\prime \prime }:=\{W^n\left(1\right)=w^n,W^n\left(l\right)={\tilde{w}}^n,U^n=u^n,V^n=v^n\}.\hfill \end{array}$$

Then, the numerator and denominator of (A51) can be written as

$$\begin{array}{cc}\hfill & {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(J=1|{\mathcal{F}}^{″})\hfill \\ \hfill & ={\mathbb{E}}_{{\mu }_n}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|{\tilde{w}}_i)+{\sum }_{j\ne 1,l}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right]\hfill \\ \hfill & \le {\mathbb{E}}_{{\mu }_n}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1,l}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right],\hfill & \hfill \end{array}$$ (A56)

and

$$\begin{array}{ccc}\hfill & {\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(J=1|{\mathcal{F}}^{\prime })={\mathbb{E}}_{{\mu }_n}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right],\hfill & \hfill \end{array}$$ (A57)

respectively. The R.H.S. of (A56) (resp. (A57)) denote the average probability that $J=1$ is chosen by $P_{{\mathsf{E}}_{\mathsf{u}}}^{\left({\mathbb{B}}_n\right)}$ given $W^n\left(1\right)=w^n$, $U^n=u^n$ and $M_n^{\prime }-2$ (resp. $M_n^{\prime }-1$) other independent codewords in ${\mathbb{B}}_n$. Let

$$\begin{array}{c}\hfill {\mathcal{E}}_l:=\left\{\prod _{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(l\right))\ge max\left(\left\{\prod _{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right)),j\in \lceil M_n^{\prime }\rceil \setminus \left\{1\right\}\right\}\cup \left\{\prod _{i=1}^n{P}_{U|W}\left(u_i\right|w_i)\right\}\right)\right\}.\end{array}$$

Please note that

$$\begin{array}{cc}\hfill & {\mathbb{E}}_{{\mu }_n|{\mathcal{E}}_l^c}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right]\hfill \\ \hfill & \ge \frac{1}{2}{\mathbb{E}}_{{\mu }_n|{\mathcal{E}}_l^c}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1,l}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right].\hfill & \hfill \end{array}$$ (A58)

Hence, denoting by ${\overline{\mu }}_n$ the probability measure induced by ${\mu }_n$, we have

$$\begin{array}{cc}\hfill & \frac{{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(J=1|{\mathcal{F}}^{\prime \prime })}{{\overline{\mathbb{P}}}_{{\tilde{P}}^{({\mathbb{C}}_n,1)}}(J=1|{\mathcal{F}}^{\prime })}\hfill \\ \hfill & \le \frac{{\mathbb{E}}_{{\mu }_n}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1,l}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right]}{{\mathbb{E}}_{{\mu }_n}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right]}\hfill \\ \hfill & \le \frac{{\mathbb{E}}_{{\mu }_n}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1,l}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right]}{{\overline{\mu }}_n\left({\mathcal{E}}_l^c\right){\mathbb{E}}_{{\mu }_n|{\mathcal{E}}_l^c}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right]}\hfill \\ \hfill & \le \frac{{\mathbb{E}}_{{\mu }_n}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1,l}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right]}{\frac{1}{2}{\overline{\mu }}_n\left({\mathcal{E}}_l^c\right){\mathbb{E}}_{{\mu }_n|{\mathcal{E}}_l^c}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1,l}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right]}\hfill \end{array}$$ (A59)

$$\begin{array}{cc}\hfill & =\frac{{\mathbb{E}}_{{\mu }_n}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1,l}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right]}{\frac{1}{2}{\mathbb{E}}_{{\mu }_n}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1,l}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right]-\frac{1}{2}{\overline{\mu }}_n\left({\mathcal{E}}_l\right){\mathbb{E}}_{{\mu }_n|{\mathcal{E}}_l}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1,l}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right]}\hfill \end{array}$$ (A60)

$$\begin{array}{cc}\hfill & \le \frac{{\mathbb{E}}_{{\mu }_n}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1,l}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right]}{\frac{1}{2}{\mathbb{E}}_{{\mu }_n}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1,l}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right]-\frac{1}{2}{\overline{\mu }}_n\left({\mathcal{E}}_l\right)}\hfill \end{array}$$ (A61)

$$\begin{array}{cc}\hfill & \le \frac{{\mathbb{E}}_{{\mu }_n}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1,l}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right]}{\frac{1}{2}{\mathbb{E}}_{{\mu }_n}\left[\frac{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)}{{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|w_i)+{\sum }_{j\ne 1,l}{\prod }_{i=1}^n{P}_{U|W}\left(u_i\right|W_i\left(j\right))}\right]-e^{-e^{n(I_P(U;W)+{\eta }^{\prime })}}}\hfill \end{array}$$ (A62)

$$\begin{array}{ccc}\hfill & \le 2+o\left(1\right)\le 3,\hfill & \hfill \end{array}$$ (A63)

where (A59) is due to (A58); (A61) is since the term within ${\mathbb{E}}_{{\mu }_n|{\mathcal{E}}_l}[·]$ in (A60) is upper bounded by one; (A62) is since ${\overline{\mu }}_n\left({\mathcal{E}}_l\right)\le e^{-e^{n(I_P(U;W)+{\eta }^{\prime })}}$ for some ${\eta }^{\prime }>0$ which follows similar to ([68], Section 3.6.3), and (A63) follows since the term within the expectation which is exponential in order dominates the double exponential term. From (A52)–(A55), (A63) and (A50) follows. The analysis of the other terms in (A48) is the same as in the SHA scheme in [7], and results in the error exponent (within an additive $O\left(\delta \right)$ term) claimed in the Theorem. We refer the reader to ([13], Theorem 2) for a detailed proof (In [13], the communication channel between the observer and the detector is a DMC. However, since the coding scheme used in the achievability part of Theorem 2 in [13] is a separation-based scheme, the error exponent when the channel is noiseless can be recovered by setting $E_3(·)$ and $E_4(·)$ in Theorem 2 to ∞). By the random coding argument followed by the standard expurgation technique [72] (see ([13], Proof of Theorem 2)), there exists a deterministic codebook and binning function pair ${\mathcal{C}}_n=({\mathcal{B}}_n,f_{\mathsf{B}})$ such that the type I and type II error probabilities are within a constant multiplicative factor of their average values over the random ensemble ${\mathbb{C}}_n$, and

$$\begin{array}{cc}\hfill & S_i-(w_i\left(J\right),V_i)-(M,w^n\left(J\right),V^n,S^{i-1}),i\in \left[n\right],\hfill \end{array}$$ (A64)

$$\begin{array}{cc}\hfill & \parallel {\tilde{P}}^{({\mathcal{C}}_n,0)}-{\Psi }^{({\mathcal{C}}_n,0)}\parallel \le e^{-{\gamma }_{4}n},\hfill \end{array}$$ (A65)

$$\begin{array}{cc}\hfill & \parallel {\tilde{P}}^{({\mathcal{C}}_n,1)}-{\Psi }^{({\mathcal{C}}_n,1)}\parallel \le e^{-{\gamma }_{4}n},if{Q}_U=P_U,\hfill \end{array}$$ (A66)

$$\begin{array}{cc}\hfill \mathrm{and} & \sum _{i=1}^n\parallel P_W-{\Psi }_{w_i\left(J\right)}^{({\mathcal{C}}_n,l)}\parallel \le e^{-{\gamma }_{5}n},l=0,1,\hfill \end{array}$$ (A67)

where ${\gamma }_4$ and ${\gamma }_5$ are some positive numbers. Since the average type I error probability for our scheme tends to zero asymptotically, and the error exponent is unaffected by a constant multiplicative scaling of the type II error probability, this codebook achieves the same type I error probability and error exponent as the average over the random ensemble. Using this deterministic codebook for encoding and decoding, we first lower bound the equivocation and average distortion of $S^n$ at the detector as follows:

First consider the equivocation of $S^n$ under the null hypothesis.

$$\begin{array}{cc}\hfill H_{{\tilde{P}}^{({\mathcal{C}}_n,0)}}\left(S^n\right|M,V^n)& \ge {\mathbb{P}}_{{\tilde{P}}^{({\mathcal{C}}_n,0)}}(M\ne 0)H\left(S^n\right|M\ne 0,V^n)\hfill \end{array}$$

$$\begin{array}{cc}\hfill & \ge (1-e^{-n\Omega \left(\delta \right)})H_{{\tilde{P}}^{({\mathcal{C}}_n,0)}}\left(S^n\right|M\ne 0,V^n)\hfill \end{array}$$ (A68)

$$\begin{array}{cc}\hfill & \ge (1-e^{-n\Omega \left(\delta \right)})H_{{\tilde{P}}^{({\mathcal{C}}_n,0)}}\left(S^n\right|w^n\left(J\right),V^n)\hfill \end{array}$$ (A69)

$$\begin{array}{cc}\hfill & =(1-e^{-n\Omega \left(\delta \right)})H_{{\tilde{P}}^{({\mathcal{C}}_n,0)}}\left(S^n\right|w^n\left(J\right),V^n)\hfill \end{array}$$ (A70)

$$\begin{array}{cc}\hfill & \ge (1-e^{-n\Omega \left(\delta \right)})H_{{\Psi }^{({\mathcal{C}}_n,0)}}\left(S^n\right|w^n\left(J\right),V^n)-2e^{-{\gamma }_{4}n}log\left(\frac{{\left|\mathcal{S}\right|}^n{\left|\mathcal{V}\right|}^n}{e^{-{\gamma }_{4}n}}\right)\hfill \end{array}$$ (A71)

$$\begin{array}{cc}\hfill & =\sum _{i=1}^n{H}_{{\Psi }^{({\mathcal{C}}_n,0)}}\left(S_i\right|w_i\left(J\right),V_i)-e^{-n\Omega \left(\delta \right)}\sum _{i=1}^n{H}_{{\Psi }^{({\mathcal{C}}_n,0)}}\left(S_i\right|w_i\left(J\right),V_i)-o\left(1\right)\hfill \end{array}$$ (A72)

$$\begin{array}{cc}\hfill & \ge \sum _{i=1}^n{H}_{{\Psi }^{({\mathcal{C}}_n,0)}}\left(S_i\right|w_i\left(J\right),V_i)-n{e}^{-n\Omega \left(\delta \right)}{H}_P\left(S\right|V)-o\left(1\right)\hfill \end{array}$$ (A73)

$$\begin{array}{cc}\hfill & =\left[\sum _{i=1}^n{H}_{{\Psi }^{({\mathcal{C}}_n,0)}}\left(S_i\right|w_i\left(J\right),V_i)\right]-o\left(1\right)\hfill \end{array}$$ (A74)

$$\begin{array}{ccc}\hfill & =n{H}_P\left(S\right|W,V)-o\left(1\right).\hfill & \hfill \end{array}$$ (A75)

Here, (A68) follows from (A27); (A69) follows since M is a function of $w^n\left(J\right)$ for a deterministic codebook; (A71) follows from (A65) and Lemma 3; (A72) follows from (A24); and (A75) follows from (A67) and ${\Psi }_{S_i{V}_i|w_i}^{\left(0\right)}=P_{SV|W}^{\left(0\right)},i\in \left[n\right]$.

If $Q_U=P_U$, it follows similarly to above that

$$\begin{array}{cc}\hfill H_{{\tilde{P}}^{({\mathcal{C}}_n,1)}}\left(S^n\right|M,V^n)& \ge \left(1-e^{-n\Omega \left(\delta \right)}\right)H_{{\Psi }^{({\mathcal{C}}_n,1)}}\left(S^n\right|w^n\left(J\right),V^n)-2e^{-{\gamma }_{4}n}log\left(\frac{{\left|\mathcal{S}\right|}^n{\left|\mathcal{V}\right|}^n}{e^{-{\gamma }_{4}n}}\right)\hfill \end{array}$$ (A76)

$$\begin{array}{cc}\hfill & =\sum _{i=1}^n{H}_{{\Psi }^{({\mathcal{C}}_n,1)}}\left(S_i\right|w_i\left(J\right),V_i)-e^{-n\Omega \left(\delta \right)}\sum _{i=1}^n{H}_{{\Psi }^{({\mathcal{C}}_n,1)}}\left(S_i\right|w_i\left(J\right),V_i)-o\left(1\right)\hfill \end{array}$$ (A77)

$$\begin{array}{cc}\hfill & \ge \sum _{i=1}^n{H}_{{\Psi }^{({\mathcal{C}}_n,1)}}\left(S_i\right|w_i\left(J\right),V_i)-n{e}^{-n\Omega \left(\delta \right)}{H}_Q\left(S\right|V)-o\left(1\right)\hfill \end{array}$$ (A78)

$$\begin{array}{cc}\hfill & =\left[\sum _{i=1}^n{H}_{{\Psi }^{({\mathcal{C}}_n,1)}}\left(S_i\right|w_i\left(J\right),V_i)\right]-o\left(1\right)\hfill \end{array}$$ (A79)

$$\begin{array}{ccc}\hfill & =n{H}_Q\left(S\right|W,V)-o\left(1\right).\hfill & \hfill \end{array}$$ (A80)

Finally, consider the case $H=1$ and $Q_U\ne P_U$. We have for ${\delta }^{\prime }$ small enough that

$$\begin{array}{c}\hfill {\mathbb{P}}_{{\tilde{P}}^{({\mathcal{C}}_n,1)}}\left(M=0\right)={\mathbb{P}}_{{\tilde{P}}^{({\mathcal{C}}_n,1)}}\left(U^n\notin {\mathcal{T}}_{{\left[P_U\right]}_{{\delta }^{\prime }}}^n\right)\ge 1-e^{-n\left(D\right(P_U\left|\right|Q_U)-O\left({\delta }^{\prime }\right))}\stackrel{\left(n\right)}{\to }1.\end{array}$$ (A81)

Hence, for ${\delta }^{\prime }$ small enough, we can write

$$\begin{array}{cc}\hfill & H_{{\tilde{P}}^{({\mathcal{C}}_n,1)}}\left(S^n\right|M,V^n)\ge H_{{\tilde{P}}^{({\mathcal{C}}_n,1)}}\left(S^n\right|M,V^n,\Pi (U^n,{\delta }^{\prime },P_U))\hfill \end{array}$$

$$\begin{array}{cc}\hfill & \ge \left(1-e^{-n\left(D\right(P_U\left|\right|Q_U)-O\left({\delta }^{\prime }\right))}\right)H_{{\tilde{P}}^{({\mathcal{C}}_n,1)}}\left(S^n\right|M,V^n,\Pi (U^n,{\delta }^{\prime },P_U)=1)\hfill \end{array}$$ (A82)

$$\begin{array}{cc}\hfill & =\left(1-e^{-n\left(D\right(P_U\left|\right|Q_U)-O\left({\delta }^{\prime }\right))}\right)H_{{\tilde{P}}^{({\mathcal{C}}_n,1)}}\left(S^n\right|V^n,\Pi (U^n,{\delta }^{\prime },P_U)=1)\hfill \end{array}$$ (A83)

$$\begin{array}{cc}\hfill & \ge \left(1-e^{-n\left(D\right(P_U\left|\right|Q_U)-O\left({\delta }^{\prime }\right))}\right)\left(H_{{\tilde{P}}^{({\mathcal{C}}_n,1)}}\left(S^n\right|V^n)-o\left(1\right)\right)\hfill \end{array}$$ (A84)

$$\begin{array}{ccc}\hfill & =n{H}_Q\left(S\right|V)-n{e}^{-n\left(D\right(P_U\left|\right|Q_U)-O\left({\delta }^{\prime }\right))}{H}_Q\left(S\right|V)-o\left(1\right)=n{H}_Q\left(S\right|V)-o\left(1\right).\hfill & \hfill \end{array}$$ (A85)

Here, (A82) follows from (A81); (A83) follows since $\Pi (U^n,{\delta }^{\prime },P_U)=1$ implies $M=0$; (A84) follows from Lemma 3 and (15). Thus, since $\delta >0$ is arbitrary, we have shown that for $ϵ\in (0,1)$, $(R,\kappa ,{\Lambda }_0,{\Lambda }_1)\in {\mathcal{R}}_e\left(ϵ\right)$ if (18)–(21) holds.

On the other hand, average distortion of $S^n$ at the detector can be lower bounded under $H=0$ as follows:

$$\begin{array}{cc}\hfill & \underset{g_{i,n}^{\left(r\right)}}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=0\right]\hfill \end{array}$$

$$\begin{array}{cc}\hfill & =\underset{{\left\{{\overline{\varphi }}_{i,n}(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}{\mathbb{E}}_{{\tilde{P}}^{({\mathcal{C}}_n,0)}}\left[\sum _{i=1}^{n}d\left(S_i,{\overline{\varphi }}_i(m,v^n,s^{i-1})\right)\right]\hfill \end{array}$$ (A86)

$$\begin{array}{cc}\hfill & \ge \underset{{\left\{{\overline{\varphi }}_i(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}{\mathbb{E}}_{{\Psi }^{({\mathcal{C}}_n,0)}}\left[\sum _{i=1}^{n}d(S_i,{\overline{\varphi }}_i(m,v^n,s^{i-1}))\right]-n{e}^{-n{\gamma }_4}{D}_m\hfill \end{array}$$ (A87)

$$\begin{array}{cc}\hfill & \ge \underset{{\left\{{\overline{\varphi }}_i(·,·)\right\}}_{i=1}^n}{min}{\mathbb{E}}_{{\Psi }^{({\mathcal{C}}_n,0)}}\left[\sum _{i=1}^{n}d(S_i,{\overline{\varphi }}_i(w_i\left(J\right),V_i))\right]-n{e}^{-n{\gamma }_4}{D}_m\hfill \end{array}$$ (A88)

$$\begin{array}{cc}\hfill & \ge n\underset{\left\{\varphi (·,·)\right\}}{min}{\mathbb{E}}_P\left[d(S,\varphi (W,V\left)\right)\right]-n\left(e^{-n{\gamma }_4}+e^{-n{\gamma }_5}\right)D_m\hfill \end{array}$$ (A89)

$$\begin{array}{ccc}\hfill & =n\underset{{\left\{\varphi (·,·)\right\}}_{i=1}^n}{min}{\mathbb{E}}_P\left[d(S,\varphi (W,V\left)\right)\right]-o\left(1\right).\hfill & \hfill \end{array}$$ (A90)

Here, (A86) follows from Lemma 2; (A87) follows from ([43], Property 2(b)) due to (A65) and boundedness of distortion measure; (A88) follows from the Markov chain in (A64); (A89) follows from (A67) and the fact that ${\Psi }_{S_i{V}_i|w_i\left(J\right)}^{\left(0\right)}=P_{SV|W}^{\left(0\right)},i\in \left[n\right]$.

Next, consider the case $H=1$ and $Q_U=P_U$. Then, similarly to above, we can write

$$\begin{array}{cc}\hfill & \underset{g_{i,n}^{\left(r\right)}}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=1\right]\hfill \\ \hfill & =\underset{{\left\{{\overline{\varphi }}_i(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}{\mathbb{E}}_{{\tilde{P}}^{({\mathcal{C}}_n,1)}}\left[\sum _{i=1}^{n}d\left(S_i,{\varphi }_i(M,V^n,S^{i-1})\right)\right]\hfill \\ \hfill & \ge \underset{{\left\{{\overline{\varphi }}_i(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}{\mathbb{E}}_{{\Psi }^{({\mathcal{C}}_n,1)}}\left[\sum _{i=1}^{n}d(S_i,{\varphi }_i(M,V^n,S^{i-1}))\right]-n{e}^{-n{\gamma }_4}{D}_m\hfill \end{array}$$ (A91)

$$\begin{array}{cc}\hfill & \ge \underset{{\left\{{\varphi }_i(·,·)\right\}}_{i=1}^n}{min}{\mathbb{E}}_{{\Psi }^{({\mathcal{C}}_n,1)}}\left[\sum _{i=1}^{n}d(S_i,{\varphi }_i(w_i,V_i))\right]-n{e}^{-n{\gamma }_4}{D}_m\hfill \end{array}$$ (A92)

$$\begin{array}{cc}\hfill & \ge n\underset{{\left\{\varphi (·,·)\right\}}_{i=1}^n}{min}{\mathbb{E}}_Q\left[d(S,\varphi (W,V\left)\right)\right]-n(e^{-n{\gamma }_4}+e^{-n{\gamma }_5})D_m.\hfill \end{array}$$ (A93)

$$\begin{array}{ccc}\hfill & =n\underset{{\left\{\varphi (·,·)\right\}}_{i=1}^n}{min}{\mathbb{E}}_Q\left[d(S,\varphi (W,V\left)\right)\right]-o\left(1\right).\hfill & \hfill \end{array}$$ (A94)

If $H=1$ and $Q_U\ne P_U$, we have

$$\begin{array}{cc}\hfill & \underset{g_{i,n}^{\left(r\right)}}{min}\mathbb{E}\left[d\left(S^n,{\widehat{S}}^n\right)|H=1\right]\hfill \\ \hfill & \ge {\mathbb{P}}_{{\tilde{P}}^{({\mathcal{C}}_n,1)}}\left(M=0|H=1\right)\underset{{\left\{{\overline{\varphi }}_i(m,v^n,s^{i-1})\right\}}_{i=1}^n}{min}\sum _{i=1}^n{\mathbb{E}}_{{\tilde{P}}^{({\mathcal{C}}_n,1)}}\left[d\left(S_i,{\varphi }_i(0,V^n,S^{i-1})\right)\right]\hfill \\ \hfill & \ge {\mathbb{P}}_{{\tilde{P}}^{({\mathcal{C}}_n,1)}}\left(M=0|H=1\right)\left[\underset{{\left\{{\varphi }_i^{\prime }\left(v\right)\right\}}_{i=1}^n}{min}{\mathbb{E}}_Q\left[\sum _{i=1}^{n}d(S_i,{\varphi }_i^{\prime }\left(V_i\right))\right]-D_{m}o\left(1\right)\right]\hfill \end{array}$$ (A95)

$$\begin{array}{ccc}\hfill & =n\underset{\left\{{\varphi }^{\prime }(·)\right\}}{min}{\mathbb{E}}_Q\left[d(S,{\varphi }^{\prime }\left(V\right))\right]-o\left(1\right).\hfill & \hfill \end{array}$$ (A96)

Here, (A96) follows from (15) in Lemma 4 and (A96) follows from (A81). Thus, since $\delta >0$ is arbitrary, we have shown that $(R,\kappa ,{\Delta }_0,{\Delta }_1)\in {\mathcal{R}}_d\left(ϵ\right)$, $ϵ\in (0,1)$, provided that (18), (19), (24) and (25) are satisfied. This completes the proof of the theorem.

Appendix E. Proof of Lemma 5

Consider the $\left|\mathcal{U}\right|+2$ functions of $P_{U|W}$,

$$\begin{array}{cc}\hfill & P_U\left(u_i\right)=\sum _{w\in \mathcal{W}}{P}_W\left(w\right)P_{U|W}\left(u_i\right|w),i=1,2,\dots ,\left|\mathcal{U}\right|-1,\hfill \end{array}$$ (A97)

$$\begin{array}{cc}\hfill & H_P\left(U\right|W,Z)=\sum _w{P}_W\left(w\right)g_1(P_{U|W},w),\hfill \end{array}$$ (A98)

$$\begin{array}{cc}\hfill & H_P\left(Y\right|W,Z)=\sum _w{P}_W\left(w\right)g_2(P_{U|W},w),\hfill \end{array}$$ (A99)

$$\begin{array}{cc}\hfill & H_P\left(S\right|W,Y,Z)=\sum _w{P}_W\left(w\right)g_3(P_{U|W},w),\hfill \end{array}$$ (A100)

where

$$\begin{array}{cc}\hfill g_1(P_{U|W},w)& =-\sum _{u,z}{P}_{U|W}\left(u\right|w)P_{Z|U}\left(z\right|u)log\left(\frac{P_{U|W}\left(u\right|w)P_{Z|U}\left(z\right|u)}{{\sum }_u{P}_{U|W}\left(u\right|w)P_{Z|U}\left(z\right|u)}\right),\hfill \\ \hfill g_2(P_{U|W},w)& =-\sum _{y,z,u}{P}_{U|W}\left(u\right|w)P_{YZ|U}(y,z|u)log\left(\frac{{\sum }_u{P}_{U|W}\left(u\right|w)P_{YZ|U}(y,z|u)}{{\sum }_u{P}_{U|W}\left(u\right|w)P_{Z|U}\left(z\right|u)}\right),\hfill \\ \hfill g_3(P_{U|W},w)& =-\sum _{s,y,z,u}{P}_{U|W}\left(u\right|w)P_{SYZ|U}(s,y,z|u)log\left(\frac{{\sum }_u{P}_{U|W}\left(u\right|w)P_{SYZ|U}(s,y,z|u)}{{\sum }_u{P}_{U|W}\left(u\right|w)P_{YZ|U}(y,z|u)}\right).\hfill \end{array}$$

Thus, by the Fenchel–Eggleston–Carathéodory’s theorem [68], it is sufficient to have at most $\left|\mathcal{U}\right|-1$ points in the support of W to preserve $P_U$ and three more to preserve $H_P\left(U\right|W,Z)$, $H_P\left(Y\right|W,Z)$ and $H_P\left(S\right|W,Z,Y)$. Noting that $H_P\left(Y\right|Z)$ and $H_P\left(U\right|Z)$ are automatically preserved since $P_U$ is preserved (and $(Y,Z,S)-U-W$ holds), $\left|\mathcal{W}\right|=\left|\mathcal{U}\right|+2$ points are sufficient to preserve the R.H.S. of Equations (28)–(30). This completes the proof for the case of ${\mathcal{R}}_e$. Similarly, considering the $\left|\mathcal{U}\right|+1$ functions of $P_{W|U}$ given in (A97)–(A99) and

$$\begin{array}{cc}\hfill {\mathbb{E}}_P\left[d\left(S,\varphi (W,Y,Z)\right)\right]& =\sum _w{P}_W\left(w\right)g_4(w,P_{W|U}),\hfill \end{array}$$

where

$$\begin{array}{cc}\hfill g_4(w,P_{W|U})& =\sum _{s,u,y,z}{P}_{U|W}\left(u\right|w)P_{YZS|U}(y,z,s|u)d(s,\varphi (w,y,z)),\hfill \end{array}$$

similar result holds also for the case of ${\mathcal{R}}_d$.

Author Contributions

Conceptualization, S.S., A.C. and D.G.; writing—original draft preparation, S.S.; supervision, A.C. and D.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research was partially funded by the European Research Council Starting Grant project BEACON (grant agreement number 677854).

Conflicts of Interest

The authors declare no conflict of interest.