On the arithmetic Kakeya conjecture of Katz and Tao

Abstract

The arithmetic Kakeya conjecture, formulated by Katz and Tao (Math Res Lett 6(5–6):625–630, 1999), is a statement about addition of finite sets. It is known to imply a form of the Kakeya conjecture, namely that the upper Minkowski dimension of a Besicovitch set in ${\mathbf{R}}^n$ is n. In this note we discuss this conjecture, giving a number of equivalent forms of it. We show that a natural finite field variant of it does hold. We also give some lower bounds.

Introduction and statement of results

The arithmetic Kakeya conjecture, sometimes known as the sums-differences conjecture, was formulated by Katz and Tao around fifteen years ago. It is a purely additive-combinatorial statement which, if true, would have a deep geometric consequence—that the Minkowski dimension of Besicovitch sets in ${\mathbf{R}}^n$ is n. This is the celebrated Kakeya conjecture, discussed at length in many places: for an introduction see [21].

The arithmetic Kakeya conjecture is mentioned explicitly1 in [20]. One of the main aims of this paper is to give a number of equivalent forms of the conjecture. Here is probably the simplest formulation. It is not the original one of Katz and Tao, which is Conjecture 1.3 below.

Conjecture 1.1

Let k, N be positive integers. Write $F_k(N)$ for the size of the smallest set of integers containing, for each $d\in \{1,\cdots ,N\}$, a k-term arithmetic progression with common difference d. Then

$$\begin{array}{c}\hfill \underset{k\to \infty }{lim}\underset{N\to \infty }{lim}\frac{log{F}_k(N)}{logN}=1.\end{array}$$

This conjecture was raised by the second author as [17, Conjecture 4.2], but no links to the Kakeya problem were mentioned there.

We turn now to arguably the most natural of our formulations, concerning the entropy of random variables. As usual, the entropy $\mathbf{H}$ of a random variable $\mathsf{X}$ with finite range is defined by

$$\begin{array}{c}\hfill \mathbf{H}(\mathsf{X}):=-\sum _x\mathbf{P}(\mathsf{X}=x)log\mathbf{P}(\mathsf{X}=x),\end{array}$$

where x ranges over all values taken by $\mathsf{X}$.

Conjecture 1.2

For any $\epsilon >0$ there are2$r_1,\cdots ,r_k\in \mathbf{Q}$, none equal to $-1$, such that for any two real-valued random variables $\mathsf{X}$ and $\mathsf{Y}$ taking only finitely many values we have

$$\begin{array}{c}\hfill \mathbf{H}(\mathsf{X}-\mathsf{Y})⩽(1+\epsilon )\underset{j}{sup}\mathbf{H}(\mathsf{X}+r_j\mathsf{Y}).\end{array}$$

Next we give the original form of the conjecture discussed by Katz and Tao. Let $A\subset \mathbf{Z}\times \mathbf{Z}$ be a finite set. For rational r we write ${\pi }_r(A):=\{x+ry:(x,y)\in A\}$. We also write ${\pi }_{\infty }(A):=\{y:(x,y)\in A\}$.

Conjecture 1.3

Let $\epsilon >0$ be arbitrary. Then there are $r_1,\cdots ,r_k\in \mathbf{Q}\cup \{\infty \}$, none equal to $-1$, such that $\#{\pi }_{-1}(A)⩽{sup}_i\#{\pi }_{r_i}{(A)}^{1+\epsilon }$ for all finite sets $A\subset \mathbf{Z}\times \mathbf{Z}$.

Our fourth conjecture has not, so far as we are aware, appeared explicitly in the literature before. It is in fact a whole family of conjectures, one for each natural number n; however, we will later show that all of these are equivalent.

Conjecture 1.4

(n) Let k be a positive integer. If p is a prime, let $f_{k,n}(p)$ denote the size of the smallest set containing, for every $d\in {\mathbf{F}}_p^n\backslash \{0\}$, a k-term progression with common difference d. Then

$$\begin{array}{c}\hfill \underset{k\to \infty }{lim}\underset{p\to \infty }{lim}\frac{log{f}_{n,k}(p)}{logp}=n.\end{array}$$

Remarks

Note that $f_{p,n}(p)$ is the size of the smallest Besicovitch set in ${\mathbf{F}}_p^n$, that is to say a set containing a full line in every direction. Since $f_{p,n}(p)⩾f_{k,n}(p)$ whenever $p⩾k$, Conjecture 1.4 (n) trivially implies that

$$\begin{array}{c}\hfill \underset{p\to \infty }{lim}\frac{f_{p,n}(p)}{logp}=n,\end{array}$$

i.e. any Besicovitch set in ${\mathbf{F}}_p^n$ has size $p^{n-o_{p\to \infty }(1)}$. This is known to be true, a celebrated result of Dvir [5]. However, the only known arguments use the “polynomial method” (see, for example, [11, 22] for modern introductions). This very strongly hints that any proof of Conjecture 1.4 (and hence, by our main theorem, of the other conjectures) would have to use some form of the polynomial method.

Our fifth and final conjecture is included mainly for historical interest, as it relates very closely to a question asked by Erdős and Selfridge in the 1970s, well before the current wave of interest in the Kakeya problem and related matters.

Conjecture 1.5

Fix a positive integer k. Then, uniformly for all positive integers N, all finite sets $p_1<\cdots <p_N$ of primes and all intervals $I\subset \mathbf{N}$ of length $k{p}_N$, we have

$$\begin{array}{c}\hfill \#(I\cap \bigcup _{i=1}^N{p}_i\mathbf{Z}){\gg }_k{N}^{1-{\gamma }_k}.\end{array}$$

where ${\gamma }_k\to 0$ as $k\to \infty$.

Remark

Erdős and Selfridge [8, §6] in fact asked whether or not one can take ${\gamma }_k=0$. The second-named author [16] showed that the answer is negative, and in fact we must have ${\gamma }_k⩾\frac{1}{k}$. We note that Proposition 4.1 and Theorem 1.7 combine to give the much better bound ${\gamma }_k\gg \frac{1}{loglogk}$.

As previously stated, our main result is the equivalence of the five conjectures stated above.

Theorem 1.6

Conjectures 1.1, 1.2, 1.3, 1.4(n) (for each $n=1,2,3,\cdots$) and 1.5 are all equivalent.

Let us make some further remarks.

1. Once Theorem 1.6 is proven, it seems natural to use the term “arithmetic Kakeya conjecture” to refer to any one of the five conjectures.

2. It is known that Conjecture 1.3 (and hence all the other conjectures) implies that the upper Minkowski dimension of any Besicovitch set3 in ${\mathbf{R}}^n$ is n, a statement often referred to as the Kakeya conjecture. This follows by a straightforward generalisation of the “slicing” argument of Bourgain [4]: a sketch of this may be found in [21]. However, Bourgain [2, 3] observed that, in the notation of Conjecture 1.1, if the statement

   $$\begin{array}{c}\hfill \underset{N\to \infty }{lim}\frac{log{F}_{N^{\eta }}(N)}{logN}⩾1\end{array}$$ 1.1

   is true for all $\eta >0$ then the Kakeya conjecture follows. Since $F_k(N)$ is a nondecreasing function of k, (1.1) is immediately implied by Conjecture 1.1, whilst an implication in the reverse direction seems very unlikely without resolving both conjectures. In this sense, the arithmetic Kakeya conjecture should be considered a strictly harder problem than the Kakeya conjecture.

3. The equivalence of Conjectures 1.2 and 1.3 was proven by the second author in [18] (see also [14]). We are not aware of any references for the other implications.

Now we discuss the other results in the paper. First, we establish a lower bound showing that the convergence in Theorem 1, if it occurs, is very slow.

Theorem 1.7

In the notation of Conjecture 1.1, we have

$$\begin{array}{c}\hfill \underset{N\to \infty }{lim}\frac{log{F}_k(N)}{logN}⩽1-\frac{c}{loglogk},\end{array}$$

where the constant $c>0$ is absolute.

Second, we show that a finite field variant of Conjecture 1.2is true. Write ${\mathbf{F}}_p^{\infty }:={\bigcup }_{n=1}^{\infty }{\mathbf{F}}_p^n$.

Theorem 1.8

Suppose that $\mathsf{X}$ and $\mathsf{Y}$ are two ${\mathbf{F}}_p^{\infty }$-valued random variables, both taking only finitely many values. Then

$$\begin{array}{c}\hfill \mathbf{H}(\mathsf{X}-\mathsf{Y})⩽\left(1,+,O,\left(\frac{1}{logp}\right)\right)\underset{r\in {\mathbf{F}}_p\cup \{\infty \}\backslash \{-1\}}{sup}\mathbf{H}(\mathsf{X}+r\mathsf{Y}).\end{array}$$

Here, the constant in the O() notation is absolute.

The $O(\frac{1}{logp})$ term is best possible, as we remark in Sect. 6.

We neither discuss nor make progress on partial results towards any of Conjectures 1.1, 1.2, 1.3, 1.4 or 1.5. We believe that the best value of $\epsilon$ for which Conjecture 1.2 is known is $\epsilon \approx 0.67513\cdots$, which is equivalent to a result obtained in [13]. (The precise value here is $\alpha -1$, where $\alpha$ solves ${\alpha }^3-4\alpha +2=0$.) This bound is now 15 years old.

Notation. Most of our notation is quite standard. We use $\#X$ for the cardinality of a set X. Occasionally, if A is a set in some abelian group and k is an integer we will write $k·A$ to mean $\{ka:a\in A\}$.

Progressions, projections and entropy

In this section we establish around half of Theorem 1.6 by proving that the first three conjectures mentioned in the introduction are equivalent. Whilst at a local level the arguments are a mix of fairly unexciting linear algebra and standard tools such as Freiman isomorphisms, random projections and taking tensor powers, the large number of them makes the proof of Theorem 1.6 somewhat lengthy.

It is convenient to proceed by first showing that Conjectures 1.1, 1.3 and 1.2 are equivalent. In the course of doing so, and for later use, it is convenient to introduce a further conjecture, apparently stronger than Conjecture 1.1 but, as it turns out, equivalent to it.

Conjecture 1’. Let k be a positive integer. Write $F_k^{\prime }(N)$ for the cardinality of the smallest set $A\subset \mathbf{Z}$ which contains an arithmetic progression of length k and common difference d, for N different values of d. Then

$$\begin{array}{c}\hfill \underset{k\to \infty }{lim}\underset{N\to \infty }{lim}\frac{log{F}_k^{\prime }(N)}{logN}=1.\end{array}$$

It is obvious that Conjecture 1’ implies Conjecture 1.1, because $F_k^{\prime }(N)⩽F_k(N)$. It turns out that the reverse implication holds as well. In fact, we claim that the following is true.

Proposition 2.1

We have $F_k(N)\ll k^{3}logN·F_k^{\prime }(N)$.

Proof

Suppose we have a set

$$\begin{array}{c}\hfill A_0=\bigcup _{i=1}^N\bigcup _{j=0}^{k-1}\{a_i+j{d}_i\},\end{array}$$

where the $d_i$ are distinct. We claim that there is a set $A_1$, $\#A_1\ll k^{3}logN·\#A_0$, containing an arithmetic progression of length k and common difference d for all $d\in \{1,\cdots ,N\}$. This obviously implies the result.

Pick $\theta \in (0,1)$ uniformly at random, and define the function

$$\begin{array}{c}\hfill {\varphi }_{\theta }:\mathbf{Z}\to \{0,1,\cdots ,N-1\}\end{array}$$

by

$$\begin{array}{c}\hfill {\varphi }_{\theta }(x):=\lfloor N\{\theta x\}\rfloor .\end{array}$$

Here, $\{t\}=t-\lfloor t\rfloor$, so $0⩽\{t\}<1$.

Note that if $i\ne j$ then

$$\begin{array}{c}\hfill {\mathbf{P}}_{\theta }({\varphi }_{\theta }(d_i)={\varphi }_{\theta }(d_j))⩽{\mathbf{P}}_{\theta }(\theta (d_i-d_j)\in \left(-,\frac{1}{N},,,\frac{1}{N}\right)(\text{mod}1))=\frac{2}{N}.\end{array}$$

It follows that the expected number of pairs (i, j) with $i<j$ for which ${\varphi }_{\theta }(d_i)={\varphi }_{\theta }(d_j)$ is at most $\frac{2}{N}\left(\begin{array}{c}N\\ 2\end{array}\right)=N-1$. By linearity of expectation, there is some choice of $\theta$ for which, setting $d_i^{\prime }:={\varphi }_{\theta }(d_i)$, there are at most $N-1$ pairs (i, j) with $i<j$ and $d_i^{\prime }=d_j^{\prime }$. If $n\in \{0,1,\cdots ,N-1\}$, write f(n) for the number of i with $d_i^{\prime }=n$. Then it follows that ${\sum }_n\left(\begin{array}{c}f(n)\\ 2\end{array}\right)⩽N-1$, from which we obtain, since ${\sum }_{n}f(n)=N$, that ${\sum }_{n}f{(n)}^2⩽3N$. By Cauchy–Schwarz,

$$\begin{array}{c}\hfill N^2={\left(\sum _n,f,(n)\right)}^2⩽\#\{n:f(n)\ne 0\}\sum _{n}f{(n)}^2,\end{array}$$

and therefore there are at least N / 3 values of n for which $f(n)\ne 0$, or in other words there are at least N / 3 distinct values amongst the $d_i^{\prime }$.

Now consider the set $A_2:={\varphi }_{\theta }(A_0)$. Obviously $\#A_2⩽\#A_0$. Whilst $A_2$ itself does not obviously contain any long progressions, we observe that

$$\begin{array}{c}\hfill {\varphi }_{\theta }(a_i+(j+1)d)-{\varphi }_{\theta }(a_i+jd)-d_i^{\prime }\in \{0,1\}-\{0,N\}\end{array}$$

(In fact, ${\varphi }_{\theta }(x+y)-{\varphi }_{\theta }(x)-{\varphi }_{\theta }(y)\in \{0,1\}-\{0,N\}$ for every x, y.) By a simple induction,

$$\begin{array}{c}\hfill {\varphi }_{\theta }(a_i)+j{d}_i^{\prime }-{\varphi }_{\theta }(a_i+jd)\in \{0,1,\cdots ,k-1\}-\{0,N,\cdots ,(k-1)N\}\end{array}$$

for $j=0,1,\cdots ,k-1$, and so the set $A_3:=A_2+\{0,1,\cdots ,k-1\}-\{0,N,\cdots ,(k-1)N\}$ contains a progression of length k and common difference $d_i^{\prime }$, for all i. Note that $\#A_3⩽k^2\#A_0$.

By taking random translates (see Lemma 6.2 for details) and the fact that there are $⩾N/3$ distinct $d_i^{\prime }$, there is some set T of integers, $\#T\ll logN$, such that every element of $\{1,\cdots ,N\}$ can be written as $d_i^{\prime }+t$ with $t\in T$. Set

$$\begin{array}{c}\hfill A_1:=A_3+\{0,1,\cdots ,k-1\}·T.\end{array}$$

We have $\#A_1⩽k·\#T·\#A_3\ll k^{3}logN·\#A_0$. It is easy to see that $A_1$ contains an arithmetic progression of length k and common difference $d_i^{\prime }+t$, for all i and for all $t\in T$, and hence contains an arithmetic progression of length k and common difference d for all $d\in \{1,\cdots ,N\}$. This concludes the proof of Proposition 2.1. $\square$

Now we turn to the proof that Conjectures 1’, 1.2 and 1.3 are equivalent.

Conjecture 1’ implies Conjecture 1.3. Suppose that Conjecture 1.3 is false. Then there is some $\epsilon >0$ such that, for every k, there is a set $A_k\subset \mathbf{Z}\times \mathbf{Z}$ such that

$$\begin{array}{c}\hfill \#{\pi }_{-1}(A_k)>\underset{r\in H_k\backslash \{-1\}}{max}\#{\pi }_r{(A_k)}^{1+\epsilon },\end{array}$$ 2.1

where $H_k$ denotes the set of rationals with height at most k, that is to say

$$\begin{array}{c}\hfill H_k:=\left\{\frac{a}{b},:|a|,|b|,⩽,k\right\}\cup \{\infty \}.\end{array}$$

Our first step is to use a “tensor power” argument to show that there are arbitrarily large sets with the same property; in fact, we shall argue that for every j there is a set $A_{k,j}\subset \mathbf{Z}\times \mathbf{Z}$ such that

$$\begin{array}{c}\hfill \#{\pi }_{-1}(A_{k,j})⩾j\underset{r\in H_k\backslash \{-1\}}{max}\#{\pi }_r{(A_{k,j})}^{1+\epsilon }.\end{array}$$ 2.2

This is simple if the $A_{k,j}$ are allowed to be subsets of ${\mathbf{Z}}^n$. Indeed we may define $A_k^{(n)}$ to be the set

$$\begin{array}{c}\hfill \left\{(,(a_1,a_2,\cdots ,a_n),,,(a_1^{\prime },a_2^{\prime },\cdots ,a_n^{\prime }),),\in ,{\mathbf{Z}}^n,\times ,{\mathbf{Z}}^n,:,(a_i,a_i^{\prime }),\in ,A_k,,,\text{for},,,\text{all},,i\right\}.\end{array}$$

Then, writing ${\pi }_r^{(n)}:{\mathbf{Z}}^n\times {\mathbf{Z}}^n\to {\mathbf{Z}}^n$, for the map sending (x, y) to $x+ry$ (or, when $r=\infty$, to y) we have

$$\begin{array}{c}\hfill \#{\pi }_r^{(n)}(A_k^{(n)})=(\#{\pi }_r(A_k){)}^n\end{array}$$

for all r, n. In particular, by choosing n large enough (depending on j) we have, since the inequality (2.1) is strict,

$$\begin{array}{c}\hfill \#{\pi }_{-1}^{(n)}(A_k^{(n)})⩾j\underset{r\in H_k\backslash \{-1\}}{max}\#{\pi }_r^{(n)}{\left(A_k^{(n)}\right)}^{1+\epsilon }.\end{array}$$ 2.3

To create a subset of $\mathbf{Z}\times \mathbf{Z}$ from $A_k^{(n)}$, we take an integer t and apply a map ${\psi }_t:{\mathbf{Z}}^n\times {\mathbf{Z}}^n\to \mathbf{Z}\times \mathbf{Z}$ of the form

$$\begin{array}{c}\hfill {\psi }_t(x,y)=((t,t^2,\cdots ,t^n)·x,(t,t^2,\cdots ,t^n)·y)),\end{array}$$

where the dot denotes the usual inner product. Set $A:={\psi }_t(A_k^{(n)})$. Choose t such that for $r\in H_k$ and $(x,y),(x^{\prime },y^{\prime })\in A_k^{(n)}$ we have

$$\begin{array}{c}\hfill ({\pi }_r^{(n)}(x,y)-{\pi }_r^{(n)}(x^{\prime },y^{\prime }))·(t,t^2,\cdots ,t^n)\ne 0\end{array}$$ 2.4

unless ${\pi }_r^{(n)}(x,y)={\pi }_r^{(n)}(x^{\prime },y^{\prime })$. There is such a t, because for each of the finite number of choices of $x,y,x^{\prime },y^{\prime },r$ the left-hand side of (2.4) is a nontrivial polynomial equation in t. For such a choice of t it follows that ${\pi }_r({\psi }_t(x,y))={\pi }_r({\psi }_t(x^{\prime },y^{\prime }))$ if and only if ${\pi }_r^{(n)}(x,y)={\pi }_r^{(n)}(x^{\prime },y^{\prime })$, and so

$$\begin{array}{c}\hfill \#{\pi }_r(A)=\#{\pi }_r^{(n)}(A_k^{(n)})\end{array}$$

for all r. This establishes the existence of the sets $A_{k,j}$ satisfying (2.2).

For each j, k, consider the set $S_{k,j}\subset \mathbf{Q}$ defined by

$$\begin{array}{c}\hfill S_{k,j}:=\bigcup _{1⩽i⩽k}\bigcup _{r\in H_k\backslash \{-1\}}\frac{i}{k}·{\pi }_r(A_{k,j}).\end{array}$$

Then

$$\begin{array}{c}\hfill \#S_{k,j}⩽k·\#H_k·\underset{r\in H_k^{+}}{max}\#{\pi }_r(A_{k,j}){\ll }_k{\left(j^{-1},\#,{\pi }_{-1},(A_{k,j})\right)}^{1/(1+\epsilon )}.\end{array}$$

On the other hand, suppose that $d\in -{\pi }_{-1}(A_{k,j})$. This means that $d=y-x$ for some $(x,y)\in A_{k,j}$. If $0⩽i⩽k-1$, we have

$$\begin{array}{c}\hfill x+\frac{\mathit{id}}{k}=\frac{k-i}{k}\left(x,+,\frac{i}{k-i},y\right).\end{array}$$

Since $x+\frac{i}{k-i}y\in {\pi }_{i/(k-i)}(A_{k,j})\subset {\bigcup }_{r\in H_k^{+}}{\pi }_r(A_{k,j})$, it follows that $x+\frac{\mathit{id}}{k}\in S_{k,j}$ for $i=0,1,\cdots ,k-1$, that is to say $S_{k,j}$ contains a progression of length k and common difference $\frac{d}{k}$. Thus, writing $N_j:=\#{\pi }_{-1}(A_{k,j})$, we see that $S_{k,j}$ is a set of size $\ll {(j^{-1}{N}_j)}^{1/(1+\epsilon )}$ containing progressions of length k with at least $N_j$ distinct common differences. Since, evidently, $\#S_{k,j}⩾k$, the presence of the factor $j^{-1}$ forces $N_j\to \infty$ as $j\to \infty$. By multiplying through by an appropriate integer, we may find sets ${\tilde{S}}_{k,j}\subset \mathbf{Z}$ with the same property, contrary to Conjecture 1’.

Conjecture 1.3implies Conjecture 1.2. This implication is essentially given in [18]. The notation there takes a little unpicking and the proof is short, so we repeat the argument.

Let $\epsilon >0$, and suppose that $r_1,\cdots ,r_k\in {\mathbf{Q}}_{⩾0}\cup \{\infty \}\backslash \{-1\}$ are such that

$$\begin{array}{c}\hfill \#{\pi }_{-1}(A)⩽\underset{i}{sup}\#{\pi }_{r_i}{(A)}^{1+\epsilon }\end{array}$$ 2.5

for all finite sets $A\subset \mathbf{Z}\times \mathbf{Z}$. We claim that

$$\begin{array}{c}\hfill \mathbf{H}(\mathsf{X}-\mathsf{Y})⩽(1+\epsilon )\underset{j}{sup}\mathbf{H}(\mathsf{X}+r_j\mathsf{Y}).\end{array}$$ 2.6

for all $\mathbf{Z}$-valued random variables $\mathsf{X},\mathsf{Y}$, both taking only finitely many values. (Let us remind the reader that, by convention, $\mathbf{H}(\mathsf{X}+\infty \mathsf{Y})=\mathbf{H}(\mathsf{Y})$.)

We begin with a couple of observations. The first is that (2.5) is automatically true for sets $A\subset {\mathbf{Z}}^n\times {\mathbf{Z}}^n$, for any n. This follows from the case $n=1$ by applying a suitable map ${\psi }_t:{\mathbf{Z}}^n\times {\mathbf{Z}}^n\to \mathbf{Z}\times \mathbf{Z}$, exactly as in the argument following (2.3) above.

The second observation is that, by a simple limiting argument, we may assume that there is some q such that $q\mathbf{P}((\mathsf{X},\mathsf{Y})=(x,y))\in \mathbf{Z}$ for all (x, y): if we can prove the result for such $(\mathsf{X},\mathsf{Y})$ the same inequality for arbitrary $(\mathsf{X},\mathsf{Y})$ with finite range follows by letting $q\to \infty$.

Now let m be very large, and construct a set $A\subset {\mathbf{Z}}^{\mathit{mq}}\times {\mathbf{Z}}^{\mathit{mq}}$ as follows. Let it consist of all pairs $((x_1,\cdots ,x_{\mathit{mq}}),(y_1,\cdots ,y_{\mathit{mq}}))\in {\mathbf{Z}}^{\mathit{mq}}\times {\mathbf{Z}}^{\mathit{mq}}$ for which

$$\begin{array}{c}\hfill \#\{i:(x_i,y_i)=(x,y)\}=mq\mathbf{P}((\mathsf{X},\mathsf{Y})=(x,y)).\end{array}$$

Let us calculate $\#{\pi }_r(A)$. After a moment’s thought we see that

$$\begin{array}{c}\hfill {\pi }_r(A)=\{(z_1,\cdots ,z_{\mathit{mq}}):\#\{i:z_i=z\}=mq\mathbf{P}(\mathsf{X}+r\mathsf{Y}=z)\}.\end{array}$$

(Here, we interpret $\mathbf{P}(X+\infty \mathsf{Y}=z)$ as $\mathbf{P}(\mathsf{Y}=z)$.) Writing $n=mq$ and $p_z=\mathbf{P}(\mathsf{X}+r\mathsf{Y}=z)$ for short, it follows that

$$\begin{array}{c}\hfill \#{\pi }_r(A)=\frac{n!}{{\prod }_z(n{p}_z)!}.\end{array}$$

Note that the product over z is finite, and that each $n{p}_z$ is an integer. Taking logs and using the fact that $logN!=NlogN-N+o(N)$, we have

$$\begin{array}{c}\hfill log{\pi }_r(A)=-n\sum _z{p}_{z}log{p}_z+o(n)=n\mathbf{H}(\mathsf{X}+r\mathsf{Y})+o(n).\end{array}$$

We may assume that the o(n) term is uniform in $r\in \{r_1,\cdots ,r_k\}$ (since this is a finite set); of course, it also depends on $\mathsf{X},\mathsf{Y}$, but we are thinking of these as fixed for the duration of the argument.

Taking logs of (2.5) (which is valid for $A\subset {\mathbf{Z}}^n\times {\mathbf{Z}}^n$, as remarked), we conclude that

$$\begin{array}{c}\hfill n\mathbf{H}(\mathsf{X}-\mathsf{Y})⩽(1+\epsilon )n\underset{i}{sup}\mathbf{H}(\mathsf{X}+r_i\mathsf{Y})+o(n).\end{array}$$

Now we may simply divide through by n and let $n\to \infty$ to conclude the claim (2.6).

Conjecture 1.2implies Conjecture 1.1. This is relatively easy. Assume Conjecture 1.2. Let $\epsilon >0$ be arbitrary, and select $r_1,\cdots ,r_m\in \mathbf{Q}\cup \{\infty \}\backslash \{-1\}$ so that we have

$$\begin{array}{c}\hfill \mathbf{H}(\mathsf{X}-\mathsf{Y})⩽(1+\epsilon )\underset{i}{sup}\mathbf{H}(\mathsf{X}+r_i\mathsf{Y}).\end{array}$$ 2.7

Let Q, M be positive integers to be specified later (depending on $r_1,\cdots$, $r_m$) and suppose that $A\subset \mathbf{Z}$ contains an arithmetic progression of length $k=2MQ$ and common difference d, for every $d\in \{1,\cdots ,N\}$. Define $\mathbf{Z}$-valued random variables $\mathsf{X}$, $\mathsf{Y}$ as follows: pick d uniformly at random, and let $\{a(d),\cdots ,a(d)+(k-1)d\}$ be the progression in A for which a(d) is minimal (choosing a(d) minimal is not important, but is one way of making a definite choice). Set $\mathsf{X}=a(d)+MQd$ and $\mathsf{Y}=a(d)+(M+1)Qd$.

Then $\mathsf{X}-\mathsf{Y}$ is uniformly distributed on the set $\{-Q,-2Q,\cdots ,-NQ\}$, and so

$$\begin{array}{c}\hfill \mathbf{H}(\mathsf{X}-\mathsf{Y})=logN.\end{array}$$ 2.8

On the other hand,

$$\begin{array}{c}\hfill \mathbf{H}(X+r_j\mathsf{Y})=\mathbf{H}\left(\frac{\mathsf{X}+r_j\mathsf{Y}}{1+r_j}\right)=\mathbf{H}\left(a,(d),+,(QM+\frac{Q{r}_j}{1+r_j}),d\right).\end{array}$$

By choosing Q and then M suitably, we may ensure that all the $Q{r}_j/(1+r_j)$ are integers of magnitude $<QM$, which means that

$$\begin{array}{c}\hfill a(d)+\left(Q,M,+,\frac{Q{r}_j}{1+r_j}\right)d\in \{a(d),\cdots ,a(d)+(k-1)d\}\subset A.\end{array}$$

That is, $\mathsf{X}+r_j\mathsf{Y}$ takes values in $(1+r_j)·A$. Since $\mathbf{H}(\mathsf{W})⩽logm$ for any random variable $\mathsf{W}$ taking values in a set of size m, this implies that

$$\begin{array}{c}\hfill \mathbf{H}(\mathsf{X}+r_j\mathsf{Y})⩽log\#A.\end{array}$$

Combining this with (2.7) and (2.8) we obtain

$$\begin{array}{c}\hfill logN⩽(1+\epsilon )log\#A,\end{array}$$

or in other words

$$\begin{array}{c}\hfill \#A⩾N^{1/(1+\epsilon )}.\end{array}$$

Since $\epsilon$ was arbitrary, the implication follows.

This completes the proof that Conjectures 1.1, 1’, 1.2 and 1.3 are equivalent.

Finite fields

Next we turn to Conjecture 1.4 (n). To demonstrate its equivalence to the first three conjectures, it suffices to show that for each n we have Conjecture 1’ $\Rightarrow$ Conjecture 1.4 (n) $\Rightarrow$ Conjecture 1.1.

Conjecture 1’ implies Conjecture 1.4 (n). Suppose that $A_1\subset {\mathbf{F}}_p^n$ is a set containing a k-term arithmetic progression with common difference d, for every $d\in {\mathbf{F}}_p^n$. Define the “unwrapping” map $\psi :{\mathbf{F}}_p\to \mathbf{Z}$ to be the inverse of the natural projection map from $\{0,\cdots ,p-1\}$ to ${\mathbf{F}}_p$. Define a map ${\psi }^{(n)}:{\mathbf{F}}_p^n\to {\mathbf{Z}}^n$ by setting ${\psi }^{(n)}(x_1,\cdots ,x_n):=(\psi (x_1),\cdots ,\psi (x_n))$.

For each $d\in {\mathbf{F}}_p^n$, select a progression $\{x(d)+\lambda d,\lambda =0,1,\cdots ,k-1\}$, lying in $A_1$. Let $A_2\subset {\mathbf{Z}}^n$ be the union of all progressions $\{{\psi }^{(n)}(x(d))+\lambda {\psi }^{(n)}(d):\lambda =0,1,\cdots ,k-1\}$. By construction, $A_2\subset {\{0,1,\cdots ,k(p-1)\}}^n$, and ${\pi }^{(n)}(A_2)\subset A_1$, where ${\pi }^{(n)}:{\mathbf{Z}}^n\to {\mathbf{F}}_p^n$ is the natural map. Since $\{0,1,\cdots ,k(p-1)\}$ is covered by k discrete intervals of length p, on each of which the projection map $\pi :\mathbf{Z}\to {\mathbf{F}}_p$ is injective, we see that $\#A_2⩽k^n\#A_1$.

By construction, $A_2$ contains a progression of length k and common difference d for $p^n$ distinct values of d. Whilst $A_2$ is a subset of ${\mathbf{Z}}^n$, we can create a subset of $\mathbf{Z}$ with the same properties by looking at the image of $A_2$ under the map $f:{\mathbf{Z}}^n\to \mathbf{Z}$ defined by $f(x_1,\cdots ,x_n)={\sum }_{i=1}^n{(10kp)}^i{x}_i$. It follows that $\#A_2⩾F_k^{\prime }(p^n)$, and hence $\#A_1⩾k^{-n}{F}_k^{\prime }(p^n)$. In the notation of Conjecture 1.4, this means that $f_{n,k}(p)⩾k^{-n}{F}_k^{\prime }(p^n)$. It follows that

$$\begin{array}{c}\hfill \underset{p\to \infty }{lim}\frac{log{f}_{n,k}(p)}{logp}⩾n\underset{p\to \infty }{lim}\frac{log{F}_k^{\prime }(p^n)}{log{p}^n},\end{array}$$

and so

$$\begin{array}{c}\hfill \underset{k\to \infty }{lim}\underset{p\to \infty }{lim}\frac{log{f}_{n,k}(p)}{logp}⩾n\underset{k\to \infty }{lim}\underset{p\to \infty }{lim}\frac{log{F}_k^{\prime }(p^n)}{log{p}^n}.\end{array}$$

Assuming Conjecture 1’ (taking $N=p^n$), the right hand side here is precisely n. This implies Conjecture 1.4.

Conjecture 1.4implies Conjecture 1.1. Suppose we have a set $A_1\subset \mathbf{Z}$ containing a progression of length k and common difference d for each $d\in \{1,\cdots ,N\}$. Partition $\mathbf{Z}$ into intervals $I_j:=10kjN+\{1,\cdots ,10kN\}$, $j\in \mathbf{Z}$. Any progression of length k and common difference $d\in \{1,\cdots ,N\}$ is either wholly contained in some $I_j$, or else is split into two progressions, one in $I_j$ and the other in $I_{j+1}$, with one of these having length at least k / 2. It follows that the set $A_2\subset \mathbf{Z}$ defined by4

$$\begin{array}{c}\hfill A_2=\bigcup _j\{(A_1\cap I_j)-10kjN\}\end{array}$$

contains a progression of length at least k / 2 and common difference d, for all $d\in \{1,\cdots ,N\}$. Manifestly $\#A_2⩽\#A_1$, and by construction $A_2$ has the additional property that

$$\begin{array}{c}\hfill A_2\subset \{1,\cdots ,10kN\}.\end{array}$$ 3.1

Using $A_2$, we construct a set $A_3\subset {\mathbf{Z}}^n$. We will later use this to construct a further set $A_4\subset {\mathbf{F}}_p^n$, for a suitable prime p, by projection. To define $A_3$, let $M:=\lfloor N^{1/n}\rfloor$. Select $t\in \{-10kN,\cdots ,20kN-1\}$ uniformly at random, and define

$$\begin{array}{c}\hfill A_3(t):=\left\{(x_1,\cdots ,x_n),\in ,{\{0,\cdots ,M-1\}}^n,:,\sum _{i=1}^n,M^{i-1},x_i,\in ,A_2,+,t\right\}.\end{array}$$

Suppose that $d={\sum }_{i=1}^n{M}^{i-1}{d}_i$ with $0⩽d_i⩽M/2k$ for all i. There are at least ${(M/4k)}^n$ such values of d, and all lie in $\{0,\cdots ,N\}$. For each such d there is, by assumption, a progression $\{x(d)+\lambda d:\lambda =0,1,\cdots ,\lfloor k/2\rfloor -1\}$ lying in $A_2$. The progression $\{x(d)+t+\lambda d:\lambda =0,1,\cdots ,\lfloor k/2\rfloor -1\}$ then lies in $A_2+t$. Write

$$\begin{array}{c}\hfill S:=\left\{\sum _{i=1}^n,M^{i-1},s_i,:,0,⩽,s_i,<,M,/,2,,,\text{for},,,\text{all},,i\right\}.\end{array}$$

If it so happens that $t\in -x(d)+S$ then $A_3(t)$ contains a progression of length k and common difference $(d_1,\cdots ,d_n)$, namely $\{(s_1,\cdots ,s_n)+\lambda (d_1,\cdots ,d_n):\lambda \in \{0,1,\cdots ,k-1\}\}$, where $x(d)+t={\sum }_{i=1}^n{M}^{i-1}{s}_i$.

Since $0⩽x(d)⩽10kN$ and $S\subset \{0,1,\cdots ,M^n\}$, $-x(d)+S\subset \{-10kN,\cdots ,20kN-1\}$. It follows that

$$\begin{array}{c}\hfill \mathbf{P}(t\in -x(d)+S)=\frac{1}{30kN}\#S⩾\frac{1}{30kN}{\left(\frac{M}{2}\right)}^n{\gg }_{k,n}1.\end{array}$$

(Recall here that t is chosen uniformly at random on $\{-10kN,\cdots ,20kN-1\}$.) Summing over the ${(M/2k)}^n{\gg }_{k,n}N$ choices of d, we see that the expected number of d for which $t\in -x(d)+S$ is ${\gg }_{k,n}N$. Fix some choice of t such that $t\in -x(d)+S$ for ${\gg }_{k,n}N{\gg }_{k,n}{M}^n$ values of d, and write $A_3:=A_3(t)$. Then by construction we have

$$\begin{array}{c}\hfill \#A_3⩽\#A_2⩽\#A_1,\end{array}$$ 3.2

whilst $A_3$ contains a progression of length $⩾k/2$ and common difference d for all d in some set $\mathcal{D}\subset {\{0,\cdots ,M-1\}}^n$, $\#\mathcal{D}{\gg }_{k,n}{M}^n$.

Now choose a prime p with $M⩽p<2M$, and let $A_4\subset {\mathbf{F}}_p^n$ be the image of $A_3$ under the natural projection ${\pi }^{(n)}:{\mathbf{Z}}^n\to {\mathbf{F}}_p^n$. We have

$$\begin{array}{c}\hfill \#A_4=\#A_3,\end{array}$$ 3.3

and moreover $A_4$ contains a progression of length k and common difference d for all $d\in {\pi }^{(n)}(\mathcal{D})$, that is to say for ${\gg }_{n,k}N{\gg }_{n,k}{p}^n$ values of d. By a standard argument (taking random translations of ${\pi }^{(n)}(\mathcal{D})$, see Corollary 6.4 for details) there is a further set $A_5\subset {\mathbf{F}}_p^n$,

$$\begin{array}{c}\hfill \#A_5{\ll }_{n,k}(logp)\#A_4,\end{array}$$ 3.4

containing a progression of length k and common difference d, for all$d\in {\mathbf{F}}_p^n\backslash \{0\}$. Tracing back through (3.4), (3.3), (3.2) we see that

$$\begin{array}{c}\hfill F_k(N){\gg }_{k,n}\frac{1}{logp}{f}_{n,k}(p),\end{array}$$

where $p=p(N)\sim N^{1/n}$ is some prime. It follows that

$$\begin{array}{c}\hfill \underset{N\to \infty }{lim}\frac{log{F}_k(N)}{logN}⩾\underset{N\to \infty }{lim}\frac{log{f}_{n,k}(p(N))}{nlogp(N)}.\end{array}$$

Assuming Conjecture 1.4 (n), the limit on the right is 1. This concludes the proof that Conjecture 1.4 (n) implies Conjecture 1.1.

Before leaving this topic, we remark that it is quite possible that very strong bounds such as

$$\begin{array}{c}\hfill f_{p^{\eta },1}(p)⩾p/2\end{array}$$ 3.5

are true, provided $p⩾p_0(\eta )$ is large enough. This issue is strongly hinted at, if not explicitly conjectured, in [1]. It is pointed out there that such bounds imply vastly more than is currently known about the purely arithmetic problem of bounding the least quadratic nonresidue modulo p.

Whilst a bound of the form (3.5) is not known to imply the arithmetic Kakeya conjecture (the progressions are of length $p^{\eta }$, rather than of bounded size), the arguments of Bourgain may be adapted to show that it does imply the Euclidean Kakeya conjecture. Further details may be found in lecture notes of the first author [9, Section 10].

It is quite interesting that the innocent-looking statement (3.5) implies two famous unsolved problems in completely different mathematical areas.

A problem of Erdős and Selfridge

Finally, we turn to Conjecture 1.5. In fact, we prove the following rather tight connection between Conjecture 1’ and Conjecture 1.5.

Proposition 4.1

Write $G_k(N)$ for the minimum, over all choices $p_1<\cdots <p_N$ of primes and all intervals I of length $k{p}_N$, of $\#(I\cap {\bigcup }_{i=1}^N{p}_i\mathbf{Z})$. Then $F_k^{\prime }(N)⩽G_k(N)⩽k{F}_k^{\prime }(N)$. In particular, Conjectures 1’ and 1.5 are equivalent.

Proof

Suppose first we have a set of primes $p_1<\cdots <p_N$ and an interval I of length $k{p}_N$ so that $\#A=G_k(N)$, where $A={\bigcup }_{i=1}^N\{x\in I:p_i|x\}$. Note that A obviously contains a progression of length k and common difference $p_i$, for each i, and therefore $F_k^{\prime }(N)⩽G_k(N)$.

In the other direction, suppose we have a set A attaining the bound $F_k^{\prime }(N)$, that is to say $\#A=F_k^{\prime }(N)$ and A contains, for $i=1,\cdots ,N$, a progression $\{a_i+j{d}_i:j=0,1,\cdots ,k-1\}$. By translating if necessary, we may assume that A consists of positive integers. Let $\delta \in (0,\frac{1}{2})$ be a quantity to be specified shortly. By the theorem of the first author and T. Tao [10, Theorem 1.2], we may find positive u and v such that all of the numbers $v,u+v,\cdots d_{N}u+v$ are prime and lie in some interval $[(1-\delta )X,X]$, $X⩾100$. Set $p_i:=d_{i}u+v$. Note that $\frac{v}{u+v}⩾1-\delta$, which rearranges as $\frac{v}{u}⩾\frac{1}{\delta }-1$, hence

$$\begin{array}{c}\hfill \frac{v}{u}>4maxA\end{array}$$ 4.1

provided that $\delta$ is chosen sufficiently small. Note also that

$$\begin{array}{c}\hfill \frac{p_i}{p_N}⩾\frac{v}{v+u{d}_N}=\frac{1}{1+\frac{u}{v}{d}_N}⩾1-\frac{1}{4k}\end{array}$$ 4.2

if $\delta$ is small enough. In particular if $\delta$ is small enough then we have

$$\begin{array}{c}\hfill p_i>\frac{3}{4}{p}_N⩾\frac{1}{2}{p}_N+\frac{1}{4}v>\frac{1}{2}{p}_N+umaxA\end{array}$$ 4.3

by (4.1).

Define $A^{\prime }:=u·A+\{0,v,2v,\cdots ,(k-1)v\}$. The cardinality of $A^{\prime }$ satisfies $\#A^{\prime }⩽k{F}_k^{\prime }(N)$, and

$$\begin{array}{c}\hfill A^{\prime }\supset \bigcup _{i=1}^N\{u{a}_i+j{p}_i:j=0,1,\cdots ,k-1\}\end{array}$$ 4.4

for $i=1,\cdots ,N$. By the Chinese remainder theorem we may find w so that $p_i|w+u{a}_i$ for $i=1,\cdots ,N$.

Set $I:=w-\lfloor \frac{1}{2}{p}_N\rfloor +\{1,2,\cdots ,k{p}_N\}$. Obviously I is an interval of length $k{p}_N$. Let $i\in \{1,\cdots ,N\}$. We claim that $w+u{a}_i+j{p}_i\in I$ for an integer j if and only if $j\in \{0,1,\cdots ,k-1\}$. Since $w+u{a}_i+p_i\mathbf{Z}=p_i\mathbf{Z}$, this implies that

$$\begin{array}{c}\hfill I\cap p_i\mathbf{Z}=\{w+u{a}_i+j{p}_i:j=0,1,\cdots ,k-1\},\end{array}$$

and hence by (4.4)

$$\begin{array}{c}\hfill I\cap \bigcup _{i=1}^N{p}_i\mathbf{Z}\subset w+A^{\prime },\end{array}$$

whence

$$\begin{array}{c}\hfill G_k(N)⩽\#\left(I,\cap ,\bigcup _{i=1}^N,p_i,\mathbf{Z}\right)⩽\#A^{\prime }⩽k{F}_k^{\prime }(N).\end{array}$$

It remains to prove the claim. To prove the if implication, it suffices in view of (4.4) to show that $w+A^{\prime }\subset I$. However it is obvious that $min(w+A^{\prime })⩾minI$ (since all elements of $A^{\prime }$ are positive) and moreover

$$\begin{array}{cc}\hfill max(w+A^{\prime })& ⩽w+umaxA+(k-1)v\hfill \\ \hfill & <w+\left(k,-,\frac{1}{2}\right)v\text{by}(4.1)\hfill \\ \hfill & ⩽w+\left(k,-,\frac{1}{2}\right)p_N\hfill \\ \hfill & ⩽maxI.\hfill \end{array}$$

This establishes the if direction of the claim. To establish the only if direction, it suffices to show that

$$\begin{array}{c}\hfill w+u{a}_i-p_i<minI\end{array}$$ 4.5

and that

$$\begin{array}{c}\hfill w+u{a}_i+k{p}_i>maxI.\end{array}$$ 4.6

However by (4.3) we have

$$\begin{array}{c}\hfill w+u{a}_i-p_i<w+u(a_i-maxA)-\frac{1}{2}{p}_N⩽w-\frac{1}{2}{p}_N⩽minI,\end{array}$$

so (4.5) does hold. Also,

$$\begin{array}{cc}\hfill w+u{a}_i+k{p}_i& >w+k{p}_i\text{since}A\subset \mathbf{N}\hfill \\ \hfill & ⩾w+\left(k,-,\frac{1}{4}\right)p_N\text{by}(4.2)\hfill \\ \hfill & ⩾w+k{p}_N-⌊\frac{1}{2},p_N⌋=maxI,\hfill \end{array}$$

the last step being a consequence of the fact that $p_N⩾(1-\delta )X⩾50$. Thus (4.6) also holds, and this completes the proof of the claim. $\square$

Remark

The use of the theorem of the first author and Tao is a little excessive. One could do without it using simpler arguments if one was prepared to settle for logarithmic losses.

Small unions of progressions

In this section we prove Theorem 1.7. Write $3=p_1<p_2<\cdots$ for the odd primes, and set $Q:={\prod }_{i=1}^m{p}_i$, where $m=\lceil 10logk\rceil$. Note that $Q=k^{O(loglogk)}$.

Define a set S to be the union of all progressions $\{x_d+jd:j=0,1,\cdots ,k-1\}$ where, for $d\in \{1,\cdots ,Q-1\}$, $x_d$ is the unique element of $\{1,\cdots ,Q\}$ congruent to $d^2(\text{mod}Q)$. Evidently, S contains a progression of length k and common difference d, for all $d\in \{0,1,\cdots ,Q-1\}$.

Fix $j\in \{0,1\cdots ,k-1\}$. For each i we have

$$\begin{array}{c}\hfill x_d+jd\equiv d^2+jd\equiv {\left(d,+,\frac{j}{2}\right)}^2-\frac{j^2}{4}(\text{mod}{p}_i),\end{array}$$

and so $x_d+jd(\text{mod}{p}_i)$ takes values in a set of size $\frac{1}{2}(p_i+1)$ as d varies. Therefore $x_d+jd(\text{mod}Q)$ takes values in a set of size ${\prod }_{i=1}^m\frac{1}{2}(p_i+1)$. Since, additionally, $0<x_d+jd⩽kQ$, $x_d+jd$ takes values in a set of size $k{\prod }_{i=1}^m\frac{1}{2}(p_i+1)$. Therefore

$$\begin{array}{c}\hfill \#S⩽k^2\prod _{i=1}^m\frac{1}{2}(p_i+1)=k^2{2}^{-m}Q\prod _{i=1}^m\left(1,+,\frac{1}{p_i}\right).\end{array}$$

Recalling that $m\sim 10logk$, and using the bound ${\prod }_{i=1}^m(1+\frac{1}{p_i})\ll logm\ll k$, we see that

$$\begin{array}{c}\hfill \#S\ll k^{-7}Q\end{array}$$

and so

$$\begin{array}{c}\hfill \#S⩽Q^{1-\frac{c}{loglogk}}\end{array}$$

if k is sufficiently large, for some absolute $c>0$.

Now let n be an arbitrary positive integer, set $N_n:=Q^n$, and consider the set

$$\begin{array}{c}\hfill A_n:=\{s_0+s_{1}Q+\cdots +s_{n-1}{Q}^{n-1}:s_0,\cdots ,s_{n-1}\in S\}.\end{array}$$

Then $\#A_n⩽{(\#S)}^n⩽N_n^{1-\frac{c}{loglogk}}$. The set $A_n$ contains a progression of length k and common difference $d_0+d_{1}Q+\cdots +d_{n-1}{Q}^{n-1}$ for any choice of $d_i\in \{0,1,\cdots ,Q-1\}$, or in other words for all $d\in \{0,\cdots ,N_n-1\}$.

Finally, suppose N is an arbitrary positive integer. Choose n minimal so that $N_n>N$, and set $A:=A_n$. Then A contains a progression of length k and common difference d, for all $d\in \{1,\cdots ,N\}$. Moreover,

$$\begin{array}{c}\hfill \#A⩽N_n^{1-\frac{c}{loglogk}}⩽{(QN)}^{1-\frac{c}{loglogk}}{\ll }_k{N}^{1-\frac{c}{loglogk}}.\end{array}$$

The result follows.

Entropy inequalities in positive characteristic

In this section we give the proof of Theorem 1.8. Suppose that $\mathsf{X}$ and $\mathsf{Y}$ are two ${\mathbf{F}}_p^{\infty }$-valued random variables, both taking finitely many values. Suppose that

$$\begin{array}{c}\hfill \mathbf{H}(\mathsf{X}-\mathsf{Y})⩾(1+\epsilon )\underset{r\ne -1}{sup}\mathbf{H}(\mathsf{X}+r\mathsf{Y}).\end{array}$$ 6.1

Our aim is to prove that $\epsilon =O(\frac{1}{logp})$, which immediately implies Theorem 1.8.

The initial phases of the argument mirror the deduction of Conjecture 1.2 from Conjecture 1.3. We may assume that there is some q such that $q\mathbf{P}((\mathsf{X},\mathsf{Y})=(x,y))\in \mathbf{Z}$ for all (x, y); if (6.1) can be established in this case, uniformly in q, then the general result follows by an easy approximation argument on letting $q\to \infty$.

Now let m be very large, write $n=mq$, and construct a set $B^{(n)}\subset {({\mathbf{F}}_p^{\infty })}^{\mathit{qm}}\times {({\mathbf{F}}_p^{\infty })}^{\mathit{qm}}$ as follows. Let it consist of all pairs $((x_1,\cdots ,x_{\mathit{mq}})$, $(y_1,\cdots ,y_{\mathit{mq}}))$ for which

$$\begin{array}{c}\hfill \#\{i:(x_i,y_i)=(x,y)\}=mq\mathbf{P}((\mathsf{X},\mathsf{Y})=(x,y)).\end{array}$$

By arguments essentially the same as we saw before,

$$\begin{array}{c}\hfill \mathbf{H}(\mathsf{X}+r\mathsf{Y})=\frac{1}{n}log{\pi }_r\left(B^{(n)}\right)+o_{n\to \infty }(1).\end{array}$$

Hence, taking m sufficiently large (and observing that ${({\mathbf{F}}_p^{\infty })}^{\mathit{qm}}$ is isomorphic to ${\mathbf{F}}_p^{\infty }$ as a vector space), we obtain arbitrarily large sets $B\subset {\mathbf{F}}_p^{\infty }\times {\mathbf{F}}_p^{\infty }$ such that

$$\begin{array}{c}\hfill \#{\pi }_{-1}(B)⩾\underset{r\ne -1}{sup}{(\#{\pi }_r(B))}^{1+\epsilon /2}.\end{array}$$ 6.2

Note in particular that ${\pi }_{-1}(B)$ becomes arbitrarily large.

For such a B, we construct a finite set $A\subset {\mathbf{F}}_p^{\infty }$ as follows. If $(x,y)\in B$ and $x\ne y$, include the entire progression (line) through x and y in A. The points on this line are $\frac{x+ry}{1+r}$, for $r\ne -1$, and y. Therefore

$$\begin{array}{c}\hfill A\subset {\pi }_{\infty }(B)\cup \bigcup _{r\ne -1}\frac{1}{1+r}·{\pi }_r(B),\end{array}$$

and therefore

$$\begin{array}{c}\hfill \#A⩽p\underset{r\ne -1}{sup}{\pi }_r(B).\end{array}$$

On the other hand, A contains a progression of length p (line) and common difference d, for every $d\in {\pi }_{-1}(B)\backslash \{0\}$. Thus, writing $N:={\pi }_{-1}(B)-1$, we have

$$\begin{array}{c}\hfill \#A{\ll }_p{N}^{\frac{1}{1+\epsilon /2}}.\end{array}$$ 6.3

On the other hand we have the following result, whose proof we will supply shortly.

Proposition 6.1

Suppose that $A\subset {\mathbf{F}}_p^{\infty }$ is a finite set containing a progression of length p(that is, a line) and common difference d, for all d in some set of size N. Then $\#A{\gg }_p{N}^{1-\frac{log2}{logp}-o(1)}$.

Combining Proposition 6.1 with the construction of A satisfying (6.3) immediately gives the desired upper bound $\epsilon =O(\frac{1}{logp})$, thereby concluding the proof of Theorem 1.8.

It remains to prove Proposition 6.1.

Proof of Proposition 6.1

Set $A_1:=A$. In its initial stages, the proof of this result goes along rather similar lines to that of Proposition 2.1, only it is rather easier. The use of random projections in a similar context may be found in [7, §3]. Let n be the smallest positive integer for which $p^n⩾N$.

Since $A_1$ is finite, it is contained in some copy of ${\mathbf{F}}_p^M$. Let $\pi :{\mathbf{F}}_p^M\to {\mathbf{F}}_p^n$ be a random linear map, selected by choosing the images of the basis vectors $e_1,\cdots ,e_M$ uniformly at random from ${\mathbf{F}}_p^n$. Set $A_2:=\pi (A_1)$; evidently $\#A_2⩽\#A_1$. Let $\mathcal{D}$ be the set of common differences of progressions (of length p) lying in $A_1$. Then $A_2$ contains a progression of length p and common difference $\pi (d)$, for every $d\in \mathcal{D}$.

Put some arbitrary order $\prec$ on $\mathcal{D}$, and suppose that $d\prec d^{\prime }$. Then $\pi (d)=\pi (d^{\prime })$ if and only if $\pi (d-d^{\prime })=0$. However, $\pi (d-d^{\prime })$ is uniformly distributed in ${\mathbf{F}}_p^n$, and so the probability of this happening is $p^{-n}$. It follows that the expected number of pairs $(d,d^{\prime })$ with $d\prec d^{\prime }$ and $\pi (d)=\pi (d^{\prime })$ is $p^{-n}\left(\begin{array}{c}N\\ 2\end{array}\right)⩽\frac{1}{N}\left(\begin{array}{c}N\\ 2\end{array}\right)⩽N/2$. Pick some map $\pi$ for which the number of such pairs is at most N. For each $v\in {\mathbf{F}}_p^n$, write $f(v):=\#{\pi }^{-1}(v)$. Then we have ${\sum }_v\left(\begin{array}{c}f(v)\\ 2\end{array}\right)⩽N/2$, from which we obtain, since ${\sum }_{v}f(v)=N$, that ${\sum }_{v}f{(v)}^2⩽2N$. By Cauchy–Schwarz,

$$\begin{array}{c}\hfill N^2={\left(\sum _v,f,(v)\right)}^2⩽\#\{v:f(v)\ne 0\}\sum _{v}f{(v)}^2,\end{array}$$

and therefore there are at least N / 2 values of v for which $f(v)\ne 0$. From the choice of n it is clear that $p^n⩽pN$, and so at least $(\#{\mathbf{F}}_p^n)/2p$ elements of ${\mathbf{F}}_p^n$ lie in the image of $\pi$, or in other words are common differences of progressions in $A_2$.

By a random translation argument (see Corollary 6.4), there is a set $A_3\subset {\mathbf{F}}_p^n$, $\#A_3\ll (nplogp)\#A_2$, containing a line in every direction. That is, $A_3$ is a finite field Besicovitch set.

Now we bring in bounds on the size of such sets of a strength which, famously, are available in the finite field setting but not in characteristic zero. By the main result of [6] we have $\#A_3⩾{(p/2)}^n={(p^n)}^{1-\frac{log2}{logp}}⩾N^{1-\frac{log2}{logp}}$. The proposition follows. $\square$

Remarks

Note that here it was crucial to have an effective lower bound on the size of Kakeya sets for fixed p but with $n\to \infty$. For this, the celebrated work of Dvir [5] on the Kakeya problem would not suffice. However (at the cost of weakening the exponents slightly) we could have used the main result of [19], which has a slightly simpler proof than that of [6].

The $O(\frac{1}{logp})$ term in Theorem 1.7 is sharp. To see this, pick $a,b,b^{\prime }$ independently and uniformly from ${\mathbf{F}}_p$, and define random variables $\mathsf{X},\mathsf{Y}$ taking values in ${\mathbf{F}}_p^2$ by

$$\begin{array}{c}\hfill \mathsf{X}=(a+b,ab),\mathsf{Y}=(a+b^{\prime },a{b}^{\prime }).\end{array}$$

Then

$$\begin{array}{c}\hfill \mathsf{X}-\mathsf{Y}=(b-b^{\prime },a(b-b^{\prime })),\end{array}$$

which is almost uniformly distributed on ${\mathbf{F}}_p^2$: a short calculation gives

$$\begin{array}{c}\hfill \mathbf{H}(\mathsf{X}-\mathsf{Y})=2logp+O\left(\frac{logp}{p}\right).\end{array}$$

By contrast, if $r\ne -1$ then

$$\begin{array}{c}\hfill \frac{\mathsf{X}+r\mathsf{Y}}{1+r}=\left(a,+,\frac{b+r{b}^{\prime }}{1+r},,,a,·,\frac{b+r{b}^{\prime }}{1+r}\right),\end{array}$$

and so $\mathsf{X}+r\mathsf{Y}$ is supported on a dilate of the set $V:=\{(u+v,uv):u,v\in {\mathbf{F}}_p\}$, which has cardinality $\frac{1}{2}{p}^2+O(p)$. Therefore

$$\begin{array}{c}\hfill \mathbf{H}(\mathsf{X}+r\mathsf{Y})⩽2logp-log2+O\left(\frac{logp}{p}\right).\end{array}$$

Cognoscenti will recognise V as being equivalent to the well-known construction of optimal Kakeya sets in ${\mathbf{F}}_p^2$, due to Mockenhaupt and Tao [15].

Appendix A. Covering by translates

In this section we review some standard lemmas on random translates.

Lemma 6.2

Suppose that $S\subset \{1,\cdots ,X\}$ is a set. Then there is a set T of size $\ll \frac{X}{\#S}logX$ such that $S+T\supset \{1,\cdots ,X\}$.

Proof

We inductively define $t_1,t_2,\cdots \in \{-X+1,\cdots ,X\}$ and $A_i:=\{1,\cdots ,X\}\backslash {\bigcup }_{j=1}^i(S+t_j)$ such that, given the choice of $t_1,\cdots ,t_i$, $\#A_{i+1}$ is as small as possible. We have

$$\begin{array}{c}\hfill \sum _t\#(A_i\cap (S+t))=\#A_i\#S,\end{array}$$

and so

$$\begin{array}{c}\hfill \underset{t}{max}\#(A_i\cap (S+t))⩾\#A_i\frac{\#S}{2X}.\end{array}$$

Therefore

$$\begin{array}{c}\hfill \#A_{i+1}⩽\#A_i\left(1,-,\frac{\#S}{2X}\right).\end{array}$$

This process terminates with $\#A_i<1$ (and hence $\#A_i=0$) in $\ll \frac{X}{\#S}logX$ steps. $\square$

Lemma 6.3

Suppose that $S\subset {\mathbf{F}}_p^n$ is a set. Then there is a set $T\subset {\mathbf{F}}_p^n$ of size $\ll \frac{p^n}{\#S}nlogp$ such that $S+T={\mathbf{F}}_p^n$.

Proof

Very similar to the previous lemma, and left as an exercise. $\square$

Corollary 6.4

Suppose that $A\subset {\mathbf{F}}_p^n$ is a set containing a k-term arithmetic progression with common difference d, for all d lying in some set $\mathcal{D}$ of size $\delta p^n$. Then there is a set $A^{\prime }$, $\#A^{\prime }{\ll }_{k,n}logp·\#A$, containing a k-term arithmetic progression with every common difference.

Proof

Apply Lemma 6.3 with $S=\mathcal{D}$, and let T be the resulting set. Then take $A^{\prime }={\bigcup }_{x\in \{0\}\cup T\cup \cdots \cup (k-1)·T}(A+x)$. $\square$