Quantitative Coding and Complexity Theory of Compact Metric Spaces

Abstract

Specifying a computational problem requires fixing encodings for input and output: encoding graphs as adjacency matrices, characters as integers, integers as bit strings, and vice versa. For such discrete data, the actual encoding is usually straightforward and/or complexity-theoretically inessential (up to polynomial time, say); but concerning continuous data, already real numbers naturally suggest various encodings with very different computational properties. With respect to qualitative computability, Kreitz and Weihrauch (1985) had identified admissibility as crucial property for “reasonable” encodings over the Cantor space of infinite binary sequences, so-called representations. For (precisely) these does the Kreitz-Weihrauch representation (aka Main) Theorem apply, characterizing continuity of functions in terms of continuous realizers. We similarly identify refined criteria for representations suitable for quantitative complexity investigations. Higher type complexity is captured by replacing Cantor’s as ground space with more general compact metric spaces, similar to equilogical spaces in computability.

Introduction

Machine models formalize computation: they specify means of input, operations, and output of elements from some fixed set ; as well as measures of cost and of input/output ‘size’; such that Complexity Theory can investigate the dependence of the former on the latter. Problems over spaces X other than are treated by encoding its elements/instances over .

Example 1

1. Recall the Turing machine model operating on the set of finite (e.g. decimal or binary) sequences, and consider the space X of graphs: encoded for example as adjacency matrices’ binary entries. Operations amount to local transformations of, and in local dependence of, the tape contents. Size here is an integer: n commonly denotes the number of nodes of the graph, or the binary length of the encoded matrix, both polynomially related to each other.

2. Consider the space of natural numbers, either encoded in binary or in unary: their lengths are computably but not polynomially related, and induce computably equivalent but significantly different notions of computational complexity.

3. Recall the type-2 machine model [Wei00, §2.1] operating on the Cantor space of infinite binary sequences; and the real unit interval , equipped with various so-called representations [Wei00, §4.1]: surjective partial mappings from onto X that formalize (sequences of) approximations up to any given absolute error bound , . Different representations of X may induce non-/equivalent notions of computability [Wei00, §4.2].

4. Computational cost of a type-2 computation is commonly gauged in dependence of the index position n within the binary input/output sequence, that is, the length of the finite initial segment read/written so far [Wei00, §7.1]. For and for some of the representations, this notion of ‘size’ is polynomially (and for some even linearly) related to n occurring in the error bound [Wei00, §7.1]; for other computably equivalent representations it is not [Wei00, Examples 7.2.1+7.2.3].

5. Recall the Turing machine model with ‘variable’ oracles [KC12, §3], operating on a certain subset of string functions

   

   The ‘size’ of here is captured by an integer function [KC96]; and polynomial complexity means bounded by a second-order polynomial in and in [Meh76].

6. Equip the space of continuous functions with the surjective partial mapping from [KC12, §4.3]. Then, up to a second-order polynomial, the ‘size’ from (e) is related to a modulus of continuity (cmp. Subsect. 3.1 below) of and to the computational complexity of the application operator [KS17, KS20, NS20].

7. Spaces X of continuum cardinality beyond real numbers are also commonly encoded over Cantor space [Wei00, §3], or over ‘Baire’ space [KC12, §3]. Matthias Schröder has recommended the Hilbert Cube as domain for partial surjections onto suitable X. Also equilogical spaces serve as such domains [BBS04].

To summarize, computation on various spaces is commonly formalized by various models of computation (Turing machine, type-2 machine, oracle machine) using encodings over various domains (Cantor space, ‘Baire’ space, Hilbert Cube, etc.) with various notions of ‘size’ and of polynomial time.

Question 2

Fix two mathematical structures X and Y, expansions over topological spaces. What machine models, what encodings, what notions of size and polynomial time, are suitable to formalize computation of (multi)functions f from X to Y?

In the sequel we will focus on the part of the question concerned with encoding continuous data. Section 2 recalls classical criteria and notions: qualitative admissibility of computably ‘reasonable’ representations for the Kreitz-Weihrauch Main Theorem (Subsect. 2.1), and complexity parameters for a quantitative Main Theorem in the real case (Subsect. 2.2). Section 4 combines both towards generic quantitative admissibility and an intrinsic complexity-theoretic Main Theorem. The key is to consider metric properties of the inverse of a representation, which is inherently multivalued a ‘function’. To this end Sect. 3 adopts from [PZ13] a notion of quantitative (uniform) continuity multifunctions (Subsect. 3.1) and establishes important properties (Subsect. 3.2), including closure under a generalized conception of restriction. We close with applications to higher-type complexity.

Coding Theory of Continuous Data

Common models of computation naturally operate on some particular domain (e.g., in/finite binary sequences, string functions, etc.); processing data from another domain X (graphs, real numbers, continuous functions) requires agreeing on some way of encoding (the elements x of) X over .

Formally, a representation is a surjective partial mapping ; any is called a name of ; and for another representation of Y, computing a total function means to compute some -realizer: a transformation on names such that .

Some representations are computably ‘unsuitable’ [Tur37], including the binary expansion ; cmp. [Wei00, Exercise 7.2.7]. Others are suitable for computability investigations [Wei00, Theorem 4.3.2], but not for complexity purposes [Wei00, Examples 7.2.1+7.2.3].

Example 3

The signed digit representation of [0; 1] is the partial map

Already for the case of real numbers, it thus takes particular care to arrive at a complexity-theoretically ‘reasonable’ representation [Wei00, Theorem 7.3.1]; and even more so for continuous real functions [KC12], not to mention for more involved spaces [Ste17].

Qualitative Admissibility and Computability

Regarding computability on a large class of topological spaces X, an important criterion for a representation is admissibility [KW85, Sch02]:

Definition 4

Call admissible  iff  it is (i) continuous and satisfies (ii):

(ii) To every continuous there exists a continuous mapping with ; see [Wei00, Theorem 3.2.9.2].

Admissible representations exist (at least) for T spaces; they are Cartesian closed; and yield the Kreitz-Weihrauch (aka Main) Theorem [Wei00, Theorem 3.2.11]:

Fact 5

Let and be admissible. Then is continuous iff it admits a continuous -realizer .

In particular discontinuous functions are incomputable.

Real Quantitative Admissibility

The search for quantitative versions of admissibility and the Main Theorem is guided by above notion of qualitative admissibility. It revolves around quantitative metric versions of qualitative topological properties, such as continuity and compactness, obtained via Skolemization. Further guidance comes from reviewing the real case.

Recall that a modulus of continuity of a function between compact metric spaces (X, d) and (Y, e) is a strictly increasing mapping such that

1

In this case one says that f is -continuous. Actually we shall occasionally slightly weaken this notion and require Condition (1) only for all sufficiently large n.

Example 6

The signed digit representation from Example 3 has modulus of continuity .

Proposition 11d) below provides a converse. Together with Theorem 13 and Lemma 10 below, they yield the following quantitative strengthening of Fact 5 aka qualitative Main Theorem, where of refers to the asymptotic Landau symbol:

Theorem 7

Fix strictly increasing . A function has modulus of continuity  iff  it has a -realizer with modulus of continuity .

In particular functions f with (only) ‘large’ modulus of continuity are inherently ‘hard’ to compute; cmp. [Ko91, Theorem 2.19]. This suggests gauging the efficiency of some actual computation of f relative to it modulus of continuity, rather than absolutely [KC12, KS17, KS20, NS20]:

Definition 8

Function is polynomial-time computable  iff  it can be computed in time bounded by a (first or second order) polynomial in the output precision parameter n and in f’s modulus of continuity.

In the sequel we consider continuous total (multi)functions whose domains are compact: The latter condition ensures them to have a modulus of (uniform) continuity. Moreover computable functions with compact domains admit complexity bounds depending only on the output precision parameter n; cmp. [Ko91, Theorem 2.19] or [Wei00, Theorems 7.1.5+7.2.7] or [Sch03].

Multifunctions

Multifunctions are unavoidable in real computation [Luc77]. Their introduction simplifies several considerations; for example, every function has a (possibly multivalued) inverse .

Formally, a partial multivalued function (multifunction) F between sets X, Y is a relation that models a computational search problem: Given (any name of) , return some (name of some) with . One may identify the relation f with the single-valued total function from X to the powerset ; but we prefer the notation to emphasize that not every needs to occur as output. Letting the answer y depend on the code of x means dropping the requirement for ordinary functions to be extensional; hence, in spite of the oxymoron, such F is also called a non-extensional function. Note that no output is feasible in case .

Definition 9

Abbreviate with for the domain of F; and . F is total in case ; surjective in case . The composition of multifunctions and is

Call F pointwise compact if is compact for every .

Note that every (single-valued) function is pointwise compact. A computational problem, considered as total single-valued function , becomes ‘easier’ when restricting arguments to , that is, when proceeding to for some . A search problem, considered as total multifunction , additionally becomes ‘easier’ when proceeding to any satisfying the following: for every . We call such also a restriction of F, and write . A single-valued function is a selection of if F is a restriction of f.

Lemma 10

Fix partial multifunctions and .

1. If both F and G are pointwise compact, then so is their composition .

2. The composition of restrictions and , is again a restriction .

3. It holds . Single-valued surjective partial furthermore satisfy .

4. For representations of X and of Y, the following are equivalent: (i) is a restriction of (ii) f is a restriction of (iii) is a restriction of F.

Quantitative Continuity for Multifunctions

Every restriction of a single-valued continuous function f is again continuous. This is not true for multifunctions with respect to hemicontinuity. Instead Definition 12 below adapts, and quantitatively refines, a notion of continuity for multifunctions from [PZ13] such as to satisfy the following properties:

Proposition 11

1. A single-valued function is -continuous iff it is -continuous when considered as a multifunction.

2. Suppose that is -continuous. Then every restriction is again -continuous.

3. If additionally is -continuous, then is -continuous

4. The multivalued inverse of the signed digit representation is -continuous.

5. For every , the soft Heaviside ‘function’ is -continuous, but not for :

   

Our notion of quantitative (uniform) continuity is inspired by [BH94] and [PZ13, §4+§6]:

Definition 12

Fix metric spaces (X, d) and (Y, e) and strictly increasing . A total multifunction is called -continuous if there exists some , and to every there exists some , such that the following holds for every :

The parameter is introduced for the purpose of Proposition 11d+e). Recall that also in the single-valued case we sometimes understand Eq. (1) to hold only for all .

A continuous multifunction on Cantor space, unlike one for example on the reals [PZ13, Fig. 5], does admit a continuous selection, and even a bound on the modulus:

Theorem 13

Suppose (Y, e) is compact of diameter and satisfies the strong triangle inequality

2

If is -continuous and pointwise compact with compact domain , then G admits a -continuous selection.

Generic Quantitative Main Theorem

Generalizing both Fact 5 and Theorem 7, Lemma 10 and Proposition 11 and Theorem 13 together in fact yields the following quantitative counterpart to the qualitative Main Theorem for generic compact metric spaces:

Theorem 14

Fix compact metric spaces (X, d) and (Y, e) of . Consider representations and .

Let be strictly increasing such that is -continuous with compact domain and -continuous multivalued inverse ; is -continuous with compact domain and -continuous multivalued inverse .

1. If total multifunction has a K-continuous -realizer G, then g is -continuous.

2. If total multifunction is -continuous and pointwise compact, then it has a -continuous -realizer G.

Following up on Definition 8, this suggests gauging the efficiency of some actual computation of g relative to both it modulus of continuity and moduli of continuity of the representations (and their multivalued inverses) involved.

Generic Quantitative Admissibility

According to Theorem 14, quantitative continuity of a (multi)function g is connected to that of a (single-valued) realizer G, subject to properties of the representations under consideration.

A ‘true’ quantitative Main Theorem should replace these extrinsic parameters with ones intrinsic to the co/domains X, Y: by imposing suitable conditions on the representations as quantitative variant of qualitative admissibility [Lim19].

Definition 15

The entropy of a compact metric space (X, d) is the mapping such that X can be covered by closed balls of radius , but not by .

Introduced by Kolmogorov [KT59], thus quantitatively captures total boundedness [Koh08, Definition 18.52]. Its connections to computational complexity are well-known [Wei03, KSZ16].

Example 16

1. The d-dimensional real unit cube has linear entropy . Cantor space , equipped with the metric , has linear entropy .

2. The space of non-expansive (aka 1-Lipschitz) functions is compact when equipped with the supremum norm and has entropy .

3. More generally fix a connected compact metric space (X, d) of diameter with entropy . Then the space of non-expansive functionals is compact when equipped with the supremum norm and has entropy .

Items (b) and (c) are relevant for higher-type complexity theory.

Since computational efficiency is connected to quantitative continuity (Subsect. 2.2), in Theorem 14 one prefers and with ‘small’ moduli; similarly for and . A simple but important constraint has been identified in [Ste16, Lemma 3.1.13]—originally for single-valued functions, but its proof immediately extends to multifunctions.

Lemma 17

If surjective (multi)function is -continuous, then it holds (for all sufficiently large n).

This suggests the following tentative definition:

Definition 18

Fix some compact metric space , and recall Example 1g).

1. A representation of compact metric space (X, d) is a continuous partial surjective (single-valued) mapping .

2. Fix another compact metric space and representation . A -realizer of a total (multi)function is a (single-valued) function satisfying any/all conditions of Lemma 10d).

3. Representation is polynomially admissible if (i) It has a modulus of continuity such that is bounded by a (first or second order) polynomial in the precision parameter n and in the entropy of X. (ii) Its multivalued inverse has polynomial modulus of continuity .

4. Call total (multi)function polynomial-time computable  iff  it can be computed in time bounded by a (first or second order) polynomial in the output precision parameter n and in the entropy of X.

In view of Lemma 10c+d) we deliberately consider only single-valued representations [Wei05]. Item d) includes Definition 8 as well as higher types, such as Example 16b) and c). Note that Item (c i) indeed quantitatively strengthens Definition 4i). And Item (c ii) quantitatively strengthens Definition 4ii): For -continuous , Theorem 13 yields a -continuous selection G of , that is, with according to Lemma 10.