Reversible Programming Languages Capturing Complexity Classes

Abstract

We argue that there is a link between implicit computational complexity theory and the theory of reversible computation. We show that the complexity classes and can be captured by inherently reversible programming languages.

Introduction

The title above is inspired by the title of a paper I co-authored with Paul Voda more than 15 years ago: Programming languages capturing complexity classes [10]. In that paper we related the computational power of fragments of programming languages to complexity classes defined by imposing time and space constraints on Turing machines. Around that time, I authored and co-authored a number of related papers, e.g. [8, 9, 11], all of which were clearly inspired by work in implicit computational complexity theory from the 1990s, e.g., Bellatoni and Cook [2], Leivant [12, 13] and, particularly, Jones [5, 6].

Complexity classes like , , , , , and so on, are defined by imposing explicit resource bounds on a particular machine model, namely the Turing machine. E.g., is defined as the class of functions computable in polynomial time on a deterministic Turing machine. The definition puts constraints on the resources available to the Turing machines, but no constraints on the algorithms available to them. A Turing machine may compute a function in the class by any imaginable algorithm as long as it works in polynomial time. Implicit computational complexity theory studies classes of functions (problems, languages) that are defined without imposing explicit resource bounds on machine models, but rather by imposing linguistic constraints on the way algorithms can be formulated. When we explicitly restrict our language for formulating algorithms, that is, our programming language, then we may implicitly restrict the computational resources needed to execute algorithms. If we manage to find a restricted programming language that captures a complexity class, then we will have a so-called implicit characterization. A seminal example is Bellatoni and Cook’s [2] characterization of . They give a functional programming language (which they call a function algebra). This language consists of a few initial functions and two definition schemes (safe composition and safe primitive recursion) which allow us to define new functions. These schemes put rather severe syntactical restrictions on how we can define functions, but they do not refer to polynomially bounded Turing machines or any other kind of resource bounded computing machinery. It is not easy to write programs when we have to stick to these schemes, even experienced programmers might find it hard to multiply two numbers but, be that as it may, this is a programming language that yields an implicit characterization of a complexity class. It turns out that a function can be computed by a program written in Bellantoni & Cook’s language if and only if it belongs to the complexity class .

There is an obvious link between implicit computational complexity and reversible computing. A programming language based on natural reversible operations will impose restrictions on the way algorithms can be formulated, and thus, also restrictions on the computational resources needed to execute algorithms. Hence, the following question knocks at the door: Will it be possible find reversible programming languages that capture some of the standard complexity classes? The answer turns out to be YES. We will present a reversible language that captures, or if you like, gives an implicit characterization of, the (maybe not very well-known) complexity class . A few small modifications of this language yield a reversible language that captures the very well-known complexity class .

Our languages are based on a couple of naturally reversible operations. To increase, or decrease, a natural number by 1 modulo a base b is such an operation: . The successor of becomes 0, and then becomes the predecessor of 0. Thus, “increase” and “decrease” are the reverse of each other. To move an element from the top of one stack to the top of another stack is another such operation as we can simply move the element back to the stack it came from.

This paper addresses students and researchers interested in programming languages, reversible computations and computer science in general, they will not necessarily be experts in computability or complexity theory. We will give priority to readability over technical accuracy, but still this is a fairly technical paper, and we will assume that the reader is faintly acquainted with Turing machines and basic complexity theory (standard textbooks are Arora and Barac [1], Jones [7] and Sipser [16]).

Implicit computational complexity theory is definitely a broader and richer research area than our short discussion above may indicate. More on the subject can be found in Dal Lago [3].

Reversible Bottomless Stack (RBS) Programs

An infinite sequence of natural numbers is a bottomless stack if there exists k such that for all . We use to denote the bottomless stack where when , and when . We say that is the top element of . Observe that 0 is the top element of the stack . Furthermore, observe that is the same stack as (since and denote the same sequence of natural numbers). We will refer to as the zero stack.

The syntax of the imperative programming language RBS is given in Fig. 1. Any element in the syntactic category Command will be called a program, and we will use the word command and the word program interchangeably throughout the paper. We will now explain the semantics of RBS.

Fig. 1.

The syntax of the language RBS. The variable X in the loop command is not allowed to occur in the loop’s body.

An RBS program manipulates bottomless stacks, and each program variable holds such a stack. The input to a program is a single natural number m. When the execution of the program starts, the input m will be stored at the top of the stack hold by , that is, we have . All other variables occurring in the program hold the zero stack when the execution starts. A program is executed in a base b which is determined by the input: we have if when the execution starts. The execution base b is kept fixed during the entire execution.

Let and be program variables. We will now explain how the primitive commands work. The command pops off the top element of the stack held by and pushes it onto the stack held by , that is

The command increases the the top element of the stack held by by , that is

The command decreases the the top element of the stack held by by , that is

Observe that we have

and

when b is the base of the execution.

The semantics of the command is as expected. This is the standard composition of the commands and , that is, first is executed, then is executed. The command executes the command repeatedly k times in a row where k is the top element of the stack held by . Note that the variable is not allowed to occur in and, moreover, the command will not modify the stack held by .

Example 1

Let be the program . We have

Let be the program . We have

since the execution base is 18. All numbers stored on stacks during an execution will be strictly less than the execution base, and thus, less than or equal to where m is the input.    

Intuitively, it should be clear that programs are reversible in a very strong sense. is an inherently reversible programming language in the terminology of Matos [14]. If we like, we can of course state this insight more formally. The next definition and the following theorem will be a step in that direction.

Definition 2

We define reverse command of C, written , inductively over the structure C:

• 

• 

• 

• 

• .

   

Theorem 3

Let be a program, and let be the variables occurring in . Furthermore, let m be any natural number. We have

It is a nice, and maybe even challenging, exercise to write up a decent proof Theorem 3, even if it should be pretty clear that the theorems holds. We will offer a proof in the next section. The reader not interested in the details of the proof, may skip that section.

We will now define the set of problems that can be decided by an programs. To that end, we need to determine how an program should accept, and how an program should reject, its input. Any reasonable convention will do, and we will just pick a simple and convenient one.

Definition 4

An RBS program C accepts the natural number m if C executed with input m terminates with 0 at the top of the stack hold by , otherwise, C rejects m.

A problem is a set of natural numbers.1 An RBS program C decides the problem A if C accepts all m that belong to A and rejects all m that do not belong to A. Let denote class of problems decidable by an RBS program.    

Let A be the set of even numbers. Then A is a problem. Figure 2 shows an RBS program that decides A.

Fig. 2.

The program accepts every even number and rejects every odd number.

Now, any RBS program decides a problem, and is obviously a well-defined class of computable (decidable) problems. We have defined by a reversible programming language. We have not defined by imposing resource bounds on Turing machines or any other machine models. What can we say about the computational complexity of the problems we find in ? May it be the case that equals a complexity class?

The Proof of Theorem 3

This section is dedicated to a detailed proof of Theorem 3 (readers not interested may jump ahead to Sect. 4). First, we need some terminology and notation: We will say that a (bottomless) stack is a b-stack if every number stored on the stack is strictly smaller than b. Furthermore, we will use to denote the set of program variables occurring in the command , and for any positive integer m and any command , we define the command by and .

Now, assume that is an command with . Furthermore, assume that is executed in base b and that are b-stacks. With these assumptions in mind, we make the following claim:

claim

Theorem 3 follows straightforwardly from this claim. So all we need to do is to prove the claim.

We will of course carry out induction on the structure of the command , and our proof will split into the tree base cases (i) , (ii) and (iii) and the two inductive cases (iv) and (see Fig. 1).

Case (i). Assume

Then we also have where

for some . We have when . Thus we have . By Definition 2, we have . Now, since neither nor will modify any stack held by a variable where , we also have

This concludes the proof of case (i). The proofs of the cases (ii) and (iii) are very similar to the proof of case (i). We leave the details to the reader and proceed with the inductive cases.

Case (iv). Assume

Then there exist b-stacks such that

We apply our induction hypothesis both to and to and conclude

It follows that

Finally, as Definition 2 states that , we have

This completes the proof of case (iv).

Case (v). Assume

*

and let m be the top element of the stack .

If , we have

as the command will not be executed at all. Thus, we also have

and by Definition 2, we have

This proves that the claim holds when . We are left to prove that the claim holds when . Thus, in the remainder of this proof we assume that .

First we prove

†

by a secondary induction on m.

Let . Then we have , and an application of our main induction hypothesis to yields (). Let . Then we have

and () holds by our induction hypothesis on m and case (iv) above. This concludes the proof of ().

We are now ready to complete our proof the claim. By (*), we have

By (), we have

Since , we have , and thus, the top element of is the same as the top element of , namely m. It follows that

Finally, as Definition 2 states that , we have

This completes the proof of case (v).

Simulation of Turing Machines

A General Strategy

Let us first see how we can simulate a Turing machine in a standard way in a standard high-level language. Thereafter we will discuss how we can simulate a Turing machine in our rudimentary reversible language. In the standard language we will of course be able to simulate any Turing machine, no matter how much time and space resources the machine requires. In the reversible language we will only be able to simulate those Turing machines that run in time (where k is a constant and n is the length of the input).

We assume some familiarity with Turing machines. The reader is expected to know that a Turing machine computes by writing symbols from a finite alphabet on an infinite tape which is divided into cells; know that one of the cells is scanned by the machine’s head; know a there is a finite number of states ; and so on.

The input w will be available on the tape when a Turing machine M starts, and the actions taken by M will be governed by a finite transition table. Each entry of the table is a 5-tuple

*

where are alphabet symbols; are states; and D is ether “left” or “right”. Such a tuple is called a transition and tells M what to do when it scans the symbol in state : in that case M should write the symbol , move its head one position in the direction given by D, and then proceed in state . We restrict our attention to deterministic Turing machines, and for each alphabet symbol and each non-halting state , there will be one, and only one, transition that starts with . So a Turing machine knows exactly what to do until it reaches one its halting states, and then it simply halts (if it halts in a dedicated state , it accepts its input; if it halts in a dedicated state , it rejects its input). This entails that we can simulate a Turing machine by a sequence of if-then statements embedded into a while-loop. We need one if-then statement for each transition:

Minimum one transition will be executed each time the loop’s body is executed, and the running time of M (on input w) will more or less be the number of times the body is executed. (It might happen that more than one transition is executed when the loop’s body is executed once, but that will not cause any trouble.) In order to simulate the actions taken by the transitions, we need a representation of the computing machinery. We need to keep track of the current state, we need to keep track of the symbols on the tape, and we need to identify the scanned cell. The current state can simply be stored in a register , but how should we deal with the tape? The tape is divided into an infinite sequence of cells

where one of the cells is scanned by the head. Only finitely many of these cells will contain anything else than the blank symbol. Let us say that contains blank when . In order to simulate the machine it will obviously be sufficient to store the symbols in the cells where . In addition we need to keep track of the scanned cell . A convenient way to deal with the situation will be to use a stack , a register SCAN, another stack , and store the tape content in the following way:

Now we can mimic the movements of the head by pushing and popping alphabet symbols in the obvious way, and the transition (*) can be implemented by a program of the form

Can RBS Programs Simulate Turing Machines?

The input to an RBS program is a natural number, and we will thus discuss to what extent an RBS program can simulate a Turing machine that takes a single natural number as input.

We have seen that a program with only one while-loop can simulate a Turing machine (and we will for sure need at least one while-loop in order to simulate an arbitrary Turing machine). Now, while-loops are not available in RBS, and the best we can do in order to simulate a Turing machine is to use a fixed number of nested for-loops:

Since an RBS program cannot increase the numerical value of its input, the body of each of these loops will be executed maximum times where m is the input to the RBS program (and to the Turing machine the program simulates). Thus it is pretty clear that we cannot simulate a Turing machine if its running time is not bounded by for some constant k. This corresponds to a bound where k is a constant and |m| is the length of the input m, that is, |m| equals the number of symbols needed to represent the natural number m in binary notation. In the following we will see that any Turing machine that uses such an amount of computation time can be simulated by an RBS program.

It turns out that an RBS program can simulate the transitions of a Turing machine M in essentially the same way as the high-level program sketched above, given that the input to M is sufficiently large (on small inputs the simulation might fail). Stacks are directly available in RBS, and thus an RBS program can easily represent the tape and mimic the movements of the head. On the other hand, assignment statements and if-then statements are not directly available. This makes things a bit tricky. Let us first see how RBS programs to a certain extent can simulate programs written in a non-reversible programming language called .

Programs

The syntax of is given in Fig. 3. Any element in the syntactic category Command will be called a program. A program manipulates natural numbers, and each program variable holds a single natural number. The command assigns the fixed number k to the variable . The command assigns the number hold by the variable to the variable . The command decreases the value hold by the variable by 1 if the value is strictly greater than 0; and leave the value hold by unchanged if the value is 0. Furthermore, the command is the standard composition of the commands and , and the command executes the command repeatedly k times in a row where k in the number hold by . Note that the variable is not allowed to occur in and that the command does not modify the value held by .

Fig. 3.

The syntax of the language . The variable X in the loop command is not allowed to occur in the loop’s body.

An RBS program can represent a variable holding natural number k by a variable (we use the same variable name) holding the stack . The command can then be simulated by the program

where is an auxiliary variable ( works as a trash bin). Now, observe that this will only work if the base of execution is strictly greater than k, but that will good enough to us. The command can be simulated by the program

where is an auxiliary variable ( works as a trash bin). Furthermore, the command can be simulated by a program that uses auxiliary variables and (which represent natural numbers) and the simulations of the assignment statements given above:

This shows how programs can simulate all the primitive commands. It is easy to see that

• the RBS command simulates the command if simulates and simulates

• the RBS command simulates the command if simulates .

Hence, any program can be simulated by an program given that the input is sufficiently large. On small inputs simulations might fail since the simulation of the assignment only works if the execution base is strictly greater than k.

The language turns out to be more expressive than one might expect at a first glance, and all sorts of conditional statements and if-then constructions are available in the language. As an example, let us see how we can implement the construction

We will need some axillary variables , , , which do not occur in any of the commands and . First we execute the program

This program sets both and to 0 if and hold the same number. If and hold different numbers, one of the two variables will be set to a number strictly greater than 0. Then we execute the program

The composition of these two programs executes the program exactly once (and will not be executed at all) if and hold the same number. If and hold different numbers, will be executed exactly once (and will not be executed at all). The reader should note that this implementation of if-then-else construction does not contain any assignments of the form where .

It is proved in Kristiansen [8] that captures the complexity class , that is, the set of problems decidable in space O(n) on a deterministic Turing machine (n is the length of the input). Hence, the considerations above indicate that . However, we are on our way to proving a stronger result, namely that . The equality is one of the many notorious open problems of complexity theory. The general opinion is that the equality does not hold.

RBS Programs that Simulates Time-Bounded Turing Machines

We have seen that RBS programs (nearly) can simulate programs. can assign constants to registers and perform if-then-else constructions. This helps us to see how to an RBS program can simulate an arbitrary time Turing machine M. Such a program may be of the form

We represent the symbols in M’s alphabet by the numbers and M’s states by the numbers . We use two stacks to hold the content of the tape, and we use registers and SCAN to hold respectively the current state and the scanned cell. Each will take care of a transition and be of the form

We are left with a minor problem: This will not work for small inputs. This will only work if the base of execution is strictly greater than . Only then will the simulating program be able to perform the necessary assignments of constants to variables. In some sense we cannot deal with this problem. An RBS program will not be able to simulate (in any reasonable sense of the word) an arbitrary time Turing machine M on small inputs, but still there will be an RBS program that decides the same problem as M.

We have seen that it suffices to assign the constants 0 and 1 to variables in order to implement the if-then-else construction in . This entails that the if-then-else construction will work on small inputs as the base of execution always will be strictly greater than 1. Hence, if the problem A is decided by a time Turing machine M, there will also be an RBS program that decides A. This program will be of the form

Main Results

A Characterization of

Definition 5

Let |m| denote the number of digits required to write the natural number m in binary notation. For any natural number k, let be the class of problems decidable in time on a deterministic Turing machine. Let .    

Theorem 6

.

Proof

The proof of the inclusion should be straightforward to anyone experienced with Turing machines. Assume (we will argue that ). Then there is an program that decides A. Let m be the input to . Each loop in will be executed maximum times since the base of execution will be . Thus, there exist constants (not depending on m) such that bounds the number of primitive commands executed by on input m. A Turing machine can simulate the execution of on input m with polynomial overhead. Thus there exist constants such that bounds the number of steps a Turing machine needs to decide if m is in A. There exists k such that . Hence, . This proves the inclusion .

We turn to proof of the inclusion . Assume (we will argue that ). Then there is a time Turing machine M that decides A. Now, M will run in time when is sufficiently large. In the previous section we saw that there will be an RBS program that decides the same problem as M. Hence, . This proves the inclusion .   

A Characterization of

Would it not be nice if we could find a reversible language that captures a complexity class that is a bit more attractive than ? Now, is for a number of reasons, which the reader might be aware of, one of most popular and important complexity classes. Luckily, it turns out that a few modifications of RBS yield a characterization of .

First we modify the way programs receive input. The input will now be a string over some alphabet. Any alphabet that contains at least two symbols will do and, for convenience, we will stick to the alphabet . The base of execution will at program start be set to the length of the input. Otherwise, nearly everything is kept as before: Every variable will still hold a bottomless stack storing natural numbers. All commands available in the original version of RBS will be available in the new version. A program will still accept its input by terminating with 0 at the top of the stack held by , otherwise, the program rejects its input. Moreover, all variables including , the variable that used to import the input, hold the zero stack when the execution of a program starts.

Next we extend by two commands with the syntax

where X is a variable and com is a command which does not contain X. These commands make it possible for a program to access its input. The input is a string where b is the execution base and . Assume that holds a stack where top element is k. The command

executes the command if , otherwise, the command does nothing. The command

executes the command if , otherwise, the command does nothing.

We still have a reversible language. The two new commands are reversible. The variable is not allowed to occur in and will consequently not be modified by . Thus, for , we may extend Definition 2 by

and Theorem 3 will still hold.

To avoid confusion we will use to denote our new version of RBS. We require that the input to an program is of length at least 2 (so we exclude the empty string and the one-symbol strings a and b). This is of course a bit artificial, but it seems to be the most convenient way to deal with a few annoying problems of technical nature. Accordingly, we also require that every string in a language (see the definition below) is of length at least 2.

Definition 7

A language L is a set of strings over the alphabet , moreover, every string in L is of length at least 2.

An program C decides the language L if C accepts every string that belongs to L and rejects every string that does not belong to L. Let be class of languages decidable by an RBS’ program.

Let |w| denote the length of the string w. For any natural number k, let be the class of languages decidable in time on a deterministic Turing machine. Let .   

Figure 4 shows an program which decides the language given by the regular expression .

Fig. 4.

The program accepts any string that starts with a nonempty sequence of a’s and ends with a single b (the input to a program should at least contain two symbols). The program rejects any string that is not of this form. The program accepts by terminating with and rejects by terminating with .

The proof of the next theorem is very similar to the proof of Theorem 6, and the reader should be able to provide the details. Just recall that the execution base of an program is set to the length of the input. Hence, the number of primitive instructions executed by an program will be bounded by where |w| is the length of the input w and k is a sufficiently large constant, and moreover, an program of the form

will execute exactly times if each and one of the variables holds a stack where the top element is |w|.

Theorem 8

.

Some Final Remarks

We have argued that there is a link between implicit computational complexity theory and the theory of reversible computation, and we have showed that both and can be captured by inherently reversible programming languages. In general, implicit characterizations are meant to shed light on the nature of complexity classes and the many notoriously hard open problems involving such classes. Implicit characterizations by reversible formalisms might yield some new insights in this respect. It is beyond the scope of this paper to discuss or interpret the theorems proved above any further, but one might start to wonder how different aspects of reversibility relate to time complexity, space complexity and nondeterminism.

The author is not aware of any work in reversible computing that is closely related to the work presented above, but some work of Matos [14] is at least faintly related. Matos characterizes the primitive recursive functions by an inherently reversible loop-language.2 Paolini et al. [15] do also characterize the primitive recursive functions by a reversible formalism. Their work is of a recursion-theoretic nature and has a different flavor than ours, but it is possible that such studies might lead to interesting characterizations of complexity classes.

We finish off this paper by suggesting a small research project. It should be possible to extend to an inherently reversible higher-order language. First-order programs will be like the ones defined and explained above. Second-order programs will manipulate stacks of stacks, third-order programs will manipulate stacks of stacks of stacks, and so on. This will induce a hierarchy: the class of problems decidable by a first-order program, the class of problems decidable by a second-order program, ...by a third-order program, and so on. By the same token, will induce a hierarchy: the class of languages decidable by a first-order program, the class of languages decidable by a second-order program, and so on. These two hierarchies should be compared to the alternating time-space hierarchies studied in Goerdt [4], Jones [6], Kristiansen and Voda [10] and many other papers.